vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-06 04:30:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.06.3180 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-06-21 10:22:21 +00:00
parent 263ae9593f
commit b4b702e002
48 changed files with 109325 additions and 109325 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.06.3179 #### Version: V4.2022.06.3180
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.06.3179 #### Version: V4.2022.06.3180
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.06.3179 #### Version: V4.2022.06.3180
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13530
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.06.3179 #### Version: V4.2022.06.3180
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13534
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13534
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

13534
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13534
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

13534
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

13534
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

13534
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3179 3180

13534
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

13534
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.102.65 0; 20.253.184.42 0;

13534
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

13534
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

13534
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

13534
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.102.65 0; 20.253.184.42 0;

13534
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.102.65 0; 20.253.184.42 0;

13534
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AIBOT BBBike
Ecxi Buddy
NetSpider Domains Project
Offline Navigator IRLbot
Swiftbot IlseBot
Web Fetch Page Analyzer
WebAuto PleaseCrawl
backlink-check RebelMouse
sexsearcher ScreenerBot
voyagerx.com sysscan

280
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 360Spider
80legs 80legs
ADmantX
ASPSeek
Abonti Abonti
Aboundex Aboundexbot
AfD-Verbotsverfahren Acunetix
AhrefsBot
Aipbot Aipbot
Alexibot AllSubmitter
Anarchie
Anarchy
Anthill Anthill
AwarioSmartBot Aspiegel
BLEXBot BDFetch
BackDoorBot BackDoorBot
BackStreet BackStreet
BackWeb BacklinkCrawler
Badass Badass
Bandit Bandit
Barkrowler BetaBot
BatchFTP
Battleztar Bazinga
Bigfoot Bigfoot
Black Hole Black Hole
BlowFish Blackboard
Boardreader Boardreader
Bolt Bolt
BotALot Buck
BuiltWith BuiltWith
Bullseye
BunnySlippers BunnySlippers
BuzzSumo
CATExplorador
CODE87
CSHttp CSHttp
Calculon
CheTeam CheTeam
CheeseBot CheeseBot
CherryPicker
ChinaClaw ChinaClaw
Chlooe Cogentbot
Cliqzbot
Collector
Copier Copier
Copyscape CopyRightCheck
Cosmos Cosmos
Crawling at Home Project CrazyWebCrawler
CrunchBot
DSearch
DTS Agent DTS Agent
DataCha0s
Demon Demon
Devil
Dirbuster Dirbuster
Disco Disco
Discoverybot Discoverybot
DittoSpyder Dispatch
DomCopBot
DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
Drip Dotbot
ECCP/1.0
EMail Siphon EMail Siphon
EMail Wolf
Ecxi Ecxi
EirGrabber
EroCrawler EroCrawler
Evil
Express WebPictures Express WebPictures
ExtLinksBot
FDM FDM
FHscan
FemtosearchBot FemtosearchBot
FlashGet FlashGet
Foobot Flunky
FyberSpider Freeuploader
Genieo FrontPage
GermCrawler Fuzz
G-i-g-a-b-o-t
GetRight
GetWeb GetWeb
GoZilla Getintent
GrapeFX Go!Zilla
GridBot Gotit
Grabber
GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
HMView HTMLparser
HTTP::Lite
HTTrack
HaosouSpider
Havij
Heritrix Heritrix
HybridBot Hloader
IDBTE4M
IDBot IDBot
Iblog IRLbot
IlseBot IlseBot
Indy Library Image Fetch
Image Sucker
InfoNaviRobot InfoNaviRobot
InternetSeer InfoTekies
Iskanie InterGET
Internet Ninja
Iria
Jbrofuzz
JennyBot JennyBot
Jorgee JetCar
JustView JikeSpider
Joomla
Keyword Density Keyword Density
Kinza LexiBot
Kozmosbot
LWP::Simple
Larbin
Lftp
LibWeb LibWeb
LieBaoFast LieBaoFast
LinkWalker Likse
LinkextractorPro
LinkpadBot LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot
Litemage_walker
Lmspider Lmspider
MIDown tool MFC_Tear_Sample
MIIxpc
MJ12bot MJ12bot
MQQBrowser MQQBrowser
MSIECrawler
MTRobot MTRobot
Magnet Majestic-SEO
Mail.RU_Bot Majestic12
Majestic SEO
MarkMonitor MarkMonitor
MarkWatch
Mass Downloader Mass Downloader
MauiBot Mb2345Browser
MicroMessenger Meanpathbot
Microsoft Data Access Metauri
MolokaiBot Microsoft URL Control
Morfeus Fucking Scanner Mister PiX
Mozlila NPbot
Name Intelligence Name Intelligence
Navroad Nameprotect
NearSite NearSite
NetLyzer Needle
Netcraft Nessus
NimbleCrawler NetMechanic
NetSpider
Nibbler
Niki-bot
Nikto
Ninja
Nmap Nmap
Not Not
Nuclei Nuclei
Octopus Octopus
Offline Explorer Offline Explorer
Offline Navigator
OpenLinkProfiler OpenLinkProfiler
OpenVAS OpenVAS
Openfind
Openvas
OrangeSpider OrangeSpider
OutclicksBot PHPCrawl
PageScorer PageThing.com
Panscient
Papa Foto Papa Foto
PeoplePal Pi-Monster
Picscout PictureFinder
Pimonster
Pixray Pixray
PleaseCrawl
Pockey Pockey
Pu_iN
QueryN Metasearch QueryN Metasearch
RankFlex Quick-Crawler
RankingBot2
Rankivabot Rankivabot
RankurBot Re-re
ReGet ReGet
Reaper
RedesScrapy RedesScrapy
RepoMonkey RocketCrawler
Ripper SBIder
Rogerbot SEOkicks
SEOlyticsCrawler
SEOstats
SMTBot
ScanAlert
Scanbot
Scrapy Scrapy
ScrepyBot Seekport
Searchestate SentiBot
SeoSiteCheckup SeoSiteCheckup
Site Sucker Seomoz
SiteSucker Shodan
Sitevigil Sitebeam
Siteimprove
SmartDownload SmartDownload
Snake
Snoopy Snoopy
Sottopop Sociscraper
Spammen Spanner
SpankBot Spinn3r
SputnikBot Stripper
Sqlmap Surfbot
Sqworm
SuperBot
Suzuran Suzuran
T0PHackTeam Szukacz
T8Abot
Telesphoreo Telesphoreo
The Intraformant
TheNomad
Thumbor Thumbor
TurnitinBot Toata
Toweyabot
True_Robot
TwengaBot
Typhoeus
URLy.Warning URLy.Warning
Upflow UnisterBot
VB Project
VCI VCI
Vagabondo
VidibleScraper VidibleScraper
Virusdie VoidEYE
Voil Voil
Voltron Voltron
WASALive-Bot
WEBDAV
WISENutbot WISENutbot
WPScan
WWW-Mechanize
WWWOFFLE WWWOFFLE
Wallpapers Wallpapers
WeSEE Web Auto
Web Collage
Web Enhancer Web Enhancer
Web Fetch Web Fetch
Web Fuck
Web Pix Web Pix
Web Sucker Web Sucker
WebAuto
WebBandit WebBandit
WebCopier WebCollage
WebEnhancer WebEnhancer
WebFetch WebGo IS
WebFuck WebPix
WebLeacher WebReaper
WebStripper WebSucker
WebZIP
WebmasterWorldForumBot
Webshag Webshag
WebsiteExtractor Whack
Whacker
Whatweb Whatweb
WinHTTrack Widow
Wprecon Wprecon
Xaldon_WebSpider Xaldon_WebSpider
Zermelo Zeus
Zitebot
ZmEu ZmEu
ZumBot
ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler
arquivo.pt
clark-crawler clark-crawler
coccocbot
com.plumanalytics
dataforseo.com dataforseo.com
domainsproject.org demandbase-bot
facebookscraper facebookscraper
gopher gopher
instabid instabid
isitwp.com internetVista monitor
iubenda-radar
linkdexbot linkdexbot
magpie-crawler magpie-crawler
meanpathbot
mediawords mediawords
muhstik-scan muhstik-scan
oBot pcBrowser
page scorer seobility
satoristudio.net
scan.lol
seocompany.store seocompany.store
seoscanners seostar
sitechecker.pro serpstatbot
sogouspider spyfu
sysscan sysscan
tAkeOut
trendiction.com
trendiction.de trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
webmeup-crawler xpymep1.exe
webpros.com
webprosbot
x09Mozilla

310
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 404checker
80legs 80legs
Aboundex ADmantX
AfD-Verbotsverfahren AfD-Verbotsverfahren
Alexibot Alexibot
AllSubmitter AllSubmitter
AlphaBot Alligator
Anarchy99 Anarchy99
Anthill
Apexoo Apexoo
Aspiegel
Asterias Asterias
Atomseobot Attach
AwarioSmartBot
BBBike
BDFetch BDFetch
BackDoorBot BackStreet
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass
Bandit Bandit
BetaBot Bitacle
Bigfoot
BlackWidow
Blackboard
BlowFish
Bolt Bolt
BotALot
Buck Buck
BuiltBotTough Buddy
BuzzSumo
CCBot CCBot
CODE87
Calculon Calculon
CazoodleBot CazoodleBot
CheeseBot
Chlooe Chlooe
Cogentbot Citoid
Copier Cliqzbot
Crescent Cloud mapping
Curious Copyscape
Crawling at Home Project
CrazyWebCrawler
Custo Custo
CyotekWebCopy CyotekWebCopy
DBLBot
DSearch DSearch
DataCha0s DTS Agent
DatabaseDriverMysqli DatabaseDriverMysqli
Dirbuster Demon
Discoverybot Devil
Digincore
DigitalPebble
Discobot
Dispatch Dispatch
DittoSpyder
DnyzBot DnyzBot
DomCopBot
DomainAppender
DomainCrawler DomainCrawler
DomainStatsBot DomainSigmaCrawler
Dotbot Dotbot
Download Wonder
Dragonfly Dragonfly
Drip EasyDL
EMail Wolf Ebingbong
Ecxi EroCrawler
Exabot ExtLinksBot
Express WebPictures ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
Ezooms
FDM
FHscan
Fimap Fimap
FlashGet Foobot
Freeuploader FrontPage
Fuzz
FyberSpider FyberSpider
G-i-g-a-b-o-t GalaxyBot
Genieo
GermCrawler
GetWeb
Getintent Getintent
Gigabot
Gotit Gotit
Haansoft Grafula
Havij GridBot
Heritrix HTTrack
Harvest
Hloader
HonoluluBot HonoluluBot
HybridBot
IRLbot
Id-search
IlseBot IlseBot
Image Sucker Image Sucker
Indy Library
InfoNaviRobot InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
InternetSeer
Iria
IstellaBot
JamesBOT
JennyBot JennyBot
JetCar
JikeSpider JikeSpider
Joomla Joomla
JustView
Jyxobot
Kenjin Spider
Keybot Translation-Search-Machine
Kinza
LNSpiderguy LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot LeechGet
Leap LexiBot
LeechFTP
Lftp
LibWeb LibWeb
LieBaoFast
Lightspeedsystems
LinkScan LinkScan
Linkbot
LinkpadBot LinkpadBot
LinqiaRSSBot Lipperhey
LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Litemage_walker Lmspider
Ltx71
MIDown tool MIDown tool
MIIxpc
MJ12bot
MQQBrowser MQQBrowser
Magnet MSFrontPage
Majestic SEO Majestic SEO
Majestic-SEO Majestic-SEO
Majestic12
MarkWatch
Masscan Masscan
Mata Hari Mata Hari
Meanpathbot MeanPath Bot
Mediatoolkitbot MegaIndex.ru
Metauri
MicroMessenger
Microsoft URL Control
Minefield Minefield
Moblie Safari
Morfeus Fucking Scanner
Mozlila
Musobot Musobot
NICErsPRO NICErsPRO
NPbot NPbot
Nameprotect
Navroad Navroad
Needle NearSite
Net Vampire NetAnts
NetSpider NetSpider
NetZIP NetZIP
Nettrack Nettrack
Netvibes NextGenSearchBot
Nikto Nibbler
NimbleCrawler
Ninja
Nmap Nmap
Not Not
Offline Explorer Octopus
Offline Navigator
OpenLinkProfiler OpenLinkProfiler
OpenVAS OpenVAS
Openfind OutclicksBot
OrangeBot OutfoxBot
Pandalytics PHPCrawl
Papa Foto POE-Component-Client-HTTP
PageGrabber
PageThing.com
Panscient
Pi-Monster Pi-Monster
Picscout
Picsearch Picsearch
PictureFinder
Piepmatz Piepmatz
Psbot ProPowerBot
ProWebWalker
Probethenet
Pump Pump
PxBroker
Quick-Crawler Quick-Crawler
RSSingBot RankActive
RankActiveLinkBot RankActiveLinkBot
RankingBot RankFlex
RankingBot2 Rankivabot
RealDownload Recorder
RepoMonkey RocketCrawler
Ripper
SBIder
SEOlyticsCrawler SEOlyticsCrawler
SMTBot SEOprofiler
ScanAlert SEOstats
Screaming Scanbot
ScrepyBot ScreenerBot
SearchmetricsBot Searchestate
Semrush SemanticJuice
Siphon SeoSiteCheckup
Site Sucker SeobilityBot
Shodan
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteLockSpider
Sitebeam Sitebeam
SmartDownload Snake
Snapbot Sottopop
SocialRankIOBot SpankBot
Spbot Spbot
Spinn3r Sqlmap
SputnikBot
Steeler Steeler
Sucker
SurveyBot
Swiftbot Swiftbot
Szukacz Szukacz
TheNomad T0PHackTeam
Thumbor TeleportPro
Titan Telesphorep
Trendiction The Intraformant
True_Robot TightTwatBot
Turingos Toata
TurnitinBot TurnitinBot
TwengaBot TwengaBot
Typhoeus Typhoeus
URLy.Warning URLy Warning
UnisterBot Upflow
Virusdie Vagabondo
Voltron VidibleScraper
WASALive-Bot WBSearchBot
WEBDAV WPScan
WISENutbot
WWW-Collector-E WWW-Collector-E
WWW::Mechanize Wallpapers
WWWOFFLE WallpapersHD
WeSEE WeSEE
Web Fetch Web Fuck
Web Pix
Web Sauger
WebBandit WebBandit
WebCollage WebCopier
WebFetch WebGo IS
WebFuck
WebImageCollector WebImageCollector
WebLeacher
WebReaper
WebSauger
WebStripper
WebSucker WebSucker
WebmasterWorldForumBot WebWhacker
Webster Webalta
Whack WebsiteExtractor
Whatweb WebsiteQuester
Who.is Bot Widow
WinHTTrack WiseGuys Robot
Wotbox Woobot
Wprecon ZumBot
Xaldon_WebSpider
Xenu
YoudaoBot
Zade
Zeus
Zitebot
ZmEu
ZoomBot
adscanner
archive.org_bot archive.org_bot
autoemailspider clark-crawler
cah.io.community cognitiveseo
check1.exe com.plumanalytics
crawler.feedback crawler.feedback
crawler4j eCatch
demandbase-bot evc-batch
ips-agent facebookscraper
linkdexbot gopher
heritrix
instabid
isitwp.com
lwp-request lwp-request
lwp-trivial lwp-trivial
magpie-crawler
meanpathbot meanpathbot
mediawords mediawords
netEstate NE Crawler muhstik-scan
pcBrowser plumanalytics
polaris version ripz
s1z.ru s1z.ru
scalaj-http satoristudio.net
seobility scan.lol
sexsearcher seocompany.store
sitechecker.pro spyfu
sysscan
tAkeOut
webmeup-crawler webmeup-crawler
webpros.com
x09Mozilla x09Mozilla
x22Mozilla
xpymep1.exe
zauba.io zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
adultfriendfinder.com 256bit.by
amt-k.ru barnfurnituremart.com
aviapanda.ru dimfour.com
cctva.tv duplicashapp.com
freshsuperbloop.com himgaws.pw
orgasmatrix.com hotxnights.info
soaksoak.ru housemilan.ru
unisexjewelry.org meds-online24.com
vielporno.net oynat.info
wildnatureimages.com xn--1-8sbcpb0bdm8k6a.xn--p1ai

930
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 01apple.com
00webcams.com 1webmaster.ml
1-free-share-buttons.com 20pascals.nl
1001watch.com.ua 3-letter-domains.net
101billion.com 991mostfm.co.id
101lesbian.xyz a14download.com
13tabs.com aa14ab57a3339c4064bd9ae6fad7495b5f.com
1adult.com academia-nsk.org
2345.com adbetclickin.pink
2728fb936f0.com addtoadd.men
2daytrendingnews.com adserver-e7.com
4kpics.rocks
83net.jp
absolute-s.ru
acortarurl.es
adpremium.org
ads-seo.men
adultactioncam.com adultactioncam.com
afslankpillen2017nl.eu adultmeetonline.info
aibolita.com adzpower.com
allblogroll.com agecheckadult.com
allkrim.com aihelen.net
alot.com aimayangzhi.com
alphacarolinas.com airmaxshoes-2016.com
am15.net alessandraleone.com
ameblo.jp alfa-medosmotr.ru
amehdaily.com allwidewallpapers.com
analnoeporno.tv alt-servis.ru
anaseracresar.tk amt-k.ru
animalia-life.club animaltoplist.com
animenime.ru anime.dougasouko.com
ap.senai.br
apartamentwroclaw.eu apartamentwroclaw.eu
apiadanaknet-a.akamaihd.net apiqualitinknet-a.akamaihd.net
apioasisspacenet-a.akamaihd.net appsecurityr.com
apiwoodensealcom-a.akamaihd.net arendas.net
app5.letmacworkfaster.world
apple.com-webbrowsing-security.science
apps-analytics.net
arenanews.com.ua
arpelsreplica.xyz arpelsreplica.xyz
asrvvv-a.akamaihd.net artpicso.com
atmagroup.ru arvut.org
ausmepa.org.au astrochicks.com
aussie-prizes.com atofilms.com
australia-opening-times.com auto.rusvile.lt
auto4style.ru autoua.top
autoseo-traffic.com aviav.ru.com
avtointeres.ru avtoarenda.by
avtolombard-krasnodar.com ayodhya.co
azarttoto.com azartmsl.com
azbukafree.com azbaseimages.net
babosas.com b-styles.xyz
backlink4u.net baixar-musicas-gratis.com
bambi.ck.ua balkanfarma.ru
bamo.xsl.pt bannerconnect.net
beachfix.co bard-real.com.ua
beachpics.com bayareaaeroclub.org
beatifullandscape.co bear.gotcher.us
beget.tech beauty-bracelet.com
bestbuy.ca berrymall.ru
bestempresas.es best-seo-offer.com
bestplacetobuyeriacta.jw.lt bestchoice.cf
betshuckclean.com bestcurs.org
big-cash.net bestdraws.com
bio-optomarket.ru bestfortraders.com
biocn.dx.am bestkfiledxd.cf
bloggen.be bestmobilityscooterstoday.com
bloglag.com bestofferswalkmydogouteveryday.gq
board.f00d.de bimmerpost.com
bonanza-fortune.men bingoporno.com
bonzibuddi.com bjetjt.com
bookhome.info bkgr.se
bosman.pluto.ro blackcurranthumidifiers.site
bracketsmackdown.com blackwitchcraft.ru
bradleylive.xyz blobar.org
brainxs.us blog-hits.com
brokergid.ru blog.f00kclan.de
browserprotecter.com bloggedporn.com
bum.com.ru bloke.com
burkesales.com bluesalt.co
buyhoverboard.com bluesman.nu
buytizanidineonlinenoprescription.blogspot.com boleznikogi.com
callstevens.com bonkers.name
canacopegdl.com bonux.nextview.ru
cartechnic.ru brillianty.info
cartierlovestore.com bristolhotel.com
cartierreplica.top buketeg.xyz
cejewelry.xyz buntube.net
chatroulette.life buqyxa.rincian.info
chatroulette.online bus-offer.com
cheapbarbour.online business-made-fun.com
chidporn.com buybest1.biz
chocolateslim-en-espana.com buyk.host.sk
bvlgaribracelet.xyz
cah.io.community
calcularpagerank.com
canyougethighofftizanidine.blogspot.com
capture-room.com
carfax.com.ua
carmuffler.net
carsplate.com
cartierwatch.xyz
cb.iphantom.com
cheapjerseysa.com
cheatcode-lita12.rhcloud.com
cherrypointplace.ca
cherubinimobili.it
chinacheapelitejerseys.com
chiptuninger.com
chocoslim.pro
chuckguilford.com chuckguilford.com
clash-clans.ru cialis-samples.com
clean-start.net cialiswithoutadoctor.net
club-samodelkin.ru cindymatches.com
coldfilm.ru click2pawn.com
clickhype.com
clickprotects.com
clmforexeu.com
co.lumb.co
coderstate.com
codq.info
colehaanoutlet.store
com-onlinesupport.site com-onlinesupport.site
commoncrawl.org compliance-jess.top
compiko.info computernetworksonline.com
compliance-fred.top counciltally.com
compliance-mary.top cpabegins.ru
compucelunlock.net craftburg.ru
comsysnet.com creditwell.ru
cool-wedding.net crystalslot.com
coolwallpapers-hd.com cwetochki.ru
coprofam.org daooda.com
copypaste.traderzplanet.in ddlmega.net
covetnica.com deda-moroza-zakaz.ru
crazyboost.pro descargar-musica-gratis.net
creditnation.ru design-ideas.info
crynet.cc dichvuvesinhhanoi.com
culturevie.info digital-video-processing.com
data-mining.tk dirtpics.pw
degerlund.net dividendo.ru
diegolopezcastan.com dodlive.mil
dikx.gdn dogoimage.com
discountbarbour.online dojki-hd.com
diusyawiga.tk domoysshop.ru
dokfilms.net dostavimvdom.ru
domain2008.com downloadeer.net
domaincheck.io doxyporno.com
domaineaneblanc.com
domainsatcost.com
dotmass.top
downloadmefiranaratb1972.xpg.com.br
drev.biz
driving.kiev.ua
drugstoreforyou.com drugstoreforyou.com
dtnlyss.com dvervmoskvu.ru
dumpsmania24.com dyt.net
dvd-famille.com easyukraine.com
dynainbox.com ebooknovel.club
e-c.al editors.choice6912650.hulfingtonpost.com
earncash.com.ua electrik-avenue.com
edge.sharethis.com elektronischezigarettekaufen1.beeplog.com
eleimgo.pw elexies.info
elektrischezigarette2.devhub.com
elektronischezi.livejournal.com
elektrozigarette2.wordpressy.pl
elitesportsadvisor.com elitesportsadvisor.com
enbersoft.com eluxer.net
erectile.bid enhand.se
erotikchat-24.com erot.co
errorfixing.space esnm.ru
esfchat.tk exlarseva.webblog.es
etm-consult.de exoneration-project.us
europages.com.ru extener.org
evening-dating-club.info eyelike.com.ua
exct.net faithe.top
extremez.net fashionavenuegame.com
ezigarettekaufen.myblog.de femdom.twiclub.in
f00kclan.de fickenbumsen.net
factorynightclub.com fickenprivat.info
farmingworm.com filmgo.ru
fighrofacciufreesig.ga findanysex.com
fil.ru firma-legion.ru
filmi-onlain.info folowsite.com
financepoints.eu foxweber.com
findthe.pet freakycheats.com
finuse.com free-share-buttons.top
fivedwld.cf free-stock-illustration.com
fiverr.com freejabs.com
fix-website-errors.com freesitetest.com
flauntyoursite.com
florida-tourism.net
forms-mtm.ru
forodvd.com
fortwosmartcar.pw
forzeronly.com
fotzen-ficken.com
franecki.net
free-social-buttons.xyz
freefoto.ca
freelinkbuilding.website.tk
freenom.link
freewlan.info freewlan.info
fuckmill.com ftns.ru
fullfileaccess.com fun2cell.net
fun-mobi.pl fungirlsgames.net
fungamelands.com gamesprite.me
funnymama.com gamingspark.com
fxgallery.com gardene.ru
gallerylisting.com genericlowlatencyasiodriverhq.aircus.com
ge0ip.net girlgamerdaily.com
gentamicineyedrops.blogspot.com gne8.com
getridofstretchmarks.org gourcy.altervista.org
gigapeta.com gratuitbaise.com
glass-msk.ru gta-club.ru
globalscam.ga gta-top.ru
gorabagrata.ru hamacapty.com
gpms.org.my hdpixent.com
green-tea.tv
gyffu.com
habermetre.com
hawaiielectriclight.com
hd720kino.ru
headpharmacy.com headpharmacy.com
hentai-manga.porn helicalpile.us
hidefiles.org hit-kino.ru
hkladys.com hitcpm.com
hobbyhuren-datenbank.com hobbyhuren24.net
hobild.net hollywoodactress.info
homelygarden.com hostingclub.lk
homik.pw howmuchdoestizanidinecost.blogspot.com
hostcritique.com ictizanidinehcl4mg.blogspot.com
hotblog.top iframe-toloka.com
hoverboard360.de imk.com.ua
htmlcorner.com incest-ru.com
huimin764128.com inmate-locator.us
humanorightswatch.org inspiring-desperate.tk
ibmdatamanagement.co integritylandscapeservices.com
ifirestarter.ru interesnie-faktu.ru
ihc2015.info iopeninghours.co.uk
ilovevitaly.ru isitwp.com
ilte.info itag.pw
imadedinner.net itbc.kiev.ua
imagefinder.site j33x.com
images-graphics-pics.com javspace.net
imicrovision.com jerseywholesalebizchina.com
imperia31.ru jerseywholesalechinabiz.com
in-tandem.co
indetiske.ya.ru
indo-export.ru
informatiecentro.be
intervsem.ru
inzn.ru
isitpaleo.info
istanbulit.com
jackwolfskinoutlet.online
james13prix.info
jennyfire.ru
jongose.ninja jongose.ninja
jovencitas.gratis
jpcycles.com
jumptap.com
jus0wil.pp.ua
justucalling32211123456789.tk
kaac.ru
kalandranis.gr
kambasoft.com
kandidos.com kandidos.com
keki.info karadene.com
keywordhut.com kareliatobacco.ru
keywordsuggest.org karpun-iris.ru
kiinomaniak.pl katushka.net
kinobaks.com kemerovo.zrus.org
kinohall.ru keywords-monitoring-your-success.com
kinostorm.net kino-key.info
kinozapas.org kinoduh.ru
klitimg.pw klikbonus.com
knowsitall.info komp-pomosch.ru
konpax.com kosynka-games.ru
konturkrasoty.ru kupiproday.com.ua
koronirealestate.gr kutikomi.net
kskjco.club lafriore.ru
labvis.host.sk lancheck.net
lacasamorett.com lankarns.com
landscaping.center law-check-eight.xyz
larose.jb2c.me law-check-nine.xyz
leboard.ru laxdrills.com
letmacworkfaster.world letolove.ru
lignofix.ua levaquin750mg.blogspot.com
limtu.ifmo.ru lexixxx.com
link.ac life.biz.ua
linkredirect.biz lifebyleese.com
lifehacĸer.com
linksharingt.com
lirunet.ru lirunet.ru
littlesexdolls.com lisque.batcave.net
live-sexchat.ru live-sexcam.tk
ljusihus.se lonerangergames.com
locimge.pw lonfon.xyz
lostaruban.com longlifelomilomi.info
lottospring.com lovasszovetseg.hu
lubetube.com luckyshop.net.ua
luxmagazine.cf luxup.ru
m3gadownload.pl macdamaged.tech
macresource.co.uk macotool.com
magicdiet.gq mailemedicinals.com
makis.nu
maladot.com maladot.com
manualterap.roleforum.ru manve.info
marcoislandvacations.net
marsgatan.com marsgatan.com
meet-flirt-dating.com martlinker.com
mega-bony2017.pl maxxtor.eu
megakino.net mebel-alait.ru
menetie.ru metafilter.com
menstennisforums.com michaelkorsoutlet.store
microsearch.ru midst.eu
misandesign.se mir-limuzinov.ru
misslike.ru mobsfun.net
missvietnam.org mojpregled.com
mnogabukaff.net monarchfind-a.akamaihd.net
moivestiy.biz monclerjacketsoutlet.pw
mole.pluto.ro moneyviking-a.akamaihd.net
moskva.nodup.ru monthlywinners.com
movies-in-theaters.net moscow.online-podarki.com
moyakuhnia.ru most-kerch.org
mttwtrack.com mriyadh.com
musicas.baixar-musicas-gratis.com mundoaberrante.com
muztops.ru mural.co
myhydros.org mydownlodablefiles.com
myra.top myfreemp3.eu
nalogovyy-kodeks.ru myhealthcare.com
narkologiya-orel.ru myshopmatemac.com
narkologiya-penza.ru nacap.ru
namenectar.com
naval.jislaaik.com
neobux-bg.info neobux-bg.info
networkcheck.xyz netvouz.com
new-apps.ru nextconseil.com
news-readers.ru nhl17coinsforps3.gratisblog.biz
newstudio.tv nikkiewart.ru
nfljerseysforsalewholesaler.com nina.az
nikhilbahl.com
ningessaybe.me
nodding-passion.tk
nordvpn.com
normalegal.ru
novosibirsk.xrus.org novosibirsk.xrus.org
nufaq.com nowtorrents.com
offer.camp
offer.wpsecurity.website
offf.info offf.info
ohmyrings.com offside2.5v.pl
online-templatestore.com onescreen.cc
onlinetvseries.me online-podarki.com
onlinewritingjobs17.blogspot.ru onlineku.com
onlywoman.org onstrapon.purplesphere.in
orsonet.ru openfrost.com
oralsexfilme.net
owathemes.com
oynat.info
paleohub.info
pammik.ru
partner-trustworthy.men
pavlodar.xkaz.org
pc-virus-d0l92j2.pw
pcboa.se
pdns.download
penisvergrotendepillennl.ovh
percin.biz.ly percin.biz.ly
photo-clip.ru perl.dp.ua
pflexads.com
photo.houseofgaga.ru photo.houseofgaga.ru
photorepair.ru picquery.com
pic2fly.com pics-group.com
pierrehardysale.online picscout.com
pixell.club picsearch.com
pk-services.ru picsfair.com
pobeiranie.pl picurams.pw
pons-presse.com pijoto.net
pornonik.com pix24x7.com
pornophoto.xyz pl-top.pl
pornzone.tv platesauto.com
primedice.com pogosh.com
privhosting.com poppen-nw.net
profitwithalex.info poptool.net
prostitutki-rostova.org pornmania.pl
psoriasis-file.trade pornofeuer.com
purplesphere.in pornoslive.net
push-ad.com portadd.men
qexyfu.bugs3.com prchecker.info
qld10000.net predominant-invent.tk
quangcaons.com pricheskaonline.ru
quelle.ru prizesbook.online
rada.ru profitsport.club
rangoman.date pronekut.com
ranksonic.org proxyradar.com
ratemodels.net qitt.ru
refads.pro quebec-bin.com
registratciya-v-moskve.ru quick-seeker.com
registry-cleaner.net quizzitch.net
remont-fridge-tv.ru r-control.ru
resant.ru rainbowice.ru
rankia.com
realitykings.com
recipedays.com
redirect.trafficreceiver.club
relax.ru
remont-mobile-phones.ru
rethinkwasteni.info rethinkwasteni.info
rfserial.net revistaindustria.com
rialp.getenjoyment.net riralmolamsaca.tk
roofers.org.uk robotixix.com
rumamba.com rock-cafe.info
ruscopybook.com rubbed.us
rucrypt.com
russia-today-video.ru russia-today-video.ru
russkoe-zdorovie.ru rusvideos.su
rutor.group
s.lollypopgaming.com
sabaapress.com
saecsa.co saecsa.co
saltspray.ru sale-japan.com
satellite.maps.ilovevitaly.com samolet.fr
sbornik-zakonov.ru savememoney.co.za
sbt-aqua.ru sbwealthsolutions.ca
scanner-barak.top scanner-olga.top
scanner-julia.top se.bnt-team.com
scanner-mark.top
scanner-mary.top
scmor.ilxc.cc
scripted.com
search.1and1.com
search.alot.com
search.pch.com search.pch.com
searchengineranker.email searchtooknow-a.akamaihd.net
see-your-website-here.com seasonvar.ru
sexpartygirls.net segol.tv
sexpornotales.net semaltmedia.com
sexy-pings.com seminarykansas904.ml
shoppingmiracles.co.uk seocheckupx.com
shopvilleroyboch.com.ua sethrollins.net
sibecoprom.ru sexcamamateurchat.com
signx.info sexkvartal.com
sims-sims.ru sexobzor.info
siteaero.com sexreliz.com
sitevaluation.org shijian.ac.cn
sklad-24.ru shopcheermakeup.info
skylta.com simple-share-buttons.com
slim.sellany.ru sindragosa.comxa.com
smailik.org sitebeam.net
smart-balancewheel.com sitopreferito.it
smart-scripts.com sixcooler.de
sizeplus.work
sk.golden-praga.ru
sky-mine.ru
sladkoevideo.com
small-game.com
smartpet.ru
snjack.info snjack.info
snow.nvr163.com snts.shell-pmr.ru
soaksoak.ru snworks.com
social-buttons.com
sockshare.net sockshare.net
softomix.com sofit-dmd.ru
soietvousmaime.fr soietvousmaime.fr
space-worry.ml sortthemesitesby.com
spidtest.org space4updating.win
sport7777.net spb.afora.ru
spravka130.ru sstroy44.ru
sps-shop.com stariy-baku.com
standardchartered-forex.com stauga.altervista.org
starwars.wikia.com stroy-portal22.ru
statistici.ro studiofaca.com
stopnarco.ru sukirgenk.dvrlists.com
stpolice.com suralink.com
streha-metalko.si swinon.site
sugarkun.com swiped.su
szamponrevita.pl t3chtonic.com
tasteidea.com
tattooha.com tattooha.com
tecnoteakviareggio.it
tedxrj.com
teenforporn.com
terrafootwear.us
testbotprocessor44.com testbotprocessor44.com
tetracsaudi.com the-trader.net
thaismartloan.com the-usa-games.blogspot.com
thefarmergame.com thebestphotos.eu
theendivechronicles.com
thegalerie.eu
theplacetoupdating.pw
thepokertimer.com thepokertimer.com
thewebsitetemplate.info tilido.com
thfox.com
timecrimea.ru
timhost.ru
titslove.yopoint.in
tivolibasket.it tivolibasket.it
tizanidine6mg.blogspot.com tizanidineandgabapentin.blogspot.com
tizanidinedruginteractions.blogspot.com tizanidineforopiatewithdrawal.blogspot.com
tizanidinehcl4mgtabinfo.blogspot.com tizanidinehcl4mgtabinfo.blogspot.com
tk-assortiment.ru tizanidineinfo.blogspot.com
tizanidineorflexeril.blogspot.com
tmm-kurs.ru tmm-kurs.ru
tmtrck.com top-l2.com
top10-way.com topshef.ru
top10registrycleaners.com topsy.com
topclickguru.com totu.us
torrentgamer.net toursmaps.com
tradgardspartner.se tozup.com
trafficstars.com track112.site
tramadolandtizanidine.blogspot.com tradedeals.biz
treasuretrack-a.akamaihd.net trailer.cinemaflix.website
trion.od.ua trucri.me
trumpetedextremes.com tuckermktg.com
turbabitload.weebly.com turizm.bz
u555u.info tzritel.tk
ua.tc ucoz.ru
uac.net.au ukr-lit.com
ukkala.xyz urlopener.com
urll.eu usacasino.com
uzungil.com usadacha.net
vacuumcleanerguru.com usswrite.com
vapomnoncri.tk utiblog.fr
varikozdok.ru uytmaster.ru
vavilone.com valoresito.com
vellings.info valuado.com
velobikestock.com vchulkax.com
videos-for-your-business.com vduplo.ru
vikistars.com video-hollywood.ru
vip-dom.in videochat.bz
vipms.ru videochat.ph
visitcambridge.org videokrik.net
vitanail.ru visionwell.com.cn
vizag.kharkov.ua vksex.ru
vladhistory.com vmnmvzsmn.over-blog.com
vykup-avto-krasnodar.ru vykup-avto-krasnodar.ru
vykupavto-krasnodar.ru vzlomtw.com
w3javascript.com wareseeker.com
warningzscaler.heraeus.com webix.biz
webs.com webkeyit.com
website-speed-check.site website-speed-checker.site
whatsupinfoley.com weddingdresses.xyz
wemedinc.com
whatzmyip.net whatzmyip.net
whipme.yopoint.in wikes.20fr.com
wholesalecheapjerseysfree.com worldhistory.biz
wholesalejerseys-cheapest.com wowcasinoonline.ooo
wildnatureimages.com wurr.voila.net
wildworld.site x-true.info
windowssearch-exp.com xcombear.ru
wonderfulflowers.biz xgames-04.com
works.if.ua xmnb.net
worldlovers.ru xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
wrona.it xn----ctbigni3aj4h.xn--p1ai
www.jbetting.com xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
xn----itbkqkfiq.xn--p1ai xn--d1acah0c.xn--p1ai
xtrafficplus.com xtremeeagles.net
xtube.com xxxkaz.org
xuki.us yellowads.men
xwatt.ru yoshkarola.zrus.org
yoopsie.com youghbould.wordpress.com
youtoner.it yoursite.com
zaloro.com yuweng.info
zakon-ob-obrazovanii.ru
zambini.ru
zdesoboi.com zdesoboi.com
zeleznobeton.ru zhacker.net
znakomstva-moskva77.ru zoogdiesney.com
zoolubimets.ru zootravel.com
zvuker.net
zynax.ua

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1qingdao.com 1xbet4you.com
2728fb936f0.com 7search.com
2rich4bitches.com adimmix.com
aa625d84f1587749c1ab011d6f269f7d64.com admeasures.com
abwa.tk ads-cool.pro
academia-nsk.org adsref.men
acads.net ageofclones.com
adclickthru.net agysya.ru
adultfullhd.com aimayangzhi.com
anapa-inns.ru airmaxshoes-2016.com
apple.com-webbrowsing-security.science alert-jdh.xyz
articlesdirectoryme.info alice.it
astrochicks.com alldubai.biz
ayeartoforget.com alphahoverboards.com
azazaporn.com ameblo.top
belinvestdom.by amoi.tn
best-coupon-offer.com ample-awards-today.us
beta.hotkeys.com angkortours.vn
binomo.com anonymous-redirect.com
bio-japan.net apiwoodensealcom-a.akamaihd.net
birzha-truda.eu arate.ru
bizru.info asociatia-tipografilor-transilvania.ro
bombla.org atlant-auto.info
bonanza-fortune.men aviav.eu
bristolhostel.com avtolombard-krasnodar.ru
buqyxa.rincian.info best-mam.ru
buy-loft.ru best-seo-offer.com
buybest1.biz bestbrainprod.win
camdenmemorials.com biglistofwebsites.com
carmuffler.net bin-brokers.com
cartierring.xyz blacktwhite.com
casinorewards.info blog.axant.it
cfacarrosserie74.com blue-square.biz
check-this-out-now.online bristolhotel.com.ua
chelnytruck.ru bryansk.zrus.org
chimiver.info buyclomidonlaine.com
chocoslim.pro buyessay3.blogspot.ru
cialischmrx.com buzz-porno.info
clayvasedesigns.tk buzzsumo.com
clickintext.net calvet.altervista.org
contentsexpress.com candytiner.com
crest-poloski.ru canvas.gloverid.site
dayibiao.com carstrends2015.com
ddsoldes.fr cashonads.com
dentalpearls.com.au caulderwoodkennels.com
detoxmed24.ru cbb1smartlist12.click
dfwu1019.info cbozhe.com
disruptingdinnerparties.com cfsrating.sonicwall.com
dojki-hd.com cherrypointplace.ca
domainsatcost.com christianlouboutinreplica.pw
downloadeer.net cialiswithoutadoctor.net
downloader12.ru clickbanksites.info
ecxtracking.com clippingphotoindia.com
estibot.com clodo.ru
club-samodelkin.ru
co.lumb.co
compliance-brian.xyz
compliance-jack.top
conboy.us
copypaste.traderzplanet.in
craftinsta.ru
creativehutindia.com
customerguru.in
darkages.info
design-ideas.info
digest-project.ru
dimfour.com
doktoronline.no
elite-sex-finders.com
elvenar.com
emerson-rus.ru
eonpal.com
erotikstories.ru
escortslet.net
etur.ru etur.ru
eurosamodelki.ru extlinks.com
exci.ru f07.de
fickkontaktehobbyhuren.com filmgo.ru
findclan.org filmi-onlain.info
findpik.com firstsiteguide.com
free411games.com fitnesspiks.com
froggytube.com flytourisme.org
fsakhalin.ru free-deals.faith
fuckbuddybestgilf.info freza-sverlo.ru
game300.ru galleryawesome.com
gamedayhouse.com gamebackyard.com
gamerextra.com gaypornmovie.net
gamezblox.com get-free-traffic-now.com
gcup.ru girlsfuckdick.com
gdebestkupit.ru girlspicsa.com
ge0ip.net golaya.pw
getmyads24.com goodly.pro
getprismatic.com greatdealshop.com
gettpromos.com gsmtlf.ru
gfaq.ru guardlink.org
glass-msk.ru guruofcasino.com
gorodservis.ru hackers-crackers.tk
gsou.cf hard-porn.mobi
guigyverpo.cf hausfrauensex18.com
haikuware.com
hamilton.ca
hdapp1008-a.akamaihd.net
high-speed1.net high-speed1.net
highspeed5.net hitmuzik.ru
histock.info hotgirlhdwallpaper.com
hopeonthestreet.co.uk icaseclub.ru
howtowhitenteethfast.xyz ihc2015.info
hscsscotland.com imagui.eu
hvd-store.com immigrational.info
ifirestarter.ru in-tandem.co
iflycapetown.co.za infoupdate.org
igadgetsworld.com insta-add.pro
ihtec2019.org instasexyblog.com
iminent.com insurple.com
imperia31.ru intimshop-fantasy.ru
incomekey.net ispac.org
infowarcraft.ru itsdp3.com
investsuccess.org
ipchicken.com
itservicesthatworkforyou.com
jagg.info
japan-bearings.ru japan-bearings.ru
javidol.site jav-idol.com
jerseyschinabizwholesale.us jovencitas.gratis
kerch.site jpcycles.com
kinohit1.ru jrpmakati.com
lacloop.info kakablog.net
lakomka.com.ua kletkimehan.ru
lashstudia.ru konteiner24.com
lazymanyoga.com krasnodar.zrus.org
lkbennettstore.online kredit-blog.ru
longgreen.info ktotut.net
luckyday.world laapp.com
massage-info.nl laptop-4-less.com
maxxtor.eu larutti.ru
mcar.in.ua letslowbefast.site
megahdporno.net lider-zhaluzi.kiev.ua
metabar.ru lineshops.biz
mil-stak.com lockerz.com
mirobuvi.com.ua ltvperf.com
mixtapetorrent.com lyrster.com
mnogabukaff.net makemoneyonline.com
mobile.ok.ru master-muznachas.ru
morf.snn.gr medmajor.ru
mototsikl.org megavolt.net.ua
mybackgroundlandscape.blogspot.com memberty.com
micasainvest.com
minharevisao.com
moetomnenie.com
moi-glazki.ru
mompussy.net
monclerjacketsoutlet.pw
monetizer.com-01.site
moomi-daeri.com
moxo.com
mrvideospornogratis.xxx
my-floor.in.ua
myanyone.net
mydoctorok.ru mydoctorok.ru
natali-forex.com neobux-bg.info
nero-us.com nero-us.com
networkad.net networkcheck.xyz
nikhilbahl.com nvssf.com
noclegonline.info obiavo.uz
nullrefer.com onlythegames.com
o-o-8-o-o.com ooo-olni.ru
obnal.org oralsexfilme.net
oda.as ozoz.it
odywpjtw.bloger.index.hr
okeinfo.online
onstrapon.purplesphere.in
pacificair.com
page2rss.com page2rss.com
paramountmarble.co.uk
partner-host.men
partnerline.men
parvezmia.xyz parvezmia.xyz
pc4download.co pinapchik.com
pestomou.info
piuminiita.com
pixgood.com
pixshark.com pixshark.com
pizza-imperia.com pornogad.com
pk-pomosch.ru powc.r.ca.d.sendibm2.com
polimga.pw preg.marketingvici.com
pons-presse.com produkto.net
pornhubforum.tk proposal-engine.com
pornodojd.ru propranolol40mg.blogspot.com
pornonik.com prostitutki-almata.org
pornotubs.com psoriasis-file.trade
porto.abuilder.net pushdata.sendpulse.com
pospr.waw.pl pyrodesigns.com.au
prefersurvey.net quelle.ru
presleycollectibles.com quick-seeker.com
priscilarodrigues.com.br rada.ru
prizefestival.mobi ratemodels.net
pro-okis.ru resultshub-a.akamaihd.net
professionalwritingservices15.blogspot.ru reyel1985.webnode.fr
prokotov.com
prosperent.com
prpops.com
pulseonclick.com
puserving.com
putitin.me
qwesa.ru
raavidesigns.com
rapevideosmovies.com
rapidokbrain.com
rcpmda.ikan1080.xyz
realitykings.com
recipedays.com
refudiatethissarah.info
replica-watch.ru
resant.ru
rezeptiblud.ru rezeptiblud.ru
rollercoin.com rfd-split.hr
ru-mediaget.ru rfid-locker.co
rus-pornuha.com rimedia.org
rybalka-opt.ru ringporno.com
ryetaw.com sakhboard.ru
scanner-fred.top sarm.tk
scanner-margo.top saugatuck.com
seksvideoonlain.com savememoney.co.za
seoanalyses.com scanner-john.top
seogadget.ru scmor.ilxc.cc
scoopquest.com
search.1and1.com
seolab.top
sexkontakte-seite.com
sexreliz.com
shlyahten.ru
sierraapps.com
sim-service.net
similardeals.net similardeals.net
simon3.ru smailik.org
sindragosa.comxa.com solitaire-game.ru
slayerlife.com solution4u.com
sohoindia.net soviet-portal.do.am
srecorder.com spasswelt.net
startwp.org spidtest.org
sterva.cc spinazdrav.ru
stop-gepatit.te.ua superlist.biz
students-cheapskate.ml surffoundation.nl
swimpool.ca surflinksmedical.com
t3chtonic.com suture.co
tacbibirfa.tk thegalerie.eu
tandvardshuset.net
taraz.xkaz.org
tasteidea.com
telemetryverification.net
thebestweightlosspills.ovh
timetorelax.biz
tizanidineforopiatewithdrawal.blogspot.com tizanidineforopiatewithdrawal.blogspot.com
tizanidineinfo.blogspot.com tizanidinesideeffects.blogspot.com
tizanidinemusclerelaxant.blogspot.com toolsky.com
topcar-krasnodar.ru torrents-tracker.com
torontoplumbinggroup.com tuberkulezanet.ru
torrnada.ru typer.one
turbodsp.com uchetunet.su
turizm.bz uzporno.mobi
tvorozhnaja-zapekanka-recept.ru vacuumcleanerguru.com
twu.com.ua vacuumscleaner.com
u17795.netangels.ru vbtracker.net
ultimateclassicrock.com viagraneggrx.com
unlimitdocs.net viagroid.ru
unrealcommander.biz video-production.com
v24s.net videochat.cafe
vasileostrovsky-rayon.ru visa-china.ru
via-energy-acquistare.com vkontarkte.com
video-woman.com voloo.ru
vipsexfinders.com volume-pills.biz
vladhistory.com wallpapersdesk.info
vse-pesni.com watracker.net
w3javascript.com websiteexplorer.info
website-speed-up.site whatsappdownload10.com
weightatraining.com where-toget.com
wetgames.ru wildcattube.com
worldhistory.biz wleuaprpxuvr.ga
www.ehscloud.cn xn--80ahdheogk5l.xn--p1ai
x-mix.info youtubedownload.org
x-musics.com za-music.mymobiplanet.com
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai zaobao.com.sg
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai zebramart.ru
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai zeroredirect.com
xn--lifehacer-1rb.com zoomovies.org
yellowstonevisitortours.com zrelaya.pw
yogamatsexpert.com
zaim-pod-zalog-krasnodar.ru
zdesoboi.com
zeg-distribution.com
zoominfo.com
zoosexart.com
zvezdagedon.ru

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.247 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.06.3179 ### Version: V4.2022.06.3180
### Updated: Mon Jun 20 22:01:23 UTC 2022 ### Updated: Tue Jun 21 10:16:32 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.06.3179 ### Version: V4.2022.06.3180
### Updated: Mon Jun 20 22:01:23 UTC 2022 ### Updated: Tue Jun 21 10:16:32 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##