vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 10:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.10.2644 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-10-11 10:18:34 +00:00
parent 0f67059d18
commit 7d05ebcc37
48 changed files with 75825 additions and 75805 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2643 #### Version: V4.2021.10.2644
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2643 #### Version: V4.2021.10.2644
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2643 #### Version: V4.2021.10.2644
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9340
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2643 #### Version: V4.2021.10.2644
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9344
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9344
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

9344
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9344
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 1; 10.1.1.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 0; 10.1.1.84 0;
127.0.0.1 0; 127.0.0.1 0;

9344
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 1; 10.1.1.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 0; 10.1.1.84 0;
127.0.0.1 0; 127.0.0.1 0;

9344
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 1; 10.1.1.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 0; 10.1.1.84 0;
127.0.0.1 0; 127.0.0.1 0;

9344
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2643 2644

9344
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

9344
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.71.65.65 0; 20.94.205.253 0;

9344
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 1; 10.1.1.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 0; 10.1.1.84 0;
127.0.0.1 0; 127.0.0.1 0;

9344
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 1; 10.1.1.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 0; 10.1.1.84 0;
127.0.0.1 0; 127.0.0.1 0;

9344
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 1; 10.1.1.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 0; 10.1.1.84 0;
127.0.0.1 0; 127.0.0.1 0;

9344
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.71.65.65 0; 20.94.205.253 0;

9344
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.71.65.65 0; 20.94.205.253 0;

9344
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 1; 10.1.1.84 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
DittoSpyder Anarchy99
Go-Ahead-Got-It Asterias
JamesBOT Domains Project
JustView ExtractorPro
Nuclei Image Fetch
Re-re Msrabot
Searchestate
Siphon Siphon
Site Sucker Titan
x22Mozilla Web Sauger
Website Quester

296
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404enemy 01h4x.com
80legs 360Spider
AIBOT 404checker
ALittle Client
ASPSeek
Abonti Abonti
Aboundex
Aboundexbot
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
Aipbot AhrefsBot
Alexibot AiHitBot
Alligator Alligator
Anarchy Anthill
Anarchy99 Attach
BLEXBot AwarioRssBot
AwarioSmartBot
BBBike
BDFetch
BackDoorBot BackDoorBot
BackWeb BackWeb
Badass Backlink-Ceck
Bandit Battleztar Bazinga
BetaBot Bitacle
Black Hole Black Hole
BlackWidow
Blackboard
Blow Blow
BlowFish BlowFish
Boardreader Brandwatch
BotALot Bullseye
Brandprotect
Buddy
BuiltBotTough
BuzzSumo BuzzSumo
CATExplorador CODE87
CCBot CensysInspect
CheTeam CherryPicker
Chlooe Claritybot
Cloud mapping Cloud mapping
Cocolyzebot Copier
CopyRightCheck CopyRightCheck
Copyscape
Cosmos Cosmos
CrazyWebCrawler CrazyWebCrawler
Crescent Crescent
CrunchBot
Curious
Custo Custo
CyotekWebCopy
DBLBot
DIIbot DIIbot
DataCha0s DataCha0s
Disco DatabaseDriverMysqli
DomainSigmaCrawler Demon
Deusu
Devil
DnyzBot
DomCopBot
DomainAppender
DomainCrawler
DomainStatsBot DomainStatsBot
Domains Project Domains Project
Download Wonder Download Wonder
Dragonfly
Drip
ECCP/1.0
EMail Wolf EMail Wolf
Ecxi Ecxi
EirGrabber EirGrabber
Exabot ExtractorPro
ExtLinksBot
Extractor
EyeNetIE EyeNetIE
FemtosearchBot FemtosearchBot
FlashGet Firefox/7.0
Foobot
Freeuploader Freeuploader
FyberSpider FyberSpider
Fyrebot
GT::WWW
Genieo
GetRight
GetWeb
Getintent
Go!Zilla Go!Zilla
Go-Ahead-Got-It Gotit
GoZilla
GrabNet
Grabber
GrapeFX
GrapeshotCrawler GrapeshotCrawler
GridBot
HMView
HTMLparser HTMLparser
HTTP::Lite Haansoft
HaosouSpider HaosouSpider
Harvest
Havij Havij
Hloader HybridBot
IDBot IDBTE4M
IRLbot
Iblog Iblog
Id-search Id-search
IlseBot IlseBot
Image Fetch InfoNaviRobot
Image Sucker Intelliseek
IndeedBot
InfoTekies
InterGET InterGET
InternetSeer Internet Ninja
Iria
Iskanie
IstellaBot IstellaBot
Jbrofuzz Jbrofuzz
JennyBot JennyBot
Jetty Jetty
Jorgee
Jyxobot
Kenjin Spider Kenjin Spider
Kinza LWP::Simple
Lanshanbot Lanshanbot
Leap Larbin
LexiBot Lftp
Libwhisker
Lightspeedsystems Lightspeedsystems
LinkScan LinkScan
LinkWalker LinkpadBot
LinksManager LinqiaScrapeBot
LinqiaMetadataDownloaderBot MFC_Tear_Sample
Lipperhey Spider
Litemage_walker
Lmspider
MJ12bot MJ12bot
MQQBrowser MQQBrowser
MSIECrawler
MTRobot
Mag-Net Mag-Net
Magnet Majestic12
Mail.RU_Bot MarkMonitor
Majestic SEO
Majestic-SEO
MarkWatch
Mass Downloader
MauiBot MauiBot
Mb2345Browser Mb2345Browser
MegaIndex.ru
Microsoft Data Access Microsoft Data Access
Morfeus Fucking Scanner Microsoft URL Control
Mozlila
Msrabot Msrabot
Musobot Name Intelligence
NICErsPRO Nameprotect
Navroad Navroad
Needle
Nessus Nessus
Nettrack Net Vampire
NetAnts
NetLyzer
NetZIP
Netvibes Netvibes
Nibbler Niki-bot
NimbleCrawler NimbleCrawler
Nuclei Nimbostratus
Nutch
Octopus
Offline Explorer Offline Explorer
Openfind OnCrawl
Openvas OpenVAS
OrangeBot OrangeSpider
OutfoxBot PECL::HTTP
POE-Component-Client-HTTP PHPCrawl
Page Analyzer Page Analyzer
PageScorer PageAnalyzer
PageThing.com PageThing.com
Papa Foto Pandalytics
Pavuk Panscient
PeoplePal PeoplePal
Picscout
Pimonster Pimonster
Pockey Pockey
ProPowerBot ProPowerBot
Probethenet Pu_iN
Pump QueryN Metasearch
PyCurl RankActive
RankActiveLinkBot RankActiveLinkBot
RankingBot RankingBot
RankingBot2
ReGet ReGet
RealDownload Reaper
RebelMouse Ripper
RepoMonkey
RocketCrawler RocketCrawler
Rogerbot SBIder
SEOprofiler SEOkicks-Robot
SISTRIX SalesIntelligent
SMTBot Screaming
ScoutJet Searchestate
Scrapy
ScreenerBot
ScrepyBot
SearchmetricsBot
Semrush Semrush
SentiBot SeobilityBot
Seomoz Siphon
Shodan SiteCheckerBotCrawler
Site Sucker SiteLockSpider
SiteExplorer
SiteSnagger SiteSnagger
SiteSucker
Sitebeam Sitebeam
Sitevigil
Snake
Snapbot Snapbot
Snoopy Sociscraper
Sosospider Sogou web spider
Sottopop Sottopop
Spanner Spanner
SputnikBot SuperHTTP
Steeler Surfbot
SurveyBot
Suzuran
Swiftbot
Szukacz Szukacz
Telesoft Telesphorep
Telesphoreo
Thumbor Thumbor
TightTwatBot True_Robot
Titan
Toata
TurnitinBot TurnitinBot
TwengaBot
URLy.Warning
VCI VCI
Vacuum Vagabondo
Virusdie VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
VoidEYE VoidEYE
Voil Voltron
WASALive-Bot WASALive-Bot
WBSearchBot
WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WallpapersHD
WWWOFFLE Web Auto
Wallpapers Web Fetch
Wallpapers/3.0 Web Fuck
WeSEE
WebBandit WebBandit
WebEnhancer WebEnhancer
WebFuck WebFetch
WebLeacher WebImageCollector
WebReaper
WebStripper WebStripper
WebSucker WebSucker
WebZIP WebZIP
Website Quester Webalta
WebsiteExtractor WebmasterWorldForumBot
Whack
Whacker Whacker
Whatweb
Who.is Bot Who.is Bot
Widow
WinHTTrack
Wonderbot Wonderbot
Wprecon Woobot
Zitebot Xaldon_WebSpider
Zauba
ZoomBot ZoomBot
ZoominfoBot
ZumBot
ZyBorg
backlink-check backlink-check
cah.io.community cah.io.community
check1.exe check1.exe
coccocbot-web
cognitiveseo cognitiveseo
com.plumanalytics com.plumanalytics
crawler.feedback demandbase-bot
crawler4j domainsproject.org
evc-batch evc-batch
gopher instabid
heritrix ips-agent
isitwp.com isitwp.com
meanpathbot lwp-trivial
plumanalytics magpie-crawler
netEstate NE Crawler
pcBrowser
polaris version polaris version
probe-image-size ripz
s1z.ru satoristudio.net
scalaj-http scalaj-http
seoscanners seoscanners
sexsearcher
sitechecker.pro sitechecker.pro
sogouspider
spyfu
tAkeOut
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webmeup-crawler webmeup-crawler
x22Mozilla
zauba.io
zgrab

308
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404checker 404checker
ADmantX 404enemy
AIBOT
ASPSeek ASPSeek
Aboundexbot Aboundex
Acunetix Acunetix
AfD-Verbotsverfahren Aipbot
AhrefsBot Ankit
Alexibot Apexoo
AlphaBot Asterias
Anthill
Attach Attach
BDFetch AwarioRssBot
AwarioSmartBot
BackWeb BackWeb
Bandit Badass
Black Hole Barkrowler
Blackboard BatchFTP
Bolt Bitacle
Brandwatch Boardreader
Brandprotect
Buck Buck
Buddy BuiltBotTough
BuiltWith
Bullseye Bullseye
BunnySlippers BunnySlippers
Calculon BuzzSumo
CazoodleBot CATExplorador
CCBot
CODE87
Cegbfeieh Cegbfeieh
CensysInspect CheeseBot
CheTeam
CherryPicker
Chlooe Chlooe
Cliqzbot Claritybot
Cocolyzebot
Collector
Copier
CopyRightCheck CopyRightCheck
Copyscape Copyscape
Cosmos Cosmos
Craftbot Craftbot
CrazyWebCrawler Crawling at Home Project
Crescent Crescent
CrunchBot Curious
DIIbot DBLBot
DSearch DTS Agent
DatabaseDriverMysqli DataCha0s
Demon Devil
Dirbuster
Discobot
Discoverybot Discoverybot
Dispatch DomCopBot
DnyzBot
DomainAppender DomainAppender
DomainCrawler
DomainStatsBot
Domains Project
Dotbot
Download Wonder Download Wonder
Dragonfly ECCP/1.0
Drip Ecxi
EMail Wolf Evil
EirGrabber
Exabot Exabot
Express WebPictures
ExtractorPro ExtractorPro
FDM FDM
Firefox/7.0 FHscan
FemtosearchBot
FlashGet FlashGet
Flunky Flunky
Foobot
Freeuploader Freeuploader
FrontPage G-i-g-a-b-o-t
Fyrebot GalaxyBot
GT::WWW Genieo
Getintent GetRight
Gotit Go!Zilla
Go-Ahead-Got-It
GoZilla
GrabNet
Grafula Grafula
GrapeFX GridBot
HTTP::Lite HTMLparser
Haansoft
HaosouSpider
Harvest Harvest
Hloader Havij
IDBot Image Fetch
IRLbot
Iblog
IlseBot
Indy Library Indy Library
InfoNaviRobot
InterGET InterGET
Iria Internet Ninja
JamesBOT JamesBOT
JetCar JikeSpider
Jetty Joomla
Jorgee
JustView JustView
Jyxobot
Kenjin Spider Kenjin Spider
Keyword Density
Kinza Kinza
Lanshanbot Kozmosbot
LWP::Simple
Leap Leap
LeechFTP
LeechGet LeechGet
LibWeb
LieBaoFast LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkpadBot Likse
Linkbot
Linkdexbot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider Lipperhey
Lmspider Litemage_walker
MFC_Tear_Sample MJ12bot
MIIxpc MQQBrowser
MSIECrawler Mail.RU_Bot
Magnet Majestic12
Majestic SEO Mb2345Browser
Majestic-SEO
Mass Downloader
Masscan
MauiBot
MeanPath Bot MeanPath Bot
Meanpathbot MegaIndex.ru
Metauri Metauri
MicroMessenger
Microsoft URL Control Microsoft URL Control
Mister PiX Mister PiX
Moblie Safari Mojolicious
Mr.4x3
Msrabot Msrabot
Musobot NPbot
NICErsPRO Name Intelligence
Navroad Net Vampire
NearSite NetLyzer
NetMechanic NetMechanic
Netcraft NetZIP
Netvibes
Nibbler Nibbler
NimbleCrawler Nikto
Nimbostratus Ninja
Nmap
Octopus Octopus
Offline Explorer Offline Explorer
OpenVAS
Openvas Openvas
OrangeSpider OutclicksBot
PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer PageAnalyzer
PageScorer
Pandalytics Pandalytics
Panscient
Petalbot Petalbot
Picsearch
PictureFinder PictureFinder
Pixray
PleaseCrawl PleaseCrawl
Pockey Pockey
Pump Pu_iN
PxBroker PyCurl
RankFlex QueryN Metasearch
Re-re Quick-Crawler
Recorder RSSingBot
RocketCrawler RankActive
SEOkicks-Robot RankActiveLinkBot
Reaper
RepoMonkey
SEOkicks
SEOlyticsCrawler SEOlyticsCrawler
SalesIntelligent SEOprofiler
SMTBot
Scrapy
ScrepyBot ScrepyBot
Searchestate Semrush
SentiBot SemrushBot
SeoSiteCheckup SeoSiteCheckup
Seomoz Seomoz
Shodan Siphon
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
Siteimprove SiteSucker
Sitevigil Sitevigil
SlySearch SlySearch
SmartDownload SmartDownload
Sosospider SocialRankIOBot
Sociscraper
Sottopop
SpaceBison SpaceBison
Spammen Spammen
SpankBot
Spbot Spbot
SputnikBot Steeler
Sqlworm Stripper
Sucuri
SuperBot SuperBot
SuperHTTP T8Abot
Surfbot Teleport
Suzuran
T0PHackTeam
TeleportPro
Telesphorep
The Intraformant The Intraformant
TheNomad TheNomad
Titan Thumbor
Toweyabot Trendiction
Tracemyfile
Trendictionbot
True_Robot True_Robot
TurnitinBot Turingos
URLy Warning
URLy.Warning
UnisterBot
Upflow Upflow
VB Project
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler WBSearchBot
Virusdie
VoidEYE
WISENutbot WISENutbot
WPScan WPScan
WWW-Mechanize WWW-Collector-E
WWWOFFLE WWWOFFLE
Wallpapers
Wallpapers/3.0
WallpapersHD WallpapersHD
Web Collage Web Collage
Web Pix Web Pix
WebCollage Web Sucker
WebCopier WebAuto
WebLeacher WebFetch
WebImageCollector
WebSucker WebSucker
WebZIP WebWhacker
WebmasterWorldForumBot Webalta
Webshag Webshag
Website Quester WebsiteQuester
WebsiteExtractor
Webster Webster
Whack Whack
Widow Whacker
Woobot Who.is Bot
WinHTTrack
Wonderbot
Wotbox Wotbox
Wprecon Xaldon WebSpider
Xenu
YoudaoBot
Zade Zade
Zauba Zauba
Zermelo
Zeus
Zitebot Zitebot
ZoomBot
ZoominfoBot ZoominfoBot
ZyBorg ZumBot
arquivo-web-crawler arquivo.pt
cognitiveseo cah.io.community
dataforseo.com check1.exe
domainsproject.org clark-crawler
eCatch coccocbot-web
gopher com.plumanalytics
crawl.sogou.com
crawler.feedback
evc-batch
facebookscraper
instabid instabid
ips-agent internetVista monitor
lwp-trivial meanpathbot
mediawords
muhstik-scan muhstik-scan
polaris version oBot
ripz ripz
s1z.ru s1z.ru
satoristudio.net
scalaj-http
scan.lol scan.lol
seobility
sitechecker.pro sitechecker.pro
spyfu siteripz
sp_auditbot
sysscan
trendiction.com trendiction.com
trendiction.de webpros.com
webprosbot webprosbot
x09Mozilla
x22Mozilla x22Mozilla
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
alyeskaresort.com 3xforum.ro
ausmepa.org.au cardul.ru
escort-russian.com compliance-willy.top
ezigaretteshop.webs.com coolwallpapers4k.info
gdzkurokam.ru fotoxxxru.com
hvd-store.com hard-porn.mobi
jjbabskoe.ru liran-locks.com
macnewtech.com macdamaged.space
musezone.ru navalwiki.info
znakom.sibtest.ru toy-shop.top

928
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1000n1.ru 00go.com
24videos.tv 08800.top
3rbseyes.com 1-free-share-buttons.com
45en.ru 101flag.ru
4k-player.pl 1kdailyprofit.me
70casino.online 1pamm.ru
7yue.org 1qingdao.com
83net.jp 2345.com
991mostfm.co.id 28n2gl3wfyb0.ru
abbanreddy.com 2girls1cup-free.com
above.com 5-steps-to-start-business.com
ad-words.ru 5u.com
adanih.com 883zy.com
addbin.men 9icmzvn6.website
adidas.frwebs.fr a14download.com
adimmix.com acc.eu.org
adobereader-free.ru acortarurl.es
adserver-e7.com addblueoff.com.ua
adsfresh.men adf.ly
adtology.com ads-seo.men
agysya.ru afslankpillen2017nl.eu
alfapro.ru agahinameh.com
all-number.com akita.kz
alldezservice.kz albamargroup.com
alldownload.pw alessandraleone.com
alphavisions.net allbizne.co.ua
altamayoztourism.com allmarketsnewdayli.gdn
ameblo.top apiluckyleapnet-a.akamaihd.net
amoi.tn apisecretsaucebi-a.akamaihd.net
analytics-ads.xyz
androids-store.com
animaltoplist.com
animenime.ru
apibetweenlinesn-a.akamaihd.net
apidigidocketnet-a.akamaihd.net
apilinkswiftco-a.akamaihd.net
apishops.ru
appmsr.org appmsr.org
arabgirls.adultgalls.com
arclk.net
arenanews.com.ua arenanews.com.ua
atlasvkusov.ru arendakvartir.kz
autoblogger24.info asophoto.com
autovideobroadcast.com atlant-auto.info
avramstroy.ru autonew.biz
azadnegar.com autotrends.today
azlex.uz aviav.co
b-styles.xyz aviav.ru
babosas.com aviav.ru.com
babyfactory.fr avon-ukraine.com
babyguns.ru ayanaspa.com
bala.getenjoyment.net ayerbo.xhost.ro
bang-hotties.com azartclub.org
barbourjackets.online babieca.com
basedecor.ru backiee.com
belinvestdom.by banki76.ru
beauty-bracelet.com
belstaffstore.online belstaffstore.online
best-deals-products.com beremenyashka.com
bestofpicture.com best-mam.ru
bestwebsiteawards.com best-seo-offer.com
biketank.ga best-seo-software.xyz
bin-brokers.com bitcoinpile.com
bio-japan.net
bjetjt.com bjetjt.com
bjorkbacken.nu bongacams.com
blavia.00author.com bouda.kvalitne.cz
bloggedporn.com bpro1.top
bloggerads.net brainzpod.win
blogsfunky672.weebly.com breastaugmentation.co.za
bonanza-fortune.men brg8.com
bookhome.info brianhenry.co
braip.com.br brillianty.info
brimstonehillfortress.org bristolhotel.com
buckspar.gq btc4u.ru
bumble.cheapwebsitehoster.com buehne-fuer-menschenrechte.de
burkesales.com buttons-for-website.com
bus-offer.com buzz-porno.info
buy-cheap-online.info bycontext.com
buynorxx.com callawaygolfstore.online
buyviagraa.blogspot.com callcafe.info
cakemediahq.com.au cam-kontakte.org
cakesplus.com.au cartierlove.xyz
camdenmemorials.com celejihad.info
cardsdumps.com chcu.net
carloans.com cheap-pills-norx.com
carsplate.com cheapbarbour.online
cartierlovestore.xyz cheapjerseysa.com
castingbank.ru checkhit.com
caulderwoodkennels.com chiptuninger.com
cdncash.com chocolateslim-en-espana.com
cdncash.net choosecuisine.com
cellfun.mobi christianlouboutinreplica.win
cementaresearch.se cleanpcnow.com
checktext.org clickaider.com
cherrypointplace.ca clickbanksites.info
chiblackhawks-jerseys.com clickhype.com
children-learningreading.info clknsee.com
chimiver.info clothesforcash.com
chocoslim.pro cobaltpro.ru
ci.ua cocolyze.com
cibpenonptib.flu.cc coinsspb.com
citizenclsdriveri7.pen.io collegeessay19.blogspot.ru
cleanersoft.com comeondog.info
click2pawn.com compiko.info
clickcash.com
compliance-alex.xyz compliance-alex.xyz
compliance-barak.top compliance-elena.xyz
compliance-george.top compucelunlock.net
conboy.us connectionstrenth.com
contextualyield.com cookie-law-enforcement-hh.xyz
coprofam.org cordstrap.cc
copyrightclaims.org costablue.xyz
craftinsta.ru creativehutindia.com
cranly.net creatives.adbetclickin.pink
customboxes4less.com cscwtalkto.site
customerguru.in customchocolate.business-for-home.com
cutt.us customcollegeessays.net
cymbaltaandtizanidine.blogspot.com damedingel.ya.ru
decorationspcs.com datadepths.com
digitalassetmanagement.site ddsoldes.fr
dildofotzen.net decenttools.com
diminishedvalueoforegon.com dentalpearls.com.au
dispo.de detki-opt.ru
disruptingdinnerparties.com dfwu1013.info
dlya-android.org diegolopezcastan.com
dneprsvet.com.ua discovertreasurenow.com
dood.live domainanalyzing.xyz
doreenblog.online domaincdn.xyz
dosug-lux.ru domznaniy.ru
dotmass.top dotnetdotcom.org
doublepimp.com dscaas.website
download-wallpaper.net ebooknovel.club
dstroy.su ecookna.com.ua
dvd-famille.com ednorxmed.com
e-commerce-seo.com eldiariodeguadalajara.com
e705.net electricwheelchairsarea.com
e90post.com elektrischezigarette1.onsugar.com
eachdayisagift.review elektrozigarettekaufen2.blogse.nl
earthmagic.info elektrozigaretten1.tumblr.com
educatemetv.com elitepcgames.com
ee77ee.com
elektrischezigarettekaufen2.cowblog.fr
elektrozigarette2.webs.com
elvenar.com elvenar.com
empathica.com
empirepoker.com empirepoker.com
enbersoft.com
engines-usa.com
englate.com englate.com
erotag.com enternet.ee
essay-edu.biz erectile.bid
etotupo.ru ero-advertising.com
evrotekhservis.ru erotikstories.ru
exhibitionplus.eu estibot.com
exoclick.com everflora.ru
expertnaya-ocenka.ru expresstoplivo.ru
extremez.net extlinks.com
fanoboi.com ezigaretteshop.webs.com
fastcrawl.com factorynightclub.com
fatmaelgarny.com falcoware.com
faz99.com fantasticpixcool.com
fetroshok.ru farm26.ru
fikasound.tk farmprofi.net
filmi-onlain.info filkhbr.com
findclan.org filmfanatic.com
findpics.pw financeloan.us
finejewelryshop.ru financepoints.eu
fireads.men flashlarevista.com
flowersbazar.com floating-share-buttons.com
forexgb.ru flowwwers.com
foxinsocks.ru flprog.com
frbizlist.com forensicpsychiatry.ru
freeuploader.ml forex.osobye.ru
fsalas.com forum.tvmir.org
fxund.us forums.toucharcade.com
fotoxxxru.com
fr-bearings.ru
free-share-buttons.blogspot.com
free-today.com
funnymama.com
fym.com.ru fym.com.ru
game-top.su fz139.ttk.ru
gavuer.ru getdot.ru
gaz-voshod.ru getoutofdebtfree.org
geileweiber.tk glass-msk.ru
generousdeal-a.akamaihd.net glavprofit.ru
gfaq.ru gogps.me
gifspics.com goldenggames.com
gktt.ru gotorussia.com
go2mike.ru gov.yanao.ru
goodhumor24.com gsasearchengineranker.space
gotomontenegro.net gwebtools.com
gourcy.altervista.org hchha.com
gox.com.ua healthcarestore.info
gpms.org.my hentaiheroes.com
gwhwpxbw.bloger.index.hr
hahashka.ru
heatpower.ru
hermesbelts.xyz hermesbelts.xyz
high-speed1.net hetmanship.xyz
hmmm.cz hidefiles.org
hollywoodactress.info homeart.space
homemature.net host-protection.com
hotblog.top hotgirlhdwallpaper.com
hotel-mkad.ru
housekuba.org
houtings.xyz
howlongdoestizanidinestayinyoursystem.blogspot.com
hoztorg-opt.ru
htmlcorner.com htmlcorner.com
humbmosquina.tk hunterboots.online
ico.re hunthillfarmtrust.org
idc.com.ua iamsport.org
iflycapetown.co.za ideashome.id
ihtec2019.org ifirestarter.ru
igadgetsworld.com
iideaidekonkatu.info iideaidekonkatu.info
ilte.info ilmen.net
imfamous.info importchinacoach-teach.com
immigrational.info infobabki.ru
incolors.club insomniagamingfestival.com
infektsii.com
insider.pro
insta-add.pro
int.search.tb.ask.com
interesnie-faktu.ru
internetproviderstucson.com
intimshop-fantasy.ru
investyb.com
invitefashion.com invitefashion.com
isistaylorporn.info inzn.ru
izhevsk.zrus.org ip-guide.com
jav-way.site istizanidineanopiate.blogspot.com
jerseychinabizwholesale.com ivearchenceinflu.cf
jillepille.com jav-fetish.com
jose.mulinohouse.co jo24news.com
jurajskie.info jofucipiku.tk
kabbalah-red-bracelets.com joingames.org
kadashihotel.com kakablog.net
karting196.ru kamorel.com
kashubadesign.ru kanimage.com
kazan.xrus.org kicknights.gq
keyhantercume.com kinobaks.com
keywordblocks.com kinoplen.ru
kfon.eu kletkimehan.ru
kinky-fetishes.com krasnodar-avtolombards.ru
kino-rating.ru kvartir-remont.biz
klejonka.info lakomka.com.ua
komputers-best.ru landmania.ru
konturkrasoty.ru larose.jb2c.me
kosova.de
kupit-adenu.ru
ladsup.com
lalalove.ru
lanadelreyfans.us
landscapebackgrounds.blogspot.com
larutti.ru larutti.ru
lightinghomes.net leadwayau.com
linkredirect.biz lecbter-relationships.ga
lechtaczka.net
ledis.top
lemon-ade.site
lernur.net
letslowbefast.today
lincolntheatre.com
lineavabit.it
linkbux.ru
linksharingt.com linksharingt.com
lion.bolegapakistan.com littlesunraiser.com
lizace.com live-cam6.info
livejasmin.com
ljusihus.se ljusihus.se
louboutinreplica.xyz longadventure.com
lsex.xyz lostfilm-online.ru
m0r0zk0-krava.ru louboutinshoes.xyz
lovasszovetseg.hu
lovi-moment.com.ua
m1media.net
m292.info m292.info
magazin-pics.ru mafa-free.com
magicalfind-a.akamaihd.net mararoom.ru
maranbrinfo.com.br marblestyle.ru
marcoislandvacations.net marcogrup.com
masserect.com martlinker.com
mastershef.club massage-info.nl
mcadamssupplyco.com maxthon.com
mcar.in.ua mbiologi.ru
mearns-tractors.co.uk meccadumps.net
mediaoffers.click meendo-free-traffic.ga
medkletki.ru megaindex.ru
mesto-x.com mensandals.xyz
metarip.ru message-warning.net
mhi-systems.ru metallosajding.ru
micasainvest.com mexicosleevegastrectomy.com
michellblog.online mhtr.be
mielec.pl
mikrobiologies.ru
mil-stak.com mil-stak.com
milblueprint.com minet.club
minharevisao.com miniads.ca
minneapoliscopiers.com mirobuvi.com.ua
missis.top moesonce.com
moesen-ficken.com mojpregled.com
moncleronline.xyz moncleronline.xyz
moncleroutletonline.xyz
moneyteam24.com moneyteam24.com
moscow.xrus.org moomi-daeri.com
morefastermac.trade
moscow.online-podarki.com
most-kerch.org
mostantikor.ru
moviezbonkerssk.cf moviezbonkerssk.cf
mrbitsandbytes.com mozello.ru
mrwhite.biz mp3downloadhq.com
music.utrolive.ru mp3ringtone.info
musicdaddy.net msfsaar.de
myfreetutorials.com mtmtv.info
mygameplus.ru muabancantho.info
myhitmp3.club muizre.ru
mylesosibirsk.ru muschisexbilder.com
myperiod.club myonigroup.com
najaden.no myshopmatemac.com
nakrutka.cc na-telefon.biz
narkologiya-voronezh.ru narco24.me
nerudlogistik.ru nastydollars.com
nfvsz.com nationalbreakdown.com
nhl09.ru naturalshair.site
nightvision746.weebly.com netcheckcdn.xyz
noscrapleftbehind.co new7ob.com
notebook-pro.ru newstraveller.ru
novosibirsk.xrus.org next-dentists.tk
nuit-artisanale.com nfljerseysforsalewholesaler.com
nyfinance.ml ngps1.ru
obiavo.net nibbler.silktide.com
oddamzadarmo.eu ningessaybe.me
offtopic.biz nodding-passion.tk
okel.co nofreezingmac.work
oksrv.com npoet.ru
openfrost.net nrjmobile.fr
optikremont.ru nw-servis.ru
orel-reshka.net o-o-8-o-o.com
owathemes.com odesproperty.com
oxford-book.com.ua offer.wpsecurity.website
ohmyrings.com
ok-ua.info
olvanto.ru
omoikiri-japan.ru
onetravelguides.com
online247.ml
onlythegames.com
open-odyssey.org
openmediasoft.com
osagonline.ru
otbelivanie-zubov.com
ownshop.cf
paintingplanet.ru paintingplanet.ru
partner-stop.men paphoselectricianandplumber.com
partybunny.ru partnerads.men
pawli.eu
pdn-4.com
peekyou.com
pendelprognos.se pendelprognos.se
perso.wanadoo.es perm-profnastil.ru
pizza-tycoon.com phpdevops.com
pk-pomosch.ru pictures-and-images.com
plusnetwork.com picturesboss.com
pojdelo.weebly.com pintattoos.com
pons-presse.com pinup-casino1.ru
popugauka.ru pistonclasico.com
pornmania.pl pix24x7.com
pornolook.net pochemychka.net
portadd.men pochtovyi-index.ru
postclass.com pokerniydom.ru
potoideas.us pomoc-drogowa.cba.pl
potolokelekor.ru porno-raskazy.ru
powc.r.ca.d.sendibm2.com porno-transsexuals.ru
predominant-invent.tk pornoforadult.com
pricheskaonline.ru pornosexrolik.com
printdirectforless.com poshiv-chehol.ru
priscilarodrigues.com.br pozdrawleniya.com
privat-girl.net predmety.in.ua
privatefx-in.ru pretty-mart.com
pro-poly.ru privhosting.com
proekt-gaz.ru productarium.com
projefrio.com.br professionalwritingservices15.blogspot.ru
prombudpostach.com.ua profit-opportunity.com
prosmibank.ru promalp-universal.ru
proxyelite.biz propranolol40mg.blogspot.com
pshare.biz prostitutki-kharkova.org
pst2017.onlinewebshop.net psoriasis-file.trade
pukaporn.com pssucai.info
purplestats.com ptr.ruvds.com
q-moto.ru pushdata.sendpulse.com
questionmarque.ch puzo2arbuza.ru
qwarckoine.com qpypcx.com
rabotaetvse.ru raschtextil.com.ua
rangjued.com recipedays.ru
rapevideosmovies.com redirectme.net
rapidokbrain.com reeyanaturopathy.com
rczhan.com responsinator.com
realting-moscow.ru ring4rhino.com
rebuildermedical.com rniaeba.ga
reckonstat.info rocketchange.ru
refererx.com rockingclicks.com
registrysweeper.com royal-betting.net
relatodelpresente.com.ar rucrypt.com
remont-mobile-phones.ru runstocks.com
rentehno.ru ruscopybook.com
rn-to-bsn.com russia-tao.ru
roleforum.ru
rootandroid.org
ru-dety.ru
rukino.org
ruscoininvest.company
rusvideos.su rusvideos.su
s.lollypopgaming.com rybalka-opt.ru
sabaapress.com sady-urala.ru
sanidumps.com salmonfishingsacramentoriver.com
satellite.maps.ilovevitaly.com sanyuprojects.com
sbprabooks.com saratov.xrus.org
scalerite.co.za sashagreyblog.ga
scrapy.org satoristudio.net
sdelatmebel.ru screpy.com
seansonline24.pl se.bnt-team.com
search-error.com search.pch.com
seinterface.com securityallianceservices.com
seotoolsagency.com seogadget.ru
servethis.com seonetwizard.com
servicecenter.co.ua seorankinglinks.us
sergiorossistore.online
sevenstars7.com
sex-dating.co
sex-tracker.de
sex.hotblog.top
sexcamamateurchat.com
sexpornotales.com sexpornotales.com
sexy.babes.frontend-stack.top
sfd-chess.ru sfd-chess.ru
shemale-sex.net shakhtar-doneck.ru
shmetall.com.ua sharebutton.org
shodanhq.com
shoesonlinebuy.xyz shoesonlinebuy.xyz
shopcheermakeup.info sideeffectsoftizanidine.blogspot.com
shtaketniki.kz signal03.ru
sibecoprom.ru siha.de
sky-mine.ru simplesite.com
sluganarodu.ru sitopreferito.it
slutloadlive.com skanninge.se
smadihome.com slonechka.ru
smokewithrabbits.com snjack.info
sobecjvuwa.com.ru social-search.me
socialmonkee.com soft-terminal.ru
songoo.wz.cz songoo.wz.cz
sport-video-obzor.ru sotkal.lark.ru
stair.registrydomainservices.com splendorsearch-a.akamaihd.net
sports-supplements.us
spy-sts.com
ssconstruction.co
stairliftstrue.com stairliftstrue.com
startwp.org starwars.wikia.com
statustroll.com stathat.com
steelmaster.lv stromerrealty.com
sterva.cc superkanpo.com
storehouse.ua t3chtonic.com
streha-metalko.si tako3.com
stroy-portal22.ru td-l-market.ru
studentguide.ru telefonsex-ohne0900.net
sugarkun.com telefonsexi.com
sugvant.ru tgtclick.com
superfish.com the-torrent-tracker.blogspot.com
svetodiodoff.ru thecoolimages.net
szucs.ru theguardlan.com
teenforporn.com thepantonpractice.co.uk
teenporn18.net thewebsitetemplate.info
terrafootwear.us thin.me.pn
thecoral.com.br tizanidine4mgprice.blogspot.com
tiens2010.ru tizanidine4mguses.blogspot.com
tizanidineandhydrocodone.blogspot.com
tizanidineinteractions.blogspot.com tizanidineinteractions.blogspot.com
tizanidinemusclerelaxant.blogspot.com tizanidinepills.blogspot.com
tizanidinerestlesslegsyndrome.blogspot.com tnaionline.org
tnctrx.com topdownloads.ru
tomatis.gospartner.com topshef.ru
topappspro.com torrentdownloadhub.com
torrnada.ru
toxicwap.com toxicwap.com
toys.erolove.in traffic2cash.org
tpu.ru tsan.net
traffic2money.com tuckpointingmasonrysystems.com
triplepanda.xyz turn-up-life.life
trucri.me tvnewsclips.info
trustaffs.com tvoystartup.ru
trustedmaccleaner.com
try-rx.com
ttrraacckkrr.com
tubeline.biz
tupper-shop.ru
tversvet.ru
twinderbella.com
twitlinks.com
tzritel.tk
ua-company.ru ua-company.ru
uchil.net ufa.xrus.org
ultimateclassicrock.com unisexjewelry.org
uprour.com unrealcommander.com
uptime-delta.net uptime-alpha.net
urldelivery.com ural-buldozer.ru
vavilone.com usbggettwku.ga
viagra.pp.ua valuado.com
videogamesecrets.com vancleefreplica.pw
videooko.weebly.com vandrie-ict.nl
vchulkah.net
velpanex.ru
videositename.com
videtubs.pl videtubs.pl
vip2ch.com virtualbb.com
vipsexfinders.com vkonche.com
vipsiterip.org
vizag.kharkov.ua
vpnmouse.com vpnmouse.com
vsdshnik.com vsexkontakte.net
vvon.co.uk vzlomtw.com
vzglyadriv.kg w3data.co
wait3sec.org websites-reviews.com
wallpaperstock.net weburok.com
watchinf.com weprik.ru
watracker.net whiteelephantwellington.com
webalan.ru wildcattube.com
webenlace.com.ar wladimirpayen.com
webinstantservice.com worldhistory.biz
webradiology.ru wowas31.ucoz.ru
wesharepics.info wttavern.com
word-vorlagen.net www.arenda-yeisk.ru
wowcasinoonline.ooo www.labves.ru
www.souvenirua.com www.solartek.ru
xboxster.ru wyniki-lista.pl
xcc24.pl x-mix.info
xerox-douglas.cf x69ty.ru
xn----ctbigni3aj4h.xn--p1ai xclicks.net
xxart.ru xedserver.com
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xn----7sbaphztdjeboffeiof6c.xn--p1ai
xn--80aanaardaperhcem4a6i.com
xn--80ak6aa92e.com
xsfetish.org
xxxhdvideo.site
y8games-free.com
yachts-cruise.info yachts-cruise.info
yogamatsexpert.com yellocloud.be
yougetsignal.com youandcredit.ru
yourads.website youbrainboost.asia
yoursite.com youradexchange.com
yousense.info z-master.ru
zakazfutbolki.com zebramart.ru
zolotoy-lis.ru zed21.net
zixizop.net.ru
zona-aqua.ru zona-aqua.ru
zophim.me zrus.org
zryydi.com
zs2vm.top
zvetki.ru

488
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123movies.love 0c47f8422d3f.com
12u.info 100dollars-seo.com
13tabs.com 5ws.dating-app.ru
273-fz.ru 70casino.online
40cg.com 7fon.ru
4replicawatch.net 7minuteworkout.com
51unlim.ru 8gold.com
5i2.net aarbur.com
5tey7463.weebly.com adnotbad.com
883zy.com ads.tremorhub.com
adobereader-free.ru advertisingtag.net
aerogo.com adzerg.com
agysya.ru afslankpillen2017nl.eu
ahrefs.com amazingpic.net
aibolita.com apccargo.com
all-number.com apelsinnik.website
all4invest.ru apibrowsesmartne-a.akamaihd.net
allknow.info apimegabrowsebiz-a.akamaihd.net
allsilver925.co.il apiwoodensealcom-a.akamaihd.net
am-se.com appmsr.org
arcteryxsale.online artefakct.com
arenda-yeisk.ru articlesdirectoryme.info
asrv-a.akamoihd.net artparquet.ru
atout-energie-69.com asiengirls.net
aviav.ru.com autochoixspinelli.com
azbukafree.com autogrep.ru
backgroundpictures.net b-styles.xyz
bankcrediti.ru backlinkwatch.com
batanga.net bala.getenjoyment.net
bayareaaeroclub.org battle.net
best-seo-solution.com bingoporno.com
big-cash.net biznesschat.net
bikini-image.com blockworld.ru
bimmerpost.com bloggers.nl
blogqpot.com bmw-ark.ru
board.f00d.de bochemit.com.ua
bonzbuddy.com bolegapakistan.com
boo-arts.com bonzybuddy.com
brandov.ru brainboostingsupplements.org
buron.pw braindaily.xyz
cacheimages.com bugshoot.cn
candypeople.se cdn.walkme.com
cartierlove.xyz celejihad.info
cayado.snn.gr cheap-pills-norx.com
cete.ru coindirect.io
charmstroy.info compliance-george.xyz
chcu.net cookie-law-enforcement-hh.xyz
cheapestjerseysonwholesale.com d-black.bz
chelyabinsk.xrus.org damasarenai.info
chidporn.com detskie-zabavi.ru
chocoslim.pro dfiles.me
cleanpcnow.com digitalassetmanagement.site
corta.co digitalfaq.com
countbertwistdisp26.soup.io directrev.com
crackzplanet.com djonwatch.ru
custodieva.ru djstools.com
dawlenie.com ednorxmed.com
dealitright.click
dickssportinggoods.com
disruptingdinnerparties.com
doeco.ru
domain2008.com
dostavka-v-krym.com
e2click.com
ege-russian.ru
egvar.net egvar.net
elektronischezi.livejournal.com ekspresihati.info
elmacho.xyz elektrischezi.canalblog.com
emergencyneeds.org elitepcgames.com
etenininrade.ga elitesportsadvisor.com
eu-cookie-law.info eshop4u.jp
event-tracking.com essayassist.com
failingmarriege.blogspot.com expertblog.info
fanrto.com extremepornos.net
farmprofi.net ezaz.info
fdzone.org fashion-mk.net
fealq.com fettefrauen.net
flashahead.info findpics.pw
forex-indextop20.ru finemanteam.com
forsex.info forpostlock.ru
fotoxxxru.com free-fb-traffic.com
freshmac.space freevpn.space
fuck-paid-share-buttons.xyz fuckingawesome.com
gallery.rennlist.com fx-brokers-review.com
gamedayassist.com g33.org
gameprimary.com gay-file.com
gateway.zscalerone.net gdebestkupit.ru
gazoblok.net.ua ggiaro.com
ge0ip.com gne8.com
girlsatgames.ru gok-kasten.net
goodprotein.ru google-liar.ru
gpms.org.my goroda-vsego-mira.ru
gungamesz.com greatidea.marketing
gunsvicceadadebt.tk gsasearchenginerankerhelp.com
gwhwpxbw.bloger.index.hr heliko.no
hebr.myddns-flir.com homedecorpicture.us
honyaku.yahoofs.jp host-protection.com
hothot.ru host-tracker.com
hotloans.ru hotkeys.com
hoverboard360.es hoverboard360.at
hoverboard360.se igithab.com
huhn.altervista.org iinstalll-fii1leis.jus0wil.pp.ua
i-hobot.ru imgata.com
ikearugs.xyz impisr.edunsk.ru
incanto.in.ua intimshop-fantasy.ru
insomniagamingfestival.com ivanstroi.ru
investingclub.ru iwantmyfreecash.com
iqoption-bin.com jackpotchances.com
ispac.org jasonpartington.com
it-max.com.ua jav-fetish.com
jav-fetish.site kadashihotel.com
jerseychinabizwholesale.com kerwinandcariza.com
justdating.online keywordhouse.com
kandidos.com kicknights.gq
kchaxton.com kino-ecran.ru
kenaba.su kino2018.club
kino-rf.ru kiprinform.com
knowsitall.info klikbonus.com
konkursowo-24.pl kotaku.com
lacapilla.info krynica.info
ledpolice.ru lennatin.info
legalrc.biz listiseltemournlan.gq
longadventure.com littlesexdolls.com
luxup.ru livejasmin.com
magazin-pics.ru lkbennettoutlet.online
maglid.ru lodki-pvh.dp.ua
masterseek.com longgreen.info
masthopehomes.com louboutinreplica.xyz
lovi-moment.com.ua
mabdoola.blogspot.com
mail.allnews24.in
makemoneyonline.com
marblestyle.ru
matrixalchemy.com matrixalchemy.com
mirtorrent.net me-ke.com
mokrayakiska.com med-zdorovie.com.ua
monetizer.com-01.site medi-fitt.hu
montredemarque.nl medizinreisen.de
most-kerch.org mega-bony2017.pl
musezone.ru megaapteka.ru
musflashtv.com mielec.pl
mygameplus.ru missvietnam.org
napalm51.nut.cc moetomnenie.com
negral.pluto.ro moivestiy.biz
newstaffadsshop.club morepoweronmac.trade
newtechspb.ru moscow.online-podarki.com
nfljerseyscheapbiz.us mouselink.co
nordstar.pro moxo.com
northfacestore.online mrsdalloways.com
novgorod.xrus.org mykings.pw
nuup.info namenectar.com
nzfilecloud.weebly.com naperehresti.info
one-gear.com narosty.com
optom-deshevo.ru nlfjjunb5.ru
osagonline.ru normalegal.ru
ownshop.win obiavo.in
pathwhelp.org onetravelguides.com
pc4download.co onlythegames.com
pearlisland.ru ooomeru.ru
pic2fly.com piluli.info
plaff-go.ru pinapchik.com
pochemychka.net piratecams.com
pizza-imperia.com
plohaya-kreditnaya-istoriya.ru
popunder.ru
popup-hgd.xyz popup-hgd.xyz
porno-raskazy.ru pornokorol.com
pornoforadult.com
prefersurvey.net prefersurvey.net
prokotov.com preparevideosafesystem4unow.site
proxyelite.biz primedice.com
pushdata.sendpulse.com printie.com
quebec-bin.com prom23.ru
rankings-analytics.com prospekt-st.ru
realting-moscow.ru psvita.ru
recordpage-a.akamaihd.net ptr.ruvds.com
redirlock.com puserving.com
refudiatethissarah.info py100.ru
rethinkwasteni.info quickcashlimited.com
rosbalt.com.ua r-e-f-e-r-e-r.com
rusoft-zone.ru rapevideosmovies.com
scalerite.co.za resant.ru
scanner-mary.top rezeptiblud.ru
scanspyware.net ribieiendom.no
seryeznie-znakomstva.ru roll123.com
sexgalleries.top rozalli.com
sexytrend.ru saneyes.com
sharebutton.to sankt-peterburg.nodup.ru
shemale-sex.net scanner-jessica.top
sherlock.se searchinquire.com
shodanhq.com seasonvar.ru
silverage.ru seccioncontrabajo.com
simpoed.ufop.br security60-e.com
site.ru servisural.ru
small-game.com sex-tracker.com
smartadserver.com sex-tracker.de
smartshoppymac.com sexsaoy.com
sockshare.net sildenafilcitratemed.com
spabali.org silvermature.net
spb-plitka.ru simplesite.com
sport-video-obzor.ru sitevalued.com
srecorder.com skatestick.bid
stairliftstrue.com songplanet.ru
stauga.altervista.org spb.afora.ru
stmassage.ru starpages.net
sugarkun.com sterva.cc
suralink.com stpicks.com
sygraem.com strag-invest.ru
taaaak.com strigkaomsk.ru
taxi-v-eisk.ru stroilka.info
texbaza.by stroy-portal22.ru
thefds.net supernew.org
tizanidine4mguses.blogspot.com superoboi.com.ua
tizanidinehcl2mgtablet.blogspot.com svetlotorg.ru
tizanidinehcl4mgisitanarcotic.blogspot.com taqplayer.info
tizanidineusedfor.blogspot.com tecspb.ru
token-lab.org thegalerie.eu
tizanidineandhydrocodone.blogspot.com
toloka.hurtom.com
topquality.cf
tourismvictoria.com tourismvictoria.com
track2.shop track2.shop
tracksz.co tvand.ru
trahic.ru twincitiescarservice.com
trahvid.com udsgame.online
tupper-shop.ru uggbootsoutletsale.us
typimga.pw unpredictable.ga
uac.net.au urzedowski.eu
uchebavchehii.ru vandrie-ict.nl
unblocksit.es vbabule.net
urldelivery.com vclicks.net
urll.eu viberdownload10.com
vbtracker.net video-camer.com
videogamesecrets.com videochat.bz
videos-for-your-business.com videospornogratisx.net
videositename.com vykup-avto-krasnodar.ru
vseigru.one wahicbefa31.soup.io
vsesubwaysurfers.com wcb.su
wardreapptokone.tk web.cvut.cz
weblo.com website-speed-checker.site
webmonetizer.net wellcome2slovenia.ru
wechatdownload10.com wesharepics.info
where-toget.com westen-v.life
whos.amung.us whatsupinfoley.com
whosonmyserver.com winterclassichockeyjerseys.com
world-mmo.com wnoz.de
www.solartek.ru wpsecurity.website
x-lime.com xfluro.com
xerox-douglas.cf xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xvideosporn.biz xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xvideospornoru.com xuki.us
xxxdatinglocal.us xxx-cam.webcam
xxxkaz.org yaminecraft.ru
yatrk.xyz yellowstonesafaritours.com
youandcredit.ru youtubedownload.org
youdesigner.kz zatjmuzu.info
your-bearings.com zirondelli.it
zaidia.xhost.ro zoosexart.com
zed21.net zvezdagedon.ru
zeg-distribution.com
zeroredirect2.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.106 0; 10.1.1.84 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2643 ### Version: V4.2021.10.2644
### Updated: Sun Oct 10 22:02:40 UTC 2021 ### Updated: Mon Oct 11 10:12:41 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2643 ### Version: V4.2021.10.2644
### Updated: Sun Oct 10 22:02:40 UTC 2021 ### Updated: Mon Oct 11 10:12:41 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##