vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.12.2799 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-12-22 10:09:18 +00:00
parent e1efe880ba
commit deaf482dcd
47 changed files with 77187 additions and 77197 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2798 #### Version: V4.2021.12.2799
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2798 #### Version: V4.2021.12.2799
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.12.2798 #### Version: V4.2021.12.2799
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9516
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.12.2798 #### Version: V4.2021.12.2799
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9520
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9520
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

9520
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9520
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 1; 10.1.0.132 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 0; 10.1.0.132 0;
127.0.0.1 0; 127.0.0.1 0;

9520
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 1; 10.1.0.132 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 0; 10.1.0.132 0;
127.0.0.1 0; 127.0.0.1 0;

9520
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 1; 10.1.0.132 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 0; 10.1.0.132 0;
127.0.0.1 0; 127.0.0.1 0;

9520
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2798 2799

9520
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

9520
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.77.7.223 0; 20.112.106.111 0;

9520
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 1; 10.1.0.132 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 0; 10.1.0.132 0;
127.0.0.1 0; 127.0.0.1 0;

9520
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 1; 10.1.0.132 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 0; 10.1.0.132 0;
127.0.0.1 0; 127.0.0.1 0;

9520
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 1; 10.1.0.132 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 0; 10.1.0.132 0;
127.0.0.1 0; 127.0.0.1 0;

9520
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.77.7.223 0; 20.112.106.111 0;

9520
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.77.7.223 0; 20.112.106.111 0;

9520
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 1; 10.1.0.132 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Black Hole BBBike
EirGrabber Brandprotect
Fyrebot Gigabot
Linkbot IstellaBot
Mr.4x3 Libwhisker
Probethenet MIIxpc
Telesoft ProWebWalker
WWWOFFLE ScreenerBot
clark-crawler VeriCiteCrawler
lwp-trivial Web Fuck

292
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker
80legs 80legs
AIBOT
ALittle Client ALittle Client
ASPSeek Abonti
AfD-Verbotsverfahren Aboundexbot
AiHitBot AiHitBot
Aipbot
Alligator
AlphaBot
Anarchy Anarchy
Anarchy99
Ankit Ankit
Anthill
Apexoo Apexoo
Attach
AwarioRssBot AwarioRssBot
BDCbot BBBike
BDFetch BLEXBot
BackDoorBot
BackStreet BackStreet
BatchFTP BacklinkCrawler
Bandit
Bigfoot Bigfoot
Blackboard
Blow Blow
BotALot Buck
Brandwatch BuiltBotTough
BunnySlippers BuiltWith
CCBot CCBot
CSHttp CODE87
Cegbfeieh Calculon
CensysInspect CensysInspect
CheTeam CheTeam
ChinaClaw ChinaClaw
Citoid Cloud mapping
Claritybot Cocolyzebot
Cliqzbot
Cogentbot Cogentbot
Copier
CopyRightCheck CopyRightCheck
Crawling at Home Project CrazyWebCrawler
Curious Custo
CyotekWebCopy DSearch
DBLBot Demon
DataCha0s Devil
DatabaseDriverMysqli
Deusu
DigitalPebble
Disco
Discoverybot
Dispatch Dispatch
DnyzBot DittoSpyder
DomainAppender DomCopBot
Domains Project DomainCrawler
Dragonfly DomainStatsBot
Drip Dotbot
ECCP/1.0 ECCP/1.0
EMail Siphon
EMail Wolf
Ebingbong Ebingbong
Ecxi Ecxi
Exabot EirGrabber
EroCrawler
Express WebPictures Express WebPictures
Extractor ExtLinksBot
Extreme Picture Finder Ezooms
EyeNetIE
FHscan FHscan
FemtosearchBot
Fimap Fimap
Firefox/7.0 Foobot
Flunky Fuzz
FrontPage
FyberSpider FyberSpider
GT::WWW GT::WWW
GalaxyBot
GetWeb
Getintent Getintent
Gigabot Gigabot
Go!Zilla Gotit
GrabNet
Grabber Grabber
GrapeshotCrawler Grafula
HEADMasterSEO HEADMasterSEO
HTMLparser HTMLparser
HTTP::Lite HTTP::Lite
Harvest Harvest
Humanlinks Humanlinks
IDBTE4M IDBot
Image Sucker
Indy Library Indy Library
InfoNaviRobot
InterGET InterGET
Internet Ninja
InternetSeer InternetSeer
JOC Web Spider IstellaBot
JamesBOT JamesBOT
JennyBot Jbrofuzz
JetCar JetCar
Jetty Joomla
LNSpiderguy Kenjin Spider
Lanshanbot Kinza
Leap
LexiBot LexiBot
LieBaoFast Lftp
Lightspeedsystems LibWeb
LinkScan
LinkWalker LinkWalker
Linkdexbot Linkdexbot
LinkextractorPro LinqiaMetadataDownloaderBot
Lipperhey LinqiaRSSBot
Lipperhey Spider Litemage_walker
Lmspider Lmspider
Ltx71
MFC_Tear_Sample
MJ12bot
MQQBrowser MQQBrowser
MSFrontPage MSIECrawler
Magnet
Majestic12 Majestic12
Masscan MarkMonitor
Mediatoolkitbot Mass Downloader
Metauri MauiBot
MicroMessenger MeanPath Bot
Microsoft Data Access
Minefield Minefield
Moblie Safari Mister PiX
Mojolicious Mojeek
Mozlila Morfeus Fucking Scanner
Musobot Musobot
NICErsPRO
NPbot
Name Intelligence
Nameprotect Nameprotect
Navroad Navroad
Needle Needle
Net Vampire
NetAnts
NetLyzer
NetZIP NetZIP
Nettrack Netcraft
Netvibes
Nibbler Nibbler
Ninja
Not Not
Nutch Offline Explorer
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OutclicksBot OrangeSpider
OutfoxBot
PECL::HTTP PECL::HTTP
PageScorer POE-Component-Client-HTTP
PageThing.com PeoplePal
Pandalytics
Panscient
Piepmatz Piepmatz
Pimonster PleaseCrawl
Pixray Probethenet
QueryN Metasearch Psbot
Pu_iN
Pump
Quick-Crawler Quick-Crawler
RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
RankingBot RankurBot
RebelMouse Re-re
Reaper
RedesScrapy RedesScrapy
RepoMonkey Rogerbot
Ripper SEOlyticsCrawler
SEOprofiler
SEOstats SEOstats
SISTRIX SalesIntelligent
ScanAlert SemrushBot
ScoutJet
Scrapy
Searchestate
SearchmetricsBot
SemanticJuice
Semrush
SentiBot
SeobilityBot SeobilityBot
Seomoz Seomoz
Site Sucker Siphon
SiteExplorer
SiteSnagger SiteSnagger
Sitevigil SlySearch
Snapbot Spinn3r
Snoopy
Sociscraper
Sogou web spider
SputnikBot SputnikBot
Sqlmap
Sqworm Sqworm
Steeler Steeler
Sucuri Stripper
SuperHTTP
Surfbot
SurveyBot SurveyBot
Suzuran Suzuran
T8Abot TheNomad
Telesoft
Telesphorep
The Intraformant
Trendiction Trendiction
Trendictionbot TwengaBot
Turingos
Turnitin
TurnitinBot
Twice Twice
Typhoeus Typhoeus
URLy Warning
URLy.Warning
UnisterBot UnisterBot
Upflow
V-BOT V-BOT
VCI VCI
Vagabondo
VidibleScraper VidibleScraper
Virusdie VoidEYE
Voltron Voil
WASALive-Bot WISENutbot
WWW::Mechanize WWW-Collector-E
Web Fuck WWW-Mechanize
Wallpapers
WallpapersHD
WeSEE
Web Auto
Web Pix Web Pix
WebAuto Web Sauger
WebBandit
WebCollage
WebCopier
WebEnhancer WebEnhancer
WebFetch
WebFuck
WebImageCollector WebImageCollector
WebLeacher WebLeacher
WebReaper WebPix
WebSauger
WebSucker WebSucker
WebZIP
Webalta Webalta
WebmasterWorldForumBot
Website Quester Website Quester
WebsiteExtractor WebsiteQuester
Whacker Whack
Widow Wonderbot
WinHTTrack
WiseGuys Robot
Wotbox
Wprecon Wprecon
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zauba
Zermelo Zermelo
Zitebot ZoomBot
ZumBot
adscanner
archive.org_bot archive.org_bot
arquivo-web-crawler
arquivo.pt
backlink-check backlink-check
check1.exe
clark-crawler clark-crawler
coccocbot-web
com.plumanalytics com.plumanalytics
crawl.sogou.com crawler.feedback
crawler4j demandbase-bot
domainsproject.org
eCatch
facebookscraper facebookscraper
gopher internetVista monitor
heritrix
instabid
ips-agent
isitwp.com isitwp.com
lwp-request lwp-request
lwp-trivial
magpie-crawler
meanpathbot
mediawords mediawords
oBot oBot
page scorer page scorer
pcBrowser
plumanalytics plumanalytics
ripz polaris version
s1z.ru
satoristudio.net
scalaj-http scalaj-http
seobility seobility
seostar seostar
sysscan serpstatbot
trendiction.com sexsearcher
trendiction.de sitechecker.pro
sp_auditbot
tAkeOut
ubermetrics-technologies.com ubermetrics-technologies.com
webgains-bot webgains-bot
webmeup-crawler
webpros.com webpros.com
x09Mozilla zauba.io
x22Mozilla zgrab

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 404checker
404enemy 80legs
AIBOT
ALittle Client
Abonti
Aboundex Aboundex
Acunetix Acunetix
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot
Aipbot
AllSubmitter
Alligator Alligator
Anarchie AlphaBot
Anarchy99 Anarchy
Anthill
Apexoo Apexoo
Aspiegel
Attach Attach
BBBike AwarioRssBot
BLEXBot
BackStreet BackStreet
Badass Badass
Barkrowler Barkrowler
Black Hole Battleztar Bazinga
Blow Bigfoot
Boardreader Blackboard
Bolt
BotALot BotALot
Brandwatch
Buck Buck
Buddy
BuiltWith BuiltWith
Bullseye Bullseye
BunnySlippers BunnySlippers
CCBot
CODE87
Calculon Calculon
CazoodleBot CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect ChinaClaw
CheTeam Chlooe
Citoid
Claritybot Claritybot
Cloud mapping
Copier Copier
Crawling at Home Project Copyscape
Craftbot
CrazyWebCrawler CrazyWebCrawler
Crescent
Custo
CyotekWebCopy CyotekWebCopy
DBLBot DSearch
DIIbot
DTS Agent DTS Agent
Demon DataCha0s
Deusu Dirbuster
DigitalPebble Disco
Discobot
Dispatch Dispatch
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DomCopBot DomainSigmaCrawler
Download Wonder Dotbot
ECCP/1.0 ECCP/1.0
EMail Siphon
EMail Wolf EMail Wolf
EasyDL EasyDL
Ecxi Ebingbong
EirGrabber
EroCrawler EroCrawler
Exabot
Express WebPictures Express WebPictures
Ezooms
FDM
FHscan
FemtosearchBot
Fimap Fimap
Firefox/7.0 Firefox/7.0
FlashGet Flunky
Foobot
Freeuploader Freeuploader
FrontPage
Fuzz
FyberSpider
Fyrebot
GT::WWW
GalaxyBot
GetWeb
Getintent Getintent
Gigabot Gigabot
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla
GrabNet GrabNet
Grabber
Grafula
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
HTMLparser
Haansoft Haansoft
HaosouSpider
Harvest Harvest
Humanlinks
IDBot
IRLbot IRLbot
IlseBot IlseBot
Intelliseek IndeedBot
InterGET InfoNaviRobot
InfoTekies
InternetSeer
Iria Iria
JOC Web Spider JennyBot
Jetty JetCar
JikeSpider
Joomla Joomla
Jyxobot Jorgee
Kenjin Spider Kenjin Spider
Kozmosbot Kozmosbot
LNSpiderguy LeechFTP
LWP::Simple
Lanshanbot
LeechGet LeechGet
LibWeb LexiBot
Libwhisker Libwhisker
Lightspeedsystems
Linkdexbot Linkdexbot
LinkextractorPro
LinksManager LinksManager
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot Lipperhey
Lipperhey Spider
Lmspider Lmspider
Ltx71 Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MJ12bot
MQQBrowser MSIECrawler
MTRobot MTRobot
Mag-Net
Magnet
Majestic SEO Majestic SEO
MarkMonitor Majestic12
MarkWatch MarkWatch
Mass Downloader Masscan
Mata Hari
MauiBot MauiBot
MeanPath Bot MeanPath Bot
MegaIndex.ru MicroMessenger
Microsoft Data Access
Microsoft URL Control
Mister PiX Mister PiX
Moblie Safari Mojeek
Morfeus Fucking Scanner Mojolicious
Mr.4x3
Msrabot Msrabot
NICErsPRO NICErsPRO
NPbot
Nameprotect Nameprotect
NearSite NearSite
Net Vampire Needle
NetLyzer
NetMechanic
NetSpider
NetZIP NetZIP
Netcraft
Nettrack
NextGenSearchBot NextGenSearchBot
Nikto Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Ninja Openfind
Not
Nutch
Octopus
Offline Navigator
OutclicksBot OutclicksBot
OutfoxBot POE-Component-Client-HTTP
PageAnalyzer Page Analyzer
PageScorer
Pandalytics
Papa Foto
PeoplePal
Pi-Monster Pi-Monster
Picscout
Picsearch
PictureFinder PictureFinder
Pimonster
Pixray Pixray
PleaseCrawl
Pockey
ProPowerBot ProPowerBot
Probethenet Probethenet
Psbot
PxBroker PxBroker
PyCurl QueryN Metasearch
Quick-Crawler RankActive
RSSingBot RankurBot
RankFlex Re-re
RankingBot2 ReGet
Rankivabot RepoMonkey
RocketCrawler Ripper
SEOkicks Rogerbot
SEOkicks-Robot SEOlyticsCrawler
SMTBot SEOstats
Scanbot Scrapy
ScoutJet Screaming
SearchmetricsBot ScreenerBot
ScrepyBot
SemanticJuice SemanticJuice
Semrush Semrush
SemrushBot Seomoz
SeoSiteCheckup
Siphon
Site Sucker
SiteCheckerBotCrawler
SiteLockSpider SiteLockSpider
SiteSnagger Siteimprove
Sitevigil SmartDownload
SlySearch
Snake
Snoopy Snoopy
SocialRankIOBot Sociscraper
Sogou web spider Sottopop
SpankBot Spanner
Spbot Spbot
Spinn3r SputnikBot
Sqworm Sqlmap
Stripper
SuperBot SuperBot
Surfbot
SurveyBot SurveyBot
Szukacz Szukacz
Teleport Teleport
TheNomad The Intraformant
Thumbor
TightTwatBot TightTwatBot
Titan
Tracemyfile Tracemyfile
Trendiction
Trendictionbot Trendictionbot
Vacuum True_Robot
TurnitinBot
TwengaBot
Twice
URLy Warning
URLy.Warning
Upflow
V-BOT
VB Project
VelenPublicWebCrawler
VidibleScraper
Virusdie Virusdie
WBSearchBot
WEBDAV
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
Wallpapers/3.0 WWWOFFLE
WallpapersHD Wallpapers
Web Enhancer WeSEE
Web Fuck Web Auto
Web Sucker Web Sauger
WebAuto WebImageCollector
WebCollage WebPix
WebLeacher WebReaper
WebStripper WebSauger
WebZIP
Website Quester Website Quester
WebsiteExtractor
WebsiteQuester
Whack
Who.is Bot
Widow Widow
Woobot WinHTTrack
Wotbox Wotbox
Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zade
Zauba
Zeus Zeus
ZmEu
ZoomBot
ZumBot ZumBot
check1.exe autoemailspider
clark-crawler backlink-check
coccocbot-web com.plumanalytics
cognitiveseo dataforseo.com
crawler.feedback eCatch
domainsproject.org evc-batch
facebookscraper magpie-crawler
heritrix
isitwp.com
lwp-request
meanpathbot
mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler
plumanalytics plumanalytics
ripz scan.lol
s1z.ru seocompany.store
scalaj-http spyfu
seoscanners tAkeOut
seostar
serpstatbot
sexsearcher
siteripz
sogouspider
sp_auditbot
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webgains-bot webgains-bot
webmeup-crawler webmeup-crawler
x09Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
0lovespells0.blogspot.com freeuploader.com
bolegapakistan.com hybrid.ru
eldiariodeguadalajara.com manimpotence.com
levaquin750mg.blogspot.com plugingeorgia.com
o-o-8-o-o.com sanidumps.com
rating-bestcasino.com sexkontakteao.info
scanner-elena.top the-trader.net
serialsway.ucoz.ru webmonetizer.net
si-unique.com womens-journal.net
stretchmate.net zigarettenonlinekaufen2.drupalgardens.com

904
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0c47f8422d3f.com 000free.us
1314dh.com 178evakuator178.ru
1kinobig.ru 214jaluwobapef.cf
24-ak.ru 4senses.co
7ooo.ru 51unlim.ru
98oi.ru 70casino.online
aad1f4acb0a373420d9b0c4202d38d94fa.com 8xv8.com
absugars.com aaa38852e886ac4af1a3cff9b47cab6272.com
adanih.com abctoppictures.net
adbetclickin.pink above.com
adinterax.com actices.com
adman.gr adf.ly
adultfullhd.com adloads.com
admeasures.com
adsloads.com
adssafeprotected.com
advancedcleaner.com advancedcleaner.com
advokateg.ru aihelen.net
ageofclones.com aktivator-windows10.blogspot.com
agroeconom.kz allkrim.com
akama.com allsilver925.co.il
alert.scansafe.net alphaforum.ru
alive-ua.com ample-awards-today.us
all4invest.ru anabolics.shop
alloysteel.ru
alot.com
amateurmatch.com
amehdaily.com
amigobulls.com
amoi.tn
android-vsem.org android-vsem.org
anniemation.com angigreene.com
apessay.com anonymousfox.co
apiholdingmypage-a.akamaihd.net apimegabrowsebiz-a.akamaihd.net
apiitsthirteende-a.akamaihd.net app5.letmacworkfaster.world
aquarium-pro.ru apps-analytics.net
arendakvartir.kz apxeo.info
arius.tech arabseyes.com
arpelsreplica.xyz arclk.net
asiengirls.net auto.rusvile.lt
autodo.info autorn.ru
automobile-spec.com autovideobroadcast.com
awaybird.ru avtochehli.by
azbukafree.com azartniy-bonus.com
b-styles.xyz backiee.com
badavit.com.ua
baixar-musicas-gratis.com
balitouroffice.com balitouroffice.com
balkanfarma.org barnfurnituremart.com
bannertesting.com
battle.net battle.net
bbsoldes.fr
bcmp.org
becuo.com becuo.com
beetpics.pw best-coupon-offer.com
bensbargains.net beyan.host.sk
best-games.today bgviagrachrx.com
bestcurs.org bibys.com
bestimagecoollibrary.com big-cash.net
bestmobilityscooterstoday.com biketank.ga
bestsexyblog.com binomo.kz
beta.hotkeys.com birzha-truda.eu
blackbabesporn.com biteg.xyz
blog.f00kclan.de bitnote.co
bloggerads.net bjorkbacken.nu
bluerobot.info bloggers.nl
break-the-chains.com bolezniorganov.ru
buqayy0.livejournal.com book-bets.com
burger-tycoon.com bouda.kvalitne.cz
buy-cheap-pills-order-online.com brains2.biz
buy-loft.ru braintobe.win
breastaugmentation.co.za
budilneg.xyz
buy-cheap-online.info
buy-forum.ru
buy-meds24.com buy-meds24.com
buyessaynow.biz buyessaynow.biz
buyfriend.ru buyessayonline19.blogspot.ru
buynorxx.com buypillsorderonline.com
camsex.xxx-cam.webcam callawaygolfoutlet.online
canacopegdl.com candlehandmade.com
carloans.com canvas.gloverid.site
carson.getenjoyment.net carabela.com.do
casinopinup-wins.com carsplate.com
celebsopics.com cartierring.xyz
chatroulette.life cascadelink.org
cheapjerseysbizwholesale.us cdncash.com
cdncash.org
changde.58.com
cheapmoncler.xyz
cherrypointplace.ca
chinajerseyswholesalecoupons.com chinajerseyswholesalecoupons.com
cialischmrx.com christianlouboutinsaleoutletonline.us
clickpapa.com ci.ua
clickso.com citetick.com
clksite.com civilwartheater.com
coaching-netz.info clean-start.net
collegeessay19.blogspot.ru clickcash.com
com-secure.download cmd.kz
compiko.info cmhomestayagency.com
compliance-checker.info cntravelre.com
compliance-jessica.top compliance-checker-7.info
conocer-sanabria.com compliance-irvin.top
copblock.org compliance-jack.top
counciltally.com compliance-josh.top
counter.bloke.com compliance-olga.top
counterbot.com cookie-law-enforcement-hh.xyz
craftburg.ru countbertwistdisp26.soup.io
craftinsta.ru covetnica.com
crazyboost.pro creditnation.ru
custodieva.ru csgo4.win
d-black.bz customcatchcan.com
dailyrank.net cypernhuset.se
dairyindia.in dalavia.ru
daneshetabiat.com dev78.dioniqlabb.se
de.zapmeta.com dfwu1013.info
deanmoore.ie diamond-necklace.info
dear-diary.ru
decoratinghomes.ga
dekorkeramik.ru
derevesendeco.com
detoxmed24.ru
dieta-personalna.pl
diffbot.com diffbot.com
dimaka.info digital-video-processing.com
dimfour.com dinkolove.ya.ru
dimkino.ru divci-hry.info
dipstar.org diy-handmade-ideas.com
discovertreasure-a.akamaihd.net dlya-android.org
djstools.com dogsrun.net
dkvorota.ru dokfilms.net
docsportal.net domain-tracker.com
doctissimo.fr domain.webkeyit.com
dolohen.com dood.live
domaincdn.xyz doreenblog.online
domik-derevne.ru doska-vsem.ru
drev.biz earn-from-articles.com
drugspowerstore.com easyukraine.com
dverimegapolis.ru eets.net
dyshagi.ru elektrischezigaretten1.blogse.nl
e-poker-2005.com elektronischezigarettekaufen2.tumblr.com
eachdayisagift.review elektrozigarette1.dreamwidth.org
educatemetv.com encodable.com
educontest.net enternet.ee
elektroniksigaraankara.info es5.com
elektronischezigarettekaufen1.beeplog.com
elektrozigarette2.webs.com
elektrozigaretten2.yn.lt
elitesportsadvisor.com
elstal.com.pl
empirepoker.com
eng-lyrics.com
enthuse.computernetworksonline.com
erectile.bid
escort-russian.com
escortslet.net
estibot.com estibot.com
exlarseva.webblog.es everytalk.tv
exoclick.com executehosting.com
extremal-blog.com expertnaya-ocenka.ru
failingmarriege.blogspot.com eyes-on-you.ga
eywords-monitoring-your-success.com
ezb.elvenmachine.com
ezofest.sk
family1st.ca
fashion.stellaconstance.co
fashionavenuegame.com fashionavenuegame.com
femmesdenudees.com fba-mexico.com
ferretsoft.com fikasound.tk
fetishinside.com fillmewithhappiness.com
filerockstar298.weebly.com filmbokep69.com
filesmonster.porn filmi-v.online
filmania-x.ru financepoints.eu
fishtauto.ru find1friend.com
fitnesspiks.com finemanteam.com
formularz-konkurs.tk folowsite.com
foto-weinberger.at forumprofi.de
franch.info fotopop.club
free-deals.faith franecki.net
free-fbook-traffic.com frequiry.com
free-share-buttons.com freshberry.com.ua
free-social-buttons.com freshdz.com
freefoto.ca
freevpn.space
freza-sverlo.ru
fullfileaccess.com fullfileaccess.com
funponsel.com
fusoradio.info fusoradio.info
garciniaxt.us gael-s.ru
geileweiber.tk gambarxkata.co
get-free-social-traffic.com gavuer.ru
getpopunder.com gaypornmovie.net
glavtral.ru gepezz.info
globalsurfari.com getoutofdebtfree.org
go2mike.ru glavprofit.ru
gogps.me globalscam.ga
gojiberriess.apishops.ru goldenggames.com
google-liar.ru gotomontenegro.net
googlsucks.com gov.yanao.ru
goroda-vsego-mira.ru gpirate.com
gosmeb.ru graphicwe.org
groupmoney.ru hccoder.info
gsasearchenginerankerhelp.com hdwalls.xyz
gta-club.ru helpmymacfaster.trade
guidefs.ru hiwibyh.bugs3.com
guildebzh.info hjaoopoa.top
gungamesz.com hobild.net
guruofcasino.com home-task.com
gz2.bbsoldes.fr homeart.space
handicapbathtubarea.com honyaku.yahoofs.jp
hanink.biz.ly host-protection.com
hasshe.com hotdl.in
hazardky.net
hdimagegallery.net
hermesbirkinhandbagoutlets.com
hidefiles.org
hledejvshopech.cz
hotenergy.ru hotenergy.ru
hothor.se
hotshoppymac.com hotshoppymac.com
hptwaakw.blog.fc2.com humanorightswatch.org
humanelydrew.com hvd-store.com
husky-shop.cz iblogpress.xyz
hyipmanager.in id-forex.com
i-midias.net.br ihtec2019.org
ilmen.net imagecoolpub.com
ilovevitaly.xyz
indianmedicaltourismshop.com
infodocsportal.com infodocsportal.com
innodgfdriverhm.aircus.com infostatsvc.com
instabid.tech insomniagamingfestival.com
invest-pamm.ru insta-add.pro
int.search.mywebsearch.com
intelhdgraphicsgtdrive6w.metroblog.com
intellekt21.ru
interferencer.ru
internetproviderstucson.com
io9.com
iscblog.info
islandminingsupply.wordpress.com
isoveti.ru isoveti.ru
ivearchenceinflu.cf iyasimasennka.com
janavibekken.no jav-way.site
janerikholst.se jerseyschinabizwholesale.us
japfm.com jjbabskoe.ru
jav-idol.com jovencitas.gratis
jerseyswholesalechinalimited.com jpcycles.com
jobmarket.com.ua junketjuice.blogspot.com
jolic2.com justbcause.com
jongose.ninja kaidalibor.de
kadashihotel.com kamen-e.ru
kam-dom.ru kartiniresto.com
kanimage.com kchaxton.com
karting196.ru khadastoafarde.tk
kazrent.com kino2018.club
kerei.ru
keywesthideaways.co
keywordsdoctor.com
kino-key.info
kinoflux.net
kiskinhouse.com
knowyournextmove.com
kongoultry.net kongoultry.net
koronirealestate.gr kongruan.com
ks1234.com kretpicf.pw
kurwa.win
kutikomi.net
kvartira-sutochno.com
labafydjxa.su
labvis.host.sk
ladsblue.com
larger.io larger.io
laulini.soclog.se larutti.ru
law-check-seven.xyz
laxob.com
lernur.net lernur.net
lexixxx.com letmacworkfaster.site
lidoradio.com letslowbefast.site
lifeinsurancekb.com letslowbefast.today
lider-zhaluzi.kiev.ua
lion.bolegapakistan.com lion.bolegapakistan.com
lion.misba.us liquimondo.com
lion.playtap.us livingcanarias.com
liran-locks.com lizace.com
livefixer.com locksmith.jp
livingroomdecor.info lol-smurfs.com
lombia.co lovelycraftyhome.com
low-format.ru maglid.ru
lunamedia.co mantramusic.ru
lutherstable.org masturbate.co.uk
luxmagazine.cf meble-bogart.info
macdamaged.space meet-flirt-dating.com
magento-crew.net meetingrainstorm.bid
magicplayer-s.acestream.net mega-bony2017.pl
mainhp.net mere.host.sk
marblestyle.ru meteostate.com
markjaybeefractal.com methodsmarketing.com
medmajor.ru michellblog.online
meget.co.za microsoftportal.net
menetie.ru microstatic.pl
mexicotravelnet.com minyetki.ru
michaelkorsoutletstore.net mirmedinfo.ru
mikrobiologies.ru misslike.ru
mirzonru.net missvietnam.org
mmstat.com mobplayer.ru
mompussy.net moesen-ficken.com
mojpregled.com
monclerjacketsoutlet.pw
moncleronline.xyz moncleronline.xyz
moneyteam24.com montazhnic.ru
moneyzzz.ru moxo.com
mosrif.ru moyaterapiya.ru
mrwhite.biz mrsdalloways.com
mural.co musflashtv.com
mustwineblog.com mustwineblog.com
muycerdas.xxx myblogregistercm.tk
muz-tracker.net myhitmp3.club
myfreemp3.eu
mykings.pw mykings.pw
mypets.by mypets.by
myyour.eu myyour.eu
nacap.ru na15.ru
naperehresti.info nalogovyykodeks.ru
naphukete.ru namecrumilchlet.tk
narco24.me narco24.me
narutonaruto.ru nash-krym.info
negral.pluto.ro nelc.edu.eg
netfacet.net netfacet.net
nevansk.ru newstaffadsshop.club
nhl17coinsforps3.gratisblog.biz nfljerseyscheapchinabiz.com
nicovideo.jp nonews.co
niroo.info npoet.ru
njpalletremoval.com obiavo.su
obesidadealgarve.com
obuv-kupit.ru obuv-kupit.ru
onclkads.com ok-ua.info
online-hit.info okmusic.jp
orsonet.ru omoikiri-japan.ru
ozoz.it onclickpredictiv.com
page2rss.com onefilms.net
painting-planet.com onescreen.cc
paleohub.info onlainbesplatno.ru
papasdelivery.ru onstrapon.purplesphere.in
par-fallen.ga ooo-gotovie.ru
perfection-pleasure.ru open-odyssey.org
perm-profnastil.ru oxford-book.com.ua
perso.wanadoo.es oyster-green.com
piccshare.com pammik.ru
park.above.com
parlament.biz
partnersafe.men
paydayonlinecom.com
peekyou.com
pelfind.me
pflexads.com
pharmacyincity.com
photosaga.info
phrcialiled.com
physfunc.ru
picmoonco.pw picmoonco.pw
picsfair.com
picturesboss.com picturesboss.com
pills24h.com pinwallpaper.top
pinstake.com piski.top
pixelrz.com pk-pomosch.ru
pizda.lol play.leadzupc.com
pk-services.ru pochemychka.net
pkr1hand.com pontiacsolstice.info
pl.id-forex.com popmarker.com
playlott.com porno-dojki.net
pobeiranie.pl pornogad.com
pooleroadmedicalcentre.co.uk pornoklad.net
porn-w.org pornotubs.com
porndl.org pornoxxx.com.mx
pornhub-ru.com porto.abuilder.net
pornofiljmi.com poshiv-chehol.ru
pornowarp.info posible.net
pr0fit-b0x.com praisong.net
prlog.ru preparevideosafesystem4unow.site
proekt-gaz.ru pricheskaonline.ru
professionaldieselcare.com privacylocationforloc.com
profitsport.club producm.ru
projefrio.com.br proekt-mos.ru
prombudpostach.com.ua
pronekut.com pronekut.com
prostitutki-kharkova.org prosperent.com
pssucai.info proxyradar.com
push-ad.com prpops.com
ranksonic.org putevka24.ru
rasteniya-vs-zombi.ru pyramidlitho.webs.com
realresultslist.com realtytimes.com
reckonstat.info referencemoi.com
remmling.de registry-cleaner.net
reklama1.ru
remedyotc.com
remont-fridge-tv.ru remont-fridge-tv.ru
rentalcarnavi.info resultshub-a.akamaihd.net
restaurantlescampi.com risparmiocasa.bz.it
rewardit.com ros-ctm.ru
rossanasaavedra.net rusoft-zone.ru
rostov.xrus.org rvzr-a.akamaihd.net
rus-teh.narod.ru s-forum.biz
rybalka-opt.ru sabizonline.com
sa-live.com saitevpatorii.com
sa-rewards.co.za savememoney.co.za
sajatvelemeny.com scanner-irvin.top
savingsslider-a.akamaihd.net scanner-walt.top
sax-sex.com
scanner-margo.top
scanner-walter.top scanner-walter.top
searchimage.co searchaddis.com
seminarykansas904.ml searchkut.com
seo-2-0.com seccioncontrabajo.com
seo-platform.com securesmrt-dt.com
sevendays.com.ua selectads.men
sex-dating.co semrush.com
sexpornotales.com seoanalyses.com
sexystrippe.info seomarketings.online
shiksabd.com servethis.com
sexpornotales.net
sexxdate.net
sexy-pings.com
sexyteens.hol.es
shareyards.com
shinikiev.com.ua
shohanb.com shohanb.com
shoppingmiracles.co.uk shoppingmiracles.co.uk
sibtest.ru silvercash.com
sigmund-freud.co.uk similardeals.net
siha.de skutecznetabletkinaporostwlosow.pl
simplepooltips.com sky-mine.ru
sitechecker.pro slavkokacunko.de
siteheart.net smstraf.ru
sklad-24.ru snip.tw
skylta.com snjatie-geroinovoy-lomki.ru
skyway24.ru snyatie-lomki-v-stacionare.ru
slavic-magic.ru softomix.com
slkrm.ru souvenir.cc
smartshoppymac.com stanthonyscatholicchurch.org
socialsignals24.com steelmaster.lv
socialtrade.biz stefanbakosab.se
soft-terminal.ru stop-zavisimost.com
solmarket.by streamin.to
sonnikforme.ru stromerrealty.com
spb-plitka.ru sunflowerdrawingpaintings.blogspot.com
spb.afora.ru supernew.org
sport-video-obzor.ru sureone.pro
sribno.net
standardchartered-forex.com
start.myplaycity.com
staynplay.net
streha-metalko.si
stroimajor.ru
suture.co suture.co
svnuppsalaorebro.se svetodiodoff.ru
svolze.com
swagbucks.com swagbucks.com
tastyfoodideas.com swiped.su
tecspb.ru swsociety.se
thebitcoincode.com t-bygg.com
thecoral.com.br tandvardshuset.net
td-33.ru
tech4master.com
techart24.com
tecnoteakviareggio.it
testbotprocessor44.com
tgtclick.com
thedownloadfromwarez.blogspot.com thedownloadfromwarez.blogspot.com
thenews-today.info thefotosgratis.eu
thetoiletpaper.com
thexart.club thexart.club
timecrimea.ru thiegs.reco.ws
tizanidine4mgprice.blogspot.com threecolumnblogger.com
tizanidineformigraines.blogspot.com thruport.com
tizanidinesideeffectsweightloss.blogspot.com tiandeural.ru
tizanidinewithdrawal.blogspot.com titslove.yopoint.in
toon-families.com tizanidine4mg.blogspot.com
torrentgamer.net tk-assortiment.ru
torrents-tracker.com toondinsey.com
tootoo.to
top10-way.com
topads.men
topanasex.com
topmira.com
torrent-newgames.com
torrentdownloadhub.com
tracfone.com tracfone.com
track112.site truck-addzilla.life
trafficjunky.net trustedhealthtips.com
trafficmonetize.org tv-spoty.info
trafficmp.com tvnewsclips.info
trafficnetzwerk.de tvoystartup.ru
tramadolandtizanidine.blogspot.com twitlinks.com
turist-strani.ru tytoona.com
tx41tclega.ru u555u.info
uasb.ru ugogo.info
ukrobstep.com ukkala.xyz
unionmarkt.de ukkelberg.no
unitygame3d.com um-razum.ru
uptime-gamma.net uogonline.com
urlopener.blogspot.com.au url2image.com
usiad.net uroffer.link
usswrite.com vchulkax.com
uzungil.com via-energy-acquistare.com
valsalud.com video-woman.com
valueclick.com videochat.bz
veerotech.com videos-for-your-business.com
video-camer.com videositename.com
videonsk.com viktoria-center.ru
villacoloniale.com vip-parfumeria.ru
vinsit.ru viteonlusarezzo.it
virtualbb.com vizag.kharkov.ua
volgograd.xrus.org vkontaktemusic.ru
voloomoney.com volume-pills.biz
vozbujdenie.com vse-pesni.com
vut.com.ru vtcdns.com
wallpapersist.com vucms.com
weblibrary.win vzubah.com
weblo.com vzube.com
weboptimizes.com wallpapers-best.com
webpromotion.ae watchinf.com
websitevaluebot.com watracker.net
webadvance.club
webs.com
website-analyzer.info
website-speed-up.site
webstatsdomain.org webstatsdomain.org
wedding0venues.tk wesharepics.com
whengirlsgowild.com westum.se
winner7777.net wholinkstome.com
womensplay.net whosonmyserver.com
woodyguthrie.se winterclassichockeyjerseys.com
wladimirpayen.com
wordkeyhelper.com
worldoffiles.ru worldoffiles.ru
wrona.it wowcasinoonline.ooo
writingservices17.blogspot.ru
www.888.com www.888.com
x-diesel.com www.ehscloud.cn
xbaboon.com www.kabbalah-red-bracelets.com
xchat26.myfreecams.com xaijo.com
xgames-04.com xaylapdiendanang.com
xcombear.ru
xfire.com
xmnb.net xmnb.net
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xn----7sbho2agebbhlivy.xn--p1ai xn----itbeirbjbi7bc6bh2d.xn--p1ai
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--80abgj3a5acid6ghs.top
xn--80aodinpgi.xn--p1ai xn--80aodinpgi.xn--p1ai
xtrafficplus.com xn--oogle-wmc.com
xuki.us xnxx-n.com
xzlive.com xwatt.ru
yellowstonevisitortours.com yachts-cruise.info
yes-com.com yebocasino.com
yginekologa.com yoursearch.me
ynymnwbm.bloger.index.hr yourserverisdown.com
youandcredit.ru youtuhe.com
youporn-ru.com yun56.co
youtubedownload.org zaapplesales.blogspot.com
ypmuseum.ru zawyna.ua
zagadki.in.ua zebramart.ru
zeroredirect7.com zeg-distribution.com
zirondelli.it zeroredirect12.com
zocaparj.kz zkjovpdgxivg.ga
zoogdiesney.com zog.link
zskdla.site
zx6.ru
zytpirwai.net zytpirwai.net

474
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
007angels.com 03e.info
1008.su 108shot.com
123cha.com 24videos.tv
2itech.ru 5tey7463.weebly.com
4senses.co aasoldes.fr
a.frcls.fr ahrntal.verymes.xyz
aac500b7a15b2646968f6bd8c6305869d7.com alert-fdm.xyz
activepr.ru allknow.info
acunetix-referrer.com anal-acrobats.hol.es
adconscious.com arcteryxsale.online
adloads.com arcteryxstore.online
ads.tremorhub.com artpicso.com
akita.kz
all4wap.ru
allboard.xobor.de
amospalla.es
angkortours.vn
apartment.ru
appsaurus.com
arabsexxxtube.com
asdfz.pro asdfz.pro
asseenontvonline.ru
atmovs.com atmovs.com
atout-energie-69.com atovh.local-finders.com
audiobangout.com autochoixspinelli.com
audiofree.ru avtochehli.by
autoseo-traffic.com azartclub.org
aviav.co azartmsl.com
aviav.ru battleforupdating.site
avramstroy.ru best-seo-offer.com
azazaporn.com bestcalovebracelet.cn
bestdraws.com
bestmobilityscooterstoday.com bestmobilityscooterstoday.com
bestmouthwash.club beztuberkuleza.ru
bonux.nextview.ru blog-hits.com
bookmark4you.com boleznikogi.com
brainboostingsupplements.org boltalko.xyz
breastaugmentation.co.za bookhome.info
browsepulse-a.akamaihd.net brillianty.info
buron.pw burger-tycoon.com
buy-cheap-online.info c1.onedmp.com
buyscabiescream.com cakesplus.com.au
carapics.com camdolls.com
cartierlove2u.xyz caulderwoodkennels.com
cejewelry.xyz cercacamion.it
cete.ru cheap-pills-norx.com
changde.58.com checktext.org
chlooe.com choosecuisine.com
citizenclsdriveri7.pen.io
clickbanksites.info clickbanksites.info
compliance-julia.top compliance-jane.top
compucelunlock.net compliance-nelson.top
contextualyield.com constantaservice.net
counterbot.com covetnica.com
creams.makeforum.eu crackguru.tk
crest3d.ru
crynet.cc crynet.cc
crystalslot.com customcollegeessays.net
customboxes4less.com customergrowthsystems.com
danashop.ru db.speedup.gdn
daymusam.com decorazilla.com
detskie-zabavi.ru detskie-konstruktory.ru
deutschland123.de diesel-parts28.ru
dinosaurus.site dimaka.info
djekxa.ru dimkino.ru
dojki365.online donna7753191.ru
domaincheck.io
donvito.unas.cz donvito.unas.cz
dostavka-v-ukrainu.ru downloadmefiranaratb1972.xpg.com.br
download-of-the-warez.blogspot.com draniki.org
doyouknowtheword-flummox.ml elektrischezigarette1.blog.pl
dscaas.website elektroniksigaraankara.info
earthmagic.info elektrozigaretten1.tumblr.com
elektrozigarette1.dreamwidth.org erectile.bid
empis.magix.net essay-writing.work
eshop.md evehemming.blogspot.com.au
evening-dating-club.info ezaz.info
face.hostingx.eu ezigarettekaufen1.hpage.com
fealq.com ezigarettekaufen2.mpbloggar.se
fetishinside.com findacheaplawyers.com
fighrofacciufreesig.ga floating-share-buttons.com
firstdrugmall.ru freewebs.com
forodvd.com gambarxkata.co
forpackningsutveckling.se gamewrath.com
forumprofi.de gateway.zscalertwo.net
fotoxxxru.com gdzkurokam.ru
free-floating-buttons.com gembird.com
freefoto.ca gfaq.ru
freejabs.com glass-msk.ru
fusoradio.info gnuetella.com
gdcentre.ru googleseo.com.tr
gearcraft.us gopixdatabase.com
gepezz.info greenzaim.ru
gg-arena.ru gribokstop.com
goroda-vsego-mira.ru gsasearchengineranker.site
gosarhivrt.ru gsmlab.pl
gosreg.amchs.ru guardlink.org
graphid.com gunsvicceadadebt.tk
grupografico-pilar.com.ar gz2.bbsoldes.fr
habermetre.com hanwei.us
hamilton.ca hchha.com
hannasolution.ru hmmm.cz
healing-dysplasia.ru hollywoodactress.info
hitmuzik.ru hoverboard360.de
homemature.net hunterboots.online
hothot.ru hvd-store.com
howtoclean.club ido3.com
htmlcorner.com ifirestarter.ru
hybrid.ru igithab.com
imk.com.ua ilovevitaly.ru
in-tandem.co imperia31.ru
informatiecentro.be infostatsvc.com
ipornox.xxx instasexyblog.com
isitpaleo.info ipostroika.ru
islamtoday.co.za iqupdatetmz.win
istizanidineacontrolledsubstance.blogspot.com irkutsk.online-podarki.com
jamiembrown.com james13prix.info
javspace.net
je7.us
jerseywholesalebizchina.com
jofucipiku.tk
johannesburgsingles.co.za johannesburgsingles.co.za
jumptap.com joyceblog.top
kenaba.su kemerovo.zrus.org
khadastoafarde.tk kerch.site
kw21.org kinoplen.ru
kwzf.net kolotiloff.ru
lancheck.net konoplisemena.com
leboard.ru labelwater.se
lifebyleese.com lacloop.info
lincolntheatre.com lankarns.com
linkbux.ru laxdrills.com
lmrauction.com letsrepair.in
lockerz.com likrot.com
lombia.com liran-locks.com
liupis.com
live-cam6.info
ljusihus.se
lonely-mature.com
louisvuittonoutletstore.net
luciddiagnostics.in
m4ever.net m4ever.net
manifestyourmillion.com mamasuper.prom.ua
maranbrinfo.com.br marblestyle.ru
marktforschung-stuttgart.com mcnamaratech.com
mebel-vstroika.ru medi-fitt.hu
megahdporno.net medispainstitute.com.au
menetie.ru medizinreisen.de
mini.7zap.com medkritika.ru
misterjtbarbers.com megakino.net
mountainstream.ms metaxalonevstizanidine.blogspot.com
moz.com mex-annushka.ru
mp3ringtone.info mlvc4zzw.space
mwtpludn.review monarhs.info
mydownlodablefiles.com mosrif.ru
mygameplus.com myplaycity.com
nagdak.ru mystats.xyz
nakozhe.com nakozhe.com
nalogovyy-kodeks.ru
narco24.me narco24.me
netcheckcdn.xyz narkologiya-voronezh.ru
november-lax.com naturalbreakthroughsresearch.com
online247.ml navalwiki.info
onlyporno.ru nerudlogistik.ru
ons-add.men nudejapan.net
open-odyssey.org oakleyglassesonline.us
oyster-green.com odywpjtw.bloger.index.hr
painting-planet.com okeinfo.online
panouri-solare-acoperis.com oksrv.com
patterntrader-en.com orsonet.ru
pdn-4.com partnerline.men
phormchina.com pb-dv.ru
picsfair.com pensplan4u.com
pinstake.com picturesify.com
pivka.xyz piratecams.com
pl.aasoldes.fr pizda.lol
pornstartits.xblog.in poiskzakona.ru
prchecker.info popmarker.com
prefersurvey.net popup-hgd.xyz
prizeestates.cricket pornoklad.ru
prostitutki-astana.org portal-eu.ru
pwwysydh.com purchasepillsnorx.com
qwarckoine.com push-ad.com
registry-cleaner.net rankscanner.com
relatodelpresente.com.ar referencemoi.com
resultshub-a.akamaihd.net remmling.de
riders.ro replicaclub.ru
ring4rhino.com rospromtest.ru
robertefuller.com runstocks.com
roleforum.ru rusprostitute.com
ruex.org.ua safe-app.net
ruspdd.com sakhboard.ru
schlampen-treffen.com savefrom.com
searchengineranker.email sayyoethe.blogspot.co.za
seasaltwithfood.com sbricur.com
sex-watch.com sbt-aqua.ru
sexgalleries.top scanner-mary.top
sexyteens.hol.es seeresultshub-a.akamaihd.net
shtora66.ru seo-traffic-ranking.info
shymkent.xkaz.org seorankinglinks.xyz
sildenafil-tadalafil.info seotoolsagency.com
sim-service.net sex-foto.pw
siteaero.com sexsearch.com
smailik.org sexy.babes.frontend-stack.top
spamnuker.com silvermature.net
stats-public.grammarly.io sitopreferito.it
steamoff.net soblaznu.net
strfls.com socas.pluto.ro
subj.ukr-lit.com solnplast.ru
superstarfloraluk.com sonata-arctica.wz.cz
syndicate.fun special-porn.com
tacbelarus.ru ssconstruction.co
taqywu51.soup.io stairliftsarea.com
thenews-today.info stathat.com
thiegs.reco.ws streha-metalko.si
tizanidine6mg.blogspot.com stuff-about-money.com
tizanidineandgabapentin.blogspot.com t-bygg.com
tizanidinehcl4mgtablet.blogspot.com tecnoteakviareggio.it
tmtrck.com telsis.com
tongkatmadura.info thebluffs.com
toonfamilies.net timdreby.com
tubeline.biz tooplay.com
tula.mdverey.ru top10-online-games.com
uac.net.au touchmods.fr
ugguk.online toy-shop.top
ukrtextbook.com trichizobswiv.agddns.net
unimodemhalfduplefw.pen.io trucri.me
uptime-delta.net unece.org
vashsvet.com unpredictable.ga
viagengrarx.com usiad.net
video-camer.com varbergsvind.se
vkontaktemusic.ru veles.shop
vksaver-all.ru viandpet.com
vladimir.zrus.org videochat.tv.br
vote-up.ru vintontech.info
wallpapersist.com vitalads.net
walpaperlist.com vltai.com
webenlace.com.ar vse-pesni.com
webmasters.stackexchange.com wahicbefa31.soup.io
webs.com web.cvut.cz
where-toget.com website-audit.com.ua
wikes.20fr.com website-speed-checker.site
wjgony.com wetgames.ru
wwwadultcheck.com wholesalecheapjerseysfree.com
x-porno.video wieseversa.no
wildworld.site
wingsofrefuge.net
worldmusicfests.com
wovis.site
woweb.com.ua
ws.ampower.me
wstroika.ru
wufak.com
x-stars.ru x-stars.ru
xn----7sbifcamovvfggw9d.xn--p1ai xev.ru
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai xn--80ajbshivpvn2i.xn--p1ai
xn--d1abj0abs9d.in.ua xtube.com
xportvusbdriver8i.snack.ws
xvideosbay.com xvideosbay.com
xxxsiterips.xyz xxxhdvideo.site
youradexchange.com yougotanewdomain.com
zaim-pod-zalog-krasnodar.ru zkjovpdgxivg.ga
zarepta.com znakomstva-moskva77.ru
zhongwenlink.com zog.link
zverokruh-shop.cz zzlgxh.com

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.11 0; 10.1.0.132 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2798 ### Version: V4.2021.12.2799
### Updated: Tue Dec 21 22:01:20 UTC 2021 ### Updated: Wed Dec 22 10:03:25 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2798 ### Version: V4.2021.12.2799
### Updated: Tue Dec 21 22:01:20 UTC 2021 ### Updated: Wed Dec 22 10:03:25 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##