vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.12.2800 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-12-22 22:07:05 +00:00
parent deaf482dcd
commit f559b8a611
48 changed files with 80459 additions and 80459 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2799 #### Version: V4.2021.12.2800
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2799 #### Version: V4.2021.12.2800
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.12.2799 #### Version: V4.2021.12.2800
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9922
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.12.2799 #### Version: V4.2021.12.2800
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9926
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9926
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

9926
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9926
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 1; 10.1.0.237 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 0; 10.1.0.237 0;
127.0.0.1 0; 127.0.0.1 0;

9926
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 1; 10.1.0.237 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 0; 10.1.0.237 0;
127.0.0.1 0; 127.0.0.1 0;

9926
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 1; 10.1.0.237 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 0; 10.1.0.237 0;
127.0.0.1 0; 127.0.0.1 0;

9926
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2799 2800

9926
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

9926
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.112.106.111 0; 20.83.104.151 0;

9926
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 1; 10.1.0.237 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 0; 10.1.0.237 0;
127.0.0.1 0; 127.0.0.1 0;

9926
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 1; 10.1.0.237 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 0; 10.1.0.237 0;
127.0.0.1 0; 127.0.0.1 0;

9926
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 1; 10.1.0.237 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 0; 10.1.0.237 0;
127.0.0.1 0; 127.0.0.1 0;

9926
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.112.106.111 0; 20.83.104.151 0;

9926
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.112.106.111 0; 20.83.104.151 0;

9926
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 1; 10.1.0.237 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BBBike PageThing.com
Brandprotect Piepmatz
Gigabot SemanticJuice
IstellaBot SpaceBison
Libwhisker SputnikBot
MIIxpc Sqlworm
ProWebWalker Sucuri
ScreenerBot WebImageCollector
VeriCiteCrawler Widow
Web Fuck scalaj-http

294
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
80legs 404checker
AIBOT 404enemy
ALittle Client ASPSeek
Abonti AhrefsBot
Aboundexbot
AiHitBot AiHitBot
Aipbot Alexibot
Alligator Alligator
AlphaBot
Anarchy
Ankit Ankit
Apexoo AwarioSmartBot
Attach
AwarioRssBot
BBBike BBBike
BLEXBot BLEXBot
BackStreet BackStreet
BacklinkCrawler BacklinkCrawler
Bandit Bandit
Bigfoot Barkrowler
Bitacle
Black Hole
Blow Blow
Buck BotALot
Brandprotect
BuiltBotTough BuiltBotTough
BuiltWith CSHttp
CCBot
CODE87
Calculon Calculon
CensysInspect
CheTeam
ChinaClaw ChinaClaw
Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot Collector
Copier Copier
CopyRightCheck Crawling at Home Project
CrazyWebCrawler Crescent
Custo Custo
DSearch DSearch
DTS Agent
DataCha0s
Demon Demon
Devil Deusu
Dispatch
DittoSpyder DittoSpyder
DomCopBot DnyzBot
DomainCrawler DomainAppender
DomainStatsBot DomainSigmaCrawler
Domains Project
Dotbot Dotbot
Download Wonder
ECCP/1.0 ECCP/1.0
EMail Siphon Evil
EMail Wolf Exabot
Ebingbong Extreme Picture Finder
Ecxi
EirGrabber
EroCrawler
Express WebPictures
ExtLinksBot
Ezooms Ezooms
FHscan
FemtosearchBot
Fimap Fimap
Foobot Foobot
Fuzz Fuzz
FyberSpider FyberSpider
GT::WWW Fyrebot
GalaxyBot G-i-g-a-b-o-t
GetRight
GetWeb GetWeb
Getintent Go!Zilla
Gigabot GoZilla
Gotit
GrabNet
Grabber
Grafula Grafula
GrapeFX
GridBot
HEADMasterSEO HEADMasterSEO
HTMLparser HTMLparser
HTTP::Lite Haansoft
Harvest Harvest
Humanlinks Humanlinks
IDBot Iblog
Id-search
IlseBot
Image Fetch
Indy Library Indy Library
InfoNaviRobot InfoNaviRobot
Intelliseek
InterGET InterGET
Internet Ninja Iskanie
InternetSeer JOC Web Spider
IstellaBot
JamesBOT
Jbrofuzz
JetCar JetCar
Joomla JikeSpider
Kenjin Spider JustView
Keyword Density
Kinza Kinza
LWP::Simple
Lanshanbot
Leap
LeechGet
LexiBot LexiBot
Lftp Lftp
LibWeb LinkScan
LinkWalker LinkpadBot
Linkdexbot LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Litemage_walker LinqiaScrapeBot
Lipperhey
Lmspider Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MJ12bot MIDown tool
MQQBrowser MQQBrowser
MSIECrawler Mail.RU_Bot
Majestic SEO
Majestic-SEO
Majestic12 Majestic12
MarkMonitor MarkWatch
Mass Downloader Mass Downloader
MauiBot Mata Hari
Mb2345Browser
MeanPath Bot MeanPath Bot
Microsoft Data Access Metauri
MicroMessenger
Microsoft URL Control
Minefield Minefield
Mister PiX Moblie Safari
Mojeek Mojeek
Morfeus Fucking Scanner Mozlila
Musobot Mr.4x3
NICErsPRO
NPbot
Name Intelligence
Nameprotect
Navroad Navroad
Needle Needle
Net Vampire
NetAnts NetAnts
NetLyzer NetLyzer
NetZIP NetSpider
Netcraft Netcraft
Nettrack
Netvibes Netvibes
Nibbler NextGenSearchBot
Not Nikto
NimbleCrawler
Offline Explorer Offline Explorer
OnCrawl
OpenLinkProfiler OpenLinkProfiler
Openfind
OrangeBot
OrangeSpider OrangeSpider
OutfoxBot OutclicksBot
PECL::HTTP PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PeoplePal Page Analyzer
PageGrabber
Pandalytics
Piepmatz Piepmatz
Pixray
PleaseCrawl PleaseCrawl
Probethenet ProWebWalker
Psbot PyCurl
Pu_iN RSSingBot
Pump RankingBot
Quick-Crawler Rankivabot
RankActiveLinkBot
RankFlex
RankurBot RankurBot
Re-re Recorder
Reaper RocketCrawler
RedesScrapy SBIder
Rogerbot
SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats
SalesIntelligent SalesIntelligent
ScoutJet
Screaming
ScrepyBot
SearchmetricsBot
SemanticJuice
SemrushBot SemrushBot
SeobilityBot SeoSiteCheckup
Seomoz Seomoz
Siphon Site Sucker
SiteExplorer SiteExplorer
SiteSnagger SiteLockSpider
SiteSucker
Sitebeam
SlySearch SlySearch
Spinn3r SmartDownload
SputnikBot Snapbot
Sqlmap SocialRankIOBot
Sqworm Sogou web spider
Steeler SpankBot
Stripper Spbot
SurveyBot Sqlworm
Sucker
SuperBot
Suzuran Suzuran
TheNomad Swiftbot
T0PHackTeam
Teleport
TeleportPro
Telesphoreo
Telesphorep
Toweyabot
Trendiction Trendiction
TwengaBot Turnitin
Twice Twice
Typhoeus URLy.Warning
UnisterBot UnisterBot
V-BOT
VCI
Vagabondo Vagabondo
VidibleScraper VelenPublicWebCrawler
VoidEYE Virusdie
Voil WEBDAV
WISENutbot WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW-Mechanize
Wallpapers WWW::Mechanize
Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE WeSEE
Web Auto Web Auto
Web Pix Web Fuck
Web Sauger Web Sauger
WebBandit Web Sucker
WebAuto
WebCollage WebCollage
WebCopier
WebEnhancer
WebFetch WebFetch
WebFuck
WebImageCollector WebImageCollector
WebLeacher WebReaper
WebPix
WebSauger WebSauger
WebSucker WebSucker
WebZIP WebWhacker
Webalta Webshag
WebmasterWorldForumBot
Website Quester
WebsiteQuester WebsiteQuester
Whack Who.is Bot
Wonderbot WinHTTrack
WiseGuys Robot
Woobot
Wprecon Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot Zade
Zauba ZoominfoBot
Zermelo
ZoomBot
ZumBot
adscanner
archive.org_bot
backlink-check
clark-crawler clark-crawler
coccocbot-web crawl.sogou.com
com.plumanalytics
crawler.feedback crawler.feedback
demandbase-bot domainsproject.org
facebookscraper facebookscraper
heritrix
internetVista monitor internetVista monitor
isitwp.com isitwp.com
lwp-request lwp-trivial
mediawords magpie-crawler
muhstik-scan
oBot oBot
page scorer
pcBrowser pcBrowser
plumanalytics
polaris version polaris version
probe-image-size
s1z.ru s1z.ru
satoristudio.net satoristudio.net
scalaj-http seoscanners
seobility
seostar
serpstatbot serpstatbot
sexsearcher sexsearcher
sitechecker.pro
sp_auditbot sp_auditbot
tAkeOut trendiction.com
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webgains-bot webgains-bot
webmeup-crawler
webpros.com webpros.com
zauba.io webprosbot
zgrab x09Mozilla
xpymep1.exe

284
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 404checker
80legs
AIBOT AIBOT
ALittle Client ALittle Client
Abonti ASPSeek
Aboundex
Acunetix
AhrefsBot AhrefsBot
AllSubmitter
Alligator Alligator
AlphaBot AlphaBot
Anarchy
Anthill Anthill
Apexoo
Aspiegel Aspiegel
Asterias
Attach Attach
AwarioRssBot BackDoorBot
BLEXBot Bandit
BackStreet
Badass
Barkrowler Barkrowler
BatchFTP
Battleztar Bazinga Battleztar Bazinga
BetaBot
Bigfoot Bigfoot
Blackboard Boardreader
Bolt Bolt
BotALot
Brandwatch
Buck
Buddy Buddy
BuiltWith BuiltWith
Bullseye
BunnySlippers BunnySlippers
Calculon BuzzSumo
CCBot
CSHttp
CazoodleBot CazoodleBot
Cegbfeieh CheeseBot
ChinaClaw CherryPicker
Chlooe
Claritybot Claritybot
Cloud mapping Cloud mapping
Copier Cocolyzebot
Cogentbot
Copyscape Copyscape
Cosmos
Craftbot Craftbot
CrazyWebCrawler
CyotekWebCopy CyotekWebCopy
DSearch DSearch
DTS Agent DTS Agent
DataCha0s DataCha0s
Dirbuster
Disco Disco
Discoverybot
Dispatch Dispatch
DnBCrawler-Analytics DittoSpyder
DomainSigmaCrawler DomCopBot
Dotbot Dragonfly
ECCP/1.0 Drip
EMail Wolf
EasyDL EasyDL
Ebingbong Ecxi
EirGrabber EirGrabber
EroCrawler EroCrawler
Express WebPictures ExtLinksBot
FDM
FHscan
FemtosearchBot
Fimap Fimap
Firefox/7.0
Flunky Flunky
Freeuploader Freeuploader
FrontPage FrontPage
Fuzz G-i-g-a-b-o-t
FyberSpider
Fyrebot
GT::WWW GT::WWW
GalaxyBot GalaxyBot
GetWeb GermCrawler
Getintent GetRight
Gigabot
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GoZilla
GrabNet HTTP::Lite
GrapeshotCrawler
HEADMasterSEO
Haansoft Haansoft
HaosouSpider Hloader
Harvest
Humanlinks Humanlinks
IDBot IDBot
IRLbot Id-search
IlseBot IlseBot
Image Fetch
IndeedBot IndeedBot
InfoNaviRobot Intelliseek
InfoTekies Internet Ninja
InternetSeer Iskanie
Iria IstellaBot
JOC Web Spider
JennyBot JennyBot
JetCar Jetty
JikeSpider JikeSpider
Joomla Joomla
Jorgee JustView
Kenjin Spider Kenjin Spider
Keyword Density
Kinza
Kozmosbot Kozmosbot
LeechFTP LWP::Simple
LeechGet Lanshanbot
LexiBot LibWeb
Libwhisker Lightspeedsystems
Linkdexbot LinkScan
LinkextractorPro Linkbot
LinksManager LinksManager
LinqiaRSSBot MIIxpc
Lipperhey Mail.RU_Bot
Lmspider
Ltx71
MFC_Tear_Sample
MJ12bot
MSIECrawler
MTRobot
Mag-Net
Magnet
Majestic SEO Majestic SEO
Majestic12
MarkWatch
Masscan Masscan
Mata Hari Mata Hari
MauiBot MegaIndex.ru
MeanPath Bot Minefield
MicroMessenger
Microsoft Data Access
Microsoft URL Control
Mister PiX
Mojeek Mojeek
Mojolicious Musobot
Mr.4x3
Msrabot
NICErsPRO
NPbot NPbot
Nameprotect
NearSite NearSite
Needle Needle
Nessus
NetLyzer NetLyzer
NetMechanic
NetSpider NetSpider
NetZIP Nettrack
NextGenSearchBot Netvibes
Nikto Niki-bot
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Openfind Offline Explorer
OutclicksBot Offline Navigator
OpenLinkProfiler
OpenVAS
Openvas
OrangeBot
PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Page Analyzer
Pi-Monster PageThing.com
Picscout
Picsearch Picsearch
PictureFinder Piepmatz
Pimonster Pimonster
Pixray Pixray
PleaseCrawl PleaseCrawl
Pockey ProWebWalker
ProPowerBot Pu_iN
Probethenet
Psbot
PxBroker PxBroker
PyCurl
QueryN Metasearch QueryN Metasearch
RankActive Quick-Crawler
RankurBot RankActiveLinkBot
RankFlex
Re-re Re-re
ReGet Reaper
RepoMonkey RepoMonkey
Ripper Ripper
RocketCrawler
Rogerbot Rogerbot
SEOlyticsCrawler SEOkicks
SEOkicks-Robot
SEOstats SEOstats
Scrapy SISTRIX
SalesIntelligent
ScanAlert
Screaming Screaming
ScreenerBot ScreenerBot
ScrepyBot SearchmetricsBot
SemanticJuice SemanticJuice
Semrush SentiBot
Seomoz SeoSiteCheckup
SiteLockSpider SiteCheckerBotCrawler
SiteSucker
Sitebeam
Siteimprove Siteimprove
SmartDownload Sitevigil
Snoopy SlySearch
Sociscraper Snake
Sogou web spider
Sottopop Sottopop
Spanner SpaceBison
Spammen
Spbot Spbot
SputnikBot
Sqlmap Sqlmap
SuperBot Sqlworm
Surfbot Sqworm
SurveyBot Stripper
Szukacz SuperHTTP
Suzuran
Teleport Teleport
Telesoft
Telesphorep
The Intraformant The Intraformant
TightTwatBot TightTwatBot
Toweyabot
Tracemyfile Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot True_Robot
Turingos
Turnitin
TurnitinBot TurnitinBot
TwengaBot
Twice Twice
URLy Warning Typhoeus
URLy.Warning UnisterBot
Upflow VCI
V-BOT Vacuum
VB Project Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper VidibleScraper
Virusdie Voltron
WBSearchBot WBSearchBot
WEBDAV
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers Wallpapers/3.0
WeSEE Web Collage
Web Auto Web Sucker
Web Sauger WebBandit
WebImageCollector WebCopier
WebPix WebEnhancer
WebReaper WebReaper
WebSauger Webalta
WebZIP WebmasterWorldForumBot
Website Quester
Widow
WinHTTrack WinHTTrack
Wotbox Wonderbot
Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zermelo
Zeus
ZmEu ZmEu
ZoomBot
ZumBot ZumBot
ZyBorg
adscanner
arquivo-web-crawler
autoemailspider autoemailspider
backlink-check backlink-check
com.plumanalytics cah.io.community
clark-crawler
crawl.sogou.com
crawler.feedback
crawler4j
dataforseo.com dataforseo.com
eCatch demandbase-bot
evc-batch evc-batch
magpie-crawler instabid
internetVista monitor
isitwp.com
lwp-request
lwp-trivial
mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler page scorer
plumanalytics plumanalytics
scan.lol scalaj-http
seobility
seocompany.store seocompany.store
seoscanners
serpstatbot
sexsearcher
siteripz
sogouspider
spyfu spyfu
tAkeOut
ubermetrics-technologies.com ubermetrics-technologies.com
webgains-bot x09Mozilla
webmeup-crawler

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
freeuploader.com bestserials.com
hybrid.ru byme.se
manimpotence.com cartierreplica.pw
plugingeorgia.com centraletermice.us
sanidumps.com greatfind-a.akamaihd.net
sexkontakteao.info mundoaberrante.com
the-trader.net pandarastore.top
webmonetizer.net resellerclub.com
womens-journal.net samolet.fr
zigarettenonlinekaufen2.drupalgardens.com trafficmania.com

930
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
000free.us 03p.info
178evakuator178.ru 1001watch.com.ua
214jaluwobapef.cf 1stat.ru
4senses.co 21h2o.com
51unlim.ru 2728fb936f0.com
70casino.online 3xforum.ro
8xv8.com 404.mn
aaa38852e886ac4af1a3cff9b47cab6272.com 5ws.dating-app.ru
abctoppictures.net 72-news.com
888.com
DomainStatsBot
abclauncher.com
above.com above.com
actices.com acortarurl.es
adf.ly activepr.ru
adloads.com adconscious.com
admeasures.com addray.pro
adsloads.com adminshop.com
adssafeprotected.com adtech.us
advancedcleaner.com advokateg.ru
aihelen.net adzerg.com
aktivator-windows10.blogspot.com affrh2015.com
allkrim.com afslankpillen2017nl.eu
allsilver925.co.il agapovdg.ru
alphaforum.ru airlandsea.info
alert-fdm.xyz
alfabot.xyz
aliexpresscashback.club
all4invest.info
alphahoverboards.com
amazon-adsystem.com
amazon-seo-service.com
ample-awards-today.us ample-awards-today.us
anabolics.shop animali.deagostinipassion.it
android-vsem.org anlimebel.kiev.ua
angigreene.com api.stathat.com
anonymousfox.co apiqualitinknet-a.akamaihd.net
apimegabrowsebiz-a.akamaihd.net apper.de
app5.letmacworkfaster.world arendas.net
apps-analytics.net arkkivoltti.net
apxeo.info asdfz.pro
arabseyes.com aussie-prizes.com
arclk.net auto-zapchasti.org
auto.rusvile.lt autoua.top
autorn.ru aviav.eu
autovideobroadcast.com avon-novosib.ru
avtochehli.by azbukafree.com
azartniy-bonus.com b00kmarks.com
backiee.com babs.com.ua
balitouroffice.com back.dog
barnfurnituremart.com backgroundpictures.net
battle.net bamo.xsl.pt
becuo.com banksrf.ru
best-coupon-offer.com bedcapdealers.com
begalka.xyz
betshuckclean.com
betterscooter.com
beyan.host.sk beyan.host.sk
bgviagrachrx.com beztuberkuleza.ru
bibys.com bif-ru.info
big-cash.net bigames.online
biketank.ga bildsuche.ru
binomo.kz binomo.com
birzha-truda.eu bio.trade-jp.net
biteg.xyz bjgugu.net.ua
bitnote.co bkgr.se
bjorkbacken.nu bloke.com
bloggers.nl bluejays-jerseys.us
bolezniorganov.ru bmwhighperformers.com
book-bets.com bochemit.com.ua
bouda.kvalitne.cz bocoarchives.org
brains2.biz brateg.xyz
braintobe.win bristolhotel.com
breastaugmentation.co.za bristolhotel.com.ua
budilneg.xyz buchananshardware.com
buy-cheap-online.info buckspar.gq
bum.com.ru
bumascloset.com
bumskontakte.org
butstrap.space
buy-forum.ru buy-forum.ru
buy-meds24.com
buyessaynow.biz
buyessayonline19.blogspot.ru buyessayonline19.blogspot.ru
buypillsorderonline.com byme.se
callawaygolfoutlet.online calcularpagerank.com
candlehandmade.com candypeople.se
canvas.gloverid.site capsderfudd.tk
carabela.com.do cardiosport.com.ua
carsplate.com cardsdumps.com
cartierring.xyz cartierloveringreplica.xyz
cascadelink.org cartierlovestore.com
cdncash.com cartierreplica.win
cdncash.org
changde.58.com changde.58.com
cheapmoncler.xyz chcu.net
cherrypointplace.ca cheapjerseysbizwholesale.us
chinajerseyswholesalecoupons.com chinacheapelitejerseys.com
christianlouboutinsaleoutletonline.us chuckguilford.com
ci.ua cialgenisrx.com
citetick.com cinemaenergy-hd.ru
civilwartheater.com cityadspix.com
clean-start.net clarithromycin500mg.com
clickcash.com clmforexeu.com
cmd.kz cmd.kz
cmhomestayagency.com coin-hive.com
cntravelre.com com-onlinesupport.host
compliance-checker-7.info com-onlinesupport.site
compliance-irvin.top companies-catalog.com
compliance-jack.top compliance-andrew.top
compliance-fred.top
compliance-josh.top compliance-josh.top
compliance-olga.top compliance-willy.top
cookie-law-enforcement-hh.xyz consorzioilmosaico.org
countbertwistdisp26.soup.io contentsexpress.com
cool-wedding.net
copblock.org
cornerstone-countertops.com
covetnica.com covetnica.com
creditnation.ru crackzplanet.com
csgo4.win crazyprotocol.com
customcatchcan.com creams.makeforum.eu
cypernhuset.se daisye.top
dalavia.ru danashop.ru
dev78.dioniqlabb.se daneshetabiat.com
dfwu1013.info ddospanel.com
delta-line.men
dengi-pod-zalog-nedvizhimosti.ru
depositfiles-porn.ga
dev-seo.blog
dfiles.me
dfwu1019.info
diamond-necklace.info diamond-necklace.info
diffbot.com digital-sale.su
digital-video-processing.com digital-video-processing.com
dinkolove.ya.ru dimaka.info
divci-hry.info dispo.de
diy-handmade-ideas.com diy-handmade-ideas.com
dlya-android.org dodlive.mil
dogsrun.net
dokfilms.net dokfilms.net
domain-tracker.com domainsatcost.com
domain.webkeyit.com domoysshop.ru
dood.live dotmass.top
doreenblog.online download-wallpaper.net
doska-vsem.ru downloadmefiranaratb1972.xpg.com.br
earn-from-articles.com e-avon.ru
easyukraine.com eandsgallery.com
eets.net elektrischezigaretten2.beeplog.com
elektrischezigaretten1.blogse.nl elektronischezigarettekaufen1.beeplog.com
elektronischezigarettekaufen2.tumblr.com
elektrozigarette1.dreamwidth.org
encodable.com encodable.com
enternet.ee energydiet-info.ru
es5.com enhand.se
estibot.com erectile.bid
everytalk.tv ertelecom.ru
executehosting.com essaypro.com
expertnaya-ocenka.ru etur.ru
eyes-on-you.ga europages.com.ru
eywords-monitoring-your-success.com evrotekhservis.ru
ezb.elvenmachine.com exoneration-project.us
ezofest.sk expertblog.info
expolicenciaslatam.co
extads.net
extlabs.io
eyelike.com.ua
fable.in.ua
face.hostingx.eu
family1st.ca family1st.ca
fashion.stellaconstance.co famix.xyz
fashionavenuegame.com fearcrow.com
fba-mexico.com fickluder69.com
fikasound.tk
fillmewithhappiness.com
filmbokep69.com filmbokep69.com
filmi-v.online filmci.pro
financepoints.eu filmidivx.com
find1friend.com findacheaplawyers.com
finemanteam.com findsexguide.com
findthe.pet
flash4fun.com
flauntyoursite.com
flowersbazar.com
folowsite.com folowsite.com
forumprofi.de forex-indextop20.ru
fotopop.club forms-mtm.ru
franecki.net frameimage.org
frequiry.com free-share-buttons.blogspot.com
freshberry.com.ua freeuploader.ml
freshdz.com front.ru
fullfileaccess.com fun2cell.net
fusoradio.info galeon.com
gael-s.ru gambarkatabaru.com
gambarxkata.co gamblingnerd.com
gavuer.ru gamebackyard.com
gaypornmovie.net gameplexcity.com
gepezz.info gamerextra.com
getoutofdebtfree.org gay-site.store
glavprofit.ru ge0ip.com
globalscam.ga gearsadspromo.club
goldenggames.com geileweiber.tk
gotomontenegro.net geoads.com
gov.yanao.ru girlsatgames.ru
gpirate.com girlspicsa.com
graphicwe.org global-ics.co.za
hccoder.info go2album.com
hdwalls.xyz good-mummy.ru
helpmymacfaster.trade googlemare.com
hiwibyh.bugs3.com gorabagrata.ru
hjaoopoa.top gosreg.amchs.ru
hobild.net greenidesign.co
home-task.com greenshop.su
homeart.space grizzlysgrill.com
honyaku.yahoofs.jp gsasearchengineranker.top
host-protection.com gsasearchengineranker.xyz
hotdl.in gsasearchenginerankerdiscount.com
hotenergy.ru gwagka.com
h2monline.com
hamptonoaks.ca
hazardky.net
hermesbelts.xyz
hkdiiohi.skyrock.com
hollywoodactress.info
homedecorpicture.us
hotelcrocenzi.sm
hoterika.com
hothor.se hothor.se
hotshoppymac.com hunthillfarmtrust.org
humanorightswatch.org hydropump.su
hvd-store.com
iblogpress.xyz iblogpress.xyz
id-forex.com igithab.com
ihtec2019.org iinstalll-fii1leis.jus0wil.pp.ua
iklysha.ml
ilovevitaly.xyz
imagecoolpub.com imagecoolpub.com
inbabes.sexushost.com
infodocsportal.com infodocsportal.com
infostatsvc.com innodgfdriverhm.aircus.com
insomniagamingfestival.com izhevsk.zrus.org
insta-add.pro james13prix.info
int.search.mywebsearch.com japfm.com
intelhdgraphicsgtdrive6w.metroblog.com javcoast.com
intellekt21.ru jeremyeaton.co
interferencer.ru jerseywholesalechinabiz.com
internetproviderstucson.com jobgirl24.ru
io9.com juliadiets.com
iscblog.info jyrxd.com
islandminingsupply.wordpress.com
isoveti.ru
iyasimasennka.com
jav-way.site
jerseyschinabizwholesale.us
jjbabskoe.ru
jovencitas.gratis
jpcycles.com
junketjuice.blogspot.com
justbcause.com
kaidalibor.de
kamen-e.ru
kartiniresto.com kartiniresto.com
kchaxton.com katjimej.blog.fc2.com
khadastoafarde.tk kichenaid.ru
kino2018.club kihi.gdn
kongoultry.net kinoflux.net
kongruan.com kleine-titten.biz
kretpicf.pw klejonka.info
larger.io koleso24.com.ua
larutti.ru komputernaya-pomosh-moscow.ru
lernur.net konturkrasoty.ru
letmacworkfaster.site koral.se
letslowbefast.site krynica.info
letslowbefast.today kurbappeal.info
lider-zhaluzi.kiev.ua labplus.ru
lion.bolegapakistan.com law-check-nine.xyz
liquimondo.com lezbiyanki.net
livingcanarias.com link.ac
lizace.com lion.snapmap.us
locksmith.jp lodki-pvh.dp.ua
lol-smurfs.com longadventure.com
lovelycraftyhome.com lotto6888.com
maglid.ru louboutinshoes.xyz
mantramusic.ru luxup.ru
masturbate.co.uk magento-crew.net
meble-bogart.info managerpak204.weebly.com
meet-flirt-dating.com marketingtechniques.info
meetingrainstorm.bid marketland.ml
mega-bony2017.pl massage-info.nl
mere.host.sk mature.free-websites.com
meteostate.com medicines-choice.com
methodsmarketing.com megavolt.net.ua
michellblog.online mielec.pl
microsoftportal.net migente.com
microstatic.pl milblueprint.com
minyetki.ru militarysale.pro
mirmedinfo.ru minet.club
mir-limuzinov.ru
missis.top
misslike.ru misslike.ru
missvietnam.org monarhs.info
mobplayer.ru moneyzzz.ru
moesen-ficken.com mouselink.co
mojpregled.com
monclerjacketsoutlet.pw
moncleronline.xyz
montazhnic.ru
moxo.com moxo.com
moyaterapiya.ru muz-baza.net
mrsdalloways.com
musflashtv.com
mustwineblog.com
myblogregistercm.tk myblogregistercm.tk
myhitmp3.club myhealthcare.com
mykings.pw mylesosibirsk.ru
mypets.by myperiod.club
myyour.eu
na15.ru na15.ru
nalogovyykodeks.ru naj-filmy24.pl
namecrumilchlet.tk naturalshair.site
narco24.me naval.jislaaik.com
nash-krym.info news-readers.ru
nelc.edu.eg nicefloor.co.uk
netfacet.net novodigs.com
newstaffadsshop.club nw-servis.ru
nfljerseyscheapchinabiz.com o-o-6-o-o.com
nonews.co offer.camp
npoet.ru offf.info
obiavo.su online247.ml
obuv-kupit.ru onlinebay.ru
ok-ua.info ontargetseo.us
okmusic.jp ororodnik.goodbb.ru
omoikiri-japan.ru ostroike.org
onclickpredictiv.com ownshop.win
onefilms.net oxotl.com
onescreen.cc panamaforbeginners.com
onlainbesplatno.ru paperwritingservice17.blogspot.ru
onstrapon.purplesphere.in
ooo-gotovie.ru
open-odyssey.org
oxford-book.com.ua
oyster-green.com
pammik.ru
park.above.com
parlament.biz parlament.biz
partnersafe.men partner-pop.men
paydayonlinecom.com pastaleads.com
peekyou.com pc-virus-d0l92j2.pw
pelfind.me pendelprognos.se
pflexads.com perederni.net
pharmacyincity.com petsblogroll.com
photosaga.info pila.pl
phrcialiled.com piter.xrus.org
physfunc.ru pivka.xyz
picmoonco.pw platesauto.com
picturesboss.com polytopesexempt.com
pinwallpaper.top popup-hgd.xyz
piski.top porn555.com
pk-pomosch.ru porno-chaman.info
play.leadzupc.com porno-video-chati.ru
pochemychka.net porno.simple-image.com.ua
pontiacsolstice.info pornolook.net
popmarker.com pornosexrolik.com
porno-dojki.net
pornogad.com
pornoklad.net
pornotubs.com
pornoxxx.com.mx
porto.abuilder.net
poshiv-chehol.ru poshiv-chehol.ru
posible.net pospr.waw.pl
praisong.net predominant-invent.tk
preparevideosafesystem4unow.site prod2016.com
pricheskaonline.ru prostitutki-belgoroda.org
privacylocationforloc.com prostitutki-tyumeni.org
producm.ru pts163.ru
proekt-mos.ru pukaporn.com
pronekut.com puzzleweb.ru
prosperent.com py100.ru
proxyradar.com qpypcx.com
prpops.com quick-offer.com
putevka24.ru qwarckoine.com
pyramidlitho.webs.com rainbowice.ru
realtytimes.com rank-checker.online
referencemoi.com rankingchart.de
registry-cleaner.net razvratnoe.org
reklama1.ru realmonte.net
remedyotc.com realnye-otzyvy.info
remont-fridge-tv.ru remont-komputerov-notebook.ru
resultshub-a.akamaihd.net remorcicomerciale.ro
risparmiocasa.bz.it remybutler.fr
ros-ctm.ru replicaclub.ru
rusoft-zone.ru resellerclub.com
rvzr-a.akamaihd.net rfd-split.hr
s-forum.biz rfserial.net
sabizonline.com ribieiendom.no
saitevpatorii.com rostov.xrus.org
savememoney.co.za rp9.ru
scanner-irvin.top russia-tao.ru
scanner-walt.top rutor.group
scanner-walter.top saltspray.ru
searchaddis.com santechnik.jimdo.com
searchkut.com sawin.beth.webd.pl
scanner-andrew.top
scanner-mark.top
screpy.com
seansonline24.pl
search.xtconnect.com
searchinterneat-a.akamaihd.net
seasaltwithfood.com
seccioncontrabajo.com seccioncontrabajo.com
securesmrt-dt.com seminarykansas904.ml
selectads.men seo-smm.kz
semrush.com seonetwizard.com
seoanalyses.com serdcenebolit.com
seomarketings.online serpstat.com
servethis.com sevenstars7.com
sexpornotales.net sexobzor.info
sexxdate.net sharpchallenge.com
sexy-pings.com shillyourcoins.com
sexyteens.hol.es shitmovs.com
shareyards.com shtaketniki.ru
shinikiev.com.ua sierraapps.com
shohanb.com signal03.ru
shoppingmiracles.co.uk sispe.com.br
silvercash.com slim.sellany.ru
similardeals.net slimcdn.com
skutecznetabletkinaporostwlosow.pl slkrm.ru
sky-mine.ru smartpet.ru
slavkokacunko.de sms2x2.ru
smstraf.ru sobecjvuwa.com.ru
snip.tw social-s-hhh.xyz
snjatie-geroinovoy-lomki.ru sohoindia.net
snyatie-lomki-v-stacionare.ru solicita.info
softomix.com sotkal.lark.ru
souvenir.cc soundfrost.org
stanthonyscatholicchurch.org spammen.de
steelmaster.lv speeddream.xyz
spy-app.info
spywareit.com
stairliftstrue.com
stefanbakosab.se stefanbakosab.se
stop-zavisimost.com strangeduckfilms.com
streamin.to styro.ru
stromerrealty.com superlist.biz
sunflowerdrawingpaintings.blogspot.com surcentro.com
supernew.org survival.betteroffers.review
sureone.pro szamponrevita.pl
suture.co tattooreligion.ru
svetodiodoff.ru teastory.co
swagbucks.com telegraf.by
swiped.su teslathemes.com
swsociety.se theallgirlarcade.com
t-bygg.com theendivechronicles.com
tandvardshuset.net thefds.net
td-33.ru thenetinfo.com
tech4master.com
techart24.com
tecnoteakviareggio.it
testbotprocessor44.com
tgtclick.com
thedownloadfromwarez.blogspot.com
thefotosgratis.eu
thexart.club thexart.club
thiegs.reco.ws tizanidinedruginteractions.blogspot.com
threecolumnblogger.com tizanidinehydrochloride4mgstreetvalue.blogspot.com
thruport.com tizanidineinfo.blogspot.com
tiandeural.ru tizanidineorflexeril.blogspot.com
titslove.yopoint.in tizanidinewithdrawalsymptoms.blogspot.com
tizanidine4mg.blogspot.com tonivedu.it
tk-assortiment.ru toolsky.com
toondinsey.com toon-families.com
tootoo.to toonfamilies.net
top10-way.com tooplay.com
topads.men torontoplumbinggroup.com
topanasex.com touchmods.fr
topmira.com tracksz.co
torrent-newgames.com treasuretrack-a.akamaihd.net
torrentdownloadhub.com
tracfone.com
truck-addzilla.life
trustedhealthtips.com
tv-spoty.info
tvnewsclips.info
tvoystartup.ru
twitlinks.com
tytoona.com
u555u.info u555u.info
ugogo.info uggsale.online
ukkala.xyz uhdtv.website
ukkelberg.no ultimateclassicrock.com
um-razum.ru unlimitdocs.net
uogonline.com unpredictable.ga
url2image.com urengoy.pro
uroffer.link us-america.ru
vchulkax.com valuado.com
vancleefreplica.pw
verymes.xyz
via-energy-acquistare.com via-energy-acquistare.com
video-woman.com video-chat.in
videochat.bz video-chat.love
videos-for-your-business.com videonsk.com
videositename.com vipms.ru
viktoria-center.ru vipromoffers.com
vip-parfumeria.ru vkontarkte.com
viteonlusarezzo.it voditeltrezviy.ru
vizag.kharkov.ua wallpaperaccess.com
vkontaktemusic.ru
volume-pills.biz
vse-pesni.com
vtcdns.com
vucms.com
vzubah.com
vzube.com
wallpapers-best.com
watchinf.com
watracker.net watracker.net
webadvance.club webmasters.stackexchange.com
webs.com webpromotion.ae
website-analyzer.info website-audit.com.ua
website-speed-up.site webtherapy.ru
webstatsdomain.org weburlopener.com
wesharepics.com weclipart.com
westum.se wejdz-tu.pl
wholinkstome.com wfb.hatedriveapart.com
whosonmyserver.com white-truck.life
winterclassichockeyjerseys.com wineration.com
wladimirpayen.com withstandingheartwarming.com
wordkeyhelper.com wnoz.de
worldoffiles.ru womama.ru
wowcasinoonline.ooo wovis.site
writingservices17.blogspot.ru woweb.com.ua
www.888.com wrona.it
www.ehscloud.cn wrz0iuebwhp5fg.freeddns.com
www.kabbalah-red-bracelets.com wufak.com
xaijo.com ww2awards.info
xaylapdiendanang.com www.arenda-yeisk.ru
xcombear.ru x-lime.com
xfire.com x-musics.com
xmnb.net xchangetrak.com
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xerox-douglas.cf
xn----itbeirbjbi7bc6bh2d.xn--p1ai xjrul.com
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai xn----7sbho2agebbhlivy.xn--p1ai
xn--80abgj3a5acid6ghs.top xn--24-glceagatoq7c2a6ioc.xn--p1ai
xn--80aodinpgi.xn--p1ai xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--oogle-wmc.com xn--d1aifoe0a9a.top
xnxx-n.com xn--e1afanlbnfckd7c3d.xn--p1ai
xwatt.ru xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
yachts-cruise.info xtraffic.plus
yebocasino.com xxx-treker.ru
yoursearch.me xxxasianporn.net
yourserverisdown.com yottos.com
youtuhe.com youradexchange.com
yun56.co ypmuseum.ru
zaapplesales.blogspot.com yuarra.pluto.ro
zaidia.xhost.ro
zaimhelp.ru
zaimite.ru
zakon-ob-obrazovanii.ru
zaloadi.ru
zarenica.net
zawyna.ua zawyna.ua
zebramart.ru zigarettenonlinekaufen1.bloog.pl
zeg-distribution.com zixizop.net.ru
zeroredirect12.com zs2vm.top
zkjovpdgxivg.ga zynax.ua
zog.link
zytpirwai.net

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
03e.info 00webcams.com
108shot.com 101billion.com
24videos.tv 20pascals.nl
5tey7463.weebly.com 4inn.ru
aasoldes.fr 4webmasters.com
ahrntal.verymes.xyz acmebtn.ml
alert-fdm.xyz adsland.men
allknow.info ageofclones.com
anal-acrobats.hol.es angel1777.kz
arcteryxsale.online anime.dougasouko.com
arcteryxstore.online artparquet.ru
artpicso.com atlasvkusov.ru
asdfz.pro avtoarenda.by
atmovs.com balitouroffice.com
atovh.local-finders.com bambi.ck.ua
autochoixspinelli.com battle.net
avtochehli.by bestadbid.com
azartclub.org bibys.com
azartmsl.com bimatoprost-careprost.com.ua
battleforupdating.site bitcoinremote.com
best-seo-offer.com brewdom.ru
bestcalovebracelet.cn cablecar.us
bestdraws.com californianews.cf
bestmobilityscooterstoday.com capsderfudd.tk
beztuberkuleza.ru carapics.com
blog-hits.com chee-by.biz
boleznikogi.com cibpenonptib.flu.cc
boltalko.xyz clash-clans.ru
bookhome.info cleansearch.net
brillianty.info clksite.com
burger-tycoon.com com-onlinesupport.site
c1.onedmp.com compliance-brian.xyz
cakesplus.com.au compliance-fred.top
camdolls.com compliance-george.top
caulderwoodkennels.com comsysnet.com
cercacamion.it cv.wallhade.co
cheap-pills-norx.com cypernhuset.se
checktext.org cyprusbuyproperties.com
choosecuisine.com dalavia.ru
citizenclsdriveri7.pen.io dangphoto.trade
clickbanksites.info dating-time-now.com
compliance-jane.top dayibiao.com
compliance-nelson.top decenttools.com
constantaservice.net dekoration.us
covetnica.com dentalpearls.com.au
crackguru.tk dignitasdata.se
crynet.cc discountliv.com
customcollegeessays.net divisioncore.com
customergrowthsystems.com domain-tracker.com
db.speedup.gdn
decorazilla.com
detskie-konstruktory.ru
diesel-parts28.ru
dimaka.info
dimkino.ru
donna7753191.ru
donvito.unas.cz donvito.unas.cz
downloadmefiranaratb1972.xpg.com.br doxysexy.com
draniki.org druzhbany.ru
e-biznes.info
e-commerce-seo.com
eaplay.ru
ee77ee.com
eets.net
ekobata.ru
election.interferencer.ru
elektir.ru
elektrischezigarette1.blog.pl elektrischezigarette1.blog.pl
elektroniksigaraankara.info en.altezza.travel
elektrozigaretten1.tumblr.com enhand.se
erectile.bid eshop4u.jp
essay-writing.work eupornstar.info
evehemming.blogspot.com.au ezrvrentals.com
ezaz.info familienzahnaerzte.com
ezigarettekaufen1.hpage.com femmesdenudees.com
ezigarettekaufen2.mpbloggar.se fetishinside.com
findacheaplawyers.com fetlifeblog.com
floating-share-buttons.com filkhbr.com
freewebs.com finstroy.net
gambarxkata.co firstsiteguide.com
gamewrath.com fiverr.com
gateway.zscalertwo.net forex-indextop20.ru
gdzkurokam.ru ftns.ru
gembird.com fuckbuddybestgilf.info
gfaq.ru funponsel.com
glass-msk.ru gentamicineyedrops.blogspot.com
gepezz.info
getresponse.com
gnuetella.com gnuetella.com
googleseo.com.tr gosreg.amchs.ru
gopixdatabase.com gov.yanao.ru
greenzaim.ru gpirate.com
gribokstop.com grand-chlen.ru
gsasearchengineranker.site growmyfunds.ca
gsmlab.pl
guardlink.org guardlink.org
gunsvicceadadebt.tk guildebzh.info
gz2.bbsoldes.fr hao123.com
hanwei.us hdapp1008-a.akamaihd.net
hchha.com hosting-tracker.com
hmmm.cz hostingclub.lk
hollywoodactress.info hotxnights.info
hoverboard360.de hspline.com
hunterboots.online iflycapetown.co.za
hvd-store.com ilo134uloh.com
ido3.com ilovevitaly.com
ifirestarter.ru immigrational.info
igithab.com inbabes.sexushost.com
ilovevitaly.ru incanto.in.ua
imperia31.ru indiasourcemart.in
infostatsvc.com infokonkurs.ru
instasexyblog.com infowarcraft.ru
ipostroika.ru inspiring-desperate.tk
iqupdatetmz.win investsuccess.org
irkutsk.online-podarki.com iplusbit.blogspot.co.za
james13prix.info isitwp.com
johannesburgsingles.co.za itsdp3.com
joyceblog.top joyceblog.top
kemerovo.zrus.org junglenet-a.akamaihd.net
kerch.site kaidalibor.de
kinoplen.ru karachev-city.ru
kolotiloff.ru knowsitall.info
konoplisemena.com konica.kz
labelwater.se kosova.de
lacloop.info krasnodar.zrus.org
lankarns.com kutikomi.net
laxdrills.com ladov.ru
letsrepair.in letmacworkfaster.site
likrot.com letto.by
liran-locks.com lineavabit.it
liupis.com linkdebrideur.xyz
live-cam6.info locationdesaison.com
ljusihus.se louboutinbooties.xyz
lonely-mature.com lumb.co
louisvuittonoutletstore.net magazintiande.ru
luciddiagnostics.in mcar.in.ua
m4ever.net mearns-tractors.co.uk
mamasuper.prom.ua minneapoliscopiers.com
marblestyle.ru mirzonru.net
mcnamaratech.com misslike.ru
medi-fitt.hu missvietnam.org
medispainstitute.com.au moesonce.com
medizinreisen.de monetizer.com-01.site
medkritika.ru moneyteam24.com
megakino.net moneytop.ru
metaxalonevstizanidine.blogspot.com moneyzzz.ru
mex-annushka.ru mydearest.co
mlvc4zzw.space myiptest.com
monarhs.info myra.top
mosrif.ru nagdak.ru
myplaycity.com narkologiya-belgorod.ru
mystats.xyz newstudio.tv
nakozhe.com nikhilbahl.com
nalogovyy-kodeks.ru nippon-bearings.ru
narco24.me nonameread45.live
narkologiya-voronezh.ru o.light.d0t.ru
naturalbreakthroughsresearch.com oneshotdate.com
navalwiki.info ooomeru.ru
nerudlogistik.ru orel-reshka.net
nudejapan.net osoznanie-narkotikam.net
oakleyglassesonline.us otbelivanie-zubov.com
odywpjtw.bloger.index.hr partnerads.men
okeinfo.online pcads.ru
oksrv.com peekyou.com
orsonet.ru perederni.net
partnerline.men picturesboss.com
pb-dv.ru playfortuna-play.ru
pensplan4u.com pliks.pl
picturesify.com
piratecams.com
pizda.lol
poiskzakona.ru
popmarker.com popmarker.com
popup-hgd.xyz popugaychiki.com
pornoklad.ru pornotubs.com
portal-eu.ru prchecker.info
purchasepillsnorx.com priora-2.com
push-ad.com puserving.com
rankscanner.com puteshestvennik.com
referencemoi.com py100.ru
remmling.de radiogambling.com
replicaclub.ru redirectme.net
rospromtest.ru registrationdomainsite.com
runstocks.com rentehno.ru
repeatlogo.co.uk
replica-watch.ru
retailwith.com
rozalli.com
rusprostitute.com rusprostitute.com
safe-app.net sadaholding.com
sakhboard.ru sarafangel.ru
savefrom.com sbdl.no
sayyoethe.blogspot.co.za scanner-julianna.top
sbricur.com scrapy.org
sbt-aqua.ru search-error.com
scanner-mary.top seminarykansas904.ml
seeresultshub-a.akamaihd.net seorussian.ru
seo-traffic-ranking.info sex-dating.co
seorankinglinks.xyz sexreliz.com
seotoolsagency.com sfd-chess.ru
sex-foto.pw sharebutton.net
sexsearch.com shemalegalls.blogporn.in
sexy.babes.frontend-stack.top si-unique.com
silvermature.net sibtest.ru
sitopreferito.it smadihome.com
soblaznu.net small-game.com
socas.pluto.ro snowplanes.com
solnplast.ru socialbutton.xyz
sonata-arctica.wz.cz socialmediasuggest.com
special-porn.com sohoindia.net
ssconstruction.co soundfrost.org
stairliftsarea.com speeddream.xyz
stathat.com spl63.fr
streha-metalko.si stroilka.info
stuff-about-money.com studentguide.ru
t-bygg.com superinterstitial.com
tecnoteakviareggio.it szucs.ru
telsis.com tacbibirfa.tk
thebluffs.com tds-advert005.info
timdreby.com telegramdownload10.com
tooplay.com thebestweightlosspills.ovh
top10-online-games.com titelhelden.eu
touchmods.fr tizanidine4mgtablets.blogspot.com
toy-shop.top tizanidinehcl4mgisitanarcotic.blogspot.com
trichizobswiv.agddns.net toon-families.com
trucri.me top250movies.ru
unece.org topflownews.com
unpredictable.ga topshef.ru
usiad.net traffmonster.info
varbergsvind.se trailer.cinemaflix.website
veles.shop trustaffs.com
viandpet.com twojebook.pl
videochat.tv.br uchebavchehii.ru
vintontech.info urzedowski.eu
vitalads.net vladimir.zrus.org
vltai.com vvon.co.uk
vse-pesni.com vvpg.ru
wahicbefa31.soup.io vzube.com
web.cvut.cz wanker.us
website-audit.com.ua watch-movies.ru
website-speed-checker.site waycash.net
wetgames.ru web-betting.ru
wholesalecheapjerseysfree.com webads.co.nz
wieseversa.no webscouter.net
wildworld.site weddingdresses.xyz
wingsofrefuge.net weprik.ru
worldmusicfests.com womensplay.net
wovis.site www.timer4web.com
woweb.com.ua x-mix.info
ws.ampower.me xn----7sbaphztdjeboffeiof6c.xn--p1ai
wstroika.ru xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
wufak.com xn--b1adccaf1bzj.xn--p1ai
x-stars.ru xn--hxazdsfy.blogspot.com
xev.ru xn--oogle-wmc.com
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai xxxtube69.com
xn--80ajbshivpvn2i.xn--p1ai yellowads.men
xtube.com yellowstonesafaritours.com
xvideosbay.com yopoint.in
xxxhdvideo.site zagadki.in.ua
yougotanewdomain.com zoogdinsney.com
zkjovpdgxivg.ga zoompegs.com
znakomstva-moskva77.ru zscalerone.net
zog.link
zzlgxh.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.132 0; 10.1.0.237 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2799 ### Version: V4.2021.12.2800
### Updated: Wed Dec 22 10:03:25 UTC 2021 ### Updated: Wed Dec 22 22:01:18 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2799 ### Version: V4.2021.12.2800
### Updated: Wed Dec 22 10:03:25 UTC 2021 ### Updated: Wed Dec 22 22:01:18 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##