vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2697 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-06 10:09:21 +00:00
parent 300c408991
commit 25fb172ddd
48 changed files with 67698 additions and 67695 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2696 #### Version: V4.2021.11.2697
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2696 #### Version: V4.2021.11.2697
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2696 #### Version: V4.2021.11.2697
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8324
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2696 #### Version: V4.2021.11.2697
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8328
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8328
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8328
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8328
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.210 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.210 0;
127.0.0.1 0; 127.0.0.1 0;

8328
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.210 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.210 0;
127.0.0.1 0; 127.0.0.1 0;

8328
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.210 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.210 0;
127.0.0.1 0; 127.0.0.1 0;

8328
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2696 2697

8328
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8328
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.137.83.24 0; 20.121.211.120 0;

8328
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.210 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.210 0;
127.0.0.1 0; 127.0.0.1 0;

8328
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.210 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.210 0;
127.0.0.1 0; 127.0.0.1 0;

8328
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.210 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.210 0;
127.0.0.1 0; 127.0.0.1 0;

8328
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.137.83.24 0; 20.121.211.120 0;

8328
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.137.83.24 0; 20.121.211.120 0;

8328
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.210 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BotALot Buck
Copier Discobot
Mag-Net EasyDL
Mail.RU_Bot Jbrofuzz
Majestic12 OutfoxBot
Semrush SpankBot
SiteSucker WebCopier
Swiftbot ZoominfoBot
Wonderbot archive.org_bot
demandbase-bot spyfu

306
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker
404enemy 404enemy
80legs
ADmantX ADmantX
AIBOT ALittle Client
Abonti ASPSeek
Aboundexbot AfD-Verbotsverfahren
Aipbot Alexibot
Alligator
Anarchy Anarchy
Anthill Anthill
Apexoo Apexoo
Aspiegel
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike BLEXBot
BDCbot
BDFetch
BackWeb BackWeb
BacklinkCrawler Badass
Bandit Bandit
BatchFTP BetaBot
Bitacle Bitacle
BlowFish BlackWidow
Brandprotect Blackboard
Boardreader
Brandwatch Brandwatch
BuiltBotTough Bullseye
BuiltWith
BuzzSumo BuzzSumo
CATExplorador CCBot
CODE87 CSHttp
Calculon CazoodleBot
CensysInspect Cegbfeieh
Chlooe CheeseBot
ChinaClaw
Claritybot Claritybot
Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Copier Cogentbot
Copyscape
Craftbot Craftbot
Crawling at Home Project CrazyWebCrawler
Crescent Curious
CrunchBot CyotekWebCopy
Custo DSearch
DTS Agent
DatabaseDriverMysqli
Demon
Deusu
Devil Devil
DigitalPebble Digincore
Discobot Discobot
Discoverybot Discoverybot
Dispatch DnBCrawler-Analytics
DittoSpyder
DnyzBot
DomainAppender DomainAppender
DomainStatsBot
Domains Project
Dotbot Dotbot
Download Wonder
Dragonfly Dragonfly
Drip
ECCP/1.0 ECCP/1.0
EMail Siphon EMail Wolf
EasyDL
Ebingbong Ebingbong
EirGrabber
Evil
ExtractorPro ExtractorPro
Extreme Picture Finder EyeNetIE
FDM Ezooms
FHscan FHscan
Firefox/7.0 Fimap
Flunky
Foobot Foobot
G-i-g-a-b-o-t Fyrebot
Genieo
GermCrawler GermCrawler
GetRight
GetWeb
Go!Zilla
Go-Ahead-Got-It
GoZilla GoZilla
GrabNet GrabNet
Grabber HEADMasterSEO
Grafula Harvest
GrapeFX Havij
GridBot Humanlinks
HTMLparser IDBTE4M
HTTP::Lite
HTTrack
Haansoft
Hloader
IDBot
IRLbot IRLbot
Iblog Iblog
IlseBot IlseBot
Indy Library Image Sucker
InfoNaviRobot Internet Ninja
InfoTekies
Intelliseek
Iskanie Iskanie
IstellaBot JOC Web Spider
Jbrofuzz JamesBOT
JetCar JetCar
JikeSpider
Jorgee Jorgee
Kozmosbot JustView
Jyxobot
LWP::Simple LWP::Simple
Larbin Larbin
LeechGet Leap
LeechFTP
Libwhisker
LieBaoFast LieBaoFast
Likse Likse
Linkdexbot Linkdexbot
LinksManager LinqiaScrapeBot
LinqiaMetadataDownloaderBot
Litemage_walker
Lmspider Lmspider
Ltx71 Ltx71
MFC_Tear_Sample
MJ12bot MJ12bot
MSFrontPage MSFrontPage
Mag-Net MTRobot
Majestic SEO Majestic-SEO
Masscan Majestic12
Mata Hari MauiBot
Mb2345Browser MeanPath Bot
Metauri Mediatoolkitbot
MegaIndex.ru
MicroMessenger
Microsoft Data Access Microsoft Data Access
Moblie Safari Moblie Safari
Mojeek
Mojolicious Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila
Mr.4x3
Msrabot
Musobot Musobot
NICErsPRO NPbot
Nameprotect Navroad
Needle
Net Vampire
NetAnts NetAnts
NetLyzer NetMechanic
NetSpider
Netcraft Netcraft
NextGenSearchBot Nibbler
Niki-bot Nikto
Nimbostratus Nimbostratus
Nmap
Nutch
Offline Explorer Offline Explorer
OpenLinkProfiler OnCrawl
OpenVAS
Openfind Openfind
OrangeSpider OutclicksBot
PHPCrawl OutfoxBot
PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
Papa Foto Papa Foto
PeoplePal Pavuk
Petalbot
Pi-Monster
Picscout
Picsearch Picsearch
PictureFinder
Pimonster Pimonster
Pockey PleaseCrawl
ProPowerBot ProWebWalker
Probethenet Probethenet
Psbot Quick-Crawler
Pump RSSingBot
PxBroker RankActiveLinkBot
RankActive RankFlex
RankingBot2 RankingBot
Reaper ReGet
Rogerbot RebelMouse
Recorder
Ripper
SBIder SBIder
SEOlyticsCrawler
SEOstats SEOstats
SMTBot
SalesIntelligent SalesIntelligent
ScanAlert
Scanbot Scanbot
Scrapy
Screaming
ScrepyBot
SearchmetricsBot
Semrush Semrush
SemrushBot SemrushBot
SeobilityBot
Seomoz Seomoz
Shodan SiteLockSpider
Siphon
SiteCheckerBotCrawler
SiteExplorer
SiteSnagger
SiteSucker
Sitebeam Sitebeam
SmartDownload SlySearch
Snake Snapbot
Snoopy Snoopy
Sociscraper
Sosospider
SpaceBison SpaceBison
Spanner
Spbot
Spinn3r Spinn3r
Sqlworm Sqlworm
Sqworm Stripper
Steeler Sucker
Sucuri Surfbot
SurveyBot
T0PHackTeam
T8Abot T8Abot
Telesoft Telesphoreo
Toata TightTwatBot
Trendiction Titan
Trendictionbot
True_Robot True_Robot
TurnitinBot
TwengaBot TwengaBot
UnisterBot UnisterBot
Upflow Upflow
V-BOT
VB Project VB Project
VeriCiteCrawler Vagabondo
Virusdie
VoidEYE VoidEYE
Voltron WASALive-Bot
WBSearchBot
WEBDAV
WWW-Collector-E WWW-Collector-E
Wallpapers WWWOFFLE
WallpapersHD Web Auto
WeSEE
Web Enhancer
Web Fetch Web Fetch
Web Fuck Web Sauger
Web Sucker Web Sucker
WebAuto WebAuto
WebBandit WebBandit
WebCopier WebFetch
WebFuck WebFuck
WebReaper
WebSauger
WebStripper WebStripper
WebsiteQuester WebZIP
Whack WebmasterWorldForumBot
Widow Whatweb
Who.is Bot
WinHTTrack
Wonderbot
Woobot Woobot
Wotbox Wotbox
Wprecon Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Zauba Xenu
Zeus Zermelo
ZoomBot
ZyBorg
arquivo-web-crawler arquivo-web-crawler
check1.exe cah.io.community
coccocbot-web coccocbot-web
cognitiveseo crawler4j
com.plumanalytics
dataforseo.com
demandbase-bot
domainsproject.org domainsproject.org
evc-batch evc-batch
instabid facebookscraper
ips-agent internetVista monitor
lwp-trivial lwp-request
mediawords meanpathbot
s1z.ru muhstik-scan
netEstate NE Crawler
oBot
page scorer
pcBrowser
plumanalytics
scalaj-http
scan.lol scan.lol
seobility
seoscanners seoscanners
seostar
serpstatbot
sexsearcher
sitechecker.pro
sogouspider sogouspider
sp_auditbot
spyfu spyfu
sysscan sysscan
ubermetrics-technologies.com tAkeOut
webpros.com trendiction.com
webprosbot voyagerx.com
x09Mozilla zauba.io
zgrab

286
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
ALittle Client 404enemy
Abonti 80legs
AfD-Verbotsverfahren Aboundexbot
Alexibot Alexibot
AllSubmitter
Anarchie Anarchie
Anarchy99
Ankit
Anthill Anthill
AwarioRssBot BDCbot
AwarioSmartBot
BBBike
BDFetch
BLEXBot
BackDoorBot BackDoorBot
Bandit BackStreet
Bigfoot BackWeb
Bolt BacklinkCrawler
Barkrowler
Battleztar Bazinga
BetaBot
Bitacle
Blackboard
Blow
BotALot BotALot
Brandprotect Brandprotect
BuiltWith BuiltWith
Bullseye Bullseye
CCBot CCBot
Calculon Calculon
CheeseBot CazoodleBot
Cegbfeieh
ChinaClaw
Cloud mapping
Copier Copier
CopyRightCheck
Cosmos Cosmos
Craftbot Craftbot
Crawling at Home Project
CrazyWebCrawler
Crescent Crescent
CrunchBot CrunchBot
Curious
Custo Custo
DTS Agent DTS Agent
DataCha0s Demon
DatabaseDriverMysqli Deusu
Devil Discobot
Digincore Dispatch
DigitalPebble
Disco
Discoverybot
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnyzBot
DomainCrawler
DomainStatsBot
Domains Project
Dotbot Dotbot
Download Wonder Download Wonder
Dragonfly Drip
ECCP/1.0 EMail Siphon
EMail Wolf EMail Wolf
EasyDL
Ebingbong Ebingbong
Ecxi EirGrabber
EroCrawler Exabot
Evil Extractor
ExtLinksBot
EyeNetIE EyeNetIE
Flunky Fimap
G-i-g-a-b-o-t Firefox/7.0
Foobot
FrontPage
Fuzz
GT::WWW GT::WWW
GalaxyBot GalaxyBot
GetRight
GetWeb GetWeb
Getintent
Gigabot Gigabot
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GoZilla
GrabNet
Grabber Grabber
Grafula Grafula
GrapeshotCrawler GridBot
HMView
HaosouSpider HaosouSpider
Iblog Heritrix
Id-search Humanlinks
HybridBot
IRLbot
IlseBot
Indy Library Indy Library
InfoNaviRobot
Intelliseek
Internet Ninja
InternetSeer
Iria Iria
JamesBOT Iskanie
IstellaBot
Jbrofuzz
JennyBot JennyBot
Jetty JetCar
JikeSpider JustView
Kenjin Spider Kenjin Spider
Keyword Density
Kozmosbot Kozmosbot
Larbin LNSpiderguy
LeechGet LWP::Simple
Lanshanbot
LeechFTP
LexiBot LexiBot
Lftp LibWeb
LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkScan
Linkbot Linkbot
LinkextractorPro LinkextractorPro
LinkpadBot
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider Lipperhey Spider
MFC_Tear_Sample Lmspider
MIIxpc Ltx71
MJ12bot MSFrontPage
MQQBrowser MTRobot
Majestic-SEO Mag-Net
Majestic SEO
Majestic12 Majestic12
Masscan MarkWatch
MauiBot Mass Downloader
Mata Hari
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot Metauri
Microsoft URL Control Microsoft URL Control
Morfeus Fucking Scanner
Msrabot Msrabot
NICErsPRO
Name Intelligence
Navroad Navroad
NearSite NetAnts
Nessus
Net Vampire
NetLyzer NetLyzer
NetZIP NetMechanic
Netcraft
Netvibes Netvibes
Nibbler Nibbler
Nimbostratus NimbleCrawler
Ninja Nuclei
Nutch Nutch
Offline Navigator Octopus
OpenLinkProfiler Offline Explorer
OpenVAS OpenVAS
Openfind
Openvas
OrangeSpider OrangeSpider
OutclicksBot
Page Analyzer Page Analyzer
PageAnalyzer PageGrabber
PageScorer PageScorer
PageThing.com
Pandalytics
Papa Foto
Pavuk Pavuk
PeoplePal
Picsearch Picsearch
PictureFinder PleaseCrawl
Pimonster ProWebWalker
Pixray
Probethenet Probethenet
Psbot Psbot
Pump Pump
PxBroker PxBroker
QueryN Metasearch PyCurl
RSSingBot
RankActive RankActive
RankFlex RankFlex
RankingBot
RankingBot2
Rankivabot Rankivabot
RealDownload RealDownload
RepoMonkey Reaper
RocketCrawler RebelMouse
RedesScrapy
SEOkicks SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOstats
SISTRIX SISTRIX
SMTBot
ScanAlert ScanAlert
Scanbot
ScoutJet ScoutJet
Screaming Searchestate
SearchmetricsBot SearchmetricsBot
SemrushBot Semrush
SeoSiteCheckup SeoSiteCheckup
Seomoz SeobilityBot
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer SiteLockSpider
Snake SiteSucker
Sociscraper Sitebeam
Snapbot
SocialRankIOBot
Sogou web spider Sogou web spider
Sosospider Sosospider
Sottopop
SpaceBison SpaceBison
Spinn3r SputnikBot
Sqlworm Sqlworm
Sucuri
SuperBot SuperBot
Surfbot Surfbot
Teleport Suzuran
Telesoft Swiftbot
Thumbor T0PHackTeam
Toata Telesphoreo
Trendiction The Intraformant
Trendictionbot TheNomad
Turingos Titan
Turnitin Tracemyfile
TurnitinBot
URLy Warning URLy Warning
URLy.Warning URLy.Warning
V-BOT
VB Project VB Project
VCI VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
Virusdie
VoidEYE VoidEYE
Voil
WASALive-Bot
WEBDAV WEBDAV
WISENutbot
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD Web Auto
WeSEE Web Collage
Web Enhancer Web Enhancer
Web Fuck Web Sauger
Web Sucker WebBandit
WebCopier
WebFetch WebFetch
WebFuck
WebGo IS
WebImageCollector WebImageCollector
WebStripper WebZIP
WebWhacker
WebmasterWorldForumBot WebmasterWorldForumBot
Webster Website Quester
Whack WebsiteExtractor
Whacker Whacker
Whatweb Whatweb
WinHTTrack Who.is Bot
Xaldon WebSpider Widow
WiseGuys Robot
Wonderbot
Woobot
Xaldon_WebSpider Xaldon_WebSpider
Zade YoudaoBot
ZmEu Zauba
ZumBot Zermelo
Zeus
Zitebot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt check1.exe
clark-crawler
coccocbot-web
crawler.feedback crawler.feedback
crawler4j crawler4j
eCatch domainsproject.org
evc-batch
facebookscraper
gopher gopher
lwp-request heritrix
meanpathbot ips-agent
mediawords mediawords
muhstik-scan pcBrowser
netEstate NE Crawler
plumanalytics plumanalytics
polaris version scalaj-http
ripz
s1z.ru
satoristudio.net
scan.lol
seobility
serpstatbot
sexsearcher sexsearcher
sitechecker.pro
siteripz
sogouspider sogouspider
sp_auditbot
sysscan sysscan
trendiction.com voyagerx.com
trendiction.de
webmeup-crawler
webprosbot webprosbot
x09Mozilla
xpymep1.exe
zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
allcredits.su bezlimitko.xyz
cheap-trusted-backlinks.com buypillsorderonline.com
clevernt.com galaxy-family.ru
floating-share-buttons.com paphoselectricianandplumber.com
love-baby.cz predominant-invent.tk
medizinreisen.de sbornik-zakonov.ru
mylovelibrabry.com simply.net
site5.com sptslmtrafms.com
trahic.ru thecoral.com.br
wnhjavlhezp.gq vrnelectro.ru

924
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0n-line.tv 1001desert.com
1008.su 2ads.co.uk
12u.info 300richmond.co.nz
178evakuator178.ru 3weekdiet.com
2020iscoming.info 4kplayer.pl
3dracergames.com 6hopping.com
4senses.co aa81bf391151884adfa3dd677e41f94be1.com
7search.com aa8780bb28a1de4eb5bff33c28a218a930.com
999webdesign.com aac52006ec82a24e08b665f4db2b5013f7.com
a14download.com abbanreddy.com
addray.pro abcguru.xyz
adloads.com acc.eu.org
adserver-e7.com addbin.men
agapovdg.ru admanaerofoil.com
agreda.pluto.ro adult-shop.com.ua
airlandsea.info adultnet.in
akusherok.ru agahinameh.com
alldubai.biz agysya.ru
alloysteel.ru aihelen.net
allsilver925.co.il aimayangzhi.com
amazon-seo-service.com akama.com
anatomiy.com alarmobninsk.ru
animal36.com alert-hgd.xyz
alienwheel.es
all-streaming-media.com
allproblog.com
alta-realestate.com
anal-acrobats.hol.es
android-systems.ru
android-vsem.org
angel1777.kz
anjalika.co.in
apartmentbay.ru apartmentbay.ru
apiallgeniusinfo-a.akamaihd.net apimegabrowsebiz-a.akamaihd.net
apidiamondatanet-a.akamaihd.net appearance-cool.com
apilinkswiftco-a.akamaihd.net appleid-verification.com
apioutoboxnet-a.akamaihd.net architecturebest.com
app-ready.xyz arendakvartir.kz
apparel-offer.com arewater.com
appsecurityr.com artavenuegardenstudios.com
arendovalka.xyz ausmepa.org.au
articlesdirectoryme.info aviav.org
asscenihotosidea.blogspot.com aviav.ru
atdedinotuho.tk
audiofree.ru
aviav.eu
avirasecureserver.com
avitocars.ru avitocars.ru
ayodhya.co avkzarabotok.info
azartmix.com avon-novosib.ru
azlex.uz azbuka-mo.ru
backgroundpictures.net b00kmarks.com
backiee.com babespcs.com
baikaleminer.com bad-stars.net
balkanfarma.org baersaratov.ru
balois.worldbreak.com balkanfarma.ru
bannerads.de bavsac.com
bannerbank.ru bazaronline24.ru
basedecor.ru beclean-nn.ru
bayareaaeroclub.org bestmarriages.com
belsetirehafi.tk binomo.kz
berlininsl.com bioinnovate.co
bestbrainprod.win
bestwebsiteawards.com
beta.hotkeys.com
betonka.pro
bidr.trellian.com
bigcareer.info
bigpictures.club
bio-optomarket.ru
biocn.dx.am
bitcoins-live.ru bitcoins-live.ru
biznesluxe.ru bluerobot.info
bloggen.be bocoarchives.org
bluesman.nu bolezniorganov.ru
bmsco.co bongiornos.info
brainboostingsupplements.org book-bets.com
brakehawk.com bosefux.esy.es
brandbucket.com bouda.kvalitne.cz
break-the-chains.com breastaugmentation.co.za
brewdom.ru brewdom.ru
browsepulse-a.akamaihd.net brianhenry.co
buigas.00it.com budpost.com.ua
burger-imperia.com builtwith.com
buyhoverboard.com businesn.men
buynorxx.com buy-forum.ru
buypillsorderonline.com candlehandmade.com
candelluxsklep.pl cannibalcheerleader.com
candypeople.se canvas.gloverid.site
cartierloveringreplica.xyz cartierlove.xyz
cartierreplica.xyz cartierreplica.top
catalogs-parts.com cartiertrinity.xyz
ccbill.com cascadelink.org
celejihad.info castingbank.ru
cementaresearch.se cbcseward.com
centrdebut.ru chatroulette.world
centre-indigo.org.ua cheapsergiorossi.online
centrumcoachingu.com checkpagerank.net
cheapestjerseys-wholesale.com chiblackhawks-jerseys.com
cheapestjerseysonwholesale.com
chlooe.com chlooe.com
chocoslim.pro christianlouboutinshoes.xyz
cialisndbrx.com circlesl.com
cicaki.net clash-clans.ru
clarithromycin500mg.com cleansearch.net
com-supportcenter.website clicksor.com
compiko.info clmforexeu.com
compliance-alex.xyz club-samodelkin.ru
compliance-andrew.xyz coinsspb.com
compliance-jane.top community.allhiphop.com
compliance-walter.top compliance-alexa.top
converse.ddsoldes.fr contentsexpress.com
coslab.club copypaste.traderzplanet.in
covadhosting.biz covadhosting.biz
cowblog.fr cpajunkies.com
crd.clan.su crackzplanet.com
creativehutindia.com cryptoswap.biz
creditnation.ru customsua.com.ua
crest-poloski.ru cymbaltaandtizanidine.blogspot.com
d2jsp.org dailyfinancefix.com
daretodonate.co dayibiao.com
de.zapmeta.com deanmoore.ie
delfin-aqua.com.ua dekoration.us
dimaka.info demian.kz
dimfour.com den-noch24.ru
dimkino.ru dikx.gdn
discountbarbour.online disruptingdinnerparties.com
diusyawiga.tk divku.ru
dkvorota.ru doctormakarova.ru
dnepropetrovsk.mistr-x.org domcran.net
dogclothing.store domik-derevne.ru
dojki365.online
domashneeruporno.com
dominateforex.ml dominateforex.ml
dotmass.top domznaniy.ru
dptaughtme.com dorratex.tn
druzhininevgeniy63.blogspot.com doska-vsem.ru
e-avon.ru downtuptv.gq
easync.io dvrlists.com
e-c.al
e-collantes.com
e-commerce-seo1.com
ednorxmed.com
ee77ee.com
ekaterinburg.xrus.org ekaterinburg.xrus.org
elektrozigarette1.dreamwidth.org el-nation.com
emailaccountlogin.co eldiariodeguadalajara.com
embedle.com elektrozigaretten2.yn.lt
encodable.com elidelcream.weebly.com
energy-ua.com eropho.com
espaceinventoristes.com erotiktreff24.info
este-line.com.ua event-tracking.com
europeanwatches.ru everflora.ru
evening-dating-club.info
exchanges-bet.com exchanges-bet.com
exoclick.com exhibitionplus.eu
exportshop.us eyelike.com.ua
eyessurgery.ru
eywords-monitoring-your-success.com
ezigarettenshop2.myblog.de ezigarettenshop2.myblog.de
factorynightclub.com ezigaretteshop2.mywapblog.com
famix.xyz falcoware.com
fandlr.com fapgon.com
fastcrawl.com faracontrol.ir
fearcrow.com fashion-stickers.ru
faz99.com
fickkontaktehobbyhuren.com fickkontaktehobbyhuren.com
film-one.ru fidelityfunding.com
finansov.info fikasound.tk
fitfloponline.store finstroy.net
fiverr.com floridahuntingfishingadventures.com
flagcounter.me flowersbazar.com
floating-share-buttons.com footballfarrago.com
fordsonmajbor.cf forensicpsychiatry.ru
forumprofi.de frameimage.org
foto-telok.net free-laptop-reward.com
foxweber.com free-share-buttons.com
free-floating-buttons.com free411games.com
ftns.ru freshberry.com.ua
funnypica.com freshwallpapers.info
fxgallery.com front.ru
gallerily.com fsakhalin.ru
getoutofdebtfree.org fuel-gas.com
getridofstretchmarks.org fugarif.ga
gifspics.com fun2cell.net
gigapeta.com gallery.rennlist.com
girls-ufa.ru gambarxkata.co
given2.com gamewrath.com
glavprofit.ru gaypornmovie.net
glogow.pl gayxperience.com
gemgrab-a.akamaihd.net
generic-pills-online.com
gfaq.ru
ghernnqr.skyrock.com
gilsonchiro.xyz
girlgamerdaily.com
girlsatgames.ru
gktt.ru
go2mike.ru go2mike.ru
gonextmedia.com goatse.ru
googglet.com goblacked.com
gobongo.info
golden-catalog.pro
goodhousekeeping.com
goodwriterssales.com
googleseo.com.tr googleseo.com.tr
gratuitbaise.com greatgrace.ru
gratuitxblcodes.com growmyfunds.ca
greatdealshop.com guardlink.org
greatfind-a.akamaihd.net guruofcasino.com
growshop.es handsandlegs.ru
grtyi.com hanink.biz.ly
gyffu.com hccoder.info
hd-filmy.net
hd720kino.ru hd720kino.ru
healgastro.com healgastro.com
healmytrauma.info healmytrauma.info
health-medical-portal.info hermesbracelets.xyz
heartofbeijing.blogspot.com hitmuzik.ru
hentai-manga.porn hobild.net
hentaiheroes.com honyaku.yahoofs.jp
histats.com hotchatdate.com
hitcpm.com hothor.se
hostsshop.ru hotsocialz.com
hotdl.in houseofrose.com
houston-vikings.com hspline.com
hscsscotland.com hulfingtonpost.com
ideawheel.com humanorightswatch.org
ikritikimou.gr hundejo.com
impresagaia.it hunterboots.online
insider.pro i-hobot.ru
intermesh.net ilmexico.com
ipchicken.com imabase.com
iphantom.com images.gyffu.com
iweblist.info imfamous.info
javspace.net imgarcade.com
jimmychoosale.online imitex-plus.ru
jofucipiku.tk imperiafilm.ru
junketjuice.blogspot.com inboxdollars.com
kam-dom.ru infazavr.ru
kemerovo.zrus.org invest-pamm.ru
kevblog.top investyb.com
kidd.reunionwatch.com invitefashion.com
kidskunst.info iptool.xyz
kinozapas.com istizanidineanopiate.blogspot.com
klumba55.ru javatex.co.id
knigonosha.net javidol.site
kostenlos-sexvideos.com javitas.info
kotaku.com job.icivil.ir
krasivye-devushki.net kabbalah-red-bracelets.com
krasnodar.zrus.org kambasoft.com
kskjco.club kashubadesign.ru
kumuk.info katadhin.co
labvis.host.sk kenaba.su
lampokrat.ws keywordsuggest.org
lapolis.it kinky-fetishes.com
laptop-4-less.com kino-filmi.com
larchik.net kino-key.info
komputers-best.ru
konpax.com
kovrenok.ru
kurwa.win
kvartir-remont.biz
kwzf.net
landreferat.ru
languagecode.com
laparfumotec.com
larose.jb2c.me
larutti.ru larutti.ru
lechenie-gemorroya.com ldrtrack.com
lenvred.org lecbter-relationships.ga
lemon-ade.site
letslowbefast.today letslowbefast.today
lexixxx.com lineavabit.it
lida-ru.com listiseltemournlan.gq
liran-locks.com littlesexdolls.com
littlesunraiser.com
liver-chintai.org liver-chintai.org
livetsomudvekslingstudent.bloggersdelight.dk llastbuy.ru
localflirtbuddies.com
loginduepunti.it loginduepunti.it
machicon-akihabara.info lost-alpha.ru
ltvperf.com
macrotek.ru
mahnwachen-helfen.info
mail.allnews24.in
maju.bluesalt.co
makemoneyonline.com makemoneyonline.com
manimpotence.com makis.nu
marblestyle.ru malwareremovalcenter.com
maridan.com.ua mcar.in.ua
marketingvici.com meccadumps.net
maxthon.com medicinacom.ru
maxxtor.eu medicine-4u.org
mazda-roadsters.com
mecash.ru
medkletki.ru
meetingrainstorm.bid
megakino.net megakino.net
meget.co.za mesto-x.com
metabar.ru
michaelkorssaleoutletonline.net michaelkorssaleoutletonline.net
mirmedinfo.ru mil-stak.com
missclub.info millionare.com
mobifunapp.weebly.com mir-betting.ru
moetomnenie.com misswell.net
mojaocena.com mister-shop.com
mojpreskumanie.com mmofreegames.online
moncleroutletonline.pw monarhs.info
montredemarque.nl moneymaster.ru
mostantikor.ru morepoweronmac.trade
musicstock.me mossmesi.com
mustat.com mozello.ru
muzaporn.com mp3films.ru
mylida.org mrmoneymustache.com
naperehresti.info musicas.baixar-musicas-gratis.com
narkologiya-peterburg.ru mxgetcode.com
neobux-bg.info mybackgroundlandscape.blogspot.com
nerudlogistik.ru mydownlodablefiles.com
netanalytics.xyz nacap.ru
next-dentists.tk naturalbreakthroughsresearch.com
nfljerseys.online navalwiki.info
novodigs.com ningessaybe.me
novosti-hi-tech.ru noclegonline.info
nsatc.net novgorod.xrus.org
o-o-6-o-o.com nuit-artisanale.com
o.light.d0t.ru nullrefer.com
obiavo.by obiavo.uz
oconto.ru obnal.org
ofanda.com obsessionphrases.com
odywpjtw.bloger.index.hr
oecnhs.info
of-ireland.info
omgtnc.com
onclkads.com onclkads.com
onlinadverts.com openfrost.net
online-hd.pl oren-cats.ru
online-hit.info ownshop.cf
online-x.ru ownshop.win
onlinewritingjobs17.blogspot.ru p-business.ru
osnova3.ru page2rss.com
osoznanie-narkotikam.net
oxford-book.com.ua
paclitor.com
paleohub.info paleohub.info
palocco.it parajumpersjakkesalgnorge.info
parajumpersstore.online partners-ship.pro
pateaswing.com pay2me.pl
pertlocogasilk.tk photochki.com
php-market.ru picurams.pw
pics-group.com pinapchik.com
picsforkeywordsuggestion.com pinstake.com
pipki.r.acdnpro.com pinwallpaper.xyz
pirateday.ru pio.polytopesexempt.com
playtap.us piter.xrus.org
plugingeorgia.com plastjulgranar.se
pogodnyyeavarii.gq platezhka.net
pons-presse.com play.leadzupc.com
popugauka.ru plohaya-kreditnaya-istoriya.ru
porno-raskazy.ru pochtovyi-index.ru
pornobrazzers.biz podshipniki-nsk.ru
pornoelita.info poquoson.org
pornokorol.com porno.simple-image.com.ua
pornophoto.xyz pornzone.tv
pornosee.info
pornosemki.info
pornotubexxx.name
portadd.men portadd.men
prefersurvey.net potolokelekor.ru
preg.marketingvici.com pozdravleniya-c.ru
pregnant.guru predmety.in.ua
priora-2.com priceg.com
privacyassistant.net profitfx.online
privetsochi.ru promalp-universal.ru
profitkode.com promoheads.com
projefrio.com.br push-ad.com
prosmibank.ru pushdata.sendpulse.com
qpypcx.com pussyspace.net
quizzitch.net quality-traffic.com
qwesa.ru quickchange.cc
randki-sex.com raschtextil.com.ua
ranking2017.ga rasteniya-vs-zombi.ru
razamicroelectronics.com remybutler.fr
rebrand.ly rixpix.ru
recipedays.com robot-forex.biz
reining.lovasszovetseg.hu rrutw.com
remedyotc.com running-line.ru
rennlist.com ruscopybook.com
replicaclub.ru
research.ifmo.ru
ricorsogiustizia.org
robertefuller.com
rocis.site
rogervivierforsale.com
roll123.com
romhacking.ru
rossanasaavedra.net
royal-betting.net
ruex.org.ua
runofilms.ru
rusoft-zone.ru
rustic-quiver.win
s1z.ru
sabizonline.com sabizonline.com
saecsa.co safe-app.net
samara.rosfirm.ru sardinie.us
savefrom.com sbornik-zakonov.ru
sbtdesign.co.uk sbricur.com
scanner-alexa.top sbt-aqua.ru
scanner-margo.top scanner-margo.top
school-diplomat.ru scanner-marwin.top
scmor.ilxc.cc scmor.ilxc.cc
searchkut.com searchimage.co
see-your-website-here.com securesmrt-dt.com
sell-fb-group-here.com seeingmeerkat.com
seorank.info seeresultshub-a.akamaihd.net
seorussian.ru seoanalyses.com
serptehnika.ru seozoom.it
service-core.ru service.adtech.fr
sevendays.com.ua
sexcamamateurchat.com sexcamamateurchat.com
sexkontakte-seite.com sexkontakte-seite.com
sexpartygirls.net sexstream.pl
sexpornotales.net shinikiev.com.ua
sexuria.net shoesonlinebuy.xyz
shoppingjequiti.com.br shop.acim.org
siha.de sientalyric.co
simpoed.ufop.br sitechecker.pro
site.ru siteexpress.co.il
sitevaluation.com siteonomy.com
skuteczna-dieta.co.pl sitopreferito.it
slavkokacunko.de skanninge.se
smartadserver.com slonechka.ru
smartpet.ru
snegozaderzhatel.ru
sobecjvuwa.com.ru sobecjvuwa.com.ru
soc-econom-problems.ru
social-buttons.com
sockshare.net
sofit-dmd.ru sofit-dmd.ru
sogimlecal.tk softwaretrend.net
spamnuker.com solnplast.ru
spb.ru soserfis.com
spbchampionat.ru sovinsteel.ru
spravka130.ru speeddream.xyz
spy-app.info splendorsearch-a.akamaihd.net
spy-sts.com sports-supplements.us
ssconstruction.co spymac.net
stairliftsarea.com stard.shop
stariy-baku.com stardevine.com
stop-gepatit.te.ua stoki.ru
stpolice.com stpolice.com
strana-solnca.ru streamin.to
stroy-portal22.ru supers.com.ua
studiofmp.com surintech.ac.th
sucsesofinspiration.com svolze.com
sumo.com szucs.ru
superinterstitial.com taihouse.ru
tacbelarus.ru tam-gde-more.ru
telefonsex-ohne0900.net taraz.xkaz.org
teletype.in td-33.ru
tetracsaudi.com telefonsexsofort.tk
thegioixekhach.com template-kid.com
ticketsys.inetwd.com tfxiq.com
time-japan.ru themestotal.com
tizanidine6mg.blogspot.com thepokertimer.com
tizanidinehcl2mg.blogspot.com thexart.club
tizanidinehydrochloride4mgstreetvalue.blogspot.com tizanidineinfo.blogspot.com
tizanidinenarcotic.blogspot.com tizanidinepain.blogspot.com
tizanidinevscyclobenzaprine.blogspot.com tizanidineshowupondrugtest.blogspot.com
tk-assortiment.ru tizanidinezanaflex.blogspot.com
tmm-kurs.ru toon-families.com
todohr.com toondinsey.com
token-lab.org topvidos.ru
topappspro.com trackzapper.com
totu.us trafaret74.ru
toyota.7zap.com traffic100.com
tracfone.com trafficnetzwerk.de
trkdf.com trainoffend.ml
trkur.com trichizobswiv.agddns.net
trucri.me trustedhealthtips.com
tubeline.biz tubo360.com
tupper-posuda.ru tzritel.tk
twojebook.pl uhodzalijami.ru
tyumen.xrus.org underthesite.com
uhod-za-sobakoj.ru uptime-delta.net
ukr-lit.com urzedowski.eu
ukrobstep.com us-america.ru
unmaroll.ya.ru usswrite.com
upstore.me utrolive.ru
ural-buldozer.ru uytmaster.ru
urccvfmc.bloger.index.hr valkiria-tk.ru
vacuumcleanerguru.com vbtracker.net
vavilone.com vesnatehno.ru
vekzdorov.ru vigrx-original.ru
vellings.info virus-schutzmasken.de
velpanex.ru viteonlusarezzo.it
verymes.xyz vizitki.net
video-camer.com vladhistory.com
videos-for-your-business.com vsexkontakte.net
viel.su vzubah.com
vintontech.info wallpapers-best.com
vinylvault.co.uk warezkeeper.com
vip-dom.in
vipsiterip.org
vkmusics.ru
vonradio.com
vozbujdenie.com
vvpg.ru
vysigy.su
vzglyadriv.kg
vzube.com
wait3sec.org
waterefficiency.co waterefficiency.co
weboptimizes.com web-analytics.date
website-speed-check.site webalan.ru
websites-reviews.com webnode.me
wedding-salon.net website-analyzer.info
weightatraining.com websiteaccountant.de
wholinkstome.com westsextube.com
wildworld.site wetgames.ru
wladimirpayen.com wholesalejerseychinashop.com
wleuaprpxuvr.ga winterclassichockeyjerseys.com
woman-h.ru woman-h.ru
workle.website womensterritory.ru
writingservices17.blogspot.ru world-mmo.com
www.get-free-traffic-now.com worldinternetauthority.com
www.timer4web.com wowcasinoonline.ooo
x-mix.info wsgames.ru
xclicks.net ww1943.ru
x-true.info
xbaboon.com
xfluro.com
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xn----9sbebi2bvzr7h.xn--p1ai xn----9sbebi2bvzr7h.xn--p1ai
xn--e1aggki3c.xn--80adxhks xn--80abgj3a5acid6ghs.top
xpresscare.ru xn--80aodinpgi.xn--p1ai
xxxmania.top xn--e1afanlbnfckd7c3d.xn--p1ai
xxxrus.org xn--hxazdsfy.blogspot.com
xz618.com xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
yoluxuryevents.com xxxhdvideo.site
yopoint.in yachts-cruise.info
youandcredit.ru yadro.ru
yubikk.info yoopsie.com
zaapplesales.blogspot.com yougotanewdomain.com
zaidia.xhost.ro yoursearch.me
yur-p.ru
z-master.ru
zajm-pod-zalog-nedvizhimosti.ru
zakazfutbolki.com zakazfutbolki.com
zakonobosago.ru zeroredirect8.com
zaloro.com zetmaster.ru
zaobao.com.sg zigarettenonlinekaufen2.drupalgardens.com
zarabotat-na-sajte.ru zona-aqua.ru
zdesoboi.com zoomovies.org
zerocash.msk.ru zryydi.com
zetgie.com.pl zvezdagedon.ru
zigarettenonlinekaufen2.bloog.pl
zvooq.eu
zzlgxh.com

490
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
03p.info 00it.com
1008.su 123movies.love
100dollars-seo.com
1kinobig.ru
2pxg8bcf.top
4inn.ru 4inn.ru
4kpics.rocks 5kstore.com
aa625d84f1587749c1ab011d6f269f7d64.com a.frcls.fr
actices.com accessoires-mode.in
adconscious.com adanih.com
adidas.frwebs.fr
adloads.com
adrenalin-stk.ru
adssafeprotected.com adssafeprotected.com
adviceforum.info ahrefs.com
advokat-grodno.by akusherok.ru
afmuseum.com alert-fjg.xyz
allblogroll.com all-streaming-media.com
alot.com allboard.xobor.de
animenime.ru allornamenti.com
apibrowsesmartne-a.akamaihd.net animalrank.com
apifasterlightin-a.akamaihd.net anime.dougasouko.com
apisecretsaucebi-a.akamaihd.net anlimebel.kiev.ua
approved.su anonymous-redirect.com
approvedlocal.co.za anonymousfox.co
autogrep.ru apccargo.com
beachdriveblog.com apibatbrowsecom-a.akamaihd.net
beachtoday.ru apiwoodensealcom-a.akamaihd.net
best-seo-software.xyz arate.ru
betshuckclean.com arcteryxsale.online
bg6s0.com arpelsreplica.xyz
bioca.org arraty.altervista.org
bizfly.info artel116.ru
biznesluxe.ru as5000.com
book-bets.com astana.xxxkaz.org
bradleylive.xyz auto4style.ru
brandehk.dk avtolombard-krasnodar.ru
bscodecs.com azte.ch
buttons-for-your-website.com bambi.ck.ua
cakemediahq.com.au bannerconnect.net
cartierlovebracelet.xyz bausparen.bz.it
cezartabac.ro best-seo-offer.com
cfjump.com bestcoin.cc
cheapjerseysbizwholesale.us bharatdefencekavach.com
cialischstgerts.com bizcheapjerseyswholesalechina.com
cialisovercounteratwalmartusa.com bizru.info
citetick.com bonux.nextview.ru
cityadspix.com bosman.pluto.ro
clickso.com brainboostingsupplements.org
brandedleadgeneration.com
brimstonehillfortress.org
castingbank.ru
cctva.tv
celebsopics.com
cheapbarbour.online
choosecuisine.com
christianlouboutinreplica.pw
clknsee.com clknsee.com
cornerstone-countertops.com club-lukojl.ru
crackguru.tk comissionka.net
crazyboost.pro compliance-alex.top
daooda.com compliance-george.xyz
dawlenie.com compliance-walter.top
delta-line.men converse.ddsoldes.fr
devochki.top coolyarddecorations.com
dimkino.ru cordstrap.cc
dinkolove.ya.ru courtshipgift.com
dirtpics.pw crafthubs.com
diusyawiga.tk creditnation.ru
djekxa.ru dieswaene.com
dominterior.org digital-sale.su
druzhbany.ru dimfour.com
eaplay.ru dividendo.ru
edudocs.net dogclothing.store
elektrischezigarettekaufen2.cowblog.fr doktoronline.no
eluxer.net domainsatcost.com
en.altezza.travel dyshagi.ru
enthuse.computernetworksonline.com ebonyporn.site
eralph.tk ed-shop01.ru
family1st.ca editors.choice6912650.hulfingtonpost.com
felizporno.com ekspresihati.info
finansov.info elektrischezigarette2.onsugar.com
fitfloponline.store eorogo.top
formularz-konkurs.tk errorfixing.space
fortevidyoze.net escort-russian.com
forum.doctissimo.fr essaypro.com
frankofficial.ru eu-cookie-law.blogspot.com
freelotto.com expertnaya-ocenka.ru
freemags.cc fable.in.ua
freemaintenancesysforpcandmac.top factorynightclub.com
freenom.link fanrto.com
fuckbuddybestgilf.info fetroshok.ru
gallerylisting.com filmania-x.ru
gallictures.com finstroy.net
gambarkatabaru.com fitness-video.net
gamezblox.com flex4launch.ru
gardene.ru forex21.ru
gdzkurokam.ru forums.d2jsp.org
get-seo-domain.com foto-weinberger.at
getlamborghini.ga fr-bearings.ru
gogps.me franklinfire.co
google-liar.ru freetangodownload.com
graphicwe.org frighteningremain.cf
handicapbathtubarea.com fsakhalin.ru
hawaiielectriclight.com funtoonez.com
hazardky.net get-your-social-buttons.info
heliko.no ghernnqr.skyrock.com
home.myplaycity.com gok-kasten.net
hotchatdate.com greatgrace.ru
howlongdoestizanidinestayinyoursystem.blogspot.com hdpixent.com
hoztorg-opt.ru hostcritique.com
idealtits.net hoverboardforsaledirect.com
igithab.com iboss.com
imitex-plus.ru ideibiznesa2015.ru
in-tandem.co iklysha.ml
incep.imagine-ex.co irkutsk.online-podarki.com
infodocsportal.com istock-mebel.ru
investzalog.ru itag.pw
iusstf.org itronics.ca
janerikholst.se iwanttodeliver.com
jerseyschinabizwholesale.us javrip.net
jongose.ninja joingames.org
kamalsinha.com katadhin.co
karting196.ru kinohit1.ru
keywordglobal.co.za klejonka.info
kinoduh.ru knowyournextmove.com
laulini.soclog.se kolotiloff.ru
law-check-seven.xyz komputers-best.ru
letslowbefast.today kostenlos-sexvideos.com
lionking-1994.blogspot.com ladov.ru
landscapebackgrounds.blogspot.com
lashstudia.ru
likesuccess.com
linerdrilling.com
lion.playtap.us
lirunet.ru
livefixer.com
llastbuy.ru llastbuy.ru
longlifelomilomi.info lost-alpha.ru
luciddiagnostics.in mafa-free.com
manimpotence.com magda-gadalka.ru
maranbrinfo.com.br makis.nu
masturbate.co.uk marketland.ml
mcar.in.ua matchpal-a.akamaihd.net
mebel-vstroika.ru med-zdorovie.com.ua
medizinreisen.de member-quiz.com
menetie.ru michaelkorsoutletstore.net
menhealed.net mir-limuzinov.ru
minecraft-rus.org mp3ringtone.info
mmstat.com mp3ritm.top
moetomnenie.com mulberryoutletonlineeu.com
moi-glazki.ru music.utrolive.ru
monarhs.info myliveblog.ru
monclerjacketsoutlet.win nastydollars.com
na15.ru negociosdasha.com
news-readers.ru njkmznnb.ru
nfljerseys.online nodding-passion.tk
no-fuel.org nyfinance.ml
okayimage.com oivcvx.website
okmusic.jp online-x.ru
oksrv.com openmultipleurl.com
painting-planet.com ostrovtaxi.ru
partnerline.men paighambot.com
patterntrader-en.com pandroid.co
perm.xrus.org partner-pop.men
picturesmania.com pcgroup.com.uy
pistonclasico.com photorepair.ru
pl-top.pl picquery.com
pl-vouchers.com picturesfrom.com
pochtovyi-index.ru pk-services.ru
poems.com.ua placid-rounded-coast.glitch.me
popugauka.ru plusnetwork.com
pornhive.org pochemychka.net
pornophoto.xyz podshipniki-ntn.ru
pornoreino.com poptool.net
pozdrawleniya.ru porno.simple-image.com.ua
pravoholding.ru pornofiljmi.com
profolan.pl portside.cc
proftests.net profitkode.com
promoheads.com pskcijdc.bloger.index.hr
prostitutki-almata.org quelle.ru
rada.ru realmonte.net
randki-sex.com richinvestmonitor.com
rankings-analytics.com roma-kukareku.livejournal.com
ranksonic.info rukino.org
raschtextil.com.ua saltspray.ru
razvratnoe.org samoiedo.it
recipedays.com sardinie.us
rethinkwasteni.info scanner-donald.top
rock-cafe.info
rubanners.com
russintv.fr
scansafe.net scansafe.net
scottbywater.com seansonline24.pl
seasonvar.ru seeresultshub-a.akamaihd.net
seomarketings.online seo-smm.kz
sexgalleries.top seotoolsagency.com
sextracker.be servicecenter.co.ua
shivafurnishings.com servisural.ru
sildenafil-tadalafil.info sientalyric.co
socas.pluto.ro simply.net
softlinesolutions.me sledstvie-veli.net
solinf.co slujbauborki.ru
soviet-portal.do.am smallseotools.com
sps-shop.com soc-proof.su
stevemonsen.com social-s-iii.xyz
stopnarco.ru socialbuttons.xyz
sundrugstore.com statistici.ro
supermodni.com.ua stats-collector.org
surintech.ac.th streetfooduncovered.com
tako3.com studiokamyk.com.pl
tedy.su suchenindeutschland.com
thaismartloan.com taaaak.com
thedownloadfromwarez.blogspot.com teastory.co
thefarmergame.com teesdaleflyballclub.co.uk
thiegs.reco.ws texbaza.by
thin.me.pn thaisamkok.com
timdreby.com thenetinfo.com
tizanidinerecreationaluse.blogspot.com time-japan.ru
toondinsey.com tizanidine4mgprice.blogspot.com
top-deal.com.pl top10-online-games.com
top10-way.com toxicwap.com
topcasinoratings.ru traffic2cash.org
topdownloads.ru trkur.com
topvidos.ru trustl.life
treasuretrack-a.akamaihd.net tubeoffline.com
tubo360.com tvorozhnaja-zapekanka-recept.ru
tversvet.ru twojebook.pl
tvnewsclips.info uchetunet.su
twincitiescarservice.com unacittaconte.org
twitlinks.com unitygame3d.com
txxx.com valoresito.com
tytoona.com vgoloveboli.net
usiad.net viagraneggrx.com
utrolive.ru videosbox.ru
veerotech.com vip.51.la
versaut.xxx-cam.webcam voluumtracker1.com
vids18.site web-betting.ru
vipps.com.my webcamdevochka.com
vladimir.xrus.org website-datenbank.de
volgograd.xrus.org websitevaluebot.com
vtc.pw whereiskentoday.com
vtcdns.com wholesalenfljerseys.us.com
vzlom-na-zakaz.com womens-journal.net
wallpaperaccess.com ww2awards.info
warning.or.kr www.get-free-traffic-now.com
waycash.net www.timer4web.com
waysbetter.cn xjlottery.com
web-analytics.date xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
webmonetizer.net xn--90acenikpebbdd4f6d.xn--p1ai
webpromotion.ae xn--oogle-wmc.com
wemarketing.se xsion.net
white-truck.life xxxguitars.com
wordkeyhelper.com yellowstonevisitortours.com
wpsecurity.website yes-com.com
wufak.com yottos.com
www.jbetting.com yur-p.ru
xn--hxazdsfy.blogspot.com zakonobosago.ru
xnxx-n.com zigarettenonlinekaufen2.drupalgardens.com
xtremeeagles.net
yaaknaa.info
youghbould.wordpress.com
yourmovies.pl
zerocash.msk.ru
zytpirwai.net

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

41
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,6 +8,23 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -477,20 +494,6 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -581,14 +584,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.210 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2696 ### Version: V4.2021.11.2697
### Updated: Fri Nov 5 22:01:15 UTC 2021 ### Updated: Sat Nov 6 10:03:35 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2696 ### Version: V4.2021.11.2697
### Updated: Fri Nov 5 22:01:15 UTC 2021 ### Updated: Sat Nov 6 10:03:35 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##