vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2719 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-16 22:07:07 +00:00
parent f02cb62658
commit 275fd252fa
47 changed files with 63686 additions and 63693 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2718 #### Version: V4.2021.11.2719
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2718 #### Version: V4.2021.11.2719
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2718 #### Version: V4.2021.11.2719
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7804
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2718 #### Version: V4.2021.11.2719
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7808
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7808
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

7808
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7808
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

7808
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

7808
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

7808
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2718 2719

7808
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

7808
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.175.203.137 0; 52.142.59.40 0;

7808
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

7808
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

7808
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

7808
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.175.203.137 0; 52.142.59.40 0;

7808
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.175.203.137 0; 52.142.59.40 0;

7808
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.1.76 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Barkrowler CherryPicker
Bolt JOC Web Spider
Crawling at Home Project MarkMonitor
DomainAppender MicroMessenger
FDM Picscout
LinqiaScrapeBot Psbot
NetSpider RankFlex
Nibbler SEOkicks
Spinn3r Spammen
plumanalytics seobility

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
80legs 80legs
ADmantX ALittle Client
AIBOT ASPSeek
Aboundexbot
Acunetix
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot AiHitBot
Aipbot
AlphaBot AlphaBot
Anarchie
Anarchy Anarchy
Ankit Anthill
Asterias AwarioSmartBot
Attach
BBBike BBBike
BDCbot
BDFetch BDFetch
BackWeb Badass
BacklinkCrawler
Bandit
Barkrowler Barkrowler
BatchFTP Battleztar Bazinga
Bitacle BetaBot
Black Hole Bigfoot
BlackWidow
Blow Blow
BlowFish
Boardreader Boardreader
Bolt
BotALot BotALot
Brandprotect
Brandwatch
Buck
Buddy Buddy
CCBot BuiltBotTough
CODE87 CODE87
CSHttp
Calculon Calculon
CazoodleBot CazoodleBot
Cegbfeieh ChinaClaw
CensysInspect Cliqzbot
CheeseBot
CherryPicker
Chlooe
Cloud mapping
Cocolyzebot Cocolyzebot
Collector Collector
CopyRightCheck CopyRightCheck
Craftbot Copyscape
Crawling at Home Project
Crescent Crescent
CyotekWebCopy Custo
DIIbot
DSearch
DataCha0s
DatabaseDriverMysqli
Demon Demon
Deusu Devil
Digincore
DigitalPebble DigitalPebble
Disco
Discobot Discobot
Discoverybot DomainAppender
DittoSpyder DomainCrawler
DomCopBot
DomainStatsBot
Download Wonder
Dragonfly Dragonfly
EMail Siphon Drip
ECCP/1.0
EMail Wolf EMail Wolf
EasyDL
Ebingbong Ebingbong
EirGrabber Ecxi
EroCrawler
Evil Evil
Exabot Express WebPictures
Extractor ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
FDM FDM
FHscan
FemtosearchBot FemtosearchBot
Firefox/7.0
Flunky Flunky
Freeuploader FyberSpider
FrontPage GalaxyBot
Fuzz GermCrawler
Fyrebot Getintent
Go!Zilla Grabber
Go-Ahead-Got-It
GoZilla
Grafula Grafula
GrapeshotCrawler HMView
GridBot Havij
HEADMasterSEO Humanlinks
HTMLparser IDBTE4M
HaosouSpider
Harvest
HybridBot
IDBot
IRLbot IRLbot
Image Sucker Iblog
Id-search
Image Fetch
IndeedBot
InfoNaviRobot InfoNaviRobot
InfoTekies
Intelliseek
InterGET
Internet Ninja Internet Ninja
InternetSeer InternetSeer
Iskanie
JennyBot
JetCar
Jetty
JikeSpider
Jorgee
Jyxobot Jyxobot
Kenjin Spider Kenjin Spider
Keyword Density Kinza
Kozmosbot
LWP::Simple
Lanshanbot Lanshanbot
LeechGet
Lftp
LibWeb LibWeb
Lightspeedsystems LinkScan
LinkextractorPro LinqiaMetadataDownloaderBot
LinksManager Lipperhey
Lipperhey Spider Lipperhey Spider
MIDown tool
MIIxpc
MJ12bot
MQQBrowser
MSFrontPage
MTRobot
Mag-Net Mag-Net
Magnet Magnet
Majestic SEO
Majestic-SEO
MarkWatch MarkWatch
Masscan Masscan
MauiBot Mata Hari
Meanpathbot
Mediatoolkitbot Mediatoolkitbot
Metauri
MicroMessenger
Microsoft Data Access Microsoft Data Access
Mister PiX Mister PiX
Moblie Safari
Morfeus Fucking Scanner
Mozlila Mozlila
Musobot Mr.4x3
NPbot
Name Intelligence Name Intelligence
Nameprotect Nameprotect
NearSite
Nessus Nessus
NetLyzer
Netcraft
Nettrack Nettrack
Nimbostratus Netvibes
Nmap NextGenSearchBot
Nutch Niki-bot
OpenLinkProfiler NimbleCrawler
Ninja
Nuclei
Octopus
Offline Navigator
OpenVAS OpenVAS
Openvas
OrangeBot OrangeBot
OrangeSpider
OutclicksBot OutclicksBot
OutfoxBot OutfoxBot
PECL::HTTP PHPCrawl
Pandalytics POE-Component-Client-HTTP
Petalbot Panscient
Piepmatz PeoplePal
Pixray Pi-Monster
ProWebWalker Picsearch
Probethenet Psbot
Pu_iN
Pump Pump
QueryN Metasearch PyCurl
RankActive
RankActiveLinkBot
RankingBot
RankingBot2
RankurBot
ReGet ReGet
RealDownload RealDownload
Reaper RebelMouse
SBIder RepoMonkey
SEOkicks-Robot Ripper
SEOstats RocketCrawler
SalesIntelligent SMTBot
ScanAlert ScoutJet
Scanbot Scrapy
SearchmetricsBot Screaming
SemanticJuice
Semrush
SemrushBot
SentiBot SentiBot
SeoSiteCheckup
Shodan Shodan
SiteCheckerBotCrawler
SiteLockSpider
SiteSucker SiteSucker
Sitebeam SlySearch
Siteimprove Snake
Sitevigil
SmartDownload
Snapbot Snapbot
SocialRankIOBot
Sogou web spider
Sosospider Sosospider
SpaceBison SpaceBison
Spammen Sqlmap
SpankBot
Spbot
Spinn3r
Sqlworm
Stripper Stripper
Sucker
SuperBot SuperBot
SuperHTTP SuperHTTP
SurveyBot SurveyBot
Suzuran Suzuran
Swiftbot T0PHackTeam
Teleport Teleport
TeleportPro TeleportPro
Telesphorep Telesoft
TightTwatBot Thumbor
Toata
Tracemyfile
Trendictionbot
True_Robot True_Robot
Turingos Turingos
Typhoeus Turnitin
UnisterBot
Upflow Upflow
V-BOT
VB Project VB Project
VCI
Vagabondo Vagabondo
VelenPublicWebCrawler
VidibleScraper VidibleScraper
VoidEYE
Voltron
WASALive-Bot WASALive-Bot
WBSearchBot
WEBDAV WEBDAV
WWW-Collector-E WISENutbot
WWW-Mechanize WWWOFFLE
Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE
Web Auto
Web Collage Web Collage
Web Enhancer
Web Fetch
Web Fuck Web Fuck
Web Pix
Web Sucker Web Sucker
WebCopier WebBandit
WebEnhancer WebFuck
WebSauger WebGo IS
WebSucker WebReaper
WebWhacker WebZIP
Webalta
Webshag Webshag
WebsiteExtractor
WebsiteQuester WebsiteQuester
Whack
Whacker Whacker
Whatweb Widow
Who.is Bot
WinHTTrack
WiseGuys Robot WiseGuys Robot
Xenu Wprecon
Zade Zauba
Zermelo Zermelo
ZoomBot
ZoominfoBot ZoominfoBot
adscanner ZyBorg
arquivo-web-crawler arquivo.pt
autoemailspider autoemailspider
backlink-check
cah.io.community
check1.exe
coccocbot-web
com.plumanalytics com.plumanalytics
crawl.sogou.com crawl.sogou.com
evc-batch crawler4j
ips-agent dataforseo.com
isitwp.com eCatch
muhstik-scan facebookscraper
heritrix
internetVista monitor
lwp-request
magpie-crawler
oBot
page scorer
pcBrowser
plumanalytics
polaris version
probe-image-size
scalaj-http
scan.lol scan.lol
seobility
seocompany.store
seoscanners
seostar
sexsearcher
trendiction.de trendiction.de
webmeup-crawler webmeup-crawler
webpros.com
webprosbot webprosbot
x22Mozilla
zgrab

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404enemy 404enemy
80legs
ADmantX ADmantX
AIBOT AfD-Verbotsverfahren
ASPSeek Alexibot
Aipbot Alligator
AlphaBot Anarchy
Anarchie Aspiegel
Apexoo
Asterias Asterias
AwarioRssBot AwarioRssBot
BDCbot AwarioSmartBot
Backlink-Ceck BLEXBot
BackStreet
BackWeb
Badass Badass
Bandit Barkrowler
BatchFTP BatchFTP
Battleztar Bazinga
BetaBot BetaBot
Bigfoot Bigfoot
Bitacle Bitacle
Blackboard Black Hole
BotALot Blow
Brandprotect Brandprotect
Buddy Buck
Bullseye Bullseye
CATExplorador CATExplorador
CSHttp CCBot
CODE87
Calculon Calculon
Cegbfeieh Cegbfeieh
CheeseBot CensysInspect
CherryPicker CherryPicker
ChinaClaw
Chlooe Chlooe
Claritybot Cliqzbot
Cocolyzebot Cloud mapping
Collector Cogentbot
Copyscape Cosmos
Craftbot Craftbot
DBLBot Crawling at Home Project
Curious
CyotekWebCopy
DIIbot
DTS Agent DTS Agent
Deusu
Devil Devil
Digincore Disco
Discoverybot Discoverybot
DnBCrawler-Analytics DittoSpyder
DomainSigmaCrawler DomainCrawler
DomainStatsBot DomainStatsBot
Domains Project Download Wonder
Ebingbong EMail Siphon
EMail Wolf
EasyDL
EirGrabber
EroCrawler EroCrawler
Exabot
Express WebPictures Express WebPictures
Extractor
FDM
FHscan
Fimap Fimap
Firefox/7.0 Firefox/7.0
FlashGet FlashGet
Foobot Foobot
Freeuploader Freeuploader
Fuzz FrontPage
Fyrebot
G-i-g-a-b-o-t
GT::WWW
GalaxyBot GalaxyBot
Genieo GetRight
GetWeb
Getintent Getintent
Go-Ahead-Got-It Gigabot
GoZilla GoZilla
Gotit Gotit
Grabber
Grafula
GrapeshotCrawler GrapeshotCrawler
GridBot
HEADMasterSEO HEADMasterSEO
HTMLparser HMView
HTTP::Lite
HTTrack HTTrack
Haansoft Haansoft
HaosouSpider
Havij
Heritrix Heritrix
Hloader Iblog
Id-search
Image Fetch Image Fetch
Image Sucker
IndeedBot
Indy Library
InternetSeer
Iria Iria
IstellaBot IstellaBot
Jbrofuzz JOC Web Spider
JamesBOT
JennyBot
JetCar JetCar
Jetty
JikeSpider JikeSpider
Kenjin Spider Joomla
Kozmosbot Jyxobot
Kinza
LNSpiderguy
Leap
LeechFTP LeechFTP
LeechGet
Lftp Lftp
LieBaoFast
Lightspeedsystems
Likse
LinkScan
LinkWalker LinkWalker
Linkbot
Linkdexbot Linkdexbot
LinkpadBot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker Lmspider
Ltx71 Ltx71
MIIxpc MFC_Tear_Sample
MJ12bot MIDown tool
MTRobot MQQBrowser
Mag-Net MSFrontPage
Magnet Majestic SEO
Mata Hari Mass Downloader
MicroMessenger Mb2345Browser
Mr.4x3 MeanPath Bot
Musobot Metauri
Nameprotect Moblie Safari
Needle Mojeek
Morfeus Fucking Scanner
NICErsPRO
NPbot
Name Intelligence
NetAnts NetAnts
NetSpider NetZIP
Netcraft Netcraft
Nettrack Nettrack
Nikto NextGenSearchBot
NimbleCrawler
Nimbostratus
Ninja Ninja
Nmap Nmap
Nuclei
Octopus Octopus
Offline Explorer OrangeBot
OnCrawl
Openfind
Openvas
OrangeSpider OrangeSpider
OutfoxBot OutclicksBot
POE-Component-Client-HTTP PageGrabber
PageScorer PageThing.com
Pandalytics
PeoplePal PeoplePal
Picscout Picscout
Pixray
PleaseCrawl
Pockey
ProPowerBot ProPowerBot
Psbot ProWebWalker
Pu_iN Probethenet
RankFlex PyCurl
RebelMouse QueryN Metasearch
RankActive
RankingBot
RankurBot
Re-re
Reaper
RedesScrapy RedesScrapy
SEOkicks-Robot Rogerbot
SEOlyticsCrawler SBIder
SEOprofiler SEOprofiler
SEOstats
SMTBot SMTBot
ScanAlert ScoutJet
Scrapy
Screaming
Searchestate
SemanticJuice
Semrush Semrush
SemrushBot SeobilityBot
Seomoz Seomoz
Site Sucker Shodan
SiteExplorer Siphon
SiteLockSpider SiteCheckerBotCrawler
SiteSucker SiteSnagger
Sitevigil Siteimprove
SmartDownload SmartDownload
Snapbot
Snoopy Snoopy
SocialRankIOBot
Sociscraper
Sosospider Sosospider
Sottopop SpankBot
Spanner Sqworm
Steeler SurveyBot
SuperBot
SuperHTTP
Surfbot
Suzuran
Swiftbot Swiftbot
Szukacz T0PHackTeam
T8Abot
Teleport Teleport
TeleportPro
Telesoft Telesoft
Telesphoreo
The Intraformant The Intraformant
TheNomad Titan
Toata Toata
Toweyabot
Trendiction
Trendictionbot Trendictionbot
Turnitin TwengaBot
URLy Warning V-BOT
Upflow VCI
VB Project Vagabondo
Vacuum
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Virusdie Virusdie
Voltron Voltron
WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV
WISENutbot WISENutbot
WPScan
WWW-Mechanize
WWWOFFLE WWWOFFLE
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
Web Auto WallpapersHD
Web Enhancer Web Collage
Web Fetch Web Pix
Web Sauger
WebAuto
WebCopier WebCopier
WebEnhancer
WebFuck WebFuck
WebGo IS
WebImageCollector WebImageCollector
WebPix WebReaper
WebSauger WebSauger
WebWhacker Webshag
WebZIP
Webalta
Website Quester Website Quester
WebsiteExtractor Whacker
Webster Who.is Bot
WinHTTrack WiseGuys Robot
Xaldon_WebSpider Wotbox
YoudaoBot Wprecon
Zade Xaldon WebSpider
ZoomBot Xenu
Zeus
ZoominfoBot ZoominfoBot
ZyBorg
adscanner
archive.org_bot
autoemailspider
cah.io.community
check1.exe check1.exe
clark-crawler
coccocbot-web coccocbot-web
cognitiveseo
crawler4j
dataforseo.com dataforseo.com
demandbase-bot demandbase-bot
facebookscraper
gopher gopher
instabid instabid
internetVista monitor internetVista monitor
ips-agent lwp-request
isitwp.com
magpie-crawler magpie-crawler
muhstik-scan meanpathbot
netEstate NE Crawler mediawords
pcBrowser
plumanalytics plumanalytics
ripz ripz
satoristudio.net s1z.ru
scalaj-http scalaj-http
seobility scan.lol
seocompany.store seostar
seoscanners
serpstatbot serpstatbot
sitechecker.pro sitechecker.pro
siteripz trendiction.com
sogouspider trendiction.de
sysscan
tAkeOut
ubermetrics-technologies.com
voyagerx.com voyagerx.com
x09Mozilla webpros.com

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
01apple.com aibolita.com
apiholdingmypage-a.akamaihd.net cand.jp
appsecurityr.com dokfilms.net
custom-product-labels.com fickkontakte.org
elektronischezigarettekaufen1.beeplog.com glogow.pl
hit-men.men goodnightjournal.com
mesto-x.com musirc.com
mini.7zap.com mycaf.it
sexystrippe.info scanner-elena.top
tengohydar.tk uzporno.mobi

926
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
007angels.com
0daymusic.org 0daymusic.org
123movies.love 100searchengines.com
101billion.com
123cha.com
123locker.com
18ps.ru 18ps.ru
2015god.org 1j7740kd.website
2daytrendingnews.com 202ch.com
4pp13.com 5u.com
5kstore.com 66cpwgln.space
8lufu.com 76brighton.co.uk
a14download.com 883zy.com
aac500b7a15b2646968f6bd8c6305869d7.com 888.com
absolutelyanalog.com 98oi.ru
acc.eu.org 991mostfm.co.id
acheterviagrafr24.com aab94f698f36684c5a852a2ef272e031bb.com
addtoadd.men abctoppictures.net
adbetclickin.pink
adsland.men adsland.men
affordablewebsitesandmobileapps.com adult-shop.com.ua
ageofclones.com aghanyna.com
aihelen.net akita.kz
alert-hgd.xyz akusherok.ru
all4invest.info all-streaming-media.com
allboard.xobor.de alldezservice.kz
allornamenti.com aludecor.info
allwomen.info alveris.ru
alot.com amospalla.es
ameblo.jp angigreene.com
amung.us apidiamondatanet-a.akamaihd.net
animal-drawings.com apiwoodensealcom-a.akamaihd.net
apartmentratings.com arcadepages.com
apimossnetinfo-a.akamaihd.net arenda-avtoprokat-krasnodar.ru
appfixing.space
apps-analytics.net
aramaicmedia.org
arcadeplayhouse.com
arendakvartir.kz arendakvartir.kz
asdfz.pro artavenuegardenstudios.com
asophoto.com artefakct.com
auto-zapchasti.org asociatia-tipografilor-transilvania.ro
autobudpostach.club aucoinhomes.com
autolombard-krasnodar.ru audiobangout.com
availit.weebly.com autochoixspinelli.com
aviav.eu aviav.eu
avon-novosib.ru avirasecureserver.com
avon-ukraine.com avkzarabotok.info
avtolombard-krasnodar.ru avon-severozapad.ru
backlinkwatch.com awaybird.ru
balook.com azbaseimages.net
banki76.ru badavit.com.ua
bannerbank.ru bag77.ru
bannerspace.com bamo.xsl.pt
bashtime.ru
battlecarnival.su
beachtoday.ru
becuo.com becuo.com
bensbargains.net beremenyashka.com
berdasovivan.ru berrymall.ru
bestdraws.com bestdraws.com
bestofferswalkmydogouteveryday.gq
bestofpicture.com
bestssaker.com bestssaker.com
betteroffers.review bezcmexa.ru
bigpenisguide.com bitcoinpile.com
billiard-classic.com.ua bjanshee.ru
binaryoptionscops.info bjgugu.net.ua
biznesluxe.ru blackhatworth.com
bjetjt.com bloggen.be
blpmovies.com bmsco.co
bluejays-jerseys.us boanonihaca.tk
bluerobot.info bonzbuddy.com
bluesman.nu bonzybuddy.com
bmwhighperformers.com bookhome.info
boc.kz braintobe.win
boo-arts.com bukleteg.xyz
boom-celebs.com buy-meds24.com
bouda.kvalitne.cz buzzsumo.com
brainsandeggs.blogspot.com bycontext.com
brewdom.ru cakesplus.com.au
brianhenry.co callstevens.com
briomotor.co cand.jp
burger-tycoon.com candlehandmade.com
burn-fat.ga candy-glam-hp.com
buron.pw cartierlovestore.com
businesxxl.com
buy-cheap-online.info
buybest1.biz
buzzzg.men
byme.se
c-english.ru
camsex.xxx-cam.webcam
candiceloves.us
cardsdumps.com
carloans.com
cctva.tv
celejihad.info celejihad.info
cercacamion.it centraletermice.us
certifywebsite.win cfcl.co.uk
cheap-trusted-backlinks.com cheapcigarettesc.info
cheapjerseysa.com cheapjerseysap.com
cheapjerseysfootballshop.com chinacheapelitejerseys.com
chimiver.info christianlouboutinreplica.win
ciceros.co
ciekawinki.pl ciekawinki.pl
clayvasedesigns.tk cinemaflix.website
cleanmypc.com cobaltpro.ru
cleansearch.net codq.info
cocyq.inwtrade.com cognitiveseo.com
compiko.info com-onlinesupport.host
compliance-walt.top commoncrawl.org
companies-catalog.com
compliance-ivan.xyz
compliance-jessica.top
compliance-julianna.top
constantaservice.net constantaservice.net
cookie-law-enforcement-ff.xyz copblock.org
coolbar.pro counciltally.com
copypaste.traderzplanet.in
cornerstone-countertops.com
covetnica.com
cracksplanet.com cracksplanet.com
creams.makeforum.eu crest3d.ru
custodieva.ru cscwtalkto.site
customerpromos-a.akamaihd.net d0t.ru
daretodonate.co damedingel.ya.ru
darkbooks.org daneshetabiat.com
de.zapmeta.com data.vtc.pw
dealitright.click daydream-studio.ru
decorationspcs.com db.speedup.gdn
deshevo-nedorogo.ru dealwifi.com
decoratinghomes.ga
defenderxtactical.com
delta-line.men
deniven.1bb.ru
detective01.ru detective01.ru
dichvuvesinhhanoi.com
dignitasdata.se dignitasdata.se
dimaka.info diusyawiga.tk
dipstar.org djihispano.com
discountliv.com dmmspy.com
discovertreasure-a.akamaihd.net doctormakarova.ru
disruptingdinnerparties.com dogbreedspicture.net
divci-hry.info dostavimvdom.ru
dneprsvet.com.ua doxysexy.com
dolohen.com dvd-famille.com
domaincheck.io dynainbox.com
domik-derevne.ru earncash.com.ua
donvito.unas.cz editmedios.com
downloadeer.net ekspresihati.info
druzhininevgeniy63.blogspot.com eleimgo.pw
dstroy.su elektrozigaretten1.webs.com
dyt.net elektrozigaretten2.yn.lt
eaplay.ru elitepcgames.com
ec-file.info enternet.ee
econom.co estibot.com
egvar.net europages.com.ru
electro-prom.com exchanges-bet.com
elektir.ru exmasters.com
elektrischeziga.livejournal.com expertblog.info
elektrischezigarette1.onsugar.com extrabot.com
elkacentr.ru extremez.net
eluxer.net familienzahnaerzte.com
elvenar.com fashionindeed.ml
enhand.se femmesdenudees.com
epicbrogaming.com filmania-x.ru
eropornosex.ru finejewelryshop.ru
erotag.com fiuxy.com
erotische-geschichten-xxl.com flipper.top
escortslet.net floridamobilebillboards.com
essay-edu.biz footbalive.org
eurosamodelki.ru formulaf1results.blogspot.com
excitacion.info fortwosmartcar.pw
ezigarettekaufen2.mpbloggar.se forum.tvmir.org
f00kclan.de foxinsocks.ru
f012.de franch.info
farm26.ru freakycheats.com
farmprofi.net free-laptop-reward.com
fashion-stickers.ru freelotto.com
fashionavenuegame.com freewhatsappload.com
feel-planet.com front.to
fertilitetsradgivningen.se fun-mobi.pl
filmetricsasia.com
financehint.eu
flashbannernow.com
flashlarevista.com
flowersbazar.com
footballfarrago.com
foto-sisek.porngalleries.top
fr.netlog.com
free-stock-illustration.com
fres-news.com
frivgame250.com
fun2cell.net fun2cell.net
fyxabomiw.ru galeon.com
gaming-journal.com gallery.rennlist.com
gdzkurokam.ru gamedayhouse.com
ge0ip.net gay-file.com
ge0ip.org
gearsadspromo.club
generalporn.org generalporn.org
generousdeal-a.akamaihd.net gepezz.info
get-free-traffic-now.com getaclueamerica.com
girlporn.ru getmyads24.com
getoutofdebtfree.org
girlgamerdaily.com
girlspicsa.com
glall.ru
glass-msk.ru
globalsurfari.com
golden-praga.ru golden-praga.ru
gopixdatabase.com goldpanningtools.com
gosreg.amchs.ru good-mummy.ru
gotcher.us goodwinmetals.co
gourcy.altervista.org gotomontenegro.net
gq-catalog.gq gsasearchengineranker.top
greatfind-a.akamaihd.net gsasearchenginerankerhelp.com
greenzaim.ru hackers-crackers.tk
gwagka.com hamptonoaks.ca
gwebtools.com hannasolution.ru
haikuware.com hausfrauensex18.com
hamacapty.com hayate.biz
handicapvantoday.com hd-filmy.net
hanink.biz.ly hdimagelib.com
hdxnxxtube.mobi heartofpayne.xyz
healing-dysplasia.ru hermesreplica.win
hem.passagen.se hmywwogw.bloger.index.hr
hidefiles.org hobbyhuren-datenbank.com
hit-kino.ru hol.es
hitmuzik.ru homelygarden.com
hotblognetwork.com homemade.gq
howlongdoestizanidinestayinyoursystem.blogspot.com honyaku.yahoofs.jp
hscsscotland.com hoste.octopis.com
hustoon.over-blog.com hotblog.top
hut1.ru hyiphunter.org
idc.com.ua i-hobot.ru
ilbe.club
images-free.net
imfamous.info
iminent.com
imk.com.ua imk.com.ua
inbabes.sexushost.com immobiliaremassaro.com
incolors.club impresagaia.it
innoslicon.com incest-ru.com
intellekt21.ru inet-traffic.com
ipostroika.ru intelhdgraphicsgtdrive6w.metroblog.com
interferencer.ru
iplusbit.blogspot.co.za
iqbazar.ru iqbazar.ru
iqoption.pro irkutsk.online-podarki.com
iradiology.ru
istizanidineanopiate.blogspot.com
ivoiretechnocom.ci
jackpotchances.com
japfm.com
jerseychinabizwholesale.us jerseychinabizwholesale.us
jikoman.info jimmychoosale.online
jrcigars.com
juliaworld.net
justbcause.com justbcause.com
ketoanhanoi.info justkillingti.me
keyword-suggestions.com justucalling32211123456789.tk
kfon.eu kabbalah-red-bracelets.com
kinomaniatv.pl kareliatobacco.ru
klosetkitten.com kazan.xrus.org
kollesa.ru kellyonline.xyz
kotaku.com keywordbasket.com
krasnodar-avtolombards.ru keywordglobal.co.za
krasnodar.zrus.org keywordhouse.com
krassh.ru keywordsking.com
kredit-pod-zalog-krasnodar.ru kinosed.net
kuzinsp.ru komputers-best.ru
kvartir-remont.biz koopilka.com
ladov.ru kosova.de
laexotic.com kretpicf.pw
lafourmiliaire.com kriokomora.info
landoftracking.com languagecode.com
legionalpha.com laptop-4-less.com
law-check-nine.xyz
lenpipet.ru lenpipet.ru
levaquin750mg.blogspot.com letto.by
libertybilisim.com
lightinghomes.net lightinghomes.net
linkbolic.com likesuccess.com
linkdebrideur.xyz linkarena.com
linkpulse.com linkpulse.com
littleberry.ru linkwithin.com
littlesunraiser.com lion.bolegapakistan.com
liumimgx.pw live-cam6.info
lombia.com livefixer.com
longgreen.info lkbennettstore.online
lotto6888.com love-baby.cz
louboutinbooties.xyz lulea-auktionsverk.se
luchshie-topcasino.ru lutherstable.org
luxup.ru m1media.net
mac-shield.com mahnwachen-helfen.info
mafa-free.com map028.com
mafcards.ru mcnamaratech.com
manipulyator-peterburg.ru
maranbrinfo.com.br
marinetraffic.com
marketland.ml
masserect.com
maxxtor.eu
mebelcomplekt.ru
medi-fitt.hu medi-fitt.hu
mex-annushka.ru mediawhirl.net
mexicotravelnet.com medicines-choice.com
micasainvest.com meetingrainstorm.bid
michaelkorsoutletstore.net mexicosleevegastrectomy.com
mirmedinfo.ru milblueprint.com
missclub.info mini-modus.ru
mnogabukaff.net miniature.io
monclercoats.xyz misswell.net
moncleroutletonline.win mixed-wrestling.ru
moscow-region.ru moscow.online-podarki.com
mrlmedia.net mostantikor.ru
mustwineblog.com movies-in-theaters.net
mydirtyhobby.com mrwhite.biz
mylesosibirsk.ru mycaf.it
myliveblog.ru mykings.pw
mysexpics.ru nalogovyykodeks.ru
na-telefon.biz nardulan.com
nalogovyy-kodeks.ru narkologiya-peterburg.ru
narkologiya-orel.ru national-today-winning-winner.club
naughtyconnect.com netvouz.com
notebook-pro.ru newfilmsonline.ru
onescreen.cc newtechspb.ru
onko-24.com ngps1.ru
online7777.com niki-mlt.ru
onthemarch.co nomuos.it
oralsexfilme.net nonews.co
orsonet.ru nsatc.net
pammik.ru nudepatch.net
parajumpersjakkesalgnorge.info o00.in
parlament.biz obiavo.uz
partnerline.men obsessionphrases.com
pcimforum.com okuos.com
one-gear.com
oneclickfiles.com
oneshotdate.com
onlineku.com
onlineslotmaschine.com
oops-cinema.ru
outrageousdeal-a.akamaihd.net
paperwritingservice17.blogspot.ru
parajumpersstore.online
partner-print.men
partybunny.ru
patterntrader-en.com
pepperstyle.ru pepperstyle.ru
petedrummond.com percin.biz.ly
photorepair.ru perl.dp.ua
phuketscreen.com perm-profnastil.ru
pic2fly.com perso.wanadoo.es
piccdata.com petrovka-online.com
picsfair.com photo-clip.ru
picturesfrom.com picphotos.net
pigrafix.at pinkduck.ga
plastgranar.nu pivka.xyz
platinumdeals.gr pizdeishn.com
playmsn.com pl-top.pl
plugingeorgia.com platesauto.com
pobeiranie.pl playfortuna-play.ru
podshipniki-ntn.ru pliks.pl
poem-paying.gq pogosh.com
polcin.de pons-presse.com
popunder.ru porndairy.in
pornobrazzers.biz pornhubforum.tk
pornohd1080.online pornoforadult.com
pornoklad.net pornoklad.ru
portadd.men pornophoto.xyz
pozvonim.com pornstartits.xblog.in
poshiv-chehol.ru
positive2b.ru
postclass.com
powc.r.ca.d.sendibm2.com
pozdravleniya-c.ru
pozdrawleniya.com
pregnant.guru pregnant.guru
preparevideosafesystem4unow.site printdirectforless.com
presleycollectibles.com promgirldresses.xyz
printie.com propranolol40mg.blogspot.com
privatbank46.ru prostitutki-belgoroda.org
prizefestival.mobi prostitutki-tyumeni.org
prizestohandle.club prostitutki-yaroslavlya.org
pro-tec.kz pskcijdc.bloger.index.hr
proekt-gaz.ru
promoheads.com
prostitutki-astana.org
proxyradar.com
pst2017.onlinewebshop.net
psvita.ru
pussyspace.net pussyspace.net
qcstrtvt.bloger.index.hr q-moto.ru
qexyfu.bugs3.com quickloanbank.com
qitt.ru randki-sex.com
quizzitch.net ranksignals.com
rangjued.com rapidsites.pro
redirect.trafficreceiver.club razorweb-a.akamaihd.net
rednise.com regdefense.com
remmling.de registratciya-v-moskve.ru
remont-comp-pomosh.ru remedyotc.com
renecaovillasale.online remont-mobile-phones.ru
rentehno.ru remont-ustanovka-tehniki.ru
respublica-otel.ru rezeptiblud.ru
reward-survey.net rockma.se
ringtonepartner.com rome2rio.com
riralmolamsaca.tk rustic-quiver.win
roll123.com saecsa.co
rossanasaavedra.net sanidumps.com
royal-betting.net sanjosestartups.com
royalads.net sbwealthsolutions.ca
rus-pornuha.com scalerite.co.za
rutor.vip scanner-don.top
ryetaw.com scanner-george.top
sadaholding.com scanner-viktor.top
sbetodiodnye-lampy.ru scansafe.net
sbtdesign.co.uk scottbywater.com
scanner-alexa.top scripted.com
scanner-john.top scrnet.biz.ua
scanner-willy.top search.pch.com
search-error.com searchengineranker.email
seasaltwithfood.com searchmywindow-a.akamaihd.net
seokicks.de securesmrt-dt.com
sergiorossistore.online seksotur.ru
seoheap.com
seonetwizard.com
servicecenter.co.ua
sex-tracker.de sex-tracker.de
sexiporno.net sexobzor.info
sherlock.se shoesonlinebuy.cn
shiksabd.com
shopwme.ru shopwme.ru
sideeffectsoftizanidine.blogspot.com shymkent.xkaz.org
sierraapps.com signx.info
simple-image.com.ua siha.de
simple-share-buttons.com sk.golden-praga.ru
sisiynas.ru snsdeainavi.info
site.ru soc-econom-problems.ru
sitevalued.com
sixcooler.de
sladkoevideo.com
slim.sellany.ru
smartpet.ru
soc-proof.su
social-button.xyz social-button.xyz
social-buttons.xyz soietvousmaime.fr
social-s-ggg.xyz soundfrost.org
social-s-iii.xyz
softlinesolutions.me
softonicads.com
softtor.com
sophang8.com
spaceshipad.com
specialfinanceoffers.com specialfinanceoffers.com
sports-supplements.us speedup-my.site
sprttrack.com spinnerco.ca
sribno.net sport7777.net
start.myplaycity.com spravka-medosmotr.ru
startufa.ru spyfu.com
staticfs.host stard.shop
stretchingabuckblog.com stats-public.grammarly.io
stretchmate.net stauga.altervista.org
studentguide.ru steamoff.net
superoboi.com.ua store-rx.com
surflinksmedical.com stroiminsk.org
swinger-mobil.net strongholdsb.ru
taqplayer.info strongsignal-a.akamaihd.net
taxi-v-eisk.ru success-seo.com
telemetryverification.net sucsesofinspiration.com
teslathemes.com svbur.ru
testbotprocessor44.com tandvardshuset.net
thebestphotos.eu teencastingporn.com
theguardlan.com tengohydar.tk
textads.men
thedownloadfreeonlinegames.blogspot.com
thegolfclub.info
thepokertimer.com thepokertimer.com
thepornsex.org
thetoiletpaper.com
thexart.club
timetorelax.biz
titangel-vietnam.com
tizanidine4mgstreetprice.blogspot.com tizanidine4mgstreetprice.blogspot.com
tizanidine6mg.blogspot.com tizanidineingredients.blogspot.com
tizanidineandcipro.blogspot.com tizanidinerecreationaluse.blogspot.com
toloka.hurtom.com tizanidinevscyclobenzaprine.blogspot.com
tooplay.com tn811.us
top10-way.com topanasex.com
torrent-to-magnet.com tozup.com
totu.us trafaret74.ru
tovaroboom.vast.ru trafficreceiver.club
track-rankings.online try-rx.com
triumf-realty.ru tube8.com
tula.mdverey.ru turizm.bz
turbo-suslik.org turvgori.ru
turist-strani.ru tv-spoty.info
turkeyreport.tk tzritel.tk
twelvevisionspartyofcolorado.com uamtrk.com
u-cheats.ru uchebavchehii.ru
ufa.xrus.org uchil.net
uhod-za-sobakoj.ru uk-zheu20.ru
uptime-as.net ukrobstep.com
url-img.link uogonline.com
utrolive.ru uprour.com
vadimkravtcov.ru upstore.me
valuado.com usacasino.com
vashsvet.com usadacha.net
vduplo.ru vapsy.com
vegascosmetics.ru veles.shop
viandpet.com viagra-soft.ru
videochat.ph video-hollywood.ru
videochat.world videojam.tv
vkak.ru videospornogratisx.net
vladimir.zrus.org vip-file.com
vozbujdenie.com vip2ch.com
vpnmouse.com virus-schutzmasken.de
vseigru.one visitcambridge.org
vsexkontakte.net voloomoney.com
vtcdns.com vonradio.com
vzubah.com voprosotvet24.ru
walkme.com wallpapersist.com
wdrake.com walpaperlist.com
websitevaluebot.com wareseeker.com
weburlopener.com webix.me
weightatraining.com weblibrary.win
westermarkanjou.se webradiology.ru
whatistizanidine2mg.blogspot.com webs.com
wholesalejerseysgaa.com website-audit.com.ua
win-spy.com wejdz-tu.pl
winterclassichockeyjerseys.com welcomeauto.ru
wpsecurity.website wfb.hatedriveapart.com
www.pinnacle-bets.com whiteproduct.com
wieseversa.no
wma-x.com
woman-tampon.ru
woodyguthrie.se
wordpresscore.com
worldmusicfests.com
www.ehscloud.cn
www.solartek.ru www.solartek.ru
wygraj-skiny.win x-diesel.biz
x-diesel.org
x-mix.info
x-rates.ru x-rates.ru
xblognetwork.com xgftnlrt.bloger.index.hr
xfluro.com xmnb.net
xmladserver.com xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
xn----7sbho2agebbhlivy.xn--p1ai xn----7sbahjd3btneuw1joc.xn--p1ai
xn----7sbifcamovvfggw9d.xn--p1ai xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn----ctbbcjd3dbsehgi.xn--p1ai
xn--80ahdheogk5l.xn--p1ai xn--80ahdheogk5l.xn--p1ai
xn--b1adccaf1bzj.xn--p1ai xn--d1abj0abs9d.in.ua
xsion.net yachts-cruise.info
xxxrus.org yebocasino.co.za
yellowproxy.net ynymnwbm.bloger.index.hr
yopoint.in yougotanewdomain.com
yunque.pluto.ro youtoner.it
zaobao.com.sg yuarra.pluto.ro
zastroyka.org yurgorod.ru
zoomovies.org zaapplesales.blogspot.com
zscalertwo.net zastenchivosti.net
zverokruh-shop.cz zdesformula.ru
zero1.it
zeroredirect.com
zeroredirect10.com
zeroredirect7.com
zophim.me
zreloeporno.tv
zx6.ru

474
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
101billion.com 123movies.love
2000k.ru 1j7740kd.website
24videos.tv 404.mn
4senses.co 7search.com
aadroid.net 8jn.dating-app.ru
aaronabel.com
abcguru.xyz abcguru.xyz
abctoppictures.net absolute-s.ru
above.com academiacsmendoza.org
adclickservice.com acc.eu.org
add-add.men actices.com
advertisingtag.net adf.ly
aero2.ru admitad.com
affiliate-fr.com adtology.com
agecheckadult.com akuhni.by
ahrefs.com albumsuper.info
alert-hgd.xyz
alevinefesleri.com alevinefesleri.com
aliexpresscashback.club alfa-img.com
all-streaming-media.com alltheviews.com
alldubai.biz alyeskaresort.com
allornamenti.com angel1777.kz
apimountainbikei-a.akamaihd.net apccargo.com
apple.com-webbrowsing-security.science appfastplay.com
aramaicmedia.org apple.com-cleaner.systems
arendakvartir.kz arewater.com
asdfg.pro arquivo.pt
asociatia-tipografilor-transilvania.ro asociatia-tipografilor-transilvania.ro
ausergrubhof.info autovideobroadcast.com
ausmepa.org.au ayodhya.co
auto.rusvile.lt azarttoto.com
autoua.top b00kmarks.com
avirasecureserver.com bankofthewext.com
azartmix.com bestcurs.org
bad-stars.net bestwebsitesawards.com
bannerswap.com betterhdporn.com
barbourjackets.online betteroffers.review
battlecarnival.su beyan.host.sk
battleforupdating.site big-boards.info
beauty-b0x.pl bioinnovate.co
benchmarkcommunications.co.uk bitcoinpile.com
bestmarriages.com
beta.hotkeys.com
bget.ru
bitcoin-ua.top
blog-hits.com blog-hits.com
blog.koorg.ru blog.remote-computer.de
bosefux.esy.es brainsandeggs.blogspot.com
budilneg.xyz buyclomidonlaine.com
buy-meds24.com buyscabiescream.com
buyessaynow.biz bvlgariring.xyz
buypanicdisorderpill.com
buypillsorderonline.com
buzzzg.men
cablecar.us
cactussoft.biz cactussoft.biz
carmuffler.net call-of-duty.info
charmstroy.info callcafe.info
cheapsergiorossi.online carabela.com.do
co.lumb.co carapics.com
compliance-alexa.xyz cdn.walkme.com
compliance-don.xyz cdnanalytics.xyz
compliance-olga.top cdncash.org
cookie-law-enforcement-ee.xyz cfsrating.sonicwall.com
coolwallpapers4k.info cherubinimobili.it
cubook.supernew.org chocolateslim-en-espana.com
cyclobenzaprinevstizanidine.blogspot.com clickhype.com
d2jsp.org coderstate.com
comissionka.net
compliance-checker.info
connectionstrenth.com
cookie-law-enforcement-ff.xyz
cosmeticswomens-womensports.rhcloud.com
cpajunkies.com
darkages.info
dating-app.ru
ddpills.com ddpills.com
deadlinkchecker.com dignitasdata.se
decoratinghomes.ga directrev.com
deda-moroza-zakaz.ru diy-handmade-ideas.com
dikx.gdn dktr.ru
dildofotzen.net doctormakarova.ru
doctissimo.fr dojki-hd.com
documentbase.net drugs-no-rx.info
domain-tracker.com dyshagi.ru
domain.webkeyit.com earn-from-articles.com
domaineaneblanc.com ee77ee.com
dorratex.tn erotik0049.com
dptaughtme.com ertelecom.ru
duawitchrarato.tk eupornstar.info
elektir.ru everypony.ru
elvenar.com exoclick.com
evehemming.blogspot.com.au f-loaded.de
exct.net fakehandbags.xyz
eyemagination.com family1st.ca
ferretsoft.com farmprofi.net
financepoints.eu fashion-stickers.ru
find1friend.com fatfasts-4tmz.com
finejewelryshop.ru fearcrow.com
free-laptop-reward.com financeloan.us
fsalas.com flashlarevista.com
geile-lelly.eu forum20.smailik.org
free-share-buttons.blogspot.com
free-social-buttons.xyz
gallerily.com
geileweiber.tk geileweiber.tk
generic-pills-online.com gepezz.info
germes-trans.com germetiki.com.ua
girlgamerdaily.com glavprofit.ru
go2mike.ru gorabagrata.ru
gq-catalog.gq gorodservis.ru
guge.io gsasearchengineranker.top
guardlink.com
guitar-master.org guitar-master.org
gwagka.com gwhwpxbw.bloger.index.hr
hdimagelib.com h2monline.com
heartofbeijing.blogspot.com handsandlegs.ru
highland-homes.com hd-film.pl
hkladys.com heliko.no
hollywoodactress.info hem.passagen.se
housediz.com hesteel.pl
housekuba.org hildinghr.se
hvd-store.com hplaserjetpdriver8y.pen.io
hyhj.info hyipmanager.in
i4track.net imk.com.ua
images-graphics-pics.com infobanks.ru
iqoption-bin.com informatiecentro.be
inzn.ru
iplogger.org
istizanidineanopiate.blogspot.com istizanidineanopiate.blogspot.com
itis4you.com itch.io
iwanttodeliver.com izhevsk.xrus.org
johnnyhaley.top janerikholst.se
kabbalah-red-bracelets.com jasonpartington.com
keywords-monitoring-success.com jaxcube.info
kihi.gdn jerseyssportsshop.com
kiwe-analytics.com jetsli.de
kliksaya.com karpun-iris.ru
kochanelli.com karting196.ru
kovrenok.ru keywordblocks.com
kredit-pod-zalog-krasnodar.ru kongruan.com
laxdrills.com ladsup.com
letsrepair.in lashstudia.ru
linkpulse.com ldrtrack.com
liupis.com life.biz.ua
locimge.pw linksharingt.com
lombardfinder.ru listiseltemournlan.gq
louboutinshoes.xyz littleberry.ru
love-baby.cz loginduepunti.it
luckyshop.net.ua lonely-mature.com
magicdiet.gq luchshie-topcasino.ru
marinetraffic.com lunamedia.co
me-ke.com luxmagazine.cf
mebelcomplekt.ru madot.onlinewebshop.net
medizinreisen.de makis.nu
mere.host.sk marcowebonyodziezowe.pl
methodsmarketing.com marmotstore.online
minegam.com miniads.ca
mojpregled.com mnimmigrantrights.net
monarchfind-a.akamaihd.net monclerboots.xyz
moonci.ru monclercheap.xyz
moscow-region.ru movies-in-theaters.net
myliveblog.ru mp3ritm.top
myshopmatemac.com mrbojikobi4.biz
naturalbreakthroughsresearch.com multgo.ru
nodding-passion.tk mydoctorok.ru
novodigs.com nacap.ru
nudepatch.net nailsimg.com
oneminutesite.it narkologiya-belgorod.ru
onko-24.com newhairstylesformen2014.com
onlineku.com nfljerseyscheapchinabiz.com
oynat.info no-rx.info
paparazzistudios.com.au nvformula.ru
nw-servis.ru
okroshki.ru
onlinedomains.ru
openmultipleurl.com
oranga.host.sk
pagesense.com
paperwritingservice17.blogspot.ru
parajumpersstore.online parajumpersstore.online
partner-cdn.men park.above.com
pc-services.ru pelfind.me
pdamods.ru picquery.com
perfection-pleasure.ru picturesify.com
petitions.whitehouse.gov picturesmania.com
phpdevops.com platinumdeals.gr
polybuild.ru poem-paying.gq
popads.net poffet.net
prizesbook.online pokerniydom.ru
prlog.ru popugauka.ru
pro-poly.ru pornokorol.com
pro-tec.kz pornophoto.xyz
prostitutki-odessa.org portadd.men
purchasepillsnorx.com princeadvantagesales.com
ranking2017.ga prodess.ru
razorweb-a.akamaihd.net quick-offer.com
remont-mobile-phones.ru registrationdomainsite.com
replica-watch.ru reining.lovasszovetseg.hu
replicaclub.ru remont-fridge-tv.ru
reversing.cc remont-komputerov-notebook.ru
ric.info ros-ctm.ru
royalcar-ufa.ru royalads.net
scanner-barak.top sandhillsonline.com
scanner-walt.top sarm.tk
seeingmeerkat.com seccioncontrabajo.com
seemoreresultshu-a.akamaihd.net sexstream.pl
sell-fb-group-here.com slayerlife.com
semalt.com souvenir.cc
seoheap.com space2019.top
sex-znakomstva.online space4update.pw
sexxdate.net sribno.net
shoesonlinebuy.cn stroydetali.ru
shtora66.ru swinger-mobil.net
slavic-magic.ru swingerseiten.com
soblaznu.net tabletkinaodchudzanie.com.pl
splendorsearch-a.akamaihd.net tagil.zrus.org
staynplay.net titan-ads.life
strana-solnca.ru tizanidinehcl4mgisitanarcotic.blogspot.com
strigkaomsk.ru
strv.se
superinterstitial.com
superlist.biz
swiped.su
takeflyte.com
tam-gde-more.ru
taqywu51.soup.io
theheroes.ru
thejournal.ru
themeforest.net
titelhelden.eu
tizanidineinteractions.blogspot.com
tizanidineoral.blogspot.com tizanidineoral.blogspot.com
tizanidinepain.blogspot.com tizanidinevssoma.blogspot.com
torrents-tracker.com trafaret74.ru
torrents.cd traffic-club.info
toursmaps.com traffic2money.com
traffmonster.info trafficnetzwerk.de
trailer.cinemaflix.website treasuretrack-a.akamaihd.net
ttsq.fr
tuberkuleznik.ru tuberkuleznik.ru
txxx.com tuckermktg.com
ultimatesetnewfreeallsoftupgradesystems.pw turn-up-life.life
unitexindia.com u555u.info
univerfiles.com undergroundcityphoto.com
vandrie-ict.nl uptimechecker.com
videochat.ph urccvfmc.bloger.index.hr
videogamesecrets.com valmetrundan.se
videospornogratisx.net via-energy-acquistare.com
vip2ch.com via-energy-order.com
videos-for-your-business.com
viel.su
vipps.com.my
virus-schutzmasken.de virus-schutzmasken.de
viteonlusarezzo.it vltai.com
vladimir.xrus.org voronezh.xrus.org
voluumtracker1.com warningwar.ru
vut.com.ru website-datenbank.de
wcb.su wemarketing.se
webshoppermac.com wheelchairliftsarea.com
website-speed-checker.site wildnatureimages.com
weightatraining.com win-spy.com
whiteproduct.com winterclassichockeyjerseys.com
wildcattube.com x-diesel.biz
woman-h.ru xn----7sbho2agebbhlivy.xn--p1ai
www.bookmaker-bets.com xn----ctbbcjd3dbsehgi.xn--p1ai
xclicks.net xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai xn--d1abj0abs9d.in.ua
xrus.org xsion.net
xuki.us xtubeporno.net
xvideosporn.biz yebocasino.co.za
yaminecraft.ru zawyna.ua
zetmaster.ru zigarettenonlinekaufen2.bloog.pl
zoolubimets.ru
zzbroya.com.ua

363
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -8,6 +8,184 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -316,181 +494,6 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -581,14 +584,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.1.76 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2718 ### Version: V4.2021.11.2719
### Updated: Tue Nov 16 10:03:30 UTC 2021 ### Updated: Tue Nov 16 22:01:14 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2718 ### Version: V4.2021.11.2719
### Updated: Tue Nov 16 10:03:30 UTC 2021 ### Updated: Tue Nov 16 22:01:14 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##