vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 11:10:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.08.2548 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-08-25 10:09:20 +00:00
parent 1f4754cda7
commit bb4652acb2
47 changed files with 65742 additions and 65732 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2547 #### Version: V4.2021.08.2548
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2547 #### Version: V4.2021.08.2548
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.08.2547 #### Version: V4.2021.08.2548
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8082
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.08.2547 #### Version: V4.2021.08.2548
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8086
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8086
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8086
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8086
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8086
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8086
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8086
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2547 2548

8086
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8086
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.94.206.155 0; 52.158.232.191 0;

8086
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8086
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8086
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8086
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.94.206.155 0; 52.158.232.191 0;

8086
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.94.206.155 0; 52.158.232.191 0;

8086
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Bandit Barkrowler
CSHttp Bolt
Gotit CherryPicker
IDBTE4M DomainStatsBot
Image Sucker HaosouSpider
MeanPath Bot Mozlila
NetZIP Nimbostratus
Pandalytics Re-re
PeoplePal RealDownload
Snake ripz

288
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 360Spider
AIBOT
Abonti Abonti
Aboundex Aboundex
Aboundexbot
AfD-Verbotsverfahren
AiHitBot AiHitBot
Aipbot
Alexibot
AllSubmitter
Alligator Alligator
AlphaBot AlphaBot
Anarchy
Ankit
Anthill Anthill
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BDFetch BackDoorBot
BackStreet
Backlink-Ceck Backlink-Ceck
Badass BacklinkCrawler
Bandit
Barkrowler
Bigfoot Bigfoot
BlackWidow Bitacle
Blackboard Blackboard
Blow
Boardreader
Bolt Bolt
Brandprotect BuiltBotTough
Buddy
BunnySlippers
BuzzSumo BuzzSumo
CazoodleBot CSHttp
Calculon
Cegbfeieh Cegbfeieh
CheeseBot CensysInspect
ChinaClaw CheTeam
Cloud mapping Claritybot
Cosmos Collector
Copier
Craftbot Craftbot
Crawling at Home Project
Curious Curious
Custo Custo
DBLBot DTS Agent
Deusu DatabaseDriverMysqli
Digincore Devil
Disco DigitalPebble
Discobot
Discoverybot
Dispatch
DittoSpyder DittoSpyder
DnyzBot DnyzBot
DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Dotbot Dragonfly
Drip
ECCP/1.0 ECCP/1.0
EasyDL EMail Wolf
Ebingbong Ebingbong
Ecxi Ecxi
EirGrabber EirGrabber
Express WebPictures
ExtLinksBot ExtLinksBot
ExtractorPro
EyeNetIE EyeNetIE
FDM
FHscan FHscan
Fimap Fimap
Flunky FyberSpider
Foobot
FrontPage
G-i-g-a-b-o-t
GT::WWW
GermCrawler GermCrawler
Gigablast Go-Ahead-Got-It
Go!Zilla
GoZilla GoZilla
Gotit Gotit
Grafula Grafula
GrapeFX
GrapeshotCrawler GrapeshotCrawler
GridBot
HMView HMView
Haansoft HTMLparser
HaosouSpider HaosouSpider
Harvest Harvest
Hloader Hloader
Humanlinks IDBot
HybridBot IRLbot
Iblog Iblog
Id-search
IlseBot IlseBot
Image Fetch InfoTekies
IndeedBot Intelliseek
Internet Ninja Internet Ninja
Iskanie
IstellaBot
Jbrofuzz
JennyBot JennyBot
Jetty JetCar
Jorgee Jorgee
JustView
Jyxobot Jyxobot
Kozmosbot Kenjin Spider
LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
Leap
LeechFTP LeechFTP
LexiBot Lftp
Libwhisker
Likse
LinkScan
Linkdexbot
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaRSSBot
Lipperhey Lipperhey Spider
Litemage_walker MQQBrowser
Ltx71 MTRobot
MFC_Tear_Sample Majestic SEO
Mag-Net
Magnet
Mail.RU_Bot
MarkMonitor
MarkWatch MarkWatch
Mass Downloader
Masscan Masscan
MeanPath Bot Mb2345Browser
MicroMessenger Meanpathbot
Mediatoolkitbot
MegaIndex.ru
Microsoft Data Access
Microsoft URL Control
Moblie Safari Moblie Safari
Morfeus Fucking Scanner Mozlila
Msrabot Mr.4x3
NICErsPRO NICErsPRO
NPbot
Name Intelligence
Needle
Nessus
Net Vampire Net Vampire
NetAnts
NetLyzer
NetSpider
Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler Niki-bot
Nikto Nikto
Ninja Octopus
Nmap
Offline Explorer Offline Explorer
Offline Navigator OnCrawl
OpenVAS
Openfind Openfind
Openvas Openvas
PHPCrawl OrangeBot
Page Analyzer OutclicksBot
POE-Component-Client-HTTP
PageAnalyzer
PageGrabber PageGrabber
Pavuk PageScorer
Picscout Panscient
Pi-Monster
Picsearch
Piepmatz Piepmatz
PleaseCrawl Pixray
Pockey
ProPowerBot
ProWebWalker
Probethenet Probethenet
Psbot
PxBroker PxBroker
RankActive RankFlex
RankingBot
RankingBot2 RankingBot2
Rankivabot RankurBot
Re-re
ReGet ReGet
RealDownload
Recorder Recorder
RepoMonkey RepoMonkey
Ripper Ripper
SEOstats RocketCrawler
SISTRIX SEOkicks-Robot
SEOlyticsCrawler
SMTBot SMTBot
SalesIntelligent Scanbot
Scrapy Scrapy
ScrepyBot ScreenerBot
Searchestate
SearchmetricsBot SearchmetricsBot
Semrush
SentiBot SentiBot
SiteCheckerBotCrawler SeoSiteCheckup
SiteSnagger Shodan
Sitebeam Site Sucker
SlySearch
SmartDownload SmartDownload
Sociscraper Snoopy
Sosospider Spammen
SpaceBison Spinn3r
Sqlworm
Steeler
Stripper Stripper
Sucker Sucker
Sucuri SuperBot
SuperHTTP SuperHTTP
Surfbot
SurveyBot
Suzuran Suzuran
Szukacz Swiftbot
T0PHackTeam T8Abot
Teleport
TeleportPro
Telesoft Telesoft
Telesphoreo Telesphorep
The Intraformant The Intraformant
TheNomad TheNomad
Titan Thumbor
Trendiction TightTwatBot
Tracemyfile
Trendictionbot
True_Robot
Turingos Turingos
TurnitinBot TurnitinBot
TwengaBot TwengaBot
Typhoeus Twice
URLy.Warning URLy Warning
UnisterBot Upflow
V-BOT
VB Project
VCI VCI
Vagabondo Vagabondo
WEBDAV VelenPublicWebCrawler
WISENutbot VeriCiteCrawler
VidibleScraper
WWW-Collector-E
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0
Web Auto Web Auto
Web Collage Web Sucker
Web Sauger
WebAuto WebAuto
WebCollage WebCollage
WebCopier
WebEnhancer WebEnhancer
WebFetch WebFetch
WebGo IS WebPix
WebImageCollector WebReaper
WebLeacher WebStripper
WebSauger WebSucker
WebWhacker WebWhacker
WebZIP
Webalta Webalta
Webshag WebsiteExtractor
Website Quester
WebsiteQuester
Webster
Whack
Whacker Whacker
Whatweb Widow
WiseGuys Robot WiseGuys Robot
Wprecon Wotbox
Xaldon_WebSpider
Xenu
Zauba
Zermelo Zermelo
ZumBot Zeus
ZyBorg backlink-check
arquivo-web-crawler cah.io.community
arquivo.pt clark-crawler
check1.exe
com.plumanalytics
crawl.sogou.com crawl.sogou.com
demandbase-bot
domainsproject.org domainsproject.org
gopher eCatch
instabid internetVista monitor
isitwp.com ips-agent
magpie-crawler lwp-request
meanpathbot lwp-trivial
mediawords netEstate NE Crawler
page scorer
pcBrowser
plumanalytics plumanalytics
ripz
s1z.ru s1z.ru
scan.lol satoristudio.net
seobility seobility
sitechecker.pro sitechecker.pro
sogouspider
sp_auditbot sp_auditbot
ubermetrics-technologies.com sysscan
trendiction.com
trendiction.de
voyagerx.com voyagerx.com
xpymep1.exe webprosbot
zauba.io zauba.io
zgrab

294
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
80legs 01h4x.com
404enemy
AIBOT AIBOT
ALittle Client
Abonti Abonti
Aboundex Aboundex
AfD-Verbotsverfahren
AiHitBot AiHitBot
Aipbot
Alexibot Alexibot
AllSubmitter AllSubmitter
AlphaBot Anarchy
Anarchie Anthill
Ankit Asterias
Apexoo
AwarioRssBot AwarioRssBot
BBBike BBBike
BDCbot
BLEXBot
BackDoorBot BackDoorBot
Backlink-Ceck BackStreet
BacklinkCrawler BacklinkCrawler
Barkrowler
Bigfoot Bigfoot
Black Hole
BlackWidow BlackWidow
Boardreader Brandprotect
Bolt Buck
BotALot BuiltBotTough
Brandwatch
Buddy
BunnySlippers
BuzzSumo BuzzSumo
CATExplorador CSHttp
CCBot CazoodleBot
Calculon
Cegbfeieh Cegbfeieh
CheeseBot CensysInspect
CheTeam
CherryPicker CherryPicker
ChinaClaw ChinaClaw
Chlooe Cloud mapping
Cliqzbot Collector
Cocolyzebot Copier
Cosmos
Crawling at Home Project
CrunchBot CrunchBot
Custo Curious
CyotekWebCopy CyotekWebCopy
DBLBot DBLBot
DIIbot DIIbot
DSearch DSearch
Deusu DatabaseDriverMysqli
Devil Digincore
DigitalPebble Discobot
Disco Discoverybot
Dispatch Dispatch
DittoSpyder DittoSpyder
DomCopBot DomCopBot
DomainStatsBot DomainSigmaCrawler
Dotbot Domains Project
ECCP/1.0 Dragonfly
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
EasyDL EasyDL
Ebingbong
Ecxi Ecxi
EirGrabber EroCrawler
Evil
Exabot
ExtLinksBot ExtLinksBot
Extreme Picture Finder ExtractorPro
EyeNetIE EyeNetIE
Ezooms Ezooms
Fimap FemtosearchBot
Firefox/7.0
FlashGet FlashGet
FrontPage
Fuzz Fuzz
GetWeb GalaxyBot
GetRight
Getintent Getintent
Gigabot Gigabot
Go!Zilla
Gotit Gotit
GrabNet Grafula
Grabber GrapeshotCrawler
HaosouSpider GridBot
HEADMasterSEO
HMView
HTTP::Lite
Harvest
Havij Havij
Heritrix Hloader
Humanlinks Humanlinks
IDBot IDBTE4M
IRLbot
Iblog Iblog
Id-search
Image Fetch Image Fetch
Image Sucker
IndeedBot IndeedBot
Indy Library InfoTekies
InfoNaviRobot
InterGET InterGET
Internet Ninja
InternetSeer InternetSeer
Iria Iria
JOC Web Spider JetCar
JamesBOT JikeSpider
Jbrofuzz
Joomla Joomla
JustView JustView
Jyxobot Keyword Density
Kinza Larbin
LWP::Simple
Leap Leap
LeechFTP
LexiBot
LibWeb
Libwhisker Libwhisker
Lightspeedsystems
Likse Likse
Linkbot LinkWalker
Linkdexbot Linkdexbot
LinqiaRSSBot LinkextractorPro
LinqiaScrapeBot LinqiaMetadataDownloaderBot
Litemage_walker Lipperhey
Lmspider Lipperhey Spider
MIDown tool Ltx71
MSFrontPage MSIECrawler
Mail.RU_Bot Majestic12
Majestic SEO
Majestic-SEO
MarkMonitor
Masscan Masscan
Meanpathbot Mata Hari
Mediatoolkitbot Mb2345Browser
MeanPath Bot
Metauri Metauri
Microsoft URL Control
Mister PiX Mister PiX
Mojolicious Mojolicious
Morfeus Fucking Scanner
Mozlila Mozlila
Nameprotect Name Intelligence
Navroad
NearSite
Needle
Nessus Nessus
Net Vampire Net Vampire
NetLyzer NetSpider
NetMechanic NetZIP
NextGenSearchBot Nettrack
Netvibes
Nibbler Nibbler
NimbleCrawler Niki-bot
Nikto
Nimbostratus
Nuclei Nuclei
Nutch
Octopus Octopus
Offline Explorer OnCrawl
Offline Navigator Openvas
OpenLinkProfiler OutclicksBot
OutfoxBot OutfoxBot
PageGrabber PECL::HTTP
PageScorer PHPCrawl
Pandalytics Page Analyzer
Petalbot Panscient
Papa Foto
Pi-Monster
Picscout Picscout
Piepmatz Piepmatz
Pimonster Pimonster
PleaseCrawl PleaseCrawl
Pockey
ProPowerBot ProPowerBot
Probethenet Psbot
Pump Pu_iN
PxBroker PxBroker
PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler RankActive
RankActiveLinkBot RankActiveLinkBot
Rankivabot RankingBot
RankurBot RankingBot2
Re-re Re-re
RealDownload
Recorder
RedesScrapy
RepoMonkey RepoMonkey
RocketCrawler Ripper
Rogerbot SBIder
SEOprofiler SEOkicks-Robot
ScanAlert SISTRIX
ScoutJet ScoutJet
Screaming
ScreenerBot ScreenerBot
Searchestate SearchmetricsBot
Semrush SemrushBot
SeoSiteCheckup
Shodan Shodan
Siphon Siteimprove
SiteSnagger
SiteSucker
Sitevigil
SlySearch
Snake
SocialRankIOBot
Sociscraper Sociscraper
SpaceBison Sogou web spider
Sosospider
Spbot Spbot
Sqlmap Spinn3r
Sqlworm Sucuri
Stripper SuperBot
Sucker Swiftbot
SurveyBot T0PHackTeam
Suzuran
Szukacz
T8Abot T8Abot
Telesoft Teleport
Thumbor Titan
Toata
Toweyabot Toweyabot
Tracemyfile
True_Robot True_Robot
Turingos
Turnitin Turnitin
TwengaBot TwengaBot
URLy Warning URLy.Warning
V-BOT Upflow
Vacuum VeriCiteCrawler
VidibleScraper
Voltron Voltron
WISENutbot WISENutbot
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WeSEE
Web Collage
Web Enhancer
Web Fetch Web Fetch
Web Sucker Web Fuck
WebCopier Web Pix
Web Sauger
WebBandit
WebFuck WebFuck
WebGo IS WebImageCollector
WebLeacher Website Quester
WebPix
Webalta
WebmasterWorldForumBot
Webshag
Webster Webster
Whack
Whatweb Whatweb
Widow WiseGuys Robot
Woobot
Xaldon WebSpider
Xenu
YoudaoBot YoudaoBot
Zeus Zade
Zitebot Zitebot
ZoomBot ZoomBot
ZoominfoBot
ZumBot ZumBot
arquivo.pt archive.org_bot
backlink-check arquivo-web-crawler
cah.io.community
check1.exe check1.exe
crawl.sogou.com clark-crawler
eCatch coccocbot-web
cognitiveseo
crawler4j
demandbase-bot
facebookscraper
heritrix
instabid instabid
internetVista monitor lwp-trivial
lwp-request
meanpathbot
mediawords mediawords
netEstate NE Crawler page scorer
polaris version
probe-image-size probe-image-size
satoristudio.net
scalaj-http scalaj-http
scan.lol scan.lol
sexsearcher seoscanners
seostar
serpstatbot
siteripz siteripz
sogouspider sogouspider
sysscan sysscan
tAkeOut ubermetrics-technologies.com
trendiction.de webprosbot
webmeup-crawler x22Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
12u.info 83net.jp
domination.ml enskedesquashclub.se
gribokstop.com freelifetimefuckbook.com
medictube.ru homemature.net
nfljerseyscheapchinabiz.com iqoption-bin.com
siteexpress.co.il lion.conboy.us
tako3.com netvouz.com
vavilone.com onlyforemont.ru
venta-prom.ru staff.prairiesouth.ca
womensplay.net zigarettenonlinekaufen2.drupalgardens.com

934
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
01apple.com 00author.com
10bet.com 00go.com
1adult.com 03e.info
1bet.com 0lovespells0.blogspot.com
1kinobig.ru 0n-line.tv
1-free-share-buttons.com
13tabs.com
300richmond.co.nz 300richmond.co.nz
4inn.ru 45en.ru
4senses.co 4ur.click
4webmasters.com
5-steps-to-start-business.com
5000-cotydzien.com
5kstore.com 5kstore.com
7yue.org 777-club.ru
8jn.dating-app.ru 999webdesign.com
8kisses.eu 9icmzvn6.website
aac52006ec82a24e08b665f4db2b5013f7.com a96527gi.beget.tech
aadroid.net aaa38852e886ac4af1a3cff9b47cab6272.com
absugars.com aanapa.ru
abtasty.com abcguru.xyz
accessoires-mode.in adanih.com
adclickthru.net adclickservice.com
admanaerofoil.com adconscious.com
adservme.com addblueoff.com.ua
adsland.men adexprts.com
adtech.de ads.gold
afora.ru adultwebhosting.info
affrh2015.com
ageofclones.com ageofclones.com
air-edem.ru agreda.pluto.ro
allesohnegirls.net ahmedabadwebs.com
alloysteel.ru akuhni.by
alltheviews.com akvamaster.dp.ua
alpharma.net alibestsale.com
amazon-adsystem.com all-number.com
ananumous.ru allcredits.su
allsilver925.co.il
aloofly.com
aludecor.info
am15.net
amazingninja.com
ameblo.top
amt-k.ru
anapa-inns.ru anapa-inns.ru
angkortours.vn android-style.com
apiallgeniusinfo-a.akamaihd.net android-vsem.org
apibrowsesmartne-a.akamaihd.net apioasisspacenet-a.akamaihd.net
apiportalmorecom-a.akamaihd.net arabgirls.adultgalls.com
apisecretsaucebi-a.akamaihd.net
apple.com-webbrowsing-security.review
appsecurityr.com
aproposde.com
ardimobileinfo.ml ardimobileinfo.ml
artparquet.ru arenanews.com.ua
asia-forum.ru aruplighting.com
asrv-a.akamaihd.net ascat.porn
atdedinotuho.tk asmxsatadriverin.aircus.com
auto-moto-elektronika.cz astana.xxxkaz.org
avtointeres.ru audiobangout.com
ayakino.net auto.rusvile.lt
azartmix.com automatic-seo.com
azbukadiets.ru aviav.org
b2b-lounge.com aviav.ru.com
babespcs.com bank.uz
bannerspace.com bankofthewext.com
bashtime.ru
bcmp.org
beachfix.co
beachpics.com
beauby.ru beauby.ru
beauty-bracelet.com bensbargains.net
beauty-things.com bestwebsitesawards.com
best-seo-offer.com betterscooter.com
bestdooz.com betwinservice.com
bestfortraders.com bezlimitko.xyz
besthoro.ru bikini-image.com
bgviagrachrx.com billiard-classic.com.ua
bimatoprost-careprost.com bingo8888.com
bizlist.com.de bingoporno.com
blogs.rediff.com bio-optomarket.ru
bloke.com bizfly.info
bmwhighperformers.com bjanshee.ru
boostmyppc.com blog.f00kclan.de
brainzpod.win blogig.org
brandedleadgeneration.com blogqpot.com
brandimensions.com bluerobot.info
bravegirlsclub.com bluesman.nu
buckspar.gq bolitgorlo.net
buron.pw bonky.biz
camdolls.com boo-arts.com
cand.jp boobsimge.com
boole.onlinewebshop.net
brandehk.dk
brillianty.info
bristolhostel.com
budilneg.xyz
burn-fat.ga
bus-offer.com
camsex.xxx-cam.webcam
canoncdriverq3.pen.io
carfax.com.ua
cartierlove.xyz cartierlove.xyz
casinopinup-wins.com cartierreplica.top
cctva.tv cfcl.co.uk
centraletermice.us cheapjerseysbizwholesale.us
charmstroy.info cheapmoncler.win
chcu.net chinaelitecheapjerseys.com
cheapjerseysap.com chocolateslim-original.com
cheapjerseysfootballshop.com cialiswithoutadoctor.net
cheapmarmot.online cienum.fr
chelnytruck.ru cleanersoft.com
chiptuninger.com clips.ua.ac.be
chlooe.com
christianlouboutinsaleoutletonline.us
cigarpass.com
citetick.com
citysecurity.nu
clickbank.net
clickprotects.com
coin-hive.com coin-hive.com
coindirect.io
com-onlinesupport.site
com-secure.download com-secure.download
comeondog.info concordexoticrentals.com
cometorussia.net cookie-law-enforcement-ee.xyz
communisave.co.za cool-wedding.net
compliance-george.top coop-gamers.ru
compliance-julianna.top country-chic.ru
comsysnet.com covadhosting.biz
copenergo.ru covetnica.com
coslab.club d0t.ru
countbertwistdisp26.soup.io dairyindia.in
creativehutindia.com dandiyabeats.in
creditnation.ru dear-diary.ru
cubook.supernew.org deluxedumps.com
cupday.com dev33.dioniqlabb.se
customchocolate.business-for-home.com digital-sale.su
cv.wallhade.co discovertreasure-a.akamaihd.net
cvety24.by doctissimo.fr
daptravel.com dodlive.mil
darkbooks.org dojki-hd.com
data-mining.tk domainsatcost.com
decenttools.com
dikqlyremy.info
diminishedvalueoforegon.com
documentsite.net
dojki365.online
domaineaneblanc.com
domainsphoto.com
dominateforex.ml
doska-vsem.ru doska-vsem.ru
doyouknowtheword-flummox.ml dosug-lux.ru
drupa.com doublepimp.com
duplicashapp.com downtuptv.gq
dvrlists.com drev.biz
e-commerce-seo1.com duawitchrarato.tk
e-poker-2005.com e-kwiaciarz.pl
e705.net
e90post.com
ecommerce-seo.org
editors.choice6912650.hulfingtonpost.com
eduserver.net
efkt.jp
efnor-ac.com
egvar.net
el-nation.com el-nation.com
electronicadirect.com
elektronischezigarettekaufen1.beeplog.com
elitedollars.com elitedollars.com
englishgamer.com emailaccountlogin.co
eralph.tk eralph.tk
eropho.com erectile.bid
es5.com erotische-geschichten-xxl.com
everypony.ru eu-cookie-law.blogspot.com
everytalk.tv event-tracking.com
evidencecleanergold.com executehosting.com
extrabot.com ezigarettekaufen.myblog.de
extractorandburner.com ezigarettekaufen1.hpage.com
f-online.de family1st.ca
fashionavenuegame.com famix.xyz
fast-torrent.ru farmprofi.net
fermersovet.ru fashion-mk.net
ff30236ddef1465f88547e760973d70a.com filesmonster.porn
filmi-v.online filmgo.ru
filunika.com.ru financepoints.eu
finuse.com findacheaplawyers.com
flauntyoursite.com findpik.com
forensicpsychiatry.ru fitnesspiks.com
fortwosmartcar.pw flprog.com
fodelsedagspresenter.nu
foto-sisek.porngalleries.top foto-sisek.porngalleries.top
fotzen-ficken.com free-social-buttons.com
foxinsocks.ru freshsuperbloop.com
fr.netlog.com freshwallpapers.info
free-laptop-reward.com frvo.alptandem.ru
freejabs.com fuckingawesome.com
freshberry.com.ua funponsel.com
freshdz.com fxund.us
ftns.ru g.starmoe.xyz
fuckmill.com gagrasector.ru
funnel.co.za gambarkataku.co
funnymama.com gameonasia.com
fusoradio.info gay-site.store
gallerily.com gdzkurokam.ru
gamebackyard.com gepezz.info
gamedayassist.com gerhardhealer.com
gamesprite.me
gardene.ru
gate5.co.za
gavuer.ru
gay-file.com
get-your-social-buttons.info
getaclueamerica.com
getlamborghini.ga
getprismatic.com getprismatic.com
gheus.altervista.org gheus.altervista.org
glall.ru gilbertbanda.net
glopages.ru gktt.ru
goblacked.com glass-msk.ru
golden-praga.ru glicol.kz
gopixdatabase.com goodwriterssales.com
goroda-vsego-mira.ru gopro-online.info
grand-chlen.ru gribokstop.com
greatfind-a.akamaihd.net growshop.es
greendream.com.ua gsou.cf
grizzlysgrill.com gtopstats.com
growboxbl.ru guge.io
gsasearchengineranker.site handicapbathtubarea.com
gsasearchenginerankerhelp.com
guidefs.ru
hanwei.us
hauleddes.com
hawaiielectriclight.com
hd-film.pl
hdimagegallery.net hdimagegallery.net
heartofbeijing.blogspot.com headpress.ru
heatpower.ru hebr.myddns-flir.com
helpmymacfaster.trade heygidday.biz
hermesbirkinhandbagoutlets.com high-speed1.net
hidefiles.org hiwibyh.bugs3.com
historichometeam.com hjaoopoa.top
hobbyhuren-datenbank.com hkladys.com
homegardenlova.com hollywoodactress.info
hopeonthestreet.co.uk homeart.space
hotloans.ru hostingclub.lk
houseofgaga.ru hotcore.info
howmuchdoestizanidinecost.blogspot.com hotelcrocenzi.sm
hspline.com hotgirlhdwallpaper.com
hut1.ru hotshoppymac.com
iboss.com howtoclean.club
icaseclub.ru hscsscotland.com
idc.com.ua hundejo.com
ideashome.id iamsport.org
ilovevitaly.ru imabase.com
imagerydatabase.com imagerydatabase.com
images-free.net immobiliaremassaro.com
imagez.co importchinacoach-teach.com
insider.pro impresagaia.it
insomniagamingfestival.com incep.imagine-ex.co
inzn.ru indiasourcemart.in
iscblog.info instakink.com
isitwp.com integritylandscapeservices.com
istanbulit.com interior-stickers.ru
intervsem.ru
iplogger.org
isistaylorporn.info
istizanidinelikexanax.blogspot.com istizanidinelikexanax.blogspot.com
istock-mebel.ru
jackwolfskinoutlet.online jackwolfskinoutlet.online
janerikholst.se jav-fetish.com
javatex.co.id javxxx18.com
jobgirl24.ru jjbabskoe.ru
katushka.net jofucipiku.tk
keywordsking.com johnrobertsoninc.com
kidd.reunionwatch.com kandidos.com
kiinomaniak.pl karaganda.xkaz.org
kimcurlrvsms.com kartiniresto.com
kinocccp.net kemerovo.zrus.org
kleine-titten.biz ketoanhanoi.info
knowsitall.info keywords-monitoring-your-success.com
konica.kz kihi.gdn
kountrylife.com kino-rating.ru
krasnodar.xrus.org kochanelli.com
kumuk.info konoplisemena.com
kunstaktien.info koopilka.com
lapitec.eu kreativperlen.ch
laptoper.net kretpicf.pw
laudit.ru lacave.ntic.fr
letsart.ru ladsup.com
lineshops.biz lawyers.cafe
linkdebrideur.xyz laxdrills.com
liumimgx.pw lazy-z.com
lombia.co leadwayau.com
longadventure.com letslowbefast.today
luciddiagnostics.in letsrepair.in
magicalfind-a.akamaihd.net lineavabit.it
managerpak204.weebly.com lion.playtap.us
mantramusic.ru liquimondo.com
mastershef.club littlecity.ch
matchpal-a.akamaihd.net llastbuy.ru
lrac.info
lutherstable.org
m4ever.net
maju.bluesalt.co
manimpotence.com
manve.info
map028.com
marmotstore.online
matpre.top
max-p.men
mb140.ru
mcadamssupplyco.com mcadamssupplyco.com
megakino.net mediawhirl.net
meget.co.za medmajor.ru
merryhouse.co.uk mega-bony-2017.pl
mexicosleevegastrectomy.com mesbuta.info
microsoftportal.net michaelkorsoutlet.store
miniads.ca mindbox.co.za
mlf.hordo.win missvietnam.org
mnogolok.info mistr-x.org
mobilemedia.md moje-recenze.cz
monetizationking.net moncleroutletonline.win
morpicert.pw moneyteam24.com
monitorwebsitespeed.com
morf.snn.gr
moskva.nodup.ru moskva.nodup.ru
mp3downloadhq.com mototsikl.org
moxo.com
mrbitsandbytes.com
mrlmedia.net mrlmedia.net
musicstock.me mrsdalloways.com
muizre.ru
multgo.ru
muzaporn.com
myfreemp3.eu myfreemp3.eu
myonlinepayday.co mypets.by
myyour.eu mystats.xyz
nardulan.com nac-bearings.ru
needtosellmyhousefast.com nalogovyykodeks.ru
news-speaker.com narkologiya-orel.ru
nextconseil.com narkologiya-penza.ru
nfljerseyscheapchinabiz.com news-readers.ru
ningessaybe.me next-dentists.tk
nl.netlog.com nicovideo.jp
nonameread45.live nikitabuch.com
nudejapan.net normalegal.ru
obiavo.in nudepatch.net
ok.ru oda.as
oneminutesite.it oecnhs.info
oneshotdate.com offer.camp
openlibrary.org office2web.com
orhonit.com okayimage.com
outshop.ru okmedia.sk
owohho.com onemactrckr.com
oxford-book.com.ua onetravelguides.com
oz-offers.com online-hd.pl
parajumpersoutlet.online online7777.com
partnersafe.men onload.pw
pateaswing.com onlyporno.ru
pay2me.pl openstat.com
pepperstyle.ru oranga.host.sk
perm-profnastil.ru osnova3.ru
perosan.com osoznanie-narkotikam.net
perso.wanadoo.es ovirus.ru
phpdevops.com pagesense.com
pic-re.blogspot.com pandarastore.top
pic2fly.com parfusale.se
picquery.com pearlisland.ru
pictures-and-images.com penisvergrotendepillennl.ovh
pijoto.net petrovka-online.com
pintattoos.com
piter.xrus.org
piuminiita.com piuminiita.com
pixelrz.com pk-pomosch.ru
plastjulgranar.se
play-mp3.com
play.leadzupc.com play.leadzupc.com
ploenjitmedia.azurewebsites.net playmsn.com
pogosh.com pogodnyyeavarii.gq
porno-play.net polybuild.ru
pornokajf.com pomoc-drogowa.cba.pl
pregnant.guru popugaychiki.com
preparevideosafesystem4unow.site popup-jdh.xyz
privat-girl.net pornhive.org
prizefestival.mobi porno-chaman.info
porno-video-chati.ru
portadd.men
positive2b.ru
pp-budpostach.com.ua
preconnubial.usuby.site
priscilarodrigues.com.br
privatefx-in.ru
prizeestates.cricket
proekt-mos.ru
profitwithalex.info profitwithalex.info
prom23.ru progonrumarket.ru
promoforum.ru promover.org
prostitutki-astana.org pron.pro
prostitutki-novgoroda.org prospekt-st.ru
prostitutki-yaroslavlya.org pst2017.onlinewebshop.net
realting-moscow.ru purplestats.com
recipedays.com pushdata.sendpulse.com
refudiatethissarah.info putevka24.ru
puzzleweb.ru
pxhdwsm.com
ranksonic.info
rating-bestcasino.com
razyboard.com
realitykings.com
realnye-otzyvy.info
redbottomheels.xyz
relatodelpresente.com.ar relatodelpresente.com.ar
remont-mobile-phones.ru renecaovilla.online
remote-dba.de rent2spb.ru
replica-watch.ru revistaindustria.com
rfid-locker.co ribieiendom.no
rollercoin.com ringporno.com
roofers.org.uk riralmolamsaca.tk
rozalli.com robotixix.com
rvtv.ru russian-postindex.ru
sa-live.com russkie-gorki.ru
safe-app.net s-iwantyou.com
sanyuprojects.com samara.rosfirm.ru
sbornik-zakonov.ru samolet.fr
scanner-donald.top scanner-george.top
scanner-margo.top scanner-ivan.top
scenicmissouri.us scanner-josh.top
sdsjweb.com scoopquest.com
searchaddis.com scrapy.org
searchimage.co secretscook.ru
searchwebknow-a.akamaihd.net seeresultshub-a.akamaihd.net
seoboxes.com semalt.com
seocheki.net semprofile.com
serptehnika.ru seo-prof1.xyz
service.adtech.us seoanalyses.com
seryeznie-znakomstva.ru seofirmreviewsus.info
seolab.top
seoprofiler.com
seorank.info
sevendays.com.ua sevendays.com.ua
sexvporno.ru sextracker.de
shell-pmr.ru sexuria.net
shemalegalls.blogporn.in sexy-pings.com
si-unique.com sharebutton.to
shivafurnishings.com
shop-electron.ru
shtora66.ru
sibtest.ru
sibvitr.ru sibvitr.ru
siteheart.net sildenafil-tadalafil.info
sixcooler.de simplepooltips.com
slavic-magic.ru sisi-go.ru
smichovbike.cz slowmacfaster.trade
socialsignals24.com smartshoppymac.com
softomix.net snip.tw
soochi.co snjatie-geroinovoy-lomki.ru
spaceshipad.com socialmonkee.com
spinnerco.ca soda.media
stardevine.com sohoindia.net
staticfs.host star61.de
stpicks.com stathat.com
stpolice.com statistici.ro
stroydetali.ru stats-public.grammarly.io
studentguide.ru stauga.altervista.org
sukarame.net streetfire.net
supermodni.com.ua stroyplus.ru
stuff-about-money.com
suchenindeutschland.com
suggest-keywords.com
suralink.com
surintech.ac.th
survival.betteroffers.review survival.betteroffers.review
swingerseiten.com svnuppsalaorebro.se
syndicate.fun svtrd.com
tackletarts.co taboola.com
taihouse.ru tacbibirfa.tk
tattooreligion.ru takeflyte.com
teencastingporn.com tastyfoodideas.com
telefonsexi.com technika-remont.ru
tengohydar.tk tedy.su
thedownloadfromwarez.blogspot.com teenbbw.yopoint.in
theendivechronicles.com theheroes.ru
theplacetoupdating.pw thiegs.reco.ws
thexart.club thruport.com
timecrimea.ru timetorelax.biz
titan-ads.life titelhelden.eu
tizanidine4mgprice.blogspot.com tizanidineandcipro.blogspot.com
tizanidine6mg.blogspot.com tizanidinecost.blogspot.com
tizanidineandgabapentin.blogspot.com tizanidinedrugtest.blogspot.com
tizanidinecapsules.blogspot.com tizanidinehcl4mgtabinfo.blogspot.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com tizanidinesleepaid.blogspot.com
tkanorganizma.ru tjkckpytpnje.com
top-deal.com.pl tn811.us
topads.men tooplay.com
toproadrunner5.info torrentdownloadhub.com
totu.us toxicwap.com
tourismvictoria.com tozup.com
toursmaps.com tracksz.co
toys.erolove.in trafficstars.com
tracfone.com truemfilelj.gq
traffic-club.info uamtrk.com
trafficmania.com uchebavchehii.ru
traffixer.com uggbootsoutletsale.us
traffpartners.com uggsale.online
triumf-realty.ru ukr-lit.com
turizm.bz ultimateclassicrock.com
twodollarshows.com urlopener.com
txxx.com us-america.ru
tyumen.xrus.org vabasa.inwtrade.com
u555u.info videochat.bz
unitygame3d.com videochat.world
urengoy.pro vids18.site
ustion.ru vita.com.hr
vacuumscleaner.com vse-pesni.com
validccseller.com vut.com.ru
vapomnoncri.tk vzube.com
varikozdok.ru
viandpet.com
video-hollywood.ru
videospornogratisx.net
vielporno.net
virtualbb.com
virus-schutzmasken.de
vitoriacabos.com
vizag.kharkov.ua
volgograd.xrus.org
vseigry.fun
vzubah.com
wahicbefa31.soup.io
wait3sec.org
wapsite.me wapsite.me
wareseeker.com weblo.com
wdfdocando.com
webadvance.club
webmasterhome.cn
weboptimizes.com weboptimizes.com
webstatsdomain.org welck.octopis.com
whatistizanidinehclusedfor.blogspot.com wemedinc.com
wingsoffury2.com wesharepics.info
woodyguthrie.se wholesalecheapjerseysfree.com
word-vorlagen.net worldhistory.biz
worldtraveler.world woweb.com.ua
wowas31.ucoz.ru www.timer4web.com
wpsecurity.website x-porno.video
writersgroup580.web.fc2.com x-stars.ru
ws.ampower.me xmnb.net
wstroika.ru xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
www.wohnkabinen-shop.de xn----7sbaphztdjeboffeiof6c.xn--p1ai
xcc24.pl xn----7sbifcamovvfggw9d.xn--p1ai
xdoza.com xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xfluro.com xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
xgames-04.com xn----itbeirbjbi7bc6bh2d.xn--p1ai
xn--80abgj3a5acid6ghs.top xn--1-8sbcpb0bdm8k6a.xn--p1ai
xn--hxazdsfy.blogspot.com xn--80ahvj9e.xn--p1ai
xtrafficplus.com xn--80ajbshivpvn2i.xn--p1ai
xxart.ru xn--90acenikpebbdd4f6d.xn--p1ai
xxxhdvideo.site xnxxandxvideos.com
xxxsiterips.xyz xuki.us
yoopsie.com xxxkaz.org
youdesigner.kz youdao.com
zagadki.in.ua yoursearch.me
zaobao.com.sg youtoner.it
zarepta.com zaloadi.ru
zawyna.ua zaloro.com
zero1.it zazagames.org
zeroredirect6.com zhacker.net
zkjovpdgxivg.ga zhongwenlink.com
zlatnajesen.com zhorapankratov7.blogspot.com
znakomstvaonlain.ru znaturaloriginal.com
zojirushi-products.ru zolotoy-lis.ru
zx6.ru zone-kev717.info
zreloeporno.tv

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00it.com 101lesbian.xyz
03e.info 123any.com
1314dh.com 123kuma.com
2020iscoming.info 2daytrendingnews.com
2pxg8bcf.top 8kisses.eu
3w24.com 8lufu.com
5forex.ru aa9bd78f328a6a41279d0fad0a88df1901.com
6128786.com aanapa.ru
aa9d046aab36af4ff182f097f840430d51.com addray.pro
adf.ly adssafeprotected.com
adultmeetonline.info aibolita.com
ahrntal.verymes.xyz aktivator-windows10.blogspot.com
amateurgalls.com alert-jdh.xyz
anal-acrobats.hol.es alibestsale.com
arewater.com aloofly.com
asiengirls.net ananumous.ru
asociatia-tipografilor-transilvania.ro appiq.mobi
ausmepa.org.au applicationg29.com
auto.rusvile.lt asseenontv.ru
autoplate.info autolombard-krasnodar.ru
ayeartoforget.com b-styles.xyz
ayerbo.xhost.ro balans.shahterworld.org
bang-hotties.com baltgem.com
banki76.ru bbtec.net
bannerspace.com beauty-things.com
bensbargains.net berdasovivan.ru
betterscooter.com betterhdporn.com
bfz.biz bio.trade-jp.net
blobar.org
blog.koreadaily.com blog.koreadaily.com
blpmovies.com bonkers.name
bristolhotel.com bravegirlsclub.com
bumble.cheapwebsitehoster.com buqayy0.livejournal.com
buyessay3.blogspot.ru buybest1.biz
buzzsumo.com buyessaynow.biz
bwlx.prepedu.cn californianews.cf
canyougethighofftizanidine.blogspot.com canrioloadm.gq
cartierreplica.pw canvas.gloverid.site
cartierring.xyz cartierjusteunclou.xyz
cartiertrinity.xyz centre-indigo.org.ua
cheap-pills-norx.com chidporn.com
chiptuninger.com cinemaenergy-hd.ru
choice-credit.ru click2pawn.com
cjmc.info clmforexeu.com
clarithromycin500mg.com colehaanoutlet.store
codysbbq.com communisave.co.za
coindirect.io compucelunlock.net
compliance-alexa.xyz converse.ddsoldes.fr
compliance-don.xyz cwetochki.ru
compliance-jessica.top daydream-studio.ru
compliance-john.top de.zapmeta.com
conboy.us dentalpearls.com.au
cowblog.fr descargar-musica-gratis.net
credit-online.ws dispo.de
crest-poloski.ru dnsrsearch.com
cvety24.by domoysshop.ru
cyse.tk donvito.unas.cz
daretodonate.co doxyporno.com
dear-diary.ru drev.biz
dikqlyremy.info dumpsmania24.com
divci-hry.info easyshoppermac.com
domain-tracker.com efnor-ac.com
dominateforex.ml el-nation.com
e-biznes.info elektrischezigarettekaufen2.cowblog.fr
education-cz.ru elektrischezigaretten1.blogse.nl
eloconcream.blogspot.com envaseslotusama.com
englate.com escortslet.net
essaypro.com essayassist.com
eugenevaultstorage.com fanoboi.com
executehosting.com ferrotodo.com
ezigarettenshop2.myblog.de finansov.info
facebook-mobile.xyz findpik.com
familienzahnaerzte.com fishtauto.ru
faracontrol.ir flprog.com
findthe.pet focalink.com
fingerprintjs.com fortunejack.com
free-gluten.ru freesitetest.com
freecamdollars.com gdcentre.ru
froggytube.com germetiki.com.ua
fxtips.ru getlaid-xxxhookupdirect.com
gameprimary.com go2jump.org
ge0ip.com goldenggames.com
gelezki.com goodly.pro
get-seo-domain.com
googlemare.com googlemare.com
graphicwe.org greatgrace.ru
greendream.com.ua guitar-master.org
greenidesign.co guruofcasino.com
gyffu.com
hamilton.ca hamilton.ca
handicapbathtubarea.com hanink.biz.ly
hasfun.com hard-porn.mobi
hdpixion.com hasshe.com
hebr.myddns-flir.com hausfrauensex18.com
heliko.no helvetia.com.ua
heroz.fr hoporno.com
hitcpm.com hostsshop.ru
homedecorpicture.us howlongdoestizanidinestayinyoursystem.blogspot.com
hoverboard360.es https-legalrc.biz
ictizanidinehcl4mg.blogspot.com hubbble.com
ideoworld.org iamsport.org
ilovevitaly.com ibb.com.ua
inclk.com imitex-plus.ru
indo-export.ru indiasourcemart.in
itrevolution.cf infazavr.ru
izoll.ru interior-stickers.ru
jovencitas.gratis ispaniya-costa-blanca.ru
joyceblog.top jabimgo.pw
juliadiets.com japan-bearings.ru
justucalling32211123456789.tk justucalling32211123456789.tk
keyhantercume.com kamalsinha.com
keywords-monitoring-your-success.com koleso24.com.ua
kometars.xyz kollesa.ru
komputernaya-pomosh-moscow.ru kredit-blog.ru
konpax.com la-fa.ru
konteiner24.com lazymanyoga.com
kostenloser-sex.com letsrepair.in
krasnodar-avtolombards.ru lider-zhaluzi.kiev.ua
kupiproday.com.ua lifeinsurancekb.com
laptop-4-less.com loftdigital.eu
liffa.ru louisvuittonoutletstore.net
lion.playtap.us m3gadownload.pl
lollypopgaming.com mainlinehobby.net
lomza.info mansiondelrio.co
ltvperf.com mcnamaratech.com
luckyshop.net.ua medizinreisen.de
maranbrinfo.com.br member-quiz.com
marmotstore.online members.ghanaweb.com
masterseek.com michaelkorsoutletstore.net
mazda-roadsters.com minecraft-rus.org
mediawhirl.net mjchamonix.org
mhtr.be mncrftpcs.com
minet.club mnogabukaff.net
mirzonru.net mobile-appster.ru
mixtapetorrent.com monarchfind-a.akamaihd.net
mrsdalloways.com moncleroutletonline.pw
mycouponizemac.com motherboard.vice.com
mygameplus.ru mrinsidesales.com
mylida.org myonigroup.com
nastroyke.net myperiod.club
naturalbreakthroughsresearch.com mypets.by
nero-us.com na-telefon.biz
nhl09.ru narkologiya-orel.ru
nicovideo.jp new-post.tk
nikitabuch.com odesproperty.com
nuker.com ok.ru
o-o-6-o-o.ru onlinemegax.com
oconto.ru oops-cinema.ru
ohmyrings.com openlibrary.org
online247.ml orel-reshka.net
oralsexfilme.net osoznanie-narkotikam.net
oren-cats.ru
ossmalta.com
oyster-green.com oyster-green.com
patol01.pw painting-planet.com
pensplan4u.com panamaforbeginners.com
perl.dp.ua partner-host.men
pic-re.blogspot.com pelfind.me
picswe.com petrovka-online.com
pintattoos.com pinkduck.ga
pizdeishn.com poppen-nw.net
poffet.net porn555.com
pogruztehnik.ru porndroids.com
pornowarp.info pornmania.pl
pp-budpostach.com.ua pravoholding.ru
princeadvantagesales.com privatefx.all4invest.info
projectforte.ru profitkode.com
pussyspace.net prosmibank.ru
quelle.ru quebec-bin.com
qwesa.ru quick-seeker.com
rcb101.ru
rcpmda.ikan1080.xyz rcpmda.ikan1080.xyz
reelheroes.net relayblog.com
regdefense.com rfd-split.hr
reimageplus.com rocketchange.ru
relax.ru royalads.net
remmling.de s1z.ru
resultshub-a.akamaihd.net s8-nowy-wygraj.comli.com
risparmiocasa.bz.it sabizonline.com
ruspdd.com savingsslider-a.akamaihd.net
sajatvelemeny.com sbwealthsolutions.ca
scanmyphones.com scanner-barak.top
security60-e.com scanner-fred.top
sell-fb-group-here.com scanner-marwin.top
seoprofiler.com schalke04fc.info
servingnotice.com schoolfiles.net
sexy-pings.com scmor.ilxc.cc
sibdevice.ru scopich.com
sivs.ru sdrescher.net
skytraf.xyz seasonvar.ru
slujbauborki.ru selectads.men
semprofile.com
seozoom.it
sex-pr.net
shillyourcoins.com
sideeffectsoftizanidine.blogspot.com
sinestesia.host.sk
skatestick.bid
sledstvie-veli.net
slimcdn.com
slonechka.ru
small-game.com
smichovbike.cz smichovbike.cz
soblaznu.net snip.tw
social-search.me sohoindia.net
sodexo.com souvenir.cc
soserfis.com space4update.pw
stal-rulon.ru stathat.com
stefanbakosab.se statustroll.com
stmassage.ru stockquotes.wooeb.com
stoki.ru streetfooduncovered.com
streetfire.net studiokamyk.com.pl
stroimajor.ru sugvant.ru
suture.co supervesti.ru
svarkagid.com svolze.com
taboola.com swingerseiten.com
takeprofitsystem.com taihouse.ru
taxi-v-eisk.ru taranerymagesswa.blogspot.com
tecnoteakviareggio.it taraz.xkaz.org
tedy.su telemetryverification.net
thaisamkok.com textads.men
thedownloadfromwarez.blogspot.com tizanidine4mgstreetprice.blogspot.com
tizanidine4mg.blogspot.com tizanidinehydrochloride2mg.blogspot.com
tizanidinecost.blogspot.com toondinsey.com
tizanidinedosage.blogspot.com topmira.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com tradgardspartner.se
tizanidinevscyclobenzaprine.blogspot.com trafficnetzwerk.de
tmtrck.com trafficstars.com
topdownloads.ru trkdf.com
topquality.cf tubo360.com
topsy.com unitygame3d.com
track-rankings.online url-img.link
traderzplanet.in usswrite.com
ttrraacckkrr.com videochat.bz
tvorozhnaja-zapekanka-recept.ru viel.su
tx41tclega.ru visa-pasport.ru
umg-stroy.ru vkontaktemusic.ru
uogonline.com vseigru.one
uptime-alpha.net wardreapptokone.tk
ural-buldozer.ru watchinf.com
urdoot.win websites-reviews.com
usacasino.com woman-h.ru
v720hd.ru www.souvenirua.com
vepad.com xmladserver.com
videonsk.com xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
vids18.site xn----7sbifcamovvfggw9d.xn--p1ai
webstatsdomain.org xn--1-8sbcpb0bdm8k6a.xn--p1ai
wholesalejerseys-cheapest.com xn--d1aifoe0a9a.top
wovis.site xvideosbay.com
ww2awards.info yhit.press
x-diesel.com zahvat.ru
xmlinde.com
xn--b1adccaf1bzj.xn--p1ai
yachts-cruise.info
yourads.website
yourserverisdown.com
youthreaders.com
yugk.net
zaimhelp.ru
zakonobosago.ru
zarepta.com
zlatnajesen.com
zona-aqua.ru zona-aqua.ru
zreloeporno.tv
zryydi.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2547 ### Version: V4.2021.08.2548
### Updated: Tue Aug 24 22:01:16 UTC 2021 ### Updated: Wed Aug 25 10:03:29 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2547 ### Version: V4.2021.08.2548
### Updated: Tue Aug 24 22:01:16 UTC 2021 ### Updated: Wed Aug 25 10:03:29 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##