vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 02:59:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.09.3340 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-09-09 10:09:21 +00:00
parent f303163545
commit cbe285fbad
47 changed files with 94602 additions and 94703 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.09.3339 #### Version: V4.2022.09.3340
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.09.3339 #### Version: V4.2022.09.3340
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.09.3339 #### Version: V4.2022.09.3340
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11690
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.09.3339 #### Version: V4.2022.09.3340
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11694
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11694
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11694
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11694
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

11694
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

11694
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

11694
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3339 3340

11694
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11694
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.65.147.232 0; 20.29.115.225 0;

11694
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

11694
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

11694
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

11694
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.65.147.232 0; 20.29.115.225 0;

11694
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.65.147.232 0; 20.29.115.225 0;

11694
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BackStreet Copyscape
Iria DatabaseDriverMysqli
MJ12bot GalaxyBot
Magnet Jorgee
NetMechanic Moblie Safari
OnCrawl PictureFinder
PECL::HTTP RankingBot
Seomoz Siteimprove
TheNomad Trendiction
WASALive-Bot pcBrowser

276
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 360Spider
80legs ASPSeek
Acunetix Abonti
AfD-Verbotsverfahren AfD-Verbotsverfahren
Anarchie Alexibot
AllSubmitter
Anarchy99 Anarchy99
Apexoo
Asterias Asterias
Atomseobot Atomseobot
Attach
AwarioSmartBot AwarioSmartBot
BBBike BDCbot
BDFetch BDFetch
BackStreet BLEXBot
BackWeb BackWeb
Backlink-Ceck Barkrowler
Bandit BetaBot
Bitacle
Black Hole Black Hole
Blow BlackWidow
Boardreader Boardreader
Bolt BotALot
Buck Brandprotect
Bullseye Buddy
BunnySlippers BunnySlippers
BuzzSumo BuzzSumo
CATExplorador
CSHttp CSHttp
Calculon
CazoodleBot CazoodleBot
Cogentbot CherryPicker
ChinaClaw
Chlooe
Cliqzbot
Collector Collector
Copier Copier
Copyscape
Craftbot Craftbot
CrazyWebCrawler CrazyWebCrawler
Crescent Crescent
Curious CyotekWebCopy
DIIbot DatabaseDriverMysqli
DSearch Demon
DataCha0s Deusu
Disco Disco
Discoverybot DittoSpyder
Dispatch DnBCrawler-Analytics
DomainAppender DomainAppender
DomainSigmaCrawler Dotbot
Domains Project Download Wonder
Dragonfly
ECCP/1.0
EMail Wolf EMail Wolf
EasyDL
EirGrabber
EroCrawler
Exabot Exabot
FHscan ExtLinksBot
FemtosearchBot ExtractorPro
FlashGet Fimap
Flunky
Freeuploader Freeuploader
FrontPage
Fuzz
Fyrebot Fyrebot
GT::WWW G-i-g-a-b-o-t
GalaxyBot GetRight
GetWeb GetWeb
Getintent
Gigabot
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
Gotit Gotit
GrabNet GrapeshotCrawler
GridBot HMView
HEADMasterSEO HTTrack
HTMLparser Haansoft
HaosouSpider
Heritrix Heritrix
Hloader Humanlinks
Iblog HybridBot
IlseBot
IndeedBot IndeedBot
Indy Library InfoNaviRobot
Internet Ninja InterGET
InternetSeer
IstellaBot IstellaBot
JamesBOT
JikeSpider JikeSpider
Joomla
Jorgee Jorgee
JustView JustView
Kenjin Spider Kenjin Spider
LNSpiderguy Kinza
LWP::Simple
Lanshanbot Lanshanbot
LexiBot LeechFTP
LeechGet
Lftp Lftp
Likse LibWeb
LieBaoFast
Lightspeedsystems
LinkScan LinkScan
LinkWalker Linkbot
LinqiaScrapeBot LinkextractorPro
LinkpadBot
LinksManager
Lipperhey Spider Lipperhey Spider
MIDown tool Lmspider
MQQBrowser MIIxpc
Magnet MJ12bot
Mail.RU_Bot Mail.RU_Bot
Majestic-SEO
Majestic12 Majestic12
MarkMonitor MicroMessenger
MarkWatch Microsoft Data Access
Mass Downloader
Masscan
Mata Hari
Mediatoolkitbot
Metauri
Minefield Minefield
Mojolicious Moblie Safari
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Name Intelligence Mr.4x3
Msrabot
Musobot
Nameprotect
Navroad
NearSite NearSite
Needle
Net Vampire
NetAnts NetAnts
NetZIP NetZIP
NextGenSearchBot NextGenSearchBot
Nibbler Nikto
NimbleCrawler NimbleCrawler
Ninja Nmap
Not Not
Nutch Nuclei
Octopus Offline Explorer
Offline Navigator Offline Navigator
OpenLinkProfiler OpenVAS
OrangeSpider Openfind
OutclicksBot
PECL::HTTP PECL::HTTP
PageAnalyzer PHPCrawl
PageScorer
PageThing.com
Pandalytics Pandalytics
Papa Foto Papa Foto
Pavuk
PeoplePal
Picscout Picscout
PictureFinder ProPowerBot
Piepmatz Probethenet
ProWebWalker Pump
Pu_iN PxBroker
PyCurl PyCurl
QueryN Metasearch
RSSingBot RSSingBot
RankFlex
Rankivabot Rankivabot
RankurBot RankurBot
ReGet
RealDownload RealDownload
RedesScrapy
RepoMonkey
Rogerbot Rogerbot
SEOkicks SBIder
SEOkicks-Robot SISTRIX
Scanbot ScanAlert
ScoutJet
Scrapy Scrapy
Screaming Screaming
Searchestate
SearchmetricsBot SearchmetricsBot
SemanticJuice Seekport
Semrush Semrush
SemrushBot SeobilityBot
Shodan Shodan
SiteCheckerBotCrawler Site Sucker
SiteExplorer SiteExplorer
SiteSnagger
Sitevigil
SlySearch SlySearch
SmartDownload
Snake
Snoopy Snoopy
SocialRankIOBot SocialRankIOBot
Sogou web spider Sosospider
Sottopop
SpaceBison
Spammen Spammen
SpankBot
Spanner
Sqlmap Sqlmap
Steeler Steeler
Sucker Sucuri
SuperBot SuperBot
SurveyBot SurveyBot
Suzuran
Szukacz
T0PHackTeam T0PHackTeam
TeleportPro Teleport
Telesoft Telesphorep
The Intraformant TightTwatBot
Tracemyfile Titan
Trendiction Toata
Trendictionbot
TwengaBot TwengaBot
Twice
URLy Warning
URLy.Warning URLy.Warning
Upflow Upflow
V-BOT
VB Project VB Project
VCI
Vacuum Vacuum
VidibleScraper VoidEYE
Voil Voil
Voltron Voltron
WBSearchBot WASALive-Bot
WEBDAV WEBDAV
WISENutbot
WPScan WPScan
WWW-Collector-E WWWOFFLE
Wallpapers WeSEE
Wallpapers/3.0 Web Auto
Web Enhancer Web Pix
Web Fuck WebBandit
Web Sauger WebCopier
WebEnhancer WebImageCollector
WebLeacher WebPix
WebReaper
WebSauger WebSauger
WebStripper
WebWhacker
WebZIP WebZIP
WebmasterWorldForumBot Webalta
Webshag Webshag
Website Quester Website Quester
Whack Whatweb
Whacker
Who.is Bot
Widow
WinHTTrack
Woobot Woobot
Xaldon WebSpider YoudaoBot
Zade
Zermelo
Zeus Zeus
ZoomBot
ZoominfoBot
ZumBot ZumBot
adscanner adscanner
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
autoemailspider
cah.io.community cah.io.community
coccocbot
cognitiveseo cognitiveseo
com.plumanalytics crawler4j
crawl.sogou.com
crawler.feedback
dataforseo.com
evc-batch evc-batch
gopher gopher
heritrix instabid
internetVista monitor
iubenda-radar iubenda-radar
linkdexbot
magpie-crawler magpie-crawler
meanpathbot
netEstate NE Crawler netEstate NE Crawler
page scorer oBot
pcBrowser pcBrowser
plumanalytics
probe-image-size probe-image-size
s1z.ru s1z.ru
satoristudio.net satoristudio.net
scan.lol
seocompany.store
seoscanners seoscanners
seostar serpstatbot
sexsearcher
sitechecker.pro
siteripz siteripz
sogouspider
spyfu
sysscan sysscan
tAkeOut trendiction.com
ubermetrics-technologies.com ubermetrics-technologies.com
webpros.com webprosbot
x09Mozilla
zauba.io zauba.io
zgrab zgrab

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 404enemy
80legs
ADmantX ADmantX
ALittle Client ALittle Client
Acunetix Aboundexbot
AiHitBot AfD-Verbotsverfahren
AhrefsBot
Aipbot
Alexibot Alexibot
Alligator Alligator
Anarchie Anarchy
Anarchy99 Ankit
Apexoo Apexoo
Aspiegel
Atomseobot
Attach
AwarioRssBot AwarioRssBot
BDCbot
BLEXBot BLEXBot
Backlink-Ceck Backlink-Ceck
BacklinkCrawler Bandit
Badass
Barkrowler
BatchFTP BatchFTP
Black Hole BlackWidow
Blackboard
Blow
BlowFish
Boardreader
Bolt Bolt
Brandwatch
Buck Buck
CATExplorador BuiltBotTough
Calculon BunnySlippers
CODE87
CSHttp
CazoodleBot
Cegbfeieh
CensysInspect
CheTeam CheTeam
CherryPicker Cocolyzebot
Cogentbot
Collector
CopyRightCheck CopyRightCheck
Copyscape Copyscape
Cosmos
Craftbot
CrazyWebCrawler CrazyWebCrawler
Crescent Crescent
Curious
Custo Custo
DBLBot
DIIbot
DSearch
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Deusu Demon
Devil
Digincore Digincore
Discoverybot DnyzBot
DittoSpyder
DomCopBot DomCopBot
DomainAppender DomainSigmaCrawler
DomainCrawler DomainStatsBot
Domains Project Dotbot
Dragonfly Dragonfly
Drip EMail Siphon
Ebingbong
EirGrabber EirGrabber
Express WebPictures Evil
ExtractorPro ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
FDM FDM
Fimap
Firefox/7.0 Firefox/7.0
FlashGet
Flunky Flunky
Foobot Freeuploader
FrontPage FrontPage
Fuzz Fyrebot
FyberSpider
GalaxyBot GalaxyBot
Genieo
GetRight GetRight
Gigabot Go-Ahead-Got-It
GoZilla
Grafula Grafula
GrapeFX GrapeFX
GrapeshotCrawler GrapeshotCrawler
HMView HTTP::Lite
HTTrack HaosouSpider
Haansoft Harvest
Havij Havij
Heritrix Humanlinks
Hloader IDBTE4M
IDBot
Iblog
Id-search
IlseBot IlseBot
Image Sucker Image Fetch
IndeedBot Indy Library
InfoTekies InfoNaviRobot
Intelliseek
InterGET
InternetSeer InternetSeer
Iskanie
IstellaBot
JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz
JennyBot JennyBot
JetCar
JikeSpider JikeSpider
Jorgee Jorgee
Jyxobot
Kenjin Spider
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
LWP::Simple LNSpiderguy
Lanshanbot Larbin
LeechFTP LeechFTP
LeechGet LeechGet
LexiBot
Lftp
LieBaoFast LieBaoFast
LinkScan
LinkWalker LinkWalker
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Lipperhey
MJ12bot Litemage_walker
MFC_Tear_Sample
MIIxpc
MSFrontPage MSFrontPage
MTRobot
Mag-Net
Magnet
Majestic12 Majestic12
MarkMonitor
MarkWatch
Mass Downloader Mass Downloader
Meanpathbot MauiBot
Mediatoolkitbot Mb2345Browser
MegaIndex.ru MicroMessenger
Metauri Microsoft Data Access
Microsoft URL Control Minefield
Mister PiX Mister PiX
Moblie Safari Mojeek
Msrabot MolokaiBot
Musobot Musobot
Nessus NICErsPRO
NetLyzer Nameprotect
NetMechanic NearSite
NetSpider NetSpider
Nettrack NetZIP
Netcraft
Nibbler
Niki-bot
Nikto
NimbleCrawler NimbleCrawler
Nimbostratus
Nuclei Nuclei
Nutch Nutch
OpenLinkProfiler Offline Explorer
Offline Navigator
OrangeBot OrangeBot
OutclicksBot OrangeSpider
OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl PeoplePal
Page Analyzer
PageGrabber
PageScorer
Pandalytics
Papa Foto
Pavuk
Petalbot
Pi-Monster Pi-Monster
PleaseCrawl Picscout
ProPowerBot Picsearch
ProWebWalker Pu_iN
PyCurl
QueryN Metasearch QueryN Metasearch
RSSingBot
RankFlex
RankingBot RankingBot
RankurBot Rankivabot
RealDownload RealDownload
RebelMouse RebelMouse
RepoMonkey RepoMonkey
RocketCrawler Ripper
SEOkicks SBIder
SEOkicks-Robot SEOkicks-Robot
SEOstats SEOprofiler
SISTRIX Scrapy
SMTBot
SalesIntelligent
ScanAlert
ScreenerBot
ScrepyBot
Searchestate Searchestate
Semrush Seekport
SeobilityBot SemrushBot
Shodan SentiBot
Seomoz
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteLockSpider SiteLockSpider
Sitebeam SiteSnagger
SiteSucker
Sitevigil Sitevigil
SmartDownload SmartDownload
Spanner Snake
SputnikBot Sociscraper
SpaceBison
Spammen
Sqlmap Sqlmap
Stripper Sqlworm
Sucuri Sqworm
SuperBot SuperBot
Surfbot SuperHTTP
Suzuran Suzuran
Swiftbot Swiftbot
Szukacz Szukacz
T0PHackTeam
T8Abot T8Abot
Telesoft Teleport
The Intraformant
TheNomad TheNomad
TightTwatBot Thumbor
Trendictionbot Titan
Turingos Toata
TurnitinBot Tracemyfile
Trendiction
TwengaBot TwengaBot
URLy Warning VB Project
UnisterBot
Vacuum Vacuum
Vagabondo
VeriCiteCrawler VeriCiteCrawler
VidibleScraper Voil
Virusdie WEBDAV
WASALive-Bot WISENutbot
WPScan WPScan
WWW-Mechanize WWW-Mechanize
WWW::Mechanize Wallpapers/3.0
Web Collage Web Enhancer
Web Fetch Web Pix
Web Fuck WebCollage
Web Sauger WebImageCollector
Web Sucker WebLeacher
WebBandit WebSauger
WebEnhancer
WebFetch
WebFuck
WebGo IS
WebReaper
WebStripper WebStripper
WebWhacker
Webshag
Website Quester Website Quester
WebsiteQuester WebsiteExtractor
Whacker
Widow
Wonderbot Wonderbot
Wprecon Wprecon
Xenu
YoudaoBot
Zauba
Zermelo Zermelo
Zitebot
ZmEu ZmEu
arquivo.pt arquivo.pt
check1.exe autoemailspider
clark-crawler backlink-check
cognitiveseo coccocbot
crawler.feedback crawler.feedback
crawler4j crawler4j
eCatch eCatch
facebookscraper facebookscraper
gopher heritrix
instabid isitwp.com
internetVista monitor iubenda-radar
pcBrowser linkdexbot
polaris version lwp-request
s1z.ru meanpathbot
satoristudio.net mediawords
scalaj-http muhstik-scan
scan.lol oBot
seocompany.store
seoscanners seoscanners
seostar seostar
sexsearcher sexsearcher
sitechecker.pro ubermetrics-technologies.com
siteripz voyagerx.com
sogouspider webpros.com
sp_auditbot
sysscan
trendiction.de
x22Mozilla
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
akama.com bscodecs.com
amyfoxfitness.com filesvine.com
design-ideas.info hledejvshopech.cz
doeco.ru igru-xbox.net
mozello.ru jurajskie.info
openfrost.com na-telefon.biz
sitebeam.net responsinator.com
tizanidinefibromyalgia.blogspot.com seeingmeerkat.com
webinstantservice.com space4updating.win
world-mmo.com xn--80aodinpgi.xn--p1ai

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 007angels.com
108shot.com 00webcams.com
123cha.com 13tabs.com
1kinobig.ru 1xbet4you.com
1qingdao.com 2020iscoming.info
2drittel.de 2girls1cup-free.com
3-letter-domains.net 45en.ru
3dracergames.com 4kplayer.pl
4webmasters.org 4rent.ru
6128786.com 51unlim.ru
7fon.ru 55wmz.ru
9med.net 5tey7463.weebly.com
aaa38852e886ac4af1a3cff9b47cab6272.com 999webdesign.com
aac52006ec82a24e08b665f4db2b5013f7.com a.frcls.fr
acmebtn.ml aa14ab57a3339c4064bd9ae6fad7495b5f.com
addray.pro adexprts.com
adelly.bg adloads.net
adman.se admanaerofoil.com
adobereader-free.ru admatic.com.tr
adpremium.org admeasures.com
advancedsoftwaresupport.com adnotbad.com
adzpower.com adprotect.net
afftrack001.com adultwebhosting.info
ahmedabadwebs.com advancedmassagebysara.com
ahrntal.verymes.xyz affiliate-fr.com
airlandsea.info agreda.pluto.ro
airmaxshoes-2016.com akama.com
albumsuper.info alfabot.xyz
albuteroli.com algerianembassy.co.in
alf-img.com alive-ua.com
alfa-img.com alt-servis.ru
algarveglobal.com alveris.ru
alyeskaresort.com amazingpic.net
amehdaily.com amazon-adsystem.com
amoi.tn anabolics.shop
animaltoplist.com animalrank.com
apartment.ru anime.dougasouko.com
apibetweenlinesn-a.akamaihd.net animenime.ru
appleid-verification.com anlimebel.kiev.ua
appmsr.org antons-transporte.de
arenda-yeisk.ru aosexkontakte.net
arewater.com aosheng-tech.com
arkkivoltti.net apidiamondatanet-a.akamaihd.net
arpelsreplica.xyz apimountainbikei-a.akamaihd.net
artdeko.info apiqualitinknet-a.akamaihd.net
atlasvkusov.ru apispringsmartne-a.akamaihd.net
autoua.top apiwebwebgetcom-a.akamaihd.net
availit.weebly.com app5.letmacworkfaster.world
avon-novosib.ru appiq.mobi
ayerbo.xhost.ro arius.tech
azarttoto.com artel116.ru
b2b-lounge.com aruplighting.com
babespcs.com asophoto.com
babyfactory.fr asrv-a.akamaihd.net
bagcionderlab.com asscenihotosidea.blogspot.com
asseenontv.ru
azbukadiets.ru
b00kmarks.com
baladur.ru
balkanfarma.ru balkanfarma.ru
balois.worldbreak.com balois.worldbreak.com
barbourjackets.online baltgem.com
beauty-clinic.ru bbsoldes.fr
beget.tech beachdriveblog.com
bestplacetobuyeriacta.jw.lt bidr.trellian.com
betwinservice.com binaryoptionscops.info
biglistofwebsites.com bioscorp.ru
bimmerpost.com bjetjt.com
bitnote.co bkns.vn
bjanshee.ru blackcurranthumidifiers.site
bkgr.se blobar.org
blacktwhite.com blockety.co
blavia.00author.com blogsfunky672.weebly.com
bobba.dzaba.com
bonanza-fortune.men bonanza-fortune.men
bracketsmackdown.com
brainboosting.club
brainboostingsupplements.org
brandedleadgeneration.com brandedleadgeneration.com
bugof.gq brandehk.dk
buy-cheap-pills-order-online.com brateg.xyz
buy-forum.ru brewdom.ru
buy-loft.ru briomotor.co
bvlgariring.xyz bryansk.zrus.org
c-english.ru buketeg.xyz
candelluxsklep.pl bumascloset.com
candycrushshop.com burkesales.com
candytiner.com bus-offer.com
cartierlovestore.xyz candypeople.se
cartierreplica.pw cartierlove2u.xyz
cctva.tv cartierring.xyz
chatroulette.life cartierwatch.xyz
cheap-trusted-backlinks.com casablancamanor.co.za
cheapmoncler.pw cdnanalytics.xyz
cheapmoncler.xyz cdnnetwok.xyz
chelyabinsk.xrus.org celejihad.info
chocolateslim-original.com chatroulette.video
choice-credit.ru cheapestjerseys-wholesale.com
cheapjerseysap.com
cherubinimobili.it
christianlouboutinreplica.pw
christianlouboutinshoes.xyz
civilwartheater.com civilwartheater.com
cjmc.info cleanersoft.com
classicakuhni.ru clevernt.com
clean-virus-mac.com clickintext.net
cleanpcnow.com clippingphotoindia.com
clickaider.com codq.info
clickzzs.nl cognitiveseo.com
collegeessay19.blogspot.ru collegeessay19.blogspot.ru
compiko.info collegerentals.com
continent-e.tv compliance-don.top
copyrightclaims.org compliance-jess.top
cottageofgrace.com compliance-julia.top
countbertwistdisp26.soup.io comsysnet.com
craftburg.ru craftinsta.ru
crest3d.ru cubook.supernew.org
culturevie.info cv.wallhade.co
custodieva.ru czat.wp.pl
cutalltheshit.com daooda.com
damasarenai.info darkbooks.org
dame-ns.kz dataloading.net
daptravel.com
data.vtc.pw
data1.scopich.com
dating-app.ru
dbmkfhqk.bloger.index.hr dbmkfhqk.bloger.index.hr
dcj-nn.ru dealighted.com
delitime.info dms-sw.ru
demenageur.com dogclothing.store
deniven.1bb.ru domain.webkeyit.com
deutschehobbyhuren.net domik-derevne.ru
dfwu1019.info dominterior.org
djstools.com donna7753191.ru
dkvorota.ru doska-vsem.ru
dlya-android.org dostavka-v-ukrainu.ru
dominateforex.ml doublepimp.com
domoysshop.ru drivotracker.com
domznaniy.ru duawitchrarato.tk
driving.kiev.ua dumuelave.xyz
dwomlink.info e-c.al
e90post.com eandsgallery.com
earn-from-articles.com ekto.ee
earthmagic.info electrik-avenue.com
ec-file.info electro-prom.com
ecookna.com.ua elektrozigarette2.webs.com
educatemetv.com elektrozigaretten1.webs.com
ee77ee.com emailaccountlogin.co
election.interferencer.ru embedle.com
elektrischezi.canalblog.com emerson-rus.ru
enhand.se empathica.com
eshop.md en.altezza.travel
essaypro.com enginebay.ru
etur.ru eropornosex.ru
eu-cookie-law.blogspot.com etm-consult.de
exct.net excitacion.info
exhibitionplus.eu expertnaya-ocenka.ru
ezigarettekaufen2.yolasite.com eywords-monitoring-your-success.com
ezigarettenshop2.postbit.com ezigarettekaufen2.blox.pl
f012.de f0815.de
f07.de facebook-mobile.xyz
family1st.ca falcoware.com
fantasticpixcool.com famix.xyz
farmprofi.net fanrto.com
fastcrawl.com filmidivx.com
fba-mexico.com firstdrugmall.ru
feorina.ru fitness-video.net
fickblock18.com flirt4free.com
fidelityfunding.com
financehint.eu
flashahead.info
floridahuntingfishingadventures.com floridahuntingfishingadventures.com
floridamobilebillboards.com fordsonmajbor.cf
fm-upgrade.ru fortwosmartcar.pw
forex-procto.ru
forum.poker4life.ru forum.poker4life.ru
foto-weinberger.at forum69.info
fototravel.eu free-deals.faith
freenom.link free-video-tool.com
front.to freemaintenancesysforpcandmac.top
fullfileaccess.com freewareseek.com
fyl.com.ru freewhatsappload.com
gabeshop.ru fres-news.com
game-mmorpg.net fuckmill.com
gayxperience.com g33.org
gearcraft.us gamewrath.com
gamezblox.com
gateway.zscalerone.net
gay-file.com
genericlowlatencyasiodriverhq.aircus.com genericlowlatencyasiodriverhq.aircus.com
genetworx.com gepezz.info
getdot.ru getmyads24.com
gilsonchiro.xyz getpopunder.com
glicol.kz gigixo.com
good-mummy.ru girlspicsa.com
gopro-online.info gk170.ru
gosreg.amchs.ru glall.ru
graphid.com globalscam.ga
gribokstop.com goodwriterssales.com
growboxbl.ru googleseo.com.tr
gsasearchengineranker.pw greatdealshop.com
gsasearchengineranker.site greenidesign.co
gsasearchengineranker.space
gsmlab.pl gsmlab.pl
guardlink.org gstatey.net
gwagka.com gta-club.ru
guitar-master.org
h2monline.com
harmonyglen.us
hatdc.org
hayate.biz hayate.biz
hcate.com hchha.com
hdapp1008-a.akamaihd.net hdapp1008-a.akamaihd.net
hit-men.men healbio.ru
hobild.net healgastro.com
homemature.net heliko.no
horoshieokna.com hiwibyh.bugs3.com
hotblog.top hotel-mkad.ru
hotchatdate.com hptwaakw.blog.fc2.com
hotkeys.com hyhj.info
hotxnights.info ico.re
housediz.com iinstalll-fii1leis.jus0wil.pp.ua
housekuba.org
housemilan.ru
houtings.xyz
hoverboard360.se
https-legalrc.biz
huimin764128.com
ibmdatamanagement.co
ikearugs.xyz ikearugs.xyz
ilo134uloh.com ilmen.net
immobiliaremassaro.com imagerydatabase.com
impisr.ru imk.com.ua
incanto.in.ua insider.pro
incolors.club insta-add.pro
inet-traffic.com intimshop-fantasy.ru
infoupdate.org investzalog.ru
intellektmedia.at invivo.hu
iptool.xyz ip-guide.com
irunfar.com itbc.kiev.ua
iscblog.info itsdp3.com
istizanidineanarcoticdrug.blogspot.com iwanttodeliver.com
itag.pw jerseyschinabizwholesale.us
izismile.com jerseyswholesalechinalimited.com
j33x.com jerseywholesalebizchina.com
jasonpartington.com jestr.org
jo24news.com kaidalibor.de
johnnyhaley.top kaliningrad.zrus.org
jumptap.com karachev-city.ru
kakadu-interior.com.ua kartiniresto.com
kasino-money.pw kenaba.su
katushka.net keywordsuggests.com
kellyonline.xyz kidd.reunionwatch.com
keywordhouse.com kino-key.info
keywordsuggest.org kino-rf.ru
kichenaid.ru kinobaks.com
kino2018.club kinogolos.ru
kinozapas.com klikbonus.com
knogg.net koleso24.com.ua
kollesa.ru
konoplisemena.com konoplisemena.com
konpax.com konpax.com
kostenloser-sex.com kretpicf.pw
kozhniebolezni.com kskjco.club
kupiproday.com.ua landscaping.center
kw21.org letsart.ru
lacasamorett.com likesuccess.com
lanadelreyfans.us linkrr.com
law-check-eight.xyz listiseltemournlan.gq
lemon-ade.site littlesexdolls.com
letslowbefast.site littlesunraiser.com
lexixxx.com lob.com.ru
lincolntheatre.com localmatchbook.com
liveinternet.ro lol-smurfs.com
lonerangergames.com lolnex.us
lostfilm-online.ru lonely-mature.com
lunchrock.co losangeles-ads.com
m0r0zk0-krava.ru lottospring.com
magicalfind-a.akamaihd.net louboutinreplica.pw
magicplayer-s.acestream.net louboutinreplica.xyz
mall.uk lovi-moment.com.ua
mansiondelrio.co m.b00kmarks.com
mebelcomplekt.ru macdamaged.space
medi-fitt.hu macotool.com
medtherapy.ru magicdiet.gq
meetlocalchicks.com matpre.top
mega-polis.biz.ua me-ke.com
megakino.net medicine-4u.org
megavolt.net.ua megagrabber.ru
mesbuta.info menhealed.net
meteocast.net
mex-annushka.ru
mil-stak.com mil-stak.com
miniature.io mir-limuzinov.ru
minneapoliscopiers.com mirmedinfo.ru
mjchamonix.org mirobuvi.com.ua
mlf.hordo.win missis.top
mobile-appster.ru misslike.ru
mobplayer.net misterjtbarbers.com
monetizationking.net mistr-x.org
money-every-day.com mmstat.com
musezone.ru monclerboots.xyz
monitorwebsitespeed.com
moroccosurfadventures.com
moscow-region.ru
most.gov.iq
moviemail-online.co.uk
mowser.com
mp3ringtone.info
mrinsidesales.com
mrvideospornogratis.xxx
musirc.com musirc.com
muzaporn.com myblogregistercm.tk
my-big-family.com mydirtystuff.com
my-cash-bot.co myindospace.com
my-floor.in.ua
myanyone.net
mydeathspace.com
mylesosibirsk.ru mylesosibirsk.ru
mylida.org najaden.no
mymercy.info
myperiod.club
mypets.by
mysex21.com
myxdate.info
nalogovyy-kodeks.ru
narco24.me narco24.me
netallergy.ru narkologiya-voronezh.ru
netpics.org narutonaruto.ru
newfilmsonline.ru naturtreenspicerx.pw
newhairstylesformen2014.com nbsproject.ru
nicefloor.co.uk nelc.edu.eg
nordstar.pro newstraveller.ru
obiavo.in nfljerseyscheapchinabiz.com
nlfjjunb5.ru
nmrk.ru
northfacestore.online
november-lax.com
odesproperty.com
ohmyrings.com
okayimage.com okayimage.com
olvanto.ru okout.ru
online.ktc45.ru oktube.ru
onlyforemont.ru onlinemeetingnow.com
ooomeru.ru onstrapon.purplesphere.in
orsonet.ru open-odyssey.org
osnova3.ru oralsexfilme.net
outshop.ru orel-reshka.net
oynat.info osagonline.ru
panicatack.com ourtherapy.ru
pastaleads.com oyster-green.com
papasdelivery.ru
partnerline.men
pattersonsweb.com pattersonsweb.com
pc-services.ru
pdfprof.com
pdn-4.com pdn-4.com
pensplan.com penisvergrotendepillennl.ovh
petitions.whitehouse.gov perm-profnastil.ru
phelissota.xyz pharmacyincity.com
phobia.us photo.houseofgaga.ru
photostudiolightings.com
picanalyzer.data-ox.com
piccshare.com piccshare.com
pihl.se pl.id-forex.com
pinstake.com podshipniki-ntn.ru
pinup-casino1.ru pokemongooo.ml
pk-pomosch.ru pooleroadmedicalcentre.co.uk
pobeiranie.pl popander.mobi
porno-raskazy.ru popunder.net
pornodojd.ru popup-jdh.xyz
pornolook.net porn555.com
pornonik.com pornophoto.xyz
pornoslive.net privatefx-in.ru
pornoxxx.com.mx professionaldieselcare.com
pr-ten.de pron.pro
profit-opportunity.com pronorm.fr
profolan.pl propranolol40mg.blogspot.com
pshare.biz prostitutki-almata.org
pulse33.ru prostitutki-yaroslavlya.org
pxhdwsm.com proxyelite.biz
quangcaons.com pyrodesigns.com.au
raavidesigns.com qitt.ru
rabotaetvse.ru rabot.host.sk
rating-casino2021.ru rapidgator-porn.ga
realresultslist.com razvratnoe.org
rebelmouse.com remont-mobile-phones.ru
recipedays.ru renewablewealth.com
redirectme.net rentehno.ru
reelheroes.net replicalouboutin.xyz
remybutler.fr respublica-otel.ru
resultshub-a.akamaihd.net ringporno.com
richinvestmonitor.com rocis.site
rollercoin.com
rome2rio.com
rossanasaavedra.net
rozalli.com rozalli.com
rubanners.com roznica.com.ua
rucrypt.com
running-line.ru running-line.ru
rutor.vip ruscoininvest.company
rvi.biz s1z.ru
safe-app.net
sajatvelemeny.com sajatvelemeny.com
sakhboard.ru salutmontreal.com
sammlungfotos.online scanner-john.top
santasgift.ml searchtooknow-a.akamaihd.net
scanner-brian.top serpstat.com
scrapy.org sexpornotales.net
sdrescher.net sextracker.com
search-error.com sexy-pings.com
seminarygeorgia59.ga shama-rc.net
sergiorossistore.online shitting.pro
service.adtech.fr sierraapps.com
sex.hotblog.top sim-service.net
sexkontakteao.info simpoed.ufop.br
sexreliz.com sirpornogratis.xxx
sexvideo-sex.com site-analyzer.com
shoppingmiracles.co.uk sklad-24.ru
siha.de skladvaz.ru
sispe.com.br smailik.org
siteaero.com smokewithrabbits.com
sitechecker.pro snip.tw
sladkoevideo.com socialbookmarksubmission.org
slim.sellany.ru soft-program.com
smartshoppymac.com softxaker.ru
social-button.xyz sohoindia.net
spamnuker.com solnplast.ru
spb-plitka.ru sonata-arctica.wz.cz
special-porn.com sortthemesitesby.com
sptslmtrafms.com spinazdrav.ru
spywarenuker.com sprttrack.com
sr-rekneskap.no spyfu.com
staff.prairiesouth.ca stackthatbucks.com
sterva.cc stair.registrydomainservices.com
stoki.ru stathat.com
store-rx.com stillmiracle.com
stpolice.com stmassage.ru
streetfire.net stpicks.com
stroiminsk.org stretchmate.net
stroy-portal22.ru stroiminsk.com
styro.ru superfish.com
sukarame.net superstats.com
supermesta.ru swsociety.se
survival.betteroffers.review takeflyte.com
swingerseiten.com tarad.com
syndicate.fun td-l-market.ru
szucs.ru teletype.in
tabakur77.com templates.franklinfire.co
tackletarts.co thebluenoodle.com
tecspb.ru thecoolimages.net
telegraf.by thefds.net
thebitcoincode.com thegioixekhach.com
thenews-today.info thewomenlife.com
theporndude.com thiegs.reco.ws
theprofitsmaker.net timdreby.com
thetardistimes.ovh
thetattoohut.com
thexart.club
titan-cloud.life
tizanidinecapsules.blogspot.com
tizanidinecost.blogspot.com
tizanidinedrugtest.blogspot.com tizanidinedrugtest.blogspot.com
tizanidinehclsideeffects.blogspot.com tizanidinehcl2mgtablet.blogspot.com
tmearegion26.com tizanidinemusclerelaxant.blogspot.com
topappspro.com tomatis.gospartner.com
topsy.com tomck.com
touchmods.fr tonivedu.it
tracxn.com toonfamilies.net
traderzplanet.in torrentgamer.net
tradgardspartner.se toy-shop.top
trafficjunky.net
trafficnetzwerk.de trafficnetzwerk.de
trafficshaper.com traffixer.com
trafficstars.com traxdom.ru
tuckermktg.com trichizobswiv.agddns.net
turizm.bz truck-land.life
tvoystartup.ru tyumen.xrus.org
twojebook.pl uamtrk.com
u555u.info uchebavchehii.ru
ua.tc ugguk.online
ukrtvir.com.ua um-razum.ru
umg-stroy.ru uni.me
unacittaconte.org univerfiles.com
uogonline.com urdoot.win
uptime-as.net usbggettwku.ga
v720hd.ru vabasa.inwtrade.com
vegan-foods.us vapomnoncri.tk
venerologiya.com vchulkax.com
ventelnos.com vegascosmetics.ru
viberdownload10.com video--production.com
videochat.bz videochat.world
videochat.life virtualbb.com
videochat.ph virus-schutzmasken.de
voloslove.ru visitcambridge.org
voprosotvet24.ru vodaodessa.com
vtc.pw w3data.co
vvpg.ru webalan.ru
vzubah.com webnode.me
w3javascript.com webpromotion.ae
watchinf.com website-speed-up.site
watchmygf.net weburok.com
waycash.net wejdz-tu.pl
webenlace.com.ar wemedinc.com
webshoppermac.com westen-z.life
wedding0venues.tk westermarkanjou.se
whengirlsgowild.com white-truck.life
whereverdesperate.gq wholinkstome.com
wonderfulflowers.biz woman-h.ru
wordpress-crew.net word-vorlagen.net
workle.website worldinternetauthority.com
worldlovers.ru worldmusicfests.com
wpsecurity.website wovis.site
www.get-free-traffic-now.com woweb.com.ua
x-diesel.com ws.ampower.me
xgftnlrt.bloger.index.hr www.888.com
xmlinde.com wzgyyq.com
xn----7sbaphztdjeboffeiof6c.xn--p1ai x-stars.ru
xn----ctbbcjd3dbsehgi.xn--p1ai x-true.info
xn----dtbndd4ae7eub.top xcc24.pl
xn--1-8sbcpb0bdm8k6a.xn--p1ai xchat26.myfreecams.com
xjlottery.com
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xn--80abgj3a5acid6ghs.top xn--80abgj3a5acid6ghs.top
xn--80aeahghtf8ac5i.xn--p1ai
xn--80atua3d.xn--p1ai xn--80atua3d.xn--p1ai
xn--b1agm2d.net xolodremont.ru
xvideosbay.com xxxnatelefon.ru
xvideospornoru.com ynymnwbm.bloger.index.hr
xxxdatinglocal.us youandcredit.ru
yaderenergy.ru youtube-downloader.savetubevideo.com
yellowads.men yugk.net
youthreaders.com zajm-pod-zalog-nedvizhimosti.ru
zaidia.xhost.ro zarabiaj-dzis.pl
zaloadi.ru zazagames.org
zastenchivosti.net zdesformula.ru
zeleznobeton.ru zeroredirect7.com
zeroredirect6.com zmoda.hostreo.com
zlatnajesen.com zoogdinsney.com
zvezdagedon.ru
zytpirwai.net zytpirwai.net
zzbroya.com.ua

496
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
108shot.com 1001desert.com
2pxg8bcf.top 202ch.com
7xc4n.com 4ureyesonly.com
9icmzvn6.website 4webmasters.com
a.frcls.fr 8gold.com
actices.com aa81bf391151884adfa3dd677e41f94be1.com
addtoadd.men aaa38852e886ac4af1a3cff9b47cab6272.com
adinterax.com adelly.bg
aero2.ru advancedcleaner.com
alibestsale.com aibolita.com
alphacarolinas.com alert.scansafe.net
altamayoztourism.com allknow.info
amehdaily.com allwidewallpapers.com
amt-k.ru angkortours.vn
animal36.com animaltoplist.com
arabsexxxtube.com anniemation.com
asiengirls.net anonymousfox.co
asrvvv-a.akamaihd.net appsaurus.com
australia-opening-times.com arcadeplayhouse.com
autoblogger24.info artdestshop.eu
avirasecureserver.com asseenontv.ru
babieca.com atdedinotuho.tk
backuperwebcam.weebly.com atelielembrancaqueencanta.com.br
bangla-chat-uk-paralud.ga atley.eu.pn
bannerbank.ru backiee.com
basisches-wasser.net bagcionderlab.com
bcmp.org baoxaydung.com.vn
beatifullandscape.co bear.gotcher.us
bestchoice.cf belinka.com.ua
beyan.host.sk bestadbid.com
bimatoprost-careprost.com binaryoptionscops.info
biographiya.com bluesalt.co
bizru.info bmw-ark.ru
blog.axant.it boanonihaca.tk
bpro1.top boeuklad.com
btc4u.ru bonzibuddi.com
buscarfoto.com burger-tycoon.com
buyessay3.blogspot.ru buypillsorderonline.com
calvet.altervista.org calc-for-credit.ru
candypeople.se callstevens.com
cartierbraceletsreplica.pw cam-kontakte.org
cartierlovestore.xyz cartierreplica.top
castingbank.ru cartiertrinity.xyz
check-host.net civilwartheater.com
cherrypointplace.ca classiquebijoux.ru
children-learningreading.info clickcash.com
cialischmrx.com clippingphotoindia.com
cleanmypc.com club-lukojl.ru
coindirect.io codq.info
compliance-irvin.top coldfilm.ru
compliance-julia.top coop-gamers.ru
compliance-mary.top cottageofgrace.com
compucelunlock.net countercrazy.com
continent-e.tv cyprusbuyproperties.com
cookingmeat.ru d2jsp.org
covetnica.com ddpills.com
cowlmash.com delayreferat.ru
de.zapmeta.com demon-tweeks.com
deniven.1bb.ru devochki.top
dev.citetick.com dichvuvesinhhanoi.com
discovertreasure-a.akamaihd.net dickssportinggoods.com
docsportal.net dieswaene.com
dyt.net dikqlyremy.info
e2click.com domaincheck.io
ebonyporn.site duawitchrarato.tk
ekaterinburg.xrus.org eachdayisagift.review
ekn-art.se eets.net
ekobata.ru elektrischezigarette2.devhub.com
elektrischeziga.livejournal.com enews.tech
elektrozigarettekaufen2.blogse.nl epicbrogaming.com
emailaccountlogin.co erectile.bid
energy-ua.com es5.com
eurosamodelki.ru excaliburfilms.com
ezigarettekaufen2.blox.pl extads.net
f07.de ezigarettenshop2.myblog.de
fakehandbags.xyz ezigaretteshop2.mywapblog.com
fashion-stickers.ru filmgo.ru
fba-mexico.com find1friend.com
finuse.com flagcounter.me
firma-legion.ru floridamobilebillboards.com
flashlarevista.com fortevidyoze.net
flipper.top forum.tvmir.org
free411games.com foto-basa.com
freecamdollars.com freegamesplay.online
fun2cell.net fun-mobi.pl
funtoonez.com g7m.pl
galleryawesome.com gamblingnerd.com
gamebackyard.com games.kolossale.ru
garciniaxt.us gaming-journal.com
getpopunder.com generic-pills-online.com
ghostvisitor.com getaclueamerica.com
gktt.ru hdxnxxtube.mobi
go2jump.org heartofbeijing.blogspot.com
goodhousekeeping.com histats.com
gotcher.us hit-kino.ru
groupmoney.ru hitsbox.info
growshop.es hollywoodactress.info
grupografico-pilar.com.ar housekuba.org
gsasearchengineranker.top hut1.ru
gsmtlf.ru id-forex.com
hamacapty.com ihc2015.info
hazardky.net ilovevitaly.com
heliko.no immobiliaremassaro.com
helpmymacfaster.trade inboxdollars.com
hermesreplica.pw indo-export.ru
hexpilot.com instabid.tech
hop.clickbank.net japfm.com
hotenergy.ru jikoman.info
huhn.altervista.org kamagragelusa.net
igadgetsworld.com kazinogames.lv
imediadesk.com kino2018.club
imperiafilm.ru kinotorka.ru
infostatsvc.com komputernaya-pomosh-moscow.ru
investsuccess.org kosynka-games.ru
iphantom.com kreativperlen.ch
iqoption.pro lacave.ntic.fr
iqupdatetmz.win lafriore.ru
irkutsk.online-podarki.com
izhevsk.zrus.org
kamalsinha.com
keywords-monitoring-success.com
kollekcioner.ru
lancheck.net
landscaping.center landscaping.center
ldrtrack.com legalrc.biz
leboard.ru link.ac
libertybilisim.com lion.playtap.us
lignofix.ua ljusihus.se
lionking-1994.blogspot.com lolnex.us
liveu.infoteka.hu long-beach-air-conditioning.com
mail.allnews24.in lost-alpha.ru
mall.uk lutherstable.org
marketingvici.com maladot.com
matrixalchemy.com masterseek.com
mesto-x.com mebel-ekb.com
mmoguider.ru microstatic.pl
mobilierland.com mnimmigrantrights.net
modnie-futbolki.net mobplayer.ru
morepoweronmac.trade mobstarr.com
muizre.ru mojaocena.com
musicktab.com moncleroutletonline.pw
musicpro.monster moonci.ru
musirc.com muz-tracker.net
mustwineblog.com my-bc.ru
nardulan.com my-cash-bot.co
nhl17coinsforps3.gratisblog.biz my-floor.in.ua
npoet.ru mydearest.co
nuit-artisanale.com myindospace.com
onlywoman.org nalogovyy-kodeks.ru
oxford-book.com.ua nastydollars.com
panchro.co.uk netpics.org
parajumpersstore.online newyorkhotelsmotels.info
park.above.com no-rx.info
pathwhelp.org nofreezingmac.work
patterntrader-en.com nomuos.it
picsfair.com nuup.info
pierrehardysale.online o-o-6-o-o.ru
plastjulgranar.se ochistka-stokov.ru
pogosh.com online247.ml
powitania.pl openfrost.com
promover.org ownshop.cf
pssucai.info paperwritingservice17.blogspot.ru
ptr.ruvds.com partner-high.men
pulseonclick.com perfectplanned.com
qwertty.net php-market.ru
rapidgator-porn.ga pipki.r.acdnpro.com
rcb101.ru pixelrz.com
redhotfreebies.co.uk pk-pomosch.ru
registratciya-v-moskve.ru pourvous.info
reining.lovasszovetseg.hu printingpeach.com
rosbalt.com.ua pxhdwsm.com
royal-betting.net qualitymarketzone.com
sarahmilne.top redirectme.net
servicecenter.co.ua reklama1.ru
servisural.ru relatodelpresente.com.ar
sethrollins.net remontgruzovik.ru
sexkrasivo.net rennlist.com
sextracker.de rff-cfal.info
sexuria.net rfserial.net
sexvideo-sex.com rubanners.com
shopperifymac.com rumamba.com
slimcdn.com runofilms.ru
slowmac.tech saveindex.xyz
solitaire-game.ru scanner-walt.top
spacash.com searchlock.com
spanking.to seccioncontrabajo.com
sstroy44.ru sexfreepornoxxx.com
starwars.wikia.com sexsaoy.com
stevemonsen.com sharebutton.net
strfls.com sharpchallenge.com
sugarlyflex.pw shivafurnishings.com
superiends.org sierraapps.com
supermesta.ru similardeals.net
supers.com.ua site5.com
survival.betteroffers.review siteaero.com
swinginwithme.ru siteripz.net
taximytishi.ru sixcooler.de
tfxiq.com smart-scripts.com
thecoral.com.br sobecjvuwa.com.ru
thecounter.com sockshare.net
thefarmergame.com sotechco.co
tizanidine4mgprice.blogspot.com strv.se
tizanidinecost.blogspot.com subj.ukr-lit.com
tizanidinedrugtest.blogspot.com superfish.com
tizanidinehcl4mgisitanarcotic.blogspot.com superstarfloraluk.com
tizanidinevssoma.blogspot.com surflinksmedical.com
toonfamilies.net svetlotorg.ru
track2.shop svtrd.com
trafaret74.ru terrafootwear.us
trustaffs.com thegalerie.eu
themestotal.com
thetardistimes.ovh
tizanidine4mg.blogspot.com
tizanidinehcl4mgtablet.blogspot.com
tizanidinestreetprice.blogspot.com
topappspro.com
topsiteminecraft.com
tourismvictoria.com
traffic-club.info
trafficgenius.xyz
treasuretrack-a.akamaihd.net
ttsq.fr ttsq.fr
ukkelberg.no tubeoffline.com
ukr-lit.com tversvet.ru
um-razum.ru ua.tc
umityangin.net unrealcommander.biz
unmaroll.ya.ru ural-buldozer.ru
usdx.us vashsvet.com
vedomstvo.net vbtracker.net
veerotech.com video-chat.love
viagra.pp.ua videogamesecrets.com
videositename.com vids18.site
villacoloniale.com vkgaleria.com
villakohlanta.nu voluumtracker1.com
vipsexfinders.com voronezh.xrus.org
vita.com.hr website-datenbank.de
vykup-avto-krasnodar.ru weekes.biz.tc
watracker.net whatistizanidinehclusedfor.blogspot.com
webcamtalk.net wholesalejerseychinaoutlet.com
webmasters.stackexchange.com winx-play.ru
webpromotion.ae xn----7sbho2agebbhlivy.xn--p1ai
website-speed-check.site xtrafficplus.com
wnoz.de xvideospornoru.com
wordpress-crew.net youjizz.com
www.arenda-yeisk.ru youjizz.vc
www.get-free-traffic-now.com yourads.website
x-rates.ru youraticles.pl
xdoza.com yourothersite.com
xn----7sbaphztdjeboffeiof6c.xn--p1ai youtube-downloader.savetubevideo.com
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai zakazfutbolki.com
xn--24-glceagatoq7c2a6ioc.xn--p1ai zaloadi.ru
xn--90acenikpebbdd4f6d.xn--p1ai zastenchivosti.net
xpresscare.ru zeroredirect8.com
xtube.com zhongwenlink.com
yoluxuryevents.com zoogdisany.com
yuarra.pluto.ro zzbroya.com.ua
zaimite.ru
zeroredirect10.com
zhacker.net
zigzog.ru
zvuker.net

101
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -1,104 +1,3 @@
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html> <html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.32 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.09.3339 ### Version: V4.2022.09.3340
### Updated: Thu Sep 8 22:01:26 UTC 2022 ### Updated: Fri Sep 9 10:03:26 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.09.3339 ### Version: V4.2022.09.3340
### Updated: Thu Sep 8 22:01:26 UTC 2022 ### Updated: Fri Sep 9 10:03:26 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##