vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-06 12:40:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.10.3412 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-10-15 22:06:54 +00:00
parent 70f8ae031f
commit 035ef86423
47 changed files with 91273 additions and 91266 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3411 #### Version: V4.2022.10.3412
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3411 #### Version: V4.2022.10.3412
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.10.3411 #### Version: V4.2022.10.3412
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11270
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.10.3411 #### Version: V4.2022.10.3412
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11274
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11274
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11274
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11274
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 1; 10.1.0.28 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 0; 10.1.0.28 0;
127.0.0.1 0; 127.0.0.1 0;

11274
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 1; 10.1.0.28 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 0; 10.1.0.28 0;
127.0.0.1 0; 127.0.0.1 0;

11274
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 1; 10.1.0.28 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 0; 10.1.0.28 0;
127.0.0.1 0; 127.0.0.1 0;

11274
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3411 3412

11274
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11274
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.25.1.144 0; 20.242.54.201 0;

11274
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 1; 10.1.0.28 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 0; 10.1.0.28 0;
127.0.0.1 0; 127.0.0.1 0;

11274
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 1; 10.1.0.28 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 0; 10.1.0.28 0;
127.0.0.1 0; 127.0.0.1 0;

11274
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 1; 10.1.0.28 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 0; 10.1.0.28 0;
127.0.0.1 0; 127.0.0.1 0;

11274
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.25.1.144 0; 20.242.54.201 0;

11274
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.25.1.144 0; 20.242.54.201 0;

11274
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 1; 10.1.0.28 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Aipbot Attach
Boardreader Bullseye
CSHttp PleaseCrawl
CazoodleBot PyCurl
LinqiaMetadataDownloaderBot Surfbot
PageAnalyzer TightTwatBot
SEOstats com.plumanalytics
Sogou web spider gopher
Voltron muhstik-scan
WebsiteQuester page scorer

312
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
404enemy
80legs 80legs
ADmantX ALittle Client
AIBOT
Abonti
AfD-Verbotsverfahren AfD-Verbotsverfahren
AiHitBot
Aipbot Aipbot
Alexibot
AllSubmitter
Alligator Alligator
AlphaBot AlphaBot
Anarchie Apexoo
Anarchy Attach
Anarchy99 AwarioRssBot
Ankit BBBike
Anthill
Aspiegel
AwarioSmartBot
BDCbot
BLEXBot BLEXBot
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass
Bandit Bandit
BatchFTP
Battleztar Bazinga Battleztar Bazinga
Black Hole Black Hole
BlackWidow
Blow Blow
BlowFish
Boardreader
Bolt Bolt
BotALot
Brandprotect Brandprotect
Bullseye BuiltBotTough
BunnySlippers
CATExplorador CATExplorador
CCBot CCBot
CSHttp CODE87
CherryPicker Calculon
Chlooe CazoodleBot
Cogentbot Cegbfeieh
CheeseBot
Citoid
Cliqzbot
CopyRightCheck CopyRightCheck
Craftbot
Crawling at Home Project Crawling at Home Project
Crescent
Custo
CyotekWebCopy CyotekWebCopy
DIIbot DIIbot
DSearch Digincore
DTS Agent DigitalPebble
Disco Dispatch
Discobot
Discoverybot
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot
DomCopBot DomCopBot
DomainAppender DomainAppender
DomainCrawler DomainStatsBot
Dotbot
Download Wonder Download Wonder
Dragonfly
EMail Wolf
EasyDL EasyDL
Ebingbong
Ecxi
EirGrabber EirGrabber
EroCrawler
Express WebPictures Express WebPictures
Extractor Extractor
ExtractorPro EyeNetIE
Extreme Picture Finder FlashGet
FHscan
FemtosearchBot
Firefox/7.0
Flunky Flunky
Foobot Freeuploader
FrontPage
Fuzz
FyberSpider FyberSpider
Fyrebot Fyrebot
GalaxyBot Getintent
Genieo
GetRight
Gigabot Gigabot
Go!Zilla
Gotit Gotit
GrapeFX
GridBot
HEADMasterSEO
HMView HMView
HTMLparser
HTTrack HTTrack
Haansoft HonoluluBot
HaosouSpider Humanlinks
IlseBot HybridBot
IndeedBot IRLbot
Iblog
Image Fetch
Indy Library Indy Library
InfoTekies Intelliseek
Internet Ninja Internet Ninja
IstellaBot Iria
Kenjin Spider JamesBOT
JennyBot
Joomla
Jorgee
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Keyword Density Keyword Density
LWP::Simple Kinza
Leap LeechGet
LexiBot LexiBot
LibWeb LibWeb
LieBaoFast Lightspeedsystems
LinkpadBot LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider
Litemage_walker
Lmspider
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MIIxpc
MJ12bot
MQQBrowser
MSFrontPage
MSIECrawler
Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic-SEO Majestic SEO
MarkMonitor Majestic12
Mediatoolkitbot MauiBot
MicroMessenger Mb2345Browser
Metauri
Microsoft Data Access Microsoft Data Access
Minefield
Mister PiX
Mojeek
MolokaiBot
Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3
Msrabot Msrabot
Musobot NICErsPRO
Nessus NPbot
Navroad
Net Vampire
NetAnts NetAnts
NetLyzer NetMechanic
NetSpider
Netcraft
Netvibes
NextGenSearchBot
Niki-bot Niki-bot
Nimbostratus
Nuclei
Nutch Nutch
Octopus Octopus
Offline Explorer Offline Explorer
OnCrawl Offline Navigator
Openfind OpenLinkProfiler
Openvas
OrangeBot OrangeBot
OrangeSpider
OutfoxBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP PageAnalyzer
PageGrabber PageThing.com
Pandalytics Pi-Monster
Pavuk
Picsearch Picsearch
PictureFinder PictureFinder
Piepmatz Pimonster
Pixray Pixray
Pockey
ProPowerBot
ProWebWalker
PxBroker PxBroker
QueryN Metasearch PyCurl
Quick-Crawler Quick-Crawler
RankFlex RSSingBot
RankingBot Rankivabot
Re-re ReGet
RepoMonkey RebelMouse
RocketCrawler Recorder
RedesScrapy
Ripper
Rogerbot Rogerbot
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOprofiler
SISTRIX
SMTBot
ScanAlert
Scanbot
ScrepyBot
Searchestate
SearchmetricsBot
SemanticJuice
Semrush Semrush
SentiBot SeoSiteCheckup
SiteExplorer SeobilityBot
SiteLockSpider Seomoz
SiteSnagger SiteCheckerBotCrawler
Sitebeam
Siteimprove Siteimprove
SmartDownload
Snake Snake
Snapbot SocialRankIOBot
Sosospider Sosospider
Sottopop SpankBot
SpaceBison
Spbot Spbot
Sqlworm SputnikBot
Sqworm
Steeler
Stripper Stripper
Sucuri
SuperBot
SuperHTTP SuperHTTP
Surfbot SurveyBot
T0PHackTeam Suzuran
Teleport Swiftbot
Telesphorep Telesphorep
Thumbor TightTwatBot
Toata Tracemyfile
Toweyabot Trendiction
TurnitinBot Trendictionbot
Turingos
Turnitin
TwengaBot TwengaBot
Twice URLy Warning
URLy.Warning
UnisterBot UnisterBot
V-BOT Upflow
VB Project VB Project
VCI VCI
Vagabondo Vacuum
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper VidibleScraper
Voltron Virusdie
WEBDAV Voil
WASALive-Bot
WWW-Mechanize WWW-Mechanize
WWW::Mechanize
WWWOFFLE WWWOFFLE
Web Auto Wallpapers
Web Fetch WeSEE
Web Fuck
Web Sucker Web Sucker
WebAuto
WebCopier WebCopier
WebEnhancer WebPix
WebGo IS WebStripper
WebImageCollector WebSucker
WebReaper WebWhacker
Webalta Webalta
Website Quester Webshag
WebsiteQuester WebsiteQuester
WiseGuys Robot Webster
Wonderbot Whack
Xaldon WebSpider Whatweb
Xaldon_WebSpider Widow
YoudaoBot YoudaoBot
Zauba Zauba
Zeus Zitebot
adscanner ZmEu
ZoomBot
archive.org_bot archive.org_bot
check1.exe backlink-check
clark-crawler crawler.feedback
coccocbot crawler4j
com.plumanalytics demandbase-bot
dataforseo.com eCatch
heritrix evc-batch
gopher
internetVista monitor internetVista monitor
ips-agent
isitwp.com
linkdexbot linkdexbot
lwp-request magpie-crawler
mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler
oBot oBot
page scorer
pcBrowser pcBrowser
polaris version s1z.ru
ripz
scalaj-http
seobility
seocompany.store seocompany.store
sitechecker.pro seoscanners
seostar
sexsearcher
siteripz siteripz
sogouspider spyfu
sp_auditbot
trendiction.com
trendiction.de trendiction.de
webmeup-crawler webprosbot
webpros.com
x09Mozilla x09Mozilla
xpymep1.exe x22Mozilla
zauba.io

302
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
ADmantX 80legs
ASPSeek ALittle Client
Abonti
Aboundex Aboundex
Acunetix AhrefsBot
AllSubmitter Alexibot
AlphaBot AlphaBot
Anarchy Anarchie
Anthill Anarchy99
Ankit
Aspiegel
Asterias
Attach Attach
BBBike AwarioSmartBot
BDCbot BDCbot
BDFetch BDFetch
BLEXBot
BackDoorBot BackDoorBot
BackStreet BacklinkCrawler
Barkrowler Bandit
BatchFTP
Battleztar Bazinga
BetaBot BetaBot
Bigfoot
Bitacle Bitacle
Black Hole Blackboard
BlowFish BotALot
Bolt
Brandprotect Brandprotect
BunnySlippers Brandwatch
Buck
BuzzSumo BuzzSumo
CATExplorador
CODE87
CazoodleBot CazoodleBot
Cegbfeieh CensysInspect
CherryPicker CheTeam
Claritybot ChinaClaw
Cocolyzebot Cliqzbot
Collector
CopyRightCheck
Copyscape Copyscape
Cosmos
Craftbot
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Curious Crescent
Custo
CyotekWebCopy CyotekWebCopy
DBLBot DBLBot
DIIbot DIIbot
Demon DatabaseDriverMysqli
Disco Devil
Digincore
Dirbuster
Discobot Discobot
Dispatch
DittoSpyder DittoSpyder
DomCopBot DnyzBot
DomainAppender
DomainCrawler DomainCrawler
Domains Project
Dotbot Dotbot
EasyDL Dragonfly
Ebingbong Drip
Exabot EMail Siphon
EroCrawler
Express WebPictures Express WebPictures
FlashGet ExtractorPro
Fyrebot EyeNetIE
GalaxyBot Foobot
Grabber FyberSpider
GT::WWW
GermCrawler
GetWeb
Getintent
GoZilla
Grafula
GrapeshotCrawler GrapeshotCrawler
GridBot HEADMasterSEO
HMView
HTTP::Lite HTTP::Lite
HTTrack HTTrack
Haansoft
HaosouSpider HaosouSpider
Harvest Hloader
Heritrix IDBTE4M
Iblog
IlseBot IlseBot
Image Fetch
Image Sucker Image Sucker
IndeedBot
InfoNaviRobot InfoNaviRobot
Internet Ninja Internet Ninja
IstellaBot Iskanie
Jbrofuzz JOC Web Spider
JamesBOT
JennyBot JennyBot
JikeSpider
Jorgee Jorgee
JustView JustView
Jyxobot
Kenjin Spider Kenjin Spider
Keyword Density
Kinza
LNSpiderguy LNSpiderguy
Larbin Larbin
Leap
LeechFTP LeechFTP
Lftp Lftp
LibWeb
Libwhisker Libwhisker
Lightspeedsystems
LinkScan
LinkWalker
LinkextractorPro LinkextractorPro
LinkpadBot LinkpadBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey Spider Lipperhey
Litemage_walker Litemage_walker
Lmspider Lmspider
MIDown tool MFC_Tear_Sample
MSFrontPage MQQBrowser
MTRobot MTRobot
Mail.RU_Bot Mag-Net
Magnet
Majestic-SEO Majestic-SEO
Majestic12
MarkWatch MarkWatch
Masscan Mediatoolkitbot
MeanPath Bot
Meanpathbot
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Moblie Safari Minefield
Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner
Mr.4x3
Msrabot Msrabot
Name Intelligence Name Intelligence
Needle Net Vampire
Nessus
NetLyzer NetLyzer
NetMechanic
NetZIP NetZIP
Nettrack Nettrack
Netvibes Netvibes
Nibbler Niki-bot
Nikto Nikto
Nimbostratus Nimbostratus
Not Ninja
Offline Explorer OpenLinkProfiler
Openfind Openfind
OrangeSpider Openvas
OrangeBot
OutclicksBot OutclicksBot
PECL::HTTP OutfoxBot
Page Analyzer PageGrabber
PageAnalyzer Pixray
PageThing.com Pockey
Pandalytics ProWebWalker
PeoplePal Pu_iN
Picscout Pump
Piepmatz
Psbot
PxBroker PxBroker
RSSingBot
RankActive RankActive
RankingBot RankFlex
RankingBot2
Rankivabot Rankivabot
RealDownload RankurBot
RebelMouse Re-re
RepoMonkey ReGet
RocketCrawler Reaper
Recorder
RedesScrapy
Ripper
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler
SEOstats
SISTRIX SISTRIX
Scanbot SMTBot
Scrapy SalesIntelligent
Screaming ScoutJet
ScreenerBot ScrepyBot
SearchmetricsBot SearchmetricsBot
SemanticJuice SeoSiteCheckup
SentiBot Seomoz
SeobilityBot
Shodan
SiteExplorer SiteExplorer
SiteLockSpider Siteimprove
SiteSnagger
SlySearch SlySearch
SmartDownload
Snake
Snapbot Snapbot
Snoopy Snoopy
Sogou web spider Sociscraper
Sosospider Sottopop
Spammen SpaceBison
SpankBot Sqlworm
Spanner Sucker
Spinn3r
Steeler
Sucuri
SuperHTTP SuperHTTP
Surfbot Surfbot
SurveyBot
Suzuran Suzuran
Swiftbot T0PHackTeam
Teleport
TeleportPro
Telesoft
The Intraformant
TheNomad TheNomad
Toata TightTwatBot
Tracemyfile Titan
Trendiction
Trendictionbot Trendictionbot
True_Robot True_Robot
TwengaBot
Twice Twice
Typhoeus Typhoeus
URLy Warning UnisterBot
V-BOT V-BOT
Vacuum VB Project
VelenPublicWebCrawler VelenPublicWebCrawler
Virusdie VeriCiteCrawler
VidibleScraper
Voil
WBSearchBot
WISENutbot
WPScan WPScan
WWW-Mechanize WWW-Mechanize
WWWOFFLE Wallpapers
WallpapersHD Web Auto
Web Enhancer Web Enhancer
Web Fetch Web Sauger
WebCollage
WebCopier
WebEnhancer WebEnhancer
WebFetch WebFetch
WebGo IS WebGo IS
WebImageCollector WebPix
WebLeacher
WebReaper WebReaper
WebSauger WebSauger
WebStripper WebStripper
Webalta Website Quester
Webshag
WebsiteExtractor
WebsiteQuester WebsiteQuester
Whack Webster
Who.is Bot Whacker
WiseGuys Robot Widow
Wonderbot Wonderbot
Wotbox Woobot
YoudaoBot Wprecon
Zade Xaldon WebSpider
Zitebot Xaldon_WebSpider
ZoomBot Zermelo
archive.org_bot Zeus
arquivo.pt ZmEu
backlink-check ZyBorg
adscanner
arquivo-web-crawler
check1.exe check1.exe
coccocbot clark-crawler
cognitiveseo com.plumanalytics
crawler.feedback crawler.feedback
dataforseo.com
domainsproject.org domainsproject.org
eCatch eCatch
gopher
heritrix heritrix
instabid
internetVista monitor
ips-agent
iubenda-radar iubenda-radar
lwp-trivial lwp-trivial
magpie-crawler
netEstate NE Crawler
oBot oBot
pcBrowser ripz
polaris version satoristudio.net
scalaj-http scan.lol
seobility sexsearcher
seocompany.store
sitechecker.pro sitechecker.pro
sysscan sp_auditbot
trendiction.de spyfu
webgains-bot tAkeOut
webprosbot voyagerx.com
x09Mozilla
xpymep1.exe xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
clicksor.com canzoni.ru
compliance-julianna.top energydiet24.ru
czeshop.info frbizlist.com
elektrozigaretten1.webs.com freegamesplay.online
itbc.kiev.ua imagine-ex.co
sethrollins.net mexicosleevegastrectomy.com
silvermature.net muycerdas.xxx
spl63.fr paydayloanslocal.com
teslathemes.com smichovbike.cz
trackzapper.com turizmus.us

916
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com
03p.info 03p.info
101flag.ru 1001desert.com
123locker.com 12u.info
123movies.love 14b.info
1688.com 2000k.ru
1kinobig.ru 28n2gl3wfyb0.ru
202ch.com 3weekdiet.com
2kata.ru 4rent.ru
2nt.ru 4ur.click
3th.co.in 55wmz.ru
5ws.dating-app.ru 5ws.dating-app.ru
91abcw.com 7fon.ru
DomainStatsBot 9med.net
a96527gi.beget.tech aa08daf7e13b6345e09e92f771507fa5f4.com
academiacsmendoza.org abclauncher.com
acads.net academia-nsk.org
adf.ly
adsland.men adsland.men
advocatemsk.ru adultfullhd.com
afmuseum.com adultmeetonline.info
agapovdg.ru adultwebhosting.info
aghanyna.com ahrefs.com
ahmedabadwebs.com aimayangzhi.com
alarmobninsk.ru alert-jdh.xyz
algarveglobal.com
alive-ua.com alive-ua.com
all4invest.ru
alldownload.pw
allmarketsnewdayli.gdn
alloysteel.ru
allproblog.com
allwomen.info allwomen.info
aludecor.info alphahoverboards.com
amazon-adsystem.com android-vsem.org
an-donut.com android4fun.org
anal-acrobats.com
anal-acrobats.hol.es
animali.deagostinipassion.it animali.deagostinipassion.it
anjalika.co.in anniemation.com
apartment.ru apioasisspacenet-a.akamaihd.net
appiq.mobi app-ready.xyz
approvedlocal.co.za appfastplay.com
architecturebest.com appmsr.org
arendakvartir.kz asseenontv.ru
arvut.org audiofree.ru
astrochicks.com autodo.info
atleticpharm.org autonew.biz
aucoinhomes.com aviva-limoux.com
autobudpostach.club avkzarabotok.com
autoplate.info ayakino.net
avitocars.ru azlex.uz
ayodhya.co bagsonsale.online
azazaporn.com baladur.ru
back.dog balkanfarma.ru
backlink4u.net bang-hotties.com
baikaleminer.com bannerswap.com
balitouroffice.com bannertesting.com
bbsoldes.fr beachpics.com
best-seo-solution.com begalka.xyz
best-way.men beremenyashka.com
bestadbid.com bestdooz.com
bestempresas.es
besthoro.ru
bestimagecoollibrary.com
bestmouthwash.club bestmouthwash.club
bestofpicture.com bestsexyblog.com
besttorrentknifta.weebly.com
bestwrinklecreamnow.com bestwrinklecreamnow.com
bidbuy.co.kr bet365.com
bif-ru.info bigcities.org
bingoporno.com binomo.kz
bioinnovate.co
bioscorp.ru
bird1.ru bird1.ru
birzha-truda.eu bizcheapjerseyswholesalechina.com
blog-hits.com biznesluxe.ru
bobba.dzaba.com blogqpot.com
bolitgorlo.net
book-bets.com book-bets.com
bookmaker-bet.com bosefux.esy.es
bukleteg.xyz bracketsmackdown.com
burger-tycoon.com brainzpod.win
business-made-fun.com braip.com.br
butstrap.space bristolhotel.com.ua
buttons-for-your-website.com buigas.00it.com
buketeg.xyz
buy-forum.ru
buyantiviralwp.com buyantiviralwp.com
buyhoverboard.com buytizanidineonlinenoprescription.blogspot.com
buypanicdisorderpill.com bytimedance.ru
cacheimages.com bzero1jewelry.net
candcstuccoandstone.com
candiceloves.us candiceloves.us
candlewooddentalcentre.com.au cardul.ru
candy-glam-hp.com carson.getenjoyment.net
candypeople.se cartierlovebraceletreplica.xyz
canvas.gloverid.site cartierlovestore.xyz
carfax.com.ua cashkitten-a.akamaihd.net
carsdined.org casinorewards.info
cartierreplica.xyz cbb1smartlist12.click
cayado.snn.gr centrumcoachingu.com
cctva.tv cfcl.co.uk
cdncash.org checktext.org
chatroulette.video
cheapbarbour.online
chinajerseyswholesalecoupons.com chinajerseyswholesalecoupons.com
chinese-amezon.com cigarpass.com
choice-credit.ru citetick.com
chorus.terakeet.com citizenclsdriveri7.pen.io
christianlouboutinsaleoutletonline.us cleanallspyware.com
cinemaflix.website clickhype.com
clubfashionista.com clicksotrk.com
coaching-netz.info club-musics.ru
cocyq.inwtrade.com codq.info
codysbbq.com compliance-andrew.xyz
collegerentals.com compliance-brian.xyz
com-onlinesupport.host compliance-don.top
compliance-alex.top
compliance-alexa.top
compliance-brian.top
compliance-checker.info
compliance-ivan.top
compliance-jane.top compliance-jane.top
compliance-mary.top compucelunlock.net
compliance-walter.top computernetworksonline.com
concordexoticrentals.com construmac.com.mx
converse.ddsoldes.fr cookie-law-enforcement-ee.xyz
creditmoney.com.ua coolingoods.com
culmimg.pw copenergo.ru
culturevie.info cougfan.info
custom-electric-guitar.com cracksplanet.com
customerguru.in craftburg.ru
cutt.us creditnation.ru
cwetochki.ru crest-poloski.ru
datsun-do.com czat.wp.pl
daydream-studio.ru day-news.info
dayibiao.com
ddospanel.com ddospanel.com
dikqlyremy.info dev.citetick.com
dimaka.info devochki.top
digest-project.ru
dimkino.ru
dinkolove.ya.ru dinkolove.ya.ru
dnsrsearch.com distver.ru
domainsatcost.com dkvorota.ru
domainsphoto.com docstoc.com
donna7753191.ru dogsrun.net
dorratex.tn domain-tracker.com
dostavka-v-krym.com domaincheck.io
dotnetdotcom.org dostavka-v-ukrainu.ru
draniki.org downloaddy.net
drivotracker.com doxysexy.com
dtnlyss.com drpornogratisx.xxx
dvd-famille.com duplicashapp.com
e-kwiaciarz.pl dvervmoskvu.ru
earncash.com.ua dwomlink.info
edge.sharethis.com dyshagi.ru
efkt.jp e-buyeasy.com
elektrischezigaretten1.blogse.nl ecommerce-seo.com
elektronischezi.livejournal.com ednorxmed.com
engines-usa.com efnor-ac.com
englate.com ekaterinburg.xrus.org
elektrischeziga.livejournal.com
elektrischezigaretten2.beeplog.com
elite-sex-finders.com
elmifarhangi.com
emergencyneeds.org
englishgamer.com
enpolis.ru enpolis.ru
eralph.tk espaceinventoristes.com
essaypro.com
eu-cookie-law.blogspot.com
eugenevaultstorage.com
euromasterclass.ru
evening-dating-club.info
everypony.ru everypony.ru
evogarage.com exchangeit.gq
exchanges-bet.com exonline.info
exct.net eywords-monitoring-your-success.com
executehosting.com f1nder.org
eyelike.com.ua factorynightclub.com
ezigarettekaufen.myblog.de faithe.top
ezigarettenshop2.postbit.com
f-loaded.de
familyholiday.ml
fanpagerobot.com
fapgon.com fapgon.com
faptitans.com farmingworm.com
faracontrol.ir ferretsoft.com
filerockstar298.weebly.com fighrofacciufreesig.ga
filkhbr.com filesvine.com
filmania-x.ru findanysex.com
flauntyoursite.com fingerprintjs.com
flirt4free.com flavors.me
flooringinstallation-edmonton.com floridahuntingfishingadventures.com
fodelsedagspresenter.nu forum-engineering.ru
forms-mtm.ru fr.netlog.com
fotosfotos.eu free-floating-buttons.com
frcls.fr free-stock-illustration.com
free-traffic.xyz freenom.link
free-video-tool.com froggytube.com
freelifetimefuckbook.com
freza-sverlo.ru
fxtips.ru
fxund.us
fyl.com.ru fyl.com.ru
gambarkatabaru.com galleryawesome.com
game300.ru game-top.su
gateway.zscalertwo.net gayxperience.com
gazporno.com gazobeton-p.com.ua
germes-trans.com gdebestkupit.ru
genericlowlatencyasiodriverhq.aircus.com
getaclueamerica.com
getoutofdebtfree.org getoutofdebtfree.org
gg.zzyjxs.com glall.ru
googleseo.com.tr glasof.es
gopro-online.info glass-msk.ru
gowreckdiving.com glcomputers.ru
gpirate.com glopages.ru
gratis-sexkontakte.com goldandcard.ru
greatzip.com gomusix.com
gsasearchengineranker.xyz goodhousekeeping.com
h2monline.com goodhumor24.com
hamilton.ca googst2.ru
hdseriale.pl gorabagrata.ru
healing-dysplasia.ru gourcy.altervista.org
hentaiheroes.com greenidesign.co
hermesreplica.win growmyfunds.ca
hjaoopoa.top growshop.es
hkdiiohi.skyrock.com gsmlab.pl
hoholikik.club gz2.bbsoldes.fr
homik.pw habermetre.com
hosting-tracker.com haikuware.com
houseofrose.com handicapbathtubarea.com
houston-vikings.com hasfun.com
hspline.com hayate.biz
https-legalrc.biz hetmanship.xyz
humanelydrew.com hit-men.men
ie.57883.net home.myplaycity.com
iflycapetown.co.za hoste.octopis.com
ikritikimou.gr hoverboard360.at
ilmexico.com hustoon.over-blog.com
imagefinder.site i-midias.net.br
imagez.co iblogpress.xyz
inet-traffic.com idc.com.ua
infospot.pt ideawheel.com
imagine-ex.co
impotentik.com
infokonkurs.ru
infowarcraft.ru infowarcraft.ru
inspiring-desperate.tk inome.com.ua
instasexyblog.com instakink.com
ipornox.xxx int.search.mywebsearch.com
iqoption-bin.com interior-stickers.ru
iradiology.ru investpamm.ru
ix20.ru inzn.ru
j-times.ru irkutsk.zrus.org
jav-fetish.com istripper.com
johannesburgsingles.co.za itbc.kiev.ua
johnnyhaley.top iweblist.info
joyceblog.top javlibrary.cc
jyrxd.com javstock.com
kareliatobacco.ru jillepille.com
karpun-iris.ru jofucipiku.tk
kerei.ru johnrobertsoninc.com
kerwinandcariza.com justkillingti.me
keyword-suggestions.com kakadu-interior.com.ua
keywords-monitoring-success.com karaganda.xkaz.org
kichenaid.ru kchaxton.com
kinomaniatv.pl kino-key.info
kometars.xyz klikbonus.com
kostenloser-sex.com kliksaya.com
krasivye-devushki.net klosetkitten.com
krasula.pp.ua klumba55.ru
kursy-ege.ru kovrenok.ru
landscapebackgrounds.blogspot.com krynica.info
legalrc.biz ku6.com
kumuk.info
ladsup.com
lapitec.eu
larchik.net
larger.io
laxob.com
lemon-ade.site
lenpipet.ru lenpipet.ru
letmacwork.world lenvred.org
lflash.ru letmacworkfaster.world
likrot.com levaquin750mg.blogspot.com
lineavabit.it lion.bolegapakistan.com
linkbolic.com lion.misba.us
linkbuddies.com liquimondo.com
linksharingt.com littlesunraiser.com
liveshoppersmac.com liumimgx.pw
loadingpages.me live-sexcam.tk
locimge.pw liveu.infoteka.hu
lob.com.ru
locksmith.jp
lonfon.xyz lonfon.xyz
lsex.xyz love-baby.cz
luchshie-topcasino.ru mail.allnews24.in
lyrster.com makis.nu
madot.onlinewebshop.net
mafcards.ru
mall.uk mall.uk
malls.com maslenka.kz
mamasuper.prom.ua matematikus.info
marcoislandvacations.net meddesk.ru
marinetraffic.com medtherapy.ru
marmotstore.online megaapteka.ru
masserect.com mesto-x.com
matb3aa.com mex-annushka.ru
medictube.ru mhtr.be
meduza-consult.ru minyetki.ru
megavolt.net.ua mirmedinfo.ru
mensandals.xyz missvietnam.org
microsoftportal.net misswell.net
microstatic.pl mlf.hordo.win
mikozstop.com mlvc4zzw.space
mirobuvi.com.ua
mmog-play.ru
mobifunapp.weebly.com
mobilemedia.md
mobilierland.com
mobplayer.ru
mobstarr.com
modabutik.ru modabutik.ru
moivestiy.biz moi-glazki.ru
mojowhois.com mojpregled.com
moneyzzz.ru monclercoats.xyz
moonci.ru moncleroutletonline.pw
moscow.online-podarki.com monetizer.com-01.site
movpod.in moroccosurfadventures.com
moscow-region.ru
mtmtv.info
mug-na-chas-moscow.ru mug-na-chas-moscow.ru
mundoaberrante.com myhealthcare.com
muz-baza.net
mwtpludn.review
mybinaryoptionsrobot.com
mylovelibrabry.com
myseoconsultant.com myseoconsultant.com
mysex21.com myshopmatemac.com
mysexpics.ru nastroyke.net
nationalbreakdown.com
nerudlogistik.ru
new-post.tk new-post.tk
newstudio.tv nfljerseyscheapchinabiz.com
nextbackgroundcheck.gq njkmznnb.ru
nikitabuch.com nodup.ru
no-fuel.org nordstar.pro
norththeface.store oconto.ru
noumeda.com offtime.ru
novosti-hi-tech.ru one.net.in
nuup.info online7777.com
nvssf.com onlywoman.org
of-ireland.info openfrost.net
onclkads.com openlibrary.org
onlinemegax.com ops.picscout.com
onlineslotmaschine.com ordernorxx.com
onlinetvseries.me origin-my.ru
onlinewritingjobs17.blogspot.ru orion-v.com
orsonet.ru ostrovtaxi.ru
oyster-green.com ozoz.it
paphoselectricianandplumber.com paydayonlinecom.com
partner-pop.men
partner-trustworthy.men
partybunny.ru
patterntrader-en.com
pc-services.ru
pc4download.co pc4download.co
pepperstyle.ru pepperstyle.ru
percin.biz.ly photochki.com
perm.xrus.org piccdata.com
pihl.se picphotos.net
pills24h.com pixgood.com
pixshark.com plastweb.ru
playboyfiles.xblog.in play.leadzupc.com
porno-chaman.info pliks.pl
ploenjitmedia.azurewebsites.net
poisk-zakona.ru
porn555.com
pornodojd.ru
pornokorol.com
pornonik.com pornonik.com
pornosmola.info pornosmola.info
pornstartits.xblog.in portadd.men
poshiv-chehol.ru potoideas.us
predominant-invent.tk prizefestival.mobi
prefersurvey.net proftests.net
pro-okis.ru promalp-universal.ru
prosmibank.ru promoforum.ru
prospekt-st.ru pron.pro
prostitutki-astana.org pronekut.com
prostitutki-tolyatti.org pufip.com
pulseonclick.com purchasepillsnorx.com
quickloanbank.com putevka24.ru
rangoman.date qitt.ru
ranksonic.com qpypcx.com
rating-bestcasino.com quelle.ru
rczhan.com raavidesigns.com
reckonstat.info radiodigital.co
recordpage-a.akamaihd.net reeyanaturopathy.com
registry-cleaner.net refudiatethissarah.info
reining.lovasszovetseg.hu rentehno.ru
reklama-i-rabota.ru rewardit.com
remont-comp-pomosh.ru richinvestmonitor.com
research.ifmo.ru rockma.se
resellerclub.com roleforum.ru
rn-to-bsn.com royalcar-ufa.ru
rossmark.ru rozalli.com
royalvegascasino.com ru-dety.ru
rvtv.ru sabizonline.com
ryetaw.com sad-torg.com.ua
sandhillsonline.com samolet.fr
saneitconsulting.com sampleletters.net
sape.top scanner-elena.top
sarafangel.ru scanner-fred.top
saratov.xrus.org scenarii-1-sentyabrya.uroki.org.ua
savememoney.co.za schlampen-treffen.com
scalerite.co.za scmor.ilxc.cc
scanmyphones.com scrnet.biz.ua
scanner-nelson.top seoexperimenty.ru
sdsjweb.com servicecenter.co.ua
seadragonherbery.com sex-dating.co
seccioncontrabajo.com sexyebonyteen.com
securityallianceservices.com
segol.tv
seo-tools-optimizing.com
seofied.com
seoholding.com
seorankinglinks.xyz
sevendays.com.ua
sexkvartal.com
sexsaoy.com
sharebutton.to
shareyards.com
sheerseo.com sheerseo.com
shillyourcoins.com shivafurnishings.com
shopcheermakeup.info
signoredom.com signoredom.com
simon3.ru
simple-image.com.ua simple-image.com.ua
sisi-go.ru sitevaluation.org
sixcooler.de sladkoevideo.com
skladvaz.ru slavkokacunko.de
slowmac.tech sluganarodu.ru
smadihome.com soblaznu.net
smart-balancewheel.com social-fun.ru
smartpet.ru socialbookmarksubmission.org
sneakyboy.com soda.media
snip.to sodexo.com
snts.shell-pmr.ru spammen.de
soc-proof.su spywarenuker.com
socialmonkee.com srecorder.com
sortthemesitesby.com stackthatbucks.com
sovetogorod.ru stats-public.grammarly.io
speechfoodie.com steamoff.net
sports-supplements.us stockquotes.wooeb.com
sptslmtrafms.com students-cheapskate.ml
spylog.com styro.ru
spywareit.com
spywarespy.com
srdrvp.com
start.myplaycity.com
steelmaster.lv
stjamesschool.info
strv.se
success-seo.com success-seo.com
sudexpert66.ru sundrugstore.com
suhanpacktech.com supermama.top
superlist.biz supermodni.com.ua
supermesta.ru superoboi.com.ua
superstarfloraluk.com surfbuyermac.com
surffoundation.nl svensk-poesi.com
tattomedia.com szucs.ru
tattoo33.ru taranerymagesswa.blogspot.com
td-l-market.ru teastory.co
telefonsexi.com telefonsexi.com
terrafootwear.us telefonsexsofort.tk
thebitcoincode.com templates.franklinfire.co
thebluenoodle.com tengohydar.tk
thedownloadfreeonlinegames.blogspot.com thebestphotos.eu
theheroes.ru thebluffs.com
thin.me.pn thefarmergame.com
timhost.ru thenews-today.info
tizanidine4mgstreetprice.blogspot.com timecrimea.ru
tizanidineingredients.blogspot.com torrentred.games
tizanidinewithdrawal.blogspot.com torrents.life
tkanorganizma.ru totu.info
torrents-tracker.com tovaroboom.vast.ru
trafaret74.ru trafficreceiver.club
traffixer.com traffictrade.life
trahvid.com truck-land.life
tubeoffline.com tube8.com
turist-strani.ru
twincitiescarservice.com twincitiescarservice.com
twojebook.pl twodollarshows.com
ublaze.ru ublaze.ru
uggbootsoutletsale.us ukrtvir.com.ua
uhod-za-sobakoj.ru uogonline.com
underthesite.com urlcut.ru
unimodemhalfduplefw.pen.io usacasino.com
unitexindia.com validdomain.xyz
valmetrundan.se vasileostrovsky-rayon.ru
valsalud.com vavilone.com
vchulkah.net
vclicks.net vclicks.net
vektorpress.ru vegan-foods.us
velobikestock.com veles.shop
vesnatehno.ru venerologiya.com
video-chat.in vesnatehno.com
vip.51.la viktoria-center.ru
visa-pasport.ru vkontarkte.com
vk-mus.ru voltrknc1.com
vladhistory.com voluumtrk.com
vmnmvzsmn.over-blog.com vpnhowto.info
vodaodessa.com warezaccess.com
vriel.batcave.net web-betting.ru
vzubah.com weboptimizes.com
wallpapersist.com website-speed-up.top
wareseeker.com websites-reviews.com
waycash.net weclipart.com
webmonetizer.net weekes.biz.tc
website-stealer.nufaq.com wemedinc.com
wesharepics.com williamrobsonproperty.com
wesharepics.info wineration.com
westen-v.life witclub.info
whatistizanidine2mg.blogspot.com witherrom55.eklablog.fr
win-spy.com wnhjavlhezp.gq
withstandingheartwarming.com wordpresscore.com
wurr.voila.net workle.website
x-mix.info worldhistory.biz
x-stars.ru wormix-cheats.ru
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai wosik-dach.service-for-web.de
xn--b1adccaf1bzj.xn--p1ai wowas31.ucoz.ru
xn--oogle-wmc.com wrz0iuebwhp5fg.freeddns.com
xsfetish.org wtsindia.in
x-lime.com
xcc24.pl
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
xn----7sbahjd3btneuw1joc.xn--p1ai
xn----7sbho2agebbhlivy.xn--p1ai
xn----9sbebi2bvzr7h.xn--p1ai
xn--80aikhbrhr.net
xn--90acenikpebbdd4f6d.xn--p1ai
xnxxandxvideos.com
xtube.com
xxxhdvideo.site
xxxkaz.org xxxkaz.org
xxxsiterips.xyz xz618.com
youbrainboost.asia yachts-cruise.info
yougotanewdomain.com yogamatsexpert.com
youporn-ru.com yottos.com
ypmuseum.ru youresponsive.com
zawyna.ua youthreaders.com
zed21.net yuarra.pluto.ro
znakomstvaonlain.ru zakon-ob-obrazovanii.ru
zona-aqua.ru zdesoboi.com
zeroredirect9.com
zixizop.net.ru
zoosexart.com
zscalertwo.net
zzlgxh.com

496
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
0912701309f8ce.com 00go.com
214jaluwobapef.cf 1flag.co.za
2pxg8bcf.top 2020iscoming.info
83net.jp 202ch.com
8si.ru 4k-player.pl
aa8780bb28a1de4eb5bff33c28a218a930.com 4kepics.com
addblueoff.com.ua 5-steps-to-start-business.com
adserver-e7.com aa9d046aab36af4ff182f097f840430d51.com
adsloads.com aaa38852e886ac4af1a3cff9b47cab6272.com
adultfullhd.com aac500b7a15b2646968f6bd8c6305869d7.com
advancedmassagebysara.com aac52006ec82a24e08b665f4db2b5013f7.com
advokateg.ru adpremium.org
affordablewebsitesandmobileapps.com agahinameh.com
agecheckadult.com aibolita.com
agysya.ru alessandraleone.com
allblogroll.com aliexpresscashback.club
allnews.md allbizne.co.ua
alveris.ru amazon-adsystem.com
android4fun.org apiallgeniusinfo-a.akamaihd.net
aosexkontakte.net app5.letmacworkfaster.world
apelsinnik.website asacopaco.tk
atlasvkusov.ru asseenontvonline.ru
ayeartoforget.com auto-moto-elektronika.cz
balkanfarma.org autoplate.info
balois.worldbreak.com baersaratov.ru
beclean-nn.ru bala.getenjoyment.net
bestpriceninja.com baltgem.com
betshuckclean.com bankiem.pl
bitporno.sx batanga.net
blogqpot.com best-deals-products.com
bobinoz.com bezcmexa.ru
book-bets.com bikini-image.com
brainboostingsupplements.org bio-japan.net
brianhenry.co blackcurranthumidifiers.site
carrauterie.be blog.koorg.ru
cartujano-pre.de bloggedporn.com
chlooe.com boanonihaca.tk
christianlouboutinsaleoutletonline.us bookmark4you.com
cineacademy.ru boom-celebs.com
cjs.com.ru bumascloset.com
click2pawn.com buqyxa.rincian.info
clickcash.com burkesales.com
clippingphotoindia.com butstrap.space
compliance-alexa.xyz buyhoverboard.com
conocer-sanabria.com buytizanidineonline.blogspot.com
constantaservice.net canrioloadm.gq
coolbar.pro carson.getenjoyment.net
cordstrap.cc cashkitten-a.akamaihd.net
crafthubs.com check-host.net
credit-online.ws chee-by.biz
creditnation.ru children-learningreading.info
cwetochki.ru cienum.fr
data-mining.tk cntravelre.com
davidovic.info codysbbq.com
dbmkfhqk.bloger.index.hr counter.bloke.com
devochki.top customcatchcan.com
dimkino.ru cyclobenzaprinevstizanidine.blogspot.com
dirtpics.pw dandiyabeats.in
dividendo.ru defenderxtactical.com
dogclothing.store dogbreedspicture.net
domashneeruporno.com drhomes.biz
dominterior.org drillsaw.ru
donna7753191.ru editors.choice6912650.hulfingtonpost.com
dprtb.com ednorxmed.com
earn-from-articles.com elexies.info
elektrischezigarette2.devhub.com ereko.ru
elektronischezigarettekaufen2.tumblr.com es5.com
elite-sex-finders.com esnm.ru
elvel.com.ua eugenevaultstorage.com
enpolis.ru expertnaya-ocenka.ru
epngo.bz extractorandburner.com
exci.ru ezigarettekaufen.myblog.de
failingmarriege.blogspot.com ezigarettenkaufen2.dreamwidth.org
faithe.top ezigarettenshop2.postbit.com
fetlifeblog.com ezigaretteshop2.mywapblog.com
filmbokep69.com fandlr.com
financehint.eu finemanteam.com
finansov.info formseo.com
forex-indextop20.ru forum69.info
free-social-buttons.com franch.info
freelotto.com free-fbook-traffic.com
friendflnder.com free-share-buttons.top
fyxabomiw.ru fsakhalin.ru
gazoblok.net.ua gambarkataku.co
gazporno.com gamevalue7.weebly.com
gearsadspromo.club gardene.ru
getlamborghini.ga gateway.zscalerone.net
ghernnqr.skyrock.com gay.adultgalls.com
given2.com gelezki.com
googst2.ru ggiaro.com
gorabagrata.ru gox.com.ua
haikuware.com guildebzh.info
hatdc.org gymi.name
hebr.myddns-flir.com hamacapty.com
hifidesign.ru happy.new.yeartwit.com
histats.com hard-porn.mobi
hit-men.men heartofbeijing.blogspot.com
hjaoopoa.top himazin.info
hobild.net hiwibyh.bugs3.com
houseofgaga.ru hkdiiohi.skyrock.com
i-service.kz hmywwogw.bloger.index.hr
ibb.com.ua homeinns.com
imk.com.ua hoste.octopis.com
incomekey.net hotel-mkad.ru
infostatsvc.com hotxnights.info
interesnie-faktu.ru houston-vikings.com
islamtoday.co.za huhn.altervista.org
istizanidineanopiate.blogspot.com iboss.com
jav-idol.com id-forex.com
jikoman.info impresagaia.it
journeydownthescale.info infobabki.ru
kakadu-interior.com.ua internetproviderstucson.com
kino-ecran.ru investmac.com
kinoflux.net inzn.ru
krafte.ru iweblist.info
krasnodar.ru javstock.com
labvis.host.sk justucalling32211123456789.tk
letslowbefast.today keywords-monitoring-success.com
lflash.ru keywordsuggests.com
li-er.ru kidskunst.info
lion.playtap.us kongruan.com
magnetpress.sk koral.se
makis.nu koronirealestate.gr
manifestation.betteroffers.review lampokrat.ws
manifestyourmillion.com letslowbefast.site
masturbate.co.uk lifebyleese.com
mebel-vstroika.ru livejasmin.com
meddesk.ru localflirtbuddies.com
member-quiz.com locustdesign.co
metascephe.com luckyshop.net.ua
lyrster.com
magento-crew.net
magicalfind-a.akamaihd.net
mantramusic.ru
mastershef.club
mathgym.com.au
max-p.men
mebelcomplekt.ru
medictube.ru
meteocast.net meteocast.net
methodsmarketing.com millionare.com
mexicotravelnet.com mirmedinfo.ru
michaelkorsoutletstore.net mixed-wrestling.ru
minegam.com mobile-appster.ru
minharevisao.com moncleroutletonline.win
mmog-play.ru moviezbonkerssk.cf
mojpregled.com mp3s.club
mp3ritm.top mwtpludn.review
mrwhite.biz naj-filmy24.pl
musicvidz.ru networkcheck.xyz
muztops.ru newsperuse.com
mydeathspace.com o00.in
mymercy.info odywpjtw.bloger.index.hr
narco24.me office2web.com
narosty.com okmedia.sk
natali-forex.com onescreen.cc
naval.jislaaik.com oops-cinema.ru
nerudlogistik.ru origin-my.ru
newstudio.tv
nw-servis.ru
ofanda.com
ok.ru
ooomeru.ru
open-odyssey.org
openmediasoft.com
osagonline.ru osagonline.ru
paclitor.com paramountmarble.co.uk
panchro.xyz physfunc.ru
pateaswing.com pinwallpaper.top
patterntrader-en.com pons-presse.com
pc-services.ru pornoreino.com
pewit.pw poshiv-chehol.ru
photo.houseofgaga.ru pourvous.info
pl-vouchers.com pr0fit-b0x.com
playlott.com preparevideosafesystem4unow.space
poiskzakona.ru princevc.com
porno-raskazy.ru privatbank46.ru
pornogig.com proftests.net
potolokelekor.ru push-ad.com
progress-upakovka.ru r-e-f-e-r-e-r.com
prostitutki-almata.org razyboard.com
prostitutki-belgoroda.org realitykings.com
proxyelite.biz referencemoi.com
ptr.ruvds.com refererx.com
putitin.me rock-cafe.info
qld10000.net rootandroid.org
rczhan.com rospromtest.ru
real-time-analytics.com s8-nowy-wygraj.comli.com
redhotfreebies.co.uk salutmontreal.com
remmling.de seasonvar.ru
renecaovilla.online securesmrt-dt.com
restorator-msk.ru seinterface.com
russian-postindex.ru seo-tools-optimizing.com
sammyweaver.com seorank.info
santasgift.ml seorussian.ru
scanhub.ru serpstat.com
scanner-jessica.top sexy-screen-savers.com
scanner-marwin.top shakhtar-doneck.ru
screpy.com shijian.ac.cn
se.bnt-team.com sispe.com.br
searchtooknow-a.akamaihd.net slomm.ru
seasaltwithfood.com slowmacfaster.trade
sex-sex-sex5.com smadihome.com
shopvilleroyboch.com.ua snow.nvr163.com
sim-service.net social-s-ggg.xyz
simplesite.com softtor.com
sitopreferito.it souvenir.cc
slavic-magic.ru star61.de
slujbauborki.ru steelmaster.lv
sms2x2.ru streha-metalko.si
sneakyboy.com stroicol.net
snowplanes.com success-seo.com
social-buttons.com summerlinhomes411.info
socialbuttons.xyz survival.betteroffers.review
speedup-my.site t3chtonic.com
spinnerco.ca tastyfoodideas.com
suchenindeutschland.com tattooha.com
superkanpo.com tehngr.ru
susanholtphotography.com telefonsexkostenlos.tk
tandvardshuset.net thegalerie.eu
teenporn18.net thegameriders.com
thedownloadfromwarez.blogspot.com thfox.com
titslove.yopoint.in time-japan.ru
tizanidineingredients.blogspot.com titan-cloud.life
top10-way.com tizanidine4mgstreetvalue.blogspot.com
tramadolandtizanidine.blogspot.com tizanidinedrugtest.blogspot.com
truebeauty.cc tootoo.to
twinderbella.com topbestgames.com
twu.com.ua torrentgamer.net
ukrobstep.com torrentred.games
unitexindia.com totu.us
universals.com.ua toxicwap.com
unrealcommander.biz turbabitload.weebly.com
uroffer.link turkeyreport.tk
vacuumscleaner.com twojebook.pl
valmetrundan.se tytoona.com
versaut.xxx-cam.webcam urlopener.blogspot.com.au
voditeltrezviy.ru v720hd.ru
vrotike.ru varikoz24.com
waycash.net vchulkax.com
wedding-salon.net volgograd.xrus.org
wildworld.site vysigy.su
wleuaprpxuvr.ga vzlomtw.com
woman-h.ru we-are-gamers.com
worst-sites.online website-analytics.online
ww2awards.info weclipart.com
wwwadultcheck.com westvilletowingservices.co.za
wygraj-teraz.com wikes.20fr.com
xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks worldoffiles.ru
xn--80abgj3a5acid6ghs.top wufak.com
yellowfootprints.com xxx-treker.ru
youjizz.com xz618.com
youradexchange.com yeartwit.com
yourporn.com yopoint.in
youtubologia.it zazagames.org
zarabotat-na-sajte.ru zeroredirect.com
zarenica.net znakom.sibtest.ru
zebradudka.com
zhuravlev.info

41
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,23 +8,6 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -494,6 +477,20 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -584,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.140 0; 10.1.0.28 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3411 ### Version: V4.2022.10.3412
### Updated: Sat Oct 15 10:03:24 UTC 2022 ### Updated: Sat Oct 15 22:01:09 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3411 ### Version: V4.2022.10.3412
### Updated: Sat Oct 15 10:03:24 UTC 2022 ### Updated: Sat Oct 15 22:01:09 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##