vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.09.2610 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-09-24 10:09:14 +00:00
parent 02933a2be4
commit 87d01cd2bb
47 changed files with 85627 additions and 85617 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2609 #### Version: V4.2021.09.2610
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2609 #### Version: V4.2021.09.2610
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.09.2609 #### Version: V4.2021.09.2610
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10566
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.09.2609 #### Version: V4.2021.09.2610
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10570
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10570
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10570
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10570
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 1; 10.1.0.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 0; 10.1.0.96 0;
127.0.0.1 0; 127.0.0.1 0;

10570
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 1; 10.1.0.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 0; 10.1.0.96 0;
127.0.0.1 0; 127.0.0.1 0;

10570
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 1; 10.1.0.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 0; 10.1.0.96 0;
127.0.0.1 0; 127.0.0.1 0;

10570
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2609 2610

10570
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10570
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.158.224.31 0; 52.183.126.138 0;

10570
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 1; 10.1.0.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 0; 10.1.0.96 0;
127.0.0.1 0; 127.0.0.1 0;

10570
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 1; 10.1.0.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 0; 10.1.0.96 0;
127.0.0.1 0; 127.0.0.1 0;

10570
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 1; 10.1.0.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 0; 10.1.0.96 0;
127.0.0.1 0; 127.0.0.1 0;

10570
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.158.224.31 0; 52.183.126.138 0;

10570
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.158.224.31 0; 52.183.126.138 0;

10570
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 1; 10.1.0.96 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
GrabNet AiHitBot
LinkextractorPro CrazyWebCrawler
Msrabot DatabaseDriverMysqli
Nuclei Discobot
ProWebWalker ExtractorPro
SiteLockSpider Linkdexbot
SputnikBot MTRobot
Voil Morfeus Fucking Scanner
Web Auto Sqlmap
Webster Zauba

300
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
80legs
AIBOT AIBOT
ASPSeek ALittle Client
Abonti
Aboundex Aboundex
Aboundexbot AhrefsBot
Apexoo AiHitBot
AllSubmitter
AlphaBot
Anarchie
Anarchy99
Ankit
Anthill
Asterias Asterias
BBBike AwarioRssBot
BDCbot BDCbot
BackDoorBot BackDoorBot
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler
Badass Badass
BatchFTP BatchFTP
Battleztar Bazinga BetaBot
Bigfoot
Black Hole Black Hole
Blow BlackWidow
BlowFish Brandprotect
Bolt
Brandwatch Brandwatch
Bullseye BuiltBotTough
BuzzSumo BuzzSumo
CCBot CATExplorador
CODE87 CODE87
CazoodleBot
CheTeam CheTeam
ChinaClaw
Chlooe Chlooe
Cloud mapping Claritybot
Cocolyzebot Copier
Collector Copyscape
CopyRightCheck CrazyWebCrawler
Crescent
CrunchBot
Custo Custo
DataCha0s CyotekWebCopy
Demon DIIbot
Deusu DTS Agent
Dirbuster DigitalPebble
Disco
Discobot
Dispatch
DnyzBot
DomCopBot
DomainAppender
DomainCrawler DomainCrawler
DomainStatsBot Dragonfly
Domains Project Ebingbong
Drip EroCrawler
EasyDL
Ecxi
Evil Evil
Express WebPictures Exabot
Extractor ExtLinksBot
ExtractorPro
Ezooms Ezooms
FDM FDM
FemtosearchBot
Firefox/7.0
FlashGet FlashGet
FyberSpider Foobot
FrontPage
Fuzz
Fyrebot
GT::WWW GT::WWW
GalaxyBot GalaxyBot
GetRight GetRight
GetWeb Getintent
Go-Ahead-Got-It Go!Zilla
GoZilla
GrabNet GrabNet
Grabber
Grafula Grafula
GrapeFX
GrapeshotCrawler GrapeshotCrawler
GridBot HTTP::Lite
HTMLparser
HTTrack HTTrack
Haansoft Haansoft
HaosouSpider
Harvest Harvest
Humanlinks Havij
HybridBot
IDBTE4M IDBTE4M
Id-search
IlseBot
Image Fetch Image Fetch
IndeedBot
Indy Library Indy Library
InfoTekies
Internet Ninja
InternetSeer
Iria
Iskanie
IstellaBot IstellaBot
JikeSpider JamesBOT
Joomla Jbrofuzz
Jorgee Jorgee
Jyxobot JustView
Kinza Kozmosbot
LNSpiderguy LNSpiderguy
Leap LWP::Simple
LeechGet Larbin
LexiBot LexiBot
Libwhisker LibWeb
LieBaoFast
Lightspeedsystems
Likse
LinkScan LinkScan
Linkdexbot Linkbot
LinkextractorPro LinkextractorPro
LinkpadBot LinqiaMetadataDownloaderBot
Litemage_walker LinqiaRSSBot
Lmspider Lmspider
Mag-Net MIDown tool
Magnet MJ12bot
Majestic SEO MSFrontPage
MauiBot MSIECrawler
Majestic-SEO
Mass Downloader
MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru
MicroMessenger MicroMessenger
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Moblie Safari
Mojolicious Mojeek
Msrabot Mr.4x3
NICErsPRO
NPbot
Name Intelligence Name Intelligence
NearSite
Needle Needle
Net Vampire Nessus
NetAnts NetAnts
NetLyzer NetMechanic
NetSpider NetZIP
Netcraft
Nettrack
Niki-bot Niki-bot
Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Ninja Octopus
Nmap Offline Explorer
Nuclei Offline Navigator
Nutch OpenLinkProfiler
OpenVAS OpenVAS
Openfind
PECL::HTTP PECL::HTTP
PHPCrawl PageGrabber
POE-Component-Client-HTTP
PageAnalyzer
PageScorer PageScorer
Pandalytics Pandalytics
Papa Foto Pavuk
PeoplePal PeoplePal
Petalbot
Pi-Monster Pi-Monster
PictureFinder Picsearch
Pimonster
PleaseCrawl PleaseCrawl
ProPowerBot
ProWebWalker ProWebWalker
Probethenet Pump
Pu_iN
PxBroker
Quick-Crawler Quick-Crawler
RSSingBot RSSingBot
Rankivabot RankActive
Re-re RankFlex
Reaper RankingBot
Recorder RepoMonkey
Ripper
RocketCrawler RocketCrawler
Rogerbot
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler
SISTRIX
SMTBot SMTBot
ScanAlert SalesIntelligent
ScoutJet
Screaming
ScrepyBot ScrepyBot
Searchestate Semrush
Seomoz SeoSiteCheckup
Shodan
Siphon Siphon
Sitebeam Site Sucker
SmartDownload SiteCheckerBotCrawler
SiteSucker
Sitevigil
SlySearch
Snake
Snapbot Snapbot
SocialRankIOBot SocialRankIOBot
Sosospider Sogou web spider
Sottopop Sottopop
SpaceBison SpaceBison
Spammen
SpankBot SpankBot
Spanner Spbot
Spinn3r SputnikBot
Sqlworm
Sqworm
Sucker
SuperBot
SuperHTTP
Suzuran Suzuran
Teleport T8Abot
TeleportPro Telesoft
Telesphorep
The Intraformant
TheNomad
TightTwatBot TightTwatBot
Toweyabot Toweyabot
Tracemyfile Tracemyfile
Trendiction Turnitin
Trendictionbot TurnitinBot
True_Robot Typhoeus
URLy Warning
UnisterBot
V-BOT V-BOT
VB Project
VCI
Vagabondo
VidibleScraper VidibleScraper
VoidEYE Virusdie
WASALive-Bot
WBSearchBot
WEBDAV
WISENutbot WISENutbot
WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
Web Auto
Web Collage
Web Enhancer
Web Fuck Web Fuck
Web Sauger Web Sauger
WebBandit
WebCopier WebCopier
WebFuck WebFuck
WebGo IS WebGo IS
WebImageCollector WebImageCollector
WebPix WebLeacher
Webalta WebSucker
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Website Quester
Whatweb WebsiteExtractor
WinHTTrack Webster
Wonderbot Whack
Woobot WiseGuys Robot
Wotbox Wotbox
Xaldon WebSpider Xaldon_WebSpider
YoudaoBot Xenu
Zeus Zeus
ZmEu Zitebot
ZoominfoBot ZumBot
ZyBorg
autoemailspider autoemailspider
check1.exe
clark-crawler
coccocbot-web
crawl.sogou.com crawl.sogou.com
crawler.feedback
crawler4j crawler4j
demandbase-bot dataforseo.com
eCatch
evc-batch evc-batch
lwp-trivial facebookscraper
instabid
ips-agent
isitwp.com
magpie-crawler magpie-crawler
netEstate NE Crawler netEstate NE Crawler
oBot page scorer
plumanalytics plumanalytics
satoristudio.net polaris version
probe-image-size
scalaj-http
seobility seobility
seostar
siteripz
sogouspider sogouspider
sp_auditbot spyfu
sysscan
tAkeOut tAkeOut
trendiction.com trendiction.com
voyagerx.com webprosbot
x09Mozilla x09Mozilla
x22Mozilla zauba.io
zgrab zgrab

312
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider ALittle Client
ASPSeek ASPSeek
Abonti Abonti
Aboundex Acunetix
AiHitBot AfD-Verbotsverfahren
Anarchie Aipbot
Anarchy Alexibot
Anarchy99 AllSubmitter
Ankit Ankit
Aspiegel
Asterias
Attach
AwarioRssBot
BDFetch BDFetch
BLEXBot
BackDoorBot BackDoorBot
Backlink-Ceck Backlink-Ceck
Barkrowler Badass
BatchFTP Battleztar Bazinga
BetaBot
Bigfoot
Black Hole
BlackWidow
Blackboard Blackboard
BlowFish
Boardreader Boardreader
Bolt Bullseye
BotALot
Brandwatch
Buddy
BuiltBotTough
BunnySlippers BunnySlippers
CSHttp CATExplorador
Calculon CensysInspect
CheeseBot CheTeam
ChinaClaw CherryPicker
Chlooe
Claritybot
Cloud mapping Cloud mapping
Cocolyzebot Cogentbot
Collector
Copier
Copyscape Copyscape
Craftbot Cosmos
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Custo Custo
DBLBot DBLBot
DSearch
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli
Deusu Deusu
Dirbuster Devil
Digincore
Disco Disco
Dispatch Dispatch
DittoSpyder
DnyzBot DnyzBot
DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler
Dotbot
Download Wonder Download Wonder
Dragonfly
Drip Drip
ECCP/1.0 ECCP/1.0
EMail Wolf EasyDL
Ebingbong EirGrabber
EroCrawler Exabot
Express WebPictures ExtLinksBot
ExtractorPro ExtractorPro
Ezooms Ezooms
FHscan FHscan
FemtosearchBot Firefox/7.0
FlashGet FlashGet
Flunky
Freeuploader Freeuploader
FrontPage FrontPage
Fuzz Fuzz
Fyrebot GT::WWW
G-i-g-a-b-o-t
GalaxyBot
Genieo
GermCrawler
Getintent Getintent
GoZilla GoZilla
GrabNet Gotit
Grafula Grafula
HTMLparser GrapeFX
GrapeshotCrawler
GridBot
HMView
HTTP::Lite HTTP::Lite
HaosouSpider HaosouSpider
Harvest Harvest
Havij
Heritrix Heritrix
Hloader
Humanlinks Humanlinks
HybridBot
IDBTE4M IDBTE4M
IDBot
Iblog Iblog
Id-search Id-search
IlseBot
Image Sucker
IndeedBot IndeedBot
InterGET Intelliseek
InternetSeer
Iria Iria
Iskanie Jbrofuzz
JennyBot Jorgee
JetCar Jyxobot
Jetty Larbin
JikeSpider LeechFTP
Kinza
LWP::Simple
Lanshanbot
LeechGet
Lftp Lftp
LibWeb
Libwhisker Libwhisker
LieBaoFast LieBaoFast
Linkdexbot Likse
LinkWalker
LinkextractorPro
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey
Lipperhey Spider Lipperhey Spider
MFC_Tear_Sample Lmspider
MIIxpc
MJ12bot MJ12bot
MQQBrowser Mag-Net
MSFrontPage Magnet
Mail.RU_Bot
Majestic12 Majestic12
MarkMonitor Mass Downloader
MarkWatch Mb2345Browser
Mata Hari Mediatoolkitbot
MegaIndex.ru Metauri
MicroMessenger
Microsoft Data Access Microsoft Data Access
Moblie Safari Microsoft URL Control
Msrabot Mister PiX
NICErsPRO Mojeek
Name Intelligence Mr.4x3
Nameprotect NearSite
NetZIP Needle
Netcraft Nessus
Netvibes Net Vampire
NetAnts
NetLyzer
NetMechanic
NetSpider
NextGenSearchBot NextGenSearchBot
Nibbler Nibbler
Niki-bot Niki-bot
NimbleCrawler
Ninja Ninja
Octopus Nutch
Offline Explorer Offline Explorer
Offline Navigator
OpenLinkProfiler OpenLinkProfiler
Openfind OpenVAS
OrangeBot OrangeSpider
PECL::HTTP OutclicksBot
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Pandalytics
PageAnalyzer Panscient
PageGrabber
PageThing.com
Papa Foto Papa Foto
Pi-Monster Pavuk
PeoplePal
Picscout Picscout
PictureFinder
Piepmatz Piepmatz
Pimonster Pixray
PleaseCrawl
ProPowerBot
Psbot Psbot
Pump Pu_iN
Quick-Crawler PyCurl
RSSingBot
RankActive RankActive
Ripper RankFlex
Rogerbot RebelMouse
SEOkicks-Robot RepoMonkey
SEOprofiler SBIder
SMTBot SEOkicks
Scrapy SEOlyticsCrawler
Screaming SEOstats
Searchestate Scanbot
SentiBot ScoutJet
Site Sucker ScrepyBot
SiteExplorer SemrushBot
SiteLockSpider SeobilityBot
SiteSucker Seomoz
Siphon
Sitebeam
Sitevigil Sitevigil
SlySearch
SmartDownload
Snake
Snoopy Snoopy
Sociscraper Sociscraper
Sogou web spider
Sottopop Sottopop
Spanner Spammen
SpankBot
Spbot Spbot
Sqworm SputnikBot
Sucuri Sqlmap
Suzuran Stripper
T0PHackTeam Sucker
SuperBot
Surfbot
SurveyBot
Teleport Teleport
TeleportPro Telesphorep
Telesoft
Telesphoreo
TheNomad
Thumbor
Titan
Toata Toata
Tracemyfile Tracemyfile
True_Robot Trendictionbot
Turingos Turingos
Turnitin TurnitinBot
URLy.Warning Twice
Typhoeus
URLy Warning
UnisterBot UnisterBot
Upflow V-BOT
VB Project VB Project
VCI
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper VeriCiteCrawler
Virusdie
VoidEYE VoidEYE
WASALive-Bot WWW-Collector-E
WWW::Mechanize WWW-Mechanize
Wallpapers WWWOFFLE
WallpapersHD Wallpapers/3.0
WeSEE Web Auto
Web Sucker
WebCopier WebCopier
WebFetch WebEnhancer
WebImageCollector WebGo IS
WebSucker WebReaper
WebSauger
WebWhacker WebWhacker
WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
WebsiteQuester
Webster Webster
Widow Who.is Bot
WinHTTrack
WiseGuys Robot WiseGuys Robot
Wonderbot Wonderbot
Woobot Xaldon_WebSpider
Xaldon WebSpider Xenu
Zade
Zauba Zauba
Zermelo
Zeus Zeus
ZoomBot Zitebot
arquivo.pt ZyBorg
backlink-check archive.org_bot
coccocbot-web autoemailspider
crawl.sogou.com cah.io.community
check1.exe
cognitiveseo
crawler4j crawler4j
demandbase-bot demandbase-bot
domainsproject.org
eCatch
facebookscraper facebookscraper
isitwp.com instabid
meanpathbot ips-agent
lwp-request
mediawords mediawords
muhstik-scan muhstik-scan
page scorer pcBrowser
plumanalytics plumanalytics
s1z.ru polaris version
satoristudio.net
scalaj-http scalaj-http
seobility seoscanners
sexsearcher serpstatbot
sitechecker.pro sitechecker.pro
sogouspider sogouspider
webprosbot spyfu
x09Mozilla voyagerx.com
x22Mozilla webmeup-crawler
xpymep1.exe zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
counter.bloke.com adinterax.com
dev-seo.blog balla-boo.se
imgarcade.com blog-hits.com
imguramx.pw ciproandtizanidine.blogspot.com
kostenlos-sexvideos.com pila.pl
rique.host.sk scanner-fred.top
uginekologa.com skanninge.se
weblo.com slutloadlive.com
worldhistory.biz spravka-medosmotr.ru
ynymnwbm.bloger.index.hr us-america.ru

926
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 007angels.com
101billion.com 1hwy.com
123cha.com 2020iscoming.info
23kw.ru 24x7-server-support.site
300richmond.co.nz 2drittel.de
4inn.ru 4kplayer.pl
4replicawatch.net 4ur.click
4webmasters.org a.frcls.fr
57883.net aa8780bb28a1de4eb5bff33c28a218a930.com
7yue.org aac500b7a15b2646968f6bd8c6305869d7.com
883zy.com
8xv8.com
91abcw.com
a96527gi.beget.tech
abctoppictures.net
abovetherivernc.com
actionnooz.com actionnooz.com
adloads.net actulite.com
acunetix-referrer.com
adrunnr.com
adultactioncam.com adultactioncam.com
adultmeetonline.info advokat-grodno.by
advertisingtag.net advokateg.xyz
affrh2015.com agreda.pluto.ro
aliexpresscashback.club agroeconom.kz
alldubai.biz akama.com
alpinism.ru alborzan.com
aludecor.info alekseevec.ru
amateurmatch.com alfa-medosmotr.ru
angigreene.com all4invest.ru
animal-drawings.com alphavisions.net
anime.dougasouko.com apartmentratings.com
apartment.ru apidigidocketnet-a.akamaihd.net
apple.com-webbrowsing-security.review appfastplay.com
aproposde.com approved.su
aquarium-pro.ru appsecurityr.com
ascat.porn arendas.net
arkkivoltti.net
articlesdirectoryme.info
asrvrep-a.akamaihd.net
asscenihotosidea.blogspot.com
atlasvkusov.ru atlasvkusov.ru
autochoixspinelli.com aviav.eu
autoloans.com avramstroy.ru
autolombard-krasnodar.ru babosas.com
aversis.net bagcionderlab.com
azartniy-bonus.com balans.shahterworld.org
backgroundpictures.net balitouroffice.com
bad-stars.net bankofthewext.com
balois.worldbreak.com banksrf.ru
bannerspace.com bannerconnect.net
bausparen.bz.it beatifullandscape.co
bayareaaeroclub.org best-mam.ru
bbtec.net best-seo-software.xyz
beauby.ru
best-deals-products.com
bestdooz.com
bestmarriages.com
bestofupload.info bestofupload.info
bimatoprost-careprost.com betwinservice.com
bingo8888.com bfz.biz
biznesluxe.ru bidbuy.co.kr
blavia.00author.com bio-market.kz
blockworld.ru bio-optomarket.ru
bloggerads.net bizfly.info
blogqpot.com blackwitchcraft.ru
blogrankers.com blog.f00kclan.de
bluesalt.co blogos.kz
blogs.rediff.com
bmw-ark.ru bmw-ark.ru
bodybuilding-shop.biz bocoarchives.org
brandedleadgeneration.com boo-arts.com
brianhenry.co brainxs.us
buntube.net breastaugmentation.co.za
buqayy0.livejournal.com briomotor.co
bus-offer.com buckspar.gq
businescoop.men burn-fat.ga
buttons-for-website.com buttons-for-website.com
buyk.host.sk buy-cheap-online.info
buyparajumpers.online buydissertation.net
bvlgariwallet.xyz buyessaynow.biz
bytimedance.ru buzz-porno.info
cablecar.us buzzurl.jp
calvet.altervista.org buzzzg.men
carsplate.com bvlgariring.xyz
cartierring.xyz c-gainsbourg.com
cartierwatch.xyz candy-glam-hp.com
chatroulette.life cannibalcheerleader.com
cheapestjerseys-wholesale.com carfax.com.ua
cheapsergiorossi.online carsdined.org
checkhit.com cashonads.com
catalogs-parts.com
caulderwoodkennels.com
cdncash.net
centraletermice.us
centre-indigo.org.ua
checkpagerank.net
checktext.org checktext.org
chidporn.com children-learningreading.info
cialis-samples.com christianlouboutinsaleoutletonline.us
cigarpass.com cialischsrx.com
claytransformations.info cialiswithoutadoctor.net
cleanallspyware.com ciekawinki.pl
cleansearch.net clickbanksites.info
clickpapa.com club-lukojl.ru
clothesforcash.com coderstate.com
club-musics.ru coinsspb.com
coin-hive.com compliance-alex.xyz
coindirect.io compliance-fred.top
coldfilm.ru compliance-irvin.top
com-onlinesupport.host compliance-jessica.top
communisave.co.za contentlook.co
compliance-alex.top cosmetic.donna7753191.ru
compliance-alexa.xyz craftburg.ru
compliance-brian.xyz creativehutindia.com
compliance-don.top credit-respect.ru
compliance-julia.top cubook.supernew.org
conboy.us custodieva.ru
cookie-law-enforcement-ee.xyz dailystrength.org
covetnica.com dating-time-now.com
crackguru.tk datract.com
crest3d.ru ddsoldes.fr
cryptoswap.biz decenttools.com
culmimg.pw decorationspcs.com
customchocolate.business-for-home.com demian.kz
customcollegeessays.net demon-tweeks.com
cv.wallhade.co deniven.1bb.ru
cyse.tk destinationrealestate.com
danashop.ru detki-opt.ru
deadlinkchecker.com digitalassetmanagement.site
delfin-aqua.com.ua domini.cat
demenageur.com dotmass.top
devochki.top downloader12.ru
digilander.libero.it downloadmefiranaratb1972.xpg.com.br
dikqlyremy.info
divci-hry.info
dnepr-avtospar.com.ua
docs4all.com
dogbreedspicture.net
dojki-hd.com
domain2008.com
domaincheck.io
dood.live
doreenblog.online
drivotracker.com drivotracker.com
drugspowerstore.com
duawitchrarato.tk
dwomlink.info
e-buyeasy.com e-buyeasy.com
eachdayisagift.review
earncash.com.ua earncash.com.ua
ecookna.com.ua ecommerce-seo.org
editors.choice6912650.hulfingtonpost.com elektronischezigarettekaufen1.myblog.de
eduinfosite.com elektrozigarette2.webs.com
elektronischezigarette2.mex.tl elektrozigaretten2.yn.lt
elite-sex-finders.com
elmifarhangi.com
eloconcream.blogspot.com
eluxer.net
embedle.com
emediate.eu
enhand.se
eralph.tk eralph.tk
euromasterclass.ru eropornosex.ru
expertnaya-ocenka.ru erotikstories.ru
expolicenciaslatam.co essaypro.com
expresstoplivo.ru etenininrade.ga
extstat.com etm-consult.de
ezigarettenshop2.myblog.de exct.net
ezigaretteshop.webs.com extremal-blog.com
ezigaretteshop2.mywapblog.com f0815.de
f5mtrack.com f5mtrack.com
facebook-mobile.xyz falcon-images.blogspot.com
fashion-stickers.ru falcoware.com
fenoyl.batcave.net fatmaelgarny.com
fermersovet.ru fdzone.org
fetlifeblog.com femmesdenudees.com
filmbokep69.com fighrofacciufreesig.ga
filmi-v.online filkhbr.com
financehint.eu findclan.org
fitfloponline.store flavors.me
flowwwers.com formseo.com
forms-mtm.ru formularz-konkurs.tk
free-share-buttons.blogspot.com forumprofi.de
freegamesplay.online fotoxxxru.com
freevpn.space freenode.info
frequiry.com frequiry.com
friendflnder.com
frivgame250.com frivgame250.com
fugarif.ga fsalas.com
fx-brokers-review.com fungamelands.com
gagrasector.ru fyxabomiw.ru
galaxyflowers.ru gallerylisting.com
galeon.com gambarkataku.co
game-mmorpg.net gate5.co.za
gamerextra.com gateway.zscalertwo.net
gay-file.com gdzkurokam.ru
get-seo-domain.com
getprismatic.com getprismatic.com
gettpromos.com girlgamerdaily.com
gigixo.com girls-ufa.ru
gk170.ru glastecfilms.com.my
glogow.pl glicol.kz
goatse.ru goldandcard.ru
gogps.me goodhousekeeping.com
good-mummy.ru goodwriterssales.com
goodhumor24.com googglet.com
goodwinmetals.co gopixdatabase.com
gosreg.amchs.ru gosarhivrt.ru
gratuitbaise.com green-tea.tv
gsmlab.pl guardlink.com
guidefs.ru
guigyverpo.cf guigyverpo.cf
gungamesz.com gungamesz.com
guod.me
h2monline.com h2monline.com
haikuware.com hanwei.us
hannasolution.ru hdpixion.com
hasshe.com healbio.ru
hd720kino.ru healing-dysplasia.ru
hdhc.ru hermesreplica.pw
heartofbeijing.blogspot.com homeandhealth.ru
heygidday.biz homemade.gq
highland-homes.com
hkladys.com
homeart.space
hostcritique.com
hostnow.men hostnow.men
hotblog.top hothor.se
housediz.com house.sieraddns.com
hoverboard360.es hyipmanager.in
hoverboard360.nl hystersister.com
hspline.com ibb.com.ua
ikearugs.xyz igrovyeavtomaty777.ru
imitex-plus.ru imediadesk.com
interior-stickers.ru imgarit.pw
iminent.com
incest-ru.com
innodgfdriverhm.aircus.com
insider.pro
insomniagamingfestival.com
intellekt21.ru
intermesh.net intermesh.net
ipornox.xxx intervsem.ru
itis4you.com ipchicken.com
ivanstroi.ru istizanidinelikexanax.blogspot.com
iyasimasennka.com ix20.ru
izoll.ru jackpotchances.com
j33x.com javspace.net
jabimgo.pw jjbabskoe.ru
javlibrary.cc jrpmakati.com
jerseychinabizwholesale.us jwcialislrt.com
jestr.org jyrxd.com
jmat.cn
johnrobertsoninc.com
journeydownthescale.info
jus0wil.pp.ua
justprofit.xyz
kaac.ru
kalb.ru kalb.ru
kazan.xrus.org kanimage.com
kazrent.com keenoutlet.online
kidd.reunionwatch.com keki.info
kinky-fetishes.com kenaba.su
ketoanhanoi.info
keywordsuggests.com
kiinomaniak.pl
kino-fun.ru kino-fun.ru
kinotorka.ru kinoflux.net
knogg.net kiskinhouse.com
komp-pomosch.ru kit-opt.ru
kotaku.com kiwi237au.tk
kriokomora.info klin3952.ru
ktotut.net klitimg.pw
kursy-ege.ru kollesa.ru
ladsup.com konkursov.net
lafourmiliaire.com krasnodar.ru
landmania.ru kvartiry-remont.ucoz.ru
laulini.soclog.se lafriore.ru
lawyers.cafe landinez.co
leftofcentrist.com larutti.ru
law-check-nine.xyz
legalrc.biz
lezbiyanki.net lezbiyanki.net
libertybilisim.com lider-zhaluzi.kiev.ua
lifehacĸer.com liky.co.ua
linkbuddies.com linerdrilling.com
llastbuy.ru linkdebrideur.xyz
lonfon.xyz linkpulse.com
luckyshop.net.ua ljusihus.se
lyrster.com lovelycraftyhome.com
mackeeper-land-672695126.us-east-1.elb.amazonaws.com lovi-moment.com.ua
magnetpress.sk luxmagazine.cf
mahnwachen-helfen.info malwareremovalcenter.com
mainlinehobby.net
managerpak204.weebly.com managerpak204.weebly.com
manifestyourmillion.com marketingtechniques.info
manyresultshub-a.akamaihd.net meble-bogart.info
marcogrup.com medicovi.com
maridan.com.ua meet-flirt-dating.com
massage-info.nl miniads.ca
mastershef.club mmofreegames.online
matb3aa.com mobstarr.com
mebel-ekb.com modnie-futbolki.net
medi-fitt.hu moesen-ficken.com
medicineseasybuy.com moncleroutletonline.win
meduza-consult.ru moneyzzz.ru
mexicosleevegastrectomy.com monitorwebsitespeed.com
middlerush-a.akamaihd.net
misswell.net
mistr-x.org
mitrasound.ru
mixtapetorrent.com
mixx.com
mjchamonix.org
mlvc4zzw.space
mobsfun.net
mompussy.net
money-for-placing-articles.com
moomi-daeri.com moomi-daeri.com
morlat.altervista.org moviezbonkerssk.cf
mug-na-chas-moscow.ru mowser.com
muzaporn.com moyakuhnia.ru
mydeathspace.com mundoaberrante.com
nacap.ru myblogregistercm.tk
myftpupload.com
myprintscreen.com
namenectar.com
narkologiya-belgorod.ru narkologiya-belgorod.ru
narkologiya-penza.ru
natali-forex.com
net-profits.xyz net-profits.xyz
nhl09.ru newstudio.tv
nikitsyringedrivelg.pen.io nextconseil.com
nippon-bearings.ru nomuos.it
nordvpn.com nordvpn.com
nucia.biz.ly nudo.ca
nw-servis.ru obiavo.kz
o333o.com obnal.org
obiavo.com obuv-kupit.ru
officedocuments.net offer.camp
offtopic.biz okroshki.ru
onlainbesplatno.ru one.net.in
online-sbank.ru online-hd.pl
ontargetseo.us orion-v.com
ooomeru.ru osnova3.ru
oren-cats.ru ozas.net
origin-my.ru pandarastore.top
ourtherapy.ru partner-print.men
oyster-green.com picsforkeywordsuggestion.com
pandroid.co piter.xrus.org
pcboa.se pizdopletka.club
petsblogroll.com playlott.com
photo-clip.ru playtap.us
pinup-casino1.ru pokerniydom.ru
plusnetwork.com pornohd1080.online
poligon.com pornolook.net
pornhive.org pornoplen.com
pornhub-ru.com posible.net
porno-chaman.info predominant-invent.tk
porno-raskazy.ru primedice.com
pornokorol.com printdirectforless.com
prlog.ru privacylocationforloc.com
promgirldresses.xyz privatov-zapisi.ru
promoforum.ru prizefestival.mobi
proxyelite.biz profitsport.club
ptr.ruvds.com progonrumarket.ru
pufip.com pron.pro
purplesphere.in prostitutki-yaroslavlya.org
pussyspace.net pulse33.ru
quality-traffic.com pyrodesigns.com.au
quickbuck.com qld10000.net
ragecash.com ranksignals.com
randalljhoward.com rating-bestcasino.com
rangjued.com rczhan.com
rcb101.ru redirect.trafficreceiver.club
registrydomainservices.com refererx.com
remont-comp-pomosh.ru registrationdomainsite.com
remont-ustanovka-tehniki.ru reklama-i-rabota.ru
remybutler.fr respublica-otel.ru
renecaovilla.online reversing.cc
rennlist.com rewardpoll.com
reward-survey.net rezeptiblud.ru
roleforum.ru rique.host.sk
rostov.xrus.org royalads.net
ruex.org.ua rustic-quiver.win
ruf777.com
rumamba.com
rutor.group
ryetaw.com ryetaw.com
s-iwantyou.com s8-nowy-wygraj.comli.com
s1z.ru samo-soznanie.ru
sa-rewards.co.za sanyuprojects.com
sarafangel.ru saveriopiazza.it
sax-sex.com scalerite.co.za
scanner-barak.top
scanner-fred.top
scanner-john.top scanner-john.top
scat.porn scanner-josh.top
scopich.com scenarii-1-sentyabrya.uroki.org.ua
scrapinghub.com se.bnt-team.com
sdi-pme.com search.pch.com
secretscook.ru securesmrt-dt.com
securityallianceservices.com seeingmeerkat.com
seinterface.com seo-prof1.xyz
seminarykansas904.ml seo18.su
semp.net seobility.net
seocdvig.ru seoboxes.com
seogadget.ru seolab.top
serviporno.com seomarketings.online
sex-dating.co seoprofiler.com
sexpartygirls.net seorussian.ru
sexreliz.com service.adtech.fr
sfj-ror.no servicecenter.co.ua
shisha-swag.de sex.hotblog.top
shopvilleroyboch.com.ua sexgalleries.top
sibvitr.ru sexpornotales.net
silktide.com sexstream.pl
simplepooltips.com sexvporno.ru
siteexpress.co.il shakhtar-doneck.ru
skuteczna-dieta.co.pl sharebutton.org
smailik.org shitmovs.com
shohanb.com
shopfishing.com.ua
sicfor.bcu.cc
sildenafil-tadalafil.info
silvercash.com
sims-sims.ru
sirpornogratis.xxx
sisi-go.ru
siteimprove.com
skladvaz.ru
skytraf.xyz
slayerlife.com
slkrm.ru
small-games.biz
sms2x2.ru
snip.to
snomer1.ru
social-button.xyz social-button.xyz
softomix.com socialmediasuggest.com
softxaker.ru soda.media
sotechco.co spb.afora.ru
space2019.top
spamnuker.com
spasswelt.xyz
sport7777.net sport7777.net
sports-supplements.us stariy-baku.com
spy-sts.com starwars.wikia.com
stal-rulon.ru stroyhelp-dv.ru
starpages.net studentguide.ru
stockspmb.info stuff-about-money.com
stoki.ru sucsesofinspiration.com
stretchmate.net sugvant.ru
stroimajor.ru sundrugstore.com
stromerrealty.com supers.com.ua
superoboi.com.ua superstats.com
surflinksmedical.com surintech.ac.th
susanholtphotography.com svetodiodoff.ru
svbur.ru tako3.com
takethatad.com tanieaukcje.com.pl
taranerymagesswa.blogspot.com
tattooha.com tattooha.com
tattooreligion.ru tech4master.com
techart24.com teenfuck.tv
tecspb.ru teguh.info
the-trader.net telemetryverification.net
thedownloadfreeonlinegames.blogspot.com textads.men
thegalerie.eu tfxiq.com
thegameriders.com thecoral.com.br
titelhelden.eu theendivechronicles.com
tizanidine4mguses.blogspot.com thegioixekhach.com
tizanidineandhydrocodone.blogspot.com ticketsys.inetwd.com
tizanidinevsvalium.blogspot.com titslove.yopoint.in
toproadrunner5.info tizanidinedrugtest.blogspot.com
toy-shop.top tizanidinehclsideeffects.blogspot.com
traffixer.com tmearegion26.com
trailer.cinemaflix.website tmm-kurs.ru
trainoffend.ml tooplay.com
tramadolandtizanidine.blogspot.com torrents-tracker.com
tripper.de tracksz.co
tryrating.com trackzapper.com
tuberkuleznik.ru traderzplanet.in
tvoystartup.ru trafaret74.ru
twittruth.com traffic2cash.org
u17795.netangels.ru trafficnetzwerk.de
ua-company.ru trk-4.net
unece.org trkdf.com
uprour.com tubeline.biz
uptimebot.net twelvevisionspartyofcolorado.com
urengoy.pro twinderbella.com
urldelivery.com twu.com.ua
vacances-voyages.info uginekologa.com
vbikse.com ukrtvory.in.ua
vchulkah.net ultimatesetnewfreeallsoftupgradesystems.pw
veerotech.com unimodemhalfduplefw.pen.io
verymes.xyz unionmarkt.de
viberdownload10.com unmaroll.ya.ru
video-camer.com urlopener.blogspot.com.au
videochat.life uroki.net
videonsk.com uzporno.mobi
videooko.weebly.com vapeface.club
virus-respirators.com vbabule.net
vekzdorov.ru
vepad.com
vigrx-original.ru
vizag.kharkov.ua vizag.kharkov.ua
voloomoney.com vkgaleria.com
vzlom-na-zakaz.com vkonche.com
wallpapers-best.com vladimir.xrus.org
webix.biz w3javascript.com
website-analytics.online warezaccess.com
websiteexplorer.info watsonrealtycorp.com
websitevaluebot.com web.cvut.cz
weburok.com webads.co.nz
weekes.biz.tc webcamtalk.net
weightatraining.com webnode.me
wemedinc.com westvilletowingservices.co.za
whipme.yopoint.in wetgames.ru
wholesalejerseys-cheapest.com whatsappdownload10.com
win-spy.com whiteproduct.com
wingsoffury2.com wildworld.site
winner7777.net wingsofrefuge.net
wma-x.com winterclassichockeyjerseys.com
workle.website withstandingheartwarming.com
world-mmo.com wleuaprpxuvr.ga
worldhistory.biz wonderfulflowers.biz
woodyguthrie.se
worst-sites.online
writersgroup580.web.fc2.com writersgroup580.web.fc2.com
writingservices17.blogspot.ru wtsindia.in
wrz0iuebwhp5fg.freeddns.com x-diesel.com
wstroika.ru x-stars.ru
www.labves.ru xn--80aaagvmjabrs1aoc9luc.xn--p1ai
wygraj-teraz.com xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
x-lime.net
x69ty.ru
xdoza.com
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn----7sbahjd3btneuw1joc.xn--p1ai
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn--80aanaardaperhcem4a6i.com
xn--80ahdheogk5l.xn--p1ai xn--80ahdheogk5l.xn--p1ai
xn--h1ahbi.com.ua
xn--l1aengat.xn--p1ai
xolodremont.ru xolodremont.ru
xpresscare.ru xrus.org
xtremeeagles.net xvideosbay.com
xtubeporno.net xxart.ru
xvideospornoru.com xxxkaz.org
xxxtubesafari.com xzlive.com
yaoguangdj.com yadro.ru
yebocasino.co.za yousense.info
yoshkarola.zrus.org youthreaders.com
yougetsignal.com yuarra.pluto.ro
youradulthosting.com zahvat.ru
yoursearch.me zaobao.com.sg
yourtemplatefinder.com zastroyka.org
zoolubimets.ru zeroredirect12.com
zootoplist.com zolotoy-lis.ru
zophim.me zoompegs.com
zvooq.eu
zytpirwai.net
zzbroya.com.ua

488
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00go.com 0n-line.tv
123movies.love 101flag.ru
1qingdao.com 108shot.com
8gold.com 10bet.com
8lufu.com 14b.info
above.com 202ch.com
acmebtn.ml 273-fz.ru
adclickthru.net 3-letter-domains.net
addtoadd.men 4senses.co
adhome.biz 76brighton.co.uk
adinterax.com 888.com
adserver-e7.com adktrailmap.com
adsland.men adultnet.in
adult-shop.com.ua advancedcleaner.com
advertisingtag.net adzerg.com
adviceforum.info albamargroup.com
agapovdg.ru alert.scansafe.net
ahhjf.com all4invest.info
ahmedabadwebs.com allkrim.com
alienwheel.es aludecor.info
am15.net anabolics.shop
ananumous.ru anonymousfox.co
anime.dougasouko.com apartment.ru
aosexkontakte.net apiportalmorecom-a.akamaihd.net
apibrowsesmartne-a.akamaihd.net approved.su
atovh.local-finders.com arate.ru
automatic-seo.com arendatora.ru
avitocars.ru artdestshop.eu
azarttoto.com artefakct.com
bad-stars.net artel116.ru
bdsmgalls.net as5000.com
berlininsl.com asscenihotosidea.blogspot.com
bestdraws.com astana.xxxkaz.org
bestkfiledxd.cf aucoinhomes.com
bestserials.com autoseo-traffic.com
bidr.trellian.com autowebmarket.com.ua
bitcoin-ua.top aversis.net
boole.onlinewebshop.net avkzarabotok.info
breastaugmentation.co.za awency.com
buqyxa.rincian.info ayodhya.co
bus-offer.com bablonow.ru
buy-cheap-pills-order-online.com bazaronline24.ru
belsetirehafi.tk
besthoro.ru
bestimagecoollibrary.com
bestplacetobuyeriacta.jw.lt
bet-prognoz.com
bizzliving.com
bonvillan.com
brewdom.ru
bronzeaid-a.akamaihd.net
btc4u.ru
business-suggested.tk
buydissertation.net buydissertation.net
buzz-porno.info bzero1jewelry.net
byme.se cartierlove2u.com
candiceloves.us cartierloveringreplica.xyz
cartierreplica.xyz cartierlovestore.xyz
caulderwoodkennels.com certifywebsite.win
cbox.ws chatroulette.si
cdncash.org cheapcigarettesc.info
cejewelry.xyz cheapmoncler.pw
centraletermice.us checkpagerank.net
ceotrk.com cialischstgerts.com
cete.ru cleanallspyware.com
cheapbarbour.online clickhype.com
cheapmarmot.online compliance-elena.top
chinaelitecheapjerseys.com
cialischmrx.com
cibpenonptib.flu.cc
cigarpass.com
citizenclsdriveri7.pen.io
cjmc.info
clickintext.net
cmd.kz
compliance-ivan.xyz
compliance-jess.top compliance-jess.top
compliance-josh.top compliance-walter.top
compliance-mark.top coolbar.pro
contentsexpress.com creditwell.ru
cookie-law-enforcement-ee.xyz ddospanel.com
cosmeticswomens-womensports.rhcloud.com deluxedumps.com
creatives.adbetclickin.pink dengi-pod-zalog-nedvizhimosti.ru
danceuniverse.ru dfiles.me
darodar.com domain-submit.info
dating-app.ru dosug-lux.ru
datsun-do.com dosugrostov.site
de.zapmeta.com downvids.net
detki-opt.ru easyshoppermac.com
domainroam.win elektronischezigarettekaufen2.tumblr.com
domainsatcost.com elektrozigarettekaufen1.devhub.com
drpornogratisx.xxx elitepcgames.com
dtnlyss.com eloconcream.blogspot.com
econom.co enbersoft.com
educontest.net enginebay.ru
efnor-ac.com ero-advertising.com
elektrozigarettekaufen2.blogse.nl
elkacentr.ru
en.altezza.travel
envaseslotusama.com
esfchat.tk esfchat.tk
eshop.md ewebarticle.info
excitacion.info farm26.ru
f012.de
farmprofi.net farmprofi.net
filmi-onlain.info fermersovet.ru
findacheaplawyers.com floridamobilebillboards.com
flex4launch.ru fotosfotos.eu
foto-sisek.porngalleries.top freshwallpapers.info
freevpn.space front.ru
fun-mobi.pl ftns.ru
fx-brokers-review.com gambarkatabaru.com
gallerylisting.com geilehausfrauen.net
gamezblox.com gelezki.com
generousdeal-a.akamaihd.net ghernnqr.skyrock.com
gigapeta.com glopages.ru
girlsatgames.ru gomusix.com
globalsurfari.com greenzaim.ru
globatur.ru gymi.name
goroda-vsego-mira.ru headpharmacy.com
gwagka.com heartofbeijing.blogspot.com
heatpower.ru highstairs-a.akamaihd.net
hjaoopoa.top hit-kino.ru
hunterboots.online hoste.octopis.com
ikritikimou.gr hotenergy.ru
imicrovision.com hotloans.ru
intellego.info hspline.com
io9.com hystersister.com
iqs.biz.ua ilte.info
istripper.com innoslicon.com
izismile.com internetproviderstucson.com
jose.mulinohouse.co istizanidinelikexanax.blogspot.com
kinomaniatv.pl jaxcube.info
kinoplen.ru jumptap.com
kiwi237au.tk junketjuice.blogspot.com
komp-pomosch.ru kanimage.com
laudit.ru khafre.us
lazymanyoga.com kinky-fetishes.com
lechenie-gemorroya.com kit-opt.ru
light.ifmo.ru kladrus.ru
lmrauction.com krassh.ru
locimge.pw le-clos-des-alouettes.com
lovelycraftyhome.com ledpolice.ru
luckyshop.net.ua letslowbefast.site
m.b00kmarks.com lflash.ru
mackeeper-center.club lion.playtap.us
magazintiande.ru liumimgx.pw
makis.nu live-sexchat.ru
markjaybeefractal.com lizace.com
megahdporno.net mararoom.ru
mere.host.sk mirtorrent.net
mikrobiologies.ru mlf.hordo.win
mindeyegames.com mmofreegames.online
minet.club mojaocena.com
modnie-futbolki.net moncleronline.xyz
mulberryoutletonlineeu.com monsterdivx.tv
my-cash-bot.co mosdverka.ru
mykings.pw motherboard.vice.com
mylovelibrabry.com musflashtv.com
needtosellmyhousefast.com mylida.org
nfljerseyscheapchinabiz.com nerudlogistik.ru
nfljerseysforsalewholesaler.com nextbackgroundcheck.gq
nikhilbahl.com nrv.co.za
nonews.co o-o-6-o-o.com
ntic.fr offer.camp
nufaq.com offf.info
o.light.d0t.ru olvanto.ru
odywpjtw.bloger.index.hr omgtnc.com
online247.ml onlinebay.ru
openfrost.com opinionreelle.com
osagonline.ru optom-deshevo.ru
osb.se11.ru oren-cats.ru
ovirus.ru pensplan4u.com
oxford-book.com.ua petitions.whitehouse.gov
page2rss.com pewit.pw
parajumpersoutlet.online phormchina.com
pateaswing.com pisanieprac.info
paydayonlinecom.com poligon.com
pb-dv.ru polytopesexempt.com
picturesboss.com pornonik.com
pixell.club positive2b.ru
pobeiranie.pl princeadvantagesales.com
pomoc-drogowa.cba.pl privacylocationforloc.com
pornowarp.info prod2016.com
porto.abuilder.net progonrumarket.ru
powc.r.ca.d.sendibm2.com promodj.com
preparevideosafesystem4unow.space pshare.biz
prostitutki-astana.org rebuildermedical.com
puzo2arbuza.ru remont-ustanovka-tehniki.ru
qualitymarketzone.com renewablewealth.com
qwesa.ru retreatia.com
reckonstat.info ring4rhino.com
rfid-locker.co robertefuller.com
ritlweb.com rossanasaavedra.net
royal-betting.net royal-betting.net
rrutw.com running-line.ru
salutmontreal.com rusenvironmental.net
sc-specialhost.com rusprostitute.com
seocheki.net ryetaw.com
seokicks.de sadaholding.com
seonetwizard.com scanner-alex.top
seorussian.ru scat.porn
serialsway.ucoz.ru segol.tv
sexiporno.net semaltmedia.com
sexobzor.info semprofile.com
sharebutton.to senger.atspace.co.uk
shemalegalls.blogporn.in seofirmreviewsus.info
shisha-swag.de serpstat.com
shop.acim.org sexkrasivo.net
signx.info sexpornotales.com
site-speed-check.site sexvideo-sex.com
shareyards.com
shivafurnishings.com
signoredom.com
sindragosa.comxa.com
siteimprove.com
slimcdn.com slimcdn.com
smadihome.com sloopyjoes.com
socialsignals24.com sovetogorod.ru
soheavyblog.com sports-supplements.us
space-worry.ml stillmiracle.com
spaceshipad.com stroiminsk.org
speedup-my.site taaaak.com
spywarebegone.com telegraf.by
spywarenuker.com thebluffs.com
summerlinhomes411.info thegalerie.eu
tedy.su thelottosecrets.com
teesdaleflyballclub.co.uk theporndude.com
thecounter.com theprofitsmaker.net
tizanidinehcl4mgtablet.blogspot.com timeallnews.ru
tizanidinehclsideeffects.blogspot.com tobeyouday.win
tizanidinepills.blogspot.com tongkatmadura.info
tizanidinevsvalium.blogspot.com tracksz.co
torrents-tracker.com traffic100.com
torrnada.ru trafficjunky.com
trackmedia101.com trafficmp.com
truck-addzilla.life twojebook.pl
uptime-gamma.net umekana.ru
url-img.link upstore.me
vacances-voyages.info vabasa.inwtrade.com
vchulkax.com vduplo.ru
via-energy-acquistare.com website-stealer.nufaq.com
video-camer.com websites-reviews.com
video-chat.in whipme.yopoint.in
videochat.bz wholesalejerseychinashop.com
vipsexfinders.com wikes.20fr.com
wapsite.me womama.ru
watchmyfb.pl worldtraveler.world
waysbetter.cn wttavern.com
webadvance.club www.arenda-yeisk.ru
websitevaluebot.com x-diesel.biz
wesharepics.site x-stars.ru
wetgames.ru xgames-04.com
whengirlsgowild.com xn----7sbaphztdjeboffeiof6c.xn--p1ai
wholesalejerseychinaoutlet.com xn----7sbifcamovvfggw9d.xn--p1ai
worldoffiles.ru xn--l1aengat.xn--p1ai
www.wohnkabinen-shop.de yottos.com
xn----itbeirbjbi7bc6bh2d.xn--p1ai youresponsive.com
yginekologa.com yurgorod.ru
youbloodyripper.com zarabotok--doma.ru
yougame.biz zeroredirect2.com
yur-p.ru zetmaster.ru
zakon-ob-obrazovanii.ru zhongwenlink.com
zeleznobeton.ru zvooq.eu
znaturaloriginal.com
zzlgxh.com

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.136 0; 10.1.0.96 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2609 ### Version: V4.2021.09.2610
### Updated: Thu Sep 23 22:01:31 UTC 2021 ### Updated: Fri Sep 24 10:03:24 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2609 ### Version: V4.2021.09.2610
### Updated: Thu Sep 23 22:01:31 UTC 2021 ### Updated: Fri Sep 24 10:03:24 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##