vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.04.3057 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-04-21 10:09:23 +00:00
parent 08a38cf01f
commit e2834713a6
47 changed files with 80659 additions and 80669 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3056 #### Version: V4.2022.04.3057
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3056 #### Version: V4.2022.04.3057
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.04.3056 #### Version: V4.2022.04.3057
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9948
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.04.3056 #### Version: V4.2022.04.3057
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9952
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9952
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

9952
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9952
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.105 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.105 0;
127.0.0.1 0; 127.0.0.1 0;

9952
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.105 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.105 0;
127.0.0.1 0; 127.0.0.1 0;

9952
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.105 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.105 0;
127.0.0.1 0; 127.0.0.1 0;

9952
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3056 3057

9952
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

9952
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.232.159.186 0; 52.155.59.8 0;

9952
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.105 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.105 0;
127.0.0.1 0; 127.0.0.1 0;

9952
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.105 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.105 0;
127.0.0.1 0; 127.0.0.1 0;

9952
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.105 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.105 0;
127.0.0.1 0; 127.0.0.1 0;

9952
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.232.159.186 0; 52.155.59.8 0;

9952
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.232.159.186 0; 52.155.59.8 0;

9952
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.105 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
GrabNet G-i-g-a-b-o-t
HMView Nikto
Microsoft URL Control OnCrawl
Name Intelligence OpenVAS
NetAnts ProWebWalker
Openvas SiteSnagger
SlySearch Suzuran
TheNomad Web Pix
coccocbot seocompany.store
satoristudio.net sexsearcher

310
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider
404checker 404checker
404enemy
80legs
AIBOT
ALittle Client ALittle Client
ASPSeek
Abonti Abonti
Aboundex Aboundex
Aboundexbot AiHitBot
Acunetix
AhrefsBot
Aipbot
Alexibot Alexibot
AlphaBot AlphaBot
Anarchie Anarchie
Anarchy
Anarchy99 Anarchy99
Ankit Ankit
Anthill Apexoo
Asterias Aspiegel
Attach Atomseobot
AwarioRssBot AwarioRssBot
AwarioSmartBot
BDCbot BDCbot
BLEXBot
BackDoorBot
BackWeb BackWeb
Backlink-Ceck
BacklinkCrawler
Badass Badass
Barkrowler
Battleztar Bazinga Battleztar Bazinga
BetaBot Bigfoot
Boardreader Black Hole
BlackWidow
BlowFish
Bolt Bolt
BotALot BotALot
Brandprotect
Buck Buck
Buddy Bullseye
BuiltWith
BunnySlippers
BuzzSumo
CATExplorador CATExplorador
CODE87 CSHttp
CensysInspect CazoodleBot
CheTeam
CherryPicker CherryPicker
ChinaClaw
Citoid Citoid
Claritybot Claritybot
Cliqzbot Cloud mapping
Cogentbot Cogentbot
Collector Collector
CopyRightCheck
Cosmos
Craftbot Craftbot
Crawling at Home Project Crawling at Home Project
CrunchBot Crescent
DSearch Custo
CyotekWebCopy
DBLBot
DTS Agent
DatabaseDriverMysqli
Deusu Deusu
Devil Discoverybot
Digincore Dispatch
Discobot DomainCrawler
DnyzBot
DomCopBot
DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
Dragonfly DomainStatsBot
EMail Siphon EMail Siphon
EMail Wolf Ebingbong
EasyDL Ecxi
EirGrabber Evil
Exabot ExtLinksBot
Express WebPictures
Extractor Extractor
Ezooms ExtractorPro
FHscan Extreme Picture Finder
FemtosearchBot Firefox/7.0
Fimap FlashGet
Flunky
Foobot Foobot
Freeuploader Freeuploader
Fuzz GT::WWW
Fyrebot GalaxyBot
GermCrawler GermCrawler
GetRight
GetWeb GetWeb
Getintent Getintent
Gigabot
GoZilla GoZilla
Gotit Gotit
GrabNet Grabber
Grafula HMView
GrapeshotCrawler
GridBot
HTTP::Lite HTTP::Lite
HTTrack HaosouSpider
Haansoft Harvest
HonoluluBot Humanlinks
IDBTE4M HybridBot
IDBot
Iblog Iblog
Id-search
IlseBot
Image Sucker
Indy Library Indy Library
InfoNaviRobot InfoTekies
Iria JennyBot
JamesBOT
JetCar JetCar
Jetty Joomla
JikeSpider Kinza
Jyxobot LNSpiderguy
Keyword Density
LWP::Simple
Larbin
Leap Leap
Lftp Libwhisker
LieBaoFast Lightspeedsystems
LinkScan
LinkWalker LinkWalker
Linkbot
LinkextractorPro LinkextractorPro
LinkpadBot LinksManager
LinqiaMetadataDownloaderBot LinqiaRSSBot
Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker
Ltx71 Ltx71
MIIxpc
MSIECrawler MSIECrawler
Majestic SEO Mag-Net
MarkMonitor Magnet
Mail.RU_Bot
Majestic-SEO
Majestic12
Mata Hari Mata Hari
MauiBot MauiBot
Mb2345Browser Meanpathbot
Mediatoolkitbot MicroMessenger
Microsoft Data Access
Microsoft URL Control
Minefield
Mister PiX
Mojolicious Mojolicious
Morfeus Fucking Scanner
Mozlila
Musobot
NPbot
Name Intelligence
Nameprotect Nameprotect
NearSite NearSite
Needle
Nessus
Net Vampire
NetLyzer
NetMechanic NetMechanic
Netvibes NetZIP
NextGenSearchBot NextGenSearchBot
Nuclei Niki-bot
Octopus NimbleCrawler
Nmap
Offline Explorer Offline Explorer
OnCrawl OrangeBot
OutfoxBot OrangeSpider
PHPCrawl PECL::HTTP
POE-Component-Client-HTTP
PageScorer PageScorer
Pandalytics Panscient
Papa Foto Papa Foto
Petalbot
Pi-Monster Pi-Monster
Picsearch
Pimonster Pimonster
Pixray
ProPowerBot ProPowerBot
Probethenet Probethenet
Psbot QueryN Metasearch
Pu_iN
Quick-Crawler Quick-Crawler
RankActiveLinkBot RankActiveLinkBot
RankingBot RankingBot
RankurBot RankingBot2
ReGet Re-re
RocketCrawler RebelMouse
SEOkicks-Robot Recorder
SEOlyticsCrawler SEOkicks
SMTBot SMTBot
SalesIntelligent ScanAlert
ScoutJet
Scrapy Scrapy
ScreenerBot ScreenerBot
ScrepyBot ScrepyBot
Searchestate
Seekport Seekport
SemanticJuice SemanticJuice
SentiBot
SeobilityBot SeobilityBot
SiteLockSpider SiteCheckerBotCrawler
Sitebeam Sitebeam
Siteimprove Sociscraper
Snoopy
SocialRankIOBot
Sogou web spider Sogou web spider
Sosospider Sosospider
SpaceBison
Spanner Spanner
Spbot Spinn3r
SputnikBot SputnikBot
Sucker Sqlworm
Sucuri Sqworm
SuperBot Stripper
SuperHTTP
Surfbot Surfbot
Suzuran
Swiftbot Swiftbot
T0PHackTeam Szukacz
Teleport TeleportPro
Telesphoreo TightTwatBot
Thumbor
Titan
Toata Toata
Tracemyfile Turingos
Trendiction Typhoeus
True_Robot URLy Warning
TwengaBot
Twice
URLy.Warning URLy.Warning
Upflow Upflow
V-BOT VCI
VB Project VelenPublicWebCrawler
VeriCiteCrawler Virusdie
VoidEYE Voltron
WWW-Mechanize WBSearchBot
WEBDAV
WISENutbot
WWW-Collector-E
WWWOFFLE WWWOFFLE
Web Fetch Wallpapers/3.0
Web Sucker WebBandit
WebAuto
WebCollage
WebEnhancer
WebFetch WebFetch
WebGo IS WebLeacher
WebStripper WebStripper
WebZIP WebZIP
Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Webshag
WebsiteQuester Website Quester
Webster Widow
Whacker WinHTTrack
Whatweb Woobot
Xenu Xaldon WebSpider
Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zitebot
Zauba ZmEu
Zermelo ZumBot
Zeus archive.org_bot
ZyBorg arquivo.pt
adscanner
autoemailspider
backlink-check backlink-check
cah.io.community
check1.exe check1.exe
dataforseo.com coccocbot
com.plumanalytics
crawler4j
demandbase-bot
gopher
heritrix
instabid
internetVista monitor internetVista monitor
ips-agent
isitwp.com
iubenda-radar iubenda-radar
linkdexbot lwp-trivial
lwp-request muhstik-scan
meanpathbot
mediawords
netEstate NE Crawler
probe-image-size
satoristudio.net
scalaj-http
scan.lol scan.lol
seocompany.store serpstatbot
seoscanners
sexsearcher sexsearcher
siteripz sitechecker.pro
sysscan sogouspider
tAkeOut tAkeOut
trendiction.com
trendiction.de trendiction.de
webgains-bot voyagerx.com
x09Mozilla webmeup-crawler
xpymep1.exe webpros.com
webprosbot
zgrab

292
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 360Spider
404enemy 404enemy
80legs
ADmantX ADmantX
ALittle Client ALittle Client
ASPSeek
Abonti
Aboundex
Aboundexbot Aboundexbot
AfD-Verbotsverfahren
AhrefsBot
AiHitBot
Aipbot Aipbot
Alligator
AlphaBot AlphaBot
Asterias Anarchie
Attach Anarchy99
AwarioSmartBot Ankit
Anthill
Atomseobot
BDCbot
BDFetch BDFetch
BackDoorBot
BackStreet
Backlink-Ceck Backlink-Ceck
BacklinkCrawler Bandit
Barkrowler
BetaBot BetaBot
Bitacle BlackWidow
Blackboard
Blow
BlowFish
Boardreader Boardreader
Buck BotALot
Brandwatch
Buddy
BuiltBotTough BuiltBotTough
BuiltWith BuiltWith
Bullseye
BunnySlippers BunnySlippers
BuzzSumo BuzzSumo
CATExplorador CCBot
CODE87 CODE87
Calculon CazoodleBot
Cegbfeieh
CheTeam
CheeseBot CheeseBot
CherryPicker Copier
Chlooe
Citoid
Cliqzbot
Cloud mapping
Cocolyzebot
CopyRightCheck
Copyscape Copyscape
Crescent Craftbot
Curious Custo
DBLBot
DIIbot DIIbot
DSearch
DataCha0s
Deusu Deusu
Digincore
DigitalPebble DigitalPebble
Disco Discobot
Discoverybot
Dispatch
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot
DomainAppender DomainAppender
DomainSigmaCrawler DomainCrawler
DomainStatsBot DomainStatsBot
Domains Project Domains Project
Dotbot Drip
Download Wonder EasyDL
EMail Siphon
Ecxi
EirGrabber
EroCrawler EroCrawler
Express WebPictures ExtLinksBot
Extractor FlashGet
Extreme Picture Finder FrontPage
Ezooms
FDM
Freeuploader
Fuzz
Fyrebot Fyrebot
GT::WWW G-i-g-a-b-o-t
GermCrawler GetWeb
Getintent Gigabot
GrabNet Gotit
Grabber Grabber
Grafula GridBot
GrapeshotCrawler HTMLparser
HMView HTTrack
Haansoft HaosouSpider
Harvest Harvest
Havij Heritrix
HonoluluBot
Humanlinks Humanlinks
Iblog Iblog
Id-search
Image Sucker
Indy Library Indy Library
InfoNaviRobot
InfoTekies InfoTekies
Intelliseek
InterGET InterGET
Internet Ninja
InternetSeer
Iskanie Iskanie
IstellaBot
JamesBOT JamesBOT
Jbrofuzz
Jetty
JikeSpider
Joomla
Jorgee
JustView JustView
Kenjin Spider Jyxobot
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
LNSpiderguy Keyword Density
LWP::Simple Kinza
Lanshanbot
LexiBot LexiBot
Lftp Lftp
LibWeb LibWeb
Libwhisker LieBaoFast
Lightspeedsystems
LinkextractorPro
LinkpadBot LinkpadBot
LinqiaScrapeBot LinksManager
Lmspider Lipperhey Spider
Litemage_walker
MIIxpc MIIxpc
MSFrontPage MSFrontPage
Mag-Net
Mail.RU_Bot Mail.RU_Bot
Mass Downloader Majestic-SEO
Mata Hari Majestic12
Masscan
MauiBot MauiBot
Mb2345Browser Mb2345Browser
Meanpathbot
MegaIndex.ru MegaIndex.ru
Metauri Metauri
Moblie Safari MicroMessenger
Mojeek Microsoft URL Control
Minefield
Mojolicious
MolokaiBot MolokaiBot
Msrabot Mozlila
Mr.4x3
Musobot
NICErsPRO NICErsPRO
NPbot Nameprotect
Navroad Needle
Net Vampire
NetAnts NetAnts
NetMechanic NetZIP
NetSpider Nettrack
Netcraft Netvibes
NextGenSearchBot
Niki-bot
NimbleCrawler
Nimbostratus
Nuclei Nuclei
Octopus Nutch
Offline Explorer
OnCrawl
OpenLinkProfiler
Openfind Openfind
OrangeBot Openvas
OrangeSpider
OutclicksBot OutclicksBot
POE-Component-Client-HTTP OutfoxBot
Pandalytics Pandalytics
Panscient Panscient
Papa Foto
Pavuk
PeoplePal PeoplePal
Pi-Monster Petalbot
Picsearch
Piepmatz
ProWebWalker ProWebWalker
Probethenet Probethenet
Psbot Pump
Pu_iN RankingBot
QueryN Metasearch
Quick-Crawler
RankActiveLinkBot
RankingBot2 RankingBot2
ReGet RankurBot
RealDownload Reaper
RebelMouse RebelMouse
RedesScrapy Rogerbot
RepoMonkey SEOkicks-Robot
SBIder SEOprofiler
SISTRIX
ScanAlert ScanAlert
Scanbot
ScoutJet ScoutJet
SemanticJuice Scrapy
SemrushBot Screaming
Seekport
SentiBot
Siphon Siphon
SiteExplorer Siteimprove
SiteLockSpider Snake
Sitebeam SocialRankIOBot
Sociscraper Sociscraper
Sottopop SpankBot
SputnikBot Sqlmap
Steeler Steeler
Sucker SuperHTTP
Sucuri SurveyBot
Suzuran Suzuran
Swiftbot Szukacz
T0PHackTeam
T8Abot T8Abot
Telesphoreo Telesoft
The Intraformant The Intraformant
TheNomad TheNomad
Thumbor Thumbor
Toata Toata
Trendiction Trendictionbot
True_Robot
Turingos Turingos
TurnitinBot TwengaBot
Twice Twice
Typhoeus
URLy Warning URLy Warning
V-BOT URLy.Warning
Vagabondo VB Project
Vacuum
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
Virusdie
Voil
WASALive-Bot WASALive-Bot
WBSearchBot WBSearchBot
WWW-Collector-E WWW-Mechanize
WWWOFFLE
Wallpapers
WallpapersHD WallpapersHD
WeSEE WeSEE
Web Auto Web Fetch
Web Collage Web Pix
Web Sauger Web Sucker
WebAuto WebAuto
WebEnhancer WebBandit
WebFetch WebFetch
WebGo IS WebGo IS
WebReaper WebSauger
Webalta WebSucker
WebWhacker
WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Webshag
WebsiteExtractor WebsiteExtractor
Webster Webster
Whack
Whacker Whacker
Whatweb Whatweb
Who.is Bot
Widow Widow
WinHTTrack WiseGuys Robot
Wonderbot Wonderbot
Woobot
Wotbox
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider YoudaoBot
Zermelo Zeus
Zitebot
ZyBorg ZyBorg
arquivo-web-crawler adscanner
cah.io.community autoemailspider
coccocbot backlink-check
com.plumanalytics cognitiveseo
crawler.feedback
dataforseo.com dataforseo.com
eCatch
evc-batch
facebookscraper facebookscraper
gopher gopher
iubenda-radar instabid
linkdexbot ips-agent
isitwp.com
lwp-trivial
magpie-crawler magpie-crawler
oBot
page scorer page scorer
plumanalytics plumanalytics
probe-image-size ripz
s1z.ru scan.lol
seobility seobility
seostar seostar
sitechecker.pro siteripz
voyagerx.com sp_auditbot
webpros.com spyfu
x22Mozilla webgains-bot
zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
checkhit.com 2itech.ru
custodieva.ru automatic-seo.com
icaseclub.ru bizru.info
montredemarque.nl cialgenisrx.com
morlat.altervista.org musezone.ru
shop-electron.ru nikkiewart.ru
shopwme.ru pyrodesigns.com.au
studentguide.ru simul.co
teenforporn.com xrus.org
tiandeural.ru zoolubimets.ru

910
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com
03p.info 03p.info
108shot.com 108shot.com
123any.com 123any.com
2drittel.de 24-ak.ru
2itech.ru 3w24.com
2nt.ru 45en.ru
5u.com 4kepics.com
7minuteworkout.com 4webmasters.com
4webmasters.org
72-news.com
7search.com 7search.com
8jn.dating-app.ru 8jn.dating-app.ru
abwa.tk 91abcw.com
adimmix.com a1security.com.ua
adman.se aa8b68101d388c446389283820863176e7.com
adsfresh.men aac52006ec82a24e08b665f4db2b5013f7.com
adssafeprotected.com absolute-s.ru
agadelha.com.br academia-nsk.org
ahrntal.verymes.xyz adobereader-free.ru
akuhni.by adrunnr.com
alert-hgd.xyz ads-seo.men
altamayoztourism.com adzpower.com
amazon-adsystem.com afslankpillen2017nl.eu
amigobulls.com agysya.ru
anaseracresar.tk ahhjf.com
android4fun.org alfa-img.com
animalia-life.club algerianembassy.co.in
anjalika.co.in allwidewallpapers.com
allwomen.info
alvtank.se
amateurlivechat.org
amazon-seo-service.com
ameblo.jp
an-donut.com
anaksma.info
android-style.com
anonymousfox.co anonymousfox.co
apidiamondatanet-a.akamaihd.net apessay.com
apioasisspacenet-a.akamaihd.net apibatbrowsecom-a.akamaihd.net
apioutoboxnet-a.akamaihd.net apilinkswiftco-a.akamaihd.net
apple.com-webbrowsing-security.review apimegabrowsebiz-a.akamaihd.net
apple.com-webbrowsing-security.science approved.su
appleid-verification.com apxeo.info
aquarium-pro.ru
arabseyes.com arabseyes.com
arendas.net arcteryxstore.online
asdfg.pro ardimobileinfo.ml
autobrennero.it artdestshop.eu
artefakct.com
asrvrep-a.akamaihd.net
audiofree.ru
autochoixspinelli.com
autoloans.com
autolombard-krasnodar.ru autolombard-krasnodar.ru
autoseo-traffic.com autonew.biz
autotop.com.ua autorn.ru
autoua.top azbukadiets.ru
autowebmarket.com.ua azinoofficial777.ru
aviapanda.ru babyguns.ru
ayeartoforget.com bangla-chat-uk-paralud.ga
ayerbo.xhost.ro barnfurnituremart.com
balkanfarma.org bayareaaeroclub.org
balkanfarma.ru bazaronline24.ru
bankmib.ru bbsoldes.fr
bankofthewext.com bdsmgalls.net
bannerconnect.net belinvestdom.by
batanga.net bestchoice.cf
beauty-clinic.ru bestofupload.info
beauty-things.com besttorrentknifta.weebly.com
best-seo-offer.com bezsporno.ru
bestadbid.com big-cash.net
bestbrainprod.win biglistofwebsites.com
bestbuy.ca bingo8888.com
besthoro.ru birzha-truda.eu
bestkfiledxd.cf bkns.vn
bestpornuha.com blackle.com
bestsexyblog.com blobar.org
bestssaker.com blogporn.in
bestwaystogetpaid.us bobinoz.com
bidbuy.co.kr breastaugmentation.co.za
bin-brokers.com brillianty.info
bitcoin-ua.top builtwith.com
bitcoinremote.com buqyxa.rincian.info
bitnote.co buron.pw
blackcurranthumidifiers.site calc-for-credit.ru
blackplanet.com candypeople.se
blagovest-med.ru cardiosport.com.ua
blockworld.ru cartierreplica.top
bloggers.nl cartierreplica.win
bloglag.com cartierring.xyz
blogsfunky672.weebly.com cartujano-pre.de
boo-arts.com castingbank.ru
brainxs.us censys.io
braip.com.br cheapbarbour.online
breget74.ru cheapjerseysbizwholesale.us
bristolhotel.com.ua checkm8.com
buscarfoto.com chocolateslim-u-srbiji.com
buyessay3.blogspot.ru cialiswithoutadoctor.net
bwlx.prepedu.cn ciproandtizanidine.blogspot.com
cactussoft.biz claytransformations.info
cakemediahq.com.au clickbank.net
calcularpagerank.com
carabela.com.do
cdncash.net
chatroulette.life
cheapestjerseys-wholesale.com
cheapmoncler.xyz
cheapsergiorossi.online
cheatcode-lita12.rhcloud.com
check-this-out-now.online
chinaelitecheapjerseys.com
chinese-amezon.com
christianlouboutinreplica.win
chuckguilford.com
ci.ua
cialisndbrx.com
circlesl.com
clean-start.net
click2pawn.com
club-samodelkin.ru club-samodelkin.ru
coldfilm.ru cntravelre.com
compliance-checker-7.info cocyq.inwtrade.com
compliance-don.top codysbbq.com
compliance-elena.xyz com-cleaner.systems
compliance-irvin.xyz cometorussia.net
companies-catalog.com
compliance-fred.xyz
compliance-jess.top compliance-jess.top
compliance-john.top compliance-josh.top
compliance-margo.top
compliance-mark.top compliance-mark.top
cookie-law-enforcement-ee.xyz comsysnet.com
coolwallpapers-hd.com cool-wedding.net
coolwallpapers4k.info copenergo.ru
crackguru.tk copro.pw
cracksplanet.com cracksplanet.com
crackzplanet.com cryptoswap.biz
craftinsta.ru cs-passion.pl
crystalslot.com csgo4.win
custodieva.ru customcatchcan.com
customboxes4less.com customerguru.in
cutt.us customsua.com.ua
d0t.ru cwetochki.ru
daptravel.com
datadepths.com
dealighted.com
decorazilla.com
dentalpearls.com.au dentalpearls.com.au
detskie-zabavi.ru dikqlyremy.info
deutschland123.de dimkino.ru
dev.citetick.com dkvorota.ru
dfwu1019.info documentsite.net
dimaka.info
dirtpics.pw
doctissimo.fr
documentbase.net
dood.live
dostavka-v-krym.com
downloadkakaotalk.com
downloadmefiranaratb1972.xpg.com.br downloadmefiranaratb1972.xpg.com.br
dptaughtme.com downtuptv.gq
draniki.org drunkenstepfather.com
driving.kiev.ua dustyorate.com
dvr.biz.ua
e-commerce-seo1.com
ecommerce-seo.org ecommerce-seo.org
editmedios.com
edudocs.net
eets.net
ekosmetyki.net.pl ekosmetyki.net.pl
elektir.ru
elektrischezigarette1.blog.pl
elektronischezigarettekaufen1.beeplog.com
elitepcgames.com elitepcgames.com
elitesportsadvisor.com enhand.se
empis.magix.net essaypro.com
enbersoft.com estibot.com
energy-ua.com everytalk.tv
enge-fotzen.info
enternet.ee
eshop.md
esnm.ru
euromasterclass.ru
exci.ru exci.ru
ezigarettenshop2.myblog.de expertnaya-ocenka.ru
ezigaretteshop2.mywapblog.com expolicenciaslatam.co
ezofest.sk eyessurgery.ru
f00kclan.de ezigarettekaufen2.blox.pl
farmingworm.com ezigaretteshop2.vefblog.net
fashionavenuegame.com f07.de
feeriaclub.ru f5mtrack.com
findclan.org fable.in.ua
flashbannernow.com fashion-stickers.ru
fodelsedagspresenter.nu fastcrawl.com
forex.osobye.ru favorcosmetics.com
fotosfotos.eu fettefrauen.net
freshwallpapers.info filmidivx.com
fun-mobi.pl findacheaplawyers.com
fusoradio.info forex-procto.ru
g33.org forminecrafters.ru
gameprimary.com foto-sisek.porngalleries.top
games.kolossale.ru foxydeal.com
gateway.zscalerone.net frameimage.org
gdzkurokam.ru frbizlist.com
gelezki.com free-share-buttons.top
germetiki.com.ua freshmac.space
getmiro.com fun2cell.net
gettpromos.com fx-brokers-review.com
girlgamerdaily.com gamevalue7.weebly.com
globalscam.ga gateway.zscalertwo.net
globalsurfari.com gaz-voshod.ru
gobongo.info gembird.com
gonextmedia.com genetworx.com
gopixdatabase.com getyourimage.club
gta-top.ru ggiaro.com
guruofcasino.com gigapeta.com
hdapp1008-a.akamaihd.net girlspicsa.com
hdfreeporno.net gnuetella.com
hdpixent.com goldenggames.com
helpmymacfaster.trade googlepositions.com
hidemyass.com gopro-online.info
high-speed1.net goroda-vsego-mira.ru
highstairs-a.akamaihd.net gotcher.us
himazin.info gotwebsite1.com
hit-men.men guidefs.ru
hol.es h2monline.com
homelygarden.com hawaiisurf.com
houtings.xyz hdwallpapers-free.com
hoverboard360.se hdxnxxtube.mobi
humbmosquina.tk healbio.ru
hyipmanager.in healmytrauma.info
ifmo.ru heartofpayne.xyz
ilikevitaly.com hermesbirkinhandbagoutlets.com
imfamous.info hikesearch.net
integritylandscapeservices.com hobbyhuren-datenbank.com
internetartfair.com hobild.net
invest-pamm.ru hostingclub.lk
hotblog.top
housediz.com
hoverboard360.at
hoztorg-opt.ru
id-forex.com
ideashome.id
igadgetsworld.com
img.wallpaperstock.net
indianmedicaltourismshop.com
insomniagamingfestival.com
intermesh.net
investsuccess.org investsuccess.org
investyb.com
invitefashion.com
iplusbit.blogspot.co.za
iptool.xyz iptool.xyz
it-max.com.ua iqoption.pro
itch.io iradiology.ru
ivearchenceinflu.cf istizanidineanarcoticdrug.blogspot.com
itag.pw
iusstf.org
japfm.com japfm.com
javrip.net javitas.info
javxxx18.com joessmogtestonly.com
jeremyeaton.co johnnyhaley.top
jmat.cn kakablog.net
johnrobertsoninc.com kamalsinha.com
joy-penguin.com kamen-e.ru
jyrxd.com kamorel.com
keki.info karadene.com
keywordsking.com kartiniresto.com
kicknights.gq keywords-monitoring-success.com
kino-ecran.ru keywordteam.net
kino2018.club kichenaid.ru
kinofak.net kinofak.net
kirov.zrus.org kinogonew.ru
kmgamex.cf klumba55.ru
kollesa.ru konoplisemena.com
kosova.de konturkrasoty.ru
kotaku.com kosynka-games.ru
krasivoe-hd.com krasivoe-hd.com
kretpicf.pw lacapilla.info
lacasamorett.com ladov.ru
ladsblue.com ladsblue.com
lakomka.com.ua landinez.co
law-check-seven.xyz laudit.ru
lexixxx.com lazymanyoga.com
lineavabit.it lenpipet.ru
linkbuddies.com lidoradio.com
lkbennettoutlet.online lifehacĸer.com
lolnex.us linkarena.com
lombia.com lisque.batcave.net
lonerangergames.com livesex-amateure.info
lonfon.xyz lodki-pvh.dp.ua
lostaruban.com losangeles-ads.com
lottospring.com louboutinshoes.xyz
lovi-moment.com.ua lunchrock.co
m1media.net m292.info
machicon-ueno.info macfix.life
mactechinfo.info mainlinehobby.net
make-money-online.com makedo.ru
makis.nu mall.uk
maladot.com marktforschung-stuttgart.com
manimpotence.com maximilitary.ru
mansiondelrio.co mb140.ru
matsdale.com mebelcomplekt.ru
max-eclat.men med-zdorovie.com.ua
medicine-4u.org medicine-4u.org
metallosajding.ru medispainstitute.com.au
mexicotravelnet.com medizinreisen.de
mitrasound.ru member-quiz.com
mlf.hordo.win metalonly.info
mmog-play.ru mil-stak.com
moivestiy.biz mini.7zap.com
mokrayakiska.com mmostrike.ru
monclerjacketsoutlet.pw mobicover.com.ua
mostantikor.ru moneyzzz.ru
mural.co moscow-clining.ru
mustat.com mustat.com
muzaporn.com mycouponizemac.com
mwtpludn.review mydearest.co
myghillie.info mydownlodablefiles.com
myindospace.com myxdate.info
myliveblog.ru mzdish.site
myra.top naj-filmy24.pl
naphukete.ru
narco24.me narco24.me
narkologiya-belgorod.ru nelc.edu.eg
nikhilbahl.com new7ob.com
nina.az newtechspb.ru
njkmznnb.ru nl.netlog.com
novatech.vn nodding-passion.tk
o-o-6-o-o.com nosecret.com.ua
o-o-8-o-o.ru noumeda.com
obnal.org o333o.com
obsessionphrases.com obiavo.net
officedocuments.net odesproperty.com
okeinfo.online of-ireland.info
oksrv.com offer.wpsecurity.website
onblastblog.online offside2.5v.pl
online-podarki.com oneminutesite.it
online-zaymy.ru onlinefilmz.net
onlineslotmaschine.com onstrapon.purplesphere.in
ontargetseo.us
ooo-gotovie.ru ooo-gotovie.ru
orsonet.ru ooo-olni.ru
ovirus.ru oyster-green.com
paleohub.info partner-stop.men
palvira.com.ua pawli.eu
pandroid.co petedrummond.com
panouri-solare-acoperis.com peugeot-club.org
partner-trustworthy.men
pb-dv.ru
pdfprof.com
perl.dp.ua
perosan.com
photokitchendesign.com photokitchendesign.com
phuketscreen.com piccdata.com
pintattoos.com picurams.pw
pirateday.ru pila.pl
pixgood.com pizza-tycoon.com
plastgran.com play.leadzupc.com
plastgranar.nu playtap.us
playmsn.com poem-paying.gq
pliks.pl popads.net
plugingeorgia.com poptool.net
pokemon-go-play.online pornhive.org
pokerniydom.ru porno-video-chati.ru
pony-business.com pornoblood.com
porno-home365.com pornoreino.com
pornophoto.xyz pornoted.com
pornotubexxx.name portnoff.od.ua
princeadvantagesales.com porto.abuilder.net
privatamateure.com powenlite24.ru
privatov-zapisi.ru pozdrawleniya.com
professionaldieselcare.com prize44.com
profitfx.online produkto.net
prohoster.info pufip.com
pronorm.fr puteshestvennik.com
prostitutki-astana.org pwwysydh.com
psoriasis-file.trade qpypcx.com
pssucai.info quick-seeker.com
pulse33.ru quickchange.cc
purplesphere.in ranking2017.ga
puzo2arbuza.ru ranksonic.org
quickbuck.com
quickloanbank.com
quizzitch.net
rarbg.to rarbg.to
razamicroelectronics.com realting-moscow.ru
realitykings.com remont-fridge-tv.ru
rebelmouse.com remont-komputerov-notebook.ru
recipedays.com responsinator.com
relax.ru reversing.cc
renecaovillasale.online rewardpoll.com
rentalcarnavi.info rialp.getenjoyment.net
resellerclub.com ribieiendom.no
ringporno.com
rockma.se rockma.se
rollercoin.com
royal-betting.net royal-betting.net
russian-postindex.ru ru-mediaget.ru
rusvideos.su rubbed.us
sady-urala.ru ruex.org.ua
saitevpatorii.com sakhboard.ru
sajatvelemeny.com
sardinie.us
sarf3omlat.com sarf3omlat.com
savefrom.com sbornik-zakonov.ru
saveriopiazza.it scanner-john.top
sbf441.com scanner-nelson.top
scanner-brian.top scanner-viktor.top
scmor.ilxc.cc scanspyware.net
scopich.com secret.xn--oogle-wmc.com
screpy.com seminarykansas904.ml
se-welding.ru seoprofiler.com
sei80.com sex-znakomstva.online
seofied.com sexad.net
seokicks.de sexobzor.info
seomarketings.online sexsearch.com
serving.adbetclickin.pink sexuria.net
servisural.ru signx.info
sex-tracker.com simplepooltips.com
sexiporno.net siteaero.com
sexpartygirls.net
sexspornotub.com
sexy-pings.com
sexytrend.ru
shopcheermakeup.info
shopfishing.com.ua
sideeffectsoftizanidine.blogspot.com
sierraapps.com
silverage.ru
silvercash.com
simple-image.com.ua
site3.free-share-buttons.com
sk.golden-praga.ru
skladvaz.ru skladvaz.ru
sky-mine.ru
smallseotools.com smallseotools.com
snsdeainavi.info sms2x2.ru
snow.nvr163.com
snts.shell-pmr.ru snts.shell-pmr.ru
snworks.com
sobecjvuwa.com.ru
social-s-hhh.xyz social-s-hhh.xyz
social-s-iii.xyz softomix.com
soda.media soheavyblog.com
soserfis.com solution4u.com
specialfinanceoffers.com sotkal.lark.ru
speedup-my.site spammen.de
spidtest.org spasswelt.xyz
sport-video-obzor.ru spbchampionat.ru
spywarenuker.com spidtest.space
stairliftsarea.com sportbetfair.com
starwars.wikia.com
steebook.com
storehouse.ua storehouse.ua
stroiminsk.com
stroyplus.ru
sucsesofinspiration.com
suggest-keywords.com
superiends.org superiends.org
svtrd.com supermama.top
swinginwithme.ru surgut.zrus.org
sygraem.com svensk-poesi.com
swingerseiten.com
swsociety.se
t3chtonic.com t3chtonic.com
takeprofitsystem.com tacbibirfa.tk
tedy.su tamada69.com
tehngr.ru tampabaywatch.org
telegraf.by tattoo33.ru
telesvoboda.ru tds-advert002.info
texbaza.by teslathemes.com
thedownloadfreeonlinegames.blogspot.com textads.men
thegameriders.com theprofitsmaker.net
thetardistimes.ovh thiegs.reco.ws
tizanidineforopiatewithdrawal.blogspot.com tiandeural.ru
tizanidineonline.blogspot.com titangel-vietnam.com
tmtrck.com tizanidinefibromyalgia.blogspot.com
toon-families.com tizanidinehcl4mgtab.blogspot.com
top1-seo-service.com tizanidineinteractions.blogspot.com
top10-way.com tizanidinewithdrawalsymptoms.blogspot.com
torrents.cd tjkckpytpnje.com
totu.info tnctrx.com
toxicwap.com tomck.com
tradedeals.biz totu.us
traderzplanet.in toyota.7zap.com
trafficmonetize.org tozup.com
tpu.ru
tracfone.com
trackzapper.com
trafficmania.com
traffictrade.life traffictrade.life
traffixer.com treasuretrack-a.akamaihd.net
traffpartners.com trichizobswiv.agddns.net
truebeauty.cc trubywriting.com
trumpetedextremes.com
tula.mdverey.ru
tupper-shop.ru tupper-shop.ru
turizmus.us turist-strani.ru
tv-spoty.info twiclub.in
tvand.ru uggbootsoutletsale.us
tvorozhnaja-zapekanka-recept.ru uginekologa.com
tyumen.xrus.org
udsgame.online
uhod-za-sobakoj.ru uhod-za-sobakoj.ru
ultimatesetnewfreeallsoftupgradesystems.pw ultramart.biz
unitexindia.com uni.me
uptime-alpha.net unlimitdocs.net
url-img.link upproar.com
url2image.com urll.eu
vabasa.inwtrade.com usdx.us
velobikestock.com v720hd.ru
vezuviy.su vacuumcleanerguru.com
videospornogratisx.net vadimkravtcov.ru
videtubs.pl validccseller.com
vikistars.com vchulkax.com
vkak.ru vedomstvo.net
vpnhowto.info vektorpress.ru
vpnmouse.com viagengrarx.com
we-are-gamers.com videojam.tv
web-betting.ru videonsk.com
webmasters.stackexchange.com vip2ch.com
webnode.me vipsiterip.org
webstatsdomain.org vita.com.hr
wieseversa.no vkmusics.ru
witclub.info voditeltrezviy.ru
wordpress-crew.net vodkoved.ru
vzlom-na-zakaz.com
wdrake.com
webalan.ru
webix.me
weblo.com
websiteexplorer.info
websitevaluebot.com
weekes.biz.tc
wemarketing.se
whiteproduct.com
wholesalecheapjerseysfree.com
whos.amung.us
wildworld.site
williamrobsonproperty.com
wingsofrefuge.net
wladimirpayen.com
wowcasinoonline.ooo
wrona.it wrona.it
www.bookmaker-bets.com www.888.com
wyniki-lista.pl www.labves.ru
x5market.ru www.timer4web.com
xblognetwork.com xclicks.net
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai xingzi-vision.com
xn----ctbbcjd3dbsehgi.xn--p1ai xitjw.info
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xpresscare.ru xn--80ahvj9e.xn--p1ai
xsfetish.org xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
xxxkaz.org xtube.com
yoluxuryevents.com xxxdatinglocal.us
youporn-forum.ga y8games-free.com
yourporngay.com yeartwit.com
yunque.pluto.ro yebocasino.co.za
z-master.ru yebocasino.com
zambini.ru yellowstonesafaritours.com
zapatosenventa.info youandcredit.ru
zirondelli.it youradulthosting.com
zkjovpdgxivg.ga yubikk.info
znakomstva-piter78.ru zigarettenonlinekaufen1.bloog.pl
zoolubimets.ru znakomstvaonlain.ru
zophim.me zoogdisany.com
zygophyceous.womanstars.site zscalerone.net
zytpirwai.net zvezdagedon.ru
zynax.ua

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00go.com 101raccoon.ru
2rich4bitches.com 12bet.com
3-letter-domains.net 12masterov.com
3dracergames.com 13tabs.com
5000-cotydzien.com 1xbet4you.com
777-club.ru 273-fz.ru
7a2rnuey1tw9ar.ru 4kepics.com
91abcw.com 8si.ru
aa8780bb28a1de4eb5bff33c28a218a930.com a.frcls.fr
abcdeg.xyz aac52006ec82a24e08b665f4db2b5013f7.com
adclickthru.net acgs.tk
adminshop.com adelly.bg
adtology.com adf.ly
advancedmassagebysara.com ads.tremorhub.com
an-donut.com adtech.de
apparel-offer.com aibolita.com
arabsexxxtube.com aihelen.net
arcteryxsale.online aktobe.xkaz.org
ascat.porn akuhni.by
asmxsatadriverin.aircus.com akvamaster.dp.ua
asscenihotosidea.blogspot.com allkrim.com
asseenontv.ru amazon-seo-service.com
australia-opening-times.com angel1777.kz
avitocars.ru angigreene.com
aviva-limoux.com apidiamondatanet-a.akamaihd.net
baersaratov.ru appartement-stumm.at
balla-boo.se apps-analytics.net
bankiem.pl atlant-auto.info
best-way.men atlasvkusov.ru
bestcoin.cc bbtec.net
bestprofits-there.com beauby.ru
beta.hotkeys.com beclean-nn.ru
betterhdporn.com best-deals-products.com
bikini-image.com bestserials.com
bin-brokers.com bio.trade-jp.net
blackle.com birzha-truda.eu
bluejays-jerseys.us bitcoinremote.com
boazpower.com blogqpot.com
bookmark4you.com.biz blpmovies.com
budpost.com.ua boleznikogi.com
buketeg.xyz bookmark4you.com
buyhoverboard.com boom-celebs.com
calc-for-credit.ru bosefux.esy.es
cannibalcheerleader.com buy-forum.ru
ccbill.com buzzzg.men
cheap-pills-norx.com callcafe.info
chocoslim.pro callstevens.com
ciceros.co cctva.tv
cleanersoft.com cheapestjerseysonwholesale.com
clickaider.com clash-clans.ru
clickbanksites.info cognitiveseo.com
clickintext.net compliance-fred.top
collegeessay19.blogspot.ru connectingsingles.com
compliance-don.xyz contextualyield.com
connectionstrenth.com converse.ddsoldes.fr
covid-schutzmasken.de cougfan.info
dinosaurus.site counter.bloke.com
dirtpics.pw culmimg.pw
dispo.de dawlenie.com
domainsphoto.com ddpills.com
domznaniy.ru defenderxtactical.com
dotnetdotcom.org dengi-pod-zalog-nedvizhimosti.ru
draniki.org detsky-nabytek.info
dstroy.su divatraffic.com
dvd-famille.com dktr.ru
ebonyporn.site dnepr-avtospar.com.ua
ecommerce-seo.com drivotracker.com
ee77ee.com e-biznes.info
ekaterinburg.xrus.org ec-file.info
eets.net
ekosmetyki.net.pl ekosmetyki.net.pl
elektrischezigarette1.blog.pl energy-ua.com
elektrischezigaretten2.beeplog.com exmasters.com
elektroniksigaraankara.info extremez.net
elektrozigarettekaufen2.blogse.nl ezigarettenshop1.yolasite.com
elvenmachine.com ezigaretteshop2.vefblog.net
enge-fotzen.info ezofest.sk
escort-russian.com faithe.top
esnm.ru falllow.gq
esopini.com famix.xyz
etenininrade.ga faz99.com
eurosamodelki.ru filerockstar298.weebly.com
ezigarettekaufen2.yolasite.com filmi-onlain.info
farmingworm.com findclan.org
fashion-mk.net finejewelryshop.ru
fenoyl.batcave.net finstroy.net
ferretsoft.com flauntyoursite.com
fotoxxxru.com fortwosmartcar.pw
free-stock-illustration.com foto-basa.com
gay.adultgalls.com foto-telok.net
genericviagrasildenafiled.net foxjuegos.com
get-free-traffic-now.com free-social-buttons.xyz
getmiro.com genericlowlatencyasiodriverhq.aircus.com
glavtral.ru get-seo-domain.com
goatse.ru getpopunder.com
goblacked.com ggiaro.com
gomusix.com gheus.altervista.org
googlemare.com gilsonchiro.xyz
gorodservis.ru haikuware.com
gosmeb.ru hamacapty.com
greatidea.marketing hawaiielectriclight.com
gta-club.ru hdapp1008-a.akamaihd.net
guardlink.com herehloadibs.cf
habermetre.com high-speed1.net
hawaiisurf.com homedecorpicture.us
healing-dysplasia.ru housemilan.ru
hermesbirkinhandbagoutlets.com huimin764128.com
hiwibyh.bugs3.com id-forex.com
hotchatdate.com imicrovision.com
howlongdoestizanidinestayinyoursystem.blogspot.com immigrational.info
hspline.com incolors.club
humanelydrew.com investzalog.ru
iboss.com ip-guide.com
ihc2015.info ispaniya-costa-blanca.ru
images-free.net izhevsk.xrus.org
imfamous.info j-times.ru
imguramx.pw javrip.net
impresagaia.it jaxcube.info
increasewwwtraffic.info juliadiets.com
infostatsvc.com kakadu-interior.com.ua
interferencer.ru keywords-monitoring-success.com
janerikholst.se kicknights.gq
jewelryandfiligree.com kinofak.net
jjbabskoe.ru kiwi237au.tk
jobgirl24.ru knowyournextmove.com
keywesthideaways.co koversite.info
kinoflux.net kumuk.info
kmgamex.cf kurbappeal.info
ktotut.net larchik.net
leftofcentrist.com law-check-seven.xyz
lenvred.org lernur.net
letmacwork.world lisque.batcave.net
lexaprogeneric.link livingroomdecor.info
linkarena.com lol-smurfs.com
littleberry.ru longlifelomilomi.info
longgreen.info luciddiagnostics.in
mail.allnews24.in marketingtechniques.info
makenahartlin.com matchpal-a.akamaihd.net
menstennisforums.com mega-bony-2017.pl
mexicosleevegastrectomy.com mex-annushka.ru
misswell.net minneapoliscopiers.com
modnie-futbolki.net mirmedinfo.ru
monclercheap.xyz missis.top
monclerjacketsoutlet.pw mncrftpcs.com
musicvidz.ru moneytop.ru
my-bc.ru monsterdivx.com
mydirtyhobby.com montredemarque.nl
narkologiya-penza.ru mp3s.club
narkologiya-voronezh.ru msk.afora.ru
oakridgemo.com musicstock.me
olgacvetmet.com mydirtystuff.com
online-hd.pl mydoctorok.ru
osb.se11.ru mymobilemoneypages.com
pdns.download njpalletremoval.com
novgorod.xrus.org
onlyforemont.ru
openfrost.com
owohho.com
pandroid.co
partnerworkroom.men
paydayloanslocal.com
pelfind.me pelfind.me
pflexads.com perosan.com
php-market.ru photokitchendesign.com
pobeiranie.pl pk-pomosch.ru
placid-rounded-coast.glitch.me
platezhka.net
play.leadzupc.com
polcin.de polcin.de
pomoc-drogowa.cba.pl polytopesexempt.com
popmarker.com pozdrawleniya.com
popup-fdm.xyz pregnant.guru
pornhub-ru.com prlog.ru
portadd.men profitwithalex.info
portal-eu.ru prohoster.info
preg.marketingvici.com putevka24.ru
pretty-mart.com quizzitch.net
priscilarodrigues.com.br realmonte.net
pro-okis.ru reklama1.ru
profitfx.online respublica-otel.ru
progonrumarket.ru reyel1985.webnode.fr
promoforum.ru rfd-split.hr
prostitutki-tolyatti.org sady-urala.ru
qwertty.net salmonfishingsacramentoriver.com
raavidesigns.com saratov.xrus.org
radiogambling.com scanner-donald.top
responsinator.com scanner-jess.top
rethinkwasteni.info scenicmissouri.us
reversing.cc search.pch.com
roofers.org.uk searchtooknow-a.akamaihd.net
rukino.org seccioncontrabajo.com
russian-postindex.ru seksvideoonlain.com
s-iwantyou.com seoholding.com
scanner-barak.top
seomarketings.online seomarketings.online
sexvideo-sex.com sexkontakte-seite.com
shohanb.com sexvporno.ru
sideeffectsoftizanidine.blogspot.com sfd-chess.ru
site5.com sispe.com.br
skuteczna-dieta.co.pl skylta.com
slayerlife.com slayerlife.com
slimcdn.com sledstvie-veli.net
soda.media slujbauborki.ru
sport7777.net songoo.wz.cz
sportbetfair.com spammen.de
sweepstakes.rewardit.com staynplay.net
symphonyintegratedhealthcare.com storehouse.ua
taaaak.com stretchingabuckblog.com
tam-gde-more.ru stroiminsk.com
teenporn18.net sucsesofinspiration.com
thedownloadfromwarez.blogspot.com surflinksmedical.com
theprofitsmaker.net svnuppsalaorebro.se
threecolumnblogger.com theallgirlarcade.com
timeallnews.ru tiens2010.ru
timer4web.com tk-assortiment.ru
tizanidine4mgprice.blogspot.com topkarkas.com
tizanidineorflexeril.blogspot.com torrent-newgames.com
tizanidinevssoma.blogspot.com uprour.com
tongkatmadura.info urengoy.pro
top-deal.com.pl uvozdeckych.info
topvidos.ru valsalud.com
torrents-tracker.com velobikestock.com
tracklead.net viagra.pp.ua
trafficfactory.biz viddyoze.com
trailer.cinemaflix.website videospornogratisx.net
trustedmaccleaner.com visionwell.com.cn
turbabitload.weebly.com vodaodessa.com
turbodsp.com vvon.co.uk
u555u.info w-journal.ru
umaseh.com weddingdresses.xyz
unacittaconte.org wellcome2slovenia.ru
unpredictable.ga wingsofrefuge.net
usdx.us wnhjavlhezp.gq
utrolive.ru wordpresscore.com
viagra-soft.ru x-diesel.org
voloslove.ru x-lime.net
vroze.com xingzi-vision.com
vucms.com xkaz.org
vzube.com xn--b1agm2d.net
watchinf.com xn--oogle-wmc.com
waycash.net xnxx-n.com
webcamdevochka.com yellowproxy.net
website-speed-check.site yes-com.com
websitevaluebot.com yhit.press
wedding0venues.tk ypmuseum.ru
whiteproduct.com
wjgony.com
xev.ru
xgftnlrt.bloger.index.hr
xmnb.net
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn--e1afanlbnfckd7c3d.xn--p1ai
yebocasino.com
yourothersite.com
yourporn.com
yunque.pluto.ro yunque.pluto.ro
zeg-distribution.com zaapplesales.blogspot.com
zeikopay.com zaim-pod-zalog-krasnodar.ru
znakomstvaonlain.ru zawyna.ua
zero1.it
znaturaloriginal.com znaturaloriginal.com
zoogdiesney.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.105 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3056 ### Version: V4.2022.04.3057
### Updated: Wed Apr 20 22:01:32 UTC 2022 ### Updated: Thu Apr 21 10:03:33 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3056 ### Version: V4.2022.04.3057
### Updated: Wed Apr 20 22:01:32 UTC 2022 ### Updated: Thu Apr 21 10:03:33 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##