vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2938 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-22 22:07:19 +00:00
parent 1f681ec42f
commit 786320717f
47 changed files with 91049 additions and 91059 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2937 #### Version: V4.2022.02.2938
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2937 #### Version: V4.2022.02.2938
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2937 #### Version: V4.2022.02.2938
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11248
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2937 #### Version: V4.2022.02.2938
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11252
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11252
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11252
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11252
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 1; 10.1.1.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 0; 10.1.1.68 0;
127.0.0.1 0; 127.0.0.1 0;

11252
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 1; 10.1.1.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 0; 10.1.1.68 0;
127.0.0.1 0; 127.0.0.1 0;

11252
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 1; 10.1.1.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 0; 10.1.1.68 0;
127.0.0.1 0; 127.0.0.1 0;

11252
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2937 2938

11252
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11252
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.121.162.178 0; 20.190.34.94 0;

11252
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 1; 10.1.1.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 0; 10.1.1.68 0;
127.0.0.1 0; 127.0.0.1 0;

11252
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 1; 10.1.1.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 0; 10.1.1.68 0;
127.0.0.1 0; 127.0.0.1 0;

11252
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 1; 10.1.1.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 0; 10.1.1.68 0;
127.0.0.1 0; 127.0.0.1 0;

11252
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.121.162.178 0; 20.190.34.94 0;

11252
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.121.162.178 0; 20.190.34.94 0;

11252
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 1; 10.1.1.68 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BDFetch Custo
Bigfoot MTRobot
BlackWidow Offline Navigator
Discobot ReGet
Ecxi Seekport
Morfeus Fucking Scanner TightTwatBot
Teleport TwengaBot
Turnitin WPScan
TurnitinBot eCatch
Web Sauger lwp-trivial

296
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 360Spider
404enemy
80legs 80legs
ADmantX
Abonti Abonti
Aboundexbot Aboundexbot
AfD-Verbotsverfahren AllSubmitter
AiHitBot Alligator
Anarchie
Ankit Ankit
Aspiegel Anthill
Asterias Apexoo
Atomseobot
AwarioRssBot AwarioRssBot
AwarioSmartBot
BBBike BBBike
Backlink-Ceck BLEXBot
Badass BackDoorBot
BackStreet
Bandit
Barkrowler Barkrowler
BetaBot BetaBot
Bigfoot Bigfoot
Black Hole Blackboard
BlackWidow Blow
BlowFish Buddy
Brandprotect BuzzSumo
BuiltBotTough
BuiltWith
Bullseye
CATExplorador CATExplorador
CCBot
CODE87
CSHttp CSHttp
CensysInspect
CheTeam CheTeam
CheeseBot
ChinaClaw ChinaClaw
Chlooe Chlooe
Citoid
Claritybot
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot Cogentbot
Copier
Copyscape Copyscape
Cosmos
CrazyWebCrawler CrazyWebCrawler
Crescent
Custo Custo
CyotekWebCopy DIIbot
DSearch Deusu
DataCha0s Dispatch
Devil
DigitalPebble
Dirbuster
Discobot
Discoverybot
DnBCrawler-Analytics DnBCrawler-Analytics
DomCopBot
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Dragonfly DomainStatsBot
Drip Domains Project
Dotbot
Download Wonder
ECCP/1.0
EMail Wolf
EasyDL EasyDL
Ecxi EroCrawler
Evil ExtractorPro
Exabot EyeNetIE
ExtLinksBot Ezooms
Extractor Firefox/7.0
FemtosearchBot Flunky
FlashGet
Freeuploader Freeuploader
FrontPage FrontPage
GalaxyBot Fuzz
G-i-g-a-b-o-t
Genieo
GermCrawler GermCrawler
Gigabot GetRight
Go-Ahead-Got-It GetWeb
Gotit
GrabNet GrabNet
Grabber
GrapeFX GrapeFX
GrapeshotCrawler
HEADMasterSEO
HMView HMView
HTMLparser
HTTP::Lite HTTP::Lite
HaosouSpider Harvest
Heritrix Heritrix
Humanlinks Humanlinks
HybridBot
IDBTE4M
IlseBot IlseBot
Image Fetch Image Fetch
Image Sucker
IndeedBot
Indy Library Indy Library
InterGET InternetSeer
IstellaBot IstellaBot
JOC Web Spider Jbrofuzz
Jetty JennyBot
JikeSpider Kenjin Spider
Joomla
Jyxobot
Keyword Density Keyword Density
Kinza
Kozmosbot Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple Lanshanbot
LieBaoFast LeechGet
Likse Libwhisker
LinkpadBot Lightspeedsystems
LinksManager LinkScan
LinqiaRSSBot Linkbot
Ltx71 LinqiaScrapeBot
Lmspider
MIDown tool MIDown tool
MIIxpc MQQBrowser
MSFrontPage
MSIECrawler MSIECrawler
Mail.RU_Bot Mag-Net
Majestic SEO Magnet
Meanpathbot Majestic-SEO
Microsoft Data Access Majestic12
Minefield MarkWatch
Mass Downloader
Mb2345Browser
MeanPath Bot
Mister PiX Mister PiX
Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Musobot
Mr.4x3 NICErsPRO
Name Intelligence NPbot
Net Vampire Navroad
Nettrack Needle
Nessus
NetAnts
Netcraft
NextGenSearchBot NextGenSearchBot
Nibbler Ninja
Nimbostratus Not
Nuclei
Offline Navigator Offline Navigator
OrangeSpider OpenLinkProfiler
Openfind
Openvas
OrangeBot
OutfoxBot OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl PHPCrawl
POE-Component-Client-HTTP PageGrabber
Panscient PageScorer
PageThing.com
Pavuk Pavuk
PeoplePal
Petalbot Petalbot
Picscout Pi-Monster
Picsearch Picsearch
Piepmatz Piepmatz
Probethenet Pimonster
Pockey
ProPowerBot
ProWebWalker
Pump Pump
PxBroker
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
RankingBot RealDownload
RankurBot
Reaper Reaper
Ripper Recorder
Rogerbot RepoMonkey
SEOkicks-Robot RocketCrawler
SBIder
SEOlyticsCrawler SEOlyticsCrawler
SalesIntelligent SMTBot
Scrapy Scanbot
Screaming Seekport
Searchestate
SemanticJuice
Semrush Semrush
SentiBot SeoSiteCheckup
Shodan
Siphon
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteSnagger SiteSnagger
Siteimprove SiteSucker
Sitevigil SlySearch
Snake
Snapbot Snapbot
Snoopy SocialRankIOBot
Sociscraper
Sogou web spider Sogou web spider
Sosospider
Sottopop Sottopop
Spammen SpaceBison
Spanner
Spbot
Spinn3r Spinn3r
SputnikBot Stripper
Sqlmap
Sqlworm
Sqworm
Sucker
Sucuri Sucuri
Surfbot
SurveyBot
Suzuran Suzuran
Szukacz Swiftbot
TeleportPro T0PHackTeam
Toata Teleport
Toweyabot Telesphorep
Tracemyfile Tracemyfile
Trendictionbot Trendictionbot
Turingos TwengaBot
Twice URLy Warning
Typhoeus
URLy.Warning URLy.Warning
UnisterBot UnisterBot
Upflow
VB Project VB Project
VCI VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie
VoidEYE VoidEYE
Voil Voil
WASALive-Bot Voltron
WBSearchBot WBSearchBot
WPScan
WWW::Mechanize WWW::Mechanize
WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WeSEE Web Collage
Web Auto
Web Enhancer Web Enhancer
Web Pix Web Fuck
WebBandit
WebCopier WebCopier
WebFetch
WebFuck
WebGo IS
WebImageCollector WebImageCollector
WebPix WebSauger
WebStripper WebWhacker
WebSucker Webster
WebZIP
Webshag
Whatweb Whatweb
Who.is Bot Who.is Bot
Zade Woobot
Zeus Wprecon
Xaldon WebSpider
Zauba
Zermelo
Zitebot Zitebot
ZmEu
ZoomBot ZoomBot
ZyBorg
adscanner adscanner
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
autoemailspider
backlink-check
coccocbot coccocbot
cognitiveseo cognitiveseo
crawl.sogou.com com.plumanalytics
crawler.feedback dataforseo.com
eCatch demandbase-bot
evc-batch gopher
facebookscraper
internetVista monitor
isitwp.com isitwp.com
lwp-request
lwp-trivial lwp-trivial
magpie-crawler
meanpathbot
muhstik-scan muhstik-scan
netEstate NE Crawler
oBot
pcBrowser pcBrowser
plumanalytics
s1z.ru s1z.ru
satoristudio.net satoristudio.net
scalaj-http scalaj-http
seobility scan.lol
seocompany.store seocompany.store
seoscanners
seostar seostar
sitechecker.pro sitechecker.pro
siteripz
sogouspider sogouspider
sysscan trendiction.com
voyagerx.com trendiction.de
webmeup-crawler webgains-bot
webpros.com
webprosbot webprosbot
zgrab x09Mozilla
xpymep1.exe

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 360Spider
ADmantX 80legs
Abonti Abonti
Aboundex
Aboundexbot Aboundexbot
AfD-Verbotsverfahren Acunetix
AlphaBot AiHitBot
Anarchie Aipbot
Ankit Alexibot
Anthill Badass
Aspiegel
AwarioRssBot
AwarioSmartBot
BBBike
BackDoorBot
BackStreet
BackWeb
BacklinkCrawler
Barkrowler Barkrowler
BatchFTP
Battleztar Bazinga Battleztar Bazinga
BetaBot Boardreader
Bigfoot Bolt
Bitacle
Black Hole
BotALot
Brandprotect Brandprotect
Buck
Buddy
BuiltBotTough
BuiltWith
Bullseye Bullseye
BunnySlippers
CATExplorador CATExplorador
Calculon CCBot
Cliqzbot CODE87
Cocolyzebot CensysInspect
Cogentbot CheeseBot
Collector Claritybot
Cloud mapping
Copier Copier
CopyRightCheck Copyscape
Cosmos CrazyWebCrawler
CyotekWebCopy CrunchBot
Custo
DBLBot
DIIbot DIIbot
DSearch
DTS Agent DTS Agent
DataCha0s DatabaseDriverMysqli
Demon
Digincore
DigitalPebble
Disco Disco
Discobot
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DomainAppender
DomCopBot DomainCrawler
DomainStatsBot
Domains Project Domains Project
Dotbot
Download Wonder Download Wonder
Dragonfly Dragonfly
ECCP/1.0 ECCP/1.0
EMail Wolf EMail Wolf
Ebingbong EasyDL
Ecxi EirGrabber
EroCrawler EroCrawler
Express WebPictures ExtLinksBot
Extractor
EyeNetIE
Ezooms
FDM FDM
FHscan Fimap
FemtosearchBot
Firefox/7.0 Firefox/7.0
Flunky Foobot
Freeuploader Fuzz
FrontPage FyberSpider
Fyrebot
GalaxyBot GalaxyBot
Genieo
GetRight GetRight
GetWeb
Getintent Getintent
GoZilla
Gotit
GrapeFX GrapeFX
GridBot
HTTP::Lite HTTP::Lite
HTTrack HTTrack
Haansoft HonoluluBot
Heritrix HybridBot
IDBTE4M IDBTE4M
Id-search Iblog
IndeedBot Indy Library
Intelliseek
InterGET InterGET
InternetSeer Jbrofuzz
Iskanie JustView
IstellaBot Jyxobot
JennyBot
JetCar
Kinza Kinza
Larbin LWP::Simple
LeechGet Leap
LeechFTP
LexiBot LexiBot
Linkbot Libwhisker
LinkextractorPro Likse
LinksManager
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider
Litemage_walker
MFC_Tear_Sample
MIDown tool MIDown tool
MIIxpc MQQBrowser
MJ12bot MTRobot
MSIECrawler Mag-Net
Magnet
Majestic SEO
Majestic-SEO Majestic-SEO
Majestic12 Majestic12
MarkWatch MarkWatch
Mass Downloader Mass Downloader
Masscan
Mata Hari Mata Hari
Mb2345Browser Mb2345Browser
MicroMessenger MeanPath Bot
Microsoft Data Access Microsoft URL Control
Mister PiX
Moblie Safari
Mojolicious Mojolicious
Morfeus Fucking Scanner MolokaiBot
Mozlila
Mr.4x3
Musobot
NICErsPRO
Name Intelligence Name Intelligence
Nameprotect NearSite
Navroad
Net Vampire Net Vampire
NetZIP NetAnts
Nettrack NetMechanic
Netvibes Netcraft
Nibbler Niki-bot
Nikto
NimbleCrawler NimbleCrawler
Ninja Nmap
Nutch
Octopus
Offline Navigator Offline Navigator
OnCrawl
OrangeBot OrangeBot
OrangeSpider
PECL::HTTP PECL::HTTP
PHPCrawl PageGrabber
Page Analyzer
PageAnalyzer
PageScorer PageScorer
PageThing.com
Pandalytics Pandalytics
Panscient Panscient
PeoplePal Pavuk
Petalbot Pi-Monster
Picsearch Picsearch
PictureFinder PictureFinder
Pimonster ProPowerBot
Pockey Pump
PxBroker
RankActive RankActive
RankActiveLinkBot
RankFlex RankFlex
RankingBot RankingBot
ReGet Rankivabot
RedesScrapy RankurBot
Re-re
RepoMonkey RepoMonkey
Rogerbot
SBIder
SEOkicks-Robot
SEOprofiler SEOprofiler
SEOstats SMTBot
Screaming SalesIntelligent
ScreenerBot
Searchestate Searchestate
SearchmetricsBot SearchmetricsBot
Seekport Seekport
SemanticJuice SemanticJuice
SentiBot SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz
Siphon
SiteCheckerBotCrawler
SiteSnagger SiteSnagger
SiteSucker
Siteimprove
SlySearch SlySearch
SmartDownload Snapbot
Snake Sogou web spider
Snoopy Spammen
SocialRankIOBot Spbot
SpaceBison
SpankBot
Spinn3r Spinn3r
SputnikBot
Sqlworm
Sqworm Sqworm
Steeler
Sucker
SuperBot SuperBot
SuperHTTP SuperHTTP
Swiftbot Szukacz
T8Abot T0PHackTeam
Telesphoreo TeleportPro
The Intraformant
TheNomad
Titan Titan
Tracemyfile Trendiction
Turnitin True_Robot
Turingos
TurnitinBot TurnitinBot
URLy Warning TwengaBot
Twice
URLy.Warning
UnisterBot UnisterBot
Upflow VB Project
VCI
Vacuum Vacuum
VelenPublicWebCrawler Vagabondo
Virusdie VidibleScraper
WBSearchBot Voil
WPScan WASALive-Bot
WISENutbot
WWW::Mechanize WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0 WeSEE
Web Auto
Web Collage Web Collage
Web Fetch Web Fetch
Web Fuck Web Fuck
Web Pix Web Pix
Web Sauger Web Sauger
WebBandit WebCollage
WebCopier
WebEnhancer WebEnhancer
WebGo IS WebFuck
WebPix WebImageCollector
WebSauger WebStripper
WebSucker
WebZIP
Webalta Webalta
Website Quester WebmasterWorldForumBot
WebsiteExtractor
WebsiteQuester WebsiteQuester
Whatweb Webster
Wotbox Whack
Widow
WiseGuys Robot
Wprecon Wprecon
Zitebot Xaldon WebSpider
adscanner Xaldon_WebSpider
Xenu
YoudaoBot
Zade
ZmEu
ZoomBot
ZyBorg
arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider backlink-check
cah.io.community coccocbot
clark-crawler
com.plumanalytics com.plumanalytics
crawler.feedback crawler.feedback
demandbase-bot
domainsproject.org domainsproject.org
gopher gopher
ips-agent
iubenda-radar iubenda-radar
lwp-trivial mediawords
page scorer muhstik-scan
pcBrowser netEstate NE Crawler
plumanalytics
polaris version
probe-image-size
ripz ripz
scalaj-http scalaj-http
scan.lol scan.lol
seobility
seocompany.store seocompany.store
seoscanners
sitechecker.pro sitechecker.pro
siteripz siteripz
sp_auditbot
spyfu spyfu
sysscan
tAkeOut tAkeOut
trendiction.com
trendiction.de trendiction.de
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webprosbot webprosbot
x09Mozilla
xpymep1.exe xpymep1.exe
zauba.io zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
discountbarbour.online buy-meds24.com
frankofficial.ru candypeople.se
hiwibyh.bugs3.com gsasearchengineranker.site
hotkeys.com gz2.bbsoldes.fr
ilmexico.com omsk.xrus.org
kiwe-analytics.com parajumpersoutlet.online
more-letom.ru sexstream.pl
mylida.org shopsellcardsdumps.com
preparevideosafesystem4unow.space svarkagid.com
seminarykansas904.ml u555u.info

914
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00webcams.com 0n-line.tv
123kuma.com 1bet.com
28n2gl3wfyb0.ru 2000k.ru
3-letter-domains.net 3-letter-domains.net
300richmond.co.nz 4inn.ru
4replicawatch.net 7fon.ru
7minuteworkout.com 7zap.com
883zy.com aa8b68101d388c446389283820863176e7.com
abctoppictures.net aaronabel.com
absolutelyanalog.com absolutelyanalog.com
actulite.com addtoadd.men
adman.gr admitad.com
ads-seo.men adnotbad.com
adservme.com
adsfresh.men adsfresh.men
aibolita.com adultfullhd.com
akuhni.by adventureparkcostarica.com
akusherok.ru affordablewebsitesandmobileapps.com
alert-fjg.xyz alert-jdh.xyz
alert.scansafe.net alfapro.ru
algarveglobal.com alibestsale.com
allfinweb.com alpinism.ru
alphacarolinas.com ameblo.jp
amazon-adsystem.com amehdaily.com
anaseracresar.tk ampower.me
android-style.com anal-acrobats.hol.es
animenime.ru animalrank.com
anniemation.com apiqualitinknet-a.akamaihd.net
apccargo.com appartement-stumm.at
apiluckyleapnet-a.akamaihd.net apper.de
arendakvartir.kz appiq.mobi
arpelsreplica.xyz arcteryxstore.online
artavenuegardenstudios.com articlesdirectoryme.info
artel116.ru asdfz.pro
aruplighting.com aviav.eu
asmxsatadriverin.aircus.com aviav.org
asociatia-tipografilor-transilvania.ro avon-severozapad.ru
asophoto.com avon-ukraine.com
atmovs.com azadnegar.com
ausergrubhof.info
auto-zapchasti.org
autovideobroadcast.com
avtointeres.ru
ayerbo.xhost.ro
azartclub.org
azartmix.com
azazaporn.com azazaporn.com
azinoofficial777.ru azinoofficial777.ru
bala.getenjoyment.net azte.ch
bamo.xsl.pt baladur.ru
balkanfarma.org
balook.com
banki76.ru
banksrf.ru
bannerads.de
bayareaaeroclub.org bayareaaeroclub.org
beauty-bracelet.com bensbargains.net
beauty-things.com best-way.men
belinvestdom.by bestwebsitesawards.com
beremenyashka.com bet365.com
bestcalovebracelet.cn bigpenisguide.com
bestmouthwash.club bioscorp.ru
bezlimitko.xyz biznesluxe.ru
bibys.com biznesrealnost.ru
bigames.online blog.koreadaily.com
bin-brokers.com blogos.kz
bioca.org blpmovies.com
bjetjt.com bluejays-jerseys.us
blog.remote-computer.de bluesalt.co
bolegapakistan.com bongiornos.info
bosefux.esy.es bonux.nextview.ru
bouda.kvalitne.cz bookmaker-bet.com
brandedleadgeneration.com brainxs.us
briomotor.co
bristolhostel.com
brk-rti.ru brk-rti.ru
browsepulse-a.akamaihd.net browsepulse-a.akamaihd.net
bugshoot.cn burn-fat.ga
bumascloset.com buy-cheap-online.info
buntube.net buypanicdisorderpill.com
buyessay3.blogspot.ru buypillsorderonline.com
buypuppies.ca
buytizanidineonline.blogspot.com
buzz-porno.info buzz-porno.info
cakesplus.com.au bytimedance.ru
calvet.altervista.org c-gainsbourg.com
carabela.com.do callawaygolfstore.online
carapics.com calstaterealty.us
carina-sy.de cartierreplica.xyz
cashkitten-a.akamaihd.net cashkitten-a.akamaihd.net
casinopinup-wins.com
casinos4dummies.co.uk casinos4dummies.co.uk
cbcseward.com
cdnanalytics.xyz
cdncash.org cdncash.org
cete.ru cementaresearch.se
cheapjerseysa.com centrdebut.ru
cheapwebsitehoster.com charmstroy.info
chinacheapelitejerseys.com chatroulette.online
choosecuisine.com cheapmarmot.online
cialgenisrx.com christianlouboutinreplica.pw
cjmc.info chuckguilford.com
ci.ua
circlesl.com
clarithromycin500mg.com clarithromycin500mg.com
classicakuhni.ru clayvasedesigns.tk
co.lumb.co club-lukojl.ru
cocolyze.com club-samodelkin.ru
cobaltpro.ru
cocyq.inwtrade.com
com-cleaner.systems
concordexoticrentals.com concordexoticrentals.com
contentsexpress.com
continent-e.tv
cool-wedding.net
coolbar.pro
copblock.org
cornomase.win cornomase.win
coslab.club cosmetic.donna7753191.ru
costablue.xyz craftinsta.ru
crd.clan.su
creatives.adbetclickin.pink
crest-poloski.ru
cubook.supernew.org cubook.supernew.org
culad.com cubs-jerseys.us
custodieva.ru custom-product-labels.com
cutalltheshit.com customergrowthsystems.com
customerpromos-a.akamaihd.net
cutt.us cutt.us
cxpromote.com cyprusbuyproperties.com
dafatiri.com damedingel.ya.ru
darmebel.com.ua ddlmega.net
dawlenie.com dealighted.com
delta-line.men dealitright.click
deluxedumps.com derevesendeco.com
den-noch24.ru dgfitness.co
dentalpearls.com.au dieta-personalna.pl
designdevise.com djonwatch.ru
detmebel.su dmmspy.com
detskie-zabavi.ru downloaddy.net
dichvuvesinhhanoi.com downvids.net
digest-project.ru drillsaw.ru
diminishedvalueoforegon.com e-commerce-seo1.com
dinosaurus.site ebooknovel.club
documentsite.net econom.co
dogclothing.store editors.choice6912650.hulfingtonpost.com
dojki-hd.com educontest.net
dojki365.online
dtnlyss.com
e2click.com
egovaleo.it egovaleo.it
ekn-art.se elmifarhangi.com
electro-prom.com elvel.com.ua
elektronischezigarettekaufen1.beeplog.com erectile.bid
emergencyneeds.org erotag.com
engines-usa.com erotikstories.ru
es5.com essayservicewriting.org
essayassist.com este-line.com.ua
eugenevaultstorage.com everflora.ru
evaashop.ru evrotekhservis.ru
ewebarticle.info exhibitionplus.eu
excaliburfilms.com exmasters.com
extlabs.io expolicenciaslatam.co
extremez.net f-loaded.de
ezb.elvenmachine.com f0815.de
ezigarettekaufen1.hpage.com farmprofi.net
facebook-mobile.xyz fashionindeed.ml
fakehandbags.xyz fastcrawl.com
falopicm.pw fetroshok.ru
fanpagerobot.com fighrofacciufreesig.ga
faz99.com filmbokep69.com
fikasound.tk filmfanatic.com
financepoints.eu filmi-onlain.info
find1friend.com filmi-v.online
fiuxy.com findpik.com
flashbannernow.com footballfarrago.com
flytourisme.org
fordsonmajbor.cf fordsonmajbor.cf
formulaf1results.blogspot.com forexgb.ru
forum.poker4life.ru forminecrafters.ru
forum20.smailik.org forums.toucharcade.com
fotoxxxru.com fr.netlog.com
foxydeal.com frbizlist.com
frameimage.org free-social-buttons.com
free-floating-buttons.com freenode.info
freemags.cc freetangodownload.com
freenom.link friendflnder.com
freewhatsappload.com froggytube.com
frequiry.com fun-mobi.pl
front.to fungamelands.com
fsakhalin.ru fz139.ttk.ru
futbolkisales.ru galaxyflowers.ru
gallerily.com gamesprite.me
galleryawesome.com getdot.ru
gamblingnerd.com girlporn.ru
gk-atlant.info gk-atlant.info
goldadpremium.com glavprofit.ru
golfresa.lucania.se glicol.kz
glogow.pl
gnuetella.com
goo.ne.jp goo.ne.jp
goosefishpost.bid goosefishpost.bid
graphicwe.org gopixdatabase.com
gyffu.com gratis-sexkontakte.com
hacktougroup.ru handicapvansarea.com
haveinc.xyz headpress.ru
hazardky.net healing-dysplasia.ru
hchha.com hentaiheroes.com
healmytrauma.info hermesbelts.xyz
heatpower.ru hermesbracelets.xyz
hikesearch.net hexpilot.com
homeinns.com hidefiles.org
hosting-tracker.com hitcpm.com
hotel-mkad.ru hobbyhuren-datenbank.com
hotsocialz.com hobild.net
ideibiznesa2015.ru homemade.gq
hscsscotland.com
humbmosquina.tk
ifmo.ru ifmo.ru
ilmen.net ilikevitaly.com
ilovevitaly.ru imabase.com
ilovevitaly.xyz imadedinner.net
imagine-ex.co images.gyffu.com
inboundlinks.win imagui.eu
infoupdate.org indiasourcemart.in
inmate-locator.us inmate-locator.us
insomniagamingfestival.com intelhdgraphicsgtdrive6w.metroblog.com
investmac.com interesnie-faktu.ru
invitefashion.com interferencer.ru
irunfar.com interfucks.net
istock-mebel.ru investingclub.ru
itservicesthatworkforyou.com investyb.com
jewelryandfiligree.com ipostroika.ru
jofucipiku.tk izhevsk.xrus.org
izismile.com
izoll.ru
james13prix.info
javatex.co.id
jerseychinabizwholesale.com
johnrobertsoninc.com
journalhome.com journalhome.com
jyrxd.com kadashihotel.com
kabbalah-red-bracelets.com
kalb.ru kalb.ru
kamagragelusa.net katadhin.co
kamorel.com keywordhouse.com
ketrzyn.pl kit-opt.ru
kino-key.info klin3952.ru
klosetkitten.com konkursowo-24.pl
knigonosha.net konoplisemena.com
kol-energo.ru kosmetyki.tm.pl
koral.se kountrylife.com
krasnodar.xrus.org kredit-blog.ru
la-fa.ru kuzinsp.ru
labelwater.se kwzf.net
labplus.ru lacapilla.info
lanasshop.ru
lancheck.net
laptop-4-less.com laptop-4-less.com
larchik.net
larger.io larger.io
law-check-nine.xyz lazymanyoga.com
law-check-seven.xyz lerporn.info
life-instyle.com letslowbefast.today
lightinghomes.net letto.by
linksharingt.com lflash.ru
lida-ru.com
lietaer.com
linkbolic.com
linkpulse.com
linkredirect.biz
littlesunraiser.com littlesunraiser.com
liupis.com lockerz.com
livejasmin.com locksmith.jp
liverpool.gsofootball.com lostaruban.com
locimge.pw lovelycraftyhome.com
luckyday.world macdamaged.tech
luckyshop.net.ua machicon-ueno.info
mackeeper-center.club madot.onlinewebshop.net
magicplayer-s.acestream.net make-money-online.com
mansiondelrio.co makenahartlin.com
maridan.com.ua mantramusic.ru
masturbate.co.uk manyresultshub-a.akamaihd.net
max-p.men marcowebonyodziezowe.pl
mazda-roadsters.com masserect.com
mcadamssupplyco.com mb140.ru
mcar.in.ua mebelcomplekt.ru
mediawhirl.net mediaoffers.click
medictube.ru meds-online24.com
medmajor.ru meet-flirt-dating.com
mega-bony2017.pl mensandals.xyz
metaxalonevstizanidine.blogspot.com mesto-x.com
miniature.io metafilter.com
minyetki.ru mexicosleevegastrectomy.com
mirzonru.net mexicotravelnet.com
mmofreegames.online monclerjacketsoutlet.win
mobifunapp.weebly.com moncleronline.xyz
mobioffertrck.com morpicert.pw
mole.pluto.ro mountainstream.ms
moroccosurfadventures.com msk-diplomat.com
mostantikor.ru musezone.su
moviezbonkerssk.cf
mrmoneymustache.com
mrwhite.biz
multgo.ru
musirc.com
muz-baza.net muz-baza.net
mydoctorok.ru muztops.ru
myfreetutorials.com myhydros.org
myperiod.club nelc.edu.eg
nac-bearings.ru net-profits.xyz
naperehresti.info netanalytics.xyz
nardulan.com netvouz.com
nastroyke.net newfilmsonline.ru
new-apps.ru news-readers.ru
newstaffadsshop.club newstraveller.ru
nightvision746.weebly.com nfljerseyscheapchinabiz.com
nlfjjunb5.ru nibbler.silktide.com
nvformula.ru nikhilbahl.com
nzfilecloud.weebly.com no-fuel.org
obesidadealgarve.com nonews.co
oddamzadarmo.eu noumeda.com
onefilms.net ntic.fr
onlinadverts.com obiavo.uz
online-podarki.com ochistka-stokov.ru
openmultipleurl.com oecnhs.info
outshop.ru ok-ua.info
palvira.com.ua oksrv.com
pandarastore.top olgacvetmet.com
partnerads.men online.ktc45.ru
onlineslotmaschine.com
onlinetvseries.me
onlyforemont.ru
onthemarch.co
open-odyssey.org
orion-code-access.net
ossmalta.com
otbelivanie-zubov.com
paceform.com
pagesense.com
par-fallen.ga
partner-pop.men
patol01.pw patol01.pw
perfectplanned.com perederni.net
perosan.com perfectpracticeweb.com
peugeot-club.org photo.houseofgaga.ru
photokitchendesign.com photokitchendesign.com
piccshare.com picswe.com
picsforkeywordsuggestion.com pierrehardysale.online
pillscheap24h.com pila.pl
pintattoos.com
pistonclasico.com
pix24x7.com pix24x7.com
pizza-tycoon.com plastjulgranar.se
pk-services.ru pod-muzyku.club
plohaya-kreditnaya-istoriya.ru poisk-zakona.ru
podshipniki-nsk.ru pomoc-drogowa.cba.pl
pokemongooo.ml popunder.ru
pooleroadmedicalcentre.co.uk porno-home365.com
popunder.net pornokorol.com
porno-dojki.net
porno-video-chati.ru
pornodojd.ru
pornoklad.ru
pornoreino.com
pornosmola.info
pornosok.ru pornosok.ru
pornoted.com posible.net
pornstartits.xblog.in pospr.waw.pl
pr0fit-b0x.com praisong.net
privacylocationforloc.com privatefx.all4invest.info
privat-girl.net proftests.net
prizestohandle.club progonrumarket.ru
projefrio.com.br psa48.ru
ptr.ruvds.com pssucai.info
purplesphere.in pulse33.ru
queerspace.com q-moto.ru
quickbuck.com qexyfu.bugs3.com
radiogambling.com quebec-bin.com
rankexperience.com quickchange.cc
redbottomheels.xyz r-control.ru
refudiatethissarah.info rangoman.date
remont-ustanovka-tehniki.ru rank-checker.online
remontvsamare.su rapidgator-porn.ga
rentehno.ru reklama-i-rabota.ru
retailwith.com remontgruzovik.ru
remote-dba.de
ring4rhino.com ring4rhino.com
risparmiocasa.bz.it rique.host.sk
robot-forex.biz rocis.site
rockprogblog.com ru-dety.ru
rosbalt.com.ua rukino.org
ruclicks.com s-iwantyou.com
rus-pornuha.com sandhillsonline.com
ruscoininvest.company sawin.beth.webd.pl
rusoft-zone.ru scanner-jane.top
rutor.group scanner-margo.top
s.lollypopgaming.com scenarii-1-sentyabrya.uroki.org.ua
sammlungfotos.online schlampen-treffen.com
samolet.fr sdi-pme.com
sape.top se.bnt-team.com
sarf3omlat.com searchaddis.com
satellite.maps.ilovevitaly.com securesmrt-dt.com
sbf441.com selectads.men
scanner-jack.top seotoolsagency.com
scat.porn servethis.com
score-ads.men service-core.ru
searchinterneat-a.akamaihd.net seryeznie-znakomstva.ru
seemoreresultshu-a.akamaihd.net sethrollins.net
sell-fb-group-here.com sex-dating.co
semaltmedia.com sex-watch.com
seo-prof1.xyz sexxdate.net
seoboxes.com shitmovs.com
seoholding.com shoesonlinebuy.xyz
seoprofiler.com shopcheermakeup.info
seorank.info sibvitr.ru
serptehnika.ru signx.info
sexpartygirls.net simply.net
sextracker.be sitebeam.net
sexystrippe.info slavkokacunko.de
sexyteens.hol.es snyatie-lomki-v-stacionare.ru
sfd-chess.ru socialbuttons.xyz
shemalegalls.blogporn.in socialtrade.biz
shmetall.com.ua sofit-dmd.ru
sigmund-freud.co.uk softwaretrend.net
silverage.ru spabali.org
simplepooltips.com spidtest.org
sinestesia.host.sk srgwebmail.nl
site-analyzer.com sribno.net
siteheart.net standardchartered-forex.com
social-search.me
socialbutton.xyz
sockshares.tv
softonicads.com
solmarket.by
songoo.wz.cz
sonnikforme.ru
spasswelt.xyz
spbchampionat.ru
speeddream.xyz
sprttrack.com
staff.prairiesouth.ca
stairliftstrue.com
stariy-baku.com
stillmiracle.com
stockquotes.wooeb.com stockquotes.wooeb.com
stoki.ru
strag-invest.ru
streha-metalko.si
stroimajor.ru stroimajor.ru
styro.ru stroymonolit.su
supermodni.com.ua sunflowerdrawingpaintings.blogspot.com
survival.betteroffers.review susanholtphotography.com
swinginwithme.ru suture.co
tabletkinaodchudzanie.com.pl svnuppsalaorebro.se
tastyfoodideas.com swsociety.se
tehngr.ru symbaloo.com
telefonsex-ohne0900.net tagil.zrus.org
teletype.in tam-gde-more.ru
tengohydar.tk tampabaywatch.org
terrafootwear.us tasteidea.com
teslathemes.com telemetryverification.net
tgtclick.com testbotprocessor44.com
the-usa-games.blogspot.com thaismartloan.com
thedownloadfromwarez.blogspot.com theallgirlarcade.com
thegameriders.com
thegioixekhach.com thegioixekhach.com
theplacetoupdating.pw theporndude.com
thewomenlife.com thesmartsearch.net
thin.me.pn
tiandeural.ru tiandeural.ru
timhost.ru titangel-vietnam.com
tizanidineandcipro.blogspot.com tizanidine4mgstreetvalue.blogspot.com
tizanidinedosageforsleep.blogspot.com tizanidineandhydrocodone.blogspot.com
tizanidinehcl4mgisitanarcotic.blogspot.com tizanidineingredients.blogspot.com
tizanidineinteractions.blogspot.com
tizanidinemusclerelaxant.blogspot.com tizanidinemusclerelaxant.blogspot.com
tizanidineshowupondrugtest.blogspot.com tizanidinesleepaid.blogspot.com
tizanidinesideeffects.blogspot.com tizanidinevssoma.blogspot.com
tizanidinewithdrawalsymptoms.blogspot.com tmtrck.com
tootoo.to token-lab.org
topbestgames.com torture.ml
topkarkas.com
topshef.ru
tourcroatia.co.uk
toy-shop.top toy-shop.top
tracklead.net tracfone.com
trafficfactory.biz tubeline.biz
trion.od.ua tuckermktg.com
trkur.com turist-strani.ru
truck-addzilla.life twojebook.pl
trustedmaccleaner.com twu.com.ua
tuberkuleznik.ru u17795.netangels.ru
turbodsp.com uchil.net
tvnewsclips.info ukrtextbook.com
txxx.com
tytoona.com
tyumen.xrus.org
ukrtvir.com.ua
ultimateclassicrock.com ultimateclassicrock.com
underthesite.com unece.org
uptime-alpha.net uprour.com
usadacha.net url-extractor.xyz
vacances-voyages.info url2image.com
viagra.pp.ua urlopener.blogspot.com.au
viddyoze.com uvozdeckych.info
video-camer.com v720hd.ru
vinsit.ru vashsvet.com
vip2ch.com via-gra.webstarts.com
virus-schutzmasken.de viagraneggrx.com
visa-pasport.ru video-chat.love
vonradio.com video-hollywood.ru
vvpg.ru videochat.life
vykup-avto-krasnodar.ru videochat.world
web-analytics.date videokrik.net
web-betting.ru vitanail.ru
webcamtalk.net vkak.ru
website-speed-up.top vltai.com
websites-reviews.com vmnmvzsmn.over-blog.com
while.cheapwebsitehoster.com voloslove.ru
wholesalejerseychinashop.com walkme.com
wholesalejerseys-cheapest.com wallpapersdesk.info
wholesalejerseysgaa.com wallpapersist.com
wholinkstome.com wardreapptokone.tk
wikes.20fr.com warning.or.kr
wildcattube.com website-stealer.nufaq.com
websiteexplorer.info
welck.octopis.com
whatsupinfoley.com
winterclassichockeyjerseys.com winterclassichockeyjerseys.com
worldlovers.ru winwotgold.pl
wwwadultcheck.com womensterritory.ru
wzgyyq.com works.if.ua
x-mix.info world-mmo.com
x-musics.com wstroika.ru
xfluro.com wurr.voila.net
xmladserver.com xev.ru
xn----7sbaphztdjeboffeiof6c.xn--p1ai xn--80ahdheogk5l.xn--p1ai
xn--80aanaardaperhcem4a6i.com xn--d1abj0abs9d.in.ua
xn--b1agm2d.net xxxhdvideo.site
xn--l1aengat.xn--p1ai xxxtube69.com
xportvusbdriver8i.snack.ws xzlive.com
xxxkaz.org y8games-free.com
yachts-cruise.info yeartwit.com
yatrk.xyz yellowstonevisitortours.com
yourporngay.com youporn-ru.com
yuweng.info yubikk.info
zakazvzloma.com zaapplesales.blogspot.com
zarajbuilders.com zajm-zalog-krasnodar.ru
zastenchivosti.net zambini.ru
zawyna.ua zarenica.net
zeroredirect5.com zeroredirect1.com
zeroredirect7.com zigarettenonlinekaufen1.blox.pl
zigarettenonlinekaufen2.bloog.pl zrizvtrnpale.tk
zionstar.net zvooq.eu
znaturaloriginal.com
zoogdiesney.com
zootravel.com
zverokruh-shop.cz

472
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1kdailyprofit.me 00author.com
20pascals.nl 0lovespells0.blogspot.com
2itech.ru 100searchengines.com
4kplayer.pl 1314dh.com
abcdefh.xyz 202ch.com
acmebtn.ml 404.mn
adconscious.com 4senses.co
adsland.men 5u.com
aktobe.xkaz.org 7yue.org
albumsuper.info 7zap.com
all-streaming-media.com 8558l.jobs.net
alldezservice.kz abctoppictures.net
alta-realestate.com adtech.de
amt-k.ru adultactioncam.com
amung.us alert-jdh.xyz
animenime.ru allcredits.su
aosexkontakte.net amoi.tn
apiluckyleapnet-a.akamaihd.net anal-acrobats.hol.es
appsaurus.com apartment.ru
arendatora.ru arendas.net
artavenuegardenstudios.com arewater.com
arvut.org arquivo.pt
asdfg.pro asrvvv-a.akamaihd.net
auto-zapchasti.org atdedinotuho.tk
autobrennero.it atleticpharm.org
automobile-spec.com atout-energie-69.com
awaybird.ru
azartclub.org azartclub.org
azbaseimages.net babieca.com
baladur.ru balois.worldbreak.com
balans.shahterworld.org bestmarriages.com
beta.hotkeys.com bestofferswalkmydogouteveryday.gq
bjetjt.com betshuckclean.com
bristolhotel.com bimatoprost-careprost.com
budpost.com.ua bmsco.co
buybest1.biz bombla.org
buyessayonline19.blogspot.ru budilneg.xyz
buyhoverboard.com buketeg.xyz
cah.io.community buyfriend.ru
cannibalcheerleader.com buypuppies.ca
celejihad.info californianews.cf
centrumcoachingu.com cardsdumps.com
ciceros.co chatroulette.world
chiblackhawks-jerseys.com
chocolateslim-u-srbiji.com
choice-credit.ru
ciekawinki.pl ciekawinki.pl
cityadspix.com
cleanallspyware.com cleanallspyware.com
collegeessay19.blogspot.ru clicksor.net
compliance-alex.xyz colehaanoutlet.store
compliance-ivan.xyz collegerentals.com
compliance-olga.top concordexoticrentals.com
comsysnet.com constantaservice.net
cookingmeat.ru cookielawblog.wordpress.com
coolingoods.com creditnation.ru
cosmeticswomens-womensports.rhcloud.com
costablue.xyz
cottageofgrace.com
craftburg.ru
cscwtalkto.site
customerpromos-a.akamaihd.net customerpromos-a.akamaihd.net
d-black.bz damedingel.ya.ru
danschawbel.com de.zapmeta.com
dayibiao.com defenderxtactical.com
deanmoore.ie den-noch24.ru
degerlund.net downloader12.ru
destinationrealestate.com draniki.org
directrev.com elektrischezigarette2.devhub.com
dms-sw.ru elmifarhangi.com
doctormakarova.ru empis.magix.net
eandsgallery.com energy-ua.com
elitesportsadvisor.com erectile.bid
embedle.com
emediate.eu
eropornosex.ru
evening-dating-club.info evening-dating-club.info
evogarage.com exhibitionplus.eu
f012.de exmasters.com
findacheaplawyers.com exonline.info
findclan.org extads.net
fivedwld.cf ezigarettenkaufen2.dreamwidth.org
foxweber.com fast-wordpress-start.com
freenode.info ferrotodo.com
froggytube.com fidelityfunding.com
gabeshop.ru figuringmoneyout.com
gael-s.ru filmci.pro
gallery.rennlist.com filmetricsasia.com
gameonasia.com fishingwholesale.us
gelezki.com fix-website-errors.com
girlgamerdaily.com flipper.top
fortevidyoze.net
fotopop.club
foxinsocks.ru
freefoto.ca
fungirlsgames.net
galaxy-family.ru
gamesprite.me
generousdeal-a.akamaihd.net
get-seo-domain.com
gilbertbanda.net
girlsatgames.ru
glasof.es
glass-msk.ru glass-msk.ru
gratis-sexkontakte.com googglet.com
grupografico-pilar.com.ar googst2.ru
gsasearchengineranker.pw haikuware.com
guardlink.org hanink.biz.ly
guitar-master.org hasshe.com
gwagka.com heroz.fr
hamptonoaks.ca hit-kino.ru
hdxnxxtube.mobi hkdiiohi.skyrock.com
hentaiheroes.com hollywoodactress.info
hitmuzik.ru ideibiznesa2015.ru
home.myplaycity.com isitwp.com
hoverboard360.at islamtoday.co.za
i-midias.net.br iwantedmoney.com
ikritikimou.gr jurajskie.info
intermesh.net justdating.online
istizanidineanopiate.blogspot.com karusel-market.ru
itbc.kiev.ua kino2018.club
itch.io kinohall.ru
iusstf.org klejonka.info
izhevsk.zrus.org kongruan.com
johannesburgsingles.co.za kostenlos-sexvideos.com
jolic2.com kostenloser-sex.com
kiprinform.com kosynka-games.ru
kmd-pto.ru
krasivoe-hd.com
krynica.info krynica.info
le-clos-des-alouettes.com lankarns.com
lexiangwan.com lion.bolegapakistan.com
lidoradio.com liran-locks.com
lightinghomes.net livefixer.com
linkarena.com liver-chintai.org
linkbuddies.com lonely-mature.com
live-cam6.info louboutinbooties.xyz
loadingpages.me louisvuittonoutletstore.net
lostfilm-online.ru mabdoola.blogspot.com
maju.bluesalt.co mall.uk
manipulyator-peterburg.ru master-muznachas.ru
manuscript.su maxthon.com
mastershef.club medicovi.com
matpre.top medispainstitute.com.au
mcadamssupplyco.com modabutik.ru
mere.host.sk moneymaster.ru
michaelkorssaleoutletonline.net
migente.com
mini-modus.ru
mistr-x.org
mobilemedia.md
moncleroutletonline.win
montazhnic.ru montazhnic.ru
muznachas-service.ru morefastermac.trade
nakozhe.com most.gov.iq
naperehresti.info muizre.ru
narosty.com mustwineblog.com
national-today-winning-winner.club myhitmp3.club
neobux-bg.info mymercy.info
nfljerseysforsalewholesaler.com myseoconsultant.com
nofreezingmac.click navalwiki.info
notasprensa.info nlfjjunb5.ru
oktube.ru nomuos.it
oneclickfiles.com northfacestore.online
online-sbank.ru npoet.ru
ooo-olni.ru oakridgemo.com
p-business.ru offer.wpsecurity.website
pacificair.com offf.info
pagesense.com okeinfo.online
palvira.com.ua optibuymac.com
pandroid.co orsonet.ru
parajumpersstore.online paclitor.com
partnerline.men panouri-solare-acoperis.com
perosan.com paparazzistudios.com.au
pewit.pw partner-stop.men
pharmacyincity.com pc-virus-d0l92j2.pw
pobeiranie.pl picsfair.com
podshipniki-nsk.ru pigrafix.at
poshiv-chehol.ru pix24x7.com
pozdrawleniya.com pizdopletka.club
preg.marketingvici.com play-movie.pl
proekt-mos.ru poppen-nw.net
profitfx.online porndairy.in
prohoster.info pornhub-forum.ga
prokotov.com praisong.net
puzzleweb.ru pretty-mart.com
py100.ru proprostatit.com
qualitymarketzone.com prostitutki-odessa.org
pshare.biz
qor360.com
rainbowice.ru rainbowice.ru
randalljhoward.com redirectme.net
rasteniya-vs-zombi.ru remmling.de
remont-mobile-phones.ru remybutler.fr
rent2spb.ru robotixix.com
rethinkwasteni.info russia-today-video.ru
sabaapress.com rvi.biz
sabizonline.com sa-live.com
scanner-margo.top samo-soznanie.ru
scanner-nelson.top sanidumps.com
scanner-walter.top sankt-peterburg.nodup.ru
scanspyware.net sbf441.com
search.xtconnect.com scanner-walt.top
searchlock.com searchimpression.com
seo-platform.com seksvideoonlain.com
seocheki.net seocheki.net
serialsway.ucoz.ru seofied.com
serving.adbetclickin.pink seomarketings.online
sexreliz.com service.adtech.us
sibtest.ru sex-znakomstva.online
sideeffectsoftizanidine.blogspot.com shopperifymac.com
site-speed-checker.site sientalyric.co
siteexpress.co.il simply.net
slowmac.tech sisiynas.ru
snjatie-geroinovoy-lomki.ru sixcooler.de
sodexo.com smstraf.ru
songplanet.ru soft1.ru
spbchampionat.ru spb-plitka.ru
srdrvp.com speedup-my.site
srecorder.com stjamesschool.info
starpages.net stpolice.com
staynplay.net superstats.com
steebook.com surffoundation.nl
sticken.co teenbbw.yopoint.in
streetfooduncovered.com templates.radiodigital.co
sureone.pro the-torrent-tracker.blogspot.com
taboola.com thegolfclub.info
tako3.com
teesdaleflyballclub.co.uk
telefonsex-ohne0900.net
terrafootwear.us
thfox.com thfox.com
time-japan.ru tizanidineduringpregnancy.blogspot.com
tizanidineforopiatewithdrawal.blogspot.com torontoplumbinggroup.com
tonerbox.kz trahic.ru
trustedhealthtips.com traxdom.ru
tula.mdverey.ru tripper.de
twelvevisionspartyofcolorado.com ttsq.fr
u-cheats.ru turvgori.ru
umaseh.com tx41tclega.ru
unionmarkt.de uchil.net
uptimechecker.com ultimatesetnewfreeallsoftupgradesystems.pw
valoresito.com uytmaster.ru
venta-prom.ru valuado.com
viagroid.ru viagroid.ru
videochat.cafe videochaty.ru
videochat.mx vietimgy.pw
vitanail.ru vigrx-original.ru
warezaccess.com vilingstore.net
websites-reviews.com virtualbb.com
whatsappdownload10.com vitalads.net
vse-pesni.com
waterefficiency.co
webix.biz
webstatsdomain.org
wesharepics.com
wfb.hatedriveapart.com
while.cheapwebsitehoster.com while.cheapwebsitehoster.com
woman-h.ru womama.ru
www.get-free-traffic-now.com wsgames.ru
xbaboon.com www.labves.ru
xchat26.myfreecams.com xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai xn--1-8sbcpb0bdm8k6a.xn--p1ai
xn----7sbifcamovvfggw9d.xn--p1ai xportvusbdriver8i.snack.ws
xn--80aaagvmjabrs1aoc9luc.xn--p1ai xz618.com
yaminecraft.ru yogamatsexpert.com
yhit.press
yourporngay.com
yoursite.com yoursite.com
youtuhe.com youtubologia.it
zastroyka.org zerocash.msk.ru
zeg-distribution.com znakomstvaonlain.ru
zhorapankratov7.blogspot.com zoominfo.com
zx6.ru

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.184 0; 10.1.1.68 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2937 ### Version: V4.2022.02.2938
### Updated: Tue Feb 22 10:03:37 UTC 2022 ### Updated: Tue Feb 22 22:01:25 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2937 ### Version: V4.2022.02.2938
### Updated: Tue Feb 22 10:03:37 UTC 2022 ### Updated: Tue Feb 22 22:01:25 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##