vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-07 13:10:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.06.3191 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-06-26 10:09:28 +00:00
parent 486416ff11
commit 8659c7349d
47 changed files with 110623 additions and 110613 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.06.3190 #### Version: V4.2022.06.3191
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.06.3190 #### Version: V4.2022.06.3191
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.06.3190 #### Version: V4.2022.06.3191
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13692
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.06.3190 #### Version: V4.2022.06.3191
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13696
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13696
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

13696
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13696
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

13696
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

13696
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

13696
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3190 3191

13696
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

13696
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.1.172.87 0; 20.66.70.106 0;

13696
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

13696
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

13696
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

13696
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.1.172.87 0; 20.66.70.106 0;

13696
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.1.172.87 0; 20.66.70.106 0;

13696
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BLEXBot Aboundex
CherryPicker Brandwatch
Craftbot ChinaClaw
Dirbuster DigitalPebble
Express WebPictures Ebingbong
Jetty Linkbot
LexiBot Sosospider
Pi-Monster Toata
SurveyBot Website Quester
internetVista monitor meanpathbot

292
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
ADmantX 01h4x.com
360Spider
80legs
ALittle Client ALittle Client
ASPSeek Abonti
Aboundexbot Acunetix
AhrefsBot AiHitBot
Alexibot Alexibot
Anthill AlphaBot
Aspiegel Anarchy99
Atomseobot
AwarioSmartBot AwarioSmartBot
BBBike BBBike
BLEXBot
Backlink-Ceck Backlink-Ceck
Barkrowler BacklinkCrawler
BatchFTP Battleztar Bazinga
BetaBot
Bitacle
Black Hole Black Hole
Blackboard Blackboard
Blow
BlowFish BlowFish
BotALot Bolt
Brandprotect
Brandwatch Brandwatch
BuiltBotTough BuiltBotTough
BuiltWith
Bullseye Bullseye
BunnySlippers BunnySlippers
CATExplorador BuzzSumo
CCBot
CSHttp CSHttp
CazoodleBot
Cegbfeieh Cegbfeieh
CheeseBot CherryPicker
ChinaClaw Cloud mapping
Citoid Cocolyzebot
Claritybot
Cliqzbot
Cogentbot
Collector Collector
Craftbot Copier
Crawling at Home Project Cosmos
CrunchBot DataCha0s
DatabaseDriverMysqli Deusu
Devil
Digincore Digincore
DigitalPebble DigitalPebble
Disco Dirbuster
Discobot DnBCrawler-Analytics
Discoverybot
Dispatch
DnyzBot DnyzBot
DomainAppender DomainStatsBot
Domains Project
Dotbot Dotbot
EasyDL Drip
EirGrabber EirGrabber
Express WebPictures Exabot
ExtLinksBot ExtLinksBot
Foobot ExtractorPro
Freeuploader Extreme Picture Finder
GT::WWW EyeNetIE
GermCrawler Ezooms
GoZilla FDM
FemtosearchBot
Fimap
FyberSpider
Getintent
GrabNet GrabNet
Grafula GrapeshotCrawler
GridBot HMView
HEADMasterSEO
HTMLparser HTMLparser
HTTrack HTTP::Lite
Haansoft Harvest
IRLbot
Id-search
IlseBot IlseBot
Image Fetch Image Fetch
Image Sucker
IndeedBot IndeedBot
Indy Library Indy Library
InfoTekies InfoTekies
Intelliseek
InterGET InterGET
InternetSeer Internet Ninja
Iskanie Iria
IstellaBot
JamesBOT JamesBOT
JennyBot JennyBot
JetCar Joomla
JikeSpider JustView
Jorgee
Jyxobot Jyxobot
Kenjin Spider Kenjin Spider
Keybot Translation-Search-Machine Kinza
Keyword Density LNSpiderguy
Kozmosbot Leap
LWP::Simple
LeechFTP
LeechGet
Lftp Lftp
Lightspeedsystems Libwhisker
LinkWalker LieBaoFast
Linkbot Linkbot
LinkpadBot LinkpadBot
LinksManager LinqiaScrapeBot
Litemage_walker Litemage_walker
Lmspider Ltx71
MIIxpc
MJ12bot MJ12bot
MQQBrowser Magnet
MSFrontPage
MSIECrawler
MTRobot
Majestic SEO Majestic SEO
Majestic-SEO Majestic-SEO
MarkMonitor
Masscan
Mata Hari Mata Hari
MeanPath Bot Mb2345Browser
MegaIndex.ru
Metauri Metauri
MicroMessenger MicroMessenger
Microsoft Data Access
Minefield
Mojeek Mojeek
Mojolicious Mojolicious
MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3 Mr.4x3
Msrabot Msrabot
Musobot Musobot
NICErsPRO NPbot
Nameprotect Nameprotect
Navroad NearSite
Needle NetLyzer
NetMechanic NetMechanic
NextGenSearchBot NetSpider
Netvibes
Niki-bot Niki-bot
Nikto NimbleCrawler
Nmap
Not
Nuclei
Nutch Nutch
Octopus Offline Explorer
Offline Navigator Offline Navigator
OnCrawl
OpenVAS OpenVAS
Openvas
OutfoxBot OutfoxBot
POE-Component-Client-HTTP PECL::HTTP
PHPCrawl
PageAnalyzer
PageGrabber PageGrabber
Pandalytics Petalbot
Pavuk Pi-Monster
PeoplePal Picsearch
Picscout Pimonster
PictureFinder PleaseCrawl
Pockey Pockey
ProWebWalker ProPowerBot
Probethenet
Pump Pump
PxBroker
PyCurl PyCurl
QueryN Metasearch
RankFlex
RankurBot RankurBot
Re-re
ReGet
Reaper Reaper
RebelMouse Recorder
RedesScrapy Ripper
RocketCrawler SEOstats
SBIder
SEOkicks-Robot
SEOlyticsCrawler
SISTRIX
SMTBot
ScoutJet ScoutJet
Scrapy Scrapy
Screaming SearchmetricsBot
ScreenerBot SemanticJuice
Searchestate SemrushBot
Semrush SeoSiteCheckup
SentiBot SeobilityBot
Siphon Seomoz
SiteExplorer Shodan
SiteSnagger SiteCheckerBotCrawler
SiteSucker
Sitebeam Sitebeam
Siteimprove SlySearch
Sitevigil SmartDownload
Snake
Snapbot
Snoopy
Sociscraper Sociscraper
Sogou web spider Sogou web spider
Spanner SpankBot
Sqworm Sqlworm
Steeler
Sucuri Sucuri
Suzuran SuperBot
T8Abot Szukacz
Teleport Teleport
Telesoft Telesphorep
Telesphoreo Thumbor
The Intraformant
TheNomad
Titan
Toata Toata
Toweyabot
Tracemyfile Tracemyfile
Trendiction Trendiction
Turingos
Turnitin Turnitin
TurnitinBot
TwengaBot TwengaBot
Twice Twice
Typhoeus
URLy Warning URLy Warning
URLy.Warning V-BOT
UnisterBot VB Project
VCI VidibleScraper
Virusdie
VoidEYE VoidEYE
Voil Voil
WEBDAV WEBDAV
WPScan WPScan
WWWOFFLE WWWOFFLE
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
Web Collage Web Collage
Web Sucker Web Enhancer
Web Pix
Web Sauger
WebAuto WebAuto
WebFetch WebCopier
WebGo IS
WebReaper
WebStripper WebStripper
WebSucker WebSucker
Webshag WebmasterWorldForumBot
Website Quester
WebsiteQuester
Webster Webster
WinHTTrack Whacker
Whatweb
Who.is Bot
Wonderbot
Wotbox Wotbox
YoudaoBot Wprecon
Zade Zermelo
Zitebot Zitebot
ZmEu ZoominfoBot
ZyBorg
archive.org_bot
arquivo-web-crawler
arquivo.pt arquivo.pt
check1.exe autoemailspider
clark-crawler cah.io.community
coccocbot coccocbot
com.plumanalytics
crawler.feedback crawler.feedback
demandbase-bot demandbase-bot
facebookscraper eCatch
evc-batch
gopher
heritrix heritrix
instabid linkdexbot
lwp-request lwp-request
lwp-trivial plumanalytics
magpie-crawler seocompany.store
meanpathbot
netEstate NE Crawler
page scorer
ripz
scalaj-http
scan.lol
seoscanners seoscanners
serpstatbot seostar
sexsearcher sexsearcher
sitechecker.pro
siteripz siteripz
spyfu sogouspider
sp_auditbot
sysscan
tAkeOut tAkeOut
trendiction.com
trendiction.de trendiction.de
voyagerx.com
webgains-bot webgains-bot
webmeup-crawler
webpros.com webpros.com
webprosbot webprosbot
x09Mozilla
x22Mozilla

300
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
AIBOT ALittle Client
Abonti Aboundexbot
AfD-Verbotsverfahren AhrefsBot
AiHitBot Aipbot
Alexibot
AllSubmitter AllSubmitter
Alligator Alligator
Asterias AlphaBot
Attach Anarchie
BBBike Apexoo
BDFetch Aspiegel
BLEXBot Atomseobot
BackDoorBot AwarioRssBot
BackStreet BackStreet
BacklinkCrawler Backlink-Ceck
Barkrowler
Battleztar Bazinga
Bigfoot Bigfoot
Bitacle Bitacle
Blackboard Black Hole
Blow BlowFish
Boardreader Boardreader
Bolt
BotALot
Brandprotect
Buck Buck
Buddy
BuiltBotTough BuiltBotTough
BuiltWith BunnySlippers
Bullseye
CCBot
CSHttp CSHttp
CensysInspect
CherryPicker
ChinaClaw ChinaClaw
Chlooe Cloud mapping
Cocolyzebot Cocolyzebot
Collector CopyRightCheck
Craftbot Craftbot
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
CrunchBot CyotekWebCopy
DSearch
DTS Agent DTS Agent
DataCha0s
DatabaseDriverMysqli
Demon
Deusu Deusu
Devil Digincore
Dispatch Discoverybot
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
ECCP/1.0 Dotbot
Dragonfly
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
Ecxi EasyDL
Extreme Picture Finder EirGrabber
FDM Express WebPictures
FlashGet Extractor
Ezooms
FHscan
FemtosearchBot
Fimap
Freeuploader
FrontPage FrontPage
Fuzz
FyberSpider FyberSpider
Fyrebot Fyrebot
GT::WWW G-i-g-a-b-o-t
GalaxyBot Genieo
GermCrawler
GetRight
GetWeb GetWeb
Getintent
Gigabot
Go!Zilla Go!Zilla
GoZilla
Gotit
Grabber Grabber
GrapeshotCrawler Grafula
GridBot
HEADMasterSEO HEADMasterSEO
HMView HTMLparser
HaosouSpider HTTrack
Heritrix
Hloader
Humanlinks Humanlinks
HybridBot
IDBTE4M
IDBot
Image Sucker
IndeedBot IndeedBot
InfoNaviRobot
InfoTekies InfoTekies
Intelliseek Intelliseek
Internet Ninja Internet Ninja
Iskanie Iria
IstellaBot IstellaBot
JetCar Jbrofuzz
Jetty JustView
Joomla Keybot Translation-Search-Machine
Jorgee Keyword Density
Jyxobot
Kenjin Spider
Kinza Kinza
LNSpiderguy Kozmosbot
LWP::Simple LWP::Simple
LeechGet Leap
LeechFTP
LexiBot LexiBot
LibWeb LieBaoFast
Linkbot LinkWalker
LinkpadBot
Lipperhey Lipperhey
Litemage_walker
Lmspider
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc
MJ12bot
MQQBrowser MQQBrowser
MSFrontPage MSFrontPage
MTRobot MTRobot
Magnet Magnet
Majestic-SEO Mail.RU_Bot
MarkWatch
Masscan
Mata Hari Mata Hari
MauiBot Mb2345Browser
Meanpathbot Meanpathbot
Microsoft URL Control Metauri
Minefield Minefield
Mister PiX
Moblie Safari
Mojeek Mojeek
Mojolicious
MolokaiBot
Mozlila Mozlila
Mr.4x3
Musobot Musobot
Name Intelligence Nameprotect
Navroad
NearSite
Net Vampire Net Vampire
NetAnts
NetSpider NetSpider
NetZIP NetZIP
Netcraft Netcraft
Nettrack Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler
Niki-bot Niki-bot
Nimbostratus Nimbostratus
Ninja
Nmap Nmap
Not
Nuclei Nuclei
Offline Navigator Nutch
Octopus
Openfind Openfind
Openvas Openvas
OrangeBot OrangeBot
OrangeSpider POE-Component-Client-HTTP
OutfoxBot PageAnalyzer
PageGrabber PageGrabber
PageThing.com
Pandalytics Pandalytics
Papa Foto Panscient
Pavuk
PeoplePal
Petalbot
PleaseCrawl
ProPowerBot
Probethenet Probethenet
PxBroker PyCurl
QueryN Metasearch
Quick-Crawler
RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex RankingBot2
RankurBot Rankivabot
Re-re Re-re
RebelMouse ReGet
Recorder RocketCrawler
Ripper SEOkicks-Robot
SBIder SEOlyticsCrawler
SEOkicks
SEOstats SEOstats
SISTRIX
SalesIntelligent
Scrapy Scrapy
ScrepyBot
Seekport
SemrushBot
SeoSiteCheckup
SeobilityBot SeobilityBot
Site Sucker Seomoz
SiteCheckerBotCrawler SiteLockSpider
Sitebeam SiteSucker
Siteimprove Sitevigil
SlySearch
SmartDownload SmartDownload
Snake
Snapbot Snapbot
Snoopy
Sociscraper
Sosospider Sosospider
Sottopop Sottopop
SpaceBison SpaceBison
SpankBot SpankBot
Spbot Sqlmap
Spinn3r Sqworm
SputnikBot
Sqlworm
Sucker
Sucuri
Surfbot Surfbot
T0PHackTeam
Teleport Teleport
TeleportPro Telesoft
The Intraformant Thumbor
TightTwatBot
Toata Toata
Trendiction
True_Robot
Turingos Turingos
Turnitin Turnitin
TwengaBot URLy Warning
Twice Upflow
UnisterBot VB Project
VeriCiteCrawler
Virusdie
VoidEYE VoidEYE
Voltron Voil
WASALive-Bot WBSearchBot
WEBDAV
WISENutbot WISENutbot
WWW-Collector-E WWW::Mechanize
WWW-Mechanize WWWOFFLE
Wallpapers/3.0 Wallpapers
WallpapersHD WallpapersHD
Web Auto
Web Collage Web Collage
Web Enhancer
Web Fetch
Web Pix
WebAuto WebAuto
WebEnhancer WebEnhancer
WebPix WebGo IS
WebImageCollector
WebReaper
WebSauger WebSauger
WebStripper WebStripper
WebSucker
WebWhacker WebWhacker
WebmasterWorldForumBot Webalta
Webshag WebsiteQuester
Website Quester Webster
WebsiteExtractor Whatweb
Whack
Who.is Bot
Widow Widow
WinHTTrack
Wonderbot Wonderbot
Woobot Woobot
Xaldon WebSpider Xaldon WebSpider
Xenu Zitebot
Zermelo ZmEu
Zeus ZoomBot
ZumBot
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo.pt
autoemailspider backlink-check
check1.exe cah.io.community
com.plumanalytics clark-crawler
coccocbot
crawl.sogou.com
crawler.feedback crawler.feedback
domainsproject.org
eCatch eCatch
evc-batch facebookscraper
instabid gopher
heritrix
ips-agent
iubenda-radar iubenda-radar
linkdexbot
lwp-request
lwp-trivial lwp-trivial
mediawords mediawords
oBot page scorer
polaris version plumanalytics
ripz probe-image-size
satoristudio.net scan.lol
seoscanners seocompany.store
serpstatbot seostar
siteripz sitechecker.pro
tAkeOut sogouspider
voyagerx.com ubermetrics-technologies.com
webmeup-crawler webprosbot
x09Mozilla x09Mozilla
x22Mozilla
xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
70casino.online dieswaene.com
agardomains.com gov.yanao.ru
analnoeporno.tv hd-filmy.net
arraty.altervista.org instabid.tech
face.hostingx.eu kostenloser-sex.com
games.kolossale.ru lakomka.com.ua
istizanidinelikexanax.blogspot.com manualterap.roleforum.ru
lirunet.ru popugaychiki.com
movpod.in seozoom.it
sbt-aqua.ru voltrknc1.com

928
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 00go.com
01apple.com
100dollars-seo.com
101lesbian.xyz 101lesbian.xyz
1qingdao.com 123any.com
1stat.ru 12u.info
273-fz.ru 1millionusd.xyz
3dracergames.com
4istoshop.com
4ureyesonly.com
57883.net 57883.net
76brighton.co.uk 8lufu.com
7zap.com 8si.ru
83net.jp DomainStatsBot
8gold.com aa14ab57a3339c4064bd9ae6fad7495b5f.com
98oi.ru abcdefh.xyz
aac500b7a15b2646968f6bd8c6305869d7.com above.com
aad1f4acb0a373420d9b0c4202d38d94fa.com abwa.tk
acmebtn.ml admitad.com
adamoads.com adsfresh.men
addray.pro adtech.de
adexprts.com advancedmassagebysara.com
adserver-e7.com affordablewebsitesandmobileapps.com
adservme.com ahhjf.com
agadelha.com.br aihelen.net
agysya.ru alarmobninsk.ru
ahrntal.verymes.xyz allcredits.su
aimayangzhi.com alphahoverboards.com
air-edem.ru
aktivator-windows10.blogspot.com
albamargroup.com
allbizne.co.ua
allnews24.in
alt-servis.ru alt-servis.ru
amung.us amoi.tn
anaksma.info android-systems.ru
anatomiy.com animal-drawings.com
android4fun.org aosheng-tech.com
anticrawler.org apartmentbay.ru
apelsinnik.website apidigidocketnet-a.akamaihd.net
apibrowsesmartne-a.akamaihd.net arius.tech
apidiamondatanet-a.akamaihd.net as5000.com
appearance-cool.com auto-moto-elektronika.cz
arenda-yeisk.ru auto.rusvile.lt
articlesdirectoryme.info automobile-spec.com
arvut.org autotop.com.ua
avitocars.ru aviav.org
avtovolop.ru awaybird.ru
ayeartoforget.com ayakino.net
ayodhya.co azartclub.org
azbukadiets.ru azartmsl.com
backuperwebcam.weebly.com b00kmarks.com
bag77.ru bagsonsale.online
balois.worldbreak.com baltgem.com
banksrf.ru bangla-chat-uk-paralud.ga
bannerswap.com begalka.xyz
bbsoldes.fr besthomemadepornsites.com
beachpics.com bestwebsiteawards.com
bear.gotcher.us betterscooter.com
beauty-bracelet.com bezlimitko.xyz
beauty-things.com bif-ru.info
beremenyashka.com bioscorp.ru
bestimagecoollibrary.com bitporno.sx
besttorrentknifta.weebly.com blackcurranthumidifiers.site
bezsporno.ru blacktwhite.com
bget.ru
big-boards.info
bitcoin-ua.top
bizzliving.com
blog-hits.com blog-hits.com
blog.axant.it bloglag.com
bloggerads.net blogs.rediff.com
bmwhighperformers.com
bolitgorlo.net
bombla.org
bonvillan.com bonvillan.com
bpro1.top braintobe.win
bradleylive.xyz brandov.ru
brandbucket.com brendbutik.ru
btnativenav.com brillianty.info
budpost.com.ua brimstonehillfortress.org
buehne-fuer-menschenrechte.de businesn.men
burkesales.com businesxxl.com
buytizanidineonline.blogspot.com buypillsorderonline.com
buzz-porno.info bzero1jewelry.net
buzzsumo.com canacopegdl.com
candlehandmade.com capsderfudd.tk
candycrushshop.com
candypeople.se
canrole.com
carina-sy.de
carstrends2015.com
cartierreplica.pw cartierreplica.pw
cbb1smartlist12.click cartierreplica.top
cenokos.ru cbozhe.com
chatroulette.world cfsrating.sonicwall.com
cheapsergiorossi.online chatroulette.online
chinese-amezon.com cheap-pills-norx.com
ci.ua chinaelitecheapjerseys.com
cialiswithoutadoctor.net chiptuninger.com
cicaki.net choice-credit.ru
co.lumb.co christianlouboutinoutlet.win
compliance-julianna.top cibpenonptib.flu.cc
compliance-walter.top citysecurity.nu
clipartnew.com
clubfashionista.com
cmd.kz
collegerentals.com
colobit.biz
compliance-donald.xyz
compliance-jane.top
compliance-mary.top
compucelunlock.net compucelunlock.net
confib.ifmo.ru copyrightclaims.org
coolgamechannel.com counciltally.com
coslab.club
cowlmash.com
crazyprotocol.com
crd.clan.su crd.clan.su
credit.co.ua credit.co.ua
cscwtalkto.site custom-electric-guitar.com
cupday.com cypernhuset.se
daisye.top darmebel.com.ua
dbmkfhqk.bloger.index.hr davebestdeals.com
ddpills.com delitime.info
dealighted.com detsky-nabytek.info
dealitright.click dickssportinggoods.com
decorationspcs.com diesel-parts28.ru
deutschehobbyhuren.net dieswaene.com
digitalfaq.com domain.webkeyit.com
djekxa.ru domain2008.com
dnepr-avtospar.com.ua downloadme.life
dneprsvet.com.ua doxyporno.com
doctissimo.fr drhomes.biz
dogoimage.com dustyorate.com
domaincheck.io dvervmoskvu.ru
domznaniy.ru dwomlink.info
doreenblog.online e-collantes.com
doublepimp.com e-kwiaciarz.pl
drupa.com earncash.com.ua
dvrlists.com ecommerce-seo.org
dynainbox.com ecookna.com.ua
dyt.net ednorxmed.com
e-buyeasy.com educatemetv.com
earl-brown.info eduserver.net
eets.net ekobata.ru
ekaterinburg.xrus.org electronicadirect.com
electricwheelchairsarea.com elektrischeziga.livejournal.com
en.altezza.travel elmacho.xyz
enge-fotzen.info eloxal.ru
erotische-geschichten-xxl.com emerson-rus.ru
etur.ru escortslet.net
extlinks.com eshop.md
extractorandburner.com estibot.com
eyelike.com.ua euroskat.ru
ezigarettekaufen.myblog.de excaliburfilms.com
ezigarettenkaufen1.deviantart.com exct.net
expertnaya-ocenka.ru
expolicenciaslatam.co
expresstoplivo.ru
extads.net
extremepornos.net
ezigarettekaufen2.blox.pl
ezigaretteshop2.mywapblog.com
f-loaded.de f-loaded.de
f07.de failingmarriege.blogspot.com
faithe.top fastcrawl.com
fba-mexico.com felizporno.com
fetishinside.com filmania-x.ru
fickkontakte.org financetip.eu
fickluder69.com findsexguide.com
fidelityfunding.com flauntyoursite.com
financehint.eu flirt4free.com
findacheaplawyers.com fotoxxxru.com
florida-tourism.net fr.netlog.com
forex-procto.ru freefoto.ca
forex21.ru freesitetest.com
forum-engineering.ru fsalas.com
freejabs.com
front.ru
ftns.ru
fugarif.ga fugarif.ga
fusoradio.info fullgirl.ru
galeon.com funcrushgames.com
gambarkatabaru.com gallery.rennlist.com
gamesprite.me gallerylisting.com
gamevalue7.weebly.com game-mmorpg.net
gamingspark.com gamebackyard.com
ge0ip.org garciniaxt.us
germetiki.com.ua gepezz.info
getmyads24.com get-seo-domain.com
gheus.altervista.org getmiro.com
girls-ufa.ru glastecfilms.com.my
gkvector.ru go2mike.ru
go2jump.org goatse.ru
goldpanningtools.com goodhousekeeping.com
gonextmedia.com goodhumor24.com
goodnightjournal.com gotcher.us
gotwebsite1.com graphicwe.org
greatfind-a.akamaihd.net
greatidea.marketing
greenidesign.co
gribkovye-zabolevaniya.com gribkovye-zabolevaniya.com
gsasearchengineranker.top gta-club.ru
gsmlab.pl guildebzh.info
hard-porn.mobi guitar-master.org
hatdc.org gwebtools.com.br
hdhc.ru hauleddes.com
headpress.ru hayate.biz
helicalpile.us hd720kino.ru
heliko.no hdapp1008-a.akamaihd.net
hermesbirkinhandbagoutlets.com hdxnxxtube.mobi
herokuapp.com headpharmacy.com
healgastro.com
healing-dysplasia.ru
hermesreplica.win
hornymatches.com hornymatches.com
host-protection.com hoterika.com
hosting-tracker.com howmuchdoestizanidinecost.blogspot.com
hotel-mkad.ru howtoclean.club
hotkeys.com hunterboots.online
hptwaakw.blog.fc2.com hyhj.info
hreade.com hyipmanager.in
htmlcorner.com img.wallpaperstock.net
iklysha.ml immobiliaremassaro.com
ilmexico.com infospot.pt
images.gyffu.com int.search.tb.ask.com
imagez.co interior-stickers.ru
imagui.eu intervsem.ru
imguramx.pw
imk.com.ua
immigrational.info
incanto.in.ua
indiakino.net
infobabki.ru
infodocsportal.com
infokonkurs.ru
infoupdate.org
infowarcraft.ru
insider.pro
intermesh.net
investmac.com
investpamm.ru investpamm.ru
iplusbit.blogspot.co.za iomoio.net
iqupdatetmz.win isistaylorporn.info
iradiology.ru istanbulit.com
iscblog.info itag.pw
ivoiretechnocom.ci iusstf.org
j-times.ru
jackwolfskinoutlet.online
jagg.info
janavibekken.no
japfm.com japfm.com
jav-idol.com jav-idol.com
javcoast.com javitas.info
jerseyssportsshop.com javrip.net
jetsli.de javspace.net
jillepille.com jikoman.info
johannesburgsingles.co.za jolic2.com
jose.mulinohouse.co kadashihotel.com
joyceblog.top kerei.ru
kamagragelusa.net kimcurlrvsms.com
karting196.ru kinopolet.net
katadhin.co knogg.net
keywords-monitoring-success.com
kino-fun.ru
kinoflux.net
kinoplen.ru
kiwi237au.tk
kliksaya.com
koleso24.com.ua koleso24.com.ua
kovrenok.ru komputers-best.ru
kredit-blog.ru konkursowo-24.pl
kurwa.win koral.se
labplus.ru kostenlos-sexvideos.com
lacloop.info kostenloser-sex.com
ladsblue.com kozhniebolezni.com
lakomka.com.ua krasivye-devushki.net
laparfumotec.com kurbappeal.info
larose.jb2c.me lancheck.net
laserpen.club laudit.ru
law-check-eight.xyz lazymanyoga.com
lawyers.cafe
le-clos-des-alouettes.com le-clos-des-alouettes.com
legalrc.biz lesbian.xyz
lietaer.com lexaprogeneric.link
liky.co.ua linkarena.com
limtu.ifmo.ru linkpulse.com
lineavabit.it liver-chintai.org
lion.snapmap.us loadingpages.me
live-sexchat.ru longlifelomilomi.info
livingroomdecor.info
lookingglassemb.com lookingglassemb.com
lunamedia.co louboutinreplica.xyz
luxmagazine.cf luxmagazine.cf
lyrics.home-task.com mackeeper-land-672695126.us-east-1.elb.amazonaws.com
m1media.net magda-gadalka.ru
macdamaged.tech magiadinamibia.blogspot.com
macotool.com mainlinehobby.net
madot.onlinewebshop.net masterseek.com
matb3aa.com max-eclat.men
mature.free-websites.com mebel-vstroika.ru
max-p.men medicine-4u.org
maxthon.com meendo-free-traffic.ga
maxxtor.eu meetingrainstorm.bid
medkritika.ru mega-bony-2017.pl
megaindex.ru mejoresfotos.eu
memberty.com menstennisforums.com
microsearch.ru metaxalonevstizanidine.blogspot.com
minegam.com meteostate.com
missvietnam.org mezaruk.info
mixtapetorrent.com mhi-systems.ru
mnogabukaff.net michellblog.online
mobstarr.com microstatic.pl
modenamebel.ru mindbox.co.za
mojaocena.com mini.7zap.com
moje-recenze.cz minyetki.ru
mojpregled.com mnogolok.info
moscow.xrus.org moesonce.com
mouselink.co mokrayakiska.com
mozello.ru moskva.nodup.ru
mp3films.ru movies-in-theaters.net
mug-na-chas-moscow.ru moviezbonkerssk.cf
movpod.in
mrlmedia.net
mturkcontent.com
musicdaddy.net
mustat.com mustat.com
namenectar.com mydeathspace.com
narutonaruto.ru nakozhe.com
new7ob.com narkologiya-peterburg.ru
next-dentists.tk natali-forex.com
nfvsz.com naval.jislaaik.com
nhl17coins.exblog.jp nelc.edu.eg
niki-mlt.ru newyorkhotelsmotels.info
nikitabuch.com nhl09.ru
nikitsyringedrivelg.pen.io nikkiewart.ru
notaria-desalas.com nmrk.ru
npoet.ru nonameread45.live
nudejapan.net notebook-pro.ru
november-lax.com
o-o-6-o-o.com
o-o-8-o-o.ru
oakridgemo.com
obesidadealgarve.com
ochistka-stokov.ru
odesproperty.com odesproperty.com
okonich.com.ua
oksrv.com
onclickpredictiv.com onclickpredictiv.com
online.ktc45.ru oneminutesite.it
online7777.com online-hit.info
onlineku.com online-zaymy.ru
onload.pw ooo-olni.ru
onlyporno.ru ossmalta.com
ons-add.men ownshop.cf
oops-cinema.ru paceform.com
oren-cats.ru pacificair.com
osoznanie-narkotikam.net paclitor.com
ovirus.ru pathwhelp.org
ozas.net pc-virus-d0l92j2.pw
pateaswing.com pdfprof.com
pawli.eu penisvergrotendepillennl.ovh
pcgroup.com.uy perfectplanned.com
pekori.to petrovka-online.com
photorepair.ru photorepair.ru
phuketscreen.com piccdata.com
picmoonco.pw pics-group.com
podshipniki-ntn.ru picturesboss.com
polska-poezja.com pinkduck.ga
popander.mobi pinwallpaper.top
poptool.net pipki.r.acdnpro.com
pornosemki.info plastgranar.nu
pornotubs.com play.leadzupc.com
portadd.men pod-muzyku.club
postclass.com pontiacsolstice.info
popugaychiki.com
popup.matchmaker.com
portnoff.od.ua
pozdrawleniya.ru
pozvonim.com pozvonim.com
preparevideosafesystem4unow.space pp-budpostach.com.ua
privatefx.all4invest.info privatamateure.com
pro-okis.ru prize44.com
promalp-universal.ru prizestohandle.club
pssucai.info prod2016.com
pussyspace.net progress-upakovka.ru
pwwysydh.com psvita.ru
q-moto.ru queerspace.com
qld10000.net quick-offer.com
r-control.ru qwertty.net
ragecash.com raavidesigns.com
recipedays.ru ranksignals.com
relax.ru ranksonic.info
remedyotc.com registrydomainservices.com
remontgruzovik.ru rentehno.ru
reward-survey.net research.ifmo.ru
rialp.getenjoyment.net rfd-split.hr
romhacking.ru rimedia.org
rusenvironmental.net ritlweb.com
s.lollypopgaming.com rogervivierforsale.com
sakhboard.ru rootandroid.org
royalcar-ufa.ru
royalvegascasino.com
ruf777.com
runofilms.ru
russkie-gorki.ru
s1z.ru
sale-japan.com
salutmontreal.com
sammlungfotos.online
santasgift.ml
sarafangel.ru sarafangel.ru
sbt-aqua.ru
sc-specialhost.com sc-specialhost.com
scanner-ivan.top scanner-barak.top
scanner-willy.top score-ads.men
scmor.ilxc.cc sdelatmebel.ru
screentoolkit.com searchengineranker.email
search.pch.com searchtooknow-a.akamaihd.net
searchimpression.com seeingmeerkat.com
sendearnings.com segol.tv
serw.clicksor.com seinterface.com
sevenstars7.com seokicks.de
sexflirtbook.com sex-pr.net
sexsearch.com sexuria.net
sexspornotub.com sexvideo-sex.com
sextracker.be shopvilleroyboch.com.ua
shama-rc.net
sibtest.ru
silktide.com
sim-service.net
similardeals.net similardeals.net
simple-share-buttons.com sinel.info
sinestesia.host.sk sisiynas.ru
site-analyzer.com sitechecker.pro
site3.free-share-buttons.com skanninge.se
sivs.ru slutloadlive.com
skladvaz.ru smokewithrabbits.com
skylta.com social-buttons.com
slkrm.ru socialmadesimple.com
snip.to softlinesolutions.me
socialbookmarksubmission.org
softonicads.com
softwaretrend.net softwaretrend.net
sogimlecal.tk soheavyblog.com
soochi.co sonnikforme.ru
souvenir.cc sotkal.lark.ru
spb.ru sovetogorod.ru
speechfoodie.com spaceshipad.com
spy-app.info spidtest.space
spinazdrav.ru
sstroy44.ru sstroy44.ru
stairliftstrue.com stair.registrydomainservices.com
stard.shop starpages.net
stats-public.grammarly.io statistici.ro
stauga.altervista.org statustroll.com
strongholdsb.ru storehouse.ua
studiofmp.com stpolice.com
sudexpert66.ru stroy-portal22.ru
sugarlyflex.pw studentguide.ru
superfish.com superiends.org
swinginwithme.ru superstarfloraluk.com
swinon.site superstats.com
tamada69.com svbur.ru
tanieaukcje.com.pl tackletarts.co
td-33.ru taqplayer.info
tdsing.ru technika-remont.ru
tecnoteakviareggio.it tedy.su
tetracsaudi.com templates.radiodigital.co
thefds.net testbotprocessor44.com
theheroes.ru tfxiq.com
thruport.com the-trader.net
timer4web.com thegameriders.com
tizanidineformigraines.blogspot.com thenews-today.info
tizanidinehcl4mgisitanarcotic.blogspot.com thetoiletpaper.com
thfox.com
thiegs.reco.ws
titelhelden.eu
tizanidine4mg.blogspot.com
tizanidine4mguses.blogspot.com
tizanidinecost.blogspot.com
tizanidinedruginteractions.blogspot.com
tizanidinehcl2mgsideeffects.blogspot.com
tizanidinehydrochloride2mg.blogspot.com tizanidinehydrochloride2mg.blogspot.com
tizanidinenarcotic.blogspot.com tizanidineingredients.blogspot.com
top10registrycleaners.com tizanidinerestlesslegsyndrome.blogspot.com
topshef.ru tnctrx.com
tourismvictoria.com tonerbox.kz
toursmaps.com tootoo.to
track112.site top-karkas.ru
trafficjunky.net topmira.com
trafficstars.com toys.erolove.in
trafaret74.ru
traffique.net
trailer.cinemaflix.website
truck-land.life
trustedhealthtips.com trustedhealthtips.com
turizm.bz ttrraacckkrr.com
unrealcommander.org tubeline.biz
url-extractor.xyz tvorozhnaja-zapekanka-recept.ru
urldelivery.com uchetunet.su
ustion.ru udayavani.com
uytmaster.ru uhod-za-sobakoj.ru
v24s.net uhodzalijami.ru
undergroundcityphoto.com
upproar.com
uptime-as.net
uptime-delta.net
usiad.net
uzungil.com
vclicks.net vclicks.net
vektorpress.ru vektorpress.ru
vesnatehno.com veles.shop
vgoloveboli.net videos-for-your-business.com
via-energy-order.com videtubs.pl
videochat.mx vikistars.com
videokrik.net vip2ch.com
visitcambridge.org vipps.com.my
vpnmouse.com viteonlusarezzo.it
vriel.batcave.net voditeltrezviy.ru
vsesubwaysurfers.com vozbujdenie.com
vucms.com vvpg.ru
w7s.ru vzlomsn.org
wallpapers-best.com wdfdocando.com
wallpapersist.com web-betting.ru
wallpaperstock.net webads.co.nz
wanker.us webshoppermac.com
wareseeker.com websites-reviews.com
watsonrealtycorp.com weightatraining.com
webcamdevochka.com winterclassichockeyjerseys.com
weboptimizes.com wjgony.com
webs.com workle.website
weburlopener.com
wheelchairliftsarea.com
wildnatureimages.com
witclub.info
works.if.ua
worldinternetauthority.com worldinternetauthority.com
ws.ampower.me x-diesel.org
wtsindia.in x5market.ru
ww1943.ru xaylapdiendanang.com
www.888.com xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
www.labves.ru xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
www.solartek.ru xn--80aodinpgi.xn--p1ai
www.timer4web.com xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
wyniki-lista.pl xolodremont.ru
x-diesel.info xtubeporno.net
xclicks.net yellowfootprints.com
xedserver.com yellowstonesafaritours.com
xerox-douglas.cf youresponsive.com
xitjw.info yourtemplatefinder.com
xn--80aanaardaperhcem4a6i.com za-music.mymobiplanet.com
xn--80ab4aa2g.xn--p1ai zaimite.ru
xn--80ajbshivpvn2i.xn--p1ai zajm-zalog-krasnodar.ru
xn--80ak6aa92e.com zazagames.org
xn--80atua3d.xn--p1ai zeroredirect.com
xnxxandxvideos.com zskdla.site
yeartwit.com zygophyceous.womanstars.site
yougame.biz
youporn-ru.com
yousense.info
yuarra.pluto.ro
yugo-star.ru
zarenica.net
zeroredirect12.com
zigarettenonl.canalblog.com
zigarettenonlinekaufen2.drupalgardens.com
zoogdisany.com
zooggames.com
zophim.me
zs2vm.top

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123locker.com 1j7740kd.website
1adult.com 4kpics.rocks
1bet.com 4rent.ru
1hwy.com 5tey7463.weebly.com
2020iscoming.info abctoppictures.net
2kata.ru abiente.ru
83net.jp adtech.fr
aac500b7a15b2646968f6bd8c6305869d7.com aero2.ru
aadroid.net afftrack001.com
aarbur.com agecheckadult.com
acheterviagrafr24.com alert-jdh.xyz
adult3dgames.com alkoravto.ru
adultactioncam.com all-streaming-media.com
adverster.com amanda-porn.ga
adzpower.com amung.us
affiliate-fr.com anabolics.shop
agadelha.com.br apiholdingmypage-a.akamaihd.net
agreda.pluto.ro appsecurityr.com
alborzan.com ardimobileinfo.ml
alfa-img.com arraty.altervista.org
alfabot.xyz artel116.ru
allornamenti.com autotrends.today
alphahoverboards.com aviapanda.ru
alveris.ru ayakino.net
amateurlivechat.org baladur.ru
an-donut.com bestcalovebracelet.cn
animenime.ru bestdooz.com
apibetweenlinesn-a.akamaihd.net bestmarriages.com
apimountainbikei-a.akamaihd.net bezcmexa.ru
arcteryxsale.online bgviagrachrx.com
arendovalka.xyz bidbuy.co.kr
asdfz.pro biteg.xyz
atmagroup.ru bkns.vn
atofilms.com bloglag.com
auto-moto-elektronika.cz boanonihaca.tk
automobile-spec.com bosman.pluto.ro
avon-novosib.ru bronzeaid-a.akamaihd.net
avtolombard-krasnodar.ru btvn.ru
ayeartoforget.com buigas.00it.com
azbukadiets.ru bumble.cheapwebsitehoster.com
azinoofficial777.ru burger-tycoon.com
backgroundpictures.net buyessayonline19.blogspot.ru
backlink4u.net buyparajumpers.online
backuperwebcam.weebly.com bycontext.com
best-businessman.ru capacitacionyaprendizaje.com
big-boards.info cheapjerseysbizwholesale.us
bigcities.org
bimmerpost.com
bmsco.co
bmw.afora.ru
bonux.nextview.ru
brateg.xyz
brimstonehillfortress.org
bugshoot.cn
buytizanidineonline.blogspot.com
c-english.ru
calcularpagerank.com
call-of-duty.info
candcstuccoandstone.com
cheapmoncler.win cheapmoncler.win
chinaelitecheapjerseys.com citysecurity.nu
choice-credit.ru clksite.com
cleansearch.net
clodo.ru clodo.ru
cntravelre.com compliance-checker.info
compliance-donald.xyz cookie-law-enforcement-aa.xyz
cosmetic.donna7753191.ru counciltally.com
crafthubs.com covadhosting.biz
cutalltheshit.com credit-online.ws
cv.wallhade.co cscwtalkto.site
cwetochki.ru custom-product-labels.com
cyprusbuyproperties.com dbmkfhqk.bloger.index.hr
danschawbel.com ddospanel.com
detalizaciya-tut.biz dimfour.com
detective01.ru dojki365.online
dinosaurus.site earthmagic.info
docs4all.com econom.co
dokfilms.net
downloader12.ru
drunkenstepfather.com
earncash.com.ua
edudocs.net
eleimgo.pw
elektronischezi.livejournal.com
elitedollars.com
elitesportsadvisor.com
englishgamer.com
enhand.se enhand.se
everflora.ru esnm.ru
expertblog.info essay-edu.biz
ezigarettenshop1.yolasite.com estibot.com
ezrvrentals.com eyelike.com.ua
fakehandbags.xyz ezigaretteshop2.mywapblog.com
falcon-images.blogspot.com favorcosmetics.com
fandlr.com fikasound.tk
florida-tourism.net formseo.com
freefoto.ca foto-weinberger.at
freenode.info frameimage.org
freshsuperbloop.com fun-mobi.pl
geileweiber.tk funnymama.com
glopages.ru funponsel.com
gopro-online.info galleryawesome.com
gotorussia.com gallerylisting.com
gwebtools.com garciniaxt.us
hamacapty.com ge0ip.net
happy.new.yeartwit.com getyourimage.club
hd-film.pl goo.ne.jp
hdpixion.com gotwebsite1.com
hdwallpapers-free.com hamilton.ca
headpress.ru hatdc.org
helvetia.com.ua hayate.biz
hentai-manga.porn heatpower.ru
hentaiheroes.com hoholikik.club
highstairs-a.akamaihd.net hotsocialz.com
homegardenlova.com howtowhitenteethfast.xyz
host-protection.com hundejo.com
hoste.octopis.com ibmdatamanagement.co
hothot.ru imagerydatabase.com
housekuba.org imagine-ex.co
htmlcorner.com impotentik.com
imgarit.pw incest-ru.com
itbc.kiev.ua infogame.name
iyasimasennka.com infoupdate.org
izhevsk.xrus.org integritylandscapeservices.com
junketjuice.blogspot.com intim-uslugi.info
justkillingti.me iqupdatetmz.win
keywordglobal.co.za iwantmyfreecash.com
klitimg.pw iweblist.info
kochanelli.com joyceblog.top
krasivye-devushki.net kazka.ru
krassh.ru keyword-suggestions.com
ku6.com kidd.reunionwatch.com
kinopolet.net
kirov.zrus.org
kunstaktien.info kunstaktien.info
labplus.ru kutikomi.net
ledpolice.ru larchik.net
lenpipet.ru larose.jb2c.me
lidoradio.com lego4x4.ru
likesuccess.com letmacworkfaster.site
likrot.com liky.co.ua
limtu.ifmo.ru lol-smurfs.com
linkpulse.com luxup.ru
longlifelomilomi.info machicon-akihabara.info
lost-alpha.ru
mackeeper-land-672695126.us-east-1.elb.amazonaws.com
magicalfind-a.akamaihd.net magicalfind-a.akamaihd.net
marketland.ml makis.nu
marmitaco.cat maxxtor.eu
masterseek.com mbiologi.ru
medicine-4u.org mebeldekor.com.ua
medkritika.ru medanestesia.ru
mega-polis.biz.ua medictube.ru
megagrabber.ru menetie.ru
memberty.com middlerush-a.akamaihd.net
missis.top mirobuvi.com.ua
mobot.site mixx.com
morepoweronmac.trade mobile-appster.ru
most.gov.iq mp3ritm.top
mwtpludn.review my-big-family.com
mxgetcode.com mycaf.it
mystats.xyz mymobilemoneypages.com
nakrutka.cc mysex21.com
nelc.edu.eg mzdish.site
newsperuse.com nac-bearings.ru
northfacestore.online naturalbreakthroughsresearch.com
naughtyconnect.com
networkcheck.xyz
no-rx.info
notfastfood.ru
nuker.com nuker.com
okroshki.ru nuup.info
onlinetvseries.me obiavo.su
opinionreelle.com onlinefilmz.net
outrageousdeal-a.akamaihd.net onlineku.com
paclitor.com onlyforemont.ru
parlament.biz orion-v.com
pavlodar.xkaz.org patterntrader-en.com
photokitchendesign.com perso.wanadoo.es
photosaga.info picquery.com
platesauto.com pogosh.com
play-movie.pl
pomoc-drogowa.cba.pl pomoc-drogowa.cba.pl
popugauka.ru popads.net
porndl.org praisong.net
posible.net progonrumarket.ru
predmety.in.ua prostitutki-yaroslavlya.org
professionaldieselcare.com prpops.com
pufip.com qexyfu.bugs3.com
pulse33.ru quickbuck.com
rainbowice.ru quit-smoking.ga
runstocks.com r-control.ru
rus-pornuha.com ranksonic.com
saltspray.ru razamicroelectronics.com
scalerite.co.za realizmobi.com
scanner-ivan.top registry-clean-up.net
scansafe.net reklama-i-rabota.ru
scat.porn replica-watch.ru
sdelai-prosto.ru ribieiendom.no
seoboxes.com rome2rio.com
shitting.pro ruf777.com
shoppingmiracles.co.uk rus-teh.narod.ru
shoppytoolmac.com sadaholding.com
shopvilleroyboch.com.ua scanmarine.info
smstraf.ru scanner-walt.top
sonnikforme.ru scanner-walter.top
soserfis.com seminarykansas904.ml
spb-plitka.ru seokicks.de
squidoo.com seoprofiler.com
staff.prairiesouth.ca sex-dating.co
steebook.com sexcamamateurchat.com
stillmiracle.com shoesonlinebuy.cn
strfls.com simon3.ru
stroy-portal22.ru skincrate.net
suture.co skytraf.xyz
technopellet.gr smartbalanceworld.com
snowplanes.com
snsdeainavi.info
soc-econom-problems.ru
socialbookmarksubmission.org
soda.media
spravka130.ru
ssconstruction.co
stockspmb.info
studworks.org
symbaloo.com
taxi-v-eisk.ru
tedxrj.com
tetracsaudi.com
textads.men textads.men
thejournal.ru theendivechronicles.com
tizanidinehcl2mg.blogspot.com thesmartsearch.net
tizanidinerecreationaluse.blogspot.com titelhelden.eu
tizanidinestreetprice.blogspot.com tizanidine6mg.blogspot.com
tobeyouday.win tizanidinedosage.blogspot.com
toloka.hurtom.com tizanidinerestlesslegsyndrome.blogspot.com
tootoo.to tizanidinewithdrawal.blogspot.com
traffixer.com top-karkas.ru
tramadolandtizanidine.blogspot.com topads.men
treasuretrack-a.akamaihd.net topmira.com
toy-shop.top
trackmedia101.com
traffic2money.com
trichizobswiv.agddns.net
truck-addzilla.life
truebeauty.cc truebeauty.cc
tubeline.biz tt-ipd.info
turbodsp.com tvoystartup.ru
twodollarshows.com usiad.net
tzritel.tk vapomnoncri.tk
u17795.netangels.ru vertaform.com
ukrobstep.com vilingstore.net
umnovocaminho.com virus-schutzmasken.de
usadacha.net viveresaniesnelli.it
utrolive.ru vodaodessa.com
valkiria-tk.ru voloo.ru
vedomstvo.net voloslove.ru
vinsit.ru vostoktrade.info
virus-respirators.com warningwar.ru
viven.host.sk webalan.ru
vizag.kharkov.ua wladimirpayen.com
wallpapersdesk.info worldhistory.biz
wallpaperstock.net worldlovers.ru
we-are-gamers.com wwwadultcheck.com
white-truck.life x-diesel.com
wineitudes.wordpress.com xbaboon.com
wordkeyhelper.com xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
x-rates.ru xn--b1agm2d.net
xedserver.com xsfetish.org
xvideosporn.biz yebocasino.co.za
yaaknaa.info yellowproxy.net
yes-com.com youradexchange.com
youdao.com youthreaders.com
zygophyceous.womanstars.site yurgorod.ru
zeroredirect.com
zoosexart.com
zskdla.site

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.06.3190 ### Version: V4.2022.06.3191
### Updated: Sun Jun 26 06:44:34 UTC 2022 ### Updated: Sun Jun 26 10:03:35 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.06.3190 ### Version: V4.2022.06.3191
### Updated: Sun Jun 26 06:44:34 UTC 2022 ### Updated: Sun Jun 26 10:03:35 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##