vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.10.3386 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-10-02 22:06:57 +00:00
parent 5f24e5cb63
commit 51f6b5b141
47 changed files with 96533 additions and 96523 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3385 #### Version: V4.2022.10.3386
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3385 #### Version: V4.2022.10.3386
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.10.3385 #### Version: V4.2022.10.3386
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11932
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.10.3385 #### Version: V4.2022.10.3386
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11936
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11936
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11936
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11936
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

11936
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

11936
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

11936
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3385 3386

11936
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11936
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
168.61.4.191 0; 20.165.36.187 0;

11936
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

11936
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

11936
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

11936
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
168.61.4.191 0; 20.165.36.187 0;

11936
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
168.61.4.191 0; 20.165.36.187 0;

11936
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 1; 10.1.0.39 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AiHitBot CensysInspect
BBBike Jorgee
Grafula
InternetSeer
LinkWalker LinkWalker
Mr.4x3 Magnet
Rankivabot Minefield
RocketCrawler Petalbot
Upflow Psbot
WWW-Collector-E SpaceBison
Toata
Web Auto

284
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404enemy 404enemy
80legs ADmantX
ALittle Client Aboundex
ASPSeek
Aboundexbot Aboundexbot
Acunetix Acunetix
AfD-Verbotsverfahren AhrefsBot
AllSubmitter Alligator
AlphaBot Anarchy
Anarchie Anarchy99
AwarioRssBot Atomseobot
Attach
AwarioSmartBot
BDCbot BDCbot
BLEXBot BackStreet
BackDoorBot BackWeb
BacklinkCrawler BacklinkCrawler
Battleztar Bazinga
Bigfoot
Black Hole
Blackboard
Blow
Boardreader
Bolt
BotALot
Brandprotect
Brandwatch Brandwatch
Bullseye
BunnySlippers BunnySlippers
CATExplorador
CCBot CCBot
CODE87
Calculon Calculon
CazoodleBot CazoodleBot
ChinaClaw CensysInspect
CheTeam
CherryPicker
Chlooe Chlooe
Cloud mapping Cocolyzebot
Copier Cogentbot
Collector
CopyRightCheck
Copyscape Copyscape
Craftbot Crawling at Home Project
Crescent
CrunchBot
Curious Curious
Custo DBLBot
DIIbot DIIbot
DataCha0s DataCha0s
DatabaseDriverMysqli
Demon
Deusu
Digincore Digincore
DigitalPebble DigitalPebble
Dirbuster
Discoverybot
Dispatch
DittoSpyder
DomCopBot
DomainAppender DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot Dotbot
Dragonfly
Drip Drip
ECCP/1.0
EMail Wolf EMail Wolf
EasyDL Ecxi
EroCrawler
Evil Evil
Exabot
Express WebPictures Express WebPictures
Extractor Extreme Picture Finder
ExtractorPro
EyeNetIE EyeNetIE
FDM Ezooms
FHscan
Fimap
FlashGet FlashGet
Flunky Flunky
Fuzz Fuzz
Fyrebot Fyrebot
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot
Genieo
GetRight GetRight
GetWeb GetWeb
Getintent
Gotit Gotit
GrabNet GrabNet
Grabber GrapeshotCrawler
HMView
HTTP::Lite HTTP::Lite
HTTrack HTTrack
HaosouSpider HaosouSpider
Humanlinks Havij
HybridBot HybridBot
IDBTE4M IlseBot
Image Sucker
IndeedBot
Indy Library Indy Library
InterGET Intelliseek
Iria Iria
Iskanie Iskanie
IstellaBot
JOC Web Spider JOC Web Spider
JikeSpider Jbrofuzz
Joomla Jorgee
JustView JustView
Jyxobot Jyxobot
Keybot Translation-Search-Machine LNSpiderguy
Keyword Density Larbin
Kinza LeechGet
Kozmosbot LexiBot
Lanshanbot LibWeb
Lftp
Likse
LinkScan
LinkextractorPro LinkextractorPro
LinkpadBot LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey Spider
Litemage_walker Litemage_walker
Lmspider MIIxpc
Ltx71 Mag-Net
Majestic12 Mail.RU_Bot
Masscan Majestic-SEO
MauiBot Mass Downloader
Mata Hari
MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot Metauri
MicroMessenger
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Moblie Safari Minefield
Mojeek
MolokaiBot
Morfeus Fucking Scanner
Mr.4x3 Mr.4x3
NPbot
Name Intelligence
Nameprotect Nameprotect
NearSite Needle
NetAnts NetMechanic
NetLyzer NetZIP
Netcraft Nettrack
Niki-bot
Nikto Nikto
Ninja Ninja
Not
Nuclei Nuclei
Offline Explorer Octopus
OpenLinkProfiler
OpenVAS
OrangeSpider OrangeSpider
OutclicksBot
PHPCrawl PHPCrawl
Page Analyzer Page Analyzer
PageAnalyzer PageAnalyzer
PageGrabber PageGrabber
Papa Foto
Pavuk Pavuk
Petalbot PeoplePal
Picsearch Picsearch
PictureFinder PictureFinder
ProWebWalker Pockey
Pump ProPowerBot
Probethenet
PxBroker PxBroker
PyCurl
QueryN Metasearch
RankActive RankActive
RankActiveLinkBot RankFlex
Re-re Rankivabot
RealDownload RebelMouse
RepoMonkey RedesScrapy
Ripper
Rogerbot
SBIder SBIder
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler
SEOstats SEOstats
SISTRIX SalesIntelligent
SMTBot Scrapy
ScoutJet
Searchestate
SearchmetricsBot
Seekport
SeoSiteCheckup SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz
Shodan
Siphon
SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker
Sitebeam Sitebeam
Siteimprove Siteimprove
Sitevigil
SlySearch
Snapbot Snapbot
Sociscraper Sogou web spider
Sosospider Sottopop
SpaceBison Sqlmap
Spammen
Sqlworm Sqlworm
Sqworm Sqworm
Steeler
Stripper Stripper
Sucuri
SuperBot SuperBot
T0PHackTeam Telesoft
TeleportPro
Telesphoreo
Telesphorep
The Intraformant
TheNomad
TightTwatBot TightTwatBot
Toata Toweyabot
Tracemyfile Tracemyfile
Trendiction Trendictionbot
TurnitinBot True_Robot
TwengaBot Twice
URLy.Warning URLy Warning
V-BOT
VB Project
VCI VCI
VidibleScraper
Virusdie
VoidEYE
Voltron Voltron
WASALive-Bot
WBSearchBot WBSearchBot
WISENutbot
WPScan WPScan
WWW-Collector-E
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers Web Fuck
Web Enhancer
Web Sauger
WebAuto WebAuto
WebBandit
WebCollage
WebCopier WebCopier
WebEnhancer
WebFetch WebFetch
WebLeacher WebLeacher
WebReaper WebReaper
WebSauger
WebWhacker WebWhacker
WebZIP
Webalta
WebmasterWorldForumBot
Webshag Webshag
Website Quester
WebsiteQuester WebsiteQuester
Webster
Whacker
Whatweb Whatweb
Who.is Bot WinHTTrack
Widow WiseGuys Robot
Wonderbot
Wotbox Wotbox
Wprecon Wprecon
Xaldon_WebSpider
Xenu Xenu
YoudaoBot
Zade Zade
ZoomBot Zeus
ZumBot Zitebot
adscanner ZoominfoBot
archive.org_bot ZyBorg
arquivo.pt autoemailspider
coccocbot check1.exe
crawler4j crawl.sogou.com
domainsproject.org crawler.feedback
demandbase-bot
eCatch eCatch
evc-batch internetVista monitor
facebookscraper
heritrix
ips-agent ips-agent
isitwp.com
iubenda-radar iubenda-radar
linkdexbot linkdexbot
magpie-crawler lwp-trivial
meanpathbot
oBot oBot
polaris version pcBrowser
ripz probe-image-size
s1z.ru s1z.ru
scalaj-http satoristudio.net
scan.lol
seobility seobility
seocompany.store
seoscanners seoscanners
serpstatbot serpstatbot
sogouspider sogouspider
sp_auditbot sp_auditbot
sysscan spyfu
ubermetrics-technologies.com trendiction.de
webmeup-crawler x09Mozilla
webprosbot x22Mozilla
xpymep1.exe zauba.io
zgrab

276
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
404enemy 404enemy
ALittle Client ADmantX
Alexibot AIBOT
AllSubmitter AllSubmitter
Anarchy Alligator
Aspiegel Ankit
Asterias Anthill
Attach Apexoo
AwarioSmartBot BDFetch
BDCbot BackStreet
BacklinkCrawler Backlink-Ceck
Badass
Bandit Bandit
BatchFTP
Battleztar Bazinga
Bitacle Bitacle
Blackboard Black Hole
BotALot Brandprotect
Brandwatch
Buck Buck
BuiltWith Buddy
Bullseye
BuzzSumo BuzzSumo
CCBot CATExplorador
CODE87
CSHttp CSHttp
Calculon
CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect CensysInspect
CheTeam
CherryPicker CherryPicker
ChinaClaw Claritybot
Chlooe Cocolyzebot
Citoid Cogentbot
Cloud mapping Collector
Copier
Copyscape Copyscape
Cosmos Cosmos
Craftbot Craftbot
CrazyWebCrawler
Crescent Crescent
CrunchBot
Curious
DTS Agent DTS Agent
Deusu
Devil Devil
DigitalPebble
Disco
Discoverybot Discoverybot
Dispatch
DnyzBot DnyzBot
DomCopBot DomainCrawler
Dotbot
Download Wonder Download Wonder
EasyDL Dragonfly
Ebingbong Drip
Ecxi EMail Siphon
ExtLinksBot Express WebPictures
Extractor Extractor
Extreme Picture Finder
EyeNetIE
FemtosearchBot
Fimap Fimap
FlashGet Firefox/7.0
Flunky
Freeuploader
FyberSpider FyberSpider
Getintent G-i-g-a-b-o-t
Gigabot GermCrawler
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GoZilla
Gotit Gotit
GrabNet
HEADMasterSEO
HaosouSpider HaosouSpider
Havij Havij
Heritrix Heritrix
Humanlinks HonoluluBot
HybridBot HybridBot
Iblog IRLbot
Image Fetch Id-search
Image Sucker
IndeedBot IndeedBot
Indy Library
InfoTekies
Intelliseek Intelliseek
Internet Ninja Internet Ninja
IstellaBot InternetSeer
JamesBOT Iria
Jbrofuzz
JennyBot JennyBot
Jetty
JikeSpider
Joomla
Jyxobot
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Kinza Kinza
Kozmosbot LNSpiderguy
Lanshanbot Lanshanbot
Larbin
Leap
LeechFTP
LeechGet
LexiBot
LibWeb LibWeb
Libwhisker
Lightspeedsystems
LinkWalker
Linkbot Linkbot
LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider
Litemage_walker
Lmspider
Ltx71 Ltx71
MIDown tool
MJ12bot
MQQBrowser MQQBrowser
Mail.RU_Bot MSFrontPage
Majestic12 MSIECrawler
Mag-Net
MarkMonitor
MarkWatch MarkWatch
MeanPath Bot MauiBot
Meanpathbot
Minefield Minefield
Mister PiX Mister PiX
Mojolicious Mojeek
Mozlila
NICErsPRO NICErsPRO
NPbot NPbot
Navroad Navroad
Nessus
NetMechanic NetMechanic
Netcraft NetSpider
Netvibes Netvibes
Niki-bot
Nikto Nikto
Nimbostratus Not
Nutch
Offline Explorer Offline Explorer
OpenLinkProfiler OpenLinkProfiler
Openfind OpenVAS
Openvas
OrangeBot OrangeBot
OutclicksBot
OutfoxBot OutfoxBot
PECL::HTTP PageAnalyzer
PHPCrawl PageGrabber
POE-Component-Client-HTTP
Page Analyzer
PageScorer PageScorer
PageThing.com Panscient
Papa Foto
Pavuk Pavuk
PeoplePal PeoplePal
Petalbot Pi-Monster
Picscout PictureFinder
Piepmatz Piepmatz
Pixray Pimonster
PleaseCrawl Pockey
ProPowerBot PyCurl
ProWebWalker
Psbot
Pump
PxBroker
QueryN Metasearch QueryN Metasearch
Quick-Crawler RankFlex
RankActiveLinkBot Rankivabot
ReGet ReGet
RealDownload
Reaper
Recorder Recorder
SEOprofiler RedesScrapy
RepoMonkey
SBIder
SEOkicks-Robot
SEOstats SEOstats
SISTRIX
SMTBot SMTBot
ScreenerBot ScoutJet
SearchmetricsBot Scrapy
SemanticJuice
Semrush Semrush
SeoSiteCheckup SeobilityBot
Seomoz
Shodan
Siphon Siphon
SiteLockSpider Site Sucker
Sitebeam SiteExplorer
Siteimprove Siteimprove
SlySearch SmartDownload
Snoopy Snoopy
SocialRankIOBot Sociscraper
Sogou web spider Sottopop
Sosospider
Spammen Spammen
SpankBot Spanner
Spinn3r
SputnikBot
Sqlmap Sqlmap
Sqworm
Sucker Sucker
Sucuri
SuperBot SuperBot
SuperHTTP SuperHTTP
SurveyBot
Suzuran
Szukacz
T0PHackTeam
T8Abot T8Abot
TeleportPro Teleport
Telesoft
Telesphorep Telesphorep
The Intraformant The Intraformant
TheNomad
Thumbor
TightTwatBot TightTwatBot
Trendiction Titan
True_Robot
Turingos Turingos
Turnitin
TurnitinBot TurnitinBot
TwengaBot Twice
URLy Warning Typhoeus
UnisterBot
V-BOT V-BOT
Vacuum
VelenPublicWebCrawler
VidibleScraper VidibleScraper
Virusdie Virusdie
Voil
Voltron Voltron
WBSearchBot
WEBDAV
WISENutbot
WPScan WPScan
WWW::Mechanize WWW::Mechanize
Wallpapers/3.0 WWWOFFLE
Wallpapers
Web Auto Web Auto
Web Fetch Web Pix
Web Sauger
Web Sucker Web Sucker
WebEnhancer WebAuto
WebBandit
WebCollage
WebGo IS WebGo IS
WebImageCollector WebImageCollector
WebLeacher WebLeacher
WebPix WebPix
WebStripper WebStripper
WebSucker
WebWhacker WebWhacker
WebZIP WebZIP
Webalta WebmasterWorldForumBot
Website Quester Webshag
WebsiteQuester WebsiteQuester
Whack
Whacker Whacker
Widow WinHTTrack
WiseGuys Robot
Woobot Woobot
Wprecon Wotbox
Xaldon_WebSpider
Xenu Xenu
YoudaoBot
Zade
Zauba Zauba
Zermelo Zermelo
ZoominfoBot Zitebot
ZmEu
ZyBorg ZyBorg
arquivo.pt archive.org_bot
backlink-check backlink-check
cah.io.community
clark-crawler
coccocbot
com.plumanalytics
crawler.feedback
crawler4j
dataforseo.com
demandbase-bot demandbase-bot
domainsproject.org
evc-batch evc-batch
gopher
isitwp.com
iubenda-radar
linkdexbot
lwp-request lwp-request
lwp-trivial lwp-trivial
magpie-crawler magpie-crawler
muhstik-scan
oBot
plumanalytics plumanalytics
polaris version satoristudio.net
ripz scan.lol
scalaj-http seoscanners
sp_auditbot sp_auditbot
spyfu
tAkeOut
voyagerx.com voyagerx.com
webmeup-crawler
webpros.com webpros.com
x09Mozilla
x22Mozilla
zauba.io zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
abbanreddy.com brothers-smaller.ru
gotcher.us excitacion.info
keywesthideaways.co internet-apteka.ru
kiwi237au.tk linkpulse.com
rogervivierforsale.com myfreecams.com
social-buttons.com parfusale.se
tako3.com perm.xrus.org
themestotal.com phrcialiled.com
turist-strani.ru swinginwithme.ru
unmaroll.ya.ru worldoffiles.ru

944
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0lovespells0.blogspot.com 1001desert.com
1-99seo.com 1001watch.com.ua
1-free-share-buttons.com 11pikachu.ru
123any.com 1flag.co.za
2girls1cup-free.com 1stat.ru
4kepics.com 1webmaster.ml
51.la 20pascals.nl
aa8780bb28a1de4eb5bff33c28a218a930.com 24videos.tv
acc.eu.org 2daytrendingnews.com
ad-words.ru 2pxg8bcf.top
adloads.com 404.mn
admatic.com.tr 40cg.com
adobereader-free.ru 45en.ru
adrenalin-stk.ru 6128786.com
adssafeprotected.com 72-news.com
advancedmassagebysara.com 7yue.org
advertex.info 8kisses.eu
affordablewebsitesandmobileapps.com 91abcw.com
ahrefs.com aa81bf391151884adfa3dd677e41f94be1.com
alborzan.com aa9d046aab36af4ff182f097f840430d51.com
albumsuper.info abclauncher.com
alert-fdm.xyz acortarurl.es
alf-img.com adsloads.com
all-streaming-media.com adultgalls.com
alt-servis.ru aerogo.com
altamayoztourism.com afmuseum.com
amanda-porn.ga allnews24.in
amospalla.es alta-realestate.com
anabolics.shop android-systems.ru
android-style.com animenime.ru
animal-drawings.com apartmentbay.ru
anniemation.com apibrowsesmartne-a.akamaihd.net
apiqualitinknet-a.akamaihd.net apimountainbikei-a.akamaihd.net
artparquet.ru arabgirls.adultgalls.com
asophoto.com architecturebest.com
arewater.com
arraty.altervista.org
astana.xxxkaz.org astana.xxxkaz.org
atley.eu.pn atdedinotuho.tk
atofilms.com auspolice.com
audiobangout.com autochoixspinelli.com
australia-opening-times.com autoloans.com
autogrep.ru autoplate.info
autoseo-traffic.com avkzarabotok.info
aviav.ru avtoarenda.by
avramstroy.ru avtolombard-krasnodar.com
azazu.ru azartclub.org
azbuka-mo.ru
babieca.com babieca.com
backiee.com batanga.net
bamo.xsl.pt bbsoldes.fr
bazaronline24.ru beclean-nn.ru
bestmarriages.com beget.tech
bestmobilityscooterstoday.com best-businessman.ru
betune.onlinewebshop.net betshuckclean.com
bigcareer.info bg6s0.com
bin-brokers.com bibys.com
bio.trade-jp.net bioca.org
biographiya.com biographiya.com
bioinnovate.co biznesschat.net
blogos.kz blog.remote-computer.de
blogporn.in bloglag.com
bmsco.co blogs.rediff.com
board.f00d.de boanonihaca.tk
brains2.biz bolezniorganov.ru
brakehawk.com bonvillan.com
brandehk.dk boo-arts.com
bristolhotel.com bpro1.top
brokergid.ru brandov.ru
burger-tycoon.com briomotor.co
bscodecs.com
bugof.gq
buketeg.xyz
businesxxl.com businesxxl.com
buypillsorderonline.com buy-cheap-online.info
bytimedance.ru buy-loft.ru
c-english.ru buyantiviralwp.com
carapics.com buyclomidonlaine.com
cardul.ru buydissertation.net
carrauterie.be buytizanidineonlinenoprescription.blogspot.com
carsdined.org calc-for-credit.ru
candcstuccoandstone.com
carsplate.com carsplate.com
casinos4dummies.co.uk cdnanalytics.xyz
cbox.ws cdnnetwok.xyz
cdncash.net cenokos.ru
celejihad.info chatroulette.video
cfsrating.sonicwall.com cheapmoncler.xyz
children-learningreading.info checkhit.com
choice-credit.ru chocoslim.pro
citysecurity.nu christianlouboutinreplica.win
cialgenisrx.com
cigarpass.com
citetick.com
civilwartheater.com civilwartheater.com
cleanallvirus.com cl.s7.exct.net
click2pawn.com clean-virus-mac.com
clicksor.net cleanpcnow.com
cobaltpro.ru clippingphotoindia.com
club-lukojl.ru
coin-hive.com
collegerentals.com
colobit.biz
com-onlinesupport.site
compliance-don.top
compliance-don.xyz compliance-don.xyz
conboy.us compliance-fred.xyz
conocer-sanabria.com comsysnet.com
construmac.com.mx concordexoticrentals.com
cool-wedding.net cookie-law-enforcement-hh.xyz
coolwallpapers-hd.com coolwallpapers-hd.com
copenergo.ru cougfan.info
cottageofgrace.com counterbot.com
counter.yadro.ru country-chic.ru
creams.makeforum.eu cymbaltaandtizanidine.blogspot.com
culad.com daptravel.com
culmimg.pw darkbooks.org
cyse.tk darodar.com
damasarenai.info dating-time-now.com
damedingel.ya.ru ddpills.com
data-ox.com detsky-nabytek.info
data1.scopich.com devochki.top
datract.com diesel-parts28.ru
datsun-do.com digital-video-processing.com
dcj-nn.ru
ddospanel.com
deadlinkchecker.com
design-lands.ru
diamond-necklace.info
dichvuvesinhhanoi.com
dikqlyremy.info dikqlyremy.info
discountliv.com dms-sw.ru
diy-handmade-ideas.com documentbase.net
doctissimo.fr dodlive.mil
doctormakarova.ru dojki-hd.com
dokfilms.net dominateforex.ml
domainroam.win domoysshop.ru
donvito.unas.cz dvr.biz.ua
dotnetdotcom.org earncash.com.ua
downtuptv.gq
dprtb.com
e-buyeasy.com
e-collantes.com
eager-nash.188-93-233-196.plesk.page
eas-seo.com
editors.choice6912650.hulfingtonpost.com
elektrischezigarette1.blog.pl elektrischezigarette1.blog.pl
elektroniksigaraankara.info elektrischezigaretten2.beeplog.com
elektronischezigarettekaufen1.beeplog.com elektrozigaretten1.tumblr.com
elektrozigaretten2.yn.lt elstal.com.pl
elitedollars.com emediate.eu
eloconcream.blogspot.com emerson-rus.ru
elvel.com.ua empis.magix.net
energy-ua.com en.altezza.travel
eonpal.com enge-fotzen.info
eredijovon.com epngo.bz
erotik0049.com errorfixing.space
eshop.md
essay-edu.biz
etenininrade.ga etenininrade.ga
etur.ru evrotekhservis.ru
europages.com.ru extener.org
european-torches.ru ezigarettekaufen.myblog.de
excaliburfilms.com ezigarettekaufen2.yolasite.com
extstat.com
eyessurgery.ru
ezofest.sk ezofest.sk
f5mtrack.com falopicm.pw
facialporntube.com
family1st.ca
fantasticpixcool.com fantasticpixcool.com
fashion-stickers.ru fashion-mk.net
fatfasts-4tmz.com fast-torrent.ru
favoritemoney.ru fastcrawl.com
fetlifeblog.com fastfixing.tech
fickenprivat.info fatmaelgarny.com
filerockstar298.weebly.com fermersovet.ru
filkhbr.com fidelityfunding.com
filmidivx.com findclan.org
finemanteam.com
firstsiteguide.com firstsiteguide.com
fiuxy.com flex4launch.ru
flash4fun.com forpackningsutveckling.se
flashahead.info fortevidyoze.net
flooringinstallation-edmonton.com forums.d2jsp.org
fok.nl franch.info
forminecrafters.ru free-laptop-reward.com
foxweber.com free-traffic.xyz
free-today.com freeuploader.ml
freewlan.info ftns.ru
fuck-paid-share-buttons.xyz fugarif.ga
fullfileaccess.com galaxy-family.ru
fungamelands.com galleryawesome.com
fxgallery.com gamblingnerd.com
g33.org gamedayassist.com
galeria-zdjec.com gameonasia.com
gallerylisting.com garciniaxt.us
gazobeton-p.com.ua ge0ip.net
genetworx.com genericlowlatencyasiodriverhq.aircus.com
getaclueamerica.com ghazel.ru
girlsatgames.ru go2mike.ru
goforexvps.com gonextmedia.com
goldpanningtools.com goodwinmetals.co
goodwriterssales.com googlepositions.com
googlefeud.com googleseo.com.tr
googlemare.com gotwebsite1.com
gpms.org.my graphid.com
greenidesign.co gsasearchengineranker.xyz
growshop.es gsasearchenginerankerdiscount.com
gsou.cf gsbs.com.ua
guardlink.com gwebtools.com.br
handicapbathtubarea.com heliko.no
hasshe.com heroz.fr
hayate.biz heygidday.biz
hccoder.info hitsbox.info
healmytrauma.info holidaypics.org
hobild.net home.myplaycity.com
hol.es homedo.fabpage.com
homedecoguide.info homelygarden.com
hop.clickbank.net homik.pw
hostsshop.ru horoshieokna.com
hotblognetwork.com humanorightswatch.org
housekuba.org hundejo.com
housemilan.ru husky-shop.cz
hoverboard360.de i4track.net
hplaserjetpdriver8y.pen.io iblogpress.xyz
hptwaakw.blog.fc2.com id-forex.com
hully.altervista.org ikearugs.xyz
hyipmanager.in ilo134uloh.com
ideibiznesa2015.ru instakink.com
ido3.com instasexyblog.com
igrovyeavtomaty777.ru interior-stickers.ru
imagerydatabase.com investpamm.ru
inboundlinks.win inzn.ru
inmate-locator.us irkutsk.online-podarki.com
intermesh.net
ipostroika.ru
iqbazar.ru
iskalko.ru iskalko.ru
islamtoday.co.za istizanidineanopiate.blogspot.com
istizanidinelikexanax.blogspot.com iwanttodeliver.com
itrevolution.cf jackwolfskinoutlet.online
janerikholst.se jav-way.site
javidol.site jerseywholesalechinabiz.com
jillepille.com jestr.org
jongose.ninja jo24news.com
jovencitas.gratis joingames.org
jrcigars.com jose.mulinohouse.co
junketjuice.blogspot.com juliadiets.com
jurajskie.info jyvopys.com
kadashihotel.com kalandranis.gr
kamalsinha.com katadhin.co
karting196.ru kazrent.com
kazan.zrus.org
kerei.ru
kerwinandcariza.com kerwinandcariza.com
keywesthideaways.co keywordteam.net
keywords-monitoring-success.com kinozapas.org
kicknights.gq konkursowo-24.pl
kino-key.info konturkrasoty.ru
kino-rf.ru krasivoe-hd.net
kletkimehan.ru krasnodar.xrus.org
knogg.net
kollekcioner.ru
koptims.tiu.ru
koral.se
kosova.de
kostenlos-sexvideos.com
krasivoe-hd.com
kretpicf.pw kretpicf.pw
kvartiry-remont.ucoz.ru kumuk.info
lacapilla.info kupit-adenu.ru
lafourmiliaire.com kursy-ege.ru
lalalove.ru labafydjxa.su
larutti.ru lasvegaslockandsafe.com
leboard.ru levaquin750mg.blogspot.com
lechenie-gemorroya.com life.biz.ua
leftofcentrist.com lion.playtap.us
letmacworkfaster.world lion.snapmap.us
lincolntheatre.com livetsomudvekslingstudent.bloggersdelight.dk
linkarena.com lkbennettoutlet.online
littlesunraiser.com lob.com.ru
live-sexchat.ru locatellicorretor.com.br
ljusihus.se longgreen.info
lombia.com losangeles-ads.com
lonfon.xyz
lostaruban.com
louboutinreplica.pw
lutherstable.org lutherstable.org
max-eclat.men luxup.ru
mcar.in.ua makenahartlin.com
mearns-tractors.co.uk malwareremovalcenter.com
medicinacom.ru marwer.info
megavolt.net.ua masthopehomes.com
members.ghanaweb.com mathgym.com.au
menetie.ru mhi-systems.ru
mhtr.be
michaelkorsoutlet.store
microstatic.pl
mikrobiologies.ru mikrobiologies.ru
mindeyegames.com mini-modus.ru
minecraft-neo.ru mitrasound.ru
misandesign.se mnogabukaff.net
missclub.info monarhs.info
mmoguider.ru moncleroutletonline.xyz
mmostrike.ru monsterdivx.tv
mobile-appster.ru moroccosurfadventures.com
mobplayer.ru most.gov.iq
modenamebel.ru mouselink.co
moinozhki.com mp3s.club
monclercheap.xyz
monclercoats.xyz
monclerjacketsoutlet.win
moscow.online-podarki.com
mototsikl.org
moviemail-online.co.uk
mp3ringtone.info
mrsdalloways.com
multgo.ru
mundoaberrante.com mundoaberrante.com
musflashtv.com
mustwineblog.com mustwineblog.com
muycerdas.xxx muznachas-service.ru
my-floor.in.ua mykings.pw
mydirtyhobby.com mylovelibrabry.com
mydownlodablefiles.com myonigroup.com
nailsimg.com myxdate.info
narutonaruto.ru naj-filmy24.pl
naval.jislaaik.com najaden.no
netpics.org naphukete.ru
nfljerseys.online naturalshair.site
nfvsz.com nerudlogistik.ru
nippon-bearings.ru netanalytics.xyz
nootrino.com newfilmsonline.ru
novosti-hi-tech.ru newstraveller.ru
o.light.d0t.ru nisuturnetdgu.tk
obesidadealgarve.com nlfjjunb5.ru
obiavo.net nmrk.ru
oklogistic.ru no-rx.info
okroshki.ru nodup.ru
olvanto.ru nonameread45.live
oneclickfiles.com noumeda.com
online-hd.pl novgorod.xrus.org
online-templatestore.com novodigs.com
onlinefilmz.net nrjmobile.fr
onlineku.com o-o-8-o-o.com
palvira.com.ua o-o-8-o-o.ru
pammik.ru obiavo.kz
partner-trustworthy.men oksrv.com
partnersafe.men omgtnc.com
perederni.net onemactrckr.com
photorepair.ru online-hit.info
picanalyzer.data-ox.com onlinemegax.com
picturesmania.com oralsexfilme.net
picurams.pw owohho.com
pizdeishn.com oynat.info
pl-vouchers.com p-business.ru
poem-paying.gq pacificair.com
poisk-zakona.ru panchro.xyz
poiskzakona.ru paphoselectricianandplumber.com
popunder.net parajumpersoutlet.online
pornokorol.com patol01.pw
posible.net patterntrader-en.com
pospr.waw.pl paydayloanslocal.com
potoideas.us paydayonlinecom.com
pozdrawleniya.ru pdfprof.com
pravoholding.ru pdns.cz
princeadvantagesales.com pelfind.me
privatefx-in.ru perfectplanned.com
privatov-zapisi.ru petrovka-online.com
prlog.ru picphotos.net
pro-okis.ru pihl.se
prom23.ru pinup-casino1.ru
promodj.com pisanieprac.info
pst2017.onlinewebshop.net placid-rounded-coast.glitch.me
pulseonclick.com play.leadzupc.com
pussyspace.net popunder.ru
q-moto.ru pornhub-forum.ga
qualitymarketzone.com pornoelita.info
pornoinn.com
pornonik.com
pornoted.com
pr-ten.de
privhosting.com
prostitutki-odessa.org
prostitutki-tyumeni.org
prpops.com
ptr.ruvds.com
purplesphere.in
puserving.com
pushdata.sendpulse.com
puzzleweb.ru
quickcashlimited.com quickcashlimited.com
quizzitch.net quickchange.cc
randki-sex.com rarbg.to
rangjued.com raschtextil.com.ua
ranksignals.com
razvratnoe.org
rcb101.ru rcb101.ru
regionshop.biz redirlock.com
registry-clean-up.net reeyanaturopathy.com
remont-mobile-phones.ru refads.pro
remybutler.fr reimageplus.com
reward-survey.net reklamuss.ru
rockingclicks.com remontvsamare.su
rubanners.com rennlist.com
rus-teh.narod.ru rentalcarnavi.info
rusvideos.su rentehno.ru
sad-torg.com.ua rniaeba.ga
sammyweaver.com rockma.se
saugatuck.com rucrypt.com
sbf441.com rumamba.com
sbornik-zakonov.ru samolet.fr
scanner-julia.top sashagreyblog.ga
scopich.com savefrom.com
sdsjweb.com savingsslider-a.akamaihd.net
search-goo.com sawin.beth.webd.pl
secret.xn--oogle-wmc.com scanner-brian.top
see-your-website-here.com scanner-donald.top
serialsway.ucoz.ru scanner-jessica.top
sexfreepornoxxx.com scanner-josh.top
sexpornotales.net schalke04fc.info
sextracker.de sdrescher.net
ship-marvel.co.ua searchtooknow-a.akamaihd.net
shoppingjequiti.com.br sell-fb-group-here.com
shtaketniki.ru seoanalyses.com
simul.co serptehnika.ru
service.adtech.us
sex-foto.pw
sex-sex-sex5.com
sexsaoy.com
sexsearch.com
sexy-pings.com
sharebutton.net
shareyards.com
shillyourcoins.com
shop.xz618.com
shopcheermakeup.info
sierraapps.com
sim-service.net
sirpornogratis.xxx
sispe.com.br
site-speed-check.site site-speed-check.site
sitopreferito.it sitevalued.com
skachat-besplatno-obrazcy.ru skincrate.net
skatestick.bid sky-mine.ru
snworks.com slavia.info
social-s-iii.xyz slimcdn.com
soietvousmaime.fr sluganarodu.ru
solicita.info smallseotools.com
sovinsteel.ru socialmadesimple.com
spb.afora.ru sofit-dmd.ru
sports-supplements.us soheavyblog.com
spy-app.info souvenir.cc
stairliftsarea.com sovetogorod.ru
stard.shop spb-plitka.ru
stariy-baku.com speedup-my.site
statistici.ro spravka-medosmotr.ru
steamoff.net stair.registrydomainservices.com
steebook.com start.myplaycity.com
sterva.cc students-cheapskate.ml
streetfooduncovered.com subj.ukr-lit.com
stromerrealty.com sucsesofinspiration.com
stroy-portal22.ru
studworks.org
sugvant.ru
superiends.org
superlist.biz superlist.biz
swinger-mobil.net suralink.com
telsis.com svolze.com
terrafootwear.us tabletkinaodchudzanie.com.pl
the-trader.net tamada69.com
theheroes.ru tandvardshuset.net
titslove.yopoint.in technopellet.gr
tizanidineoral.blogspot.com teesdaleflyballclub.co.uk
tizanidinevscyclobenzaprine.blogspot.com template-kid.com
tomatis.gospartner.com templates.radiodigital.co
topcasinoratings.ru thelottosecrets.com
torrents-tracker.com themeforest.net
trackmedia101.com ticketsys.inetwd.com
tsstcorpcddvdwshbbdriverfb.aircus.com timeallnews.ru
tula.howotorg.ru tizanidineinteractions.blogspot.com
tupper-posuda.ru tizanidinepain.blogspot.com
turbodsp.com tizanidinerecreationaluse.blogspot.com
twitlinks.com top-study.work
txxx.com top250movies.ru
tyumen.xrus.org topmira.com
ustion.ru topsiteminecraft.com
vapomnoncri.tk torrnada.ru
velen.io torture.ml
vepad.com trahvid.com
vesnatehno.ru trkur.com
via-energy-acquistare.com tsyndicate.com
viagraneggrx.com tube8.com
videochat.ph tuberkulezanet.ru
turkeyreport.tk
uhod-za-sobakoj.ru
uni.me
upstore.me
urccvfmc.bloger.index.hr
urdoot.win
valmetrundan.se
vandrie-ict.nl
vedomstvo.net
videochat.bz
videochat.world videochat.world
vinylvault.co.uk videogamesecrets.com
vip2ch.com videos-for-your-business.com
visa-pasport.ru vids18.site
vksaver-all.ru vielporno.net
vostoktrade.info vikistars.com
vozbujdenie.com vilingstore.net
vremya.eu vip-file.com
vuclip.com vipsiterip.org
wcb.su virtualbb.com
wdfdocando.com visa-china.ru
webalan.ru vrnelectro.ru
webcamtalk.net vsdshnik.com
webmonetizer.net vucms.com
wedding-salon.net vzlomfb.com
weprik.ru w3javascript.com
whatistizanidine2mg.blogspot.com wallpaperaccess.com
whatistizanidinehclusedfor.blogspot.com watchmygf.net
wholesalejerseychinashop.com webcamdevochka.com
wingsoffury2.com webs.com
winner7777.net wechatdownload10.com
wormix-cheats.ru westermarkanjou.se
www.ehscloud.cn whos.amung.us
www.solartek.ru wordpresscore.com
xboxster.ru world-mmo.com
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai worldis.me
xn----ctbigni3aj4h.xn--p1ai wowcasinoonline.ooo
www.souvenirua.com
x-stars.ru
xblognetwork.com
xjlottery.com
xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xn----btbdvdh4aafrfciljm6k.xn--p1ai
xn--1-8sbcpb0bdm8k6a.xn--p1ai xn--1-8sbcpb0bdm8k6a.xn--p1ai
xn--80atua3d.xn--p1ai xn--h1aakne2ba.xn--p1ai
xn--c1acygb.xn--p1ai xn--l1aengat.xn--p1ai
xsfetish.org xn--lifehacer-1rb.com
xtubeporno.net xnxx-n.com
xxx-cam.webcam xnxxandxvideos.com
xzlive.com yopoint.in
yachts-cruise.info yourmovies.pl
yogamatsexpert.com youtoner.it
yougetsignal.com ypmuseum.ru
youjizz.vc zagadki.in.ua
youporn-forum.ga
youresponsive.com
yousense.info
youtube-downloader.savetubevideo.com
zakonobosago.ru
zapatosenventa.info
zawyna.ua
zdesoboi.com
zmoda.hostreo.com
zolotoy-lis.ru
zone-kev717.info
zoominfo.com zoominfo.com
zootoplist.com zygophyceous.womanstars.site
zx6.ru

480
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
20pascals.nl 007angels.com
2ads.co.uk 1000n1.ru
adbetclickin.pink 4kplayer.pl
adobereader-free.ru 7minuteworkout.com
aibolita.com 83net.jp
albamargroup.com abcguru.xyz
academia-nsk.org
academiacsmendoza.org
acmebtn.ml
ads.gold
affordablewebsitesandmobileapps.com
alarmobninsk.ru
albuteroli.com albuteroli.com
amigobulls.com all4invest.info
anlimebel.kiev.ua allbizne.co.ua
anniemation.com allstatesugarbowl.org
anticrawler.org allwidewallpapers.com
alphahoverboards.com
amateurmatch.com
amospalla.es
androids-store.com
angigreene.com
apartmentbay.ru apartmentbay.ru
apessay.com apiallgeniusinfo-a.akamaihd.net
apiadanaknet-a.akamaihd.net apiappenableinfo-a.akamaihd.net
apiqualitinknet-a.akamaihd.net appartement-stumm.at
arabseyes.com arpelsreplica.xyz
asdfg.pro ayeartoforget.com
asrvvv-a.akamaihd.net azbukafree.com
asscenihotosidea.blogspot.co.za bad-stars.net
asseenontv.ru baersaratov.ru
atout-energie-69.com basisches-wasser.net
azlex.uz bazaronline24.ru
bag77.ru
barbourjackets.online
bashtime.ru
beauty-clinic.ru
best-mam.ru best-mam.ru
bingoporno.com bezsporno.ru
bioca.org bidbuy.co.kr
biznesschat.net bizru.info
bjgugu.net.ua bloggedporn.com
bloglag.com bmsco.co
bloke.com boobsimge.com
bolitgorlo.net brains2.biz
bookhome.info brewdom.ru
bradleylive.xyz brk-rti.ru
brainboosting.club buybest1.biz
braindaily.xyz buyessayonline19.blogspot.ru
brandimensions.com buyhoverboard.com
bumble.cheapwebsitehoster.com cakemediahq.com.au
businesxxl.com cartierreplica.pw
buytizanidineonlinenoprescription.blogspot.com ccjp.eu
bycontext.com cdnnetwok.xyz
c-gainsbourg.com ceotrk.com
c1.onedmp.com chcu.net
canyougethighofftizanidine.blogspot.com cialischsrx.com
cardsdumps.com cobaltpro.ru
cartierlovebracelet.xyz coldfilm.ru
cashkitten-a.akamaihd.net compliance-fred.top
cete.ru continent-e.tv
cherrypointplace.ca cookie-law-enforcement-hh.xyz
choosecuisine.com cordstrap.cc
citysecurity.nu creams.makeforum.eu
compliance-fred.xyz customcatchcan.com
compliance-jessica.top davidovic.info
concordexoticrentals.com
cookie-law-enforcement-ee.xyz
creditmoney.com.ua
crest3d.ru
cutt.us
cvety24.by
dalavia.ru
decoratinghomes.ga decoratinghomes.ga
deshevo-nedorogo.ru delta-line.men
digitalassetmanagement.site downloadmefiranaratb1972.xpg.com.br
donna7753191.ru ec-file.info
dostavka-v-ukrainu.ru elektrischezigaretten1.blogse.nl
downvids.net elektrozigarette2.wordpressy.pl
dyshagi.ru empirepoker.com
e2click.com erotikstories.ru
elektronischezi.livejournal.com errorfixing.space
erotag.com essay-writing.work
erotik0049.com europeanwatches.ru
estelight.ru fanoboi.com
f012.de fapgon.com
facecup.top feeriaclub.ru
fitfloponline.store fertilitetsradgivningen.se
foodcrafts.website fidelityfunding.com
fortevidyoze.net fighrofacciufreesig.ga
freeuploader.com flooringinstallation-edmonton.com
fuckingawesome.com floridamobilebillboards.com
fun2cell.net folowsite.com
get-free-traffic-now.com forum20.smailik.org
getridofstretchmarks.org foto-basa.com
ghazel.ru foto-weinberger.at
girlgamerdaily.com frameimage.org
goldadpremium.com frbizlist.com
googlefeud.com frcls.fr
gox.com.ua freelinkbuilding.website.tk
gratuitxblcodes.com fyl.com.ru
gribokstop.com fyxabomiw.ru
grupografico-pilar.com.ar galeon.com
hatdc.org gay-file.com
havepussy.com genericlowlatencyasiodriverhq.aircus.com
hidefiles.org given2.com
himazin.info gopixdatabase.com
hobbyhuren-datenbank.com graphics8.info
hobbyhuren24.net gribkovye-zabolevaniya.com
hotxnights.info gwagka.com
ikearugs.xyz gyffu.com
ilbe.club h2monline.com
img.wallpaperstock.net hasshe.com
instabid.tech hmywwogw.bloger.index.hr
hspline.com
ilo134uloh.com
ilovevitaly.com
import-sales.com
infokonkurs.ru
intellektmedia.at intellektmedia.at
isitwp.com iscblog.info
istmira.ru islamtoday.co.za
ivanovo.zrus.org javstock.com
ivearchenceinflu.cf justprofit.xyz
izismile.com jwss.cc
jabimgo.pw jyrxd.com
jagg.info kalandranis.gr
javrip.net kam-dom.ru
justdating.online kerei.ru
justkillingti.me kevblog.top
kaac.ru kidskunst.info
kabbalah-red-bracelets.com ktotut.net
kazinogames.lv laexotic.com
kchaxton.com
kino-key.info
kinobest.pl
kiwe-analytics.com
knowyournextmove.com
koronirealestate.gr
languagecode.com
lashstudia.ru
lechenie-gemorroya.com lechenie-gemorroya.com
lightinghomes.net llastbuy.ru
linkrr.com lombia.co
lion.misba.us lrac.info
lkbennettoutlet.online malls.com
luxmagazine.cf malwareremovalcenter.com
madot.onlinewebshop.net marketingvici.com
mainlinehobby.net
manve.info
masterseek.com masterseek.com
maxthon.com matsdale.com
mbiologi.ru mielec.pl
mikozstop.com mobile.ok.ru
minet.club moinozhki.com
mmoguider.ru money-for-placing-articles.com
monthlywinners.com moscow-clining.ru
mossmesi.com mrlmedia.net
musflashtv.com mxgetcode.com
musicspire.online myiptest.com
mybackgroundlandscape.blogspot.com myra.top
narutonaruto.ru nastydollars.com
net-radar.com naturalbreakthroughsresearch.com
njkmznnb.ru novodigs.com
notaria-desalas.com nufaq.com
notfastfood.ru obiavo.su
nyfinance.ml openfrost.net
o-o-8-o-o.com openmultipleurl.com
obiavo.com orion-v.com
one-gear.com ossmalta.com
online.ktc45.ru palocco.it
online247.ml palvira.com.ua
onlineserialy.ru pepperstyle.ru
openfrost.com pila.pl
pavlodar.xkaz.org pl.id-forex.com
picsfair.com popander.mobi
porno-chaman.info pornogad.com
pornofiljmi.com
pornohd1080.online
pornosemki.info
pornosexrolik.com pornosexrolik.com
postclass.com pornoted.com
pozdrawleniya.ru priceg.com
pricheski-video.com pukaporn.com
prlog.ru push-ad.com
produkto.net ranksonic.com
pron.pro rating-bestcasino.com
propranolol40mg.blogspot.com razleton.com
ptr.ruvds.com razvratnoe.org
qcstrtvt.bloger.index.hr razyboard.com
real-time-analytics.com recipedays.com
realmonte.net relax.ru
realting-moscow.ru resant.ru
rep-am.com rethinkwasteni.info
restaurantlescampi.com rn-to-bsn.com
ringtonepartner.com roma-kukareku.livejournal.com
robertefuller.com rumamba.com
rp9.ru saletool.ru
ruex.org.ua sawin.beth.webd.pl
rus-teh.narod.ru sbt-aqua.ru
russkoe-zdorovie.ru scmor.ilxc.cc
rvtv.ru scrapinghub.com
scanmarine.info se-welding.ru
scanner-marwin.top secret.xn--oogle-wmc.com
semalt.com seeingmeerkat.com
senger.atspace.co.uk sei80.com
sibdevice.ru sell-fb-group-here.com
sitevaluation.com semrush.com
sitevaluation.org seoheap.com
skladvaz.ru sergiorossistore.online
smstraf.ru shariki-zuma-lines.ru
shopfishing.com.ua
signoredom.com
sitebeam.net
skutecznetabletkinaporostwlosow.pl
skypasss.com
slayerlife.com
slomm.ru
smadihome.com
snjatie-geroinovoy-lomki.ru snjatie-geroinovoy-lomki.ru
soundfrost.org social-s-iii.xyz
spin2016.cf socialsignals24.com
spinnerco.ca softlinesolutions.me
sps-shop.com softxaker.ru
sptslmtrafms.com sonnikforme.ru
statustroll.com spasswelt.net
storehouse.ua spb-plitka.ru
strana-krasoty.ru stauga.altervista.org
stroilka.info svtrd.com
students-cheapskate.ml syndicate.fun
sukirgenk.dvrlists.com tabakur77.com
svarkagid.com taraz.xkaz.org
taqplayer.info technika-remont.ru
tastyfoodideas.com telefonsex-ohne0900.net
teenfuck.tv templates.franklinfire.co
thecounter.com tgtclick.com
thedownloadfromwarez.blogspot.com titan-ads.life
thefds.net tizanidinecost.blogspot.com
tivolibasket.it topquality.cf
tmearegion26.com
toon-families.com
tourismvictoria.com
trichizobswiv.agddns.net
trk-4.net trk-4.net
tx41tclega.ru trubywriting.com
typimga.pw tv-spoty.info
ultimatesetnewfreeallsoftupgradesystems.pw tvand.ru
ultramart.biz unrealcommander.com
unece.org uogonline.com
uvozdeckych.info valmetrundan.se
vapeface.club vapomnoncri.tk
vclicks.net varikozdok.ru
velen.io vashsvet.com
velpanex.ru vchulkax.com
videochat.ph video-chat.in
vipsiterip.org videochaty.ru
womama.ru videositename.com
worldhistory.biz virtualbb.com
worldinternetauthority.com viveresaniesnelli.it
woweb.com.ua vk-mus.ru
x-diesel.org vkmusics.ru
xclicks.net voloo.ru
xep.info voprosotvet24.ru
xerox-douglas.cf warezaccess.com
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai wfb.hatedriveapart.com
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai wildworld.site
xvideosporn.biz woman-tampon.ru
y8games-free.com wpsecurity.website
youdao.com wurr.voila.net
zaapplesales.blogspot.com xn----7sbho2agebbhlivy.xn--p1ai
zaidia.xhost.ro xn--80ahdheogk5l.xn--p1ai
zeroredirect1.com xn--80ak6aa92e.com
zoogdisany.com yoshkarola.zrus.org
zzbroya.com.ua youghbould.wordpress.com
yugo-star.ru
zakon-ob-obrazovanii.ru
zigarettenonlinekaufen1.blox.pl
zoodrawings.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.29 0; 10.1.0.39 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3385 ### Version: V4.2022.10.3386
### Updated: Sun Oct 2 10:03:32 UTC 2022 ### Updated: Sun Oct 2 22:01:10 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3385 ### Version: V4.2022.10.3386
### Updated: Sun Oct 2 10:03:32 UTC 2022 ### Updated: Sun Oct 2 22:01:10 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##