vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-06 12:40:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.01.3560 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-01-28 22:07:08 +00:00
parent 89f55f4135
commit ea2761d643
47 changed files with 100863 additions and 100873 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3559 #### Version: V4.2023.01.3560
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3559 #### Version: V4.2023.01.3560
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.01.3559 #### Version: V4.2023.01.3560
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12472
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.01.3559 #### Version: V4.2023.01.3560
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12476
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12476
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12476
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12476
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 1; 10.1.0.82 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 0; 10.1.0.82 0;
127.0.0.1 0; 127.0.0.1 0;

12476
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 1; 10.1.0.82 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 0; 10.1.0.82 0;
127.0.0.1 0; 127.0.0.1 0;

12476
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 1; 10.1.0.82 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 0; 10.1.0.82 0;
127.0.0.1 0; 127.0.0.1 0;

12476
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3559 3560

12476
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12476
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.176.205.113 0; 13.91.160.240 0;

12476
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 1; 10.1.0.82 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 0; 10.1.0.82 0;
127.0.0.1 0; 127.0.0.1 0;

12476
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 1; 10.1.0.82 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 0; 10.1.0.82 0;
127.0.0.1 0; 127.0.0.1 0;

12476
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 1; 10.1.0.82 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 0; 10.1.0.82 0;
127.0.0.1 0; 127.0.0.1 0;

12476
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.176.205.113 0; 13.91.160.240 0;

12476
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.176.205.113 0; 13.91.160.240 0;

12476
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 1; 10.1.0.82 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Aspiegel 404enemy
Asterias AlphaBot
InfoTekies Brandprotect
Rankivabot BunnySlippers
Siteimprove Ezooms
Spbot Lanshanbot
Sucuri Ltx71
TwengaBot Nutch
WebsiteExtractor Recorder
sysscan Wotbox

298
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
ADmantX
ASPSeek
Aboundex Aboundex
Aboundexbot Aboundexbot
Acunetix
AhrefsBot
AiHitBot
Aipbot
Alexibot Alexibot
Alligator AllSubmitter
AlphaBot AlphaBot
Anarchie Aspiegel
Anarchy Asterias
Anarchy99 AwarioRssBot
Anthill
Atomseobot
AwarioSmartBot AwarioSmartBot
BackStreet BBBike
BackWeb BDCbot
Backlink-Ceck BDFetch
BacklinkCrawler BLEXBot
BetaBot Bitacle
BlowFish Boardreader
Bolt Buck
Brandwatch BuiltWith
Buddy BuzzSumo
BuiltBotTough
CATExplorador CATExplorador
CCBot
CSHttp
CheeseBot CheeseBot
CherryPicker ChinaClaw
Chlooe Claritybot
Cliqzbot Cliqzbot
Cogentbot Cocolyzebot
Collector
Copier
Cosmos Cosmos
Craftbot
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Crescent Crescent
CrunchBot
Curious Curious
DIIbot DIIbot
DSearch
DTS Agent DTS Agent
Devil DatabaseDriverMysqli
Disco Digincore
DnyzBot Dirbuster
Discobot
Discoverybot
Dispatch
DomainAppender
Domains Project
Dotbot Dotbot
Download Wonder
Drip Drip
ECCP/1.0
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
Ecxi
EirGrabber
EroCrawler EroCrawler
Exabot
ExtLinksBot
ExtractorPro
EyeNetIE
Ezooms Ezooms
FemtosearchBot
Fimap
Flunky Flunky
Freeuploader Fuzz
FyberSpider FyberSpider
Fyrebot Fyrebot
GermCrawler GT::WWW
GetRight GalaxyBot
GetWeb Go-Ahead-Got-It
Gigabot GoZilla
Go!Zilla Gotit
GrabNet Grabber
Grafula GrapeFX
GrapeshotCrawler HTMLparser
HMView
HTTP::Lite HTTP::Lite
Haansoft Havij
HaosouSpider IDBot
IRLbot
Iblog
Id-search
Image Fetch Image Fetch
Image Sucker IndeedBot
InfoTekies Indy Library
InterGET Intelliseek
Internet Ninja
Iria Iria
JOC Web Spider IstellaBot
Jbrofuzz Jbrofuzz
JennyBot
JikeSpider JikeSpider
Joomla Joomla
Kenjin Spider Leap
LWP::Simple
LeechFTP LeechFTP
Lftp LexiBot
Libwhisker
LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkWalker Likse
Linkbot Linkbot
LinkextractorPro
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MIDown tool
MQQBrowser MSIECrawler
MSFrontPage MTRobot
Mag-Net
Majestic-SEO
MarkMonitor
MarkWatch MarkWatch
Masscan Mata Hari
MauiBot MauiBot
Mb2345Browser Mediatoolkitbot
MeanPath Bot
MegaIndex.ru
Metauri Metauri
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Minefield Minefield
Mister PiX Mojeek
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila
Msrabot
NICErsPRO NICErsPRO
NPbot Name Intelligence
Nameprotect
Needle
Nessus
Net Vampire Net Vampire
NetMechanic
NetSpider NetSpider
NetZIP NetZIP
Netcraft
Netvibes
Nibbler Nibbler
Nikto Niki-bot
Octopus Nuclei
OnCrawl Offline Navigator
OpenVAS Openfind
OrangeBot OrangeBot
OutfoxBot OutclicksBot
PHPCrawl
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageGrabber PageAnalyzer
PageThing.com PageThing.com
Panscient
PeoplePal
Petalbot Petalbot
Picsearch Picscout
PictureFinder PictureFinder
PleaseCrawl Pimonster
Probethenet Pixray
Pu_iN
PxBroker PxBroker
PyCurl PyCurl
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot
RankActiveLinkBot
RankingBot RankingBot
RankurBot Rankivabot
RedesScrapy Re-re
RocketCrawler Reaper
Rogerbot RepoMonkey
Ripper
SBIder SBIder
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler
SEOstats
SISTRIX SISTRIX
SalesIntelligent ScanAlert
Scanbot Scanbot
ScoutJet Screaming
Scrapy
SearchmetricsBot SearchmetricsBot
Seekport
Semrush Semrush
SeoSiteCheckup SemrushBot
SentiBot
Siphon
Site Sucker Site Sucker
SiteExplorer
SiteLockSpider
SiteSucker SiteSucker
Sitebeam
Siteimprove
Snake Snake
Snapbot Snapbot
SocialRankIOBot
Sogou web spider
Sottopop Sottopop
Spammen
Spanner Spanner
Spinn3r Spbot
SputnikBot
Sqlworm Sqlworm
Stripper Sucker
Sucuri Sucuri
SuperBot SuperHTTP
Surfbot Surfbot
Suzuran
Swiftbot
Szukacz Szukacz
T0PHackTeam The Intraformant
T8Abot
Telesphoreo
Telesphorep
TheNomad TheNomad
Thumbor
Titan
Toata
Trendictionbot Trendictionbot
True_Robot True_Robot
Turingos TwengaBot
TurnitinBot Typhoeus
Twice URLy.Warning
URLy Warning
Upflow Upflow
V-BOT
VB Project VB Project
VCI
Vacuum
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
Virusdie Virusdie
Voil Voil
WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV WWW-Collector-E
WISENutbot WWW::Mechanize
WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD WallpapersHD
WebAuto Web Fuck
Web Sauger
Web Sucker
WebFetch
WebFuck WebFuck
WebGo IS WebGo IS
WebImageCollector Webalta
WebLeacher
WebStripper
WebWhacker
WebmasterWorldForumBot
Webshag
Website Quester Website Quester
WebsiteQuester WebsiteExtractor
Webster Whack
Whacker Whacker
Whatweb Whatweb
Widow Who.is Bot
WinHTTrack Wonderbot
Xaldon_WebSpider Woobot
Zade Wotbox
Zermelo Xaldon WebSpider
ZoomBot Xenu
archive.org_bot Zitebot
ZoominfoBot
adscanner
arquivo.pt
autoemailspider autoemailspider
cah.io.community cah.io.community
check1.exe
cognitiveseo
com.plumanalytics com.plumanalytics
crawl.sogou.com
crawler.feedback crawler.feedback
dataforseo.com demandbase-bot
evc-batch evc-batch
heritrix gopher
internetVista monitor internetVista monitor
ips-agent ips-agent
isitwp.com isitwp.com
lwp-trivial lwp-trivial
magpie-crawler mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler ripz
oBot satoristudio.net
plumanalytics sitechecker.pro
scalaj-http
seocompany.store
seostar
sexsearcher
siteripz
sogouspider sogouspider
sysscan sp_auditbot
tAkeOut tAkeOut
trendiction.com
webmeup-crawler
webpros.com webpros.com
webprosbot
x22Mozilla
xpymep1.exe
zgrab

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker
80legs 80legs
ADmantX ADmantX
ASPSeek
Abonti
Aboundex Aboundex
Acunetix Acunetix
AiHitBot AfD-Verbotsverfahren
Aipbot AhrefsBot
Alexibot AllSubmitter
Alligator Ankit
Anarchy
Anarchy99
Apexoo Apexoo
Aspiegel
Asterias Asterias
BBBike Atomseobot
BDCbot AwarioRssBot
BackStreet Backlink-Ceck
Badass Badass
Barkrowler
BatchFTP BatchFTP
Bitacle Bitacle
BlackWidow
Blackboard
Blow Blow
BlowFish Bolt
BotALot
Brandprotect Brandprotect
Brandwatch Brandwatch
Buck Buck
Buddy
BuiltBotTough BuiltBotTough
BuiltWith BuzzSumo
CCBot CATExplorador
CODE87 CazoodleBot
Cegbfeieh CensysInspect
CheTeam CheTeam
CheeseBot
CherryPicker
ChinaClaw ChinaClaw
Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Collector Collector
CrazyWebCrawler Copier
Custo Craftbot
CyotekWebCopy Crescent
DTS Agent CrunchBot
DatabaseDriverMysqli DBLBot
Demon DIIbot
Deusu Deusu
Devil Digincore
Dirbuster Disco
DnyzBot Discobot
DomainCrawler Discoverybot
DomCopBot
DomainAppender
DomainStatsBot
Domains Project Domains Project
Dotbot
Download Wonder
Dragonfly
Drip
ECCP/1.0
EMail Wolf EMail Wolf
EasyDL
Ebingbong Ebingbong
Ecxi Ecxi
EirGrabber EirGrabber
Exabot Evil
Express WebPictures
ExtLinksBot ExtLinksBot
Extractor Extractor
ExtractorPro ExtractorPro
Ezooms Ezooms
FHscan FDM
Flunky FemtosearchBot
Freeuploader Foobot
Genieo Fuzz
FyberSpider
GetRight
Getintent
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla
Gotit Gotit
GrabNet Grabber
Grafula
GrapeshotCrawler GrapeshotCrawler
HTMLparser GridBot
HaosouSpider HTTrack
HonoluluBot Haansoft
HybridBot Havij
Iblog Heritrix
Id-search IDBTE4M
IlseBot IndeedBot
InfoTekies InfoTekies
Intelliseek Intelliseek
InterGET
Internet Ninja
Iria Iria
JamesBOT Jbrofuzz
JikeSpider JennyBot
Joomla JetCar
Kenjin Spider Jetty
Kozmosbot Jorgee
LNSpiderguy LNSpiderguy
Larbin Leap
LeechFTP LibWeb
LeechGet
LexiBot
Libwhisker Libwhisker
Lightspeedsystems
Likse
LinkScan LinkScan
Linkbot LinkWalker
LinkextractorPro LinksManager
LinkpadBot Lipperhey
LinqiaRSSBot
Lipperhey Spider Lipperhey Spider
MIDown tool MIDown tool
MIIxpc
MJ12bot MJ12bot
MQQBrowser MQQBrowser
MSFrontPage
MTRobot MTRobot
Mag-Net Mail.RU_Bot
Majestic SEO
Majestic-SEO Majestic-SEO
Majestic12
MarkWatch
Mass Downloader Mass Downloader
MauiBot MauiBot
MicroMessenger Mb2345Browser
MolokaiBot Microsoft URL Control
Mozlila Minefield
Mr.4x3 Mojeek
Musobot Mojolicious
NPbot NICErsPRO
Navroad Nameprotect
NearSite NearSite
Needle
NetAnts
NetMechanic
NetZIP NetZIP
Netcraft
Nettrack Nettrack
Netvibes Netvibes
Nikto NextGenSearchBot
Nimbostratus
Ninja
Nmap Nmap
Offline Explorer
Offline Navigator Offline Navigator
OnCrawl
OpenVAS
Openfind Openfind
Openvas
OutfoxBot OutfoxBot
Page Analyzer POE-Component-Client-HTTP
PageThing.com PageThing.com
Pandalytics Panscient
Petalbot Papa Foto
Picscout Picscout
Piepmatz Pockey
Pimonster
Probethenet Probethenet
PxBroker Pu_iN
PyCurl Pump
Quick-Crawler QueryN Metasearch
RSSingBot RankFlex
RankurBot RankingBot
ReGet Re-re
RebelMouse Reaper
Recorder Recorder
RedesScrapy RedesScrapy
Rogerbot
SBIder SBIder
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler
SMTBot SMTBot
ScanAlert ScoutJet
ScrepyBot
SemanticJuice SemanticJuice
SemrushBot
SentiBot SentiBot
SeoSiteCheckup
SeobilityBot
Seomoz
Shodan Shodan
Siphon
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider
SiteSnagger
SiteSucker
Sitevigil Sitevigil
SlySearch Sociscraper
Snake
Snapbot
Sosospider
Sottopop Sottopop
Spammen SpaceBison
Spbot
Spinn3r Spinn3r
Steeler SputnikBot
Sqlmap
Sqworm
Sucker Sucker
Surfbot SuperBot
Swiftbot Suzuran
T0PHackTeam Teleport
T8Abot
Telesoft Telesoft
Telesphorep Telesphoreo
Thumbor
TightTwatBot TightTwatBot
Titan Turingos
Toata Typhoeus
Toweyabot URLy Warning
Trendictionbot Vagabondo
Turnitin Virusdie
URLy.Warning Voltron
UnisterBot WPScan
Upflow
V-BOT
Vacuum
VidibleScraper
WBSearchBot
WISENutbot
WWW-Collector-E WWW-Collector-E
WWWOFFLE WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD
WeSEE WeSEE
Web Auto Web Enhancer
Web Fuck Web Sucker
Web Sauger WebAuto
WebBandit WebBandit
WebEnhancer
WebLeacher WebLeacher
WebPix
WebSauger WebSauger
WebStripper
WebSucker
WebWhacker WebWhacker
WebZIP WebZIP
Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Webshag
WebsiteExtractor WebsiteQuester
Whack Whatweb
Who.is Bot Who.is Bot
Wotbox Widow
Xaldon WebSpider WinHTTrack
Xaldon_WebSpider Wonderbot
Zade
Zauba
Zeus
Zitebot Zitebot
ZmEu
ZoomBot
ZoominfoBot
ZumBot ZumBot
adscanner archive.org_bot
check1.exe cognitiveseo
clark-crawler com.plumanalytics
coccocbot crawler.feedback
crawler4j
dataforseo.com
demandbase-bot
eCatch
evc-batch
facebookscraper
heritrix heritrix
instabid instabid
iubenda-radar
lwp-request lwp-request
meanpathbot
muhstik-scan muhstik-scan
netEstate NE Crawler
oBot
pcBrowser pcBrowser
plumanalytics plumanalytics
polaris version
probe-image-size probe-image-size
ripz s1z.ru
satoristudio.net
scalaj-http scalaj-http
seocompany.store seocompany.store
seostar seoscanners
sexsearcher
sitechecker.pro sitechecker.pro
sysscan siteripz
sogouspider
sp_auditbot
trendiction.com trendiction.com
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webgains-bot webgains-bot
xpymep1.exe webmeup-crawler
x09Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
allmarketsnewdayli.gdn 108shot.com
cdnnetwok.xyz arclk.net
fishtauto.ru bizlist.com.de
jrcigars.com kinogolos.ru
ldrtrack.com mysexpics.ru
metascephe.com naturalshair.site
paceform.com noscrapleftbehind.co
robotixix.com prize44.com
shmetall.com.ua proxyradar.com
tizanidineshowupondrugtest.blogspot.com timdreby.com

944
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
000free.us 0c47f8422d3f.com
007angels.com 1000n1.ru
00it.com 100dollars-seo.com
03e.info 123cha.com
101flag.ru 123movies.love
1688.com
18ps.ru 18ps.ru
1kdailyprofit.me 1millionusd.xyz
5forex.ru 21h2o.com
7search.com 2nt.ru
7wind.ru 5-steps-to-start-business.com
7xc4n.com 8si.ru
8lufu.com 9med.net
aasoldes.fr aasoldes.fr
abiente.ru abbanreddy.com
adelly.bg adanih.com
admitad.com adclickthru.net
adservme.com adservme.com
adssafeprotected.com adtology.com
afora.ru adultfriendfinder.com
agroeconom.kz adultnet.in
allstatesugarbowl.org affrh2015.com
amanda-porn.ga afftrack001.com
amigobulls.com ahhjf.com
apiallgeniusinfo-a.akamaihd.net ahrefs.com
apiportalmorecom-a.akamaihd.net allpdfmags.net
appfixing.space alpinism.ru
aproposde.com amazon-adsystem.com
arenda-avtoprokat-krasnodar.ru amyfoxfitness.com
arraty.altervista.org android4fun.org
atmovs.com apiqualitinknet-a.akamaihd.net
autobudpostach.club approved.su
azartmix.com appsecurityr.com
azartniy-bonus.com arabsexxxtube.com
azinoofficial777.ru arcteryxsale.online
baersaratov.ru aruplighting.com
baikaleminer.com asacopaco.tk
bang-hotties.com asmxsatadriverin.aircus.com
bankcrediti.ru asseenontv.ru
bankmib.ru atley.eu.pn
bedcapdealers.com auspolice.com
beremenyashka.com autoblogger24.info
bestbrainprod.win babs.com.ua
bestbuy.ca bear.gotcher.us
betterscooter.com bestdooz.com
bfz.biz bestofpicture.com
bget.ru bestpornuha.com
biketank.ga bestprofits-there.com
bet-prognoz.com
betune.onlinewebshop.net
bigpictures.club
bioscorp.ru bioscorp.ru
bird1.ru
bizcheapjerseyswholesalechina.com
bkns.vn
blackhatworth.com
blacktwhite.com blacktwhite.com
bloggen.be book-bets.com
bloggerads.net brainboosting.club
blogig.org brianhenry.co
blogs.rediff.com brothers-smaller.ru
blpmovies.com bwlx.prepedu.cn
boeuklad.com byme.se
boleznikogi.com calc-for-credit.ru
bolitgorlo.net callstevens.com
boobsimge.com calstaterealty.us
boostmyppc.com
bosefux.esy.es
bpro1.top
brandov.ru
breget74.ru
bronzeaid-a.akamaihd.net
business-suggested.tk
buyantiviralwp.com
buydissertation.net
buynorxx.com
buyparajumpers.online
buzz-porno.info
bzero1jewelry.net
cakesplus.com.au
call-of-duty.info
candlehandmade.com candlehandmade.com
carfax.com.ua cashkitten-a.akamaihd.net
cartierlovebraceletreplica.xyz cayado.snn.gr
castingbank.ru cbb1smartlist12.click
cdnnetwok.xyz cheapestjerseysonwholesale.com
cheapjerseysfootballshop.com cheapwebsitehoster.com
cheapsergiorossi.online checkhit.com
cleanpcnow.com chelnytruck.ru
chocolateslim-u-srbiji.com
cialgenisrx.com
ciceros.co
cleanallvirus.com
clickintext.net
clickpapa.com clickpapa.com
codysbbq.com
coin-hive.com coin-hive.com
collegerentals.com
compiko.info
compliance-alex.xyz
compliance-alexa.xyz
compliance-barak.top compliance-barak.top
compliance-don.xyz compliance-elena.top
compliance-fred.top compliance-george.xyz
compliance-john.top compliance-ivan.top
conciergegroup.org consorzioilmosaico.org
connectingsingles.com
constantaservice.net
coolyarddecorations.com coolyarddecorations.com
costablue.xyz coop-gamers.ru
cowblog.fr crafthubs.com
creditwell.ru cranly.net
cubs-jerseys.us cvety24.by
cupday.com czeshop.info
customsua.com.ua daneshetabiat.com
dailyrank.net decenttools.com
dalavia.ru decorazilla.com
danashop.ru dentfidemountpreach.tk
danceuniverse.ru deshevo-nedorogo.ru
daptravel.com detmebel.su
decoratinghomes.ga detsky-nabytek.info
dengi-pod-zalog-nedvizhimosti.ru dikx.gdn
diegolopezcastan.com diusyawiga.tk
digest-project.ru djihispano.com
discovertreasurenow.com documentsite.net
disruptingdinnerparties.com downloaddy.net
dnepropetrovsk.mistr-x.org dptaughtme.com
dnsrsearch.com dumuelave.xyz
domini.cat e-collantes.com
dotmass.top e-commerce-seo.com
downloader12.ru
drillsaw.ru
drunkmoms.net
drupa.com
dstroy.su
easyshoppermac.com
easytuningshop.ru easytuningshop.ru
ebonyporn.site ec-file.info
ecookna.com.ua ednorxmed.com
edudocs.net ege-essay.ru
elektronischezigarettekaufen1.myblog.de electronicadirect.com
elite-sex-finders.com elektrischeziga.livejournal.com
elvenar.com elektronischezigarette2.mex.tl
en.home-task.com elmacho.xyz
eonpal.com eluxer.net
eralph.tk energydiet-info.ru
este-line.com.ua eng-lyrics.com
etm-consult.de eropho.net
eupornstar.info errorfixing.space
expdom.com eugenevaultstorage.com
extremal-blog.com excaliburfilms.com
f07.de exchanges-bet.com
face.hostingx.eu exct.net
fashion-mk.net ezrvrentals.com
feel-planet.com f0815.de
fetroshok.ru facebook-mobile.xyz
financetip.eu fanpagerobot.com
findanysex.com favornews.com
fishtauto.ru fickenbumsen.net
fodelsedagspresenter.nu filesclub.net
filmbokep69.com
filmfanatic.com
filmgo.ru
finejewelryshop.ru
fok.nl fok.nl
frcls.fr folowsite.com
free-laptop-reward.com forex-indextop20.ru
free-share-buttons.top forminecrafters.ru
free411games.com forodvd.com
freegamesplay.online forpostlock.ru
fres-news.com free-fb-traffic.com
freza-sverlo.ru free-share-buttons.com
front.to free-stock-illustration.com
fx-brokers-review.com freejabs.com
gagrasector.ru fuckmill.com
galaxy-family.ru fym.com.ru
gambarkatabaru.com g7m.pl
gameprimary.com galleryawesome.com
gamerscorps.com gallictures.com
gamewrath.com game-top.su
gaz-voshod.ru gate5.co.za
gemara.com gateway.zscalerone.net
getridofstretchmarks.org gazoblok.net.ua
getyourimage.club gdcentre.ru
given2.com gdebestkupit.ru
gktt.ru ge0ip.org
go2jump.org geckoandfly.com
googlepositions.com gemgrab-a.akamaihd.net
gotwebsite1.com getlaid-xxxhookupdirect.com
gpirate.com getpopunder.com
grand-chlen.ru gidonline.one
gratuitxblcodes.com girlsfuckdick.com
guruofcasino.com glavtral.ru
gwhwpxbw.bloger.index.hr gne8.com
gymi.name gnuetella.com
hanwei.us gogalleryawesome.com
haveinc.xyz goldenggames.com
goodwriterssales.com
googlefeud.com
graphics8.info
growshop.es
guigyverpo.cf
gwebtools.com
gyffu.com
hacktougroup.ru
hanink.biz.ly
hayate.biz hayate.biz
hdapp1008-a.akamaihd.net hazardky.net
hdimagelib.com hentai-manga.porn
health-medical-portal.info hermesbirkinhandbagoutlets.com
hem.passagen.se high-speed1.net
hentaiheroes.com
highland-homes.com highland-homes.com
hit-kino.ru hmywwogw.bloger.index.hr
hit-men.men hobbyhuren24.net
hopeonthestreet.co.uk homedecoguide.info
hornymatches.com hotel-mkad.ru
hoste.octopis.com hotenergy.ru
hotblognetwork.com hothot.ru
hotcore.info
housediz.com
houston-vikings.com
hoverboard360.de
howmuchdoestizanidinecost.blogspot.com
howtoclean.club howtoclean.club
ifirestarter.ru hoztorg-opt.ru
iideaidekonkatu.info hydropump.su
ilmen.net iblogpress.xyz
iloveitaly.ru ideashome.id
imagefinder.site ido3.com
imagui.eu iflycapetown.co.za
insurple.com ilikevitaly.com
isitwp.com ilo134uloh.com
islandminingsupply.wordpress.com imfamous.info
ispaniya-costa-blanca.ru imperiafilm.ru
ivanovo.zrus.org indetiske.ya.ru
ivoiretechnocom.ci indiakino.net
ix20.ru indiasourcemart.in
izoll.ru internet-apteka.ru
javstock.com invest-pamm.ru
javxxx18.com invivo.hu
jerseychinabizwholesale.com iomoio.net
jerseyschinabizwholesale.us iptool.xyz
jobgirl24.ru irkutsk.online-podarki.com
jofucipiku.tk iron-age.info
junketjuice.blogspot.com istanbulit.com
kamorel.com istripper.com
karachev-city.ru itsdp3.com
karusel-market.ru iwantedmoney.com
ixora.pro
jerseywholesalebizchina.com
kalandranis.gr
kaliningrad.zrus.org
kambasoft.com
ketoanhanoi.info
ketrzyn.pl
keywordhouse.com keywordhouse.com
keywordhut.com kidskunst.info
keywordsuggest.org kinocccp.net
keywordsuggests.com kiwi237au.tk
kinogolos.ru klejonka.info
kol-energo.ru kollesa.ru
konteiner24.com kosmetyki.tm.pl
kozhniebolezni.com kriokomora.info
kvartira-sutochno.com kskjco.club
la-fa.ru kunstaktien.info
lacloop.info legalrc.biz
landinez.co legionalpha.com
laxob.com lesbian.xyz
lida-ru.com lescinq.com
lidoradio.com
lightinghomes.net lightinghomes.net
linkredirect.biz likrot.com
lion.bolegapakistan.com linerdrilling.com
liveinternet.ru lineshops.biz
livingcanarias.com link.ac
loadopia.com littlecity.ch
loftdigital.eu littlesunraiser.com
lollypopgaming.com liupis.com
lombia.co lmrauction.com
longadventure.com locatellicorretor.com.br
lunamedia.co lookingglassemb.com
lunchrock.co lovelycraftyhome.com
m-google.xyz mac-shield.com
m0r0zk0-krava.ru macresource.co.uk
macfix.life madot.onlinewebshop.net
manipulyator-peterburg.ru mafa-free.com
malls.com
mansiondelrio.co
mansparskats.com
map028.com
marktforschung-stuttgart.com marktforschung-stuttgart.com
marmotstore.online mediaoffers.click
marsgatan.com mhtr.be
masthopehomes.com mindbox.co.za
me-ke.com mini.7zap.com
med-zdorovie.com.ua mobot.site
medanestesia.ru mobplayer.ru
meddesk.ru moje-recenze.cz
medictube.ru moneyviking-a.akamaihd.net
megahdporno.net moscow-region.ru
megakino.net mountainstream.ms
megavolt.net.ua musezone.ru
merryhouse.co.uk music.utrolive.ru
metarip.ru myblogregistercm.tk
mex-annushka.ru mydearest.co
minet.club myfreetutorials.com
mini-modus.ru myghillie.info
missis.top myonigroup.com
mmostrike.ru newfilmsonline.ru
mobstarr.com newhairstylesformen2014.com
montredemarque.nl newstaffadsshop.club
moroccosurfadventures.com nextconseil.com
moscow.online-podarki.com nhl17coinsforps3.gratisblog.biz
mosrif.ru nisuturnetdgu.tk
mostantikor.ru njkmznnb.ru
mp3films.ru nomuos.it
mp3s.club nsatc.net
mrvideospornogratis.xxx nuker.com
muabancantho.info odesproperty.com
music7s.me olvanto.ru
musicktab.com onclickpredictiv.com
musirc.com online-x.ru
muz-shoes.ru onlinebay.ru
mycaf.it onlinemeetingnow.com
mydeathspace.com onload.pw
mysexpics.ru onstrapon.purplesphere.in
namecrumilchlet.tk oren-cats.ru
netanalytics.xyz origin-my.ru
netfacet.net osb.se11.ru
nibbler.silktide.com oyster-green.com
no-rx.info p-business.ru
norththeface.store paighambot.com
notfastfood.ru partner-host.men
novgorod.xrus.org partner-pop.men
ntic.fr pc-test.net
nullrefer.com pdfprof.com
oballergiya.ru pearlisland.ru
obiavo.ru pechikamini.ru
obiavo.su
offside2.5v.pl
okout.ru
online-hit.info
online-podarki.com
open-odyssey.org
opinionreelle.com
orhonit.com
outclicks.net
outpersonals.com
pammik.ru
paperwritingservice17.blogspot.ru
par-fallen.ga
parajumpersjakkesalgnorge.info
paramountmarble.co.uk
park.above.com
patol01.pw
pdamods.ru
pdn-4.com
pensplan.com
perederni.net
perfectpracticeweb.com perfectpracticeweb.com
petedrummond.com petrovka-online.com
picturesboss.com pharmacyincity.com
pillscheap24h.com picanalyzer.data-ox.com
pinkduck.ga
piski.top piski.top
pistonclasico.com pk-services.ru
plastweb.ru pobeiranie.pl
poiskzakona.ru pod-muzyku.club
pojdelo.weebly.com poppen-nw.net
polybuild.ru porn-w.org
popcash.net porn555.com
porndl.org pornobrazzers.biz
porno-play.net pornofeuer.com
pornotubexxx.name pornophoto.xyz
powc.r.ca.d.sendibm2.com pornotubs.com
priceg.com pornpost.in
privatefx-in.ru portadd.men
producm.ru powitania.pl
proekt-mos.ru prchecker.info
promalp-universal.ru prefersurvey.net
prosperent.com preparevideosafesystem4unow.site
prostitutki-tyumeni.org promgirldresses.xyz
psa48.ru promover.org
pufip.com proposal-engine.com
qcstrtvt.bloger.index.hr propranolol40mg.blogspot.com
quickbuck.com prosmibank.ru
quit-smoking.ga prostitutki-belgoroda.org
randalljhoward.com pskcijdc.bloger.index.hr
realmonte.net ptr.ruvds.com
recipedays.ru pulse33.ru
rednise.com putevka24.ru
remorcicomerciale.ro py100.ru
rennlist.com qor360.com
reversing.cc quick-seeker.com
rfserial.net rabot.host.sk
ric.info rcb101.ru
ritlweb.com realitykings.com
roznica.com.ua refererx.com
samo-soznanie.ru registrydomainservices.com
saneyes.com riralmolamsaca.tk
sbf441.com rniaeba.ga
sbricur.com rogervivierforsale.com
scanner-fred.top roleforum.ru
scanner-ivan.top roll123.com
scanner-josh.top rrutw.com
screentoolkit.com ru-mediaget.ru
sdi-pme.com runofilms.ru
sdrescher.net rus-teh.narod.ru
search.xtconnect.com rusoft-zone.ru
seoanalyses.com rutor.vip
seoexperimenty.ru ryetaw.com
seofirmreviewsus.info sampleletters.net
sergiorossistore.online sax-sex.com
serviporno.com scripted.com
sex-pr.net scrnet.biz.ua
sex-watch.com see-your-website-here.com
sexkontakteao.info seinterface.com
sexs-foto.com seksvideoonlain.com
semalt.com
seo-2-0.com
seo-traffic-ranking.info
sevendays.com.ua
sex-tracker.com
sexobzor.info
sexs-foto.top
sexy-screen-savers.com
share-buttons-for-free.com
shemale-sex.net
shisha-swag.de
shoesonlinebuy.xyz shoesonlinebuy.xyz
shopfishing.com.ua silvermature.net
shopvilleroyboch.com.ua site-auditor.online
sientalyric.co slimcdn.com
sierraapps.com smartadserver.com
simpoed.ufop.br smokewithrabbits.com
simul.co soaksoak.ru
sinestesia.host.sk social-s-ggg.xyz
site3.free-share-buttons.com social-search.me
sitevaluation.com socialmadesimple.com
sitevaluation.org
slomm.ru
smartpet.ru
sneakyboy.com
snowplanes.com
social-buttons.xyz
social-s-hhh.xyz
soda.media soda.media
sortthemesitesby.com softomix.net
spamnuker.com softonicads.com
stairliftstrue.com soietvousmaime.fr
star61.de soochi.co
statistici.ro sophang8.com
statoutlook.info spabali.org
steelmaster.lv spanking.to
stockspmb.info spasswelt.net
strangeduckfilms.com spasswelt.xyz
suchenindeutschland.com spin2016.cf
suggest-keywords.com spywarespy.com
superinterstitial.com startufa.ru
swinginwithme.ru staynplay.net
stop-gepatit.te.ua
stpolice.com
strana-krasoty.ru
stroicol.net
stroyhelp-dv.ru
sucsesofinspiration.com
sugarkun.com
sumo.com
superkanpo.com
superstats.com
surflinksmedical.com
svarbit.com
swiped.su swiped.su
t-bygg.com szamponrevita.pl
tattoo33.ru tattooha.com
taxi-v-eisk.ru teletype.in
template-kid.com thedownloadfromwarez.blogspot.com
terraclicks.com thelottosecrets.com
textads.men thepokertimer.com
thegioixekhach.com thepornsex.org
theheroes.ru thiegs.reco.ws
themeforest.net tilido.com
thewomenlife.com timeallnews.ru
thin.me.pn timer4web.com
timetorelax.biz tizanidine4mg.blogspot.com
tizanidinedosage.blogspot.com tizanidinehcl2mg.blogspot.com
tizanidinefibromyalgia.blogspot.com tizanidinehcl4mgtab.blogspot.com
tizanidinehcl4mgtabinfo.blogspot.com tizanidinehclsideeffects.blogspot.com
tizanidineinteractions.blogspot.com tizanidinerecreationaluse.blogspot.com
tizanidineshowupondrugtest.blogspot.com tizanidinesideeffects.blogspot.com
tobeyouday.win tkanorganizma.ru
topmira.com tksn.ru
tourcroatia.co.uk tnaionline.org
tourismvictoria.com tomatis.gospartner.com
traffic-club.info tonerbox.kz
trk-4.net top-deal.com.pl
top1-seo-service.com
topanasex.com
topappspro.com
topcar-krasnodar.ru
torrnada.ru
toy-shop.top
tracksurf.daooda.com
tracxn.com
trafficnetzwerk.de
trafficshaper.com
trudogolik.net trudogolik.net
tubeline.biz ufa.xrus.org
tupper-posuda.ru ukrtvir.com.ua
tvnewsclips.info unisexjewelry.org
twincitiescarservice.com uogonline.com
tyumen.xrus.org uptimebot.net
uamtrk.com
udsgame.online
uggsale.online
ugogo.info
universals.com.ua
unpredictable.ga
uptime-delta.net
urccvfmc.bloger.index.hr urccvfmc.bloger.index.hr
url-extractor.xyz urlcut.ru
utrolive.ru ussearche.cf
vacuumscleaner.com valoresito.com
viagra.pp.ua valsalud.com
videogamesecrets.com vasileostrovsky-rayon.ru
vietimgy.pw vertaform.com
viktoria-center.ru vgoloveboli.net
vinylvault.co.uk via-gra.webstarts.com
vip-parfumeria.ru videosbox.ru
vip.51.la vipsiterip.org
vipps.com.my vroze.com
virtualbb.com vsdshnik.com
visa-pasport.ru
visionwell.com.cn
vk-mus.ru
vod.com.ua
voditeltrezviy.ru
vtcdns.com vtcdns.com
vucms.com vvpg.ru
webextract.profound.net we-are-gamers.com
weekes.biz.tc webalan.ru
wesharepics.com website-speed-check.site
whengirlsgowild.com weprik.ru
where-toget.com wetgames.ru
winner7777.net whatsupinfoley.com
womensplay.net wholesalejerseysgaa.com
wonderfulflowers.biz withstandingheartwarming.com
wordpress-crew.net workle.website
wpsecurity.website wpsecurity.website
www.labves.ru wurr.voila.net
www.pinnacle-bets.com www.get-free-traffic-now.com
x-mix.info x-diesel.com
xboxster.ru x-diesel.info
xclicks.net xbaboon.com
xdoza.com
xgftnlrt.bloger.index.hr
xkaz.org xkaz.org
xmladserver.com xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xtraffic.plus xn----btbdvdh4aafrfciljm6k.xn--p1ai
xtrafficplus.com xn----dtbndd4ae7eub.top
yebocasino.com xn--24-glceagatoq7c2a6ioc.xn--p1ai
yes-com.com xn--80aikhbrhr.net
yoursearch.me xnxx-n.com
ypmuseum.ru xtube.com
yuarra.pluto.ro xvideospornoru.com
zebramart.ru xxxrus.org
zkjovpdgxivg.ga ynymnwbm.bloger.index.hr
znakomstva-moskva77.ru youjizz.com
zone-kev717.info yourtemplatefinder.com
yunque.pluto.ro
yur-p.ru
zapatosenventa.info
zeroredirect2.com
zionstar.net
zmoda.hostreo.com
zocaparj.kz
zreloeporno.tv

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
03p.info 123cha.com
08800.top 18ps.ru
1001desert.com 2girls1cup-free.com
13tabs.com aa08daf7e13b6345e09e92f771507fa5f4.com
1stat.ru aa14ab57a3339c4064bd9ae6fad7495b5f.com
256bit.by abcdeg.xyz
2daytrendingnews.com addbin.men
51unlim.ru adrunnr.com
7xc4n.com adultwebhosting.info
883zy.com agahinameh.com
aa9bd78f328a6a41279d0fad0a88df1901.com algerianembassy.co.in
aab94f698f36684c5a852a2ef272e031bb.com ample-awards-today.us
admatic.com.tr animal-drawings.com
admeasures.com apccargo.com
adtech.de apiadanaknet-a.akamaihd.net
albuteroli.com appfastplay.com
allkrim.com atelielembrancaqueencanta.com.br
alphacarolinas.com australia-opening-times.com
alphahoverboards.com auto-moto-elektronika.cz
apiluckyleapnet-a.akamaihd.net avirasecureserver.com
apioasisspacenet-a.akamaihd.net ayanaspa.com
app-ready.xyz bankmib.ru
applyneedy.xyz banksrf.ru
asophoto.com begalka.xyz
aucoinhomes.com bestfortraders.com
audiofree.ru bigames.online
autoblogger24.info bio.trade-jp.net
autoplate.info bitporno.sx
autotrends.today blogs.rediff.com
babyfactory.fr blue-square.biz
bdsmgalls.net bonzybuddy.com
best-way.men brandedleadgeneration.com
bestcalovebracelet.cn bravegirlsclub.com
bharatdefencekavach.com break-the-chains.com
bidr.trellian.com cablecar.us
bildsuche.ru
bjanshee.ru
bjetjt.com
blog.f00kclan.de
blog.koreadaily.com
blogqpot.com
bonvillan.com
brokergid.ru
burn-fat.ga
business-suggested.tk
buyessay3.blogspot.ru
buyparajumpers.online
calvet.altervista.org
carmuffler.net carmuffler.net
casablancamanor.co.za cdn.walkme.com
christianlouboutinoutlet.win chlooe.com
christianlouboutinreplica.win christianlouboutinsaleonline.us
clarithromycin500mg.com christianlouboutinsaleoutletonline.us
clicksotrk.com cigarpass.com
compliance-julia.top cinemaflix.website
cookie-law-enforcement-ff.xyz comeondog.info
copblock.org compliance-irvin.xyz
cottageofgrace.com concealthyself.com
creditnation.ru confib.ifmo.ru
customboxes4less.com culturevie.info
danschawbel.com db.speedup.gdn
dawlenie.com demon-tweeks.com
daymusam.com deplim.com
delitime.info descargar-musica-gratis.net
digital-sale.su detective01.ru
discovertreasure-a.akamaihd.net dfwu1019.info
documentbase.net
domain-submit.info
downloader12.ru downloader12.ru
downloadkakaotalk.com
drunkmoms.net drunkmoms.net
drupa.com dyshagi.ru
dumuelave.xyz dyt.net
e-avon.ru earncash.com.ua
elektronischezigarettekaufen1.beeplog.com edwinkonijn.com.au
estelight.ru eets.net
eurosamodelki.ru efkt.jp
evidencecleanergold.com elektir.ru
expertnaya-ocenka.ru elektrischezigarette1.onsugar.com
faracontrol.ir elkacentr.ru
favoritemoney.ru ertelecom.ru
favornews.com esnm.ru
filerockstar298.weebly.com factorynightclub.com
filmania-x.ru fickkontaktehobbyhuren.com
forensicpsychiatry.ru filmetricsasia.com
formseo.com filunika.com.ru
front.ru financepoints.eu
funcrushgames.com firesub.pl
gambarxkata.co floating-share-buttons.com
gameonasia.com forum.poker4life.ru
gamerextra.com foxtechfpv.com
gentamicineyedrops.blogspot.com freshdz.com
ghostvisitor.com frighteningremain.cf
goldenggames.com g.starmoe.xyz
gowreckdiving.com gaz-voshod.ru
greenzaim.ru girlgamerdaily.com
gsasearchengineranker.site gribkovye-zabolevaniya.com
hahashka.ru hermesreplica.pw
hanwei.us
healing-dysplasia.ru
heatpower.ru
highland-homes.com
himazin.info himazin.info
holidaypics.org hledejvshopech.cz
horoshieokna.com homeart.space
hostingclub.lk hornymatches.com
housemilan.ru hully.altervista.org
houtings.xyz hundejo.com
howmuchdoestizanidinecost.blogspot.com i4track.net
howtoclean.club iklysha.ml
humanorightswatch.org impresagaia.it
i-midias.net.br instabid.tech
imicrovision.com interior-stickers.ru
imperia31.ru intermesh.net
inboxdollars.com internet-apteka.ru
incomekey.net ivoiretechnocom.ci
infogame.name iwantmyfreecash.com
infowarcraft.ru jerseywholesalebizchina.com
insta-add.pro jolic2.com
instasexyblog.com juliaworld.net
istizanidineanarcoticdrug.blogspot.com keywordsdoctor.com
j33x.com kino-rating.ru
javatex.co.id kino2018.cc
johnrobertsoninc.com klejonka.info
junglenet-a.akamaihd.net kochanelli.com
keywordblocks.com konkursov.net
keywordhut.com kraljeva-sutjeska.com
keywordsuggests.com ks1234.com
kichenaid.ru ktotut.net
kino-filmi.com lapolis.it
kinogolos.ru larutti.ru
kollesa.ru
kupiproday.com.ua
legalrc.biz legalrc.biz
lietaer.com letmacwork.world
likesdesign.com letslowbefast.site
linkbux.ru linkarena.com
loadingpages.me linkdebrideur.xyz
loadopia.com lion.conboy.us
magiadinamibia.blogspot.com localflirtbuddies.com
mansiondelrio.co lonerangergames.com
marketingtechniques.info luxup.ru
mcadamssupplyco.com mac-shield.com
mcar.in.ua machicon-akihabara.info
maglid.ru
mamasuper.prom.ua
marmotstore.online
matchpal-a.akamaihd.net
mazda-roadsters.com
mearns-tractors.co.uk mearns-tractors.co.uk
mebel-alait.ru member-quiz.com
medosmotr-ufa.ru metalonly.info
misswell.net missclub.info
mmgq.ru moesen-ficken.com
mobile-appster.ru moneymaster.ru
mobile.ok.ru morefastermac.trade
mobioffertrck.com mosrif.ru
more-letom.ru mrbitsandbytes.com
moscow.online-podarki.com msk-diplomat.com
mp3ritm.top mvpicton.co.uk
muschisexbilder.com nailsimg.com
muz-baza.net nosecret.com.ua
mybinaryoptionsrobot.com obuv-kupit.ru
mysexpics.ru odywpjtw.bloger.index.hr
nakrutka.cc orion-code-access.net
naphukete.ru osnova3.ru
newstaffadsshop.club paphoselectricianandplumber.com
nordstar.pro partner-high.men
o333o.com
oakridgemo.com
oddamzadarmo.eu
onescreen.cc
onko-24.com
ooomeru.ru
orenburg-gsm.ru
paleohub.info
partner-trustworthy.men partner-trustworthy.men
penisvergrotendepillennl.ovh pc4download.co
photostudiolightings.com pcads.ru
piccshare.com
pinsdaddy.com pinsdaddy.com
pistonclasico.com piuminiita.com
pornokajf.com plastweb.ru
profitwithalex.info pliks.pl
poffet.net
pogodnyyeavarii.gq
popup-fdm.xyz
popup-hgd.xyz
popup.matchmaker.com
pornhub-ru.com
porodasobak.net
prchecker.info
prom23.ru prom23.ru
prostitutki-rostova.org promover.org
questionmarque.ch prospekt-st.ru
r-e-f-e-r-e-r.com prostitutki-belgoroda.org
ranksignals.com pshare.biz
rogervivierforsale.com pukaporn.com
royal-investments.net quizzitch.net
rutor.vip ragecash.com
saitevpatorii.com ranksays.com
sankt-peterburg.nodup.ru raschtextil.com.ua
scanmyphones.com realizmobi.com
recordpage-a.akamaihd.net
remybutler.fr
rockprogblog.com
ros-ctm.ru
saneitconsulting.com
santechnik.jimdo.com
savetubevideo.com
savingsslider-a.akamaihd.net
scanner-barak.top scanner-barak.top
searchlock.com score-ads.men
security60-e.com sdelatmebel.ru
sei80.com seccioncontrabajo.com
seorussian.ru seo-prof1.xyz
service.adtech.fr seorankinglinks.us
serving.adbetclickin.pink sex-tracker.de
seryeznie-znakomstva.ru sexblog.pw
sex-znakomstva.online shariki-zuma-lines.ru
sexkontakteao.info
sexobzor.info
sexpartygirls.net
sfj-ror.no
shivafurnishings.com shivafurnishings.com
sideeffectsoftizanidine.blogspot.com shoesonlinebuy.xyz
site-analyzer.com shoppytoolmac.com
snowplanes.com sierraapps.com
social-search.me sitevalued.com
skyway24.ru
slavic-magic.ru
smallseotools.com
soc-econom-problems.ru
soc-proof.su
sodexo.com sodexo.com
solicita.info sport7777.net
sotechco.co sports-supplements.us
steame.ru srgwebmail.nl
strongholdsb.ru startwp.org
svbur.ru stjamesschool.info
swimpool.ca strongsignal-a.akamaihd.net
tecnoteakviareggio.it stroy-portal22.ru
telefonsex-ohne0900.net stroyplus.ru
thecoolimages.net sundrugstore.com
thefarmergame.com surfbuyermac.com
thin.me.pn taranerymagesswa.blogspot.com
tizanidinecost.blogspot.com teastory.co
tizanidineoral.blogspot.com thaismartloan.com
toonfamilies.net thegamerznetwork.com
top10-online-games.com totu.info
topsiteminecraft.com tube8.com
track.deriv.com turizmus.us
tramadolandtizanidine.blogspot.com turvgori.ru
treasuretrack-a.akamaihd.net tvoystartup.ru
typer.one tyumen.xrus.org
urlcut.ru valkiria-tk.ru
usdx.us varikoz24.com
vacuumscleaner.com versaut.xxx-cam.webcam
vepad.com vip-dom.in
vkak.ru vip-file.com
wallpapers-best.com visionwell.com.cn
walpaperlist.com wallpapersist.com
webradiology.ru webalan.ru
websiteaccountant.de wetgames.ru
wfb.hatedriveapart.com wma-x.com
wingsoffury2.com wordpresscore.com
wsgames.ru www.jbetting.com
www.pinnacle-bets.com xchat26.myfreecams.com
x-diesel.biz xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
xn--80aanaardaperhcem4a6i.com xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai xn--80ajbshivpvn2i.xn--p1ai
xn--90acenikpebbdd4f6d.xn--p1ai xn--oogle-wmc.com
xn--d1abj0abs9d.in.ua xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
yellowstonevisitortours.com youbloodyripper.com
yginekologa.com youporn-ru.com
yur-p.ru yourporn.com
zirondelli.it yun56.co
zvooq.eu zakazfutbolki.com
zeroredirect7.com
zlatnajesen.com
znakomstva-piter78.ru
zootoplist.com
zreloeporno.tv

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.79 0; 10.1.0.82 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3559 ### Version: V4.2023.01.3560
### Updated: Sat Jan 28 10:03:12 UTC 2023 ### Updated: Sat Jan 28 22:01:13 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3559 ### Version: V4.2023.01.3560
### Updated: Sat Jan 28 10:03:12 UTC 2023 ### Updated: Sat Jan 28 22:01:13 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##