vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3491 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-25 10:08:56 +00:00
parent 5b301b1193
commit 65485c705e
47 changed files with 95302 additions and 95312 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3490 #### Version: V4.2022.11.3491
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3490 #### Version: V4.2022.11.3491
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3490 #### Version: V4.2022.11.3491
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11778
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3490 #### Version: V4.2022.11.3491
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11782
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11782
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11782
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11782
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

11782
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

11782
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

11782
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3490 3491

11782
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11782
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.169.221.175 0; 20.172.197.130 0;

11782
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

11782
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

11782
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

11782
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.169.221.175 0; 20.172.197.130 0;

11782
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.169.221.175 0; 20.172.197.130 0;

11782
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BackStreet DTS Agent
ChinaClaw DatabaseDriverMysqli
DataCha0s Ezooms
DomainAppender GridBot
GalaxyBot LinkWalker
MolokaiBot Openvas
Surfbot Seomoz
TightTwatBot WebAuto
instabid YoudaoBot
lwp-request Zermelo

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider
404checker
404enemy 404enemy
AIBOT AIBOT
ASPSeek
Abonti
Aboundexbot Aboundexbot
Acunetix
AfD-Verbotsverfahren
AiHitBot AiHitBot
Aipbot
Alexibot
AllSubmitter AllSubmitter
Alligator AlphaBot
Anarchie
Anarchy
Ankit
Anthill Anthill
Atomseobot Atomseobot
Attach
AwarioSmartBot AwarioSmartBot
BBBike BackStreet
BackDoorBot BackWeb
Backlink-Ceck Backlink-Ceck
Badass Barkrowler
Bandit
BatchFTP BatchFTP
BetaBot Battleztar Bazinga
Bigfoot Bigfoot
Boardreader Bitacle
Bolt BlackWidow
Buck Blow
Brandprotect
Brandwatch
Buddy Buddy
BuiltBotTough Bullseye
BuiltWith BunnySlippers
CODE87 BuzzSumo
CCBot
CSHttp
Calculon Calculon
CazoodleBot
CheTeam
CheeseBot CheeseBot
CherryPicker Citoid
ChinaClaw Cliqzbot
Claritybot
Copier Copier
CopyRightCheck CopyRightCheck
Copyscape
Craftbot Craftbot
CrazyWebCrawler Custo
Curious DSearch
DIIbot DataCha0s
DatabaseDriverMysqli
Digincore Digincore
Dirbuster Dirbuster
Discoverybot
Dispatch
DomCopBot DomCopBot
DomainAppender
DomainStatsBot DomainStatsBot
Download Wonder Download Wonder
Dragonfly Dragonfly
ECCP/1.0 Drip
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
EasyDL
Ecxi
EirGrabber EirGrabber
EroCrawler
Evil Evil
Extreme Picture Finder Exabot
EyeNetIE Express WebPictures
Ezooms ExtLinksBot
Firefox/7.0 Extractor
ExtractorPro
FHscan
FemtosearchBot
Fimap
FlashGet
Foobot
Freeuploader Freeuploader
GalaxyBot Fuzz
Genieo Fyrebot
G-i-g-a-b-o-t
GermCrawler
GetRight
GetWeb GetWeb
Gigabot
Go!Zilla
GoZilla
Gotit
GrabNet GrabNet
GrapeshotCrawler HTMLparser
HTTrack Havij
Haansoft Hloader
IDBTE4M HybridBot
Id-search IDBot
IlseBot IRLbot
Intelliseek Iblog
InterGET Image Fetch
IndeedBot
Internet Ninja Internet Ninja
InternetSeer
Iria
Iskanie
IstellaBot IstellaBot
JennyBot
JetCar
Jetty
JikeSpider JikeSpider
LNSpiderguy Jyxobot
Kenjin Spider
Kozmosbot
LWP::Simple LWP::Simple
Larbin Larbin
Lftp Leap
LeechFTP
LexiBot
LibWeb LibWeb
Libwhisker Libwhisker
LieBaoFast
Lightspeedsystems Lightspeedsystems
Likse Likse
LinkScan LinkScan
Linkbot LinksManager
LinkextractorPro LinqiaRSSBot
LinkpadBot Lipperhey
LinqiaMetadataDownloaderBot Lipperhey Spider
Lmspider MIDown tool
MFC_Tear_Sample
MIIxpc MIIxpc
MSIECrawler Mail.RU_Bot
MTRobot Majestic SEO
Mag-Net MarkWatch
Mass Downloader
Masscan
Mata Hari Mata Hari
MauiBot
Mb2345Browser Mb2345Browser
Meanpathbot
Mediatoolkitbot
Metauri Metauri
MicroMessenger
Microsoft URL Control Microsoft URL Control
Minefield Minefield
Mister PiX Mister PiX
Moblie Safari
MolokaiBot
Mr.4x3
Msrabot Msrabot
NICErsPRO NICErsPRO
Nameprotect
NearSite
Needle Needle
NetMechanic NetLyzer
NetZIP
Netcraft
Nettrack Nettrack
NextGenSearchBot
Niki-bot
Nikto Nikto
NimbleCrawler Ninja
Not
OnCrawl
Openfind
OrangeBot OrangeBot
OutfoxBot OutclicksBot
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageGrabber PageAnalyzer
Pandalytics Pandalytics
Pi-Monster Panscient
Papa Foto
Pavuk
PeoplePal
Petalbot
Picscout
Picsearch Picsearch
PictureFinder PictureFinder
Piepmatz PleaseCrawl
Pump Pockey
PyCurl PyCurl
QueryN Metasearch
RSSingBot RSSingBot
RankActive
Rankivabot
RankurBot RankurBot
Re-re
ReGet
Reaper Reaper
Recorder RebelMouse
RepoMonkey RedesScrapy
SEOkicks-Robot RocketCrawler
SEOprofiler SEOlyticsCrawler
SEOstats SEOstats
SISTRIX SISTRIX
ScanAlert SalesIntelligent
Scanbot Screaming
ScoutJet
Scrapy
ScreenerBot ScreenerBot
Seekport Searchestate
SemanticJuice SemanticJuice
SemrushBot
SentiBot
SeoSiteCheckup
SeobilityBot
Seomoz Seomoz
Siphon Site Sucker
SiteLockSpider
Siteimprove
Sitevigil Sitevigil
Snapbot SmartDownload
SocialRankIOBot
Sociscraper
Sosospider Sosospider
Spinn3r SpankBot
Spanner
Spbot
Sqlmap Sqlmap
Steeler Sqlworm
Stripper
SuperBot
SuperHTTP
Surfbot
SurveyBot SurveyBot
Szukacz Suzuran
Swiftbot
Teleport
Telesoft Telesoft
Telesphoreo Titan
The Intraformant Toata
Thumbor
Toweyabot
Tracemyfile Tracemyfile
Trendictionbot
Turingos
Turnitin Turnitin
TurnitinBot
TwengaBot
Twice
Typhoeus Typhoeus
URLy Warning
UnisterBot
Upflow Upflow
V-BOT VCI
VB Project
Vacuum Vacuum
VeriCiteCrawler
VidibleScraper VidibleScraper
VoidEYE Virusdie
WISENutbot Voil
WPScan
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
Wallpapers Wallpapers
WallpapersHD Wallpapers/3.0
WeSEE
Web Collage Web Collage
Web Sauger WebCopier
WebGo IS WebEnhancer
WebPix WebPix
WebReaper
WebSauger WebSauger
WebStripper
WebWhacker WebWhacker
Webalta Website Quester
WebsiteExtractor
Webster Webster
WiseGuys Robot Whack
Wprecon WinHTTrack
Xaldon_WebSpider Wonderbot
Xenu Woobot
YoudaoBot
Zermelo Zermelo
Zeus
Zitebot
ZmEu ZmEu
ZoomBot ZoominfoBot
arquivo.pt arquivo.pt
autoemailspider autoemailspider
backlink-check
check1.exe check1.exe
com.plumanalytics crawler.feedback
demandbase-bot eCatch
evc-batch
facebookscraper facebookscraper
gopher
heritrix
instabid
isitwp.com
iubenda-radar
lwp-trivial lwp-trivial
meanpathbot
mediawords mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler
oBot oBot
page scorer
pcBrowser pcBrowser
plumanalytics
ripz ripz
sexsearcher s1z.ru
scan.lol
seocompany.store
sitechecker.pro sitechecker.pro
siteripz
spyfu
sysscan
tAkeOut tAkeOut
trendiction.de trendiction.com
voyagerx.com webpros.com
webgains-bot zgrab
webmeup-crawler
webprosbot
x09Mozilla
xpymep1.exe
zauba.io

292
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 404checker
80legs ADmantX
AhrefsBot AIBOT
ALittle Client
Abonti
AfD-Verbotsverfahren
AiHitBot AiHitBot
Aipbot Aipbot
Alexibot Alexibot
AllSubmitter AllSubmitter
Alligator
Anarchie
Anarchy Anarchy
Anarchy99
Ankit Ankit
Apexoo
Aspiegel Aspiegel
Asterias Asterias
AwarioRssBot Attach
AwarioSmartBot AwarioSmartBot
BBBike BBBike
BLEXBot BDFetch
BackDoorBot BackDoorBot
BackStreet BackStreet
BackWeb BatchFTP
Barkrowler BetaBot
Battleztar Bazinga Black Hole
BlackWidow
Blow
BlowFish
Boardreader Boardreader
Bolt
Brandwatch
Buck Buck
Bullseye
BunnySlippers BunnySlippers
CATExplorador
CODE87
Calculon
Cegbfeieh Cegbfeieh
Claritybot CheeseBot
Chlooe
Citoid
Collector
Copyscape Copyscape
Crescent
CrunchBot
CyotekWebCopy
DBLBot DBLBot
DIIbot DTS Agent
DSearch
DatabaseDriverMysqli
Demon Demon
Devil Devil
DigitalPebble
Disco Disco
Discobot DnyzBot
Discoverybot DomainCrawler
DnBCrawler-Analytics
DomCopBot
DomainAppender
DomainSigmaCrawler
Dotbot Dotbot
Download Wonder
Dragonfly Dragonfly
Drip
ECCP/1.0 ECCP/1.0
EMail Wolf EMail Siphon
EasyDL EasyDL
Ebingbong EirGrabber
EroCrawler Extractor
Evil ExtractorPro
EyeNetIE EyeNetIE
FDM Ezooms
Firefox/7.0 Firefox/7.0
Foobot FlashGet
Fuzz Freeuploader
G-i-g-a-b-o-t
GT::WWW GT::WWW
Genieo GetRight
Go!Zilla Gigabot
Go-Ahead-Got-It GrapeshotCrawler
Gotit HTTP::Lite
Grafula
GrapeFX
GridBot
HMView
HTMLparser
HTTrack
Harvest Harvest
Havij
Hloader Hloader
IDBot IDBTE4M
Iblog Image Sucker
Id-search IndeedBot
Indy Library
InfoNaviRobot
InfoTekies InfoTekies
Intelliseek Intelliseek
InterGET JetCar
InternetSeer
Iria
JOC Web Spider
Jbrofuzz
JikeSpider JikeSpider
Joomla
JustView JustView
Jyxobot Jyxobot
Kenjin Spider Kenjin Spider
Kinza Keyword Density
Kozmosbot Kozmosbot
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot Lanshanbot
Larbin
LeechGet LeechGet
Libwhisker LexiBot
Lftp
LieBaoFast LieBaoFast
Lightspeedsystems Likse
LinkScan LinkextractorPro
Linkbot
LinqiaRSSBot
Lipperhey Lipperhey
Litemage_walker Litemage_walker
MQQBrowser Ltx71
MIDown tool
MJ12bot
MSFrontPage MSFrontPage
Mag-Net MTRobot
Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic-SEO Majestic12
MarkWatch Mass Downloader
Masscan
Mata Hari
MeanPath Bot MeanPath Bot
MegaIndex.ru MegaIndex.ru
Metauri Metauri
MicroMessenger Microsoft Data Access
Microsoft URL Control Minefield
Moblie Safari Mister PiX
Mojeek
MolokaiBot MolokaiBot
Mr.4x3 Morfeus Fucking Scanner
NPbot Mozlila
Msrabot
Musobot
Navroad Navroad
Needle Needle
NetZIP NetAnts
Netcraft Nikto
Nettrack
Niki-bot
Ninja Ninja
Nmap
Not Not
Octopus Nutch
Offline Explorer Offline Navigator
OpenVAS OpenVAS
OrangeSpider
OutclicksBot OutclicksBot
PHPCrawl PHPCrawl
Page Analyzer POE-Component-Client-HTTP
PageAnalyzer
PageGrabber
Pandalytics Pandalytics
Panscient Pavuk
PeoplePal Picsearch
Pi-Monster Pimonster
Picscout
PictureFinder
Pixray Pixray
PleaseCrawl PleaseCrawl
Pockey
ProPowerBot ProPowerBot
ProWebWalker ProWebWalker
Psbot Probethenet
Pu_iN
Pump
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot
RankActiveLinkBot
RankingBot RankingBot
RankurBot RealDownload
Re-re
RedesScrapy
RepoMonkey RepoMonkey
Ripper
Rogerbot Rogerbot
SEOkicks
SEOstats SEOstats
ScanAlert SalesIntelligent
Scanbot Scrapy
Screaming ScreenerBot
Seekport
SemanticJuice SemanticJuice
SemrushBot
SeoSiteCheckup SeoSiteCheckup
Shodan SeobilityBot
Seomoz
Siphon
Site Sucker Site Sucker
SiteLockSpider SiteExplorer
SiteSnagger SiteSnagger
SiteSucker Snapbot
Sottopop Sogou web spider
SpaceBison SpaceBison
Spbot Spammen
Spanner
SputnikBot SputnikBot
Sqlmap
Sqworm Sqworm
Surfbot
T8Abot T8Abot
TeleportPro TeleportPro
Telesoft
Telesphoreo
Telesphorep Telesphorep
The Intraformant TightTwatBot
TheNomad Tracemyfile
Toata
Trendiction Trendiction
Trendictionbot TwengaBot
True_Robot UnisterBot
Turingos VB Project
Turnitin
URLy Warning
URLy.Warning
Upflow
V-BOT
VCI VCI
Vacuum Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler VidibleScraper
VeriCiteCrawler Virusdie
Voil
Voltron
WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV WEBDAV
WISENutbot
WWW::Mechanize
WWWOFFLE WWWOFFLE
Web Enhancer Web Enhancer
Web Fetch Web Fetch
Web Sauger Web Pix
WebAuto
WebBandit WebBandit
WebEnhancer WebCollage
WebCopier
WebFetch
WebFuck
WebGo IS WebGo IS
WebImageCollector WebImageCollector
WebReaper WebLeacher
WebStripper WebSauger
WebZIP WebSucker
Website Quester WebWhacker
WebsiteExtractor Webalta
Whatweb Whacker
Who.is Bot
WinHTTrack WinHTTrack
Wonderbot WiseGuys Robot
Woobot Woobot
Xaldon_WebSpider Wprecon
Zade YoudaoBot
Zauba Zeus
ZmEu Zitebot
ZoomBot
ZyBorg ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt
autoemailspider autoemailspider
cah.io.community
coccocbot coccocbot
com.plumanalytics com.plumanalytics
crawl.sogou.com
crawler.feedback
dataforseo.com
domainsproject.org
eCatch
evc-batch evc-batch
gopher internetVista monitor
heritrix
ips-agent ips-agent
iubenda-radar
linkdexbot
lwp-request
lwp-trivial lwp-trivial
muhstik-scan mediawords
oBot netEstate NE Crawler
pcBrowser probe-image-size
ripz satoristudio.net
scan.lol scan.lol
seoscanners seobility
seostar seostar
serpstatbot serpstatbot
siteripz sogouspider
spyfu trendiction.com
tAkeOut trendiction.de
ubermetrics-technologies.com
webgains-bot
webmeup-crawler webmeup-crawler
webpros.com x22Mozilla
xpymep1.exe zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
girlsfuckdick.com 2daytrendingnews.com
hubbble.com boeuklad.com
larose.jb2c.me children-learningreading.info
morlat.altervista.org downloadeer.net
perso.wanadoo.es kellyonline.xyz
popup-jdh.xyz khadastoafarde.tk
revistaindustria.com lovi-moment.com.ua
supermodni.com.ua rusenvironmental.net
tizanidinesideeffects.blogspot.com sexy.babes.frontend-stack.top
usacasino.com zionstar.net

922
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
100searchengines.com 00go.com
24x7-server-support.site 03e.info
2kata.ru 1008.su
34.gs 101lesbian.xyz
3w24.com 178evakuator178.ru
45en.ru 1flag.co.za
4ur.click 28n2gl3wfyb0.ru
4webmasters.org 2girls1cup-free.com
5i2.net 2itech.ru
6128786.com 3rbseyes.com
70casino.online 4kpics.rocks
5u.com
72-news.com 72-news.com
7a2rnuey1tw9ar.ru 777-club.ru
7fon.ru aac52006ec82a24e08b665f4db2b5013f7.com
a14download.com aadroid.net
abcdeg.xyz abclauncher.com
adclickthru.net above.com
adktrailmap.com acmebtn.ml
adtiger.tk actionnooz.com
advertisingtag.net adexprts.com
aksonural.ru adloads.com
aktobe.xkaz.org adsland.men
alarmobninsk.ru adsref.men
alert-fjg.xyz adviceforum.info
alfapro.ru adzpower.com
alibestsale.com afmuseum.com
alkoravto.ru agahinameh.com
alphavisions.net aliexpresscashback.club
alveris.ru alldownload.pw
alvtank.se allmarketsnewdayli.gdn
ameblo.top allproblog.com
ample-awards-today.us allstatesugarbowl.org
anaseracresar.tk alot.com
anatomiy.com am-se.com
androids-store.com amospalla.es
anmysite.com analnoeporno.tv
apccargo.com analytics-ads.xyz
apiluckyleapnet-a.akamaihd.net android-vsem.org
anime.dougasouko.com
aosheng-tech.com
apelsinnik.website
apibetweenlinesn-a.akamaihd.net
apiqualitinknet-a.akamaihd.net apiqualitinknet-a.akamaihd.net
apple.com-webbrowsing-security.science appearance-cool.com
applyneedy.xyz apper.de
arcadeplayhouse.com artdestshop.eu
arcteryxsale.online asdfg.pro
articlesdirectoryme.info atdedinotuho.tk
ausergrubhof.info autoloans.com
auto.rusvile.lt automatic-seo.com
autowebmarket.com.ua bag77.ru
avek.ru
aviav.ru.com
avkzarabotok.info
avtointeres.ru
azazu.ru
azbaseimages.net
azlex.uz
badavit.com.ua
bala.getenjoyment.net
balkanfarma.org
baltgem.com baltgem.com
bannerconnect.net bang-hotties.com
battleforupdating.site barnfurnituremart.com
bayareaaeroclub.org bbsoldes.fr
beatifullandscape.co beauty-clinic.ru
bestbuy.ca best-seo-solution.com
bestwrinklecreamnow.com best-way.men
bezcmexa.ru bestpornuha.com
bfz.biz besttorrentknifta.weebly.com
bg6s0.com bet-prognoz.com
billiard-classic.com.ua biglistofwebsites.com
birzha-truda.eu bildsuche.ru
bimatoprost-careprost.com.ua
bimmerpost.com
bird1.ru
bitcoinpile.com
bitcoins-live.ru bitcoins-live.ru
bizcheapjerseyswholesalechina.com biznesschat.net
bjanshee.ru blog.axant.it
blackcurranthumidifiers.site blogporn.in
bloglag.com blogrankers.com
bmsco.co blogs.rediff.com
bmw.afora.ru bonzbuddy.com
boazpower.com bracketsmackdown.com
brandbucket.com brainsandeggs.blogspot.com
budpost.com.ua braintobe.win
bum.com.ru brandedleadgeneration.com
buyantiviralwp.com brandov.ru
buytizanidineonline.blogspot.com bravegirlsclub.com
buzzonclick.com bristolhostel.com
call-of-duty.info buy-forum.ru
calstaterealty.us buytizanidineonlinenoprescription.blogspot.com
candelluxsklep.pl buzz-porno.info
candlewooddentalcentre.com.au bycontext.com
canvas.gloverid.site c1.onedmp.com
cardul.ru candiceloves.us
casablancamanor.co.za candlehandmade.com
cenokos.ru carabela.com.do
certifywebsite.win carloans.com
cezartabac.ro carmuffler.net
chatroulette.life carrauterie.be
cheapbarbour.online cars-modification.net
cheapjerseysfootballshop.com cartierlovebraceletreplica.xyz
chee-by.biz cartierlovestore.com
catalogs-parts.com
ccbill.com
cementaresearch.se
cheapwebsitehoster.com
choice-credit.ru
christianlouboutinreplica.win
ci.ua
clean-start.net clean-start.net
clicksotrk.com cleanallspyware.com
clmforexeu.com clickbanksites.info
codysbbq.com clickpapa.com
cometorussia.net clickprotects.com
communisave.co.za clickso.com
compliance-elena.xyz clicksor.net
compliance-jess.top cmd.kz
compliance-nelson.top coldfilm.ru
conboy.us collegeessay19.blogspot.ru
cookie-law-enforcement-ff.xyz compiko.info
covid-schutzmasken.de compliance-john.top
cowblog.fr compucelunlock.net
crest3d.ru comsysnet.com
customsua.com.ua contentlook.co
czat.wp.pl cornomase.win
daydream-studio.ru counterbot.com
covetnica.com
customcatchcan.com
daneshetabiat.com
datract.com
datsun-do.com
de.zapmeta.com de.zapmeta.com
dealighted.com decenttools.com
deanmoore.ie deda-moroza-zakaz.ru
degerlund.net delayreferat.ru
dfwu1013.info dickssportinggoods.com
diffbot.com dneprsvet.com.ua
dimkino.ru doctissimo.fr
discovertreasurenow.com domain-tracker.com
divisioncore.com
documentsite.net
dostavimvdom.ru dostavimvdom.ru
dosugrostov.site downvids.net
downloadme.life dverimegapolis.ru
downtuptv.gq e-c.al
drugs-no-rx.info e-collantes.com
drugspowerstore.com ebooknovel.club
e-commerce-seo1.com ed-shop01.ru
e-kwiaciarz.pl edge.sharethis.com
e705.net edwinkonijn.com.au
ednorxmed.com elektronischezi.livejournal.com
edudocs.net elitesportsadvisor.com
egovaleo.it
elektrischezigarette2.devhub.com
elektronischezigarette2.mex.tl
elektrozigarette2.webs.com
elektrozigarettekaufen1.devhub.com
elkacentr.ru elkacentr.ru
energy-ua.com elvenmachine.com
eredijovon.com emerson-rus.ru
ereko.ru enge-fotzen.info
englishdictionaryfree.com
eropho.net
erotikstories.ru erotikstories.ru
essay-writing.work escort-russian.com
essayservicewriting.org etenininrade.ga
este-line.com.ua eu-cookie-law.blogspot.com
expertblog.info eupornstar.info
european-torches.ru
evening-dating-club.info
exchangeit.gq
exct.net
extlinks.com extlinks.com
eyes-on-you.ga eyes-on-you.ga
f012.de f-online.de
famix.xyz faracontrol.ir
faptitans.com fatfasts-4tmz.com
farm26.ru fearcrow.com
fashion.stellaconstance.co fertilitetsradgivningen.se
fast-wordpress-start.com fetlifeblog.com
fefo.gdn fidelityfunding.com
filkhbr.com filkhbr.com
findpik.com findacheaplawyers.com
fireads.men finteks.ru
foodgid.net firma-legion.ru
flooringinstallation-edmonton.com
footballfarrago.com
forminecrafters.ru forminecrafters.ru
forum-engineering.ru formularz-konkurs.tk
free-gluten.ru fr-bearings.ru
freenode.info franklinfire.co
freshmac.space frbizlist.com
freza-sverlo.ru frivgame250.com
front.to fuckmill.com
fuckbuddybestgilf.info fullfileaccess.com
funnymama.com fullgirl.ru
fxtips.ru funcrushgames.com
fyxabomiw.ru fungamelands.com
galeria-zdjec.com gamesprite.me
gallerily.com gaypornmovie.net
game300.ru gifspics.com
garciniaxt.us gkvector.ru
gcup.ru
ghazel.ru
gigapeta.com
glavprofit.ru
glcomputers.ru
glicol.kz
globetrotting-culture.ru globetrotting-culture.ru
goldpanningtools.com gne8.com
gnuetella.com
googleseo.com.tr googleseo.com.tr
gosmeb.ru goosefishpost.bid
gratis-sexkontakte.com growmyfunds.ca
greatdealshop.com grtyi.com
gsasearchengineranker.pw guiadeserraazul.com
gsbs.com.ua hamptonoaks.ca
guildebzh.info
hatdc.org hatdc.org
helvetia.com.ua hdimagegallery.net
hkladys.com hdwallpapers-free.com
hop.clickbank.net hexpilot.com
host-tracker.com hifidesign.ru
howlongdoestizanidinestayinyoursystem.blogspot.com hildinghr.se
htmlcorner.com hitsbox.info
huhn.altervista.org hiwibyh.bugs3.com
i-midias.net.br hoste.octopis.com
ilovevitaly.com hotxnights.info
immobiliaremassaro.com howtoclean.club
impotentik.com hyiphunter.org
incomekey.net hystersister.com
intervsem.ru i-service.kz
investpamm.ru idegenvezeto.eu
iplogger.org iloveitaly.ru
iqoption.com imagerydatabase.com
iqupdatetmz.win images-free.net
islamtoday.co.za imgarit.pw
ispac.org imperia31.ru
iwantedmoney.com infostatsvc.com
ix20.ru insider.pro
javrip.net integritylandscapeservices.com
jo24news.com investsuccess.org
joyceblog.top iomoio.net
junketjuice.blogspot.com istizanidineanarcoticdrug.blogspot.com
jus0wil.pp.ua ivoiretechnocom.ci
justucalling32211123456789.tk izhevsk.xrus.org
kashubadesign.ru jabimgo.pw
keyhantercume.com james13prix.info
kinofak.net javlibrary.cc
javspace.net
jerseyswholesalechinalimited.com
joingames.org
jolic2.com
kakablog.net
karachev-city.ru
karpun-iris.ru
kazan.xrus.org
kazan.zrus.org
keki.info
keywordbasket.com
keywordhut.com
kinogolos.ru kinogolos.ru
kinozapas.com kiwi237au.tk
kit-opt.ru konkursov.net
kmd-pto.ru kostenlos-sexvideos.com
konpax.com kredit-blog.ru
kraljeva-sutjeska.com kretpicf.pw
krasivoe-hd.com kvartiry-remont.ucoz.ru
krasivoe-hd.net kw21.org
kupit-adenu.ru lampokrat.ws
lawyersinfo.org laudit.ru
layola.biz.tc lego4x4.ru
ldrtrack.com lezbiyanki.net
letmacwork.world li-er.ru
life-instyle.com lider-zhaluzi.kiev.ua
lignofix.ua lifehacĸer.com
likesuccess.com lineshops.biz
lineavabit.it lionking-1994.blogspot.com
linkwithin.com livesex-amateure.info
livefixer.com liveshoppersmac.com
liverpool.gsofootball.com locatellicorretor.com.br
localflirtbuddies.com loftdigital.eu
lombardfinder.ru lonerangergames.com
lovelycraftyhome.com luchshie-topcasino.ru
lowephotos.info magento-crew.net
lsp-awak-perikanan.com maju.bluesalt.co
lyngdalhudterapi.no managerpak204.weebly.com
macresource.co.uk mansiondelrio.co
mafa-free.com mansparskats.com
magda-gadalka.ru manualterap.roleforum.ru
makenahartlin.com manyresultshub-a.akamaihd.net
manve.info
mapquestz.us
marcogrup.com marcogrup.com
matrixalchemy.com marcowebonyodziezowe.pl
meble-bogart.info mathgym.com.au
medicinacom.ru maxthon.com
meetlocalchicks.com medkletki.ru
megavolt.net.ua meds-online24.com
meltwater.com mesto-x.com
mjchamonix.org michaelkorsoutlet.store
mnogolok.info millionare.com
minyetki.ru
mitrasound.ru
mobifunapp.weebly.com
mobilemedia.md mobilemedia.md
mobilierland.com modabutik.ru
mojpregled.com moje-recenze.cz
mompussy.net mokrayakiska.com
monetizationking.net moneytop.ru
montredemarque.nl moneyzzz.ru
mototsikl.org morocco-nomad-excursions.com
movies-in-theaters.net morpicert.pw
msk.afora.ru mowser.com
moyakuhnia.ru
mp3ritm.top
mp3s.club
mtmtv.info
musflashtv.com
musicktab.com musicktab.com
musicspire.online musicspire.online
muznachas-service.ru mydownlodablefiles.com
myfreetutorials.com myliveblog.ru
myghillie.info myperiod.club
mymercy.info naval.jislaaik.com
myplaycity.com newfilmsonline.ru
nacap.ru nextbackgroundcheck.gq
nastydollars.com nextlnk12.com
netcheckcdn.xyz nextrent-crimea.ru
news-readers.ru niroo.info
newyorkhotelsmotels.info ntic.fr
nfvsz.com nw-servis.ru
novgorod.xrus.org o00.in
o333o.com oakridgemo.com
ochistka-stokov.ru offside2.5v.pl
odywpjtw.bloger.index.hr ooo-olni.ru
oecnhs.info open-odyssey.org
of-ireland.info outrageousdeal-a.akamaihd.net
offtopic.biz oxotl.com
omsk.xrus.org paceform.com
oneminutesite.it pammik.ru
online-podarki.com
onlinetvseries.me
onlywoman.org
openmultipleurl.com
ororodnik.goodbb.ru
ostrovtaxi.ru
outclicks.net
ownshop.cf
oz-offers.com
ozas.net
panamaforbeginners.com
paperwritingservice17.blogspot.ru paperwritingservice17.blogspot.ru
partner-stop.men patterntrader-en.com
pavlodar.xkaz.org pavlodar.xkaz.org
paydayonlinecom.com
pc-services.ru
pelfind.me pelfind.me
pensplan4u.com pendelprognos.se
perederni.net perl.dp.ua
pharmacyincity.com perm.xrus.org
php-market.ru picanalyzer.data-ox.com
picmoonco.pw piccdata.com
picswe.com
pihl.se
pills24h.com
pillscheap24h.com
pinsdaddy.com
pintattoos.com pintattoos.com
piratecams.com pix-hd.com
pisanieprac.info play-movie.pl
pixshark.com
pizda.lol
platesauto.com
platezhka.net
playfortuna-play.ru playfortuna-play.ru
pliks.pl ploenjitmedia.azurewebsites.net
poffet.net
poisk-zakona.ru
poiskzakona.ru
polcin.de polcin.de
porn-w.org polimga.pw
polybuild.ru
porno-home365.com porno-home365.com
pornogig.com pornodojd.ru
pornoklad.net pornpost.in
pornonik.com powenlite24.ru
pornoslive.net predmety.in.ua
pornowarp.info preventheadacheguide.info
porodasobak.net princeadvantagesales.com
pricheskaonline.ru printingpeach.com
produkto.net privacyassistant.net
progonrumarket.ru profitwithalex.info
progress-upakovka.ru prokotov.com
qor360.com py100.ru
quangcaons.com pyrodesigns.com.au
queerspace.com quickloanbank.com
qwesa.ru randalljhoward.com
rapidsites.pro realnye-otzyvy.info
real-time-analytics.com recipedays.ru
realresultslist.com registry-cleaner.net
realting-moscow.ru remont-comp-pomosh.ru
realtytimes.com responsive-test.net
rebrand.ly ric.info
refudiatethissarah.info riders.ro
registrysweeper.com ritlweb.com
reimageplus.com rixpix.ru
reining.lovasszovetseg.hu saecsa.co
resellerclub.com sakhboard.ru
rfserial.net sc-specialhost.com
ricorsogiustizia.org scanhub.ru
rn-to-bsn.com scanner-elena.top
robot-forex.biz se-welding.ru
roofers.org.uk search.alot.com
royalcar-ufa.ru search.pch.com
rubanners.com searchinterneat-a.akamaihd.net
rubbed.us searchtooknow-a.akamaihd.net
russintv.fr seotoolsagency.com
rutor.vip
rvi.biz
s8-nowy-wygraj.comli.com
sa-rewards.co.za
salut-camp.ru
salutmontreal.com
saneyes.com
savingsslider-a.akamaihd.net
scanner-mark.top
scanner-nelson.top
search.1and1.com
seksotur.ru
semp.net
seoholding.com
seomarketings.online
seonetwizard.com
serpstat.com serpstat.com
servethis.com servethis.com
servingnotice.com serw.clicksor.com
sexfreepornoxxx.com sexs-foto.top
sexkvartal.com shariki-zuma-lines.ru
sexspornotub.com shivafurnishings.com
sexy-screen-savers.com sibecoprom.ru
shell-pmr.ru sideeffectsoftizanidine.blogspot.com
shopperifymac.com sinestesia.host.sk
shopsellcardsdumps.com sisi-go.ru
shtora66.ru sitebeam.net
signoredom.com smailik.org
simply.net smart-balancewheel.com
siteonomy.com social-fun.ru
slavkokacunko.de social-s-hhh.xyz
smartbalanceworld.com socialbookmarksubmission.org
smartpet.ru sofit-dmd.ru
snip.to softtor.com
soc-proof.su sortthemesitesby.com
sockshares.tv srgwebmail.nl
softomix.com
softxaker.ru
sonnikforme.ru
sophang8.com
spaceshipad.com
spb.ru
sps-shop.com
srecorder.com
stairliftstrue.com stairliftstrue.com
stal-rulon.ru start.myplaycity.com
stretchingabuckblog.com staynplay.net
stromerrealty.com steebook.com
sucsesofinspiration.com stjamesschool.info
survival.betteroffers.review stopnarco.ru
susanholtphotography.com streetfooduncovered.com
tanieaukcje.com.pl sugarkun.com
tattoo33.ru supermodni.com.ua
tech4master.com supervesti.ru
telefonsexkostenlos.tk surfbuyermac.com
testingads.pro syndicate.fun
tfxiq.com tackletarts.co
thebestphotos.eu takeprofitsystem.com
thefotosgratis.eu tandvardshuset.net
timdreby.com taqplayer.info
tattomedia.com
teastory.co
telemetryverification.net
thebluenoodle.com
thedownloadfreeonlinegames.blogspot.com
themeforest.net
theporndude.com
thfox.com
tiandeural.ru
tizanidine4mgstreetprice.blogspot.com
tizanidine4mgtablets.blogspot.com tizanidine4mgtablets.blogspot.com
tizanidinecapsules.blogspot.com tizanidinecost.blogspot.com
tizanidinehcl4mgtablet.blogspot.com
tizanidineinfo.blogspot.com
tizanidinesleepaid.blogspot.com tizanidinesleepaid.blogspot.com
tobeyouday.win tn811.us
top250movies.ru tnaionline.org
toxicwap.com topmira.com
toy-shop.top toy-shop.top
track112.site tpu.ru
trafficjunky.net trackmedia101.com
trafficnetzwerk.de trafficfactory.biz
traffmonster.info traffictrade.life
truck-land.life tramadolandtizanidine.blogspot.com
tsan.net trion.od.ua
tupper-shop.ru uhod-za-sobakoj.ru
turizm.bz unisexjewelry.org
tv-spoty.info unrealcommander.org
tvnewsclips.info uptime-delta.net
tvteleport.ru uptime.com
u17795.netangels.ru url-extractor.xyz
uchil.net
uggbootsoutletsale.us
uggsale.online
uhdtv.website
ultramart.biz
unece.org
unimodemhalfduplefw.pen.io
url2image.com url2image.com
urlopener.com validdomain.xyz
uzporno.mobi varikozdok.ru
vedomstvo.net veerotech.com
ventelnos.com velobikestock.com
vilingstore.net video-chat.cn
vip.51.la videochat.cafe
videochat.life
videospornogratisx.net
visa-china.ru visa-china.ru
vladimir.zrus.org vita.com.hr
volgograd.xrus.org viveresaniesnelli.it
volume-pills.biz voloo.ru
vroze.com vremya.eu
vuclip.com vvpg.ru
vut.com.ru vzlomfb.com
vzlomtw.com vzlomtw.com
wallpapersdesk.info w-journal.ru
website-analyzer.info watchdogs-2.ru
weightatraining.com watchmyfb.pl
wholesalenfljerseys.us.com wcb.su
williamrobsonproperty.com web-analytics.date
wineration.com web-betting.ru
winterclassichockeyjerseys.com webmasters.stackexchange.com
worldinternetauthority.com wedding0venues.tk
www.bookmaker-bets.com wejdz-tu.pl
wesharepics.com
where-toget.com
whereiskentoday.com
while.cheapwebsitehoster.com
wieseversa.no
wildcattube.com
wordpresscore.com
www.get-free-traffic-now.com www.get-free-traffic-now.com
xclicks.net www.kabbalah-red-bracelets.com
xedserver.com wyniki-lista.pl
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xboxster.ru
xn----btbdvdh4aafrfciljm6k.xn--p1ai xgames-04.com
xn--80aafb2a.xn--p1ai xitjw.info
xn--80aanaardaperhcem4a6i.com xkaz.org
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn--1-8sbcpb0bdm8k6a.xn--p1ai
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--80ab4aa2g.xn--p1ai
xn--80abgj3a5acid6ghs.top
xn--80aeahghtf8ac5i.xn--p1ai xn--80aeahghtf8ac5i.xn--p1ai
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai xn--h1ahbi.com.ua
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai xportvusbdriver8i.snack.ws
yadro.ru xxx-cam.webcam
yeartwit.com xxxmania.top
yellowproxy.net yachts-cruise.info
yottos.com yourothersite.com
youghbould.wordpress.com yourserverisdown.com
youtube-downloader.savetubevideo.com youtubologia.it
ypmuseum.ru yugo-star.ru
zahvat.ru yun56.co
zaim-pod-zalog-krasnodar.ru zaloro.com
zaimite.ru zhacker.net
zmoda.hostreo.com zygophyceous.womanstars.site
zoosexart.com

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123movies.love 01apple.com
12masterov.com 1stat.ru
1qingdao.com 3weekdiet.com
3w24.com 72-news.com
45en.ru 8558l.jobs.net
aa14ab57a3339c4064bd9ae6fad7495b5f.com 8gold.com
aa625d84f1587749c1ab011d6f269f7d64.com aa8780bb28a1de4eb5bff33c28a218a930.com
acmebtn.ml abbanreddy.com
adloads.net adanih.com
adobereader-free.ru adloads.com
adssafeprotected.com adman.gr
akvamaster.dp.ua adnotbad.com
alfa9.com ads-seo.men
alibestsale.com air-edem.ru
alloysteel.ru airmaxshoes-2016.com
amazingninja.com alborzan.com
anaksma.info anal-acrobats.com
app-ready.xyz animali.deagostinipassion.it
apparel-offer.com apispringsmartne-a.akamaihd.net
applyneedy.xyz arate.ru
asrv-a.akamoihd.net ardimobileinfo.ml
ausergrubhof.info arquapetrarca.info
availit.weebly.com arquivo.pt
avtolombard-krasnodar.com asscenihotosidea.blogspot.com
avtolombard-krasnodar.ru atdedinotuho.tk
autoua.top
autowebmarket.com.ua
avtoarenda.by
azbukafree.com azbukafree.com
bbtec.net azinoofficial777.ru
berdasovivan.ru bannertesting.com
bfz.biz bestcoin.cc
bigpenisguide.com bestpriceninja.com
bio-market.kz bird1.ru
bitcoinpile.com blogig.org
blackcurranthumidifiers.site bookmaker-bet.com
board.f00d.de boostmyppc.com
boleznikogi.com buyessaynow.biz
bonux.nextview.ru
browsepulse-a.akamaihd.net
buttons-for-your-website.com
buzz-porno.info
callejondelpozo.es callejondelpozo.es
cartechnic.ru candlehandmade.com
cartierlove2u.xyz capture-room.com
cementaresearch.se carrauterie.be
christianlouboutinoutlet.win casablancamanor.co.za
christianlouboutinsaleoutletonline.us cenokos.ru
ciproandtizanidine.blogspot.com chcu.net
clickbank.net check-host.net
clickzzs.nl chocolateslim-en-espana.com
compliance-alexa.top circlesl.com
compliance-brian.top clickhype.com
compliance-don.top compliance-george.top
compliance-elena.xyz cookie-law-enforcement-ee.xyz
contentsexpress.com counter.bloke.com
coop-gamers.ru creditmoney.com.ua
copyrightclaims.org cscwtalkto.site
cordstrap.cc custodieva.ru
countbertwistdisp26.soup.io daymusam.com
crazyprotocol.com ddpills.com
crest3d.ru dimaka.info
damedingel.ya.ru dimkino.ru
delfin-aqua.com.ua discovertreasurenow.com
detki-opt.ru driving.kiev.ua
detskie-zabavi.ru e-c.al
dividendo.ru
divisioncore.com
domainsphoto.com
eets.net eets.net
el-nation.com elektronischezi.livejournal.com
elidelcream.weebly.com elektrozigarette2.webs.com
elitepcgames.com eluxer.net
empis.magix.net elvenmachine.com
enge-fotzen.info enbersoft.com
eropornosex.ru eralph.tk
esfchat.tk excaliburfilms.com
fashionindeed.ml exoneration-project.us
felizporno.com ezigarettenkaufen1.pagina.gr
findacheaplawyers.com feel-planet.com
flashlarevista.com flytourisme.org
frbizlist.com fortwosmartcar.pw
free-laptop-reward.com forum20.smailik.org
freewebs.com fun2cell.net
g33.org fxgallery.com
gagrasector.ru gallictures.com
gallery.rennlist.com gamerscorps.com
gambarkataku.co
gamevalue7.weebly.com
ge0ip.net ge0ip.net
generalporn.org get-seo-domain.com
gigixo.com habermetre.com
go2album.com hackers-crackers.tk
gosreg.amchs.ru hcate.com
guod.me healgastro.com
handicapbathtubarea.com hidefiles.org
handicapvansarea.com hitsbox.info
hatedriveapart.com hoverboard360.es
hawaiielectriclight.com howlongdoestizanidinestayinyoursystem.blogspot.com
hd-film.pl htmlcorner.com
hdseriale.pl https-legalrc.biz
health-medical-portal.info iinstalll-fii1leis.jus0wil.pp.ua
heartofbeijing.blogspot.com ikearugs.xyz
hermesreplica.pw ilmexico.com
hitcpm.com imicrovision.com
hotelcrocenzi.sm importchinacoach-teach.com
hyhj.info indianmedicaltourismshop.com
ideashome.id inet-traffic.com
ilte.info intellekt21.ru
impisr.edunsk.ru internet-apteka.ru
inbabes.sexushost.com invitefashion.com
int.search.tb.ask.com ip-guide.com
interesnie-faktu.ru iphantom.com
itservicesthatworkforyou.com ipornox.xxx
jbl-charge.info jerseywholesalechinabiz.com
kanimage.com jofucipiku.tk
kidskunst.info kicknights.gq
kinopolet.net
kinosed.net
kiprinform.com
kongoultry.net
konkursov.net
kostenlos-sexvideos.com kostenlos-sexvideos.com
krasnodar.xrus.org kretpicf.pw
labafydjxa.su kskjco.club
laexotic.com landmania.ru
lalalove.ru landscaping.center
letsart.ru laulini.soclog.se
live-sexcam.tk lechtaczka.net
lollypopgaming.com ledis.top
lovi-moment.com.ua likrot.com
lsex.xyz lolnex.us
luchshie-topcasino.ru lomb.co
lyrics.home-task.com lowephotos.info
m5home.ru lunamedia.co
mansiondelrio.co macdamaged.tech
marketingvici.com makedo.ru
matb3aa.com managerpak204.weebly.com
meendo-free-traffic.ga manimpotence.com
megakino.net marmotstore.online
meltwater.com masterseek.com
message-warning.net masthopehomes.com
metascephe.com meccadumps.net
microsearch.ru megagrabber.ru
minecraft-neo.ru member-quiz.com
moncleroutletonline.win menetie.ru
more-letom.ru mere.host.sk
movies-in-theaters.net micasainvest.com
mrinsidesales.com mixtapetorrent.com
myfreemp3.eu morocco-nomad-excursions.com
myhealthcare.com morpicert.pw
myonigroup.com muizre.ru
netanalytics.xyz myindospace.com
newstraveller.ru mysexpics.ru
niroo.info mzdish.site
nucia.biz.ly narkologiya-voronezh.ru
officedocuments.net nextconseil.com
ok.ru ningessaybe.me
onlainbesplatno.ru nowtorrents.com
open-odyssey.org o.light.d0t.ru
paidonlinesites.com ochistka-stokov.ru
palocco.it odoratus.net
panouri-solare-acoperis.com office2web.com
partnersafe.men onthemarch.co
pdamods.ru ooo-olni.ru
picturesify.com osnova3.ru
pinup-casino1.ru page2rss.com
pisanieprac.info panamaforbeginners.com
playtap.us parajumpersjakkesalgnorge.info
plohaya-kreditnaya-istoriya.ru partner-pop.men
pochemychka.net parvezmia.xyz
pomoc-drogowa.cba.pl pathwhelp.org
predominant-invent.tk pcgroup.com.uy
prointer.net.ua pinwallpaper.top
prostitutki-kharkova.org plastjulgranar.se
py100.ru pogosh.com
reklamuss.ru porndairy.in
renewablewealth.com pornoblood.com
ringporno.com pornobrazzers.biz
rixpix.ru pornokajf.com
ruscopybook.com preg.marketingvici.com
sad-torg.com.ua preparevideosafesystem4unow.space
sadaholding.com pro-tec.kz
profitfx.online
proftests.net
prostitutki-tyumeni.org
proxyelite.biz
quickcashlimited.com
quickloanbank.com
rimedia.org
rocketchange.ru
rp9.ru
running-line.ru
sa-live.com
sammyweaver.com
sarm.tk sarm.tk
scanner-barak.top sawin.beth.webd.pl
scanspyware.net search-error.com
scat.porn seo-2-0.com
serialsway.ucoz.ru seo-traffic-ranking.info
sexreliz.com seorussian.ru
shemale-sex.net sex-tracker.de
shopperifymac.com sexstream.pl
shopvilleroyboch.com.ua share-buttons-for-free.com
signx.info shikiso.info
soc-proof.su shoppingjequiti.com.br
socialmadesimple.com shopwme.ru
socialseet.ru sladkoevideo.com
stickers-market.ru snomer1.ru
stoki.ru snworks.com
stpolice.com solicita.info
strangeduckfilms.com songoo.wz.cz
stroymonolit.su spyfu.com
studiokamyk.com.pl spywarenuker.com
superlist.biz stuff-about-money.com
superstats.com swimpool.ca
surintech.ac.th taihouse.ru
svbur.ru taqplayer.info
svtrd.com taraz.xkaz.org
syndicate.fun telesvoboda.ru
takeprofitsystem.com
td-33.ru
tedxrj.com
telegramdownload10.com
tengohydar.tk
terrafootwear.us
texbaza.by
tgtclick.com tgtclick.com
thedownloadfreeonlinegames.blogspot.com theplacetoupdating.pw
thegioixekhach.com timeallnews.ru
themeforest.net tizanidinecost.blogspot.com
themestotal.com tizanidinerecreationaluse.blogspot.com
thetoiletpaper.com tizanidinesleepaid.blogspot.com
tizanidinedrugtest.blogspot.com tizanidinestreetprice.blogspot.com
trafficjunky.com tjkckpytpnje.com
trafficnetzwerk.de tongkatmadura.info
traffique.net toon-families.com
trudogolik.net topkarkas.com
tx41tclega.ru toys.erolove.in
ukrobstep.com tradgardspartner.se
ukrtextbook.com trafaret74.ru
uptime-gamma.net twittruth.com
usdx.us uggsale.online
vkontaktemusic.ru unrealcommander.biz
vladhistory.com uroki.net
voloslove.ru valuado.com
voluumtrk.com vellings.info
voronezh.xrus.org via-energy-acquistare.com
vzlomsn.org vkgaleria.com
wallpaperaccess.com voluumtracker1.com
webcamdevochka.com vseuznaem.com
webcamtalk.net vtc.pw
webshoppermac.com vzglyadriv.kg
wfb.hatedriveapart.com vzlomtw.com
wineration.com wallpapersdesk.info
winterclassichockeyjerseys.com webmasterhome.cn
woweb.com.ua weboptimizes.com
www.888.com webpromotion.ae
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai wordpresscore.com
xn----ctbbcjd3dbsehgi.xn--p1ai wowcasinoonline.ooo
xn--c1acygb.xn--p1ai www.jbetting.com
zazagames.org xgftnlrt.bloger.index.hr
zhuravlev.info xmlinde.com
zoosexart.com xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
zzlgxh.com xn----itbeirbjbi7bc6bh2d.xn--p1ai
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
xwatt.ru
yaminecraft.ru
youtoner.it
znaturaloriginal.com
zskdla.site

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.2 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3490 ### Version: V4.2022.11.3491
### Updated: Thu Nov 24 22:01:16 UTC 2022 ### Updated: Fri Nov 25 10:03:13 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3490 ### Version: V4.2022.11.3491
### Updated: Thu Nov 24 22:01:16 UTC 2022 ### Updated: Fri Nov 25 10:03:13 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##