vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 15:34:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3493 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-26 10:09:13 +00:00
parent 2664cf8cd9
commit c55d175c5f
47 changed files with 95175 additions and 95172 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3492 #### Version: V4.2022.11.3493
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3492 #### Version: V4.2022.11.3493
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3492 #### Version: V4.2022.11.3493
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11758
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3492 #### Version: V4.2022.11.3493
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11762
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11762
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11762
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11762
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.37 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.37 0;
127.0.0.1 0; 127.0.0.1 0;

11762
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.37 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.37 0;
127.0.0.1 0; 127.0.0.1 0;

11762
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.37 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.37 0;
127.0.0.1 0; 127.0.0.1 0;

11762
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3492 3493

11762
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11762
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.174.69.167 0; 20.236.131.43 0;

11762
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.37 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.37 0;
127.0.0.1 0; 127.0.0.1 0;

11762
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.37 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.37 0;
127.0.0.1 0; 127.0.0.1 0;

11762
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.37 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.37 0;
127.0.0.1 0; 127.0.0.1 0;

11762
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.174.69.167 0; 20.236.131.43 0;

11762
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.174.69.167 0; 20.236.131.43 0;

11762
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.37 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
FHscan Anarchy
GrapeFX Cliqzbot
Image Sucker CyotekWebCopy
JamesBOT RankActiveLinkBot
Lipperhey Ripper
NextGenSearchBot Steeler
PictureFinder Webalta
Probethenet cognitiveseo
lwp-trivial
sexsearcher sexsearcher
siteripz

306
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 360Spider
404checker
80legs 80legs
ADmantX
AIBOT
ALittle Client ALittle Client
ASPSeek
Abonti
Aboundex
Aboundexbot Aboundexbot
AhrefsBot Acunetix
Aipbot AfD-Verbotsverfahren
Alexibot
AllSubmitter AllSubmitter
Alligator
Anarchy99 Anarchy99
Ankit Ankit
Atomseobot Anthill
BBBike Apexoo
BDFetch Asterias
BLEXBot Backlink-Ceck
BackDoorBot Badass
BackStreet Bandit
BackWeb
Barkrowler
BatchFTP BatchFTP
Bigfoot BetaBot
Bitacle Bitacle
Black Hole Black Hole
Blackboard Blackboard
BuiltWith Bolt
BunnySlippers BotALot
Bullseye
CODE87
CSHttp CSHttp
Calculon Calculon
Cegbfeieh Cegbfeieh
CensysInspect
CheTeam CheTeam
Claritybot
Cliqzbot Cliqzbot
Collector Cosmos
Copier
CopyRightCheck
Copyscape
Craftbot Craftbot
Crawling at Home Project Custo
CrazyWebCrawler DBLBot
CrunchBot DataCha0s
CyotekWebCopy Devil
DatabaseDriverMysqli DittoSpyder
Digincore
DigitalPebble
Disco
Discobot
Discoverybot
Dispatch
DnyzBot
DomCopBot
DomainStatsBot DomainStatsBot
Domains Project Dotbot
Download Wonder ECCP/1.0
Dragonfly EMail Wolf
EMail Siphon
EasyDL EasyDL
Ebingbong Ecxi
EirGrabber Exabot
Express WebPictures Express WebPictures
ExtLinksBot
Extractor Extractor
Ezooms Extreme Picture Finder
Firefox/7.0
Flunky Flunky
Fyrebot GermCrawler
GT::WWW GetRight
GalaxyBot GetWeb
Genieo Go!Zilla
Getintent Gotit
GrabNet
Grabber Grabber
GrapeshotCrawler Grafula
HTMLparser GridBot
HTTP::Lite
HTTrack HTTrack
HaosouSpider Havij
HonoluluBot Hloader
Humanlinks Humanlinks
HybridBot
IDBTE4M
IDBot IDBot
IRLbot
Iblog
Image Fetch
Image Sucker
IndeedBot IndeedBot
InfoNaviRobot InfoNaviRobot
Intelliseek InfoTekies
InternetSeer InterGET
IstellaBot Internet Ninja
JamesBOT Iria
Jbrofuzz Iskanie
JennyBot
JikeSpider
Joomla Joomla
Jorgee
JustView JustView
Jyxobot Jyxobot
Keyword Density Keyword Density
LNSpiderguy Kinza
LWP::Simple
Larbin Larbin
LeechGet LibWeb
Libwhisker
LieBaoFast
Lightspeedsystems
Likse Likse
LinkScan LinkWalker
LinkextractorPro Linkbot
LinksManager LinkpadBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey Lipperhey
MIDown tool Litemage_walker
MIIxpc Lmspider
MSFrontPage Ltx71
MTRobot Masscan
Majestic SEO Mata Hari
MauiBot
Meanpathbot Meanpathbot
MegaIndex.ru
MicroMessenger MicroMessenger
Microsoft URL Control Microsoft URL Control
Moblie Safari
Mojeek Mojeek
Mozlila Mojolicious
Mr.4x3 MolokaiBot
NICErsPRO
NPbot NPbot
Nameprotect
Net Vampire Net Vampire
NetLyzer Netcraft
NetZIP
Netvibes Netvibes
NextGenSearchBot
Nibbler Nibbler
NimbleCrawler NimbleCrawler
Ninja Ninja
Nmap Nmap
Nutch Not
Offline Explorer Offline Navigator
OnCrawl OpenLinkProfiler
OpenVAS OpenVAS
Openvas Openfind
OrangeBot
OrangeSpider OrangeSpider
PECL::HTTP OutfoxBot
PageGrabber PHPCrawl
Panscient POE-Component-Client-HTTP
PageThing.com
Papa Foto Papa Foto
Pavuk Pavuk
PeoplePal PeoplePal
Pi-Monster
Picscout Picscout
Picsearch
PictureFinder
Pimonster Pimonster
Pixray
ProPowerBot ProPowerBot
ProWebWalker
Psbot Psbot
Quick-Crawler Pump
RankActiveLinkBot PxBroker
RSSingBot
RankActive
RankingBot RankingBot
RankingBot2 Rankivabot
SEOprofiler Reaper
SEOstats RedesScrapy
SalesIntelligent RepoMonkey
ScanAlert Ripper
RocketCrawler
Rogerbot
SBIder
SEOkicks-Robot
SEOlyticsCrawler
SISTRIX
SMTBot
Scrapy
Screaming Screaming
Searchestate
SearchmetricsBot
Seekport
SemanticJuice
SemrushBot SemrushBot
SeoSiteCheckup Shodan
Seomoz Site Sucker
SiteCheckerBotCrawler SiteSucker
SiteExplorer
SiteLockSpider
Siteimprove Siteimprove
Snake SlySearch
Sociscraper SocialRankIOBot
Sosospider Sosospider
Spanner Spammen
Spbot Spbot
Spinn3r Spinn3r
SputnikBot
Sqlmap
Steeler
Stripper
Sucker
Sucuri Sucuri
Surfbot SuperBot
SurveyBot
Suzuran
T0PHackTeam T0PHackTeam
T8Abot
Teleport
Telesoft Telesoft
Telesphorep
The Intraformant The Intraformant
Toweyabot Toweyabot
Tracemyfile Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot Turingos
Turnitin
Twice Twice
Typhoeus
URLy Warning
UnisterBot
Upflow
V-BOT V-BOT
Vacuum VB Project
VCI
Vagabondo
VeriCiteCrawler VeriCiteCrawler
Virusdie Virusdie
VoidEYE
Voil Voil
Voltron
WASALive-Bot
WPScan WPScan
Wallpapers/3.0 WallpapersHD
Web Auto WeSEE
Web Sauger Web Collage
Web Fetch
Web Fuck
Web Sucker
WebEnhancer WebEnhancer
WebImageCollector WebFuck
WebReaper WebGo IS
WebSauger
WebStripper WebStripper
WebWhacker WebZIP
Website Quester Webalta
WebsiteExtractor Whatweb
WebsiteQuester
Whack
Whacker
Widow
WinHTTrack
Wonderbot
Woobot
Xenu
YoudaoBot YoudaoBot
Zade
Zauba Zauba
Zermelo
Zeus Zeus
Zitebot
ZoominfoBot ZoominfoBot
arquivo-web-crawler ZyBorg
arquivo.pt
autoemailspider autoemailspider
backlink-check backlink-check
clark-crawler clark-crawler
coccocbot
cognitiveseo cognitiveseo
crawler.feedback crawl.sogou.com
crawler4j
dataforseo.com
demandbase-bot
domainsproject.org domainsproject.org
eCatch
evc-batch evc-batch
gopher heritrix
instabid
ips-agent
isitwp.com
iubenda-radar
linkdexbot linkdexbot
lwp-trivial
magpie-crawler
meanpathbot meanpathbot
mediawords netEstate NE Crawler
oBot oBot
page scorer pcBrowser
ripz
s1z.ru
scan.lol scan.lol
seobility seoscanners
seostar
serpstatbot serpstatbot
sitechecker.pro sitechecker.pro
sogouspider siteripz
sysscan
trendiction.com trendiction.com
trendiction.de ubermetrics-technologies.com
webmeup-crawler
webpros.com webpros.com
webprosbot zgrab

274
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404enemy
80legs
AIBOT
Aboundex Aboundex
Aboundexbot Aboundexbot
AhrefsBot AhrefsBot
Alexibot
Alligator Alligator
Anthill
Aspiegel Aspiegel
Asterias
Atomseobot
Attach Attach
AwarioSmartBot
BDCbot BDCbot
BLEXBot BDFetch
BackWeb
Backlink-Ceck Backlink-Ceck
Badass Bandit
Battleztar Bazinga Battleztar Bazinga
BetaBot
Bigfoot Bigfoot
Bitacle Bitacle
BlackWidow
Blackboard Blackboard
Blow
Bolt
Brandprotect Brandprotect
Brandwatch Brandwatch
Buck BuiltBotTough
Buddy BuiltWith
Bullseye
BunnySlippers BunnySlippers
CATExplorador CATExplorador
CCBot CODE87
Cegbfeieh CSHttp
CheTeam
CheeseBot CheeseBot
CherryPicker CherryPicker
Claritybot Citoid
Cliqzbot Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot
CopyRightCheck
Copyscape Copyscape
Craftbot Craftbot
CyotekWebCopy Crawling at Home Project
CrazyWebCrawler
Crescent
CrunchBot
DBLBot
DIIbot
DSearch DSearch
DataCha0s DTS Agent
DatabaseDriverMysqli DatabaseDriverMysqli
Devil Demon
Deusu
Digincore
Dirbuster
Disco
Dispatch
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DomCopBot
DomainAppender DomainAppender
DomainCrawler DomainCrawler
Domains Project Drip
Download Wonder
Dragonfly
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
EasyDL
Ebingbong
Ecxi
ExtLinksBot ExtLinksBot
FemtosearchBot ExtractorPro
FlashGet EyeNetIE
Flunky Fimap
FrontPage Freeuploader
GT::WWW Fuzz
Fyrebot
Genieo
GetRight
GetWeb GetWeb
Gigabot
Go!Zilla Go!Zilla
GoZilla
Gotit
GrapeFX
GrapeshotCrawler GrapeshotCrawler
GridBot Hloader
HMView Humanlinks
Haansoft Iblog
Harvest Image Fetch
Heritrix InfoNaviRobot
IDBTE4M
Id-search
IndeedBot
Indy Library
InfoTekies InfoTekies
Intelliseek Intelliseek
InterGET InterGET
InternetSeer Iria
Iskanie
IstellaBot IstellaBot
Jbrofuzz Jbrofuzz
JennyBot JikeSpider
Jetty
Joomla Joomla
JustView
Jyxobot
Kinza Kinza
LNSpiderguy LNSpiderguy
LWP::Simple LWP::Simple
Larbin LeechFTP
Leap Likse
LeechGet LinkWalker
LibWeb
LinkScan
Linkbot
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lipperhey
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MIIxpc
MSFrontPage
MSIECrawler
MTRobot MTRobot
Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic SEO
Majestic-SEO Majestic-SEO
Majestic12 Majestic12
MarkMonitor
MarkWatch MarkWatch
Mass Downloader Mass Downloader
Masscan Masscan
Mata Hari Mata Hari
Mediatoolkitbot MegaIndex.ru
Microsoft Data Access Metauri
Microsoft URL Control
Mojeek Mojeek
Mozlila MolokaiBot
Morfeus Fucking Scanner
Mr.4x3 Mr.4x3
Navroad
NearSite
Needle Needle
Nessus Nessus
Net Vampire Net Vampire
NetAnts NetAnts
NetLyzer NetSpider
NetMechanic NetZIP
Netcraft
Nettrack
Niki-bot Niki-bot
Nikto Nimbostratus
Not Ninja
Nmap
Octopus
Offline Explorer Offline Explorer
Offline Navigator
OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
Openfind OrangeSpider
OrangeBot OutfoxBot
PECL::HTTP
PHPCrawl PHPCrawl
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageAnalyzer PageGrabber
PageThing.com PageThing.com
Pandalytics
Panscient Panscient
Pi-Monster Pi-Monster
Picsearch Picscout
Pixray
PleaseCrawl PleaseCrawl
Psbot Pockey
Pump ProPowerBot
PyCurl ProWebWalker
QueryN Metasearch Probethenet
RSSingBot Pu_iN
RankingBot Quick-Crawler
Rankivabot RankFlex
RankurBot RankurBot
Re-re
ReGet
RealDownload
Reaper
RebelMouse
Recorder Recorder
RedesScrapy
RepoMonkey
RocketCrawler
Rogerbot Rogerbot
SMTBot SEOprofiler
Scanbot Scanbot
ScoutJet ScrepyBot
Searchestate SearchmetricsBot
Seekport Seekport
SemanticJuice SemrushBot
SentiBot Site Sucker
SeoSiteCheckup
SeobilityBot
Seomoz
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteSnagger SiteLockSpider
Sitebeam SiteSucker
Sitevigil Sitevigil
SmartDownload Snoopy
SocialRankIOBot Spanner
SpankBot
Spbot Spbot
Sqlmap Steeler
Sucker
Sucuri Sucuri
SuperHTTP SuperBot
SurveyBot
Swiftbot Swiftbot
Szukacz
T0PHackTeam T0PHackTeam
T8Abot Telesoft
Telesphoreo Telesphoreo
TheNomad Telesphorep
Toata Thumbor
Tracemyfile
Trendiction Trendiction
True_Robot
Turingos
Turnitin Turnitin
TwengaBot TwengaBot
Twice V-BOT
VB Project
VCI VCI
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper WWW::Mechanize
Virusdie Wallpapers/3.0
WBSearchBot WallpapersHD
WEBDAV
WISENutbot
WPScan
WWW-Collector-E
WWW-Mechanize
WeSEE WeSEE
Web Fetch
Web Fuck Web Fuck
Web Sucker
WebBandit WebBandit
WebCollage WebCollage
WebEnhancer
WebFetch WebFetch
WebImageCollector WebImageCollector
WebLeacher WebSucker
WebReaper
WebSauger
WebZIP
Webshag Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
Widow Whacker
WiseGuys Robot WiseGuys Robot
Wonderbot Woobot
Wotbox Xaldon_WebSpider
YoudaoBot YoudaoBot
Zauba Zauba
Zeus Zeus
Zitebot
ZmEu
ZoomBot
ZoominfoBot
archive.org_bot
arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider
check1.exe check1.exe
cognitiveseo cognitiveseo
com.plumanalytics crawl.sogou.com
domainsproject.org domainsproject.org
eCatch eCatch
facebookscraper heritrix
gopher
internetVista monitor internetVista monitor
ips-agent ips-agent
isitwp.com isitwp.com
lwp-trivial iubenda-radar
lwp-request
magpie-crawler magpie-crawler
oBot meanpathbot
polaris version polaris version
s1z.ru probe-image-size
satoristudio.net satoristudio.net
scan.lol scalaj-http
seoscanners siteripz
seostar
sexsearcher
sitechecker.pro
sogouspider
spyfu
tAkeOut tAkeOut
trendiction.com
webgains-bot
webmeup-crawler webmeup-crawler
webpros.com
webprosbot
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
alphavisions.net 202ch.com
bigpenisguide.com 999webdesign.com
dimaka.info cjs.com.ru
european-torches.ru ff30236ddef1465f88547e760973d70a.com
hkladys.com homeinns.com
jaxcube.info lyngdalhudterapi.no
kupiproday.com.ua nmrk.ru
remontvsamare.su podshipniki-ntn.ru
savetubevideo.com sextracker.de
usswrite.com sk.golden-praga.ru

940
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00it.com 0912701309f8ce.com
03p.info 12bet.com
11pikachu.ru 1hwy.com
18ps.ru 2000k.ru
2015god.org 202ch.com
2728fb936f0.com 24-ak.ru
273-fz.ru 4kplayer.pl
2ads.co.uk 5000-cotydzien.com
2pxg8bcf.top 5u.com
3weekdiet.com 72-news.com
4istoshop.com 8558l.jobs.net
4kepics.com DomainStatsBot
4webmasters.com abcdefh.xyz
a14download.com absugars.com
a96527gi.beget.tech acmebtn.ml
aa14ab57a3339c4064bd9ae6fad7495b5f.com activepr.ru
aa625d84f1587749c1ab011d6f269f7d64.com adamoads.com
adclickservice.com adcash.com
adexprts.com adexprts.com
admeasures.com adnotbad.com
adsref.men advancedsoftwaresupport.com
adultactioncam.com agapovdg.ru
adultmeetonline.info ahmedabadwebs.com
adultnet.in aibolita.com
advokat-grodno.by alborzan.com
adzerg.com alekseevec.ru
afora.ru alf-img.com
agroeconom.kz all4invest.info
algarveglobal.com alloysteel.ru
alibestsale.com amazingpic.net
alldezservice.kz anal-acrobats.hol.es
amospalla.es
ananumous.ru ananumous.ru
animalia-life.club apccargo.com
animebox.com.ua apibatbrowsecom-a.akamaihd.net
apartmentbay.ru appleid-verification.com
apessay.com approved.su
apibrowsesmartne-a.akamaihd.net arabsexxxtube.com
apimossnetinfo-a.akamaihd.net arenanews.com.ua
applyneedy.xyz ascat.porn
approvedlocal.co.za asdfz.pro
artefakct.com asrvrep-a.akamaihd.net
asmxsatadriverin.aircus.com ausmepa.org.au
atmagroup.ru auto4style.ru
aucoinhomes.com
autobrennero.it autobrennero.it
autolombard-krasnodar.ru azbukafree.com
autotrends.today baladur.ru
avtoarenda.by balla-boo.se
azbuka-mo.ru bambi.ck.ua
bard-real.com.ua bank.uz
barnfurnituremart.com bankiem.pl
beauby.ru bdsmgalls.net
becuo.com belinvestdom.by
bestpornuha.com best-coupon-offer.com
bestssaker.com bestfortraders.com
bestwaystogetpaid.us betterscooter.com
betteroffers.review bibys.com
bg6s0.com bikini-image.com
big-boards.info blackplanet.com
biketank.ga blagovest-med.ru
bildsuche.ru blockworld.ru
bizlist.com.de blog.koreadaily.com
bjgugu.net.ua bluejays-jerseys.us
blavia.00author.com bluesman.nu
bmwhighperformers.com bmw.afora.ru
boanonihaca.tk bolitgorlo.net
bonux.nextview.ru
bradleylive.xyz
bristolhotel.com bristolhotel.com
bupropion-sr-150-mg.us burger-imperia.com
buy-meds24.com burger-tycoon.com
buybest1.biz businesxxl.com
buyk.host.sk buyessaynow.biz
buyparajumpers.online calstaterealty.us
buypillsorderonline.com calvet.altervista.org
cakemediahq.com.au cand.jp
candelluxsklep.pl candy-glam-hp.com
candytiner.com cartujano-pre.de
cartierlove.xyz cashonads.com
cartierlovestore.com casinopinup-wins.com
casablancamanor.co.za casinorewards.info
caulderwoodkennels.com cb.iphantom.com
cdncash.org cdnanalytics.xyz
cdnnetwok.xyz cheapmoncler.win
cete.ru checkhit.com
changde.58.com chlooe.com
cheapcigarettesc.info chuckguilford.com
check-this-out-now.online club-samodelkin.ru
children-learningreading.info
chinese-amezon.com
choice-credit.ru
christianlouboutinreplica.win
cmhomestayagency.com
coin-hive.com coin-hive.com
compliance-andrew.top colobit.biz
compliance-irvin.top com-onlinesupport.host
compliance-ivan.xyz compliance-nelson.top
contentsexpress.com compliance-olga.top
courtshipgift.com compliance-walt.top
covetnica.com coolwallpapers4k.info
creatives.adbetclickin.pink copypaste.traderzplanet.in
cubook.supernew.org country-chic.ru
customchocolate.business-for-home.com covadhosting.biz
customsua.com.ua credit.co.ua
damedingel.ya.ru crest3d.ru
data-ox.com crirussian.ru
date-withme.com cryptoswap.biz
ddospanel.com culturevie.info
de.zapmeta.com cupday.com
dealitright.click cutalltheshit.com
dear-diary.ru cxpromote.com
deda-moroza-zakaz.ru czeshop.info
delfin-aqua.com.ua decorationspcs.com
dengi-pod-zalog-nedvizhimosti.ru den-noch24.ru
descargar-musica-gratis.net dentalpearls.com.au
deshevo-nedorogo.ru detskie-konstruktory.ru
detmebel.su dev.citetick.com
detoxmed24.ru diamond-necklace.info
diegolopezcastan.com directivepub.com
digitalassetmanagement.site discovertreasurenow.com
dkvorota.ru dneprsvet.com.ua
donna7753191.ru documentsite.net
dood.live dosug-lux.ru
doreenblog.online dotnetdotcom.org
dostavka-v-ukrainu.ru drugstoreforyou.com
downloader12.ru dtnlyss.com
dtm-spain.com dverimegapolis.ru
duplicashapp.com dvr.biz.ua
dvrlists.com
e-avon.ru
e-biznes.info e-biznes.info
eas-seo.com e90post.com
easync.io ed-shop01.ru
easyshoppermac.com ekspresihati.info
ebonyporn.site eleimgo.pw
ekn-art.se elektrischeziga.livejournal.com
elektrischezi.canalblog.com elite-sex-finders.com
elektrozigaretten1.tumblr.com elkacentr.ru
elmacho.xyz eloxal.ru
empathica.com eluxer.net
energydiet24.ru empirepoker.com
englate.com enthuse.computernetworksonline.com
enskedesquashclub.se epngo.bz
envaseslotusama.com estelight.ru
erotik-kostenlos.net eugenevaultstorage.com
europages.com.ru europages.com.ru
evehemming.blogspot.com.au extlabs.io
event-tracking.com extrabot.com
everflora.ru extremepornos.net
exdocsfiles.com eyessurgery.ru
executehosting.com ezigaretteshop2.mywapblog.com
extener.org faithe.top
ezb.elvenmachine.com faptitans.com
f012.de farm26.ru
f5mtrack.com fba-mexico.com
fanpagerobot.com fifa-coins.online
fertilitetsradgivningen.se findclan.org
fil.ru findsexguide.com
filesclub.net finejewelryshop.ru
filmidivx.com flauntyoursite.com
findthe.pet flirt4free.com
firstdrugmall.ru floridamhca.org
flashlarevista.com folowsite.com
fm-upgrade.ru footbalive.org
funcrushgames.com fordsonmajbor.cf
fyl.com.ru forexgb.ru
g.starmoe.xyz formulaantiuban.com
gaming-journal.com forzeronly.com
gelezki.com foto-weinberger.at
get-your-social-buttons.info fotosfotos.eu
getmiro.com free-social-buttons.xyz
gobongo.info freewhatsappload.com
freshmac.space
futbolkisales.ru
gearcraft.us
geileweiber.tk
generalporn.org
generousdeal-a.akamaihd.net
get-free-social-traffic.com
getyourimage.club
goldandcard.ru
gombita.info
goodwinmetals.co
goroda-vsego-mira.ru
gpms.org.my
greamimgo.pw
greatfind-a.akamaihd.net greatfind-a.akamaihd.net
guardlink.com gribkovye-zabolevaniya.com
guiadeserraazul.com grizzlysgrill.com
gwebtools.com gsou.cf
haikuware.com gwebtools.com.br
hannasolution.ru hidefiles.org
hawaiielectriclight.com hledejvshopech.cz
hd-filmy.net hotenergy.ru
hdpixion.com hotshoppymac.com
help.tpu.ru houseofgaga.ru
homedo.fabpage.com houseofrose.com
homik.pw hoverboard360.nl
hosting-tracker.com howtoclean.club
housemilan.ru hptwaakw.blog.fc2.com
hscsscotland.com hscsscotland.com
hunthillfarmtrust.org hunterboots.online
hybrid.ru hut1.ru
ictizanidinehcl4mg.blogspot.com iblogpress.xyz
idc.com.ua ilovevitaly.xyz
ido3.com imagine-ex.co
imadedinner.net import-sales.com
images-graphics-pics.com impresagaia.it
imgata.com indiasourcemart.in
imk.com.ua infokonkurs.ru
impisr.ru intervsem.ru
indo-export.ru invest-pamm.ru
informatiecentro.be investmac.com
intellektmedia.at islamtoday.co.za
interesnie-faktu.ru
interior-stickers.ru
intim-uslugi.info
invivo.hu
iopeninghours.co.uk
iradiology.ru
iskalko.ru
isoveti.ru
istizanidinelikexanax.blogspot.com
itservicesthatworkforyou.com itservicesthatworkforyou.com
j33x.com japfm.com
jbl-charge.info javrip.net
jo24news.com javxxx18.com
johannesburgsingles.co.za jerseyschinabizwholesale.us
jongose.ninja jewelryandfiligree.com
jpcycles.com joingames.org
junketjuice.blogspot.com juliadiets.com
kaac.ru kakablog.net
katjimej.blog.fc2.com kambasoft.com
kazrent.com kasino-money.pw
kerwinandcariza.com katadhin.co
keywordsking.com keywesthideaways.co
keywordsuggest.org keywordhut.com
keywordsuggests.com keywordteam.net
kino-filmi.com kinomaniatv.pl
kino2018.club kinotorka.ru
kliksaya.com knigonosha.net
kndxbkdx.bloger.index.hr komputernaya-pomosh-moscow.ru
krassh.ru kotaku.com
kustanay.kz krasnodar.xrus.org
kuzinsp.ru kutikomi.net
kvartir-remont.biz kvartir-remont.biz
la-fa.ru lanadelreyfans.us
labelwater.se larutti.ru
labvis.host.sk leadn.pl
lanasshop.ru lemon-ade.site
landinez.co lexaprogeneric.link
laptop-4-less.com lightinghomes.net
lasvegaslockandsafe.com link.ac
lenvred.org linkredirect.biz
letsart.ru linksharingt.com
liky.co.ua liquimondo.com
liveu.infoteka.hu live-sexcam.tk
locksmith.jp live-sexchat.ru
lol-smurfs.com llastbuy.ru
lostfilm-online.ru longgreen.info
lsp-awak-perikanan.com longlifelomilomi.info
lunchrock.co lovasszovetseg.hu
lyrics.home-task.com ltvperf.com
mainlinehobby.net luckyshop.net.ua
mamasuper.prom.ua lyngdalhudterapi.no
masterseek.com m5home.ru
maxxtor.eu mabdoola.blogspot.com
mearns-tractors.co.uk macdamaged.space
medizinreisen.de magicalfind-a.akamaihd.net
medkritika.ru manyresultshub-a.akamaihd.net
megagrabber.ru marketland.ml
metabar.ru megaapteka.ru
methodsmarketing.com member-quiz.com
mex-annushka.ru mesbuta.info
mhi-systems.ru middlerush-a.akamaihd.net
mindbox.co.za midst.eu
minet.club mielec.pl
mini.7zap.com
mir-business-24.ru mir-business-24.ru
mlvc4zzw.space missclub.info
mmgq.ru moesen-ficken.com
moinozhki.com moomi-daeri.com
mojaocena.com moscow-clining.ru
monclervests.xyz moviemail-online.co.uk
morf.snn.gr mrmoneymustache.com
morocco-nomad-excursions.com muabancantho.info
moyakuhnia.ru my-big-family.com
mtmtv.info myhitmp3.club
mttwtrack.com mylovelibrabry.com
multgo.ru mymobilemoneypages.com
musezone.ru myperiod.club
muz-tracker.net
myiptest.com
myprintscreen.com
nac-bearings.ru
nalogovyykodeks.ru nalogovyykodeks.ru
nastroyke.net namecrumilchlet.tk
nationalbreakdown.com nardulan.com
networkcheck.xyz narkologiya-voronezh.ru
new7ob.com naughtyconnect.com
newstaffadsshop.club needtosellmyhousefast.com
newstraveller.ru net-profits.xyz
niki-mlt.ru nfljerseyscheapchinabiz.com
nootrino.com nl.netlog.com
noscrapleftbehind.co nlfjjunb5.ru
novatech.vn odoratus.net
novgorod.xrus.org of-ireland.info
novosti-hi-tech.ru officedocuments.net
npoet.ru ok.ru
ntic.fr okayimage.com
nuker.com onlainbesplatno.ru
nvformula.ru onlinadverts.com
nvssf.com onlinebay.ru
obiavo.kz
obsessionphrases.com
odesproperty.com
offer.camp
offergroup.info
office-windows.ru
offside2.5v.pl
olympescort.com
omoikiri-japan.ru
onlineku.com
onlinemeetingnow.com onlinemeetingnow.com
onlineslotmaschine.com onlinewritingjobs17.blogspot.ru
onthemarch.co open-odyssey.org
orenburg-gsm.ru oren-cats.ru
ostrovtaxi.ru ourville.info
palvira.com.ua outrageousdeal-a.akamaihd.net
parajumpersjakkesalgnorge.info outshop.ru
paydayloanslocal.com paparazzistudios.com.au
pcads.ru partner-cdn.men
peekyou.com patterntrader-en.com
perosan.com perfectplanned.com
pic2fly.com photo.houseofgaga.ru
pila.pl pic-re.blogspot.com
pinstake.com picsearch.com
pipki.r.acdnpro.com picswe.com
poem-paying.gq picturesboss.com
poiskzakona.ru pl-vouchers.com
pl.id-forex.com
platezhka.net
play-movie.pl
pod-muzyku.club
pogosh.com
polytopesexempt.com polytopesexempt.com
pony-business.com popander.mobi
popup-hgd.xyz porndl.org
porno-chaman.info
pornofeuer.com pornofeuer.com
pornogad.com pornolook.net
pornosmola.info pornotubexxx.name
porto.abuilder.net pornstartits.xblog.in
praisong.net posible.net
prefersurvey.net positive2b.ru
promover.org pravoholding.ru
pufip.com printingpeach.com
pukaporn.com privatov-zapisi.ru
purplesphere.in prombudpostach.com.ua
puzo2arbuza.ru promoforum.ru
prostitutki-rostova.org
purchasepillsnorx.com
puteshestvennik.com
quality-traffic.com
qualitymarketzone.com qualitymarketzone.com
quickbuck.com quebec-bin.com
quickcashlimited.com quickchange.cc
quit-smoking.ga rcb101.ru
quizzitch.net redhotfreebies.co.uk
rankings-analytics.com registratciya-v-moskve.ru
realnye-otzyvy.info registrydomainservices.com
realresultslist.com relatodelpresente.com.ar
redirlock.com remont-mobile-phones.ru
refads.pro renecaovilla.online
remote-dba.de repeatlogo.co.uk
rentalcarnavi.info respublica-otel.ru
ric.info rezeptiblud.ru
rossmark.ru ring4rhino.com
rogervivierforsale.com
rospromtest.ru
rubbed.us
rumamba.com rumamba.com
saugatuck.com rusenvironmental.net
savingsslider-a.akamaihd.net russkie-gorki.ru
sbwealthsolutions.ca saddiechoua.com
scanner-ivan.top saletool.ru
scanner-jane.top sandhillsonline.com
score-ads.men sardinie.us
seadragonherbery.com satellite.maps.ilovevitaly.com
scanner-nelson.top
scrapinghub.com
seansonline24.pl
search.1and1.com search.1and1.com
searchimpression.com
searchlock.com searchlock.com
searchwebknow-a.akamaihd.net segol.tv
secret.xn--oogle-wmc.com seoanalyses.com
sei80.com seogadget.ru
senger.atspace.co.uk seokicks.de
sexkvartal.com servingnotice.com
sexpornotales.com sevenstars7.com
sexreliz.com sex-tracker.com
sexvporno.ru sex.hotblog.top
sfj-ror.no sexiporno.net
shopperifymac.com sexy-screen-savers.com
shtaketniki.kz sexytrend.ru
shymkent.xkaz.org shikiso.info
sibvitr.ru shodanhq.com
sirpornogratis.xxx shohanb.com
siteexpress.co.il shop.acim.org
sklad-24.ru shopfishing.com.ua
smailik.org shoppingjequiti.com.br
smartshoppymac.com shopsellcardsdumps.com
socialseet.ru sideeffectsoftizanidine.blogspot.com
soft-program.com sigmund-freud.co.uk
spabali.org simple-image.com.ua
spacash.com sitevalued.com
speechfoodie.com skatestick.bid
sport-video-obzor.ru skincrate.net
sports-supplements.us slim.sellany.ru
ssconstruction.co slowmacfaster.trade
stauga.altervista.org smallseotools.com
sticken.co smstraf.ru
stretchingabuckblog.com softonicads.com
stroyplus.ru softxaker.ru
superfish.com solinf.co
superiends.org sotkal.lark.ru
suture.co sovetogorod.ru
swinon.site sovinsteel.ru
tabletkinaodchudzanie.com.pl space2019.top
tampabaywatch.org sportbetfair.com
technopellet.gr spylog.com
teenforporn.com stackthatbucks.com
templates.radiodigital.co stal-rulon.ru
stanthonyscatholicchurch.org
staticfs.host
steelmaster.lv
stmassage.ru
stockspmb.info
stop-gepatit.te.ua
stpicks.com
strag-invest.ru
strangeduckfilms.com
stroilka.info
sugarkun.com
svetodiodoff.ru
sygraem.com
tattooreligion.ru
the-torrent-tracker.blogspot.com the-torrent-tracker.blogspot.com
thetoiletpaper.com thegameriders.com
thetattoohut.com
timeallnews.ru timeallnews.ru
tizanidinecost.blogspot.com titangel-vietnam.com
tizanidinehclsideeffects.blogspot.com titslove.yopoint.in
tizanidineingredients.blogspot.com tizanidinehcl4mgtabinfo.blogspot.com
tizanidinerestlesslegsyndrome.blogspot.com tizanidinerecreationaluse.blogspot.com
tizanidineusedfor.blogspot.com tootoo.to
tizanidinevssoma.blogspot.com topbestgames.com
tmearegion26.com
todohr.com
tomck.com
toolsky.com
top-l2.com
topkarkas.com topkarkas.com
tour-line.net torrent-to-magnet.com
toys.erolove.in traffic2money.com
traderzplanet.in trafficnetzwerk.de
traffic-club.info trafficshaper.com
traffic100.com trkur.com
trafficjunky.net tube8.com
trafficmania.com tuberkulezanet.ru
trk-4.net tuckermktg.com
tuberkuleznik.ru ukrtvir.com.ua
udayavani.com ultimateclassicrock.com
ultramart.biz unitygame3d.com
undergroundcityphoto.com univerfiles.com
unpredictable.ga universals.com.ua
upproar.com urldelivery.com
uptime-delta.net
urccvfmc.bloger.index.hr
userequip.com userequip.com
utiblog.fr uzungil.com
vadimkravtcov.ru v720hd.ru
validdomain.xyz vepad.com
vchulkax.com video-woman.com
vclicks.net videochat.world
vgoloveboli.net videochaty.ru
viagroid.ru videojam.tv
video--production.com videonsk.com
vikistars.com vip-file.com
villacoloniale.com vkgaleria.com
vip-parfumeria.ru vkonche.com
vipcallsgirls.com voloomoney.com
vipms.ru vuclip.com
vipsiterip.org vvpg.ru
virus-respirators.com wait3sec.org
vk-mus.ru
vrnelectro.ru
vseigry.fun
vsesubwaysurfers.com
vtcdns.com
vzlomfb.com
wallpaperaccess.com
wareseeker.com wareseeker.com
waterefficiency.co wdrake.com
waycash.net webcamdevochka.com
webcamtalk.net webinstantservice.com
webkeyit.com wedding-salon.net
webmasters.stackexchange.com weprik.ru
website-audit.com.ua
westvilletowingservices.co.za westvilletowingservices.co.za
whiteproduct.com wholesalejerseychinashop.com
wholinkstome.com wineitudes.wordpress.com
womensplay.net womensterritory.ru
worldis.me wordpresscore.com
writersgroup580.web.fc2.com wosik-dach.service-for-web.de
www.arenda-yeisk.ru wurr.voila.net
www.kabbalah-red-bracelets.com www.bookmaker-bets.com
www.souvenirua.com xblog.in
wzgyyq.com xev.ru
xblognetwork.com xingzi-vision.com
xdoza.com xitjw.info
xmlinde.com xlovecam.com
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai xmnb.net
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn----7sbho2agebbhlivy.xn--p1ai xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xn----9sbubg3ambdfl1j.xn--p1ai xn----9sbubg3ambdfl1j.xn--p1ai
xn----dtbndd4ae7eub.top xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn--hxazdsfy.blogspot.com xn--h1aakne2ba.xn--p1ai
xxxtubesafari.com xxxtubesafari.com
yebocasino.co.za yachts-cruise.info
yellowproxy.net yottos.com
yes-com.com yougetsignal.com
youporn-forum.ga your-bearings.com
yourporn.com youradulthosting.com
yun56.co yourserverisdown.com
z-master.ru zaim-pod-zalog-krasnodar.ru
zajm-pod-zalog-nedvizhimosti.ru zeroredirect6.com
zeroredirect8.com
zetgie.com.pl zetgie.com.pl
zigarettenonlinekaufen2.drupalgardens.com
zlatnajesen.com
znakomstva-moskva77.ru
znaturaloriginal.com
zocaparj.kz zocaparj.kz
zoogdiesney.com zolotoy-lis.ru
zreloeporno.tv zona-aqua.ru
zryydi.com zooggames.com
zytpirwai.net zvetki.ru
zvuker.net

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00author.com 4webmasters.com
12masterov.com 777-club.ru
1adult.com 888.com
2728fb936f0.com 9icmzvn6.website
273-fz.ru aaronabel.com
3dracergames.com aero2.ru
4ureyesonly.com affordablewebsitesandmobileapps.com
991mostfm.co.id aktivator-windows10.blogspot.com
999webdesign.com android-systems.ru
aa81bf391151884adfa3dd677e41f94be1.com anticrawler.org
abbanreddy.com apidigidocketnet-a.akamaihd.net
activepr.ru approved.su
adman.gr arate.ru
adrenalin-stk.ru asdfg.pro
alevinefesleri.com asseenontvonline.ru
alfabot.xyz autobrennero.it
alvtank.se autowebmarket.com.ua
anlimebel.kiev.ua azarttoto.com
apartmentratings.com azbukafree.com
apiholdingmypage-a.akamaihd.net bannerbank.ru
app5.letmacworkfaster.world
appsecurityr.com
arcadeplayhouse.com
atdedinotuho.tk
atovh.local-finders.com
ayerbo.xhost.ro
backlinkwatch.com
bannertesting.com
battlecarnival.su battlecarnival.su
bestfortraders.com bavsac.com
big-boards.info bcmp.org
bikini-image.com beremenyashka.com
bin-brokers.com bestbrainprod.win
binomo.kz bigpenisguide.com
bizlist.com.de bimmerpost.com
bloggen.be binomo.com
bloggerads.net bkgr.se
bloggers.nl blagovest-med.ru
blpmovies.com blogporn.in
bonzybuddy.com blogs.rediff.com
buron.pw boltalko.xyz
buynorxx.com bonky.biz
buypillsorderonline.com book-bets.com
buyscabiescream.com brothers-smaller.ru
carfax.com.ua buypanicdisorderpill.com
chastnoeporno.com buytizanidineonlinenoprescription.blogspot.com
cheapjerseysfootballshop.com bzero1jewelry.net
check-this-out-now.online cakesplus.com.au
cherubinimobili.it canrole.com
chocolateslim-en-espana.com cbcseward.com
cibpenonptib.flu.cc celebsopics.com
cicaki.net cementaresearch.se
ciceros.co chatroulette.si
clknsee.com cheap-pills-norx.com
comeondog.info cheapjerseysbizwholesale.us
cookie-law-enforcement-ee.xyz chiblackhawks-jerseys.com
copblock.org circlesl.com
covadhosting.biz cjs.com.ru
cracksplanet.com cleanallspyware.com
cranly.net cocyq.inwtrade.com
customcollegeessays.net comissionka.net
czat.wp.pl compliance-alexa.xyz
d2jsp.org compliance-willy.top
data-mining.tk cool-wedding.net
credit-respect.ru
customchocolate.business-for-home.com
cyclobenzaprinevstizanidine.blogspot.com
cyse.tk
dailystrength.org
deplim.com deplim.com
deshevo-nedorogo.ru dignitasdata.se
diamond-necklace.info dominateforex.ml
domini.cat domznaniy.ru
downloaddy.net easytuningshop.ru
eaplay.ru easyukraine.com
elektrischezigarettekaufen2.cowblog.fr eets.net
elektronischezi.livejournal.com efnor-ac.com
escortslet.net englate.com
etur.ru erectile.bid
exonline.info erotik0049.com
falopicm.pw everflora.ru
familyholiday.ml exmasters.com
fermersovet.ru exoneration-project.us
fickkontaktehobbyhuren.com famix.xyz
fidelityfunding.com fitfloponline.store
filmania-x.ru forum.tvmir.org
filmi-onlain.info freza-sverlo.ru
firesub.pl fxtips.ru
flex4launch.ru generousdeal-a.akamaihd.net
fotzen-ficken.com gepezz.info
fusoradio.info get-free-traffic-now.com
galeon.com getdot.ru
geile-lelly.eu gilsonchiro.xyz
gheus.altervista.org girls-ufa.ru
gombita.info girlsatgames.ru
gonextmedia.com globalsurfari.com
googglet.com googst2.ru
hackers-crackers.tk gpirate.com
hermesbirkinhandbagoutlets.com gratuitbaise.com
herokuapp.com guge.io
highstairs-a.akamaihd.net hamilton.ca
hothot.ru hesteel.pl
humbmosquina.tk hkladys.com
images.gyffu.com howopen.ru
inboxdollars.com hvd-store.com
incanto.in.ua icaseclub.ru
istizanidinelikexanax.blogspot.com iideaidekonkatu.info
izhevsk.xrus.org iinstalll-fii1leis.jus0wil.pp.ua
izismile.com infowarcraft.ru
jacago.com internetartfair.com
jav-idol.com jamiembrown.com
kareliatobacco.ru janerikholst.se
kerwinandcariza.com javitas.info
keywordsdoctor.com jerseychinabizwholesale.com
kfon.eu jerseysbizwholesalecheap.com
konica.kz jose.mulinohouse.co
konoplisemena.com justdating.online
kskjco.club jwcialislrt.com
kakadu-interior.com.ua
kaz.kz
keywordbasket.com
keywords-monitoring-success.com
kiinomaniak.pl
kinofak.net
krasnodar-avtolombards.ru
kriokomora.info
kunstaktien.info kunstaktien.info
kursy-ege.ru linkpulse.com
lacloop.info livingroomdecor.info
ledpolice.ru magento-crew.net
linkdebrideur.xyz mararoom.ru
lionking-1994.blogspot.com medi-fitt.hu
livetsomudvekslingstudent.bloggersdelight.dk medkritika.ru
livingcanarias.com meet-flirt-dating.com
lizace.com megaapteka.ru
lmrauction.com mil-stak.com
lomb.co mmgq.ru
lostaruban.com
lotto6888.com
macotool.com
magnetic-bracelets.ru
matrixalchemy.com
med-zdorovie.com.ua
medicinacom.ru
members.ghanaweb.com
microsearch.ru
miniature.io
mixx.com
mobicover.com.ua mobicover.com.ua
modnie-futbolki.net mobot.site
monclerjacketsoutlet.win mobplayer.ru
moskva.nodup.ru moncleroutletonline.win
musicas.baixar-musicas-gratis.com moneyzzz.ru
mydoctorok.ru moomi-daeri.com
morocco-nomad-excursions.com
mrvideospornogratis.xxx
mug-na-chas-moscow.ru
my-cash-bot.co
mydirtystuff.com
myfreetutorials.com
mygameplus.ru
myhitmp3.club myhitmp3.club
mysexpics.ru
namecrumilchlet.tk namecrumilchlet.tk
newyorkhotelsmotels.info namenectar.com
nofreezingmac.work networkcheck.xyz
novosibirsk.xrus.org noclegonline.info
oklogistic.ru obiavo.by
onlywoman.org oda.as
owathemes.com omgtnc.com
paramountmarble.co.uk oneminutesite.it
partner-host.men online.ktc45.ru
partners-ship.pro oops-cinema.ru
patterntrader-en.com orion-code-access.net
pensplan.com pagesense.com
phelissota.xyz paparazzistudios.com.au
picsfair.com partner-print.men
pisanieprac.info pcads.ru
pizza-imperia.com pdns.download
playmsn.com perso.wanadoo.es
powitania.pl pestomou.info
pozvonim.com pharmacyincity.com
praisong.net pijoto.net
prize44.com plastweb.ru
proxyradar.com pooleroadmedicalcentre.co.uk
pwwysydh.com porno-play.net
pxhdwsm.com porno.simple-image.com.ua
pornosee.info
pornosemki.info
portside.cc
princeadvantagesales.com
printingpeach.com
pro-okis.ru
pukaporn.com
pulseonclick.com
quit-smoking.ga
qwertty.net
rada.ru
rankia.com
rankingchart.de
rating-casino2021.ru
razvratnoe.org
real-time-analytics.com real-time-analytics.com
recipedays.ru realitykings.com
redirect.trafficreceiver.club redirlock.com
refererx.com registrationdomainsite.com
remontbiz.ru reklama1.ru
rennlist.com remmling.de
replicaclub.ru remont-fridge-tv.ru
rezeptiblud.ru russia-today-video.ru
rossmark.ru s.lollypopgaming.com
s8-nowy-wygraj.comli.com sa-rewards.co.za
samolet.fr selectads.men
sawin.beth.webd.pl seo-prof1.xyz
sbornik-zakonov.ru seocheki.net
scanner-mark.top seoexperimenty.ru
scanspyware.net seolab.top
search.xtconnect.com sharebutton.org
securityallianceservices.com shell-pmr.ru
sei80.com shoppingjequiti.com.br
seofied.com sierraapps.com
seorankinglinks.us socialmediasuggest.com
sexxdate.net soft-program.com
shtaketniki.kz stanthonyscatholicchurch.org
simply.net stjamesschool.info
sinestesia.host.sk styro.ru
site3.free-share-buttons.com tagil.zrus.org
slujbauborki.ru tedy.su
smartbalanceworld.com teenporn18.net
socas.pluto.ro thebitcoincode.com
spyfu.com thfox.com
statistici.ro titangel-vietnam.com
stickers-market.ru tizanidine4mg.blogspot.com
strag-invest.ru tizanidine4mgprice.blogspot.com
sukirgenk.dvrlists.com tizanidine4mgtablets.blogspot.com
superfish.com tizanidineingredients.blogspot.com
tanieaukcje.com.pl tmearegion26.com
technopellet.gr topsiteminecraft.com
teenforporn.com torrentgamer.net
the-usa-games.blogspot.com trackzapper.com
thepantonpractice.co.uk trafficmp.com
thexart.club
tizanidineduringpregnancy.blogspot.com
tizanidinehcl2mgsideeffects.blogspot.com
tizanidinesideeffectsweightloss.blogspot.com
tjkckpytpnje.com
tooplay.com
topkarkas.com
torrents.life
tourcroatia.co.uk
traffpartners.com traffpartners.com
tri-slona.org tula.mdverey.ru
tt-ipd.info
tubeline.biz
tuberkuleznik.ru
tzritel.tk tzritel.tk
ultramart.biz ua-company.ru
umekana.ru uptimebot.net
utiblog.fr us-america.ru
valueclick.com videtubs.pl
veerotech.com vitanail.ru
videonsk.com voronezh.xrus.org
videosbox.ru watchmygf.net
vladimir.zrus.org website-speed-check.site
vzube.com wholesalejerseychinaoutlet.com
webalan.ru wikes.20fr.com
webstatsdomain.org
winwotgold.pl
woodyguthrie.se woodyguthrie.se
wordkeyhelper.com
wowcasinoonline.ooo
www.get-free-traffic-now.com www.get-free-traffic-now.com
x-lime.net
xbaboon.com
xblognetwork.com xblognetwork.com
xfire.com xcc24.pl
xingzi-vision.com xep.info
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn--h1ahbi.com.ua xn--80atua3d.xn--p1ai
xuki.us xn--h1aakne2ba.xn--p1ai
yoluxuryevents.com xxxguitars.com
youghbould.wordpress.com yourporn.com
youtuhe.com youtube-downloader.savetubevideo.com
yubikk.info zeroredirect12.com
zawyna.ua znaniyapolza.ru
zeroredirect1.com
zoosexart.com
zzlgxh.com

73
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -8,6 +8,44 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -456,41 +494,6 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.37 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3492 ### Version: V4.2022.11.3493
### Updated: Fri Nov 25 22:01:22 UTC 2022 ### Updated: Sat Nov 26 10:03:24 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3492 ### Version: V4.2022.11.3493
### Updated: Fri Nov 25 22:01:22 UTC 2022 ### Updated: Sat Nov 26 10:03:24 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##