vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.10.3408 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-10-13 22:07:05 +00:00
parent 3e02b1dab6
commit 1d57b54e53
47 changed files with 102583 additions and 102573 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3407 #### Version: V4.2022.10.3408
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3407 #### Version: V4.2022.10.3408
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.10.3407 #### Version: V4.2022.10.3408
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12688
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.10.3407 #### Version: V4.2022.10.3408
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12692
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12692
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12692
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12692
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.40 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.40 0;
127.0.0.1 0; 127.0.0.1 0;

12692
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.40 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.40 0;
127.0.0.1 0; 127.0.0.1 0;

12692
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.40 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.40 0;
127.0.0.1 0; 127.0.0.1 0;

12692
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3407 3408

12692
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12692
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.66.29.227 0; 157.55.188.69 0;

12692
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.40 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.40 0;
127.0.0.1 0; 127.0.0.1 0;

12692
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.40 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.40 0;
127.0.0.1 0; 127.0.0.1 0;

12692
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.40 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.40 0;
127.0.0.1 0; 127.0.0.1 0;

12692
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.66.29.227 0; 157.55.188.69 0;

12692
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.66.29.227 0; 157.55.188.69 0;

12692
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 1; 10.1.0.40 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Aipbot Alligator
CODE87 BDFetch
Cegbfeieh DBLBot
Getintent DomainCrawler
HTTrack SpankBot
Minefield Wallpapers/3.0
MolokaiBot WebGo IS
Morfeus Fucking Scanner ZoominfoBot
Sitebeam coccocbot
lwp-trivial crawl.sogou.com

266
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
404enemy
80legs
ADmantX
AIBOT AIBOT
ASPSeek ASPSeek
Aboundex
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot
AiHitBot AiHitBot
Aipbot AlphaBot
Anarchy99
Apexoo Apexoo
Asterias
Atomseobot
Attach
AwarioRssBot AwarioRssBot
BBBike
BDCbot
BackDoorBot
BackStreet
BackWeb BackWeb
Bandit BacklinkCrawler
BetaBot Badass
Bitacle BatchFTP
BlackWidow Bigfoot
Blackboard Blackboard
Blow
BlowFish
Boardreader
Brandprotect Brandprotect
Brandwatch
Buck
BuiltBotTough
BuiltWith BuiltWith
Bullseye
CATExplorador CATExplorador
CCBot
CODE87 CODE87
CSHttp CSHttp
Calculon Calculon
CazoodleBot
CheTeam
CheeseBot
ChinaClaw
Chlooe Chlooe
Citoid
Claritybot Claritybot
Cliqzbot Cliqzbot
Cloud mapping Collector
Cocolyzebot Cosmos
Copier CrunchBot
CopyRightCheck CyotekWebCopy
Craftbot DBLBot
Crawling at Home Project
Crescent
Curious
DIIbot DIIbot
DSearch DataCha0s
Devil DatabaseDriverMysqli
Disco Demon
Discobot DittoSpyder
DnBCrawler-Analytics
DnyzBot DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Dotbot Dotbot
Download Wonder Dragonfly
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
EasyDL Ebingbong
EirGrabber EirGrabber
FDM EroCrawler
Evil
Exabot
Extractor
FHscan
FemtosearchBot FemtosearchBot
Fimap
Foobot Foobot
FrontPage FyberSpider
G-i-g-a-b-o-t
GT::WWW GT::WWW
GetRight GermCrawler
Gigabot Gigabot
Grabber Go!Zilla
GridBot GridBot
HEADMasterSEO HEADMasterSEO
HTMLparser HTMLparser
Haansoft Haansoft
HaosouSpider
Harvest
Havij
Hloader
HonoluluBot HonoluluBot
Humanlinks Humanlinks
IDBTE4M
IDBot
IRLbot IRLbot
IlseBot Id-search
Image Fetch
IndeedBot
InfoNaviRobot InfoNaviRobot
Internet Ninja
InternetSeer
Iria
IstellaBot IstellaBot
JOC Web Spider JOC Web Spider
Jbrofuzz JennyBot
Jetty
JikeSpider
Joomla
Jorgee Jorgee
Jyxobot Keybot Translation-Search-Machine
Kenjin Spider Keyword Density
LNSpiderguy
Lanshanbot Lanshanbot
Larbin
LeechFTP LeechFTP
LexiBot
LibWeb LibWeb
LieBaoFast
Lightspeedsystems Lightspeedsystems
LinksManager LinkWalker
LinqiaMetadataDownloaderBot Linkbot
LinkpadBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
MFC_Tear_Sample
MIDown tool MIDown tool
MIIxpc
MQQBrowser MQQBrowser
MSIECrawler MSIECrawler
MTRobot MTRobot
Majestic SEO
Majestic-SEO
Majestic12 Majestic12
MarkMonitor MarkWatch
Masscan Mata Hari
Mister PiX Meanpathbot
Metauri
Microsoft URL Control
Moblie Safari Moblie Safari
Morfeus Fucking Scanner Morfeus Fucking Scanner
Msrabot NPbot
Musobot
Navroad Navroad
NearSite
Nessus
Net Vampire Net Vampire
NetAnts NextGenSearchBot
NetZIP Ninja
Netvibes
Nibbler
Nikto
Not
Nuclei Nuclei
Nutch Offline Explorer
OpenLinkProfiler OnCrawl
Openvas Openvas
OrangeBot OrangeSpider
OutfoxBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageGrabber
PageScorer PageScorer
PageThing.com PageThing.com
Pandalytics Pandalytics
Panscient
Papa Foto Papa Foto
Petalbot Pavuk
PeoplePal
Picscout Picscout
Pimonster Piepmatz
Pixray
PleaseCrawl PleaseCrawl
ProPowerBot ProPowerBot
Probethenet ProWebWalker
Psbot Psbot
Pu_iN
Pump Pump
PxBroker
PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot RSSingBot
RankActive
RankActiveLinkBot RankActiveLinkBot
RankingBot RankingBot
RankingBot2 RankingBot2
Re-re Rankivabot
ReGet ReGet
RebelMouse RebelMouse
RedesScrapy
RepoMonkey RepoMonkey
Ripper Rogerbot
SBIder
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats SISTRIX
SMTBot SMTBot
ScanAlert
Scanbot Scanbot
Screaming ScoutJet
Searchestate
SearchmetricsBot SearchmetricsBot
Semrush
SentiBot SentiBot
SeoSiteCheckup
Seomoz Seomoz
Shodan Shodan
Siphon
Site Sucker Site Sucker
SiteExplorer SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker Siteimprove
Sitebeam Sitevigil
SlySearch Sociscraper
Snake
Sogou web spider Sogou web spider
Sosospider Sottopop
Spbot SputnikBot
Steeler Sqlmap
Stripper
Sucuri
SuperBot
SuperHTTP SuperHTTP
Surfbot TheNomad
Suzuran Thumbor
Swiftbot Toweyabot
Szukacz
T0PHackTeam
Telesoft
The Intraformant
TightTwatBot
Toata
Tracemyfile
Trendictionbot Trendictionbot
Turnitin
TurnitinBot
Twice Twice
Vacuum Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler
Virusdie Virusdie
WEBDAV Voltron
WISENutbot
WWW-Collector-E WWW-Collector-E
WWW::Mechanize WWWOFFLE
Wallpapers/3.0
WeSEE WeSEE
Web Fetch
Web Fuck
WebBandit WebBandit
WebPix WebCollage
WebSucker WebFetch
WebGo IS
WebImageCollector
WebWhacker
Webalta
WebmasterWorldForumBot
Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
WebsiteQuester WebsiteQuester
Widow Whack
Zauba Xaldon WebSpider
Xaldon_WebSpider
Xenu
Zermelo Zermelo
ZmEu
ZoomBot ZoomBot
ZoominfoBot ZyBorg
autoemailspider cah.io.community
check1.exe check1.exe
clark-crawler cognitiveseo
crawler.feedback crawl.sogou.com
crawler4j crawler4j
demandbase-bot demandbase-bot
gopher internetVista monitor
heritrix lwp-request
instabid magpie-crawler
mediawords meanpathbot
muhstik-scan muhstik-scan
oBot oBot
pcBrowser
probe-image-size probe-image-size
satoristudio.net ripz
s1z.ru
scalaj-http scalaj-http
scan.lol scan.lol
seobility
seocompany.store
serpstatbot serpstatbot
sitechecker.pro sitechecker.pro
siteripz siteripz
sp_auditbot sogouspider
spyfu spyfu
trendiction.com tAkeOut
voyagerx.com
webgains-bot webgains-bot
webpros.com webmeup-crawler
webprosbot
x09Mozilla
xpymep1.exe xpymep1.exe
zauba.io

312
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
404checker ADmantX
404enemy
ALittle Client ALittle Client
ASPSeek ASPSeek
Aboundex AhrefsBot
Aboundexbot AiHitBot
Alligator
AlphaBot
Anarchie Anarchie
Anarchy Anarchy
Anarchy99
Anthill Anthill
Apexoo AwarioRssBot
Asterias
AwarioSmartBot
BBBike BBBike
BDFetch BackDoorBot
BacklinkCrawler
Bandit
Barkrowler Barkrowler
BatchFTP Battleztar Bazinga
BetaBot BetaBot
Bigfoot BlackWidow
Bolt Boardreader
Brandwatch Brandprotect
Buck
Buddy Buddy
BuiltBotTough
Bullseye Bullseye
CODE87 CATExplorador
Cegbfeieh CCBot
CSHttp
Calculon
CensysInspect CensysInspect
CheTeam
CheeseBot CheeseBot
CherryPicker
ChinaClaw ChinaClaw
Citoid Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Collector
CopyRightCheck
Cosmos Cosmos
CrunchBot
Curious Curious
Custo DIIbot
Demon DataCha0s
Digincore Deusu
Disco DigitalPebble
Discobot Dirbuster
Discoverybot
Dispatch Dispatch
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot
DomainAppender DomainAppender
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project
Dragonfly
EMail Wolf
EasyDL
Ebingbong Ebingbong
EirGrabber Ecxi
Exabot EroCrawler
Extractor Extractor
ExtractorPro
EyeNetIE
Ezooms Ezooms
Fimap FDM
Foobot FHscan
Firefox/7.0
Freeuploader Freeuploader
Fyrebot
G-i-g-a-b-o-t
GT::WWW
Genieo
GermCrawler GermCrawler
GetRight
GetWeb GetWeb
Getintent Go-Ahead-Got-It
Gigabot Gotit
GoZilla GrabNet
Grafula Grabber
GridBot
HMView HMView
HTMLparser
HTTP::Lite HTTP::Lite
Haansoft Haansoft
Harvest Havij
Hloader Hloader
Humanlinks
HybridBot HybridBot
IDBot IRLbot
Iblog
Id-search
IlseBot IlseBot
Indy Library Image Fetch
InfoNaviRobot Image Sucker
IndeedBot
InfoTekies
Intelliseek
InternetSeer
Iria Iria
Iskanie
JennyBot JennyBot
Jetty JetCar
Jorgee Jorgee
Keyword Density Keyword Density
Lanshanbot Kinza
Larbin LNSpiderguy
LWP::Simple
Leap Leap
LeechGet LeechGet
Libwhisker Lftp
LieBaoFast
Likse Likse
LinkWalker
Linkbot Linkbot
LinkextractorPro
LinkpadBot
LinksManager LinksManager
LinqiaRSSBot LinqiaMetadataDownloaderBot
Lipperhey Spider Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MJ12bot
MQQBrowser MSIECrawler
Mail.RU_Bot Mag-Net
Majestic-SEO Majestic12
MauiBot Mass Downloader
Mb2345Browser Masscan
Meanpathbot Mata Hari
Mediatoolkitbot MeanPath Bot
MicroMessenger Microsoft Data Access
Microsoft URL Control
Mister PiX
Moblie Safari Moblie Safari
Mojeek Mr.4x3
Msrabot
Nameprotect
NearSite NearSite
Needle
Nessus
NetAnts
NetMechanic NetMechanic
Nettrack NetSpider
Nibbler NetZIP
Nikto Netcraft
Not Netvibes
Octopus NextGenSearchBot
OrangeBot NimbleCrawler
PHPCrawl Nutch
Offline Navigator
OpenLinkProfiler
OpenVAS
OrangeSpider
OutclicksBot
POE-Component-Client-HTTP
PageGrabber
PageThing.com PageThing.com
Pandalytics Panscient
Papa Foto
Pavuk Pavuk
PeoplePal
Petalbot Petalbot
PictureFinder
Piepmatz
Pixray
Pockey
ProPowerBot
ProWebWalker
Probethenet
Psbot Psbot
PyCurl Quick-Crawler
QueryN Metasearch RSSingBot
RankActive RankFlex
RankActiveLinkBot
Rankivabot Rankivabot
Re-re Re-re
ReGet ReGet
Recorder RebelMouse
RedesScrapy
RepoMonkey
Ripper
RocketCrawler
Rogerbot
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats SalesIntelligent
SISTRIX ScanAlert
Screaming Searchestate
SearchmetricsBot SearchmetricsBot
SemanticJuice
SentiBot
SeoSiteCheckup SeoSiteCheckup
Siphon
SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
Sitebeam Sitebeam
Sitevigil Siteimprove
SmartDownload SlySearch
Snapbot Snoopy
SocialRankIOBot SocialRankIOBot
Sociscraper Sogou web spider
Sosospider Spammen
Sottopop SpankBot
SputnikBot Spbot
Sqlmap
Sqlworm Sqlworm
Steeler Sqworm
Sucuri Stripper
SuperBot Sucker
SurveyBot Surfbot
Swiftbot
Szukacz Szukacz
T0PHackTeam Telesoft
T8Abot
Telesphoreo Telesphoreo
Telesphorep Telesphorep
The Intraformant
TheNomad TheNomad
TightTwatBot TightTwatBot
Toata Titan
Toweyabot
Trendictionbot Trendictionbot
Turnitin
TwengaBot TwengaBot
Twice
Typhoeus
URLy Warning
Upflow
V-BOT
VB Project VB Project
VCI Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
VoidEYE VoidEYE
Voltron Voil
WEBDAV
WISENutbot WISENutbot
WWWOFFLE WWW-Collector-E
WWW::Mechanize
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD
Web Enhancer Web Enhancer
Web Fuck
Web Pix Web Pix
Web Sauger WebAuto
WebFetch WebBandit
WebGo IS WebFuck
WebLeacher
WebReaper
WebStripper WebStripper
WebWhacker
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Webshag
Website Quester Website Quester
WebsiteExtractor
WebsiteQuester
Webster Webster
Whack Whatweb
Whacker
Who.is Bot Who.is Bot
WiseGuys Robot
Woobot Woobot
Wprecon Xenu
Xaldon_WebSpider YoudaoBot
Zade Zade
Zauba
Zermelo
Zitebot Zitebot
ZoomBot ZoomBot
ZumBot ZoominfoBot
ZyBorg ZyBorg
archive.org_bot arquivo.pt
arquivo-web-crawler
autoemailspider
backlink-check backlink-check
cah.io.community cah.io.community
clark-crawler check1.exe
coccocbot
cognitiveseo cognitiveseo
com.plumanalytics
crawler.feedback crawler.feedback
dataforseo.com crawler4j
demandbase-bot domainsproject.org
eCatch
facebookscraper facebookscraper
heritrix gopher
ips-agent internetVista monitor
isitwp.com linkdexbot
lwp-request lwp-request
magpie-crawler lwp-trivial
oBot
page scorer
polaris version polaris version
probe-image-size
ripz
satoristudio.net
scalaj-http
seobility
seoscanners seoscanners
seostar serpstatbot
spyfu spyfu
sysscan trendiction.de
zgrab xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
balois.worldbreak.com begalka.xyz
freelifetimefuckbook.com chee-by.biz
lonfon.xyz getmyads24.com
master-muznachas.ru igtools.club
net-profits.xyz kutikomi.net
social-buttons.xyz sbprabooks.com
solution4u.com scanner-alex.top
spb-plitka.ru sex-tracker.de
whatzmyip.net tandvardshuset.net
znakomstva-piter78.ru witclub.info

920
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
101billion.com 1001watch.com.ua
123cha.com 123movies.love
13tabs.com 12u.info
1kinobig.ru 1688.com
2020iscoming.info 178evakuator178.ru
24-ak.ru
24x7-server-support.site 24x7-server-support.site
2daytrendingnews.com 34.gs
4istoshop.com 4rent.ru
5ws.dating-app.ru 4replicawatch.net
66cpwgln.space 777-club.ru
6hopping.com 7makemoneyonline.com
76brighton.co.uk 7search.com
83net.jp 8lufu.com
8558l.jobs.net aa08daf7e13b6345e09e92f771507fa5f4.com
8gold.com acmebtn.ml
9med.net adobereader-free.ru
a.frcls.fr adsloads.com
aa9d046aab36af4ff182f097f840430d51.com adultgalls.com
aasoldes.fr adultnet.in
abwa.tk adventureparkcostarica.com
actices.com adverster.com
adbetclickin.pink albuteroli.com
add-add.men alert-fjg.xyz
adsfresh.men alibestsale.com
advokateg.xyz allblogroll.com
adzpower.com allwidewallpapers.com
agapovdg.ru aloofly.com
agysya.ru amateurlivechat.org
alevinefesleri.com amyfoxfitness.com
alf-img.com angel1777.kz
alfapro.ru angigreene.com
allsilver925.co.il anonymizeme.pro
alpinism.ru aosexkontakte.net
alyeskaresort.com apartamentwroclaw.eu
amazon-adsystem.com apartment.ru
anaseracresar.tk apartmentbay.ru
apccargo.com apessay.com
apidigidocketnet-a.akamaihd.net apidiamondatanet-a.akamaihd.net
apiholdingmypage-a.akamaihd.net app5.letmacworkfaster.world
apiwebwebgetcom-a.akamaihd.net
apparel-offer.com apparel-offer.com
appfastplay.com applyneedy.xyz
apple.com-webbrowsing-security.science arabseyes.com
apptester.tk architecturebest.com
arendakvartir.kz arenda-avtoprokat-krasnodar.ru
as5000.com
asiengirls.net
asociatia-tipografilor-transilvania.ro
atdedinotuho.tk
audiobangout.com
ausergrubhof.info
ausmepa.org.au
auto.rusvile.lt auto.rusvile.lt
aversis.net aviav.ru.com
avtoarenda.by avitocars.ru
avkzarabotok.com
avtointeres.ru
awency.com
axbocz.net axbocz.net
ayodhya.co ayeartoforget.com
babespcs.com azte.ch
bad-stars.net babs.com.ua
balkanfarma.ru backuperwebcam.weebly.com
bankiem.pl baixar-musicas-gratis.com
bayareaaeroclub.org battlecarnival.su
bbsoldes.fr
beachdriveblog.com
beachpics.com
bear.gotcher.us
beauby.ru
bedandbreakfast.com bedandbreakfast.com
best-games.today bestchoice.cf
bestwebsiteawards.com bestdraws.com
big-boards.info besthomemadepornsites.com
bigpenisguide.com bestimagecoollibrary.com
bin-brokers.com betterscooter.com
birzha-truda.eu big-cash.net
bloggers.nl bioca.org
bluesman.nu blavia.00author.com
bolitgorlo.net boeuklad.com
bongiornos.info boobsimge.com
bookmark4you.com bookmark4you.com
braintobe.win boole.onlinewebshop.net
brandimensions.com braindaily.xyz
briomotor.co brianhenry.co
bumascloset.com bronzeaid-a.akamaihd.net
buqyxa.rincian.info brus-vsem.ru
buypuppies.ca btc4u.ru
buzzsumo.com buchananshardware.com
byme.se bupropion-sr-150-mg.us
c-gainsbourg.com buy-cheap-online.info
cactussoft.biz buyclomidonlaine.com
carloans.com buydissertation.net
buzzzg.men
camdolls.com
cannibalcheerleader.com
carsplate.com carsplate.com
cartierlove2u.com cartierlovebraceletreplica.xyz
caulderwoodkennels.com cartierlovestore.xyz
cdncash.com casite-513049.cloudaccess.net
chatroulette.online cctva.tv
cheapjerseysap.com cdn.walkme.com
cheapmoncler.win centre-indigo.org.ua
chocolateslim-u-srbiji.com cgi2.nintendo.co.jp
chorus.terakeet.com cheapwebsitehoster.com
cjs.com.ru chiblackhawks-jerseys.com
chinacheapelitejerseys.com
chinajerseyswholesalecoupons.com
chocolateslim-en-espana.com
chocoslim.pro
cicaki.net
ciekawinki.pl
civilwartheater.com
clarithromycin500mg.com clarithromycin500mg.com
cleanallvirus.com clash-clans.ru
clipartnew.com clickbank.net
cognitiveseo.com clicksotrk.com
collegerentals.com coccoc.com
compliance-andrew.top codq.info
compliance-barak.top coindirect.io
compliance-checker-7.info compliance-don.top
compliance-jess.top compliance-olga.top
compliance-josh.top coolingoods.com
compliance-willy.top cougfan.info
cookielawblog.wordpress.com crynet.cc
coolwallpapers4k.info customboxes4less.com
covetnica.com cvety24.by
crazyprotocol.com
customerpromos-a.akamaihd.net
customsua.com.ua
cypernhuset.se
d0t.ru
dailystrength.org
darkages.info
dataloading.net
davebestdeals.com
dcj-nn.ru dcj-nn.ru
dekorkeramik.ru deda-moroza-zakaz.ru
den-noch24.ru den-noch24.ru
dentalpearls.com.au dfwu1013.info
deplim.com discountbarbour.online
diegolopezcastan.com distver.ru
dinosaurus.site djihispano.com
discovertreasure-a.akamaihd.net documentbase.net
dodlive.mil dojki-hd.com
dogclothing.store dokfilms.net
drhomes.biz domaineaneblanc.com
dustyorate.com domashneeruporno.com
e-avon.ru dominateforex.ml
e-poker-2005.com dood.live
download-wallpaper.net
draniki.org
drunkmoms.net
dverimegapolis.ru
eager-nash.188-93-233-196.plesk.page eager-nash.188-93-233-196.plesk.page
easync.io earncash.com.ua
ebonyporn.site easyukraine.com
ed-shop01.ru edudocs.net
ekto.ee efnor-ac.com
electricwheelchairsarea.com ekobata.ru
electro-prom.com elektrozigaretten2.yn.lt
elite-sex-finders.com elidelcream.weebly.com
en.home-task.com englate.com
enternet.ee enskedesquashclub.se
epicbrogaming.com ero-advertising.com
escortplius.com erotische-geschichten-xxl.com
estibot.com escort-russian.com
etotupo.ru evaashop.ru
everypony.ru expdom.com
exchanges-bet.com expolicenciaslatam.co
exmasters.com eyessurgery.ru
expresstoplivo.ru ezigarettenshop2.myblog.de
extrabot.com ezigaretteshop2.vefblog.net
ezaz.info f012.de
ezofest.sk facialporntube.com
f00kclan.de falcon-images.blogspot.com
family1st.ca fanpagerobot.com
farm26.ru fanrto.com
fetishinside.com fashion.stellaconstance.co
ff30236ddef1465f88547e760973d70a.com felizporno.com
filerockstar298.weebly.com feorina.ru
finteks.ru fetroshok.ru
fishtauto.ru fickblock18.com
fickkontaktehobbyhuren.com
figuringmoneyout.com
filesvine.com
filmi-v.online
financehint.eu
findclan.org
fingerprintjs.com
flagcounter.me
flirt4free.com
flowwwers.com flowwwers.com
foodgid.net forensicpsychiatry.ru
forex-indextop20.ru freakycheats.com
forpostlock.ru freeseedsonline.com
forum.tvmir.org friendflnder.com
fototravel.eu funnypica.com
freecamdollars.com gagrasector.ru
freelotto.com gambarkatabaru.com
front.ru get-free-social-traffic.com
fuckmill.com getmyads24.com
fugarif.ga getridofstretchmarks.org
fym.com.ru gettpromos.com
gael-s.ru ghostvisitor.com
gamedayassist.com gkvector.ru
geile-lelly.eu goblacked.com
genetworx.com golden-praga.ru
gheus.altervista.org gourcy.altervista.org
girlgamerdaily.com graphicwe.org
globalscam.ga greatgrace.ru
glogow.pl greatzip.com
googlemare.com gsasearchengineranker.space
greatfind-a.akamaihd.net guidefs.ru
gsasearchengineranker.pw
gsmtlf.ru
guge.io
guigyverpo.cf guigyverpo.cf
guildebzh.info gwebtools.com.br
h2monline.com handicapbathtubarea.com
hanink.biz.ly
hard-porn.mobi
hatdc.org hatdc.org
hd720kino.ru hauleddes.com
helicalpile.us highstairs-a.akamaihd.net
hopeonthestreet.co.uk hornymatches.com
host-tracker.com hotblog.top
hosting-tracker.com howmuchdoestizanidinecost.blogspot.com
hostingclub.lk huhn.altervista.org
hotkeys.com
humanorightswatch.org
hustoon.over-blog.com hustoon.over-blog.com
hvd-store.com ico.re
hydropump.su imagerydatabase.com
icaseclub.ru imguramx.pw
id-forex.com infektsii.com
ilikevitaly.com
imagez.co
imgata.com
infazavr.ru
infoupdate.org
instasexyblog.com
insurple.com
io9.com
iomoio.net
iplusbit.blogspot.co.za iplusbit.blogspot.co.za
irkutsk.online-podarki.com ipornox.xxx
iskalko.ru iqoption.pro
ispac.org iron-age.info
itsdp3.com it-max.com.ua
jasonpartington.com itch.io
jerseywholesaleelitestore.com itis4you.com
jestr.org ivoiretechnocom.ci
jillepille.com iyasimasennka.com
jmat.cn jav-fetish.com
jrcigars.com jav-way.site
justdating.online javrip.net
javxxx18.com
journeydownthescale.info
juliaworld.net
kamagragelusa.net kamagragelusa.net
kambasoft.com karting196.ru
katjimej.blog.fc2.com kazrent.com
kevblog.top kerch.site
keywordhut.com ketrzyn.pl
keywordsuggests.com kidd.reunionwatch.com
kino-ecran.ru kiinomaniak.pl
kinobest.pl kino2018.cc
kinocccp.net kinoduh.ru
kit-opt.ru kinogolos.ru
knowsitall.info klejonka.info
knowyournextmove.com klosetkitten.com
kongruan.com kosynka-games.ru
konica.kz kountrylife.com
ktotut.net kovesszucs.atw.hu
kutikomi.net krafte.ru
lafourmiliaire.com kupit-adenu.ru
lanadelreyfans.us kurbappeal.info
lawyers.cafe ladsup.com
le-clos-des-alouettes.com lasvegaslockandsafe.com
ldrtrack.com
ledis.top ledis.top
ledpolice.ru
lida-ru.com
lidoradio.com lidoradio.com
light.ifmo.ru life-instyle.com
lineavabit.it lifeinsurancekb.com
linkarena.com likrot.com
linkbux.ru linksharingt.com
linkdebrideur.xyz
lion.misba.us lion.misba.us
lion.snapmap.us littlesexdolls.com
lionking-1994.blogspot.com liver-chintai.org
listiseltemournlan.gq
ljusihus.se
lol-smurfs.com lol-smurfs.com
longgreen.info lonerangergames.com
lordzfilmz.me lostaruban.com
louboutinbooties.xyz lovasszovetseg.hu
m3gadownload.pl m-google.xyz
magicplayer-s.acestream.net m.b00kmarks.com
mastershef.club macdamaged.tech
masthopehomes.com mailemedicinals.com
matb3aa.com manualterap.roleforum.ru
matchpal-a.akamaihd.net maridan.com.ua
mature.free-websites.com matpre.top
medicinacom.ru medmajor.ru
meendo-free-traffic.ga meendo-free-traffic.ga
mejoresfotos.eu mere.host.sk
meltwater.com mexicosleevegastrectomy.com
memberty.com mielec.pl
mexicotravelnet.com mikozstop.com
minneapoliscopiers.com minegam.com
mncrftpcs.com
mobioffertrck.com
mobplayer.ru mobplayer.ru
moinozhki.com mockupui.com
mompussy.net moivestiy.biz
mountainstream.ms mojowhois.com
mrbojikobi4.biz moneytop.ru
muycerdas.xxx moroccosurfadventures.com
myonigroup.com moscow.online-podarki.com
myphotopipe.com mp3films.ru
myprintscreen.com mp3ringtone.info
nakozhe.com musicktab.com
nalogovyykodeks.ru musicvidz.ru
new-apps.ru muztops.ru
new-post.tk mybinaryoptionsrobot.com
newstraveller.ru mydirtyhobby.com
newstudio.tv mydoctorok.ru
newyorkhotelsmotels.info mypets.by
normalegal.ru myra.top
notaria-desalas.com narco24.me
notebook-pro.ru naturalshair.site
nuker.com negociosdasha.com
oakleyglassesonline.us neobux-bg.info
obiavo.com nero-us.com
offer.camp netanalytics.xyz
oneclickfiles.com niki-mlt.ru
online-hit.info nootrino.com
onlinebay.ru notfastfood.ru
onlinefilmz.net novodigs.com
onlinewritingjobs17.blogspot.ru npoet.ru
onload.pw nudejapan.net
onthemarch.co obiavo.kz
ooomeru.ru offf.info
outpersonals.com office-windows.ru
oktube.ru
oneminutesite.it
onlainbesplatno.ru
onlineslotmaschine.com
onlywoman.org
oops-cinema.ru
outrageousdeal-a.akamaihd.net outrageousdeal-a.akamaihd.net
pc-services.ru ovirus.ru
pcimforum.com paidonlinesites.com
pechikamini.ru partner-trustworthy.men
pinup-casino1.ru partnersafe.men
pc-virus-d0l92j2.pw
peekyou.com
pendelprognos.se
perosan.com
phobia.us
photorepair.ru
picturesify.com
piluli.info
pio.polytopesexempt.com pio.polytopesexempt.com
pixgood.com pixgood.com
pizda.lol pod-muzyku.club
plastgran.com polska-poezja.com
play.leadzupc.com pons-presse.com
polybuild.ru pooleroadmedicalcentre.co.uk
popserve.adscpm.net porndl.org
pornhubforum.tk pornosemki.info
pornofeuer.com pornotubs.com
pornoinn.com pornzone.tv
pornonik.com powitania.pl
powc.r.ca.d.sendibm2.com praisong.net
pozdravleniya-c.ru princevc.com
pozdrawleniya.ru printdirectforless.com
pozvonim.com privatbank46.ru
prefersurvey.net prodess.ru
preparevideosafesystem4unow.site producm.ru
prize44.com progonrumarket.ru
pro-tec.kz prohoster.info
profitfx.online prointer.net.ua
profitwithalex.info promgirldresses.xyz
promoforum.ru psoriasis-file.trade
prostitutki-kiev.org qitt.ru
pssucai.info quality-traffic.com
purchasepillsnorx.com radiodigital.co
purplestats.com razleton.com
pussysaga.com
pussyspace.net
quick-seeker.com
rangjued.com
rankia.com
ranksignals.com
ranksonic.com
rasteniya-vs-zombi.ru
razorweb-a.akamaihd.net razorweb-a.akamaihd.net
redirectingat.com realnye-otzyvy.info
reimageplus.com remont-mobile-phones.ru
reklama-i-rabota.ru rent2spb.ru
relatodelpresente.com.ar rff-cfal.info
remont-komputerov-notebook.ru rfserial.net
remont-ustanovka-tehniki.ru rp9.ru
remybutler.fr
rniaeba.ga
robotixix.com
royalads.net
rozalli.com
ru-mediaget.ru
rubbed.us rubbed.us
rucrypt.com rucrypt.com
russintv.fr rumamba.com
rvtv.ru rustic-quiver.win
sa-rewards.co.za s1z.ru
saletool.ru sajatvelemeny.com
samara.rosfirm.ru salmonfishingsacramentoriver.com
sanjosestartups.com sankt-peterburg.nodup.ru
santasgift.ml santasgift.ml
sardinie.us sardinie.us
satoristudio.net savefrom.com
scanner-don.top scanner-don.top
scanner-jack.top scanner-john.top
scanner-nelson.top scansafe.net
scopich.com se-welding.ru
searchwebknow-a.akamaihd.net search.pch.com
semaltmedia.com securityallianceservices.com
seokicks.de see-your-website-here.com
seorankinglinks.us seeingmeerkat.com
serdcenebolit.com semxiu.com
seoanalyses.com
seocheckupx.com
seonetwizard.com
serpstat.com serpstat.com
service.adtech.us sevenstars7.com
sex-znakomstva.online sexblog.pw
sexspornotub.com sexsaoy.com
sharebutton.to sextracker.de
shop-electron.ru sexxdate.net
sibvitr.ru sexy-pings.com
sildenafilcitratemed.com shitmovs.com
sisiynas.ru sildenafil-tadalafil.info
site-analyzer.com sms2x2.ru
slutloadlive.com sneakyboy.com
snjatie-geroinovoy-lomki.ru soc-econom-problems.ru
soaksoak.ru sodexo.com
socialmonkee.com sofit-dmd.ru
solnplast.ru soft-terminal.ru
soserfis.com
sotechco.co
spamnuker.com spamnuker.com
spb-plitka.ru spy-sts.com
spidtest.org
spylog.com spylog.com
spymac.net streamin.to
sterva.cc streetfooduncovered.com
stillmiracle.com
stop-gepatit.te.ua
stromerrealty.com
stroymonolit.su
superiends.org superiends.org
svnuppsalaorebro.se supernew.org
sweepstakes.rewardit.com superstarfloraluk.com
swimpool.ca svensk-poesi.com
tagil.zrus.org t-bygg.com
tattooreligion.ru tds-advert005.info
teastory.co teesdaleflyballclub.co.uk
tecnoteakviareggio.it texbaza.by
tedxrj.com thecoolimages.net
teenporn18.net tizanidinemusclerelaxant.blogspot.com
telefonsexkostenlos.tk tizanidinepain.blogspot.com
terrafootwear.us tizanidinepills.blogspot.com
testingads.pro tizanidinestreetprice.blogspot.com
thebestweightlosspills.ovh tizanidinewithdrawalsymptoms.blogspot.com
thedownloadfromwarez.blogspot.com tkanorganizma.ru
thepokertimer.com tonerbox.kz
timhost.ru toonfamilies.net
tizanidineonline.blogspot.com tooplay.com
tmearegion26.com topappspro.com
tmtrck.com topmira.com
token-lab.org tozup.com
top10registrycleaners.com trafficgenius.xyz
topanasex.com tri-slona.org
topsiteminecraft.com
track112.site
triplepanda.xyz
truck-rece.life truck-rece.life
trustl.life trucri.me
tubo360.com trustedhealthtips.com
tupper-posuda.ru turkeyreport.tk
twincitiescarservice.com turvgori.ru
ua-company.ru tvoystartup.ru
ucoz.ru tvteleport.ru
udayavani.com twitlinks.com
ukrtextbook.com tx41tclega.ru
unacittaconte.org typer.one
usacasino.com uac.net.au
v720hd.ru umaseh.com
vacances-voyages.info univerfiles.com
valkiria-tk.ru uptime-alpha.net
uptimebot.net
valueclick.com
vapsy.com vapsy.com
vduplo.ru vesnatehno.ru
ventelnos.com video-chat.in
vilingstore.net video-hollywood.ru
vintontech.info videotuber.ru
vinylvault.co.uk vip-dom.in
vizag.kharkov.ua vip.51.la
vkgaleria.com vipcallsgirls.com
vmnmvzsmn.over-blog.com virus-respirators.com
vod.com.ua voloslove.ru
vodkoved.ru vsesubwaysurfers.com
vseigru.one vzlomsn.org
vtcdns.com wanker.us
vucms.com warning.or.kr
vzlomtw.com webcamdevochka.com
walkme.com webmonetizer.net
web-betting.ru weboptimizes.com
web.cvut.cz website-audit.com.ua
webalan.ru websiteaccountant.de
weprik.ru wesharepics.info
westum.se wesharepics.site
whatsappdownload10.com
whatsupinfoley.com
whereverdesperate.gq whereverdesperate.gq
wieseversa.no wholesalejerseychinaoutlet.com
wingsofrefuge.net wholesalejerseys-cheapest.com
woman-h.ru whos.amung.us
worldmusicfests.com witclub.info
writingservices17.blogspot.ru www.souvenirua.com
wstroika.ru wyniki-lista.pl
www.get-free-traffic-now.com x-lime.com
xchat26.myfreecams.com xgames-04.com
xn--80aaagvmjabrs1aoc9luc.xn--p1ai xn----9sbubg3ambdfl1j.xn--p1ai
xn--80aafb2a.xn--p1ai xn----ctbigni3aj4h.xn--p1ai
xn--b1agm2d.net xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
xolodremont.ru xn--e1afanlbnfckd7c3d.xn--p1ai
xtrafficplus.com xnxx699.com
xxxnatelefon.ru xvideosporn.biz
yellowads.men xxxasianporn.net
yougetsignal.com xxxkaz.org
youjizz.com xxxsiterips.xyz
yurgorod.ru xz618.com
zajm-pod-zalog-nedvizhimosti.ru yatrk.xyz
zaloadi.ru yoluxuryevents.com
zarabotat-na-sajte.ru youtoner.it
zebradudka.com zajm-zalog-krasnodar.ru
zeroredirect6.com zeroredirect5.com
zeroredirect8.com zetmaster.ru
zhuravlev.info zigarettenonlinekaufen2.bloog.pl
zmoda.hostreo.com zog.link
zoominfo.com zojirushi-products.ru
zvezdagedon.ru zoolubimets.ru
zrizvtrnpale.tk
zvooq.eu
zx6.ru zx6.ru
zytpirwai.net

488
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00it.com 1001watch.com.ua
03e.info 123movies.love
0daymusic.org 1bet.com
101raccoon.ru 34.gs
1millionusd.xyz 45en.ru
21h2o.com 4replicawatch.net
273-fz.ru 51unlim.ru
4webmasters.org 6hopping.com
addray.pro 8558l.jobs.net
adtiger.tk a14download.com
advokateg.ru adclickservice.com
alfapro.ru adimmix.com
alice.it advancedmassagebysara.com
allsilver925.co.il alarmobninsk.ru
allstatesugarbowl.org alienwheel.es
allwidewallpapers.com allboard.xobor.de
am-se.com alldubai.biz
amyfoxfitness.com allornamenti.com
androids-store.com allpdfmags.net
animaltoplist.com alphavisions.net
api.stathat.com am15.net
anlimebel.kiev.ua
app-ready.xyz app-ready.xyz
appfastplay.com arabsexxxtube.com
applyneedy.xyz articlesdirectoryme.info
ayerbo.xhost.ro asseenontv.ru
backiee.com atlasvkusov.ru
balans.shahterworld.org ausergrubhof.info
bankmib.ru azlex.uz
basedecor.ru bambi.ck.ua
becuo.com bavsac.com
belinvestdom.by beatifullandscape.co
bestwaystogetpaid.us bestbrainprod.win
betwinservice.com betterscooter.com
bnt-team.com beyan.host.sk
bolitgorlo.net big-cash.net
bongacams.com bizcheapjerseyswholesalechina.com
bizfly.info
bizlist.com.de
bizru.info
blobar.org
bonky.biz bonky.biz
book-bets.com boole.onlinewebshop.net
bosman.pluto.ro buttons-for-your-website.com
businesn.men buzzurl.jp
buy-cheap-pills-order-online.com c-gainsbourg.com
buyantiviralwp.com cardul.ru
buyclomidonlaine.com carmuffler.net
buzzonclick.com cayado.snn.gr
callawaygolfoutlet.online ccjp.eu
carsplate.com cenokos.ru
cete.ru cheapjerseysap.com
chee-by.biz cmd.kz
chinaelitecheapjerseys.com co.lumb.co
citysecurity.nu collegerentals.com
cleansearch.net compliance-barak.xyz
clickbank.net conocer-sanabria.com
clickso.com
colehaanoutlet.store
com-supportcenter.website
compliance-mary.top
compucelunlock.net
coolwallpapers-hd.com
coolyarddecorations.com coolyarddecorations.com
copro.pw creditnation.ru
counterbot.com damedingel.ya.ru
crackguru.tk dengi-pod-zalog-nedvizhimosti.ru
crackzplanet.com directrev.com
cranly.net dlya-android.org
crirussian.ru doctormakarova.ru
cubook.supernew.org documentbase.net
cvety24.by doeco.ru
darmebel.com.ua dogoimage.com
daymusam.com domashneeruporno.com
dcj-nn.ru download-of-the-warez.blogspot.com
dev78.dioniqlabb.se dwomlink.info
digital-sale.su easyshoppermac.com
divatraffic.com ekn-art.se
dojki365.online ekobata.ru
dumuelave.xyz ekto.ee
elkacentr.ru engines-usa.com
essaypro.com erotag.com
estelight.ru evogarage.com
euroskat.ru extlabs.io
exchangeit.gq f-online.de
expresstoplivo.ru fakehandbags.xyz
extremal-blog.com fanpagerobot.com
ezigarettenshop2.postbit.com feel-planet.com
f-loaded.de felizporno.com
fatfasts-4tmz.com finansov.info
fermersovet.ru findacheaplawyers.com
ferretsoft.com finteks.ru
flashlarevista.com flprog.com
footbalive.org forminecrafters.ru
fotosfotos.eu fortevidyoze.net
fotoxxxru.com forum20.smailik.org
foxydeal.com foto-sisek.porngalleries.top
freejabs.com fotzen-ficken.com
freshsuperbloop.com free-deals.faith
frustrated-favorable.gq freenode.info
fullfileaccess.com front.ru
g7m.pl fx-brokers-review.com
gamebackyard.com gamebackyard.com
garciniaxt.us gaming-journal.com
gaytube.com geilehausfrauen.net
ge0ip.com genetworx.com
geoads.com getridofstretchmarks.org
girlsatgames.ru gg-arena.ru
greatgrace.ru glogow.pl
gsasearchengineranker.space goodprotein.ru
gta-club.ru gowreckdiving.com
hausfrauensex18.com grand-chlen.ru
hd-film.pl gsasearchengineranker.xyz
hdxnxxtube.mobi guardlink.com
helvetia.com.ua himazin.info
herokuapp.com homeinns.com
histats.com homelygarden.com
homeart.space imagecoolpub.com
hotcore.info imediadesk.com
hotelcrocenzi.sm importchinacoach-teach.com
hothot.ru infobanks.ru
hoverboard360.de infokonkurs.ru
hulfingtonpost.com iscblog.info
ideawheel.com itis4you.com
imguramx.pw jasonpartington.com
insider.pro jeremyeaton.co
investmac.com jetsli.de
inzn.ru jpcycles.com
istanbulit.com kakadu-interior.com.ua
izhevsk.zrus.org karadene.com
janavibekken.no kazka.ru
jose.mulinohouse.co kevblog.top
kchaxton.com keywords-monitoring-success.com
kenaba.su kino-rf.ru
keywordsuggests.com klitimg.pw
kino2018.cc konpax.com
kinosed.net kraljeva-sutjeska.com
krasivye-devushki.net larutti.ru
ktotut.net life.biz.ua
ku6.com lifebyleese.com
kvartira-sutochno.com livesex-amateure.info
lacloop.info ljusihus.se
leadwayau.com machicon-akihabara.info
lerporn.info macotool.com
lignofix.ua mebel-vstroika.ru
lomza.info mesto-x.com
lumb.co meteocast.net
m3gadownload.pl milblueprint.com
mac-shield.com mixed-wrestling.ru
marcoislandvacations.net mmstat.com
martlinker.com mobicover.com.ua
maximilitary.ru mobilierland.com
mb140.ru mosrif.ru
meddesk.ru mrinsidesales.com
megahdporno.net mycouponizemac.com
meget.co.za myra.top
michaelkorssaleoutletonline.net nalogovyykodeks.ru
mir-betting.ru narkologiya-belgorod.ru
mister-shop.com narosty.com
mnogolok.info negociosdasha.com
modenamebel.ru newfilmsonline.ru
mojaocena.com notasprensa.info
moncleroutletonline.win o-o-6-o-o.com
mosdverka.ru office2web.com
mozello.ru ohmyrings.com
mrlmedia.net online-x.ru
musezone.ru online-zaymy.ru
musirc.com outshop.ru
national-today-winning-winner.club pathwhelp.org
net-radar.com perederni.net
notfastfood.ru pinapchik.com
novodigs.com pl-vouchers.com
novosti-hi-tech.ru popmarker.com
nsatc.net pornokajf.com
olgacvetmet.com pornonik.com
online247.ml pornotubs.com
ororodnik.goodbb.ru prizesbook.online
outpersonals.com prosperent.com
painting-planet.com pwwysydh.com
paparazzistudios.com.au quickcashlimited.com
pattersonsweb.com qwertty.net
perosan.com relax.ru
peugeot-club.org remedyotc.com
phpdevops.com sack.net
pijoto.net saltspray.ru
pinstake.com sbtdesign.co.uk
porn-w.org school-diplomat.ru
porndroids.com scrnet.biz.ua
pornolook.net search.xtconnect.com
pornowarp.info searchimpression.com
porto.abuilder.net sexphoto.site
princevc.com shikiso.info
promalp-universal.ru siha.de
pyramidlitho.webs.com sim-service.net
quebec-bin.com sirpornogratis.xxx
ranksays.com sitopreferito.it
rapidsites.pro
redirlock.com
robotixix.com
sakhboard.ru
scanner-elena.top
scanner-jane.top
schoolfiles.net
searchtooknow-a.akamaihd.net
seorankinglinks.us
sex-foto.pw
sexs-foto.com
shemalegalls.blogporn.in
sherlock.se
skladvaz.ru skladvaz.ru
softxaker.ru softtor.com
sribno.net solution4u.com
stockspmb.info sosdepotdebilan.com
stop-zavisimost.com souvenirua.com
start.myplaycity.com
steebook.com
stpicks.com stpicks.com
suchenindeutschland.com success-seo.com
supermodni.com.ua suhanpacktech.com
sureone.pro superinterstitial.com
svetodiodoff.ru swiped.su
tattooha.com tastyfoodideas.com
templates.radiodigital.co telemetryverification.net
tvand.ru thesmartsearch.net
twinderbella.com thewebsitetemplate.info
twitlinks.com tizanidinepills.blogspot.com
ukkala.xyz tozup.com
ukrtvir.com.ua tri-slona.org
unblocksit.es triumf-realty.ru
velen.io tsyndicate.com
versaut.xxx-cam.webcam tvorozhnaja-zapekanka-recept.ru
viagraneggrx.com tx41tclega.ru
viberdownload10.com ukrtvory.in.ua
vipromoffers.com ultimatesetnewfreeallsoftupgradesystems.pw
vkontarkte.com urlcut.ru
voltrknc1.com vapsy.com
vozbujdenie.com video--production.com
webads.co.nz videochat.mx
webcamtalk.net vietimgy.pw
welck.octopis.com vip-parfumeria.ru
wholinkstome.com vip2ch.com
wikes.20fr.com virus-schutzmasken.de
womens-journal.net vladhistory.com
xev.ru volume-pills.biz
xkaz.org voronezh.xrus.org
xn--80aafb2a.xn--p1ai vzglyadriv.kg
xsfetish.org walkme.com
xxxdatinglocal.us wallpaperstock.net
youradulthosting.com warningwar.ru
yourtemplatefinder.com waycash.net
youtubedownload.org website-analytics.online
zahvat.ru westen-z.life
zebradudka.com white-truck.life
zero1.it www.bookmaker-bets.com
zeroredirect5.com xitjw.info
zolotoy-lis.ru xlolitka.com
zophim.me xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--90acenikpebbdd4f6d.xn--p1ai
xtremeeagles.net
xxlargepop.com
yachts-cruise.info
yeartwit.com
yur-p.ru
zambini.ru
zlatnajesen.com
zoomovies.org

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.170 0; 10.1.0.40 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3407 ### Version: V4.2022.10.3408
### Updated: Thu Oct 13 10:04:21 UTC 2022 ### Updated: Thu Oct 13 22:01:17 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3407 ### Version: V4.2022.10.3408
### Updated: Thu Oct 13 10:04:21 UTC 2022 ### Updated: Thu Oct 13 22:01:17 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##