vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 10:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2903 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-07 10:09:26 +00:00
parent 8d355e2025
commit 6099561f4d
46 changed files with 96158 additions and 96158 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2902 #### Version: V4.2022.02.2903
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2902 #### Version: V4.2022.02.2903
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2902 #### Version: V4.2022.02.2903
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11884
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2902 #### Version: V4.2022.02.2903
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11888
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11888
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11888
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11888
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11888
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11888
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11888
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2902 2903

11888
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11888
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.229.17.53 0; 104.43.164.81 0;

11888
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11888
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11888
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11888
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.229.17.53 0; 104.43.164.81 0;

11888
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.229.17.53 0; 104.43.164.81 0;

11888
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AiHitBot
Bolt Bolt
Disco Extractor
Download Wonder GrapeshotCrawler
LinkScan Pockey
Ltx71 Pump
Quick-Crawler Reaper
WallpapersHD WebFetch
ZoominfoBot ZyBorg
s1z.ru eCatch
scalaj-http

314
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider
404enemy 404enemy
80legs
ADmantX
ALittle Client ALittle Client
Abonti Aboundex
Acunetix Acunetix
AfD-Verbotsverfahren AhrefsBot
AiHitBot AiHitBot
AllSubmitter AllSubmitter
AlphaBot
Anarchie Anarchie
Ankit Ankit
Anthill Anthill
Apexoo Aspiegel
Atomseobot Atomseobot
AwarioSmartBot AwarioSmartBot
BDCbot BBBike
BDFetch BDFetch
BackStreet
Badass Badass
Bandit Bandit
BatchFTP BatchFTP
Battleztar Bazinga
BetaBot BetaBot
BlackWidow
Blow Blow
BlowFish
BotALot BotALot
Brandprotect Brandprotect
BuiltWith Brandwatch
Bullseye Buck
BuzzSumo BuiltBotTough
CCBot CATExplorador
CODE87 CSHttp
Calculon Calculon
CazoodleBot
CensysInspect CensysInspect
CheeseBot Cocolyzebot
CherryPicker Copier
Chlooe CopyRightCheck
Claritybot
Cliqzbot
Collector
Copyscape
Cosmos
CrazyWebCrawler CrazyWebCrawler
Crescent Crescent
Curious
Custo Custo
CyotekWebCopy
DTS Agent
Devil Devil
Dirbuster Digincore
DigitalPebble
Disco
Discobot Discobot
DnBCrawler-Analytics Dispatch
DomainCrawler DittoSpyder
Drip DomCopBot
DomainAppender
DomainSigmaCrawler
DomainStatsBot
Dotbot
ECCP/1.0 ECCP/1.0
EMail Siphon Ecxi
Evil EirGrabber
Exabot EroCrawler
Ezooms Extractor
FDM FDM
FHscan Firefox/7.0
FemtosearchBot FlashGet
Fimap Fuzz
Foobot FyberSpider
Freeuploader GalaxyBot
FrontPage GetRight
Fyrebot GoZilla
GT::WWW GrabNet
Genieo GrapeFX
GermCrawler
GetWeb
Go!Zilla
GrapeshotCrawler
GridBot
HMView HMView
HTMLparser HTMLparser
HTTP::Lite
HTTrack
Haansoft Haansoft
Havij Havij
Heritrix Humanlinks
Hloader HybridBot
IDBTE4M IRLbot
Iblog Iria
Id-search
IlseBot
Image Fetch
Image Sucker
IndeedBot
Indy Library
InfoNaviRobot
Intelliseek
Internet Ninja
Iskanie Iskanie
IstellaBot IstellaBot
Jbrofuzz JOC Web Spider
Jetty Jetty
Joomla Jorgee
JustView Keyword Density
Jyxobot Kinza
Kenjin Spider LNSpiderguy
Leap LWP::Simple
LexiBot LeechFTP
Lftp Lightspeedsystems
LibWeb LinkWalker
Libwhisker Linkbot
LieBaoFast LinkpadBot
Likse
LinkScan
Linkdexbot
LinkextractorPro
LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
Ltx71 Lmspider
MQQBrowser MIIxpc
Magnet MSIECrawler
Majestic SEO Mag-Net
Majestic-SEO
MarkMonitor MarkMonitor
Mass Downloader
Masscan
Mata Hari Mata Hari
MauiBot
MeanPath Bot MeanPath Bot
Meanpathbot
Metauri
MicroMessenger
Microsoft Data Access Microsoft Data Access
Mister PiX Moblie Safari
Mojeek Mojeek
MolokaiBot Mojolicious
Morfeus Fucking Scanner Mozlila
NPbot Mr.4x3
Musobot
NICErsPRO
Name Intelligence
Navroad
NearSite NearSite
Needle
Nessus
NetAnts
NetLyzer
NetMechanic
Netcraft Netcraft
Nettrack
Netvibes Netvibes
Nimbostratus NextGenSearchBot
Nuclei Nibbler
Nutch Niki-bot
Not
OpenLinkProfiler
OpenVAS OpenVAS
Openfind
Openvas
OrangeBot OrangeBot
OrangeSpider OrangeSpider
OutclicksBot OutclicksBot
POE-Component-Client-HTTP PECL::HTTP
PageScorer Pavuk
Papa Foto PeoplePal
Picsearch Petalbot
PictureFinder Picscout
Piepmatz
Pimonster Pimonster
ProWebWalker Pockey
Pu_iN Pu_iN
Pump
PyCurl
QueryN Metasearch
Quick-Crawler
RSSingBot RSSingBot
Rankivabot RankFlex
ReGet RankingBot2
RepoMonkey Re-re
RealDownload
Reaper
RebelMouse
Recorder
RocketCrawler RocketCrawler
Rogerbot SBIder
SEOprofiler SEOprofiler
SISTRIX SISTRIX
SalesIntelligent SalesIntelligent
Screaming ScoutJet
Scrapy
ScreenerBot ScreenerBot
ScrepyBot SemanticJuice
Searchestate Semrush
SentiBot Seomoz
SeoSiteCheckup
Shodan
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
Sitebeam Siteimprove
SmartDownload Sitevigil
Snake Sociscraper
Sogou web spider
Sosospider
SpaceBison SpaceBison
SpankBot Stripper
Spanner
Spbot
Spinn3r
SuperBot SuperBot
SuperHTTP SuperHTTP
Suzuran Surfbot
T8Abot SurveyBot
Telesoft Swiftbot
The Intraformant Szukacz
Thumbor T0PHackTeam
TeleportPro
Telesphoreo
TheNomad
Titan Titan
Toata Toweyabot
Tracemyfile Tracemyfile
Trendictionbot Trendiction
True_Robot
Turingos
Twice Twice
Upflow UnisterBot
V-BOT
VB Project VB Project
Vagabondo VelenPublicWebCrawler
Virusdie Virusdie
Voil
Voltron
WASALive-Bot WASALive-Bot
WBSearchBot
WEBDAV WEBDAV
WISENutbot WISENutbot
WWW-Mechanize WWW-Mechanize
WWWOFFLE Wallpapers
Web Sauger WallpapersHD
Web Sucker Web Collage
Web Pix
WebCopier
WebEnhancer
WebFetch WebFetch
WebFuck WebGo IS
WebImageCollector
WebPix WebPix
WebSauger
WebStripper
WebSucker
WebWhacker
WebZIP WebZIP
Website Quester Webalta
Webshag
WebsiteExtractor
Webster
Whatweb Whatweb
Wonderbot Widow
WinHTTrack
Woobot Woobot
Wprecon
Xaldon WebSpider Xaldon WebSpider
Zermelo Xenu
Zeus Zauba
Zitebot
ZoomBot ZoomBot
ZoominfoBot ZumBot
ZyBorg adscanner
arquivo-web-crawler arquivo-web-crawler
arquivo.pt backlink-check
coccocbot-web coccocbot-web
crawl.sogou.com crawler.feedback
demandbase-bot
domainsproject.org
evc-batch
gopher gopher
heritrix instabid
internetVista monitor
ips-agent
isitwp.com
iubenda-radar iubenda-radar
lwp-request lwp-request
lwp-trivial magpie-crawler
mediawords mediawords
netEstate NE Crawler
oBot
pcBrowser
s1z.ru s1z.ru
seocompany.store seocompany.store
seoscanners seoscanners
seostar seostar
serpstatbot serpstatbot
siteripz siteripz
sp_auditbot sogouspider
spyfu
sysscan sysscan
trendiction.com trendiction.com
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webprosbot
xpymep1.exe
zgrab

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker 80legs
404enemy ADmantX
AIBOT Abonti
Aboundex
Acunetix Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
Alexibot Alexibot
Anarchy AllSubmitter
Ankit Alligator
AlphaBot
Anarchie
Anthill Anthill
Apexoo
Aspiegel
Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot BDCbot
BBBike BDFetch
BLEXBot
BackWeb BackWeb
Backlink-Ceck Badass
Barkrowler
BatchFTP BatchFTP
Battleztar Bazinga Battleztar Bazinga
BetaBot BetaBot
Black Hole
BlackWidow BlackWidow
Blackboard
BlowFish BlowFish
BotALot BotALot
Brandprotect Brandprotect
Brandwatch Buddy
BuiltBotTough BuiltBotTough
BuiltWith BuiltWith
BunnySlippers Bullseye
BuzzSumo
CATExplorador CATExplorador
CazoodleBot Calculon
CensysInspect
CheeseBot CheeseBot
CherryPicker
Chlooe Chlooe
Citoid Copier
Claritybot
Cocolyzebot
CopyRightCheck CopyRightCheck
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
CrunchBot
Curious Curious
Custo
DBLBot DBLBot
DTS Agent DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Devil Devil
DigitalPebble Discobot
Discoverybot DnBCrawler-Analytics
Dispatch DnyzBot
DittoSpyder DomainAppender
DomainCrawler DomainStatsBot
DomainSigmaCrawler Download Wonder
EMail Siphon Drip
EirGrabber EMail Wolf
Ebingbong
EroCrawler EroCrawler
Exabot Exabot
Extractor
ExtractorPro ExtractorPro
Extreme Picture Finder Ezooms
EyeNetIE FHscan
FDM
FemtosearchBot FemtosearchBot
Firefox/7.0 Fimap
Freeuploader Fyrebot
FrontPage GalaxyBot
FyberSpider GetRight
G-i-g-a-b-o-t GetWeb
GermCrawler
Getintent Getintent
Gigabot Go!Zilla
GoZilla Go-Ahead-Got-It
Gotit GrabNet
Grabber GrapeFX
Grafula
HMView
HTMLparser HTMLparser
HTTP::Lite HTTP::Lite
Haansoft HTTrack
Harvest Havij
Hloader Hloader
Humanlinks
IDBTE4M IDBTE4M
Iblog
Id-search Id-search
Image Fetch Image Sucker
IndeedBot InfoNaviRobot
Indy Library InfoTekies
InterGET Iskanie
Internet Ninja IstellaBot
JamesBOT JOC Web Spider
Jbrofuzz Jbrofuzz
JennyBot JikeSpider
Jetty Kinza
Jorgee
Kozmosbot Kozmosbot
LeechGet LNSpiderguy
Larbin
LexiBot LexiBot
Lftp Libwhisker
LieBaoFast
LinkWalker LinkWalker
Linkbot
Linkdexbot Linkdexbot
LinkpadBot LinkpadBot
LinqiaScrapeBot LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey Lipperhey
Lmspider Litemage_walker
MIDown tool Ltx71
MQQBrowser MFC_Tear_Sample
MJ12bot
MSFrontPage
MSIECrawler MSIECrawler
MarkWatch Majestic-SEO
Mass Downloader Majestic12
Mata Hari MauiBot
Mb2345Browser Mb2345Browser
MicroMessenger Mediatoolkitbot
MegaIndex.ru
Microsoft Data Access Microsoft Data Access
Mister PiX Mister PiX
Moblie Safari
Mojeek Mojeek
MolokaiBot Morfeus Fucking Scanner
Mozlila Mozlila
Msrabot
Musobot Musobot
NPbot Navroad
NearSite NearSite
Needle Needle
Nessus NetLyzer
NetMechanic NetMechanic
NextGenSearchBot NetZIP
Nibbler Netvibes
NimbleCrawler Nikto
Ninja Ninja
Not Not
Nuclei Nuclei
Octopus Nutch
Offline Explorer
OnCrawl OnCrawl
Openfind OpenLinkProfiler
OrangeSpider OpenVAS
OutfoxBot OutfoxBot
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageScorer PageThing.com
Picscout Pavuk
PictureFinder Picsearch
Pimonster Pimonster
Pixray
PleaseCrawl PleaseCrawl
ProPowerBot ProWebWalker
PyCurl Psbot
Pump
QueryN Metasearch QueryN Metasearch
RealDownload Quick-Crawler
RepoMonkey RankActiveLinkBot
SBIder RankurBot
SISTRIX Ripper
SEOkicks-Robot
SEOprofiler
SEOstats
SalesIntelligent SalesIntelligent
Scrapy ScanAlert
Screaming ScoutJet
ScrepyBot ScreenerBot
Searchestate Searchestate
SearchmetricsBot
Seekport Seekport
SemanticJuice Semrush
Seomoz SentiBot
Shodan SeoSiteCheckup
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteLockSpider
SiteExplorer
SiteSnagger SiteSnagger
SiteSucker
Sitebeam
Siteimprove Siteimprove
Sitevigil
SlySearch
Snake Snake
Snapbot Snapbot
SocialRankIOBot Sociscraper
Sogou web spider Sosospider
Sottopop Spammen
SpaceBison Spinn3r
SputnikBot
Sqworm Sqworm
SuperHTTP Stripper
Surfbot Sucker
SuperBot
SurveyBot SurveyBot
T0PHackTeam Swiftbot
Teleport Teleport
Telesphorep
TheNomad TheNomad
Thumbor Titan
TightTwatBot Toata
True_Robot Toweyabot
Turingos TurnitinBot
Turnitin
URLy Warning URLy Warning
UnisterBot URLy.Warning
Upflow V-BOT
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
Voltron VeriCiteCrawler
VidibleScraper
Virusdie
WBSearchBot
WEBDAV WEBDAV
WISENutbot
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE
Wallpapers/3.0
WeSEE WeSEE
Web Sucker Web Collage
Web Fuck
Web Sauger
WebAuto
WebEnhancer
WebFetch WebFetch
WebFuck WebImageCollector
WebLeacher
WebPix WebPix
WebWhacker WebWhacker
Webalta
WebsiteQuester
Whack
Whacker Whacker
Whatweb Widow
WinHTTrack WinHTTrack
WiseGuys Robot WiseGuys Robot
Wonderbot
Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Xenu ZmEu
YoudaoBot ZoomBot
Zauba
Zermelo
ZoominfoBot ZoominfoBot
ZumBot ZumBot
adscanner ZyBorg
autoemailspider arquivo-web-crawler
arquivo.pt
backlink-check
check1.exe
coccocbot-web
cognitiveseo cognitiveseo
com.plumanalytics com.plumanalytics
crawl.sogou.com
crawler.feedback crawler.feedback
eCatch crawler4j
evc-batch evc-batch
facebookscraper facebookscraper
gopher gopher
heritrix
meanpathbot
mediawords
muhstik-scan muhstik-scan
pcBrowser
polaris version polaris version
ripz ripz
scan.lol satoristudio.net
seobility
seocompany.store seocompany.store
seoscanners
seostar
sexsearcher
sogouspider
sp_auditbot
trendiction.com trendiction.com
ubermetrics-technologies.com webpros.com
voyagerx.com webprosbot
webmeup-crawler x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe xpymep1.exe
zgrab zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
12u.info callawaygolfoutlet.online
b00kmarks.com dailyfinancefix.com
compliance-ivan.xyz members.ghanaweb.com
erotiktreff24.info ruf777.com
jerseywholesalebizchina.com russintv.fr
nvssf.com strigkaomsk.ru
ororodnik.goodbb.ru svarkagid.com
profolan.pl traxdom.ru
simply.net turbodsp.com
xlovecam.com tvteleport.ru

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
178evakuator178.ru 007angels.com
1hwy.com 123movies.love
51unlim.ru 2728fb936f0.com
5forex.ru 2itech.ru
3-letter-domains.net
4k-player.pl
5ws.dating-app.ru
72-news.com
7fon.ru 7fon.ru
aa8b68101d388c446389283820863176e7.com 8jn.dating-app.ru
aaronabel.com a96527gi.beget.tech
aa625d84f1587749c1ab011d6f269f7d64.com
abctoppictures.net abctoppictures.net
absolute-s.ru abiente.ru
acunetix-referrer.com above.com
adman.gr absugars.com
adnotbad.com acmebtn.ml
adsref.men actices.com
adbetclickin.pink
adidas.frwebs.fr
adrenalin-stk.ru
adultnet.in adultnet.in
adultwebhosting.info advertex.info
advokateg.ru advokateg.xyz
aero2.ru afmuseum.com
aerogo.com alert-fdm.xyz
agreda.pluto.ro alert-fjg.xyz
airmaxshoes-2016.com alice.it
alert.scansafe.net alphaforum.ru
allboard.xobor.de alyeskaresort.com
allnews24.in amateurlivechat.org
alpharma.net amazon-seo-service.com
ameblo.top android4fun.org
andpolice.com animal36.com
android-systems.ru animali.deagostinipassion.it
anime.dougasouko.com animalrank.com
anticrawler.org anonymous-redirect.com
aosheng-tech.com aosexkontakte.net
apiallgeniusinfo-a.akamaihd.net apiappenableinfo-a.akamaihd.net
appartement-stumm.at apibetweenlinesn-a.akamaihd.net
apxeo.info apidigidocketnet-a.akamaihd.net
arabgirls.adultgalls.com apiwebwebgetcom-a.akamaihd.net
artavenuegardenstudios.com appfixing.space
asscenihotosidea.blogspot.com apps-analytics.net
ausergrubhof.info aramaicmedia.org
autoua.top arendas.net
availit.weebly.com autoseo-traffic.com
babs.com.ua autotop.com.ua
baditri.com avkzarabotok.info
baersaratov.ru azartclub.org
balois.worldbreak.com azlex.uz
banksrf.ru badavit.com.ua
barbourjackets.online bannerpower.com
bashtime.ru
bayareaaeroclub.org
bdsmgalls.net
beachdriveblog.com
beachfix.co beachfix.co
bear.gotcher.us berlininsl.com
becuo.com best-seo-offer.com
bedcapdealers.com
bestadbid.com
bestmarriages.com bestmarriages.com
betteroffers.review bestmouthwash.club
bg6s0.com bestpornuha.com
bharatdefencekavach.com bet365.com
billiard-classic.com.ua betune.onlinewebshop.net
binomo.com bigames.online
bio-japan.net bigpictures.club
bitcoin-ua.top biketank.ga
bitcoins-live.ru bio-market.kz
bitnote.co birzha-truda.eu
blogs.rediff.com bitporno.sx
bocoarchives.org biznesluxe.ru
boobsimge.com bjgugu.net.ua
boom-celebs.com blog.f00kclan.de
bouda.kvalitne.cz blog.yam.com
brateg.xyz bloggerads.net
buckspar.gq brainboosting.club
burger-imperia.com braindaily.xyz
business-suggested.tk breastaugmentation.co.za
buttons-for-your-website.com brimstonehillfortress.org
buyclomidonlaine.com buy-cheap-pills-order-online.com
call-of-duty.info buy-forum.ru
camsex.xxx-cam.webcam buy-loft.ru
candycrushshop.com buynorxx.com
cbb1smartlist12.click bytimedance.ru
cheapcigarettesc.info callcafe.info
cheapmarmot.online calstaterealty.us
chinajerseyswholesalecoupons.com cam-kontakte.org
chocolateslim-en-espana.com camdolls.com
chocolateslim-en-france.com carabela.com.do
chocolateslim-original.com cartierlovebraceletreplica.xyz
cb.iphantom.com
cdncash.net
cdnnetwok.xyz
cementaresearch.se
cenokos.ru
charmstroy.info
cheapjerseysbizwholesale.us
choice-credit.ru
christianlouboutinsaleoutletonline.us christianlouboutinsaleoutletonline.us
cialgenisrx.com
cialisovercounteratwalmartusa.com cialisovercounteratwalmartusa.com
claytransformations.info cjs.com.ru
clean-start.net clayvasedesigns.tk
clean-virus-mac.com clickintext.net
cleanersoft.com coaching-netz.info
clicksor.com collegeessay19.blogspot.ru
clips.ua.ac.be compliance-alexa.top
club-musics.ru compliance-barak.top
cobaltpro.ru compliance-george.top
colobit.biz compliance-irvin.xyz
com-onlinesupport.host compliance-olga.top
com-supportcenter.website coprofam.org
compliance-don.top cornerstone-countertops.com
compliance-ivan.xyz
construmac.com.mx
cookie-law-enforcement-aa.xyz
cookielawblog.wordpress.com
coolgamechannel.com
cosmeticswomens-womensports.rhcloud.com
counter.bloke.com
crazyboost.pro
credit-respect.ru credit-respect.ru
cryptoswap.biz credit.co.ua
danceuniverse.ru cxpromote.com
darodar.com cyprusbuyproperties.com
deda-moroza-zakaz.ru dairyindia.in
detoxmed24.ru dangphoto.trade
diegolopezcastan.com dating-app.ru
dikqlyremy.info decenttools.com
dildofotzen.net decorationspcs.com
discountbarbour.online digital-video-processing.com
discountliv.com dlya-android.org
diusyawiga.tk dnepropetrovsk.mistr-x.org
dmmspy.com domaincdn.xyz
dojki365.online domcran.net
domain2008.com drivotracker.com
dominateforex.ml dynainbox.com
domination.ml
donna7753191.ru
downloadmefiranaratb1972.xpg.com.br
dpihatinh.gov.vn
dprtb.com
dptaughtme.com
draniki.org
drev.biz
e-collantes.com e-collantes.com
easycommerce.cf easyshoppermac.com
editmedios.com educontest.net
ee77ee.com egovaleo.it
ekn-art.se
ekobata.ru
ekosmetyki.net.pl
election.interferencer.ru election.interferencer.ru
electrik-avenue.com elektir.ru
elektrischezigarette2.devhub.com
elektrischezigaretten1.blogse.nl elektrischezigaretten1.blogse.nl
energydiet-info.ru elektronischezigarettekaufen1.beeplog.com
esnm.ru elektronischezigarettekaufen2.tumblr.com
elektrozigarette2.webs.com
elektrozigarettekaufen1.devhub.com
emailaccountlogin.co
empis.magix.net
en.altezza.travel
enskedesquashclub.se
erotik-kostenlos.net
erotiktreff24.info
essayservicewriting.org
eupornstar.info eupornstar.info
exoclick.com euronis-free.com
exportshop.us europages.com.ru
extads.net executehosting.com
extremal-blog.com expertnaya-ocenka.ru
ezigarettekaufen2.mpbloggar.se eyessurgery.ru
ezigarettenkaufen2.dreamwidth.org ezaz.info
ezofest.sk ezigarettekaufen2.blox.pl
f1nder.org ezigarettenkaufen1.deviantart.com
faptitans.com ezigarettenkaufen1.pagina.gr
fashionindeed.ml ezrvrentals.com
favornews.com f012.de
ferieboligkbh.dk familienzahnaerzte.com
fickenbumsen.net fanrto.com
finansov.info farm26.ru
find1friend.com fastcrawl.com
firesub.pl feeriaclub.ru
flowwwers.com fetlifeblog.com
foodcrafts.website filefilter.weebly.com
forexgb.ru filesmonster.porn
forpostlock.ru fingerprintjs.com
forum-engineering.ru fiverr.com
foto-basa.com flipper.top
foxweber.com folowsite.com
freeseedsonline.com forex.osobye.ru
fuck-paid-share-buttons.xyz forex21.ru
funnel.co.za forms-mtm.ru
funnymama.com fortevidyoze.net
fz139.ttk.ru forum.doctissimo.fr
galleryawesome.com forum69.info
gamedayhouse.com foxydeal.com
fr-bearings.ru
frankofficial.ru
free-share-buttons.blogspot.com
freelinkbuilding.website.tk
freewareseek.com
freshberry.com.ua
freshsuperbloop.com
frustrated-favorable.gq
fuckmill.com
funtoonez.com
g7m.pl
gamewrath.com gamewrath.com
gateway.zscalerone.net gateway.zscalertwo.net
gembird.com gaygalls.net
generalporn.org gdebestkupit.ru
generousdeal-a.akamaihd.net geckoandfly.com
get-seo-domain.com getridofstretchmarks.org
getdot.ru ghazel.ru
getmyads24.com ghernnqr.skyrock.com
getprismatic.com glavtral.ru
gonextmedia.com glogow.pl
googlsucks.com gnuetella.com
gosarhivrt.ru gowreckdiving.com
gourcy.altervista.org gox.com.ua
gratuitbaise.com gratuitxblcodes.com
guardlink.org gwebtools.com.br
guod.me haikuware.com
healbio.ru handicapvantoday.com
hentaiheroes.com heatpower.ru
hitmuzik.ru helicalpile.us
hjaoopoa.top highland-homes.com
homedecoguide.info hitcpm.com
hitsbox.info
homelygarden.com
hotdl.in
hothot.ru hothot.ru
howmuchdoestizanidinecost.blogspot.com housemilan.ru
hspline.com iccornacircri.cf
hully.altervista.org iloveitaly.ru
hyipmanager.in imagecoolpub.com
i4track.net imediadesk.com
iboss.com img.wallpaperstock.net
igithab.com imgarit.pw
ikritikimou.gr importchinacoach-teach.com
ilmen.net increasewwwtraffic.info
ilovevitaly.xyz indetiske.ya.ru
imagui.eu infoupdate.org
imgarcade.com inspiring-desperate.tk
imk.com.ua intellego.info
immigrational.info interferencer.ru
inet-traffic.com iqoption-bin.com
infokonkurs.ru iqoption.com
interesnie-faktu.ru ispaniya-costa-blanca.ru
interior-stickers.ru izhevsk.zrus.org
invitefashion.com japfm.com
iron-age.info jav-fetish.site
iscblog.info jerseysbizwholesalecheap.com
isotoner.com jjbabskoe.ru
it-max.com.ua jpcycles.com
itis4you.com jrpmakati.com
itronics.ca juliaworld.net
javlibrary.cc justkillingti.me
javrip.net justucalling32211123456789.tk
jetsli.de
jillepille.com
joy-penguin.com
kaac.ru kaac.ru
kam-dom.ru kakablog.net
karachev-city.ru kandidos.com
kaz.kz kinopolet.net
keywordglobal.co.za kinozapas.com
keywords-monitoring-success.com klejonka.info
kino-rating.ru klosetkitten.com
kinobaks.com knigonosha.net
kinohit1.ru kochanelli.com
klumba55.ru kupit-adenu.ru
komp-pomosch.ru kustanay.kz
konica.kz labelwater.se
koptims.tiu.ru law-check-two.xyz
lampokrat.ws lescinq.com
laptoper.net listiseltemournlan.gq
larose.jb2c.me lizace.com
lenpipet.ru lovasszovetseg.hu
levaquin750mg.blogspot.com lyrster.com
lflash.ru m292.info
linkbux.ru m3gadownload.pl
liveshoppersmac.com marcogrup.com
lkbennettoutlet.online maslenka.kz
lkbennettstore.online master-muznachas.ru
llastbuy.ru masterseek.com
lmrauction.com mastershef.club
loadingpages.me matematikus.info
lob.com.ru mebel-alait.ru
lol-smurfs.com meccadumps.net
lonfon.xyz medkletki.ru
lumb.co memberty.com
magento-crew.net meteostate.com
magiadinamibia.blogspot.com minegam.com
maglid.ru mirzonru.net
maladot.com mlvc4zzw.space
marsgatan.com monclerjacketsoutlet.pw
maxthon.com moneyzzz.ru
medmajor.ru morf.snn.gr
medtherapy.ru movies-in-theaters.net
mega-bony-2017.pl mturkcontent.com
member-quiz.com mulberryoutletonlineeu.com
merryhouse.co.uk my-cash-bot.co
mex-annushka.ru mydearest.co
middlerush-a.akamaihd.net mydownloadengine.com
mil-stak.com mymobilemoneypages.com
millionare.com nacap.ru
minet.club nationalbreakdown.com
missis.top news-readers.ru
mobplayer.net newtechspb.ru
moesonce.com
mokrayakiska.com
mole.pluto.ro
monclercheap.xyz
monclerjacketsoutlet.win
moskva.nodup.ru
moviemail-online.co.uk
mrwhite.biz
mug-na-chas-moscow.ru
mundoaberrante.com
muz-baza.net
muz-tracker.net
mwtpludn.review
mylida.org
mzdish.site
na-telefon.biz
netoil.no
newhairstylesformen2014.com
newyorkhotelsmotels.info
nextlnk12.com
ngps1.ru
nhl17coinsforps3.gratisblog.biz nhl17coinsforps3.gratisblog.biz
nodup.ru nodding-passion.tk
norththeface.store nofreezingmac.click
notebook-pro.ru nomuos.it
november-lax.com nonameread45.live
nullrefer.com nordstar.pro
obiavo.in noumeda.com
obnal.org novatech.vn
obsessionphrases.com nrv.co.za
oda.as o-o-6-o-o.com
odywpjtw.bloger.index.hr obiavo.kz
offer.camp oecnhs.info
onetravelguides.com offside2.5v.pl
okroshki.ru
old-rock.com
olympescort.com
online-hit.info
onlinebay.ru
onlinefilmz.net
onload.pw
onlywoman.org
openmediasoft.com
openmultipleurl.com openmultipleurl.com
orenburg-gsm.ru orion-v.com
osagonline.ru oxotl.com
osb.se11.ru
otbelivanie-zubov.com
p-business.ru
paidonlinesites.com
painting-planet.com painting-planet.com
palocco.it panamaforbeginners.com
panchro.xyz parlament.biz
paramountmarble.co.uk partnerads.men
perfection-pleasure.ru partnerworkroom.men
photokitchendesign.com pdamods.ru
pictures-and-images.net perm-profnastil.ru
pinwallpaper.top pflexads.com
pl.id-forex.com pharmacyincity.com
plastweb.ru piccdata.com
play.leadzupc.com pigrafix.at
pinstake.com
platesauto.com
playfortuna-play.ru playfortuna-play.ru
pliks.pl plugingeorgia.com
popugauka.ru polcin.de
pornoinn.com polimga.pw
poshiv-chehol.ru pornoblood.com
pornotubexxx.name
predmety.in.ua predmety.in.ua
privatefx.all4invest.info pregnant.guru
primedice.com
princeadvantagesales.com
princevc.com
prizesbook.online prizesbook.online
pro-tec.kz prlog.ru
produkto.net prodvigator.ua
projefrio.com.br professionalwritingservices15.blogspot.ru
pron.pro prom23.ru
prostitutki-kharkova.org promgirldresses.xyz
q-moto.ru pronekut.com
qpypcx.com prostitutki-almata.org
ragecash.com pts163.ru
rainbowice.ru pulseonclick.com
rangoman.date pwwysydh.com
rapidokbrain.com pyrodesigns.com.au
rcb101.ru qor360.com
rcpmda.ikan1080.xyz rabot.host.sk
redbottomheels.xyz rankexperience.com
redirect.trafficreceiver.club rczhan.com
relatodelpresente.com.ar real-time-analytics.com
replica-watch.ru realnye-otzyvy.info
revistaindustria.com rebuildermedical.com
robot-forex.biz redhotfreebies.co.uk
roll123.com rednise.com
rome2rio.com registrysweeper.com
royalcar-ufa.ru reining.lovasszovetseg.hu
rucrypt.com relax.ru
rusexy.xyz rennlist.com
sale-japan.com resant.ru
sankt-peterburg.nodup.ru respublica-otel.ru
scanner-donald.top rethinkwasteni.info
score-ads.men ringtonepartner.com
sdsjweb.com rocis.site
seansonline24.pl rocketchange.ru
search-goo.com rockingclicks.com
search.alot.com rogervivierforsale.com
searchengineranker.email ru-dety.ru
sell-fb-group-here.com sabizonline.com
seocheckupx.com sack.net
seocheki.net sad-torg.com.ua
seofied.com saletool.ru
seofirmreviewsus.info samoiedo.it
seoholding.com scanmarine.info
sex-dating.co scanmyphones.com
sexflirtbook.com scanner-alex.top
sexy.babes.frontend-stack.top scanner-josh.top
shareyards.com scanner-walt.top
shell-pmr.ru school-diplomat.ru
shinikiev.com.ua scripted.com
shodanhq.com seksotur.ru
shop.xz618.com seoboxes.com
sibecoprom.ru seomarketings.online
sientalyric.co seozoom.it
silktide.com serving.adbetclickin.pink
sinestesia.host.sk sex-foto.pw
skuteczna-dieta.co.pl sex-znakomstva.online
skypasss.com sexkvartal.com
skyway24.ru sexs-foto.com
sexsaoy.com
sexyteens.hol.es
shijian.ac.cn
shoesonlinebuy.cn
siha.de
sim-service.net
siteaero.com
skatestick.bid
sladkoevideo.com sladkoevideo.com
small-game.com slowmac.tech
soblaznu.net sms2x2.ru
spammen.de smsactivator.ru
specialfinanceoffers.com sneakyboy.com
sptslmtrafms.com snowplanes.com
spy-app.info soc-econom-problems.ru
socialseet.ru
sockshare.net
sofit-dmd.ru
soft-terminal.ru
sohoindia.net
sovinsteel.ru
space4updating.win
spravka-medosmotr.ru
spy-sts.com
srdrvp.com srdrvp.com
ssconstruction.co stanthonyscatholicchurch.org
stairliftsarea.com stardevine.com
strana-solnca.ru starpages.net
strangeduckfilms.com sukirgenk.dvrlists.com
stroymonolit.su supermesta.ru
subj.ukr-lit.com
supermodni.com.ua
supernew.org supernew.org
svtrd.com surfbuyermac.com
swingerseiten.com symphonyintegratedhealthcare.com
tackletarts.co szamponrevita.pl
taxi-v-eisk.ru tasteidea.com
technopellet.gr telesvoboda.ru
tecspb.ru theautoprofit.ml
templates.radiodigital.co thedownloadfromwarez.blogspot.com
thepornsex.org theendivechronicles.com
thewomenlife.com thefds.net
timecrimea.ru tizanidineandhydrocodone.blogspot.com
timhost.ru tizanidineforopiatewithdrawal.blogspot.com
titangel-vietnam.com tizanidinehydrochloride2mg.blogspot.com
tizanidinehcl2mgtablet.blogspot.com tnctrx.com
tizanidineorflexeril.blogspot.com
tizanidinepain.blogspot.com
tizanidinesleepaid.blogspot.com
token-lab.org
toolsky.com
tootoo.to
top-l2.com
top10registrycleaners.com top10registrycleaners.com
topcar-krasnodar.ru topbestgames.com
tovaroboom.vast.ru torontoplumbinggroup.com
toxicwap.com torrents-tracker.com
tracxn.com track2.shop
trafficgenius.xyz tracksurf.daooda.com
trafficjunky.com trafficfactory.biz
trainoffend.ml trudogolik.net
trustl.life trustl.life
tsstcorpcddvdwshbbdriverfb.aircus.com
tula.mdverey.ru
turist-strani.ru turist-strani.ru
turkeyreport.tk tversvet.ru
turn-up-life.life twojebook.pl
twiclub.in tyumen.xrus.org
typer.one uchil.net
ua-company.ru umnovocaminho.com
unionmarkt.de universals.com.ua
unrealcommander.com ural-buldozer.ru
ustion.ru v720hd.ru
vancleefreplica.pw
vapomnoncri.tk vapomnoncri.tk
vbabule.net vduplo.ru
via-energy-order.com viagra-soft.ru
viktoria-center.ru viberdownload10.com
video--production.com
videooko.weebly.com
vielporno.net
vinylvault.co.uk
vip-file.com vip-file.com
virus-schutzmasken.de voditeltrezviy.ru
visa-china.ru voloo.ru
viveresaniesnelli.it
voltrknc1.com
vostoktrade.info vostoktrade.info
vremya.eu vzlom-na-zakaz.com
vse-pesni.com we-are-gamers.com
vtcdns.com webkeyit.com
vykup-avto-krasnodar.ru webmasterhome.cn
vykupavto-krasnodar.ru webnode.me
wardreapptokone.tk websites-reviews.com
watchdogs-2.ru weburlopener.com
waterpurifier.club whatzmyip.net
wcb.su wildcattube.com
web.cvut.cz woweb.com.ua
webalan.ru xaylapdiendanang.com
website-analyzer.info xjlottery.com
wedding0venues.tk xmronta.com
weprik.ru xn----7sbaphztdjeboffeiof6c.xn--p1ai
westen-v.life xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
westvilletowingservices.co.za xn--90acenikpebbdd4f6d.xn--p1ai
while.cheapwebsitehoster.com xn--e1aggki3c.xn--80adxhks
wnoz.de xxxmania.top
womens-journal.net youradulthosting.com
woodyguthrie.se youresponsive.com
worldis.me yourmovies.pl
worldtraveler.world yourothersite.com
wormix-cheats.ru yousense.info
wyniki-lista.pl ypmuseum.ru
x-diesel.info zeikopay.com
x-mix.info zero1.it
xitjw.info zeroredirect12.com
xkaz.org zeroredirect8.com
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai zeroredirect9.com
xn----9sbubg3ambdfl1j.xn--p1ai
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--b1adccaf1bzj.xn--p1ai
xn--h1aakne2ba.xn--p1ai
xnxx699.com
xxxtube69.com
y8games-free.com
yaminecraft.ru
youjizz.vc
yubikk.info
zakon-ob-obrazovanii.ru
zeroredirect1.com
zeroredirect7.com
zigarettenonlinekaufen1.blox.pl
zigarettenonlinekaufen2.bloog.pl zigarettenonlinekaufen2.bloog.pl
zolotoy-lis.ru zionstar.net
zoogdiesney.com
zoogdisany.com
zootoplist.com zootoplist.com

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01apple.com 101lesbian.xyz
1-99seo.com 10bet.com
101flag.ru 11235813.webzdarma.cz
12masterov.com 123kuma.com
1kdailyprofit.me 12bet.com
20pascals.nl 2kata.ru
3th.co.in 2rich4bitches.com
5elementov.ru 3-letter-domains.net
98oi.ru 4senses.co
acads.net 5forex.ru
accessoires-mode.in adtiger.tk
ads.gold adtology.com
adsloads.com affordablewebsitesandmobileapps.com
aktivator-windows10.blogspot.com alfa-medosmotr.ru
akuhni.by all4invest.ru
alessandraleone.com alpinism.ru
antons-transporte.de amazon-seo-service.com
apiportalmorecom-a.akamaihd.net amigobulls.com
apple.com-cleaner.systems anaseracresar.tk
anmysite.com
app-ready.xyz
approvedlocal.co.za
aramaicmedia.org
architecturebest.com
arclk.net
arenda-avtoprokat-krasnodar.ru arenda-avtoprokat-krasnodar.ru
artdestshop.eu arquapetrarca.info
atleticpharm.org asrvrep-a.akamaihd.net
atmagroup.ru autotop.com.ua
auto-zapchasti.org avon-ukraine.com
avirasecureserver.com bankiem.pl
ayerbo.xhost.ro bear.gotcher.us
baksman.com beauty-things.com
bankmib.ru bestcoin.cc
bestpriceninja.com bestsexyblog.com
betonka.pro blackbabesporn.com
betterhdporn.com bloggerads.net
bezlimitko.xyz blogos.kz
blog.koreadaily.com bolegapakistan.com
bmw.afora.ru boltalko.xyz
bouda.kvalitne.cz bonux.nextview.ru
brewdom.ru bookmaker-bet.com
buehne-fuer-menschenrechte.de browsepulse-a.akamaihd.net
buketeg.xyz cakemediahq.com.au
buydissertation.net cartujano-pre.de
buzzzg.men catalogs-parts.com
calstaterealty.us cfcl.co.uk
cardsharp1.ru cjmc.info
cartierbraceletsreplica.pw clayvasedesigns.tk
changde.58.com cleanallvirus.com
cheatcode-lita12.rhcloud.com clickbank.net
choosecuisine.com collegeessay19.blogspot.ru
classiquebijoux.ru compliance-jack.top
clevernt.com compliance-mark.top
cmd.kz coolyarddecorations.com
com-cleaner.systems
cosmetic.donna7753191.ru cosmetic.donna7753191.ru
cpajunkies.com crafthubs.com
culad.com culturevie.info
customchocolate.business-for-home.com dodlive.mil
customergrowthsystems.com domination.ml
customerpromos-a.akamaihd.net downloadeer.net
cvety24.by drupa.com
defenderxtactical.com dyt.net
delayreferat.ru e-kwiaciarz.pl
deniven.1bb.ru earn-from-articles.com
dickssportinggoods.com ege-russian.ru
digitalassetmanagement.site elektrischezi.canalblog.com
div.as emergencyneeds.org
dividendo.ru englate.com
divku.ru erotiktreff24.info
dnepropetrovsk.mistr-x.org escort-russian.com
dogsrun.net fanpagerobot.com
domik-derevne.ru fast-wordpress-start.com
driving.kiev.ua faz99.com
eager-nash.188-93-233-196.plesk.page filmetricsasia.com
earncash.com.ua filmgo.ru
eas-seo.com firma-legion.ru
ec-file.info free-floating-buttons.com
ecommerce-seo.com free-share-buttons.blogspot.com
edge.sharethis.com friendflnder.com
ekspresihati.info
elektroniksigaraankara.info
energydiet-info.ru
esopini.com
executehosting.com
exmasters.com
exoclick.com
extrabot.com
failingmarriege.blogspot.com
farm26.ru
fastcrawl.com
fbdownloader.com
fickluder69.com
filesdatabase.net
finemanteam.com
flex4launch.ru
forex.osobye.ru
free-fbook-traffic.com
freemags.cc
freshdz.com
freshwallpapers.info
gambarkataku.co gambarkataku.co
game-mmorpg.net gamblingnerd.com
games.kolossale.ru gamevalue7.weebly.com
gelezki.com gaytube.com
gepezz.info generalporn.org
getpopunder.com ghostvisitor.com
ghernnqr.skyrock.com goodwriterssales.com
glass-msk.ru graphid.com
gloverid.site gsasearchengineranker.site
goblacked.com gsasearchenginerankerdiscount.com
goodprotein.ru gsbs.com.ua
guge.io gstatey.net
hackers-crackers.tk gta-club.ru
hcate.com guiadeserraazul.com
h2monline.com
hawaiielectriclight.com
hdapp1008-a.akamaihd.net hdapp1008-a.akamaihd.net
hoterika.com hildinghr.se
ifirestarter.ru host-protection.com
importchinacoach-teach.com hyipmanager.in
indetiske.ya.ru igru-xbox.net
investzalog.ru imagez.co
iradiology.ru immobiliaremassaro.com
istizanidineanopiate.blogspot.com instasexyblog.com
istizanidinelikexanax.blogspot.com intelhdgraphicsgtdrive6w.metroblog.com
iyasimasennka.com intermesh.net
javatex.co.id investingclub.ru
kladrus.ru iqs.biz.ua
krasivoe-hd.net istanbulit.com
lanadelreyfans.us istock-mebel.ru
laparfumotec.com james13prix.info
lesbian.xyz je7.us
light.ifmo.ru jennyfire.ru
locksmith.jp joyceblog.top
lostaruban.com kidskunst.info
mansiondelrio.co koleso24.com.ua
manve.info kongruan.com
martlinker.com konoplisemena.com
marwer.info koptims.tiu.ru
maslenka.kz krasivye-devushki.net
medicovi.com kustanay.kz
meduza-consult.ru lampokrat.ws
mejoresfotos.eu letslowbefast.today
metalonly.info linkbolic.com
mikozstop.com listiseltemournlan.gq
miniature.io liverpool.gsofootball.com
mnimmigrantrights.net lmrauction.com
mole.pluto.ro lonely-mature.com
moncleronline.xyz luchshie-topcasino.ru
morf.snn.gr magicdiet.gq
marcoislandvacations.net
medizinreisen.de
mega-bony-2017.pl
meteostate.com
milblueprint.com
mnogabukaff.net
mobplayer.net
modabutik.ru
moyaterapiya.ru moyaterapiya.ru
multgo.ru muz-baza.net
mybackgroundlandscape.blogspot.com mymercy.info
myfreecams.com narco24.me
myiptest.com narutonaruto.ru
mzdish.site nastydollars.com
nakrutka.cc net-radar.com
napalm51.nut.cc newstaffadsshop.club
naughtyconnect.com offtopic.biz
neobux-bg.info onlainbesplatno.ru
nmrk.ru
nordvpn.com
novatech.vn
novgorod.xrus.org
nsatc.net
nuker.com
offer.camp
online-x.ru
online247.ml online247.ml
onlinedomains.ru onlinedomains.ru
outshop.ru
ozoz.it
paceform.com paceform.com
pandroid.co panouri-solare-acoperis.com
paradontozanet.ru pc-services.ru
pateaswing.com perm.xrus.org
pcgroup.com.uy pills24h.com
perfectplanned.com pivka.xyz
perm-profnastil.ru pochemychka.net
picsfair.com popunder.ru
picswe.com poquoson.org
pierrehardysale.online pornogad.com
pihl.se
pisanieprac.info
ploenjitmedia.azurewebsites.net
pod-muzyku.club
pornoted.com pornoted.com
predominant-invent.tk preconnubial.usuby.site
prizestohandle.club privatbank46.ru
proftests.net proftests.net
prpops.com prom23.ru
psvita.ru proxyradar.com
r-control.ru pssucai.info
radiodigital.co pts163.ru
ratemodels.net radiogambling.com
reining.lovasszovetseg.hu ranksonic.info
remont-ustanovka-tehniki.ru razleton.com
reyel1985.webnode.fr remorcicomerciale.ro
sajatvelemeny.com replicalouboutin.xyz
sale-japan.com resant.ru
sanyuprojects.com resellerclub.com
scanner-elena.top responsive-test.net
scanner-willy.top rique.host.sk
seemoreresultshu-a.akamaihd.net rostov.xrus.org
senger.atspace.co.uk rustic-quiver.win
sexyteens.hol.es rvi.biz
shell-pmr.ru rvtv.ru
sientalyric.co sarahmilne.top
slkrm.ru sashagreyblog.ga
solinf.co scanmyphones.com
sr-rekneskap.no scat.porn
stairliftstrue.com scenicmissouri.us
styro.ru school-diplomat.ru
telemetryverification.net screentoolkit.com
telesvoboda.ru sdsjweb.com
tgtclick.com seoboxes.com
thegioixekhach.com sexflirtbook.com
thepornsex.org simpoed.ufop.br
thin.me.pn sluganarodu.ru
smokewithrabbits.com
social-button.xyz
social-vestnik.ru
spravka130.ru
srdrvp.com
stpicks.com
sugarkun.com
surcentro.com
swiped.su
taranerymagesswa.blogspot.com
tdsing.ru
tech4master.com
techart24.com
thecoral.com.br
thegameriders.com
tizanidine4mgprice.blogspot.com
tizanidinecapsules.blogspot.com
tizanidinehcl4mgtablet.blogspot.com
tizanidinevsvalium.blogspot.com tizanidinevsvalium.blogspot.com
traffixer.com topcar-krasnodar.ru
tramadolandtizanidine.blogspot.com tpu.ru
twitlinks.com tradgardspartner.se
uptime-gamma.net traffic2cash.org
vadimkravtcov.ru try-rx.com
vapsy.com turvgori.ru
veerotech.com twittruth.com
vektorpress.ru valueclick.com
vekzdorov.ru vedomstvo.net
veopornogratis.xxx vepad.com
veselokloun.ru vertaform.com
vgoloveboli.net vesnatehno.ru
video-chat.love videochat.life
vids18.site videochat.tv.br
villacoloniale.com vinsit.ru
vip-dom.in virus-respirators.com
vkgaleria.com vitoriacabos.com
vodkoved.ru vodkoved.ru
voprosotvet24.ru vseuznaem.com
walkme.com w-journal.ru
webix.me w3data.co
website-analyzer.info webmasterhome.cn
websitevaluebot.com webstatsdomain.org
weburok.com weclipart.com
whiteproduct.com weightatraining.com
wholesalenfljerseys.us.com whatistizanidinehclusedfor.blogspot.com
wieseversa.no winner7777.net
woweb.com.ua wrona.it
ws.ampower.me www.ehscloud.cn
x-true.info www.pinnacle-bets.com
xmlinde.com www.solartek.ru
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai x-diesel.com
xn--d1acah0c.xn--p1ai x69ty.ru
youtubologia.it xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
zhuravlev.info xn--1-8sbcpb0bdm8k6a.xn--p1ai
zigarettenonl.canalblog.com xn--h1ahbi.com.ua
yaminecraft.ru
znakom.sibtest.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2902 ### Version: V4.2022.02.2903
### Updated: Sun Feb 6 22:01:17 UTC 2022 ### Updated: Mon Feb 7 10:03:33 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2902 ### Version: V4.2022.02.2903
### Updated: Sun Feb 6 22:01:17 UTC 2022 ### Updated: Mon Feb 7 10:03:33 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##