vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-06 12:40:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2694 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-04 22:07:21 +00:00
parent 1704941b0f
commit 2fd24e8a6a
46 changed files with 68786 additions and 68786 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2693 #### Version: V4.2021.11.2694
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2693 #### Version: V4.2021.11.2694
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2693 #### Version: V4.2021.11.2694
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8464
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2693 #### Version: V4.2021.11.2694
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8468
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8468
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8468
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8468
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.1.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.1.52 0;
127.0.0.1 0; 127.0.0.1 0;

8468
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.1.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.1.52 0;
127.0.0.1 0; 127.0.0.1 0;

8468
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.1.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.1.52 0;
127.0.0.1 0; 127.0.0.1 0;

8468
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2693 2694

8468
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8468
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.112.115.138 0; 13.77.152.73 0;

8468
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.1.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.1.52 0;
127.0.0.1 0; 127.0.0.1 0;

8468
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.1.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.1.52 0;
127.0.0.1 0; 127.0.0.1 0;

8468
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.1.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.1.52 0;
127.0.0.1 0; 127.0.0.1 0;

8468
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.112.115.138 0; 13.77.152.73 0;

8468
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.112.115.138 0; 13.77.152.73 0;

8468
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.1.52 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Cogentbot BackWeb
DnyzBot Drip
Iblog HTMLparser
InfoNaviRobot LinkpadBot
MeanPath Bot LinksManager
RocketCrawler Mister PiX
SEOprofiler Musobot
Shodan Niki-bot
WISENutbot OpenVAS
archive.org_bot webmeup-crawler

280
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker ADmantX
404enemy AIBOT
ALittle Client ALittle Client
Aboundex AllSubmitter
AiHitBot AlphaBot
Anarchy Anarchie
Anthill Anarchy99
Aspiegel Ankit
AwarioRssBot Attach
BDCbot AwarioSmartBot
BLEXBot
BackStreet BackStreet
Backlink-Ceck Backlink-Ceck
BatchFTP Bandit
Barkrowler
BetaBot BetaBot
Bigfoot Black Hole
Bitacle Bolt
Blackboard
Blow
Brandwatch Brandwatch
BuiltBotTough Buck
Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CODE87 CATExplorador
CazoodleBot CCBot
CheeseBot CSHttp
CherryPicker CherryPicker
Claritybot
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot Cogentbot
Copier Copier
Cosmos Cosmos
Craftbot Craftbot
CrunchBot CrunchBot
Curious
CyotekWebCopy CyotekWebCopy
DIIbot DIIbot
DataCha0s
DatabaseDriverMysqli
Demon
Deusu Deusu
Devil
DigitalPebble DigitalPebble
Discobot Discoverybot
Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnyzBot
DomainCrawler DomainCrawler
DomainSigmaCrawler DomainStatsBot
Domains Project Domains Project
Dragonfly EMail Siphon
EasyDL EasyDL
Ecxi EirGrabber
EroCrawler EroCrawler
Exabot Express WebPictures
ExtLinksBot ExtLinksBot
Extreme Picture Finder Extractor
ExtractorPro
EyeNetIE EyeNetIE
Fimap FDM
FlashGet Flunky
Foobot
Freeuploader Freeuploader
FrontPage
Fuzz Fuzz
FyberSpider
GT::WWW GT::WWW
Genieo Grafula
GermCrawler
Getintent
Go-Ahead-Got-It
GoZilla
GridBot GridBot
HTTP::Lite HTTP::Lite
HTTrack
HaosouSpider
Harvest
Havij
HybridBot
IDBTE4M
IRLbot IRLbot
Image Fetch Id-search
Image Sucker Indy Library
IndeedBot IstellaBot
InfoNaviRobot JOC Web Spider
InfoTekies
Intelliseek
Internet Ninja
InternetSeer
JennyBot JennyBot
JetCar
Jetty
JikeSpider JikeSpider
Joomla
Jorgee Jorgee
JustView
Jyxobot Jyxobot
LNSpiderguy Kinza
LWP::Simple
Lanshanbot
Larbin
Leap
LeechFTP LeechFTP
LibWeb LibWeb
Libwhisker
LieBaoFast LieBaoFast
Linkdexbot Linkdexbot
LinkextractorPro LinkextractorPro
LinksManager LinkpadBot
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey Spider Lipperhey Spider
Litemage_walker
Lmspider
Ltx71 Ltx71
MIIxpc MSFrontPage
MJ12bot MSIECrawler
MQQBrowser Mag-Net
MTRobot
Mail.RU_Bot
Majestic SEO Majestic SEO
MarkWatch Majestic12
Mass Downloader MarkMonitor
Masscan Masscan
Meanpathbot Mb2345Browser
Metauri Mediatoolkitbot
MicroMessenger Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Mister PiX
Moblie Safari Moblie Safari
Musobot Mojeek
NICErsPRO Mojolicious
Morfeus Fucking Scanner
Msrabot
Name Intelligence Name Intelligence
NearSite Nessus
Net Vampire Net Vampire
NetAnts
NetMechanic
NetZIP NetZIP
NextGenSearchBot Netcraft
Nuclei Nettrack
Octopus Niki-bot
Offline Explorer Nimbostratus
Nutch
Offline Navigator Offline Navigator
OpenVAS OpenLinkProfiler
Openfind
Openvas
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageGrabber PageAnalyzer
PageThing.com PageScorer
PeoplePal Pandalytics
Panscient
Papa Foto
Petalbot Petalbot
Picsearch Picsearch
Piepmatz Piepmatz
Pixray Pixray
PleaseCrawl
Probethenet Probethenet
Psbot
Pump
PxBroker
QueryN Metasearch QueryN Metasearch
Quick-Crawler RankActive
RSSingBot
RankingBot2 RankingBot2
Rankivabot RankurBot
ReGet Re-re
Reaper RebelMouse
Ripper Recorder
RocketCrawler RedesScrapy
Rogerbot SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler SMTBot
SISTRIX
SalesIntelligent SalesIntelligent
ScanAlert
ScoutJet ScoutJet
Screaming Screaming
ScreenerBot
ScrepyBot
Searchestate
Semrush Semrush
SemrushBot
SeoSiteCheckup SeoSiteCheckup
Seomoz Shodan
Siphon Site Sucker
SiteExplorer SiteCheckerBotCrawler
SiteSucker SiteLockSpider
Sitebeam
SlySearch
Snake Snake
Snoopy Snoopy
SocialRankIOBot SocialRankIOBot
Sottopop Sottopop
Spbot SpaceBison
SpankBot
Spanner
Spinn3r
Sqworm Sqworm
Steeler
Sucker Sucker
Sucuri Sucuri
SuperBot SuperBot
SuperHTTP SuperHTTP
Swiftbot Szukacz
T8Abot T8Abot
Teleport Teleport
Telesoft Telesoft
Telesphoreo Telesphoreo
Telesphorep
The Intraformant
TheNomad TheNomad
Toata Toweyabot
Trendictionbot Trendictionbot
Turingos
TurnitinBot TurnitinBot
Typhoeus TwengaBot
URLy.Warning URLy.Warning
UnisterBot VB Project
Upflow VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VoidEYE VidibleScraper
Voil WEBDAV
Voltron
WISENutbot WISENutbot
WPScan WPScan
WWWOFFLE WWW-Collector-E
Web Fetch WWW::Mechanize
WallpapersHD
WeSEE
Web Fuck Web Fuck
Web Sauger WebAuto
Web Sucker WebCopier
WebBandit
WebFetch WebFetch
WebGo IS
WebImageCollector WebImageCollector
WebLeacher
WebPix WebPix
WebReaper WebStripper
WebWhacker WebWhacker
Webalta Webalta
Webshag Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
WebsiteQuester Who.is Bot
Widow Wonderbot
Wotbox
Wprecon Wprecon
Xaldon WebSpider Xaldon WebSpider
Xenu Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zeus
Zauba Zitebot
ZoomBot archive.org_bot
arquivo.pt arquivo.pt
autoemailspider backlink-check
cah.io.community
check1.exe check1.exe
clark-crawler coccocbot-web
com.plumanalytics demandbase-bot
crawler.feedback
domainsproject.org domainsproject.org
gopher evc-batch
internetVista monitor facebookscraper
heritrix
ips-agent ips-agent
isitwp.com isitwp.com
lwp-request lwp-request
meanpathbot lwp-trivial
magpie-crawler
muhstik-scan muhstik-scan
oBot oBot
page scorer pcBrowser
probe-image-size polaris version
s1z.ru satoristudio.net
scalaj-http seostar
seoscanners
serpstatbot
sitechecker.pro sitechecker.pro
sogouspider
spyfu
tAkeOut tAkeOut
trendiction.de trendiction.de
webmeup-crawler
webpros.com webpros.com
x22Mozilla x09Mozilla
xpymep1.exe xpymep1.exe
zauba.io zauba.io
zgrab zgrab

316
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 80legs
404checker
404enemy
AIBOT AIBOT
ALittle Client ASPSeek
Abonti Abonti
AfD-Verbotsverfahren Aboundexbot
AiHitBot
Alexibot Alexibot
Anarchy Anarchy99
Ankit Apexoo
Anthill
Aspiegel Aspiegel
Asterias
Attach Attach
AwarioRssBot
BBBike BBBike
BDCbot BDCbot
BLEXBot BDFetch
Battleztar Bazinga BackDoorBot
BetaBot BackStreet
Bigfoot BackWeb
Black Hole Backlink-Ceck
BlackWidow BatchFTP
Blackboard
Blow Blow
BlowFish BlowFish
Boardreader Boardreader
Brandprotect
Buck Buck
BunnySlippers
CATExplorador CATExplorador
CCBot CCBot
CensysInspect CheeseBot
CheTeam ChinaClaw
CherryPicker Cliqzbot
Cocolyzebot
Collector Collector
Copyscape CopyRightCheck
CrazyWebCrawler Cosmos
Craftbot
Crawling at Home Project
CrunchBot CrunchBot
CyotekWebCopy Custo
DBLBot DBLBot
DIIbot Devil
DSearch Discobot
DTS Agent DomainAppender
Deusu DomainCrawler
Discoverybot Dragonfly
DittoSpyder Drip
DomainSigmaCrawler
Domains Project
Dotbot
Download Wonder
EMail Siphon EMail Siphon
Ecxi EMail Wolf
EroCrawler EasyDL
Extractor Ebingbong
ExtractorPro Evil
Exabot
Extreme Picture Finder Extreme Picture Finder
FemtosearchBot EyeNetIE
Fimap Ezooms
FrontPage FDM
Firefox/7.0
Flunky
FyberSpider
Fyrebot
GT::WWW GT::WWW
GalaxyBot GalaxyBot
GermCrawler
GetRight
GetWeb GetWeb
Getintent
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GoZilla
Grabber GrabNet
GrapeFX Grafula
GrapeshotCrawler GrapeshotCrawler
GridBot HTTrack
HEADMasterSEO Heritrix
HMView IDBTE4M
HTMLparser IRLbot
HaosouSpider
Harvest
Havij
Humanlinks
Iblog
Id-search Id-search
Image Fetch InfoNaviRobot
Indy Library InfoTekies
JOC Web Spider Intelliseek
JamesBOT Iria
JetCar Iskanie
Jetty Jetty
JikeSpider
JustView
Kenjin Spider Kenjin Spider
Kozmosbot
LWP::Simple LWP::Simple
Lanshanbot Lanshanbot
LeechFTP LeechFTP
LibWeb LexiBot
LieBaoFast Lftp
Lightspeedsystems Libwhisker
LinkScan
LinkextractorPro LinkextractorPro
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lmspider
Litemage_walker Ltx71
MFC_Tear_Sample MIDown tool
MJ12bot
MQQBrowser MQQBrowser
MSIECrawler MSFrontPage
MTRobot Magnet
Mag-Net Majestic-SEO
Majestic12
MarkMonitor MarkMonitor
MarkWatch Masscan
Mb2345Browser Mata Hari
MeanPath Bot
Meanpathbot
Mediatoolkitbot
Microsoft URL Control Microsoft URL Control
Mister PiX Mister PiX
Mojolicious Mojeek
Morfeus Fucking Scanner NICErsPRO
Mozlila NetAnts
NPbot Nettrack
Navroad Nibbler
NearSite
Needle
Net Vampire
NetMechanic
Netcraft
NextGenSearchBot
Niki-bot Niki-bot
NimbleCrawler NimbleCrawler
Ninja Ninja
Nmap Nutch
Nuclei
Octopus Octopus
OpenLinkProfiler OpenLinkProfiler
OpenVAS Openfind
OrangeBot OrangeBot
OrangeSpider
OutclicksBot OutclicksBot
OutfoxBot
PECL::HTTP
PHPCrawl PHPCrawl
POE-Component-Client-HTTP
Page Analyzer
PageScorer
Panscient Panscient
Papa Foto
Pavuk
Pi-Monster Pi-Monster
Picsearch Picsearch
PictureFinder PictureFinder
Piepmatz
Pimonster Pimonster
ProWebWalker Pixray
Probethenet PleaseCrawl
Pu_iN ProPowerBot
PyCurl Pump
RSSingBot QueryN Metasearch
Quick-Crawler
RankActive RankActive
RankActiveLinkBot RankFlex
Rankivabot RankingBot
Re-re RankingBot2
RealDownload ReGet
RepoMonkey Reaper
SEOlyticsCrawler RebelMouse
SEOstats RedesScrapy
SalesIntelligent RocketCrawler
Scanbot SEOprofiler
ScoutJet Scrapy
Screaming ScreenerBot
Semrush Searchestate
SearchmetricsBot
SeoSiteCheckup SeoSiteCheckup
Seomoz Seomoz
Siphon Site Sucker
Siteimprove SiteCheckerBotCrawler
Sitevigil SiteLockSpider
Snake Sitebeam
Snapbot SlySearch
Snoopy Snoopy
Sosospider
SpaceBison SpaceBison
SpankBot Spammen
Spbot
SputnikBot
Sqlmap Sqlmap
Sqlworm Sqlworm
Stripper Stripper
SuperBot SuperBot
Surfbot SuperHTTP
Suzuran Swiftbot
T0PHackTeam Szukacz
TeleportPro TeleportPro
Telesoft
Telesphoreo
Telesphorep Telesphorep
Thumbor
Toata
Toweyabot
Tracemyfile Tracemyfile
True_Robot TurnitinBot
Turingos
Turnitin
TwengaBot TwengaBot
URLy Warning Twice
URLy.Warning URLy.Warning
UnisterBot UnisterBot
V-BOT Upflow
VB Project VB Project
Vagabondo Vagabondo
VelenPublicWebCrawler WASALive-Bot
VidibleScraper WWW-Collector-E
Voltron WWW::Mechanize
WPScan WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WeSEE Web Auto
Web Fetch Web Collage
Web Fuck
Web Pix Web Pix
Web Sauger
WebAuto WebAuto
WebBandit
WebCollage WebCollage
WebEnhancer WebFetch
WebFuck
WebGo IS WebGo IS
WebImageCollector WebReaper
WebLeacher
WebSauger WebSauger
WebSucker WebSucker
Webalta
Webshag Webshag
Website Quester WebsiteExtractor
Whack WebsiteQuester
Whatweb Whatweb
WinHTTrack Wonderbot
WiseGuys Robot Woobot
Wotbox Wotbox
Xaldon WebSpider Zade
Xaldon_WebSpider
YoudaoBot
Zauba Zauba
Zitebot Zeus
ZmEu
ZoomBot ZoomBot
archive.org_bot ZoominfoBot
ZyBorg
autoemailspider autoemailspider
cah.io.community check1.exe
clark-crawler
coccocbot-web coccocbot-web
cognitiveseo crawl.sogou.com
crawler4j crawler4j
dataforseo.com dataforseo.com
internetVista monitor eCatch
magpie-crawler evc-batch
isitwp.com
lwp-trivial
meanpathbot meanpathbot
netEstate NE Crawler muhstik-scan
plumanalytics oBot
ripz pcBrowser
s1z.ru
satoristudio.net satoristudio.net
scalaj-http
seobility seobility
seoscanners
seostar seostar
sitechecker.pro sexsearcher
trendiction.de sogouspider
spyfu
sysscan
tAkeOut
ubermetrics-technologies.com ubermetrics-technologies.com
webmeup-crawler
webprosbot webprosbot
x09Mozilla
x22Mozilla
xpymep1.exe xpymep1.exe
zgrab zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
clksite.com aad1f4acb0a373420d9b0c4202d38d94fa.com
hamilton.ca cdncash.org
healmytrauma.info cgi2.nintendo.co.jp
ideibiznesa2015.ru f012.de
keenoutlet.online handicapbathtubarea.com
oecnhs.info kandidos.com
prchecker.info live-sexchat.ru
simply.net shemalegalls.blogporn.in
stairliftstrue.com slimcdn.com
theprofitsmaker.net u-cheats.ru

924
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0lovespells0.blogspot.com 101lesbian.xyz
1001desert.com 1bet.com
1pamm.ru 2020iscoming.info
1webmaster.ml 256bit.by
20pascals.nl 70casino.online
24videos.tv 7a2rnuey1tw9ar.ru
3dracergames.com 7makemoneyonline.com
404.mn aa81bf391151884adfa3dd677e41f94be1.com
45en.ru aa9bd78f328a6a41279d0fad0a88df1901.com
4k-player.pl
7minuteworkout.com
7yue.org
abcdefh.xyz abcdefh.xyz
abtasty.com academia-nsk.org
actionnooz.com acortarurl.es
addtoadd.men adclickthru.net
adman.se adminshop.com
ads-cool.pro ads.gold
adtech.de advokateg.ru
adultgalls.com alert-fjg.xyz
affiliate-fr.com allkrim.com
ahmedabadwebs.com alphahoverboards.com
alarmobninsk.ru alpharma.net
albumsuper.info ameblo.jp
alpinism.ru
aludecor.info
amehdaily.com
anaksma.info
anatomiy.com anatomiy.com
anime.dougasouko.com angkortours.vn
anmysite.com animal-drawings.com
anticrawler.org apartamentwroclaw.eu
ap.senai.br
apiitsthirteende-a.akamaihd.net apiitsthirteende-a.akamaihd.net
apiluckyleapnet-a.akamaihd.net apiwebwebgetcom-a.akamaihd.net
apiwoodensealcom-a.akamaihd.net appearance-cool.com
appfastplay.com appfixing.space
apple.com-cleaner.systems apple.com-cleaner.systems
appsecurityr.com apple.com-webbrowsing-security.science
applyneedy.xyz
arabseyes.com
arclk.net arclk.net
arraty.altervista.org atmovs.com
artavenuegardenstudios.com auto-moto-elektronika.cz
auspolice.com autobrennero.it
auto-zapchasti.org autobudpostach.club
availit.weebly.com aversis.net
avon-severozapad.ru avkzarabotok.info
avtoarenda.by ayakino.net
babyfactory.fr ayerbo.xhost.ro
backuperwebcam.weebly.com azazu.ru
bagsonsale.online azinoofficial777.ru
b00kmarks.com
b2b-lounge.com
balla-boo.se balla-boo.se
bannerpower.com bamo.xsl.pt
bcmp.org
bdsmgalls.net bdsmgalls.net
begalka.xyz beauty-things.com
bestmarriages.com bensbargains.net
bestprofits-there.com bestofupload.info
betterhdporn.com bet-prognoz.com
betwinservice.com
bibys.com
big-boards.info
bigames.online bigames.online
bigpenisguide.com bigpenisguide.com
bingoporno.com bitcoin-ua.top
bio.trade-jp.net bitcoinpile.com
blackhatworth.com biznesrealnost.ru
blavia.00author.com blagovest-med.ru
blogs.rediff.com blog.f00kclan.de
bnt-team.com blog.yam.com
bonzbuddy.com blogsfunky672.weebly.com
bonzybuddy.com bmw-ark.ru
brakehawk.com bmw.afora.ru
bristolhotel.com.ua book-bets.com
brokergid.ru boom-celebs.com
buyk.host.sk brainboosting.club
bytimedance.ru brainsandeggs.blogspot.com
carabela.com.do brillianty.info
carfax.com.ua bum.com.ru
cars-modification.net burger-imperia.com
carsplate.com buy-loft.ru
cartierbracelet.xyz camsex.xxx-cam.webcam
cartierreplica.pw candlehandmade.com
cashkitten-a.akamaihd.net cartierjusteunclou.xyz
casinopinup-wins.com casablancamanor.co.za
caveavins.fr casite-513049.cloudaccess.net
cb.iphantom.com ccjp.eu
cdnnetwok.xyz cenoval.ru
cfcl.co.uk certifywebsite.win
cheap-pills-norx.com chatroulette.world
checkm8.com cheapcigarettesc.info
christianlouboutinsaleonline.us cheapmoncler.win
cialischsrx.com cherrypointplace.ca
cityadspix.com choosecuisine.com
cleanallvirus.com clickprotects.com
clickso.com clicksor.net
clipartnew.com clmforexeu.com
communisave.co.za coldfilm.ru
compliance-jack.top cometorussia.net
compliance-jane.top compiko.info
compliance-barak.xyz
compliance-viktor.top
connectionstrenth.com
copenergo.ru
coprofam.org coprofam.org
copyrightclaims.org corta.co
cranly.net crazyprotocol.com
cyprusbuyproperties.com cs-passion.pl
dailyfinancefix.com customerpromos-a.akamaihd.net
daisye.top cyse.tk
dame-ns.kz dafatiri.com
data-mining.tk data1.scopich.com
dcj-nn.ru dating-time-now.com
deplim.com datract.com
detoxmed24.ru decorationspcs.com
dev.citetick.com dentalpearls.com.au
dgfitness.co destinationrealestate.com
diminishedvalueoforegon.com dfwu1019.info
diamond-necklace.info
dieta-personalna.pl
digest-project.ru
dimkino.ru dimkino.ru
directivepub.com discovertreasure-a.akamaihd.net
divku.ru dispo.de
dktr.ru djstools.com
dnsrsearch.com
domainanalyzing.xyz
domashneeruporno.com
domoysshop.ru
dorratex.tn dorratex.tn
doublepimp.com dosugrostov.site
earl-brown.info doxyporno.com
earthmagic.info dptaughtme.com
ecookna.com.ua druzhbany.ru
education-cz.ru dwomlink.info
ege-russian.ru eduserver.net
ekto.ee efnor-ac.com
elektrischezigarette1.blog.pl ekaterinburg.xrus.org
elektrischezigarettekaufen2.cowblog.fr elektrozigarette2.webs.com
elektrozigaretten1.postbit.com elektrozigaretten2.yn.lt
encodable.com elitepcgames.com
englate.com eralph.tk
ereko.ru ereko.ru
ero-advertising.com evogarage.com
eropho.com exhibitionplus.eu
eropornosex.ru eyessurgery.ru
essay-edu.biz f-online.de
everflora.ru falcon-images.blogspot.com
exdocsfiles.com femdom.twiclub.in
executehosting.com filesvine.com
extractorandburner.com fitnesspiks.com
extremepornos.net fix-website-errors.com
eyemagination.com flauntyoursite.com
eywords-monitoring-your-success.com floridamobilebillboards.com
ezigaretteshop2.vefblog.net
f00kclan.de
feel-planet.com
ferieboligkbh.dk
filmetricsasia.com
financehint.eu
financepoints.eu
finansov.info
fodelsedagspresenter.nu
food.dtu.dk
forex-indextop20.ru forex-indextop20.ru
forex-procto.ru foto-weinberger.at
formseo.com
forum.doctissimo.fr
fotosfotos.eu
free-deals.faith free-deals.faith
free-share-buttons.com free-today.com
freemags.cc freegamesplay.online
freewhatsappload.com freejabs.com
frustrated-favorable.gq freelifetimefuckbook.com
fsalas.com frequiry.com
g33.org fres-news.com
gayxperience.com freshsuperbloop.com
ge0ip.net ftns.ru
ge0ip.org galaxys6manual.info
get-free-traffic-now.com games.kolossale.ru
gaming-journal.com
gateway.zscalerone.net
gaygalls.net
gazobeton-p.com.ua
gazporno.com
gearcraft.us
gentamicineyedrops.blogspot.com
getresponse.com
gfaq.ru
ghostvisitor.com
gktt.ru gktt.ru
gnuetella.com glasof.es
goldpanningtools.com globatur.ru
goodwriterssales.com gobongo.info
googglet.com good-mummy.ru
goroda-vsego-mira.ru groupmoney.ru
graphics8.info growboxbl.ru
greatdealshop.com gsasearchengineranker.top
greatgrace.ru gsasearchenginerankerdiscount.com
gsmtlf.ru gsasearchenginerankerhelp.com
gta-top.ru
guardlink.com
guge.io guge.io
gyffu.com guitar-master.org
habermetre.com hannasolution.ru
hatdc.org happy.new.yeartwit.com
hd-filmy.net hidefiles.org
hem.passagen.se hotdl.in
hermesreplica.pw hoverboard360.se
hetmanship.xyz howtowhitenteethfast.xyz
heygidday.biz ictizanidinehcl4mg.blogspot.com
home-task.com ilovevitaly.ru
homeart.space imagez.co
homegardenlova.com imitex-plus.ru
homemade.gq impisr.edunsk.ru
hosting-tracker.com incanto.in.ua
hostsshop.ru
hotcore.info
hotkeys.com
hotxnights.info
housemilan.ru
humanorightswatch.org
humbmosquina.tk
id-forex.com
ifirestarter.ru
ilmexico.com
ilo134uloh.com
ilovevitaly.com
images.gyffu.com
inclk.com
increasewwwtraffic.info
indiakino.net
indiasourcemart.in indiasourcemart.in
infoupdate.org informatiecentro.be
innoslicon.com intellekt21.ru
interesnie-faktu.ru intimshop-fantasy.ru
internetproviderstucson.com jav-fetish.site
intim-uslugi.info jerseychinabizwholesale.com
investyb.com joy-penguin.com
iqoption.pro justkillingti.me
iradiology.ru kamalsinha.com
javidol.site kamorel.com
javlibrary.cc kaz.kz
jimmychoosale.online kchaxton.com
johnrobertsoninc.com keenoutlet.online
jongose.ninja kellyonline.xyz
joyceblog.top keywordhouse.com
kadashihotel.com kinotorka.ru
kalb.ru kinozapas.com
kasino-money.pw kmgamex.cf
katjimej.blog.fc2.com kochanelli.com
kerch.site konturkrasoty.ru
kerwinandcariza.com kotaku.com
keywordhut.com kredit-pod-zalog-krasnodar.ru
keywordteam.net lalalove.ru
kimcurlrvsms.com larchik.net
kino-fun.ru larger.io
kino-rating.ru
kinomaniatv.pl
kinoplen.ru
kliksaya.com
komp-pomosch.ru
kraljeva-sutjeska.com
krasnodar.ru
kutikomi.net
landscaping.center
lapolis.it
laptoper.net
lasvegaslockandsafe.com lasvegaslockandsafe.com
law-check-eight.xyz leadwayau.com
leboard.ru letolove.ru
lightinghomes.net letslowbefast.site
likesuccess.com lietaer.com
linkbolic.com likrot.com
listiseltemournlan.gq link.ac
liumimgx.pw
live-sexchat.ru live-sexchat.ru
liveinternet.ro livejasmin.com
lmrauction.com loftdigital.eu
loadingpages.me lol-smurfs.com
locatellicorretor.com.br longadventure.com
locksmith.jp lottospring.com
lolnex.us louboutinreplica.xyz
lonerangergames.com m292.info
long-beach-air-conditioning.com makis.nu
louboutinreplica.pw mall.uk
m-google.xyz manyresultshub-a.akamaihd.net
machicon-akihabara.info marmitaco.cat
magnetic-bracelets.ru master-muznachas.ru
manimpotence.com matematikus.info
mapquestz.us matrixalchemy.com
marketingtechniques.info mbiologi.ru
markjaybeefractal.com mere.host.sk
masturbate.co.uk metarip.ru
maxthon.com microsearch.ru
mcnamaratech.com microstatic.pl
mebel-ekb.com midst.eu
mebelcomplekt.ru
megaapteka.ru
meget.co.za
milblueprint.com
mindeyegames.com
minecraft-neo.ru
mirzonru.net mirzonru.net
mncrftpcs.com misandesign.se
monetizationking.net missclub.info
moneyteam24.com mobilierland.com
mobstarr.com
modenamebel.ru
moncleroutletonline.win
monthlywinners.com monthlywinners.com
moskva.nodup.ru msfsaar.de
most.gov.iq mtmtv.info
moxo.com mttwtrack.com
mrvideospornogratis.xxx mundoaberrante.com
musicspire.online
musicstock.me
muz-shoes.ru muz-shoes.ru
mydoctorok.ru mydeathspace.com
mydownloadengine.com mydownloadengine.com
myprintscreen.com myftpupload.com
mysexpics.ru mygameplus.com
naj-filmy24.pl mykings.pw
narco24.me mymobilemoneypages.com
myshopmatemac.com
myyour.eu
nagdak.ru
naperehresti.info
nastydollars.com
naturalbreakthroughsresearch.com
net-profits.xyz net-profits.xyz
next-dentists.tk newstudio.tv
nfljerseysforsalewholesaler.com nisuturnetdgu.tk
nhl17coinsforps3.gratisblog.biz no-fuel.org
nina.az obiavo.by
njkmznnb.ru obiavo.com
nodding-passion.tk obiavo.net
nordvpn.com ohmyrings.com
noumeda.com okeinfo.online
novodigs.com omgtnc.com
nsatc.net online-podarki.com
nw-servis.ru online247.ml
offer.camp online7777.com
ok-ua.info onstrapon.purplesphere.in
oksrv.com orion-v.com
online-hit.info oxotl.com
online-x.ru
onlinetvseries.me
onload.pw
optikremont.ru
orgasmus-virtual.com
ororodnik.goodbb.ru
ostrovtaxi.ru
outrageousdeal-a.akamaihd.net
owohho.com
paceform.com paceform.com
papasdelivery.ru paclitor.com
parajumpersjakkesalgnorge.info panamaforbeginners.com
parajumpersstore.online pastaleads.com
parfusale.se pattersonsweb.com
partner-host.men pc-test.net
partners-ship.pro pdns.cz
pdns.download
perfectpracticeweb.com
perl.dp.ua perl.dp.ua
pinup-casino1.ru photo.houseofgaga.ru
pipki.r.acdnpro.com photochki.com
piski.top photorepair.ru
pk-pomosch.ru php-market.ru
plastweb.ru pics-group.com
plusnetwork.com picsforkeywordsuggestion.com
pod-muzyku.club picturesify.com
poems.com.ua picurams.pw
poisk-zakona.ru pills24h.com
polytopesexempt.com pinwallpaper.top
popads.net pix-hd.com
popup-fdm.xyz pizdeishn.com
porndairy.in porngalleries.top
pornokajf.com pornosemki.info
pornoplen.com pospr.waw.pl
pornosee.info potolokelekor.ru
pornosexrolik.com pregnant.guru
pornoslive.net preparevideosafesystem4unow.space
postclass.com pretty-mart.com
profit-opportunity.com priceg.com
prohoster.info priscilarodrigues.com.br
prostitutki-odessa.org professionalwritingservices15.blogspot.ru
prostitutki-tolyatti.org projectforte.ru
proxyelite.biz pronekut.com
pshare.biz prospekt-st.ru
pts163.ru prostitutki-novgoroda.org
puserving.com pussyspace.net
pussysaga.com qor360.com
qpypcx.com quality-traffic.com
r-e-f-e-r-e-r.com quickbuck.com
radiodigital.co r-control.ru
rapidokbrain.com rankscanner.com
rentaremotecomputer.com rasteniya-vs-zombi.ru
rff-cfal.info rczhan.com
rimedia.org realizmobi.com
rockingclicks.com recipedays.com
rootandroid.org reckonstat.info
royalvegascasino.com redirlock.com
rozalli.com regionshop.biz
rusenvironmental.net registrysweeper.com
russia-today-video.ru replica-watch.ru
sabizonline.com replicalouboutin.xyz
safe-app.net retreatia.com
samara.rosfirm.ru rfserial.net
sammyweaver.com rossanasaavedra.net
scanmarine.info ruex.org.ua
scanner-julianna.top runtnc.net
scanner-nelson.top ruscoininvest.company
scansafe.net russkoe-zdorovie.ru
scopich.com rusvideos.su
sei80.com rvzr-a.akamaihd.net
seksotur.ru sa-rewards.co.za
semaltmedia.com sadaholding.com
semrush.com saitevpatorii.com
seocheki.net savefrom.com
seolab.top scalerite.co.za
seorankinglinks.xyz scanner-george.top
seotoolsagency.com scanner-mark.top
sex-foto.pw scanner-viktor.top
sexspornotub.com schalke04fc.info
sexytrend.ru scrnet.biz.ua
sharpchallenge.com search.xtconnect.com
shemale-sex.net semp.net
seo18.su
service.adtech.us
sextracker.com
sexyteens.hol.es
sfj-ror.no
shijian.ac.cn shijian.ac.cn
shitmovs.com shitting.pro
shodanhq.com shop-electron.ru
shopperifymac.com shopsellcardsdumps.com
sicfor.bcu.cc shopwme.ru
signoredom.com sibvitr.ru
silverage.ru signx.info
site-analyzer.com site-analyzer.com
sixcooler.de siteimprove.com
sklad-24.ru sitevalued.com
slkrm.ru slkrm.ru
sodexo.com slutloadlive.com
sonata-arctica.wz.cz smadihome.com
sonnikforme.ru smichovbike.cz
space4update.pw social-button.xyz
srdrvp.com social-search.me
sterva.cc socialmadesimple.com
stroiminsk.com solnplast.ru
sovetogorod.ru
spbchampionat.ru
spidtest.space
spywarespy.com
ssconstruction.co
stop-gepatit.te.ua
stroydetali.ru stroydetali.ru
stroyhelp-dv.ru studworks.org
stroymonolit.su success-seo.com
studentguide.ru superinterstitial.com
subj.ukr-lit.com support.nopeas.sk
sugarkun.com tattomedia.com
supervesti.ru tengohydar.tk
svetodiodoff.ru
tamada69.com
tasteidea.com
terrafootwear.us
teslathemes.com teslathemes.com
theallgirlarcade.com tetracsaudi.com
thepantonpractice.co.uk thedownloadfreeonlinegames.blogspot.com
time-japan.ru thfox.com
tizanidine6mg.blogspot.com tilido.com
tizanidinehcl2mgtablet.blogspot.com timhost.ru
tizanidine4mgstreetprice.blogspot.com
tizanidine4mgstreetvalue.blogspot.com
tizanidine4mgtablets.blogspot.com
tizanidinestreetprice.blogspot.com
tizanidinestreetvalue.blogspot.com
tizanidinewithdrawal.blogspot.com
tizanidinezanaflex.blogspot.com tizanidinezanaflex.blogspot.com
tomatis.gospartner.com tmm-kurs.ru
top-deal.com.pl toonfamilies.net
top10registrycleaners.com top-karkas.ru
torture.ml top-l2.com
track.deriv.com top10-online-games.com
trackmedia101.com torrentred.games
trafficmonetize.org torrents.cd
traffictrade.life tovaroboom.vast.ru
tuberkuleznik.ru tracksz.co
tuckermktg.com trackzapper.com
trafficjunky.com
truck-rece.life
truemfilelj.gq
ttrraacckkrr.com
turbo-suslik.org turbo-suslik.org
turvgori.ru tversvet.ru
twojebook.pl twelvevisionspartyofcolorado.com
uac.net.au typer.one
uamtrk.com u-cheats.ru
umg-stroy.ru ukrtextbook.com
umnovocaminho.com ultimatesetnewfreeallsoftupgradesystems.pw
unlimitdocs.net ultramart.biz
unrealcommander.org upupa.net
uptime-delta.net urll.eu
uptimechecker.com uroffer.link
urccvfmc.bloger.index.hr ustion.ru
uroki.net vadimkravtcov.ru
userequip.com vancleefreplica.pw
usswrite.com varbergsvind.se
viagroid.ru vduplo.ru
video-hollywood.ru vedomstvo.net
videochat.tv.br vgoloveboli.net
vipsexfinders.com via-energy-cumpara.com
voditeltrezviy.ru via-energy-order.com
webalan.ru viberdownload10.com
webpromotion.ae videochat.cafe
websiteaccountant.de videogamesecrets.com
websitevaluebot.com videos-for-your-business.com
weddingdresses.xyz viktoria-center.ru
wesharepics.info vip2ch.com
wineration.com visa-pasport.ru
winx-play.ru viteonlusarezzo.it
wma-x.com vkonche.com
wnhjavlhezp.gq vkontarkte.com
wowas31.ucoz.ru vladimir.zrus.org
wpsecurity.website vodkoved.ru
wpthemedetector.co.uk vote-up.ru
wrz0iuebwhp5fg.freeddns.com wallpaperaccess.com
ws.ampower.me waterefficiency.co
wufak.com watracker.net
www.bookmaker-bets.com waysbetter.cn
www.ehscloud.cn webix.biz
x-diesel.com webmonetizer.net
x5market.ru website-stealer.nufaq.com
xedserver.com wedding-salon.net
xfluro.com wejdz-tu.pl
wemarketing.se
womensterritory.ru
workle.website
wovis.site
wrona.it
wurr.voila.net
www.arenda-yeisk.ru
www.labves.ru
xmlinde.com
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn----7sbifcamovvfggw9d.xn--p1ai xn----7sbifcamovvfggw9d.xn--p1ai
xn--80aodinpgi.xn--p1ai xn----dtbndd4ae7eub.top
xn--90acenikpebbdd4f6d.xn--p1ai xn--90acenikpebbdd4f6d.xn--p1ai
xn--h1aakne2ba.xn--p1ai xxxdatinglocal.us
xxart.ru xxxsiterips.xyz
xxxtubesafari.com yaminecraft.ru
yebocasino.com you-shall-not-pass.is74.ru
yginekologa.com youghbould.wordpress.com
your-bearings.com yourdesires.ru
yuarra.pluto.ro yoursearch.me
zajm-pod-zalog-nedvizhimosti.ru z-master.ru
zastroyka.org zaapplesales.blogspot.com
zebradudka.com zagadki.in.ua
zhorapankratov7.blogspot.com zeroredirect2.com
zirondelli.it zigarettenonlinekaufen.tumblr.com
zmoda.hostreo.com zocaparj.kz
znaniyapolza.ru zooggames.com
znaturaloriginal.com
zone-kev717.info
zoogdiesney.com
zoolubimets.ru zoolubimets.ru
zophim.me
zscaler.net
zskdla.site zskdla.site

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1-99seo.com 0lovespells0.blogspot.com
1pamm.ru 101flag.ru
24-ak.ru 3w24.com
2itech.ru 4webmasters.org
2x2fan.ru 8xv8.com
991mostfm.co.id abiente.ru
a96527gi.beget.tech adbetclickin.pink
aac52006ec82a24e08b665f4db2b5013f7.com
abclauncher.com
addblueoff.com.ua addblueoff.com.ua
admitad.com adidas.frwebs.fr
adultgalls.com adimmix.com
agecheckadult.com adsref.men
alborzan.com aerogo.com
alevinefesleri.com agapovdg.ru
all-number.com aimayangzhi.com
alldubai.biz akama.com
albumsuper.info
allfinweb.com
allsilver925.co.il allsilver925.co.il
alvtank.se alyeskaresort.com
anapa-inns.ru android4fun.org
arendakvartir.kz arenanews.com.ua
aruplighting.com asacopaco.tk
asseenontv.ru asociatia-tipografilor-transilvania.ro
ausergrubhof.info asrvrep-a.akamaihd.net
autotop.com.ua astana.xxxkaz.org
avtointeres.ru atout-energie-69.com
azartclub.org atovh.local-finders.com
azbuka-mo.ru audiobangout.com
baersaratov.ru azartmsl.com
balois.worldbreak.com bag77.ru
bankmib.ru bagsonsale.online
baikaleminer.com
bard-real.com.ua
battle.net battle.net
berdasovivan.ru beget.tech
besthomemadepornsites.com bestadbid.com
bezlimitko.xyz biocn.dx.am
biglistofwebsites.com blog.yam.com
biketank.ga blogs.rediff.com
blog.koorg.ru bonky.biz
blog.remote-computer.de bookhome.info
bloke.com brandov.ru
bnt-team.com browserprotecter.com
bombla.org buckspar.gq
bum.com.ru buyk.host.sk
buntube.net buypuppies.ca
buytizanidineonline.blogspot.com cartierbracelet.xyz
cdnnetwok.xyz cartierlovebraceletreplica.xyz
centraletermice.us cayado.snn.gr
ceotrk.com cellfun.mobi
cinemaenergy-hd.ru cenoval.ru
classiquebijoux.ru ci.ua
clickpapa.com clickaider.com
compliance-jess.top collegeessay19.blogspot.ru
coolgramgoods.com credit-online.ws
ddsoldes.fr cubook.supernew.org
diegolopezcastan.com customergrowthsystems.com
digitalassetmanagement.site d-black.bz
danashop.ru
danschawbel.com
data.vtc.pw
day-news.info
dfiles.me
dimkino.ru dimkino.ru
divatraffic.com doctormakarova.ru
dojki-hd.com dosug-lux.ru
domashneeruporno.com ekobata.ru
dood.live
downloads-whatsapp.com
droidlook.net
duawitchrarato.tk
eachdayisagift.review
election.interferencer.ru election.interferencer.ru
elitesportsadvisor.com electricwheelchairsarea.com
empathica.com elektrozigaretten1.postbit.com
en.home-task.com engines-usa.com
euronis-free.com etotupo.ru
exoclick.com everypony.ru
exonline.info exhibitionplus.eu
extremez.net falopicm.pw
faptitans.com finuse.com
fealq.com flowersbazar.com
feeriaclub.ru fordsonmajbor.cf
flashahead.info formularz-konkurs.tk
fm-upgrade.ru forum.doctissimo.fr
fok.nl fotosfotos.eu
forex-procto.ru fr.netlog.com
freemags.cc frameimage.org
freenode.info freeuploader.com
freshberry.com.ua fym.com.ru
freza-sverlo.ru galaxyflowers.ru
girlspicsa.com gambarxkata.co
golden-catalog.pro gearcraft.us
grand-chlen.ru getprismatic.com
guge.io glogow.pl
hamilton.ca go2album.com
harmonyglen.us good-mummy.ru
healgastro.com graphid.com
helpmymacfaster.trade gsasearchengineranker.xyz
hostcritique.com hanink.biz.ly
hyipmanager.in hayate.biz
ideawheel.com headpress.ru
ideibiznesa2015.ru histock.info
ie.57883.net hitmuzik.ru
home-task.com
hosting-tracker.com
ilovevitaly.xyz ilovevitaly.xyz
immigrational.info images-free.net
intim-uslugi.info infoupdate.org
itrevolution.cf inome.com.ua
jackpotchances.com interior-stickers.ru
jeremyeaton.co invest-pamm.ru
jolic2.com investpamm.ru
kadashihotel.com ivanovo.zrus.org
kazan.zrus.org james13prix.info
ketrzyn.pl janettabridal.com
kinozapas.com javspace.net
konica.kz jerseywholesalebizchina.com
labplus.ru joingames.org
lancheck.net joy-penguin.com
landmania.ru junketjuice.blogspot.com
lapolis.it kanimage.com
law-check-two.xyz katushka.net
lexiangwan.com keyword-suggestions.com
life-instyle.com kichenaid.ru
lombardfinder.ru kit-opt.ru
lottospring.com konoplisemena.com
macdamaged.tech konteiner24.com
machicon-ueno.info lacave.ntic.fr
magicalfind-a.akamaihd.net laexotic.com
metabar.ru lanadelreyfans.us
michaelkorssaleoutletonline.net larger.io
mirzonru.net lflash.ru
missis.top light.ifmo.ru
mixed-wrestling.ru lion.conboy.us
modenamebel.ru lonerangergames.com
moesexy.com lookingglassemb.com
moi-glazki.ru lunchrock.co
monsterdivx.tv m-google.xyz
moyakuhnia.ru magicplayer-s.acestream.net
mozello.ru marketingvici.com
muizre.ru mathgym.com.au
musirc.com maximpartnerspr.com
mycouponizemac.com mcnamaratech.com
mydirtystuff.com mebel-ekb.com
myfreetutorials.com megakino.net
myphotopipe.com mikrobiologies.ru
najaden.no misswell.net
nakozhe.com mockupui.com
napalm51.nut.cc movpod.in
narkologiya-peterburg.ru moz.com
nationalbreakdown.com mwtpludn.review
netpics.org namenectar.com
news-speaker.com new-apps.ru
nfljerseyscheapbiz.us nicovideo.jp
nicefloor.co.uk nordvpn.com
nordstar.pro nuker.com
nudepatch.net nw-servis.ru
nvssf.com o00.in
ochistka-stokov.ru obiavo.by
offf.info omoikiri-japan.ru
omgtnc.com oneclickfiles.com
pcimforum.com onlinadverts.com
pendelprognos.se online-sbank.ru
picturesfrom.com pacificair.com
pix24x7.com peekyou.com
pl-top.pl perl.dp.ua
polytopesexempt.com pharmacyincity.com
popmarker.com phormchina.com
pornodojd.ru picmoonco.pw
portside.cc picswe.com
predominant-invent.tk pipki.r.acdnpro.com
preg.marketingvici.com play-movie.pl
prizeestates.cricket playmsn.com
prlog.ru porndairy.in
proposal-engine.com porno-raskazy.ru
relayblog.com potolokelekor.ru
roleforum.ru proekt-mos.ru
rozalli.com promodj.com
salmonfishingsacramentoriver.com pron.pro
saneyes.com prostitutki-belgoroda.org
santechnik.jimdo.com prostitutki-kharkova.org
scanner-don.top proxyradar.com
seasonvar.ru pskcijdc.bloger.index.hr
sexkrasivo.net pts163.ru
quickloanbank.com
rabot.host.sk
raschtextil.com.ua
realmonte.net
registrysweeper.com
saitevpatorii.com
scanner-andrew.top
scenarii-1-sentyabrya.uroki.org.ua
seocdvig.ru
sethrollins.net
sex-watch.com
sexiporno.net
sexs-foto.top sexs-foto.top
sexstream.pl ship-marvel.co.ua
sextracker.be shtaketniki.kz
sexyteens.hol.es
shisha-swag.de
shopperifymac.com
signoredom.com signoredom.com
sims-sims.ru sinel.info
sirpornogratis.xxx skutecznetabletkinaporostwlosow.pl
siteimprove.com snomer1.ru
slavic-magic.ru soc-proof.su
small-games.biz social-s-ggg.xyz
soft1.ru socialbuttons.xyz
soheavyblog.com speedup-my.site
streamin.to splendorsearch-a.akamaihd.net
spywarebegone.com
srdrvp.com
sstroy44.ru
streetfire.net streetfire.net
sunflowerdrawingpaintings.blogspot.com sucsesofinspiration.com
supervesti.ru suture.co
surflinksmedical.com t-bygg.com
szucs.ru tattooha.com
tagil.zrus.org teastory.co
taximytishi.ru tiens2010.ru
templates.franklinfire.co
teslathemes.com
tgtclick.com
the-usa-games.blogspot.com
thejournal.ru
thenews-today.info
thepokertimer.com
thetattoohut.com
thetoiletpaper.com
timhost.ru
titangel-vietnam.com titangel-vietnam.com
tizanidinehcl2mg.blogspot.com titslove.yopoint.in
tizanidinevssoma.blogspot.com tizanidineinfo.blogspot.com
tkanorganizma.ru tizanidinemusclerelaxant.blogspot.com
tksn.ru topappspro.com
torrentred.games toproadrunner5.info
trafficshaper.com triumf-realty.ru
validdomain.xyz tuckermktg.com
valkiria-tk.ru tversvet.ru
vapeface.club twincitiescarservice.com
vchulkax.com twu.com.ua
verymes.xyz unmaroll.ya.ru
vip-dom.in uptime-alpha.net
virtualbb.com usdx.us
viteonlusarezzo.it vasileostrovsky-rayon.ru
vizag.kharkov.ua vitoriacabos.com
vladhistory.com
vladimir.zrus.org
volgograd.xrus.org
vremya.eu vremya.eu
watch-movies.ru waterefficiency.co
webmasterhome.cn waterpurifier.club
webmasters.stackexchange.com where-toget.com
wedding0venues.tk whereiskentoday.com
welck.octopis.com wholesalecheapjerseysfree.com
westvilletowingservices.co.za winner7777.net
wineration.com
wingsofrefuge.net
writingservices17.blogspot.ru writingservices17.blogspot.ru
wufak.com wzgyyq.com
x-stars.ru xchat26.myfreecams.com
xcc24.pl xn--hxazdsfy.blogspot.com
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xxxtube69.com
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf yadro.ru
xn--90acenikpebbdd4f6d.xn--p1ai zaidia.xhost.ro
xn--b1addnj3cah.xn--p1ai zajm-zalog-krasnodar.ru
xxxnatelefon.ru zakazvzloma.com
zagadki.in.ua zatjmuzu.info
zarajbuilders.com zdesformula.ru
zeroredirect6.com zeroredirect7.com
zigarettenonlinekaufen.tumblr.com zoominfo.com
znakomstva-piter78.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.1.52 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2693 ### Version: V4.2021.11.2694
### Updated: Thu Nov 4 10:03:22 UTC 2021 ### Updated: Thu Nov 4 22:01:26 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2693 ### Version: V4.2021.11.2694
### Updated: Thu Nov 4 10:03:22 UTC 2021 ### Updated: Thu Nov 4 22:01:26 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##