vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-09 22:05:46 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2741 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-27 10:09:39 +00:00
parent fb89d46744
commit 3700cfd493
46 changed files with 70217 additions and 70217 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2740 #### Version: V4.2021.11.2741
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2740 #### Version: V4.2021.11.2741
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2740 #### Version: V4.2021.11.2741
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8642
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2740 #### Version: V4.2021.11.2741
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8646
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8646
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8646
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8646
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8646
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8646
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8646
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2740 2741

8646
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8646
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.89.37.221 0; 40.78.141.192 0;

8646
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8646
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8646
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

8646
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.89.37.221 0; 40.78.141.192 0;

8646
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.89.37.221 0; 40.78.141.192 0;

8646
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.0.107 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AhrefsBot BunnySlippers
Apexoo Ecxi
DTS Agent HTMLparser
Heritrix Ltx71
Jetty Msrabot
Nmap SiteSnagger
Teleport
TurnitinBot TurnitinBot
gopher Webshag
webmeup-crawler ZoominfoBot
com.plumanalytics

314
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
80legs 80legs
ADmantX
AIBOT AIBOT
Abonti
Aboundex
Aboundexbot
Acunetix Acunetix
AiHitBot AiHitBot
Anarchy99 AlphaBot
Apexoo Anarchy
Ankit
Asterias Asterias
AwarioRssBot
BDFetch BDFetch
BackStreet BLEXBot
Bandit BackDoorBot
BatchFTP Backlink-Ceck
Battleztar Bazinga Barkrowler
BetaBot Bigfoot
Bitacle
Black Hole
BlackWidow BlackWidow
BlowFish BlowFish
BotALot Bolt
Brandwatch
Buck Buck
BunnySlippers Bullseye
CCBot
CODE87
Calculon Calculon
Cegbfeieh
CensysInspect
CheTeam CheTeam
CheeseBot Collector
Cliqzbot Copier
Cocolyzebot Cosmos
Cogentbot
Craftbot
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler
Crescent
CrunchBot CrunchBot
Curious Curious
CyotekWebCopy
DBLBot
DIIbot DIIbot
DTS Agent Deusu
DataCha0s Devil
DatabaseDriverMysqli DomCopBot
Discoverybot DomainAppender
DittoSpyder
DomainCrawler DomainCrawler
DomainStatsBot
Domains Project
Download Wonder Download Wonder
Dragonfly
Drip Drip
ECCP/1.0
EMail Wolf
Ebingbong
EirGrabber
Evil
Exabot Exabot
Express WebPictures Express WebPictures
ExtLinksBot ExtLinksBot
Extractor EyeNetIE
Extreme Picture Finder Firefox/7.0
Ezooms FlashGet
FDM Foobot
FemtosearchBot Freeuploader
FrontPage FrontPage
Fuzz
FyberSpider FyberSpider
Fyrebot G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot
GermCrawler GermCrawler
GetRight GetRight
GetWeb Getintent
GoZilla Gigabot
GrabNet Go!Zilla
Grabber Go-Ahead-Got-It
HEADMasterSEO GridBot
HTTrack
Harvest Harvest
Havij
Heritrix Heritrix
HybridBot Humanlinks
IDBTE4M IRLbot
IDBot
Id-search Id-search
Image Fetch Image Sucker
InfoTekies Indy Library
Intelliseek InfoNaviRobot
InterGET Internet Ninja
InternetSeer
JennyBot
JetCar JetCar
Jetty JustView
JikeSpider Jyxobot
Jorgee
Kenjin Spider Kenjin Spider
Kinza Keyword Density
Kozmosbot
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot Larbin
Leap
LeechFTP LeechFTP
LexiBot LexiBot
Lftp
LibWeb
LieBaoFast LieBaoFast
Lightspeedsystems Likse
LinkWalker LinkWalker
Linkbot LinkpadBot
LinkextractorPro LinqiaRSSBot
LinksManager LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Litemage_walker
Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MJ12bot
MSFrontPage MSFrontPage
MarkMonitor MSIECrawler
Masscan MTRobot
Magnet
Majestic-SEO
Majestic12
MarkWatch
Mata Hari Mata Hari
MeanPath Bot MauiBot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
MicroMessenger Microsoft URL Control
Microsoft Data Access Mojeek
Minefield Mojolicious
Mozlila Morfeus Fucking Scanner
Mr.4x3
Msrabot
NICErsPRO
NPbot NPbot
Navroad Needle
NearSite Nessus
Net Vampire
NetAnts
NetLyzer NetLyzer
NetMechanic
NetSpider
Nettrack
NextGenSearchBot NextGenSearchBot
Nibbler
Niki-bot Niki-bot
Nikto Nimbostratus
NimbleCrawler
Ninja
Nmap Nmap
Not
Nuclei Nuclei
Openvas OpenLinkProfiler
Openfind
OrangeBot
OutfoxBot
PECL::HTTP
PHPCrawl PHPCrawl
POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer
PageGrabber
PageScorer PageScorer
PageThing.com
Pandalytics Pandalytics
PeoplePal Panscient
PictureFinder Papa Foto
Piepmatz PleaseCrawl
Pimonster
Pixray
Pockey Pockey
Probethenet ProPowerBot
PxBroker
PyCurl PyCurl
QueryN Metasearch
Quick-Crawler
RSSingBot RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankingBot RankFlex
Rankivabot RankurBot
RebelMouse Re-re
RedesScrapy Reaper
Recorder
RepoMonkey RepoMonkey
RocketCrawler SalesIntelligent
Rogerbot ScanAlert
SBIder
SEOkicks
SEOlyticsCrawler
SEOprofiler
SMTBot
Scanbot Scanbot
ScoutJet Searchestate
Screaming
SemanticJuice
Semrush
SeoSiteCheckup SeoSiteCheckup
Site Sucker SeobilityBot
SiteCheckerBotCrawler Siphon
SiteLockSpider SiteLockSpider
Sitebeam SiteSucker
Siteimprove Siteimprove
Snoopy SmartDownload
SocialRankIOBot Spbot
Sottopop SputnikBot
SpaceBison Sqlworm
Sqworm
Stripper Stripper
Szukacz Sucker
SuperBot
SurveyBot
T8Abot
Teleport
TeleportPro
Telesoft Telesoft
Telesphorep The Intraformant
TheNomad
Thumbor Thumbor
TightTwatBot
Trendiction Trendiction
Trendictionbot True_Robot
Turingos
Turnitin Turnitin
TwengaBot Typhoeus
Upflow URLy Warning
V-BOT V-BOT
VeriCiteCrawler Vagabondo
VidibleScraper
Virusdie Virusdie
VoidEYE
Voil
Voltron Voltron
WISENutbot WASALive-Bot
WEBDAV
WPScan WPScan
WWW::Mechanize WWW-Collector-E
Wallpapers WWW-Mechanize
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD WallpapersHD
Web Auto
Web Collage Web Collage
Web Fetch Web Fetch
Web Fuck Web Fuck
Web Pix
Web Sucker Web Sucker
WebGo IS WebAuto
WebLeacher WebCollage
WebWhacker WebEnhancer
WebZIP WebFetch
WebmasterWorldForumBot WebFuck
WebImageCollector
Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
Whack
Whacker Whacker
Who.is Bot Who.is Bot
WinHTTrack WiseGuys Robot
Wonderbot Wonderbot
Woobot
Wprecon Wprecon
Xenu Zauba
ZoomBot
ZumBot
adscanner adscanner
autoemailspider
check1.exe
clark-crawler
com.plumanalytics com.plumanalytics
crawl.sogou.com crawler4j
evc-batch dataforseo.com
domainsproject.org
eCatch
instabid instabid
internetVista monitor internetVista monitor
ips-agent
lwp-request lwp-request
lwp-trivial lwp-trivial
magpie-crawler meanpathbot
mediawords
netEstate NE Crawler netEstate NE Crawler
pcBrowser pcBrowser
probe-image-size
ripz
s1z.ru
satoristudio.net
scalaj-http scalaj-http
seobility seobility
seoscanners serpstatbot
seostar sogouspider
sexsearcher sysscan
siteripz
tAkeOut
trendiction.de
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webgains-bot webgains-bot
webmeup-crawler webpros.com
x09Mozilla webprosbot
x22Mozilla
zauba.io zauba.io
zgrab

286
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 01h4x.com
404checker 404checker
80legs
AIBOT AIBOT
ALittle Client Aboundexbot
ASPSeek Acunetix
Aboundex AfD-Verbotsverfahren
AhrefsBot Aipbot
AiHitBot
Alexibot Alexibot
AllSubmitter
AlphaBot
Anarchy Anarchy
Anarchy99 Ankit
Asterias Anthill
Attach
AwarioSmartBot AwarioSmartBot
BLEXBot BDCbot
BackDoorBot BackDoorBot
BackWeb BackStreet
BacklinkCrawler
Badass
Bandit Bandit
Barkrowler Barkrowler
BlackWidow BetaBot
Blackboard Bigfoot
Bitacle
BlowFish BlowFish
Boardreader Brandprotect
Bolt
Brandwatch
Buddy Buddy
BuiltBotTough
BuiltWith
Bullseye Bullseye
BuzzSumo
CCBot
CODE87
CazoodleBot CazoodleBot
Cegbfeieh CherryPicker
CheTeam
Claritybot
Cliqzbot Cliqzbot
Cloud mapping Cloud mapping
Collector
CopyRightCheck CopyRightCheck
CrazyWebCrawler Cosmos
Curious Crawling at Home Project
CrunchBot
Custo
DBLBot
DIIbot DIIbot
DSearch DSearch
DTS Agent
Demon
Disco
Discobot Discobot
DnBCrawler-Analytics Discoverybot
Dispatch
DomainAppender
DomainStatsBot DomainStatsBot
Dragonfly Dragonfly
EMail Wolf Drip
Ecxi EMail Siphon
EasyDL
EirGrabber EirGrabber
EroCrawler
Evil Evil
ExtLinksBot
ExtractorPro ExtractorPro
Extreme Picture Finder Extreme Picture Finder
FDM Fimap
FrontPage G-i-g-a-b-o-t
FyberSpider GalaxyBot
Genieo Genieo
GermCrawler Gigabot
Getintent
Go!Zilla Go!Zilla
Go-Ahead-Got-It
GoZilla GoZilla
GrabNet GrabNet
GrapeFX
GrapeshotCrawler GrapeshotCrawler
GridBot
HEADMasterSEO HEADMasterSEO
HTTP::Lite HTTP::Lite
HTTrack HTTrack
Havij
Heritrix
Hloader Hloader
IDBTE4M IDBTE4M
IRLbot IDBot
Id-search Iblog
IlseBot
IndeedBot
InfoNaviRobot InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
Iria Iria
IstellaBot Iskanie
JamesBOT
JennyBot
JetCar JetCar
Joomla
Jorgee Jorgee
JustView JustView
Keyword Density Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
Larbin
LeechGet LeechGet
LexiBot
Lftp Lftp
Libwhisker LibWeb
LieBaoFast LinkextractorPro
Lightspeedsystems LinqiaRSSBot
Likse
LinkWalker
Linkdexbot
LinkpadBot
Lipperhey
Ltx71 Ltx71
MFC_Tear_Sample
MIIxpc
MJ12bot MJ12bot
MQQBrowser MQQBrowser
MSFrontPage
MSIECrawler MSIECrawler
MTRobot Mag-Net
Magnet Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic-SEO Majestic-SEO
MarkWatch MarkMonitor
Mata Hari Mata Hari
MauiBot Microsoft URL Control
Meanpathbot Moblie Safari
MegaIndex.ru
Metauri
MicroMessenger
Microsoft Data Access
Mister PiX
Mojeek
Mojolicious Mojolicious
Mr.4x3 Msrabot
Musobot Musobot
Nameprotect NICErsPRO
Nessus Name Intelligence
Net Vampire Navroad
Needle
NetLyzer NetLyzer
NetMechanic NetMechanic
NetZIP NetSpider
Netvibes Nettrack
NextGenSearchBot
Nibbler
Nimbostratus Nimbostratus
Nutch Ninja
Nmap
Nuclei
Octopus Octopus
Offline Explorer Offline Navigator
OnCrawl OpenLinkProfiler
OpenVAS
Page Analyzer Page Analyzer
PageAnalyzer PageAnalyzer
PageThing.com PageScorer
Pandalytics
Panscient Panscient
Papa Foto Petalbot
PeoplePal Picsearch
PictureFinder
Piepmatz Piepmatz
Pixray PleaseCrawl
Pockey
ProPowerBot ProPowerBot
ProWebWalker
Probethenet
PxBroker PxBroker
RankingBot2 RankActive
RebelMouse RankFlex
Re-re
ReGet
RocketCrawler
Rogerbot Rogerbot
SBIder SISTRIX
SEOkicks
SMTBot SMTBot
SalesIntelligent
ScanAlert ScanAlert
Scanbot
ScoutJet
Scrapy Scrapy
ScreenerBot Screaming
Searchestate
Semrush Semrush
SemrushBot SeoSiteCheckup
SentiBot Shodan
Seomoz SiteExplorer
SiteLockSpider
SiteSnagger
SiteSucker SiteSucker
Sitebeam
Siteimprove
Sitevigil Sitevigil
SlySearch SlySearch
SmartDownload
Snake Snake
Snapbot SocialRankIOBot
Sociscraper
Sosospider Sosospider
SpaceBison
Spammen
SpankBot SpankBot
Spbot Spbot
Sqlmap Sqlmap
Steeler Sqlworm
Stripper Sqworm
Sucker
SuperBot
SuperHTTP
SurveyBot
Suzuran Suzuran
Szukacz Telesoft
TeleportPro The Intraformant
Thumbor Thumbor
Trendictionbot
Turnitin
TurnitinBot TurnitinBot
TwengaBot Twice
URLy.Warning Typhoeus
V-BOT URLy Warning
VB Project Upflow
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie Virusdie
Voil
WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV WEBDAV
WPScan Wallpapers
WallpapersHD WallpapersHD
WeSEE Web Auto
Web Enhancer
Web Fetch
Web Fuck Web Fuck
Web Pix Web Pix
Web Sauger Web Sucker
WebAuto WebCollage
WebCopier WebCopier
WebFetch WebEnhancer
WebGo IS WebGo IS
WebImageCollector WebImageCollector
WebLeacher WebLeacher
WebReaper WebReaper
WebStripper WebStripper
WebSucker WebSucker
WebWhacker
WebmasterWorldForumBot
Webshag
Website Quester Website Quester
WebsiteQuester WebsiteExtractor
Who.is Bot Webster
Widow Whacker
Wonderbot Woobot
Wotbox
Wprecon Wprecon
Xaldon WebSpider Xenu
Xaldon_WebSpider YoudaoBot
Zermelo
Zeus
Zitebot Zitebot
ZoominfoBot ZmEu
ZoomBot
adscanner adscanner
backlink-check arquivo-web-crawler
cah.io.community cah.io.community
check1.exe check1.exe
clark-crawler clark-crawler
coccocbot-web coccocbot-web
com.plumanalytics com.plumanalytics
crawler.feedback crawler4j
domainsproject.org
gopher
heritrix
internetVista monitor internetVista monitor
ips-agent ips-agent
lwp-trivial lwp-request
magpie-crawler oBot
scalaj-http pcBrowser
serpstatbot probe-image-size
sitechecker.pro ripz
spyfu s1z.ru
satoristudio.net
seobility
seocompany.store
sogouspider
sp_auditbot
sysscan sysscan
tAkeOut
trendiction.com trendiction.com
voyagerx.com webgains-bot
webprosbot x09Mozilla
zgrab x22Mozilla
xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
arendakvartir.kz 0daymusic.org
arewater.com 12masterov.com
aussie-prizes.com blogs.rediff.com
compliance-mary.top fearcrow.com
erectile.bid fym.com.ru
johannesburgsingles.co.za htmlcorner.com
lotto6888.com pipki.r.acdnpro.com
mega-bony-2017.pl pivka.xyz
profitwithalex.info prostitutki-kharkova.org
speedup-my.site site-speed-check.site

934
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
123kuma.com 1001watch.com.ua
178evakuator178.ru 101billion.com
2345.com 1webmaster.ml
2015god.org
300richmond.co.nz
40cg.com 40cg.com
4replicawatch.net 4webmasters.org
5tey7463.weebly.com 5-steps-to-start-business.com
66cpwgln.space 5i2.net
91abcw.com 5kstore.com
a1security.com.ua 76brighton.co.uk
aa14ab57a3339c4064bd9ae6fad7495b5f.com 7xc4n.com
aa8b68101d388c446389283820863176e7.com 888.com
aasoldes.fr 98oi.ru
abcdefh.xyz 9med.net
abcdeg.xyz aa8780bb28a1de4eb5bff33c28a218a930.com
accessoires-mode.in addblueoff.com.ua
actices.com adf.ly
ads.tremorhub.com adobereader-free.ru
adsloads.com adpremium.org
adzpower.com adtech.us
aerogo.com adult3dgames.com
agroeconom.kz advancedmassagebysara.com
aktobe.xkaz.org advancedsoftwaresupport.com
alborzan.com affordablewebsitesandmobileapps.com
alf-img.com agreda.pluto.ro
algarveglobal.com agysya.ru
alive-ua.com akvamaster.dp.ua
allkrim.com alarmobninsk.ru
alphahoverboards.com aliexpresscashback.club
am-se.com all-streaming-media.com
amanda-porn.ga allbizne.co.ua
amazingninja.com allpdfmags.net
an-donut.com aloofly.com
anniemation.com alot.com
anonymous-redirect.com amehdaily.com
apibrowsesmartne-a.akamaihd.net ananumous.ru
apidiamondatanet-a.akamaihd.net apidiamondatanet-a.akamaihd.net
apple.com-webbrowsing-security.review
arabgirls.adultgalls.com arabgirls.adultgalls.com
artel116.ru asscenihotosidea.blogspot.com
arvut.org audiofree.ru
atmovs.com ausergrubhof.info
auspolice.com avkzarabotok.com
autonew.biz avtolombard-krasnodar.ru
aviav.ru azinoofficial777.ru
backiee.com babosas.com
baltgem.com baixar-musicas-gratis.com
bank.uz bard-real.com.ua
bannerconnect.net battlecarnival.su
bannertesting.com beachtoday.ru
baoxaydung.com.vn beatifullandscape.co
best-way.men beauty-clinic.ru
bestcalovebracelet.cn belinka.com.ua
bestplacetobuyeriacta.jw.lt besthomemadepornsites.com
betonka.pro bestofpicture.com
betwinservice.com bet-prognoz.com
bezcmexa.ru bgviagrachrx.com
bigcities.org bimmerpost.com
biketank.ga bitcoinpile.com
biteg.xyz bizfly.info
bkgr.se blacktwhite.com
blackbabesporn.com blagovest-med.ru
blog.axant.it bmw.afora.ru
blogqpot.com board.f00d.de
blogrankers.com bodybuilding-shop.biz
bobinoz.com bonux.nextview.ru
boc.kz bonzybuddy.com
brandedleadgeneration.com boom-celebs.com
bristolhostel.com brendbutik.ru
brillianty.info
bristolhotel.com.ua bristolhotel.com.ua
buyclomidonlaine.com buchananshardware.com
buypillsorderonline.com budilneg.xyz
buzzurl.jp burger-tycoon.com
callawaygolfoutlet.online butstrap.space
callawaygolfstore.online buytizanidineonline.blogspot.com
canacopegdl.com cam-kontakte.org
candcstuccoandstone.com
cartechnic.ru cartechnic.ru
cartierreplica.top cartierloveringreplica.xyz
cartierreplica.xyz cdncash.net
cartiertrinity.xyz cellfun.mobi
casablancamanor.co.za cercacamion.it
catalogs-parts.com cezartabac.ro
ccjp.eu chatroulette.video
cfacarrosserie74.com cheapmoncler.win
chatroulette.life chocolateslim-en-france.com
cheap-pills-norx.com christianlouboutinsaleonline.us
chiblackhawks-jerseys.com ci.ua
chinajerseyswholesalecoupons.com cialischstgerts.com
choice-credit.ru cialisovercounteratwalmartusa.com
cienum.fr civilwartheater.com
ciproandtizanidine.blogspot.com cjmc.info
citetick.com cocyq.inwtrade.com
cleanpcnow.com compliance-barak.top
clickpapa.com compliance-jess.top
clubfashionista.com compliance-mark.top
coindirect.io compliance-olga.top
compliance-alex.xyz conciergegroup.org
compliance-don.top copro.pw
compliance-don.xyz costablue.xyz
compliance-ivan.xyz covadhosting.biz
compliance-jack.top credit-respect.ru
connectionstrenth.com cryptoswap.biz
copenergo.ru cs-passion.pl
cordstrap.cc
corta.co
creams.makeforum.eu
creativehutindia.com
custodieva.ru custodieva.ru
dairyindia.in cymbaltaandtizanidine.blogspot.com
dame-ns.kz cyprusbuyproperties.com
daooda.com
darodar.com darodar.com
data.vtc.pw dating-app.ru
day-news.info decoratinghomes.ga
daydream-studio.ru dimaka.info
delta-line.men dimfour.com
dengi-pod-zalog-nedvizhimosti.ru dinosaurus.site
detki-opt.ru directrev.com
dichvuvesinhhanoi.com discountbarbour.online
diesel-parts28.ru distver.ru
digital-video-processing.com djekxa.ru
dignitasdata.se domain2008.com
dkvorota.ru
dmmspy.com
dogclothing.store
dogsrun.net
dokfilms.net
domaincheck.io
domainroam.win domainroam.win
donna7753191.ru domainsatcost.com
dorratex.tn dostavka-v-krym.com
dostavka-v-ukrainu.ru downloaddy.net
dotmass.top dvervmoskvu.ru
downvids.net e2click.com
doxyporno.com e705.net
doxysexy.com edge.sharethis.com
dpihatinh.gov.vn ee77ee.com
dprtb.com efkt.jp
dptaughtme.com ege-essay.ru
drivotracker.com ekspresihati.info
drugstoreforyou.com electro-prom.com
druzhbany.ru eluxer.net
dvd-famille.com energydiet-info.ru
e-commerce-seo.com engines-usa.com
e-poker-2005.com enskedesquashclub.se
eas-seo.com enternet.ee
ekn-art.se eredijovon.com
ekosmetyki.net.pl escortslet.net
eleimgo.pw essay-writing.work
elektrozigarettekaufen2.blogse.nl estelight.ru
elkacentr.ru etenininrade.ga
elstal.com.pl exlarseva.webblog.es
empirepoker.com exoneration-project.us
enpolis.ru expertblog.info
eropornosex.ru extremal-blog.com
esfchat.tk ezigarettekaufen.myblog.de
eupornstar.info f5mtrack.com
euroskat.ru falcoware.com
executehosting.com fba-mexico.com
extractorandburner.com fbdownloader.com
eyes-on-you.ga fetishinside.com
ezaz.info fetlifeblog.com
failingmarriege.blogspot.com
fanpagerobot.com
fashion-mk.net
fastcrawl.com
fatfasts-4tmz.com
favoritemoney.ru
feargames.ru
feel-planet.com
fefo.gdn
femmesdenudees.com
fickenbumsen.net fickenbumsen.net
findanysex.com filmania-x.ru
fishtauto.ru findpik.com
fitnesspiks.com fix-website-errors.com
footbalive.org fordsonmajbor.cf
forensicpsychiatry.ru foto-weinberger.at
forex-indextop20.ru foxinsocks.ru
fotosfotos.eu free-gluten.ru
free-share-buttons.top
free-social-buttons.com free-social-buttons.com
fuckingawesome.com free-stock-illustration.com
gearcraft.us free-today.com
genericlowlatencyasiodriverhq.aircus.com freelotto.com
gentamicineyedrops.blogspot.com freewhatsappload.com
getmyads24.com fuckbuddybestgilf.info
gkvector.ru fungirlsgames.net
glopages.ru galaxy-family.ru
go2album.com gate5.co.za
golden-catalog.pro gembird.com
goodnightjournal.com generalporn.org
goodwriterssales.com gg.zzyjxs.com
gox.com.ua ghernnqr.skyrock.com
greatfind-a.akamaihd.net golden-praga.ru
greatzip.com goodly.pro
grtyi.com goodprotein.ru
gsasearchenginerankerhelp.com googlemare.com
gsmtlf.ru gotcher.us
grand-chlen.ru
gsasearchengineranker.pw
gsasearchenginerankerdiscount.com
gsmlab.pl
guardlink.org
guge.io guge.io
guruofcasino.com gwagka.com
hd-filmy.net gwebtools.com
hdwallpapers-free.com gymi.name
hdxnxxtube.mobi gz2.bbsoldes.fr
healmytrauma.info hd720kino.ru
herehloadibs.cf hdimagelib.com
homemade.gq healing-dysplasia.ru
hotblog.top heartofbeijing.blogspot.com
hotdl.in hentaiheroes.com
hotxnights.info hermesbirkinhandbagoutlets.com
housekuba.org hesteel.pl
howtoclean.club hexpilot.com
hulfingtonpost.com hikesearch.net
hyipmanager.in homeart.space
ibb.com.ua host-protection.com
ictizanidinehcl4mg.blogspot.com host-tracker.com
igadgetsworld.com hoste.octopis.com
ilmexico.com hosting-tracker.com
imagerydatabase.com hotblognetwork.com
impresagaia.it hothot.ru
infokonkurs.ru hotloans.ru
infospot.pt houseofrose.com
interior-stickers.ru humbmosquina.tk
internetartfair.com ido3.com
investingclub.ru iframe-toloka.com
investsuccess.org igrovyeavtomaty777.ru
iqbazar.ru ilovevitaly.xyz
iqupdatetmz.win imabase.com
iusstf.org imagefinder.site
imgarcade.com
imperia31.ru
incest-ru.com
incolors.club
increasewwwtraffic.info
indetiske.ya.ru
instasexyblog.com
internet-apteka.ru
investmac.com
invitefashion.com
iphantom.com
istizanidineanopiate.blogspot.com
ivanstroi.ru ivanstroi.ru
izismile.com iyasimasennka.com
jacago.com jackpotchances.com
jackwolfskinoutlet.online
japan-bearings.ru
jmat.cn
jobgirl24.ru jobgirl24.ru
joy-penguin.com journeydownthescale.info
jrcigars.com junketjuice.blogspot.com
kakadu-interior.com.ua kazan.zrus.org
karpun-iris.ru keywordsdoctor.com
kartiniresto.com keywordsuggests.com
kazka.ru kino-ecran.ru
kerei.ru kino-rating.ru
kevblog.top kiskinhouse.com
keywordhouse.com komp-pomosch.ru
kinohall.ru kongruan.com
klejonka.info kopihijau.info
kriokomora.info koral.se
kvartir-remont.biz kosova.de
kvartira-sutochno.com kvartira-sutochno.com
lalalove.ru laexotic.com
landoftracking.com laxdrills.com
landscaping.center levaquin750mg.blogspot.com
lemon-ade.site lexiangwan.com
letmacworkfaster.site lflash.ru
letto.by li-er.ru
liupis.com lietaer.com
lmrauction.com linkbux.ru
localmatchbook.com linkdebrideur.xyz
locksmith.jp liveshoppersmac.com
lonfon.xyz lordzfilmz.me
louboutinshoes.xyz louboutinreplica.xyz
lutherstable.org lsex.xyz
luxmagazine.cf ltvperf.com
macotool.com lunchrock.co
magazintiande.ru m4ever.net
magnetic-bracelets.ru macfix.life
mafa-free.com
mafcards.ru
mahnwachen-helfen.info mahnwachen-helfen.info
maladot.com manuscript.su
mature.free-websites.com masturbate.co.uk
max-eclat.men maxxtor.eu
mcadamssupplyco.com mbiologi.ru
medanestesia.ru mebel-ekb.com
medicinacom.ru mebeldekor.com.ua
medmajor.ru mega-bony2017.pl
meds-online24.com mensandals.xyz
metaxalonevstizanidine.blogspot.com mesto-x.com
mikrobiologies.ru michaelkorsoutlet.store
microsearch.ru
migente.com
militarysale.pro
minecraft-neo.ru
miniature.io
mir-business-24.ru mir-business-24.ru
misswell.net mitrasound.ru
mmostrike.ru moesonce.com
mobicover.com.ua mokrayakiska.com
moesexy.com mole.pluto.ro
monclerjacketsoutlet.pw monetizer.com-01.site
most-kerch.org moneyviking-a.akamaihd.net
moyaterapiya.ru moscow.online-podarki.com
mundoaberrante.com mostantikor.ru
mycaf.it mouselink.co
myghillie.info moviemail-online.co.uk
myliveblog.ru mrinsidesales.com
mypornfree.ru musezone.su
myseoconsultant.com mybackgroundlandscape.blogspot.com
mysex21.com myfreetutorials.com
nakrutka.cc nacap.ru
nalogovyy-kodeks.ru namenectar.com
narkologiya-penza.ru natali-forex.com
newyorkhotelsmotels.info new-post.tk
next-dentists.tk newfilmsonline.ru
november-lax.com news-speaker.com
nuit-artisanale.com newstaffadsshop.club
nuup.info nextlnk12.com
o00.in nibbler.silktide.com
odesproperty.com nina.az
office2web.com nlfjjunb5.ru
nonews.co
notfastfood.ru
nrjmobile.fr
nw-servis.ru
offtopic.biz offtopic.biz
online-zaymy.ru olgacvetmet.com
openstat.com one-gear.com
orenburg-gsm.ru online-templatestore.com
origin-my.ru online.ktc45.ru
outclicks.net onload.pw
ownshop.win openmediasoft.com
paleohub.info optibuymac.com
panchro.co.uk orgasmatrix.com
panicatack.com osoznanie-narkotikam.net
par-fallen.ga ourville.info
park.above.com oyster-green.com
partners-ship.pro ozoz.it
partnersafe.men page2rss.com
petitions.whitehouse.gov partybunny.ru
pc-virus-d0l92j2.pw
pdamods.ru
perfectplanned.com
petrovka-online.com
phelissota.xyz phelissota.xyz
picturesfrom.com pijoto.net
pinup-casino1.ru pillscheap24h.com
pisanieprac.info piski.top
pizda.lol pizza-tycoon.com
pizza-imperia.com plaff-go.ru
plastweb.ru
platesauto.com
playtap.us playtap.us
pochemychka.net pornmania.pl
pons-presse.com porno-dojki.net
popup-fdm.xyz porno-play.net
pornhub-ru.com pornogad.com
porno-home365.com pornstartits.xblog.in
porno.simple-image.com.ua powc.r.ca.d.sendibm2.com
pornogig.com pr0fit-b0x.com
pornokorol.com pregnant.guru
pornonik.com pro-tec.kz
pornoreino.com producm.ru
pornotubexxx.name promoheads.com
portal-eu.ru quit-smoking.ga
positive2b.ru qwarckoine.com
pospr.waw.pl ranksonic.org
pozvonim.com remote-dba.de
pp-budpostach.com.ua retreatia.com
pravoholding.ru
preconnubial.usuby.site
preparevideosafesystem4unow.site
prizefestival.mobi
pro-poly.ru
professionalwritingservices15.blogspot.ru
prom23.ru
prostitutki-odessa.org
pufip.com
puserving.com
puteshestvennik.com
randalljhoward.com
rapevideosmovies.com
rasteniya-vs-zombi.ru
realizmobi.com
recipedays.com
regdefense.com
registratciya-v-moskve.ru
registrydomainservices.com
registrysweeper.com
restorator-msk.ru
reward-survey.net reward-survey.net
rfserial.net rewardit.com
rocketchange.ru ribieiendom.no
rockingclicks.com rn-to-bsn.com
roma-kukareku.livejournal.com
rp9.ru rp9.ru
ru-dety.ru ru-mediaget.ru
rucrypt.com running-line.ru
rukino.org rusenvironmental.net
rumamba.com russia-today-video.ru
rus-pornuha.com rvtv.ru
ryetaw.com
sack.net
sad-torg.com.ua
samara.rosfirm.ru samara.rosfirm.ru
sanidumps.com sape.top
sanyuprojects.com sarahmilne.top
scalerite.co.za sashagreyblog.ga
scanmarine.info sbetodiodnye-lampy.ru
scanner-andrew.top sbricur.com
scanner-barak.top
scanner-don.top scanner-don.top
scanner-nelson.top scanner-john.top
scansafe.net school-diplomat.ru
scmor.ilxc.cc search.1and1.com
searchinterneat-a.akamaihd.net searchencrypt.com
seeingmeerkat.com searchwebknow-a.akamaihd.net
seoholding.com seminarykansas904.ml
seozoom.it seo-2-0.com
sexkontakteao.info sexsaoy.com
sextracker.de signoredom.com
sexvideo-sex.com similardeals.net
sexy-pings.com sirpornogratis.xxx
sexy.babes.frontend-stack.top skatestick.bid
shillyourcoins.com slavkokacunko.de
siha.de slimcdn.com
sisi-go.ru smsactivator.ru
skuteczna-dieta.co.pl social-buttons.xyz
skylta.com socialmediasuggest.com
small-game.com solicita.info
smartshoppymac.com sonata-arctica.wz.cz
sneakyboy.com spacash.com
snsdeainavi.info srecorder.com
soblaznu.net stard.shop
solinf.co statistici.ro
solmarket.by steebook.com
soochi.co stockspmb.info
soserfis.com streetfire.net
souvenir.cc strongsignal-a.akamaihd.net
space4updating.win studentguide.ru
speechfoodie.com supermama.top
spin2016.cf
sport7777.net
sps-shop.com
spy-app.info
stroydetali.ru
suggest-keywords.com
sukarame.net
svtrd.com
taaaak.com
tacbelarus.ru tacbelarus.ru
tarad.com takethatad.com
tattooreligion.ru tech4master.com
telegramdownload10.com teletype.in
templates.franklinfire.co terrafootwear.us
templates.radiodigital.co teslathemes.com
terraclicks.com the-trader.net
thesmartsearch.net thefds.net
thfox.com tizanidine4mg.blogspot.com
timecrimea.ru tizanidinecost.blogspot.com
tizanidinesideeffectsweightloss.blogspot.com tizanidinehcl4mgtabinfo.blogspot.com
tongkatmadura.info tizanidinestreetvalue.blogspot.com
top10-way.com tizanidinevscyclobenzaprine.blogspot.com
topcar-krasnodar.ru tizanidinezanaflex.blogspot.com
topshef.ru tksn.ru
tradedeals.biz tmearegion26.com
trafaret74.ru topkarkas.com
trafficshaper.com torrents.cd
tramadolandtizanidine.blogspot.com tovaroboom.vast.ru
traxdom.ru toys.erolove.in
trkdf.com traffic2money.com
truck-addzilla.life trafficmp.com
trudogolik.net truebeauty.cc
trumpetedextremes.com tubo360.com
tula.howotorg.ru
turbo-suslik.org
turizmus.us
turn-up-life.life turn-up-life.life
tvoystartup.ru tvoystartup.ru
tx41tclega.ru
u17795.netangels.ru
ua-company.ru
uginekologa.com
ukkelberg.no
unece.org unece.org
urldelivery.com unmaroll.ya.ru
uroffer.link unrealcommander.biz
urzedowski.eu us-america.ru
usbggettwku.ga
v-doc.co
valkiria-tk.ru
vandrie-ict.nl vandrie-ict.nl
vbabule.net vavilone.com
vchulkax.com ventelnos.com
vids18.site versaut.xxx-cam.webcam
visa-china.ru vertaform.com
viagengrarx.com
viagra-soft.ru
video-chat.cn
vip-file.com
vipms.ru
vipps.com.my
virus-respirators.com
visionwell.com.cn visionwell.com.cn
vkontaktemusic.ru vitoriacabos.com
vksex.ru viven.host.sk
vonradio.com vizitki.net
voronezh.xrus.org voloomoney.com
wallpapers-best.com vrnelectro.ru
watchmygf.net vsexkontakte.net
warningzscaler.heraeus.com
watchdogs-2.ru
watchinf.com
waterefficiency.co waterefficiency.co
waterpurifier.club webmonetizer.net
webshoppermac.com website-speed-up.top
website-analytics.online website-stealer.nufaq.com
webtherapy.ru websitevaluebot.com
westermarkanjou.se
westvilletowingservices.co.za westvilletowingservices.co.za
while.cheapwebsitehoster.com whatistizanidine2mg.blogspot.com
wleuaprpxuvr.ga wholesalejerseys-cheapest.com
wowas31.ucoz.ru wholesalejerseysgaa.com
wrz0iuebwhp5fg.freeddns.com win-spy.com
wtsindia.in wineration.com
winx-play.ru
witclub.info
wnoz.de
womensterritory.ru
wordpress-crew.net
www.labves.ru www.labves.ru
www.wohnkabinen-shop.de xgames-04.com
wygraj-teraz.com xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
x-lime.net xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
x-mix.info
xkaz.org
xmladserver.com
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xn----9sbebi2bvzr7h.xn--p1ai xn----9sbubg3ambdfl1j.xn--p1ai
xxxhdvideo.site xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
yaderenergy.ru xn--b1agm2d.net
yellocloud.be xvideosporn.biz
youjizz.com y8games-free.com
yurgorod.ru yogamatsexpert.com
zacreditom.ru yoluxuryevents.com
zajm-zalog-krasnodar.ru yoursite.com
zaloadi.ru youtube-downloader.savetubevideo.com
zarabotat-na-sajte.ru za-fun-offer.com
zigarettenonl.canalblog.com zaidia.xhost.ro
zigarettenonlinekaufen.tumblr.com zakazfutbolki.com
zone-kev717.info zeroredirect1.com
zskdla.site zeroredirect7.com
zytpirwai.net zetmaster.ru
zigarettenonlinekaufen2.drupalgardens.com
znakom.sibtest.ru
zona-aqua.ru

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1qingdao.com 11235813.webzdarma.cz
2girls1cup-free.com 18ps.ru
7wind.ru 202ch.com
adrenalin-stk.ru 2728fb936f0.com
adventureparkcostarica.com 2nt.ru
aero2.ru 300richmond.co.nz
affrh2015.com 40cg.com
alive-ua.com 4kpics.rocks
all4wap.ru 4replicawatch.net
allnews.md 8kisses.eu
anaksma.info 8si.ru
apessay.com 98oi.ru
astana.xxxkaz.org aasoldes.fr
autoblogger24.info adktrailmap.com
autotrends.today adman.se
aviav.co adsfresh.men
bestplacetobuyeriacta.jw.lt alfa-img.com
beta.hotkeys.com allstatesugarbowl.org
bizlist.com.de animali.deagostinipassion.it
blog.axant.it anjalika.co.in
bloggen.be anti-virus-removal.info
bobba.dzaba.com approved.su
boltalko.xyz artparquet.ru
buy-cheap-pills-order-online.com aviav.org
buzzonclick.com avtolombard-krasnodar.ru
buzzurl.jp awaybird.ru
byme.se azbukadiets.ru
bannerspace.com
battle.net
bbtec.net
bestfortraders.com
bestserials.com
bigpictures.club
bimatoprost-careprost.com.ua
bluesman.nu
bonky.biz
brainzpod.win
buttons-for-your-website.com
buyantiviralwp.com
calvet.altervista.org
cam-kontakte.org
carsdined.org
cartierlovebraceletreplica.xyz
cartierreplica.xyz cartierreplica.xyz
cheapwebsitehoster.com casite-513049.cloudaccess.net
christianlouboutinreplica.pw cialischstgerts.com
citysecurity.nu clickpapa.com
compliance-ivan.top clksite.com
compliance-walter.top coderstate.com
concordexoticrentals.com cognitiveseo.com
constantaservice.net constantaservice.net
copypaste.traderzplanet.in coolbar.pro
copyrightclaims.org covid-schutzmasken.de
coslab.club cranly.net
covadhosting.biz culmimg.pw
cs-passion.pl customerpromos-a.akamaihd.net
culturevie.info danashop.ru
cypernhuset.se dating-app.ru
data-mining.tk dating-time-now.com
ddospanel.com ddpills.com
deda-moroza-zakaz.ru
defenderxtactical.com
dekoration.us
den-noch24.ru
dev.citetick.com dev.citetick.com
dimaka.info devochki.top
docs4all.com divatraffic.com
doctissimo.fr dokumentalkino.net
dolohen.com domainsatcost.com
downloaddy.net dostavimvdom.ru
downloads-whatsapp.com doublepimp.com
dprtb.com earncash.com.ua
drpornogratisx.xxx econom.co
dumpsmania24.com eduserver.net
eachdayisagift.review ekspertmed.com
ecommerce-seo.org elektrozigaretten1.tumblr.com
embedle.com emediate.eu
enbersoft.com encodable.com
energy-ua.com erotikchat-24.com
enpolis.ru errorfixing.space
enskedesquashclub.se evrotekhservis.ru
erotag.com feel-planet.com
escortplius.com ferrotodo.com
evehemming.blogspot.com.au fickluder69.com
f-online.de fivedwld.cf
f5mtrack.com forums.toucharcade.com
femmesdenudees.com foxinsocks.ru
financeloan.us frbizlist.com
findsexguide.com freakycheats.com
flashbannernow.com
formularz-konkurs.tk
fortevidyoze.net
free-fbook-traffic.com
free-video-tool.com
free411games.com free411games.com
freshwallpapers.info frustrated-favorable.gq
funnypica.com fusoradio.info
fyl.com.ru g.starmoe.xyz
gamedayassist.com gallery.rennlist.com
gamerscorps.com gameprimary.com
gdebestkupit.ru
ge0ip.net
geile-lelly.eu geile-lelly.eu
getridofstretchmarks.org gogps.me
gigixo.com goodwinmetals.co
glall.ru googlemare.com
globalsurfari.com gourcy.altervista.org
globatur.ru gpirate.com
gopro-online.info gsasearchengineranker.space
gorabagrata.ru hackers-crackers.tk
growshop.es hahashka.ru
hannasolution.ru handsandlegs.ru
hermesbelts.xyz happy.new.yeartwit.com
hothot.ru hd-filmy.net
hoztorg-opt.ru hermesbracelets.xyz
ideawheel.com hkdiiohi.skyrock.com
idegenvezeto.eu hoporno.com
iminent.com hotblognetwork.com
in-tandem.co hotloans.ru
inclk.com houdom.net
infowarcraft.ru huimin764128.com
ispaniya-costa-blanca.ru hydropump.su
istizanidineanarcoticdrug.blogspot.com i-service.kz
izhevsk.zrus.org idc.com.ua
jennyfire.ru ilovevitaly.com
jestr.org imabase.com
joessmogtestonly.com immobiliaremassaro.com
jovencitas.gratis inbabes.sexushost.com
kasino-money.pw instabid.tech
kazan.xrus.org int.search.tb.ask.com
kinky-fetishes.com iwantedmoney.com
janavibekken.no
job.icivil.ir
karpun-iris.ru
kartiniresto.com
keywordteam.net
kino-rating.ru kino-rating.ru
kollekcioner.ru kmgamex.cf
kozhniebolezni.com kovesszucs.atw.hu
kraljeva-sutjeska.com kriokomora.info
landoftracking.com ladsblue.com
lexaprogeneric.link larutti.ru
lietaer.com laserpen.club
lineshops.biz law-check-seven.xyz
littleberry.ru legalrc.biz
lmrauction.com livingcanarias.com
lulea-auktionsverk.se lombardfinder.ru
m1media.net longgreen.info
m4ever.net losangeles-ads.com
magazintiande.ru lovelycraftyhome.com
manyresultshub-a.akamaihd.net magicdiet.gq
maslenka.kz maximilitary.ru
medtherapy.ru mb140.ru
meduza-consult.ru meccadumps.net
meetingrainstorm.bid meget.co.za
merryhouse.co.uk menstennisforums.com
mixx.com microstatic.pl
moneytop.ru mirmedinfo.ru
monitorwebsitespeed.com mister-shop.com
moscow-clining.ru mmofreegames.online
mototsikl.org mobsfun.net
muabancantho.info mojaocena.com
mundoaberrante.com moncleronline.xyz
myfreetutorials.com msfsaar.de
nakrutka.cc myindospace.com
nalogovyy-kodeks.ru myonigroup.com
nero-us.com mypornfree.ru
new-apps.ru mystats.xyz
nicovideo.jp nhl09.ru
njkmznnb.ru nlfjjunb5.ru
nuit-artisanale.com
o333o.com o333o.com
obiavo.by olympescort.com
oklogistic.ru park.above.com
onclickpredictiv.com pattersonsweb.com
onko-24.com picture-group.com
orion-code-access.net piulatte.cz
ossmalta.com pix-hd.com
paighambot.com plaff-go.ru
panouri-solare-acoperis.com popup-hgd.xyz
paparazzistudios.com.au pregnant.guru
partnerline.men privatefx-in.ru
pastaleads.com proekt-mos.ru
patterntrader-en.com profolan.pl
paydayonlinecom.com progress-upakovka.ru
pcimforum.com puzzleweb.ru
pdns.download qcstrtvt.bloger.index.hr
perso.wanadoo.es recipedays.com
photochki.com registratciya-v-moskve.ru
pinsdaddy.com resultshub-a.akamaihd.net
piter.xrus.org rome2rio.com
pl-vouchers.com rubbed.us
plastweb.ru rukino.org
pornhubforum.tk running-line.ru
porno-play.net s8-nowy-wygraj.comli.com
porno-raskazy.ru saletool.ru
porno-video-chati.ru sarm.tk
powitania.pl saveindex.xyz
privhosting.com sbprabooks.com
prostitutki-tyumeni.org scanner-ivan.top
pussyfleet.com scat.porn
raavidesigns.com score-ads.men
ranksignals.com searchaddis.com
regionshop.biz security60-e.com
reimageplus.com seoexperimenty.ru
reklama1.ru serpstat.com
retailwith.com sex-znakomstva.online
ribieiendom.no sharebutton.net
roofers.org.uk shisha-swag.de
rootandroid.org
sa-rewards.co.za
sad-torg.com.ua
santasgift.ml
santechnik.jimdo.com
scrapinghub.com
secret.xn--oogle-wmc.com
seorankinglinks.us
shop-electron.ru
shoppingjequiti.com.br
sicfor.bcu.cc sicfor.bcu.cc
sientalyric.co slayerlife.com
silverage.ru
siteheart.net
sitevalued.com
sizeplus.work
snworks.com snworks.com
social-s-iii.xyz social-buttons.xyz
soft-terminal.ru solicita.info
sotkal.lark.ru
spammen.de
special-porn.com
spitfiremusic.com
steamoff.net steamoff.net
stroy-portal22.ru steelmaster.lv
superinterstitial.com stockquotes.wooeb.com
swsociety.se stockspmb.info
tako3.com stroyhelp-dv.ru
technika-remont.ru supermodni.com.ua
tedxrj.com surffoundation.nl
tizanidinedosageforsleep.blogspot.com svensk-poesi.com
topclickguru.com tds-advert005.info
tour-line.net telegramdownload10.com
toursmaps.com theallgirlarcade.com
track-rankings.online theautoprofit.ml
tsstcorpcddvdwshbbdriverfb.aircus.com thenetinfo.com
umekana.ru ticketsys.inetwd.com
unlimitdocs.net titan-cloud.life
upproar.com tizanidinedrugtest.blogspot.com
urdoot.win tizanidineingredients.blogspot.com
ussearche.cf tizanidinepain.blogspot.com
tmtrck.com
toolsky.com
uasb.ru
um-razum.ru
universals.com.ua
utiblog.fr
v24s.net
vapomnoncri.tk
vapsy.com
vduplo.ru
vekzdorov.ru vekzdorov.ru
video-camer.com via-gra.webstarts.com
videochat.cafe videochat.life
vzlomtw.com vids18.site
webradiology.ru viktoria-center.ru
weburlopener.com visionwell.com.cn
wnhjavlhezp.gq vkonche.com
world-mmo.com vladimir.xrus.org
www.solartek.ru warezkeeper.com
xaylapdiendanang.com webstatsdomain.org
xev.ru weburok.com
xn----btbdvdh4aafrfciljm6k.xn--p1ai woman-orgasm.ru
xn--80abgj3a5acid6ghs.top wurr.voila.net
xn--d1acah0c.xn--p1ai x-lime.net
xn--h1ahbi.com.ua x69ty.ru
xsfetish.org xn--80ak6aa92e.com
xxx-treker.ru xn--80atua3d.xn--p1ai
yaaknaa.info yougetsignal.com
yaoguangdj.com
yebocasino.co.za
youtubedownload.org
zog.link
zynax.ua

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.0.107 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2740 ### Version: V4.2021.11.2741
### Updated: Fri Nov 26 22:01:17 UTC 2021 ### Updated: Sat Nov 27 10:03:37 UTC 2021
### Bad Bot Count: 630 ### Bad Bot Count: 630
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2740 ### Version: V4.2021.11.2741
### Updated: Fri Nov 26 22:01:17 UTC 2021 ### Updated: Sat Nov 27 10:03:37 UTC 2021
### Bad Bot Count: 630 ### Bad Bot Count: 630
################################################### ###################################################
### Version Information ## ### Version Information ##