vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 06:15:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2902 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-06 22:07:10 +00:00
parent 34f48cac7e
commit 8d355e2025
46 changed files with 81319 additions and 81319 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2901 #### Version: V4.2022.02.2902
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2901 #### Version: V4.2022.02.2902
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2901 #### Version: V4.2022.02.2902
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10028
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2901 #### Version: V4.2022.02.2902
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10032
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10032
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10032
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10032
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

10032
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

10032
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

10032
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2901 2902

10032
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10032
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.69.146.73 0; 52.229.17.53 0;

10032
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

10032
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

10032
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

10032
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.69.146.73 0; 52.229.17.53 0;

10032
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.69.146.73 0; 52.229.17.53 0;

10032
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Abonti AiHitBot
BBBike Bolt
Copier Disco
IndeedBot Download Wonder
Probethenet LinkScan
RankurBot Ltx71
Titan Quick-Crawler
WebGo IS WallpapersHD
ZoomBot ZoominfoBot
ips-agent s1z.ru

302
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 01h4x.com
80legs 404enemy
ADmantX
ALittle Client ALittle Client
ASPSeek Abonti
Aboundex Acunetix
AfD-Verbotsverfahren
AiHitBot AiHitBot
Anarchy99 AllSubmitter
Asterias AlphaBot
Attach Anarchie
AwarioRssBot Ankit
Anthill
Apexoo
Atomseobot
AwarioSmartBot AwarioSmartBot
BBBike BDCbot
BDFetch
BackStreet BackStreet
BackWeb Badass
Backlink-Ceck Bandit
BacklinkCrawler
Barkrowler
BatchFTP BatchFTP
Battleztar Bazinga BetaBot
Black Hole
Blow Blow
BlowFish BotALot
Brandprotect Brandprotect
Brandwatch BuiltWith
BuiltBotTough Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CATExplorador
CCBot CCBot
CODE87 CODE87
CSHttp Calculon
CheTeam CensysInspect
CheeseBot CheeseBot
ChinaClaw CherryPicker
Citoid Chlooe
Claritybot Claritybot
Cliqzbot
Collector Collector
Crawling at Home Project Copyscape
Cosmos
CrazyWebCrawler CrazyWebCrawler
CrunchBot Crescent
DIIbot Custo
DSearch Devil
Deusu Dirbuster
DigitalPebble
Discobot Discobot
Discoverybot DnBCrawler-Analytics
Dispatch
DittoSpyder
DomCopBot
DomainCrawler DomainCrawler
Domains Project
Dotbot
Drip Drip
ECCP/1.0 ECCP/1.0
EasyDL EMail Siphon
EirGrabber Evil
Exabot Exabot
Express WebPictures Ezooms
ExtLinksBot
Extractor
FDM FDM
FHscan FHscan
FemtosearchBot FemtosearchBot
Fimap Fimap
FlashGet Foobot
Freeuploader Freeuploader
FyberSpider FrontPage
Fyrebot Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
Gigabot Genieo
Grafula GermCrawler
GrapeFX GetWeb
HTTP::Lite Go!Zilla
HTTrack GrapeshotCrawler
GridBot
HMView
HTMLparser
Haansoft Haansoft
HaosouSpider Havij
Heritrix Heritrix
IRLbot Hloader
IDBTE4M
Iblog
Id-search
IlseBot
Image Fetch
Image Sucker
IndeedBot
Indy Library
InfoNaviRobot InfoNaviRobot
Intelliseek Intelliseek
InternetSeer Internet Ninja
Iria
Iskanie Iskanie
IstellaBot IstellaBot
JamesBOT Jbrofuzz
JennyBot Jetty
Joomla
JustView JustView
Jyxobot
Kenjin Spider Kenjin Spider
LWP::Simple
Lanshanbot
Larbin
Leap Leap
LeechGet
LexiBot LexiBot
Lftp
LibWeb
Libwhisker Libwhisker
LieBaoFast LieBaoFast
Likse
LinkScan LinkScan
LinkWalker Linkdexbot
Linkbot LinkextractorPro
LinqiaMetadataDownloaderBot LinqiaScrapeBot
Lipperhey Lipperhey Spider
MFC_Tear_Sample Litemage_walker
MJ12bot Ltx71
MTRobot MQQBrowser
Magnet Magnet
Majestic-SEO Majestic SEO
MarkMonitor MarkMonitor
Mass Downloader
Masscan Masscan
Mb2345Browser Mata Hari
Mediatoolkitbot MeanPath Bot
Microsoft URL Control Meanpathbot
Minefield Metauri
MicroMessenger
Microsoft Data Access
Mister PiX
Mojeek
MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila
Mr.4x3
NICErsPRO
NPbot NPbot
NearSite
Needle
Nessus
NetAnts NetAnts
NetSpider NetLyzer
NetZIP NetMechanic
Netcraft Netcraft
NextGenSearchBot Netvibes
Nibbler Nimbostratus
Ninja Nuclei
Nmap
Not
Nutch Nutch
Offline Explorer OpenVAS
OnCrawl
Openfind Openfind
Openvas
OrangeBot OrangeBot
OrangeSpider OrangeSpider
OutclicksBot OutclicksBot
PHPCrawl POE-Component-Client-HTTP
Page Analyzer PageScorer
PageGrabber Papa Foto
PageThing.com Picsearch
Pandalytics PictureFinder
Panscient
Picscout
Piepmatz
Pimonster Pimonster
Pixray ProWebWalker
PleaseCrawl
Psbot
Pu_iN Pu_iN
Pump
PyCurl
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot
Rankivabot
ReGet ReGet
RealDownload
Reaper
RepoMonkey RepoMonkey
RocketCrawler RocketCrawler
Rogerbot Rogerbot
SEOkicks-Robot SEOprofiler
SMTBot SISTRIX
SalesIntelligent SalesIntelligent
Screaming
ScreenerBot ScreenerBot
Semrush ScrepyBot
SemrushBot Searchestate
SentiBot SentiBot
SeobilityBot SeoSiteCheckup
Siphon Shodan
SiteSnagger SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider
Sitebeam Sitebeam
Siteimprove SmartDownload
Snake Snake
SocialRankIOBot SpaceBison
Sosospider
SpankBot SpankBot
SputnikBot Spanner
Stripper Spbot
Sucker Spinn3r
Sucuri
SuperBot SuperBot
SuperHTTP SuperHTTP
Suzuran Suzuran
Telesphoreo T8Abot
Telesphorep Telesoft
TheNomad The Intraformant
Thumbor
Titan Titan
Toata
Tracemyfile Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot
Turingos Turingos
TwengaBot Twice
URLy.Warning Upflow
V-BOT
VB Project VB Project
VelenPublicWebCrawler Vagabondo
VeriCiteCrawler
Virusdie Virusdie
VoidEYE Voil
Voltron Voltron
WBSearchBot WASALive-Bot
WEBDAV WEBDAV
WISENutbot WISENutbot
WWW-Collector-E WWW-Mechanize
WWW::Mechanize
WWWOFFLE WWWOFFLE
Web Pix
Web Sauger Web Sauger
Web Sucker Web Sucker
WebBandit
WebEnhancer
WebFetch WebFetch
WebGo IS WebFuck
WebImageCollector
WebLeacher
WebPix WebPix
WebReaper WebZIP
WebStripper
Website Quester Website Quester
WebsiteExtractor Whatweb
WebsiteQuester Wonderbot
Widow
Woobot Woobot
Xaldon_WebSpider Wprecon
Xaldon WebSpider
Zermelo
Zeus
Zitebot Zitebot
ZoomBot ZoomBot
ZoominfoBot ZoominfoBot
ZyBorg ZyBorg
arquivo-web-crawler
arquivo.pt arquivo.pt
cah.io.community
clark-crawler
coccocbot-web coccocbot-web
com.plumanalytics
crawl.sogou.com crawl.sogou.com
crawler.feedback gopher
dataforseo.com
evc-batch
facebookscraper
heritrix heritrix
instabid
iubenda-radar iubenda-radar
meanpathbot lwp-request
lwp-trivial
mediawords mediawords
muhstik-scan s1z.ru
polaris version seocompany.store
satoristudio.net seoscanners
seostar
serpstatbot serpstatbot
siteripz
sp_auditbot
spyfu spyfu
tAkeOut sysscan
trendiction.com trendiction.com
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
webmeup-crawler voyagerx.com
webprosbot webprosbot
xpymep1.exe xpymep1.exe
zauba.io zgrab

310
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 404checker
ADmantX 404enemy
ALittle Client AIBOT
ASPSeek
Aboundex Aboundex
Acunetix Acunetix
Aipbot AfD-Verbotsverfahren
Alexibot
Anarchy
Ankit Ankit
Anthill
Apexoo Apexoo
Asterias Aspiegel
Attach Attach
BDCbot AwarioRssBot
BackDoorBot AwarioSmartBot
BackStreet BBBike
BLEXBot
BackWeb
Backlink-Ceck Backlink-Ceck
Badass Barkrowler
Blackboard BatchFTP
Battleztar Bazinga
BetaBot
Black Hole
BlackWidow
BlowFish BlowFish
BotALot
Brandprotect Brandprotect
Buddy Brandwatch
BuiltBotTough BuiltBotTough
BuiltWith BuiltWith
CODE87 BunnySlippers
BuzzSumo
CATExplorador
CazoodleBot CazoodleBot
CensysInspect CheeseBot
CheTeam
CherryPicker
Chlooe Chlooe
Citoid Citoid
Claritybot Claritybot
Cocolyzebot Cocolyzebot
Collector CopyRightCheck
Copyscape Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
CrunchBot CrunchBot
CyotekWebCopy Curious
DIIbot Custo
DSearch DBLBot
DTS Agent DTS Agent
DatabaseDriverMysqli
Devil Devil
Digincore DigitalPebble
Disco
Discoverybot Discoverybot
DomainAppender Dispatch
DomainStatsBot DittoSpyder
Download Wonder DomainCrawler
Drip DomainSigmaCrawler
EMail Siphon EMail Siphon
Ecxi EirGrabber
Evil EroCrawler
Express WebPictures Exabot
ExtLinksBot Extractor
Ezooms ExtractorPro
Extreme Picture Finder
EyeNetIE
FDM FDM
FHscan FemtosearchBot
Firefox/7.0
Freeuploader Freeuploader
FrontPage FrontPage
FyberSpider FyberSpider
GT::WWW G-i-g-a-b-o-t
GalaxyBot GermCrawler
GetWeb
Getintent Getintent
Gigabot Gigabot
Go!Zilla GoZilla
Go-Ahead-Got-It Gotit
GrabNet Grabber
HEADMasterSEO Grafula
HMView HMView
HTTrack HTMLparser
HTTP::Lite
Haansoft
Harvest Harvest
Havij Hloader
HybridBot Humanlinks
IRLbot IDBTE4M
Image Sucker Id-search
Image Fetch
IndeedBot
Indy Library
InterGET InterGET
InternetSeer Internet Ninja
IstellaBot JamesBOT
JOC Web Spider Jbrofuzz
JetCar JennyBot
Jetty Jetty
JustView Jorgee
Keyword Density
Kozmosbot Kozmosbot
LWP::Simple LeechGet
Lanshanbot LexiBot
Larbin
Leap
LeechFTP
Lftp Lftp
LibWeb
Libwhisker
LinkScan
LinkWalker LinkWalker
Linkbot
Linkdexbot
LinkpadBot LinkpadBot
LinqiaMetadataDownloaderBot LinqiaScrapeBot
LinqiaRSSBot Lipperhey
Lipperhey Spider
Lmspider Lmspider
Ltx71 MIDown tool
MIIxpc
MQQBrowser MQQBrowser
MSFrontPage MSIECrawler
MTRobot MarkWatch
Magnet
Mass Downloader Mass Downloader
Masscan Mata Hari
MegaIndex.ru Mb2345Browser
Microsoft URL Control MicroMessenger
Minefield Microsoft Data Access
Mister PiX
Mojeek Mojeek
Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Mozlila
NICErsPRO Msrabot
Musobot
NPbot NPbot
Name Intelligence NearSite
Nameprotect
Navroad
Needle Needle
NetAnts Nessus
NetZIP NetMechanic
Netcraft
Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler
NimbleCrawler
Ninja Ninja
Not Not
OpenVAS Nuclei
OrangeBot Octopus
PECL::HTTP OnCrawl
PHPCrawl Openfind
OrangeSpider
OutfoxBot
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer
PageGrabber
PageScorer PageScorer
PageThing.com Picscout
Petalbot
Pi-Monster
Picsearch
PictureFinder PictureFinder
Piepmatz Pimonster
PleaseCrawl PleaseCrawl
Pockey
ProPowerBot ProPowerBot
PyCurl PyCurl
QueryN Metasearch QueryN Metasearch
RSSingBot RealDownload
RankActive RepoMonkey
RankActiveLinkBot
Re-re
ReGet
Recorder
RedesScrapy
RocketCrawler
SBIder SBIder
SEOstats SISTRIX
ScanAlert SalesIntelligent
ScoutJet Scrapy
SeoSiteCheckup Screaming
ScrepyBot
Searchestate
Seekport
SemanticJuice
Seomoz
Shodan Shodan
Site Sucker
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteSnagger
SiteSucker SiteSucker
Sitebeam Sitebeam
Siteimprove
Sitevigil Sitevigil
SlySearch
Snake
Snapbot Snapbot
SocialRankIOBot SocialRankIOBot
Spammen Sogou web spider
Spbot Sottopop
Spinn3r SpaceBison
SputnikBot Sqworm
Sqlworm
Steeler
Sucuri
SuperBot
SuperHTTP SuperHTTP
Suzuran Surfbot
Swiftbot SurveyBot
Szukacz T0PHackTeam
Teleport Teleport
Telesphorep
The Intraformant
TheNomad TheNomad
Thumbor
TightTwatBot TightTwatBot
Toata
Toweyabot
Trendiction
True_Robot True_Robot
Turingos Turingos
VCI Turnitin
Vacuum URLy Warning
UnisterBot
Upflow
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler Voltron
VidibleScraper
Virusdie
Voil
WASALive-Bot
WBSearchBot
WEBDAV WEBDAV
WPScan WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0
WeSEE WeSEE
Web Auto Web Sucker
WebFetch WebFetch
WebCopier WebFuck
WebEnhancer
WebGo IS
WebPix WebPix
WebReaper WebWhacker
WebZIP
Webalta Webalta
Webshag
WebsiteQuester WebsiteQuester
Webster Whack
Whacker
Whatweb Whatweb
Widow
WinHTTrack WinHTTrack
WiseGuys Robot
Wonderbot Wonderbot
Woobot Wprecon
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Xenu Xenu
Zade YoudaoBot
Zauba
Zermelo Zermelo
ZmEu ZoominfoBot
ZumBot
adscanner adscanner
arquivo.pt
autoemailspider autoemailspider
cah.io.community
clark-crawler
cognitiveseo cognitiveseo
crawl.sogou.com com.plumanalytics
domainsproject.org crawler.feedback
ips-agent eCatch
isitwp.com evc-batch
facebookscraper
gopher
muhstik-scan muhstik-scan
pcBrowser pcBrowser
polaris version polaris version
ripz ripz
s1z.ru scan.lol
satoristudio.net
seobility seobility
seocompany.store
seoscanners
seostar seostar
siteripz sexsearcher
spyfu sogouspider
tAkeOut sp_auditbot
trendiction.com
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webmeup-crawler webmeup-crawler
x22Mozilla
xpymep1.exe xpymep1.exe
zauba.io zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
best-businessman.ru 12u.info
eropho.net b00kmarks.com
franklinfire.co compliance-ivan.xyz
high-speed1.net erotiktreff24.info
host-protection.com jerseywholesalebizchina.com
ladov.ru nvssf.com
piccshare.com ororodnik.goodbb.ru
slomm.ru profolan.pl
vip-file.com simply.net
yubikk.info xlovecam.com

932
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0daymusic.org 178evakuator178.ru
101flag.ru
123cha.com
123movies.love
12bet.com
1hwy.com 1hwy.com
21h2o.com 51unlim.ru
2ads.co.uk 5forex.ru
5u.com 7fon.ru
66cpwgln.space aa8b68101d388c446389283820863176e7.com
8lufu.com aaronabel.com
adcash.com abctoppictures.net
addtoadd.men absolute-s.ru
ahhjf.com acunetix-referrer.com
alert-fdm.xyz adman.gr
algerianembassy.co.in adnotbad.com
alienwheels.de adsref.men
allbizne.co.ua adultnet.in
adultwebhosting.info
advokateg.ru
aero2.ru
aerogo.com
agreda.pluto.ro
airmaxshoes-2016.com
alert.scansafe.net
allboard.xobor.de allboard.xobor.de
alldubai.biz allnews24.in
allknow.info alpharma.net
allstatesugarbowl.org ameblo.top
amehdaily.com andpolice.com
amigobulls.com
android-systems.ru android-systems.ru
animal-drawings.com anime.dougasouko.com
animenime.ru anticrawler.org
aosexkontakte.net aosheng-tech.com
apartamentwroclaw.eu apiallgeniusinfo-a.akamaihd.net
apartmentratings.com appartement-stumm.at
apccargo.com apxeo.info
apiholdingmypage-a.akamaihd.net arabgirls.adultgalls.com
apioutoboxnet-a.akamaihd.net artavenuegardenstudios.com
apishops.ru asscenihotosidea.blogspot.com
aquarium-pro.ru ausergrubhof.info
arcadepages.com autoua.top
artpicso.com
asacopaco.tk
asrvvv-a.akamaihd.net
auto-zapchasti.org
autoloans.com
autotrends.today
availit.weebly.com availit.weebly.com
ayanaspa.com babs.com.ua
azbukadiets.ru baditri.com
azinoofficial777.ru baersaratov.ru
backgroundpictures.net balois.worldbreak.com
bambi.ck.ua banksrf.ru
barbourjackets.online
bashtime.ru
bayareaaeroclub.org
bdsmgalls.net
beachdriveblog.com
beachfix.co
bear.gotcher.us
becuo.com
bedcapdealers.com bedcapdealers.com
belinvestdom.by bestadbid.com
benchmarkcommunications.co.uk bestmarriages.com
bestpornuha.com betteroffers.review
bimatoprost-careprost.com bg6s0.com
birzha-truda.eu bharatdefencekavach.com
biznesschat.net billiard-classic.com.ua
bjorkbacken.nu binomo.com
blogig.org bio-japan.net
blogporn.in bitcoin-ua.top
bonanza-fortune.men bitcoins-live.ru
bonzybuddy.com bitnote.co
blogs.rediff.com
bocoarchives.org
boobsimge.com boobsimge.com
bracketsmackdown.com boom-celebs.com
bradleylive.xyz bouda.kvalitne.cz
brus-vsem.ru brateg.xyz
businesxxl.com buckspar.gq
buy-cheap-pills-order-online.com burger-imperia.com
buy-loft.ru business-suggested.tk
buyparajumpers.online buttons-for-your-website.com
calcularpagerank.com buyclomidonlaine.com
candelluxsklep.pl call-of-duty.info
cartierlove2u.com camsex.xxx-cam.webcam
ccbill.com candycrushshop.com
checkhit.com cbb1smartlist12.click
chiptuninger.com cheapcigarettesc.info
cheapmarmot.online
chinajerseyswholesalecoupons.com
chocolateslim-en-espana.com
chocolateslim-en-france.com chocolateslim-en-france.com
cigarpass.com chocolateslim-original.com
cinemaflix.website christianlouboutinsaleoutletonline.us
civilwartheater.com cialisovercounteratwalmartusa.com
clickbanksites.info claytransformations.info
clicksotrk.com clean-start.net
coderstate.com clean-virus-mac.com
colehaanoutlet.store cleanersoft.com
collegerentals.com clicksor.com
com-cleaner.systems clips.ua.ac.be
companies-catalog.com club-musics.ru
compliance-jack.top cobaltpro.ru
compliance-john.top colobit.biz
compliance-walter.top com-onlinesupport.host
compucelunlock.net com-supportcenter.website
computernetworksonline.com compliance-don.top
contentlook.co compliance-ivan.xyz
cosmetic.donna7753191.ru construmac.com.mx
cowblog.fr cookie-law-enforcement-aa.xyz
creativehutindia.com cookielawblog.wordpress.com
crirussian.ru coolgamechannel.com
czat.wp.pl cosmeticswomens-womensports.rhcloud.com
counter.bloke.com
crazyboost.pro
credit-respect.ru
cryptoswap.biz
danceuniverse.ru
darodar.com darodar.com
datsun-do.com deda-moroza-zakaz.ru
decorazilla.com detoxmed24.ru
dengi-pod-zalog-nedvizhimosti.ru diegolopezcastan.com
detmebel.su dikqlyremy.info
detskie-konstruktory.ru dildofotzen.net
discountbarbour.online
discountliv.com discountliv.com
divisioncore.com diusyawiga.tk
documentsite.net dmmspy.com
dogbreedspicture.net dojki365.online
dogoimage.com domain2008.com
dolohen.com dominateforex.ml
domain-submit.info domination.ml
dood.live donna7753191.ru
dosug-lux.ru downloadmefiranaratb1972.xpg.com.br
e-buyeasy.com dpihatinh.gov.vn
e-kwiaciarz.pl dprtb.com
easyukraine.com dptaughtme.com
educatemetv.com draniki.org
eets.net drev.biz
ekobata.ru e-collantes.com
el-nation.com easycommerce.cf
electronicadirect.com editmedios.com
elitedollars.com ee77ee.com
elitepcgames.com election.interferencer.ru
enskedesquashclub.se electrik-avenue.com
escort-russian.com elektrischezigarette2.devhub.com
etenininrade.ga elektrischezigaretten1.blogse.nl
eurosamodelki.ru energydiet-info.ru
exonline.info esnm.ru
eupornstar.info
exoclick.com
exportshop.us
extads.net
extremal-blog.com extremal-blog.com
ezigarettekaufen.myblog.de ezigarettekaufen2.mpbloggar.se
ezigarettenkaufen2.dreamwidth.org ezigarettenkaufen2.dreamwidth.org
f-online.de ezofest.sk
f07.de f1nder.org
fashionavenuegame.com faptitans.com
fast-wordpress-start.com fashionindeed.ml
fatfasts-4tmz.com favornews.com
fdzone.org ferieboligkbh.dk
filkhbr.com fickenbumsen.net
financetip.eu finansov.info
findsexguide.com find1friend.com
firstdrugmall.ru firesub.pl
flprog.com flowwwers.com
flytourisme.org foodcrafts.website
foodgid.net forexgb.ru
fortevidyoze.net forpostlock.ru
forum.doctissimo.fr forum-engineering.ru
forum69.info foto-basa.com
forumprofi.de foxweber.com
foxinsocks.ru freeseedsonline.com
fr-bearings.ru fuck-paid-share-buttons.xyz
franklinfire.co funnel.co.za
freecamdollars.com funnymama.com
freelifetimefuckbook.com fz139.ttk.ru
freemags.cc galleryawesome.com
freshsuperbloop.com gamedayhouse.com
g33.org gamewrath.com
galeria-zdjec.com gateway.zscalerone.net
gallerily.com gembird.com
game-top.su generalporn.org
gameplexcity.com generousdeal-a.akamaihd.net
gay.adultgalls.com get-seo-domain.com
ge0ip.com getdot.ru
gentamicineyedrops.blogspot.com getmyads24.com
getmiro.com getprismatic.com
ghernnqr.skyrock.com gonextmedia.com
gigapeta.com googlsucks.com
gilsonchiro.xyz gosarhivrt.ru
girlgamerdaily.com gourcy.altervista.org
gogalleryawesome.com gratuitbaise.com
goldadpremium.com guardlink.org
golden-catalog.pro guod.me
goldpanningtools.com healbio.ru
googleseo.com.tr hentaiheroes.com
gopixdatabase.com hitmuzik.ru
gorabagrata.ru hjaoopoa.top
gox.com.ua homedecoguide.info
greatfind-a.akamaihd.net hothot.ru
groupmoney.ru howmuchdoestizanidinecost.blogspot.com
gsmlab.pl hspline.com
guarrasdelporno.xxx hully.altervista.org
guitar-master.org hyipmanager.in
gunsvicceadadebt.tk i4track.net
handicapbathtubarea.com iboss.com
happy.new.yeartwit.com igithab.com
hcate.com ikritikimou.gr
helicalpile.us ilmen.net
heliko.no ilovevitaly.xyz
hidemyass.com imagui.eu
hikesearch.net imgarcade.com
hitcpm.com imk.com.ua
hosting-tracker.com immigrational.info
hotel-mkad.ru inet-traffic.com
ictizanidinehcl4mg.blogspot.com infokonkurs.ru
id-forex.com interesnie-faktu.ru
ilovevitaly.ru interior-stickers.ru
images-free.net invitefashion.com
infodocsportal.com iron-age.info
infoupdate.org
intermesh.net
irkutsk.online-podarki.com
iscblog.info iscblog.info
islandminingsupply.wordpress.com
isotoner.com isotoner.com
isoveti.ru it-max.com.ua
istripper.com itis4you.com
iyasimasennka.com itronics.ca
jasonpartington.com javlibrary.cc
javcoast.com javrip.net
kamorel.com jetsli.de
kanimage.com jillepille.com
karaganda.xkaz.org joy-penguin.com
kevblog.top kaac.ru
kam-dom.ru
karachev-city.ru
kaz.kz
keywordglobal.co.za
keywords-monitoring-success.com keywords-monitoring-success.com
keywordsking.com kino-rating.ru
kino-fun.ru kinobaks.com
kinosed.net kinohit1.ru
kinotorka.ru klumba55.ru
kladrus.ru komp-pomosch.ru
klikbonus.com konica.kz
klosetkitten.com koptims.tiu.ru
koversite.info
kovrenok.ru
kreativperlen.ch
kvartira-sutochno.com
lakomka.com.ua
lampokrat.ws lampokrat.ws
landscapebackgrounds.blogspot.com laptoper.net
larose.jb2c.me larose.jb2c.me
law-check-seven.xyz lenpipet.ru
leboard.ru levaquin750mg.blogspot.com
lesbian.xyz lflash.ru
lider-zhaluzi.kiev.ua linkbux.ru
lifeinsurancekb.com liveshoppersmac.com
lincolntheatre.com lkbennettoutlet.online
liveinternet.ro lkbennettstore.online
lovasszovetseg.hu llastbuy.ru
mactechinfo.info lmrauction.com
magazin-pics.ru loadingpages.me
magicdiet.gq lob.com.ru
maju.bluesalt.co lol-smurfs.com
matsdale.com lonfon.xyz
mazda-roadsters.com lumb.co
magento-crew.net
magiadinamibia.blogspot.com
maglid.ru
maladot.com
marsgatan.com
maxthon.com
medmajor.ru
medtherapy.ru
mega-bony-2017.pl mega-bony-2017.pl
mere.host.sk member-quiz.com
mhi-systems.ru merryhouse.co.uk
mikrobiologies.ru mex-annushka.ru
moivestiy.biz middlerush-a.akamaihd.net
mil-stak.com
millionare.com
minet.club
missis.top
mobplayer.net
moesonce.com
mokrayakiska.com
mole.pluto.ro mole.pluto.ro
monarchfind-a.akamaihd.net monclercheap.xyz
monclerjacketsoutlet.pw monclerjacketsoutlet.win
moneymaster.ru moskva.nodup.ru
moneytop.ru moviemail-online.co.uk
morocco-nomad-excursions.com
moz.com
mriyadh.com
mrvideospornogratis.xxx
mrwhite.biz mrwhite.biz
multgo.ru mug-na-chas-moscow.ru
mundoaberrante.com
muz-baza.net muz-baza.net
muz-tracker.net muz-tracker.net
my-floor.in.ua mwtpludn.review
myfreecams.com
myftpupload.com
mylesosibirsk.ru
mylida.org mylida.org
narosty.com mzdish.site
nash-krym.info na-telefon.biz
netcheckcdn.xyz netoil.no
networkad.net newhairstylesformen2014.com
njpalletremoval.com newyorkhotelsmotels.info
nmrk.ru nextlnk12.com
no-rx.info ngps1.ru
nootrino.com nhl17coinsforps3.gratisblog.biz
noumeda.com nodup.ru
novgorod.xrus.org norththeface.store
nvssf.com notebook-pro.ru
nw-servis.ru november-lax.com
o-dachnik.ru nullrefer.com
ochistka-stokov.ru obiavo.in
offer.wpsecurity.website obnal.org
okroshki.ru obsessionphrases.com
onstrapon.purplesphere.in oda.as
odywpjtw.bloger.index.hr
offer.camp
onetravelguides.com
openmultipleurl.com openmultipleurl.com
optibuymac.com orenburg-gsm.ru
orgasmus-virtual.com osagonline.ru
ostroike.org osb.se11.ru
otbelivanie-zubov.com otbelivanie-zubov.com
ozoz.it p-business.ru
paclitor.com paidonlinesites.com
panchro.co.uk painting-planet.com
palocco.it
panchro.xyz
paramountmarble.co.uk paramountmarble.co.uk
pathwhelp.org perfection-pleasure.ru
pcimforum.com photokitchendesign.com
pdns.download pictures-and-images.net
pensplan.com pinwallpaper.top
perfectpracticeweb.com pl.id-forex.com
phormchina.com plastweb.ru
photosaga.info play.leadzupc.com
picphotos.net playfortuna-play.ru
pihl.se pliks.pl
pillscheap24h.com popugauka.ru
pinkduck.ga pornoinn.com
pizda.lol poshiv-chehol.ru
pizdopletka.club predmety.in.ua
pk-services.ru privatefx.all4invest.info
platezhka.net prizesbook.online
play-mp3.com pro-tec.kz
poffet.net produkto.net
poiskzakona.ru projefrio.com.br
pojdelo.weebly.com pron.pro
pokemon-go-play.online
pooleroadmedicalcentre.co.uk
popads.net
popup-jdh.xyz
porn-w.org
porno-dojki.net
porno-video-chati.ru
pornohub.me
pornoplen.com
pornzone.tv
postclass.com
powitania.pl
preg.marketingvici.com
priceg.com
princeadvantagesales.com
privacyassistant.net
privatefx-in.ru
professionaldieselcare.com
prostitutki-belgoroda.org
prostitutki-kharkova.org prostitutki-kharkova.org
radiodigital.co q-moto.ru
rankia.com qpypcx.com
ranking2017.ga ragecash.com
rankingchart.de rainbowice.ru
rarbg.to rangoman.date
rentehno.ru rapidokbrain.com
repeatlogo.co.uk rcb101.ru
replicaclub.ru rcpmda.ikan1080.xyz
rewardpoll.com redbottomheels.xyz
rfserial.net redirect.trafficreceiver.club
ritlweb.com relatodelpresente.com.ar
replica-watch.ru
revistaindustria.com
robot-forex.biz robot-forex.biz
roll123.com roll123.com
rumamba.com rome2rio.com
rusenvironmental.net royalcar-ufa.ru
russia-today-video.ru rucrypt.com
russian-postindex.ru rusexy.xyz
sa-live.com sale-japan.com
saddiechoua.com sankt-peterburg.nodup.ru
salutmontreal.com scanner-donald.top
samolet.fr score-ads.men
sanidumps.com sdsjweb.com
savetubevideo.com seansonline24.pl
scanhub.ru search-goo.com
scanner-alex.top search.alot.com
scanner-fred.top searchengineranker.email
scanner-josh.top sell-fb-group-here.com
semxiu.com seocheckupx.com
seo-prof1.xyz seocheki.net
seo18.su seofied.com
seojokes.net seofirmreviewsus.info
seokicks.de seoholding.com
sex-pr.net sex-dating.co
sexblog.pw sexflirtbook.com
share-buttons-for-free.com sexy.babes.frontend-stack.top
sharpchallenge.com shareyards.com
shikiso.info shell-pmr.ru
shopfishing.com.ua shinikiev.com.ua
shtora66.ru shodanhq.com
signal03.ru shop.xz618.com
simul.co sibecoprom.ru
sirpornogratis.xxx sientalyric.co
site.ru silktide.com
siteonomy.com sinestesia.host.sk
sitopreferito.it skuteczna-dieta.co.pl
slavic-magic.ru skypasss.com
smartbalanceworld.com skyway24.ru
social-search.me sladkoevideo.com
socialmonkee.com small-game.com
songplanet.ru soblaznu.net
sophang8.com spammen.de
splendorsearch-a.akamaihd.net specialfinanceoffers.com
stackthatbucks.com sptslmtrafms.com
stard.shop spy-app.info
stillmiracle.com srdrvp.com
stmassage.ru ssconstruction.co
stop-gepatit.te.ua stairliftsarea.com
superkanpo.com strana-solnca.ru
strangeduckfilms.com
stroymonolit.su
subj.ukr-lit.com
supermodni.com.ua
supernew.org supernew.org
svarbit.com svtrd.com
svetodiodoff.ru swingerseiten.com
svolze.com tackletarts.co
swinginwithme.ru taxi-v-eisk.ru
symbaloo.com technopellet.gr
syvertsen-da.no tecspb.ru
t-bygg.com templates.radiodigital.co
tampabaywatch.org thepornsex.org
tech4master.com thewomenlife.com
teenbbw.yopoint.in timecrimea.ru
teenforporn.com timhost.ru
teenfuck.tv titangel-vietnam.com
the-trader.net tizanidinehcl2mgtablet.blogspot.com
theautoprofit.ml tizanidineorflexeril.blogspot.com
thedownloadfreeonlinegames.blogspot.com tizanidinepain.blogspot.com
theendivechronicles.com tizanidinesleepaid.blogspot.com
thegalerie.eu token-lab.org
thetattoohut.com toolsky.com
thewebsitetemplate.info
threecolumnblogger.com
tiandeural.ru
ticketsys.inetwd.com
tiens2010.ru
tizanidineinteractions.blogspot.com
tizanidineusedfor.blogspot.com
tizanidinevssoma.blogspot.com
tonerbox.kz
tootoo.to tootoo.to
topkarkas.com top-l2.com
totu.info top10registrycleaners.com
topcar-krasnodar.ru
tovaroboom.vast.ru tovaroboom.vast.ru
track.deriv.com toxicwap.com
tracklead.net tracxn.com
trafficmp.com trafficgenius.xyz
trichizobswiv.agddns.net trafficjunky.com
trion.od.ua trainoffend.ml
tripper.de trustl.life
tvand.ru tsstcorpcddvdwshbbdriverfb.aircus.com
uchebavchehii.ru tula.mdverey.ru
ucsol.ru turist-strani.ru
ufa.xrus.org turkeyreport.tk
ukkala.xyz turn-up-life.life
ukrtvir.com.ua twiclub.in
ultimateclassicrock.com typer.one
ural-buldozer.ru ua-company.ru
url-extractor.xyz unionmarkt.de
urlopener.blogspot.com.au unrealcommander.com
uzungil.com ustion.ru
vektorpress.ru vapomnoncri.tk
vekzdorov.ru vbabule.net
viagra-soft.ru via-energy-order.com
video-woman.com viktoria-center.ru
videonsk.com
videooko.weebly.com
vip-file.com vip-file.com
vipsiterip.org virus-schutzmasken.de
visa-china.ru visa-china.ru
visitcambridge.org viveresaniesnelli.it
vitanail.ru voltrknc1.com
voditeltrezviy.ru vostoktrade.info
vonradio.com vremya.eu
vsexkontakte.net vse-pesni.com
vuclip.com vtcdns.com
vucms.com
vykup-avto-krasnodar.ru vykup-avto-krasnodar.ru
w3data.co vykupavto-krasnodar.ru
webshoppermac.com wardreapptokone.tk
website-speed-check.site watchdogs-2.ru
wejdz-tu.pl waterpurifier.club
wfb.hatedriveapart.com wcb.su
wholesalecheapjerseysfree.com web.cvut.cz
wieseversa.no webalan.ru
winterclassichockeyjerseys.com website-analyzer.info
wladimirpayen.com wedding0venues.tk
womama.ru weprik.ru
woman-tampon.ru westen-v.life
wovis.site westvilletowingservices.co.za
wpthemedetector.co.uk while.cheapwebsitehoster.com
www.ehscloud.cn wnoz.de
www.jbetting.com womens-journal.net
xchangetrak.com woodyguthrie.se
xedserver.com worldis.me
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai worldtraveler.world
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai wormix-cheats.ru
xn----btbdvdh4aafrfciljm6k.xn--p1ai wyniki-lista.pl
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai x-diesel.info
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai x-mix.info
xn--80ak6aa92e.com xitjw.info
xn--e1afanlbnfckd7c3d.xn--p1ai xkaz.org
xn--h1ahbi.com.ua xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
xtraffic.plus xn----9sbubg3ambdfl1j.xn--p1ai
xtremeeagles.net xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xxx-cam.webcam xn--b1adccaf1bzj.xn--p1ai
xxxguitars.com xn--h1aakne2ba.xn--p1ai
yebocasino.co.za xnxx699.com
yellowads.men xxxtube69.com
yottos.com y8games-free.com
youjizz.com yaminecraft.ru
yourads.website youjizz.vc
yuweng.info yubikk.info
za-fun-offer.com zakon-ob-obrazovanii.ru
zaapplesales.blogspot.com zeroredirect1.com
zarajbuilders.com zeroredirect7.com
zazagames.org zigarettenonlinekaufen1.blox.pl
zeroredirect5.com zigarettenonlinekaufen2.bloog.pl
zigarettenonl.canalblog.com zolotoy-lis.ru
zmoda.hostreo.com zoogdiesney.com
zojirushi-products.ru zoogdisany.com
zoolubimets.ru zootoplist.com
zrizvtrnpale.tk
zryydi.com
zscaler.net
zvetki.ru

490
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123movies.love 01apple.com
1adult.com 1-99seo.com
5tey7463.weebly.com 101flag.ru
72-news.com 12masterov.com
7a2rnuey1tw9ar.ru 1kdailyprofit.me
a14download.com 20pascals.nl
aab94f698f36684c5a852a2ef272e031bb.com 3th.co.in
aanapa.ru 5elementov.ru
ad-words.ru 98oi.ru
aktobe.xkaz.org acads.net
akvamaster.dp.ua accessoires-mode.in
alert.scansafe.net ads.gold
allstatesugarbowl.org adsloads.com
animalrank.com aktivator-windows10.blogspot.com
arquivo.pt akuhni.by
as5000.com alessandraleone.com
aussie-prizes.com antons-transporte.de
avramstroy.ru apiportalmorecom-a.akamaihd.net
avtolombard-krasnodar.com apple.com-cleaner.systems
babs.com.ua arenda-avtoprokat-krasnodar.ru
bashtime.ru artdestshop.eu
beauty-clinic.ru atleticpharm.org
bestbuy.ca atmagroup.ru
bestserials.com auto-zapchasti.org
bestwebsiteawards.com avirasecureserver.com
beyan.host.sk ayerbo.xhost.ro
binomo.com baksman.com
birzha-truda.eu bankmib.ru
blacktwhite.com bestpriceninja.com
blavia.00author.com betonka.pro
bluejays-jerseys.us betterhdporn.com
bmw-ark.ru bezlimitko.xyz
board.f00d.de blog.koreadaily.com
boazpower.com bmw.afora.ru
bonvillan.com bouda.kvalitne.cz
brateg.xyz brewdom.ru
brk-rti.ru buehne-fuer-menschenrechte.de
brus-vsem.ru buketeg.xyz
btnativenav.com buydissertation.net
business-made-fun.com buzzzg.men
cartierring.xyz calstaterealty.us
castingbank.ru cardsharp1.ru
cbcseward.com cartierbraceletsreplica.pw
cfcl.co.uk changde.58.com
chatroulette.world cheatcode-lita12.rhcloud.com
cheapbarbour.online choosecuisine.com
cheapmoncler.xyz classiquebijoux.ru
cheapsergiorossi.online clevernt.com
com-secure.download cmd.kz
coolyarddecorations.com com-cleaner.systems
cv.wallhade.co cosmetic.donna7753191.ru
czeshop.info cpajunkies.com
dairyindia.in culad.com
dealighted.com customchocolate.business-for-home.com
deutschehobbyhuren.net customergrowthsystems.com
domini.cat customerpromos-a.akamaihd.net
downloaddy.net cvety24.by
drugstoreforyou.com defenderxtactical.com
ebonyporn.site delayreferat.ru
elektrischezigaretten2.beeplog.com deniven.1bb.ru
dickssportinggoods.com
digitalassetmanagement.site
div.as
dividendo.ru
divku.ru
dnepropetrovsk.mistr-x.org
dogsrun.net
domik-derevne.ru
driving.kiev.ua
eager-nash.188-93-233-196.plesk.page
earncash.com.ua
eas-seo.com
ec-file.info
ecommerce-seo.com
edge.sharethis.com
ekspresihati.info
elektroniksigaraankara.info
energydiet-info.ru energydiet-info.ru
eredijovon.com esopini.com
extremez.net executehosting.com
eyessurgery.ru exmasters.com
fast-wordpress-start.com exoclick.com
favorcosmetics.com extrabot.com
fearcrow.com failingmarriege.blogspot.com
filesclub.net farm26.ru
filmgo.ru fastcrawl.com
flipper.top fbdownloader.com
forexunion.net fickluder69.com
formseo.com filesdatabase.net
fortevidyoze.net finemanteam.com
foto-telok.net flex4launch.ru
frcls.fr forex.osobye.ru
freecamdollars.com free-fbook-traffic.com
freeseedsonline.com freemags.cc
fsalas.com freshdz.com
fxtips.ru freshwallpapers.info
gabeshop.ru gambarkataku.co
galaxy-family.ru game-mmorpg.net
ge0ip.org games.kolossale.ru
getlaid-xxxhookupdirect.com gelezki.com
getoutofdebtfree.org gepezz.info
gg.zzyjxs.com getpopunder.com
gnuetella.com ghernnqr.skyrock.com
googlsucks.com glass-msk.ru
googst2.ru gloverid.site
gsasearchengineranker.site goblacked.com
gsasearchengineranker.top goodprotein.ru
guardlink.com guge.io
hkdiiohi.skyrock.com hackers-crackers.tk
hobbyhuren24.net hcate.com
hotgirlhdwallpaper.com hdapp1008-a.akamaihd.net
housediz.com hoterika.com
hoverboard360.es ifirestarter.ru
howtoclean.club importchinacoach-teach.com
htmlcorner.com indetiske.ya.ru
huhn.altervista.org investzalog.ru
ibb.com.ua iradiology.ru
iboss.com istizanidineanopiate.blogspot.com
ie.57883.net istizanidinelikexanax.blogspot.com
investyb.com
islandminingsupply.wordpress.com
iyasimasennka.com iyasimasennka.com
jav-fetish.com javatex.co.id
jerseywholesalebizchina.com kladrus.ru
jobgirl24.ru krasivoe-hd.net
johnrobertsoninc.com lanadelreyfans.us
justprofit.xyz laparfumotec.com
kabbalah-red-bracelets.com lesbian.xyz
kambasoft.com light.ifmo.ru
kfon.eu locksmith.jp
kimcurlrvsms.com lostaruban.com
koptims.tiu.ru mansiondelrio.co
kovrenok.ru manve.info
lapitec.eu martlinker.com
law-check-seven.xyz marwer.info
layola.biz.tc maslenka.kz
linkdebrideur.xyz medicovi.com
lirunet.ru meduza-consult.ru
longgreen.info mejoresfotos.eu
low-format.ru metalonly.info
ltvperf.com
m4ever.net
matb3aa.com
mazda-roadsters.com
midst.eu
mielec.pl
mikozstop.com mikozstop.com
moje-recenze.cz miniature.io
mnimmigrantrights.net
mole.pluto.ro mole.pluto.ro
moskva.nodup.ru moncleronline.xyz
mossmesi.com morf.snn.gr
moxo.com moyaterapiya.ru
mp3s.club multgo.ru
mydeathspace.com mybackgroundlandscape.blogspot.com
myhealthcare.com myfreecams.com
na15.ru myiptest.com
national-today-winning-winner.club mzdish.site
needtosellmyhousefast.com nakrutka.cc
netpics.org napalm51.nut.cc
new-post.tk naughtyconnect.com
nikitsyringedrivelg.pen.io neobux-bg.info
noumeda.com nmrk.ru
o-o-6-o-o.ru nordvpn.com
o.light.d0t.ru novatech.vn
offers.bycontext.com novgorod.xrus.org
offtopic.biz nsatc.net
olgacvetmet.com nuker.com
omoikiri-japan.ru offer.camp
osb.se11.ru online-x.ru
parajumpersjakkesalgnorge.info online247.ml
pathwhelp.org onlinedomains.ru
pattersonsweb.com outshop.ru
phormchina.com ozoz.it
phrcialiled.com paceform.com
pipki.r.acdnpro.com pandroid.co
pornogad.com paradontozanet.ru
pornosee.info pateaswing.com
praisong.net pcgroup.com.uy
proekt-mos.ru perfectplanned.com
promgirldresses.xyz perm-profnastil.ru
ptr.ruvds.com picsfair.com
quangcaons.com picswe.com
raavidesigns.com pierrehardysale.online
randalljhoward.com pihl.se
rangjued.com pisanieprac.info
registry-clean-up.net ploenjitmedia.azurewebsites.net
remontgruzovik.ru pod-muzyku.club
responsinator.com pornoted.com
ricorsogiustizia.org predominant-invent.tk
rossanasaavedra.net prizestohandle.club
russian-postindex.ru proftests.net
rutor.vip prpops.com
sa-rewards.co.za psvita.ru
sabizonline.com r-control.ru
saecsa.co radiodigital.co
sanjosestartups.com ratemodels.net
sexblog.pw reining.lovasszovetseg.hu
sharpchallenge.com remont-ustanovka-tehniki.ru
shiksabd.com reyel1985.webnode.fr
silktide.com sajatvelemeny.com
simul.co sale-japan.com
skutecznetabletkinaporostwlosow.pl sanyuprojects.com
sloopyjoes.com scanner-elena.top
smailik.org scanner-willy.top
snsdeainavi.info seemoreresultshu-a.akamaihd.net
softxaker.ru senger.atspace.co.uk
spamnuker.com sexyteens.hol.es
spinazdrav.ru shell-pmr.ru
sport-video-obzor.ru sientalyric.co
sps-shop.com slkrm.ru
spywarespy.com solinf.co
stairliftsarea.com sr-rekneskap.no
stairliftstrue.com stairliftstrue.com
strongsignal-a.akamaihd.net styro.ru
suhanpacktech.com telemetryverification.net
sunflowerdrawingpaintings.blogspot.com telesvoboda.ru
tagil.zrus.org tgtclick.com
tecnoteakviareggio.it thegioixekhach.com
telegraf.by thepornsex.org
thegamerznetwork.com thin.me.pn
themestotal.com tizanidinevsvalium.blogspot.com
timhost.ru traffixer.com
tizanidine4mgprice.blogspot.com tramadolandtizanidine.blogspot.com
tizanidinerecreationaluse.blogspot.com twitlinks.com
tizanidineshowupondrugtest.blogspot.com uptime-gamma.net
tizanidinevscyclobenzaprine.blogspot.com vadimkravtcov.ru
tizanidinezanaflex.blogspot.com vapsy.com
topshef.ru veerotech.com
torrents.life vektorpress.ru
totu.us vekzdorov.ru
truebeauty.cc veopornogratis.xxx
trustl.life veselokloun.ru
turbo-suslik.org vgoloveboli.net
ublaze.ru video-chat.love
uhodzalijami.ru vids18.site
urccvfmc.bloger.index.hr villacoloniale.com
vabasa.inwtrade.com vip-dom.in
venerologiya.com vkgaleria.com
vip-file.com vodkoved.ru
visa-pasport.ru voprosotvet24.ru
vykup-avto-krasnodar.ru walkme.com
wallpapersist.com webix.me
warezkeeper.com website-analyzer.info
waysbetter.cn websitevaluebot.com
webmasters.stackexchange.com weburok.com
webpromotion.ae whiteproduct.com
weburlopener.com wholesalenfljerseys.us.com
wesharepics.com wieseversa.no
winwotgold.pl woweb.com.ua
worldtraveler.world ws.ampower.me
wowcasinoonline.ooo x-true.info
wrz0iuebwhp5fg.freeddns.com xmlinde.com
www.jbetting.com xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
x-lime.com xn--d1acah0c.xn--p1ai
x-lime.net youtubologia.it
xfluro.com zhuravlev.info
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai zigarettenonl.canalblog.com
yaaknaa.info
yourserverisdown.com
za-fun-offer.com
zakazvzloma.com
zapiszto.pl
zigarettenonlinekaufen2.bloog.pl
zoolubimets.ru
zrelaya.pw
zs2vm.top
zscalertwo.net

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.180 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2901 ### Version: V4.2022.02.2902
### Updated: Sun Feb 6 10:06:59 UTC 2022 ### Updated: Sun Feb 6 22:01:17 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2901 ### Version: V4.2022.02.2902
### Updated: Sun Feb 6 10:06:59 UTC 2022 ### Updated: Sun Feb 6 22:01:17 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##