vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2695 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-05 10:09:15 +00:00
parent 2fd24e8a6a
commit 1d113c30c3
46 changed files with 70066 additions and 70066 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2694 #### Version: V4.2021.11.2695
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2694 #### Version: V4.2021.11.2695
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2694 #### Version: V4.2021.11.2695
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8622
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2694 #### Version: V4.2021.11.2695
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8626
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8626
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8626
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8626
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 1; 10.1.1.100 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 0; 10.1.1.100 0;
127.0.0.1 0; 127.0.0.1 0;

8626
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 1; 10.1.1.100 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 0; 10.1.1.100 0;
127.0.0.1 0; 127.0.0.1 0;

8626
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 1; 10.1.1.100 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 0; 10.1.1.100 0;
127.0.0.1 0; 127.0.0.1 0;

8626
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2694 2695

8626
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8626
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.77.152.73 0; 20.112.94.21 0;

8626
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 1; 10.1.1.100 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 0; 10.1.1.100 0;
127.0.0.1 0; 127.0.0.1 0;

8626
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 1; 10.1.1.100 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 0; 10.1.1.100 0;
127.0.0.1 0; 127.0.0.1 0;

8626
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 1; 10.1.1.100 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 0; 10.1.1.100 0;
127.0.0.1 0; 127.0.0.1 0;

8626
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.77.152.73 0; 20.112.94.21 0;

8626
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.77.152.73 0; 20.112.94.21 0;

8626
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 1; 10.1.1.100 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BackWeb Alligator
Anarchy
Craftbot
Dragonfly
Drip Drip
HTMLparser LieBaoFast
LinkpadBot Recorder
LinksManager TheNomad
Mister PiX Toweyabot
Musobot internetVista monitor
Niki-bot
OpenVAS
webmeup-crawler

290
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
ADmantX 360Spider
AIBOT
ALittle Client ALittle Client
AllSubmitter Abonti
AlphaBot Aboundex
Anarchie Acunetix
AfD-Verbotsverfahren
Aipbot
Alligator
Anarchy99 Anarchy99
Ankit Ankit
Attach Anthill
AwarioSmartBot Apexoo
BLEXBot Asterias
AwarioRssBot
BDCbot
BackStreet BackStreet
Backlink-Ceck BackWeb
Bandit BacklinkCrawler
Barkrowler Badass
Battleztar Bazinga
BetaBot BetaBot
Black Hole Blow
Bolt BotALot
Brandwatch
Buck Buck
Bullseye Bullseye
BunnySlippers BunnySlippers
BuzzSumo
CATExplorador CATExplorador
CCBot
CSHttp CSHttp
CherryPicker Calculon
CazoodleBot
CheTeam
CheeseBot
Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot Collector
Copier Copier
CopyRightCheck
Copyscape
Cosmos Cosmos
Craftbot Craftbot
CrunchBot Crawling at Home Project
CyotekWebCopy Crescent
DIIbot Curious
DataCha0s DBLBot
DatabaseDriverMysqli
Demon
Deusu
Devil Devil
DigitalPebble Digincore
Discoverybot Discoverybot
Dispatch DnBCrawler-Analytics
DittoSpyder DomainSigmaCrawler
DnyzBot Dotbot
DomainCrawler Dragonfly
DomainStatsBot Ebingbong
Domains Project
EMail Siphon
EasyDL
EirGrabber EirGrabber
EroCrawler Evil
Express WebPictures
ExtLinksBot ExtLinksBot
Extractor Extractor
ExtractorPro ExtractorPro
EyeNetIE Ezooms
FDM FDM
FlashGet
Flunky Flunky
Freeuploader Foobot
FrontPage FrontPage
Fuzz Fuzz
FyberSpider GalaxyBot
GT::WWW GermCrawler
Grafula Getintent
GridBot Gigabot
Go!Zilla
Go-Ahead-Got-It
GrabNet
Grabber
HMView
HTTP::Lite HTTP::Lite
HaosouSpider
Harvest
Havij
IDBTE4M
IDBot
IRLbot IRLbot
Id-search IlseBot
Indy Library IndeedBot
IstellaBot InfoNaviRobot
JOC Web Spider JamesBOT
JennyBot
JetCar
Jetty
JikeSpider
Joomla Joomla
Jorgee JustView
Jyxobot Jyxobot
Kinza Kinza
LeechFTP LWP::Simple
LibWeb Larbin
Libwhisker LeechGet
LieBaoFast Lightspeedsystems
Linkdexbot LinkScan
LinkextractorPro LinkextractorPro
LinkpadBot
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey Spider Litemage_walker
Ltx71 Ltx71
MSFrontPage MJ12bot
MSIECrawler MSIECrawler
Mag-Net Magnet
Majestic SEO Mail.RU_Bot
Majestic12 Majestic12
MarkMonitor
Masscan Masscan
Mb2345Browser Mb2345Browser
Mediatoolkitbot MegaIndex.ru
Microsoft Data Access Metauri
Microsoft URL Control
Moblie Safari
Mojeek
Mojolicious
Morfeus Fucking Scanner
Msrabot Msrabot
Name Intelligence NICErsPRO
Nessus Nameprotect
Net Vampire NearSite
NetAnts NetAnts
NetMechanic NetLyzer
NetSpider
NetZIP NetZIP
Netcraft
Nettrack Nettrack
Niki-bot Nmap
Nimbostratus
Nutch Nutch
Offline Navigator Octopus
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS
OrangeSpider
OutclicksBot
PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer Page Analyzer
PageScorer PageScorer
PageThing.com
Pandalytics Pandalytics
Panscient Pavuk
Papa Foto
Petalbot Petalbot
Picsearch Picsearch
PictureFinder
Piepmatz Piepmatz
Pixray Pimonster
PleaseCrawl Pockey
Probethenet Probethenet
Psbot Psbot
Pump Pu_iN
PxBroker Quick-Crawler
QueryN Metasearch
RankActive RankActive
RankingBot2 RankActiveLinkBot
RankFlex
RankurBot RankurBot
Re-re
RebelMouse RebelMouse
Recorder Recorder
RedesScrapy RepoMonkey
Ripper
SEOkicks SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler
SEOstats
SMTBot SMTBot
SalesIntelligent SalesIntelligent
ScanAlert Scanbot
ScoutJet
Screaming Screaming
ScreenerBot ScreenerBot
ScrepyBot
Searchestate
Semrush Semrush
SemrushBot SeobilityBot
SeoSiteCheckup
Shodan Shodan
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteExplorer
SiteLockSpider SiteLockSpider
SiteSucker
Snake Snake
Snoopy Snapbot
SocialRankIOBot SocialRankIOBot
Sottopop Sottopop
SpaceBison SpaceBison
Spammen
SpankBot SpankBot
Spanner Sqlworm
Spinn3r
Sqworm
Steeler
Sucker
Sucuri Sucuri
SuperBot
SuperHTTP SuperHTTP
Szukacz
T8Abot T8Abot
Teleport Teleport
Telesoft
Telesphoreo Telesphoreo
The Intraformant
TheNomad TheNomad
Toweyabot Thumbor
Trendictionbot TightTwatBot
Turingos Toata
TurnitinBot Tracemyfile
TwengaBot TwengaBot
Twice
URLy.Warning URLy.Warning
UnisterBot
Upflow
VB Project VB Project
VelenPublicWebCrawler WBSearchBot
VeriCiteCrawler
VidibleScraper
WEBDAV
WISENutbot
WPScan WPScan
WWW-Collector-E
WWW::Mechanize WWW::Mechanize
WallpapersHD Wallpapers/3.0
WeSEE Web Enhancer
Web Fuck Web Fetch
Web Pix
Web Sucker
WebAuto WebAuto
WebCopier WebCopier
WebFetch WebEnhancer
WebGo IS
WebImageCollector WebImageCollector
WebLeacher WebLeacher
WebPix
WebStripper
WebWhacker WebWhacker
Webalta
Webshag Webshag
Website Quester Website Quester
WebsiteExtractor WebsiteQuester
Whacker
Who.is Bot Who.is Bot
Widow
WiseGuys Robot
Wonderbot Wonderbot
Wotbox
Wprecon Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot Xenu
Zeus Zauba
Zitebot ZoominfoBot
archive.org_bot ZumBot
ZyBorg
arquivo-web-crawler
arquivo.pt arquivo.pt
backlink-check autoemailspider
cah.io.community
check1.exe check1.exe
coccocbot-web coccocbot-web
com.plumanalytics
crawler4j
demandbase-bot demandbase-bot
domainsproject.org domainsproject.org
evc-batch
facebookscraper facebookscraper
gopher
heritrix heritrix
instabid
internetVista monitor
ips-agent ips-agent
isitwp.com
lwp-request lwp-request
lwp-trivial meanpathbot
magpie-crawler mediawords
muhstik-scan muhstik-scan
oBot oBot
pcBrowser page scorer
polaris version scalaj-http
satoristudio.net
seostar seostar
sitechecker.pro sexsearcher
spyfu
sysscan
tAkeOut tAkeOut
trendiction.de trendiction.com
webpros.com ubermetrics-technologies.com
x09Mozilla
xpymep1.exe
zauba.io
zgrab

312
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404checker
80legs 80legs
AIBOT ALittle Client
ASPSeek
Abonti
Aboundexbot Aboundexbot
AhrefsBot
Alexibot Alexibot
AlphaBot
Anarchy
Anarchy99 Anarchy99
Apexoo Ankit
Aspiegel Anthill
Asterias AwarioRssBot
Attach
BBBike BBBike
BDCbot BDCbot
BDFetch BLEXBot
BackDoorBot BackDoorBot
BackStreet BackStreet
BackWeb BackWeb
Backlink-Ceck BacklinkCrawler
BatchFTP Badass
Blow BetaBot
Bitacle
Black Hole
Blackboard
BlowFish BlowFish
Boardreader Bolt
Brandprotect Buddy
Buck BuiltBotTough
BuiltWith
BunnySlippers BunnySlippers
BuzzSumo
CATExplorador CATExplorador
CCBot Calculon
CheeseBot Cegbfeieh
ChinaClaw CensysInspect
Cliqzbot Claritybot
Collector Collector
CopyRightCheck Copier
Cosmos Copyscape
Craftbot Craftbot
Crawling at Home Project Crawling at Home Project
CrunchBot Curious
Custo Dirbuster
DBLBot Disco
Devil Dispatch
Discobot DittoSpyder
DomainAppender DnyzBot
DomainCrawler DomCopBot
Dragonfly DomainSigmaCrawler
Drip Dotbot
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
EasyDL EasyDL
Ebingbong Ecxi
Evil EirGrabber
Exabot Exabot
Express WebPictures
Extractor
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
Ezooms
FDM FDM
Firefox/7.0 FHscan
Flunky FemtosearchBot
Fimap
FlashGet
FrontPage
FyberSpider FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot Genieo
GermCrawler
GetRight
GetWeb GetWeb
Getintent
Go-Ahead-Got-It
GoZilla GoZilla
GrabNet GrabNet
Grafula Grabber
GrapeFX
GrapeshotCrawler GrapeshotCrawler
HTMLparser
HTTP::Lite
HTTrack HTTrack
Heritrix Haansoft
IDBTE4M HaosouSpider
Harvest
Havij
IDBot
IRLbot IRLbot
Id-search Id-search
InfoNaviRobot IlseBot
InfoTekies InterGET
Intelliseek Internet Ninja
Iria
Iskanie Iskanie
IstellaBot
JennyBot
Jetty Jetty
JikeSpider JikeSpider
Jorgee
JustView JustView
Kenjin Spider Keyword Density
Kozmosbot LeechGet
LWP::Simple
Lanshanbot
LeechFTP
LexiBot
Lftp
Libwhisker Libwhisker
LieBaoFast
Lightspeedsystems
LinkScan
Linkdexbot
LinkextractorPro LinkextractorPro
LinksManager LinqiaMetadataDownloaderBot
LinqiaRSSBot Lipperhey Spider
LinqiaScrapeBot
Lmspider
Ltx71
MIDown tool MIDown tool
MQQBrowser MIIxpc
MSFrontPage MJ12bot
MSIECrawler
MTRobot
Mag-Net
Magnet Magnet
Majestic-SEO
Majestic12
MarkMonitor MarkMonitor
Masscan MarkWatch
Mata Hari MauiBot
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot Metauri
Microsoft URL Control Microsoft Data Access
Mister PiX Mister PiX
Mojeek Mojolicious
Msrabot
NICErsPRO NICErsPRO
NetAnts Nameprotect
Nettrack NearSite
Netvibes
NextGenSearchBot
Nibbler Nibbler
Niki-bot Niki-bot
NimbleCrawler Nimbostratus
Ninja Openvas
Nutch
Octopus
OpenLinkProfiler
Openfind
OrangeBot
OutclicksBot OutclicksBot
OutfoxBot
PECL::HTTP
PHPCrawl PHPCrawl
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageScorer PageScorer
Panscient Pandalytics
Papa Foto PeoplePal
Pavuk
Pi-Monster
Picsearch
PictureFinder PictureFinder
Pimonster Piepmatz
Pixray
PleaseCrawl
ProPowerBot ProPowerBot
Pump Probethenet
QueryN Metasearch RankActiveLinkBot
Quick-Crawler
RankActive
RankFlex
RankingBot RankingBot
RankingBot2 Re-re
ReGet ReGet
Reaper
RebelMouse
RedesScrapy RedesScrapy
RocketCrawler RepoMonkey
SEOprofiler Rogerbot
SBIder
SEOkicks
SEOkicks-Robot
SEOstats
SISTRIX
SMTBot
SalesIntelligent
ScoutJet
Scrapy Scrapy
ScreenerBot Semrush
Searchestate SemrushBot
SearchmetricsBot
SeoSiteCheckup SeoSiteCheckup
Seomoz Seomoz
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
Sitebeam Sitebeam
SlySearch SocialRankIOBot
Snoopy Sogou web spider
Sosospider
SpaceBison
Spammen
Spbot Spbot
SputnikBot
Sqlmap
Sqlworm Sqlworm
Stripper Stripper
SuperBot SuperBot
SuperHTTP Surfbot
SurveyBot
Swiftbot Swiftbot
Szukacz Teleport
TeleportPro
Telesoft
Telesphoreo Telesphoreo
Telesphorep Telesphorep
Tracemyfile The Intraformant
TurnitinBot TightTwatBot
TwengaBot Toata
Twice Turnitin
URLy Warning
URLy.Warning URLy.Warning
UnisterBot UnisterBot
Upflow VelenPublicWebCrawler
VB Project VeriCiteCrawler
Vagabondo VoidEYE
WASALive-Bot WASALive-Bot
WBSearchBot
WWW-Collector-E WWW-Collector-E
WWW::Mechanize WWW-Mechanize
WWWOFFLE
Wallpapers
Wallpapers/3.0
Web Auto Web Auto
Web Collage Web Sauger
Web Fuck Web Sucker
Web Pix
WebAuto
WebBandit WebBandit
WebCollage WebEnhancer
WebFetch
WebFuck WebFuck
WebGo IS WebImageCollector
WebLeacher
WebReaper WebReaper
WebSauger WebSauger
WebStripper
WebSucker WebSucker
Webshag WebWhacker
Webalta
Website Quester
WebsiteExtractor WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster
Whack
Whacker
Whatweb Whatweb
Widow
WinHTTrack
WiseGuys Robot
Wonderbot Wonderbot
Woobot
Wotbox Wotbox
Zade YoudaoBot
Zauba Zermelo
Zeus ZmEu
ZoomBot ZoomBot
ZoominfoBot
ZyBorg
autoemailspider autoemailspider
check1.exe cognitiveseo
clark-crawler
coccocbot-web
crawl.sogou.com
crawler4j
dataforseo.com
eCatch eCatch
evc-batch gopher
isitwp.com instabid
lwp-trivial lwp-request
meanpathbot magpie-crawler
muhstik-scan
oBot oBot
page scorer
pcBrowser pcBrowser
ripz
s1z.ru s1z.ru
satoristudio.net satoristudio.net
seobility scalaj-http
seostar scan.lol
sexsearcher sexsearcher
sogouspider sitechecker.pro
spyfu sp_auditbot
sysscan trendiction.com
tAkeOut trendiction.de
ubermetrics-technologies.com
webmeup-crawler webmeup-crawler
webpros.com
webprosbot webprosbot
xpymep1.exe zgrab
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
aad1f4acb0a373420d9b0c4202d38d94fa.com animalia-life.club
cdncash.org compliance-margo.top
cgi2.nintendo.co.jp extlinks.com
f012.de fun2cell.net
handicapbathtubarea.com hkdiiohi.skyrock.com
kandidos.com jillepille.com
live-sexchat.ru smartpet.ru
shemalegalls.blogporn.in udsgame.online
slimcdn.com urzedowski.eu
u-cheats.ru varbergsvind.se

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
101lesbian.xyz 007angels.com
1bet.com 0c47f8422d3f.com
2020iscoming.info 12u.info
256bit.by 1webmaster.ml
70casino.online 214jaluwobapef.cf
7a2rnuey1tw9ar.ru 21h2o.com
7makemoneyonline.com 24videos.tv
aa81bf391151884adfa3dd677e41f94be1.com 2drittel.de
aa9bd78f328a6a41279d0fad0a88df1901.com 2nt.ru
abcdefh.xyz 2pxg8bcf.top
academia-nsk.org 3xforum.ro
acortarurl.es 40cg.com
adclickthru.net 55wmz.ru
adminshop.com 5ws.dating-app.ru
ads.gold 6hopping.com
advokateg.ru 8gold.com
alert-fjg.xyz 991mostfm.co.id
allkrim.com addray.pro
alphahoverboards.com adserver-e7.com
alpharma.net adtech.us
ameblo.jp adventureparkcostarica.com
anatomiy.com advocatemsk.ru
angkortours.vn ahhjf.com
animal-drawings.com ahrefs.com
apartamentwroclaw.eu aimayangzhi.com
apiitsthirteende-a.akamaihd.net alf-img.com
altamayoztourism.com
alvtank.se
amateurmatch.com
amospalla.es
anaksma.info
anal-acrobats.hol.es
analnoeporno.tv
ananumous.ru
androids-store.com
angel1777.kz
animali.deagostinipassion.it
animalrank.com
aosheng-tech.com
apiwebwebgetcom-a.akamaihd.net apiwebwebgetcom-a.akamaihd.net
appearance-cool.com appartement-stumm.at
appfixing.space
apple.com-cleaner.systems
apple.com-webbrowsing-security.science apple.com-webbrowsing-security.science
applyneedy.xyz
arabseyes.com arabseyes.com
arclk.net arpelsreplica.xyz
atmovs.com arquivo.pt
auto-moto-elektronika.cz atofilms.com
autobrennero.it auto4style.ru
autobudpostach.club autodo.info
aversis.net autorn.ru
avkzarabotok.info autoua.top
ayakino.net aviapanda.ru
ayerbo.xhost.ro avtointeres.ru
azazu.ru azartmsl.com
azinoofficial777.ru azazaporn.com
b00kmarks.com babieca.com
b2b-lounge.com balook.com
balla-boo.se barnfurnituremart.com
bamo.xsl.pt bayareaaeroclub.org
bdsmgalls.net bbsoldes.fr
beauty-things.com beatifullandscape.co
bensbargains.net beauty-b0x.pl
bestofupload.info beclean-nn.ru
bet-prognoz.com bestimagecoollibrary.com
bigames.online bestssaker.com
bigpenisguide.com bezcmexa.ru
bitcoin-ua.top bgviagrachrx.com
bitcoinpile.com bikini-image.com
biznesrealnost.ru bildsuche.ru
blagovest-med.ru bin-brokers.com
blog.f00kclan.de bingoporno.com
blog.yam.com binomo.kz
blogsfunky672.weebly.com bird1.ru
bmw-ark.ru blackcurranthumidifiers.site
bmw.afora.ru blogrankers.com
book-bets.com blpmovies.com
boom-celebs.com bluesalt.co
brainboosting.club boleznikogi.com
brainsandeggs.blogspot.com bonzbuddy.com
brillianty.info bonzybuddy.com
bum.com.ru bosman.pluto.ro
burger-imperia.com brandedleadgeneration.com
buy-loft.ru breget74.ru
camsex.xxx-cam.webcam bryansk.zrus.org
candlehandmade.com bscodecs.com
cartierjusteunclou.xyz bumskontakte.org
casablancamanor.co.za buttons-for-your-website.com
casite-513049.cloudaccess.net buyk.host.sk
ccjp.eu buynorxx.com
cenoval.ru buzzurl.jp
certifywebsite.win bvlgariring.xyz
chatroulette.world candycrushshop.com
cheapcigarettesc.info capacitacionyaprendizaje.com
cheapmoncler.win cartierbracelet.xyz
cherrypointplace.ca cartierloveringreplica.xyz
choosecuisine.com cbozhe.com
clickprotects.com ccbill.com
clicksor.net cctva.tv
clmforexeu.com cheapestjerseys-wholesale.com
coldfilm.ru cheapestjerseysonwholesale.com
cometorussia.net chimiver.info
compiko.info chinaelitecheapjerseys.com
compliance-barak.xyz citizenclsdriveri7.pen.io
compliance-viktor.top club-musics.ru
connectionstrenth.com coinsspb.com
copenergo.ru compliance-checker-7.info
coprofam.org compliance-irvin.top
corta.co cookielawblog.wordpress.com
crazyprotocol.com courtshipgift.com
cs-passion.pl craftburg.ru
customerpromos-a.akamaihd.net culturevie.info
cyse.tk custodieva.ru
dafatiri.com dawlenie.com
data1.scopich.com dekoration.us
dating-time-now.com
datract.com
decorationspcs.com
dentalpearls.com.au dentalpearls.com.au
destinationrealestate.com devochki.top
dfwu1019.info digitalassetmanagement.site
diamond-necklace.info disruptingdinnerparties.com
dieta-personalna.pl dnepr-avtospar.com.ua
digest-project.ru doctormakarova.ru
dimkino.ru documentbase.net
discovertreasure-a.akamaihd.net domcran.net
dispo.de doreenblog.online
djstools.com dpihatinh.gov.vn
dnsrsearch.com drillsaw.ru
domainanalyzing.xyz drugstoreforyou.com
domashneeruporno.com dvr.biz.ua
domoysshop.ru ecookna.com.ua
dorratex.tn edge.sharethis.com
dosugrostov.site
doxyporno.com
dptaughtme.com
druzhbany.ru
dwomlink.info
eduserver.net
efnor-ac.com efnor-ac.com
ekaterinburg.xrus.org egovaleo.it
elektrozigarette2.webs.com ekn-art.se
elektrozigaretten2.yn.lt elektrischezigarette2.onsugar.com
elitepcgames.com elektrischezigarettekaufen2.cowblog.fr
eralph.tk elexies.info
elstal.com.pl
englate.com
ereko.ru ereko.ru
evogarage.com etur.ru
exhibitionplus.eu ewebarticle.info
eyessurgery.ru exct.net
f-online.de expdom.com
falcon-images.blogspot.com extlabs.io
femdom.twiclub.in ezofest.sk
fatfasts-4tmz.com
fatmaelgarny.com
favornews.com
ferrotodo.com
figuringmoneyout.com
filesvine.com filesvine.com
fitnesspiks.com findanysex.com
fix-website-errors.com finstroy.net
flauntyoursite.com flashbannernow.com
floridamobilebillboards.com forex21.ru
forex-indextop20.ru forexgb.ru
forodvd.com
forpostlock.ru
foto-basa.com
foto-weinberger.at foto-weinberger.at
free-deals.faith foxinsocks.ru
free-today.com foxydeal.com
freegamesplay.online franklinfire.co
freejabs.com freenom.link
freelifetimefuckbook.com freshmac.space
frequiry.com fuck-paid-share-buttons.xyz
fres-news.com fungamelands.com
freshsuperbloop.com fungirlsgames.net
ftns.ru funnel.co.za
galaxys6manual.info funtoonez.com
games.kolossale.ru fusoradio.info
gaming-journal.com fx-brokers-review.com
gateway.zscalerone.net fym.com.ru
gaygalls.net gagrasector.ru
gazobeton-p.com.ua galaxy-family.ru
gazporno.com galaxyflowers.ru
gearcraft.us gallery.rennlist.com
gentamicineyedrops.blogspot.com ge0ip.net
getresponse.com ge0ip.org
gfaq.ru gearsadspromo.club
ghostvisitor.com gelezki.com
gktt.ru generic-pills-online.com
glasof.es girlgamerdaily.com
globatur.ru goblacked.com
gobongo.info goforexvps.com
good-mummy.ru goodhumor24.com
groupmoney.ru gosmeb.ru
growboxbl.ru greatzip.com
gsasearchengineranker.top grtyi.com
gsasearchenginerankerdiscount.com
gsasearchenginerankerhelp.com
gta-top.ru
guardlink.com
guge.io
guitar-master.org guitar-master.org
hannasolution.ru gyffu.com
happy.new.yeartwit.com hamilton.ca
hidefiles.org hdwallpapers-free.com
hotdl.in helpmymacfaster.trade
hoverboard360.se hentai-manga.porn
howtowhitenteethfast.xyz hexpilot.com
ictizanidinehcl4mg.blogspot.com hit-kino.ru
ilovevitaly.ru hkdiiohi.skyrock.com
imagez.co hmmm.cz
imitex-plus.ru homeart.space
impisr.edunsk.ru homemature.net
incanto.in.ua host-protection.com
indiasourcemart.in hostingclub.lk
informatiecentro.be hostnow.men
hotblognetwork.com
housemilan.ru
ibb.com.ua
ideibiznesa2015.ru
iflycapetown.co.za
igru-xbox.net
iloveitaly.ru
ilovevitaly.com
imagui.eu
infowarcraft.ru
intellego.info
intellekt21.ru intellekt21.ru
intimshop-fantasy.ru isitpaleo.info
jav-fetish.site islamtoday.co.za
ispac.org
itbc.kiev.ua
ivanstroi.ru
iwantmyfreecash.com
izhevsk.xrus.org
javlibrary.cc
jerseychinabizwholesale.com jerseychinabizwholesale.com
joy-penguin.com joyceblog.top
justkillingti.me kam-dom.ru
kamalsinha.com
kamorel.com
kaz.kz
kchaxton.com
keenoutlet.online
kellyonline.xyz kellyonline.xyz
keywordhouse.com keywesthideaways.co
kinotorka.ru keywordblocks.com
kinozapas.com kino-key.info
kmgamex.cf kinobest.pl
kochanelli.com kinomaniatv.pl
klumba55.ru
knigonosha.net
kollekcioner.ru
konturkrasoty.ru konturkrasoty.ru
kotaku.com koopilka.com
kredit-pod-zalog-krasnodar.ru krasnodar-avtolombards.ru
lalalove.ru kupit-adenu.ru
larchik.net kursy-ege.ru
larger.io landinez.co
lasvegaslockandsafe.com lapolis.it
leadwayau.com lifeinsurancekb.com
letolove.ru lincolntheatre.com
letslowbefast.site lion.playtap.us
lietaer.com lionking-1994.blogspot.com
likrot.com liverpool.gsofootball.com
link.ac localmatchbook.com
liumimgx.pw loginduepunti.it
live-sexchat.ru lombardfinder.ru
livejasmin.com lombia.com
loftdigital.eu m-google.xyz
lol-smurfs.com m3gadownload.pl
longadventure.com magda-gadalka.ru
lottospring.com magnetic-bracelets.ru
louboutinreplica.xyz mainlinehobby.net
m292.info managerpak204.weebly.com
makis.nu marinetraffic.com
mall.uk marmotstore.online
manyresultshub-a.akamaihd.net marwer.info
marmitaco.cat masserect.com
master-muznachas.ru masterseek.com
matematikus.info matpre.top
matrixalchemy.com maximilitary.ru
mbiologi.ru mazda-roadsters.com
mere.host.sk meduza-consult.ru
metarip.ru metarip.ru
microsearch.ru middlerush-a.akamaihd.net
microstatic.pl millionare.com
midst.eu minecraft-neo.ru
mirzonru.net mini-modus.ru
misandesign.se minyetki.ru
missclub.info mnogolok.info
mobilierland.com mobile.ok.ru
mobstarr.com modabutik.ru
modenamebel.ru monclervests.xyz
moncleroutletonline.win monsterdivx.tv
monthlywinners.com moscow.xrus.org
msfsaar.de mrinsidesales.com
mtmtv.info mrmoneymustache.com
mttwtrack.com mrpornogratis.xxx
mundoaberrante.com mundoaberrante.com
musicspire.online mycaf.it
musicstock.me myliveblog.ru
muz-shoes.ru namecrumilchlet.tk
mydeathspace.com naphukete.ru
mydownloadengine.com net-radar.com
myftpupload.com netcheckcdn.xyz
mygameplus.com o-o-11-o-o.com
mykings.pw oakridgemo.com
mymobilemoneypages.com obesidadealgarve.com
myshopmatemac.com
myyour.eu
nagdak.ru
naperehresti.info
nastydollars.com
naturalbreakthroughsresearch.com
net-profits.xyz
newstudio.tv
nisuturnetdgu.tk
no-fuel.org
obiavo.by
obiavo.com obiavo.com
obiavo.net oksrv.com
ohmyrings.com one-gear.com
okeinfo.online ooo-olni.ru
omgtnc.com ossmalta.com
online-podarki.com otbelivanie-zubov.com
online247.ml oxford-book.com.ua
online7777.com p-business.ru
onstrapon.purplesphere.in painting-planet.com
orion-v.com paintingplanet.ru
oxotl.com panchro.co.uk
paceform.com paparazzistudios.com.au
paclitor.com parajumpersoutlet.online
panamaforbeginners.com partner-cdn.men
pastaleads.com partner-high.men
pattersonsweb.com partnerline.men
pc-test.net partners-ship.pro
pdns.cz partnersafe.men
pdns.download paydayonlinecom.com
perfectpracticeweb.com pensplan4u.com
perl.dp.ua perso.wanadoo.es
photo.houseofgaga.ru phrcialiled.com
photochki.com picturesfrom.com
photorepair.ru
php-market.ru
pics-group.com
picsforkeywordsuggestion.com
picturesify.com picturesify.com
picurams.pw pillscheap24h.com
pills24h.com pinsdaddy.com
pinwallpaper.top piski.top
pix-hd.com pizdopletka.club
pizdeishn.com playmsn.com
porngalleries.top pokemongooo.ml
popander.mobi
pornoblood.com
pornosemki.info pornosemki.info
pospr.waw.pl privatefx.all4invest.info
potolokelekor.ru
pregnant.guru
preparevideosafesystem4unow.space
pretty-mart.com
priceg.com
priscilarodrigues.com.br
professionalwritingservices15.blogspot.ru professionalwritingservices15.blogspot.ru
projectforte.ru rabotaetvse.ru
pronekut.com radiodigital.co
prospekt-st.ru ranksays.com
prostitutki-novgoroda.org ranksonic.com
pussyspace.net ranksonic.info
qor360.com raschtextil.com.ua
quality-traffic.com recinziireale.com
quickbuck.com redhotfreebies.co.uk
r-control.ru renecaovilla.online
rankscanner.com renecaovillasale.online
rasteniya-vs-zombi.ru research.ifmo.ru
rczhan.com restorator-msk.ru
realizmobi.com resultshub-a.akamaihd.net
recipedays.com risparmiocasa.bz.it
reckonstat.info robertefuller.com
redirlock.com royalcar-ufa.ru
regionshop.biz rrutw.com
registrysweeper.com rubbed.us
replica-watch.ru rvi.biz
replicalouboutin.xyz scanner-jane.top
retreatia.com scanner-julianna.top
rfserial.net scanspyware.net
rossanasaavedra.net school-diplomat.ru
ruex.org.ua scottbywater.com
runtnc.net searchimage.co
ruscoininvest.company security60-e.com
russkoe-zdorovie.ru seksvideoonlain.com
rusvideos.su seminarygeorgia59.ga
rvzr-a.akamaihd.net seomarketings.online
sa-rewards.co.za seozoom.it
sadaholding.com serpstat.com
saitevpatorii.com
savefrom.com
scalerite.co.za
scanner-george.top
scanner-mark.top
scanner-viktor.top
schalke04fc.info
scrnet.biz.ua
search.xtconnect.com
semp.net
seo18.su
service.adtech.us service.adtech.us
sextracker.com sex-foto.pw
sexyteens.hol.es sexkvartal.com
sfj-ror.no sexs-foto.com
shijian.ac.cn sharpchallenge.com
shitting.pro shopfishing.com.ua
shop-electron.ru shoppytoolmac.com
shopsellcardsdumps.com si-unique.com
shopwme.ru sicfor.bcu.cc
sibvitr.ru sientalyric.co
signx.info silverage.ru
site-analyzer.com sim-service.net
siteimprove.com sitiz.club
sitevalued.com skatestick.bid
slkrm.ru slowmac.tech
slutloadlive.com smartbalanceworld.com
smadihome.com smsactivator.ru
smichovbike.cz snjack.info
social-button.xyz snomer1.ru
social-search.me snsdeainavi.info
socialmadesimple.com soblaznu.net
solnplast.ru soft1.ru
sovetogorod.ru solitaire-game.ru
spbchampionat.ru spbchampionat.ru
spidtest.space srgwebmail.nl
spywarespy.com
ssconstruction.co ssconstruction.co
stair.registrydomainservices.com
stal-rulon.ru
stathat.com
statoutlook.info
steebook.com
sterva.cc
sticken.co
stickers-market.ru
stop-gepatit.te.ua stop-gepatit.te.ua
stroydetali.ru storehouse.ua
studworks.org strangeduckfilms.com
success-seo.com stroilka.info
superinterstitial.com styro.ru
superiends.org
supernew.org
superstats.com
support.nopeas.sk support.nopeas.sk
tattomedia.com svolze.com
tengohydar.tk swingerseiten.com
teslathemes.com tandvardshuset.net
tetracsaudi.com tasteidea.com
thedownloadfreeonlinegames.blogspot.com tattooreligion.ru
thfox.com technika-remont.ru
tilido.com teguh.info
timhost.ru templates.radiodigital.co
tizanidine4mgstreetprice.blogspot.com terraclicks.com
tizanidine4mgstreetvalue.blogspot.com thaisamkok.com
tizanidine4mgtablets.blogspot.com thebluffs.com
tizanidinestreetprice.blogspot.com theprofitsmaker.net
tizanidinestreetvalue.blogspot.com timdreby.com
tizanidinewithdrawal.blogspot.com titslove.yopoint.in
tizanidinezanaflex.blogspot.com tizanidine6mg.blogspot.com
tmm-kurs.ru tizanidinehcl2mgtablet.blogspot.com
toonfamilies.net tizanidinehcl4mgtabinfo.blogspot.com
top-karkas.ru tizanidinehcl4mgtablet.blogspot.com
top-l2.com tizanidinenarcotic.blogspot.com
top10-online-games.com tizanidinepain.blogspot.com
torrentred.games tizanidinepills.blogspot.com
torrents.cd tizanidinesideeffectsweightloss.blogspot.com
tovaroboom.vast.ru tizanidinesleepaid.blogspot.com
tkanorganizma.ru
topflownews.com
topquality.cf
torrent-to-magnet.com
tourismvictoria.com
toys.erolove.in
tracksz.co tracksz.co
trackzapper.com trafficjunky.net
trafficjunky.com trahic.ru
truck-rece.life tsan.net
truemfilelj.gq tube8.com
ttrraacckkrr.com tuberkuleznik.ru
turbo-suslik.org tvoystartup.ru
tversvet.ru twojebook.pl
twelvevisionspartyofcolorado.com tx41tclega.ru
typer.one uhdtv.website
u-cheats.ru uk-zheu20.ru
ukrtextbook.com ukrtvory.in.ua
ultimatesetnewfreeallsoftupgradesystems.pw uptimechecker.com
ultramart.biz
upupa.net upupa.net
urll.eu ural-buldozer.ru
url-extractor.xyz
urlopener.com
uroffer.link uroffer.link
ustion.ru usiad.net
vadimkravtcov.ru validdomain.xyz
vancleefreplica.pw
varbergsvind.se varbergsvind.se
vduplo.ru vduplo.ru
vedomstvo.net vekzdorov.ru
vgoloveboli.net vereo.eu
via-energy-cumpara.com verymes.xyz
via-energy-order.com videochat.life
viberdownload10.com vielporno.net
videochat.cafe vigrx-original.ru
videogamesecrets.com vipps.com.my
videos-for-your-business.com virus-respirators.com
viktoria-center.ru
vip2ch.com
visa-pasport.ru
viteonlusarezzo.it viteonlusarezzo.it
vitoriacabos.com
vkonche.com vkonche.com
vkontarkte.com vkontaktemusic.ru
vladimir.zrus.org voditeltrezviy.ru
vodkoved.ru vse-pesni.com
vote-up.ru vsexkontakte.net
wallpaperaccess.com vvon.co.uk
waterefficiency.co wanker.us
watracker.net warezaccess.com
waysbetter.cn web.cvut.cz
webix.biz webpromotion.ae
webmonetizer.net wfb.hatedriveapart.com
website-stealer.nufaq.com wholesalejerseysgaa.com
wedding-salon.net worldlovers.ru
wejdz-tu.pl
wemarketing.se
womensterritory.ru
workle.website
wovis.site
wrona.it
wurr.voila.net
www.arenda-yeisk.ru www.arenda-yeisk.ru
www.labves.ru x69ty.ru
xmlinde.com xkaz.org
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai xn----ctbigni3aj4h.xn--p1ai
xn----7sbifcamovvfggw9d.xn--p1ai xn----itbkqkfiq.xn--p1ai
xn----dtbndd4ae7eub.top xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--90acenikpebbdd4f6d.xn--p1ai xn--80aeahghtf8ac5i.xn--p1ai
xxxdatinglocal.us xuki.us
xxxsiterips.xyz yaderenergy.ru
yaminecraft.ru yebocasino.co.za
you-shall-not-pass.is74.ru yougame.biz
youghbould.wordpress.com yourads.website
yourdesires.ru yuarra.pluto.ro
yoursearch.me yur-p.ru
z-master.ru zaimhelp.ru
zaapplesales.blogspot.com zaloro.com
zagadki.in.ua zapiszto.pl
zeroredirect2.com zarabotok--doma.ru
zigarettenonlinekaufen.tumblr.com zarenica.net
zocaparj.kz zeroredirect12.com
zooggames.com zeroredirect5.com
zoolubimets.ru zeroredirect6.com
zophim.me zeroredirect8.com
zscaler.net zeroredirect9.com
zskdla.site zog.link
zrus.org
zverokruh-shop.cz

492
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
0lovespells0.blogspot.com 08800.top
101flag.ru 1pamm.ru
3w24.com 2daytrendingnews.com
4webmasters.org 51unlim.ru
8xv8.com aa14ab57a3339c4064bd9ae6fad7495b5f.com
abiente.ru aasoldes.fr
adbetclickin.pink adclickservice.com
addblueoff.com.ua adelly.bg
adidas.frwebs.fr aero2.ru
adimmix.com all4wap.ru
adsref.men alltheviews.com
aerogo.com anonymous-redirect.com
agapovdg.ru apccargo.com
aimayangzhi.com apimegabrowsebiz-a.akamaihd.net
akama.com appsaurus.com
albumsuper.info
allfinweb.com
allsilver925.co.il
alyeskaresort.com
android4fun.org
arenanews.com.ua arenanews.com.ua
asacopaco.tk asseenontv.ru
asociatia-tipografilor-transilvania.ro atlasvkusov.ru
asrvrep-a.akamaihd.net autobrennero.it
astana.xxxkaz.org autolombard-krasnodar.ru
atout-energie-69.com avek.ru
atovh.local-finders.com aviav.eu
audiobangout.com
azartmsl.com azartmsl.com
bag77.ru backiee.com
bagsonsale.online balkanfarma.ru
baikaleminer.com bestmarriages.com
bard-real.com.ua bestwebsitesawards.com
battle.net betune.onlinewebshop.net
beget.tech bidr.trellian.com
bestadbid.com bizru.info
biocn.dx.am blog-hits.com
blog.yam.com bocoarchives.org
blogs.rediff.com bradleylive.xyz
bonky.biz brainboosting.club
bookhome.info brakehawk.com
brandov.ru bronzeaid-a.akamaihd.net
browserprotecter.com btc4u.ru
buckspar.gq capsderfudd.tk
buyk.host.sk cartierlove.xyz
buypuppies.ca cgi2.nintendo.co.jp
cartierbracelet.xyz cheatcode-lita12.rhcloud.com
cartierlovebraceletreplica.xyz cialiswithoutadoctor.net
cayado.snn.gr cibpenonptib.flu.cc
cellfun.mobi cinemaflix.website
cenoval.ru citizenclsdriveri7.pen.io
ci.ua clean-virus-mac.com
clickaider.com clickhype.com
collegeessay19.blogspot.ru club-samodelkin.ru
credit-online.ws cognitiveseo.com
cubook.supernew.org compliance-andrew.top
customergrowthsystems.com compliance-irvin.xyz
d-black.bz converse.ddsoldes.fr
danashop.ru coolwallpapers4k.info
danschawbel.com coop-gamers.ru
data.vtc.pw coprofam.org
day-news.info creditmoney.com.ua
dfiles.me date-withme.com
dimkino.ru dbmkfhqk.bloger.index.hr
doctormakarova.ru dekorkeramik.ru
dosug-lux.ru delayreferat.ru
ekobata.ru descargar-musica-gratis.net
election.interferencer.ru dichvuvesinhhanoi.com
electricwheelchairsarea.com docsportal.net
elektrozigaretten1.postbit.com drev.biz
engines-usa.com druzhbany.ru
etotupo.ru e-poker-2005.com
everypony.ru egovaleo.it
exhibitionplus.eu elvel.com.ua
falopicm.pw enge-fotzen.info
finuse.com ereko.ru
flowersbazar.com european-torches.ru
fordsonmajbor.cf evrotekhservis.ru
formularz-konkurs.tk exlarseva.webblog.es
forum.doctissimo.fr exoclick.com
fotosfotos.eu fast-torrent.ru
fr.netlog.com fbdownloader.com
frameimage.org felizporno.com
freeuploader.com fermersovet.ru
fym.com.ru fikasound.tk
galaxyflowers.ru financehint.eu
gambarxkata.co fivedwld.cf
gearcraft.us floridamobilebillboards.com
getprismatic.com free-stock-illustration.com
glogow.pl ftns.ru
go2album.com funnymama.com
fyl.com.ru
game300.ru
gay-file.com
gay-site.store
goldpanningtools.com
good-mummy.ru good-mummy.ru
graphid.com gosmeb.ru
gsasearchengineranker.xyz gox.com.ua
hanink.biz.ly guod.me
hayate.biz hamilton.ca
headpress.ru hasfun.com
histock.info heatpower.ru
hitmuzik.ru highland-homes.com
home-task.com highspeed5.net
hosting-tracker.com hop.clickbank.net
ilovevitaly.xyz hystersister.com
images-free.net i-midias.net.br
infoupdate.org iinstalll-fii1leis.jus0wil.pp.ua
inome.com.ua ilbe.club
interior-stickers.ru jabimgo.pw
invest-pamm.ru johnrobertsoninc.com
investpamm.ru justprofit.xyz
ivanovo.zrus.org
james13prix.info
janettabridal.com
javspace.net
jerseywholesalebizchina.com
joingames.org
joy-penguin.com
junketjuice.blogspot.com
kanimage.com
katushka.net katushka.net
keyword-suggestions.com kaz.kz
kichenaid.ru kazka.ru
kit-opt.ru kchaxton.com
konoplisemena.com kuzinsp.ru
konteiner24.com landmania.ru
lacave.ntic.fr law-check-seven.xyz
laexotic.com legionalpha.com
lanadelreyfans.us letto.by
larger.io linkbolic.com
lflash.ru linkbux.ru
light.ifmo.ru littlesunraiser.com
lion.conboy.us locustdesign.co
lonerangergames.com longlifelomilomi.info
lookingglassemb.com lyrster.com
lunchrock.co mail.allnews24.in
m-google.xyz mainhp.net
magicplayer-s.acestream.net maju.bluesalt.co
marketingvici.com makenahartlin.com
mathgym.com.au mediaoffers.click
maximpartnerspr.com megagrabber.ru
mcnamaratech.com mejoresfotos.eu
mebel-ekb.com miniads.ca
megakino.net mir-business-24.ru
mikrobiologies.ru mobilierland.com
misswell.net moi-glazki.ru
mockupui.com mountainstream.ms
movpod.in mozello.ru
moz.com mrlmedia.net
mwtpludn.review mrpornogratis.xxx
namenectar.com mundoaberrante.com
new-apps.ru nero-us.com
nicovideo.jp nonews.co
nordvpn.com nucia.biz.ly
nuker.com ofanda.com
nw-servis.ru okmusic.jp
o00.in oneshotdate.com
obiavo.by onlinefilmz.net
omoikiri-japan.ru onlythegames.com
oneclickfiles.com oranga.host.sk
onlinadverts.com outclicks.net
online-sbank.ru panamaforbeginners.com
pacificair.com panchro.co.uk
peekyou.com partner-print.men
perl.dp.ua partners-ship.pro
pharmacyincity.com pivka.xyz
phormchina.com poffet.net
picmoonco.pw porno-play.net
picswe.com pricheskaonline.ru
pipki.r.acdnpro.com produkto.net
play-movie.pl proposal-engine.com
playmsn.com pshare.biz
porndairy.in radiogambling.com
porno-raskazy.ru rankings-analytics.com
potolokelekor.ru rapidsites.pro
proekt-mos.ru remontvsamare.su
promodj.com rfid-locker.co
pron.pro ribieiendom.no
prostitutki-belgoroda.org ricorsogiustizia.org
prostitutki-kharkova.org rockma.se
proxyradar.com roleforum.ru
pskcijdc.bloger.index.hr rootandroid.org
pts163.ru royalvegascasino.com
quickloanbank.com ruf777.com
rabot.host.sk sack.net
raschtextil.com.ua sawin.beth.webd.pl
realmonte.net scanner-elena.top
registrysweeper.com scanner-marwin.top
saitevpatorii.com scat.porn
scanner-andrew.top searchlock.com
scenarii-1-sentyabrya.uroki.org.ua seasaltwithfood.com
seocdvig.ru selectads.men
sethrollins.net seo18.su
sex-watch.com seoexperimenty.ru
sexiporno.net sexsearch.com
sexs-foto.top shemalegalls.blogporn.in
ship-marvel.co.ua shohanb.com
shtaketniki.kz sizeplus.work
signoredom.com skachat-besplatno-obrazcy.ru
sinel.info slonechka.ru
skutecznetabletkinaporostwlosow.pl snowplanes.com
snomer1.ru snsdeainavi.info
soc-proof.su softtor.com
social-s-ggg.xyz sovinsteel.ru
socialbuttons.xyz stardevine.com
speedup-my.site steebook.com
splendorsearch-a.akamaihd.net strfls.com
spywarebegone.com superstarfloraluk.com
srdrvp.com svetodiodoff.ru
sstroy44.ru sweepstakes.rewardit.com
streetfire.net tecnoteakviareggio.it
sucsesofinspiration.com teguh.info
suture.co template-kid.com
t-bygg.com testingads.pro
tattooha.com theprofitsmaker.net
teastory.co thetoiletpaper.com
tiens2010.ru tomck.com
titangel-vietnam.com tozup.com
titslove.yopoint.in trafficgenius.xyz
tizanidineinfo.blogspot.com trahvid.com
tizanidinemusclerelaxant.blogspot.com trustedmaccleaner.com
topappspro.com ttrraacckkrr.com
toproadrunner5.info ural-buldozer.ru
triumf-realty.ru vacuumscleaner.com
tuckermktg.com vduplo.ru
tversvet.ru vedomstvo.net
twincitiescarservice.com video--production.com
twu.com.ua vinylvault.co.uk
unmaroll.ya.ru vipsiterip.org
uptime-alpha.net virtuagirl.com
usdx.us virus-respirators.com
vasileostrovsky-rayon.ru viven.host.sk
vitoriacabos.com vsexkontakte.net
vremya.eu vut.com.ru
waterefficiency.co vzube.com
waterpurifier.club walpaperlist.com
where-toget.com warningzscaler.heraeus.com
whereiskentoday.com watchinf.com
wholesalecheapjerseysfree.com web.cvut.cz
winner7777.net website-speed-checker.site
writingservices17.blogspot.ru website-stealer.nufaq.com
wzgyyq.com whatzmyip.net
xchat26.myfreecams.com wholesalejerseysgaa.com
xn--hxazdsfy.blogspot.com wineration.com
xxxtube69.com worldinternetauthority.com
yadro.ru wrz0iuebwhp5fg.freeddns.com
zaidia.xhost.ro ws.ampower.me
zajm-zalog-krasnodar.ru wyniki-lista.pl
zakazvzloma.com xbaboon.com
zatjmuzu.info xclicks.net
zdesformula.ru xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
zeroredirect7.com xn--lifehacer-1rb.com
zoominfo.com xtrafficplus.com
yatrk.xyz
yebocasino.com
yellowstonesafaritours.com
yougotanewdomain.com
youjizz.vc
yourdesires.ru
zeroredirect9.com
zhorapankratov7.blogspot.com
zoosexart.com
zytpirwai.net

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.52 0; 10.1.1.100 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2694 ### Version: V4.2021.11.2695
### Updated: Thu Nov 4 22:01:26 UTC 2021 ### Updated: Fri Nov 5 10:03:23 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2694 ### Version: V4.2021.11.2695
### Updated: Thu Nov 4 22:01:26 UTC 2021 ### Updated: Fri Nov 5 10:03:23 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##