vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 06:15:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.12.3523 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-12-20 22:07:32 +00:00
parent 984c9fa8a2
commit 794e251b48
46 changed files with 95748 additions and 95748 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3522 #### Version: V4.2022.12.3523
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3522 #### Version: V4.2022.12.3523
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.12.3522 #### Version: V4.2022.12.3523
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11834
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.12.3522 #### Version: V4.2022.12.3523
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11838
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11838
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11838
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11838
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

11838
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

11838
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

11838
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3522 3523

11838
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11838
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.65.83.83 0; 20.119.180.194 0;

11838
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

11838
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

11838
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

11838
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.65.83.83 0; 20.119.180.194 0;

11838
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.65.83.83 0; 20.119.180.194 0;

11838
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Kinza BDFetch
Larbin BuiltBotTough
Netvibes EasyDL
PyCurl InterGET
RocketCrawler PleaseCrawl
SBIder PxBroker
WeSEE ZoomBot
Web Pix crawler.feedback
ZyBorg seobility
ips-agent xpymep1.exe

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker
404enemy
ADmantX
ASPSeek
Abonti
Aboundex
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot AiHitBot
Alligator Alligator
Anarchie Anarchy99
Anarchy
Anthill
Apexoo
Aspiegel Aspiegel
AwarioSmartBot Asterias
BBBike AwarioRssBot
BDCbot BLEXBot
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Bandit Bandit
Barkrowler BatchFTP
BetaBot BetaBot
Black Hole Bigfoot
BlackWidow Bitacle
Brandwatch Blow
Boardreader
Bolt
BotALot
Buck Buck
Buddy Buddy
BunnySlippers
BuzzSumo BuzzSumo
CCBot
CODE87 CODE87
CSHttp
CazoodleBot CazoodleBot
Cegbfeieh
CherryPicker
Chlooe Chlooe
Claritybot Cocolyzebot
Cliqzbot CopyRightCheck
Copier Copyscape
CrunchBot Cosmos
Curious Craftbot
DSearch Crescent
DTS Agent CyotekWebCopy
DataCha0s DBLBot
DIIbot
Demon
Deusu Deusu
Discobot DigitalPebble
Discoverybot
Dispatch
DittoSpyder DittoSpyder
DomainStatsBot DnyzBot
DomainCrawler
Domains Project Domains Project
Dotbot
Dragonfly Dragonfly
Drip
ECCP/1.0
EMail Siphon
EMail Wolf EMail Wolf
EasyDL EasyDL
EirGrabber
Exabot Exabot
Express WebPictures
ExtractorPro
Extreme Picture Finder
Ezooms
FDM
FHscan
FemtosearchBot FemtosearchBot
FlashGet Fimap
Flunky Freeuploader
FrontPage
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW GT::WWW
Genieo
GermCrawler GermCrawler
GetRight GetRight
GetWeb GetWeb
Getintent
Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
Gotit Gotit
GrabNet GrabNet
GrapeshotCrawler Grabber
HEADMasterSEO Grafula
HMView
HTMLparser HTMLparser
HTTrack Heritrix
Haansoft Hloader
HonoluluBot
IDBot IDBot
Iblog IRLbot
Id-search Id-search
InfoNaviRobot
Internet Ninja Internet Ninja
InternetSeer InternetSeer
Iria IstellaBot
JamesBOT JOC Web Spider
Jbrofuzz
JetCar
Jetty
JikeSpider
Joomla Joomla
JustView Jorgee
Jyxobot
Kenjin Spider
Keybot Translation-Search-Machine
Keyword Density
Kinza Kinza
Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple Larbin
Lanshanbot
Leap Leap
LeechFTP LibWeb
Libwhisker Likse
LieBaoFast
LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
Lmspider LinqiaRSSBot
MJ12bot Lipperhey
Majestic SEO MSFrontPage
Mail.RU_Bot
MarkWatch MarkWatch
MauiBot Masscan
Mb2345Browser MeanPath Bot
Mediatoolkitbot Mediatoolkitbot
MicroMessenger
Mojeek
Mozlila
Mr.4x3
Musobot
NICErsPRO NICErsPRO
NPbot NPbot
Name Intelligence
Nameprotect Nameprotect
NearSite
Nessus Nessus
NetLyzer NetAnts
NetMechanic NetZIP
Netvibes Netcraft
NextGenSearchBot Nibbler
Niki-bot Nikto
Nimbostratus
Ninja
Nmap
Not
Nutch Nutch
Octopus Offline Explorer
Offline Navigator OnCrawl
OpenVAS
Openfind Openfind
OrangeBot OutclicksBot
PageThing.com Page Analyzer
PageAnalyzer
PageGrabber
PageScorer
Pandalytics Pandalytics
Papa Foto Petalbot
PeoplePal
Pi-Monster Pi-Monster
Picsearch
Pimonster Pimonster
PleaseCrawl Pixray
Pockey Pockey
ProPowerBot
ProWebWalker ProWebWalker
Pu_iN Pu_iN
PxBroker Pump
PyCurl
QueryN Metasearch
Quick-Crawler Quick-Crawler
RankActive RankActive
RankingBot2 RankActiveLinkBot
Rankivabot
RankurBot RankurBot
ReGet Recorder
Rogerbot RedesScrapy
SBIder SBIder
SEOkicks-Robot SEOkicks
SEOlyticsCrawler
SMTBot SMTBot
SalesIntelligent
ScoutJet ScoutJet
Scrapy SearchmetricsBot
ScreenerBot Seekport
ScrepyBot SemanticJuice
Searchestate Semrush
SentiBot SeobilityBot
SeoSiteCheckup Site Sucker
Shodan
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer Sitebeam
SiteLockSpider SmartDownload
Siteimprove Snapbot
SlySearch SocialRankIOBot
Snake Sogou web spider
Sottopop Sottopop
SputnikBot SpaceBison
SpankBot
Spbot
Spinn3r
Steeler
Sucker
Sucuri Sucuri
SuperHTTP SuperHTTP
SurveyBot
Szukacz Szukacz
Telesphoreo Teleport
Telesphorep TeleportPro
The Intraformant The Intraformant
TheNomad TheNomad
Titan Thumbor
Toweyabot Toata
Tracemyfile Tracemyfile
Trendiction True_Robot
Turingos
TurnitinBot TurnitinBot
TwengaBot
Twice
Typhoeus Typhoeus
URLy Warning UnisterBot
V-BOT
VB Project
VCI VCI
Vagabondo Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
WPScan VidibleScraper
Virusdie
Voltron
WEBDAV
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWWOFFLE
Wallpapers WeSEE
Wallpapers/3.0 Web Collage
Web Auto
Web Enhancer
Web Fetch Web Fetch
Web Pix Web Sauger
WebCollage WebAuto
WebGo IS WebFuck
WebImageCollector WebImageCollector
WebLeacher
WebPix WebPix
WebStripper
WebZIP
Webalta Webalta
WebmasterWorldForumBot
Webshag Webshag
Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
WinHTTrack Webster
Whack
Widow
WiseGuys Robot WiseGuys Robot
Wotbox Xaldon WebSpider
Wprecon Xaldon_WebSpider
Xenu
YoudaoBot
Zauba Zauba
Zeus
Zitebot Zitebot
ZoomBot ZoomBot
ZoominfoBot
adscanner adscanner
archive.org_bot
arquivo.pt
autoemailspider autoemailspider
backlink-check crawler4j
cah.io.community dataforseo.com
check1.exe
com.plumanalytics
demandbase-bot demandbase-bot
domainsproject.org domainsproject.org
eCatch evc-batch
heritrix gopher
internetVista monitor ips-agent
iubenda-radar
linkdexbot linkdexbot
lwp-request
lwp-trivial lwp-trivial
magpie-crawler
meanpathbot meanpathbot
mediawords
muhstik-scan muhstik-scan
page scorer netEstate NE Crawler
satoristudio.net pcBrowser
scan.lol plumanalytics
polaris version
seobility
seocompany.store seocompany.store
seostar seoscanners
sogouspider sogouspider
sp_auditbot spyfu
trendiction.de sysscan
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webgains-bot x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe
zgrab zgrab

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -3,248 +3,248 @@
80legs 80legs
ADmantX ADmantX
AIBOT AIBOT
Abonti AhrefsBot
Aboundex AiHitBot
Aboundexbot
Alexibot Alexibot
Ankit AllSubmitter
Asterias Anarchy
Atomseobot Anthill
AwarioSmartBot Aspiegel
BDCbot BDCbot
BLEXBot BDFetch
BackDoorBot BackDoorBot
BacklinkCrawler BackStreet
Barkrowler Barkrowler
BatchFTP Battleztar Bazinga
Bitacle Blackboard
Black Hole Blow
BlackWidow
Bolt
Brandprotect Brandprotect
Brandwatch Buddy
Buck
BuiltBotTough
Bullseye Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CSHttp
Calculon
CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect
CheeseBot CheeseBot
ChinaClaw CherryPicker
Chlooe Chlooe
Cloud mapping Claritybot
Collector Collector
Copier
Cosmos Cosmos
Craftbot Crescent
Curious
Custo Custo
CyotekWebCopy DIIbot
DTS Agent
Demon
Devil Devil
Digincore Digincore
DigitalPebble
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnyzBot
DomCopBot
DomainStatsBot
Download Wonder Download Wonder
Dragonfly Drip
ECCP/1.0 ECCP/1.0
Evil EMail Wolf
Ecxi
ExtLinksBot ExtLinksBot
ExtractorPro ExtractorPro
Ezooms Extreme Picture Finder
Firefox/7.0 EyeNetIE
FlashGet FDM
Fyrebot Fimap
GT::WWW FrontPage
GermCrawler FyberSpider
GetRight Genieo
Gigabot Gigabot
Go!Zilla Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GrabNet GoZilla
Gotit
Grabber
GrapeFX
GridBot
HMView HMView
HTMLparser
HTTP::Lite HTTP::Lite
Havij Heritrix
Hloader IRLbot
HonoluluBot Iblog
Humanlinks Id-search
IDBot
Image Fetch
IndeedBot
Indy Library
InfoNaviRobot InfoNaviRobot
Intelliseek Intelliseek
Internet Ninja Iria
Iskanie
JOC Web Spider JOC Web Spider
Jbrofuzz JamesBOT
JennyBot JetCar
Jetty Jetty
Jorgee Jorgee
Keyword Density Keyword Density
Kozmosbot Kozmosbot
LWP::Simple LeechFTP
Lanshanbot LeechGet
LibWeb
Libwhisker
Lightspeedsystems Lightspeedsystems
Linkbot LinkWalker
LinkextractorPro LinqiaMetadataDownloaderBot
LinkpadBot Lipperhey
Litemage_walker Litemage_walker
MSIECrawler MFC_Tear_Sample
Magnet MIDown tool
Mail.RU_Bot MIIxpc
MQQBrowser
MSFrontPage
Mag-Net
Majestic-SEO
Majestic12 Majestic12
MarkMonitor MarkMonitor
Mass Downloader Masscan
MauiBot Mata Hari
Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri Metauri
Minefield Mister PiX
Moblie Safari Moblie Safari
Mojeek
MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
Msrabot Musobot
NPbot NICErsPRO
Name Intelligence
Nameprotect Nameprotect
Navroad NearSite
Needle Needle
Net Vampire
NetAnts NetAnts
NextGenSearchBot NetLyzer
NetMechanic
NetSpider
Netcraft
Nibbler Nibbler
Niki-bot
Nikto
NimbleCrawler NimbleCrawler
Nimbostratus Nmap
Ninja Offline Explorer
Not
Nuclei
Octopus
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
OutfoxBot OrangeSpider
PHPCrawl OutclicksBot
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageThing.com Page Analyzer
Pandalytics PageGrabber
Pi-Monster Panscient
Piepmatz Papa Foto
Pavuk
PeoplePal
Picscout
PleaseCrawl PleaseCrawl
Pockey Pockey
ProWebWalker ProPowerBot
Psbot
Pu_iN Pu_iN
PyCurl
Quick-Crawler
RSSingBot RSSingBot
Rankivabot RankingBot2
Re-re RebelMouse
RocketCrawler Rogerbot
SBIder SBIder
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SalesIntelligent SEOlyticsCrawler
SEOprofiler
SEOstats
Scanbot Scanbot
ScoutJet Screaming
ScreenerBot ScreenerBot
ScrepyBot SearchmetricsBot
Searchestate SentiBot
Seekport Shodan
Semrush
SemrushBot
Seomoz
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteLockSpider
SiteSnagger
SiteSucker
Sitebeam Sitebeam
SlySearch SlySearch
SmartDownload Sosospider
Snoopy
SpaceBison
Spammen Spammen
SpankBot
Spanner Spanner
Spbot Spbot
Sqlworm
Sqworm Sqworm
Stripper
Sucker Sucker
SuperBot Sucuri
SuperHTTP Suzuran
SurveyBot
Swiftbot
Szukacz
T0PHackTeam T0PHackTeam
TeleportPro TeleportPro
Telesphoreo Telesphoreo
Telesphorep Toata
The Intraformant Turingos
TightTwatBot
Titan
Toweyabot
Tracemyfile
Turnitin Turnitin
TwengaBot URLy.Warning
Typhoeus
Upflow Upflow
VB Project Virusdie
Vacuum
VidibleScraper
VoidEYE VoidEYE
Voil
Voltron
WEBDAV
WISENutbot
WPScan
WWW-Mechanize WWW-Mechanize
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers
WallpapersHD WallpapersHD
Web Fuck Web Pix
WebAuto Web Sucker
WebBandit
WebCopier
WebEnhancer
WebFuck WebFuck
WebLeacher
WebPix WebPix
WebReaper WebReaper
WebSauger
WebStripper WebStripper
WebSucker WebSucker
WebWhacker
WebZIP WebZIP
Website Quester Webalta
Webster WebmasterWorldForumBot
Webshag
WebsiteExtractor
WebsiteQuester
Whack Whack
WinHTTrack Whacker
Whatweb
Woobot Woobot
Wotbox
Wprecon
Xaldon_WebSpider
Zauba Zauba
Zermelo
Zeus
Zitebot Zitebot
ZmEu ZoomBot
ZumBot ZoominfoBot
arquivo-web-crawler ZyBorg
adscanner
archive.org_bot
arquivo.pt
autoemailspider autoemailspider
backlink-check
cah.io.community
check1.exe check1.exe
clark-crawler crawl.sogou.com
cognitiveseo
crawler.feedback crawler.feedback
dataforseo.com
demandbase-bot demandbase-bot
evc-batch
facebookscraper
gopher
heritrix
iubenda-radar iubenda-radar
linkdexbot
magpie-crawler magpie-crawler
plumanalytics meanpathbot
pcBrowser
polaris version
satoristudio.net satoristudio.net
scalaj-http seostar
seobility
seocompany.store
sexsearcher sexsearcher
sitechecker.pro siteripz
sogouspider
spyfu spyfu
sysscan trendiction.de
voyagerx.com
webpros.com webpros.com
x22Mozilla webprosbot
xpymep1.exe
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
artavenuegardenstudios.com bitcoins-live.ru
bugof.gq cleanpcnow.com
bumascloset.com hacktougroup.ru
easync.io infodocsportal.com
gta-top.ru pornhive.org
kinogonew.ru qor360.com
okuos.com rangjued.com
rostov.xrus.org seogadget.ru
stuff-about-money.com snts.shell-pmr.ru
yaaknaa.info whatistizanidinehclusedfor.blogspot.com

914
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
000free.us
03e.info 03e.info
1314dh.com 0n-line.tv
1j7740kd.website 1qingdao.com
2000k.ru 2345.com
20pascals.nl 300richmond.co.nz
2nt.ru 4inn.ru
3th.co.in
4rent.ru
4ureyesonly.com 4ureyesonly.com
4webmasters.org 4webmasters.org
6hopping.com 5-steps-to-start-business.com
a.pr-cy.ru 70casino.online
a1security.com.ua a1security.com.ua
aab94f698f36684c5a852a2ef272e031bb.com aa14ab57a3339c4064bd9ae6fad7495b5f.com
aa9d046aab36af4ff182f097f840430d51.com
aaa38852e886ac4af1a3cff9b47cab6272.com
abiente.ru
abwa.tk
accessoires-mode.in
actices.com actices.com
adbetclickin.pink actulite.com
admitad.com add-add.men
afftrack001.com addblueoff.com.ua
agardomains.com addray.pro
ahrntal.verymes.xyz admanaerofoil.com
aihelen.net adpremium.org
akita.kz adtech.de
alarmobninsk.ru adultgalls.com
all-streaming-media.com advokateg.ru
alltheviews.com ahmedabadwebs.com
alphaforum.ru aktivator-windows10.blogspot.com
anjalika.co.in alf-img.com
apartamentwroclaw.eu alibestsale.com
api.stathat.com allblogroll.com
alphahoverboards.com
am-se.com
amateurgalls.com
anonymizeme.pro
apimossnetinfo-a.akamaihd.net
apioutoboxnet-a.akamaihd.net
app-ready.xyz app-ready.xyz
appfastplay.com apparel-offer.com
apple.com-webbrowsing-security.science applicationg29.com
apptester.tk aquarium-pro.ru
arenda-avtoprokat-krasnodar.ru architecturebest.com
articlesdirectoryme.info arendas.net
as5000.com artefakct.com
atlasvkusov.ru asia-forum.ru
auspolice.com asrvvv-a.akamaihd.net
autodo.info asscenihotosidea.blogspot.com
availit.weebly.com atlant-auto.info
avkzarabotok.com aviav.org
avtolombard-krasnodar.ru axbocz.net
ayodhya.co
azarttoto.com
azbaseimages.net
babosas.com babosas.com
babyfactory.fr babyguns.ru
bad-stars.net baersaratov.ru
balook.com
bank.uz
bankcrediti.ru
bannerswap.com bannerswap.com
bannertesting.com bashtime.ru
bard-real.com.ua basisches-wasser.net
battle.net
bdsmgalls.net
beauby.ru beauby.ru
bestmouthwash.club best-way.men
big-boards.info besthoro.ru
bestpornuha.com
bestserials.com
bestssaker.com
betune.onlinewebshop.net
bildsuche.ru
binaryoptionscops.info binaryoptionscops.info
binomo.com bitnote.co
bitcoinremote.com blog-hits.com
bkgr.se blog.axant.it
blog.f00kclan.de bocoarchives.org
boltalko.xyz bongiornos.info
brainboostingsupplements.org bonkers.name
brandehk.dk bookmaker-bet.com
brandov.ru bookmark4you.com
browserprotecter.com bpro1.top
brianhenry.co
brk-rti.ru
bryansk.zrus.org bryansk.zrus.org
buqayy0.livejournal.com bscodecs.com
burger-imperia.com bumble.cheapwebsitehoster.com
buy-cheap-online.info buydissertation.net
buyantiviralwp.com
buyfriend.ru buyfriend.ru
buytizanidineonline.blogspot.com buypuppies.ca
cablecar.us
cacheimages.com cacheimages.com
calvet.altervista.org callmd5map.com
candcstuccoandstone.com camdenmemorials.com
canoncdriverq3.pen.io canacopegdl.com
cardul.ru cars-modification.net
carfax.com.ua cezartabac.ro
caveavins.fr chatroulette.si
chatseo.com
cheapestjerseysonwholesale.com
cheapjerseysap.com
chiblackhawks-jerseys.com
christianlouboutinsaleonline.us
cialisovercounteratwalmartusa.com
cialiswithoutadoctor.net
cinemaenergy-hd.ru cinemaenergy-hd.ru
classicakuhni.ru citysecurity.nu
claytransformations.info clarithromycin500mg.com
clickpapa.com cleanallspyware.com
clicksotrk.com cleanallvirus.com
clmforexeu.com clipartnew.com
club-musics.ru
coccoc.com coccoc.com
communisave.co.za colehaanoutlet.store
compliance-irvin.top compliance-alexa.top
compliance-ivan.xyz compliance-barak.top
compliance-jess.top compliance-josh.top
compliance-walt.top
compliance-willy.top
computernetworksonline.com
contentsexpress.com contentsexpress.com
coslab.club cookie-law-enforcement-ee.xyz
cosmetic.donna7753191.ru
costablue.xyz
countbertwistdisp26.soup.io
cowblog.fr
customboxes4less.com
customerguru.in
cutalltheshit.com
cvety24.by cvety24.by
cypernhuset.se dalavia.ru
dailyrank.net damedingel.ya.ru
dame-ns.kz dating-time-now.com
dcj-nn.ru datsun-do.com
ddpills.com ddospanel.com
dealitright.click delitime.info
deanmoore.ie dev78.dioniqlabb.se
decoratinghomes.ga dgfitness.co
den-noch24.ru dieta-personalna.pl
deplim.com discovertreasure-a.akamaihd.net
depositfiles-porn.ga divisioncore.com
design-ideas.info divku.ru
design-lands.ru
dikx.gdn
divatraffic.com
djonwatch.ru djonwatch.ru
dneprsvet.com.ua downloads-whatsapp.com
dnsrsearch.com
dogclothing.store
domain-tracker.com
domaincrawler.com
domznaniy.ru
dotnetdotcom.org
downtuptv.gq
draniki.org
drev.biz drev.biz
duawitchrarato.tk droidlook.net
earncash.com.ua dyt.net
easyshoppermac.com eager-nash.188-93-233-196.plesk.page
ebooknovel.club eaplay.ru
elektrischezigarette2.devhub.com easycommerce.cf
elektronischezigarettekaufen2.tumblr.com edge.sharethis.com
elkacentr.ru editmedios.com
emailaccountlogin.co eksprognoz.ru
enews.tech elektronischezigarettekaufen1.myblog.de
englate.com empis.magix.net
englishdictionaryfree.com
eorogo.top
ero-advertising.com ero-advertising.com
eropornosex.ru errorfixing.space
essaypro.com ertelecom.ru
etenininrade.ga
eupornstar.info eupornstar.info
european-torches.ru exoclick.com
extlinks.com ezigarettekaufen.myblog.de
ez8motelseaworldsandiego.com ezigarettenkaufen1.deviantart.com
ezigarettenshop2.myblog.de familyphysician.ru
facebook-mobile.xyz farmprofi.net
familyholiday.ml fenoyl.batcave.net
fashion.stellaconstance.co feorina.ru
favoritemoney.ru fertilitetsradgivningen.se
faz99.com fickblock18.com
feeriaclub.ru fickluder69.com
femdom.twiclub.in filkhbr.com
fickenbumsen.net fillmewithhappiness.com
filefilter.weebly.com finemanteam.com
filmci.pro fiuxy.com
findclan.org flagcounter.me
floating-share-buttons.com flowersbazar.com
forpostlock.ru footballfarrago.com
free-laptop-reward.com forex-indextop20.ru
forsex.info
forumprofi.de
foxtechfpv.com
franecki.net
freejabs.com freejabs.com
freshwallpapers.info freeseedsonline.com
frighteningremain.cf friendflnder.com
fym.com.ru fungirlsgames.net
gameonasia.com gabeshop.ru
gamewrath.com gagrasector.ru
gazporno.com gamedayassist.com
gdcentre.ru gay-file.com
gearsadspromo.club germes-trans.com
get-seo-domain.com getaclueamerica.com
getdot.ru getridofstretchmarks.org
getoutofdebtfree.org
getresponse.com
gg.zzyjxs.com gg.zzyjxs.com
gilsonchiro.xyz gigapeta.com
globalsurfari.com glavtral.ru
golfresa.lucania.se gobongo.info
googlefeud.com gsasearchengineranker.top
gox.com.ua gsbs.com.ua
gribkovye-zabolevaniya.com gwebtools.com
gsasearchengineranker.xyz hazardky.net
gsmtlf.ru hdimagegallery.net
guardlink.org
guitar-master.org
hdpixion.com
healthcarestore.info
hebr.myddns-flir.com hebr.myddns-flir.com
histock.info herehloadibs.cf
hit-men.men heroz.fr
home-task.com hetmanship.xyz
hobbyhuren24.net
hoholikik.club
hollywoodactress.info
homeandhealth.ru
homeart.space
homemature.net
hoporno.com
hotel-mkad.ru
hoverboard360.es
hoverboard360.se
howopen.ru howopen.ru
howtowhitenteethfast.xyz hptwaakw.blog.fc2.com
hubbble.com hreade.com
hully.altervista.org huimin764128.com
humanelydrew.com ictizanidinehcl4mg.blogspot.com
husky-shop.cz idealtits.net
iblogpress.xyz igithab.com
iframe-toloka.com igrovyeavtomaty777.ru
igtools.club ikritikimou.gr
iloveitaly.ru ilbe.club
ilovevitaly.com imagine-ex.co
images-free.net imperia31.ru
imicrovision.com inclk.com
impisr.ru incomekey.net
impresagaia.it infostatsvc.com
incest-ru.com insomniagamingfestival.com
intellego.info interesnie-faktu.ru
interfucks.net internetproviderstucson.com
iqoption.com inzn.ru
it-max.com.ua iron-age.info
itag.pw irunfar.com
ivanovo.zrus.org isitpaleo.info
janavibekken.no islandminingsupply.wordpress.com
jav-idol.com itbc.kiev.ua
javxxx18.com itrevolution.cf
jerseyschinabizwholesale.us ivoiretechnocom.ci
jerseywholesaleelitestore.com ix20.ru
jobmarket.com.ua japfm.com
journalhome.com jikoman.info
jus0wil.pp.ua joyceblog.top
kamagragelusa.net justkillingti.me
kandidos.com jyvopys.com
karting196.ru kabbalah-red-bracelets.com
kasino-money.pw katushka.net
keywordblocks.com ketrzyn.pl
keywordsuggests.com
keywordteam.net keywordteam.net
kichenaid.ru kinky-fetishes.com
kihi.gdn kinocccp.net
kiinomaniak.pl kinotorka.ru
kino2018.cc
kino2018.club
klejonka.info
klikbonus.com
klitimg.pw klitimg.pw
klumba55.ru klosetkitten.com
konkursov.net kol-energo.ru
koptims.tiu.ru komputernaya-pomosh-moscow.ru
kredit-blog.ru krasula.pp.ua
kupiproday.com.ua kretpicf.pw
lafriore.ru kutikomi.net
lancheck.net labafydjxa.su
landreferat.ru ladsup.com
laserpen.club
lasvegaslockandsafe.com
lazy-z.com
lechtaczka.net
legalrc.biz
legionalpha.com legionalpha.com
letolove.ru lego4x4.ru
lida-ru.com limads.men
life-instyle.com linerdrilling.com
lincolntheatre.com link.ac
linksharingt.com liveinternet.ru
liumimgx.pw livingcanarias.com
liver-chintai.org llastbuy.ru
loginduepunti.it lob.com.ru
long-beach-air-conditioning.com locatellicorretor.com.br
lostaruban.com lol-smurfs.com
lovelycraftyhome.com lombia.co
m292.info lost-alpha.ru
magicplayer-s.acestream.net lovasszovetseg.hu
makemoneyonline.com lsex.xyz
makenahartlin.com lutherstable.org
managerpak204.weebly.com m.b00kmarks.com
marmotstore.online m3gadownload.pl
marsgatan.com macdamaged.space
massage-info.nl makis.nu
maxthon.com manve.info
mbiologi.ru mararoom.ru
me-ke.com marinetraffic.com
mebel-ekb.com masturbate.co.uk
meddesk.ru matb3aa.com
mediafresh.online matpre.top
meduza-consult.ru medicovi.com
meendo-free-traffic.ga medictube.ru
mejoresfotos.eu medizinreisen.de
meltwater.com meds-online24.com
menstennisforums.com mega-bony-2017.pl
minegam.com microstatic.pl
miniads.ca miniads.ca
mmgq.ru miniature.io
mmog-play.ru missvietnam.org
mmoguider.ru mobilemedia.md
mmstat.com mobplayer.ru
moi-glazki.ru modenamebel.ru
mototsikl.org mojowhois.com
moyaterapiya.ru mountainstream.ms
mrmoneymustache.com mrvideospornogratis.xxx
mvpicton.co.uk mtmtv.info
my-bc.ru musicvidz.ru
mydearest.co myanyone.net
mydirtyhobby.com mydownloadengine.com
myfreecams.com mydownlodablefiles.com
narco24.me myftpupload.com
naturalpharm.com.ua myiptest.com
negociosdasha.com myliveblog.ru
negral.pluto.ro mypets.by
neodownload.webcam mywallpaper.top
netallergy.ru nacap.ru
nextbackgroundcheck.gq naphukete.ru
norththeface.store net-profits.xyz
oakleyglassesonline.us newtechspb.ru
obnal.org nextrent-crimea.ru
officedocuments.net nfljerseyscheapbiz.us
ok.ru ningessaybe.me
oktube.ru nodup.ru
online7777.com northfacestore.online
onlinedomains.ru nosecret.com.ua
onlinemeetingnow.com notaria-desalas.com
ons-add.men notasprensa.info
orgasmatrix.com nowtorrents.com
osoznanie-narkotikam.net nsatc.net
paclitor.com nyfinance.ml
paleohub.info o333o.com
pammik.ru odoratus.net
partner-host.men onlinadverts.com
partner-print.men onlinemegax.com
partnerline.men optikremont.ru
partnerworkroom.men oren-cats.ru
pc4download.co orenburg-gsm.ru
pestomou.info ourtherapy.ru
petsblogroll.com ovirus.ru
php-market.ru parajumpersoutlet.online
pattersonsweb.com
perederni.net
pflexads.com
picsearch.com picsearch.com
pihl.se pirateday.ru
piulatte.cz pochtovyi-index.ru
pk-services.ru podshipniki-nsk.ru
pl.id-forex.com podshipniki-ntn.ru
platezhka.net popander.mobi
play-mp3.com pornobrazzers.biz
polska-poezja.com
popup-hgd.xyz
porn555.com
pornoblood.com
pornoelita.info
pornogig.com
pornonik.com
postclass.com
potoideas.us
pourvous.info pourvous.info
predominant-invent.tk pp-budpostach.com.ua
prefersurvey.net presleycollectibles.com
privatefx-in.ru privat-girl.net
proxyradar.com privatamateure.com
psvita.ru professionalwritingservices15.blogspot.ru
puserving.com profit-opportunity.com
push-ad.com proprostatit.com
putevka24.ru prostitutki-kharkova.org
qor360.com
quality-traffic.com
quelle.ru quelle.ru
quick-offer.com
ranksays.com ranksays.com
rapidsites.pro rarbg.to
ratemodels.net
rczhan.com rczhan.com
realtytimes.com rebelmouse.com
rebuildermedical.com regionshop.biz
redbottomheels.xyz registrydomainservices.com
redirlock.com registrysweeper.com
retreatia.com reklamuss.ru
rollercoin.com ringtonepartner.com
rockingclicks.com
rogervivierforsale.com
rospromtest.ru rospromtest.ru
rossmark.ru royal-investments.net
ru-dety.ru ruclicks.com
ru-mediaget.ru saddiechoua.com
rumamba.com sanatorrii.ru
russkie-gorki.ru
saecsa.co
sandhillsonline.com
sashagreyblog.ga
satellite.maps.ilovevitaly.com
sax-sex.com
sbetodiodnye-lampy.ru sbetodiodnye-lampy.ru
scanner-brian.top sbf441.com
scanner-donald.top scenarii-1-sentyabrya.uroki.org.ua
scanner-john.top schoolfiles.net
score-ads.men scoopquest.com
secret.xn--oogle-wmc.com scrapinghub.com
security60-e.com sdelai-prosto.ru
see-your-website-here.com seeingmeerkat.com
semrush.com sendearnings.com
seo-prof1.xyz seo-prof1.xyz
sexuria.net servingnotice.com
sfd-chess.ru seryeznie-znakomstva.ru
shop-electron.ru sevenstars7.com
sildenafilcitratemed.com sex-pr.net
silvermature.net sex-sex-sex5.com
simple-share-buttons.com sexvideo-sex.com
shillyourcoins.com
shoppytoolmac.com
silverage.ru
simpoed.ufop.br
site-auditor.online
sitevaluation.org sitevaluation.org
sklad-24.ru sky-mine.ru
skypasss.com skypasss.com
slomm.ru slowmacfaster.trade
snworks.com smartadserver.com
snyatie-lomki-v-stacionare.ru smokewithrabbits.com
social-button.xyz social-buttons.com
socialbuttons.xyz socialbookmarksubmission.org
softomix.net soft-program.com
soheavyblog.com softomix.com
songplanet.ru solmarket.by
soviet-portal.do.am special-porn.com
space2019.top stal-rulon.ru
specialfinanceoffers.com stariy-baku.com
speedup-my.site steebook.com
spinnerco.ca stickers-market.ru
spl63.fr stockspmb.info
spy-app.info stopnarco.ru
staticfs.host store-rx.com
strana-krasoty.ru sumo.com
stroydetali.ru
studiofmp.com
suhanpacktech.com
sukirgenk.dvrlists.com
superstats.com superstats.com
taaaak.com svbur.ru
tdsing.ru takethatad.com
tecspb.ru teenbbw.yopoint.in
tehngr.ru tetracsaudi.com
telefonsexi.com theallgirlarcade.com
telefonsexsofort.tk thedownloadfromwarez.blogspot.com
template-kid.com thepantonpractice.co.uk
terraclicks.com thin.me.pn
texbaza.by tizanidineandhydrocodone.blogspot.com
thenetinfo.com tizanidineusedfor.blogspot.com
thepokertimer.com tjkckpytpnje.com
thewebsitetemplate.info tobeyouday.win
tilido.com token-lab.org
time-japan.ru toon-families.com
timeallnews.ru topads.men
timhost.ru topbestgames.com
tivolibasket.it topclickguru.com
tizanidine4mguses.blogspot.com topsiteminecraft.com
tizanidinehcl4mgtab.blogspot.com torrentdownloadhub.com
tizanidinemusclerelaxant.blogspot.com torrnada.ru
tizanidinesleepaid.blogspot.com totu.info
topshef.ru toys.erolove.in
torrent-to-magnet.com tracksz.co
torrents.life trackzapper.com
tovaroboom.vast.ru tradgardspartner.se
tpu.ru traffic100.com
track-rankings.online traffic2cash.org
trustaffs.com trafficcentr.xyz
tuberkuleznik.ru trafficmonetize.org
turizm.bz trailer.cinemaflix.website
tvnewsclips.info trkdf.com
tyumen.xrus.org trubywriting.com
uk-zheu20.ru tula.howotorg.ru
unmaroll.ya.ru turn-up-life.life
uprour.com ucsol.ru
uptime-delta.net ukrup.com
urengoy.pro upstore.me
urlcut.ru uzporno.mobi
valoresito.com v24s.net
vedomstvo.net
veles.shop veles.shop
venerologiya.com venerologiya.com
vesnatehno.com venta-prom.ru
via-energy-acquistare.com viagra.pp.ua
videochat.mx videochat.life
videogamesecrets.com videochat.ph
videotuber.ru viel.su
vipcallsgirls.com vitoriacabos.com
vipsiterip.org vladimir.zrus.org
vitalads.net vodaodessa.com
vkmusics.ru vtcdns.com
vladimir.xrus.org wallpaperstock.net
vonradio.com watchdogs-2.ru
vzlom-na-zakaz.com wdfdocando.com
wanker.us webix.me
watch-movies.ru webmonetizer.net
wcb.su whatsappdownload10.com
wdrake.com wholesalecheapjerseysfree.com
web.cvut.cz windowssearch-exp.com
websiteaccountant.de wma-x.com
wellcome2slovenia.ru wpthemedetector.co.uk
westen-z.life wrz0iuebwhp5fg.freeddns.com
wineration.com www.888.com
withstandingheartwarming.com www.pinnacle-bets.com
wleuaprpxuvr.ga x-lime.net
woman-h.ru xblog.in
worldhistory.biz xmlinde.com
worldlovers.ru xn----7sbaphztdjeboffeiof6c.xn--p1ai
www.ehscloud.cn xn----btbdvdh4aafrfciljm6k.xn--p1ai
x69ty.ru xn----ctbbcjd3dbsehgi.xn--p1ai
xblognetwork.com xn--24-glceagatoq7c2a6ioc.xn--p1ai
xedserver.com xn--90acenikpebbdd4f6d.xn--p1ai
xjlottery.com xolodremont.ru
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai xrus.org
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai yaderenergy.ru
xn----7sbho2agebbhlivy.xn--p1ai youbloodyripper.com
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf yourserverisdown.com
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai yousense.info
xn----itbeirbjbi7bc6bh2d.xn--p1ai youtubologia.it
xn--80abgj3a5acid6ghs.top yunque.pluto.ro
xxart.ru zeroredirect2.com
youbrainboost.asia
youraticles.pl
zacreditom.ru
zakazvzloma.com
zazagames.org
zeroredirect7.com zeroredirect7.com
zigarettenonlinekaufen1.blox.pl znakomstva-piter78.ru
zojirushi-products.ru zoompegs.com
zvuker.net zryydi.com
zzlgxh.com zscalerone.net

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00it.com 0n-line.tv
100dollars-seo.com 1001desert.com
23kw.ru 1xbet4you.com
3xforum.ro 5elementov.ru
5ws.dating-app.ru 7fon.ru
accessoires-mode.in 7xc4n.com
ads-cool.pro 98oi.ru
advancedmassagebysara.com adclickservice.com
adverster.com adconscious.com
affordablewebsitesandmobileapps.com adktrailmap.com
afslankpillen2017nl.eu admanaerofoil.com
agardomains.com adssafeprotected.com
affiliate-fr.com
ahrefs.com
alarmobninsk.ru alarmobninsk.ru
alborzan.com allnews24.in
albumsuper.info alphahoverboards.com
alert-hgd.xyz android4fun.org
amanda-porn.ga animaltoplist.com
amoi.tn apartmentbay.ru
animenime.ru appfixing.space
anonymizeme.pro appleid-verification.com
apifasterlightin-a.akamaihd.net applyneedy.xyz
apiholdingmypage-a.akamaihd.net autoseo-traffic.com
apparel-offer.com awency.com
arenanews.com.ua azbukafree.com
asmxsatadriverin.aircus.com azinoofficial777.ru
atlant-auto.info b00kmarks.com
autoloans.com backlinkwatch.com
aversis.net balitouroffice.com
avramstroy.ru bannerswap.com
azbukadiets.ru bausparen.bz.it
bag77.ru bestdraws.com
beachfix.co bigcareer.info
beclean-nn.ru bitcoin-ua.top
bmw-ark.ru blavia.00author.com
bonvillan.com blobar.org
btnativenav.com bongacams.com
bumskontakte.org bonky.biz
camel-beach.com brains2.biz
canrioloadm.gq breastaugmentation.co.za
caveavins.fr bus-offer.com
chidporn.com buttons-for-website.com
children-learningreading.info buyessaynow.biz
chocolateslim-u-srbiji.com buzzurl.jp
ciekawinki.pl buzzzg.men
cinemaflix.website cardul.ru
citetick.com carsdined.org
classicakuhni.ru chastnoeporno.com
clean-start.net checkpagerank.net
chinese-amezon.com
cl.s7.exct.net
clayvasedesigns.tk
clodo.ru
cocolyze.com
coderstate.com
compliance-barak.top
compliance-margo.top
coolgamechannel.com
coolingoods.com
cosmeticswomens-womensports.rhcloud.com
crazyboost.pro
crd.clan.su crd.clan.su
creatives.adbetclickin.pink culturevie.info
crynet.cc darkbooks.org
cyclobenzaprinevstizanidine.blogspot.com deda-moroza-zakaz.ru
czat.wp.pl deluxedumps.com
davidovic.info dengi-pod-zalog-nedvizhimosti.ru
deutschland123.de devochki.top
dignitasdata.se divatraffic.com
dogoimage.com dogbreedspicture.net
dostavka-v-ukrainu.ru domain-tracker.com
duplicashapp.com doreenblog.online
e-biznes.info dostavka-v-krym.com
egovaleo.it download-wallpaper.net
downtuptv.gq
drillsaw.ru
duawitchrarato.tk
dvr.biz.ua
ekto.ee ekto.ee
electricwheelchairsarea.com elektrozigaretten2.yn.lt
elektrischezigarette1.blog.pl enge-fotzen.info
enews.tech epngo.bz
erotiktreff24.info
erotische-geschichten-xxl.com erotische-geschichten-xxl.com
essay-writing.work extlinks.com
etotupo.ru falcoware.com
ezigarettekaufen2.mpbloggar.se finstroy.net
face.hostingx.eu flauntyoursite.com
fanrto.com freefoto.ca
fbdownloader.com freevpn.space
fefo.gdn freewebs.com
filkhbr.com freshwallpapers.info
fix-website-errors.com gamewrath.com
flooringinstallation-edmonton.com gazporno.com
flowwwers.com genericviagrasildenafiled.net
fototravel.eu ggiaro.com
geileweiber.tk glicol.kz
gemara.com glogow.pl
genericlowlatencyasiodriverhq.aircus.com gotcher.us
generousdeal-a.akamaihd.net gq-catalog.gq
getpopunder.com guiadeserraazul.com
gettpromos.com hcate.com
ghazel.ru hchha.com
girls-ufa.ru heygidday.biz
gk-atlant.info highstairs-a.akamaihd.net
glavprofit.ru hotxnights.info
glopages.ru hreade.com
hdseriale.pl ie.57883.net
heartofpayne.xyz inet-traffic.com
hermesbracelets.xyz istanbulit.com
herokuapp.com itrevolution.cf
hidefiles.org itronics.ca
homedecoguide.info ivearchenceinflu.cf
hotblog.top izhevsk.xrus.org
iamsport.org junglenet-a.akamaihd.net
importchinacoach-teach.com kadashihotel.com
incest-ru.com kalandranis.gr
increasewwwtraffic.info keywordglobal.co.za
inome.com.ua
iqoption-bin.com
iscblog.info
isistaylorporn.info
islamtoday.co.za
ispac.org
it-max.com.ua
ixora.pro
jav-fetish.site
jerseywholesaleelitestore.com
jo24news.com
juliaworld.net
kaidalibor.de
kakablog.net
kemerovo.zrus.org
keywordsuggests.com keywordsuggests.com
khafre.us koral.se
kihi.gdn kosynka-games.ru
kinomaniatv.pl krynica.info
kinopolet.net kvartir-remont.biz
klikbonus.com li-er.ru
knogg.net linkpulse.com
knowsitall.info linkwithin.com
kopihijau.info lmrauction.com
koptims.tiu.ru loginduepunti.it
kostenlos-sexvideos.com lost-alpha.ru
larose.jb2c.me mackeeper-land-672695126.us-east-1.elb.amazonaws.com
lazy-z.com magazin-pics.ru
legionalpha.com magda-gadalka.ru
lenvred.org mailemedicinals.com
lerporn.info manve.info
listiseltemournlan.gq marwer.info
littleberry.ru memberty.com
louboutinreplica.xyz mere.host.sk
lowephotos.info michellblog.online
lsex.xyz minecraft-rus.org
luckyday.world mirzonru.net
maridan.com.ua mixtapetorrent.com
matchpal-a.akamaihd.net moje-recenze.cz
me-ke.com moneymaster.ru
meetingrainstorm.bid montredemarque.nl
megaindex.ru msfsaar.de
mexicosleevegastrectomy.com msk.afora.ru
michaelkorsoutletstore.net mturkcontent.com
missvietnam.org musicktab.com
monarchfind-a.akamaihd.net musicstock.me
monclercoats.xyz narkologiya-orel.ru
mrwhite.biz nomuos.it
multgo.ru norththeface.store
musicspire.online novgorod.xrus.org
muz-baza.net odesproperty.com
mygameplus.ru oksrv.com
myonigroup.com
net-radar.com
nextlnk12.com
njpalletremoval.com
nmrk.ru
nodding-passion.tk
nuker.com
omgtnc.com
omoikiri-japan.ru omoikiri-japan.ru
openfrost.com openlibrary.org
ozas.net pavlodar.xkaz.org
ozoz.it pensplan4u.com
panchro.xyz perm-profnastil.ru
patol01.pw perso.wanadoo.es
pdn-4.com pictures-and-images.com
peugeot-club.org pochemychka.net
phuketscreen.com
poisk-zakona.ru poisk-zakona.ru
porno.simple-image.com.ua popcash.net
pornofiljmi.com pornohub.me
pornoklad.ru pricheskaonline.ru
pozdrawleniya.com profitfx.online
pravoholding.ru prom23.ru
preventheadacheguide.info pronekut.com
prize44.com randki-sex.com
promodj.com rapidsites.pro
prosmibank.ru rebuildermedical.com
registratciya-v-moskve.ru relayblog.com
rentaremotecomputer.com remmling.de
replicalouboutin.xyz renecaovillasale.online
research.ifmo.ru restaurantlescampi.com
resellerclub.com rumamba.com
robertefuller.com ruspdd.com
ru-dety.ru sc-specialhost.com
ru-mediaget.ru scrapy.org
rusexy.xyz searchlock.com
rvtv.ru selectads.men
sankt-peterburg.nodup.ru seoprofiler.com
seeingmeerkat.com seorussian.ru
seeresultshub-a.akamaihd.net sexpornotales.net
sei80.com sextracker.com
seocheckupx.com sexuria.net
servingnotice.com sibtest.ru
servisural.ru sim-service.net
sextracker.de skytraf.xyz
sharpchallenge.com slavkokacunko.de
shop-electron.ru sledstvie-veli.net
silvermature.net snjatie-geroinovoy-lomki.ru
siteonomy.com sonata-arctica.wz.cz
social-s-hhh.xyz spamnuker.com
spl63.fr spidtest.org
spywarespy.com squidoo.com
srecorder.com sucsesofinspiration.com
sudexpert66.ru syndicate.fun
superfish.com taboola.com
superoboi.com.ua taraz.xkaz.org
susanholtphotography.com teslathemes.com
swinger-mobil.net tgtclick.com
tattomedia.com tizanidine4mgprice.blogspot.com
tds-advert005.info tizanidinedosageforsleep.blogspot.com
tedxrj.com tizanidinedruginteractions.blogspot.com
teguh.info tizanidineonline.blogspot.com
textads.men tizanidineoral.blogspot.com
the-torrent-tracker.blogspot.com tizanidineorflexeril.blogspot.com
thebluffs.com tizanidinesideeffectsweightloss.blogspot.com
tizanidineandgabapentin.blogspot.com tmm-kurs.ru
top-deal.com.pl tonerbox.kz
topads.men top250movies.ru
tourcroatia.co.uk torture.ml
trustedmaccleaner.com trafficgenius.xyz
tuckpointingmasonrysystems.com trahic.ru
ukrup.com turkeyreport.tk
url-extractor.xyz ucoz.ru
vgoloveboli.net uk-zheu20.ru
videos-for-your-business.com v-doc.co
viktoria-center.ru vbikse.com
vkontaktemusic.ru vclicks.net
vladhistory.com vektorpress.ru
voloomoney.com vip2ch.com
wallpapersist.com voronezh.xrus.org
watracker.net vozbujdenie.com
webadvance.club vzlom-na-zakaz.com
webstatsdomain.org warningzscaler.heraeus.com
wholesalejerseychinashop.com webcamtalk.net
www.labves.ru whatsupinfoley.com
www.souvenirua.com wieseversa.no
x-true.info writersgroup580.web.fc2.com
xcombear.ru wyniki-lista.pl
xmronta.com x-diesel.org
xn--80aodinpgi.xn--p1ai x5market.ru
xn--80atua3d.xn--p1ai xaijo.com
xn--lifehacer-1rb.com xn----7sbahjd3btneuw1joc.xn--p1ai
yellowstonevisitortours.com xn--80ahvj9e.xn--p1ai
yopoint.in xn--80aikhbrhr.net
youtube-downloader.savetubevideo.com xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
zambini.ru xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
znakomstva-piter78.ru xxxdatinglocal.us
yellowstonesafaritours.com
yugk.net
za-music.mymobiplanet.com
zaobao.com.sg
zastenchivosti.net
zerocash.msk.ru
zetmaster.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.25 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3522 ### Version: V4.2022.12.3523
### Updated: Sat Dec 17 22:01:16 UTC 2022 ### Updated: Tue Dec 20 22:01:37 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3522 ### Version: V4.2022.12.3523
### Updated: Sat Dec 17 22:01:16 UTC 2022 ### Updated: Tue Dec 20 22:01:37 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##