vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 04:00:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.04.3050 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-04-17 22:07:00 +00:00
parent 8686e3406e
commit 87099e7602
46 changed files with 97593 additions and 97593 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3049 #### Version: V4.2022.04.3050
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3049 #### Version: V4.2022.04.3050
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.04.3049 #### Version: V4.2022.04.3050
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12064
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.04.3049 #### Version: V4.2022.04.3050
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12068
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12068
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12068
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12068
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

12068
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

12068
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

12068
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3049 3050

12068
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12068
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.231.230.15 0; 20.121.15.133 0;

12068
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

12068
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

12068
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

12068
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.231.230.15 0; 20.121.15.133 0;

12068
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.231.230.15 0; 20.121.15.133 0;

12068
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Deusu AlphaBot
DomainStatsBot Aspiegel
Evil CSHttp
JOC Web Spider LinkWalker
Masscan Litemage_walker
Name Intelligence MicroMessenger
Nessus SalesIntelligent
NetAnts Teleport
SISTRIX WebAuto
check1.exe linkdexbot

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
ADmantX 404enemy
AIBOT
ASPSeek
Abonti Abonti
Aboundex Aboundex
AhrefsBot Aboundexbot
AfD-Verbotsverfahren
Alexibot Alexibot
Alligator
AlphaBot AlphaBot
Apexoo Anarchie
Aspiegel
Atomseobot Atomseobot
Attach Attach
BLEXBot AwarioRssBot
BackWeb AwarioSmartBot
BacklinkCrawler BBBike
BDFetch
BackStreet
Backlink-Ceck
Barkrowler
Battleztar Bazinga
BetaBot
Bigfoot Bigfoot
Black Hole
Blackboard Blackboard
Brandwatch Blow
BlowFish
BotALot
Buddy
BuiltBotTough BuiltBotTough
BuiltWith
BunnySlippers BunnySlippers
CATExplorador BuzzSumo
CODE87
Calculon Calculon
Cegbfeieh
CheTeam CheTeam
CherryPicker ChinaClaw
Chlooe
Cliqzbot Cliqzbot
Cocolyzebot
CopyRightCheck CopyRightCheck
Copyscape Cosmos
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Curious Curious
DSearch Digincore
DigitalPebble Discobot
Dirbuster DittoSpyder
DnBCrawler-Analytics
DnyzBot DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainCrawler Domains Project
DomainSigmaCrawler ECCP/1.0
EirGrabber EMail Wolf
EroCrawler
Evil
Exabot Exabot
Extractor
EyeNetIE
FemtosearchBot FemtosearchBot
Fimap Flunky
Firefox/7.0
FlashGet
Foobot
Freeuploader
FrontPage FrontPage
Fuzz
FyberSpider FyberSpider
G-i-g-a-b-o-t GT::WWW
GalaxyBot Genieo
GermCrawler
GetRight GetRight
Gigabot GetWeb
GrapeshotCrawler
GridBot GridBot
HMView HTMLparser
HTTP::Lite HTTP::Lite
HaosouSpider Harvest
Havij
Heritrix Heritrix
Hloader
HonoluluBot
Humanlinks
HybridBot
IDBTE4M
IDBot
IRLbot IRLbot
IndeedBot IlseBot
Image Fetch
Image Sucker
Indy Library
Intelliseek
InterGET
Internet Ninja Internet Ninja
InternetSeer InternetSeer
Iria
Iskanie
IstellaBot
JOC Web Spider JOC Web Spider
Jetty
JikeSpider JikeSpider
Jorgee Joomla
JustView JustView
Kenjin Spider Jyxobot
Keyword Density Larbin
Kinza LeechFTP
LWP::Simple LexiBot
Leap LibWeb
LeechGet Libwhisker
LieBaoFast Lightspeedsystems
LinkScan LinkScan
LinkWalker
Linkbot
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
Lipperhey Lipperhey
Lipperhey Spider Litemage_walker
MIDown tool MIDown tool
MQQBrowser Magnet
MSFrontPage
Mag-Net
Majestic SEO
Majestic-SEO Majestic-SEO
Majestic12
MarkWatch MarkWatch
Mass Downloader Mass Downloader
Masscan
Mata Hari
MauiBot
Meanpathbot Meanpathbot
MicroMessenger MicroMessenger
Microsoft Data Access Microsoft Data Access
Minefield Moblie Safari
Mister PiX Mojolicious
Mojeek
MolokaiBot MolokaiBot
Mozlila
Mr.4x3 Mr.4x3
Musobot
NICErsPRO
NPbot
Needle
Net Vampire Net Vampire
Netvibes NetAnts
NextGenSearchBot NetMechanic
Nibbler Netcraft
Nikto Nettrack
NimbleCrawler Ninja
Nimbostratus Nuclei
Nmap Octopus
Not Offline Explorer
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS
Openfind
Openvas Openvas
OrangeBot OrangeBot
OrangeSpider PageThing.com
OutclicksBot
OutfoxBot
PHPCrawl
POE-Component-Client-HTTP
Page Analyzer
Papa Foto Papa Foto
Petalbot PeoplePal
Picscout
Picsearch Picsearch
Piepmatz Piepmatz
PleaseCrawl Probethenet
PyCurl Psbot
QueryN Metasearch Pu_iN
Rankivabot RSSingBot
RankurBot RankActive
RankingBot
Re-re Re-re
ReGet ReGet
RealDownload
RebelMouse RebelMouse
RocketCrawler Recorder
RedesScrapy
Ripper
Rogerbot
SEOstats SEOstats
SalesIntelligent SalesIntelligent
ScanAlert
Scanbot
Scrapy
Screaming Screaming
ScrepyBot Seekport
SemanticJuice SentiBot
Seomoz Siphon
Shodan Site Sucker
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteLockSpider
SiteSnagger SiteSnagger
SlySearch Sitebeam
Snake Siteimprove
Snoopy Snapbot
Sociscraper Sosospider
Sogou web spider Spammen
Sottopop
SpaceBison
Spanner
Spbot Spbot
Surfbot SputnikBot
Swiftbot Sqlworm
Steeler
Stripper
Sucker
Sucuri
T0PHackTeam T0PHackTeam
Teleport
TeleportPro TeleportPro
Telesphoreo Telesphoreo
Telesphorep Telesphorep
Thumbor TheNomad
TightTwatBot TightTwatBot
Titan True_Robot
Trendiction
Trendictionbot
Turnitin Turnitin
TwengaBot TurnitinBot
Upflow Twice
V-BOT V-BOT
VB Project
Vagabondo
Virusdie
VoidEYE VoidEYE
Voil WEBDAV
Voltron WPScan
WASALive-Bot WWW-Collector-E
WBSearchBot Wallpapers
WISENutbot
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
WallpapersHD WallpapersHD
Web Auto Web Enhancer
Web Fetch
Web Fuck Web Fuck
Web Sauger Web Sauger
WebAuto WebAuto
WebCollage WebCollage
WebCopier WebLeacher
WebEnhancer WebReaper
WebFuck WebStripper
WebGo IS
WebSucker WebSucker
WebWhacker WebWhacker
WebZIP
Webalta Webalta
WebmasterWorldForumBot WebsiteExtractor
Webshag Whatweb
Whacker
Widow Widow
WinHTTrack WinHTTrack
WiseGuys Robot Xaldon WebSpider
Wotbox
Xaldon_WebSpider
YoudaoBot YoudaoBot
Zermelo Zade
Zeus Zitebot
ZmEu ZmEu
ZoominfoBot
ZyBorg
adscanner adscanner
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
backlink-check arquivo.pt
cah.io.community autoemailspider
coccocbot
crawl.sogou.com crawl.sogou.com
crawler4j
dataforseo.com
demandbase-bot demandbase-bot
gopher facebookscraper
heritrix ips-agent
instabid isitwp.com
internetVista monitor
iubenda-radar iubenda-radar
mediawords lwp-request
netEstate NE Crawler lwp-trivial
page scorer meanpathbot
muhstik-scan
oBot
ripz ripz
satoristudio.net s1z.ru
scan.lol scan.lol
seoscanners seocompany.store
seostar seostar
serpstatbot
sexsearcher sexsearcher
sitechecker.pro sitechecker.pro
sogouspider
spyfu
trendiction.com trendiction.com
voyagerx.com trendiction.de
ubermetrics-technologies.com
webmeup-crawler
webpros.com
webprosbot webprosbot
x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe xpymep1.exe
zauba.io zauba.io
zgrab

302
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
404enemy 404enemy
ASPSeek 80legs
AfD-Verbotsverfahren ADmantX
Aipbot Aboundex
Alexibot Acunetix
Alligator AhrefsBot
AiHitBot
AllSubmitter
AlphaBot
Anarchy Anarchy
Anarchy99 Anarchy99
Apexoo Anthill
Aspiegel
Atomseobot
Attach Attach
AwarioRssBot BBBike
BackStreet BDFetch
Backlink-Ceck BLEXBot
Badass BackDoorBot
Bandit BacklinkCrawler
Barkrowler
BatchFTP BatchFTP
Black Hole Black Hole
Bolt Blackboard
Blow
BlowFish
BotALot
BuiltBotTough BuiltBotTough
BuiltWith BuiltWith
Bullseye Bullseye
CATExplorador BuzzSumo
CCBot Cegbfeieh
CODE87 CheTeam
CSHttp Chlooe
Calculon
CazoodleBot
CensysInspect
ChinaClaw
Citoid
Claritybot Claritybot
Cliqzbot Cliqzbot
Cloud mapping
Cogentbot
CopyRightCheck
Cosmos Cosmos
Craftbot
Crawling at Home Project Crawling at Home Project
Crescent Custo
CyotekWebCopy
DBLBot DBLBot
DIIbot DataCha0s
DTS Agent Deusu
Digincore Digincore
Disco Discobot
Discoverybot
Dispatch Dispatch
DnBCrawler-Analytics DomainCrawler
DomCopBot DomainSigmaCrawler
DomainAppender Domains Project
DomainStatsBot
Dotbot Dotbot
EMail Siphon Download Wonder
Ecxi ECCP/1.0
EirGrabber EirGrabber
EroCrawler
Exabot Exabot
Extractor Extreme Picture Finder
EyeNetIE FDM
Ezooms FHscan
FemtosearchBot FemtosearchBot
Foobot Fimap
FrontPage FlashGet
FyberSpider Flunky
G-i-g-a-b-o-t Fuzz
GT::WWW
GalaxyBot
Genieo Genieo
GermCrawler Gotit
GetRight GrapeshotCrawler
Go-Ahead-Got-It HMView
Grabber
GrapeFX
GridBot
HTMLparser HTMLparser
HTTP::Lite HTTP::Lite
HTTrack Harvest
Havij Heritrix
Humanlinks Hloader
IDBot HybridBot
IDBTE4M
IRLbot IRLbot
Image Fetch Iblog
IlseBot
Image Sucker
IndeedBot
Indy Library
InterGET InterGET
InternetSeer Internet Ninja
Iria Iria
IstellaBot IstellaBot
JamesBOT JamesBOT
JennyBot JetCar
JikeSpider
Joomla Joomla
Jorgee JustView
Kenjin Spider Jyxobot
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Keyword Density
Kinza Kinza
Kozmosbot Kozmosbot
LNSpiderguy LNSpiderguy
Lanshanbot Larbin
LibWeb LeechFTP
Libwhisker LeechGet
Likse Lightspeedsystems
LinkScan LinkScan
Linkbot
LinkextractorPro LinkextractorPro
LinkpadBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Lmspider
Ltx71 Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MQQBrowser MQQBrowser
MSIECrawler MSFrontPage
Majestic SEO MTRobot
Majestic-SEO Magnet
Mail.RU_Bot
Majestic12 Majestic12
MarkMonitor MarkMonitor
Mass Downloader
Masscan Masscan
Mata Hari MegaIndex.ru
Metauri Mojolicious
Microsoft URL Control MolokaiBot
Minefield
Mojeek
Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3 Msrabot
Musobot Musobot
NICErsPRO NICErsPRO
Name Intelligence Name Intelligence
Navroad Navroad
NearSite
NetAnts NetAnts
NetMechanic NetLyzer
NetSpider Nettrack
NetZIP
Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler Nibbler
Ninja Niki-bot
Octopus Nimbostratus
Offline Explorer Nuclei
Openfind OnCrawl
Openvas
OrangeBot
OutclicksBot OutclicksBot
PECL::HTTP
Page Analyzer Page Analyzer
PageAnalyzer
Panscient
Papa Foto Papa Foto
Pavuk
Petalbot
Pi-Monster Pi-Monster
PictureFinder Picsearch
Pixray Pimonster
PleaseCrawl
Probethenet
QueryN Metasearch QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot RankFlex
RankActiveLinkBot
RankingBot RankingBot
RankingBot2 Rankivabot
ReGet ReGet
RealDownload
Reaper
RebelMouse RebelMouse
Recorder RedesScrapy
SEOkicks RepoMonkey
SEOkicks-Robot SBIder
SEOlyticsCrawler SEOlyticsCrawler
SISTRIX
ScanAlert ScanAlert
Scanbot
SearchmetricsBot
Seekport
SemanticJuice SemanticJuice
Semrush
Seomoz Seomoz
Shodan Sitebeam
Siphon SlySearch
Site Sucker
SiteLockSpider
SiteSucker
Sitevigil
SmartDownload SmartDownload
Snapbot SocialRankIOBot
Sosospider Sogou web spider
Spanner Sottopop
Sqlworm SpankBot
SputnikBot
Sqworm Sqworm
Stripper Stripper
Sucker
Sucuri Sucuri
SuperHTTP SuperHTTP
Surfbot Surfbot
T0PHackTeam SurveyBot
T8Abot Suzuran
Teleport Szukacz
Thumbor TeleportPro
TightTwatBot The Intraformant
Tracemyfile Titan
Turingos Turingos
URLy Warning Turnitin
TurnitinBot
URLy.Warning
UnisterBot
V-BOT V-BOT
VB Project
VCI VCI
Vacuum Vacuum
VelenPublicWebCrawler Vagabondo
VeriCiteCrawler
Virusdie Virusdie
Voltron Voil
WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV WEBDAV
WWW-Mechanize
WWWOFFLE WWWOFFLE
Wallpapers Wallpapers/3.0
Web Fuck Web Auto
Web Pix Web Collage
Web Fetch
Web Sauger
Web Sucker Web Sucker
WebBandit
WebCopier WebCopier
WebFetch WebFetch
WebFuck WebPix
WebReaper WebStripper
WebSauger WebWhacker
WebSucker WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Webshag
Whack Website Quester
WebsiteExtractor
Whacker Whacker
Widow Widow
WinHTTrack WinHTTrack
Wotbox Wonderbot
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider
Zade Zade
Zauba
Zermelo
Zitebot
ZmEu ZmEu
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt
autoemailspider autoemailspider
cah.io.community clark-crawler
cognitiveseo cognitiveseo
crawl.sogou.com com.plumanalytics
crawler.feedback crawler.feedback
crawler4j demandbase-bot
evc-batch
heritrix heritrix
internetVista monitor instabid
isitwp.com
lwp-trivial lwp-trivial
magpie-crawler
meanpathbot
mediawords
oBot
page scorer
pcBrowser pcBrowser
plumanalytics
polaris version polaris version
scalaj-http
scan.lol
seocompany.store seocompany.store
seoscanners seoscanners
serpstatbot seostar
siteripz spyfu
trendiction.com
trendiction.de trendiction.de
webmeup-crawler webpros.com
zauba.io x09Mozilla
zgrab xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
alyeskaresort.com avtovolop.ru
binomo.com buy-forum.ru
get-free-traffic-now.com candelluxsklep.pl
javidol.site f-online.de
ledpolice.ru get-seo-domain.com
marcowebonyodziezowe.pl hopeonthestreet.co.uk
oil-td.ru photosaga.info
paramountmarble.co.uk torrents.cd
stuffpride.com ultimateclassicrock.com
veselokloun.ru zapatosenventa.info

928
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
123kuma.com 0912701309f8ce.com
1j7740kd.website 0daymusic.org
100dollars-seo.com
1314dh.com
13tabs.com
18ps.ru
1adult.com
1xbet4you.com
2020iscoming.info
2girls1cup-free.com
34.gs 34.gs
45en.ru 7fon.ru
6128786.com a1security.com.ua
777-club.ru abcguru.xyz
aad1f4acb0a373420d9b0c4202d38d94fa.com adimmix.com
abclauncher.com adpremium.org
abovetherivernc.com adultfullhd.com
addbin.men adultwebhosting.info
adexprts.com advertisingtag.net
adloads.net
adsref.men
adtology.com
adverster.com
affrh2015.com
afslankpillen2017nl.eu afslankpillen2017nl.eu
agahinameh.com agapovdg.ru
aghanyna.com akama.com
alert.scansafe.net albamargroup.com
alice.it
alif-ba-ta.com alif-ba-ta.com
allesohnegirls.net allkrim.com
alphacarolinas.com alloysteel.ru
alpinism.ru alltheviews.com
ameblo.top allwomen.info
amt-k.ru am15.net
anime.dougasouko.com amehdaily.com
animenime.ru angel1777.kz
anjalika.co.in animebox.com.ua
apparel-offer.com aosexkontakte.net
arcteryxstore.online apibrowsesmartne-a.akamaihd.net
articlesdirectoryme.info apimossnetinfo-a.akamaihd.net
arvut.org apps-analytics.net
asdfg.pro appsaurus.com
architecturebest.com
arkkivoltti.net
ascat.porn
asiengirls.net asiengirls.net
asseenontvonline.ru atdedinotuho.tk
atovh.local-finders.com atleticpharm.org
autovideobroadcast.com aucoinhomes.com
autowebmarket.com.ua autobrennero.it
avkzarabotok.com autonew.biz
avtovolop.ru autoplate.info
bankcrediti.ru aviapanda.ru
banki76.ru aviav.org
bankiem.pl avon-severozapad.ru
bannerads.de azartmsl.com
bannerpower.com azartniy-bonus.com
bannerspace.com azbukadiets.ru
beachtoday.ru azinoofficial777.ru
bedcapdealers.com babespcs.com
begalka.xyz bagcionderlab.com
best-coupon-offer.com balitouroffice.com
beztuberkuleza.ru bambi.ck.ua
bikini-image.com basisches-wasser.net
bio-japan.net battlecarnival.su
blog.f00kclan.de battleforupdating.site
blog.koreadaily.com beachdriveblog.com
bloggers.nl benchmarkcommunications.co.uk
bensbargains.net
berrymall.ru
bestimagecoollibrary.com
besttorrentknifta.weebly.com
betterhdporn.com
bigpenisguide.com
binaryoptionscops.info
bizlist.com.de
bizzliving.com
bkgr.se
blogqpot.com blogqpot.com
bluerobot.info blogrankers.com
bmsco.co bluejays-jerseys.us
brainxs.us book-bets.com
brandbucket.com brainboosting.club
brillianty.info brakehawk.com
bristolhotel.com bscodecs.com
bronzeaid-a.akamaihd.net
buigas.00it.com
bumascloset.com bumascloset.com
burn-fat.ga bumskontakte.org
buron.pw buttons-for-your-website.com
buy-meds24.com buypuppies.ca
buypanicdisorderpill.com camsex.xxx-cam.webcam
bycontext.com cartierreplica.top
call-of-duty.info cbb1smartlist12.click
callawaygolfoutlet.online centrumcoachingu.com
candypeople.se cgi2.nintendo.co.jp
candytiner.com chatseo.com
canrole.com chimiver.info
canzoni.ru chinese-amezon.com
carina-sy.de chocolateslim-en-france.com
carstrends2015.com ci.ua
cashonads.com cicaki.net
cheapmoncler.xyz cindymatches.com
checktext.org cjs.com.ru
choice-credit.ru clksite.com
cineacademy.ru clodo.ru
clean-virus-mac.com compliance-fred.top
clickso.com compliance-irvin.top
clicksor.net
coccoc.com
coinsspb.com
collegerentals.com
compliance-irvin.xyz compliance-irvin.xyz
compliance-julianna.top comsysnet.com
conboy.us
conocer-sanabria.com conocer-sanabria.com
constantaservice.net cool-wedding.net
construmac.com.mx coolwallpapers4k.info
cookie-law-enforcement-aa.xyz costablue.xyz
counterbot.com counter.yadro.ru
credit-respect.ru covetnica.com
customchocolate.business-for-home.com culad.com
customerguru.in cymbaltaandtizanidine.blogspot.com
customerpromos-a.akamaihd.net czat.wp.pl
cutt.us daneshetabiat.com
cxpromote.com darkbooks.org
cyclobenzaprinevstizanidine.blogspot.com
dailyrank.net
darmebel.com.ua
dating-time-now.com dating-time-now.com
datsun-do.com datsun-do.com
davebestdeals.com daydream-studio.ru
ddospanel.com dayibiao.com
deplim.com dentalpearls.com.au
descargar-musica-gratis.net deshevo-nedorogo.ru
dgfitness.co diegolopezcastan.com
dieta-personalna.pl digest-project.ru
div.as dimfour.com
dokfilms.net divku.ru
domainsphoto.com dnsrsearch.com
downloadmefiranaratb1972.xpg.com.br documentsite.net
dvd-famille.com domainanalyzing.xyz
e-kwiaciarz.pl domoysshop.ru
e2click.com dostavimvdom.ru
eaplay.ru dosug-lux.ru
easyukraine.com download-wallpaper.net
ecommerce-seo.org droidlook.net
ekn-art.se dtnlyss.com
elektrischezigaretten1.blogse.nl dvervmoskvu.ru
elstal.com.pl dvrlists.com
emediate.eu eandsgallery.com
empis.magix.net ec-file.info
ednorxmed.com
efnor-ac.com
egvar.net
el-nation.com
election.interferencer.ru
elektrozigaretten1.postbit.com
elektrozigaretten1.tumblr.com
eloxal.ru
emergencyneeds.org
energydiet-info.ru energydiet-info.ru
englate.com enhand.se
esopini.com eredijovon.com
espaceinventoristes.com escortplius.com
este-line.com.ua eshop4u.jp
european-torches.ru essay-edu.biz
expertblog.info europages.com.ru
extremal-blog.com eurosamodelki.ru
eywords-monitoring-your-success.com euroskat.ru
f07.de extrabot.com
fdzone.org ezigarettekaufen1.hpage.com
fikasound.tk ezigarettenshop1.yolasite.com
filerockstar298.weebly.com facialporntube.com
filkhbr.com fandlr.com
filmetricsasia.com farmingworm.com
findanysex.com felizporno.com
finteks.ru filesmonster.porn
flprog.com fillmewithhappiness.com
forsex.info financeloan.us
fototravel.eu firstdrugmall.ru
free-deals.faith flowersbazar.com
free-fbook-traffic.com forexgb.ru
freelotto.com forzeronly.com
fusoradio.info frbizlist.com
futbolkisales.ru freeuploader.ml
g33.org funnymama.com
games.kolossale.ru funtoonez.com
gamewrath.com galaxy-family.ru
gaming-journal.com galeria-zdjec.com
gaypornmovie.net gaytube.com
gdzkurokam.ru germetiki.com.ua
ge0ip.org getmiro.com
gearcraft.us getpopunder.com
geileweiber.tk gilsonchiro.xyz
getaclueamerica.com glavtral.ru
getlaid-xxxhookupdirect.com golfresa.lucania.se
getprismatic.com gosarhivrt.ru
gettpromos.com graphics8.info
ghazel.ru gratuitxblcodes.com
gheus.altervista.org
gigixo.com
gk-atlant.info
gkvector.ru
global-ics.co.za
gogalleryawesome.com
gopixdatabase.com
gratuitbaise.com
gsasearchengineranker.xyz
gta-club.ru
guardlink.org guardlink.org
gungamesz.com hd720kino.ru
hackers-crackers.tk hdimagelib.com
handicapbathtubarea.com hdwallpapers-free.com
hanink.biz.ly healing-dysplasia.ru
hannasolution.ru heartofbeijing.blogspot.com
hdapp1008-a.akamaihd.net hebr.myddns-flir.com
hdpixion.com high-speed1.net
heartofpayne.xyz highland-homes.com
hesteel.pl
histock.info histock.info
hit-men.men home-task.com
hitsbox.info homegardenlova.com
hkdiiohi.skyrock.com homemature.net
host-protection.com hop.clickbank.net
hostnow.men hostnow.men
hotblognetwork.com houseofgaga.ru
hotsocialz.com hoverboardforsaledirect.com
hunterboots.online icaseclub.ru
husky-shop.cz iframe-toloka.com
hut1.ru
ideibiznesa2015.ru
iinstalll-fii1leis.jus0wil.pp.ua iinstalll-fii1leis.jus0wil.pp.ua
infobanks.ru ilikevitaly.com
interferencer.ru iloveitaly.ru
irkutsk.zrus.org ilovevitaly.com
it-max.com.ua imgarcade.com
japfm.com impisr.ru
javcoast.com importchinacoach-teach.com
javstock.com incanto.in.ua
jbl-charge.info inclk.com
jerseychinabizwholesale.us indianmedicaltourismshop.com
jose.mulinohouse.co infokonkurs.ru
karachev-city.ru investingclub.ru
iqupdatetmz.win
itag.pw
itis4you.com
javxxx18.com
jolic2.com
jongose.ninja
junglenet-a.akamaihd.net
jyvopys.com
kam-dom.ru
katjimej.blog.fc2.com
ketoanhanoi.info ketoanhanoi.info
keywordsking.com kidskunst.info
kino-fun.ru
kinobest.pl kinobest.pl
kladrus.ru kinogonew.ru
komp-pomosch.ru kollesa.ru
kretpicf.pw komputers-best.ru
kuzinsp.ru konkursov.net
lakomka.com.ua koversite.info
ldrtrack.com kredit-blog.ru
lenpipet.ru kwzf.net
lanadelreyfans.us
landscaping.center
laudit.ru
lflash.ru
linksharingt.com linksharingt.com
livefixer.com linkwithin.com
liveshoppersmac.com lion.misba.us
lockerz.com littleberry.ru
lordzfilmz.me localmatchbook.com
lsp-awak-perikanan.com lodki-pvh.dp.ua
m.b00kmarks.com lookingglassemb.com
macdamaged.tech louboutinbooties.xyz
machicon-akihabara.info m0r0zk0-krava.ru
macrotek.ru m1media.net
mainlinehobby.net macdamaged.space
makis.nu magicdiet.gq
manifestation.betteroffers.review makedo.ru
manifestyourmillion.com marcoislandvacations.net
mansparskats.com marketingtechniques.info
mapquestz.us masserect.com
marcogrup.com maxxtor.eu
matsdale.com medizinreisen.de
maximpartnerspr.com megakino.net
mcadamssupplyco.com memberty.com
mcnamaratech.com metallosajding.ru
meddesk.ru methodsmarketing.com
medispainstitute.com.au micasainvest.com
meetlocalchicks.com mixtapetorrent.com
mejoresfotos.eu
meltwater.com
mesbuta.info
militarysale.pro
minet.club
mlf.hordo.win mlf.hordo.win
mncrftpcs.com mobile.ok.ru
mnogabukaff.net moinozhki.com
mobot.site more-letom.ru
mobplayer.net moroccosurfadventures.com
moesen-ficken.com mosdverka.ru
monclervests.xyz mp3ringtone.info
money-for-placing-articles.com mrwhite.biz
monitorwebsitespeed.com mturkcontent.com
moomi-daeri.com musezone.ru
morepoweronmac.trade musicas.baixar-musicas-gratis.com
mozello.ru muztops.ru
mrsdalloways.com my-aladin.com
muizre.ru mycaf.it
mundoaberrante.com mydirtyhobby.com
music.utrolive.ru myftpupload.com
mybackgroundlandscape.blogspot.com myshopmatemac.com
mylida.org mzdish.site
nagdak.ru na-telefon.biz
nalogovyy-kodeks.ru naperehresti.info
namecrumilchlet.tk nardulan.com
narco24.me
naturalbreakthroughsresearch.com
naturalshair.site naturalshair.site
negociosdasha.com naughtyconnect.com
negral.pluto.ro neodownload.webcam
new-apps.ru niroo.info
nfljerseyscheapbiz.us o-o-6-o-o.com
noclegonline.info
nofreezingmac.click
nosecret.com.ua
o333o.com
obsessionphrases.com obsessionphrases.com
oktube.ru obuv-kupit.ru
onescreen.cc olympescort.com
openfrost.com oneclickfiles.com
onlinemegax.com
onlythegames.com
openlibrary.org
openmultipleurl.com
ops.picscout.com ops.picscout.com
optibuymac.com paparazzistudios.com.au
optom-deshevo.ru parajumpersoutlet.online
oralsexfilme.net parfusale.se
orion-code-access.net partner-high.men
orsonet.ru partner-stop.men
ozoz.it pdamods.ru
paccohichetoti.ml
paighambot.com
paleohub.info
pay2me.pl
paydayloanslocal.com
pdfprof.com pdfprof.com
perfection-pleasure.ru pdns.cz
perfectplanned.com perederni.net
pictures-and-images.net perfectpracticeweb.com
perso.wanadoo.es
pestomou.info
pewit.pw
php-market.ru
picsfair.com
pihl.se
pijoto.net pijoto.net
pinwallpaper.top pillscheap24h.com
piratecams.com piulatte.cz
piski.top
pk-pomosch.ru pk-pomosch.ru
pl-top.pl
placid-rounded-coast.glitch.me placid-rounded-coast.glitch.me
plastgranar.nu
playlott.com
pliks.pl
ploenjitmedia.azurewebsites.net
podshipniki-nsk.ru
poisk-zakona.ru poisk-zakona.ru
pokerniydom.ru pokemon-go-play.online
pons-presse.com polska-poezja.com
porngalleries.top pontiacsolstice.info
pornodojd.ru popads.net
pornosee.info popmarker.com
pornosemki.info popup-jdh.xyz
pornosok.ru pornoelita.info
portal-eu.ru powenlite24.ru
portside.xyz prchecker.info
pr-ten.de princevc.com
privacylocationforloc.com printdirectforless.com
prizefestival.mobi profitsport.club
prlog.ru projefrio.com.br
prodess.ru prostitutki-tolyatti.org
proekt-mos.ru psoriasis-file.trade
proprostatit.com
prostitutki-astana.org
prostitutki-kiev.org
pst2017.onlinewebshop.net
pulse33.ru
pulseonclick.com pulseonclick.com
pussysaga.com purchasepillsnorx.com
pxhdwsm.com quick-offer.com
qor360.com quickloanbank.com
qpypcx.com
qwertty.net
randalljhoward.com randalljhoward.com
rapevideosmovies.com rapidokbrain.com
redirectingat.com rebrand.ly
registrationdomainsite.com recipedays.com
reklama-i-rabota.ru remedyotc.com
remont-fridge-tv.ru
remontgruzovik.ru
renewablewealth.com
rentaremotecomputer.com
replica-watch.ru replica-watch.ru
replicaclub.ru
rethinkwasteni.info rethinkwasteni.info
rfd-split.hr rezeptiblud.ru
rialp.getenjoyment.net rimedia.org
romhacking.ru robot-forex.biz
rospromtest.ru ros-ctm.ru
rossanasaavedra.net royal-investments.net
rubanners.com rp9.ru
ruspdd.com ru-mediaget.ru
russintv.fr ruscoininvest.company
rybalka-opt.ru ruscopybook.com
sa-rewards.co.za rusoft-zone.ru
sack.net russkie-gorki.ru
sajatvelemeny.com s-iwantyou.com
sammyweaver.com sankt-peterburg.nodup.ru
saneyes.com sbf441.com
satoristudio.net sbt-aqua.ru
scanner-mary.top score-ads.men
se.bnt-team.com search.xtconnect.com
seasaltwithfood.com searchkut.com
seeingmeerkat.com semalt.com
selectads.men semp.net
seoexperimenty.ru sendearnings.com
seorankinglinks.xyz seolab.top
serialsway.ucoz.ru servingnotice.com
sexblog.pw sex-sex-sex5.com
sexobzor.info shakhtar-doneck.ru
sextracker.com shama-rc.net
sexy-screen-savers.com sharebutton.org
share-buttons-for-free.com shoesonlinebuy.xyz
shikiso.info shoplvlv.us
sigmund-freud.co.uk shoppytoolmac.com
sim-service.net shymkent.xkaz.org
simul.co sibtest.ru
sinel.info siha.de
sirpornogratis.xxx sindragosa.comxa.com
sitopreferito.it skanninge.se
smartbalanceworld.com skincrate.net
smartshoppymac.com smart-balancewheel.com
snomer1.ru socialbuttons.xyz
soblaznu.net socialmonkee.com
soc-proof.su sockshares.tv
social-s-hhh.xyz soda.media
social-vestnik.ru soft1.ru
socialmadesimple.com softwaretrend.net
sogimlecal.tk sogimlecal.tk
solnplast.ru songoo.wz.cz
sosdepotdebilan.com sonnikforme.ru
space2019.top sophang8.com
speeddream.xyz spywarenuker.com
spylog.com spywarespy.com
stats-public.grammarly.io strigkaomsk.ru
staynplay.net stroicol.net
steelmaster.lv stroilka.info
stillmiracle.com suggest-keywords.com
stockspmb.info swinger-mobil.net
stopnarco.ru sygraem.com
strana-krasoty.ru tampabaywatch.org
strfls.com tanieaukcje.com.pl
stroiminsk.com tattooreligion.ru
stroyhelp-dv.ru technika-remont.ru
summerlinhomes411.info telefonsexkostenlos.tk
superinterstitial.com testingads.pro
svetodiodoff.ru texbaza.by
t-bygg.com tgtclick.com
takethatad.com theendivechronicles.com
teenforporn.com thefarmergame.com
teenfuck.tv theporndude.com
tehngr.ru thewomenlife.com
telemetryverification.net timer4web.com
tengohydar.tk tizanidinedosage.blogspot.com
tetracsaudi.com tizanidinedrugtest.blogspot.com
thetattoohut.com topdownloads.ru
thiegs.reco.ws
tizanidine6mg.blogspot.com
tizanidinecapsules.blogspot.com
tizanidinehcl4mgtabinfo.blogspot.com
todohr.com
top10-online-games.com
topflownews.com topflownews.com
track-rankings.online torontoplumbinggroup.com
trafaret74.ru torrent-to-magnet.com
trafficjunky.com tovaroboom.vast.ru
trafficjunky.net trackmedia101.com
trafficreceiver.club tracksurf.daooda.com
trailer.cinemaflix.website traffic-club.info
trichizobswiv.agddns.net trk-4.net
twojebook.pl trubywriting.com
tyumen.xrus.org trumpetedextremes.com
uchebavchehii.ru turn-up-life.life
tversvet.ru
txxx.com
tytoona.com
ucoz.ru ucoz.ru
umekana.ru uhod-za-sobakoj.ru
upproar.com ukrobstep.com
uptimechecker.com ultimateclassicrock.com
urlcut.ru unece.org
urll.eu unionmarkt.de
userequip.com unmaroll.ya.ru
uzporno.mobi uptime.com
vadimkravtcov.ru vacances-voyages.info
valmetrundan.se varbergsvind.se
valuado.com varikozdok.ru
vancleefreplica.pw vegascosmetics.ru
vclicks.net ventelnos.com
veles.shop via-energy-cumpara.com
video-chat.in viagra-soft.ru
video-production.com videochaty.ru
videos-for-your-business.com videos-for-your-business.com
videtubs.pl videositename.com
vip2ch.com vip-parfumeria.ru
voluumtrk.com visa-pasport.ru
vzglyadriv.kg vitalads.net
wallpaperaccess.com viteonlusarezzo.it
wallpapersist.com vizag.kharkov.ua
vladhistory.com
vodkoved.ru
voltrknc1.com
vroze.com
vseuznaem.com
vzlomtw.com
wait3sec.org
wareseeker.com
watchdogs-2.ru watchdogs-2.ru
waterefficiency.co website-audit.com.ua
webadvance.club wesharepics.site
weblo.com wetgames.ru
webmasterhome.cn wieseversa.no
websiteexplorer.info wjgony.com
wedding-salon.net wnoz.de
wemedinc.com woman-h.ru
wesharepics.com woman-tampon.ru
westen-z.life wordpress-crew.net
whereiskentoday.com woweb.com.ua
wildcattube.com ww1943.ru
wladimirpayen.com www.solartek.ru
wleuaprpxuvr.ga xbaboon.com
word-vorlagen.net xerox-douglas.cf
wovis.site xfluro.com
x-lime.com xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
x-rates.ru xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
x-stars.ru xn--d1aifoe0a9a.top
xmlinde.com xtube.com
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai xxxasianporn.net
xn----7sbifcamovvfggw9d.xn--p1ai xxxdatinglocal.us
xn----btbdvdh4aafrfciljm6k.xn--p1ai xxxmania.top
xn--80ajbshivpvn2i.xn--p1ai zaim-pod-zalog-krasnodar.ru
xn--h1ahbi.com.ua zakon-ob-obrazovanii.ru
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai zero1.it
xtremeeagles.net zerocash.msk.ru
yadro.ru zigarettenonl.canalblog.com
yougetsignal.com zone-kev717.info
youporn-ru.com zoodrawings.com
zeikopay.com zscalerone.net
znakom.sibtest.ru
zooggames.com
zoompegs.com
zootoplist.com
zynax.ua
zzbroya.com.ua

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
0c47f8422d3f.com 123movies.love
12masterov.com 14b.info
2015god.org 1bet.com
202ch.com 1qingdao.com
404.mn 4replicawatch.net
5ws.dating-app.ru 66cpwgln.space
9icmzvn6.website 8xv8.com
a96527gi.beget.tech aa9d046aab36af4ff182f097f840430d51.com
aadroid.net academiacsmendoza.org
abcdeg.xyz acheterviagrafr24.com
actulite.com
adbetclickin.pink adbetclickin.pink
afftrack001.com adexprts.com
agecheckadult.com aktobe.xkaz.org
aktivator-windows10.blogspot.com alert.scansafe.net
albumsuper.info alfa-img.com
alibestsale.com alfa-medosmotr.ru
amazingninja.com algarveglobal.com
amazingpic.net ameblo.top
apper.de amospalla.es
apple.com-webbrowsing-security.science anaseracresar.tk
applicationg29.com arenda-avtoprokat-krasnodar.ru
arcteryxsale.online artpicso.com
ardimobileinfo.ml atout-energie-69.com
arendas.net automatic-seo.com
arkkivoltti.net autorn.ru
arraty.altervista.org azte.ch
atlasvkusov.ru baersaratov.ru
b00kmarks.com bestpornuha.com
banan.tv
beauty-b0x.pl
benchmarkcommunications.co.uk
bestbuy.ca
bezsporno.ru
bg6s0.com bg6s0.com
billiard-classic.com.ua big-cash.net
bimatoprost-careprost.com.ua bioca.org
bitcoinpile.com bmsco.co
blackbabesporn.com brainxs.us
blockety.co bristolhotel.com
bobinoz.com bvlgariwallet.xyz
bradleylive.xyz cartierloveringreplica.xyz
bronzeaid-a.akamaihd.net celebsopics.com
buttons-for-website.com chcu.net
buyantiviralwp.com check-this-out-now.online
cablecar.us chimiver.info
cartierreplica.top chorus.terakeet.com
cejewelry.xyz
centraletermice.us
cheapmoncler.win
cheapmoncler.xyz
christianlouboutinreplica.win
cibpenonptib.flu.cc
ciekawinki.pl
cityadspix.com cityadspix.com
clickhype.com citysecurity.nu
clickprotects.com clash-clans.ru
com-onlinesupport.site coaching-netz.info
compliance-donald.xyz communisave.co.za
compliance-julia.top connectingsingles.com
compliance-walt.top cookielawblog.wordpress.com
coolwallpapers4k.info cpabegins.ru
creatives.adbetclickin.pink customcatchcan.com
cscwtalkto.site cv.wallhade.co
dairyindia.in dailystrength.org
daydream-studio.ru daptravel.com
dildofotzen.net data-mining.tk
discovertreasure-a.akamaihd.net dengi-pod-zalog-nedvizhimosti.ru
doska-vsem.ru diamond-necklace.info
dvd-famille.com directivepub.com
eduserver.net dojki-hd.com
ege-russian.ru domain.webkeyit.com
elite-sex-finders.com dostavka-v-ukrainu.ru
elvenar.com dumuelave.xyz
empathica.com edudocs.net
epngo.bz elektrischezigarette2.onsugar.com
eropho.com erot.co
errorfixing.space etur.ru
eugenevaultstorage.com extremez.net
f-loaded.de fidelityfunding.com
farmingworm.com foxinsocks.ru
fashionavenuegame.com freelifetimefuckbook.com
felizporno.com fun2cell.net
filmi-v.online fungirlsgames.net
finstroy.net funnymama.com
florida-tourism.net gamezblox.com
formulaantiuban.com ge0ip.com
fr-bearings.ru get-free-traffic-now.com
frcls.fr gifspics.com
fsalas.com gk170.ru
fuckmill.com glopages.ru
game-mmorpg.net gombita.info
gay-site.store gwebtools.com
gaypornmovie.net hasshe.com
geckoandfly.com hdpixent.com
get-free-social-traffic.com helvetia.com.ua
get-your-social-buttons.info hmywwogw.bloger.index.hr
gettpromos.com host-tracker.com
ghernnqr.skyrock.com
gigapeta.com
golden-catalog.pro
googlsucks.com
goosefishpost.bid
grtyi.com
gwhwpxbw.bloger.index.hr
healbio.ru
heartofpayne.xyz
heliko.no
hit-kino.ru
hitsbox.info
homedo.fabpage.com
homelygarden.com
hoztorg-opt.ru
humanelydrew.com
hystersister.com hystersister.com
igrovyeavtomaty777.ru i-service.kz
inmate-locator.us impotentik.com
interfucks.net inome.com.ua
invitefashion.com integritylandscapeservices.com
iyasimasennka.com interferencer.ru
jav-idol.com iqbazar.ru
javatex.co.id itsdp3.com
joingames.org joy-penguin.com
jpcycles.com kadashihotel.com
kellyonline.xyz keyhantercume.com
kinoflux.net keywordblocks.com
kollekcioner.ru kino2018.club
komputernaya-pomosh-moscow.ru kinohit1.ru
kurbappeal.info kleine-titten.biz
languagecode.com klin3952.ru
laxob.com kountrylife.com
liveinternet.ru krynica.info
loftdigital.eu kumuk.info
kung-fu-ru.com
kvartiry-remont.ucoz.ru
lanasshop.ru
lapolis.it
law-check-two.xyz
lesbian.xyz
live-sexchat.ru
longgreen.info
lost-alpha.ru lost-alpha.ru
lostfilm-online.ru magicdiet.gq
mafa-free.com makis.nu
mafcards.ru meble-bogart.info
magento-crew.net medmajor.ru
maju.bluesalt.co mobicover.com.ua
makenahartlin.com mobplayer.ru
master-muznachas.ru moetomnenie.com
maxthon.com mojpregled.com
meduza-consult.ru mompussy.net
mezaruk.info moncleronline.xyz
millionare.com mostantikor.ru
monclerboots.xyz mrcsa.com.au
moomi-daeri.com mvpicton.co.uk
my-bc.ru
mybackgroundlandscape.blogspot.com
myperiod.club myperiod.club
myprintscreen.com naperehresti.info
naughtyconnect.com narkologiya-belgorod.ru
nextrent-crimea.ru
nofreezingmac.work nofreezingmac.work
obnal.org oakleyglassesonline.us
oil-td.ru oddamzadarmo.eu
online-zaymy.ru onlinadverts.com
ooomeru.ru online-x.ru
opinionreelle.com onlinedomains.ru
orion-code-access.net onlyforemont.ru
outshop.ru ooo-olni.ru
oxotl.com orion-v.com
panamaforbeginners.com panicatack.com
panchro.xyz partnerworkroom.men
par-fallen.ga petrovka-online.com
partner-trustworthy.men picturesmania.com
perosan.com pinsdaddy.com
photokitchendesign.com poptool.net
photorepair.ru popup-fdm.xyz
platinumdeals.gr pornoklad.net
porndairy.in pornotubexxx.name
pornowarp.info pornozhara.com
positive2b.ru predominant-invent.tk
privatamateure.com pricheskaonline.ru
prize44.com privhosting.com
prostitutki-odessa.org promover.org
prostitutki-yaroslavlya.org psvita.ru
ptr.ruvds.com pukaporn.com
pxhdwsm.com puteshestvennik.com
qualitymarketzone.com puzzleweb.ru
quickbuck.com qexyfu.bugs3.com
rangoman.date quality-traffic.com
rasteniya-vs-zombi.ru rethinkwasteni.info
rcb101.ru reward-survey.net
realizmobi.com rosbalt.com.ua
revistaindustria.com ruscoininvest.company
sadaholding.com ruspdd.com
satellite.maps.ilovevitaly.com s-forum.biz
sbprabooks.com sa-rewards.co.za
scanner-margo.top scanner-john.top
sdi-pme.com
searchlock.com
securesmrt-dt.com
seksvideoonlain.com seksvideoonlain.com
semaltmedia.com sendearnings.com
senger.atspace.co.uk seocheckupx.com
servingnotice.com seorankinglinks.com
sexkontakteao.info sexfreepornoxxx.com
sexspornotub.com shakhtar-doneck.ru
shodanhq.com sheerseo.com
si-unique.com shivafurnishings.com
sindragosa.comxa.com sigmund-freud.co.uk
slim.sellany.ru silktide.com
simpoed.ufop.br
snjatie-geroinovoy-lomki.ru snjatie-geroinovoy-lomki.ru
sockshare.net soblaznu.net
softomix.net sofit-dmd.ru
solinf.co solmarket.by
sotkal.lark.ru sosdepotdebilan.com
stackthatbucks.com space-worry.ml
tako3.com sportbetfair.com
tampabaywatch.org srecorder.com
tattomedia.com strongsignal-a.akamaihd.net
telesvoboda.ru strv.se
thaismartloan.com supers.com.ua
the-trader.net tam-gde-more.ru
tandvardshuset.net
tecnoteakviareggio.it
telegramdownload10.com
teletype.in
tfxiq.com
theguardlan.com
thelottosecrets.com thelottosecrets.com
ticketsys.inetwd.com timdreby.com
tizanidinehcl4mgtabinfo.blogspot.com titslove.yopoint.in
tizanidinesleepaid.blogspot.com tizanidinedrugtest.blogspot.com
tksn.ru top10registrycleaners.com
toondinsey.com topsy.com
toonfamilies.net tour-line.net
topads.men tpu.ru
torrent-newgames.com trafaret74.ru
toxicwap.com tuckpointingmasonrysystems.com
track2.shop twittruth.com
trackzapper.com twodollarshows.com
tupper-shop.ru uasb.ru
u555u.info umityangin.net
unrealcommander.com unpredictable.ga
upstore.me upupa.net
uptime.com url2image.com
veerotech.com valsalud.com
valueclick.com
vasileostrovsky-rayon.ru
velobikestock.com velobikestock.com
via-energy-acquistare.com velpanex.ru
videokrik.net vielporno.net
viktoria-center.ru vigrx-original.ru
vipromoffers.com vkgaleria.com
vladimir.xrus.org voprosotvet24.ru
warningwar.ru vseigry.fun
wesharepics.info warezkeeper.com
whatsupinfoley.com warningzscaler.heraeus.com
wingsoffury2.com webnode.me
woman-orgasm.ru worldmusicfests.com
world-mmo.com wovis.site
www.wohnkabinen-shop.de ws.ampower.me
x-lime.com xfire.com
xdoza.com xfluro.com
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xn--h1aakne2ba.xn--p1ai
xn----dtbndd4ae7eub.top xpresscare.ru
xtube.com
xvideosbay.com
xvideospornoru.com
xwatt.ru
xxxtubesafari.com xxxtubesafari.com
yellowproxy.net youradulthosting.com
yopoint.in youtoner.it
zebradudka.com ypmuseum.ru
zed21.net yurgorod.ru
zaobao.com.sg
zeroredirect7.com
zionstar.net
zscalerone.net

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.73 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3049 ### Version: V4.2022.04.3050
### Updated: Sun Apr 17 10:03:40 UTC 2022 ### Updated: Sun Apr 17 22:01:17 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3049 ### Version: V4.2022.04.3050
### Updated: Sun Apr 17 10:03:40 UTC 2022 ### Updated: Sun Apr 17 22:01:17 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##