vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 06:15:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3450 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-04 10:09:27 +00:00
parent 6bb946914d
commit 8f161873bf
46 changed files with 99925 additions and 99925 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3449 #### Version: V4.2022.11.3450
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3449 #### Version: V4.2022.11.3450
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3449 #### Version: V4.2022.11.3450
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12354
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3449 #### Version: V4.2022.11.3450
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12358
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12358
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12358
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12358
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

12358
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

12358
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

12358
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3449 3450

12358
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12358
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.214.92.133 0; 20.22.194.135 0;

12358
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

12358
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

12358
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

12358
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.214.92.133 0; 20.22.194.135 0;

12358
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.214.92.133 0; 20.22.194.135 0;

12358
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.101 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
ADmantX AiHitBot
EMail Siphon Domains Project
LinkWalker FrontPage
PyCurl Ltx71
RankingBot2 MegaIndex.ru
Tracemyfile Sitevigil
Web Fetch SurveyBot
ZyBorg Telesphoreo
sitechecker.pro WebFetch
webprosbot WebLeacher

332
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker
404enemy 404enemy
80legs ASPSeek
ALittle Client AiHitBot
Aboundex AllSubmitter
Aboundexbot Alligator
AlphaBot
Ankit Ankit
Anthill Anthill
Aspiegel Aspiegel
Asterias Asterias
Attach BDCbot
AwarioRssBot BackWeb
BBBike
BLEXBot
BackDoorBot
BacklinkCrawler BacklinkCrawler
Badass Barkrowler
Battleztar Bazinga BatchFTP
Bigfoot BlackWidow
Bitacle Brandprotect
Blackboard
Blow
Boardreader
Brandwatch Brandwatch
Buck
BuiltBotTough BuiltBotTough
Bullseye BuiltWith
CATExplorador BuzzSumo
CODE87 CCBot
CSHttp
Calculon Calculon
CazoodleBot Cegbfeieh
CensysInspect CensysInspect
CheTeam
CheeseBot CheeseBot
CherryPicker Chlooe
ChinaClaw
Citoid
Claritybot Claritybot
Cliqzbot Cliqzbot
Cloud mapping Collector
Cogentbot
Copier Copier
CopyRightCheck
Cosmos Cosmos
Crawling at Home Project CrunchBot
Crescent DBLBot
Curious DTS Agent
Custo DataCha0s
Devil DigitalPebble
Discoverybot Disco
Dispatch Dispatch
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot Dotbot
DomCopBot Download Wonder
DomainCrawler Dragonfly
DomainStatsBot ECCP/1.0
Domains Project EasyDL
Drip EroCrawler
EMail Siphon Evil
Ebingbong Exabot
Ecxi ExtLinksBot
EirGrabber Extractor
ExtractorPro ExtractorPro
Extreme Picture Finder Extreme Picture Finder
FDM EyeNetIE
FHscan FemtosearchBot
Firefox/7.0
FlashGet FlashGet
Fuzz FyberSpider
GetRight Fyrebot
Getintent
Gigabot Gigabot
Go!Zilla Go!Zilla
Go-Ahead-Got-It
GoZilla GoZilla
GridBot Gotit
HTMLparser GrabNet
Grafula
GrapeFX
GrapeshotCrawler
HEADMasterSEO
HMView
HTTP::Lite HTTP::Lite
HTTrack
Haansoft Haansoft
HybridBot Harvest
Havij
Hloader
HonoluluBot
Humanlinks
IDBot IDBot
Id-search
IlseBot IlseBot
Image Fetch
Image Sucker
IndeedBot
InfoNaviRobot InfoNaviRobot
InfoTekies Intelliseek
InterGET InterGET
InternetSeer
JOC Web Spider JOC Web Spider
JamesBOT Jbrofuzz
Jyxobot Joomla
Kinza JustView
LWP::Simple Keyword Density
Larbin Kozmosbot
LeechGet LNSpiderguy
LibWeb Lanshanbot
Lightspeedsystems LeechFTP
Linkbot LexiBot
LinksManager LinkWalker
Lipperhey LinkextractorPro
Lipperhey Spider LinqiaMetadataDownloaderBot
LinqiaRSSBot
Litemage_walker
Ltx71
MIIxpc
MJ12bot MJ12bot
MSFrontPage MQQBrowser
Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic12 Majestic-SEO
MarkWatch
MauiBot MauiBot
Mb2345Browser Mb2345Browser
Metauri Metauri
Microsoft URL Control MicroMessenger
Mojolicious Mister PiX
Morfeus Fucking Scanner Mojeek
Msrabot Msrabot
Musobot
NPbot NPbot
Nameprotect Name Intelligence
Navroad NearSite
Needle Needle
Net Vampire
NetLyzer NetLyzer
NetMechanic Netcraft
NetSpider Nettrack
NetZIP Netvibes
NextGenSearchBot Nibbler
Niki-bot Niki-bot
NimbleCrawler
Nimbostratus
Ninja
Nmap
Not
Nuclei Nuclei
Octopus Octopus
Offline Explorer OnCrawl
Offline Navigator
OpenLinkProfiler
OpenVAS
Openfind Openfind
PECL::HTTP OrangeSpider
PHPCrawl
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageGrabber PageAnalyzer
Panscient PageScorer
Pandalytics
Pavuk
PeoplePal PeoplePal
Piepmatz PictureFinder
Pimonster Pixray
PleaseCrawl PleaseCrawl
Pockey
ProWebWalker
Probethenet Probethenet
Psbot Pu_iN
Pump
PxBroker PxBroker
PyCurl RSSingBot
QueryN Metasearch
Quick-Crawler
RankActive RankActive
RankFlex RankActiveLinkBot
RebelMouse RankingBot2
Recorder RankurBot
SEOprofiler ReGet
SMTBot RepoMonkey
Rogerbot
SBIder
SEOstats
SalesIntelligent SalesIntelligent
Scrapy ScanAlert
ScoutJet
Screaming Screaming
ScreenerBot ScrepyBot
Searchestate Searchestate
Seekport SearchmetricsBot
Semrush Semrush
SeoSiteCheckup Shodan
SeobilityBot
Siphon
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteLockSpider SiteLockSpider
SiteSnagger
Sitebeam
Sitevigil
SlySearch SlySearch
Snapbot Snapbot
Snoopy SocialRankIOBot
Sociscraper Sogou web spider
SpaceBison
Spammen Spammen
SpankBot
Spanner Spanner
Spbot SputnikBot
Sqlworm Sqlworm
SuperHTTP Sqworm
Suzuran Stripper
Swiftbot SuperBot
T0PHackTeam SurveyBot
T8Abot Teleport
TeleportPro
Telesoft
Telesphoreo Telesphoreo
Telesphorep Telesphorep
The Intraformant Thumbor
Toata Trendictionbot
Turnitin True_Robot
VB Project Turingos
VeriCiteCrawler URLy.Warning
VelenPublicWebCrawler
VoidEYE
WASALive-Bot WASALive-Bot
WBSearchBot
WEBDAV
WISENutbot WISENutbot
WWW-Mechanize WallpapersHD
Wallpapers
WeSEE WeSEE
Web Enhancer Web Auto
Web Sauger
WebAuto
WebCollage
WebCopier
WebEnhancer WebEnhancer
WebFuck
WebGo IS WebGo IS
WebImageCollector WebImageCollector
WebReaper WebWhacker
Webshag WebZIP
Website Quester Webalta
WebsiteExtractor Webster
Whack Whacker
Whatweb Whatweb
Who.is Bot WinHTTrack
Wonderbot WiseGuys Robot
Xaldon WebSpider Wotbox
Xaldon_WebSpider Xaldon_WebSpider
Xenu Zade
Zauba Zermelo
Zeus Zitebot
ZumBot ZmEu
ZyBorg arquivo-web-crawler
archive.org_bot cah.io.community
arquivo.pt
autoemailspider
backlink-check
check1.exe check1.exe
coccocbot coccocbot
cognitiveseo com.plumanalytics
crawler4j crawl.sogou.com
dataforseo.com demandbase-bot
domainsproject.org
evc-batch evc-batch
internetVista monitor gopher
iubenda-radar instabid
ips-agent
isitwp.com
linkdexbot
lwp-trivial
mediawords mediawords
oBot muhstik-scan
page scorer page scorer
plumanalytics
polaris version polaris version
ripz ripz
satoristudio.net
scalaj-http scalaj-http
scan.lol scan.lol
seocompany.store seocompany.store
seoscanners seoscanners
seostar seostar
spyfu sp_auditbot
tAkeOut trendiction.com
ubermetrics-technologies.com webmeup-crawler
webpros.com
x09Mozilla x09Mozilla
x22Mozilla zgrab
zauba.io

300
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404enemy
80legs 80legs
ALittle Client ALittle Client
ASPSeek
Abonti Abonti
Aboundex Aboundex
Aboundexbot Aboundexbot
AhrefsBot Acunetix
Alexibot AllSubmitter
Alligator
AlphaBot AlphaBot
Anarchy99 Anarchy99
Ankit
Apexoo Apexoo
Aspiegel Aspiegel
AwarioRssBot
BBBike
BDCbot
BDFetch BDFetch
BacklinkCrawler BackStreet
Badass BackWeb
Battleztar Bazinga
BetaBot BetaBot
Black Hole Bitacle
BlackWidow
Blackboard Blackboard
BlowFish
Boardreader Boardreader
Bolt
Buck Buck
BuiltWith BuiltWith
BunnySlippers
CATExplorador CATExplorador
CCBot
CSHttp CSHttp
CazoodleBot
Cegbfeieh
CensysInspect CensysInspect
ChinaClaw CheeseBot
Cliqzbot
Cocolyzebot Cocolyzebot
Collector Collector
Cosmos Copier
CrunchBot Craftbot
Curious Crawling at Home Project
Custo CrazyWebCrawler
DIIbot Crescent
DataCha0s CyotekWebCopy
DatabaseDriverMysqli DTS Agent
Deusu
Digincore Digincore
DigitalPebble DigitalPebble
Dirbuster Dirbuster
Discobot Disco
DnBCrawler-Analytics
DomainAppender DomainAppender
DomainCrawler
DomainStatsBot DomainStatsBot
Domains Project
Dotbot Dotbot
Dragonfly
ECCP/1.0 ECCP/1.0
EMail Siphon
EMail Wolf EMail Wolf
EasyDL EasyDL
EirGrabber
EroCrawler EroCrawler
Evil Exabot
Express WebPictures Extractor
ExtractorPro ExtractorPro
Extreme Picture Finder
FemtosearchBot
Fimap Fimap
Foobot Foobot
FrontPage FyberSpider
Fuzz Genieo
Fyrebot Getintent
GT::WWW Gigabot
GermCrawler
GetWeb
Go-Ahead-Got-It
Gotit Gotit
GrabNet
Grafula
GrapeFX
GrapeshotCrawler
GridBot
HTTrack
Haansoft
HaosouSpider HaosouSpider
Havij
Hloader Hloader
HybridBot
IDBTE4M
IDBot
Id-search Id-search
IlseBot Image Fetch
IndeedBot InfoNaviRobot
Indy Library Intelliseek
InfoTekies
Internet Ninja
Iskanie
IstellaBot
JamesBOT JamesBOT
Jbrofuzz
JennyBot JennyBot
Joomla JikeSpider
Keyword Density JustView
Kinza Jyxobot
Kenjin Spider
Kozmosbot
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot Larbin
Leap
LeechFTP LeechFTP
LeechGet
Lftp
LibWeb
Libwhisker
LieBaoFast LieBaoFast
Linkbot
LinkextractorPro LinkextractorPro
Ltx71 LinkpadBot
MIIxpc LinksManager
MSIECrawler LinqiaMetadataDownloaderBot
Lipperhey Spider
Lmspider
MJ12bot
MSFrontPage
MTRobot
Mag-Net
Magnet Magnet
Majestic SEO Majestic SEO
Majestic-SEO
Majestic12 Majestic12
MarkMonitor MarkMonitor
Mass Downloader MarkWatch
Mata Hari
MauiBot
Mb2345Browser Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot Microsoft URL Control
Mister PiX Minefield
Moblie Safari
Mojeek Mojeek
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila
Mr.4x3 Mr.4x3
NPbot Msrabot
Musobot
Navroad Navroad
Nessus NearSite
Net Vampire
NetAnts NetAnts
NetMechanic NetMechanic
NetZIP NetZIP
Nettrack Netcraft
Netvibes Niki-bot
NextGenSearchBot
Nikto
Nimbostratus Nimbostratus
Nmap
Not Not
Nuclei
Nutch Nutch
Offline Navigator Offline Navigator
OpenLinkProfiler Openfind
Openvas
OrangeSpider OrangeSpider
OutclicksBot OutclicksBot
OutfoxBot
Page Analyzer Page Analyzer
PageGrabber
PageScorer PageScorer
PageThing.com Panscient
Pavuk
PeoplePal
Petalbot
Pi-Monster Pi-Monster
Picscout Picscout
PictureFinder Pu_iN
Pimonster PxBroker
Pixray
PleaseCrawl
Pump
QueryN Metasearch
Quick-Crawler
RankFlex RankFlex
RankingBot2 Rankivabot
Recorder Reaper
Ripper RedesScrapy
SBIder
SEOkicks-Robot SEOkicks-Robot
SEOprofiler SEOprofiler
SEOstats SEOstats
SalesIntelligent ScoutJet
ScanAlert Scrapy
ScreenerBot Searchestate
SemanticJuice SearchmetricsBot
Shodan SeoSiteCheckup
Site Sucker SiteLockSpider
SiteCheckerBotCrawler SiteSucker
SiteExplorer Siteimprove
Sitebeam Sitevigil
Snapbot SlySearch
SmartDownload
SocialRankIOBot SocialRankIOBot
Sociscraper
Sosospider Sosospider
SputnikBot Spbot
Sqlworm Sqlmap
Steeler Sqworm
Stripper
Sucuri Sucuri
SuperBot SuperBot
SurveyBot SuperHTTP
Suzuran Swiftbot
Szukacz Szukacz
T8Abot
Teleport
TeleportPro TeleportPro
Telesphorep The Intraformant
TheNomad TheNomad
Thumbor
TightTwatBot
Titan Titan
Toata Toweyabot
Trendictionbot URLy Warning
True_Robot VCI
TwengaBot
Twice
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler VidibleScraper
VeriCiteCrawler Virusdie
Voltron
WBSearchBot WBSearchBot
WEBDAV WEBDAV
WISENutbot WWW-Mechanize
WPScan Wallpapers/3.0
WWW-Collector-E WeSEE
WWWOFFLE Web Auto
Wallpapers Web Collage
Web Fetch Web Fuck
Web Pix Web Pix
Web Sucker Web Sucker
WebAuto WebAuto
WebCollage WebEnhancer
WebCopier WebGo IS
WebFetch
WebFuck
WebImageCollector WebImageCollector
WebPix
WebSauger WebSauger
WebWhacker Webalta
WebZIP WebsiteExtractor
WebsiteQuester
Webster
Whatweb Whatweb
Who.is Bot
WinHTTrack WinHTTrack
WiseGuys Robot WiseGuys Robot
Wotbox Xaldon WebSpider
Xenu YoudaoBot
Zade Zade
Zermelo
Zeus
ZmEu
ZoominfoBot ZoominfoBot
backlink-check ZumBot
clark-crawler ZyBorg
adscanner
archive.org_bot
arquivo.pt
autoemailspider
check1.exe
coccocbot coccocbot
crawler.feedback
crawler4j
dataforseo.com
demandbase-bot demandbase-bot
heritrix eCatch
instabid
internetVista monitor internetVista monitor
linkdexbot ips-agent
iubenda-radar
lwp-trivial lwp-trivial
muhstik-scan mediawords
netEstate NE Crawler oBot
plumanalytics pcBrowser
probe-image-size probe-image-size
s1z.ru ripz
satoristudio.net scalaj-http
scan.lol scan.lol
seobility serpstatbot
sitechecker.pro sexsearcher
spyfu
sysscan sysscan
tAkeOut
trendiction.com trendiction.com
webprosbot trendiction.de
x22Mozilla ubermetrics-technologies.com
voyagerx.com
webmeup-crawler
xpymep1.exe
zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
applyneedy.xyz 20pascals.nl
buytizanidineonlinenoprescription.blogspot.com autoua.top
ciceros.co classiquebijoux.ru
coolgamechannel.com digest-project.ru
dataloading.net ekosmetyki.net.pl
harmonyglen.us pipki.r.acdnpro.com
ihtec2019.org se-welding.ru
m292.info smartshoppymac.com
pospr.waw.pl twiclub.in
writersgroup580.web.fc2.com xchangetrak.com

922
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00webcams.com 08800.top
0lovespells0.blogspot.com 0lovespells0.blogspot.com
12bet.com 1-99seo.com
12u.info 1webmaster.ml
2015god.org 2345.com
214jaluwobapef.cf 2nt.ru
273-fz.ru 300richmond.co.nz
3w24.com 5000-cotydzien.com
4istoshop.com 7xc4n.com
7fon.ru 7zap.com
7makemoneyonline.com 8lufu.com
7minuteworkout.com a.frcls.fr
7search.com aa81bf391151884adfa3dd677e41f94be1.com
8xv8.com above.com
aa9bd78f328a6a41279d0fad0a88df1901.com adexprts.com
absugars.com adrunnr.com
academiacsmendoza.org ads.gold
activecampaign.dreamhosters.com adsref.men
ad-words.ru adtology.com
adtech.de adzerg.com
advokateg.ru agahinameh.com
aerogo.com ageofclones.com
aghanyna.com
airlandsea.info airlandsea.info
aktobe.xkaz.org
alevinefesleri.com
allkrim.com
alloysteel.ru alloysteel.ru
alpinism.ru alvtank.se
am15.net amazon-seo-service.com
apiadanaknet-a.akamaihd.net amoi.tn
apiwebwebgetcom-a.akamaihd.net angel1777.kz
appleid-verification.com apiluckyleapnet-a.akamaihd.net
arcadeplayhouse.com appartement-stumm.at
arenda-avtoprokat-krasnodar.ru applicationg29.com
apxeo.info
aquarium-pro.ru
arcadepages.com
arcteryxsale.online
arendakvartir.kz arendakvartir.kz
arquapetrarca.info
artdestshop.eu
artefakct.com artefakct.com
autobudpostach.club atlant-auto.info
autolombard-krasnodar.ru atovh.local-finders.com
avkzarabotok.info audiofree.ru
avtointeres.ru autobrennero.it
azartmsl.com autoloans.com
balans.shahterworld.org aviav.eu
balook.com aviav.ru.com
bangla-chat-uk-paralud.ga avtochehli.by
bankcrediti.ru babs.com.ua
bashtime.ru balkanfarma.org
bbsoldes.fr bankofthewext.com
bdsmgalls.net bannerbank.ru
becuo.com bannerspace.com
bestadbid.com bannerswap.com
bestcurs.org bazaronline24.ru
bestserials.com bestmobilityscooterstoday.com
betwinservice.com betterscooter.com
bgviagrachrx.com bigpictures.club
bigcareer.info blockety.co
biglistofwebsites.com blog.koreadaily.com
bimatoprost-careprost.com.ua bloggen.be
binaryoptionscops.info bloggerads.net
bitnote.co blogos.kz
board.f00d.de boltalko.xyz
boazpower.com bongacams.com
bombla.org bonzbuddy.com
bonux.nextview.ru bonzybuddy.com
brendbutik.ru boobsimge.com
browsepulse-a.akamaihd.net braip.com.br
btvn.ru brus.city
bupropion-sr-150-mg.us bugshoot.cn
buytizanidineonlinenoprescription.blogspot.com bumascloset.com
callejondelpozo.es buyk.host.sk
candytiner.com buzzsumo.com
cardul.ru bzero1jewelry.net
calcularpagerank.com
carabela.com.do
cartierbraceletsreplica.pw
cartierreplica.top cartierreplica.top
cctva.tv cbcseward.com
cdncash.org centre-indigo.org.ua
cenokos.ru certifywebsite.win
chastnoeporno.com chcu.net
cheapjerseysap.com cheap-pills-norx.com
chidporn.com cheapmarmot.online
chinaelitecheapjerseys.com checktext.org
chiptuninger.com chimiver.info
classicakuhni.ru chocolateslim-en-espana.com
clickbank.net choice-credit.ru
clickprotects.com claytransformations.info
clips.ua.ac.be coaching-netz.info
clothesforcash.com codysbbq.com
cocyq.inwtrade.com compliance-don.top
colobit.biz compliance-ivan.top
com-onlinesupport.site compliance-olga.top
compliance-alexa.top
compliance-checker.info
compliance-jane.top
compliance-walt.top
construmac.com.mx
copro.pw copro.pw
counterbot.com costablue.xyz
courtshipgift.com
covid-schutzmasken.de
crackguru.tk
cracksplanet.com cracksplanet.com
credit.co.ua
crest-poloski.ru
csgo4.win csgo4.win
cv.wallhade.co
d2jsp.org
dafatiri.com dafatiri.com
dailystrength.org damasarenai.info
daneshetabiat.com daptravel.com
darkbooks.org delta-line.men
dataloading.net descargar-musica-gratis.net
dealighted.com detskie-konstruktory.ru
decorazilla.com detskie-zabavi.ru
defenderxtactical.com detsky-nabytek.info
dentalpearls.com.au dfiles.me
dfwu1019.info dichvuvesinhhanoi.com
dgfitness.co digital-sale.su
diesel-parts28.ru dikqlyremy.info
dimkino.ru discountliv.com
dipstar.org diy-handmade-ideas.com
directrev.com djekxa.ru
disruptingdinnerparties.com djihispano.com
divisioncore.com doctormakarova.ru
dokfilms.net dojki-hd.com
domain.webkeyit.com dokumentalkino.net
doreenblog.online domaincheck.io
downloadeer.net domaineaneblanc.com
druzhbany.ru dostavka-v-krym.com
dtnlyss.com downtuptv.gq
dyt.net draniki.org
e-c.al duplicashapp.com
ednorxmed.com dverimegapolis.ru
ekobata.ru dyshagi.ru
electricwheelchairsarea.com eager-nash.188-93-233-196.plesk.page
electronicadirect.com efnor-ac.com
elektrozigarettekaufen2.blogse.nl egovaleo.it
elektrozigaretten1.webs.com ekspertmed.com
empathica.com eldiariodeguadalajara.com
enternet.ee elitesportsadvisor.com
epicbrogaming.com elvenmachine.com
erot.co emailaccountlogin.co
escortplius.com energydiet-info.ru
eshop.md energydiet24.ru
exchanges-bet.com eropornosex.ru
exct.net ertelecom.ru
expertnaya-ocenka.ru essay-writing.work
este-line.com.ua
europages.com.ru
exmasters.com
exoclick.com
expdom.com
expresstoplivo.ru
extener.org extener.org
extractorandburner.com ezigarettenkaufen1.pagina.gr
extremepornos.net f1nder.org
ezigarettekaufen2.yolasite.com family1st.ca
f-online.de familyholiday.ml
face.hostingx.eu fanrto.com
familienzahnaerzte.com farm26.ru
fashionindeed.ml fetroshok.ru
fatfasts-4tmz.com findacheaplawyers.com
feorina.ru
fickkontaktehobbyhuren.com
fingerprintjs.com
fireads.men
fitfloponline.store fitfloponline.store
flavors.me fitnesspiks.com
floridamhca.org flashlarevista.com
forums.toucharcade.com fordsonmajbor.cf
free-fb-traffic.com forensicpsychiatry.ru
forexgb.ru
front.to
funnypica.com
funtoonez.com
fxtips.ru
fyxabomiw.ru fyxabomiw.ru
galaxyflowers.ru gallery.rennlist.com
gateway.zscalerone.net garciniaxt.us
gearsadspromo.club geckoandfly.com
gemgrab-a.akamaihd.net generousdeal-a.akamaihd.net
gerhardhealer.com genetworx.com
get-seo-domain.com getmiro.com
getlaid-xxxhookupdirect.com gfaq.ru
getridofstretchmarks.org gheus.altervista.org
gg-arena.ru gifspics.com
ghostvisitor.com gk-atlant.info
gigapeta.com
girlspicsa.com
glass-msk.ru
glastecfilms.com.my glastecfilms.com.my
gnuetella.com globatur.ru
gogalleryawesome.com go2album.com
gojiberriess.apishops.ru golden-praga.ru
golaya.pw gonextmedia.com
good-mummy.ru gpirate.com
googglet.com greatzip.com
googst2.ru hao123.com
goosefishpost.bid hard-porn.mobi
gosarhivrt.ru harmonyglen.us
grand-chlen.ru havepussy.com
greatidea.marketing hdxnxxtube.mobi
gwebtools.com healgastro.com
habermetre.com highland-homes.com
handicapbathtubarea.com highstairs-a.akamaihd.net
happy.new.yeartwit.com histats.com
hausfrauensex18.com
hd-filmy.net
helpmymacfaster.trade
himazin.info
hol.es hol.es
home.myplaycity.com horoshieokna.com
hosting-tracker.com hosting-tracker.com
hostsshop.ru hotelcrocenzi.sm
hotgirlhdwallpaper.com hotshoppymac.com
houtings.xyz housemilan.ru
howopen.ru hspline.com
howtowhitenteethfast.xyz
hulfingtonpost.com hulfingtonpost.com
humbmosquina.tk hundejo.com
iinstalll-fii1leis.jus0wil.pp.ua hyiphunter.org
idc.com.ua
ideoworld.org
ilovevitaly.xyz ilovevitaly.xyz
imagine-ex.co iminent.com
infobabki.ru import-sales.com
infokonkurs.ru in-tandem.co
insta-add.pro inboundlinks.win
investyb.com informatiecentro.be
investzalog.ru insider.pro
iqs.biz.ua internetartfair.com
iqupdatetmz.win investsuccess.org
ivoiretechnocom.ci ip-guide.com
iwantmyfreecash.com iplogger.org
javitas.info istanbulit.com
jerseysbizwholesalecheap.com javatex.co.id
jmat.cn jeremyeaton.co
jewelryandfiligree.com
johannesburgsingles.co.za johannesburgsingles.co.za
johnnyhaley.top jrcigars.com
juliadiets.com jumptap.com
kanimage.com
kareliatobacco.ru
karting196.ru
karusel-market.ru karusel-market.ru
katjimej.blog.fc2.com kerch.site
keywords-monitoring-success.com keywordbasket.com
kino-key.info khafre.us
kletkimehan.ru kinoflux.net
kliksaya.com klumba55.ru
kollesa.ru kochanelli.com
konkursowo-24.pl komputers-best.ru
laxdrills.com konoplisemena.com
leadn.pl koopilka.com
lechtaczka.net koptims.tiu.ru
lennatin.info kriokomora.info
lidoradio.com kurwa.win
link.ac kvartir-remont.biz
linkwithin.com kwzf.net
liumimgx.pw lacapilla.info
livingcanarias.com ladov.ru
lonerangergames.com laexotic.com
lafriore.ru
lasvegaslockandsafe.com
law-check-eight.xyz
leadwayau.com
letmacwork.world
lexaprogeneric.link
lignofix.ua
live-cam6.info
lkbennettstore.online
llastbuy.ru
loadingpages.me
lomb.co
lonely-mature.com
long-beach-air-conditioning.com
longgreen.info longgreen.info
lostfilm-online.ru
louboutinreplica.xyz
lovelycraftyhome.com
ltvperf.com
lumb.co
mackeeper-land-672695126.us-east-1.elb.amazonaws.com mackeeper-land-672695126.us-east-1.elb.amazonaws.com
macrotek.ru macresource.co.uk
magento-crew.net mafcards.ru
manifestyourmillion.com martlinker.com
mantramusic.ru maxxtor.eu
map028.com megaindex.ru
mapquestz.us mensandals.xyz
maslenka.kz menstennisforums.com
matsdale.com merryhouse.co.uk
max-p.men methodsmarketing.com
mazda-roadsters.com mindbox.co.za
mcar.in.ua
meltwater.com
mere.host.sk
metarip.ru
mexicosleevegastrectomy.com
mhi-systems.ru
minet.club
miniature.io miniature.io
mjchamonix.org minneapoliscopiers.com
mobile.ok.ru mobstarr.com
modenamebel.ru
modnie-futbolki.net modnie-futbolki.net
moesonce.com moesonce.com
monclerboots.xyz money-for-placing-articles.com
monclercheap.xyz moneyviking-a.akamaihd.net
monclerjacketsoutlet.pw montazhnic.ru
mrlmedia.net most-kerch.org
mustwineblog.com mountainstream.ms
mxgetcode.com moviemail-online.co.uk
moxo.com
mttwtrack.com
mulberryoutletonlineeu.com
musflashtv.com
music7s.me
muz-tracker.net
myblogregistercm.tk myblogregistercm.tk
mydownlodablefiles.com
myhitmp3.club
mylovelibrabry.com mylovelibrabry.com
myxdate.info mymobilemoneypages.com
narkologiya-orel.ru myseoconsultant.com
nationalbreakdown.com natali-forex.com
negral.pluto.ro negral.pluto.ro
netcheckcdn.xyz netvouz.com
nextrent-crimea.ru newyorkhotelsmotels.info
nfljerseysforsalewholesaler.com nhl17coins.exblog.jp
nightvision746.weebly.com nhl17coinsforps3.gratisblog.biz
noumeda.com nlfjjunb5.ru
nufaq.com nodding-passion.tk
o-o-8-o-o.ru nodup.ru
oakleyglassesonline.us norththeface.store
obesidadealgarve.com nuup.info
oddamzadarmo.eu offers.bycontext.com
odesproperty.com office-windows.ru
offergroup.info office2web.com
offside2.5v.pl ok-ua.info
olgacvetmet.com oklogistic.ru
onblastblog.online okmedia.sk
online-hd.pl online-templatestore.com
onlinebay.ru online247.ml
onlineporno.site open-odyssey.org
openmultipleurl.com opinionreelle.com
osoznanie-narkotikam.net oranga.host.sk
oxford-book.com.ua ourville.info
pandarastore.top outpersonals.com
panicatack.com oynat.info
papasdelivery.ru pacificair.com
partner-stop.men palvira.com.ua
partners-ship.pro paphoselectricianandplumber.com
perosan.com partner-trustworthy.men
partnerads.men
pastaleads.com
pavlodar.xkaz.org
pc-services.ru
pc4download.co
pekori.to
pendelprognos.se
percin.biz.ly
petedrummond.com petedrummond.com
photosaga.info phormchina.com
piccdata.com piccshare.com
picsforkeywordsuggestion.com picquery.com
picswe.com pio.polytopesexempt.com
pictures-and-images.com pipki.r.acdnpro.com
pierrehardysale.online pixgood.com
pigrafix.at pkr1hand.com
pila.pl placid-rounded-coast.glitch.me
piluli.info popmarker.com
pirateday.ru popugauka.ru
pizza-tycoon.com popunder.net
plastjulgranar.se pornodojd.ru
playtap.us pornoklad.net
pochemychka.net pornophoto.xyz
pogruztehnik.ru pornotubexxx.name
pokemongooo.ml pornowarp.info
pontiacsolstice.info privat-girl.net
popcash.net privatefx.all4invest.info
popup-jdh.xyz pro-tec.kz
pornoforadult.com proekt-mos.ru
poshiv-chehol.ru
pr0fit-b0x.com
prchecker.info
presleycollectibles.com
primedice.com
prizestohandle.club
progonrumarket.ru progonrumarket.ru
pronekut.com promoheads.com
prostitutki-novgoroda.org pukaporn.com
prpops.com pushdata.sendpulse.com
quick-seeker.com qcstrtvt.bloger.index.hr
rapidgator-porn.ga qld10000.net
rarbg.to rabotaetvse.ru
razvratnoe.org radiogambling.com
rankings-analytics.com
rapidokbrain.com
rczhan.com
realizmobi.com realizmobi.com
realmonte.net redbottomheels.xyz
recipedays.com reelheroes.net
refudiatethissarah.info refudiatethissarah.info
relatodelpresente.com.ar regionshop.biz
renhacklids.tk reklama-i-rabota.ru
resellerclub.com relayblog.com
rixpix.ru remedyotc.com
rn-to-bsn.com research.ifmo.ru
rniaeba.ga restorator-msk.ru
ru-mediaget.ru revistaindustria.com
rusexy.xyz ricorsogiustizia.org
rustic-quiver.win rockprogblog.com
rvtv.ru roznica.com.ua
s8-nowy-wygraj.comli.com rusenvironmental.net
saddiechoua.com sammyweaver.com
safe-app.net samolet.fr
santasgift.ml sarf3omlat.com
sbtdesign.co.uk sax-sex.com
scanmyphones.com scanner-ivan.top
scanner-nelson.top school-diplomat.ru
scripted.com se.bnt-team.com
secret.xn--oogle-wmc.com searchtooknow-a.akamaihd.net
seeingmeerkat.com security60-e.com
semprofile.com seksvideoonlain.com
sex-tracker.com seo-platform.com
sex.hotblog.top seotoolsagency.com
sexyteens.hol.es sexkontakte-seite.com
shemalegalls.blogporn.in sexphoto.site
shlyahten.ru sexy-pings.com
shymkent.xkaz.org sharebutton.net
sicfor.bcu.cc shemale-sex.net
simpoed.ufop.br shmetall.com.ua
siteaero.com shop.xz618.com
sitebeam.net shtora66.ru
sitopreferito.it sildenafil-tadalafil.info
smart-scripts.com sim-service.net
social-buttons.com sinel.info
socialbookmarksubmission.org skatestick.bid
sodexo.com skuteczna-dieta.co.pl
skylta.com
sladkoevideo.com
sloopyjoes.com
slowmacfaster.trade
smartshoppymac.com
smsactivator.ru
sobecjvuwa.com.ru
soc-proof.su
sockshares.tv
softxaker.ru softxaker.ru
solution4u.com songoo.wz.cz
soserfis.com space4update.pw
space4updating.win speeddream.xyz
spylog.com spl63.fr
spywareit.com sports-supplements.us
srdrvp.com spymac.net
starpages.net ssconstruction.co
staticfs.host staff.prairiesouth.ca
stoki.ru stair.registrydomainservices.com
strana-solnca.ru standardchartered-forex.com
studentguide.ru studiokamyk.com.pl
supermesta.ru sukarame.net
surintech.ac.th supermodni.com.ua
survival.betteroffers.review support.nopeas.sk
taaaak.com surflinksmedical.com
taranerymagesswa.blogspot.com svetodiodoff.ru
tattomedia.com symbaloo.com
tacbibirfa.tk
tattoo33.ru tattoo33.ru
telefonsexi.com tattooreligion.ru
taxi-v-eisk.ru
telefonsexkostenlos.tk telefonsexkostenlos.tk
teletype.in testbotprocessor44.com
templates.radiodigital.co theendivechronicles.com
thedownloadfromwarez.blogspot.com theheroes.ru
thesmartsearch.net thetattoohut.com
thiegs.reco.ws
thruport.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com
tmearegion26.com
tmm-kurs.ru tmm-kurs.ru
toloka.hurtom.com token-lab.org
toonfamilies.net tomck.com
topappspro.com tonivedu.it
topkarkas.com toon-families.com
torrents.cd topmira.com
totu.us topsy.com
toy-shop.top torrentgamer.net
tozup.com tour-line.net
trackmedia101.com
tracksz.co tracksz.co
trafaret74.ru trafficgenius.xyz
traffic2money.com trafficjunky.com
trafficmania.com trainoffend.ml
trafficreceiver.club tri-slona.org
trahvid.com truebeauty.cc
trailer.cinemaflix.website trustedmaccleaner.com
turist-strani.ru tvoystartup.ru
tvand.ru txxx.com
twittruth.com typer.one
unacittaconte.org uk-zheu20.ru
unimodemhalfduplefw.pen.io ukrobstep.com
universals.com.ua ukrup.com
unrealcommander.org umg-stroy.ru
us-america.ru unisexjewelry.org
valmetrundan.se uptime-gamma.net
vapomnoncri.tk usdx.us
vashsvet.com vedomstvo.net
vbabule.net veles.shop
vclicks.net via-energy-order.com
viagengrarx.com videotuber.ru
vielporno.net viveresaniesnelli.it
vitalads.net vseigry.fun
vitoriacabos.com vzlomtw.com
vladhistory.com wahicbefa31.soup.io
voloo.ru webenlace.com.ar
vonradio.com website-speed-checker.site
vtc.pw website-speed-up.site
vvon.co.uk
w-journal.ru
wallpapers-best.com
wanker.us
watch-movies.ru
webcamdevochka.com
webix.me
webmasters.stackexchange.com
webshoppermac.com
websiteexplorer.info websiteexplorer.info
westen-z.life whatsappbot.flyland.ru
westermarkanjou.se wildworld.site
whatistizanidine2mg.blogspot.com witclub.info
wholesalejerseysgaa.com wleuaprpxuvr.ga
winner7777.net woman-tampon.ru
wiosenny-bon-1500.pl wowas31.ucoz.ru
wma-x.com writingservices17.blogspot.ru
woman-h.ru wrona.it
worldoffiles.ru wttavern.com
wpthemedetector.co.uk www.bookmaker-bets.com
xcc24.pl www.get-free-traffic-now.com
xlolitka.com wzgyyq.com
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai xdoza.com
xn----itbkqkfiq.xn--p1ai xn----7sbaphztdjeboffeiof6c.xn--p1ai
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai xn--80ahvj9e.xn--p1ai
xn--80ahdheogk5l.xn--p1ai xn--b1addnj3cah.xn--p1ai
xn--b1agm2d.net xvideospornoru.com
xn--h1ahbi.com.ua xxxnatelefon.ru
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai yourmovies.pl
xnxxandxvideos.com yubikk.info
xxxdatinglocal.us zacreditom.ru
xxxtube69.com zambini.ru
yeartwit.com zeroredirect12.com
youandcredit.ru
youradulthosting.com
yoursite.com
yugo-star.ru
zajm-pod-zalog-nedvizhimosti.ru
zastroyka.org
zawyna.ua
zeroredirect9.com zeroredirect9.com
zoominfo.com znaturaloriginal.com
zverokruh-shop.cz zone-kev717.info
zvuker.net zskdla.site

480
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1008.su 214jaluwobapef.cf
57883.net 2728fb936f0.com
5tey7463.weebly.com 2pxg8bcf.top
6128786.com 3-letter-domains.net
9med.net 55wmz.ru
abiente.ru 7search.com
academiacsmendoza.org 7yue.org
actices.com 8si.ru
advertisingtag.net 991mostfm.co.id
aktivator-windows10.blogspot.com aa81bf391151884adfa3dd677e41f94be1.com
alert.scansafe.net adconscious.com
all4invest.ru addtoadd.men
alt-servis.ru adktrailmap.com
amateurlivechat.org adloads.net
android-systems.ru adobereader-free.ru
apimegabrowsebiz-a.akamaihd.net adultfullhd.com
awaybird.ru akvamaster.dp.ua
azbukafree.com alfabot.xyz
bala.getenjoyment.net allpdfmags.net
bamo.xsl.pt alta-realestate.com
banksrf.ru animalia-life.club
bannerconnect.net anjalika.co.in
bashtime.ru apartmentratings.com
best-mam.ru apessay.com
bestkfiledxd.cf apioutoboxnet-a.akamaihd.net
bibys.com arcadeplayhouse.com
bloglag.com arraty.altervista.org
boc.kz artdeko.info
brainzpod.win asmxsatadriverin.aircus.com
brothers-smaller.ru asscenihotosidea.blogspot.com
bryansk.zrus.org auspolice.com
buckspar.gq australia-opening-times.com
burn-fat.ga avirasecureserver.com
buron.pw avkzarabotok.info
callcafe.info azazaporn.com
camsex.xxx-cam.webcam berlininsl.com
carina-sy.de best-games.today
chastnoeporno.com big-cash.net
christianlouboutinreplica.pw blackplanet.com
cialischstgerts.com boanonihaca.tk
browserprotecter.com
bum.com.ru
buntube.net
callawaygolfoutlet.online
callawaygolfstore.online
candelluxsklep.pl
cartierreplica.xyz
catalogs-parts.com
chatroulette.life
cheapcigarettesc.info
chinajerseyswholesalecoupons.com
clipartnew.com clipartnew.com
com-cleaner.systems cmhomestayagency.com
commoncrawl.org commoncrawl.org
compliance-jess.top compliance-checker.info
cornomase.win conocer-sanabria.com
covetnica.com dailystrength.org
culmimg.pw danschawbel.com
customsua.com.ua dataloading.net
daooda.com dear-diary.ru
davebestdeals.com dekorkeramik.ru
delitime.info deluxedumps.com
diarioaconcagua.com detskie-konstruktory.ru
dieswaene.com dikqlyremy.info
div.as dimaka.info
docsportal.net discovertreasurenow.com
dojki365.online dnsrsearch.com
dood.live domainroam.win
dorratex.tn drunkmoms.net
downloadme.life dverimegapolis.ru
dptaughtme.com e-commerce-seo1.com
e2click.com ecxtracking.com
eropho.com eets.net
eu-cookie-law.blogspot.com ekspertmed.com
everytalk.tv elektronischezi.livejournal.com
extlabs.io elektrozigarette2.wordpressy.pl
extremez.net energydiet-info.ru
ezrvrentals.com enhand.se
ferrotodo.com erotiktreff24.info
fickenprivat.info european-torches.ru
fix-website-errors.com farm26.ru
flooringinstallation-edmonton.com fearcrow.com
foto-basa.com fenoyl.batcave.net
fotopop.club fettefrauen.net
freecamdollars.com filesvine.com
gardene.ru finstroy.net
gay-file.com forexunion.net
gembird.com forzeronly.com
getlaid-xxxhookupdirect.com freza-sverlo.ru
gheus.altervista.org fuckingawesome.com
girlspicsa.com funponsel.com
gne8.com gamblingnerd.com
goodprotein.ru gdcentre.ru
gsasearchenginerankerdiscount.com go2album.com
habermetre.com googglet.com
hatedriveapart.com gosarhivrt.ru
hdimagegallery.net hasshe.com
hkladys.com hd-filmy.net
hothor.se hdxnxxtube.mobi
hikesearch.net
hothot.ru
hotxnights.info
hplaserjetpdriver8y.pen.io hplaserjetpdriver8y.pen.io
hptwaakw.blog.fc2.com id-forex.com
hydropump.su ido3.com
iboss.com interesnie-faktu.ru
idealtits.net investsuccess.org
imgata.com itservicesthatworkforyou.com
imitex-plus.ru ivearchenceinflu.cf
interfucks.net jagg.info
iskalko.ru javidol.site
iwanttodeliver.com johannesburgsingles.co.za
kabbalah-red-bracelets.com kamalsinha.com
kakadu-interior.com.ua kazan.zrus.org
karusel-market.ru keywordsking.com
kenaba.su kidskunst.info
kino-ecran.ru ladov.ru
kinoplen.ru lafourmiliaire.com
kiprinform.com landinez.co
kollesa.ru laptop-4-less.com
lacapilla.info law-check-eight.xyz
lalalove.ru lechenie-gemorroya.com
lanasshop.ru ledpolice.ru
larger.io lenvred.org
law-check-seven.xyz letto.by
life-instyle.com lightinghomes.net
lineavabit.it limtu.ifmo.ru
linkbuddies.com link.ac
linkdebrideur.xyz linkpulse.com
lion.snapmap.us lion.snapmap.us
lol-smurfs.com littlesunraiser.com
lotto6888.com live-sexcam.tk
lowephotos.info lonfon.xyz
m1media.net long-beach-air-conditioning.com
m4ever.net lyrics.home-task.com
mabdoola.blogspot.com macrotek.ru
macdamaged.tech mailemedicinals.com
mackeeper-land-672695126.us-east-1.elb.amazonaws.com malls.com
macresource.co.uk manifestyourmillion.com
mansparskats.com mantramusic.ru
marwer.info matchpal-a.akamaihd.net
matematikus.info meget.co.za
matsdale.com midst.eu
meet-flirt-dating.com minecraft-rus.org
misslike.ru mmoguider.ru
missvietnam.org mydirtyhobby.com
mnogabukaff.net myhitmp3.club
moivestiy.biz namenectar.com
moneytop.ru narkologiya-orel.ru
moonci.ru nippon-bearings.ru
mp3ringtone.info njkmznnb.ru
musicdaddy.net nlfjjunb5.ru
muycerdas.xxx notaria-desalas.com
mylesosibirsk.ru nuit-artisanale.com
myliveblog.ru o-o-11-o-o.com
nastydollars.com okroshki.ru
nationalbreakdown.com old-rock.com
news-speaker.com partnerline.men
niroo.info pdamods.ru
o00.in
obiavo.in
obnal.org
odesproperty.com
omgtnc.com
one-gear.com
onlinadverts.com
oyster-green.com
paintingplanet.ru
paperwritingservice17.blogspot.ru
pawli.eu
pechikamini.ru
pelfind.me pelfind.me
picsfair.com picurams.pw
playmsn.com pihl.se
popugaychiki.com piski.top
porno-chaman.info ploenjitmedia.azurewebsites.net
pornolook.net pornohub.me
pornoplen.com pr0fit-b0x.com
pornoxxx.com.mx prchecker.info
predominant-invent.tk privacylocationforloc.com
prokotov.com privatbank46.ru
psa48.ru proposal-engine.com
ptr.ruvds.com pussysaga.com
puzzleweb.ru qor360.com
pyramidlitho.webs.com qwarckoine.com
q-moto.ru razleton.com
quickloanbank.com relatodelpresente.com.ar
r-e-f-e-r-e-r.com rentehno.ru
repeatlogo.co.uk rubbed.us
revistaindustria.com running-line.ru
rome2rio.com
rusoft-zone.ru
sack.net sack.net
savingsslider-a.akamaihd.net sad-torg.com.ua
searchengineranker.email scanner-alex.top
seasonvar.ru sell-fb-group-here.com
serialsway.ucoz.ru
serptehnika.ru serptehnika.ru
sexgalleries.top sex-sex-sex5.com
shoesonlinebuy.xyz skutecznetabletkinaporostwlosow.pl
signoredom.com slavia.info
signx.info smsactivator.ru
sitiz.club sneakyboy.com
sledstvie-veli.net sovetogorod.ru
social-buttons.com space4update.pw
softlinesolutions.me startufa.ru
soietvousmaime.fr steelmaster.lv
solitaire-game.ru takethatad.com
sonata-arctica.wz.cz tattomedia.com
spaceshipad.com tds-advert002.info
sport-video-obzor.ru
spravka-medosmotr.ru
staff.prairiesouth.ca
stairliftstrue.com
statistici.ro
storehouse.ua
strfls.com
stroyhelp-dv.ru
syvertsen-da.no
technika-remont.ru technika-remont.ru
teenforporn.com thebestweightlosspills.ovh
theguardlan.com thecoral.com.br
theheroes.ru theporndude.com
thin.me.pn tizanidinehydrochloride4mgstreetvalue.blogspot.com
tizanidineinfo.blogspot.com tizanidinerecreationaluse.blogspot.com
tizanidinezanaflex.blogspot.com toolsky.com
tmm-kurs.ru tooplay.com
torrnada.ru top10registrycleaners.com
toursmaps.com topflownews.com
track112.site ttrraacckkrr.com
trafficnetzwerk.de u17795.netangels.ru
trumpetedextremes.com uggsale.online
trustedmaccleaner.com uhod-za-sobakoj.ru
turvgori.ru urlcut.ru
tvory.predmety.in.ua vacuumcleanerguru.com
ultimatesetnewfreeallsoftupgradesystems.pw valsalud.com
umekana.ru vchulkax.com
unionmarkt.de vgoloveboli.net
univerfiles.com videotuber.ru
unrealcommander.com vipms.ru
vasileostrovsky-rayon.ru visa-china.ru
venerologiya.com vizitki.net
viandpet.com vkmusics.ru
videooko.weebly.com vzlomsn.org
vikistars.com wallpapersdesk.info
vilingstore.net web.cvut.cz
vipromoffers.com website-speed-up.top
vkontaktemusic.ru websites-reviews.com
vladimir.zrus.org wedding-salon.net
volgograd.xrus.org wesharepics.site
vroze.com westsextube.com
vzglyadriv.kg woman-orgasm.ru
w-journal.ru womensterritory.ru
warningwar.ru wormix-cheats.ru
webkeyit.com wygraj-teraz.com
wellcome2slovenia.ru
westermarkanjou.se
word-vorlagen.net
xclicks.net
xerox-douglas.cf
xingzi-vision.com xingzi-vision.com
xn----7sbho2agebbhlivy.xn--p1ai xmladserver.com
xxxnatelefon.ru xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn--80atua3d.xn--p1ai
xn--e1aggki3c.xn--80adxhks
xn--oogle-wmc.com
xtubeporno.net
xxxtube69.com
y8games-free.com
yatrk.xyz yatrk.xyz
youtubedownload.org yopoint.in
youjizz.vc
zastenchivosti.net
zeroredirect11.com
zirondelli.it
zolotoy-lis.ru
zryydi.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.101 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3449 ### Version: V4.2022.11.3450
### Updated: Thu Nov 3 22:01:35 UTC 2022 ### Updated: Fri Nov 4 10:03:40 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3449 ### Version: V4.2022.11.3450
### Updated: Thu Nov 3 22:01:35 UTC 2022 ### Updated: Fri Nov 4 10:03:40 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##