vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 15:34:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.05.3101 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-05-12 22:07:16 +00:00
parent d3ada70110
commit 7c32feb54e
46 changed files with 98506 additions and 98506 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3100 #### Version: V4.2022.05.3101
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3100 #### Version: V4.2022.05.3101
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.05.3100 #### Version: V4.2022.05.3101
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12176
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.05.3100 #### Version: V4.2022.05.3101
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12180
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12180
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12180
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12180
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12180
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12180
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12180
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3100 3101

12180
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12180
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.113.205.96 0; 20.232.126.243 0;

12180
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12180
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12180
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12180
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.113.205.96 0; 20.232.126.243 0;

12180
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.113.205.96 0; 20.232.126.243 0;

12180
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
DigitalPebble 01h4x.com
ExtractorPro EirGrabber
Foobot Fyrebot
GridBot GrapeshotCrawler
Iria MeanPath Bot
Lanshanbot PxBroker
RankActiveLinkBot ScoutJet
ScrepyBot ScreenerBot
SiteSucker
VCI VCI
WebSauger

312
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 360Spider
404enemy
80legs 80legs
ADmantX ADmantX
AIBOT AIBOT
ASPSeek ASPSeek
Aboundex Abonti
Acunetix AfD-Verbotsverfahren
AhrefsBot AhrefsBot
Aipbot AiHitBot
AllSubmitter
Alligator Alligator
Anarchy Anarchie
Anarchy99 Ankit
Aspiegel
AwarioSmartBot AwarioSmartBot
BDFetch BBBike
BackDoorBot Backlink-Ceck
BackStreet Badass
BatchFTP BatchFTP
BetaBot Battleztar Bazinga
Bigfoot
Bitacle Bitacle
Blackboard Blackboard
Blow
Bolt Bolt
Brandprotect Brandprotect
Brandwatch Brandwatch
Bullseye
BunnySlippers BunnySlippers
BuzzSumo
CATExplorador CATExplorador
CCBot
CODE87 CODE87
CazoodleBot Cegbfeieh
CheeseBot Citoid
ChinaClaw Cliqzbot
Chlooe
Cloud mapping Cloud mapping
Cogentbot Cocolyzebot
CopyRightCheck Collector
Copyscape
Cosmos Cosmos
Craftbot Craftbot
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Crescent
CrunchBot CrunchBot
DBLBot CyotekWebCopy
Deusu DataCha0s
DigitalPebble Digincore
Dirbuster
Disco Disco
Discobot Discobot
DnBCrawler-Analytics Discoverybot
Dragonfly
Drip Drip
EMail Siphon EMail Wolf
Ebingbong EasyDL
EirGrabber EirGrabber
Exabot Ezooms
Extractor FHscan
Extreme Picture Finder FemtosearchBot
FDM
Fimap Fimap
Firefox/7.0 Flunky
Fuzz FrontPage
FyberSpider FyberSpider
G-i-g-a-b-o-t Fyrebot
GetRight GalaxyBot
Getintent Genieo
Go-Ahead-Got-It GetWeb
Go!Zilla
GrabNet GrabNet
GridBot GrapeFX
HTMLparser Hloader
HaosouSpider
Havij
Humanlinks
HybridBot
IDBot IDBot
IRLbot Id-search
Iblog IlseBot
Image Sucker Image Sucker
IndeedBot IndeedBot
InfoNaviRobot Indy Library
InfoTekies
Intelliseek Intelliseek
Iskanie
JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
JennyBot
Joomla Joomla
Jorgee
Kenjin Spider
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Keyword Density
Kinza Kinza
LNSpiderguy
Lanshanbot
Larbin Larbin
Libwhisker Leap
LinkScan LeechFTP
LexiBot
LibWeb
Likse
Linkbot Linkbot
LinkpadBot LinkextractorPro
LinqiaMetadataDownloaderBot Lipperhey Spider
LinqiaRSSBot MFC_Tear_Sample
Ltx71 MIIxpc
MJ12bot
MSFrontPage MSFrontPage
MSIECrawler
Magnet Magnet
Meanpathbot Mail.RU_Bot
MegaIndex.ru Majestic12
Metauri MarkWatch
MauiBot
MeanPath Bot
Mediatoolkitbot
MicroMessenger MicroMessenger
Mojolicious Microsoft URL Control
Morfeus Fucking Scanner Mister PiX
Mozlila Moblie Safari
Navroad MolokaiBot
Net Vampire NPbot
NearSite
Needle
Nessus
NetAnts NetAnts
NetLyzer NetSpider
Netcraft
Nettrack Nettrack
Nibbler Netvibes
Nikto Nimbostratus
Nmap Ninja
Not Nutch
Nuclei
Octopus Octopus
OnCrawl Openfind
OpenVAS OrangeBot
OutfoxBot OutfoxBot
PECL::HTTP PECL::HTTP
PageAnalyzer POE-Component-Client-HTTP
PageGrabber Pandalytics
PageScorer
PageThing.com
Panscient Panscient
Picscout Papa Foto
Picsearch Pavuk
PeoplePal
PictureFinder PictureFinder
Piepmatz Piepmatz
PleaseCrawl PleaseCrawl
ProWebWalker Pockey
ProPowerBot
Probethenet
Psbot Psbot
Pu_iN PyCurl
Quick-Crawler QueryN Metasearch
RankFlex RSSingBot
RankActive
RankActiveLinkBot
Rankivabot
RankurBot RankurBot
ReGet ReGet
RealDownload
Reaper Reaper
RocketCrawler RebelMouse
Recorder
RepoMonkey
SBIder SBIder
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats
SalesIntelligent SalesIntelligent
ScoutJet
Scrapy
ScreenerBot ScreenerBot
SearchmetricsBot SemanticJuice
Seekport
Semrush Semrush
SeobilityBot
Seomoz Seomoz
SiteExplorer
SiteLockSpider SiteLockSpider
Sitebeam SiteSnagger
Siteimprove Snake
SlySearch SocialRankIOBot
SmartDownload Sogou web spider
Snapbot Sosospider
Snoopy Spammen
Spanner Spanner
Sqworm Spbot
Stripper Sqlworm
Sucker Steeler
Sucuri Sucuri
SuperHTTP Suzuran
Surfbot T0PHackTeam
Szukacz
T8Abot T8Abot
TeleportPro
Telesoft Telesoft
TheNomad Telesphoreo
Thumbor
Titan
Toata
Tracemyfile Tracemyfile
Trendiction Trendiction
Trendictionbot Trendictionbot
True_Robot True_Robot
Turingos
Turnitin
TurnitinBot TurnitinBot
Twice
Typhoeus Typhoeus
URLy.Warning URLy Warning
UnisterBot VelenPublicWebCrawler
Upflow VidibleScraper
V-BOT VoidEYE
VB Project WEBDAV
VCI Wallpapers
WASALive-Bot Wallpapers/3.0
WBSearchBot Web Collage
WISENutbot Web Enhancer
WPScan Web Fuck
WWW-Collector-E
WWW::Mechanize
WWWOFFLE
Web Auto
Web Fetch
Web Sauger Web Sauger
Web Sucker WebAuto
WebBandit
WebCollage WebCollage
WebCopier WebCopier
WebEnhancer
WebFetch WebFetch
WebFuck
WebGo IS WebGo IS
WebSucker WebPix
WebReaper
WebStripper
Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Webshag
Website Quester Website Quester
WebsiteExtractor
Webster
Whack Whack
Whatweb Widow
WinHTTrack
Woobot Woobot
Wprecon Wotbox
Xenu
Zauba Zauba
Zeus Zermelo
Zitebot
ZoomBot ZoomBot
arquivo-web-crawler ZyBorg
check1.exe adscanner
arquivo.pt
cah.io.community
clark-crawler
cognitiveseo cognitiveseo
com.plumanalytics
crawler.feedback crawler.feedback
crawler4j domainsproject.org
dataforseo.com eCatch
heritrix gopher
instabid
internetVista monitor internetVista monitor
ips-agent
iubenda-radar iubenda-radar
lwp-request
lwp-trivial
magpie-crawler magpie-crawler
muhstik-scan meanpathbot
probe-image-size mediawords
oBot
page scorer
pcBrowser
plumanalytics
ripz ripz
s1z.ru satoristudio.net
scalaj-http
scan.lol scan.lol
seocompany.store seobility
seoscanners seostar
serpstatbot
sexsearcher sexsearcher
siteripz sitechecker.pro
sysscan sogouspider
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
webmeup-crawler
webprosbot
x09Mozilla
zgrab zgrab

316
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404enemy 404enemy
80legs
AIBOT AIBOT
Abonti
Acunetix Acunetix
AhrefsBot AfD-Verbotsverfahren
AlphaBot AiHitBot
Anarchy99 Alexibot
AllSubmitter
Anarchie
Anarchy
Aspiegel
Asterias Asterias
Attach Atomseobot
BBBike BBBike
BackDoorBot BDFetch
Backlink-Ceck Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass Badass
Bandit Bandit
Barkrowler Barkrowler
Battleztar Bazinga Battleztar Bazinga
BetaBot
Bigfoot
Bitacle Bitacle
Black Hole Black Hole
Blackboard BlackWidow
Blow Bolt
Brandwatch Brandprotect
Buck Buck
Buddy Bullseye
BuiltBotTough
BuiltWith
BunnySlippers BunnySlippers
BuzzSumo
CATExplorador
CCBot CCBot
CODE87
CSHttp
Calculon Calculon
Claritybot CazoodleBot
Cloud mapping Cegbfeieh
Cogentbot CensysInspect
CopyRightCheck CheTeam
Copyscape Cliqzbot
Cosmos Cocolyzebot
Crawling at Home Project CrazyWebCrawler
Crescent
Curious
CyotekWebCopy CyotekWebCopy
DatabaseDriverMysqli DBLBot
Devil DSearch
Digincore DTS Agent
DigitalPebble DataCha0s
Dirbuster DnBCrawler-Analytics
Discobot
Discoverybot
Dispatch
DittoSpyder
DnyzBot DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainCrawler Dotbot
DomainStatsBot Download Wonder
Drip Drip
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
EasyDL EasyDL
Ebingbong EirGrabber
Ecxi Evil
Exabot
ExtLinksBot ExtLinksBot
Ezooms Extractor
FDM ExtractorPro
FyberSpider Extreme Picture Finder
GalaxyBot FemtosearchBot
GermCrawler Fuzz
GetWeb G-i-g-a-b-o-t
GT::WWW
Genieo
GetRight
Getintent
Go-Ahead-Got-It
GoZilla GoZilla
GrabNet GrabNet
Grafula Grabber
GrapeFX
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
HTMLparser HMView
HTTrack HTTP::Lite
Haansoft
Hloader
HonoluluBot HonoluluBot
Id-search Humanlinks
Image Fetch IDBot
IRLbot
Iblog
Image Sucker Image Sucker
Indy Library
InfoNaviRobot InfoNaviRobot
Intelliseek
InterGET InterGET
Iria
Iskanie
IstellaBot IstellaBot
JamesBOT
Jbrofuzz
JennyBot JennyBot
Jetty JetCar
Joomla JikeSpider
Jorgee
JustView JustView
Jyxobot Jyxobot
Kenjin Spider
Keyword Density Keyword Density
LNSpiderguy Lanshanbot
Larbin Lftp
LibWeb LinkextractorPro
Lightspeedsystems LinkpadBot
Likse LinqiaRSSBot
LinqiaMetadataDownloaderBot MQQBrowser
LinqiaScrapeBot MTRobot
MFC_Tear_Sample MarkMonitor
MIIxpc Mata Hari
MSIECrawler
Majestic-SEO
Masscan
MauiBot MauiBot
MeanPath Bot
Metauri
Microsoft Data Access
Microsoft URL Control
Minefield Minefield
Mojeek
Mojolicious Mojolicious
Morfeus Fucking Scanner
Mr.4x3 Mr.4x3
Musobot NPbot
Name Intelligence
Nameprotect Nameprotect
Nessus NetLyzer
NetAnts NetSpider
NetMechanic NetZIP
Netcraft NextGenSearchBot
Nettrack Nibbler
Niki-bot
Ninja
Nmap
Nutch
Octopus
Offline Explorer Offline Explorer
Offline Navigator
OnCrawl
OpenVAS OpenVAS
OrangeBot Openfind
OrangeSpider
OutfoxBot OutfoxBot
PECL::HTTP PHPCrawl
PageAnalyzer POE-Component-Client-HTTP
PageThing.com PageThing.com
Pandalytics Papa Foto
Panscient PeoplePal
Pavuk Picsearch
Petalbot Piepmatz
Pi-Monster Pixray
ProPowerBot PleaseCrawl
ProWebWalker
Psbot
Pump Pump
Re-re RSSingBot
ReGet RankActiveLinkBot
RankFlex
RankurBot
RebelMouse RebelMouse
RocketCrawler RedesScrapy
Rogerbot
SBIder SBIder
SEOlyticsCrawler SEOstats
SEOprofiler
SalesIntelligent SalesIntelligent
ScanAlert Scrapy
ScoutJet Screaming
Searchestate ScreenerBot
Semrush ScrepyBot
SearchmetricsBot
SemrushBot
SentiBot SentiBot
Siphon Seomoz
Site Sucker Site Sucker
Sitebeam SiteCheckerBotCrawler
SmartDownload SiteExplorer
SocialRankIOBot SiteSnagger
SiteSucker
Siteimprove
Snake
Snapbot
Sociscraper Sociscraper
Sogou web spider
Sosospider
Spammen Spammen
SpankBot Spinn3r
Spanner SputnikBot
Sqlworm Sqlworm
Sucuri SuperBot
Surfbot
SurveyBot SurveyBot
Suzuran Suzuran
TeleportPro T0PHackTeam
Telesoft
Telesphoreo
Telesphorep Telesphorep
TheNomad The Intraformant
TightTwatBot
Titan
Toweyabot Toweyabot
Trendiction
Trendictionbot Trendictionbot
TurnitinBot Turingos
Turnitin
TwengaBot
Typhoeus
URLy Warning
URLy.Warning
UnisterBot UnisterBot
V-BOT Vagabondo
Vacuum
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie Virusdie
Voltron
WASALive-Bot WASALive-Bot
WEBDAV
WISENutbot WISENutbot
WWW::Mechanize WWW::Mechanize
Wallpapers WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
Web Collage
Web Enhancer
Web Fetch
WebAuto WebAuto
WebEnhancer WebEnhancer
Web Pix WebFuck
Web Sucker WebImageCollector
WebAuto
WebCollage
WebFetch
WebLeacher WebLeacher
WebPix WebsiteExtractor
WebStripper
WebWhacker
WebZIP
Website Quester
Whack
Whacker
Whatweb Whatweb
Who.is Bot Widow
WinHTTrack WinHTTrack
Woobot Wprecon
Xaldon WebSpider
Xenu
YoudaoBot
Zade
Zauba
Zitebot Zitebot
ZoomBot ZmEu
ZoominfoBot
ZyBorg ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo-web-crawler
cah.io.community arquivo.pt
cognitiveseo backlink-check
crawl.sogou.com crawl.sogou.com
crawler4j eCatch
demandbase-bot evc-batch
ips-agent facebookscraper
gopher
instabid
internetVista monitor
isitwp.com isitwp.com
linkdexbot iubenda-radar
lwp-request lwp-request
magpie-crawler
mediawords
netEstate NE Crawler netEstate NE Crawler
pcBrowser pcBrowser
polaris version probe-image-size
s1z.ru
scalaj-http scalaj-http
scan.lol
seobility
seocompany.store seocompany.store
seoscanners
serpstatbot serpstatbot
sitechecker.pro sexsearcher
trendiction.de siteripz
ubermetrics-technologies.com sogouspider
sysscan
trendiction.com
voyagerx.com voyagerx.com
webgains-bot
webmeup-crawler webmeup-crawler
webpros.com webpros.com
webprosbot x09Mozilla
x22Mozilla x22Mozilla
zauba.io zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
alarmobninsk.ru 1001watch.com.ua
buytizanidineonlinenoprescription.blogspot.com arate.ru
ec-file.info banan.tv
iopeninghours.co.uk bin-brokers.com
picture-group.com freegamesplay.online
shohanb.com impotentik.com
shtaketniki.ru pawli.eu
tizanidinedruginteractions.blogspot.com realmonte.net
torrents.life validdomain.xyz
trafficgenius.xyz vipsexfinders.com

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
03e.info 03e.info
101flag.ru 101billion.com
1qingdao.com 11235813.webzdarma.cz
2000k.ru 12bet.com
3th.co.in 1bet.com
7makemoneyonline.com 214jaluwobapef.cf
8lufu.com 2ads.co.uk
a96527gi.beget.tech 40cg.com
aa8b68101d388c446389283820863176e7.com 4senses.co
aab94f698f36684c5a852a2ef272e031bb.com 4ur.click
aanapa.ru 7xc4n.com
acc.eu.org 8kisses.eu
acgs.tk 9icmzvn6.website
adexprts.com aaa38852e886ac4af1a3cff9b47cab6272.com
admatic.com.tr abcguru.xyz
adobereader-free.ru actulite.com
adrenalin-stk.ru admeasures.com
adventureparkcostarica.com adprotect.net
airmaxshoes-2016.com adservme.com
alert-fdm.xyz adssafeprotected.com
alfapro.ru adtech.de
alice.it adult-shop.com.ua
alldownload.pw adverster.com
afmuseum.com
ahhjf.com
ahrntal.verymes.xyz
aibolita.com
ald2014.org
alert-fjg.xyz
all-streaming-media.com
allornamenti.com
alloysteel.ru alloysteel.ru
alta-realestate.com alphavisions.net
alveris.ru
am15.net
amanda-porn.ga amanda-porn.ga
amateurmatch.com amateurmatch.com
anabolics.shop amazon-seo-service.com
android-vsem.org anaksma.info
angkortours.vn angigreene.com
animal36.com animal-drawings.com
anonymous-redirect.com animali.deagostinipassion.it
anonymousfox.co aosexkontakte.net
apartamentwroclaw.eu apiholdingmypage-a.akamaihd.net
apiwebwebgetcom-a.akamaihd.net apiluckyleapnet-a.akamaihd.net
applicationg29.com apimossnetinfo-a.akamaihd.net
apptester.tk asdfg.pro
aproposde.com asmxsatadriverin.aircus.com
architecturebest.com asociatia-tipografilor-transilvania.ro
ardimobileinfo.ml atout-energie-69.com
arius.tech auto.rusvile.lt
arkkivoltti.net autodo.info
atleticpharm.org
audiofree.ru
ausmepa.org.au
autogrep.ru
automatic-seo.com automatic-seo.com
autoua.top autotop.com.ua
aviapanda.ru autotrends.today
azartclub.org aviav.org
azartmix.com avirasecureserver.com
azbukafree.com avkzarabotok.info
backlink4u.net avon-ukraine.com
baksman.com avtoarenda.by
balkanfarma.ru ayanaspa.com
bannerpower.com ayeartoforget.com
best-coupon-offer.com azartmsl.com
betterhdporn.com babespcs.com
bg6s0.com bang-hotties.com
bgviagrachrx.com bannerbank.ru
bharatdefencekavach.com bear.gotcher.us
big-cash.net bedcapdealers.com
bikini-image.com beget.tech
bimatoprost-careprost.com best-games.today
bingo8888.com best-mam.ru
bestimagecoollibrary.com
bestmouthwash.club
bestofpicture.com
bestwrinklecreamnow.com
bidr.trellian.com
binomo.kz binomo.kz
biocn.dx.am bitcoinremote.com
biznesschat.net blackhatworth.com
bizru.info blog.f00kclan.de
bkns.vn bombla.org
blackbabesporn.com bonanza-fortune.men
blog.koreadaily.com bongiornos.info
bloggedporn.com
blpmovies.com
boo-arts.com
braindaily.xyz braindaily.xyz
buytizanidineonline.blogspot.com braintobe.win
buzzsumo.com bravegirlsclub.com
bytimedance.ru budpost.com.ua
calcularpagerank.com burger-tycoon.com
camel-beach.com burn-fat.ga
cannibalcheerleader.com buy-loft.ru
cartierbraceletsreplica.pw buyk.host.sk
cartujano-pre.de buyparajumpers.online
cfsrating.sonicwall.com buzzurl.jp
chatroulette.online bvlgaribracelet.xyz
chinajerseyswholesalecoupons.com candiceloves.us
chinese-amezon.com canoncdriverq3.pen.io
chocolateslim-original.com carloans.com
christianlouboutinreplica.win cdnnetwok.xyz
cityadspix.com cejewelry.xyz
cjs.com.ru cellfun.mobi
clickaider.com centrdebut.ru
clickprotects.com changde.58.com
clickzzs.nl chatroulette.video
coinsspb.com cicaki.net
cleanpcnow.com
club-samodelkin.ru
coin-hive.com
compliance-alex.top compliance-alex.top
compliance-brian.xyz compliance-fred.top
compliance-irvin.top compliance-jane.top
compliance-julia.top compliance-nelson.top
compliance-julianna.top cornomase.win
compliance-mary.top cougfan.info
concordexoticrentals.com counter.yadro.ru
coolingoods.com cranly.net
copenergo.ru creatives.adbetclickin.pink
cosmetic.donna7753191.ru cubook.supernew.org
covadhosting.biz cxpromote.com
cpabegins.ru dafatiri.com
creams.makeforum.eu daneshetabiat.com
crystalslot.com dangphoto.trade
data-ox.com darmebel.com.ua
decorationspcs.com dating-app.ru
detmebel.su dayibiao.com
dichvuvesinhhanoi.com decorazilla.com
dieswaene.com detalizaciya-tut.biz
dirtpics.pw deutschehobbyhuren.net
discovertreasure-a.akamaihd.net digest-project.ru
div.as dinosaurus.site
dms-sw.ru discovertreasurenow.com
dnepr-avtospar.com.ua distver.ru
domain-tracker.com diy-handmade-ideas.com
domaincdn.xyz domaincheck.io
domaineaneblanc.com domaineaneblanc.com
drpornogratisx.xxx dorratex.tn
doxysexy.com
drhomes.biz
drugs-no-rx.info drugs-no-rx.info
dumpsmania24.com drunkenstepfather.com
e2click.com earncash.com.ua
e90post.com
earthmagic.info earthmagic.info
ecommerce-seo.com editmedios.com
ed-shop01.ru electrik-avenue.com
efkt.jp eleimgo.pw
elektrischezigaretten1.blogse.nl
elektronischezigarettekaufen1.myblog.de
elektrozigaretten1.webs.com elektrozigaretten1.webs.com
empirepoker.com eloconcream.blogspot.com
erotag.com elstal.com.pl
es5.com en.altezza.travel
escortplius.com enge-fotzen.info
etenininrade.ga englishdictionaryfree.com
exchangeit.gq erotikchat-24.com
exci.ru erotiktreff24.info
exlarseva.webblog.es etotupo.ru
exmasters.com eupornstar.info
exoclick.com everytalk.tv
extlabs.io extlabs.io
extrabot.com ezigarettekaufen1.hpage.com
extremez.net ezigaretteshop.webs.com
ezigarettekaufen2.mpbloggar.se fakehandbags.xyz
fandlr.com
fashionindeed.ml fashionindeed.ml
feargames.ru fatmaelgarny.com
femdom.twiclub.in fbdownloader.com
fickenbumsen.net fellowshipoftheminds.com
feorina.ru
fettefrauen.net
filesmonster.porn filesmonster.porn
findclan.org filmbokep69.com
finemanteam.com finejewelryshop.ru
finstroy.net fireads.men
flagcounter.me firesub.pl
foodcrafts.website firstsiteguide.com
formulaf1results.blogspot.com flooringinstallation-edmonton.com
forums.toucharcade.com floridamobilebillboards.com
forzeronly.com fok.nl
forpostlock.ru
forum20.smailik.org
foto-weinberger.at foto-weinberger.at
fotopop.club foxtechfpv.com
fx-brokers-review.com freshdz.com
gameprimary.com froggytube.com
games.kolossale.ru fungamelands.com
gaming-journal.com fyl.com.ru
gdcentre.ru g7m.pl
get-seo-domain.com galaxys6manual.info
getlamborghini.ga galeon.com
gilsonchiro.xyz garciniaxt.us
girlsatgames.ru gay.adultgalls.com
girlspicsa.com generousdeal-a.akamaihd.net
glavtral.ru genetworx.com
gloverid.site get-free-traffic-now.com
getdot.ru
gg-arena.ru
ghernnqr.skyrock.com
glass-msk.ru
gobongo.info
goforexvps.com goforexvps.com
gomusix.com gorabagrata.ru
gorodservis.ru greenzaim.ru
gox.com.ua
graphics8.info
graphicwe.org
gymi.name
habermetre.com habermetre.com
handicapvantoday.com hannasolution.ru
hao123.com harmonyglen.us
hcate.com hccoder.info
heliko.no heartofpayne.xyz
helvetia.com.ua helpmymacfaster.trade
high-speed1.net hem.passagen.se
hiwibyh.bugs3.com hentai-manga.porn
hobbyhuren24.net hermesreplica.win
home.myplaycity.com home-task.com
hostnow.men hostcritique.com
housediz.com hoste.octopis.com
humanelydrew.com hotcore.info
idegenvezeto.eu hotshoppymac.com
igru-xbox.net hotsocialz.com
images.gyffu.com hubbble.com
imagez.co igtools.club
impresagaia.it ihc2015.info
indiakino.net ilo134uloh.com
infowarcraft.ru imediadesk.com
insurple.com impotentik.com
interfucks.net inboundlinks.win
ip-guide.com inboxdollars.com
irunfar.com innodgfdriverhm.aircus.com
islamtoday.co.za invitefashion.com
istock-mebel.ru invivo.hu
jav-fetish.site ipostroika.ru
jbl-charge.info iqupdatetmz.win
joessmogtestonly.com istizanidinelikexanax.blogspot.com
jumptap.com itag.pw
kabbalah-red-bracelets.com jav-idol.com
justucalling32211123456789.tk
kamen-e.ru kamen-e.ru
katushka.net karting196.ru
kazan.zrus.org kartiniresto.com
kevblog.top kaz.kz
keyword-suggestions.com kenaba.su
keywordblocks.com ketrzyn.pl
keywordhut.com keywordsdoctor.com
kicknights.gq khafre.us
kinocccp.net kinosed.net
kiprinform.com
kit-opt.ru kit-opt.ru
krasivoe-hd.com kiwe-analytics.com
kustanay.kz klosetkitten.com
la-fa.ru knigonosha.net
ladov.ru knowyournextmove.com
lalalove.ru kolotiloff.ru
ledpolice.ru konturkrasoty.ru
lenpipet.ru kovesszucs.atw.hu
lesbian.xyz krasnodar-avtolombards.ru
lifebyleese.com krassh.ru
linerdrilling.com kurbappeal.info
linksharingt.com kurwa.win
localmatchbook.com kvartir-remont.biz
lonely-mature.com kwzf.net
luckyshop.net.ua landscaping.center
lunamedia.co laptoper.net
m0r0zk0-krava.ru law-check-seven.xyz
mafcards.ru lechtaczka.net
magnetpress.sk letsrepair.in
mainhp.net lexixxx.com
make-money-online.com link.ac
manifestyourmillion.com linkbolic.com
lion.conboy.us
lion.playtap.us
lyngdalhudterapi.no
m5home.ru
magiadinamibia.blogspot.com
mail.allnews24.in
makis.nu
mall.uk
mansiondelrio.co mansiondelrio.co
manuscript.su marblestyle.ru
mazda-roadsters.com maridan.com.ua
mb140.ru max-p.men
meccadumps.net maxthon.com
meddesk.ru medanestesia.ru
medicovi.com medicinacom.ru
mirobuvi.com.ua megagrabber.ru
missis.top megahdporno.net
misterjtbarbers.com message-warning.net
mncrftpcs.com metascephe.com
mobilemedia.md mikozstop.com
mobilierland.com mindeyegames.com
mojpregled.com misswell.net
mnimmigrantrights.net
mobot.site
moivestiy.biz
mole.pluto.ro
mompussy.net mompussy.net
montazhnic.ru motherboard.vice.com
moscow-region.ru mozello.ru
mrcsa.com.au mp3ringtone.info
mrinsidesales.com mtmtv.info
mturkcontent.com
muizre.ru muizre.ru
music.utrolive.ru muschisexbilder.com
myghillie.info music7s.me
myhealthcare.com musicdaddy.net
mylovelibrabry.com musicvidz.ru
mysex21.com my-bc.ru
naphukete.ru myblogregistercm.tk
nardulan.com myfreecams.com
netvouz.com myindospace.com
nfljerseyscheapchinabiz.com mymobilemoneypages.com
nina.az negociosdasha.com
nudejapan.net nero-us.com
nufaq.com newtechspb.ru
obiavo.ru nisuturnetdgu.tk
omgtnc.com njkmznnb.ru
oneshotdate.com nootrino.com
onload.pw norththeface.store
ontargetseo.us noumeda.com
ooo-gotovie.ru nrv.co.za
ordernorxx.com nsatc.net
orenburg-gsm.ru nudepatch.net
outshop.ru obiavo.in
ownshop.win oda.as
owohho.com offers.bycontext.com
oxotl.com oksrv.com
paleohub.info olgacvetmet.com
paperwritingservice17.blogspot.ru optitrade24.com
pateaswing.com optom-deshevo.ru
patol01.pw orhonit.com
perso.wanadoo.es outpersonals.com
petedrummond.com ovirus.ru
petsblogroll.com page2rss.com
photo-clip.ru pertlocogasilk.tk
picturesmania.com phpdevops.com
picurams.pw physfunc.ru
pierrehardysale.online picturesboss.com
pinsdaddy.com pinsdaddy.com
pisanieprac.info pinstake.com
plastgranar.nu piter.xrus.org
platinumdeals.gr piuminiita.com
poem-paying.gq pixshark.com
poffet.net plastgran.com
pons-presse.com play-mp3.com
ploenjitmedia.azurewebsites.net
pokemon-go-play.online
pony-business.com
pooleroadmedicalcentre.co.uk
popander.mobi
poppen-nw.net poppen-nw.net
porno-transsexuals.ru popup.matchmaker.com
pornohub.me poquoson.org
pornoklad.ru porndairy.in
pornosexrolik.com pornmania.pl
poshiv-chehol.ru porno-dojki.net
positive2b.ru pornogad.com
prizestohandle.club pornohd1080.online
promalp-universal.ru pornophoto.xyz
pronekut.com pornoplen.com
prpops.com pornosemki.info
psa48.ru pornozhara.com
pskcijdc.bloger.index.hr pornzone.tv
pts163.ru potolokelekor.ru
push-ad.com pp-budpostach.com.ua
quebec-bin.com preparevideosafesystem4unow.site
qwertty.net pricheskaonline.ru
rabotaetvse.ru privacylocationforloc.com
rada.ru privatefx.all4invest.info
rainbowice.ru profitwithalex.info
rankia.com promover.org
razyboard.com propranolol40mg.blogspot.com
rcb101.ru prostitutki-kharkova.org
prostitutki-tolyatti.org
pufip.com
py100.ru
qexyfu.bugs3.com
qwesa.ru
r-e-f-e-r-e-r.com
ratemodels.net
real-time-analytics.com
realresultslist.com realresultslist.com
recipedays.com refererx.com
rednise.com reversing.cc
reelheroes.net revistaindustria.com
registry-cleaner.net rfserial.net
relatodelpresente.com.ar rostov.xrus.org
remorcicomerciale.ro royalcar-ufa.ru
remybutler.fr sampleletters.net
renecaovillasale.online scanner-alex.top
repeatlogo.co.uk scanner-george.top
rethinkwasteni.info schalke04fc.info
royalads.net
rucrypt.com
rustic-quiver.win
rvi.biz
rybalka-opt.ru
ryetaw.com
s-iwantyou.com
s.lollypopgaming.com
sa-rewards.co.za
saddiechoua.com
sandhillsonline.com
saneyes.com
sarf3omlat.com
satellite.maps.ilovevitaly.com
sbt-aqua.ru
scanner-andrew.top
scanner-ivan.top
scanner-josh.top
scanner-julia.top
scat.porn
scmor.ilxc.cc scmor.ilxc.cc
screentoolkit.com scrapinghub.com
searchimage.co se.bnt-team.com
seasaltwithfood.com sell-fb-group-here.com
secret.xn--oogle-wmc.com serdcenebolit.com
seo-tools-optimizing.com sex-znakomstva.online
seofied.com share-buttons-for-free.com
sergiorossistore.online shiksabd.com
servethis.com shopvilleroyboch.com.ua
sex-tracker.com
sexkvartal.com
sexsearch.com
sexxdate.net
shemalegalls.blogporn.in
shikiso.info
ship-marvel.co.ua
sibecoprom.ru sibecoprom.ru
simplesite.com signx.info
sisiynas.ru silvercash.com
sitechecker.pro skachat-besplatno-obrazcy.ru
sitopreferito.it slkrm.ru
skylta.com
skypasss.com
sladkoevideo.com
sms2x2.ru sms2x2.ru
smsactivator.ru social-s-iii.xyz
social-s-ggg.xyz space2019.top
socialmadesimple.com spb.afora.ru
socialmediasuggest.com sps-shop.com
soda.media sribno.net
sotkal.lark.ru stairliftsarea.com
spammen.de stuffpride.com
sport7777.net success-seo.com
spylog.com supermama.top
stats-collector.org svarbit.com
stevemonsen.com t3chtonic.com
stop-zavisimost.com talant-factory.ru
strag-invest.ru taranerymagesswa.blogspot.com
strana-krasoty.ru taximytishi.ru
stuff-about-money.com teastory.co
styro.ru tecnoteakviareggio.it
sudexpert66.ru theguardlan.com
superlist.biz thejournal.ru
surffoundation.nl thewomenlife.com
survival.betteroffers.review thexart.club
swinger-mobil.net thiegs.reco.ws
szamponrevita.pl
tacbibirfa.tk
techart24.com
telefonsexsofort.tk
telegraf.by
telesvoboda.ru
terraclicks.com
thegalerie.eu
theporndude.com
thin.me.pn
timecrimea.ru
timer4web.com timer4web.com
tizanidine4mgstreetvalue.blogspot.com titslove.yopoint.in
tizanidinedruginteractions.blogspot.com tizanidinefibromyalgia.blogspot.com
tizanidinehcl2mg.blogspot.com tizanidinehydrochloride4mgstreetvalue.blogspot.com
tizanidinestreetprice.blogspot.com tonivedu.it
tizanidinezanaflex.blogspot.com
toonfamilies.net toonfamilies.net
tooplay.com tooplay.com
top10registrycleaners.com torture.ml
topmira.com tovaroboom.vast.ru
torrentdownloadhub.com tozup.com
torrentred.games tracklead.net
trackzapper.com trafficnetzwerk.de
traderzplanet.in triplepanda.xyz
traffictrade.life try-rx.com
trubywriting.com tsan.net
trumpetedextremes.com twojebook.pl
trustaffs.com typimga.pw
tt-ipd.info udayavani.com
tula.howotorg.ru udsgame.online
tvory.predmety.in.ua umg-stroy.ru
uhod-za-sobakoj.ru unrealcommander.biz
uk-zheu20.ru urlopener.com
uni.me uvozdeckych.info
uptime-alpha.net vadimkravtcov.ru
urccvfmc.bloger.index.hr vbabule.net
urldelivery.com venta-prom.ru
uroki.net ventelnos.com
vacuumcleanerguru.com videochat.life
validccseller.com videochat.mx
varikozdok.ru videospornogratisx.net
vegan-foods.us videtubs.pl
veselokloun.ru vietimgy.pw
vgoloveboli.net vinylvault.co.uk
videogamesecrets.com vitalads.net
vids18.site vitoriacabos.com
visionwell.com.cn vkgaleria.com
vksaver-all.ru vostoktrade.info
voronezh.xrus.org warning.or.kr
vozbujdenie.com wdrake.com
vucms.com weboptimizes.com
wallpaperstock.net website-speed-up.site
watchdogs-2.ru weddingdresses.xyz
waterpurifier.club wejdz-tu.pl
waycash.net westsextube.com
wdfdocando.com whengirlsgowild.com
webalan.ru where-toget.com
webinstantservice.com whereiskentoday.com
webnode.me witclub.info
websiteexplorer.info word-vorlagen.net
whatzmyip.net wpsecurity.website
whosonmyserver.com wttavern.com
wleuaprpxuvr.ga www.event-tracking.com
worldmusicfests.com x-musics.com
wosik-dach.service-for-web.de x-rates.ru
xn----btbdvdh4aafrfciljm6k.xn--p1ai xbaboon.com
xn----dtbndd4ae7eub.top xerox-douglas.cf
xn--b1addnj3cah.xn--p1ai xev.ru
xn--d1abj0abs9d.in.ua xgames-04.com
xn--lifehacer-1rb.com xn----7sbaphztdjeboffeiof6c.xn--p1ai
xxx-treker.ru xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xz618.com xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
yes-com.com xn--80aikhbrhr.net
yhit.press xn--b1adccaf1bzj.xn--p1ai
youraticles.pl xn--hxazdsfy.blogspot.com
youtubedownload.org xtube.com
yurgorod.ru xvideospornoru.com
yuweng.info xxxkaz.org
zarabiaj-dzis.pl y8games-free.com
zerocash.msk.ru yellowfootprints.com
zeroredirect6.com yottos.com
zeroredirect7.com youandcredit.ru
zhacker.net youtubologia.it
zigarettenonlinekaufen2.drupalgardens.com zacreditom.ru
zocaparj.kz zakonobosago.ru
zone-kev717.info zarabotat-na-sajte.ru
zrizvtrnpale.tk zebramart.ru
zscalerone.net zetgie.com.pl
zvetki.ru zog.link
zzlgxh.com zoolubimets.ru
zoompegs.com
zvooq.eu

480
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1008.su
1kdailyprofit.me 1kdailyprofit.me
214jaluwobapef.cf 6128786.com
3-letter-domains.net acgs.tk
4ur.click adclickthru.net
6hopping.com admeasures.com
883zy.com adserver-e7.com
absolute-s.ru alfa-img.com
acheterviagrafr24.com allcredits.su
add-add.men apelsinnik.website
ads-seo.men appearance-cool.com
advancedcleaner.com apps-analytics.net
ageofclones.com arabsexxxtube.com
aibolita.com asociatia-tipografilor-transilvania.ro
alienwheel.es autonew.biz
allsilver925.co.il aviav.org
alot.com avkzarabotok.com
alt-servis.ru backgroundpictures.net
anonymous-redirect.com bagcionderlab.com
apartamentwroclaw.eu baksman.com
apiwebwebgetcom-a.akamaihd.net balook.com
atelielembrancaqueencanta.com.br bankofthewext.com
autolombard-krasnodar.ru bestempresas.es
awency.com bestmarriages.com
azartclub.org betterhdporn.com
balkanfarma.org beyan.host.sk
bashtime.ru bibys.com
beetpics.pw bioscorp.ru
bestadbid.com blog.koorg.ru
bestofpicture.com bonanza-fortune.men
bestssaker.com bpro1.top
betonka.pro burn-fat.ga
betshuckclean.com candcstuccoandstone.com
bjanshee.ru cartierlove2u.com
boleznikogi.com casinorewards.info
bonky.biz cbox.ws
book-bets.com chatroulette.video
cialgenisrx.com cheapmarmot.online
cialischmrx.com chocolateslim-en-france.com
citysecurity.nu cigarpass.com
clean-start.net cognitiveseo.com
cmd.kz com-cleaner.systems
cobaltpro.ru
compliance-checker-7.info
compliance-checker.info compliance-checker.info
compliance-elena.xyz coolyarddecorations.com
compliance-fred.top copenergo.ru
contentlook.co country-chic.ru
copro.pw
covadhosting.biz
crafthubs.com crafthubs.com
customerpromos-a.akamaihd.net d-black.bz
cypernhuset.se dentalpearls.com.au
dating-app.ru
detoxmed24.ru
dichvuvesinhhanoi.com
dickssportinggoods.com dickssportinggoods.com
diesel-parts28.ru dikqlyremy.info
dirtpics.pw dimaka.info
domain-tracker.com dms-sw.ru
domaineaneblanc.com dogsrun.net
dorratex.tn doktoronline.no
downloadeer.net domini.cat
domoysshop.ru
duplicashapp.com
dwomlink.info
eandsgallery.com eandsgallery.com
earthmagic.info
easyshoppermac.com
ednorxmed.com
eduserver.net eduserver.net
eldiariodeguadalajara.com elektronischezigarettekaufen1.myblog.de
elektrischezigarette2.onsugar.com englishgamer.com
empis.magix.net essay-writing.work
evehemming.blogspot.com.au evening-dating-club.info
fast-wordpress-start.com everytalk.tv
fickenprivat.info exci.ru
filesmonster.porn f00kclan.de
filmci.pro falllow.gq
fireads.men findclan.org
flytourisme.org folowsite.com
forensicpsychiatry.ru free-today.com
foxydeal.com fxgallery.com
free-share-buttons.com ghostvisitor.com
free-share-buttons.top go2jump.org
freecamdollars.com gogps.me
freeseedsonline.com goldandcard.ru
freewhatsappload.com
frequiry.com
frvo.alptandem.ru
funnymama.com
fusoradio.info
gemara.com
gemgrab-a.akamaihd.net
girls-ufa.ru
glopages.ru
goroda-vsego-mira.ru goroda-vsego-mira.ru
greatfind-a.akamaihd.net gosreg.amchs.ru
hccoder.info growshop.es
help.tpu.ru gsasearchengineranker.space
hermesbracelets.xyz gymi.name
hledejvshopech.cz harmonyglen.us
homedecoguide.info hausfrauensex18.com
hotel-mkad.ru hermesbelts.xyz
hotelcrocenzi.sm hifidesign.ru
ilbe.club hollywoodactress.info
incolors.club hotdl.in
insomniagamingfestival.com hotenergy.ru
interferencer.ru icaseclub.ru
istizanidineacontrolledsubstance.blogspot.com ie.57883.net
justdating.online images-graphics-pics.com
jwss.cc infobabki.ru
kamalsinha.com interfucks.net
kartiniresto.com intimshop-fantasy.ru
keywordsuggest.org inzn.ru
kfon.eu ip-guide.com
klin3952.ru istripper.com
kotaku.com itch.io
koversite.info izismile.com
law-check-two.xyz janavibekken.no
letto.by jeremyeaton.co
likesdesign.com johnnyhaley.top
lion.playtap.us jolic2.com
lisque.batcave.net justucalling32211123456789.tk
littlesexdolls.com kazrent.com
livefixer.com keywordglobal.co.za
liveinternet.ro kinomaniatv.pl
ljusihus.se kostenlos-sexvideos.com
magazintiande.ru krasivoe-hd.net
magicplayer-s.acestream.net krasnodar.zrus.org
kumuk.info
labelwater.se
lacave.ntic.fr
lechenie-gemorroya.com
leftofcentrist.com
lego4x4.ru
lesbian.xyz
likrot.com
linerdrilling.com
linksharingt.com
live-sexcam.tk
mahnwachen-helfen.info mahnwachen-helfen.info
mailemedicinals.com mainhp.net
marktforschung-stuttgart.com marcogrup.com
mathgym.com.au martlinker.com
meccadumps.net max-p.men
meddesk.ru me-ke.com
milblueprint.com medizinreisen.de
mistr-x.org megakino.net
mixtapetorrent.com mikozstop.com
msk.afora.ru minneapoliscopiers.com
mural.co minyetki.ru
my-aladin.com mmostrike.ru
mydeathspace.com monarhs.info
napalm51.nut.cc monclerboots.xyz
monclervests.xyz
muabancantho.info
myindospace.com
myliveblog.ru
mylovelibrabry.com
mymercy.info
nac-bearings.ru
narkologiya-penza.ru
networkcheck.xyz networkcheck.xyz
nippon-bearings.ru newtechspb.ru
nottyu.xyz nosecret.com.ua
novgorod.xrus.org okeinfo.online
nyfinance.ml onefilms.net
o-dachnik.ru online247.ml
obiavo.uz ozoz.it
office-windows.ru pandarastore.top
online-sbank.ru pcimforum.com
oren-cats.ru perederni.net
owathemes.com pharmacyincity.com
partner-high.men photostudiolightings.com
partnerads.men picswe.com
picturesboss.com pirateday.ru
piulatte.cz pisanieprac.info
podshipniki-ntn.ru pojdelo.weebly.com
poquoson.org pornoplen.com
pornozhara.com pornpost.in
preg.marketingvici.com porodasobak.net
privetsochi.ru profitwithalex.info
proekt-mos.ru projefrio.com.br
pulse33.ru pskcijdc.bloger.index.hr
qld10000.net quizzitch.net
quangcaons.com ranksonic.com
rapidgator-porn.ga raschtextil.com.ua
rebuildermedical.com redhotfreebies.co.uk
richinvestmonitor.com relax.ru
sammlungfotos.online rocis.site
scanner-andrew.top rvzr-a.akamaihd.net
scanner-josh.top saletool.ru
scenarii-1-sentyabrya.uroki.org.ua sanjosestartups.com
segol.tv sankt-peterburg.nodup.ru
semxiu.com scanner-fred.top
sendearnings.com schoolfiles.net
shinikiev.com.ua scrapinghub.com
shitting.pro search.1and1.com
sibecoprom.ru searchinterneat-a.akamaihd.net
sibtest.ru seasaltwithfood.com
simple-share-buttons.com sevenstars7.com
sizeplus.work sex-znakomstva.online
skanninge.se sexpornotales.com
smart-balancewheel.com sextracker.com
smokewithrabbits.com sexvporno.ru
sobecjvuwa.com.ru sexyteens.hol.es
solitaire-game.ru shopperifymac.com
speedup-my.site shopwme.ru
splendorsearch-a.akamaihd.net sitevaluation.org
sptslmtrafms.com sklad-24.ru
srgwebmail.nl sledstvie-veli.net
staynplay.net social-buttons.com
stjamesschool.info solicita.info
storehouse.ua soundfrost.org
stromerrealty.com souvenirua.com
tabletkinaodchudzanie.com.pl spl63.fr
tacbibirfa.tk sport-video-obzor.ru
td-33.ru stop-gepatit.te.ua
templates.radiodigital.co strangeduckfilms.com
tgtclick.com stroiminsk.org
thedownloadfromwarez.blogspot.com tam-gde-more.ru
thelottosecrets.com tehngr.ru
theprofitsmaker.net template-kid.com
thesmartsearch.net textads.men
tiandeural.ru thebluffs.com
tiens2010.ru thiegs.reco.ws
tizanidine4mguses.blogspot.com time-japan.ru
tk-assortiment.ru tizanidinehcl2mgsideeffects.blogspot.com
top1-seo-service.com tizanidinerecreationaluse.blogspot.com
torture.ml tmm-kurs.ru
tracxn.com topcar-krasnodar.ru
traffic100.com traffic2money.com
traffic2cash.org traffictrade.life
trafficmania.com tryrating.com
trafficnetzwerk.de tsan.net
triumf-realty.ru twittruth.com
trustedhealthtips.com unimodemhalfduplefw.pen.io
tv-spoty.info ural-buldozer.ru
tyumen.xrus.org vacuumscleaner.com
ucoz.ru vepad.com
ucsol.ru via-energy-acquistare.com
ultramart.biz videositename.com
umnovocaminho.com vikistars.com
unblocksit.es voloomoney.com
vabasa.inwtrade.com vonradio.com
vbikse.com vsdshnik.com
viberdownload10.com vzlomfb.com
vielporno.net webstatsdomain.org
viven.host.sk weclipart.com
vroze.com woman-h.ru
vvpg.ru wzgyyq.com
watch-movies.ru x-rates.ru
website-speed-checker.site xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
weburok.com xn--80ab4aa2g.xn--p1ai
whatzmyip.net xn--d1aifoe0a9a.top
whiteproduct.com xn--oogle-wmc.com
withstandingheartwarming.com xtrafficplus.com
wordpress-crew.net xwatt.ru
wovis.site xxx-treker.ru
writersgroup580.web.fc2.com xxxkaz.org
x69ty.ru xz618.com
xgames-04.com yaaknaa.info
xjlottery.com yaderenergy.ru
yatrk.xyz yatrk.xyz
you-shall-not-pass.is74.ru yginekologa.com
youradexchange.com youghbould.wordpress.com
yourads.website za-fun-offer.com
yourserverisdown.com zakonobosago.ru
zetmaster.ru zskdla.site
zmoda.hostreo.com
zzlgxh.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.187 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3100 ### Version: V4.2022.05.3101
### Updated: Thu May 12 10:03:33 UTC 2022 ### Updated: Thu May 12 22:01:23 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3100 ### Version: V4.2022.05.3101
### Updated: Thu May 12 10:03:33 UTC 2022 ### Updated: Thu May 12 22:01:23 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##