vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 10:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.04.3026 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-04-06 10:09:21 +00:00
parent 548dfd8d05
commit d6ee298d48
46 changed files with 88939 additions and 88939 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3025 #### Version: V4.2022.04.3026
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3025 #### Version: V4.2022.04.3026
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.04.3025 #### Version: V4.2022.04.3026
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10982
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.04.3025 #### Version: V4.2022.04.3026
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10986
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10986
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10986
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10986
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.1.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.1.101 0;
127.0.0.1 0; 127.0.0.1 0;

10986
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.1.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.1.101 0;
127.0.0.1 0; 127.0.0.1 0;

10986
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.1.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.1.101 0;
127.0.0.1 0; 127.0.0.1 0;

10986
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3025 3026

10986
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10986
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.86.70.211 0; 20.124.27.153 0;

10986
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.1.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.1.101 0;
127.0.0.1 0; 127.0.0.1 0;

10986
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.1.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.1.101 0;
127.0.0.1 0; 127.0.0.1 0;

10986
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.1.101 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.1.101 0;
127.0.0.1 0; 127.0.0.1 0;

10986
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.86.70.211 0; 20.124.27.153 0;

10986
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.86.70.211 0; 20.124.27.153 0;

10986
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.1.101 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
LinqiaMetadataDownloaderBot BacklinkCrawler
Majestic-SEO DataCha0s
Mata Hari DigitalPebble
NetSpider Lftp
RocketCrawler Litemage_walker
SpaceBison Musobot
SpankBot Name Intelligence
ZoominfoBot OutclicksBot
meanpathbot Toata
x09Mozilla WPScan

292
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404enemy 360Spider
80legs
AIBOT AIBOT
ALittle Client
Abonti Abonti
Aboundex
Acunetix Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot
Aipbot Aipbot
Alexibot
Alligator
Anarchy Anarchy
Anarchy99
Ankit
Atomseobot Atomseobot
Attach
BBBike
BDCbot BDCbot
BLEXBot
BacklinkCrawler BacklinkCrawler
Bandit Battleztar Bazinga
BatchFTP
Bitacle Bitacle
Black Hole Black Hole
Blackboard
Blow Blow
BlowFish
Boardreader
BotALot BotALot
Buddy Brandwatch
Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CATExplorador
CODE87 CODE87
CSHttp CSHttp
CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect CensysInspect
CheeseBot
CherryPicker
Citoid
Cliqzbot Cliqzbot
Cocolyzebot Cocolyzebot
Collector CopyRightCheck
Copier
Cosmos
Craftbot
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler Crescent
CrunchBot DIIbot
CyotekWebCopy
DBLBot
DSearch DSearch
DatabaseDriverMysqli DTS Agent
Demon Demon
Deusu Devil
Digincore DigitalPebble
Dirbuster Dirbuster
Discobot
Discoverybot Discoverybot
DittoSpyder Dispatch
DnBCrawler-Analytics
DnyzBot
DomCopBot DomCopBot
DomainAppender
DomainCrawler
DomainStatsBot DomainStatsBot
Dotbot Domains Project
Download Wonder Dragonfly
Drip EasyDL
ECCP/1.0 Ebingbong
EMail Siphon EirGrabber
EMail Wolf
Ecxi
EroCrawler EroCrawler
Evil Evil
Exabot
Express WebPictures
ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
FHscan
FemtosearchBot FemtosearchBot
FlashGet Fimap
G-i-g-a-b-o-t Firefox/7.0
GetWeb Flunky
Foobot
Fuzz
GalaxyBot
Getintent Getintent
Go-Ahead-Got-It
GoZilla
Gotit Gotit
Grafula Grafula
GrapeFX
GrapeshotCrawler GrapeshotCrawler
GridBot HEADMasterSEO
HTTrack HMView
HTMLparser
Haansoft Haansoft
Havij Heritrix
HonoluluBot Humanlinks
Iblog HybridBot
IDBTE4M
IlseBot
Image Fetch Image Fetch
IndeedBot Image Sucker
InfoNaviRobot Indy Library
Intelliseek
InterGET InterGET
Iria Internet Ninja
Iskanie InternetSeer
IstellaBot IstellaBot
JOC Web Spider
Jbrofuzz Jbrofuzz
JennyBot Jetty
JikeSpider JikeSpider
JustView
Jyxobot
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Keyword Density
Kozmosbot Kozmosbot
Lanshanbot Larbin
LexiBot LeechFTP
Lightspeedsystems Libwhisker
Linkbot
Linkdexbot Linkdexbot
LinqiaMetadataDownloaderBot LinkextractorPro
LinkpadBot
LinksManager
LinqiaScrapeBot
Lipperhey Spider
Litemage_walker
Lmspider Lmspider
Mag-Net MIDown tool
MIIxpc
MJ12bot
MTRobot
Magnet Magnet
Mail.RU_Bot
Majestic SEO Majestic SEO
MarkMonitor Masscan
MarkWatch MauiBot
Mass Downloader
MeanPath Bot MeanPath Bot
Mediatoolkitbot Metauri
Mojolicious MicroMessenger
Minefield
Mister PiX
MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3 NICErsPRO
Musobot NPbot
Name Intelligence Navroad
Needle Needle
Nessus Nessus
NetLyzer
NetSpider
NetZIP NetZIP
NextGenSearchBot Not
Niki-bot Nuclei
NimbleCrawler
Nimbostratus
Nutch
Octopus Octopus
Offline Explorer Offline Explorer
OrangeSpider OnCrawl
OutfoxBot OpenLinkProfiler
Page Analyzer OpenVAS
PageAnalyzer PECL::HTTP
Pavuk PHPCrawl
PeoplePal PageThing.com
Petalbot
Pixray Pixray
PleaseCrawl PleaseCrawl
Pockey Pockey
Probethenet Probethenet
Psbot PxBroker
Pu_iN
Pump
Quick-Crawler Quick-Crawler
RankActive RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
RankingBot
RankingBot2 RankingBot2
Re-re Re-re
Reaper Reaper
RepoMonkey
Ripper Ripper
SBIder SBIder
SEOkicks-Robot
SEOprofiler SEOprofiler
SEOstats
SISTRIX
SalesIntelligent SalesIntelligent
SearchmetricsBot ScanAlert
ScoutJet
Seekport
SemanticJuice
Semrush
SemrushBot SemrushBot
SentiBot SentiBot
SeoSiteCheckup Seomoz
Siphon Site Sucker
SiteExplorer SiteCheckerBotCrawler
Siteimprove SiteLockSpider
Sitevigil SiteSucker
SmartDownload SlySearch
Snake Sociscraper
SpaceBison Sogou web spider
Sottopop
Spammen Spammen
Sqlworm SpankBot
Spanner
Stripper Stripper
SuperBot
SuperHTTP SuperHTTP
Surfbot SurveyBot
Suzuran
T0PHackTeam T0PHackTeam
T8Abot
Teleport Teleport
Telesoft
Telesphoreo
Telesphorep Telesphorep
The Intraformant
TheNomad
Thumbor
TightTwatBot TightTwatBot
Toweyabot Toweyabot
Tracemyfile
Trendiction Trendiction
Trendictionbot True_Robot
Turingos Turingos
TurnitinBot Turnitin
TwengaBot
Twice Twice
Typhoeus Typhoeus
URLy Warning URLy Warning
UnisterBot URLy.Warning
V-BOT
VCI VCI
Vagabondo Vagabondo
VelenPublicWebCrawler
VidibleScraper
Virusdie
VoidEYE
Voil
Voltron Voltron
WBSearchBot WBSearchBot
WWWOFFLE WWW-Mechanize
WeSEE WallpapersHD
Web Auto Web Enhancer
Web Collage WebBandit
Web Fetch WebReaper
Web Sauger
WebEnhancer
WebFetch
WebFuck
WebGo IS
WebLeacher
WebPix
WebSauger WebSauger
WebStripper
WebZIP
WebmasterWorldForumBot
Website Quester
Webster Webster
Who.is Bot Whatweb
Widow WinHTTrack
WiseGuys Robot WiseGuys Robot
Woobot
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade
Zauba Zauba
Zitebot
ZmEu ZmEu
ZumBot ZoomBot
adscanner adscanner
backlink-check archive.org_bot
arquivo-web-crawler
arquivo.pt
cah.io.community cah.io.community
crawl.sogou.com clark-crawler
demandbase-bot com.plumanalytics
eCatch
gopher gopher
heritrix internetVista monitor
ips-agent
lwp-request lwp-request
polaris version lwp-trivial
s1z.ru magpie-crawler
satoristudio.net page scorer
seoscanners pcBrowser
plumanalytics
scan.lol
seostar
sitechecker.pro sitechecker.pro
siteripz sogouspider
trendiction.com spyfu
ubermetrics-technologies.com tAkeOut
webgains-bot webgains-bot
webmeup-crawler webmeup-crawler
x22Mozilla x09Mozilla
zauba.io xpymep1.exe
zgrab

314
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
AIBOT 404checker
ALittle Client ADmantX
Aboundex Abonti
AfD-Verbotsverfahren Aboundexbot
Aipbot Acunetix
Alexibot
Alligator Alligator
AlphaBot
Anarchie Anarchie
Anthill Aspiegel
AwarioSmartBot Atomseobot
BBBike BBBike
BDFetch BDFetch
BackWeb BacklinkCrawler
BatchFTP
BetaBot BetaBot
Bigfoot Bigfoot
Bitacle
BlackWidow BlackWidow
BlowFish Blow
Boardreader Boardreader
Bolt Bolt
BotALot BotALot
BuiltBotTough Brandprotect
CODE87 Brandwatch
CSHttp Buddy
CazoodleBot CazoodleBot
Chlooe CheTeam
Citoid CherryPicker
Claritybot ChinaClaw
Cogentbot Cliqzbot
Collector Collector
Copier Copier
CopyRightCheck Cosmos
Copyscape
Craftbot
Crawling at Home Project
CrazyWebCrawler
Crescent Crescent
CrunchBot CrunchBot
Curious
Custo
DBLBot
DIIbot DIIbot
DSearch
DTS Agent
DataCha0s
DatabaseDriverMysqli
Deusu Deusu
Dirbuster Dirbuster
DittoSpyder Discobot
DomainCrawler Discoverybot
DomCopBot
DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
Domains Project
Download Wonder Download Wonder
Dragonfly Drip
ECCP/1.0
EasyDL
Ebingbong Ebingbong
EirGrabber EirGrabber
Evil Evil
Express WebPictures Express WebPictures
Extractor EyeNetIE
Extreme Picture Finder Ezooms
FDM Firefox/7.0
FemtosearchBot
FlashGet
Flunky Flunky
Foobot
FrontPage FrontPage
Fuzz Fuzz
FyberSpider Fyrebot
G-i-g-a-b-o-t GalaxyBot
GetRight GetRight
GetWeb Gigabot
Go!Zilla Go!Zilla
GoZilla GoZilla
GrabNet GrapeshotCrawler
Grabber HTMLparser
HEADMasterSEO HTTP::Lite
HMView HTTrack
HaosouSpider HaosouSpider
Havij Harvest
Heritrix HybridBot
Humanlinks IDBot
IDBTE4M IRLbot
Iblog Iblog
Id-search IlseBot
Image Sucker Image Fetch
InternetSeer Indy Library
JOC Web Spider InfoNaviRobot
JamesBOT Intelliseek
Jbrofuzz IstellaBot
JikeSpider JennyBot
Jyxobot Jyxobot
Kenjin Spider LNSpiderguy
Kinza LWP::Simple
Kozmosbot Lanshanbot
Larbin Leap
Libwhisker Lftp
Lightspeedsystems
Likse
LinkScan LinkScan
LinkWalker
Linkdexbot
LinkextractorPro
LinkpadBot
LinksManager LinksManager
LinqiaRSSBot LinqiaMetadataDownloaderBot
Lipperhey Lipperhey
Litemage_walker
Ltx71 Ltx71
MSFrontPage MIDown tool
Magnet MJ12bot
Mail.RU_Bot MSIECrawler
MTRobot
Mag-Net
Majestic SEO Majestic SEO
Majestic12 MarkMonitor
MauiBot Masscan
Mb2345Browser
Mediatoolkitbot
MegaIndex.ru
Metauri Metauri
Minefield Microsoft URL Control
Mister PiX
Mojeek Mojeek
MolokaiBot
Mozlila Mozlila
Mr.4x3 Mr.4x3
Name Intelligence Msrabot
Musobot
NICErsPRO
NPbot
Needle Needle
Net Vampire NetLyzer
Netcraft NetSpider
Nettrack
Netvibes Netvibes
NextGenSearchBot
Nibbler Nibbler
Niki-bot
Nikto
Ninja Ninja
Nmap
Nuclei
Octopus Octopus
Offline Explorer
Offline Navigator Offline Navigator
OnCrawl Openfind
OrangeBot OrangeBot
PECL::HTTP PECL::HTTP
PHPCrawl PageGrabber
PageScorer PeoplePal
Pandalytics Petalbot
Picsearch Pi-Monster
PleaseCrawl Picscout
PictureFinder
Piepmatz
Pixray
Pockey
ProWebWalker ProWebWalker
Probethenet
Psbot Psbot
Pu_iN Pu_iN
PxBroker PxBroker
RSSingBot
RankActive
RankActiveLinkBot RankActiveLinkBot
RankingBot
RankingBot2 RankingBot2
Re-re Rankivabot
ReGet ReGet
Reaper
RebelMouse
Recorder Recorder
RocketCrawler RedesScrapy
SEOkicks Ripper
SBIder
SEOlyticsCrawler SEOlyticsCrawler
SalesIntelligent Scanbot
Scrapy ScreenerBot
ScrepyBot Searchestate
Seekport SearchmetricsBot
SemanticJuice
SemrushBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot Seomoz
Siphon SiteExplorer
SiteCheckerBotCrawler SiteSnagger
Snapbot SiteSucker
Snoopy SmartDownload
SocialRankIOBot
Sogou web spider Sogou web spider
Sosospider Sottopop
Spammen
SpankBot SpankBot
Spinn3r Spbot
SputnikBot Sqlmap
Sqworm Sqworm
Steeler
Sucker Sucker
Sucuri Sucuri
SuperBot
Surfbot Surfbot
SurveyBot Swiftbot
T8Abot
Teleport Teleport
TeleportPro
Telesoft
Telesphorep
TheNomad TheNomad
Thumbor Titan
Toweyabot
Tracemyfile
Trendiction Trendiction
Trendictionbot
True_Robot True_Robot
TurnitinBot TurnitinBot
TwengaBot Twice
Typhoeus UnisterBot
V-BOT V-BOT
VCI VB Project
Vacuum VeriCiteCrawler
VelenPublicWebCrawler Virusdie
VoidEYE
Voil Voil
WBSearchBot WPScan
WISENutbot WWW-Collector-E
WWWOFFLE Wallpapers
Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE WeSEE
Web Collage Web Collage
Web Enhancer Web Pix
Web Fuck Web Sucker
Web Sauger WebCopier
WebAuto WebImageCollector
WebBandit
WebCollage
WebFetch
WebLeacher
WebPix
WebReaper WebReaper
WebSauger
WebStripper WebStripper
WebSucker WebWhacker
Whack WebZIP
Webalta
Webster
Whatweb Whatweb
Who.is Bot Who.is Bot
Widow Widow
WiseGuys Robot
Wonderbot Wonderbot
Wotbox Wotbox
Wprecon
Xaldon_WebSpider Xaldon_WebSpider
Zade Zade
Zauba
Zitebot Zitebot
ZoomBot ZmEu
arquivo-web-crawler ZumBot
autoemailspider adscanner
backlink-check archive.org_bot
cah.io.community arquivo.pt
coccocbot
cognitiveseo cognitiveseo
dataforseo.com crawl.sogou.com
eCatch crawler4j
heritrix demandbase-bot
instabid evc-batch
internetVista monitor
isitwp.com isitwp.com
lwp-trivial meanpathbot
muhstik-scan
oBot
page scorer
pcBrowser pcBrowser
polaris version polaris version
probe-image-size probe-image-size
seoscanners s1z.ru
scalaj-http
seobility
seostar
serpstatbot serpstatbot
sitechecker.pro sexsearcher
siteripz sogouspider
sp_auditbot
spyfu
ubermetrics-technologies.com
voyagerx.com
webgains-bot webgains-bot
webmeup-crawler webpros.com
webprosbot zauba.io
x09Mozilla
xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
7zap.com adrunnr.com
aab94f698f36684c5a852a2ef272e031bb.com apiqualitinknet-a.akamaihd.net
lookingglassemb.com coolgamechannel.com
mossmesi.com domcran.net
ochistka-stokov.ru electricwheelchairsarea.com
pladform.ru intellektmedia.at
robotixix.com myiptest.com
slavia.info ownshop.win
tsan.net textads.men
xn--e1afanlbnfckd7c3d.xn--p1ai zscalerone.net

926
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00it.com 03e.info
1-free-share-buttons.com 2itech.ru
100dollars-seo.com 3w24.com
1stat.ru 3xforum.ro
2345.com 5tey7463.weebly.com
23kw.ru 7yue.org
3weekdiet.com 7zap.com
404.mn a1security.com.ua
7a2rnuey1tw9ar.ru
8si.ru
aa9bd78f328a6a41279d0fad0a88df1901.com aa9bd78f328a6a41279d0fad0a88df1901.com
abcdefh.xyz aaa38852e886ac4af1a3cff9b47cab6272.com
acads.net aaronabel.com
addbin.men abbanreddy.com
adrunnr.com abcdeg.xyz
adsref.men adanih.com
adultnet.in adservme.com
agadelha.com.br adult-shop.com.ua
air-edem.ru aktobe.xkaz.org
alert-fjg.xyz akuhni.by
alif-ba-ta.com alkoravto.ru
all4wap.ru alta-realestate.com
allsilver925.co.il amateurgalls.com
alt-servis.ru anaseracresar.tk
amanda-porn.ga
angkortours.vn
angry-fermi-7633.arukascloud.io
animali.deagostinipassion.it animali.deagostinipassion.it
anonymousfox.co anonymous-redirect.com
apiholdingmypage-a.akamaihd.net aosexkontakte.net
apiitsthirteende-a.akamaihd.net apimegabrowsebiz-a.akamaihd.net
apilinkswiftco-a.akamaihd.net apiwebwebgetcom-a.akamaihd.net
app5.letmacworkfaster.world
appsecurityr.com appsecurityr.com
astana.xxxkaz.org arcteryxsale.online
atley.eu.pn arenda-avtoprokat-krasnodar.ru
autobrennero.it arenda-yeisk.ru
aviav.org arendas.net
avtolombard-krasnodar.com arquivo.pt
azartmsl.com atlant-auto.info
azte.ch avon-novosib.ru
baksman.com babespcs.com
bankiem.pl babosas.com
bankofthewext.com balla-boo.se
bannerads.de
bannerconnect.net bannerconnect.net
batanga.net bedcapdealers.com
best-seo-solution.com belinka.com.ua
bestmarriages.com bestcalovebracelet.cn
bestwaystogetpaid.us bestdraws.com
bestwebsiteawards.com beztuberkuleza.ru
betshuckclean.com bigcities.org
betteroffers.review biographiya.com
bezlimitko.xyz
bezsporno.ru
bidbuy.co.kr
bidr.trellian.com
bigcareer.info
bigpictures.club
bin-brokers.com
bio-market.kz
bjetjt.com
blockety.co
blockworld.ru blockworld.ru
blog.f00kclan.de blogsfunky672.weebly.com
blog.koorg.ru bmsco.co
blue-square.biz
bluesman.nu
bochemit.com.ua
bolegapakistan.com
bonzibuddi.com bonzibuddi.com
boo-arts.com brainsandeggs.blogspot.com
bosefux.esy.es braip.com.br
braindaily.xyz brokergid.ru
brus.city bscodecs.com
buscarfoto.com bumskontakte.org
buyessaynow.biz burger-imperia.com
buyessayonline19.blogspot.ru buyk.host.sk
cactussoft.biz buyparajumpers.online
calc-for-credit.ru buypuppies.ca
canrioloadm.gq cakemediahq.com.au
carfax.com.ua callawaygolfoutlet.online
cbb1smartlist12.click camel-beach.com
cdnanalytics.xyz candy-glam-hp.com
cdncash.com cartechnic.ru
celebsopics.com casablancamanor.co.za
centrdebut.ru
centre-indigo.org.ua
ceotrk.com ceotrk.com
cheapjerseysbizwholesale.us cercacamion.it
check-this-out-now.online cfacarrosserie74.com
chocolateslim-en-espana.com cheapbarbour.online
ciproandtizanidine.blogspot.com cheapestjerseysonwholesale.com
clickcash.com cheapjerseysap.com
club-lukojl.ru cheapjerseysfootballshop.com
cmhomestayagency.com cialischmrx.com
codq.info cialiswithoutadoctor.net
com-secure.download civilwartheater.com
classiquebijoux.ru
cleanallspyware.com
clickbank.net
compliance-george.top
compliance-irvin.top
compliance-jess.top
compliance-julianna.top compliance-julianna.top
copblock.org compliance-walt.top
cosmeticswomens-womensports.rhcloud.com compliance-willy.top
creatives.adbetclickin.pink contentlook.co
credit.co.ua coolyarddecorations.com
crest3d.ru copro.pw
darodar.com covid-schutzmasken.de
datract.com cpajunkies.com
dealighted.com crafthubs.com
dev-seo.blog crynet.cc
diamond-necklace.info customcollegeessays.net
digilander.libero.it cvety24.by
dildofotzen.net czat.wp.pl
diy-handmade-ideas.com dating-app.ru
domaineaneblanc.com davebestdeals.com
dood.live dealwifi.com
e-kwiaciarz.pl dear-diary.ru
diesel-parts28.ru
digest-project.ru
dkvorota.ru
domaincrawler.com
domainsatcost.com
doxysexy.com
dptaughtme.com
drpornogratisx.xxx
dumuelave.xyz
e2click.com e2click.com
electricwheelchairsarea.com ege-essay.ru
egvar.net
elektir.ru
elektrischezigarette1.onsugar.com elektrischezigarette1.onsugar.com
elektrozigarettekaufen2.blogse.nl elektrozigaretten1.postbit.com
elstal.com.pl ertelecom.ru
eluxer.net es5.com
elvenmachine.com essayassist.com
embedle.com etenininrade.ga
empis.magix.net eupornstar.info
eonpal.com evening-dating-club.info
epngo.bz exlarseva.webblog.es
eropho.net
erotik0049.com
espaceinventoristes.com
essay-edu.biz
eu-cookie-law.info
exct.net
expdom.com
expresstoplivo.ru
extractorandburner.com extractorandburner.com
eyessurgery.ru ezrvrentals.com
ezigarettenkaufen1.deviantart.com failingmarriege.blogspot.com
facialporntube.com falllow.gq
fakehandbags.xyz felizporno.com
fastfixing.tech ferieboligkbh.dk
fbdownloader.com fettefrauen.net
fellowshipoftheminds.com fil.ru
fickkontakte.org filunika.com.ru
filmidivx.com financeloan.us
finemanteam.com finejewelryshop.ru
firma-legion.ru firma-legion.ru
floridamobilebillboards.com floridahuntingfishingadventures.com
fm-upgrade.ru fodelsedagspresenter.nu
folowsite.com forsex.info
footbalive.org foto-basa.com
forex-procto.ru fotoxxxru.com
formseo.com foxjuegos.com
fortevidyoze.net
forum.tvmir.org
free-deals.faith
free-share-buttons.top
free-social-buttons.com free-social-buttons.com
freelotto.com free-video-tool.com
frequiry.com freelinkbuilding.website.tk
freza-sverlo.ru freesitetest.com
fuckmill.com freshberry.com.ua
funtoonez.com freshmac.space
fxtips.ru frivgame250.com
fyxabomiw.ru fuck-paid-share-buttons.xyz
fz139.ttk.ru funnel.co.za
gael-s.ru fusoradio.info
gambarkatabaru.com g33.org
garciniaxt.us gallictures.com
getaclueamerica.com gay-site.store
getdot.ru gaytube.com
getprismatic.com gazoblok.net.ua
gg.zzyjxs.com ghazel.ru
golden-praga.ru gilbertbanda.net
googleseo.com.tr girlsatgames.ru
girlsfuckdick.com
gk-atlant.info
gnuetella.com
gojiberriess.apishops.ru
goldenggames.com
gosarhivrt.ru gosarhivrt.ru
gosmeb.ru greatzip.com
gotomontenegro.net gwhwpxbw.bloger.index.hr
gotorussia.com gz2.bbsoldes.fr
gov.yanao.ru handicapbathtubarea.com
gsasearchenginerankerdiscount.com hatdc.org
hacktougroup.ru haveinc.xyz
hahashka.ru hazardky.net
hdxnxxtube.mobi hdwalls.xyz
hermesbracelets.xyz headpress.ru
heroz.fr heatpower.ru
hifidesign.ru hentaiheroes.com
histock.info hermesreplica.pw
hmywwogw.bloger.index.hr hitmuzik.ru
homedo.fabpage.com hitsbox.info
honyaku.yahoofs.jp hornymatches.com
hostingclub.lk horoshieokna.com
hplaserjetpdriver8y.pen.io hotblognetwork.com
husky-shop.cz hotelcrocenzi.sm
hystersister.com ibmdatamanagement.co
i4track.net ictizanidinehcl4mg.blogspot.com
icaseclub.ru imabase.com
idealtits.net imk.com.ua
ikritikimou.gr inboxdollars.com
imgarcade.com indo-export.ru
imgarit.pw infobabki.ru
imicrovision.com informatiecentro.be
impisr.ru infospot.pt
indiasourcemart.in instakink.com
inet-traffic.com insurple.com
infektsii.com iqoption.pro
infogame.name irkutsk.zrus.org
investpamm.ru
irkutsk.online-podarki.com
iron-age.info iron-age.info
isitwp.com istmira.ru
iskalko.ru ivanovo.zrus.org
ispac.org ivearchenceinflu.cf
istizanidinelikexanax.blogspot.com iwantmyfreecash.com
iyasimasennka.com ix20.ru
izoll.ru justdating.online
jagg.info kalandranis.gr
javrip.net khafre.us
jewelryandfiligree.com kino-fun.ru
jumptap.com kinofak.net
kanimage.com kiwi237au.tk
ketoanhanoi.info kmgamex.cf
keywordsuggest.org koversite.info
khadastoafarde.tk kovrenok.ru
kichenaid.ru kozhniebolezni.com
kihi.gdn krasivye-devushki.net
kino-key.info krasnodar.ru
kinostorm.net
kliksaya.com
kotaku.com
krasivoe-hd.com
kredit-pod-zalog-krasnodar.ru kredit-pod-zalog-krasnodar.ru
kretpicf.pw labplus.ru
kustanay.kz lalalove.ru
landoftracking.com landreferat.ru
law-check-nine.xyz laptoper.net
layola.biz.tc ledpolice.ru
lexiangwan.com lenvred.org
lietaer.com letmacwork.world
lightinghomes.net lion.snapmap.us
linkbuddies.com lkbennettoutlet.online
linkrr.com lmrauction.com
littlesexdolls.com
lizace.com
locimge.pw
lol-smurfs.com lol-smurfs.com
lomza.info lonerangergames.com
lottospring.com lottospring.com
louboutinreplica.xyz louboutinbooties.xyz
love-baby.cz luxmagazine.cf
lowephotos.info macrotek.ru
lyngdalhudterapi.no magento-crew.net
m1media.net makenahartlin.com
mabdoola.blogspot.com makis.nu
mackeeper-center.club manve.info
magda-gadalka.ru martlinker.com
mainlinehobby.net maxxtor.eu
maladot.com
mamasuper.prom.ua
mastershef.club
mathgym.com.au
mbiologi.ru
medi-fitt.hu medi-fitt.hu
mediawhirl.net medkletki.ru
medicines-choice.com meendo-free-traffic.ga
meet-flirt-dating.com mega-bony-2017.pl
mejoresfotos.eu megaindex.ru
mesbuta.info megakino.net
misswell.net methodsmarketing.com
misterjtbarbers.com mexicosleevegastrectomy.com
michellblog.online
microsoftportal.net
milblueprint.com
minyetki.ru
mlf.hordo.win mlf.hordo.win
mobot.site mmostrike.ru
mobplayer.net moesonce.com
monetizer.com-01.site mojaocena.com
moneytop.ru monarchfind-a.akamaihd.net
moonci.ru moonci.ru
morocco-nomad-excursions.com moroccosurfadventures.com
mosdverka.ru moscow.online-podarki.com
moskva.nodup.ru moscow.xrus.org
mossmesi.com
mototsikl.org mototsikl.org
mrbitsandbytes.com mrinsidesales.com
mrbojikobi4.biz msfsaar.de
mundoaberrante.com mtmtv.info
music.utrolive.ru musirc.com
musicas.baixar-musicas-gratis.com mydirtyhobby.com
my-cash-bot.co mygameplus.com
mydownlodablefiles.com mypets.by
mylesosibirsk.ru myxdate.info
myonigroup.com najaden.no
myprintscreen.com nakozhe.com
mzdish.site
nakrutka.cc nakrutka.cc
nalogovyy-kodeks.ru narkologiya-orel.ru
namecrumilchlet.tk narkologiya-voronezh.ru
needtosellmyhousefast.com naughtyconnect.com
net-profits.xyz nbsproject.ru
new7ob.com networkcheck.xyz
newfilmsonline.ru nikitsyringedrivelg.pen.io
newyorkhotelsmotels.info nina.az
nhl17coins.exblog.jp noumeda.com
o.light.d0t.ru npoet.ru
o333o.com nrv.co.za
offergroup.info obiavo.kz
officedocuments.net office-windows.ru
okel.co offtopic.biz
onclkads.com ohmyrings.com
onemactrckr.com olgacvetmet.com
online-sbank.ru olvanto.ru
online.ktc45.ru omgtnc.com
onlinemeetingnow.com onlinemeetingnow.com
ooo-gotovie.ru opinionreelle.com
ordernorxx.com optitrade24.com
parajumpersjakkesalgnorge.info oralsexfilme.net
partner-cdn.men ororodnik.goodbb.ru
pavlodar.xkaz.org ourtherapy.ru
pb-dv.ru outpersonals.com
pearlisland.ru owohho.com
pensplan.com paidonlinesites.com
perm.xrus.org par-fallen.ga
perosan.com partner-high.men
pflexads.com partner-host.men
photochki.com partnerads.men
pic-re.blogspot.com partybunny.ru
plaff-go.ru pc4download.co
poisk-zakona.ru pdns.download
polska-poezja.com perm-profnastil.ru
popunder.ru phobia.us
popup-hgd.xyz photokitchendesign.com
porno-transsexuals.ru picsforkeywordsuggestion.com
pornobrazzers.biz pizda.lol
pornolook.net plastjulgranar.se
pornosmola.info plusnetwork.com
pospr.waw.pl pogosh.com
powc.r.ca.d.sendibm2.com pogruztehnik.ru
preventheadacheguide.info pons-presse.com
printie.com poquoson.org
privatov-zapisi.ru porno-raskazy.ru
prizefestival.mobi porno.simple-image.com.ua
productarium.com pornofiljmi.com
progress-upakovka.ru pornoplen.com
prosmibank.ru pornosexrolik.com
prostitutki-belgoroda.org pornpost.in
prostitutki-kiev.org pornzone.tv
pssucai.info portnoff.od.ua
qcstrtvt.bloger.index.hr portside.cc
quality-traffic.com positive2b.ru
printingpeach.com
profit-opportunity.com
projectforte.ru
prokotov.com
promodj.com
ptr.ruvds.com
q-moto.ru
qexyfu.bugs3.com
queerspace.com
quickbuck.com
qwesa.ru qwesa.ru
r-e-f-e-r-e-r.com rankia.com
raavidesigns.com ranksonic.com
razamicroelectronics.com rapidsites.pro
rcb101.ru realtytimes.com
redirlock.com regionshop.biz
reeyanaturopathy.com registratciya-v-moskve.ru
reimageplus.com reklamuss.ru
relayblog.com relayblog.com
remote-dba.de remont-ustanovka-tehniki.ru
rentalcarnavi.info reward-survey.net
robot-forex.biz ringtonepartner.com
robotixix.com rn-to-bsn.com
rossmark.ru rosbalt.com.ua
royal-investments.net rossanasaavedra.net
ruclicks.com rus-pornuha.com
running-line.ru russintv.fr
ruspdd.com saecsa.co
s-iwantyou.com sankt-peterburg.nodup.ru
saletool.ru sanyuprojects.com
salutmontreal.com sc-specialhost.com
sampleletters.net scanner-george.top
sanatorrii.ru scanner-john.top
sanidumps.com scanner-walter.top
saveindex.xyz schoolfiles.net
scanner-ivan.top se-welding.ru
scanner-julianna.top
scanner-margo.top
scanner-viktor.top
searchencrypt.com searchencrypt.com
searchwebknow-a.akamaihd.net sell-fb-group-here.com
seksvideoonlain.com seo-platform.com
seoanalyses.com seo-traffic-ranking.info
seoheap.com seobility.net
seonetwizard.com seorank.info
seorankinglinks.us seotoolsagency.com
sex-dating.co service-core.ru
sexyteens.hol.es sexkontakteao.info
shinikiev.com.ua sexs-foto.top
sildenafilcitratemed.com sexuria.net
sitebeam.net sexvideo-sex.com
sitevaluation.org shariki-zuma-lines.ru
snip.tw shlyahten.ru
snjack.info shoppingmiracles.co.uk
social-buttons.com silvermature.net
softtor.com simplepooltips.com
songplanet.ru simply.net
spbchampionat.ru siteonomy.com
stairliftstrue.com skatestick.bid
staynplay.net skincrate.net
steame.ru skuteczna-dieta.co.pl
stickers-market.ru small-game.com
strag-invest.ru small-games.biz
suhanpacktech.com smokewithrabbits.com
sukirgenk.dvrlists.com snow.nvr163.com
summerlinhomes411.info social-s-ggg.xyz
superfish.com socialbookmarksubmission.org
svensk-poesi.com softomix.com
syvertsen-da.no softxaker.ru
tacbelarus.ru sonata-arctica.wz.cz
tacbibirfa.tk songoo.wz.cz
tackletarts.co space4updating.win
spaceshipad.com
spylog.com
startufa.ru
starwars.wikia.com
staticfs.host
stefanbakosab.se
sterva.cc
subj.ukr-lit.com
sucsesofinspiration.com
surffoundation.nl
surflinksmedical.com
swagbucks.com
swinginwithme.ru
taaaak.com
talant-factory.ru talant-factory.ru
tds-advert005.info taqplayer.info
tecspb.ru tedxrj.com
teenforporn.com tedy.su
telefonsex-ohne0900.net
telefonsexi.com
telegramdownload10.com telegramdownload10.com
thefotosgratis.eu tgtclick.com
thegamerznetwork.com the-trader.net
thegioixekhach.com theallgirlarcade.com
threecolumnblogger.com thebitcoincode.com
tiens2010.ru thefarmergame.com
tizanidineandcipro.blogspot.com theheroes.ru
tizanidinedosage.blogspot.com thepokertimer.com
tizanidinedrugtest.blogspot.com thesmartsearch.net
tizanidineorflexeril.blogspot.com thetattoohut.com
thiegs.reco.ws
timhost.ru
tizanidineandgabapentin.blogspot.com
tizanidinedosageforsleep.blogspot.com
tizanidinedruginteractions.blogspot.com
tizanidineforopiatewithdrawal.blogspot.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com
tizanidinerestlesslegsyndrome.blogspot.com
tizanidinesideeffects.blogspot.com tizanidinesideeffects.blogspot.com
tkanorganizma.ru tizanidinestreetprice.blogspot.com
toloka.hurtom.com tizanidineusedfor.blogspot.com
tooplay.com tizanidinezanaflex.blogspot.com
top-deal.com.pl tobeyouday.win
top-l2.com top-study.work
tourismvictoria.com topcar-krasnodar.ru
trafficgenius.xyz topcasinoratings.ru
trahic.ru topkarkas.com
trailer.cinemaflix.website torrentred.games
trk-4.net tpu.ru
tuberkulezanet.ru track-rankings.online
turist-strani.ru tradgardspartner.se
turizm.bz trucri.me
tversvet.ru trumpetedextremes.com
twittruth.com trustaffs.com
uac.net.au tubeoffline.com
uchebavchehii.ru tvnewsclips.info
uchil.net tvteleport.ru
udayavani.com twojebook.pl
uk-zheu20.ru ua.tc
ukrtvory.in.ua uginekologa.com
ukrup.com unrealcommander.biz
umityangin.net utrolive.ru
unlimitdocs.net validdomain.xyz
urdoot.win valoresito.com
uroki.net
ustion.ru
vacuumcleanerguru.com
valsalud.com
valueclick.com valueclick.com
vandrie-ict.nl vapomnoncri.tk
varbergsvind.se vchulkax.com
varikoz24.com vellings.info
veopornogratis.xxx viagra.pp.ua
vesnatehno.com videochat.cafe
video-chat.in
videochat.tv.br videochat.tv.br
videochat.world vipsexfinders.com
videooko.weebly.com voprosotvet24.ru
vip-file.com
vkontarkte.com
vsexkontakte.net
vucms.com vucms.com
w3data.co w7s.ru
wareseeker.com wallpaperaccess.com
warezkeeper.com walpaperlist.com
webadvance.club waycash.net
webcamtalk.net waysbetter.cn
website-datenbank.de
website-speed-up.top
weightatraining.com
welcomeauto.ru
wesharepics.info wesharepics.info
whatzmyip.net where-toget.com
whipme.yopoint.in while.cheapwebsitehoster.com
windowssearch-exp.com whiteproduct.com
wiosenny-bon-1500.pl wildnatureimages.com
wladimirpayen.com wladimirpayen.com
woman-orgasm.ru woodyguthrie.se
worldoffiles.ru wowcasinoonline.ooo
xedserver.com wrz0iuebwhp5fg.freeddns.com
xerox-douglas.cf wtsindia.in
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai www.get-free-traffic-now.com
xn--24-glceagatoq7c2a6ioc.xn--p1ai xgames-04.com
xn--b1adccaf1bzj.xn--p1ai xitjw.info
xn--d1aifoe0a9a.top xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xxxguitars.com xn----9sbubg3ambdfl1j.xn--p1ai
yebocasino.co.za xtrafficplus.com
yebocasino.com xxxkaz.org
yellocloud.be yellocloud.be
yellowfootprints.com ynymnwbm.bloger.index.hr
yoopsie.com yogamatsexpert.com
youdesigner.kz youghbould.wordpress.com
yougotanewdomain.com
ytmnd.com
yuarra.pluto.ro yuarra.pluto.ro
zakazfutbolki.com zahvat.ru
zhorapankratov7.blogspot.com zero1.it
zigarettenonlinekaufen2.drupalgardens.com zigarettenonl.canalblog.com
zixizop.net.ru zoominfo.com
zoogdiesney.com zrus.org

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1001desert.com 108shot.com
12u.info 23kw.ru
2000k.ru 2ads.co.uk
2daytrendingnews.com 40cg.com
7makemoneyonline.com 4kpics.rocks
aa8b68101d388c446389283820863176e7.com 76brighton.co.uk
abcguru.xyz 7ooo.ru
abiente.ru abusora.com
acads.net agecheckadult.com
adloads.net ageofclones.com
adrenalin-stk.ru allforminecraft.ru
adsfresh.men allpdfmags.net
adsref.men allstatesugarbowl.org
afftrack001.com
agroeconom.kz
aimayangzhi.com
algerianembassy.co.in
alyeskaresort.com alyeskaresort.com
amanda-porn.ga amazon-seo-service.com
anapa-inns.ru asscenihotosidea.blogspot.com
andpolice.com atdedinotuho.tk
androids-store.com autobudpostach.club
anonymousfox.co autolombard-krasnodar.ru
anticrawler.org
apiitsthirteende-a.akamaihd.net
apple.com-webbrowsing-security.review
appsecurityr.com
architecturebest.com
auto-zapchasti.org
avirasecureserver.com
avon-ukraine.com avon-ukraine.com
avtolombard-krasnodar.com avtochehli.by
bambi.ck.ua azartmsl.com
basisches-wasser.net baksman.com
bezcmexa.ru balans.shahterworld.org
bjorkbacken.nu banki76.ru
blog-hits.com banksrf.ru
blog.remote-computer.de batanga.net
book-bets.com bestbuy.ca
bookmark4you.com bestcalovebracelet.cn
builtwith.com bestmouthwash.club
burger-tycoon.com biznesrealnost.ru
buyhoverboard.com bnt-team.com
bonzibuddi.com
brandbucket.com
brokergid.ru
buqyxa.rincian.info
burkesales.com
buybest1.biz
buyparajumpers.online
buytizanidineonlinenoprescription.blogspot.com buytizanidineonlinenoprescription.blogspot.com
c-english.ru bycontext.com
camdolls.com callstevens.com
cardsdumps.com candy-glam-hp.com
carstrends2015.com canyougethighofftizanidine.blogspot.com
casablancamanor.co.za casinopinup-wins.com
cementaresearch.se cete.ru
cheap-trusted-backlinks.com chelnytruck.ru
children-learningreading.info chiblackhawks-jerseys.com
chimiver.info citetick.com
christianlouboutinshoes.xyz cleanpcnow.com
cialisndbrx.com clickzzs.nl
cialisovercounteratwalmartusa.com com-onlinesupport.site
cineacademy.ru compliance-barak.xyz
cjmc.info constantaservice.net
clubfashionista.com country-chic.ru
compliance-olga.top cubook.supernew.org
compliance-willy.top customcollegeessays.net
conocer-sanabria.com cyclobenzaprinevstizanidine.blogspot.com
coprofam.org dating-app.ru
costablue.xyz decoratinghomes.ga
cubs-jerseys.us dev33.dioniqlabb.se
dafatiri.com digest-project.ru
deanmoore.ie dildofotzen.net
detalizaciya-tut.biz dispo.de
digilander.libero.it djonwatch.ru
disruptingdinnerparties.com documentbase.net
dojki365.online dogsrun.net
eager-nash.188-93-233-196.plesk.page downloads-whatsapp.com
ebooknovel.club eandsgallery.com
ekaterinburg.xrus.org eas-seo.com
ekobata.ru
en.home-task.com
energydiet-info.ru
engines-usa.com
enthuse.computernetworksonline.com
etm-consult.de etm-consult.de
evidencecleanergold.com evrotekhservis.ru
ezigarettekaufen2.yolasite.com expertblog.info
ezigarettenkaufen1.deviantart.com eyes-on-you.ga
f0815.de ezigarettekaufen2.mpbloggar.se
fakehandbags.xyz fakehandbags.xyz
fertilitetsradgivningen.se fefo.gdn
fetlifeblog.com fighrofacciufreesig.ga
figuringmoneyout.com foxweber.com
flprog.com free-deals.faith
gagrasector.ru freemags.cc
galaxy-family.ru frivgame250.com
galaxyflowers.ru garciniaxt.us
ghostvisitor.com gazobeton-p.com.ua
glastecfilms.com.my getprismatic.com
glavtral.ru goodwinmetals.co
go2jump.org gowreckdiving.com
goodwriterssales.com gq-catalog.gq
gosreg.amchs.ru gribokstop.com
gsmlab.pl gta-top.ru
gsou.cf gunsvicceadadebt.tk
gwhwpxbw.bloger.index.hr guruofcasino.com
handsandlegs.ru gz2.bbsoldes.fr
hitcpm.com healgastro.com
hornymatches.com hitsbox.info
horoshieokna.com hmywwogw.bloger.index.hr
hostnow.men homeandhealth.ru
https-legalrc.biz honyaku.yahoofs.jp
hut1.ru hoporno.com
hydropump.su house.sieraddns.com
hystersister.com hoverboard360.de
hreade.com
hspline.com
hully.altervista.org
i-hobot.ru i-hobot.ru
idc.com.ua imagui.eu
igrovyeavtomaty777.ru indo-export.ru
imadedinner.net
insurple.com insurple.com
intervsem.ru intimshop-fantasy.ru
iscblog.info iqoption.com
jasonpartington.com ispaniya-costa-blanca.ru
javxxx18.com itbc.kiev.ua
jaxcube.info izismile.com
jo24news.com jerseyssportsshop.com
kenaba.su jobmarket.com.ua
kerch.site kam-dom.ru
kichenaid.ru kazan.zrus.org
klosetkitten.com kfon.eu
koptims.tiu.ru kinobaks.com
kosova.de klitimg.pw
kozhniebolezni.com kolotiloff.ru
lashstudia.ru kongoultry.net
law-check-two.xyz laxdrills.com
lechtaczka.net laxob.com
lego4x4.ru ldrtrack.com
letmacwork.world letslowbefast.today
letmacworkfaster.world lidoradio.com
letto.by linkbuddies.com
liffa.ru lion.snapmap.us
livesex-amateure.info live-sexcam.tk
lizace.com liver-chintai.org
luciddiagnostics.in locationdesaison.com
medizinreisen.de lol-smurfs.com
mega-bony2017.pl m3gadownload.pl
meget.co.za makemoneyonline.com
mhtr.be maranbrinfo.com.br
misterjtbarbers.com masturbate.co.uk
mixx.com mediaoffers.click
moesonce.com mojowhois.com
monclercoats.xyz money-for-placing-articles.com
moskva.nodup.ru musezone.su
muycerdas.xxx music7s.me
muz-tracker.net mydirtyhobby.com
myhitmp3.club myfreecams.com
mylida.org mylida.org
mypets.by namecrumilchlet.tk
narosty.com networkad.net
nfljerseysforsalewholesaler.com nodup.ru
ok-ua.info notfastfood.ru
okeinfo.online nudejapan.net
oklogistic.ru obiavo.net
online247.ml oda.as
orion-v.com oivcvx.website
pacificair.com onthemarch.co
panicatack.com ooomeru.ru
perm.xrus.org optikremont.ru
petedrummond.com oren-cats.ru
phelissota.xyz osnova3.ru
photostudiolightings.com parajumpersstore.online
physfunc.ru patol01.pw
pics-group.com pc-test.net
pl-vouchers.com pc-virus-d0l92j2.pw
plaff-go.ru pcgroup.com.uy
pochemychka.net phobia.us
poem-paying.gq pic-re.blogspot.com
pokerniydom.ru pillscheap24h.com
popcash.net piski.top
pornofeuer.com pivka.xyz
praisong.net porndl.org
preventheadacheguide.info pornokajf.com
pro-tec.kz pornophoto.xyz
qcstrtvt.bloger.index.hr pornstartits.xblog.in
raschtextil.com.ua predominant-invent.tk
relayblog.com prosperent.com
samoiedo.it pussyfleet.com
savetubevideo.com razvratnoe.org
scenicmissouri.us reklamuss.ru
sdi-pme.com restaurantlescampi.com
seoexperimenty.ru restorator-msk.ru
sevendays.com.ua rocis.site
sex-dating.co rosbalt.com.ua
sexblog.pw royalvegascasino.com
sheerseo.com roznica.com.ua
sicfor.bcu.cc ruex.org.ua
sildenafilcitratemed.com runtnc.net
sisi-go.ru rvi.biz
sitevaluation.org sajatvelemeny.com
snjatie-geroinovoy-lomki.ru saugatuck.com
snow.nvr163.com savefrom.com
snsdeainavi.info sc-specialhost.com
space4update.pw secret.xn--oogle-wmc.com
spaceshipad.com seo-traffic-ranking.info
spravka-medosmotr.ru seofied.com
spylog.com sergiorossistore.online
stats-public.grammarly.io serpstat.com
storehouse.ua sexkontakte-seite.com
sukarame.net shakhtar-doneck.ru
swingerseiten.com shiksabd.com
taqplayer.info silverage.ru
thecoral.com.br sisiynas.ru
thegamerznetwork.com sloopyjoes.com
tilido.com slujbauborki.ru
tizanidineforopiatewithdrawal.blogspot.com snyatie-lomki-v-stacionare.ru
topappspro.com socialbutton.xyz
topflownews.com solution4u.com
topquality.cf sovetogorod.ru
torrentgamer.net spb.afora.ru
totu.us start.myplaycity.com
tula.howotorg.ru statustroll.com
stmassage.ru
suchenindeutschland.com
tagil.zrus.org
tiandeural.ru
tizanidinedrugtest.blogspot.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com
tizanidinezanaflex.blogspot.com
trafficstars.com
tuckpointingmasonrysystems.com
twu.com.ua
uchetunet.su uchetunet.su
ucsol.ru ukr-lit.com
ultimateclassicrock.com unlimitdocs.net
underthesite.com unmaroll.ya.ru
unitygame3d.com uroki.net
upproar.com ussearche.cf
urlopener.blogspot.com.au vduplo.ru
vikistars.com vegan-foods.us
vpnmouse.com vellings.info
vseuznaem.com via-energy-cumpara.com
vzglyadriv.kg viandpet.com
vzlomsn.org video--production.com
warezaccess.com vod.com.ua
warningwar.ru vvpg.ru
webmasterhome.cn vysigy.su
westen-z.life watchmyfb.pl
whengirlsgowild.com waterpurifier.club
wineitudes.wordpress.com weblo.com
winner7777.net websiteexplorer.info
wiosenny-bon-1500.pl whiteelephantwellington.com
womama.ru whiteproduct.com
www.arenda-yeisk.ru wholesalejerseyscheapjerseys.us.com
wyniki-lista.pl witherrom55.eklablog.fr
xcc24.pl xcombear.ru
xn--80ahdheogk5l.xn--p1ai xedserver.com
xtube.com xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
yellowstonesafaritours.com you-shall-not-pass.is74.ru
yoshkarola.zrus.org yourdesires.ru
youjizz.com zeroredirect2.com
yun56.co zeroredirect8.com
yuweng.info zscalerone.net
zaloadi.ru
zixizop.net.ru
zzlgxh.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.1.101 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3025 ### Version: V4.2022.04.3026
### Updated: Tue Apr 5 22:01:21 UTC 2022 ### Updated: Wed Apr 6 10:03:31 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3025 ### Version: V4.2022.04.3026
### Updated: Tue Apr 5 22:01:21 UTC 2022 ### Updated: Wed Apr 6 10:03:31 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##