vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.05.3116 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-05-20 10:09:39 +00:00
parent 4d53f7180a
commit 9bdd7a445a
46 changed files with 105582 additions and 105582 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3115 #### Version: V4.2022.05.3116
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3115 #### Version: V4.2022.05.3116
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.05.3115 #### Version: V4.2022.05.3116
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13060
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.05.3115 #### Version: V4.2022.05.3116
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13064
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13064
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

13064
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13064
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

13064
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

13064
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

13064
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3115 3116

13064
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

13064
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.112.56 0; 52.173.19.233 0;

13064
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

13064
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

13064
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

13064
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.112.56 0; 52.173.19.233 0;

13064
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.112.56 0; 52.173.19.233 0;

13064
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.245 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Cliqzbot CODE87
Flunky Fimap
Mata Hari Freeuploader
NICErsPRO Id-search
PECL::HTTP Mass Downloader
PxBroker OrangeSpider
VB Project Sqworm
WEBDAV SuperHTTP
WebsiteExtractor WebFetch
scan.lol crawler4j

300
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
404enemy
80legs
ASPSeek ASPSeek
Aboundex Aboundex
Aboundexbot
Acunetix
AfD-Verbotsverfahren
AiHitBot AiHitBot
Alexibot
AllSubmitter AllSubmitter
Alligator Alligator
Anarchy Apexoo
Ankit BDCbot
Anthill
Asterias
AwarioRssBot
BBBike
BLEXBot
BackStreet
BackWeb BackWeb
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Barkrowler Bandit
BatchFTP
BetaBot BetaBot
Blow Bitacle
Boardreader Black Hole
BlackWidow
BlowFish
Bolt
BotALot BotALot
Buddy Buck
BuiltBotTough BuiltBotTough
CATExplorador BuiltWith
BuzzSumo
CSHttp CSHttp
Cegbfeieh Calculon
ChinaClaw CheTeam
CheeseBot
CherryPicker
Chlooe
Citoid Citoid
Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot Cogentbot
Collector
Copier Copier
CopyRightCheck
Craftbot
CrunchBot CrunchBot
Curious
Custo Custo
CyotekWebCopy DBLBot
DSearch
DTS Agent
DataCha0s
DatabaseDriverMysqli
Deusu
DigitalPebble DigitalPebble
Dirbuster Dirbuster
Disco Dispatch
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DnyzBot
DomainAppender
DomainCrawler
DomainSigmaCrawler
Dotbot Dotbot
Dragonfly Dragonfly
Drip ECCP/1.0
EMail Siphon
EMail Wolf
EirGrabber
Evil Evil
Exabot
Extractor Extractor
EyeNetIE ExtractorPro
FemtosearchBot FDM
FHscan
Firefox/7.0 Firefox/7.0
FlashGet
Flunky Flunky
Foobot
FyberSpider
GalaxyBot GalaxyBot
GermCrawler
GetRight GetRight
Go!Zilla GetWeb
Go-Ahead-Got-It
GrabNet GrabNet
Grafula
GrapeFX
GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
HMView
HTMLparser HTMLparser
HTTrack HaosouSpider
Haansoft Hloader
Havij HonoluluBot
Humanlinks Humanlinks
IRLbot HybridBot
Iblog IDBTE4M
IDBot
Id-search
InfoNaviRobot InfoNaviRobot
Intelliseek InfoTekies
InterGET InterGET
Internet Ninja
InternetSeer
IstellaBot IstellaBot
Jbrofuzz JOC Web Spider
JamesBOT
JetCar
Jetty
JikeSpider
JustView
Jyxobot
Kinza
Kozmosbot
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot Larbin
LeechGet LeechFTP
LieBaoFast LexiBot
LinkextractorPro Lightspeedsystems
LinkpadBot
LinksManager LinksManager
LinqiaScrapeBot
Lipperhey
Litemage_walker Litemage_walker
Ltx71
MIIxpc MIIxpc
MQQBrowser MTRobot
MSFrontPage MarkWatch
Majestic12
Masscan Masscan
Mata Hari
Mb2345Browser Mb2345Browser
MegaIndex.ru Mediatoolkitbot
Microsoft URL Control
Minefield
Mister PiX
Moblie Safari
Mojeek
MolokaiBot
Morfeus Fucking Scanner
Mr.4x3 Mr.4x3
Msrabot Msrabot
Musobot
NICErsPRO NICErsPRO
NPbot
Nameprotect Nameprotect
NearSite NearSite
Needle
NetAnts NetAnts
NetLyzer NetLyzer
NetMechanic NetMechanic
NetSpider NetSpider
Netcraft
Nettrack Nettrack
NextGenSearchBot Nibbler
Niki-bot Niki-bot
NimbleCrawler Nikto
Not Not
Nuclei
Nutch
Octopus Octopus
OpenVAS Offline Navigator
Openfind
Openvas Openvas
OutfoxBot OrangeBot
PECL::HTTP PECL::HTTP
Page Analyzer
PageAnalyzer
PageGrabber PageGrabber
PageThing.com PageThing.com
Petalbot Panscient
Pi-Monster Pavuk
PeoplePal
Picscout Picscout
Picsearch
PictureFinder
Piepmatz Piepmatz
Pixray PleaseCrawl
Pockey
ProPowerBot
ProWebWalker ProWebWalker
Probethenet Probethenet
Psbot Pu_iN
PxBroker Pump
QueryN Metasearch RSSingBot
RankActive RankActive
RankFlex RankingBot
RankingBot2 RankingBot2
ReGet RealDownload
Reaper
RebelMouse RebelMouse
SEOkicks RepoMonkey
SEOkicks-Robot Ripper
SEOlyticsCrawler RocketCrawler
SMTBot Rogerbot
SEOprofiler
SalesIntelligent SalesIntelligent
Scanbot Scanbot
Scrapy Screaming
ScreenerBot
ScrepyBot ScrepyBot
Searchestate Seekport
SearchmetricsBot SeobilityBot
SemanticJuice Seomoz
Semrush Shodan
SeoSiteCheckup SiteSucker
Siphon Siteimprove
SiteSnagger
Sitebeam
SlySearch SlySearch
SmartDownload SmartDownload
Snake Snake
SocialRankIOBot Snapbot
Sociscraper Sociscraper
Spammen Sogou web spider
Sosospider
SpaceBison
SpankBot SpankBot
Spbot Spinn3r
Sqlworm SputnikBot
Sqlmap
Sqworm Sqworm
Sucker
Sucuri
SuperBot
SuperHTTP SuperHTTP
SurveyBot Suzuran
Swiftbot Swiftbot
T8Abot Szukacz
Teleport
TeleportPro TeleportPro
Telesphoreo Telesoft
TheNomad Titan
TightTwatBot Trendictionbot
Toweyabot Turingos
Trendiction
Turnitin Turnitin
TurnitinBot TwengaBot
Twice Twice
URLy.Warning UnisterBot
V-BOT Vacuum
VoidEYE VeriCiteCrawler
Virusdie
Voil
WPScan WPScan
WWWOFFLE Web Collage
Wallpapers/3.0 Web Fetch
Web Sucker Web Fuck
WebAuto Web Pix
Web Sauger
WebBandit WebBandit
WebCopier
WebEnhancer WebEnhancer
WebImageCollector
WebReaper WebReaper
WebSauger
WebSucker
WebWhacker WebWhacker
WebmasterWorldForumBot
Webshag
Website Quester
WebsiteQuester WebsiteQuester
Whatweb
Widow Widow
WiseGuys Robot
Wonderbot
Wprecon
Xaldon WebSpider Xaldon WebSpider
YoudaoBot Zade
Zitebot Zitebot
ZmEu
ZoominfoBot ZoominfoBot
ZumBot ZumBot
arquivo.pt adscanner
coccocbot archive.org_bot
com.plumanalytics arquivo-web-crawler
crawler4j autoemailspider
backlink-check
cah.io.community
check1.exe
cognitiveseo
dataforseo.com
eCatch eCatch
gopher
ips-agent
isitwp.com isitwp.com
iubenda-radar iubenda-radar
magpie-crawler lwp-trivial
meanpathbot muhstik-scan
mediawords pcBrowser
plumanalytics s1z.ru
polaris version scalaj-http
satoristudio.net
scan.lol scan.lol
seocompany.store
seoscanners seoscanners
seostar seostar
sitechecker.pro sitechecker.pro
sp_auditbot tAkeOut
trendiction.com ubermetrics-technologies.com
trendiction.de
webgains-bot webgains-bot
webpros.com webmeup-crawler
x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe zauba.io
zgrab

314
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider AIBOT
404checker ALittle Client
Acunetix ASPSeek
Abonti
Aboundex
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot AiHitBot
Aipbot AllSubmitter
Alexibot
Alligator
AlphaBot AlphaBot
Anarchie Anthill
Apexoo
Asterias Asterias
Atomseobot Atomseobot
AwarioSmartBot AwarioSmartBot
Backlink-Ceck BBBike
Battleztar Bazinga BackWeb
BlackWidow Badass
Blackboard Bandit
Barkrowler
Bigfoot
Boardreader Boardreader
BotALot Bullseye
Buddy BunnySlippers
BuiltBotTough
BuiltWith
CATExplorador CATExplorador
CCBot
CSHttp
CazoodleBot CazoodleBot
Cegbfeieh ChinaClaw
CheTeam
CheeseBot
Chlooe
Claritybot Claritybot
Cliqzbot Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot
Collector Collector
Craftbot Copyscape
CrazyWebCrawler Crawling at Home Project
Crescent CrunchBot
Curious
Custo Custo
CyotekWebCopy CyotekWebCopy
DSearch DTS Agent
Deusu DataCha0s
DatabaseDriverMysqli
Devil
DigitalPebble
Dirbuster Dirbuster
Disco
Discobot
Discoverybot
Dispatch
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot
DomainAppender
DomainCrawler
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project EMail Wolf
Dotbot
EasyDL EasyDL
Ecxi
EroCrawler
Evil
Express WebPictures Express WebPictures
ExtLinksBot EyeNetIE
FemtosearchBot Ezooms
Firefox/7.0 FHscan
Foobot
Fuzz
FyberSpider
GalaxyBot GalaxyBot
GermCrawler Genieo
GetRight Go-Ahead-Got-It
Gotit Gotit
Grabber HMView
GrapeshotCrawler HTTP::Lite
HTMLparser
HTTrack
Haansoft Haansoft
HaosouSpider
Havij Havij
Heritrix Heritrix
IDBTE4M Humanlinks
IDBot
IRLbot
Id-search
IlseBot IlseBot
Image Sucker Image Fetch
Indy Library
Intelliseek Intelliseek
Internet Ninja InternetSeer
IstellaBot IstellaBot
JamesBOT
JetCar JetCar
Joomla JikeSpider
Jyxobot Jorgee
Keybot Translation-Search-Machine JustView
Kenjin Spider
Kinza Kinza
Larbin LNSpiderguy
LeechGet Lanshanbot
Leap
LexiBot LexiBot
LibWeb Libwhisker
LieBaoFast
Likse
LinkScan LinkScan
LinkWalker LinkextractorPro
LinkpadBot LinkpadBot
LinksManager LinqiaMetadataDownloaderBot
LinqiaRSSBot Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
MFC_Tear_Sample Lmspider
MIIxpc Ltx71
MSIECrawler MJ12bot
MTRobot MQQBrowser
Mag-Net
Magnet Magnet
Masscan Mail.RU_Bot
MauiBot Majestic SEO
Metauri Majestic-SEO
Majestic12
Mata Hari
Mb2345Browser
Meanpathbot
MegaIndex.ru
MicroMessenger MicroMessenger
Moblie Safari
Mojeek Mojeek
Mojolicious Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Name Intelligence
Musobot
NPbot
Nameprotect Nameprotect
Navroad NearSite
Nessus Nessus
Net Vampire Net Vampire
NetAnts NetAnts
NetLyzer NetMechanic
NetSpider
Netcraft Netcraft
NextGenSearchBot
Niki-bot
Nikto Nikto
NimbleCrawler NimbleCrawler
Nimbostratus
Ninja
Nmap
Nuclei Nuclei
Nutch Nutch
Offline Explorer Offline Explorer
Offline Navigator
OnCrawl OnCrawl
Openfind OpenLinkProfiler
OpenVAS
OrangeBot OrangeBot
OrangeSpider
OutfoxBot OutfoxBot
PECL::HTTP Page Analyzer
PageScorer PageAnalyzer
PageThing.com PageThing.com
Pavuk Papa Foto
Picsearch PeoplePal
Picscout
PictureFinder PictureFinder
Piepmatz
Pimonster Pimonster
ProPowerBot PleaseCrawl
ProWebWalker
Psbot
Pu_iN
PxBroker PxBroker
QueryN Metasearch RankActive
RankingBot RankFlex
RankurBot Rankivabot
Re-re
Reaper Reaper
RebelMouse RebelMouse
Ripper RedesScrapy
RepoMonkey
RocketCrawler RocketCrawler
SalesIntelligent
Scanbot Scanbot
Scrapy ScoutJet
ScrepyBot
SearchmetricsBot SearchmetricsBot
Seekport Seekport
Semrush
SentiBot SentiBot
SeoSiteCheckup Siphon
SeobilityBot Site Sucker
SiteExplorer
SiteSnagger
SiteSucker SiteSucker
Sitebeam
SmartDownload
Snake
Snapbot Snapbot
Sosospider SocialRankIOBot
Sogou web spider
SpaceBison SpaceBison
Spammen Spammen
SpankBot Spanner
Spinn3r Spinn3r
SputnikBot Sqworm
Steeler Steeler
Stripper Sucker
Sucuri
SuperBot
SuperHTTP SuperHTTP
SurveyBot SurveyBot
Swiftbot Suzuran
Telesoft Telesphorep
Thumbor Thumbor
Toweyabot Toata
Trendiction
True_Robot
Turingos
TurnitinBot
URLy Warning
URLy.Warning URLy.Warning
UnisterBot
Upflow Upflow
VB Project VB Project
Vagabondo VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
Virusdie Virusdie
VoidEYE Voltron
WASALive-Bot WASALive-Bot
WEBDAV
WISENutbot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize Wallpapers
WallpapersHD
WeSEE
Web Collage Web Collage
Web Enhancer Web Fetch
Web Pix WebBandit
WebAuto WebFetch
WebCollage
WebCopier
WebFuck
WebImageCollector WebImageCollector
WebLeacher
WebReaper WebReaper
WebStripper WebStripper
WebsiteExtractor WebmasterWorldForumBot
WebsiteQuester Webshag
Whacker Website Quester
WinHTTrack Webster
Xaldon WebSpider Whack
Xaldon_WebSpider WiseGuys Robot
YoudaoBot Wotbox
Wprecon
Zade Zade
ZoomBot Zauba
ZumBot Zermelo
Zeus
Zitebot
ZmEu
ZoominfoBot
adscanner
archive.org_bot archive.org_bot
arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider backlink-check
cah.io.community cah.io.community
clark-crawler clark-crawler
cognitiveseo cognitiveseo
crawl.sogou.com com.plumanalytics
eCatch crawler.feedback
heritrix crawler4j
instabid domainsproject.org
internetVista monitor evc-batch
linkdexbot gopher
magpie-crawler ips-agent
mediawords isitwp.com
lwp-trivial
muhstik-scan muhstik-scan
netEstate NE Crawler netEstate NE Crawler
oBot
page scorer
pcBrowser pcBrowser
polaris version probe-image-size
s1z.ru seoscanners
seobility serpstatbot
sexsearcher sogouspider
sp_auditbot
spyfu spyfu
sysscan ubermetrics-technologies.com
tAkeOut voyagerx.com
trendiction.com webprosbot
trendiction.de x22Mozilla
webpros.com
xpymep1.exe xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
himgaws.pw alltheviews.com
lidoradio.com blog.axant.it
michellblog.online compliance-alexa.xyz
pronekut.com conboy.us
prospekt-st.ru freelotto.com
scat.porn haikuware.com
splendorsearch-a.akamaihd.net imk.com.ua
sr-rekneskap.no kinohall.ru
tracklead.net spywareit.com
zagadki.in.ua sudexpert66.ru

958
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0daymusic.org 00go.com
1flag.co.za 00webcams.com
1hwy.com 1688.com
20pascals.nl 178evakuator178.ru
2345.com 1j7740kd.website
300richmond.co.nz 2000k.ru
7fon.ru 24videos.tv
7xc4n.com 2nt.ru
aa81bf391151884adfa3dd677e41f94be1.com 4ur.click
aaa38852e886ac4af1a3cff9b47cab6272.com 72-news.com
abcdeg.xyz 76brighton.co.uk
abclauncher.com 9icmzvn6.website
adloads.com aac52006ec82a24e08b665f4db2b5013f7.com
adultfriendfinder.com abovetherivernc.com
adultfullhd.com academiacsmendoza.org
adultgalls.com acgs.tk
advertex.info adultactioncam.com
afftrack001.com adultwebhosting.info
afora.ru afora.ru
agadelha.com.br akita.kz
akvamaster.dp.ua akusherok.ru
alldownload.pw alfa-medosmotr.ru
allfinweb.com alibestsale.com
allsilver925.co.il alldubai.biz
am15.net alloysteel.ru
allstatesugarbowl.org
alveris.ru
alvtank.se
am-se.com
amazingpic.net
amehdaily.com amehdaily.com
amung.us amospalla.es
anaksma.info ample-awards-today.us
angkortours.vn anapa-inns.ru
api.stathat.com angry-fermi-7633.arukascloud.io
apiallgeniusinfo-a.akamaihd.net animal-drawings.com
apiholdingmypage-a.akamaihd.net animalrank.com
apple.com-webbrowsing-security.science anticrawler.org
appmsr.org apartamentwroclaw.eu
approvedlocal.co.za apisecretsaucebi-a.akamaihd.net
asdfz.pro app5.letmacworkfaster.world
asiengirls.net arabgirls.adultgalls.com
asscenihotosidea.blogspot.com arcadeplayhouse.com
atlant-auto.info artdeko.info
atmagroup.ru artparquet.ru
australia-opening-times.com asiavirtualsolutions.com
autoplate.info autorn.ru
avkzarabotok.com aviav.org
avon-severozapad.ru avtochehli.by
avon-ukraine.com ayanaspa.com
avtovolop.ru azartniy-bonus.com
azartmix.com azazaporn.com
backlink4u.net azazu.ru
balkanfarma.ru azbaseimages.net
bannerbank.ru b00kmarks.com
badavit.com.ua
bagcionderlab.com
baksman.com
bamo.xsl.pt
baoxaydung.com.vn baoxaydung.com.vn
barbourjackets.online beachpics.com
bard-real.com.ua beachtoday.ru
belinvestdom.by bestdooz.com
belsetirehafi.tk
bestdraws.com
bestkfiledxd.cf
bestofupload.info
besttorrentknifta.weebly.com besttorrentknifta.weebly.com
bezcmexa.ru bgviagrachrx.com
bigcities.org bigpictures.club
bitcoinpile.com bjorkbacken.nu
blackcurranthumidifiers.site blacktwhite.com
blagovest-med.ru blackwitchcraft.ru
blog.axant.it blog.remote-computer.de
blogqpot.com bmsco.co
bolegapakistan.com bolegapakistan.com
boleznikogi.com bolitgorlo.net
boole.onlinewebshop.net bookmark4you.com
brakehawk.com boostmyppc.com
brateg.xyz brendbutik.ru
buckspar.gq bristolhotel.com.ua
bumascloset.com brothers-smaller.ru
buntube.net bscodecs.com
buyclomidonlaine.com builtwith.com
buytizanidineonline.blogspot.com buketeg.xyz
bvlgariring.xyz bumble.cheapwebsitehoster.com
canacopegdl.com buron.pw
capsderfudd.tk butstrap.space
carson.getenjoyment.net buyscabiescream.com
cartierjusteunclou.xyz buyviagraa.blogspot.com
cartierlovestore.xyz camel-beach.com
cartierreplica.top candytiner.com
chatroulette.life canrole.com
christianlouboutinsaleoutletonline.us carabela.com.do
cialischmrx.com cars-modification.net
clean-start.net cartechnic.ru
clickaider.com cartierwatch.xyz
clickhype.com caulderwoodkennels.com
clicksor.net cdncash.net
clicksotrk.com celejihad.info
compliance-alexa.top choice-credit.ru
compliance-checker.info christianlouboutinsaleonline.us
compliance-don.top cialgenisrx.com
compliance-jess.top cl.s7.exct.net
compliance-viktor.top clash-clans.ru
consorzioilmosaico.org codq.info
construmac.com.mx coinsspb.com
copyrightclaims.org coldfilm.ru
crazyprotocol.com colobit.biz
crd.clan.su communisave.co.za
credit.co.ua community.allhiphop.com
crystalslot.com compliance-andrew.top
data-mining.tk compliance-donald.xyz
dawlenie.com countbertwistdisp26.soup.io
ddpills.com counter.yadro.ru
ddsoldes.fr covid-schutzmasken.de
deadlinkchecker.com creams.makeforum.eu
dealitright.click creativehutindia.com
decenttools.com credit-online.ws
defenderxtactical.com cs-passion.pl
delayreferat.ru cubs-jerseys.us
dentfidemountpreach.tk customboxes4less.com
design-ideas.info customerpromos-a.akamaihd.net
detki-opt.ru cypernhuset.se
detsky-nabytek.info dayibiao.com
dfwu1019.info dbmkfhqk.bloger.index.hr
dichvuvesinhhanoi.com demian.kz
dikqlyremy.info demon-tweeks.com
dentalpearls.com.au
deplim.com
derevesendeco.com
deshevo-nedorogo.ru
design-lands.ru
destinationrealestate.com
detective01.ru
diamond-necklace.info
digitalfaq.com
dimfour.com dimfour.com
dipstar.org dispo.de
discovertreasure-a.akamaihd.net djekxa.ru
divisioncore.com dodlive.mil
dnsrsearch.com domaincrawler.com
documentbase.net domainsatcost.com
domain2008.com downloadme.life
domoysshop.ru drunkmoms.net
dotmass.top druzhbany.ru
downtuptv.gq druzhininevgeniy63.blogspot.com
dvr.biz.ua ecookna.com.ua
eandsgallery.com eduinfosite.com
earl-brown.info ekspertmed.com
easyukraine.com elexies.info
ed-shop01.ru elvenmachine.com
ekaterinburg.xrus.org enpolis.ru
ekspresihati.info envaseslotusama.com
elektrozigaretten1.tumblr.com eorogo.top
embedle.com eropho.com
emediate.eu esopini.com
energydiet24.ru essay-writing.work
enhand.se euroskat.ru
enthuse.computernetworksonline.com extener.org
eralph.tk extstat.com
escortplius.com ezigarettenkaufen1.deviantart.com
escortslet.net face.hostingx.eu
esnm.ru fandlr.com
este-line.com.ua fanpagerobot.com
estelight.ru farmingworm.com
estibot.com
etur.ru
european-torches.ru
event-tracking.com
ewebarticle.info
exhibitionplus.eu
exoclick.com
f-online.de
f00kclan.de
facecup.top
fashionindeed.ml fashionindeed.ml
fefo.gdn fbdownloader.com
ferretsoft.com fickenbumsen.net
filunika.com.ru fickenprivat.info
financetip.eu financepoints.eu
fitness-video.net findsexguide.com
fivedwld.cf finemanteam.com
flytourisme.org firstdrugmall.ru
folowsite.com flirt4free.com
forum69.info footballfarrago.com
freejabs.com fortunejack.com
freelifetimefuckbook.com forums.toucharcade.com
freeseedsonline.com fototravel.eu
freeuploader.ml free-share-buttons.blogspot.com
freshberry.com.ua free-social-buttons.com
front.to free-video-tool.com
funnypica.com freegamesplay.online
gabeshop.ru front.ru
gamblingnerd.com fx-brokers-review.com
fyl.com.ru
gamedayassist.com
germetiki.com.ua germetiki.com.ua
getmiro.com getaclueamerica.com
getyourimage.club ghernnqr.skyrock.com
gidonline.one gifspics.com
girls-ufa.ru gk170.ru
glavprofit.ru gkvector.ru
globalsurfari.com gogps.me
gojiberriess.apishops.ru goodhumor24.com
gopro-online.info googleseo.com.tr
gosmeb.ru gosreg.amchs.ru
gourcy.altervista.org gowreckdiving.com
gq-catalog.gq gratis-sexkontakte.com
greatdealshop.com greatzip.com
gsasearchengineranker.top gsasearchengineranker.pw
gsou.cf gsbs.com.ua
gyffu.com gtopstats.com
gymi.name guiadeserraazul.com
handicapvantoday.com guigyverpo.cf
hausfrauensex18.com gz2.bbsoldes.fr
help.tpu.ru headpress.ru
herehloadibs.cf helvetia.com.ua
highspeed5.net homeart.space
himgaws.pw homelygarden.com
hiwibyh.bugs3.com hoverboard360.es
hjaoopoa.top hoztorg-opt.ru
hostingclub.lk hystersister.com
hotblognetwork.com ideawheel.com
hothor.se ie.57883.net
hscsscotland.com iflycapetown.co.za
husky-shop.cz ihc2015.info
idealtits.net ilovevitaly.ru
igadgetsworld.com imabase.com
imagecoolpub.com imadedinner.net
images.gyffu.com
imediadesk.com
imgata.com
impresagaia.it
inbabes.sexushost.com inbabes.sexushost.com
inspiring-desperate.tk infazavr.ru
intelhdgraphicsgtdrive6w.metroblog.com infowarcraft.ru
intermesh.net intellektmedia.at
iron-age.info internetproviderstucson.com
iscblog.info iqoption.com
islamtoday.co.za iqupdatetmz.win
istanbulit.com iradiology.ru
istripper.com istmira.ru
it-max.com.ua itch.io
itronics.ca jaxcube.info
iyasimasennka.com johannesburgsingles.co.za
janavibekken.no johnrobertsoninc.com
japan-bearings.ru jwss.cc
jerseywholesalebizchina.com kaac.ru
junglenet-a.akamaihd.net kaidalibor.de
jus0wil.pp.ua karusel-market.ru
justdating.online kfon.eu
kamalsinha.com kinomaniatv.pl
kashubadesign.ru klumba55.ru
keywordsking.com kndxbkdx.bloger.index.hr
khadastoafarde.tk koleso24.com.ua
kimcurlrvsms.com komputers-best.ru
kino-rf.ru koopilka.com
kinobest.pl kosmetyki.tm.pl
kinoduh.ru kountrylife.com
knogg.net krafte.ru
knowyournextmove.com kupit-adenu.ru
kozhniebolezni.com lapitec.eu
krasivoe-hd.com lawyers.cafe
krassh.ru lecbter-relationships.ga
la-fa.ru legalrc.biz
ladsblue.com letsart.ru
letmacworkfaster.site lflash.ru
libertybilisim.com lifeinsurancekb.com
life.biz.ua limads.men
lineshops.biz lion.bolegapakistan.com
linkrr.com liveinternet.ru
linksharingt.com lmrauction.com
littleberry.ru lol-smurfs.com
localmatchbook.com lombia.co
lonely-mature.com louboutinshoes.xyz
lumb.co ltvperf.com
mackeeper-center.club lulea-auktionsverk.se
macotool.com luxup.ru
maju.bluesalt.co mackeeper-land-672695126.us-east-1.elb.amazonaws.com
mansparskats.com makis.nu
marcowebonyodziezowe.pl map028.com
mastershef.club maximilitary.ru
mebel-vstroika.ru maxthon.com
mecash.ru mearns-tractors.co.uk
mebeldekor.com.ua
medispainstitute.com.au medispainstitute.com.au
megaapteka.ru meltwater.com
megahdporno.net merryhouse.co.uk
meteostate.com metaxalonevstizanidine.blogspot.com
mex-annushka.ru minecraft-neo.ru
mexicotravelnet.com mobile.ok.ru
minet.club mobplayer.net
misslike.ru mobplayer.ru
mlvc4zzw.space modabutik.ru
mobsfun.net moyaterapiya.ru
monarchfind-a.akamaihd.net myhydros.org
monclerjacketsoutlet.win mypets.by
moncleroutletonline.pw mysexpics.ru
moncleroutletonline.xyz naperehresti.info
moneyteam24.com navalwiki.info
more-letom.ru netpics.org
morefastermac.trade newhairstylesformen2014.com
mowser.com nfljerseysforsalewholesaler.com
mrcsa.com.au nomuos.it
mural.co
musicvidz.ru
mycouponizemac.com
mydearest.co
mydirtystuff.com
mykings.pw
myliveblog.ru
myshopmatemac.com
nakozhe.com
nalogovyy-kodeks.ru
nalogovyykodeks.ru
nardulan.com
nevansk.ru
nippon-bearings.ru
nmrk.ru
nosecret.com.ua
notfastfood.ru
novgorod.xrus.org novgorod.xrus.org
novosibirsk.xrus.org nrv.co.za
nuker.com nullrefer.com
offers.bycontext.com o-o-11-o-o.com
oivcvx.website o-o-6-o-o.com
olvanto.ru obiavo.uz
onemactrckr.com oconto.ru
onlinemegax.com office2web.com
opinionreelle.com ok-ua.info
optitrade24.com oneminutesite.it
optom-deshevo.ru oneshotdate.com
owohho.com onetravelguides.com
oxotl.com online-x.ru
pacificair.com onlineslotmaschine.com
pammik.ru onlyforemont.ru
pateaswing.com osagonline.ru
pay2me.pl ownshop.win
pcboa.se ozoz.it
perfectplanned.com paccohichetoti.ml
perfectpracticeweb.com palocco.it
photochki.com paparazzistudios.com.au
phuketscreen.com partner-high.men
picmoonco.pw parvezmia.xyz
picturesify.com pensplan4u.com
pihl.se piccdata.com
pinkduck.ga pinsdaddy.com
pizda.lol piuminiita.com
playlott.com plastgranar.nu
podshipniki-ntn.ru plastweb.ru
poquoson.org platesauto.com
porn555.com playmsn.com
pornodojd.ru poisk-zakona.ru
pornofeuer.com pony-business.com
pornogad.com popcash.net
pornzone.tv popugauka.ru
postclass.com popunder.net
powc.r.ca.d.sendibm2.com porndl.org
pozdravleniya-c.ru pornosexrolik.com
prefersurvey.net pornotubexxx.name
pricheskaonline.ru presleycollectibles.com
prizesbook.online princevc.com
professionalwritingservices15.blogspot.ru privacyassistant.net
prostitutki-odessa.org privacylocationforloc.com
prostitutki-rostova.org privatefx-in.ru
pskcijdc.bloger.index.hr prizefestival.mobi
push-ad.com prod2016.com
pussyfleet.com productarium.com
puzzleweb.ru profolan.pl
q-moto.ru prostitutki-kharkova.org
qld10000.net prostitutki-tolyatti.org
qor360.com randalljhoward.com
quelle.ru rankexperience.com
ratemodels.net rcpmda.ikan1080.xyz
razamicroelectronics.com recinziireale.com
realizmobi.com rednise.com
realnye-otzyvy.info rentalcarnavi.info
recipedays.ru responsinator.com
relatodelpresente.com.ar revistaindustria.com
remont-fridge-tv.ru reward-survey.net
renecaovilla.online rightenergysolutions.com.au
renecaovillasale.online rique.host.sk
rennlist.com rogervivierforsale.com
rezeptiblud.ru
rock-cafe.info
rockma.se
rumamba.com
rus-teh.narod.ru rus-teh.narod.ru
rusenvironmental.net safe-app.net
russia-tao.ru salutmontreal.com
russia-today-video.ru savetubevideo.com
rutor.group scanmarine.info
s-forum.biz scanner-andrew.top
salmonfishingsacramentoriver.com
sammlungfotos.online
sandhillsonline.com
santechnik.jimdo.com
sape.top
saratov.xrus.org
savefrom.com
saveriopiazza.it
scanner-jack.top scanner-jack.top
scat.porn scanner-walter.top
scenarii-1-sentyabrya.uroki.org.ua scenicmissouri.us
searchwebknow-a.akamaihd.net schoolfiles.net
semrush.com sdelai-prosto.ru
senger.atspace.co.uk search.pch.com
seo18.su semalt.com
seofied.com seocdvig.ru
seofirmreviewsus.info seoholding.com
seolab.top seozoom.it
sex-pr.net serialsway.ucoz.ru
sexs-foto.top servisural.ru
shell-pmr.ru sex-tracker.com
shymkent.xkaz.org sex.hotblog.top
simon3.ru sexkontakte-seite.com
skutecznetabletkinaporostwlosow.pl sexpartygirls.net
sladkoevideo.com sexuria.net
slavia.info sexy-pings.com
slavic-magic.ru shop.acim.org
smallseotools.com shopperifymac.com
smartpet.ru shoppytoolmac.com
snip.tw shopvilleroyboch.com.ua
snomer1.ru signx.info
snow.nvr163.com silvermature.net
social-buttons.xyz similardeals.net
simply.net
skincrate.net
sneakyboy.com
social-search.me
softxaker.ru softxaker.ru
solicita.info sosdepotdebilan.com
songplanet.ru sotechco.co
spb-plitka.ru spacash.com
special-porn.com
spin2016.cf spin2016.cf
spy-app.info spl63.fr
spymac.net sport-video-obzor.ru
stair.registrydomainservices.com sprttrack.com
staynplay.net spywarespy.com
success-seo.com stauga.altervista.org
suggest-keywords.com strana-krasoty.ru
sundrugstore.com strv.se
sureone.pro stuff-about-money.com
syndicate.fun sukarame.net
syvertsen-da.no superfish.com
theendivechronicles.com suralink.com
thejournal.ru surfbuyermac.com
surflinksmedical.com
swiped.su
taboola.com
taqplayer.info
tasteidea.com
teguh.info
telefonsexi.com
terraclicks.com
textads.men
thaisamkok.com
thenews-today.info thenews-today.info
theporndude.com ticketsys.inetwd.com
tizanidineandgabapentin.blogspot.com tilido.com
tizanidinemusclerelaxant.blogspot.com titelhelden.eu
tizanidinesideeffectsweightloss.blogspot.com tizanidine4mgprice.blogspot.com
tnctrx.com tk-assortiment.ru
todohr.com
token-lab.org token-lab.org
tonerbox.kz tonivedu.it
tongkatmadura.info top-study.work
toolsky.com
topbestgames.com topbestgames.com
torrent-newgames.com torrentred.games
totu.us toursmaps.com
touchmods.fr toyota.7zap.com
tourcroatia.co.uk trafaret74.ru
tracksurf.daooda.com traffic2cash.org
trafficmania.com
trafficshaper.com trafficshaper.com
traffique.net traffmonster.info
traffixer.com trubywriting.com
tsyndicate.com truck-rece.life
tt-ipd.info truebeauty.cc
ttsq.fr
turizmus.us
tvory.predmety.in.ua
twincitiescarservice.com twincitiescarservice.com
typimga.pw ua-company.ru
tzritel.tk uggsale.online
ugguk.online ukkala.xyz
uhodzalijami.ru ultimateclassicrock.com
uk-zheu20.ru ultimatesetnewfreeallsoftupgradesystems.pw
ukkelberg.no umg-stroy.ru
unimodemhalfduplefw.pen.io uogonline.com
universals.com.ua uptime-alpha.net
url-img.link uptime-as.net
userequip.com upupa.net
vasileostrovsky-rayon.ru usswrite.com
vipps.com.my uzporno.mobi
viven.host.sk vadimkravtcov.ru
vkontarkte.com validdomain.xyz
vostoktrade.info valoresito.com
vucms.com vandrie-ict.nl
warezkeeper.com vbtracker.net
watchinf.com vchulkax.com
waysbetter.cn velobikestock.com
webscouter.net viandpet.com
website-datenbank.de video-hollywood.ru
wedding0venues.tk videojam.tv
winx-play.ru vita.com.hr
witclub.info vremya.eu
worldlovers.ru vseuznaem.com
www.ehscloud.cn vzlomfb.com
wyniki-lista.pl walpaperlist.com
x-stars.ru web.cvut.cz
xlolitka.com weboptimizes.com
xn--80aaagvmjabrs1aoc9luc.xn--p1ai website-speed-up.site
xn--80aodinpgi.xn--p1ai welck.octopis.com
xnxx-n.com weprik.ru
xtubeporno.net wheelchairliftsarea.com
xxxkaz.org whengirlsgowild.com
xzlive.com woman-tampon.ru
yaderenergy.ru world-mmo.com
yellowfootprints.com www.souvenirua.com
yellowstonesafaritours.com www.timer4web.com
yginekologa.com wygraj-teraz.com
you-shall-not-pass.is74.ru wzgyyq.com
yougame.biz xaylapdiendanang.com
xblog.in
xchat26.myfreecams.com
xcombear.ru
xep.info
xjlottery.com
xjrul.com
xmlinde.com
xmnb.net
xmronta.com
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xnxxandxvideos.com
xtrafficplus.com
youdesigner.kz
youporn-forum.ga youporn-forum.ga
youthreaders.com youradulthosting.com
yuweng.info zakon-ob-obrazovanii.ru
zaimite.ru zastenchivosti.net
zajm-pod-zalog-nedvizhimosti.ru zeroredirect1.com
zerocash.msk.ru zreloeporno.tv
zeroredirect11.com zvuker.net
zeroredirect12.com
zeroredirect5.com
zocaparj.kz
zog.link
zoominfo.com
zverokruh-shop.cz

480
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00author.com 100dollars-seo.com
1-99seo.com 8gold.com
1pamm.ru aanapa.ru
2drittel.de accessoires-mode.in
4rent.ru adhome.biz
7zap.com adtech.us
a14download.com adultnet.in
aac52006ec82a24e08b665f4db2b5013f7.com agadelha.com.br
acheterviagrafr24.com airlandsea.info
adconscious.com aliexpresscashback.club
adman.se allforminecraft.ru
akama.com amazingpic.net
all-streaming-media.com anaseracresar.tk
alloysteel.ru apper.de
animalia-life.club appmsr.org
apibatbrowsecom-a.akamaihd.net aquarium-pro.ru
appleid-verification.com arabgirls.adultgalls.com
appsaurus.com asseenontvonline.ru
arendakvartir.kz audiofree.ru
aviav.ru.com ausmepa.org.au
azbuka-mo.ru balkanfarma.org
bad-stars.net beauty-bracelet.com
balook.com becuo.com
bayareaaeroclub.org bestofferswalkmydogouteveryday.gq
best-seo-solution.com bezcmexa.ru
birzha-truda.eu biznesschat.net
blackcurranthumidifiers.site bjgugu.net.ua
blacktwhite.com bjorkbacken.nu
blackwitchcraft.ru
blockworld.ru
blog.koorg.ru
bluejays-jerseys.us bluejays-jerseys.us
bobinoz.com bosman.pluto.ro
brandimensions.com bupropion-sr-150-mg.us
browserprotecter.com burger-imperia.com
btvn.ru buypuppies.ca
buchananshardware.com buytizanidineonlinenoprescription.blogspot.com
bukleteg.xyz cakesplus.com.au
bumble.cheapwebsitehoster.com candcstuccoandstone.com
buron.pw candy-glam-hp.com
buyessaynow.biz candypeople.se
buyscabiescream.com centrdebut.ru
calstaterealty.us cheapestjerseysonwholesale.com
camdenmemorials.com
camsex.xxx-cam.webcam
canrole.com
cardul.ru
cartierreplica.pw
casinorewards.info
cheapmoncler.pw
cheapmoncler.win cheapmoncler.win
checkhit.com cheapwebsitehoster.com
cialiswithoutadoctor.net chee-by.biz
club-samodelkin.ru chimiver.info
coderstate.com cicaki.net
companies-catalog.com citizenclsdriveri7.pen.io
compliance-mark.top clean-start.net
cottageofgrace.com cv.wallhade.co
cryptoswap.biz cypernhuset.se
cutt.us dangphoto.trade
cyse.tk data-ox.com
czat.wp.pl digitalassetmanagement.site
delitime.info djonwatch.ru
demian.kz docsportal.net
dfiles.me e2click.com
dollartree.info educontest.net
doxysexy.com
dscaas.website
eaplay.ru
educatemetv.com
efnor-ac.com efnor-ac.com
egovaleo.it elektrozigarettekaufen1.devhub.com
election.interferencer.ru etm-consult.de
elexies.info evaashop.ru
eluxer.net exmasters.com
elvenar.com ez8motelseaworldsandiego.com
emediate.eu favornews.com
empathica.com fefo.gdn
enternet.ee filesdatabase.net
erotag.com filkhbr.com
erotikchat-24.com
essay-edu.biz
extstat.com
ezofest.sk
fatmaelgarny.com
findanysex.com findanysex.com
findpik.com findthe.pet
firstsiteguide.com foodgid.net
flavors.me fotoxxxru.com
frankofficial.ru fr.netlog.com
franch.info
freefoto.ca freefoto.ca
freevpn.space freelotto.com
germetiki.com.ua freshwallpapers.info
getaclueamerica.com frivgame250.com
getmiro.com galaxyflowers.ru
goodly.pro gamebackyard.com
gsasearchenginerankerdiscount.com gaz-voshod.ru
gunsvicceadadebt.tk gdzkurokam.ru
hdwallpapers-free.com gearsadspromo.club
health-medical-portal.info get-seo-domain.com
help.tpu.ru girlsatgames.ru
highland-homes.com girlsfuckdick.com
hop.clickbank.net globatur.ru
hoverboard360.nl goldandcard.ru
hustoon.over-blog.com good-mummy.ru
ilo134uloh.com goodhumor24.com
images-graphics-pics.com goosefishpost.bid
imagine-ex.co gsbs.com.ua
imediadesk.com hamilton.ca
impisr.ru hentai-manga.porn
investpamm.ru hermesreplica.pw
j33x.com hikesearch.net
jeremyeaton.co hledejvshopech.cz
jus0wil.pp.ua hobild.net
kareliatobacco.ru hoholikik.club
kashubadesign.ru hotxnights.info
kinky-fetishes.com howmuchdoestizanidinecost.blogspot.com
kinofak.net hydropump.su
kinohall.ru iboss.com
kozhniebolezni.com ideashome.id
krasivye-devushki.net ihc2015.info
krasnodar-avtolombards.ru iloveitaly.ru
laptoper.net imfamous.info
livejasmin.com incolors.club
indiasourcemart.in
intimshop-fantasy.ru
invest-pamm.ru
iqs.biz.ua
izhevsk.xrus.org
jacago.com
jetsli.de
journalhome.com
justkillingti.me
kandidos.com
karadene.com
kerei.ru
knowyournextmove.com
koopilka.com
kovrenok.ru
kredit-blog.ru
kriokomora.info
landscaping.center
law-check-nine.xyz
laxdrills.com
lazy-z.com
lignofix.ua
liran-locks.com
liver-chintai.org liver-chintai.org
m4ever.net lockerz.com
mailemedicinals.com louboutinbooties.xyz
malls.com m.b00kmarks.com
max-p.men map028.com
micasainvest.com mastershef.club
microsearch.ru matsdale.com
mobicover.com.ua mediaoffers.click
moskva.nodup.ru meendo-free-traffic.ga
mp3downloadhq.com megaindex.ru
mybackgroundlandscape.blogspot.com mixx.com
mycouponizemac.com mlvc4zzw.space
mylesosibirsk.ru mmostrike.ru
netpics.org mncrftpcs.com
nfljerseyscheapbiz.us mobile.ok.ru
njpalletremoval.com mole.pluto.ro
norththeface.store mompussy.net
nsatc.net monclercheap.xyz
nuit-artisanale.com monsterdivx.com
nvssf.com monthlywinners.com
oda.as montredemarque.nl
olgacvetmet.com moonci.ru
omoikiri-japan.ru mvpicton.co.uk
onemactrckr.com myfreecams.com
online247.ml myhealthcare.com
onlinetvseries.me myra.top
openfrost.com nailsimg.com
openmultipleurl.com najaden.no
otbelivanie-zubov.com newstraveller.ru
partnersafe.men niroo.info
pb-dv.ru nw-servis.ru
pdns.download okel.co
pensplan4u.com origin-my.ru
perm.xrus.org osagonline.ru
outrageousdeal-a.akamaihd.net
perso.wanadoo.es
pharmacyincity.com pharmacyincity.com
piccdata.com pintattoos.com
pizdeishn.com pivka.xyz
poffet.net pl-top.pl
poiskzakona.ru pl.id-forex.com
popup-hgd.xyz pochemychka.net
pornobrazzers.biz pogosh.com
pornowarp.info polybuild.ru
portal-eu.ru porndroids.com
powc.r.ca.d.sendibm2.com porno-chaman.info
productarium.com pornogad.com
psoriasis-file.trade pornohub.me
qualitymarketzone.com porodasobak.net
quizzitch.net promgirldresses.xyz
recordpage-a.akamaihd.net pron.pro
ricorsogiustizia.org putitin.me
ritlweb.com raschtextil.com.ua
rocketchange.ru razvratnoe.org
rossmark.ru relatodelpresente.com.ar
rvzr-a.akamaihd.net remmling.de
samolet.fr rewardpoll.com
sayyoethe.blogspot.co.za riders.ro
sbricur.com running-line.ru
sbtdesign.co.uk rusprostitute.com
scanner-elena.top sanatorrii.ru
seksvideoonlain.com screentoolkit.com
serving.adbetclickin.pink scrnet.biz.ua
sexobzor.info seansonline24.pl
sexvporno.ru serdcenebolit.com
shareyards.com servethis.com
shoplvlv.us sex-foto.pw
shtaketniki.kz sex-pr.net
shtora66.ru sex-tracker.de
simple-image.com.ua sex.hotblog.top
sitopreferito.it shinikiev.com.ua
smartpet.ru shitmovs.com
smartshoppymac.com shohanb.com
sportbetfair.com sibdevice.ru
storehouse.ua sibvitr.ru
stroyplus.ru simplesite.com
studentguide.ru sindragosa.comxa.com
studiofaca.com sitechecker.pro
support.nopeas.sk skladvaz.ru
sureone.pro smallseotools.com
susanholtphotography.com socialmadesimple.com
svetlotorg.ru socialsignals24.com
szucs.ru sonata-arctica.wz.cz
tandvardshuset.net stroydetali.ru
taxi-v-eisk.ru superkanpo.com
teencastingporn.com supernew.org
testbotprocessor44.com svtrd.com
texbaza.by telefonsexkostenlos.tk
thewomenlife.com tizanidinedosageforsleep.blogspot.com
tizanidinehcl4mgtab.blogspot.com tnctrx.com
tizanidineinteractions.blogspot.com top10-online-games.com
torrentred.games torrentgamer.net
toursmaps.com torture.ml
truck-land.life tuberkulezanet.ru
tubeline.biz typer.one
uhdtv.website uggsale.online
uhodzalijami.ru ukrtvir.com.ua
unpredictable.ga unisexjewelry.org
vapsy.com uptime-alpha.net
vezuviy.su vbabule.net
villakohlanta.nu
virtualbb.com virtualbb.com
virus-respirators.com volgograd.xrus.org
vizitki.net vse-pesni.com
vksaver-all.ru
vzlomfb.com
wahicbefa31.soup.io wahicbefa31.soup.io
watchmygf.net warezkeeper.com
warningwar.ru
waterefficiency.co waterefficiency.co
weblo.com webradiology.ru
webmasterhome.cn weburlopener.com
williamrobsonproperty.com weclipart.com
winner7777.net wedding0venues.tk
ws.ampower.me whiteproduct.com
www.pinnacle-bets.com www.get-free-traffic-now.com
www.wohnkabinen-shop.de xn----9sbebi2bvzr7h.xn--p1ai
x-stars.ru xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xdoza.com xn--h1aakne2ba.xn--p1ai
xkaz.org yellowfootprints.com
xmnb.net youtuhe.com
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai zajm-zalog-krasnodar.ru
xuki.us zionstar.net
xxxtubesafari.com zoominfo.com
yur-p.ru
zajm-pod-zalog-nedvizhimosti.ru
zdesoboi.com
zerocash.msk.ru
zolotoy-lis.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.245 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3115 ### Version: V4.2022.05.3116
### Updated: Thu May 19 22:01:32 UTC 2022 ### Updated: Fri May 20 10:03:39 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3115 ### Version: V4.2022.05.3116
### Updated: Thu May 19 22:01:32 UTC 2022 ### Updated: Fri May 20 10:03:39 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##