vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 06:15:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.05.3094 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-05-09 10:09:43 +00:00
parent a19b9dfd6f
commit cc0c461577
46 changed files with 96108 additions and 96108 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3093 #### Version: V4.2022.05.3094
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3093 #### Version: V4.2022.05.3094
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.05.3093 #### Version: V4.2022.05.3094
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11878
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.05.3093 #### Version: V4.2022.05.3094
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11882
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11882
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11882
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11882
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 1; 10.1.0.4 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 0; 10.1.0.4 0;
127.0.0.1 0; 127.0.0.1 0;

11882
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 1; 10.1.0.4 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 0; 10.1.0.4 0;
127.0.0.1 0; 127.0.0.1 0;

11882
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 1; 10.1.0.4 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 0; 10.1.0.4 0;
127.0.0.1 0; 127.0.0.1 0;

11882
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3093 3094

11882
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11882
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.228.196.230 0; 20.245.140.23 0;

11882
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 1; 10.1.0.4 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 0; 10.1.0.4 0;
127.0.0.1 0; 127.0.0.1 0;

11882
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 1; 10.1.0.4 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 0; 10.1.0.4 0;
127.0.0.1 0; 127.0.0.1 0;

11882
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 1; 10.1.0.4 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 0; 10.1.0.4 0;
127.0.0.1 0; 127.0.0.1 0;

11882
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.228.196.230 0; 20.245.140.23 0;

11882
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.228.196.230 0; 20.245.140.23 0;

11882
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 1; 10.1.0.4 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Badass Alexibot
Barkrowler Anarchy99
CCBot Craftbot
JennyBot EMail Wolf
LieBaoFast PHPCrawl
Niki-bot SiteLockSpider
Screaming VidibleScraper
Stripper WebBandit
WebLeacher Whacker
ubermetrics-technologies.com internetVista monitor

298
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404enemy 404enemy
80legs 80legs
ADmantX ADmantX
ALittle Client AIBOT
Abonti Aboundex
Aboundexbot Acunetix
AfD-Verbotsverfahren
AhrefsBot
Aipbot
Alexibot Alexibot
AllSubmitter AllSubmitter
AlphaBot AlphaBot
Anarchie
Anarchy
Anarchy99 Anarchy99
Ankit
Anthill
Apexoo
Aspiegel
Asterias
Attach Attach
AwarioSmartBot AwarioSmartBot
BDCbot BDFetch
BLEXBot
BackDoorBot
BackWeb
BacklinkCrawler BacklinkCrawler
Badass
Bandit
Barkrowler Barkrowler
Battleztar Bazinga
BetaBot
Bitacle Bitacle
Black Hole
BlackWidow BlackWidow
BlowFish Blow
Brandprotect BotALot
Brandwatch Buck
Buddy
BuiltBotTough BuiltBotTough
Bullseye Bullseye
Calculon BuzzSumo
Cegbfeieh CazoodleBot
CensysInspect CensysInspect
CheTeam CheTeam
CheeseBot CheeseBot
CherryPicker
ChinaClaw
Chlooe Chlooe
Citoid Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot Collector
Cogentbot Copyscape
Copier Craftbot
CopyRightCheck
Cosmos
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Crescent
CrunchBot CrunchBot
DSearch DBLBot
Deusu DIIbot
DatabaseDriverMysqli
Devil Devil
Discobot
Discoverybot Discoverybot
Dispatch
DittoSpyder
DnyzBot DnyzBot
DomainAppender DomCopBot
DomainSigmaCrawler DomainStatsBot
Domains Project Domains Project
Dragonfly
Drip
EasyDL
Ecxi
EirGrabber
EroCrawler EroCrawler
Exabot Express WebPictures
Extractor ExtractorPro
Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms FDM
FHscan FHscan
FemtosearchBot Fimap
Firefox/7.0 Fuzz
FlashGet
Freeuploader
FyberSpider FyberSpider
G-i-g-a-b-o-t
GalaxyBot GalaxyBot
GoZilla Genieo
Gigabot
Gotit
GrabNet
Grabber
GridBot
HEADMasterSEO HEADMasterSEO
HMView HMView
HTTrack HTTP::Lite
Haansoft Haansoft
HaosouSpider Heritrix
Harvest HonoluluBot
Hloader
Humanlinks
HybridBot HybridBot
IDBTE4M IDBTE4M
IRLbot IDBot
Iblog
IlseBot IlseBot
Image Fetch Indy Library
InfoNaviRobot InfoTekies
Intelliseek
InterGET InterGET
InternetSeer InternetSeer
Iria Iria
JikeSpider Iskanie
Jbrofuzz
JennyBot
Jorgee Jorgee
Jyxobot Kozmosbot
Keyword Density
Kinza
LNSpiderguy
Lanshanbot Lanshanbot
Larbin
Leap Leap
LibWeb LeechFTP
LeechGet
Lftp
Libwhisker Libwhisker
Lightspeedsystems
Likse Likse
LinkWalker
Linkbot Linkbot
LinksManager LinksManager
LinqiaMetadataDownloaderBot Lipperhey
LinqiaScrapeBot Litemage_walker
Lipperhey Spider
Lmspider Lmspider
Ltx71 MJ12bot
MFC_Tear_Sample
MQQBrowser MQQBrowser
MSFrontPage MSIECrawler
MTRobot Majestic12
Mail.RU_Bot MarkMonitor
Mass Downloader Masscan
Meanpathbot MauiBot
MegaIndex.ru MegaIndex.ru
Minefield Minefield
Mister PiX Mister PiX
Moblie Safari Moblie Safari
Mozlila Mojolicious
Musobot NPbot
NearSite Needle
Nessus
Net Vampire Net Vampire
NetZIP NetMechanic
Nibbler NetSpider
Octopus NextGenSearchBot
Offline Navigator Niki-bot
OnCrawl Nimbostratus
OpenLinkProfiler Not
OpenVAS Nuclei
Openfind Nutch
OutfoxBot Openvas
POE-Component-Client-HTTP OrangeSpider
PageScorer PECL::HTTP
PageThing.com PageAnalyzer
Panscient PageGrabber
Pandalytics
Pavuk
PeoplePal
Petalbot Petalbot
Pi-Monster Pi-Monster
PictureFinder
Piepmatz Piepmatz
Pimonster Pimonster
Pixray
PleaseCrawl
Probethenet Probethenet
Pu_iN Pu_iN
Pump
PyCurl
Quick-Crawler Quick-Crawler
RankActiveLinkBot RSSingBot
Rankivabot RankurBot
RocketCrawler RedesScrapy
Rogerbot SEOkicks-Robot
SBIder SEOprofiler
SEOlyticsCrawler SEOstats
ScrepyBot ScanAlert
ScoutJet
Searchestate Searchestate
SearchmetricsBot SearchmetricsBot
Seekport
SemanticJuice
SemrushBot SemrushBot
SentiBot
SeoSiteCheckup SeoSiteCheckup
Site Sucker Seomoz
SiteExplorer Siphon
SiteSnagger SiteCheckerBotCrawler
SiteLockSpider
SiteSucker SiteSucker
Sitevigil Sitebeam
SlySearch SmartDownload
Snake Snake
SocialRankIOBot
Sociscraper Sociscraper
Sogou web spider Sogou web spider
Spammen Sosospider
SpankBot Sottopop
Spanner Spinn3r
Steeler SputnikBot
Sqlmap
Sqlworm
Stripper Stripper
SuperBot
SuperHTTP SuperHTTP
SurveyBot Surfbot
Szukacz
T0PHackTeam
T8Abot T8Abot
Teleport Teleport
The Intraformant Telesphoreo
Titan Thumbor
TightTwatBot
Toweyabot
Tracemyfile
Trendiction Trendiction
True_Robot Trendictionbot
Turingos
TurnitinBot TurnitinBot
TwengaBot
Twice Twice
Typhoeus
URLy.Warning URLy.Warning
UnisterBot
Upflow Upflow
V-BOT VelenPublicWebCrawler
Vagabondo Virusdie
Voil Voil
Voltron Voltron
WASALive-Bot WASALive-Bot
WEBDAV WEBDAV
WPScan Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE
Web Auto Web Auto
Web Enhancer Web Enhancer
Web Fetch Web Fetch
Web Fuck
Web Pix Web Pix
WebEnhancer Web Sucker
WebFetch WebFetch
WebGo IS
WebImageCollector WebImageCollector
WebPix
WebStripper
WebSucker WebSucker
Webalta Webshag
WebmasterWorldForumBot WebsiteExtractor
WebsiteQuester Whack
Whacker Widow
WinHTTrack
Wonderbot
Woobot Woobot
Wotbox Wprecon
Xenu Xaldon_WebSpider
YoudaoBot
Zade
Zeus
ZoomBot ZoomBot
archive.org_bot ZoominfoBot
ZumBot
ZyBorg
adscanner
arquivo-web-crawler arquivo-web-crawler
cah.io.community backlink-check
clark-crawler
cognitiveseo cognitiveseo
crawler.feedback com.plumanalytics
demandbase-bot facebookscraper
evc-batch heritrix
instabid instabid
lwp-request lwp-request
lwp-trivial muhstik-scan
meanpathbot oBot
plumanalytics
polaris version polaris version
scalaj-http ripz
s1z.ru
seobility
seocompany.store
sexsearcher sexsearcher
sitechecker.pro siteripz
sp_auditbot
tAkeOut tAkeOut
webprosbot webgains-bot
x09Mozilla zauba.io
zgrab zgrab

314
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
404enemy ASPSeek
AhrefsBot Aboundex
AllSubmitter Aboundexbot
Anarchy99 AfD-Verbotsverfahren
Apexoo Aipbot
Backlink-Ceck Alligator
Anarchie
Ankit
Anthill
Aspiegel
AwarioSmartBot
BBBike
BDCbot
BLEXBot
BackDoorBot
BacklinkCrawler BacklinkCrawler
BatchFTP BetaBot
Battleztar Bazinga Bitacle
Black Hole Blow
BlackWidow Bolt
Brandwatch
Buck Buck
BuiltBotTough
Bullseye Bullseye
BunnySlippers
BuzzSumo
CATExplorador CATExplorador
CCBot
CODE87 CODE87
Cegbfeieh
CensysInspect CensysInspect
CheTeam CheTeam
ChinaClaw
Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot
Copyscape
Cosmos Cosmos
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Curious Crescent
CrunchBot
Custo Custo
DBLBot
DSearch
DTS Agent DTS Agent
DatabaseDriverMysqli Demon
Deusu Digincore
Dirbuster Disco
Discoverybot Discoverybot
Dispatch DittoSpyder
DnyzBot DnyzBot
DomCopBot
DomainCrawler
DomainSigmaCrawler
DomainStatsBot
Domains Project
Download Wonder Download Wonder
EMail Siphon EasyDL
Ebingbong Ecxi
EirGrabber EirGrabber
EroCrawler
Evil
Exabot
Express WebPictures Express WebPictures
ExtLinksBot Ezooms
FDM FDM
FemtosearchBot
Fimap Fimap
Firefox/7.0 Firefox/7.0
FrontPage Freeuploader
Fuzz Fuzz
FyberSpider FyberSpider
Fyrebot Fyrebot
GetWeb G-i-g-a-b-o-t
Getintent GalaxyBot
Go!Zilla
GoZilla GoZilla
Grafula Gotit
GrapeFX
GridBot
HMView HMView
HTMLparser HTTrack
HaosouSpider
Harvest Harvest
Hloader Havij
HonoluluBot
Humanlinks Humanlinks
Iblog IDBTE4M
Id-search
Image Fetch Image Fetch
Image Sucker IndeedBot
Indy Library
InfoNaviRobot InfoNaviRobot
InfoTekies InfoTekies
Intelliseek InterGET
Internet Ninja Internet Ninja
Iskanie JennyBot
JOC Web Spider
JetCar JetCar
JikeSpider Jorgee
Jyxobot Jyxobot
Kenjin Spider Keybot Translation-Search-Machine
Keyword Density
Kinza
Kozmosbot
LNSpiderguy
Larbin Larbin
LeechGet LeechGet
LexiBot
LibWeb
Libwhisker Libwhisker
LinkWalker Likse
LinkScan
LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
Lipperhey Spider LinqiaRSSBot
Lipperhey
Litemage_walker Litemage_walker
Ltx71
MIDown tool
MIIxpc MIIxpc
MSFrontPage Magnet
MSIECrawler Majestic-SEO
Majestic SEO Majestic12
MarkMonitor
Mass Downloader
Masscan
MauiBot MauiBot
Mb2345Browser Mb2345Browser
Meanpathbot MeanPath Bot
MegaIndex.ru Mediatoolkitbot
Microsoft URL Control Microsoft URL Control
Minefield
Mister PiX
Moblie Safari Moblie Safari
Mojeek Mojeek
Mojolicious Morfeus Fucking Scanner
Mozlila Mr.4x3
Msrabot Msrabot
NICErsPRO NICErsPRO
NPbot NPbot
Nameprotect Name Intelligence
Navroad
NearSite NearSite
Needle Needle
NetAnts NetLyzer
NetMechanic
NetSpider NetSpider
Netcraft NetZIP
Nettrack
Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler NimbleCrawler
Niki-bot Nuclei
Ninja Nutch
Nmap Offline Explorer
OpenVAS Offline Navigator
OrangeBot OnCrawl
Openfind
Openvas
OrangeSpider OrangeSpider
OutclicksBot OutclicksBot
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer
PageGrabber PageGrabber
PageScorer PageScorer
Pandalytics
Petalbot Petalbot
Picscout Picsearch
Pimonster Pixray
PleaseCrawl
Pockey
ProWebWalker ProWebWalker
Pu_iN Pump
PxBroker
PyCurl PyCurl
Quick-Crawler RSSingBot
RankActive
RankActiveLinkBot RankActiveLinkBot
RankingBot RankingBot
Rankivabot RankingBot2
Re-re RankurBot
Recorder Reaper
RocketCrawler Ripper
SEOkicks Rogerbot
SEOkicks-Robot SISTRIX
SEOlyticsCrawler SMTBot
ScanAlert
ScreenerBot
Searchestate
SearchmetricsBot
SemanticJuice SemanticJuice
Semrush Semrush
SemrushBot SentiBot
Shodan
Siphon
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteSucker
Sitebeam Sitebeam
Sitevigil Siteimprove
SmartDownload SmartDownload
Snake Snoopy
Snapbot Sociscraper
SocialRankIOBot
Sosospider Sosospider
SpankBot Spammen
Spanner Sqworm
Spbot Steeler
Sucker Stripper
Sucuri T0PHackTeam
SuperBot Telesphoreo
Surfbot
Szukacz
T8Abot
Telesphorep Telesphorep
The Intraformant
TheNomad TheNomad
Thumbor Tracemyfile
Toata Trendiction
Toweyabot Turnitin
True_Robot
Turingos
TurnitinBot
TwengaBot
URLy Warning URLy Warning
URLy.Warning
V-BOT
Vacuum
Vagabondo
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie Voil
VoidEYE Voltron
WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV
WPScan WPScan
WWW-Mechanize WallpapersHD
WWWOFFLE WeSEE
Wallpapers/3.0
Web Auto
Web Collage
Web Fetch Web Fetch
Web Fuck
Web Sauger
WebBandit
WebCopier
WebEnhancer
WebFuck
WebGo IS WebGo IS
WebLeacher WebLeacher
WebStripper WebSauger
WebZIP
WebmasterWorldForumBot
Webshag Webshag
Webster Website Quester
Whack WebsiteQuester
WinHTTrack Whatweb
Wotbox Who.is Bot
Xaldon WebSpider WiseGuys Robot
Zauba Wonderbot
ZoominfoBot Wprecon
ZumBot Zeus
adscanner Zitebot
backlink-check ZoomBot
ZyBorg
arquivo-web-crawler
cah.io.community
check1.exe
clark-crawler clark-crawler
cognitiveseo
crawl.sogou.com crawl.sogou.com
demandbase-bot crawler.feedback
domainsproject.org instabid
eCatch internetVista monitor
gopher
ips-agent ips-agent
lwp-request
lwp-trivial lwp-trivial
mediawords
muhstik-scan muhstik-scan
plumanalytics netEstate NE Crawler
polaris version oBot
probe-image-size pcBrowser
scalaj-http ripz
s1z.ru
scan.lol scan.lol
seobility serpstatbot
seoscanners
seostar
sitechecker.pro sitechecker.pro
sogouspider sogouspider
sp_auditbot sp_auditbot
sysscan spyfu
tAkeOut voyagerx.com
trendiction.com
trendiction.de
ubermetrics-technologies.com
webgains-bot webgains-bot
webmeup-crawler webmeup-crawler
webprosbot webprosbot
x09Mozilla
x22Mozilla x22Mozilla
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
bvlgariring.xyz 7minuteworkout.com
glasof.es autorn.ru
hayate.biz bad-stars.net
hubbble.com boanonihaca.tk
istock-mebel.ru easync.io
kinostorm.net ilmexico.com
mediawhirl.net ipornox.xxx
runofilms.ru nowtorrents.com
xep.info top1-seo-service.com
xn--d1aifoe0a9a.top xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai

922
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
03e.info 00webcams.com
03p.info 1-free-share-buttons.com
1001watch.com.ua 101billion.com
101flag.ru 4rent.ru
10bet.com 5i2.net
1flag.co.za 6128786.com
2728fb936f0.com 999webdesign.com
3w24.com a.pr-cy.ru
40cg.com aa14ab57a3339c4064bd9ae6fad7495b5f.com
4senses.co aasoldes.fr
4ureyesonly.com abcdefh.xyz
5000-cotydzien.com
51unlim.ru
aa8b68101d388c446389283820863176e7.com
absugars.com absugars.com
academia-nsk.org accessoires-mode.in
acads.net
acheterviagrafr24.com acheterviagrafr24.com
addray.pro activecampaign.dreamhosters.com
adexprts.com adconscious.com
adhome.biz add-add.men
adventureparkcostarica.com addbin.men
adverster.com ads-seo.men
afslankpillen2017nl.eu adservme.com
agecheckadult.com advertisingtag.net
alienwheel.es aero2.ru
akvamaster.dp.ua
alborzan.com
alfapro.ru
alkoravto.ru
allbizne.co.ua allbizne.co.ua
allcredits.su allcredits.su
alphacarolinas.com allesohnegirls.net
alphavisions.net allstatesugarbowl.org
aludecor.info amazingninja.com
alyeskaresort.com
ameblo.jp ameblo.jp
ameblo.top android-systems.ru
ample-awards-today.us angel1777.kz
anal-acrobats.hol.es anti-virus-removal.info
android-vsem.org apiluckyleapnet-a.akamaihd.net
animenime.ru apioutoboxnet-a.akamaihd.net
apccargo.com
apiadanaknet-a.akamaihd.net
apiwoodensealcom-a.akamaihd.net apiwoodensealcom-a.akamaihd.net
app5.letmacworkfaster.world appartement-stumm.at
arabgirls.adultgalls.com appfastplay.com
apps-analytics.net
aramaicmedia.org aramaicmedia.org
arius.tech audiofree.ru
arpelsreplica.xyz
artpicso.com
asrvrep-a.akamaihd.net
atelielembrancaqueencanta.com.br
atovh.local-finders.com
autoblogger24.info
autochoixspinelli.com autochoixspinelli.com
avek.ru autoloans.com
aviapanda.ru aviav.ru.com
avkzarabotok.info ayerbo.xhost.ro
azazaporn.com azadnegar.com
b00kmarks.com babs.com.ua
babieca.com back.dog
backlinkwatch.com beauby.ru
balois.worldbreak.com begalka.xyz
bamo.xsl.pt belinka.com.ua
bannerswap.com
batanga.net
best-businessman.ru best-businessman.ru
bestchoice.cf bet365.com
bestpriceninja.com big-boards.info
bharatdefencekavach.com bigpenisguide.com
biglistofwebsites.com biketank.ga
bimatoprost-careprost.com.ua bio-japan.net
bioca.org biocn.dx.am
bioinnovate.co
biteg.xyz
bizlist.com.de
biznesschat.net biznesschat.net
bonkers.name blog-hits.com
bonvillan.com bloggen.be
bonzbuddy.com blogsfunky672.weebly.com
bonzibuddi.com boanonihaca.tk
bouda.kvalitne.cz boc.kz
bravegirlsclub.com bosman.pluto.ro
brimstonehillfortress.org brainsandeggs.blogspot.com
burger-imperia.com browsepulse-a.akamaihd.net
bus-offer.com buketeg.xyz
buy-meds24.com bumble.cheapwebsitehoster.com
bwlx.prepedu.cn burn-fat.ga
cah.io.community buybest1.biz
canrioloadm.gq buyclomidonlaine.com
carapics.com buyessay3.blogspot.ru
caulderwoodkennels.com buyessayonline19.blogspot.ru
caveavins.fr candypeople.se
cbcseward.com cannibalcheerleader.com
ceotrk.com cartierlovebracelet.xyz
cheap-pills-norx.com cartierlovestore.xyz
cialischmrx.com cartierreplica.pw
cialischsrx.com celebsopics.com
cigarpass.com centraletermice.us
cindymatches.com cfjump.com
cjs.com.ru chiblackhawks-jerseys.com
cl.s7.exct.net christianlouboutinshoes.xyz
clayvasedesigns.tk clash-clans.ru
coinsspb.com cleanpcnow.com
comeondog.info clevernt.com
comissionka.net click2pawn.com
compliance-brian.xyz clicksotrk.com
compliance-ivan.top club-lukojl.ru
compliance-jess.top club-musics.ru
compucelunlock.net coderstate.com
comsysnet.com companies-catalog.com
consorzioilmosaico.org compliance-alexa.xyz
compliance-mary.top
cookie-law-enforcement-ff.xyz
craftinsta.ru craftinsta.ru
crazyboost.pro
crd.clan.su crd.clan.su
custodieva.ru cs-passion.pl
customcollegeessays.net dawlenie.com
cyclobenzaprinevstizanidine.blogspot.com
czat.wp.pl
d-black.bz
daisye.top
danschawbel.com
datadepths.com
dear-diary.ru dear-diary.ru
dellalimov.com decoratinghomes.ga
deplim.com demon-tweeks.com
detskie-zabavi.ru derevesendeco.com
doctissimo.fr dev33.dioniqlabb.se
dodlive.mil devochki.top
doeco.ru dfwu1019.info
doktoronline.no directivepub.com
domaincdn.xyz discovertreasurenow.com
dominterior.org dollartree.info
domoysshop.ru domaincrawler.com
downloads-whatsapp.com domainsatcost.com
doxysexy.com donna7753191.ru
duplicashapp.com dotnetdotcom.org
dyt.net downtuptv.gq
ebonyporn.site doxyporno.com
editmedios.com drev.biz
eduinfosite.com druzhbany.ru
efnor-ac.com eets.net
elektronischezigarettekaufen1.beeplog.com eldiariodeguadalajara.com
elektronischezigarettekaufen1.myblog.de electronicadirect.com
elektronischezigarettekaufen2.tumblr.com elektrischezi.canalblog.com
elexies.info elektrischezigarette2.devhub.com
elitepcgames.com elektrozigarette2.wordpressy.pl
englishdictionaryfree.com eng-lyrics.com
eorogo.top enge-fotzen.info
esnm.ru enhand.se
erotag.com
escortslet.net
etm-consult.de etm-consult.de
exchangeit.gq everflora.ru
exmasters.com everypony.ru
expertblog.info everytalk.tv
ezigarettekaufen1.hpage.com extlabs.io
f012.de extremepornos.net
fable.in.ua fable.in.ua
falllow.gq familienzahnaerzte.com
faptitans.com farmingworm.com
farm26.ru fertilitetsradgivningen.se
fashion-stickers.ru fidelityfunding.com
favorcosmetics.com filesclub.net
feorina.ru filesmonster.porn
fermersovet.ru film-one.ru
filunika.com.ru filmi-onlain.info
fitnesspiks.com financeloan.us
flauntyoursite.com fireads.men
flytourisme.org floating-share-buttons.com
forexgb.ru florida-tourism.net
formseo.com fortevidyoze.net
fotoxxxru.com foto-basa.com
free-floating-buttons.com fotzen-ficken.com
freewareseek.com free-deals.faith
freshdz.com free-share-buttons.blogspot.com
frustrated-favorable.gq free-social-buttons.com
fungamelands.com frvo.alptandem.ru
fusoradio.info fugarif.ga
fxgallery.com fungirlsgames.net
galeon.com funnymama.com
gaming-journal.com fx-brokers-review.com
gaytube.com fz139.ttk.ru
gembird.com game-mmorpg.net
get-free-traffic-now.com gamebackyard.com
getaclueamerica.com gavuer.ru
gfaq.ru gaypornmovie.net
ggiaro.com gdebestkupit.ru
gktt.ru gdzkurokam.ru
ge0ip.org
gearsadspromo.club
geile-lelly.eu
genericlowlatencyasiodriverhq.aircus.com
generousdeal-a.akamaihd.net
gerhardhealer.com
glopages.ru glopages.ru
go2mike.ru gloverid.site
gogps.me gnuetella.com
gojiberriess.apishops.ru goatse.ru
goldpanningtools.com golfresa.lucania.se
goodhumor24.com google-liar.ru
goodwriterssales.com gotcher.us
gotomontenegro.net gowreckdiving.com
gov.yanao.ru gratuitbaise.com
gratis-sexkontakte.com gsasearchenginerankerdiscount.com
guardlink.com guiadeserraazul.com
guardlink.org hackers-crackers.tk
handicapvansarea.com hahashka.ru
hasshe.com handsandlegs.ru
haveinc.xyz hdseriale.pl
hd-film.pl hdxnxxtube.mobi
hdapp1008-a.akamaihd.net hermesreplica.pw
hdwalls.xyz hitsbox.info
helicalpile.us hobild.net
heliko.no homedecoguide.info
homedecorpicture.us homedo.fabpage.com
hornymatches.com honyaku.yahoofs.jp
hoverboard360.nl hoverboard360.nl
howmuchdoestizanidinecost.blogspot.com hscsscotland.com
icaseclub.ru hulfingtonpost.com
igadgetsworld.com ico.re
ihc2015.info id-forex.com
iinstalll-fii1leis.jus0wil.pp.ua ideibiznesa2015.ru
imguramx.pw ifmo.ru
indianmedicaltourismshop.com iminent.com
intermesh.net inbabes.sexushost.com
internetartfair.com indiakino.net
iomoio.net indiasourcemart.in
iqupdatetmz.win int.search.mywebsearch.com
izoll.ru intim-uslugi.info
japfm.com intimshop-fantasy.ru
jovencitas.gratis investingclub.ru
juliaworld.net io9.com
justbcause.com iopeninghours.co.uk
kakablog.net iplogger.org
kambasoft.com irkutsk.online-podarki.com
kamorel.com istizanidinelikexanax.blogspot.com
keywordhouse.com istmira.ru
keywordspay.com j33x.com
khadastoafarde.tk janavibekken.no
kino-rf.ru jongose.ninja
kino2018.cc journeydownthescale.info
kinostorm.net kakadu-interior.com.ua
komputernaya-pomosh-moscow.ru kamen-e.ru
krasivoe-hd.com kandidos.com
katushka.net
keyword-suggestions.com
khafre.us
kiinomaniak.pl
kino2018.club
kinoduh.ru
kinomaniatv.pl
kinosed.net
klitimg.pw
knogg.net
konteiner24.com
koptims.tiu.ru
kountrylife.com
krasnodar.xrus.org
kreativperlen.ch
kredit-blog.ru
kriokomora.info
krynica.info
kskjco.club
kurbappeal.info
kvartira-sutochno.com
kvartiry-remont.ucoz.ru
kwzf.net kwzf.net
ladsup.com labplus.ru
larchik.net lacave.ntic.fr
larger.io lanasshop.ru
lashstudia.ru landscapebackgrounds.blogspot.com
law-check-two.xyz lapolis.it
lechtaczka.net laserpen.club
lineavabit.it law-check-eight.xyz
liquimondo.com laxob.com
live-cam6.info leboard.ru
liveshoppersmac.com letolove.ru
ljusihus.se letsrepair.in
lob.com.ru letto.by
lost-alpha.ru lezbiyanki.net
lyngdalhudterapi.no lifehacĸer.com
m4ever.net lirunet.ru
m5home.ru live-sexcam.tk
mackeeper-land-672695126.us-east-1.elb.amazonaws.com livefixer.com
macotool.com livingcanarias.com
makis.nu lkbennettstore.online
maladot.com love-baby.cz
malwareremovalcenter.com lyrster.com
manifestation.betteroffers.review mafcards.ru
mainlinehobby.net
manimpotence.com manimpotence.com
masthopehomes.com mantramusic.ru
max-eclat.men manualterap.roleforum.ru
mb140.ru marcogrup.com
mcnamaratech.com masturbate.co.uk
mecash.ru matb3aa.com
medizinreisen.de maximilitary.ru
meet-flirt-dating.com maximpartnerspr.com
mega-polis.biz.ua maxxtor.eu
metabar.ru medi-fitt.hu
microsoftportal.net meduza-consult.ru
mielec.pl menstennisforums.com
migente.com misslike.ru
milblueprint.com misswell.net
mirmedinfo.ru mmgq.ru
misandesign.se mobplayer.net
mmostrike.ru mojowhois.com
mnogabukaff.net moneyteam24.com
moetomnenie.com montazhnic.ru
monclerjacketsoutlet.pw moomi-daeri.com
money-every-day.com mrvideospornogratis.xxx
money-for-placing-articles.com
monsterdivx.com
montredemarque.nl
most-kerch.org
motherboard.vice.com
mp3ritm.top
mttwtrack.com mttwtrack.com
mundoaberrante.com musicpro.monster
music7s.me
musicktab.com
musicstock.me musicstock.me
myanyone.net muycerdas.xxx
myseoconsultant.com mydeathspace.com
narkologiya-orel.ru mygameplus.ru
nero-us.com myplaycity.com
nerudlogistik.ru mysex21.com
net-profits.xyz namenectar.com
news-readers.ru naperehresti.info
nextconseil.com negral.pluto.ro
nhl17coinsforps3.gratisblog.biz new-apps.ru
nikhilbahl.com nfljerseyscheapchinabiz.com
njpalletremoval.com nfljerseysforsalewholesaler.com
nottyu.xyz ngps1.ru
novatech.vn nodding-passion.tk
novgorod.xrus.org nudejapan.net
nuker.com nuker.com
nw-servis.ru o-o-6-o-o.com
obnal.org o-o-6-o-o.ru
omgtnc.com oakleyglassesonline.us
onclickpredictiv.com oda.as
onlythegames.com offer.wpsecurity.website
orion-code-access.net onlyforemont.ru
onthemarch.co
ostroike.org
otbelivanie-zubov.com
outclicks.net outclicks.net
painting-planet.com ownshop.cf
parajumpersoutlet.online partner-high.men
partnerads.men partner-trustworthy.men
patterntrader-en.com
pavlodar.xkaz.org pavlodar.xkaz.org
pearlisland.ru pdfprof.com
perederni.net phobia.us
pewit.pw photo.houseofgaga.ru
photorepair.ru pic2fly.com
php-market.ru picquery.com
pictures-and-images.com piluli.info
porno-dojki.net play.leadzupc.com
ploenjitmedia.azurewebsites.net
polybuild.ru
porno-home365.com porno-home365.com
pornoelita.info porno-raskazy.ru
pornofeuer.com
pornogig.com
pornoplen.com pornoplen.com
pornzone.tv pornosmola.info
privat-girl.net portadd.men
privatov-zapisi.ru powc.r.ca.d.sendibm2.com
professionaldieselcare.com pozdrawleniya.com
proxyelite.biz pr0fit-b0x.com
pssucai.info pro-tec.kz
qcstrtvt.bloger.index.hr producm.ru
realresultslist.com projectforte.ru
redhotfreebies.co.uk promoheads.com
registry-cleaner.net qitt.ru
remorcicomerciale.ro rankings-analytics.com
renhacklids.tk rankscanner.com
resellerclub.com rapidgator-porn.ga
responsive-test.net rapidokbrain.com
rfid-locker.co razorweb-a.akamaihd.net
rfserial.net rebuildermedical.com
rockingclicks.com registry-clean-up.net
rospromtest.ru resant.ru
rossmark.ru research.ifmo.ru
rubbed.us rialp.getenjoyment.net
ruclicks.com ritlweb.com
rucrypt.com robotixix.com
ruf777.com rosbalt.com.ua
russintv.fr ruspdd.com
sa-rewards.co.za s-forum.biz
sack.net s-iwantyou.com
sakhboard.ru s.lollypopgaming.com
sale-japan.com safe-app.net
saltspray.ru scanner-alexa.top
saneyes.com scanner-andrew.top
sape.top scanner-donald.top
savetubevideo.com scanner-jack.top
sbt-aqua.ru scanner-jess.top
scoopquest.com scrapy.org
scopich.com sdelai-prosto.ru
sdrescher.net secretscook.ru
seasonvar.ru see-your-website-here.com
seksotur.ru seksotur.ru
semalt.com
seminarykansas904.ml seminarykansas904.ml
seo-prof1.xyz sendearnings.com
seo-smm.kz seorankinglinks.com
seorankinglinks.us seozoom.it
serving.adbetclickin.pink service.adtech.fr
serviporno.com servingnotice.com
seryeznie-znakomstva.ru sexuria.net
sexsearch.com sharebutton.net
shillyourcoins.com shinikiev.com.ua
shodanhq.com shitting.pro
shop.acim.org shoesonlinebuy.cn
shopfishing.com.ua shoplvlv.us
sildenafil-tadalafil.info shtora66.ru
siteheart.net sigmund-freud.co.uk
sitevaluation.com signx.info
skanninge.se silvermature.net
slonechka.ru sitevalued.com
sluganarodu.ru sitopreferito.it
smartadserver.com skincrate.net
snowplanes.com skuteczna-dieta.co.pl
soft-program.com snip.tw
softomix.net snjack.info
soserfis.com snomer1.ru
sotechco.co snyatie-lomki-v-stacionare.ru
space-worry.ml social-buttons.com
spidtest.space softxaker.ru
sprttrack.com spin2016.cf
spywareit.com sptslmtrafms.com
srecorder.com spy-sts.com
srgwebmail.nl srgwebmail.nl
stair.registrydomainservices.com stair.registrydomainservices.com
standardchartered-forex.com startwp.org
star61.de statistici.ro
starwars.wikia.com statoutlook.info
stop-gepatit.te.ua stroyplus.ru
stpolice.com studentguide.ru
streetfooduncovered.com styro.ru
studworks.org sugvant.ru
subj.ukr-lit.com
summerlinhomes411.info
surffoundation.nl
svolze.com svolze.com
td-33.ru tehngr.ru
tdsing.ru tgtclick.com
thebestphotos.eu the-torrent-tracker.blogspot.com
tiandeural.ru thecoral.com.br
tizanidinedosage.blogspot.com themeforest.net
tizanidinehcl2mg.blogspot.com theplacetoupdating.pw
tizanidinehcl2mgtablet.blogspot.com theprofitsmaker.net
tizanidinehydrochloride2mg.blogspot.com thin.me.pn
ticketsys.inetwd.com
timetorelax.biz
tizanidine4mguses.blogspot.com
tizanidineandgabapentin.blogspot.com
tizanidinezanaflex.blogspot.com tizanidinezanaflex.blogspot.com
tjkckpytpnje.com tjkckpytpnje.com
tksn.ru tksn.ru
tnaionline.org tnctrx.com
tooplay.com toon-families.com
top-deal.com.pl topads.men
torrent-newgames.com topshef.ru
totu.info tradedeals.biz
tracksz.co traffic-club.info
tracxn.com trion.od.ua
trafficstars.com truck-land.life
traffictrade.life tubeline.biz
traffixer.com turn-up-life.life
tupper-shop.ru tv-spoty.info
turkeyreport.tk twincitiescarservice.com
tvorozhnaja-zapekanka-recept.ru twittrading.com
tvteleport.ru ua-company.ru
twitlinks.com uchebavchehii.ru
tyumen.xrus.org udayavani.com
ugguk.online udsgame.online
ultimatesetnewfreeallsoftupgradesystems.pw ukr-lit.com
umnovocaminho.com ultimateclassicrock.com
undergroundcityphoto.com uptime-alpha.net
unitygame3d.com usbggettwku.ga
unlimitdocs.net usdx.us
upproar.com valsalud.com
ural-buldozer.ru vegascosmetics.ru
urengoy.pro velen.io
url-extractor.xyz video-hollywood.ru
urlcut.ru vikistars.com
urlopener.com vip2ch.com
utiblog.fr virtuagirl.com
utrolive.ru vitalads.net
uvozdeckych.info
vacuumscleaner.com
valkiria-tk.ru
varikozdok.ru
vashsvet.com
vasileostrovsky-rayon.ru
vbikse.com
vchulkax.com
veerotech.com
viagroid.ru
video-chat.cn
video-chat.love
videochat.bz
videochat.ph
videooko.weebly.com
vinsit.ru
vkontaktemusic.ru vkontaktemusic.ru
vladimir.zrus.org voltrknc1.com
vodkoved.ru vozbujdenie.com
vseigru.one
vykup-avto-krasnodar.ru
vykupavto-krasnodar.ru vykupavto-krasnodar.ru
wallpapersdesk.info webcamdevochka.com
welcomeauto.ru webix.biz
whengirlsgowild.com webradiology.ru
wholesalejerseyscheapjerseys.us.com webs.com
wildnatureimages.com website-audit.com.ua
wildworld.site websiteexplorer.info
winterclassichockeyjerseys.com welck.octopis.com
wjgony.com whatzmyip.net
wnhjavlhezp.gq wholinkstome.com
wordpress-crew.net wikes.20fr.com
winner7777.net
wleuaprpxuvr.ga
wormix-cheats.ru
wowas31.ucoz.ru wowas31.ucoz.ru
x-lime.net woweb.com.ua
xcc24.pl wttavern.com
wurr.voila.net
wygraj-skiny.win
xboxster.ru
xn----7sbifcamovvfggw9d.xn--p1ai
xn--24-glceagatoq7c2a6ioc.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn--80ahdheogk5l.xn--p1ai xn--h1ahbi.com.ua
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai xportvusbdriver8i.snack.ws
xtrafficplus.com xsion.net
yogamatsexpert.com yatrk.xyz
yellowproxy.net
yoopsie.com yoopsie.com
yopoint.in yoursearch.me
yougame.biz yoursite.com
youporn-forum.ga za-fun-offer.com
youtuhe.com zakazfutbolki.com
za-music.mymobiplanet.com zeroredirect.com
zagadki.in.ua zog.link
zakon-ob-obrazovanii.ru zoogdinsney.com
zarajbuilders.com
zarepta.com
zeroredirect1.com
zeroredirect12.com
zhorapankratov7.blogspot.com
zygophyceous.womanstars.site

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
007angels.com 000free.us
aa9bd78f328a6a41279d0fad0a88df1901.com 0n-line.tv
aad1f4acb0a373420d9b0c4202d38d94fa.com 100dollars-seo.com
acgs.tk 404.mn
addtoadd.men 40cg.com
adobereader-free.ru 98oi.ru
adsloads.com aanapa.ru
afmuseum.com acc.eu.org
aktivator-windows10.blogspot.com adimmix.com
aktobe.xkaz.org adloads.com
albuteroli.com alyeskaresort.com
anti-virus-removal.info angel1777.kz
apiwoodensealcom-a.akamaihd.net animal-drawings.com
avkzarabotok.com anmysite.com
awency.com arabseyes.com
azbukafree.com asacopaco.tk
atmovs.com
availit.weebly.com
avirasecureserver.com
avramstroy.ru
babyguns.ru
backiee.com backiee.com
bannerads.de balans.shahterworld.org
beachdriveblog.com balook.com
bestofupload.info bambi.ck.ua
bestwaystogetpaid.us bestofferswalkmydogouteveryday.gq
bezcmexa.ru bestplacetobuyeriacta.jw.lt
bitcoin-ua.top betonka.pro
bizfly.info bezsporno.ru
blackhatworth.com bimatoprost-careprost.com
blog.axant.it bitnote.co
bolezniorganov.ru biznesschat.net
bonanza-fortune.men buzzsumo.com
bonux.nextview.ru cacheimages.com
bosefux.esy.es camsex.xxx-cam.webcam
buy-forum.ru cartierring.xyz
buyessayonline19.blogspot.ru celejihad.info
buypanicdisorderpill.com children-learningreading.info
bvlgaribracelet.xyz choice-credit.ru
bytimedance.ru citysecurity.nu
cah.io.community cleanmypc.com
ccjp.eu cleanpcnow.com
chatroulette.life club-lukojl.ru
cheatcode-lita12.rhcloud.com community.allhiphop.com
chelyabinsk.xrus.org
cherubinimobili.it
cinemaflix.website
clicksor.com
clodo.ru
compliance-alex.xyz
compliance-george.top
compliance-jack.top
conboy.us conboy.us
cougfan.info coolyarddecorations.com
counter.yadro.ru copenergo.ru
dangphoto.trade cupday.com
danschawbel.com datract.com
db.speedup.gdn
dbmkfhqk.bloger.index.hr
deda-moroza-zakaz.ru
delta-line.men
detmebel.su detmebel.su
diffbot.com dev.citetick.com
digilander.libero.it dkvorota.ru
djonwatch.ru dokumentalkino.net
documentbase.net domain2008.com
downloadme.life doyouknowtheword-flummox.ml
draniki.org dyt.net
driving.kiev.ua e-commerce-seo1.com
droidlook.net easync.io
drugspowerstore.com edwinkonijn.com.au
dwomlink.info efnor-ac.com
elektrozigarette1.dreamwidth.org eldiariodeguadalajara.com
envaseslotusama.com elektronischezigarette2.mex.tl
elektronischezigarettekaufen1.myblog.de
elektronischezigarettekaufen2.tumblr.com
elvenmachine.com
eng-lyrics.com
enge-fotzen.info
essaypro.com essaypro.com
evidencecleanergold.com evaashop.ru
ezigarettekaufen.myblog.de event-tracking.com
ezigaretteshop2.mywapblog.com excaliburfilms.com
ezofest.sk exoclick.com
facebook-mobile.xyz extlabs.io
fashionavenuegame.com ezigarettenshop1.yolasite.com
felizporno.com ezigarettenshop2.postbit.com
fertilitetsradgivningen.se facecup.top
fickenbumsen.net fandlr.com
fil.ru filesvine.com
flauntyoursite.com friendflnder.com
flex4launch.ru fullfileaccess.com
foto-telok.net gamezblox.com
franecki.net gaming-journal.com
frighteningremain.cf greendream.com.ua
garciniaxt.us guardlink.org
gardene.ru h2monline.com
gk170.ru hamilton.ca
glopages.ru hawaiielectriclight.com
golmau.host.sk hentaiheroes.com
googleseo.com.tr hermesreplica.win
gratuitxblcodes.com hostcritique.com
greatzip.com hostsshop.ru
grizzlysgrill.com hothor.se
grtyi.com hotkeys.com
healgastro.com hoverboard360.de
heliko.no hustoon.over-blog.com
hmmm.cz i4track.net
homik.pw iccornacircri.cf
honyaku.yahoofs.jp infektsii.com
hoporno.com insomniagamingfestival.com
host-protection.com io9.com
hotdl.in iqoption-bin.com
huimin764128.com ivoiretechnocom.ci
hunterboots.online jeremyeaton.co
ideibiznesa2015.ru journalhome.com
import-sales.com jwcialislrt.com
iqoption.pro jyrxd.com
japan-bearings.ru kaliningrad.zrus.org
jpcycles.com kanimage.com
jrcigars.com karachev-city.ru
kaac.ru kareliatobacco.ru
kevblog.top keki.info
keyhantercume.com kerch.site
kinozapas.com ketoanhanoi.info
kol-energo.ru keywordteam.net
kosmetyki.tm.pl kidskunst.info
kotaku.com kinky-fetishes.com
kovrenok.ru koronirealestate.gr
kupit-adenu.ru kostenloser-sex.com
lafourmiliaire.com lacasamorett.com
landscapebackgrounds.blogspot.com lmrauction.com
larose.jb2c.me lollypopgaming.com
law-check-seven.xyz longlifelomilomi.info
letsrepair.in lotto6888.com
linkdebrideur.xyz luchshie-topcasino.ru
luxup.ru luckyday.world
magazin-pics.ru m5home.ru
marmitaco.cat magiadinamibia.blogspot.com
menetie.ru makemoneyonline.com
minecraft-rus.org malwareremovalcenter.com
missis.top meble-bogart.info
mistr-x.org mesbuta.info
mmostrike.ru metascephe.com
mobile-appster.ru mexicotravelnet.com
mriyadh.com mindeyegames.com
msk-diplomat.com mirobuvi.com.ua
musezone.su mixed-wrestling.ru
mixtapetorrent.com
mobplayer.ru
modabutik.ru
montredemarque.nl
mturkcontent.com
music7s.me music7s.me
mxgetcode.com mycaf.it
myseoconsultant.com myfreetutorials.com
nextlnk12.com nalogovyykodeks.ru
nightvision746.weebly.com naturalshair.site
noumeda.com nfvsz.com
novatech.vn nibbler.silktide.com
nrv.co.za nicefloor.co.uk
onescreen.cc npoet.ru
onlineslotmaschine.com nw-servis.ru
oralsexfilme.net o-o-8-o-o.ru
ororodnik.goodbb.ru obuv-kupit.ru
orsonet.ru ownshop.cf
outclicks.net parajumpersjakkesalgnorge.info
paccohichetoti.ml pattersonsweb.com
partner-host.men pavlodar.xkaz.org
pb-dv.ru phrcialiled.com
pearlisland.ru phuketscreen.com
phobia.us pierrehardysale.online
photo-clip.ru pijoto.net
piski.top
piuminiita.com piuminiita.com
pixshark.com placid-rounded-coast.glitch.me
pizza-tycoon.com pornoxxx.com.mx
podshipniki-nsk.ru positive2b.ru
pornokorol.com preparevideosafesystem4unow.space
porto.abuilder.net pricheski-video.com
pr-ten.de prointer.net.ua
preg.marketingvici.com ranksonic.info
profitkode.com recordpage-a.akamaihd.net
pussyspace.net
puteshestvennik.com
quebec-bin.com
radiodigital.co
ranking2017.ga
rankscanner.com
ranksonic.com
raschtextil.com.ua
razamicroelectronics.com
refererx.com
relatodelpresente.com.ar
remontbiz.ru remontbiz.ru
roll123.com rewardpoll.com
ruex.org.ua rniaeba.ga
rustic-quiver.win ros-ctm.ru
sack.net s-forum.biz
santasgift.ml saecsa.co
scanner-jane.top salutmontreal.com
scoopquest.com samolet.fr
scrapinghub.com sbwealthsolutions.ca
scripted.com scanner-brian.top
seansonline24.pl scanner-mark.top
search-goo.com scopich.com
searchlock.com semp.net
seoanalyses.com sex-dating.co
seoexperimenty.ru sex-sex-sex5.com
sextracker.com sex-znakomstva.online
sexy-screen-savers.com sexpornotales.com
sharpchallenge.com sexvideo-sex.com
shemalegalls.blogporn.in shama-rc.net
sitebeam.net sharebutton.org
slim.sellany.ru shinikiev.com.ua
slonechka.ru simplepooltips.com
smartbalanceworld.com simpoed.ufop.br
snegozaderzhatel.ru site-auditor.online
snow.nvr163.com sitevaluation.com
soft-terminal.ru skachat-besplatno-obrazcy.ru
solitaire-game.ru sledstvie-veli.net
songplanet.ru smadihome.com
souvenir.cc social-s-ggg.xyz
spabali.org socialbuttons.xyz
spacash.com softomix.net
spasswelt.xyz sohoindia.net
stathat.com solmarket.by
stauga.altervista.org sportbetfair.com
staynplay.net stackthatbucks.com
tarad.com supermodni.com.ua
telefonsexsofort.tk svbur.ru
theplacetoupdating.pw symbaloo.com
tizanidine4mguses.blogspot.com tandvardshuset.net
topanasex.com thebestphotos.eu
topappspro.com tiens2010.ru
torrentdownloadhub.com tizanidineandhydrocodone.blogspot.com
tracksz.co tizanidinestreetvalue.blogspot.com
trubywriting.com tjkckpytpnje.com
tv-spoty.info tmearegion26.com
twitlinks.com todohr.com
ugguk.online top-karkas.ru
ukr-lit.com topcar-krasnodar.ru
underthesite.com topvidos.ru
universals.com.ua torrents.life
uogonline.com trafficstars.com
url-img.link trk-4.net
usiad.net usiad.net
vashsvet.com ustion.ru
veerotech.com viagroid.ru
vegan-foods.us villacoloniale.com
vgoloveboli.net vk-mus.ru
via-gra.webstarts.com vkontaktemusic.ru
vilingstore.net vse-pesni.com
waterpurifier.club wahicbefa31.soup.io
webs.com warning.or.kr
weightatraining.com
whatsappdownload10.com whatsappdownload10.com
wikes.20fr.com wheelchairliftsarea.com
x-diesel.biz wholesalecheapjerseysfree.com
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai wnoz.de
xn----dtbndd4ae7eub.top wtsindia.in
xn--80aafb2a.xn--p1ai x-stars.ru
your-bearings.com xn----7sbaphztdjeboffeiof6c.xn--p1ai
zeg-distribution.com xn--b1ag5cfn.xn--p1ai
zzlgxh.com xn--oogle-wmc.com
xxxmania.top
youjizz.com
yourmovies.pl
youtuhe.com
zakazvzloma.com
zarabotok--doma.ru
zeroredirect11.com
zigarettenonlinekaufen1.bloog.pl
zigarettenonlinekaufen1.blox.pl
znaniyapolza.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.62 0; 10.1.0.4 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3093 ### Version: V4.2022.05.3094
### Updated: Sun May 8 22:01:25 UTC 2022 ### Updated: Mon May 9 10:03:53 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3093 ### Version: V4.2022.05.3094
### Updated: Sun May 8 22:01:25 UTC 2022 ### Updated: Mon May 9 10:03:53 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##