vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 12:09:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.06.3183 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-06-22 22:07:15 +00:00
parent 5f1905626b
commit 40a5a9b988
46 changed files with 104915 additions and 104915 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.06.3182 #### Version: V4.2022.06.3183
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.06.3182 #### Version: V4.2022.06.3183
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.06.3182 #### Version: V4.2022.06.3183
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12978
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.06.3182 #### Version: V4.2022.06.3183
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12982
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12982
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12982
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12982
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.1.111 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.1.111 0;
127.0.0.1 0; 127.0.0.1 0;

12982
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.1.111 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.1.111 0;
127.0.0.1 0; 127.0.0.1 0;

12982
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.1.111 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.1.111 0;
127.0.0.1 0; 127.0.0.1 0;

12982
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3182 3183

12982
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12982
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.125.130.227 0; 20.237.192.176 0;

12982
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.1.111 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.1.111 0;
127.0.0.1 0; 127.0.0.1 0;

12982
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.1.111 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.1.111 0;
127.0.0.1 0; 127.0.0.1 0;

12982
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.1.111 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.1.111 0;
127.0.0.1 0; 127.0.0.1 0;

12982
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.125.130.227 0; 20.237.192.176 0;

12982
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.125.130.227 0; 20.237.192.176 0;

12982
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.1.111 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Barkrowler ChinaClaw
Bullseye Discoverybot
Drip IstellaBot
EMail Siphon JikeSpider
Lmspider MauiBot
MIDown tool PageAnalyzer
RankurBot SEOlyticsCrawler
ScoutJet Scanbot
WebCopier Wallpapers/3.0
WiseGuys Robot WebFuck

316
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker
80legs 80legs
AIBOT AIBOT
Abonti ASPSeek
Aboundex Acunetix
AfD-Verbotsverfahren AlphaBot
Aipbot
AllSubmitter
Anarchie Anarchie
Anarchy Ankit
Anarchy99
Apexoo Apexoo
Asterias
Atomseobot Atomseobot
Attach AwarioRssBot
BBBike
BDCbot BDCbot
BLEXBot BDFetch
Badass BackDoorBot
Bandit BackWeb
Battleztar Bazinga Barkrowler
BatchFTP
BetaBot BetaBot
Bigfoot Bigfoot
Bitacle
Black Hole
Blow Blow
BlowFish
Boardreader
Bolt Bolt
Brandprotect BotALot
Buddy
BuiltBotTough BuiltBotTough
BuiltWith
Bullseye
BuzzSumo
CCBot CCBot
Calculon CSHttp
Cegbfeieh Cegbfeieh
CensysInspect CheeseBot
CherryPicker CherryPicker
Citoid ChinaClaw
Claritybot Chlooe
Cloud mapping Cocolyzebot
Cogentbot
Copier
CopyRightCheck
Copyscape
Cosmos Cosmos
Craftbot
Crescent
Curious
Custo Custo
DBLBot CyotekWebCopy
DIIbot Demon
DSearch
DataCha0s
DatabaseDriverMysqli
Deusu Deusu
Devil DigitalPebble
Digincore Discoverybot
Dirbuster
Disco
Dispatch Dispatch
DittoSpyder DnBCrawler-Analytics
DomCopBot
DomainAppender DomainAppender
DomainCrawler
Dotbot Dotbot
Dragonfly Drip
ECCP/1.0
Ebingbong Ebingbong
EirGrabber Ecxi
EroCrawler
Exabot
Express WebPictures
ExtLinksBot ExtLinksBot
EyeNetIE Extractor
Ezooms
FHscan FHscan
FemtosearchBot FemtosearchBot
Flunky Flunky
FyberSpider Foobot
G-i-g-a-b-o-t G-i-g-a-b-o-t
Gigabot Gigabot
GoZilla GrabNet
Grabber Grabber
HTTP::Lite GrapeFX
HTTrack GrapeshotCrawler
HEADMasterSEO
HMView
HaosouSpider
Harvest
Havij Havij
HonoluluBot Heritrix
Humanlinks Humanlinks
IDBTE4M IDBTE4M
Image Fetch IRLbot
Intelliseek Iblog
Internet Ninja IndeedBot
InternetSeer InterGET
Iria
Iskanie Iskanie
IstellaBot Jbrofuzz
JennyBot JetCar
Jetty JikeSpider
Joomla Joomla
Jorgee
Kenjin Spider Kenjin Spider
Keyword Density Kinza
Kozmosbot Kozmosbot
LWP::Simple LWP::Simple
Leap Leap
LeechFTP
LeechGet LeechGet
LibWeb LieBaoFast
Lightspeedsystems LinkWalker
Likse Linkbot
LinkpadBot LinkpadBot
LinksManager LinksManager
LinqiaRSSBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey Spider
Lmspider Lmspider
MIIxpc Ltx71
MIDown tool
MJ12bot MJ12bot
MSIECrawler Mag-Net
MTRobot Magnet
Mail.RU_Bot Majestic-SEO
MarkMonitor MeanPath Bot
Mass Downloader
Mb2345Browser
Mediatoolkitbot
Metauri
Microsoft Data Access
Moblie Safari
Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mr.4x3 Mozlila
NPbot Name Intelligence
NearSite Nameprotect
Needle Navroad
NetAnts NetLyzer
NetMechanic NetSpider
NetZIP
Netcraft
Netvibes
NextGenSearchBot NextGenSearchBot
NimbleCrawler
Nimbostratus Nimbostratus
Ninja Ninja
Octopus Nmap
Nutch
Offline Explorer Offline Explorer
OpenLinkProfiler OnCrawl
OpenVAS
OrangeSpider OrangeSpider
PECL::HTTP OutclicksBot
PageGrabber OutfoxBot
POE-Component-Client-HTTP
PageThing.com
Pandalytics
Panscient Panscient
Papa Foto Pavuk
PeoplePal Petalbot
Picsearch Picscout
Piepmatz Pixray
Pockey ProWebWalker
QueryN Metasearch Probethenet
Quick-Crawler PyCurl
RankActive RSSingBot
RankingBot
RankingBot2 RankingBot2
Rankivabot
RankurBot RankurBot
ReGet
RealDownload RealDownload
Reaper
RebelMouse RebelMouse
Recorder
RedesScrapy RedesScrapy
Ripper Ripper
Rogerbot RocketCrawler
SBIder SEOlyticsCrawler
SEOkicks-Robot SISTRIX
SEOprofiler SMTBot
SEOstats
SalesIntelligent SalesIntelligent
ScanAlert
Scanbot Scanbot
ScoutJet
Screaming
Searchestate
SearchmetricsBot SearchmetricsBot
Seekport
SemanticJuice SemanticJuice
Semrush Semrush
SemrushBot SemrushBot
SentiBot
SeobilityBot
Seomoz
Shodan
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler
SiteSnagger
SiteSucker SiteSucker
Sitebeam Sitebeam
Sitevigil
Snapbot Snapbot
Snoopy
Sosospider Sosospider
SpaceBison Sottopop
Spbot Sqworm
Spinn3r
Sqlworm
Steeler
Stripper Stripper
Sucker Sucuri
SuperBot SurveyBot
Telesoft Suzuran
Telesphoreo Szukacz
Thumbor T0PHackTeam
Toweyabot T8Abot
Teleport
TeleportPro
The Intraformant
TightTwatBot
Trendiction Trendiction
TurnitinBot Turingos
Twice TwengaBot
URLy.Warning Typhoeus
URLy Warning
VB Project VB Project
Vagabondo VCI
VoidEYE Virusdie
WWW::Mechanize WASALive-Bot
WWWOFFLE WBSearchBot
Web Auto WEBDAV
Web Enhancer WWW-Collector-E
Web Sucker WWW-Mechanize
WebBandit Wallpapers
WebEnhancer WeSEE
WebFetch Web Collage
WebSauger Web Fuck
Web Pix
Web Sauger
WebAuto
WebCollage
WebCopier
WebFuck
WebGo IS
WebStripper WebStripper
WebSucker WebZIP
Webalta Webalta
WebmasterWorldForumBot
Webshag Webshag
WebsiteQuester Website Quester
Who.is Bot WebsiteExtractor
Whack
WinHTTrack WinHTTrack
Wotbox WiseGuys Robot
Wonderbot
Woobot
Wprecon Wprecon
Xenu Xenu
YoudaoBot Zeus
Zermelo ZoomBot
Zitebot
ZoominfoBot ZoominfoBot
ZyBorg
adscanner adscanner
arquivo.pt
coccocbot coccocbot
cognitiveseo
crawl.sogou.com
dataforseo.com
demandbase-bot demandbase-bot
domainsproject.org
eCatch eCatch
evc-batch
facebookscraper
gopher
internetVista monitor internetVista monitor
lwp-trivial
magpie-crawler
meanpathbot meanpathbot
mediawords
netEstate NE Crawler
oBot oBot
pcBrowser pcBrowser
polaris version
s1z.ru s1z.ru
seobility seocompany.store
seostar serpstatbot
sexsearcher sexsearcher
sogouspider sogouspider
sysscan
tAkeOut tAkeOut
trendiction.de trendiction.de
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webpros.com webgains-bot
x09Mozilla
x22Mozilla x22Mozilla
zauba.io
zgrab

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404enemy 360Spider
80legs ADmantX
ASPSeek AIBOT
Abonti Aboundexbot
Acunetix AhrefsBot
AiHitBot AiHitBot
Aipbot
AllSubmitter AllSubmitter
Alligator Alligator
AlphaBot
Anarchie
Anarchy Anarchy
Anarchy99 Anarchy99
Apexoo Anthill
Asterias
Atomseobot Atomseobot
BBBike BLEXBot
BDCbot Backlink-Ceck
BDFetch
BackStreet
BacklinkCrawler
Bandit Bandit
Battleztar Bazinga Barkrowler
BatchFTP
Bigfoot
Bitacle Bitacle
Black Hole
Blackboard
Boardreader Boardreader
Bolt Bolt
Buck BotALot
Buddy
BuiltBotTough
Bullseye
BuzzSumo BuzzSumo
CATExplorador CATExplorador
CCBot
CSHttp CSHttp
CensysInspect
CherryPicker CherryPicker
ChinaClaw ChinaClaw
Claritybot Claritybot
Cloud mapping Cliqzbot
Copier Copyscape
Craftbot
CrazyWebCrawler
Crescent
Curious Curious
Custo
DSearch
DTS Agent DTS Agent
Deusu DataCha0s
Devil Demon
DigitalPebble Discobot
Dirbuster Discoverybot
DittoSpyder Dispatch
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DnyzBot
DomCopBot DomCopBot
DomainAppender Domains Project
DomainCrawler Dotbot
Drip Dragonfly
ECCP/1.0
EMail Wolf EMail Wolf
Ebingbong
EirGrabber
Evil Evil
EyeNetIE ExtractorPro
Ezooms
FHscan FHscan
FemtosearchBot FemtosearchBot
Flunky Fimap
Firefox/7.0
FlashGet
Foobot
FrontPage FrontPage
Fuzz Fuzz
FyberSpider Fyrebot
G-i-g-a-b-o-t
GalaxyBot GalaxyBot
GetWeb Getintent
Go!Zilla Gigabot
Go-Ahead-Got-It
GoZilla GoZilla
GrabNet GrapeshotCrawler
GrapeFX
HEADMasterSEO
HTTP::Lite HTTP::Lite
HTTrack HTTrack
Havij Haansoft
HonoluluBot Humanlinks
IDBot IDBot
IRLbot
Id-search
IlseBot IlseBot
Image Fetch Image Sucker
Intelliseek
InterGET
Internet Ninja
InternetSeer InternetSeer
Iria Iria
Iskanie
JOC Web Spider JOC Web Spider
JamesBOT
JennyBot JennyBot
JetCar
Jetty
JikeSpider JikeSpider
Joomla Joomla
Jorgee Jyxobot
Kozmosbot Kenjin Spider
LWP::Simple Keyword Density
Larbin Kinza
LeechFTP LeechGet
LexiBot
LibWeb
Lightspeedsystems Lightspeedsystems
LinkScan Linkbot
LinkextractorPro LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider
Litemage_walker
Lmspider Lmspider
MFC_Tear_Sample MFC_Tear_Sample
MSIECrawler MIIxpc
MarkMonitor MJ12bot
MauiBot Mail.RU_Bot
Mb2345Browser Majestic SEO
Mediatoolkitbot MarkWatch
Masscan
MicroMessenger
Microsoft Data Access
Minefield Minefield
Mister PiX Moblie Safari
Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Msrabot Mozlila
NICErsPRO NICErsPRO
NPbot
Name Intelligence Name Intelligence
Navroad
NearSite NearSite
Needle Needle
NetAnts NetLyzer
NetZIP NetZIP
Netcraft
Nettrack Nettrack
Niki-bot Nimbostratus
Nikto Nmap
Ninja
Not Not
Nuclei Nuclei
Nutch
Octopus
Offline Navigator Offline Navigator
OpenVAS OnCrawl
OrangeBot OrangeSpider
OutclicksBot PECL::HTTP
OutfoxBot PHPCrawl
Pavuk PageAnalyzer
PageGrabber
PageScorer
Pandalytics
Panscient
PeoplePal PeoplePal
Pi-Monster Pi-Monster
Picscout Picscout
Picsearch
PictureFinder
Pimonster Pimonster
PleaseCrawl PleaseCrawl
Pockey
ProPowerBot ProPowerBot
Probethenet
Pu_iN Pu_iN
PxBroker
QueryN Metasearch QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot RSSingBot
RankFlex
Rankivabot Rankivabot
RankurBot Recorder
RealDownload RocketCrawler
Rogerbot
SBIder
SEOkicks SEOkicks
SEOlyticsCrawler SEOkicks-Robot
SEOprofiler SMTBot
SEOstats
ScanAlert
ScoutJet
Screaming
ScreenerBot
ScrepyBot ScrepyBot
SemanticJuice Seekport
SemrushBot SemrushBot
SentiBot SentiBot
SeoSiteCheckup SeoSiteCheckup
Siphon SeobilityBot
SiteSnagger Shodan
Sitebeam Site Sucker
Siteimprove SiteCheckerBotCrawler
Snapbot SiteExplorer
SocialRankIOBot Sitevigil
Sosospider Snake
SpaceBison Snoopy
Sociscraper
SpankBot
Sqlmap Sqlmap
Sqlworm Sucker
Sqworm Surfbot
SuperBot Swiftbot
SuperHTTP
Szukacz Szukacz
T0PHackTeam
Telesoft Telesoft
The Intraformant
TheNomad TheNomad
Toata TightTwatBot
Toweyabot Toweyabot
Tracemyfile Trendictionbot
Turingos Turingos
TwengaBot TurnitinBot
URLy Warning
URLy.Warning URLy.Warning
Upflow VB Project
V-BOT
VCI VCI
Vacuum
Vagabondo Vagabondo
VeriCiteCrawler VeriCiteCrawler
Virusdie
VoidEYE
Voil Voil
Voltron WBSearchBot
WISENutbot WPScan
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
Web Auto WallpapersHD
Web Enhancer Web Collage
Web Fetch Web Fuck
Web Pix Web Sauger
WebAuto WebAuto
WebCollage WebBandit
WebImageCollector WebEnhancer
WebSucker WebLeacher
Webshag WebPix
WebWhacker
Website Quester Website Quester
Webster Webster
Whack Whacker
Who.is Bot
Widow Widow
WinHTTrack WinHTTrack
WiseGuys Robot
Wonderbot
Wprecon Wprecon
Zade Zade
Zermelo Zitebot
Zeus ZumBot
ZmEu ZyBorg
ZoominfoBot
adscanner adscanner
arquivo.pt arquivo-web-crawler
autoemailspider
backlink-check
cah.io.community cah.io.community
coccocbot clark-crawler
cognitiveseo crawl.sogou.com
com.plumanalytics
crawler4j
demandbase-bot demandbase-bot
eCatch gopher
heritrix instabid
internetVista monitor internetVista monitor
linkdexbot
lwp-request
lwp-trivial
meanpathbot meanpathbot
mediawords netEstate NE Crawler
muhstik-scan pcBrowser
oBot
plumanalytics plumanalytics
polaris version s1z.ru
probe-image-size satoristudio.net
ripz
seostar seostar
serpstatbot sogouspider
sitechecker.pro
sp_auditbot sp_auditbot
spyfu tAkeOut
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webmeup-crawler webmeup-crawler
x09Mozilla
x22Mozilla
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
boazpower.com anime.dougasouko.com
cheapjerseysfootballshop.com aosheng-tech.com
magicalfind-a.akamaihd.net cherrypointplace.ca
meccadumps.net erectile.bid
mp3ritm.top iframe-toloka.com
sarafangel.ru newyorkhotelsmotels.info
taqywu51.soup.io stroyplus.ru
ventelnos.com trafficjunky.net
witherrom55.eklablog.fr vasileostrovsky-rayon.ru
xn--80aikhbrhr.net zaimhelp.ru

928
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
000free.us 00author.com
1kdailyprofit.me 1001watch.com.ua
24videos.tv 123any.com
4kepics.com 2000k.ru
66cpwgln.space 273-fz.ru
7minuteworkout.com 2itech.ru
7search.com 4ureyesonly.com
883zy.com
DomainStatsBot
abcdefh.xyz
abclauncher.com
academia-nsk.org
activecampaign.dreamhosters.com activecampaign.dreamhosters.com
acunetix-referrer.com ad-words.ru
adidas.frwebs.fr adsloads.com
adloads.com adultfriendfinder.com
adultfullhd.com adultfullhd.com
adultwebhosting.info agapovdg.ru
adventureparkcostarica.com ageofclones.com
afslankpillen2017nl.eu aghanyna.com
ahrntal.verymes.xyz ahrefs.com
airmaxshoes-2016.com akita.kz
alf-img.com alert.scansafe.net
alkoravto.ru alfa-medosmotr.ru
alldubai.biz alfapro.ru
allkrim.com algarveglobal.com
allornamenti.com alibestsale.com
aludecor.info allbizne.co.ua
amehdaily.com allproblog.com
anapa-inns.ru alot.com
anaseracresar.tk amigobulls.com
animal36.com amyfoxfitness.com
animenime.ru angigreene.com
anmysite.com animal-drawings.com
apartment.ru anonymousfox.co
apartmentbay.ru apartmentbay.ru
apiadanaknet-a.akamaihd.net apessay.com
apple.com-cleaner.systems apparel-offer.com
applyneedy.xyz appearance-cool.com
appsecurityr.com appiq.mobi
arquivo.pt artavenuegardenstudios.com
artdestshop.eu
asiavirtualsolutions.com asiavirtualsolutions.com
asmxsatadriverin.aircus.com asiengirls.net
aucoinhomes.com autowebmarket.com.ua
audiobangout.com aviav.ru.com
auto-moto-elektronika.cz avon-severozapad.ru
autobudpostach.club axbocz.net
autogrep.ru azbukadiets.ru
avek.ru bag77.ru
avkzarabotok.com balois.worldbreak.com
backlink4u.net bambi.ck.ua
baditri.com banki76.ru
baersaratov.ru bbsoldes.fr
bagcionderlab.com becuo.com
baltgem.com bedandbreakfast.com
bausparen.bz.it best-mam.ru
beauty-b0x.pl bestadbid.com
beauty-bracelet.com bestcalovebracelet.cn
belsetirehafi.tk bestplacetobuyeriacta.jw.lt
bensbargains.net betune.onlinewebshop.net
bestmobilityscooterstoday.com
bidr.trellian.com bidr.trellian.com
bitporno.sx bigames.online
bizcheapjerseyswholesalechina.com binomo.com
bjgugu.net.ua blavia.00author.com
blog.koreadaily.com blockworld.ru
bluerobot.info bloggers.nl
bmw-ark.ru bnt-team.com
bonky.biz boazpower.com
braip.com.br bongiornos.info
brandedleadgeneration.com bonvillan.com
brandimensions.com bosefux.esy.es
bristolhostel.com bpro1.top
bus-offer.com brainboosting.club
buyfriend.ru bugof.gq
c-english.ru bupropion-sr-150-mg.us
cactussoft.biz burger-tycoon.com
calc-for-credit.ru butstrap.space
call-of-duty.info buttons-for-your-website.com
candlehandmade.com buzzurl.jp
carmuffler.net bvlgariring.xyz
cartierjusteunclou.xyz callawaygolfstore.online
cartierreplica.pw callcafe.info
cartierring.xyz candycrushshop.com
cctva.tv candypeople.se
celebsopics.com canrioloadm.gq
centre-indigo.org.ua capacitacionyaprendizaje.com
check-host.net centraletermice.us
chelnytruck.ru ceotrk.com
choosecuisine.com cheap-pills-norx.com
chorus.terakeet.com cheapbelstaff.online
cialischmrx.com cheapjerseysfootballshop.com
cialisovercounteratwalmartusa.com cheapwebsitehoster.com
ciekawinki.pl checkhit.com
cigarpass.com chee-by.biz
cleanallspyware.com chimiver.info
cntravelre.com chinacheapelitejerseys.com
compliance-alex.xyz chinese-amezon.com
compliance-checker-7.info chocolateslim-en-espana.com
compliance-checker.info chocolateslim-original.com
concealthyself.com chocoslim.pro
conciergegroup.org cicaki.net
conocer-sanabria.com cienum.fr
cookielawblog.wordpress.com civilwartheater.com
cookingmeat.ru clayvasedesigns.tk
copblock.org cleanmypc.com
crazyboost.pro click2pawn.com
credit.co.ua cocyq.inwtrade.com
custom-electric-guitar.com coderstate.com
customerguru.in com-supportcenter.website
compliance-brian.xyz
compliance-margo.top
compliance-mary.top
comsysnet.com
contentlook.co
coolgamechannel.com
country-chic.ru
csgo4.win
cubs-jerseys.us
cyprusbuyproperties.com cyprusbuyproperties.com
cyse.tk damasarenai.info
daisye.top daretodonate.co
dame-ns.kz deanmoore.ie
datadepths.com degerlund.net
datsun-do.com delfin-aqua.com.ua
davebestdeals.com detki-opt.ru
davidovic.info dikqlyremy.info
daymusam.com dinkolove.ya.ru
ddlmega.net dipstar.org
deadlinkchecker.com
detalizaciya-tut.biz
dfwu1013.info
dieswaene.com
discovertreasure-a.akamaihd.net discovertreasure-a.akamaihd.net
diy-handmade-ideas.com doeco.ru
dneprsvet.com.ua domain-tracker.com
documentbase.net dominterior.org
doktoronline.no dosug-lux.ru
domaincrawler.com draniki.org
domainsatcost.com elidelcream.weebly.com
downloaddy.net emerson-rus.ru
downtuptv.gq encodable.com
earn-from-articles.com energydiet-info.ru
editmedios.com englishgamer.com
elektrozigarettekaufen1.devhub.com ero-advertising.com
elite-sex-finders.com erotik0049.com
elitedollars.com erotiktreff24.info
emailaccountlogin.co escort-russian.com
emediate.eu eshop4u.jp
errorfixing.space espaceinventoristes.com
eshop.md estelight.ru
esopini.com european-torches.ru
essaypro.com europeanwatches.ru
etur.ru everflora.ru
event-tracking.com ezigarettenkaufen1.deviantart.com
exchanges-bet.com ezofest.sk
factorynightclub.com ezrvrentals.com
fanpagerobot.com facecup.top
faptitans.com falcoware.com
fashionavenuegame.com faracontrol.ir
femmesdenudees.com fast-wordpress-start.com
fenoyl.batcave.net feargames.ru
fermersovet.ru fefo.gdn
femdom.twiclub.in
fickblock18.com fickblock18.com
fickenprivat.info findpik.com
fifa-coins.online finstroy.net
findanysex.com
findthe.pet
fivedwld.cf fivedwld.cf
flashlarevista.com flashlarevista.com
flirt4free.com forex-procto.ru
fm-upgrade.ru
fodelsedagspresenter.nu
footballfarrago.com
formulaf1results.blogspot.com formulaf1results.blogspot.com
forum.poker4life.ru forodvd.com
fotoxxxru.com fortunejack.com
foto-sisek.porngalleries.top
fr.netlog.com fr.netlog.com
freenode.info freewareseek.com
freza-sverlo.ru froggytube.com
front.ru fsakhalin.ru
fuel-gas.com fuckmill.com
fusoradio.info funnymama.com
fyxabomiw.ru funponsel.com
gagrasector.ru fx-brokers-review.com
game300.ru galaxy-family.ru
gamingspark.com game-top.su
garciniaxt.us gazporno.com
ge0ip.net gemgrab-a.akamaihd.net
gembird.com getoutofdebtfree.org
getyourimage.club getyourimage.club
girlsatgames.ru gg.zzyjxs.com
goatse.ru gigapeta.com
golfresa.lucania.se globalsurfari.com
goodnightjournal.com gobongo.info
google-liar.ru gok-kasten.net
googlemare.com goldenggames.com
gotwebsite1.com goodly.pro
gratuitbaise.com gotcher.us
greatzip.com greatdealshop.com
greenzaim.ru gsmlab.pl
gsasearchengineranker.space gsou.cf
gstatey.net gyffu.com
guge.io heartofbeijing.blogspot.com
gwagka.com hidemyass.com
h2monline.com histock.info
hacktougroup.ru
hdapp1008-a.akamaihd.net
hdfreeporno.net
hitsbox.info
hkdiiohi.skyrock.com
homedecoguide.info homedecoguide.info
hoverboard360.nl horoshieokna.com
hostcritique.com
hostsshop.ru
hotsocialz.com
howlongdoestizanidinestayinyoursystem.blogspot.com howlongdoestizanidinestayinyoursystem.blogspot.com
huimin764128.com humanelydrew.com
humbmosquina.tk hustoon.over-blog.com
hunthillfarmtrust.org hyhj.info
i-service.kz igadgetsworld.com
ilovevitaly.xyz igithab.com
imperiafilm.ru images-free.net
inboxdollars.com inboxdollars.com
indiakino.net indiasourcemart.in
informatiecentro.be instakink.com
insider.pro instasexyblog.com
int.search.tb.ask.com intelhdgraphicsgtdrive6w.metroblog.com
intermesh.net investmac.com
investingclub.ru investpamm.ru
investyb.com investyb.com
isitpaleo.info iqs.biz.ua
islandminingsupply.wordpress.com istizanidineanopiate.blogspot.com
istizanidineacontrolledsubstance.blogspot.com itag.pw
iweblist.info javitas.info
izismile.com jjbabskoe.ru
janettabridal.com jurajskie.info
jaxcube.info justbcause.com
jongose.ninja
jrpmakati.com
juliadiets.com
jwcialislrt.com jwcialislrt.com
kaac.ru jyvopys.com
kam-dom.ru kamen-e.ru
kinocccp.net keywordsuggests.com
kinosed.net kicknights.gq
koversite.info kidskunst.info
kino2018.cc
kinohall.ru
kladrus.ru
klitimg.pw
kometars.xyz
komputernaya-pomosh-moscow.ru
konoplisemena.com
kostenloser-sex.com
kozhniebolezni.com kozhniebolezni.com
kursy-ege.ru krasnodar.zrus.org
kvartiry-remont.ucoz.ru
lacave.ntic.fr
lampokrat.ws lampokrat.ws
landmania.ru lankarns.com
lasvegaslockandsafe.com laptoper.net
law-check-seven.xyz laulini.soclog.se
lenvred.org lazy-z.com
letsrepair.in lexiangwan.com
lexaprogeneric.link life.biz.ua
linkdebrideur.xyz lifeinsurancekb.com
linkredirect.biz liffa.ru
lkbennettoutlet.online linksharingt.com
lombia.com live-sexcam.tk
lotto6888.com live-sexchat.ru
lumb.co lonely-mature.com
m.b00kmarks.com luchshie-topcasino.ru
m1media.net m0r0zk0-krava.ru
macdamaged.tech m3gadownload.pl
magento-crew.net mackeeper-land-672695126.us-east-1.elb.amazonaws.com
masthopehomes.com magiadinamibia.blogspot.com
me-ke.com managerpak204.weebly.com
mebel-alait.ru mararoom.ru
meccadumps.net matsdale.com
medictube.ru meet-flirt-dating.com
medkritika.ru meetlocalchicks.com
mega-polis.biz.ua megakino.net
megaindex.ru meteostate.com
meget.co.za methodsmarketing.com
menhealed.net michellblog.online
mex-annushka.ru middlerush-a.akamaihd.net
mirobuvi.com.ua milblueprint.com
monclerjacketsoutlet.win missclub.info
morf.snn.gr mjchamonix.org
morlat.altervista.org monarhs.info
moviezbonkerssk.cf monclerjacketsoutlet.pw
moxo.com monclervests.xyz
msk-diplomat.com monsterdivx.com
muz-shoes.ru more-letom.ru
mydeathspace.com moscow.xrus.org
mydownlodablefiles.com mostantikor.ru
mymercy.info mouselink.co
net-radar.com moyakuhnia.ru
netallergy.ru mrmoneymustache.com
newstaffadsshop.club my-bc.ru
nfljerseyscheapbiz.us myfreecams.com
nhl17coins.exblog.jp myphotopipe.com
nhl17coinsforps3.gratisblog.biz nailsimg.com
notasprensa.info nalogovyykodeks.ru
novatech.vn naperehresti.info
o-o-8-o-o.ru narkologiya-belgorod.ru
oakleyglassesonline.us naturalbreakthroughsresearch.com
obiavo.in newhairstylesformen2014.com
of-ireland.info news-speaker.com
offf.info nofreezingmac.work
olympescort.com notaria-desalas.com
nuup.info
o00.in
oakridgemo.com
offer.wpsecurity.website
offergroup.info
old-rock.com
omgtnc.com
omoikiri-japan.ru omoikiri-japan.ru
onclkads.com onblastblog.online
online-zaymy.ru onlinebay.ru
onlineku.com onlinewritingjobs17.blogspot.ru
onlyforemont.ru ons-add.men
ontargetseo.us open-odyssey.org
ourville.info openfrost.net
oynat.info opinionreelle.com
oz-offers.com optibuymac.com
ozas.net optom-deshevo.ru
page2rss.com osnova3.ru
paphoselectricianandplumber.com paceform.com
parajumpersoutlet.online parfusale.se
partner-cdn.men
partner-pop.men
partner-stop.men
pateaswing.com pateaswing.com
patol01.pw pc4download.co
patterntrader-en.com pdamods.ru
pavlodar.xkaz.org pestomou.info
penisvergrotendepillennl.ovh piccshare.com
perfectplanned.com picmoonco.pw
photorepair.ru pijoto.net
photostudiolightings.com pipki.r.acdnpro.com
php-market.ru playboyfiles.xblog.in
pic2fly.com playmsn.com
pinstake.com pobeiranie.pl
pinup-casino1.ru pochtovyi-index.ru
pladform.ru poem-paying.gq
plastgran.com pogruztehnik.ru
play-movie.pl pokemongooo.ml
plohaya-kreditnaya-istoriya.ru popup.matchmaker.com
pojdelo.weebly.com pornhive.org
poligon.com pornokorol.com
popander.mobi pornosee.info
popserve.adscpm.net pozdravleniya-c.ru
popup-fdm.xyz printdirectforless.com
pornhub-forum.ga privatbank46.ru
pornolook.net privatefx-in.ru
pornotubexxx.name prostitutki-almata.org
porto.abuilder.net purplesphere.in
pregnant.guru purplestats.com
pricheskaonline.ru pxhdwsm.com
priscilarodrigues.com.br quangcaons.com
prizefestival.mobi rankingchart.de
professionaldieselcare.com
prostitutki-astana.org
prostitutki-belgoroda.org
prostitutki-kharkova.org
purchasepillsnorx.com
putevka24.ru
puzzleweb.ru
rankexperience.com
rapidokbrain.com rapidokbrain.com
rapidsites.pro realting-moscow.ru
rarbg.to recipedays.ru
razvratnoe.org refudiatethissarah.info
rcb101.ru reklama1.ru
registrydomainservices.com remont-mobile-phones.ru
remont-fridge-tv.ru renewablewealth.com
replicaclub.ru rezeptiblud.ru
responsinator.com riders.ro
retailwith.com rn-to-bsn.com
rockingclicks.com rukino.org
roleforum.ru sampleletters.net
royal-betting.net sarf3omlat.com
royalcar-ufa.ru
ru-mediaget.ru
rumamba.com
s-forum.biz
sa-rewards.co.za
savememoney.co.za
saveriopiazza.it
sbdl.no
scanner-brian.top scanner-brian.top
scanner-julianna.top scanner-donald.top
schoolfiles.net scanner-fred.top
scmor.ilxc.cc scanner-jess.top
searchaddis.com scanner-mary.top
seeingmeerkat.com scanner-olga.top
seinterface.com scanner-viktor.top
search.1and1.com
sel-hoz.com
semalt.com semalt.com
sendearnings.com seoexperimenty.ru
seoboxes.com seotoolsagency.com
seoprofiler.com service.adtech.us
seorussian.ru sevendays.com.ua
serw.clicksor.com sevenstars7.com
sexstream.pl sexkvartal.com
sexytrend.ru sextracker.be
sfj-ror.no
shareyards.com
shijian.ac.cn shijian.ac.cn
shiksabd.com shop.acim.org
shopwme.ru shopcheermakeup.info
simpoed.ufop.br shtaketniki.ru
sisi-go.ru signal03.ru
skypasss.com silverage.ru
skytraf.xyz sitebeam.net
slavic-magic.ru skachat-besplatno-obrazcy.ru
snjack.info sklad-24.ru
social-buttons.com slkrm.ru
sockshare.net sluganarodu.ru
snegozaderzhatel.ru
soft-terminal.ru soft-terminal.ru
space4update.pw softwaretrend.net
spidtest.space sogimlecal.tk
sport-video-obzor.ru soviet-portal.do.am
spylog.com sovinsteel.ru
sribno.net space4updating.win
spb.afora.ru
speechfoodie.com
spidtest.org
srecorder.com
stairliftsarea.com
star61.de
stariy-baku.com stariy-baku.com
stockquotes.wooeb.com starpages.net
stopnarco.ru stmassage.ru
stroiminsk.org stoki.ru
sugarlyflex.pw stroydetali.ru
superlist.biz stroymonolit.su
suralink.com suhanpacktech.com
surflinksmedical.com sumo.com
suture.co sweepstakes.rewardit.com
svtrd.com tacbelarus.ru
swinon.site td-33.ru
szucs.ru tds-advert005.info
takethatad.com teencastingporn.com
taranerymagesswa.blogspot.com teesdaleflyballclub.co.uk
telemetryverification.net teguh.info
telsis.com templates.franklinfire.co
theautoprofit.ml templates.radiodigital.co
theendivechronicles.com terraclicks.com
thenetinfo.com thecoolimages.net
thiegs.reco.ws thefarmergame.com
timetorelax.biz thegalerie.eu
titan-cloud.life thegameriders.com
titangel-vietnam.com titelhelden.eu
tivolibasket.it
tizanidine4mg.blogspot.com
tizanidineinteractions.blogspot.com tizanidineinteractions.blogspot.com
tizanidinemusclerelaxant.blogspot.com tizanidinepain.blogspot.com
tizanidinenarcotic.blogspot.com tizanidinezanaflex.blogspot.com
tizanidineoral.blogspot.com topflownews.com
tizanidineshowupondrugtest.blogspot.com torrents.life
tizanidinesideeffects.blogspot.com totu.info
tizanidinewithdrawal.blogspot.com toy-shop.top
tkanorganizma.ru tpu.ru
tksn.ru trafficmonetize.org
tmearegion26.com ttsq.fr
toon-families.com tuberkuleznik.ru
tootoo.to turist-strani.ru
topmira.com ugogo.info
topsiteminecraft.com uhodzalijami.ru
touchmods.fr univerfiles.com
toxicwap.com unrealcommander.org
tracksz.co uptime-delta.net
tracxn.com upupa.net
trahvid.com userequip.com
trustedmaccleaner.com uvozdeckych.info
tryrating.com validdomain.xyz
tsan.net vclicks.net
tsstcorpcddvdwshbbdriverfb.aircus.com ventelnos.com
tytoona.com videochat.world
tyumen.xrus.org videospornogratisx.net
ukrtvir.com.ua videtubs.pl
ultimatesetnewfreeallsoftupgradesystems.pw
unmaroll.ya.ru
ural-buldozer.ru
urlopener.com
vgoloveboli.net
videokrik.net
vilingstore.net
villacoloniale.com
vip-dom.in
vipps.com.my vipps.com.my
viven.host.sk vitanail.ru
viveresaniesnelli.it vkontaktemusic.ru
vladimir.zrus.org vladimir.xrus.org
voluumtrk.com vmnmvzsmn.over-blog.com
vrnelectro.ru volgograd.xrus.org
vseuznaem.com vpnmouse.com
w7s.ru vvon.co.uk
wallpapersist.com warezaccess.com
warningzscaler.heraeus.com web-betting.ru
wdfdocando.com webcamdevochka.com
webs.com webcamtalk.net
website-audit.com.ua website-audit.com.ua
weightatraining.com website-speed-check.site
wesharepics.com websitevaluebot.com
wesharepics.info wechatdownload10.com
westen-z.life wedding0venues.tk
whengirlsgowild.com wfb.hatedriveapart.com
wineitudes.wordpress.com whatzmyip.net
wholesalecheapjerseysfree.com
wieseversa.no
womama.ru womama.ru
www.ehscloud.cn woman-h.ru
womensplay.net
wordpress-crew.net
worldoffiles.ru
worst-sites.online
writersgroup580.web.fc2.com
www.888.com
www.souvenirua.com www.souvenirua.com
x-true.info
xcombear.ru xcombear.ru
xerox-douglas.cf xfire.com
xjrul.com xitjw.info
xlolitka.com xmladserver.com
xn----7sbifcamovvfggw9d.xn--p1ai xn----7sbahjd3btneuw1joc.xn--p1ai
xn--80ak6aa92e.com xn----itbkqkfiq.xn--p1ai
xn--c1acygb.xn--p1ai xn--h1ahbi.com.ua
xtubeporno.net xvideospornoru.com
xuki.us xxxdatinglocal.us
xxart.ru xxxnatelefon.ru
yadro.ru yadro.ru
yourmovies.pl yebocasino.com
yourporngay.com youghbould.wordpress.com
youtuhe.com youjizz.com
ypmuseum.ru your-bearings.com
zaloadi.ru zarepta.com
zarabiaj-dzis.pl zero1.it
zeroredirect2.com zigarettenonlinekaufen1.bloog.pl
zixizop.net.ru
zmoda.hostreo.com
znakomstvaonlain.ru znakomstvaonlain.ru
znaniyapolza.ru zryydi.com
zskdla.site zscalerone.net
zvezdagedon.ru

490
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
100dollars-seo.com 1kdailyprofit.me
1xbet4you.com 2itech.ru
3weekdiet.com 4rent.ru
72-news.com 8kisses.eu
7wind.ru 91abcw.com
adserver-e7.com a96527gi.beget.tech
ahrntal.verymes.xyz addblueoff.com.ua
alert-jdh.xyz adhome.biz
ameblo.top admatic.com.tr
anaseracresar.tk admeasures.com
apisecretsaucebi-a.akamaihd.net allboard.xobor.de
avtochehli.by alot.com
b-styles.xyz amt-k.ru
baksman.com amyfoxfitness.com
balkanfarma.org apilinkswiftco-a.akamaihd.net
bamo.xsl.pt appearance-cool.com
bankmib.ru apple.com-webbrowsing-security.science
bestcoin.cc asdfz.pro
blavia.00author.com asseenontvonline.ru
blockety.co ausergrubhof.info
bluejays-jerseys.us autovideobroadcast.com
book-bets.com aviva-limoux.com
breastaugmentation.co.za azte.ch
breget74.ru backlink4u.net
brianhenry.co baltgem.com
bscodecs.com berrymall.ru
carstrends2015.com betune.onlinewebshop.net
cartujano-pre.de big-cash.net
cenoval.ru biglistofwebsites.com
cheapbarbour.online binaryoptionscops.info
cheapjerseysap.com bio-market.kz
checkhit.com bookmaker-bet.com
checktext.org briomotor.co
chocoslim.pro bristolhotel.com.ua
christianlouboutinshoes.xyz bugshoot.cn
cicaki.net buqayy0.livejournal.com
comissionka.net c-english.ru
concordexoticrentals.com calc-for-credit.ru
contentlook.co camdenmemorials.com
copro.pw camel-beach.com
dame-ns.kz cardul.ru
data-mining.tk carsdined.org
cartierlovestore.xyz
cdnnetwok.xyz
centraletermice.us
centre-indigo.org.ua
chocolateslim-u-srbiji.com
choosecuisine.com
cialiswithoutadoctor.net
ciceros.co
cjs.com.ru
clash-clans.ru
clickhype.com
clickintext.net
clickpapa.com
club-musics.ru
codq.info
compliance-irvin.top
conciergegroup.org
corta.co
crd.clan.su
customergrowthsystems.com
d0t.ru
defenderxtactical.com
discountliv.com
disruptingdinnerparties.com
divci-hry.info divci-hry.info
dollartree.info dokfilms.net
domaincrawler.com domaincheck.io
dostavka-v-krym.com doyouknowtheword-flummox.ml
downloadmefiranaratb1972.xpg.com.br dtnlyss.com
elektrischezigarette2.onsugar.com duplicashapp.com
enews.tech eager-nash.188-93-233-196.plesk.page
escortplius.com elidelcream.weebly.com
este-line.com.ua europages.com.ru
etm-consult.de ewebarticle.info
euroskat.ru excitacao.com
evogarage.com expdom.com
expresstoplivo.ru extrabot.com
f07.de falllow.gq
fatfasts-4tmz.com fashion-stickers.ru
fetishinside.com filmci.pro
fetroshok.ru firstsiteguide.com
fickenprivat.info froggytube.com
filmetricsasia.com
filmfanatic.com
fingerprintjs.com
firesub.pl
fiuxy.com
flashlarevista.com
flowersbazar.com
fok.nl
forminecrafters.ru
forodvd.com
forzeronly.com
freelinkbuilding.website.tk
freewareseek.com
fres-news.com
funponsel.com
galaxys6manual.info
gallery.rennlist.com
gallictures.com gallictures.com
gemara.com
generalporn.org generalporn.org
get-free-traffic-now.com given2.com
gok-kasten.net grtyi.com
goldadpremium.com gunsvicceadadebt.tk
hayate.biz gwebtools.com
healgastro.com gymi.name
hentaiheroes.com hawaiisurf.com
herokuapp.com hdxnxxtube.mobi
hkladys.com hobild.net
i-midias.net.br hoholikik.club
ihtec2019.org hollywoodactress.info
infazavr.ru homeandhealth.ru
int.search.tb.ask.com homedo.fabpage.com
investzalog.ru homemature.net
ix20.ru hotblog.top
jillepille.com housediz.com
johannesburgsingles.co.za https-legalrc.biz
juliadiets.com iframe-toloka.com
justkillingti.me ihc2015.info
kadashihotel.com inbabes.sexushost.com
kakadu-interior.com.ua interesnie-faktu.ru
katadhin.co investpamm.ru
kchaxton.com jwss.cc
kinozapas.com jyrxd.com
kol-energo.ru keywordsdoctor.com
konturkrasoty.ru kinomaniatv.pl
koopilka.com krasnodar.zrus.org
kountrylife.com kvartiry-remont.ucoz.ru
krasnodar.xrus.org labplus.ru
kung-fu-ru.com laparfumotec.com
kuzinsp.ru lazymanyoga.com
laapp.com
lacloop.info
lawyers.cafe
lazy-z.com
lenvred.org
lernur.net lernur.net
lerporn.info linkdebrideur.xyz
lflash.ru linkpulse.com
liupis.com littlesunraiser.com
live-cam6.info locimge.pw
livejasmin.com lottospring.com
luckyday.world louboutinshoes.xyz
mafcards.ru maridan.com.ua
mall.uk mearns-tractors.co.uk
marblestyle.ru med-zdorovie.com.ua
marketingvici.com metalonly.info
meble-bogart.info mhi-systems.ru
medanestesia.ru milblueprint.com
mejoresfotos.eu mjchamonix.org
mezaruk.info mmstat.com
mielec.pl mobot.site
mixx.com mojpregled.com
monthlywinners.com monclerjacketsoutlet.pw
moscow-region.ru morepoweronmac.trade
mountainstream.ms morocco-nomad-excursions.com
muschisexbilder.com mossmesi.com
mydoctorok.ru moviemail-online.co.uk
national-today-winning-winner.club musezone.su
navalwiki.info my-cash-bot.co
nero-us.com myprintscreen.com
nottyu.xyz netallergy.ru
oballergiya.ru notaria-desalas.com
oivcvx.website o-o-11-o-o.com
onload.pw onemactrckr.com
ostroike.org onlainbesplatno.ru
piter.xrus.org onlineku.com
polska-poezja.com ons-add.men
pomoc-drogowa.cba.pl ops.picscout.com
poppen-nw.net panchro.xyz
popserve.adscpm.net parajumpersoutlet.online
popup-fdm.xyz photorepair.ru
porndl.org picsfair.com
porno-play.net piluli.info
pornoreino.com pix24x7.com
pornzone.tv platinumdeals.gr
portnoff.od.ua playtap.us
prlog.ru porno-home365.com
pssucai.info pornophoto.xyz
pushdata.sendpulse.com pourvous.info
ragecash.com princevc.com
randki-sex.com privacyassistant.net
promodj.com
proprostatit.com
prostitutki-astana.org
qwertty.net
rankingchart.de rankingchart.de
rapidokbrain.com reckonstat.info
rczhan.com reeyanaturopathy.com
recinziireale.com ribieiendom.no
recordpage-a.akamaihd.net richinvestmonitor.com
refudiatethissarah.info romhacking.ru
registrysweeper.com roofers.org.uk
retailwith.com runofilms.ru
revistaindustria.com rusprostitute.com
reward-survey.net s8-nowy-wygraj.comli.com
rockprogblog.com sabizonline.com
rospromtest.ru sad-torg.com.ua
ru-mediaget.ru saltspray.ru
saneitconsulting.com savingsslider-a.akamaihd.net
sarafangel.ru scanmyphones.com
sarm.tk scanner-mary.top
sbdl.no scenarii-1-sentyabrya.uroki.org.ua
scanner-don.top school-diplomat.ru
scanner-donald.top securesmrt-dt.com
scanner-irvin.top semaltmedia.com
scanner-jack.top semp.net
score-ads.men sexkontakte-seite.com
seansonline24.pl sexreliz.com
searchengineranker.email sexyali.com
sethrollins.net sharebutton.org
sexkvartal.com shmetall.com.ua
sharebutton.to shtora66.ru
shymkent.xkaz.org simple-image.com.ua
sibdevice.ru skutecznetabletkinaporostwlosow.pl
sibecoprom.ru soc-econom-problems.ru
soft-terminal.ru socialbutton.xyz
sosdepotdebilan.com socialmonkee.com
spbchampionat.ru sodexo.com
spinazdrav.ru space4update.pw
spywarenuker.com speedup-my.site
srgwebmail.nl spidtest.org
sribno.net sports-supplements.us
startwp.org srecorder.com
stop-gepatit.te.ua stroiminsk.com
stpolice.com strv.se
sunflowerdrawingpaintings.blogspot.com styro.ru
symphonyintegratedhealthcare.com superkanpo.com
taqplayer.info surgut.zrus.org
teguh.info svbur.ru
timeallnews.ru svolze.com
tizanidinesideeffectsweightloss.blogspot.com sygraem.com
tmearegion26.com tattomedia.com
top-karkas.ru tattoo33.ru
top10-way.com thefarmergame.com
topappspro.com thelottosecrets.com
topshef.ru ticketsys.inetwd.com
torrents.life tnaionline.org
trucri.me tonivedu.it
trustedmaccleaner.com top-study.work
try-rx.com torrents-tracker.com
uginekologa.com truebeauty.cc
ultimateclassicrock.com ua-company.ru
umnovocaminho.com vedomstvo.net
uni.me vektorpress.ru
vapomnoncri.tk velpanex.ru
veles.shop video-chat.cn
veselokloun.ru videochat.cafe
viteonlusarezzo.it vintontech.info
vitoriacabos.com vip-dom.in
warning.or.kr volgograd.xrus.org
watchmygf.net vostoktrade.info
weblo.com w7s.ru
website-stealer.nufaq.com winterclassichockeyjerseys.com
wesharepics.site wjgony.com
whiteelephantwellington.com wonderfulflowers.biz
word-vorlagen.net worldmusicfests.com
wosik-dach.service-for-web.de writersgroup580.web.fc2.com
wttavern.com www.arenda-yeisk.ru
xgames-04.com x-rates.ru
xmronta.com xn----ctbigni3aj4h.xn--p1ai
xn----7sbho2agebbhlivy.xn--p1ai xn--80aodinpgi.xn--p1ai
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai xxx-cam.webcam
xn--80ab4aa2g.xn--p1ai yellocloud.be
xn--80aikhbrhr.net yoopsie.com
xn--80ajbshivpvn2i.xn--p1ai yourporngay.com
xn--80atua3d.xn--p1ai yubikk.info
xnxx-n.com zigarettenonlinekaufen.tumblr.com
xxxnatelefon.ru zrizvtrnpale.tk
you-shall-not-pass.is74.ru
zaobao.com.sg
zarenica.net
zeroredirect11.com
zigarettenonlinekaufen1.bloog.pl
zoompegs.com
zrelaya.pw

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.1.111 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.06.3182 ### Version: V4.2022.06.3183
### Updated: Wed Jun 22 10:03:26 UTC 2022 ### Updated: Wed Jun 22 22:01:28 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.06.3182 ### Version: V4.2022.06.3183
### Updated: Wed Jun 22 10:03:26 UTC 2022 ### Updated: Wed Jun 22 22:01:28 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##