vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-06 12:40:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.08.2520 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-08-12 10:09:20 +00:00
parent 38cd38afdb
commit d5ce5144fe
46 changed files with 66103 additions and 66103 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2519 #### Version: V4.2021.08.2520
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2519 #### Version: V4.2021.08.2520
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.08.2519 #### Version: V4.2021.08.2520
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8128
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.08.2519 #### Version: V4.2021.08.2520
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [620](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8132
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8132
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8132
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8132
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.95 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.95 0;
127.0.0.1 0; 127.0.0.1 0;

8132
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.95 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.95 0;
127.0.0.1 0; 127.0.0.1 0;

8132
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.95 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.95 0;
127.0.0.1 0; 127.0.0.1 0;

8132
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2519 2520

8132
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8132
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.122.52.122 0; 168.61.191.49 0;

8132
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.95 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.95 0;
127.0.0.1 0; 127.0.0.1 0;

8132
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.95 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.95 0;
127.0.0.1 0; 127.0.0.1 0;

8132
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.95 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.95 0;
127.0.0.1 0; 127.0.0.1 0;

8132
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.122.52.122 0; 168.61.191.49 0;

8132
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.122.52.122 0; 168.61.191.49 0;

8132
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 1; 10.1.0.95 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Getintent Aboundex
Go-Ahead-Got-It DnyzBot
HMView Exabot
HTMLparser Keyword Density
Heritrix LeechFTP
Kinza Microsoft URL Control
MicroMessenger Papa Foto
Openfind Web Auto
RankFlex Web Pix
Zitebot ZmEu

298
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404checker
360Spider 80legs
AIBOT Aboundex
ALittle Client Acunetix
AhrefsBot
AiHitBot
Aipbot
AllSubmitter AllSubmitter
Alligator AlphaBot
Anarchie Anarchie
Anarchy Anarchy
Ankit Ankit
Anthill Anthill
Apexoo Apexoo
Aspiegel Aspiegel
Asterias AwarioRssBot
BBBike AwarioSmartBot
BDCbot BDFetch
BackWeb
Badass Badass
BetaBot BetaBot
Bigfoot Bigfoot
Buck Bitacle
Black Hole
Blow
BotALot
BuiltBotTough BuiltBotTough
BuzzSumo BuzzSumo
CODE87
CSHttp CSHttp
Calculon Calculon
CazoodleBot CheeseBot
CheTeam CherryPicker
ChinaClaw
Chlooe
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot
Collector
Copier Copier
CopyRightCheck
Copyscape Copyscape
CrazyWebCrawler Cosmos
Craftbot
CrunchBot CrunchBot
Custo Custo
CyotekWebCopy
DBLBot DBLBot
DSearch DIIbot
DatabaseDriverMysqli DatabaseDriverMysqli
Digincore Demon
Disco Devil
DigitalPebble
Discoverybot Discoverybot
DnyzBot
DomCopBot DomCopBot
DomainAppender
DomainStatsBot DomainStatsBot
Download Wonder Domains Project
Ebingbong Drip
EMail Siphon
EMail Wolf
Ecxi
EirGrabber EirGrabber
EroCrawler Evil
FDM Extractor
Ezooms
FemtosearchBot FemtosearchBot
Firefox/7.0 Fimap
FlashGet Flunky
Foobot FrontPage
Fyrebot
GT::WWW
GalaxyBot GalaxyBot
GetRight GetRight
GetWeb GetWeb
Go!Zilla Gigablast
Go-Ahead-Got-It Gotit
GoZilla Grabber
Grafula GridBot
HMView HEADMasterSEO
HTTrack HTTrack
Haansoft
HaosouSpider
Harvest Harvest
Havij Havij
Heritrix Heritrix
Hloader Humanlinks
HybridBot
IDBTE4M
IRLbot IRLbot
Image Fetch Iblog
Image Sucker Id-search
IndeedBot IlseBot
Internet Ninja InfoNaviRobot
Jbrofuzz InterGET
JetCar IstellaBot
JennyBot
Jetty
Joomla Joomla
Jorgee Jorgee
Jyxobot Kenjin Spider
LeechGet Keyword Density
LWP::Simple
Lanshanbot
LeechFTP
LexiBot
Lftp Lftp
LibWeb Libwhisker
LinkScan LieBaoFast
Lightspeedsystems
Linkdexbot
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lmspider Lipperhey
MJ12bot MIDown tool
MSIECrawler MSIECrawler
MTRobot
Mag-Net
Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic12 Majestic-SEO
MarkMonitor
Masscan
Mata Hari Mata Hari
Mb2345Browser Metauri
MeanPath Bot Mister PiX
Meanpathbot Moblie Safari
MegaIndex.ru Mojolicious
Mr.4x3 Morfeus Fucking Scanner
Msrabot Musobot
NICErsPRO Name Intelligence
Nameprotect Navroad
Nessus
NetAnts
NetLyzer NetLyzer
Niki-bot Netcraft
Nikto Netvibes
Nibbler
Nimbostratus Nimbostratus
Nmap
Octopus Octopus
OpenLinkProfiler Offline Navigator
OpenVAS OutclicksBot
Openfind PageScorer
PECL::HTTP Pandalytics
POE-Component-Client-HTTP Panscient
Pavuk Pavuk
Petalbot PeoplePal
PictureFinder PictureFinder
Pimonster
ProPowerBot
ProWebWalker ProWebWalker
Probethenet Pu_iN
PxBroker
QueryN Metasearch QueryN Metasearch
Quick-Crawler RankActive
RankActiveLinkBot RankFlex
Re-re RankingBot2
RedesScrapy Rankivabot
RankurBot
ReGet
Reaper
RebelMouse
Recorder
RocketCrawler
SBIder SBIder
SEOlyticsCrawler SEOkicks-Robot
SEOprofiler
SISTRIX SISTRIX
SalesIntelligent ScoutJet
ScanAlert
Scanbot
Scrapy Scrapy
Screaming Screaming
ScreenerBot ScrepyBot
SearchmetricsBot SearchmetricsBot
Semrush Semrush
SemrushBot SemrushBot
SentiBot SeobilityBot
Seomoz Seomoz
SiteCheckerBotCrawler Shodan
SiteLockSpider Site Sucker
SiteExplorer
SiteSucker SiteSucker
Siteimprove Siteimprove
Sitevigil SmartDownload
Snake Snake
Snapbot Snapbot
SocialRankIOBot Sociscraper
Sogou web spider Sogou web spider
Sosospider
SpaceBison SpaceBison
SpankBot Spanner
Spbot
SputnikBot SputnikBot
Sqlmap
Sqlworm Sqlworm
Sqworm
Steeler
Stripper Stripper
Sucuri
SuperBot SuperBot
SuperHTTP Suzuran
Swiftbot T0PHackTeam
Szukacz
T8Abot
TeleportPro TeleportPro
Telesphoreo Telesphoreo
The Intraformant
TheNomad TheNomad
TightTwatBot TightTwatBot
Titan
Toweyabot
Tracemyfile Tracemyfile
Trendiction
TurnitinBot
TwengaBot
Twice Twice
UnisterBot Typhoeus
Upflow
V-BOT V-BOT
VB Project VB Project
VCI Vagabondo
VeriCiteCrawler VidibleScraper
Voil Virusdie
Voltron Voltron
WASALive-Bot WASALive-Bot
WISENutbot
WPScan
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Wallpapers Wallpapers
WallpapersHD WeSEE
Web Auto
Web Collage Web Collage
Web Fetch Web Enhancer
Web Fuck
Web Sauger
WebBandit WebBandit
WebCollage WebCollage
WebEnhancer WebCopier
WebFetch WebFetch
WebGo IS
WebLeacher
WebSauger
WebStripper WebStripper
WebZIP WebWhacker
Webalta
WebmasterWorldForumBot
WebsiteExtractor
WebsiteQuester
Webster Webster
Whacker
Who.is Bot
Widow Widow
Wonderbot WiseGuys Robot
Wotbox Woobot
Wprecon YoudaoBot
Xaldon WebSpider
Xenu
Zauba Zauba
Zermelo
Zeus Zeus
Zitebot
ZmEu
ZoomBot ZoomBot
ZoominfoBot ZumBot
arquivo-web-crawler arquivo-web-crawler
autoemailspider autoemailspider
coccocbot-web check1.exe
cognitiveseo
crawl.sogou.com crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j
eCatch eCatch
evc-batch
facebookscraper facebookscraper
heritrix gopher
internetVista monitor instabid
ips-agent
isitwp.com isitwp.com
lwp-trivial mediawords
page scorer oBot
plumanalytics pcBrowser
ripz polaris version
s1z.ru s1z.ru
scalaj-http scalaj-http
scan.lol scan.lol
seobility
seoscanners
seostar
sexsearcher sexsearcher
sitechecker.pro sp_auditbot
siteripz spyfu
voyagerx.com trendiction.com
x09Mozilla ubermetrics-technologies.com
zauba.io zauba.io
zgrab zgrab

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
ADmantX 360Spider
AIBOT
ALittle Client
Abonti
Aboundex Aboundex
Aboundexbot Aboundexbot
Acunetix
AfD-Verbotsverfahren
AhrefsBot
AiHitBot
Aipbot Aipbot
Alexibot
AllSubmitter
Alligator Alligator
Anarchy Anarchie
Anthill
Apexoo
Asterias Asterias
Attach
AwarioRssBot
BDCbot BDCbot
BDFetch BDFetch
BackStreet BackStreet
BackWeb BackWeb
Backlink-Ceck
Badass
Barkrowler Barkrowler
BatchFTP
BetaBot BetaBot
Bigfoot Bigfoot
Black Hole
BlackWidow
BlowFish
Bolt
BotALot
Buck Buck
Buddy
BuiltBotTough
BuiltWith
Bullseye Bullseye
BuzzSumo BuzzSumo
CATExplorador CCBot
Calculon CSHttp
CazoodleBot
CheeseBot CheeseBot
CherryPicker CherryPicker
ChinaClaw ChinaClaw
Chlooe Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot
Collector Collector
CopyRightCheck CopyRightCheck
Craftbot Cosmos
CrunchBot CyotekWebCopy
DIIbot DTS Agent
Demon DataCha0s
Devil Digincore
Disco Disco
Discoverybot DnyzBot
Dispatch DomCopBot
DomainAppender DomainCrawler
DomainStatsBot
Domains Project Domains Project
Dotbot
Download Wonder Download Wonder
Dragonfly Dragonfly
Drip
ECCP/1.0 ECCP/1.0
EMail Wolf Ebingbong
Evil EroCrawler
Exabot Express WebPictures
ExtLinksBot Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms Ezooms
FDM FDM
FemtosearchBot
FlashGet FlashGet
Freeuploader Foobot
FyberSpider FrontPage
G-i-g-a-b-o-t
GT::WWW
GalaxyBot GalaxyBot
Genieo Genieo
GermCrawler GermCrawler
Gigabot Getintent
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla Grabber
Grafula
GrapeFX GrapeFX
GridBot GrapeshotCrawler
HEADMasterSEO
HMView HMView
HTMLparser HTTP::Lite
HTTrack Harvest
Haansoft
Heritrix
Hloader Hloader
HybridBot IDBot
Iblog IndeedBot
IlseBot Internet Ninja
InfoNaviRobot
Intelliseek
InternetSeer
Iria
JOC Web Spider JOC Web Spider
JennyBot
JetCar JetCar
Jetty
JikeSpider JikeSpider
Joomla Kenjin Spider
Jorgee
JustView
Keyword Density Keyword Density
Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
Lightspeedsystems LeechFTP
LinkWalker LeechGet
Linkdexbot LexiBot
LibWeb
LinkScan
LinkextractorPro LinkextractorPro
LinkpadBot LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaRSSBot
Lipperhey Spider Lipperhey
Litemage_walker
Ltx71
MIIxpc MIIxpc
MQQBrowser
MSFrontPage
MSIECrawler MSIECrawler
Majestic SEO Mail.RU_Bot
MarkMonitor Majestic-SEO
MauiBot Majestic12
MeanPath Bot Mata Hari
Meanpathbot Mediatoolkitbot
MegaIndex.ru
Metauri Metauri
MicroMessenger MicroMessenger
Microsoft Data Access Microsoft Data Access
Mister PiX Mojeek
Moblie Safari Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Musobot
NPbot NICErsPRO
Name Intelligence
NearSite
Needle Needle
NetLyzer Nessus
NetMechanic NetAnts
NetZIP Nettrack
Netvibes Nimbostratus
NextGenSearchBot Nmap
Nibbler
Nutch
Octopus Octopus
OpenLinkProfiler Offline Navigator
OpenVAS Openfind
OrangeBot Openvas
OrangeSpider
OutfoxBot
PHPCrawl PHPCrawl
PageAnalyzer POE-Component-Client-HTTP
PageGrabber PageGrabber
PageThing.com Pandalytics
Panscient
Papa Foto Papa Foto
Pavuk Pi-Monster
Petalbot Picsearch
Picscout
Pixray Pixray
PleaseCrawl
Pockey
ProWebWalker
Psbot Psbot
Pu_iN PxBroker
RSSingBot QueryN Metasearch
RankActiveLinkBot
RankingBot
RankingBot2
Rankivabot
RankurBot RankurBot
Reaper Re-re
RebelMouse ReGet
RedesScrapy Recorder
RepoMonkey Ripper
RocketCrawler
Rogerbot Rogerbot
SBIder SBIder
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SISTRIX
SMTBot
ScanAlert
Scrapy
Screaming Screaming
SearchmetricsBot ScreenerBot
Semrush ScrepyBot
SeobilityBot Searchestate
SemrushBot
Site Sucker
SiteExplorer SiteExplorer
SiteSucker SiteSnagger
Siteimprove
Sitevigil
SlySearch SlySearch
Snake Snake
Snoopy Snoopy
SocialRankIOBot
Sociscraper Sociscraper
Sottopop Spammen
Spanner Spanner
SputnikBot SputnikBot
Sucker Sqlworm
Sucuri Sqworm
SuperHTTP Steeler
Swiftbot Suzuran
T0PHackTeam T8Abot
TeleportPro
Telesoft
Telesphoreo Telesphoreo
Telesphorep
The Intraformant The Intraformant
TheNomad TheNomad
Thumbor TightTwatBot
Trendiction Toweyabot
Tracemyfile
TurnitinBot
TwengaBot
Twice
Typhoeus Typhoeus
Upflow URLy Warning
VCI V-BOT
Vacuum
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie
VoidEYE VoidEYE
Voil WBSearchBot
WEBDAV WISENutbot
WWW::Mechanize WPScan
WWW-Collector-E
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD Web Auto
Web Enhancer Web Enhancer
Web Pix Web Pix
Web Sucker WebBandit
WebCollage WebEnhancer
WebCopier
WebFuck
WebImageCollector
WebPix WebPix
WebStripper WebReaper
WebSucker WebSucker
WebmasterWorldForumBot
Website Quester Website Quester
WebsiteExtractor WebsiteQuester
Who.is Bot Who.is Bot
WiseGuys Robot WiseGuys Robot
Wonderbot
Wotbox Wotbox
Wprecon Wprecon
Xaldon_WebSpider Xaldon WebSpider
YoudaoBot Zauba
Zeus
Zitebot
ZmEu ZmEu
ZoominfoBot ZoomBot
ZyBorg arquivo-web-crawler
autoemailspider arquivo.pt
backlink-check coccocbot-web
cah.io.community com.plumanalytics
cognitiveseo
crawl.sogou.com crawl.sogou.com
crawler.feedback crawler4j
domainsproject.org demandbase-bot
evc-batch
ips-agent
lwp-trivial
magpie-crawler magpie-crawler
mediawords mediawords
muhstik-scan plumanalytics
oBot
probe-image-size probe-image-size
s1z.ru
satoristudio.net
scalaj-http scalaj-http
sitechecker.pro scan.lol
sogouspider seobility
seostar
serpstatbot
sexsearcher
sp_auditbot sp_auditbot
spyfu spyfu
sysscan sysscan
trendiction.de tAkeOut
x22Mozilla voyagerx.com
webmeup-crawler
zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
cmhomestayagency.com bimatoprost-careprost.com.ua
exhibitionplus.eu centrdebut.ru
hdwalls.xyz compliance-mark.top
indetiske.ya.ru consorzioilmosaico.org
lovi-moment.com.ua picturesmania.com
matematikus.info restaurantlescampi.com
moviezbonkerssk.cf seadragonherbery.com
surintech.ac.th toy-shop.top
tds-advert005.info videotuber.ru
unblocksit.es watch-movies.ru

926
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
123movies.love 00author.com
1flag.co.za 1-99seo.com
2000k.ru 1bet.com
214jaluwobapef.cf 1pamm.ru
273-fz.ru 2345.com
2itech.ru 2nt.ru
3th.co.in 3-letter-domains.net
4kepics.com 4istoshop.com
5forex.ru 4k-player.pl
91abcw.com 4pp13.com
aasoldes.fr 4ureyesonly.com
55wmz.ru
abctoppictures.net
abwa.tk abwa.tk
acmebtn.ml academia-nsk.org
actulite.com accessoires-mode.in
addbin.men activecampaign.dreamhosters.com
adktrailmap.com admeasures.com
adpremium.org adminshop.com
adserver-e7.com adultfullhd.com
adssafeprotected.com adultnet.in
adtology.com adventureparkcostarica.com
adult-shop.com.ua advokat-grodno.by
adultgalls.com airlandsea.info
agardomains.com alphahoverboards.com
ageofclones.com anaseracresar.tk
aibolita.com android-systems.ru
alkoravto.ru apibetweenlinesn-a.akamaihd.net
all4wap.ru apimossnetinfo-a.akamaihd.net
allstatesugarbowl.org apiportalmorecom-a.akamaihd.net
allwidewallpapers.com apispringsmartne-a.akamaihd.net
allwomen.info appfastplay.com
apartment.ru appleid-verification.com
apartmentbay.ru aquarium-pro.ru
apiqualitinknet-a.akamaihd.net
apisecretsaucebi-a.akamaihd.net
approvedlocal.co.za
arabsexxxtube.com arabsexxxtube.com
ardimobileinfo.ml arcteryxstore.online
artefakct.com arewater.com
asrvvv-a.akamaihd.net artpicso.com
autoseo-traffic.com asseenontvonline.ru
avramstroy.ru atelielembrancaqueencanta.com.br
azadnegar.com auto4style.ru
azbukafree.com autodo.info
azlex.uz autoua.top
avtointeres.ru
azbuka-mo.ru
backgroundpictures.net
bagcionderlab.com bagcionderlab.com
bannerswap.com balkanfarma.org
beauty-b0x.pl bankcrediti.ru
bestimagecoollibrary.com bankiem.pl
bestmarriages.com bankofthewext.com
bestmouthwash.club basisches-wasser.net
bestofferswalkmydogouteveryday.gq becuo.com
bestofupload.info belinvestdom.by
bestserials.com belsetirehafi.tk
bioscorp.ru best-businessman.ru
bluerobot.info bestbrainprod.win
boltalko.xyz bestpriceninja.com
bookmark4you.com.biz besttorrentknifta.weebly.com
boom-celebs.com bikini-image.com
brainzpod.win bin-brokers.com
bristolhostel.com bombla.org
btnativenav.com bonzbuddy.com
buntube.net brandedleadgeneration.com
bus-offer.com buyantiviralwp.com
businescoop.men buypanicdisorderpill.com
buydissertation.net buzzonclick.com
buyessaynow.biz buzzsumo.com
buynorxx.com
buytizanidineonlinenoprescription.blogspot.com
bwlx.prepedu.cn
c-english.ru
camel-beach.com camel-beach.com
candy-glam-hp.com canyougethighofftizanidine.blogspot.com
canvas.gloverid.site capacitacionyaprendizaje.com
carabela.com.do cartierlove.xyz
carapics.com cartierlove2u.com
cartierreplica.win
cartiertrinity.xyz cartiertrinity.xyz
cartierwatch.xyz cdncash.com
cdncash.net cdncash.org
cenokos.ru cellfun.mobi
chatseo.com censys.io
cheapsergiorossi.online centre-indigo.org.ua
checkm8.com chinajerseyswholesalecoupons.com
chorus.terakeet.com
christianlouboutinshoes.xyz christianlouboutinshoes.xyz
cialischmrx.com cindymatches.com
clickso.com cineacademy.ru
comissionka.net cjs.com.ru
community.allhiphop.com clarithromycin500mg.com
compliance-alexa.top claytransformations.info
compliance-ivan.xyz clickbanksites.info
compliance-julianna.top coaching-netz.info
compliance-olga.top cometorussia.net
computernetworksonline.com compliance-mark.top
copblock.org compliance-mary.top
copenergo.ru concealthyself.com
connectionstrenth.com
cookie-law-enforcement-ff.xyz
cookingmeat.ru
cool-wedding.net
coolwallpapers-hd.com
cosmetic.donna7753191.ru
cottageofgrace.com
counter.yadro.ru counter.yadro.ru
cracksplanet.com counterbot.com
crackzplanet.com crafthubs.com
creditmoney.com.ua cs-passion.pl
culturevie.info daneshetabiat.com
customerguru.in
cutt.us
d2jsp.org
dailystrength.org
damasarenai.info
dandiyabeats.in
darkbooks.org darkbooks.org
data-mining.tk
datadepths.com
dataloading.net
dayibiao.com dayibiao.com
dbmkfhqk.bloger.index.hr
ddpills.com ddpills.com
dear-diary.ru deadlinkchecker.com
decoratinghomes.ga
delta-line.men delta-line.men
detalizaciya-tut.biz depositfiles-porn.ga
detective01.ru destinationrealestate.com
dfwu1019.info dieta-personalna.pl
dickssportinggoods.com dikx.gdn
directrev.com div.as
divci-hry.info djihispano.com
divisioncore.com dnepr-avtospar.com.ua
dogclothing.store dollartree.info
dojki-hd.com downvids.net
dojki365.online doyouknowtheword-flummox.ml
domain.webkeyit.com draniki.org
domainanalyzing.xyz drpornogratisx.xxx
domaineaneblanc.com
dostavka-v-krym.com
dosugrostov.site
dotnetdotcom.org
download-of-the-warez.blogspot.com
download-walpaperhd.blogspot.com
downloader12.ru
drugstoreforyou.com
drunkenstepfather.com
dumpsmania24.com dumpsmania24.com
dwomlink.info
dynainbox.com dynainbox.com
dyt.net econom.co
eachdayisagift.review elektroniksigaraankara.info
educatemetv.com
efkt.jp
ekspertmed.com
elektrischezigarette1.onsugar.com
elektronischezigarettekaufen1.beeplog.com elektronischezigarettekaufen1.beeplog.com
elektronischezigarettekaufen2.tumblr.com elkacentr.ru
emailaccountlogin.co elstal.com.pl
en.altezza.travel eluxer.net
enbersoft.com eng-lyrics.com
enginebay.ru ereko.ru
engines-usa.com eropho.net
erotik0049.com
erotikstories.ru erotikstories.ru
erotische-geschichten-xxl.com essayassist.com
essaypro.com evaashop.ru
etotupo.ru exct.net
europages.com.ru extads.net
evidencecleanergold.com ezigarettekaufen2.yolasite.com
excitacao.com ezigarettenkaufen1.pagina.gr
expertnaya-ocenka.ru ezigarettenshop2.postbit.com
extener.org f-online.de
extlabs.io faracontrol.ir
extrabot.com figuringmoneyout.com
ez8motelseaworldsandiego.com filmbokep69.com
ezigarettekaufen2.blox.pl findpics.pw
ezigaretteshop.webs.com finemanteam.com
factorynightclub.com finteks.ru
failingmarriege.blogspot.com firma-legion.ru
faptitans.com floating-share-buttons.com
fba-mexico.com foodcrafts.website
feargames.ru forsex.info
feel-planet.com
ferrotodo.com
filkhbr.com
findpik.com
fm-upgrade.ru
foodgid.net
formseo.com
forum.poker4life.ru
foto-telok.net
franch.info franch.info
freelifetimefuckbook.com franklinfire.co
front.to freakycheats.com
fxtips.ru free-fbook-traffic.com
fym.com.ru free-share-buttons.blogspot.com
g33.org freegamesplay.online
galaxy-family.ru freejabs.com
gaz-voshod.ru freewhatsappload.com
gemgrab-a.akamaihd.net fres-news.com
freshberry.com.ua
ftns.ru
fun2cell.net
funnel.co.za
gallerylisting.com
gallictures.com
gamerscorps.com
germetiki.com.ua germetiki.com.ua
gettpromos.com gg.zzyjxs.com
ghazel.ru gidonline.one
ghostvisitor.com gigixo.com
gifspics.com glall.ru
gilsonchiro.xyz
glavtral.ru glavtral.ru
go2jump.org go2jump.org
golmau.host.sk gombita.info
goodnightjournal.com google-liar.ru
googlemare.com gotwebsite1.com
goosefishpost.bid gowreckdiving.com
gov.yanao.ru growboxbl.ru
gratis-sexkontakte.com gsou.cf
gratuitbaise.com guiadeserraazul.com
greamimgo.pw
guidefs.ru
guruofcasino.com
hanwei.us hanwei.us
hatdc.org hdpixion.com
hawaiielectriclight.com hdwallpapers-free.com
hdhc.ru heatpower.ru
headpharmacy.com highspeed5.net
healgastro.com hmmm.cz
helpmymacfaster.trade homedo.fabpage.com
hentai-manga.porn homegardenlova.com
hifidesign.ru hostingclub.lk
histats.com hostnow.men
hitmuzik.ru hotcore.info
homedecoguide.info hothot.ru
hornymatches.com hoverboard360.es
hothor.se hplaserjetpdriver8y.pen.io
hotxnights.info https-legalrc.biz
housemilan.ru
howopen.ru
hustoon.over-blog.com hustoon.over-blog.com
ibmdatamanagement.co ibmdatamanagement.co
id-forex.com images.gyffu.com
igithab.com incanto.in.ua
ilikevitaly.com incep.imagine-ex.co
imgarit.pw inmate-locator.us
immobiliaremassaro.com intimshop-fantasy.ru
innoslicon.com investingclub.ru
inome.com.ua iqupdatetmz.win
intelhdgraphicsgtdrive6w.metroblog.com irkutsk.online-podarki.com
internetproviderstucson.com itch.io
isitpaleo.info ivearchenceinflu.cf
itronics.ca javatex.co.id
ivanstroi.ru
iwanttodeliver.com
j33x.com
jabimgo.pw
jasonpartington.com
jbl-charge.info
jerseyschinabizwholesale.us jerseyschinabizwholesale.us
juliaworld.net kalb.ru
kabbalah-red-bracelets.com kaliningrad.zrus.org
kam-dom.ru katjimej.blog.fc2.com
kandidos.com kemerovo.zrus.org
karadene.com keyword-suggestions.com
kaz.kz keywordhouse.com
kino-fun.ru keywords-monitoring-your-success.com
kino-key.info khafre.us
kino-rating.ru kinobest.pl
kinocccp.net klosetkitten.com
kollekcioner.ru kmgamex.cf
konteiner24.com kndxbkdx.bloger.index.hr
krasivoe-hd.com kongruan.com
kustanay.kz krasnodar.ru
kutikomi.net landoftracking.com
lafourmiliaire.com lapolis.it
landscaping.center
lapitec.eu
larose.jb2c.me larose.jb2c.me
larutti.ru lashstudia.ru
law-check-seven.xyz lasvegaslockandsafe.com
letolove.ru laulini.soclog.se
life.biz.ua lawyers.cafe
liveu.infoteka.hu lazy-z.com
locatellicorretor.com.br lechenie-gemorroya.com
locksmith.jp lechtaczka.net
lollypopgaming.com linkdebrideur.xyz
lomza.info lisque.batcave.net
lonfon.xyz littleberry.ru
louboutinbooties.xyz littlecity.ch
louboutinreplica.xyz lolnex.us
lunchrock.co lomb.co
machicon-akihabara.info long-beach-air-conditioning.com
machicon-ueno.info lovasszovetseg.hu
mafa-free.com lumb.co
mafcards.ru mackeeper-land-672695126.us-east-1.elb.amazonaws.com
make-money-online.com magiadinamibia.blogspot.com
marketland.ml magicalfind-a.akamaihd.net
matematikus.info maju.bluesalt.co
medi-fitt.hu managerpak204.weebly.com
medispainstitute.com.au manimpotence.com
masterseek.com
mastershef.club
mediafresh.online
medicinacom.ru
member-quiz.com
merryhouse.co.uk
methodsmarketing.com
michaelkorsoutlet.store michaelkorsoutlet.store
michellblog.online mir-betting.ru
microstatic.pl
mirtorrent.net mirtorrent.net
misterjtbarbers.com mmstat.com
mixx.com moesen-ficken.com
mnogolok.info moi-glazki.ru
mobilemedia.md mokrayakiska.com
monclerboots.xyz moncleronline.xyz
morefastermac.trade moroccosurfadventures.com
morf.snn.gr mosdverka.ru
moviezbonkerssk.cf movpod.in
mtmtv.info mriyadh.com
mygameplus.com mrwhite.biz
myhitmp3.club msk.afora.ru
mylovelibrabry.com mturkcontent.com
myphotopipe.com mulberryoutletonlineeu.com
mysexpics.ru mybinaryoptionsrobot.com
myshopmatemac.com myhealthcare.com
narkologiya-voronezh.ru myiptest.com
naturalbreakthroughsresearch.com mysex21.com
news-speaker.com mywallpaper.top
npoet.ru najaden.no
nucia.biz.ly narkologiya-penza.ru
nuup.info netallergy.ru
nw-servis.ru nevansk.ru
oballergiya.ru nibbler.silktide.com
oecnhs.info nonews.co
of-ireland.info nosecret.com.ua
ofanda.com nottyu.xyz
offtopic.biz oconto.ru
okayimage.com office-windows.ru
okmedia.sk olvanto.ru
oneminutesite.it onclkads.com
onlinewritingjobs17.blogspot.ru onetravelguides.com
ooo-gotovie.ru onko-24.com
open-odyssey.org onlinebay.ru
openmediasoft.com onlinemeetingnow.com
ops.picscout.com onlythegames.com
orgasmatrix.com optom-deshevo.ru
origin-my.ru oranga.host.sk
orion-code-access.net oren-cats.ru
osb.se11.ru orenburg-gsm.ru
owathemes.com osagonline.ru
panchro.co.uk osnova3.ru
perfection-pleasure.ru ourville.info
pewit.pw painting-planet.com
php-market.ru pandroid.co
pictures-and-images.net paramountmarble.co.uk
pinup-casino1.ru partners-ship.pro
pinwallpaper.xyz pcads.ru
pixell.club penisvergrotendepillennl.ovh
pizda.lol photo.houseofgaga.ru
pizdeishn.com piccdata.com
pl-vouchers.com picurams.pw
plastjulgranar.se pills24h.com
platesauto.com pillscheap24h.com
poppen-nw.net pix24x7.com
porno-home365.com pixgood.com
pornofeuer.com pixshark.com
pornzone.tv plastgran.com
pourvous.info poisk-zakona.ru
powc.r.ca.d.sendibm2.com poligon.com
pozdrawleniya.com popup-fdm.xyz
preconnubial.usuby.site porndairy.in
pregnant.guru porno-dojki.net
privatefx-in.ru porno-raskazy.ru
prizefestival.mobi pornobrazzers.biz
prod2016.com pornoklad.ru
profitkode.com pornosee.info
promalp-universal.ru pornosemki.info
propranolol40mg.blogspot.com pornosok.ru
prostitutki-belgoroda.org pornotubs.com
prpops.com postclass.com
r-control.ru pozdrawleniya.ru
rada.ru predmety.in.ua
ranksonic.com printie.com
rarbg.to privhosting.com
rcb101.ru promover.org
refererx.com pronorm.fr
refudiatethissarah.info prosmibank.ru
regionshop.biz proxyelite.biz
rent2spb.ru qexyfu.bugs3.com
richinvestmonitor.com queerspace.com
rocketchange.ru raisedseo.com
rogervivierforsale.com ranksonic.org
rusvideos.su rapevideosmovies.com
rvi.biz ratemodels.net
s8-nowy-wygraj.comli.com realtytimes.com
recipedays.com
redhotfreebies.co.uk
referencemoi.com
registry-cleaner.net
respublica-otel.ru
ric.info
rightenergysolutions.com.au
rock-cafe.info
rosbalt.com.ua
rossmark.ru
royalads.net
ruclicks.com
rusprostitute.com
russintv.fr
russkie-gorki.ru
saddiechoua.com
sady-urala.ru sady-urala.ru
sarm.tk saletool.ru
savetubevideo.com samara.rosfirm.ru
sbf441.com sankt-peterburg.nodup.ru
scanmarine.info savefrom.com
scanner-jess.top saveriopiazza.it
se.bnt-team.com sax-sex.com
scanner-mark.top
schalke04fc.info
score-ads.men
searchmywindow-a.akamaihd.net searchmywindow-a.akamaihd.net
seo18.su sei80.com
seoanalyses.com seksvideoonlain.com
serpstat.com seminarygeorgia59.ga
semrush.com
seo-platform.com
seorussian.ru
seotoolsagency.com
serving.adbetclickin.pink
sex-dating.co sex-dating.co
sex-watch.com sexy-screen-savers.com
sexiporno.net share-buttons-for-free.com
sextracker.com shiksabd.com
sexy.babes.frontend-stack.top shlyahten.ru
sexyebonyteen.com shopvilleroyboch.com.ua
sfd-chess.ru shtaketniki.ru
shemalegalls.blogporn.in sierraapps.com
sicfor.bcu.cc silverage.ru
simple-image.com.ua sindragosa.comxa.com
simply.net
sivs.ru
sky-mine.ru sky-mine.ru
smailik.org skypasss.com
small-game.com slutloadlive.com
snworks.com smart-scripts.com
social-s-iii.xyz smokewithrabbits.com
socialseet.ru social-vestnik.ru
softonicads.com soft1.ru
solinf.co softomix.net
sosdepotdebilan.com sonnikforme.ru
space4updating.win sovinsteel.ru
spamnuker.com spinnerco.ca
speeddream.xyz spravka130.ru
sport7777.net sps-shop.com
sportbetfair.com spy-app.info
spyfu.com srdrvp.com
stroyhelp-dv.ru stair.registrydomainservices.com
superinterstitial.com starpages.net
supermodni.com.ua steebook.com
supers.com.ua stop-gepatit.te.ua
superstarfloraluk.com styro.ru
svbur.ru sugarkun.com
svnuppsalaorebro.se sukirgenk.dvrlists.com
tacbibirfa.tk surfbuyermac.com
taraz.xkaz.org swagbucks.com
tattooreligion.ru theautoprofit.ml
tecspb.ru thegameriders.com
teguh.info thegioixekhach.com
tetracsaudi.com themeforest.net
thaisamkok.com thenetinfo.com
the-usa-games.blogspot.com thetoiletpaper.com
thelottosecrets.com time-japan.ru
themestotal.com tizanidineandcipro.blogspot.com
thepantonpractice.co.uk tizanidinehcl4mgisitanarcotic.blogspot.com
tizanidinedosage.blogspot.com tizanidinehcl4mgtab.blogspot.com
tizanidineoral.blogspot.com tizanidineingredients.blogspot.com
toloka.hurtom.com tizanidineshowupondrugtest.blogspot.com
tonerbox.kz tizanidinesideeffectsweightloss.blogspot.com
tonivedu.it tkanorganizma.ru
toon-families.com tnctrx.com
topappspro.com topcar-krasnodar.ru
topquality.cf topshef.ru
touchmods.fr topsiteminecraft.com
toursmaps.com torrents.life
toxicwap.com tradgardspartner.se
tracklead.net trafficjunky.net
traffic2money.com trahic.ru
trafficnetzwerk.de trahvid.com
traffixer.com truck-land.life
traffmonster.info ttsq.fr
tsstcorpcddvdwshbbdriverfb.aircus.com ukrobstep.com
tuckermktg.com ultimatesetnewfreeallsoftupgradesystems.pw
turist-strani.ru
turkeyreport.tk
turvgori.ru
twittrading.com
twodollarshows.com
ublaze.ru
unisexjewelry.org
uptime.com uptime.com
utrolive.ru uytmaster.ru
vacances-voyages.info valmetrundan.se
veselokloun.ru vasileostrovsky-rayon.ru
via-energy-acquistare.com video-chat.in
viddyoze.com video-hollywood.ru
videosbox.ru video-woman.com
vinylvault.co.uk viktoria-center.ru
virus-respirators.com vipsiterip.org
vltai.com vkontarkte.com
voluumtracker1.com vmnmvzsmn.over-blog.com
vseigru.one voloslove.ru
vtcdns.com vuclip.com
vucms.com
vvon.co.uk
vzlomtw.com vzlomtw.com
vzubah.com wait3sec.org
webinstantservice.com walkme.com
weboptimizes.com wallpaperstock.net
wellcome2slovenia.ru wapsite.me
wesharepics.info warningzscaler.heraeus.com
wildcattube.com watsonrealtycorp.com
womens-journal.net waysbetter.cn
worst-sites.online web-betting.ru
wpsecurity.website websiteaccountant.de
wurr.voila.net westum.se
ww1943.ru white-truck.life
x-diesel.info wildnatureimages.com
x-lime.net wineration.com
xaijo.com x-rates.ru
xcc24.pl xedserver.com
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--1-8sbcpb0bdm8k6a.xn--p1ai xn--80aeahghtf8ac5i.xn--p1ai
xn--h1aakne2ba.xn--p1ai xn--b1adccaf1bzj.xn--p1ai
xn--hxazdsfy.blogspot.com xn--d1aifoe0a9a.top
yeartwit.com xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
yoluxuryevents.com xwatt.ru
youbloodyripper.com xxxguitars.com
xxxsiterips.xyz
yatrk.xyz
yes-com.com
yopoint.in
yoshkarola.zrus.org
z-master.ru
zagadki.in.ua
zdesformula.ru zdesformula.ru
zigarettenonlinekaufen1.bloog.pl zed21.net
zigzog.ru zetgie.com.pl
znakom.sibtest.ru zkjovpdgxivg.ga
znakomstva-piter78.ru
zocaparj.kz zocaparj.kz
zynax.ua zrizvtrnpale.tk
zscalerone.net
zvetki.ru
zytpirwai.net zytpirwai.net
zzbroya.com.ua zzbroya.com.ua

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1pamm.ru 123locker.com
3-letter-domains.net 2daytrendingnews.com
3w24.com 4ur.click
4inn.ru 7fon.ru
4ureyesonly.com 7ooo.ru
aab94f698f36684c5a852a2ef272e031bb.com aa9bd78f328a6a41279d0fad0a88df1901.com
aadroid.net abctoppictures.net
adservme.com acunetix-referrer.com
adtology.com adtech.us
agardomains.com adviceforum.info
ald2014.org adzpower.com
alevinefesleri.com afmuseum.com
alfa-img.com aktivator-windows10.blogspot.com
all-number.com alfa9.com
anal-acrobats.com allesohnegirls.net
animalia-life.club allnews24.in
anmysite.com allwomen.info
app5.letmacworkfaster.world anime.dougasouko.com
arvut.org arclk.net
auto-moto-elektronika.cz avtolombard-krasnodar.com
aviav.ru.com bag77.ru
babyfactory.fr belinvestdom.by
backuperwebcam.weebly.com belstaffstore.online
bestwebsiteawards.com bestprofits-there.com
beta.hotkeys.com biglistofwebsites.com
bigpictures.club blockworld.ru
bongiornos.info bloggerads.net
bookhome.info bochemit.com.ua
brianhenry.co breastaugmentation.co.za
bryansk.zrus.org buigas.00it.com
buytizanidineonline.blogspot.com cacheimages.com
bvlgariwallet.xyz calstaterealty.us
c-gainsbourg.com cheap-trusted-backlinks.com
call-of-duty.info checkm8.com
chatroulette.video choosecuisine.com
cheapmarmot.online clash-clans.ru
clickzzs.nl clickhype.com
club-samodelkin.ru collegerentals.com
compliance-alexa.top com-secure.download
compliance-checker-7.info compliance-don.top
compliance-elena.xyz compliance-irvin.xyz
concealthyself.com compliance-ivan.top
coolingoods.com cookie-law-enforcement-hh.xyz
cordstrap.cc countbertwistdisp26.soup.io
cs-passion.pl counterbot.com
csgo4.win credit.co.ua
d2jsp.org custom-electric-guitar.com
daymusam.com
dbmkfhqk.bloger.index.hr
decenttools.com decenttools.com
descargar-musica-gratis.net dengi-pod-zalog-nedvizhimosti.ru
deutschland123.de detki-opt.ru
devochki.top dildofotzen.net
dignitasdata.se divatraffic.com
dojki-hd.com dojki-hd.com
domznaniy.ru domaincrawler.com
doreenblog.online editmedios.com
dorratex.tn ekn-art.se
drivotracker.com eldiariodeguadalajara.com
drpornogratisx.xxx elektrischezi.canalblog.com
ed-shop01.ru erotiktreff24.info
ednorxmed.com ewebarticle.info
ekobata.ru excaliburfilms.com
elektrischezigaretten1.blogse.nl exoclick.com
englate.com fashion-mk.net
eorogo.top fatmaelgarny.com
erotik0049.com fefo.gdn
erotikstories.ru filmbokep69.com
ertelecom.ru findpics.pw
european-torches.ru firstdrugmall.ru
exmasters.com fishtauto.ru
ezigarettenshop2.postbit.com foto-basa.com
fashion-stickers.ru franecki.net
fdzone.org free-share-buttons.blogspot.com
fickenbumsen.net free411games.com
flprog.com freevpn.space
food.dtu.dk freewlan.info
free-deals.faith gate5.co.za
freewhatsappload.com goodhousekeeping.com
freshsuperbloop.com goodhumor24.com
fyxabomiw.ru gosmeb.ru
gael-s.ru hackers-crackers.tk
galaxy-family.ru hawaiielectriclight.com
gameplexcity.com hdimagelib.com
gay.adultgalls.com heartofpayne.xyz
getmiro.com heliko.no
gg-arena.ru holidaypics.org
gifspics.com homeart.space
given2.com homedo.fabpage.com
glass-msk.ru hotchatdate.com
goldadpremium.com hplaserjetpdriver8y.pen.io
goldpanningtools.com hyipmanager.in
goodly.pro ibb.com.ua
gopro-online.info ilmexico.com
gorodservis.ru imitex-plus.ru
gotcher.us impresagaia.it
gotorussia.com incanto.in.ua
grtyi.com inome.com.ua
guarrasdelporno.xxx investpamm.ru
harmonyglen.us istmira.ru
housemilan.ru itservicesthatworkforyou.com
hptwaakw.blog.fc2.com jackpotchances.com
huhn.altervista.org japfm.com
hunterboots.online javitas.info
i-midias.net.br jwss.cc
ideawheel.com kinofak.net
images-free.net kskjco.club
imgarcade.com live-sexchat.ru
incomekey.net livefixer.com
instakink.com lkbennettoutlet.online
int.search.mywebsearch.com lsex.xyz
intellektmedia.at magda-gadalka.ru
jacago.com mall.uk
junketjuice.blogspot.com manve.info
kamagragelusa.net marketingtechniques.info
katjimej.blog.fc2.com mastershef.club
kinogolos.ru masthopehomes.com
kmd-pto.ru matb3aa.com
kutikomi.net mebel-ekb.com
libertybilisim.com mebelcomplekt.ru
lider-zhaluzi.kiev.ua mediaoffers.click
lietaer.com megagrabber.ru
likesuccess.com mejoresfotos.eu
lionking-1994.blogspot.com mex-annushka.ru
locksmith.jp microstatic.pl
magicalfind-a.akamaihd.net mobplayer.ru
manifestation.betteroffers.review monclercheap.xyz
manualterap.roleforum.ru money-every-day.com
matchpal-a.akamaihd.net moneyviking-a.akamaihd.net
mearns-tractors.co.uk morlat.altervista.org
medkletki.ru mrmoneymustache.com
mega-bony-2017.pl
mirobuvi.com.ua
misandesign.se
mountainstream.ms
mrvideospornogratis.xxx mrvideospornogratis.xxx
mttwtrack.com
mundoaberrante.com mundoaberrante.com
music.utrolive.ru mydoctorok.ru
mustat.com mymobilemoneypages.com
myonlinepayday.co new7ob.com
naj-filmy24.pl ok-ua.info
naperehresti.info olvanto.ru
newstudio.tv onetravelguides.com
nomuos.it online-zaymy.ru
nvformula.ru onlinemegax.com
odesproperty.com onthemarch.co
openfrost.com optikremont.ru
pdns.download owathemes.com
pekori.to pekori.to
perfectplanned.com perl.dp.ua
piccdata.com pharmacyincity.com
pirateday.ru photo.houseofgaga.ru
piski.top picsfair.com
pixshark.com pillscheap24h.com
pladform.ru piter.xrus.org
pogosh.com pivka.xyz
poisk-zakona.ru pliks.pl
pokemon-go-play.online poligon.com
porno-video-chati.ru popander.mobi
pornofeuer.com poppen-nw.net
pornoplen.com pornosee.info
pornoreino.com praisong.net
portadd.men
preparevideosafesystem4unow.site preparevideosafesystem4unow.site
prlog.ru privatamateure.com
pron.pro prombudpostach.com.ua
prospekt-st.ru pssucai.info
pts163.ru pwwysydh.com
purplesphere.in pxhdwsm.com
purplestats.com razvratnoe.org
pussysaga.com rewardit.com
puteshestvennik.com rosbalt.com.ua
quebec-bin.com runstocks.com
rankingchart.de rvzr-a.akamaihd.net
realmonte.net sanidumps.com
remorcicomerciale.ro
resultshub-a.akamaihd.net
rfserial.net
rospromtest.ru
ruclicks.com
rusoft-zone.ru
sabaapress.com
sanjosestartups.com sanjosestartups.com
savememoney.co.za sawin.beth.webd.pl
scansafe.net sax-sex.com
scoopquest.com sbornik-zakonov.ru
searchinquire.com searchimpression.com
searchkut.com searchwebknow-a.akamaihd.net
seminarygeorgia59.ga seksvideoonlain.com
semp.net service.adtech.us
servethis.com sexkontakte-seite.com
serw.clicksor.com sexspornotub.com
seryeznie-znakomstva.ru sexytrend.ru
share-buttons-for-free.com shareyards.com
shikiso.info
ship-marvel.co.ua
shmetall.com.ua
shop-electron.ru shop-electron.ru
sierraapps.com shoplvlv.us
siteimprove.com siha.de
sitiz.club siteonomy.com
smartadserver.com sklad-24.ru
social-s-ggg.xyz sluganarodu.ru
socialsignals24.com social-buttons.xyz
sotkal.lark.ru soda.media
spamnuker.com softwaretrend.net
stair.registrydomainservices.com sogimlecal.tk
stauga.altervista.org sovetogorod.ru
surintech.ac.th spidtest.org
tacbelarus.ru spravka-medosmotr.ru
tandvardshuset.net spywarenuker.com
templates.franklinfire.co squidoo.com
tengohydar.tk statoutlook.info
theautoprofit.ml stefanbakosab.se
thecoolimages.net strangeduckfilms.com
surflinksmedical.com
swingerseiten.com
teesdaleflyballclub.co.uk
thetattoohut.com
thetoiletpaper.com thetoiletpaper.com
tizanidine4mgtablets.blogspot.com tomatis.gospartner.com
tizanidineorflexeril.blogspot.com
tmearegion26.com
torrent-to-magnet.com torrent-to-magnet.com
torrents-tracker.com traffic-club.info
tovaroboom.vast.ru tramadolandtizanidine.blogspot.com
truebeauty.cc trustedmaccleaner.com
twojebook.pl tubo360.com
underthesite.com uchebavchehii.ru
uogonline.com udsgame.online
uptime-gamma.net uggbootsoutletsale.us
urlcut.ru ugguk.online
website-stealer.nufaq.com upproar.com
welck.octopis.com uptime-as.net
wemarketing.se urdoot.win
whatsupinfoley.com url-extractor.xyz
worldis.me url2image.com
wrz0iuebwhp5fg.freeddns.com urlopener.blogspot.com.au
www.souvenirua.com vedomstvo.net
xerox-douglas.cf video-chat.in
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai vietimgy.pw
xn--80abgj3a5acid6ghs.top voluumtrk.com
xn--80aeahghtf8ac5i.xn--p1ai vtcdns.com
xsfetish.org vzlom-na-zakaz.com
xtubeporno.net w7s.ru
xxart.ru webmasterhome.cn
xxxmania.top weboptimizes.com
xxxnatelefon.ru website-datenbank.de
yebocasino.com webstatsdomain.org
yellowproxy.net wineitudes.wordpress.com
youbloodyripper.com womama.ru
yoursite.com womensplay.net
youtubologia.it word-vorlagen.xyz
zarabotat-na-sajte.ru x-diesel.biz
zarabotok--doma.ru xn--b1adccaf1bzj.xn--p1ai
zeikopay.com xxx-cam.webcam
yougame.biz
youporn-ru.com
ypmuseum.ru
yurgorod.ru
zarenica.net
zhorapankratov7.blogspot.com zhorapankratov7.blogspot.com
zootoplist.com zone-kev717.info
zophim.me

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.35 0; 10.1.0.95 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2519 ### Version: V4.2021.08.2520
### Updated: Wed Aug 11 22:01:07 UTC 2021 ### Updated: Thu Aug 12 10:03:29 UTC 2021
### Bad Bot Count: 620 ### Bad Bot Count: 620
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2519 ### Version: V4.2021.08.2520
### Updated: Wed Aug 11 22:01:07 UTC 2021 ### Updated: Thu Aug 12 10:03:29 UTC 2021
### Bad Bot Count: 620 ### Bad Bot Count: 620
################################################### ###################################################
### Version Information ## ### Version Information ##