vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.09.2564 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-09-01 22:07:21 +00:00
parent df6937177d
commit 5df40eb77a
46 changed files with 60162 additions and 60162 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2563 #### Version: V4.2021.09.2564
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2563 #### Version: V4.2021.09.2564
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.09.2563 #### Version: V4.2021.09.2564
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7386
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.09.2563 #### Version: V4.2021.09.2564
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7390
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7390
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

7390
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7390
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 1; 10.1.0.198 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 0; 10.1.0.198 0;
127.0.0.1 0; 127.0.0.1 0;

7390
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 1; 10.1.0.198 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 0; 10.1.0.198 0;
127.0.0.1 0; 127.0.0.1 0;

7390
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 1; 10.1.0.198 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 0; 10.1.0.198 0;
127.0.0.1 0; 127.0.0.1 0;

7390
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2563 2564

7390
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

7390
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.69.159.17 0; 13.67.176.123 0;

7390
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 1; 10.1.0.198 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 0; 10.1.0.198 0;
127.0.0.1 0; 127.0.0.1 0;

7390
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 1; 10.1.0.198 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 0; 10.1.0.198 0;
127.0.0.1 0; 127.0.0.1 0;

7390
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 1; 10.1.0.198 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 0; 10.1.0.198 0;
127.0.0.1 0; 127.0.0.1 0;

7390
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.69.159.17 0; 13.67.176.123 0;

7390
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.69.159.17 0; 13.67.176.123 0;

7390
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 1; 10.1.0.198 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
CODE87 Aboundexbot
CSHttp CheeseBot
DBLBot Gotit
Mail.RU_Bot Ltx71
NearSite Mr.4x3
OnCrawl Netvibes
RepoMonkey Pavuk
Screaming RebelMouse
Swiftbot Voil
Who.is Bot WebsiteExtractor

294
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
404checker 404enemy
80legs ALittle Client
Aboundex ASPSeek
Aboundexbot Aboundexbot
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot
AiHitBot AiHitBot
AllSubmitter
AlphaBot AlphaBot
Anarchie Anarchie
Anarchy99
Asterias
Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot BBBike
BDFetch
BLEXBot
BackStreet BackStreet
Backlink-Ceck Backlink-Ceck
Badass Barkrowler
Bandit BlackWidow
Bigfoot Boardreader
Bitacle Brandprotect
Blow
Bolt
BotALot
Buck Buck
BuiltBotTough
Bullseye Bullseye
BunnySlippers
BuzzSumo
CCBot
CODE87 CODE87
Cegbfeieh
CheTeam CheTeam
CheeseBot
ChinaClaw
Chlooe
Cliqzbot Cliqzbot
Cloud mapping Cloud mapping
Collector Cogentbot
Copier CopyRightCheck
Craftbot Custo
Crawling at Home Project DTS Agent
Curious DataCha0s
CyotekWebCopy
DBLBot
DIIbot
DSearch
Demon
Digincore Digincore
Disco Dirbuster
DomainSigmaCrawler Discobot
Discoverybot
Dispatch
DittoSpyder
DomainStatsBot DomainStatsBot
Dotbot Dotbot
Download Wonder
Dragonfly
Drip Drip
EMail Siphon EMail Siphon
EasyDL EasyDL
Ebingbong Ecxi
EroCrawler EroCrawler
Express WebPictures Extreme Picture Finder
ExtractorPro FemtosearchBot
FHscan
Fimap
Firefox/7.0
Freeuploader
FrontPage FrontPage
Fuzz Fuzz
FyberSpider
Fyrebot Fyrebot
GT::WWW GT::WWW
GalaxyBot
Genieo
GetRight GetRight
Gigabot Getintent
Go-Ahead-Got-It Go!Zilla
GoZilla
Gotit Gotit
Grafula GrapeshotCrawler
GrapeFX GridBot
HEADMasterSEO HEADMasterSEO
HMView HTMLparser
Harvest HTTP::Lite
Humanlinks HaosouSpider
HybridBot Heritrix
IRLbot IRLbot
InfoTekies Iblog
IlseBot
Image Fetch
Image Sucker
Indy Library
Intelliseek Intelliseek
Iskanie
JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz
JennyBot JennyBot
JetCar JetCar
Joomla Joomla
JustView Jorgee
Kenjin Spider Kinza
Keyword Density Larbin
Kozmosbot
Leap
LeechFTP LeechFTP
Lightspeedsystems LibWeb
Likse Libwhisker
LinkWalker LieBaoFast
Linkbot LinkextractorPro
LinqiaMetadataDownloaderBot LinksManager
LinqiaRSSBot LinqiaScrapeBot
MFC_Tear_Sample Lipperhey
MQQBrowser Litemage_walker
MSIECrawler MSIECrawler
Magnet Magnet
Majestic SEO Majestic SEO
Majestic12 Majestic-SEO
MarkWatch
MauiBot MauiBot
Metauri Mb2345Browser
MicroMessenger Microsoft Data Access
Microsoft URL Control
Mister PiX
Moblie Safari Moblie Safari
Mojeek
Mojolicious Mojolicious
Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3
Msrabot
NPbot
Name Intelligence Name Intelligence
Nameprotect Navroad
Needle NetAnts
NetSpider NetSpider
NetZIP Netvibes
Netcraft
Nibbler Nibbler
Niki-bot
Nikto Nikto
NimbleCrawler Nimbostratus
Nmap Ninja
Nuclei
Offline Navigator Offline Navigator
OpenLinkProfiler OnCrawl
OpenVAS
OrangeBot OrangeBot
PECL::HTTP OutclicksBot
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer PageGrabber
PageScorer PageScorer
Papa Foto Pandalytics
Pavuk Pavuk
PictureFinder PeoplePal
Petalbot
Pi-Monster
Picsearch
Piepmatz
Pimonster Pimonster
Pixray
PleaseCrawl PleaseCrawl
Pockey Probethenet
ProPowerBot PxBroker
Psbot
Pu_iN
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
RankingBot RankingBot
RankingBot2
Rankivabot
RankurBot RankurBot
Re-re RealDownload
Reaper Reaper
Recorder
RepoMonkey RepoMonkey
Ripper
RocketCrawler
Rogerbot Rogerbot
SEOkicks SEOkicks
SEOlyticsCrawler
SEOprofiler
SEOstats SEOstats
SISTRIX SISTRIX
SMTBot Scrapy
ScoutJet Screaming
ScrepyBot ScreenerBot
SearchmetricsBot Searchestate
Semrush Semrush
SeoSiteCheckup SeoSiteCheckup
Site Sucker
SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker Siteimprove
Sitebeam
Sitevigil
SlySearch SlySearch
SmartDownload SmartDownload
Snake Snake
SocialRankIOBot Sosospider
SpankBot
Spinn3r Spinn3r
Steeler Sqlworm
SurveyBot Sqworm
Suzuran Sucuri
Swiftbot SuperHTTP
T8Abot Surfbot
Teleport Teleport
TeleportPro Telesoft
Telesphorep
The Intraformant The Intraformant
TheNomad
Thumbor
TightTwatBot
Toata Toata
Toweyabot
Tracemyfile Tracemyfile
Trendictionbot Trendictionbot
Turnitin TurnitinBot
TwengaBot Typhoeus
URLy Warning
URLy.Warning URLy.Warning
UnisterBot
VB Project VB Project
VCI
Vagabondo Vagabondo
VidibleScraper Voltron
Virusdie
Voil
WASALive-Bot
WWW-Collector-E
WWW::Mechanize
Wallpapers Wallpapers
WallpapersHD WeSEE
Web Enhancer Web Collage
Web Fetch Web Fetch
Web Sucker WebBandit
WebAuto
WebCollage WebCollage
WebCopier WebEnhancer
WebFuck WebFetch
WebReaper WebGo IS
WebStripper WebPix
WebWhacker WebSucker
Webalta WebZIP
WebsiteExtractor WebsiteExtractor
WebsiteQuester
Webster Webster
Whacker
Whatweb
Who.is Bot Who.is Bot
Widow Widow
WiseGuys Robot WinHTTrack
Wonderbot Wonderbot
Wprecon Wprecon
Xaldon WebSpider
Zade
Zauba
Zeus
Zitebot
ZmEu
ZyBorg ZyBorg
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
cah.io.community
clark-crawler clark-crawler
coccocbot-web
com.plumanalytics
crawl.sogou.com crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j crawler4j
demandbase-bot
eCatch
evc-batch evc-batch
facebookscraper gopher
meanpathbot heritrix
internetVista monitor
lwp-request
magpie-crawler
mediawords mediawords
page scorer netEstate NE Crawler
pcBrowser oBot
plumanalytics seoscanners
probe-image-size
satoristudio.net
scan.lol
seostar seostar
serpstatbot sogouspider
sitechecker.pro
spyfu spyfu
trendiction.com sysscan
trendiction.de trendiction.de
voyagerx.com ubermetrics-technologies.com
webpros.com
x09Mozilla x09Mozilla
x22Mozilla
xpymep1.exe xpymep1.exe
zgrab zgrab

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404enemy
404checker Abonti
80legs Aboundexbot
ASPSeek AiHitBot
Acunetix
AfD-Verbotsverfahren
Alexibot Alexibot
AlphaBot AlphaBot
Asterias Apexoo
Attach AwarioSmartBot
AwarioRssBot Badass
BBBike
BDFetch
BackStreet
Backlink-Ceck
BacklinkCrawler
Barkrowler
BatchFTP
Bigfoot Bigfoot
Black Hole Bitacle
Blow
BlowFish BlowFish
Boardreader
Bolt Bolt
BotALot
Brandprotect Brandprotect
Brandwatch
Buck Buck
Buddy
CATExplorador CATExplorador
CODE87 CODE87
Calculon CazoodleBot
CheTeam Cegbfeieh
CheeseBot CheeseBot
CherryPicker ChinaClaw
Cloud mapping Claritybot
Cocolyzebot Cogentbot
Copier Copier
CopyRightCheck Copyscape
Cosmos CrazyWebCrawler
CrunchBot Crescent
Curious Custo
DTS Agent CyotekWebCopy
Deusu DSearch
Demon
Devil
Digincore Digincore
DigitalPebble DigitalPebble
Disco Dirbuster
Discobot Discobot
Discoverybot Discoverybot
Dispatch Dispatch
DnyzBot DittoSpyder
DomainCrawler DomainSigmaCrawler
Download Wonder Download Wonder
EMail Siphon Dragonfly
EMail Wolf
EasyDL EasyDL
Ebingbong
EirGrabber EirGrabber
Evil EroCrawler
Exabot
Express WebPictures Express WebPictures
ExtLinksBot Extractor
Ezooms ExtractorPro
Fimap Extreme Picture Finder
Firefox/7.0 EyeNetIE
FDM
FHscan
FemtosearchBot
Flunky
Foobot
FrontPage FrontPage
Fuzz
FyberSpider
Fyrebot Fyrebot
GT::WWW
Genieo
GermCrawler
GetRight
GetWeb GetWeb
Gigablast
Gigabot Gigabot
Go-Ahead-Got-It Go!Zilla
Gotit GoZilla
Grabber
Grafula Grafula
HEADMasterSEO
HTTrack HTTrack
Haansoft
Harvest
Havij
Heritrix Heritrix
Hloader
Humanlinks
IDBTE4M IDBTE4M
Iblog Iblog
Image Fetch
Image Sucker Image Sucker
IndeedBot
InfoNaviRobot InfoNaviRobot
InfoTekies InfoTekies
InterGET InterGET
InternetSeer Iria
Iskanie JamesBOT
Jbrofuzz
JetCar JetCar
Jetty JustView
Keyword Density Jyxobot
Kozmosbot Kozmosbot
LNSpiderguy LWP::Simple
Lanshanbot Lanshanbot
Larbin LeechFTP
Leap LexiBot
Lftp
LibWeb
LieBaoFast
Likse Likse
LinkScan
LinkWalker
Linkdexbot
LinkextractorPro LinkextractorPro
LinkpadBot LinkpadBot
LinqiaRSSBot Lipperhey Spider
Lipperhey
Litemage_walker Litemage_walker
Ltx71 Lmspider
MQQBrowser MIDown tool
Mag-Net MIIxpc
MJ12bot
MTRobot
Magnet Magnet
Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic-SEO
Majestic12 Majestic12
MarkMonitor MarkMonitor
MarkWatch Masscan
Mass Downloader Mb2345Browser
MeanPath Bot Mister PiX
Mediatoolkitbot Moblie Safari
MicroMessenger
Mojeek
Mojolicious Mojolicious
Mozlila Morfeus Fucking Scanner
Mr.4x3 Msrabot
NPbot NPbot
Nameprotect Name Intelligence
Navroad Needle
NearSite Net Vampire
NetAnts NetAnts
NetLyzer NetLyzer
Netcraft Netcraft
Nettrack
Netvibes
Nibbler
Nikto
Nimbostratus Nimbostratus
Ninja OnCrawl
Nuclei OpenLinkProfiler
Octopus
OpenVAS OpenVAS
Openvas Openvas
OutclicksBot OutclicksBot
OutfoxBot Pavuk
PHPCrawl
Page Analyzer
PageAnalyzer
PageThing.com
Panscient
PeoplePal PeoplePal
Picscout Picscout
Picsearch
Piepmatz Piepmatz
Pimonster PleaseCrawl
ProPowerBot Pockey
PyCurl Pu_iN
RankActive RankActive
RankActiveLinkBot
RankFlex RankFlex
RankingBot
RankingBot2 RankingBot2
Rankivabot
RankurBot RankurBot
Re-re RealDownload
Reaper
RepoMonkey
Ripper Ripper
SBIder SEOkicks-Robot
SEOkicks SEOprofiler
SEOstats
SISTRIX
SMTBot SMTBot
SalesIntelligent SalesIntelligent
Scanbot ScanAlert
Scrapy ScrepyBot
ScreenerBot
Searchestate Searchestate
SentiBot SearchmetricsBot
Semrush
SemrushBot
SeoSiteCheckup SeoSiteCheckup
Seomoz Siphon
Shodan Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
Sitebeam SiteExplorer
SiteLockSpider
SiteSnagger
Siteimprove Siteimprove
Sitevigil
SmartDownload
Snapbot
SocialRankIOBot SocialRankIOBot
Sociscraper Sogou web spider
Sosospider Spammen
SpankBot
Spbot Spbot
Sqlmap Spinn3r
Sqlworm
Sqworm
Steeler Steeler
Sucker Sucker
Sucuri SuperBot
Surfbot Suzuran
Szukacz Szukacz
Telesphorep T0PHackTeam
TeleportPro
Telesoft
Telesphoreo
The Intraformant The Intraformant
Thumbor Toata
Titan Toweyabot
Tracemyfile Tracemyfile
Trendiction Trendictionbot
Turnitin Turingos
TurnitinBot Upflow
TwengaBot
URLy Warning
URLy.Warning
UnisterBot
VB Project VB Project
VoidEYE VoidEYE
Voil Voil
Voltron WBSearchBot
WEBDAV
WISENutbot
WPScan WPScan
WWW-Collector-E WWW::Mechanize
Wallpapers
Web Collage Web Collage
Web Fetch Web Fetch
Web Sucker Web Pix
WebBandit WebCollage
WebEnhancer
WebFuck WebFuck
WebReaper WebLeacher
WebSauger WebSauger
WebWhacker
WebZIP WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Website Quester Whack
WebsiteExtractor Whatweb
WebsiteQuester Wotbox
Whacker
Wonderbot
Woobot
Zauba Zauba
Zermelo Zeus
Zitebot ZoominfoBot
ZoomBot
ZyBorg ZyBorg
archive.org_bot arquivo-web-crawler
arquivo.pt autoemailspider
cah.io.community backlink-check
check1.exe check1.exe
com.plumanalytics com.plumanalytics
crawler4j crawl.sogou.com
crawler.feedback
dataforseo.com dataforseo.com
demandbase-bot demandbase-bot
domainsproject.org domainsproject.org
eCatch facebookscraper
evc-batch
gopher gopher
heritrix ips-agent
internetVista monitor lwp-trivial
isitwp.com magpie-crawler
meanpathbot
muhstik-scan muhstik-scan
netEstate NE Crawler oBot
page scorer
pcBrowser pcBrowser
plumanalytics
probe-image-size
scalaj-http scalaj-http
seobility seobility
seoscanners
serpstatbot serpstatbot
spyfu sexsearcher
trendiction.com webpros.com
ubermetrics-technologies.com x09Mozilla
x22Mozilla x22Mozilla
zgrab xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
5elementov.ru asscenihotosidea.blogspot.co.za
checktext.org bharatdefencekavach.com
christianlouboutinshoes.xyz franch.info
depositfiles-porn.ga manualterap.roleforum.ru
div.as networkad.net
ge0ip.net newstraveller.ru
nisuturnetdgu.tk pornonik.com
notebook-pro.ru relax.ru
seemoreresultshu-a.akamaihd.net reyel1985.webnode.fr
yourporngay.com seoholding.com

924
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
12bet.com 0912701309f8ce.com
14b.info 0lovespells0.blogspot.com
1stat.ru 18ps.ru
2nt.ru 20pascals.nl
3weekdiet.com 24videos.tv
3xforum.ro 5000-cotydzien.com
4kepics.com 888.com
4pp13.com a96527gi.beget.tech
aa9bd78f328a6a41279d0fad0a88df1901.com
abcdeg.xyz abcdeg.xyz
academia-nsk.org absolute-s.ru
acads.net
adcash.com adcash.com
adloads.net adrenalin-stk.ru
adminshop.com adtech.us
adtech.de adultwebhosting.info
advokateg.xyz airmaxshoes-2016.com
afmuseum.com albuteroli.com
afora.ru alice.it
akama.com all4wap.ru
akita.kz alot.com
aksonural.ru alphahoverboards.com
aktivator-windows10.blogspot.com alveris.ru
albamargroup.com alvtank.se
alessandraleone.com anaksma.info
all-streaming-media.com antons-transporte.de
all4invest.info apessay.com
alloysteel.ru apibatbrowsecom-a.akamaihd.net
aludecor.info apiholdingmypage-a.akamaihd.net
anal-acrobats.hol.es apioutoboxnet-a.akamaihd.net
anatomiy.com apiwoodensealcom-a.akamaihd.net
aosexkontakte.net appleid-verification.com
appfastplay.com appsaurus.com
appfixing.space aramaicmedia.org
applicationg29.com arenda-avtoprokat-krasnodar.ru
appmsr.org arendovalka.xyz
approvedlocal.co.za articlesdirectoryme.info
arabgirls.adultgalls.com auto-zapchasti.org
arewater.com autobrennero.it
aruplighting.com autotrends.today
asia-forum.ru autovideobroadcast.com
asrvvv-a.akamaihd.net avkzarabotok.info
asseenontv.ru avtoarenda.by
atley.eu.pn avtolombard-krasnodar.ru
automatic-seo.com azarttoto.com
autorn.ru azbuka-mo.ru
aviva-limoux.com baditri.com
avtointeres.ru bankmib.ru
babyfactory.fr bankofthewext.com
bagsonsale.online beremenyashka.com
balook.com bestprofits-there.com
bausparen.bz.it
bear.gotcher.us
beget.tech
best-coupon-offer.com
best-way.men
bestmarriages.com
bestofpicture.com
bestwaystogetpaid.us
bidr.trellian.com
big-boards.info big-boards.info
bimatoprost-careprost.com bigcities.org
binomo.com bio.trade-jp.net
bjetjt.com bioscorp.ru
blogsfunky672.weebly.com bitcoins-live.ru
bluesman.nu biznesrealnost.ru
bmw.afora.ru blog.axant.it
boom-celebs.com bloke.com
brandov.ru bmw-ark.ru
breastaugmentation.co.za boc.kz
bronzeaid-a.akamaihd.net bolezniorganov.ru
buqayy0.livejournal.com bonzibuddi.com
buron.pw brainzpod.win
buyhoverboard.com break-the-chains.com
bvlgaribracelet.xyz bristolhostel.com
callawaygolfoutlet.online bristolhotel.com
brus.city
buketeg.xyz
buytizanidineonlinenoprescription.blogspot.com
c-gainsbourg.com
cablecar.us
callmd5map.com
candcstuccoandstone.com candcstuccoandstone.com
candlewooddentalcentre.com.au cardsharp1.ru
canoncdriverq3.pen.io cartierlovebraceletreplica.xyz
cartierlovestore.xyz cartierloveringreplica.xyz
cartiertrinity.xyz cartierreplica.pw
cenoval.ru cartujano-pre.de
centraletermice.us casinos4dummies.co.uk
cete.ru cbox.ws
cheap-pills-norx.com cdncash.net
cellfun.mobi
chastnoeporno.com
cheapjerseysbizwholesale.us cheapjerseysbizwholesale.us
check-this-out-now.online
checkm8.com
chimiver.info chimiver.info
cialischsrx.com chocolateslim-en-espana.com
cjs.com.ru chocolateslim-en-france.com
click2pawn.com chocoslim.pro
clickbank.net christianlouboutinsaleoutletonline.us
ciekawinki.pl
ciproandtizanidine.blogspot.com
circlesl.com
citetick.com
clash-clans.ru
classicakuhni.ru
claytransformations.info
cleansearch.net
clicksotrk.com
cocolyze.com cocolyze.com
collegerentals.com coin-hive.com
compliance-alex.top communisave.co.za
compliance-alexa.xyz compliance-brian.xyz
compliance-fred.xyz compliance-julia.top
compliance-olga.top concordexoticrentals.com
compliance-willy.top contentlook.co
comsysnet.com coolgamechannel.com
cookingmeat.ru coolingoods.com
cool-wedding.net copblock.org
coolgramgoods.com craftinsta.ru
copro.pw customcollegeessays.net
cornomase.win cymbaltaandtizanidine.blogspot.com
country-chic.ru cyse.tk
crazyprotocol.com dailyrank.net
cubs-jerseys.us damasarenai.info
customboxes4less.com daptravel.com
cvety24.by daydream-studio.ru
d0t.ru devochki.top
dailyfinancefix.com dieta-personalna.pl
darmebel.com.ua diffbot.com
davidovic.info diusyawiga.tk
ddpills.com dividendo.ru
dealitright.click dnepr-avtospar.com.ua
deda-moroza-zakaz.ru
dengi-pod-zalog-nedvizhimosti.ru
detskie-konstruktory.ru
dfiles.me
dirtpics.pw
djekxa.ru
dms-sw.ru
dneprsvet.com.ua
doeco.ru doeco.ru
dojki365.online dollartree.info
domaincdn.xyz domaineaneblanc.com
dorratex.tn
download-of-the-warez.blogspot.com download-of-the-warez.blogspot.com
downloadeer.net
downtuptv.gq
doxyporno.com
drillsaw.ru drillsaw.ru
drugspowerstore.com drpornogratisx.xxx
drunkenstepfather.com dscaas.website
drunkmoms.net dtnlyss.com
dwomlink.info dwomlink.info
eandsgallery.com e-commerce-seo.com
earl-brown.info e-kwiaciarz.pl
eleimgo.pw e2click.com
elektrischezigaretten1.blogse.nl earn-from-articles.com
elektrozigaretten1.postbit.com eas-seo.com
elmacho.xyz easyshoppermac.com
eloxal.ru ebonyporn.site
empathica.com edelstahlschornstein-123.de
eets.net
ege-russian.ru
elektrozigarette2.wordpressy.pl
enews.tech enews.tech
epngo.bz erotik-kostenlos.net
eshop4u.jp erotik0049.com
expolicenciaslatam.co erotikchat-24.com
extremal-blog.com espaceinventoristes.com
ezigarettenkaufen1.pagina.gr etm-consult.de
fanrto.com europages.com.ru
fashion-stickers.ru euroskat.ru
faz99.com extremez.net
ezigarettekaufen.myblog.de
f-loaded.de
falcon-images.blogspot.com
falllow.gq
fashion-mk.net
favornews.com
felizporno.com felizporno.com
fidelityfunding.com fenoyl.batcave.net
financeloan.us ferieboligkbh.dk
findacheaplawyers.com fickenbumsen.net
fireads.men filesclub.net
fix-website-errors.com filmi-onlain.info
flavors.me filunika.com.ru
flipper.top financehint.eu
foodcrafts.website findpics.pw
footbalive.org fishtauto.ru
fotosfotos.eu focalink.com
franklinfire.co footballfarrago.com
freemaintenancesysforpcandmac.top foto-weinberger.at
freenode.info free-fb-traffic.com
futbolkisales.ru free-video-tool.com
gambarkataku.co freejabs.com
gardene.ru freevpn.space
gearcraft.us freewlan.info
geckoandfly.com fuel-gas.com
generalporn.org funtoonez.com
germetiki.com.ua fz139.ttk.ru
getpopunder.com g.starmoe.xyz
gettpromos.com g7m.pl
ghazel.ru gambarkatabaru.com
gkvector.ru games.kolossale.ru
glastecfilms.com.my gaming-journal.com
global-ics.co.za gamingspark.com
go2mike.ru gdzkurokam.ru
gobongo.info ge0ip.com
goodhumor24.com get-your-social-buttons.info
googst2.ru gilbertbanda.net
goosefishpost.bid gloverid.site
gotorussia.com goforexvps.com
gourcy.altervista.org goldpanningtools.com
greamimgo.pw green-tea.tv
greatdealshop.com greenidesign.co
grizzlysgrill.com gsbs.com.ua
gsasearchengineranker.top guidefs.ru
gsasearchengineranker.xyz gwhwpxbw.bloger.index.hr
gsmtlf.ru handicapbathtubarea.com
gwebtools.com hannasolution.ru
hanink.biz.ly havepussy.com
hazardky.net hawaiisurf.com
help.tpu.ru healgastro.com
herehloadibs.cf helpmymacfaster.trade
hetmanship.xyz
hildinghr.se
historichometeam.com historichometeam.com
hit-men.men homedo.fabpage.com
hornymatches.com hotgirlhdwallpaper.com
horoshieokna.com houdom.net
hotelcrocenzi.sm houseofgaga.ru
hotloans.ru iboss.com
housemilan.ru idealtits.net
hundejo.com igrovyeavtomaty777.ru
i4track.net iklysha.ml
igithab.com ilo134uloh.com
infodocsportal.com imgarcade.com
inome.com.ua imicrovision.com
interior-stickers.ru inbabes.sexushost.com
investsuccess.org incest-ru.com
investyb.com inclk.com
iqoption.pro infoupdate.org
irkutsk.online-podarki.com intellektmedia.at
istizanidineacontrolledsubstance.blogspot.com internet-apteka.ru
janavibekken.no invest-pamm.ru
javrip.net iron-age.info
joessmogtestonly.com istizanidineanarcoticdrug.blogspot.com
itronics.ca
iusstf.org
ivanstroi.ru
izismile.com
javspace.net
jerseychinabizwholesale.com
jerseywholesalechinabiz.com
johannesburgsingles.co.za
johnnyhaley.top johnnyhaley.top
juliadiets.com justprofit.xyz
kalb.ru kaliningrad.zrus.org
kambasoft.com
kamorel.com kamorel.com
keywordglobal.co.za kashubadesign.ru
kiprinform.com kenaba.su
kirov.zrus.org klosetkitten.com
komputernaya-pomosh-moscow.ru kolotiloff.ru
konica.kz krynica.info
kostenlos-sexvideos.com kwzf.net
krafte.ru lacasamorett.com
krassh.ru lafriore.ru
ks1234.com law-check-eight.xyz
kvartiry-remont.ucoz.ru letmacwork.world
la-fa.ru lexaprogeneric.link
labelwater.se likesdesign.com
ladsup.com linkbuddies.com
lafourmiliaire.com linkredirect.biz
landinez.co
landreferat.ru
lapitec.eu
lapolis.it
larchik.net
ledpolice.ru
letolove.ru
letslowbefast.today
likesuccess.com
linkrr.com
liquimondo.com liquimondo.com
live-cam6.info listiseltemournlan.gq
lmrauction.com littlesunraiser.com
lonely-mature.com liveinternet.ru
m1media.net localmatchbook.com
magazin-pics.ru locatellicorretor.com.br
magda-gadalka.ru lodki-pvh.dp.ua
managerpak204.weebly.com lunamedia.co
marketingtechniques.info macdamaged.space
martlinker.com madot.onlinewebshop.net
maslenka.kz mail.allnews24.in
master-muznachas.ru mailemedicinals.com
mcadamssupplyco.com manimpotence.com
medicines-choice.com manipulyator-peterburg.ru
medkritika.ru manualterap.roleforum.ru
mega-bony2017.pl marinetraffic.com
member-quiz.com marketingvici.com
metarip.ru marketland.ml
metascephe.com mature.free-websites.com
michellblog.online max-p.men
mini-modus.ru mearns-tractors.co.uk
mirzonru.net med-zdorovie.com.ua
mmgq.ru meddesk.ru
mediafresh.online
medtherapy.ru
meduza-consult.ru
megaindex.ru
megavolt.net.ua
merryhouse.co.uk
mesbuta.info
mhi-systems.ru
miniature.io
mobicover.com.ua
moesonce.com moesonce.com
mokrayakiska.com most-kerch.org
moneyzzz.ru muznachas-service.ru
moviemail-online.co.uk mybinaryoptionsrobot.com
moyaterapiya.ru myindospace.com
msk-diplomat.com myprintscreen.com
mttwtrack.com mzdish.site
musicpro.monster naturalpharm.com.ua
mvpicton.co.uk neobux-bg.info
myblogregistercm.tk net-profits.xyz
mycaf.it networkad.net
mydoctorok.ru newsperuse.com
myfreetutorials.com nfljerseyscheapchinabiz.com
mypornfree.ru
myyour.eu
naj-filmy24.pl
nash-krym.info
naughtyconnect.com
nevansk.ru
niki-mlt.ru
nikitabuch.com
ningessaybe.me ningessaybe.me
nodding-passion.tk nippon-bearings.ru
nisuturnetdgu.tk
nlfjjunb5.ru
nodup.ru
nomuos.it
nootrino.com
nordstar.pro nordstar.pro
nudo.ca nullrefer.com
obsessionphrases.com o00.in
offer.wpsecurity.website one-gear.com
oil-td.ru online-x.ru
onclkads.com online7777.com
online247.ml onlinemegax.com
onlyporno.ru onlyforemont.ru
ooomeru.ru paperwritingservice17.blogspot.ru
oops-cinema.ru parajumpersoutlet.online
openstat.com patterntrader-en.com
optibuymac.com pattersonsweb.com
ovirus.ru pekori.to
paleohub.info penisvergrotendepillennl.ovh
panchro.xyz pepperstyle.ru
pandarastore.top physfunc.ru
panouri-solare-acoperis.com pinstake.com
par-fallen.ga pinwallpaper.top
paramountmarble.co.uk piuminiita.com
parfusale.se pl.id-forex.com
partnerads.men plusnetwork.com
partnersafe.men popup-hgd.xyz
partybunny.ru porno-dojki.net
pawli.eu porno-transsexuals.ru
perfection-pleasure.ru pornoklad.ru
petitions.whitehouse.gov pornolook.net
picmoonco.pw pornosok.ru
pics-group.com pornzone.tv
pills24h.com
pipki.r.acdnpro.com
pistonclasico.com
pix-hd.com
pix24x7.com
placid-rounded-coast.glitch.me
plastgran.com
play-movie.pl
polcin.de
popup-fdm.xyz
pornhub-forum.ga
pornohub.me
pornowarp.info
portal-eu.ru portal-eu.ru
pozdrawleniya.ru portside.cc
pretty-mart.com pr0fit-b0x.com
presleycollectibles.com
preventheadacheguide.info
priceg.com
printdirectforless.com printdirectforless.com
privetsochi.ru prize44.com
prlog.ru professionalwritingservices15.blogspot.ru
pussyspace.net purplesphere.in
pwwysydh.com puzzleweb.ru
rebrand.ly quick-seeker.com
regionshop.biz r-e-f-e-r-e-r.com
registratciya-v-moskve.ru redhotfreebies.co.uk
registrydomainservices.com reklama1.ru
remmling.de rent2spb.ru
remybutler.fr ringporno.com
restorator-msk.ru royalads.net
revistaindustria.com
rff-cfal.info
robertefuller.com
royalcar-ufa.ru royalcar-ufa.ru
roznica.com.ua royalvegascasino.com
running-line.ru rusprostitute.com
russkoe-zdorovie.ru rutor.vip
sadaholding.com s1z.ru
samolet.fr safe-app.net
savefrom.com sale-japan.com
scanner-alex.top sanatorrii.ru
scanner-julianna.top saugatuck.com
scanner-olga.top sayyoethe.blogspot.co.za
searchaddis.com sbwealthsolutions.ca
searchengineranker.email scanner-jack.top
searchimage.co se.bnt-team.com
searchmywindow-a.akamaihd.net seasaltwithfood.com
sei80.com secret.xn--oogle-wmc.com
selectads.men seo-tools-optimizing.com
semalt.com seoanalyses.com
semxiu.com shareyards.com
seobility.net shillyourcoins.com
sex-dating.co shop.xz618.com
sex-foto.pw shoplvlv.us
sex.hotblog.top shtaketniki.kz
sexpornotales.net signal03.ru
sextracker.be siha.de
sexuria.net sildenafil-tadalafil.info
sexvporno.ru silverage.ru
sherlock.se simpoed.ufop.br
shopcheermakeup.info
simon3.ru
sims-sims.ru
site-speed-checker.site
site5.com site5.com
skylta.com siteimprove.com
sladkoevideo.com sitiz.club
slavkokacunko.de slimcdn.com
slayerlife.com smart-scripts.com
slowmac.tech social-s-hhh.xyz
slutloadlive.com socialmonkee.com
smallseotools.com
snjatie-geroinovoy-lomki.ru
sockshare.net
solnplast.ru solnplast.ru
songoo.wz.cz sophang8.com
space2019.top spasswelt.net
spaceshipad.com spb.afora.ru
spbchampionat.ru spb.ru
sport-video-obzor.ru specialfinanceoffers.com
srdrvp.com sptslmtrafms.com
stockquotes.wooeb.com startufa.ru
stop-zavisimost.com stpicks.com
strana-krasoty.ru strfls.com
stroyhelp-dv.ru stroymonolit.su
subj.ukr-lit.com
sukirgenk.dvrlists.com
supermesta.ru supermesta.ru
svarkagid.com surcentro.com
t-bygg.com szamponrevita.pl
tacbibirfa.tk tanieaukcje.com.pl
takethatad.com tech4master.com
tattooha.com teenporn18.net
tengohydar.tk testbotprocessor44.com
terrafootwear.us thepantonpractice.co.uk
thedownloadfreeonlinegames.blogspot.com thexart.club
theendivechronicles.com timer4web.com
thewebsitetemplate.info tizanidine4mg.blogspot.com
thewomenlife.com tizanidine4mgtablets.blogspot.com
titan-cloud.life tizanidinehcl2mgsideeffects.blogspot.com
tivolibasket.it tizanidinehydrochloride2mg.blogspot.com
tizanidineandgabapentin.blogspot.com tizanidinevscyclobenzaprine.blogspot.com
tizanidinecost.blogspot.com
tizanidinehcl4mgtabinfo.blogspot.com
tizanidineorflexeril.blogspot.com
tizanidinewithdrawal.blogspot.com tizanidinewithdrawal.blogspot.com
tk-assortiment.ru tn811.us
todohr.com todohr.com
toonfamilies.net
top10-online-games.com top10-online-games.com
top10-way.com
top250movies.ru top250movies.ru
topcar-krasnodar.ru torontoplumbinggroup.com
topquality.cf torrents-tracker.com
traffic2cash.org tourismvictoria.com
traffpartners.com toyota.7zap.com
trichizobswiv.agddns.net tpu.ru
truck-rece.life trafficmania.com
trustl.life trainoffend.ml
ttsq.fr trk-4.net
tuckermktg.com
tula.mdverey.ru tula.mdverey.ru
turbabitload.weebly.com tupper-shop.ru
twelvevisionspartyofcolorado.com turizm.bz
twinderbella.com twinderbella.com
twu.com.ua uhdtv.website
uasb.ru ukrobstep.com
uchil.net ukrtvir.com.ua
unlimitdocs.net underthesite.com
upstore.me uroffer.link
url-img.link valuado.com
urldelivery.com vapeface.club
uroki.net vapsy.com
userequip.com ventelnos.com
valsalud.com veopornogratis.xxx
valueclick.com
veselokloun.ru veselokloun.ru
videosbox.ru vesnatehno.ru
viel.su videochat.ph
vladimir.xrus.org videochat.tv.br
voloomoney.com vip2ch.com
vsesubwaysurfers.com vipromoffers.com
vvpg.ru vitalads.net
walkme.com vodkoved.ru
wallpapersdesk.info vostoktrade.info
warezaccess.com vote-up.ru
warningwar.ru vpnmouse.com
webjam.com vrotike.ru
vut.com.ru
vvon.co.uk
vzlom-na-zakaz.com
wallpaperstock.net
wdfdocando.com
web.cvut.cz
webextract.profound.net
webix.biz
weddingdresses.xyz
wesharepics.info wesharepics.info
wetgames.ru
whatsappbot.flyland.ru
whatzmyip.net
whos.amung.us
windowssearch-exp.com windowssearch-exp.com
wineitudes.wordpress.com woman-h.ru
winwotgold.pl works.if.ua
workle.website woweb.com.ua
wygraj-teraz.com wttavern.com
x-diesel.com xaylapdiendanang.com
xerox-douglas.cf xlolitka.com
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xmronta.com
xn----7sbho2agebbhlivy.xn--p1ai
xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--80ak6aa92e.com xn--80aanaardaperhcem4a6i.com
xsfetish.org xn--80abgj3a5acid6ghs.top
xwatt.ru xn--b1ag5cfn.xn--p1ai
xn--h1aakne2ba.xn--p1ai
xuki.us
xvideosporn.biz
xxart.ru
xxxguitars.com xxxguitars.com
yellocloud.be yadro.ru
yatrk.xyz
yes-com.com yes-com.com
yoshkarola.zrus.org yginekologa.com
youdao.com yougame.biz
yoursite.com yougetsignal.com
yubikk.info youporn-forum.ga
yugo-star.ru yourporngay.com
zajm-pod-zalog-nedvizhimosti.ru yourtemplatefinder.com
zarenica.net zaapplesales.blogspot.com
zdesformula.ru zakonobosago.ru
zeroredirect12.com zlatnajesen.com
znakomstva-piter78.ru zog.link
zolotoy-lis.ru zojirushi-products.ru
zootoplist.com zoominfo.com
zryydi.com zscaler.net
zs2vm.top zverokruh-shop.cz
zskdla.site zvooq.eu

474
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1000n1.ru 2pxg8bcf.top
3w24.com 4k-player.pl
7fon.ru 7a2rnuey1tw9ar.ru
aac500b7a15b2646968f6bd8c6305869d7.com 883zy.com
albamargroup.com acads.net
allforminecraft.ru acmebtn.ml
allsilver925.co.il addbin.men
anaseracresar.tk adnotbad.com
anlimebel.kiev.ua adtiger.tk
arendatora.ru airlandsea.info
artavenuegardenstudios.com algerianembassy.co.in
artdeko.info allwidewallpapers.com
autonew.biz anaksma.info
avitocars.ru andpolice.com
avon-novosib.ru animenime.ru
bablonow.ru apartamentwroclaw.eu
best-coupon-offer.com arabseyes.com
bigcareer.info arewater.com
bitporno.sx asmxsatadriverin.aircus.com
bizcheapjerseyswholesalechina.com atofilms.com
bizfly.info autoseo-traffic.com
blacktwhite.com azadnegar.com
bobba.dzaba.com azazaporn.com
bonkers.name babieca.com
brandbucket.com bambi.ck.ua
brus-vsem.ru barnfurnituremart.com
burger-tycoon.com bdsmgalls.net
buydissertation.net bestadbid.com
buyparajumpers.online betonka.pro
buypillsorderonline.com bimatoprost-careprost.com.ua
blockworld.ru
bmwhighperformers.com
bolezniorganov.ru
bracketsmackdown.com
brainboosting.club
builtwith.com
buqayy0.livejournal.com
buron.pw
buscarfoto.com
businesn.men
callawaygolfoutlet.online callawaygolfoutlet.online
canrole.com carson.getenjoyment.net
capsderfudd.tk chocolateslim-original.com
carfax.com.ua cialis-samples.com
carrauterie.be click2pawn.com
cartierlove2u.com clickso.com
caveavins.fr compliance-checker.info
citysecurity.nu compliance-fred.top
cjmc.info confib.ifmo.ru
classicakuhni.ru contentsexpress.com
classiquebijoux.ru cookie-law-enforcement-aa.xyz
coindirect.io courtshipgift.com
com-onlinesupport.host crazyboost.pro
compliance-alexa.xyz crd.clan.su
contentlook.co cryptoswap.biz
copblock.org data.vtc.pw
counter.bloke.com dellalimov.com
cubook.supernew.org deshevo-nedorogo.ru
custom-electric-guitar.com
demian.kz
designdevise.com designdevise.com
detskie-zabavi.ru discovertreasure-a.akamaihd.net
detsky-nabytek.info donvito.unas.cz
diarioaconcagua.com ednorxmed.com
dikqlyremy.info ekaterinburg.xrus.org
dolohen.com euronis-free.com
domainsphoto.com extlabs.io
dorratex.tn ez8motelseaworldsandiego.com
dumuelave.xyz ezigarettenshop2.postbit.com
dvr.biz.ua filmi-v.online
dwomlink.info fireads.men
earl-brown.info floridamobilebillboards.com
ecookna.com.ua forexgb.ru
eldiariodeguadalajara.com forms-mtm.ru
elektrischezigarettekaufen2.cowblog.fr free-share-buttons.com
encodable.com freelinkbuilding.website.tk
englate.com freshwallpapers.info
etotupo.ru frighteningremain.cf
europages.com.ru frvo.alptandem.ru
everypony.ru fungamelands.com
excitacao.com fz139.ttk.ru
extener.org gael-s.ru
familyholiday.ml gaygalls.net
fashionavenuegame.com gdzkurokam.ru
finteks.ru gearsadspromo.club
fortunejack.com gemara.com
forum.tvmir.org getpopunder.com
fuck-paid-share-buttons.xyz ghernnqr.skyrock.com
fuckbuddybestgilf.info gheus.altervista.org
fuckingawesome.com gigablast.com
funponsel.com globatur.ru
fx-brokers-review.com hcate.com
fym.com.ru hentaiheroes.com
gagrasector.ru hermesbirkinhandbagoutlets.com
gg.zzyjxs.com hidefiles.org
glcomputers.ru highspeed5.net
globalscam.ga host-protection.com
gne8.com hostingclub.lk
goldandcard.ru innodgfdriverhm.aircus.com
googglet.com intermesh.net
googst2.ru internet-apteka.ru
graphid.com keywordhut.com
greatdealshop.com koopilka.com
herokuapp.com kovesszucs.atw.hu
hit-men.men krasnodar.ru
homegardenlova.com laparfumotec.com
housekuba.org law-check-two.xyz
hplaserjetpdriver8y.pen.io
ico.re
ilovevitaly.com
imgata.com
iqupdatetmz.win
islandminingsupply.wordpress.com
isotoner.com
juliaworld.net
jwcialislrt.com
kakablog.net
karachev-city.ru
kemerovo.zrus.org
klin3952.ru
kochanelli.com
kotaku.com
krafte.ru
krasivye-devushki.net
kuzinsp.ru
lacave.ntic.fr
lampokrat.ws
lanadelreyfans.us
landscaping.center
laptop-4-less.com
larutti.ru
laserpen.club
libertybilisim.com
life-instyle.com life-instyle.com
lion.bolegapakistan.com liveu.infoteka.hu
lkbennettoutlet.online longadventure.com
lollypopgaming.com m3gadownload.pl
love-baby.cz macfix.life
macrotek.ru mafa-free.com
magicalfind-a.akamaihd.net
mahnwachen-helfen.info
master-muznachas.ru
mcar.in.ua mcar.in.ua
medictube.ru meccadumps.net
medicines-choice.com
medicineseasybuy.com
meetlocalchicks.com meetlocalchicks.com
megavolt.net.ua menstennisforums.com
misterjtbarbers.com misterjtbarbers.com
mixx.com mlvc4zzw.space
mobicover.com.ua mobot.site
monclerjacketsoutlet.pw mobsfun.net
monclervests.xyz monclerboots.xyz
moscow.online-podarki.com morefastermac.trade
mrlmedia.net muschisexbilder.com
mrmoneymustache.com myanyone.net
multgo.ru
mydearest.co
mydeathspace.com
myfreemp3.eu myfreemp3.eu
mygameplus.ru myxdate.info
nerudlogistik.ru myyour.eu
npoet.ru nakozhe.com
nudo.ca narkologiya-peterburg.ru
okuos.com nextlnk12.com
online-sbank.ru ngps1.ru
nrv.co.za
oda.as
of-ireland.info
offf.info
onescreen.cc
oneshotdate.com
opinionreelle.com opinionreelle.com
paighambot.com outclicks.net
painting-planet.com ownshop.cf
partner-high.men page2rss.com
pelfind.me panchro.xyz
photo.houseofgaga.ru partnerline.men
picturesify.com peugeot-club.org
piulatte.cz pinapchik.com
pornhub-ru.com pistonclasico.com
porno-chaman.info pix-hd.com
porno-transsexuals.ru plugingeorgia.com
pornohub.me
pornosok.ru
pornowarp.info pornowarp.info
pourvous.info pozdrawleniya.ru
privatamateure.com printie.com
privacylocationforloc.com
privhosting.com privhosting.com
progress-upakovka.ru pro-poly.ru
prombudpostach.com.ua pulse33.ru
promoheads.com purchasepillsnorx.com
pronorm.fr pwwysydh.com
propranolol40mg.blogspot.com qexyfu.bugs3.com
pshare.biz qor360.com
qcstrtvt.bloger.index.hr razyboard.com
quickloanbank.com rcb101.ru
rarbg.to registry-cleaner.net
rating-bestcasino.com renecaovillasale.online
razamicroelectronics.com renewablewealth.com
realmonte.net rimedia.org
realnye-otzyvy.info rixpix.ru
recipedays.com rospromtest.ru
registrysweeper.com rubbed.us
restorator-msk.ru sbtdesign.co.uk
reversing.cc scanner-george.top
risparmiocasa.bz.it scanner-nelson.top
rubanners.com search-error.com
ryetaw.com searchinquire.com
sadaholding.com seksotur.ru
sale-japan.com servisural.ru
savememoney.co.za sevenstars7.com
scanner-brian.top sex-dating.co
seansonline24.pl sexad.net
secretscook.ru sexkrasivo.net
shmetall.com.ua sexs-foto.top
shtaketniki.kz sfd-chess.ru
sierraapps.com shakhtar-doneck.ru
simple-share-buttons.com shtora66.ru
sisi-go.ru shymkent.xkaz.org
skanninge.se smartshoppymac.com
skuteczna-dieta.co.pl
socas.pluto.ro socas.pluto.ro
social-vestnik.ru social-buttons.com
sribno.net soda.media
stmassage.ru souvenirua.com
stockquotes.wooeb.com star61.de
strana-krasoty.ru start.myplaycity.com
stroiminsk.org steebook.com
studentguide.ru steelmaster.lv
sudexpert66.ru sudexpert66.ru
swagbucks.com supers.com.ua
sweepstakes.rewardit.com surintech.ac.th
swiped.su szamponrevita.pl
technopellet.gr tattooreligion.ru
teesdaleflyballclub.co.uk teenforporn.com
thefds.net telegraf.by
tivolibasket.it theallgirlarcade.com
tizanidineformigraines.blogspot.com timecrimea.ru
tizanidinehcl4mgtablet.blogspot.com tizanidine4mg.blogspot.com
trafficfactory.biz tizanidinenarcotic.blogspot.com
trafficjunky.com toon-families.com
try-rx.com top-karkas.ru
tubeoffline.com track112.site
u-cheats.ru trafficreceiver.club
ultimateclassicrock.com trk-4.net
ultimatesetnewfreeallsoftupgradesystems.pw trucri.me
uni.me turist-strani.ru
unitygame3d.com turvgori.ru
v-doc.co uhdtv.website
v24s.net ukrtextbook.com
vapeface.club ukrup.com
vchulkax.com unblocksit.es
veopornogratis.xxx uvozdeckych.info
videochat.ph velpanex.ru
webkeyit.com vertaform.com
webscouter.net via-energy-acquistare.com
wejdz-tu.pl virtuagirl.com
whiteproduct.com vizag.kharkov.ua
wholinkstome.com vkontarkte.com
workle.website weblibrary.win
x-lime.com webmonetizer.net
xaylapdiendanang.com website-stealer.nufaq.com
xingzi-vision.com wedding-salon.net
xn--80abgj3a5acid6ghs.top wellcome2slovenia.ru
xn--80ahdheogk5l.xn--p1ai www.wohnkabinen-shop.de
xedserver.com
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn--hxazdsfy.blogspot.com
xzlive.com
yaaknaa.info
yellowproxy.net
yellowstonevisitortours.com
yoshkarola.zrus.org
youbloodyripper.com youbloodyripper.com
youradexchange.com zaobao.com.sg
yourads.website zeg-distribution.com
youtoner.it zhuravlev.info
zacreditom.ru zigzog.ru
zarepta.com zoomovies.org
zeroredirect6.com zynax.ua

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.137 0; 10.1.0.198 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2563 ### Version: V4.2021.09.2564
### Updated: Wed Sep 1 10:03:11 UTC 2021 ### Updated: Wed Sep 1 22:01:26 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2563 ### Version: V4.2021.09.2564
### Updated: Wed Sep 1 10:03:11 UTC 2021 ### Updated: Wed Sep 1 22:01:26 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##