vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 06:15:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2732 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-23 10:09:09 +00:00
parent 8e96d7429a
commit ff1a4250ed
46 changed files with 80676 additions and 80676 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2731 #### Version: V4.2021.11.2732
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2731 #### Version: V4.2021.11.2732
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2731 #### Version: V4.2021.11.2732
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9950
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2731 #### Version: V4.2021.11.2732
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9954
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9954
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

9954
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9954
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.0.240 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.0.240 0;
127.0.0.1 0; 127.0.0.1 0;

9954
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.0.240 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.0.240 0;
127.0.0.1 0; 127.0.0.1 0;

9954
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.0.240 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.0.240 0;
127.0.0.1 0; 127.0.0.1 0;

9954
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2731 2732

9954
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

9954
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.106.252.91 0; 20.109.175.12 0;

9954
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.0.240 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.0.240 0;
127.0.0.1 0; 127.0.0.1 0;

9954
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.0.240 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.0.240 0;
127.0.0.1 0; 127.0.0.1 0;

9954
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.0.240 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.0.240 0;
127.0.0.1 0; 127.0.0.1 0;

9954
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.106.252.91 0; 20.109.175.12 0;

9954
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.106.252.91 0; 20.109.175.12 0;

9954
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.0.240 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
360Spider DnyzBot
Bigfoot ExtractorPro
GrapeshotCrawler FHscan
LibWeb Gotit
Mr.4x3 Netcraft
Sociscraper Trendictionbot
WWW-Mechanize Web Fetch
WWW::Mechanize lwp-trivial
WebEnhancer ripz
WebSucker seostar

296
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker
404enemy
ADmantX ADmantX
AIBOT Abonti
Aboundex
Aboundexbot Aboundexbot
AiHitBot Acunetix
AfD-Verbotsverfahren
AllSubmitter
Alligator Alligator
AlphaBot AlphaBot
Anarchie
Anarchy
Anarchy99 Anarchy99
Apexoo
Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot BBBike
BDCbot
BackStreet
Backlink-Ceck Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass Badass
BatchFTP Barkrowler
Battleztar Bazinga BetaBot
Bigfoot
Bitacle Bitacle
Blackboard Blow
BlowFish BlowFish
Buck Boardreader
Buddy Brandprotect
BuiltBotTough BuiltBotTough
BuiltWith
Bullseye Bullseye
CazoodleBot CATExplorador
CCBot
CODE87
Calculon
Cegbfeieh
CensysInspect CensysInspect
CheTeam CheTeam
CherryPicker CherryPicker
ChinaClaw
Chlooe
Claritybot Claritybot
Cocolyzebot Cliqzbot
Collector Cogentbot
Copier
CopyRightCheck
Cosmos Cosmos
Craftbot Crescent
Crawling at Home Project Curious
CrazyWebCrawler
CyotekWebCopy
DIIbot
DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Demon Devil
DigitalPebble Dirbuster
Disco
Discobot Discobot
Dispatch DittoSpyder
DnBCrawler-Analytics DomCopBot
DnyzBot
DomainAppender DomainAppender
DomainCrawler
DomainStatsBot DomainStatsBot
Dotbot Domains Project
Dragonfly
ECCP/1.0 ECCP/1.0
EMail Siphon EasyDL
Ebingbong Ebingbong
EroCrawler Ecxi
Exabot Evil
Extractor
ExtractorPro
FDM FDM
FHscan FHscan
Firefox/7.0 FemtosearchBot
Fimap
Flunky Flunky
Foobot Foobot
Freeuploader
Fyrebot Fyrebot
GetWeb G-i-g-a-b-o-t
Getintent
Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla
Gotit
Grabber Grabber
Grafula HTTrack
GridBot
HEADMasterSEO
HMView
HaosouSpider
Harvest Harvest
HybridBot Heritrix
IDBot IRLbot
Iblog Iblog
Id-search Id-search
IlseBot IlseBot
Image Fetch Image Fetch
Indy Library Image Sucker
InternetSeer Intelliseek
Internet Ninja
Iria Iria
Iskanie IstellaBot
JamesBOT
JikeSpider JikeSpider
Joomla Jorgee
Kinza JustView
LNSpiderguy
Lanshanbot Lanshanbot
LeechGet Larbin
Lftp Lftp
Lightspeedsystems LibWeb
Likse LieBaoFast
LinkWalker LinkWalker
Linkbot Linkdexbot
LinkextractorPro
LinkpadBot LinkpadBot
LinksManager LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Litemage_walker Lipperhey
Lmspider
Ltx71 Ltx71
MFC_Tear_Sample MIIxpc
MQQBrowser MQQBrowser
MTRobot MSIECrawler
Mag-Net
Magnet Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic12 Majestic SEO
Mass Downloader MarkWatch
MauiBot MauiBot
Mb2345Browser
Meanpathbot Meanpathbot
Mediatoolkitbot Mediatoolkitbot
Metauri
MicroMessenger MicroMessenger
Microsoft URL Control Moblie Safari
Morfeus Fucking Scanner
Mozlila
Mr.4x3 Mr.4x3
Msrabot Musobot
NICErsPRO
NPbot NPbot
Name Intelligence Name Intelligence
NearSite Navroad
NetMechanic Nessus
NetSpider Net Vampire
Netcraft NetZIP
Nettrack NextGenSearchBot
Nibbler Nuclei
Nikto Nutch
Nimbostratus Offline Navigator
Ninja
Offline Explorer
OnCrawl
Openfind Openfind
OrangeSpider OrangeSpider
PHPCrawl OutfoxBot
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageAnalyzer PageAnalyzer
PageGrabber PageGrabber
Pandalytics
Papa Foto Papa Foto
PeoplePal PeoplePal
Pi-Monster Petalbot
Piepmatz Picscout
PictureFinder
Pimonster
Pixray Pixray
PleaseCrawl
ProPowerBot ProPowerBot
Probethenet ProWebWalker
Psbot PxBroker
QueryN Metasearch PyCurl
Quick-Crawler
Rankivabot
ReGet ReGet
RealDownload Recorder
RedesScrapy
RepoMonkey
Ripper Ripper
RocketCrawler RocketCrawler
Rogerbot Rogerbot
SEOkicks SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
Screaming SMTBot
ScrepyBot SalesIntelligent
Searchestate Scanbot
SiteCheckerBotCrawler ScreenerBot
SiteSucker Semrush
Sitevigil SentiBot
SlySearch Seomoz
Siphon
Site Sucker
Sitebeam
SmartDownload SmartDownload
Snake
Sogou web spider
Sosospider
SpaceBison
Spammen
Spbot
Spinn3r Spinn3r
Sqlmap
Steeler Steeler
Stripper Stripper
SuperHTTP Sucker
Sucuri
Surfbot
Suzuran Suzuran
Szukacz Szukacz
Telesoft T0PHackTeam
Telesphorep T8Abot
TheNomad Teleport
Trendiction Telesphoreo
Trendictionbot The Intraformant
Turnitin Thumbor
TightTwatBot
Titan
Toweyabot
TurnitinBot TurnitinBot
TwengaBot
Twice
Typhoeus Typhoeus
UnisterBot VB Project
Upflow VCI
Vacuum Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper VidibleScraper
Virusdie VoidEYE
Voil
Voltron Voltron
WASALive-Bot
WEBDAV WEBDAV
WISENutbot WISENutbot
WWW-Collector-E WallpapersHD
Wallpapers/3.0
Web Collage Web Collage
Web Fuck Web Fuck
WebEnhancer Web Sauger
WebAuto
WebCollage
WebFetch
WebPix WebPix
WebReaper WebReaper
WebSauger WebSauger
WebSucker Webshag
WebWhacker
WebZIP
WebsiteExtractor
Whack Whack
Whacker Widow
WinHTTrack
Wonderbot Wonderbot
Woobot
Wotbox Wotbox
Wprecon adscanner
Xaldon WebSpider arquivo-web-crawler
Zeus
Zitebot
ZmEu
ZoomBot
autoemailspider autoemailspider
clark-crawler
coccocbot-web
crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j
dataforseo.com
domainsproject.org
eCatch eCatch
facebookscraper facebookscraper
instabid isitwp.com
internetVista monitor
lwp-trivial
magpie-crawler
meanpathbot
mediawords mediawords
oBot
page scorer
pcBrowser pcBrowser
polaris version ripz
probe-image-size
s1z.ru s1z.ru
scalaj-http scan.lol
seobility
seocompany.store
seostar seostar
serpstatbot serpstatbot
sexsearcher
sitechecker.pro
sogouspider
sp_auditbot sp_auditbot
webprosbot spyfu
x09Mozilla sysscan
x22Mozilla tAkeOut
trendiction.com
trendiction.de
ubermetrics-technologies.com
webmeup-crawler
zauba.io zauba.io
zgrab

300
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com AIBOT
404enemy ASPSeek
80legs Aboundex
ADmantX Acunetix
Abonti Aipbot
Aboundexbot Anarchie
AfD-Verbotsverfahren Anarchy
AhrefsBot Aspiegel
AllSubmitter BDCbot
AlphaBot BLEXBot
Anarchy99 BacklinkCrawler
Apexoo Bigfoot
Asterias BlackWidow
AwarioRssBot
BDFetch
BackDoorBot
BackWeb
Backlink-Ceck
Bandit
BatchFTP
Blackboard
Blow Blow
Boardreader Brandwatch
Bolt
BotALot
Buck Buck
Buddy Buddy
BuiltWith
BuzzSumo
CCBot
CODE87 CODE87
Calculon Calculon
Cegbfeieh
CheTeam CheTeam
CheeseBot CheeseBot
Claritybot CherryPicker
Cloud mapping Cloud mapping
CopyRightCheck Cocolyzebot
Copyscape Collector
Craftbot
CrunchBot
Curious Curious
CyotekWebCopy DBLBot
DIIbot DIIbot
DSearch
Devil
Digincore Digincore
DigitalPebble Dispatch
DittoSpyder DittoSpyder
DnyzBot DnyzBot
DomCopBot DomainCrawler
DomainAppender DomainSigmaCrawler
DomainStatsBot
Dotbot
Download Wonder Download Wonder
Drip Drip
ECCP/1.0
EMail Wolf
EirGrabber EirGrabber
Evil EroCrawler
Exabot
Express WebPictures Express WebPictures
ExtLinksBot ExtLinksBot
Extractor EyeNetIE
Ezooms Ezooms
Freeuploader FDM
FemtosearchBot
Firefox/7.0
Flunky
Foobot
Fuzz Fuzz
Fyrebot
G-i-g-a-b-o-t
Genieo Genieo
GetWeb GetWeb
Go-Ahead-Got-It
Gotit
GrabNet
Grabber
Grafula
GrapeFX GrapeFX
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO
HMView
HTTP::Lite HTTP::Lite
Haansoft
HaosouSpider HaosouSpider
Havij Harvest
Heritrix HybridBot
Hloader IDBTE4M
IDBot
IRLbot IRLbot
Id-search
IlseBot IlseBot
Image Fetch Image Fetch
Image Sucker IndeedBot
InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
InterGET InternetSeer
Iria Iskanie
JOC Web Spider
Jbrofuzz Jbrofuzz
JetCar JetCar
Jetty
Joomla
Jorgee Jorgee
JustView
Jyxobot Jyxobot
Kinza
LNSpiderguy LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
LieBaoFast Leap
LeechFTP
LeechGet
Libwhisker
Likse
LinkScan
Linkdexbot
LinkextractorPro
LinkpadBot LinkpadBot
LinksManager LinksManager
LinqiaScrapeBot LinqiaMetadataDownloaderBot
Litemage_walker LinqiaRSSBot
Lmspider MIDown tool
MFC_Tear_Sample MJ12bot
MSIECrawler MQQBrowser
Mag-Net Mag-Net
Mail.RU_Bot Majestic-SEO
Majestic12
MarkWatch
Mass Downloader Mass Downloader
Masscan
Mata Hari Mata Hari
Mb2345Browser Mb2345Browser
MeanPath Bot
Meanpathbot
Mediatoolkitbot Mediatoolkitbot
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Mojolicious Moblie Safari
Morfeus Fucking Scanner
Mozlila
Musobot
NPbot NPbot
Navroad
NearSite NearSite
Needle Needle
Net Vampire NetAnts
NetLyzer
NetSpider
NetZIP NetZIP
NextGenSearchBot
Nibbler Nibbler
Niki-bot Niki-bot
NimbleCrawler Ninja
Nimbostratus
Nuclei Nuclei
Nutch Offline Explorer
Octopus
OpenLinkProfiler
Openfind
Openvas Openvas
Page Analyzer OrangeSpider
OutfoxBot
PHPCrawl
PageAnalyzer
PageGrabber PageGrabber
PageScorer
Pandalytics
Papa Foto Papa Foto
Pavuk Pavuk
Pi-Monster
Picscout Picscout
Picsearch PictureFinder
Piepmatz Piepmatz
Pimonster
Pixray Pixray
Probethenet Psbot
Pu_iN
Pump Pump
PyCurl RankActive
QueryN Metasearch Rankivabot
RSSingBot RankurBot
RankActiveLinkBot
RankingBot
Re-re Re-re
RepoMonkey ReGet
Recorder
Ripper Ripper
RocketCrawler
SEOkicks-Robot SEOkicks-Robot
SEOstats SEOprofiler
ScanAlert SMTBot
Scrapy SalesIntelligent
ScreenerBot Scanbot
ScrepyBot ScoutJet
Searchestate Semrush
Siphon SeoSiteCheckup
SeobilityBot
Seomoz
Shodan
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteSnagger SiteSnagger
Sitebeam SiteSucker
Sitevigil Siteimprove
SlySearch SlySearch
SmartDownload
Sociscraper
Sottopop Sottopop
SpaceBison Stripper
Spanner
Spinn3r
SputnikBot
Sucker Sucker
SuperHTTP Sucuri
SurveyBot SurveyBot
Swiftbot
Szukacz
T0PHackTeam
T8Abot
Teleport
TeleportPro
Telesoft Telesoft
The Intraformant Telesphorep
Thumbor TheNomad
TightTwatBot
Toata Toata
Toweyabot Toweyabot
Trendiction
Trendictionbot Trendictionbot
True_Robot True_Robot
Turingos Turingos
Turnitin
TurnitinBot TurnitinBot
TwengaBot
URLy Warning
URLy.Warning
UnisterBot UnisterBot
Upflow Upflow
V-BOT V-BOT
VB Project Vacuum
Vagabondo Vagabondo
VeriCiteCrawler VeriCiteCrawler
Voil VidibleScraper
Virusdie
VoidEYE
WISENutbot
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0
WallpapersHD WallpapersHD
Web Auto
Web Collage Web Collage
Web Fuck Web Pix
Web Sauger
Web Sucker Web Sucker
WebBandit WebEnhancer
WebCopier WebPix
WebFetch
WebFuck
WebSauger WebSauger
WebStripper
WebSucker WebSucker
WebmasterWorldForumBot
Webshag Webshag
Website Quester
Whack Whack
Who.is Bot WinHTTrack
Widow WiseGuys Robot
Wonderbot Wprecon
Woobot Xaldon WebSpider
Xaldon_WebSpider
Xenu
YoudaoBot
Zermelo Zermelo
Zeus Zeus
ZmEu ZoominfoBot
archive.org_bot adscanner
backlink-check clark-crawler
cah.io.community
coccocbot-web coccocbot-web
cognitiveseo
crawler.feedback
dataforseo.com dataforseo.com
eCatch gopher
evc-batch
facebookscraper
heritrix heritrix
instabid internetVista monitor
ips-agent ips-agent
magpie-crawler netEstate NE Crawler
meanpathbot
oBot
page scorer page scorer
plumanalytics pcBrowser
probe-image-size probe-image-size
s1z.ru ripz
satoristudio.net
scalaj-http scalaj-http
seobility siteripz
seostar sogouspider
sexsearcher sp_auditbot
spyfu spyfu
sysscan
trendiction.com trendiction.com
trendiction.de webmeup-crawler
voyagerx.com webprosbot
x09Mozilla
x22Mozilla
xpymep1.exe xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
all4wap.ru 1stat.ru
cialischsrx.com berrymall.ru
co.lumb.co chee-by.biz
dcj-nn.ru compliance-alex.xyz
gta-top.ru jrcigars.com
shopfishing.com.ua moncleroutletonline.pw
sterva.cc pogodnyyeavarii.gq
teslathemes.com validccseller.com
usiad.net vsdshnik.com
yaminecraft.ru www.souvenirua.com

920
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1001watch.com.ua 007angels.com
101lesbian.xyz 00it.com
14b.info 0lovespells0.blogspot.com
18ps.ru 1stat.ru
1bet.com 256bit.by
1j7740kd.website 4pp13.com
20pascals.nl 57883.net
300richmond.co.nz 8gold.com
3th.co.in a.frcls.fr
40cg.com aa9bd78f328a6a41279d0fad0a88df1901.com
4senses.co aa9d046aab36af4ff182f097f840430d51.com
55wmz.ru acgs.tk
66cpwgln.space adktrailmap.com
991mostfm.co.id adpremium.org
a14download.com adserver-e7.com
aad1f4acb0a373420d9b0c4202d38d94fa.com advancedmassagebysara.com
acc.eu.org
accessoires-mode.in
ads.tremorhub.com
advancedcleaner.com
adzerg.com
afora.ru afora.ru
alert-fdm.xyz agapovdg.ru
alice.it aksonural.ru
allesohnegirls.net algarveglobal.com
allforminecraft.ru alive-ua.com
aloofly.com an-donut.com
alvtank.se ananumous.ru
am15.net android-vsem.org
animenime.ru androids-store.com
apparel-offer.com animalrank.com
appartement-stumm.at apiappenableinfo-a.akamaihd.net
appearance-cool.com apiholdingmypage-a.akamaihd.net
apple.com-webbrowsing-security.review apimegabrowsebiz-a.akamaihd.net
arquapetrarca.info apper.de
artavenuegardenstudios.com appfastplay.com
asscenihotosidea.blogspot.com apple.com-webbrowsing-security.science
astana.xxxkaz.org appmsr.org
australia-opening-times.com ardimobileinfo.ml
availit.weebly.com arvut.org
avon-novosib.ru asiavirtualsolutions.com
avtovolop.ru asscenihotosidea.blogspot.co.za
awaybird.ru asseenontv.ru
azarttoto.com autogrep.ru
azbukafree.com autovideobroadcast.com
backlinkwatch.com awency.com
ayeartoforget.com
balkanfarma.ru
bambi.ck.ua bambi.ck.ua
bamo.xsl.pt
bang-hotties.com bang-hotties.com
bankiem.pl bannerswap.com
beachpics.com battle.net
beachtoday.ru bbtec.net
bedcapdealers.com beclean-nn.ru
beget.tech beetpics.pw
best-seo-software.xyz beremenyashka.com
bestcoin.cc bestadbid.com
besthomemadepornsites.com bestimagecoollibrary.com
bestmarriages.com bestpornuha.com
bestofferswalkmydogouteveryday.gq biglistofwebsites.com
bestofupload.info bikini-image.com
bidr.trellian.com bimatoprost-careprost.com
big-boards.info bin-brokers.com
bingo8888.com bingoporno.com
bitcoinremote.com bioca.org
bjanshee.ru blog.yam.com
blog.f00kclan.de bobba.dzaba.com
bmsco.co bodybuilding-shop.biz
bmw-ark.ru boleznikogi.com
bnt-team.com bombla.org
boc.kz braindaily.xyz
boobsimge.com brandbucket.com
bookmark4you.com.biz bristolhotel.com.ua
bradleylive.xyz btnativenav.com
braintobe.win business-made-fun.com
brandedleadgeneration.com buydissertation.net
brendbutik.ru buyk.host.sk
browsepulse-a.akamaihd.net buynorxx.com
businesn.men buypuppies.ca
buttons-for-website.com buytizanidineonlinenoprescription.blogspot.com
buyantiviralwp.com buyviagraa.blogspot.com
buyhoverboard.com carsplate.com
buypillsorderonline.com cartierjusteunclou.xyz
bvlgariwallet.xyz cayado.snn.gr
cablecar.us cctva.tv
calc-for-credit.ru cdnanalytics.xyz
callcafe.info cdncash.org
canzoni.ru centre-indigo.org.ua
cars-modification.net cete.ru
carson.getenjoyment.net changde.58.com
cartierlovestore.xyz cheapjerseysa.com
cartierreplica.pw christianlouboutinshoes.xyz
cartierreplica.xyz cialischsrx.com
cartierring.xyz cienum.fr
cashonads.com clevernt.com
casinos4dummies.co.uk
cb.iphantom.com
celebsopics.com
cerev.info
chatroulette.video
chelyabinsk.xrus.org
chinacheapelitejerseys.com
choosecuisine.com
christianlouboutinreplica.pw
chuckguilford.com
cialis-samples.com
cigarpass.com
cleanallvirus.com
clickcash.com
clicksor.net
cocyq.inwtrade.com cocyq.inwtrade.com
coderstate.com coindirect.io
compiko.info colehaanoutlet.store
compliance-alex.xyz com-cleaner.systems
compliance-jane.top compliance-alexa.xyz
compliance-mark.top compliance-brian.xyz
comsysnet.com compliance-don.xyz
conciergegroup.org compliance-elena.top
contentlook.co compliance-josh.top
compliance-nelson.top
compliance-olga.top
compucelunlock.net
connectionstrenth.com
cookielawblog.wordpress.com cookielawblog.wordpress.com
coop-gamers.ru counter.bloke.com
crest3d.ru countercrazy.com
customchocolate.business-for-home.com craftburg.ru
customsua.com.ua creatives.adbetclickin.pink
cyclobenzaprinevstizanidine.blogspot.com
d-black.bz d-black.bz
ddpills.com dailyrank.net
delitime.info daptravel.com
den-noch24.ru dealighted.com
descargar-musica-gratis.net dear-diary.ru
detective01.ru dekorkeramik.ru
dev33.dioniqlabb.se detskie-konstruktory.ru
dimaka.info digital-sale.su
djstools.com dignitasdata.se
docsportal.net dikqlyremy.info
dood.live diy-handmade-ideas.com
downloadeer.net docstoc.com
doxyporno.com dodlive.mil
dptaughtme.com doeco.ru
dverimegapolis.ru dominterior.org
dynainbox.com domoysshop.ru
dyt.net download-wallpaper.net
ecommerce-seo.com downloader12.ru
elektrischezigaretten1.blogse.nl doxysexy.com
envaseslotusama.com drugspowerstore.com
escort-russian.com dumuelave.xyz
dvr.biz.ua
easyshoppermac.com
edudocs.net
ekn-art.se
elektrischezigarette2.devhub.com
elektrozigaretten1.tumblr.com
eluxer.net
embedle.com
enews.tech
enthuse.computernetworksonline.com
ertelecom.ru
estibot.com
everypony.ru
everytalk.tv everytalk.tv
f07.de evogarage.com
fakehandbags.xyz exmasters.com
falllow.gq
falopicm.pw
familyphysician.ru
fantasticpixcool.com fantasticpixcool.com
fearcrow.com fenoyl.batcave.net
feel-planet.com fok.nl
filerockstar298.weebly.com footballfarrago.com
filesmonster.porn forensicpsychiatry.ru
financetip.eu forex-indextop20.ru
flauntyoursite.com forum20.smailik.org
florida-tourism.net fotoxxxru.com
floridahuntingfishingadventures.com fotzen-ficken.com
flowwwers.com franch.info
forpostlock.ru free-floating-buttons.com
forum.doctissimo.fr free-share-buttons.top
frbizlist.com free-social-buttons.com
free-video-tool.com free-social-buttons.xyz
freecamdollars.com fun-mobi.pl
freemaintenancesysforpcandmac.top fx-brokers-review.com
freewhatsappload.com gael-s.ru
freza-sverlo.ru gamerextra.com
frivgame250.com gay-site.store
fsakhalin.ru gaygalls.net
fxgallery.com
galeria-zdjec.com
gamebackyard.com
gaypornmovie.net gaypornmovie.net
gazoblok.net.ua ge0ip.org
gembird.com gearsadspromo.club
gentamicineyedrops.blogspot.com geileweiber.tk
getlamborghini.ga get-seo-domain.com
getoutofdebtfree.org getdot.ru
getpopunder.com getmiro.com
gkvector.ru gigapeta.com
glcomputers.ru girlporn.ru
globalsurfari.com
gobongo.info gobongo.info
gogps.me gombita.info
golmau.host.sk googst2.ru
goodwinmetals.co gratuitxblcodes.com
gosmeb.ru greatdealshop.com
gowreckdiving.com gsasearchengineranker.site
greatfind-a.akamaihd.net hacktougroup.ru
greendream.com.ua hahashka.ru
hawaiisurf.com hanink.biz.ly
hayate.biz hermesbracelets.xyz
hd720kino.ru hitcpm.com
hdapp1008-a.akamaihd.net homik.pw
hdwallpapers-free.com hotblog.top
hdxnxxtube.mobi
healing-dysplasia.ru
hentaiheroes.com
highstairs-a.akamaihd.net
hitsbox.info
holidaypics.org
hotelcrocenzi.sm
hotloans.ru hotloans.ru
hoverboard360.nl housediz.com
howlongdoestizanidinestayinyoursystem.blogspot.com hptwaakw.blog.fc2.com
humbmosquina.tk hreade.com
igadgetsworld.com hspline.com
imguramx.pw hyiphunter.org
imicrovision.com i4track.net
imk.com.ua ibmdatamanagement.co
ilovevitaly.com
images.gyffu.com
immigrational.info immigrational.info
imperiafilm.ru impisr.ru
indiakino.net inet-traffic.com
infospot.pt invest-pamm.ru
intellego.info investmac.com
intellekt21.ru
invitefashion.com
ipchicken.com ipchicken.com
iqbazar.ru iscblog.info
ivanstroi.ru isitpaleo.info
ixora.pro isoveti.ru
jav-way.site istmira.ru
jerseyschinabizwholesale.us itbc.kiev.ua
jo24news.com itsdp3.com
joingames.org iyasimasennka.com
justucalling32211123456789.tk izismile.com
kambasoft.com janettabridal.com
katushka.net je7.us
kazrent.com jmat.cn
keywordbasket.com joessmogtestonly.com
kino-rating.ru junketjuice.blogspot.com
kirov.zrus.org kadashihotel.com
komputers-best.ru karpun-iris.ru
kongruan.com kazinogames.lv
koversite.info kfon.eu
krasnodar-avtolombards.ru kinocccp.net
ks1234.com kit-opt.ru
ku6.com klin3952.ru
kurbappeal.info knogg.net
lacave.ntic.fr koptims.tiu.ru
landmania.ru krasivye-devushki.net
lankarns.com kskjco.club
laptoper.net kung-fu-ru.com
larger.io laapp.com
laserpen.club lanadelreyfans.us
lashstudia.ru lashstudia.ru
laulini.soclog.se light.ifmo.ru
law-check-nine.xyz
lazymanyoga.com
ledpolice.ru
lifehacĸer.com
lignofix.ua lignofix.ua
linkbolic.com linkredirect.biz
liupis.com lion.conboy.us
liveshoppersmac.com liran-locks.com
lolnex.us ljusihus.se
longgreen.info lostfilm-online.ru
lowephotos.info lunchrock.co
lrac.info
m0r0zk0-krava.ru m0r0zk0-krava.ru
mafcards.ru magento-crew.net
mainhp.net
make-money-online.com make-money-online.com
manifestation.betteroffers.review mamasuper.prom.ua
manifestyourmillion.com mantramusic.ru
marcoislandvacations.net mararoom.ru
marketingvici.com marketingtechniques.info
mebeldekor.com.ua mebelcomplekt.ru
meble-bogart.info
mediawhirl.net
medicineseasybuy.com medicineseasybuy.com
medkritika.ru medicovi.com
medmajor.ru
medosmotr-ufa.ru
mega-bony-2017.pl
mega-polis.biz.ua mega-polis.biz.ua
megahdporno.net
menhealed.net menhealed.net
message-warning.net mhtr.be
mesto-x.com microsoftportal.net
metarip.ru mir-betting.ru
metaxalonevstizanidine.blogspot.com
michaelkorssaleoutletonline.net
migente.com
mindbox.co.za
mini.7zap.com
minneapoliscopiers.com
mistr-x.org mistr-x.org
mixtapetorrent.com mmoguider.ru
mnogolok.info moivestiy.biz
moesonce.com monarhs.info
mojaocena.com money-every-day.com
moncleroutletonline.win monsterdivx.tv
moneyviking-a.akamaihd.net monthlywinners.com
morepoweronmac.trade mozello.ru
morf.snn.gr mrbojikobi4.biz
moroccosurfadventures.com mrvideospornogratis.xxx
mp3ritm.top mtmtv.info
mriyadh.com
mturkcontent.com
muabancantho.info muabancantho.info
music.utrolive.ru mybackgroundlandscape.blogspot.com
musicdaddy.net mypornfree.ru
mvpicton.co.uk nardulan.com
mydirtystuff.com negociosdasha.com
mydownloadengine.com nerudlogistik.ru
myfreecams.com nikhilbahl.com
myra.top northfacestore.online
na15.ru nosecret.com.ua
nfljerseyscheapbiz.us novodigs.com
nfljerseysforsalewholesaler.com nsatc.net
notasprensa.info nzfilecloud.weebly.com
nyfinance.ml obiavo.in
o-o-11-o-o.com obsessionphrases.com
oil-td.ru oconto.ru
okel.co oddamzadarmo.eu
one-gear.com odesproperty.com
oneminutesite.it okonich.com.ua
online-hd.pl omgtnc.com
online-hit.info online-sbank.ru
onlinetvseries.me onthemarch.co
onlyporno.ru ooomeru.ru
optom-deshevo.ru openfrost.net
otbelivanie-zubov.com oranga.host.sk
ourville.info p-business.ru
outshop.ru
ozoz.it
paighambot.com
paintingplanet.ru paintingplanet.ru
panamaforbeginners.com parajumpersjakkesalgnorge.info
parajumpersoutlet.online
paramountmarble.co.uk
park.above.com
parlament.biz parlament.biz
pertlocogasilk.tk partner-print.men
peugeot-club.org partnerline.men
phormchina.com parvezmia.xyz
pcads.ru
pcboa.se
pelfind.me
perfectpracticeweb.com
phelissota.xyz
photorepair.ru photorepair.ru
picscout.com picphotos.net
pijoto.net pinkduck.ga
pladform.ru pipki.r.acdnpro.com
playlott.com pisanieprac.info
pooleroadmedicalcentre.co.uk placid-rounded-coast.glitch.me
popunder.ru platesauto.com
popup.matchmaker.com platezhka.net
porn-w.org podshipniki-ntn.ru
porndairy.in pogodnyyeavarii.gq
pornhub-forum.ga poptool.net
pornoblood.com porndl.org
porno-transsexuals.ru
pornofiljmi.com
pornogig.com pornogig.com
pornonik.com pornosexrolik.com
pornzone.tv porto.abuilder.net
printingpeach.com portside.xyz
privatefx-in.ru privatov-zapisi.ru
professionaldieselcare.com
prombudpostach.com.ua
promoheads.com promoheads.com
prostitutki-kiev.org promover.org
prostitutki-odessa.org proxyradar.com
purplestats.com pukaporn.com
q-moto.ru pulse33.ru
qitt.ru quangcaons.com
qwertty.net quelle.ru
qwesa.ru questionmarque.ch
randki-sex.com quick-seeker.com
quickchange.cc
rank-checker.online rank-checker.online
ranksonic.com ranksonic.org
real-time-analytics.com razleton.com
recinziireale.com razvratnoe.org
reckonstat.info rcb101.ru
referencemoi.com realmonte.net
reklama1.ru realting-moscow.ru
remedyotc.com registrysweeper.com
revistaindustria.com remybutler.fr
riralmolamsaca.tk rentaremotecomputer.com
rocketchange.ru responsinator.com
royalads.net ribieiendom.no
rrutw.com rocis.site
ryetaw.com roll123.com
s1z.ru running-line.ru
sammyweaver.com rusenvironmental.net
santasgift.ml saneyes.com
savefrom.com satellite.maps.ilovevitaly.com
savingsslider-a.akamaihd.net scanner-alex.top
sbf441.com scrapy.org
sbricur.com se-welding.ru
scanner-josh.top searchimage.co
scansafe.net searchtooknow-a.akamaihd.net
se.bnt-team.com semrush.com
searchinquire.com
searchwebknow-a.akamaihd.net
seksvideoonlain.com
semp.net
seo-2-0.com seo-2-0.com
sfj-ror.no seonetwizard.com
sharebutton.to seryeznie-znakomstva.ru
sherlock.se sex.hotblog.top
shoesonlinebuy.cn shop.xz618.com
shoplvlv.us shopperifymac.com
shtora66.ru sicfor.bcu.cc
sibvitr.ru sierraapps.com
site-analyzer.com signal03.ru
sky-mine.ru sindragosa.comxa.com
sinestesia.host.sk
slavia.info slavia.info
slimcdn.com sluganarodu.ru
slonechka.ru smartadserver.com
soaksoak.ru social-s-hhh.xyz
solmarket.by socialbookmarksubmission.org
solnplast.ru socialmadesimple.com
soserfis.com songplanet.ru
spamnuker.com sortthemesitesby.com
spb.afora.ru spb.ru
spbchampionat.ru spbchampionat.ru
spinnerco.ca spywarespy.com
srdrvp.com starpages.net
statustroll.com statistici.ro
stockquotes.wooeb.com steamoff.net
strv.se stockspmb.info
sukarame.net stroiminsk.com
supermama.top stroyplus.ru
superstarfloraluk.com studworks.org
t-bygg.com sudexpert66.ru
takethatad.com superlist.biz
tampabaywatch.org supermesta.ru
telefonsexi.com surgut.zrus.org
telefonsexkostenlos.tk susanholtphotography.com
testingads.pro svensk-poesi.com
svetlotorg.ru
swinginwithme.ru
syndicate.fun
szamponrevita.pl
tam-gde-more.ru
tasteidea.com
tattomedia.com
td-33.ru
teenfuck.tv
theautoprofit.ml
thebestphotos.eu thebestphotos.eu
thebitcoincode.com thedownloadfromwarez.blogspot.com
thejournal.ru
tiandeural.ru
ticketsys.inetwd.com ticketsys.inetwd.com
timetorelax.biz tizanidine4mg.blogspot.com
tizanidinecost.blogspot.com tizanidinedosage.blogspot.com
tizanidinestreetvalue.blogspot.com tizanidineinteractions.blogspot.com
tjkckpytpnje.com tizanidineoral.blogspot.com
todohr.com tn811.us
toonfamilies.net tomck.com
topcasinoratings.ru top1-seo-service.com
topquality.cf top10-online-games.com
torrentred.games top10-way.com
torrnada.ru torrents.cd
tpu.ru trafaret74.ru
trucri.me trafficgenius.xyz
trustedhealthtips.com trafficmonetize.org
tupper-posuda.ru traffictrade.life
turbodsp.com treasuretrack-a.akamaihd.net
twitlinks.com trkdf.com
twittrading.com trkur.com
trustedmaccleaner.com
tuberkuleznik.ru
txxx.com txxx.com
uggsale.online ucoz.ru
ultimatesetnewfreeallsoftupgradesystems.pw ukkala.xyz
unisexjewelry.org umityangin.net
unrealcommander.com unionmarkt.de
vacuumcleanerguru.com unitygame3d.com
vandrie-ict.nl uptime-alpha.net
varbergsvind.se uptime-delta.net
urzedowski.eu
ussearche.cf
vabasa.inwtrade.com
valmetrundan.se
veles.shop
verymes.xyz verymes.xyz
viberdownload10.com videochat.bz
videositename.com vigrx-original.ru
vinsit.ru vinsit.ru
vipms.ru vip2ch.com
vipromoffers.com vmnmvzsmn.over-blog.com
virtualbb.com
visa-pasport.ru
vitanail.ru
vkontarkte.com
voloslove.ru
voprosotvet24.ru voprosotvet24.ru
voronezh.xrus.org voronezh.xrus.org
vostoktrade.info vrotike.ru
vzlom-na-zakaz.com vsexkontakte.net
vtc.pw
w3data.co w3data.co
w3javascript.com wallpapers-best.com
wahicbefa31.soup.io waycash.net
webalan.ru wdfdocando.com
webnode.me webcamdevochka.com
weboptimizes.com webcamtalk.net
webscouter.net wechatdownload10.com
website-datenbank.de weclipart.com
websites-reviews.com wfb.hatedriveapart.com
wingsoffury2.com whengirlsgowild.com
worldis.me wholesalecheapjerseysfree.com
wtsindia.in wiosenny-bon-1500.pl
www.solartek.ru wordpress-crew.net
www.souvenirua.com wowas31.ucoz.ru
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai x-rates.ru
xev.ru
xjrul.com
xlolitka.com
xmronta.com
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
xn----7sbifcamovvfggw9d.xn--p1ai
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xn----ctbbcjd3dbsehgi.xn--p1ai xn----ctbbcjd3dbsehgi.xn--p1ai
xn----ctbigni3aj4h.xn--p1ai xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--80aikhbrhr.net xn--h1aakne2ba.xn--p1ai
xn--b1ag5cfn.xn--p1ai xolodremont.ru
xnxx-n.com xtubeporno.net
xtrafficplus.com xvideospornoru.com
yachts-cruise.info xwatt.ru
yaoguangdj.com xxxasianporn.net
yebocasino.com xxxhdvideo.site
yellowads.men yaaknaa.info
youghbould.wordpress.com yeartwit.com
youjizz.vc yginekologa.com
youtubologia.it youdesigner.kz
zambini.ru youporn-forum.ga
zdesformula.ru youtubedownload.org
zoosexart.com zaloadi.ru
zrizvtrnpale.tk zastenchivosti.net
zscalertwo.net zastroyka.org
zazagames.org
zed21.net
zeroredirect11.com
znakomstva-moskva77.ru
znakomstvaonlain.ru
zoogdiesney.com
zootoplist.com
zvooq.eu

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
007angels.com 03e.info
00go.com 0daymusic.org
1stat.ru 1-99seo.com
3weekdiet.com 1bet.com
abiente.ru 1xbet4you.com
acc.eu.org 3-letter-domains.net
adclickthru.net 4ur.click
adservme.com acgs.tk
adssafeprotected.com adclickservice.com
adultgalls.com addbin.men
akama.com adidas.frwebs.fr
ampower.me advancedmassagebysara.com
amyfoxfitness.com advertex.info
apelsinnik.website alevinefesleri.com
apiadanaknet-a.akamaihd.net alienwheels.de
apifasterlightin-a.akamaihd.net an-donut.com
apisecretsaucebi-a.akamaihd.net apessay.com
arquapetrarca.info app5.letmacworkfaster.world
artdestshop.eu arenanews.com.ua
ausmepa.org.au australia-opening-times.com
autoblogger24.info autobrennero.it
aversis.net avkzarabotok.com
azarttoto.com azartmix.com
baladur.ru azazu.ru
bbsoldes.fr bcmp.org
bbtec.net beetpics.pw
beauty-bracelet.com bigpictures.club
belinka.com.ua blog.axant.it
bestchoice.cf bochemit.com.ua
bestmouthwash.club bolezniorganov.ru
betune.onlinewebshop.net bookmaker-bet.com
bimatoprost-careprost.com brainboostingsupplements.org
biocn.dx.am braintobe.win
blog.koorg.ru brakehawk.com
bmwhighperformers.com brimstonehillfortress.org
business-suggested.tk btnativenav.com
buyhoverboard.com burn-fat.ga
buzz-porno.info buyantiviralwp.com
callawaygolfoutlet.online bvlgaribracelet.xyz
camdenmemorials.com bycontext.com
cardsdumps.com canzoni.ru
casinos4dummies.co.uk carrauterie.be
ceotrk.com cdnnetwok.xyz
cfsrating.sonicwall.com certifywebsite.win
checktext.org chelyabinsk.xrus.org
cialgenisrx.com cherubinimobili.it
cialischstgerts.com christianlouboutinsaleonline.us
cityadspix.com classicakuhni.ru
clickprotects.com clevernt.com
coldfilm.ru clickbanksites.info
cometorussia.net colehaanoutlet.store
connectingsingles.com com-onlinesupport.site
cookie-law-enforcement-ee.xyz compliance-brian.top
cordstrap.cc compliance-george.top
coslab.club construmac.com.mx
counciltally.com coop-gamers.ru
counter.yadro.ru dalavia.ru
creativehutindia.com datract.com
daneshetabiat.com detoxmed24.ru
dealighted.com
detective01.ru
digital-video-processing.com digital-video-processing.com
discountliv.com dikqlyremy.info
doctormakarova.ru distver.ru
domaincdn.xyz dosug-lux.ru
domini.cat drpornogratisx.xxx
downloader12.ru drugs-no-rx.info
downtuptv.gq dscaas.website
dtm-spain.com earl-brown.info
elektronischezigarettekaufen1.myblog.de ecookna.com.ua
elvenmachine.com ecxtracking.com
enews.tech energy-ua.com
engines-usa.com enthuse.computernetworksonline.com
escortslet.net eyes-on-you.ga
european-torches.ru failingmarriege.blogspot.com
everflora.ru familyphysician.ru
expolicenciaslatam.co fastcrawl.com
faithe.top floridamhca.org
famix.xyz fm-upgrade.ru
feel-planet.com foodcrafts.website
ferretsoft.com forminecrafters.ru
fidelityfunding.com freeseedsonline.com
findthe.pet fres-news.com
fireads.men friendflnder.com
flprog.com frvo.alptandem.ru
forums.d2jsp.org fsalas.com
fr-bearings.ru fym.com.ru
funnel.co.za galeon.com
gagrasector.ru game300.ru
gemara.com gamingspark.com
gg.zzyjxs.com gate5.co.za
gok-kasten.net gay.adultgalls.com
goldenggames.com gemgrab-a.akamaihd.net
gyffu.com gigixo.com
hdxnxxtube.mobi globalsurfari.com
hobbyhuren24.net gojiberriess.apishops.ru
house.sieraddns.com goldadpremium.com
houston-vikings.com goodhousekeeping.com
hoverboard360.nl hccoder.info
hyiphunter.org healthcarestore.info
ifmo.ru hledejvshopech.cz
ipornox.xxx host-protection.com
ispac.org hoverboard360.at
ivanovo.zrus.org hoverboard360.se
jacago.com hyhj.info
jackwolfskinoutlet.online iideaidekonkatu.info
jerseywholesalechinabiz.com imadedinner.net
jovencitas.gratis imperiafilm.ru
jumptap.com infodocsportal.com
jus0wil.pp.ua invest-pamm.ru
justdating.online istanbulit.com
kalb.ru istock-mebel.ru
karadene.com iwantmyfreecash.com
keenoutlet.online ix20.ru
kinozapas.org javstock.com
klosetkitten.com jillepille.com
kongruan.com joyceblog.top
ku6.com kinky-fetishes.com
labplus.ru kinogonew.ru
lacloop.info konkursov.net
larose.jb2c.me kreativperlen.ch
le-clos-des-alouettes.com laptoper.net
limtu.ifmo.ru larutti.ru
linkrr.com legionalpha.com
likesdesign.com
limads.men
lion.snapmap.us lion.snapmap.us
livesex-amateure.info liveu.infoteka.hu
liveshoppersmac.com lombia.co
loadingpages.me lonfon.xyz
lostfilm-online.ru luciddiagnostics.in
lutherstable.org m.b00kmarks.com
m1media.net m1media.net
mahnwachen-helfen.info madot.onlinewebshop.net
masthopehomes.com magicdiet.gq
max-eclat.men manualterap.roleforum.ru
mebel-vstroika.ru mecash.ru
mediawhirl.net mediaoffers.click
meet-flirt-dating.com meget.co.za
mexicotravelnet.com minecraft-neo.ru
mikrobiologies.ru mir-betting.ru
mlvc4zzw.space mistr-x.org
mmgq.ru monarchfind-a.akamaihd.net
mmostrike.ru moyakuhnia.ru
mobstarr.com mrmoneymustache.com
morefastermac.trade musezone.su
moskva.nodup.ru mydearest.co
most.gov.iq myperiod.club
mrcsa.com.au nacap.ru
mriyadh.com netvouz.com
musirc.com nippon-bearings.ru
muztops.ru nottyu.xyz
mydoctorok.ru novodigs.com
mzdish.site nudejapan.net
natali-forex.com oakleyglassesonline.us
nightvision746.weebly.com optitrade24.com
nofreezingmac.work
npoet.ru
obiavo.uz
odoratus.net
okel.co
okroshki.ru
onlineserialy.ru
orion-code-access.net
orsonet.ru
outrageousdeal-a.akamaihd.net outrageousdeal-a.akamaihd.net
ozas.net palvira.com.ua
partner-pop.men pekori.to
pc-virus-d0l92j2.pw percin.biz.ly
pensplan4u.com pic-re.blogspot.com
photostudiolightings.com picturesfrom.com
phuketscreen.com pinwallpaper.xyz
pillscheap24h.com plastgranar.nu
pinkduck.ga
pistonclasico.com
platesauto.com platesauto.com
pod-muzyku.club poquoson.org
pogodnyyeavarii.gq porndroids.com
poppen-nw.net pornoklad.ru
porno-raskazy.ru
pornobrazzers.biz
pornokorol.com pornokorol.com
preventheadacheguide.info pornotubs.com
professionaldieselcare.com pregnant.guru
projectforte.ru pro-tec.kz
pst2017.onlinewebshop.net propranolol40mg.blogspot.com
qpypcx.com pssucai.info
qwertty.net qualitymarketzone.com
rankia.com quangcaons.com
referencemoi.com razorweb-a.akamaihd.net
reklamuss.ru rednise.com
renhacklids.tk refudiatethissarah.info
rus-pornuha.com registrationdomainsite.com
rusenvironmental.net relayblog.com
sbwealthsolutions.ca remont-comp-pomosh.ru
scanner-nelson.top runofilms.ru
seccioncontrabajo.com sanyuprojects.com
securityallianceservices.com sbricur.com
seminarykansas904.ml sbtdesign.co.uk
shoesonlinebuy.xyz scanner-john.top
shopperifymac.com searchimage.co
site-analyzer.com seasonvar.ru
social-buttons.com seemoreresultshu-a.akamaihd.net
socialbookmarksubmission.org seo-prof1.xyz
socialmadesimple.com sexystrippe.info
sticken.co shtora66.ru
stockspmb.info sildenafilcitratemed.com
streetfooduncovered.com sisiynas.ru
suchenindeutschland.com sitevaluation.com
sugarkun.com sitopreferito.it
swimpool.ca skanninge.se
swinger-mobil.net socas.pluto.ro
telesvoboda.ru stairliftstrue.com
telsis.com stard.shop
thefarmergame.com stopnarco.ru
thetoiletpaper.com stroimajor.ru
time-japan.ru studentguide.ru
tizanidinecapsules.blogspot.com studworks.org
tizanidinehcl2mgtablet.blogspot.com sugvant.ru
top10-way.com tabletkinaodchudzanie.com.pl
traderzplanet.in tds-advert005.info
trainoffend.ml the-torrent-tracker.blogspot.com
triumf-realty.ru thefotosgratis.eu
tiandeural.ru
tizanidineandhydrocodone.blogspot.com
tizanidineinteractions.blogspot.com
tizanidinesleepaid.blogspot.com
token-lab.org
tozup.com
tpu.ru
trackzapper.com
traffique.net
tsstcorpcddvdwshbbdriverfb.aircus.com tsstcorpcddvdwshbbdriverfb.aircus.com
turizm.bz tvteleport.ru
typimga.pw tyumen.xrus.org
ugogo.info u555u.info
ukrtvir.com.ua ukrtvory.in.ua
upupa.net unrealcommander.com
videochat.life us-america.ru
videokrik.net validccseller.com
vizag.kharkov.ua varikoz24.com
we-are-gamers.com vedomstvo.net
website-analyzer.info videochat.world
wejdz-tu.pl vrotike.ru
westsextube.com vzube.com
westum.se wallpapers-best.com
wma-x.com watsonrealtycorp.com
ws.ampower.me websiteaccountant.de
www.888.com wechatdownload10.com
www.jbetting.com wedding0venues.tk
x-diesel.com whatistizanidine2mg.blogspot.com
xgames-04.com wildnatureimages.com
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai wildworld.site
xvideosbay.com wingsofrefuge.net
xvideospornoru.com woman-orgasm.ru
xxxdatinglocal.us worldinternetauthority.com
youdao.com www.bookmaker-bets.com
zakazfutbolki.com xn--24-glceagatoq7c2a6ioc.xn--p1ai
zhacker.net xn--80ab4aa2g.xn--p1ai
you-shall-not-pass.is74.ru
yoursite.com
zdesformula.ru
zeroredirect8.com
zmoda.hostreo.com
zolotoy-lis.ru zolotoy-lis.ru
zrelaya.pw

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.0.240 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2731 ### Version: V4.2021.11.2732
### Updated: Mon Nov 22 22:01:13 UTC 2021 ### Updated: Tue Nov 23 10:03:14 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2731 ### Version: V4.2021.11.2732
### Updated: Mon Nov 22 22:01:13 UTC 2021 ### Updated: Tue Nov 23 10:03:14 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##