vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 15:34:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3466 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-12 10:08:59 +00:00
parent e54500173c
commit 344cbb5143
46 changed files with 95508 additions and 95508 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3465 #### Version: V4.2022.11.3466
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3465 #### Version: V4.2022.11.3466
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3465 #### Version: V4.2022.11.3466
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11802
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3465 #### Version: V4.2022.11.3466
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11806
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11806
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11806
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11806
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.5 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.5 0;
127.0.0.1 0; 127.0.0.1 0;

11806
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.5 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.5 0;
127.0.0.1 0; 127.0.0.1 0;

11806
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.5 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.5 0;
127.0.0.1 0; 127.0.0.1 0;

11806
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3465 3466

11806
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11806
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.166.160 0; 20.121.12.214 0;

11806
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.5 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.5 0;
127.0.0.1 0; 127.0.0.1 0;

11806
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.5 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.5 0;
127.0.0.1 0; 127.0.0.1 0;

11806
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.5 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.5 0;
127.0.0.1 0; 127.0.0.1 0;

11806
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.166.160 0; 20.121.12.214 0;

11806
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.166.160 0; 20.121.12.214 0;

11806
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.5 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
360Spider
Acunetix Acunetix
Alligator Attach
Barkrowler Cegbfeieh
Discoverybot Drip
WWW-Mechanize Keyword Density
crawler.feedback Mozlila
demandbase-bot NPbot
webmeup-crawler PxBroker
zauba.io Website Quester
crawler4j

316
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
404enemy 404enemy
80legs
ADmantX ADmantX
AIBOT Abonti
ALittle Client
Aboundex Aboundex
Aboundexbot Aboundexbot
Acunetix Acunetix
AiHitBot AhrefsBot
Aipbot
AllSubmitter AllSubmitter
Alligator
Anarchy
Ankit
Apexoo Apexoo
Aspiegel Attach
BBBike BBBike
BDFetch BDCbot
BackWeb BLEXBot
BacklinkCrawler
Badass Badass
Bandit Barkrowler
BlackWidow BetaBot
Blow Black Hole
Blackboard
BlowFish BlowFish
BotALot
Brandwatch
Buddy
BuiltBotTough BuiltBotTough
BuiltWith
Bullseye
BunnySlippers BunnySlippers
BuzzSumo
CATExplorador CATExplorador
CCBot CCBot
Calculon CODE87
CazoodleBot
Cegbfeieh
CheTeam CheTeam
CheeseBot CherryPicker
Chlooe Chlooe
Claritybot Citoid
Cloud mapping
Cocolyzebot
Cogentbot
Collector
CopyRightCheck CopyRightCheck
Copyscape
Craftbot
Crawling at Home Project Crawling at Home Project
Crescent
CrunchBot CrunchBot
Custo Custo
DIIbot DBLBot
DTS Agent DSearch
DataCha0s DataCha0s
Demon
Deusu Deusu
Disco
Discoverybot Discoverybot
DittoSpyder Dispatch
DnBCrawler-Analytics
DomCopBot
DomainAppender
Dotbot Dotbot
EMail Siphon Download Wonder
Ebingbong Drip
Extractor ECCP/1.0
EasyDL
Ecxi
EirGrabber
Exabot
Express WebPictures
ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
Ezooms EyeNetIE
FDM Fuzz
FHscan GT::WWW
Fimap
Freeuploader
FrontPage
G-i-g-a-b-o-t
GermCrawler GermCrawler
GetRight GetRight
Gigabot
Go!Zilla Go!Zilla
Grabber Go-Ahead-Got-It
HEADMasterSEO GoZilla
Gotit
Grafula
GrapeshotCrawler
HMView HMView
HTMLparser
HTTrack
HaosouSpider HaosouSpider
Heritrix Havij
Hloader Hloader
Humanlinks
IDBTE4M IDBTE4M
IRLbot IRLbot
IlseBot
Image Fetch Image Fetch
Indy Library Image Sucker
InfoTekies IndeedBot
Intelliseek Intelliseek
InternetSeer InterGET
JamesBOT Iskanie
Jbrofuzz
JennyBot JennyBot
Jetty JetCar
Joomla Joomla
Jorgee Jorgee
Jyxobot Jyxobot
Keybot Translation-Search-Machine Kinza
Keyword Density Kozmosbot
LNSpiderguy LeechGet
Leap LexiBot
LeechFTP LieBaoFast
Lftp LinqiaScrapeBot
Lightspeedsystems
Likse
LinkScan
LinqiaMetadataDownloaderBot
Lipperhey Lipperhey
Ltx71 Ltx71
MFC_Tear_Sample MIDown tool
MIIxpc
MJ12bot MJ12bot
MSFrontPage MTRobot
Mag-Net
Magnet
Majestic SEO
Majestic-SEO Majestic-SEO
MarkMonitor MarkMonitor
Masscan MarkWatch
Mata Hari
MauiBot
MeanPath Bot MeanPath Bot
Meanpathbot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri
Microsoft Data Access
Minefield
Mister PiX
Moblie Safari
Mojeek Mojeek
Mojolicious Mojolicious
Morfeus Fucking Scanner Mr.4x3
Musobot
NICErsPRO
Nameprotect Nameprotect
NearSite
Needle Needle
NetLyzer Nessus
Net Vampire
NetAnts
NetSpider NetSpider
Nettrack
Netvibes Netvibes
NextGenSearchBot Nibbler
Niki-bot Nmap
NimbleCrawler Not
Offline Explorer Nuclei
OpenVAS Openvas
Openfind OrangeSpider
OrangeBot
OutclicksBot OutclicksBot
PHPCrawl OutfoxBot
PageScorer Page Analyzer
PageThing.com Papa Foto
Pi-Monster PictureFinder
Picscout Piepmatz
Pixray
PleaseCrawl
Pockey Pockey
ProPowerBot ProPowerBot
Probethenet ProWebWalker
RankActive Pu_iN
Pump
QueryN Metasearch
Quick-Crawler
RSSingBot
RankActiveLinkBot
RankingBot RankingBot
Re-re Rankivabot
ReGet Reaper
RealDownload
RebelMouse
Recorder Recorder
Ripper RocketCrawler
SBIder Rogerbot
SEOkicks-Robot
SEOprofiler SEOprofiler
ScanAlert SISTRIX
SalesIntelligent
ScoutJet ScoutJet
Screaming
ScreenerBot ScreenerBot
SearchmetricsBot Seekport
SemanticJuice SemrushBot
Semrush
SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Seomoz Seomoz
Shodan
Siphon Siphon
Site Sucker Sitevigil
SiteCheckerBotCrawler SmartDownload
SiteLockSpider SocialRankIOBot
SiteSnagger Sogou web spider
SiteSucker
Siteimprove
Sottopop
Spammen
Spbot Spbot
Spinn3r Sqlworm
SputnikBot Sucker
Steeler SuperHTTP
SuperBot T8Abot
Surfbot The Intraformant
T0PHackTeam Titan
Teleport Toweyabot
Telesphoreo Tracemyfile
Toata Trendictionbot
Turnitin True_Robot
URLy.Warning TurnitinBot
UnisterBot TwengaBot
Upflow Typhoeus
VB Project VB Project
Vagabondo Vacuum
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
Virusdie
VoidEYE VoidEYE
Voltron
WBSearchBot WBSearchBot
WEBDAV
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW::Mechanize
Wallpapers/3.0 Wallpapers
WallpapersHD WallpapersHD
WeSEE
Web Auto Web Auto
Web Enhancer
Web Fetch Web Fetch
Web Fuck Web Fuck
Web Pix Web Pix
Web Sucker
WebAuto WebAuto
WebBandit WebFetch
WebCollage WebFuck
WebGo IS
WebImageCollector WebImageCollector
WebLeacher WebLeacher
WebSucker WebmasterWorldForumBot
Webshag Webshag
Website Quester
WebsiteQuester
Whack Whack
Whatweb Whacker
WiseGuys Robot
Woobot Woobot
Zade Wotbox
Zitebot Xaldon_WebSpider
ZmEu Xenu
ZoomBot Zauba
ZumBot ZumBot
ZyBorg ZyBorg
adscanner adscanner
arquivo-web-crawler arquivo-web-crawler
backlink-check autoemailspider
com.plumanalytics check1.exe
cognitiveseo
crawl.sogou.com crawl.sogou.com
crawler.feedback
dataforseo.com dataforseo.com
domainsproject.org
eCatch eCatch
evc-batch evc-batch
gopher facebookscraper
instabid heritrix
internetVista monitor iubenda-radar
linkdexbot
magpie-crawler
netEstate NE Crawler
pcBrowser pcBrowser
plumanalytics
polaris version polaris version
seocompany.store probe-image-size
ripz
satoristudio.net
scalaj-http
scan.lol
seobility
seoscanners seoscanners
serpstatbot
sexsearcher sexsearcher
sitechecker.pro sp_auditbot
siteripz ubermetrics-technologies.com
sogouspider voyagerx.com
tAkeOut xpymep1.exe
trendiction.com
trendiction.de
webmeup-crawler
webprosbot

314
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
404checker 404enemy
80legs ADmantX
AIBOT AIBOT
ASPSeek ALittle Client
Aboundex Aboundexbot
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AllSubmitter AllSubmitter
Anarchie Alligator
Anarchy
Ankit
Anthill Anthill
Apexoo
Asterias Asterias
Attach Atomseobot
AwarioRssBot AwarioSmartBot
BBBike BackDoorBot
BDFetch
BLEXBot
BackWeb BackWeb
Badass Badass
Bandit
Barkrowler Barkrowler
BatchFTP
BetaBot BetaBot
BlackWidow Black Hole
BlowFish
Boardreader
BuiltWith BuiltWith
BunnySlippers BuzzSumo
CATExplorador CCBot
Calculon CODE87
Cegbfeieh Cegbfeieh
ChinaClaw CensysInspect
CheTeam
CheeseBot
CherryPicker
Chlooe Chlooe
Citoid Claritybot
Cloud mapping
Cocolyzebot
Cogentbot Cogentbot
Collector
CopyRightCheck CopyRightCheck
Craftbot Cosmos
Crawling at Home Project
CrazyWebCrawler
Curious Curious
Custo
CyotekWebCopy CyotekWebCopy
DIIbot DIIbot
DSearch DTS Agent
DataCha0s Digincore
Demon Disco
DigitalPebble Discobot
Dirbuster DnBCrawler-Analytics
Discoverybot DnyzBot
DomCopBot DomainAppender
DomainStatsBot Download Wonder
Dragonfly
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
EasyDL Ebingbong
Ecxi
EroCrawler EroCrawler
FlashGet Express WebPictures
Extractor
FemtosearchBot
Firefox/7.0
Foobot
Freeuploader
FrontPage FrontPage
Fuzz GT::WWW
FyberSpider Genieo
G-i-g-a-b-o-t Getintent
GalaxyBot Gigabot
GermCrawler Go!Zilla
GoZilla
Gotit
GrabNet GrabNet
Grabber Grafula
GrapeFX
GrapeshotCrawler GrapeshotCrawler
GridBot HMView
Haansoft Haansoft
Havij HaosouSpider
Hloader Hloader
HonoluluBot Humanlinks
IDBTE4M HybridBot
IDBot IRLbot
Iblog
IlseBot IlseBot
Image Fetch Image Fetch
Image Sucker Internet Ninja
Indy Library JennyBot
InfoNaviRobot Kenjin Spider
InfoTekies Kinza
Intelliseek Kozmosbot
InterGET Lanshanbot
InternetSeer Larbin
Iria LeechFTP
IstellaBot
Jbrofuzz
Jetty
JikeSpider
Jorgee
JustView
Keyword Density
LNSpiderguy
LWP::Simple
LeechGet
Lftp
LibWeb LibWeb
Lightspeedsystems
Likse Likse
LinkScan
LinkextractorPro
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider MFC_Tear_Sample
Litemage_walker MIDown tool
MJ12bot MIIxpc
MQQBrowser
MSFrontPage
MSIECrawler
MTRobot MTRobot
Mail.RU_Bot Mail.RU_Bot
MarkMonitor Majestic12
Masscan
MauiBot MauiBot
Meanpathbot MeanPath Bot
Metauri Metauri
MicroMessenger Microsoft Data Access
Mojolicious Moblie Safari
Mojeek
MolokaiBot MolokaiBot
Morfeus Fucking Scanner
Msrabot
NICErsPRO NICErsPRO
NPbot NPbot
Name Intelligence Name Intelligence
NetAnts Nameprotect
NetSpider Navroad
NearSite
Nessus
NetZIP
Netcraft Netcraft
Nettrack Nettrack
Netvibes Netvibes
NextGenSearchBot
Nibbler Nibbler
Nmap Nimbostratus
Not Octopus
Offline Explorer OnCrawl
Offline Navigator OpenVAS
Openvas
OrangeSpider
OutclicksBot
PECL::HTTP
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageAnalyzer PageGrabber
PageScorer
Panscient Panscient
Pavuk
PeoplePal
Pi-Monster Pi-Monster
Picsearch PleaseCrawl
Piepmatz Pump
Pimonster
Pixray
ProWebWalker
Probethenet
Pu_iN
PyCurl PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler
RSSingBot RSSingBot
RankActive RankFlex
RankurBot RankingBot2
Rankivabot
RealDownload
Reaper
RebelMouse RebelMouse
Ripper Recorder
RocketCrawler
SBIder SBIder
SEOkicks-Robot SEOkicks
SEOlyticsCrawler SEOprofiler
SEOstats
Scanbot Scanbot
Scrapy Scrapy
Screaming
Searchestate Searchestate
SearchmetricsBot
SemanticJuice
SentiBot SentiBot
SeobilityBot
Seomoz Seomoz
Shodan
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteSucker
Siteimprove Siteimprove
SlySearch SlySearch
SmartDownload
Snake
Snoopy
Sociscraper Sociscraper
SpaceBison Sogou web spider
Spammen Spanner
SpankBot
Spinn3r
Sqlmap Sqlmap
Steeler
Stripper
Sucker Sucker
Sucuri Sucuri
SuperBot
Surfbot
Suzuran Suzuran
Swiftbot Szukacz
T8Abot
TeleportPro TeleportPro
Telesphorep Telesoft
The Intraformant The Intraformant
Thumbor TightTwatBot
Titan Toweyabot
Toata Trendiction
Trendictionbot Trendictionbot
True_Robot
Turingos Turingos
TurnitinBot
Twice
Typhoeus Typhoeus
URLy.Warning Upflow
VelenPublicWebCrawler
VoidEYE
Voil Voil
Voltron
WASALive-Bot WASALive-Bot
WEBDAV Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD WeSEE
Web Auto
Web Collage
Web Sucker
WebCollage
WebEnhancer
Web Fetch Web Fetch
WebReaper Web Pix
Web Sauger
Web Sucker Web Sucker
WebCopier
WebEnhancer
WebFuck
WebGo IS
WebLeacher
WebPix
WebSauger
WebWhacker
WebZIP WebZIP
Webalta WebmasterWorldForumBot
Webshag
Website Quester Website Quester
WebsiteExtractor WebsiteExtractor
Widow WebsiteQuester
WiseGuys Robot Whack
Wonderbot Whacker
Woobot Whatweb
WinHTTrack
Wotbox
Wprecon Wprecon
Zade
ZoomBot ZoomBot
ZoominfoBot
ZumBot
ZyBorg ZyBorg
arquivo.pt archive.org_bot
arquivo-web-crawler
autoemailspider autoemailspider
backlink-check
com.plumanalytics com.plumanalytics
crawler.feedback
crawler4j crawler4j
eCatch facebookscraper
gopher ips-agent
instabid
isitwp.com
iubenda-radar iubenda-radar
linkdexbot lwp-request
lwp-trivial lwp-trivial
mediawords magpie-crawler
meanpathbot
muhstik-scan muhstik-scan
oBot netEstate NE Crawler
page scorer page scorer
pcBrowser
polaris version polaris version
probe-image-size probe-image-size
serpstatbot scalaj-http
sexsearcher seobility
seocompany.store
seoscanners
seostar
sitechecker.pro sitechecker.pro
tAkeOut sogouspider
voyagerx.com trendiction.com
ubermetrics-technologies.com
webgains-bot
webprosbot
x09Mozilla x09Mozilla
xpymep1.exe xpymep1.exe
zauba.io zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
canacopegdl.com b00kmarks.com
hoverboard360.at copro.pw
javxxx18.com dev33.dioniqlabb.se
lazy-z.com divisioncore.com
moivestiy.biz enhand.se
rankingchart.de fuck-paid-share-buttons.xyz
serialsway.ucoz.ru roznica.com.ua
tizanidinehcl4mgtab.blogspot.com sodexo.com
vitoriacabos.com syndicate.fun
weburlopener.com thejournal.ru

930
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0daymusic.org
1001desert.com
101raccoon.ru
10bet.com 10bet.com
123cha.com 1kdailyprofit.me
1j7740kd.website 1xbet4you.com
1millionusd.xyz 3w24.com
24videos.tv 3xforum.ro
2girls1cup-free.com 4pp13.com
34.gs 55wmz.ru
3rbseyes.com 5i2.net
4ur.click 5tey7463.weebly.com
66cpwgln.space 8558l.jobs.net
7makemoneyonline.com DomainStatsBot
83net.jp a14download.com
98oi.ru aa625d84f1587749c1ab011d6f269f7d64.com
aa9d046aab36af4ff182f097f840430d51.com aa8780bb28a1de4eb5bff33c28a218a930.com
aac52006ec82a24e08b665f4db2b5013f7.com adclickthru.net
adloads.net add-add.men
adobereader-free.ru adhome.biz
adsland.men adimmix.com
adultmeetonline.info adultgalls.com
affrh2015.com afora.ru
afftrack001.com agapovdg.ru
alert-hgd.xyz ahrntal.verymes.xyz
alevinefesleri.com alekseevec.ru
amateurgalls.com alibestsale.com
anal-acrobats.com allbizne.co.ua
analnoeporno.tv allforminecraft.ru
angigreene.com allkrim.com
anniemation.com allproblog.com
apiadanaknet-a.akamaihd.net allwomen.info
appfastplay.com amateurmatch.com
anatomiy.com
angry-fermi-7633.arukascloud.io
animaltoplist.com
anjalika.co.in
anlimebel.kiev.ua
aosexkontakte.net
apartmentratings.com
apisecretsaucebi-a.akamaihd.net
apple.com-cleaner.systems
arenda-avtoprokat-krasnodar.ru
arendas.net arendas.net
artefakct.com arraty.altervista.org
asscenihotosidea.blogspot.com artdestshop.eu
atdedinotuho.tk articlesdirectoryme.info
atlant-auto.info asrv-a.akamaihd.net
atlasvkusov.ru
atleticpharm.org atleticpharm.org
atley.eu.pn atmovs.com
aucoinhomes.com ausergrubhof.info
auspolice.com auspolice.com
autoua.top auto4style.ru
avtolombard-krasnodar.com avtointeres.ru
avtovolop.ru awency.com
azartclub.org ayanaspa.com
azlex.uz azartmsl.com
b2b-lounge.com azinoofficial777.ru
babosas.com b-styles.xyz
bankcrediti.ru back.dog
bannerconnect.net belinka.com.ua
bashtime.ru best-coupon-offer.com
bdsmgalls.net bestchoice.cf
beachdriveblog.com bestcurs.org
begalka.xyz bestmobilityscooterstoday.com
belsetirehafi.tk bestplacetobuyeriacta.jw.lt
best-seo-offer.com bestwaystogetpaid.us
bestdooz.com beyan.host.sk
bezcmexa.ru
bezlimitko.xyz bezlimitko.xyz
big-cash.net
biglistofwebsites.com biglistofwebsites.com
binomo.kz bingo8888.com
bio-optomarket.ru bloggen.be
bioca.org blogsfunky672.weebly.com
bioinnovate.co
blobar.org
blogporn.in
blogs.rediff.com
blue-square.biz blue-square.biz
boltalko.xyz bolitgorlo.net
bombla.org boole.onlinewebshop.net
bonvillan.com boom-celebs.com
bookmark4you.com.biz
bosman.pluto.ro bosman.pluto.ro
brg8.com brainboostingsupplements.org
buchananshardware.com brandehk.dk
bristolhotel.com.ua
brus-vsem.ru
buy-meds24.com buy-meds24.com
buyantiviralwp.com cablecar.us
buyk.host.sk callstevens.com
buzzurl.jp candytiner.com
cakemediahq.com.au carmuffler.net
calstaterealty.us cartierring.xyz
cannibalcheerleader.com casinos4dummies.co.uk
cartierreplica.pw catalogs-parts.com
cartierreplica.win cb.iphantom.com
cdncash.com cenokos.ru
cerev.info centrdebut.ru
chatroulette.video cfcl.co.uk
cheapestjerseys-wholesale.com charmstroy.info
cheapmoncler.xyz chatroulette.world
cheatcode-lita12.rhcloud.com cheapjerseysa.com
check-this-out-now.online cheapjerseysbizwholesale.us
cherubinimobili.it chinacheapelitejerseys.com
chinajerseyswholesalecoupons.com chlooe.com
chocolateslim-original.com chocolateslim-en-france.com
cibpenonptib.flu.cc chocolateslim-u-srbiji.com
codq.info chocoslim.pro
colobit.biz christianlouboutinshoes.xyz
compiko.info cindymatches.com
compliance-john.top clickso.com
compliance-julianna.top coldfilm.ru
compliance-fred.top
compliance-ivan.xyz
compliance-nelson.top compliance-nelson.top
continent-e.tv compliance-walt.top
cosmeticswomens-womensports.rhcloud.com conciergegroup.org
cottageofgrace.com converse.ddsoldes.fr
coolyarddecorations.com
copyrightclaims.org
countbertwistdisp26.soup.io
covid-schutzmasken.de covid-schutzmasken.de
cracksplanet.com
craftinsta.ru
crd.clan.su
creatives.adbetclickin.pink
creditmoney.com.ua creditmoney.com.ua
customergrowthsystems.com czeshop.info
daptravel.com dataloading.net
daymusam.com deadlinkchecker.com
decorationspcs.com delitime.info
depositfiles-porn.ga deshevo-nedorogo.ru
detsky-nabytek.info dichvuvesinhhanoi.com
diesel-parts28.ru
digilander.libero.it digilander.libero.it
digital-sale.su dikx.gdn
docsportal.net dmmspy.com
doctormakarova.ru dms-sw.ru
dojki365.online dodlive.mil
downloadme.life dostavka-v-krym.com
dptaughtme.com downloadmefiranaratb1972.xpg.com.br
dtm-spain.com drev.biz
duplicashapp.com drhomes.biz
dwomlink.info drugspowerstore.com
e-buyeasy.com druzhbany.ru
e-c.al dtnlyss.com
e-kwiaciarz.pl dvervmoskvu.ru
eaplay.ru e705.net
easytuningshop.ru eachdayisagift.review
ekosmetyki.net.pl ecommerce-seo.com
el-nation.com
elektrischezigarette2.onsugar.com
elektrischezigarettekaufen2.cowblog.fr
elektrischezigaretten1.blogse.nl elektrischezigaretten1.blogse.nl
elektrozigaretten2.yn.lt embedle.com
elmacho.xyz emerson-rus.ru
eloconcream.blogspot.com empathica.com
eluxer.net este-line.com.ua
englishdictionaryfree.com everypony.ru
enhand.se executehosting.com
epicbrogaming.com exonline.info
ereko.ru extrabot.com
eropornosex.ru ezigarettenshop2.postbit.com
erotische-geschichten-xxl.com falcoware.com
essay-edu.biz faptitans.com
etur.ru fast-wordpress-start.com
eugenevaultstorage.com fba-mexico.com
ewebarticle.info feeriaclub.ru
extads.net filmfanatic.com
extener.org find1friend.com
eyessurgery.ru findclan.org
fakehandbags.xyz finuse.com
family1st.ca firstsiteguide.com
fbdownloader.com fishtauto.ru
feargames.ru floridamobilebillboards.com
fefo.gdn forex21.ru
feorina.ru forum-engineering.ru
fettefrauen.net foto-telok.net
findsexguide.com fototravel.eu
fitfloponline.store franklinfire.co
fivedwld.cf frbizlist.com
flavors.me free-gluten.ru
floridamhca.org freenom.link
freshdz.com freewareseek.com
friendflnder.com froggytube.com
fsalas.com fuckingawesome.com
ftns.ru funcrushgames.com
g.starmoe.xyz gamebackyard.com
gaz-voshod.ru gardene.ru
gearcraft.us gazoblok.net.ua
gepezz.info genericviagrasildenafiled.net
getoutofdebtfree.org ghernnqr.skyrock.com
gidonline.one gilsonchiro.xyz
gifspics.com given2.com
glasof.es gkvector.ru
glcomputers.ru goblacked.com
gobongo.info
goforexvps.com
gogps.me gogps.me
golmau.host.sk goo.ne.jp
growshop.es good-mummy.ru
gsou.cf goodnightjournal.com
guge.io gpms.org.my
guiadeserraazul.com gratuitxblcodes.com
guildebzh.info groupmoney.ru
gwhwpxbw.bloger.index.hr hahashka.ru
gyffu.com hasfun.com
hacktougroup.ru hawaiisurf.com
hanink.biz.ly hdwalls.xyz
hannasolution.ru helvetia.com.ua
happy.new.yeartwit.com hermesbracelets.xyz
harmonyglen.us hermesreplica.pw
hazardky.net hkdiiohi.skyrock.com
heatpower.ru homemature.net
hentai-manga.porn
hermesreplica.win
heygidday.biz
hidemyass.com
highland-homes.com
himgaws.pw
homedo.fabpage.com
honyaku.yahoofs.jp honyaku.yahoofs.jp
hop.clickbank.net hostingclub.lk
hostnow.men hotcore.info
hotloans.ru hotenergy.ru
houtings.xyz hotsocialz.com
hoverboard360.at humanorightswatch.org
hoverboard360.nl
humbmosquina.tk
hunterboots.online hunterboots.online
idealtits.net ihtec2019.org
ifmo.ru ilmexico.com
iframe-toloka.com imfamous.info
imgarcade.com imicrovision.com
imk.com.ua inclk.com
imperia31.ru increasewwwtraffic.info
indiakino.net inet-traffic.com
infowarcraft.ru interferencer.ru
instasexyblog.com investingclub.ru
int.search.tb.ask.com iplusbit.blogspot.co.za
intervsem.ru ipostroika.ru
itrevolution.cf iscblog.info
janerikholst.se islandminingsupply.wordpress.com
javidol.site istock-mebel.ru
javstock.com itag.pw
jolic2.com itch.io
justdating.online ix20.ru
jwcialislrt.com j-times.ru
kaidalibor.de je7.us
keyhantercume.com jo24news.com
keywordhouse.com jose.mulinohouse.co
keywordhut.com junglenet-a.akamaihd.net
keywordsking.com justucalling32211123456789.tk
kino2018.cc kabbalah-red-bracelets.com
kinozapas.org kam-dom.ru
kleine-titten.biz kartiniresto.com
kerei.ru
kino-rf.ru
kino2018.club
kmd-pto.ru kmd-pto.ru
knowyournextmove.com knowyournextmove.com
komputernaya-pomosh-moscow.ru kollesa.ru
kongoultry.net konteiner24.com
konkursowo-24.pl koral.se
kosova.de la-fa.ru
koversite.info lacloop.info
kovesszucs.atw.hu laexotic.com
krasivye-devushki.net landinez.co
laapp.com lapolis.it
lafourmiliaire.com law-check-eight.xyz
landmania.ru law-check-seven.xyz
lasvegaslockandsafe.com layola.biz.tc
ledpolice.ru leftofcentrist.com
legalrc.biz letsrepair.in
lescinq.com levaquin750mg.blogspot.com
likesuccess.com lietaer.com
link.ac lincolntheatre.com
linkarena.com lineshops.biz
lost-alpha.ru lion.snapmap.us
louboutinreplica.xyz lirunet.ru
lyrster.com listiseltemournlan.gq
m-google.xyz loginduepunti.it
m292.info louboutinshoes.xyz
m3gadownload.pl luxup.ru
macfix.life lyrics.home-task.com
machicon-akihabara.info
magicplayer-s.acestream.net magicplayer-s.acestream.net
mamasuper.prom.ua maglid.ru
marblestyle.ru marinetraffic.com
marcowebonyodziezowe.pl marketingtechniques.info
marmotstore.online massage-info.nl
marwer.info masturbate.co.uk
matb3aa.com mearns-tractors.co.uk
me-ke.com mebel-alait.ru
meddesk.ru medi-fitt.hu
medicines-choice.com medmajor.ru
meds-online24.com medosmotr-ufa.ru
micasainvest.com megaindex.ru
mikrobiologies.ru mere.host.sk
minecraft-rus.org mesto-x.com
mir-betting.ru mex-annushka.ru
mirzonru.net mexicosleevegastrectomy.com
mmog-play.ru michaelkorsoutlet.store
mobioffertrck.com mindeyegames.com
moesexy.com monetizer.com-01.site
moinozhki.com mrinsidesales.com
monclerjacketsoutlet.pw mtmtv.info
monetizationking.net music7s.me
moneyzzz.ru mycouponizemac.com
moscow-clining.ru mydoctorok.ru
mostantikor.ru mydownloadengine.com
moviezbonkerssk.cf mymobilemoneypages.com
mulberryoutletonlineeu.com myonigroup.com
musezone.ru myphotopipe.com
musicspire.online na15.ru
mustat.com nailsimg.com
mustwineblog.com nakrutka.cc
mydownlodablefiles.com national-today-winning-winner.club
myiptest.com netcheckcdn.xyz
networkcheck.xyz nhl17coinsforps3.gratisblog.biz
newfilmsonline.ru
nextlnk12.com
ngps1.ru
nina.az
noclegonline.info noclegonline.info
novosibirsk.xrus.org nodding-passion.tk
nsatc.net nodup.ru
nufaq.com notasprensa.info
o00.in nucia.biz.ly
obiavo.in o-o-8-o-o.com
o-o-8-o-o.ru
o333o.com
obiavo.su obiavo.su
obiavo.uz odoratus.net
oconto.ru oneclickfiles.com
offer.camp online-hd.pl
offergroup.info online-hit.info
office-windows.ru onlineserialy.ru
okmedia.sk optitrade24.com
omsk.xrus.org orion-code-access.net
onclkads.com ovirus.ru
onlinewritingjobs17.blogspot.ru pacificair.com
onlyforemont.ru panamaforbeginners.com
openmediasoft.com panicatack.com
ownshop.cf par-fallen.ga
owohho.com partner-host.men
partnerline.men partner-stop.men
partnersafe.men patol01.pw
paydayloanslocal.com pc-services.ru
pc-virus-d0l92j2.pw pc4download.co
pdns.download pcgroup.com.uy
perl.dp.ua pdfprof.com
photo.houseofgaga.ru perederni.net
photostudiolightings.com perm.xrus.org
pix-hd.com photokitchendesign.com
pixell.club pic2fly.com
pizza-imperia.com pivka.xyz
play-mp3.com pixshark.com
playfortuna-play.ru pk-services.ru
playtap.us pod-muzyku.club
plohaya-kreditnaya-istoriya.ru
plugingeorgia.com
podshipniki-ntn.ru podshipniki-ntn.ru
poligon.com pojdelo.weebly.com
popander.mobi polytopesexempt.com
popup-fdm.xyz popserve.adscpm.net
pornogad.com poptool.net
pornoklad.net popup.matchmaker.com
prize44.com pornhub-forum.ga
professionaldieselcare.com pornmania.pl
promoheads.com pornoklad.ru
puzo2arbuza.ru pornosok.ru
quickchange.cc pornotubs.com
randki-sex.com positive2b.ru
razamicroelectronics.com presleycollectibles.com
refads.pro privatefx-in.ru
registry-cleaner.net privatefx.all4invest.info
remmling.de prombudpostach.com.ua
remontvsamare.su proxyelite.biz
remorcicomerciale.ro pst2017.onlinewebshop.net
resultshub-a.akamaihd.net ptr.ruvds.com
rezeptiblud.ru pxhdwsm.com
rome2rio.com quebec-bin.com
royalcar-ufa.ru quelle.ru
rukino.org questionmarque.ch
ruscoininvest.company rankings-analytics.com
rutor.group recinziireale.com
s-forum.biz reckonstat.info
sammyweaver.com referencemoi.com
savememoney.co.za reward-survey.net
savetubevideo.com rewardpoll.com
scanner-elena.top reyel1985.webnode.fr
scanner-margo.top ru-mediaget.ru
scanner-olga.top ruspdd.com
scanspyware.net savefrom.com
seadragonherbery.com scanner-donald.top
scanner-ivan.top
scanner-jessica.top
scanner-mary.top
scottbywater.com
sdrescher.net
searchimpression.com
secret.xn--oogle-wmc.com secret.xn--oogle-wmc.com
seorankinglinks.us seinterface.com
serialsway.ucoz.ru seolab.top
service.adtech.us serptehnika.ru
sex-znakomstva.online sevendays.com.ua
sexblog.pw sexpornotales.com
sexfreepornoxxx.com sheerseo.com
shemale-sex.net shemalegalls.blogporn.in
shijian.ac.cn shijian.ac.cn
shitting.pro
shivafurnishings.com
shodanhq.com shodanhq.com
shoplvlv.us sinestesia.host.sk
shymkent.xkaz.org sitevaluation.org
silvermature.net sivs.ru
site5.com sixcooler.de
siteonomy.com smartshoppymac.com
sitopreferito.it snegozaderzhatel.ru
skanninge.se socialbuttons.xyz
sladkoevideo.com socialmediasuggest.com
slayerlife.com souvenir.cc
small-games.biz space2019.top
smart-balancewheel.com spb.afora.ru
sneakyboy.com spb.ru
snjatie-geroinovoy-lomki.ru spy-app.info
sobecjvuwa.com.ru spywarebegone.com
socialseet.ru srdrvp.com
soheavyblog.com steelmaster.lv
sohoindia.net stillmiracle.com
solitaire-game.ru stmassage.ru
sonnikforme.ru stoki.ru
soochi.co stretchmate.net
sovinsteel.ru superiends.org
spanking.to superoboi.com.ua
sstroy44.ru
star61.de
steebook.com
stopnarco.ru
studentguide.ru
studiofmp.com
studiokamyk.com.pl
sudexpert66.ru
superlist.biz
surffoundation.nl surffoundation.nl
surgut.zrus.org svtrd.com
symphonyintegratedhealthcare.com taaaak.com
tds-advert002.info taboola.com
tattoo33.ru
technopellet.gr
teencastingporn.com
telefonsexi.com
telegramdownload10.com telegramdownload10.com
testbotprocessor44.com terraclicks.com
theautoprofit.ml tetracsaudi.com
thebluenoodle.com textads.men
thesmartsearch.net timeallnews.ru
tizanidinepills.blogspot.com titangel-vietnam.com
tizanidinevssoma.blogspot.com tizanidinehcl4mgtab.blogspot.com
tomatis.gospartner.com tizanidinehcl4mgtablet.blogspot.com
tonivedu.it tizanidinepain.blogspot.com
touchmods.fr tizanidineshowupondrugtest.blogspot.com
tozup.com tnaionline.org
tobeyouday.win
toon-families.com
tootoo.to
topads.men
tour-line.net
trackzapper.com trackzapper.com
trafficjunky.net trafficmania.com
trafficreceiver.club traxdom.ru
ttrraacckkrr.com triumf-realty.ru
tuberkuleznik.ru tula.howotorg.ru
turbo-suslik.org
tvteleport.ru tvteleport.ru
uasb.ru typimga.pw
udayavani.com uchil.net
uggsale.online ucsol.ru
uginekologa.com um-razum.ru
ukrtvory.in.ua umekana.ru
umityangin.net unisexjewelry.org
unrealcommander.com unitygame3d.com
unmaroll.ya.ru
uptime-gamma.net uptime-gamma.net
urlcut.ru uroffer.link
utiblog.fr uzungil.com
vacances-voyages.info vacuumscleaner.com
vandrie-ict.nl
varbergsvind.se varbergsvind.se
vchulkax.com videochaty.ru
vduplo.ru videogamesecrets.com
vertaform.com vikistars.com
viagengrarx.com
videojam.tv
videositename.com
videotuber.ru
vintontech.info
vip2ch.com
vizitki.net
vkak.ru vkak.ru
vkonche.com vltai.com
voditeltrezviy.ru voltrknc1.com
voloslove.ru vonradio.com
volume-pills.biz vozbujdenie.com
vote-up.ru vvon.co.uk
vse-pesni.com wanker.us
vseigry.fun web-betting.ru
wallpapersdesk.info welck.octopis.com
website-analyzer.info wetgames.ru
website-speed-check.site wma-x.com
wesharepics.info woman-orgasm.ru
whereiskentoday.com womens-journal.net
wholesalejerseychinashop.com worldhistory.biz
wiosenny-bon-1500.pl worldis.me
www.ehscloud.cn wowas31.ucoz.ru
wwwadultcheck.com wrz0iuebwhp5fg.freeddns.com
wyniki-lista.pl wufak.com
x-lime.com x-mix.info
xcombear.ru xmlinde.com
xep.info xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xev.ru xn--b1addnj3cah.xn--p1ai
xgames-04.com xn--h1ahbi.com.ua
xlolitka.com xn--oogle-wmc.com
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai xxart.ru
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai xxxnatelefon.ru
xn----7sbaphztdjeboffeiof6c.xn--p1ai yadro.ru
xn--80aafb2a.xn--p1ai you-shall-not-pass.is74.ru
xn--80ahvj9e.xn--p1ai yoursite.com
xn--h1aakne2ba.xn--p1ai z-master.ru
xnxxandxvideos.com za-fun-offer.com
xsfetish.org zajm-zalog-krasnodar.ru
xvideosbay.com zarabotok--doma.ru
xxxkaz.org
yginekologa.com
yoluxuryevents.com
yougotanewdomain.com
yourporn.com
yourporngay.com
za-music.mymobiplanet.com
zaapplesales.blogspot.com
zaloro.com
zarenica.net zarenica.net
zeikopay.com zerocash.msk.ru
zeroredirect9.com zkjovpdgxivg.ga
zoolubimets.ru zog.link
zophim.me zolotoy-lis.ru
zona-aqua.ru
zrizvtrnpale.tk
zs2vm.top zs2vm.top

474
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
6hopping.com 10bet.com
7wind.ru 2x2fan.ru
abiente.ru 5000-cotydzien.com
abovetherivernc.com 91abcw.com
adimmix.com affrh2015.com
adrenalin-stk.ru alf-img.com
ads.gold alfa-img.com
adultactioncam.com all4invest.info
advocatemsk.ru alveris.ru
akita.kz andpolice.com
albamargroup.com anime.dougasouko.com
albuteroli.com aosheng-tech.com
all-streaming-media.com apibetweenlinesn-a.akamaihd.net
amehdaily.com arcteryxsale.online
anonymous-redirect.com ardimobileinfo.ml
anti-virus-removal.info audiofree.ru
apidiamondatanet-a.akamaihd.net autorn.ru
apiluckyleapnet-a.akamaihd.net aviav.ru
apimountainbikei-a.akamaihd.net aviva-limoux.com
apishops.ru avto-oligarh.ru
apiwebwebgetcom-a.akamaihd.net balkanfarma.ru
appfastplay.com beauty-b0x.pl
aramaicmedia.org bestbrainprod.win
atleticpharm.org bestcurs.org
aussie-prizes.com bestempresas.es
autotop.com.ua bestplacetobuyeriacta.jw.lt
autowebmarket.com.ua bio-market.kz
azbaseimages.net biteg.xyz
beztuberkuleza.ru bizlist.com.de
big-cash.net blog.remote-computer.de
blogqpot.com bobinoz.com
bookmark4you.com.biz bombla.org
bouda.kvalitne.cz bonanza-fortune.men
breget74.ru bonux.nextview.ru
briomotor.co brandehk.dk
buron.pw brothers-smaller.ru
buyclomidonlaine.com browserprotecter.com
buyhoverboard.com cacheimages.com
buyk.host.sk camdolls.com
buypillsorderonline.com canvas.gloverid.site
byme.se carloans.com
candlewooddentalcentre.com.au casinorewards.info
cartierbraceletsreplica.pw cbcseward.com
catalogs-parts.com checkpagerank.net
cdnanalytics.xyz clksite.com
centrdebut.ru coccoc.com
check-host.net coin-hive.com
ciproandtizanidine.blogspot.com cookie-law-enforcement-aa.xyz
cityadspix.com cookie-law-enforcement-hh.xyz
clipartnew.com credit-online.ws
clmforexeu.com darkbooks.org
clothesforcash.com datadepths.com
club-musics.ru deadlinkchecker.com
compliance-elena.xyz decorazilla.com
compliance-jack.top detskie-konstruktory.ru
cubook.supernew.org dignitasdata.se
cubs-jerseys.us domcran.net
customerpromos-a.akamaihd.net doreenblog.online
cxpromote.com dosugrostov.site
danschawbel.com download-wallpaper.net
delfin-aqua.com.ua dpihatinh.gov.vn
deluxedumps.com e-buyeasy.com
detective01.ru e-kwiaciarz.pl
dev78.dioniqlabb.se ege-essay.ru
digital-video-processing.com elektrischezi.canalblog.com
dikx.gdn elektrischezigarette1.blog.pl
div.as eloxal.ru
domain.webkeyit.com eng-lyrics.com
donna7753191.ru eralph.tk
dosug-lux.ru erotik-kostenlos.net
drev.biz ezigarettenkaufen1.deviantart.com
e705.net fable.in.ua
ecommerce-seo.org facecup.top
edwinkonijn.com.au factorynightclub.com
elektrozigaretten2.yn.lt ferretsoft.com
epicbrogaming.com fetlifeblog.com
eropornosex.ru filmi-onlain.info
errorfixing.space financehint.eu
escortslet.net
eu-cookie-law.blogspot.com
f0815.de
farmingworm.com
fashionindeed.ml
ferrotodo.com
findanysex.com
findthe.pet
flavors.me flavors.me
folowsite.com food.dtu.dk
fotopop.club foto-telok.net
franecki.net free-share-buttons.top
freelifetimefuckbook.com freshsuperbloop.com
freelotto.com frvo.alptandem.ru
frequiry.com fsalas.com
fuel-gas.com fugarif.ga
gay.adultgalls.com fym.com.ru
gaz-voshod.ru goodprotein.ru
get-your-social-buttons.info gorodservis.ru
glass-msk.ru grupografico-pilar.com.ar
glcomputers.ru guod.me
goodwinmetals.co gymi.name
googglet.com hd-film.pl
googlefeud.com hdwalls.xyz
gox.com.ua helpmymacfaster.trade
gta-top.ru highland-homes.com
healmytrauma.info hoterika.com
hoporno.com hoverboard360.es
housemilan.ru hoverboard360.se
houseofgaga.ru i4track.net
hoztorg-opt.ru ifmo.ru
hvd-store.com ilbe.club
hyipmanager.in ilmexico.com
iideaidekonkatu.info
images.gyffu.com images.gyffu.com
imediadesk.com imgata.com
imgarcade.com internet-apteka.ru
inbabes.sexushost.com irkutsk.zrus.org
internetproviderstucson.com istizanidinelikexanax.blogspot.com
intervsem.ru janettabridal.com
istock-mebel.ru karusel-market.ru
iwanttodeliver.com keywordsking.com
japfm.com kidskunst.info
javrip.net
journalhome.com
keki.info
klikbonus.com
kndxbkdx.bloger.index.hr kndxbkdx.bloger.index.hr
krasnodar.zrus.org
krynica.info krynica.info
lanadelreyfans.us ladsup.com
larger.io landmania.ru
letmacworkfaster.world laxdrills.com
lesbian.xyz
lflash.ru
linkarena.com
liquimondo.com liquimondo.com
locimge.pw littleberry.ru
lumb.co locksmith.jp
magnetpress.sk lonerangergames.com
manipulyator-peterburg.ru m.b00kmarks.com
manualterap.roleforum.ru m0r0zk0-krava.ru
manve.info mafa-free.com
medicinacom.ru meteocast.net
metaxalonevstizanidine.blogspot.com meteostate.com
methodsmarketing.com mexicosleevegastrectomy.com
moskva.nodup.ru michaelkorssaleoutletonline.net
most-kerch.org mindbox.co.za
mrcsa.com.au minneapoliscopiers.com
mttwtrack.com monetizationking.net
muabancantho.info mriyadh.com
mxgetcode.com mxgetcode.com
myphotopipe.com mydownlodablefiles.com
mzdish.site narutonaruto.ru
nacap.ru negociosdasha.com
nailsimg.com neobux-bg.info
newyorkhotelsmotels.info netpics.org
new-apps.ru
new7ob.com
nextlnk12.com
nofreezingmac.click nofreezingmac.click
nrv.co.za northfacestore.online
o-o-6-o-o.ru norththeface.store
oballergiya.ru notfastfood.ru
ok-ua.info nudejapan.net
okuos.com obesidadealgarve.com
onlinebay.ru oksrv.com
onlinetvseries.me onlythegames.com
onlinewritingjobs17.blogspot.ru optom-deshevo.ru
openfrost.net oren-cats.ru
optibuymac.com origin-my.ru
paighambot.com paighambot.com
paramountmarble.co.uk
partner-pop.men
patol01.pw patol01.pw
petrovka-online.com petsblogroll.com
pharmacyincity.com phrcialiled.com
pisanieprac.info physfunc.ru
pk-services.ru plugingeorgia.com
plastgran.com poiskzakona.ru
plastweb.ru poptool.net
poisk-zakona.ru pospr.waw.pl
popads.net potolokelekor.ru
pornoxxx.com.mx privacyassistant.net
potoideas.us privatefx-in.ru
predmety.in.ua profolan.pl
promgirldresses.xyz prostitutki-rostova.org
pronorm.fr proxyelite.biz
putevka24.ru pst2017.onlinewebshop.net
py100.ru pulse33.ru
rcpmda.ikan1080.xyz recordpage-a.akamaihd.net
realmonte.net redirectme.net
rethinkwasteni.info rent2spb.ru
rewardit.com ric.info
rusexy.xyz sakhboard.ru
sandhillsonline.com scoopquest.com
santasgift.ml searchtooknow-a.akamaihd.net
scanner-nelson.top semaltmedia.com
search.1and1.com seolab.top
searchinquire.com serw.clicksor.com
seasonvar.ru shiksabd.com
seksvideoonlain.com simplesite.com
seoheap.com site-speed-check.site
sex-watch.com sk.golden-praga.ru
sklad-24.ru skladvaz.ru
sluganarodu.ru skypasss.com
slowmacfaster.trade
smartbalanceworld.com
social-fun.ru social-fun.ru
sosdepotdebilan.com socialbutton.xyz
softonicads.com
sotechco.co sotechco.co
spylog.com spasswelt.xyz
stair.registrydomainservices.com spy-sts.com
stairliftsarea.com stanthonyscatholicchurch.org
streetfire.net stillmiracle.com
stromerrealty.com stpicks.com
superinterstitial.com styro.ru
surgut.zrus.org tacbelarus.ru
swimpool.ca tattooreligion.ru
tattooha.com
tds-advert002.info tds-advert002.info
telegraf.by thecounter.com
the-trader.net thejournal.ru
thebestweightlosspills.ovh thenetinfo.com
thebluffs.com tilido.com
tizanidinehcl2mg.blogspot.com tizanidineduringpregnancy.blogspot.com
tizanidinewithdrawalsymptoms.blogspot.com tizanidinehcl4mgisitanarcotic.blogspot.com
topappspro.com topappspro.com
track2.shop torrentred.games
triumf-realty.ru toy-shop.top
tsstcorpcddvdwshbbdriverfb.aircus.com track-rankings.online
tuberkuleznik.ru tracklead.net
tversvet.ru truck-land.life
twu.com.ua tryrating.com
uginekologa.com tuberkulezanet.ru
umaseh.com tyumen.xrus.org
universals.com.ua uamtrk.com
usiad.net ukrtextbook.com
uzungil.com uogonline.com
video-chat.cn urengoy.pro
video-chat.in urlcut.ru
visa-pasport.ru uytmaster.ru
volume-pills.biz vapeface.club
voluumtrk.com vitalads.net
webalan.ru vkgaleria.com
webscouter.net vozbujdenie.com
website-audit.com.ua vzlom-na-zakaz.com
womens-journal.net wallpapersist.com
www.labves.ru wdrake.com
wzgyyq.com webcamdevochka.com
x-diesel.com webradiology.ru
x5market.ru welcomeauto.ru
xaylapdiendanang.com whatsappbot.flyland.ru
xchangetrak.com wholesalejerseychinashop.com
xn--80aafb2a.xn--p1ai wieseversa.no
xn--80aodinpgi.xn--p1ai winner7777.net
xrus.org worldmusicfests.com
xvideosporn.biz www.event-tracking.com
znakomstva-moskva77.ru xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
zoogdinsney.com xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
zzlgxh.com xn----7sbahjd3btneuw1joc.xn--p1ai
xn--c1acygb.xn--p1ai
xxxsiterips.xyz
yaderenergy.ru
yhit.press
yur-p.ru
zoosexart.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.5 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3465 ### Version: V4.2022.11.3466
### Updated: Fri Nov 11 22:01:14 UTC 2022 ### Updated: Sat Nov 12 10:03:15 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3465 ### Version: V4.2022.11.3466
### Updated: Fri Nov 11 22:01:14 UTC 2022 ### Updated: Sat Nov 12 10:03:15 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##