vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.08.3279 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-08-09 10:09:26 +00:00
parent 6e2729e800
commit 1fa34095e4
46 changed files with 88435 additions and 88435 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.08.3278 #### Version: V4.2022.08.3279
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.08.3278 #### Version: V4.2022.08.3279
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.08.3278 #### Version: V4.2022.08.3279
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10920
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.08.3278 #### Version: V4.2022.08.3279
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10924
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10924
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10924
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10924
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10924
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10924
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10924
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3278 3279

10924
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10924
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.12.224.232 0; 20.163.49.80 0;

10924
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10924
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10924
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10924
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.12.224.232 0; 20.163.49.80 0;

10924
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.12.224.232 0; 20.163.49.80 0;

10924
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BLEXBot AllSubmitter
Gigabot FyberSpider
JamesBOT HTTP::Lite
Majestic SEO IndeedBot
OpenLinkProfiler InfoNaviRobot
Vagabondo Vacuum
VidibleScraper WWW::Mechanize
archive.org_bot WallpapersHD
linkdexbot WebsiteQuester
pcBrowser ZyBorg

306
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 01h4x.com
404checker ADmantX
80legs Aboundex
AIBOT
Abonti
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot
Aipbot Aipbot
Alexibot AllSubmitter
Ankit Alligator
Anarchy
Anthill Anthill
AwarioRssBot Apexoo
AwarioSmartBot Aspiegel
BBBike Asterias
BLEXBot BDCbot
BackStreet BackWeb
Badass Backlink-Ceck
BacklinkCrawler
Barkrowler
BetaBot BetaBot
Black Hole BlowFish
BlackWidow
Blow
Boardreader Boardreader
Bolt
Brandwatch
Buck
BuiltBotTough
BuiltWith BuiltWith
BunnySlippers BuzzSumo
CSHttp CCBot
CensysInspect Calculon
CheTeam CheTeam
CheeseBot CheeseBot
CherryPicker CherryPicker
ChinaClaw ChinaClaw
Chlooe Citoid
Claritybot Collector
Cliqzbot Craftbot
Cocolyzebot CrunchBot
Copyscape Curious
CyotekWebCopy
DIIbot DIIbot
DSearch
DTS Agent DTS Agent
Demon DataCha0s
Discoverybot DatabaseDriverMysqli
Devil
Discobot
Dispatch Dispatch
DomainAppender DnBCrawler-Analytics
Dragonfly DomainCrawler
Dotbot
Drip
ECCP/1.0
EMail Siphon EMail Siphon
EasyDL EMail Wolf
Ebingbong Ebingbong
Ecxi Ecxi
EirGrabber Evil
Express WebPictures Exabot
ExtractorPro
Extreme Picture Finder
Ezooms
FHscan FHscan
FemtosearchBot FemtosearchBot
Foobot FlashGet
GetRight Freeuploader
FrontPage
G-i-g-a-b-o-t
GermCrawler
Getintent
Gigabot
GoZilla GoZilla
Gotit GrabNet
Grabber
Grafula
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
HTMLparser
HTTP::Lite HTTP::Lite
HTTrack HaosouSpider
Hloader Heritrix
HybridBot Humanlinks
IDBTE4M IDBTE4M
IRLbot
Iblog Iblog
IndeedBot IlseBot
Image Fetch
InfoTekies
InterGET InterGET
Internet Ninja
Iria Iria
IstellaBot
JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz Jbrofuzz
Jetty Jetty
Joomla
JustView
Kenjin Spider Kenjin Spider
Kinza LNSpiderguy
Leap Larbin
LexiBot LeechGet
LibWeb Libwhisker
Likse
LinkWalker LinkWalker
LinqiaMetadataDownloaderBot Linkbot
Lipperhey LinkpadBot
Lipperhey Spider LinksManager
Ltx71 LinqiaScrapeBot
Litemage_walker
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc
MJ12bot
MSFrontPage MSFrontPage
MTRobot
Magnet
Mail.RU_Bot
Majestic SEO
Mata Hari Mata Hari
MauiBot MauiBot
Mb2345Browser MeanPath Bot
Meanpathbot
MegaIndex.ru
Metauri Metauri
Microsoft URL Control Moblie Safari
Mojeek
MolokaiBot MolokaiBot
Morfeus Fucking Scanner
Mozlila Mozlila
Msrabot
Musobot Musobot
NPbot NICErsPRO
Name Intelligence
Navroad
Nessus Nessus
Net Vampire Net Vampire
NetLyzer NetSpider
Nettrack Nettrack
Netvibes Niki-bot
Nibbler
Nikto
Ninja Ninja
Nuclei Not
Nutch
Octopus
Offline Explorer Offline Explorer
Offline Navigator
OnCrawl
OpenVAS
Openfind Openfind
OrangeBot OrangeBot
OrangeSpider
OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl PHPCrawl
POE-Component-Client-HTTP
PageAnalyzer
PageGrabber
PageThing.com
Pandalytics Pandalytics
Petalbot Panscient
Picscout Papa Foto
PictureFinder PleaseCrawl
Piepmatz Pockey
ProPowerBot Probethenet
Pu_iN Pu_iN
Pump PyCurl
RankActive RankActive
RankingBot RankActiveLinkBot
RankingBot2
Rankivabot
RankurBot RankurBot
RealDownload ReGet
Recorder
RedesScrapy
RepoMonkey RepoMonkey
Rogerbot RocketCrawler
SEOprofiler SEOkicks-Robot
SEOlyticsCrawler
SEOstats SEOstats
SISTRIX SISTRIX
SMTBot SMTBot
SalesIntelligent
ScanAlert ScanAlert
Scanbot Scrapy
ScoutJet Screaming
ScreenerBot
ScrepyBot ScrepyBot
SearchmetricsBot
Seekport
SemanticJuice SemanticJuice
SeobilityBot Semrush
SemrushBot
SeoSiteCheckup
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteSnagger SiteSnagger
SiteSucker SiteSucker
Siteimprove Snake
Sitevigil Snapbot
Sogou web spider
Sosospider
Sottopop Sottopop
SpaceBison Spanner
SpankBot Spbot
Spinn3r Spinn3r
SputnikBot Sqlworm
Sqworm Sqworm
Sucker
Sucuri
Swiftbot Swiftbot
T0PHackTeam T0PHackTeam
Telesoft T8Abot
Titan Telesphoreo
Toweyabot The Intraformant
TightTwatBot
Tracemyfile
True_Robot True_Robot
TurnitinBot Turingos
TwengaBot TwengaBot
Typhoeus Typhoeus
URLy Warning URLy Warning
URLy.Warning URLy.Warning
V-BOT
VB Project VB Project
VelenPublicWebCrawler VCI
VeriCiteCrawler
Voil Voil
Voltron
WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV WISENutbot
WPScan WWW-Collector-E
WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
Web Fetch WallpapersHD
WeSEE
Web Auto
Web Fuck Web Fuck
Web Pix Web Pix
Web Sauger Web Sucker
WebCollage WebAuto
WebCopier
WebEnhancer WebEnhancer
WebGo IS WebFuck
WebImageCollector WebLeacher
WebReaper WebSauger
WebStripper
WebSucker WebSucker
WebWhacker WebmasterWorldForumBot
Webalta WebsiteExtractor
Webshag
Webster
Whack
Whacker
Whatweb Whatweb
WinHTTrack WinHTTrack
WiseGuys Robot WiseGuys Robot
Wotbox Wotbox
Xaldon_WebSpider Wprecon
Zade Xenu
Zermelo YoudaoBot
ZmEu Zauba
ZumBot Zeus
archive.org_bot ZoomBot
arquivo-web-crawler ZoominfoBot
autoemailspider ZyBorg
backlink-check adscanner
check1.exe check1.exe
clark-crawler
cognitiveseo
crawl.sogou.com crawl.sogou.com
facebookscraper crawler.feedback
evc-batch
gopher gopher
heritrix heritrix
instabid iubenda-radar
internetVista monitor linkdexbot
isitwp.com
lwp-request lwp-request
meanpathbot netEstate NE Crawler
muhstik-scan pcBrowser
oBot
page scorer
polaris version
probe-image-size
s1z.ru s1z.ru
satoristudio.net satoristudio.net
scalaj-http
scan.lol scan.lol
seocompany.store seobility
seoscanners
siteripz
sogouspider sogouspider
spyfu
sysscan
voyagerx.com voyagerx.com
webgains-bot webgains-bot
webpros.com
webprosbot webprosbot
x22Mozilla
xpymep1.exe xpymep1.exe
zauba.io
zgrab

292
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider
404checker 404checker
404enemy
AIBOT AIBOT
ASPSeek ASPSeek
Aboundex Aboundex
Aboundexbot Aboundexbot
Acunetix Acunetix
AhrefsBot
AiHitBot AiHitBot
AllSubmitter
Alligator Alligator
Ankit Anarchy
Apexoo Anarchy99
Attach Anthill
Asterias
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BDFetch BDFetch
BLEXBot
BackDoorBot
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler Bandit
Barkrowler Battleztar Bazinga
BatchFTP
Bigfoot Bigfoot
BlackWidow Black Hole
Blackboard Blackboard
Blow
BlowFish BlowFish
Boardreader Boardreader
Brandprotect Brandprotect
Brandwatch
Buddy
Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CATExplorador CODE87
CCBot
CSHttp CSHttp
Cegbfeieh Cegbfeieh
CherryPicker CensysInspect
ChinaClaw ChinaClaw
Citoid
Claritybot Claritybot
Cliqzbot Cloud mapping
Collector Cogentbot
Copier CopyRightCheck
Craftbot Copyscape
Cosmos
Crawling at Home Project Crawling at Home Project
Custo Custo
DSearch DBLBot
DTS Agent Devil
DataCha0s
DatabaseDriverMysqli
Demon
Deusu
Dirbuster
Disco Disco
Dispatch DittoSpyder
DomCopBot DnBCrawler-Analytics
DomainStatsBot DomainCrawler
Domains Project
Dotbot Dotbot
Download Wonder
Dragonfly Dragonfly
ECCP/1.0
EMail Siphon
EasyDL EasyDL
Ebingbong
EroCrawler EroCrawler
Evil
Exabot Exabot
Express WebPictures
ExtLinksBot
Extractor Extractor
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
Foobot EyeNetIE
Ezooms
FHscan
FemtosearchBot
Firefox/7.0
FlashGet
FyberSpider FyberSpider
G-i-g-a-b-o-t Fyrebot
GT::WWW GT::WWW
GalaxyBot GrabNet
Genieo Grabber
GermCrawler HEADMasterSEO
Getintent
Go!Zilla
Go-Ahead-Got-It
GrapeshotCrawler
HTMLparser
Haansoft Haansoft
HaosouSpider
Heritrix Heritrix
HonoluluBot Humanlinks
IDBTE4M
IRLbot IRLbot
IlseBot
InfoNaviRobot InfoNaviRobot
InfoTekies InterGET
Intelliseek
Iria
Iskanie Iskanie
IstellaBot
JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
JetCar JetCar
JikeSpider
Jyxobot Jyxobot
Kenjin Spider Kenjin Spider
Keybot Translation-Search-Machine
Keyword Density
Kinza Kinza
Lanshanbot Lanshanbot
Larbin Larbin
LeechFTP LeechFTP
LeechGet LeechGet
Libwhisker LexiBot
LieBaoFast Lightspeedsystems
LinkWalker LinkWalker
LinkextractorPro Linkbot
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Ltx71 Litemage_walker
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MIDown tool
Majestic12 MQQBrowser
MarkMonitor MSFrontPage
Mass Downloader Mag-Net
MarkWatch
MauiBot MauiBot
Mediatoolkitbot MegaIndex.ru
MicroMessenger MicroMessenger
Microsoft URL Control Microsoft Data Access
Mister PiX Mister PiX
Moblie Safari Moblie Safari
Mr.4x3 Mojeek
Musobot Mojolicious
Morfeus Fucking Scanner
NICErsPRO NICErsPRO
NPbot
NearSite NearSite
Net Vampire Net Vampire
NetAnts NetAnts
NetSpider NetLyzer
Netcraft NetMechanic
Nettrack Netvibes
Nibbler Nibbler
Niki-bot
Nikto
NimbleCrawler
Ninja Ninja
Nmap Nuclei
Not
Nutch
Octopus
Offline Navigator Offline Navigator
OpenVAS OpenLinkProfiler
Openfind Openfind
OutfoxBot Openvas
PHPCrawl OrangeSpider
POE-Component-Client-HTTP OutclicksBot
PageAnalyzer PECL::HTTP
PageScorer Page Analyzer
PageThing.com Panscient
Pandalytics Pi-Monster
Pavuk
PeoplePal
PictureFinder PictureFinder
Piepmatz PleaseCrawl
Pixray ProWebWalker
Pockey Pump
ProPowerBot PyCurl
Probethenet QueryN Metasearch
PxBroker
RSSingBot RSSingBot
RankActive RankActiveLinkBot
RankingBot2 RankingBot2
Rankivabot
RankurBot
Re-re Re-re
ReGet RebelMouse
RealDownload
Reaper
RedesScrapy
RepoMonkey
Ripper
SBIder
SEOkicks SEOkicks
SEOlyticsCrawler SEOkicks-Robot
Screaming Scanbot
ScreenerBot ScreenerBot
Searchestate SearchmetricsBot
SemanticJuice
Semrush
SeobilityBot
Seomoz
Shodan
Site Sucker Site Sucker
SiteCheckerBotCrawler
SiteSnagger SiteSnagger
SlySearch Siteimprove
Snake
Snoopy Snoopy
Spammen SocialRankIOBot
SpankBot SpaceBison
Sqlmap Spanner
Sqlworm Sqlworm
Sucuri Sucuri
SuperHTTP SurveyBot
Suzuran Suzuran
TheNomad T0PHackTeam
Teleport
The Intraformant
Thumbor Thumbor
Trendiction TightTwatBot
Titan
Toata
Toweyabot
True_Robot True_Robot
Turingos Typhoeus
URLy Warning
URLy.Warning URLy.Warning
UnisterBot VCI
Upflow VelenPublicWebCrawler
VidibleScraper Voil
Voltron WISENutbot
WBSearchBot WWW-Collector-E
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD WallpapersHD
Web Fuck WeSEE
Web Auto
Web Collage
Web Sauger Web Sauger
Web Sucker Web Sucker
WebAuto WebBandit
WebCollage
WebEnhancer WebEnhancer
WebFetch
WebFuck
WebGo IS
WebImageCollector WebImageCollector
WebReaper WebPix
WebsiteExtractor WebStripper
WebSucker
WebZIP
Webalta
Webshag
WebsiteQuester
Webster Webster
Whack Whack
Whacker
Whatweb Whatweb
Who.is Bot Who.is Bot
Widow Widow
WiseGuys Robot
Wonderbot Wonderbot
Wprecon Zauba
Xaldon WebSpider
Zeus Zeus
ZumBot ZumBot
ZyBorg adscanner
archive.org_bot coccocbot
autoemailspider crawler.feedback
backlink-check crawler4j
com.plumanalytics dataforseo.com
crawl.sogou.com domainsproject.org
eCatch
evc-batch evc-batch
facebookscraper facebookscraper
heritrix heritrix
ips-agent ips-agent
isitwp.com
iubenda-radar iubenda-radar
magpie-crawler
meanpathbot meanpathbot
oBot mediawords
s1z.ru netEstate NE Crawler
satoristudio.net page scorer
scalaj-http polaris version
probe-image-size
scan.lol scan.lol
seobility seobility
seostar
serpstatbot serpstatbot
sitechecker.pro siteripz
sp_auditbot
spyfu spyfu
tAkeOut sysscan
voyagerx.com webpros.com
webprosbot
x09Mozilla x09Mozilla
x22Mozilla
zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
allbizne.co.ua daretodonate.co
dustyorate.com dayibiao.com
florida-tourism.net findthe.pet
maslenka.kz meds-online24.com
pogosh.com money-for-placing-articles.com
popup-hgd.xyz pechikamini.ru
privatefx-in.ru pro-tec.kz
searchmywindow-a.akamaihd.net traffic100.com
stopnarco.ru ukrtvir.com.ua
svolze.com vykupavto-krasnodar.ru

922
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 00go.com
123any.com 14b.info
123locker.com 1hwy.com
40cg.com 1webmaster.ml
6128786.com 3rbseyes.com
7fon.ru 66cpwgln.space
9icmzvn6.website 76brighton.co.uk
abcguru.xyz 7minuteworkout.com
abusora.com 7wind.ru
adconscious.com 83net.jp
admatic.com.tr 8gold.com
adminshop.com aa9bd78f328a6a41279d0fad0a88df1901.com
ads-seo.men aac500b7a15b2646968f6bd8c6305869d7.com
adservme.com aanapa.ru
afslankpillen2017nl.eu abclauncher.com
aghanyna.com activepr.ru
alarmobninsk.ru addbin.men
albuteroli.com adserver-e7.com
alfa9.com agardomains.com
alt-servis.ru alekseevec.ru
ampower.me alert-hgd.xyz
analytics-ads.xyz alphacarolinas.com
ananumous.ru alta-realestate.com
anaseracresar.tk altamayoztourism.com
anime.dougasouko.com angkortours.vn
ap.senai.br ap.senai.br
apiholdingmypage-a.akamaihd.net apibrowsesmartne-a.akamaihd.net
arenda-avtoprokat-krasnodar.ru asociatia-tipografilor-transilvania.ro
artdeko.info asrvrep-a.akamaihd.net
aruplighting.com asseenontv.ru
asophoto.com atdedinotuho.tk
atlant-auto.info ausmepa.org.au
autobrennero.it auspolice.com
autogrep.ru autonew.biz
aviav.ru.com aversis.net
avitocars.ru avitocars.ru
avon-severozapad.ru
avramstroy.ru avramstroy.ru
avtolombard-krasnodar.com azazu.ru
avtolombard-krasnodar.ru b00kmarks.com
azartmsl.com bablonow.ru
azarttoto.com backgroundpictures.net
b2b-lounge.com badavit.com.ua
barbourjackets.online baditri.com
bayareaaeroclub.org baixar-musicas-gratis.com
bannerads.de
bannertesting.com
beachfix.co
beachpics.com beachpics.com
bear.gotcher.us beauty-clinic.ru
berrymall.ru best-games.today
best-seo-solution.com bestadbid.com
bestcurs.org
besthomemadepornsites.com besthomemadepornsites.com
bestkfiledxd.cf betonka.pro
bestssaker.com betterhdporn.com
betshuckclean.com bimatoprost-careprost.com.ua
bget.ru binomo.kz
biketank.ga bioinnovate.co
bingo8888.com blackbabesporn.com
bizcheapjerseyswholesalechina.com blackhatworth.com
blog-hits.com bocoarchives.org
bloggers.nl boleznikogi.com
bloglag.com boole.onlinewebshop.net
bolezniorganov.ru brains2.biz
boostmyppc.com brainsandeggs.blogspot.com
brandov.ru brandov.ru
bravegirlsclub.com breget74.ru
breastaugmentation.co.za businesxxl.com
buketeg.xyz buzzsumo.com
buscarfoto.com buzzurl.jp
business-suggested.tk candypeople.se
buyessay3.blogspot.ru canzoni.ru
calc-for-credit.ru cardsharp1.ru
callmd5map.com cardul.ru
canyougethighofftizanidine.blogspot.com ccbill.com
capture-room.com centraletermice.us
carrauterie.be centrdebut.ru
cars-modification.net ceotrk.com
cartierlove2u.com cheapsergiorossi.online
casinos4dummies.co.uk checktext.org
cementaresearch.se christianlouboutinsaleoutletonline.us
cete.ru chuckguilford.com
cfacarrosserie74.com cicaki.net
chatroulette.video cineacademy.ru
chatseo.com cinemaflix.website
cheapjerseysfootballshop.com
checkpagerank.net
chinajerseyswholesalecoupons.com
cialischsrx.com
cigarpass.com
clickprotects.com clickprotects.com
cognitiveseo.com clipartnew.com
coldfilm.ru co.lumb.co
com-supportcenter.website coin-hive.com
compliance-alex.xyz coinsspb.com
compliance-john.top comeondog.info
cookie-law-enforcement-ff.xyz comissionka.net
cookielawblog.wordpress.com connectionstrenth.com
conocer-sanabria.com
cookie-law-enforcement-hh.xyz
coolgamechannel.com
copro.pw copro.pw
counciltally.com cosmetic.donna7753191.ru
country-chic.ru covetnica.com
creativehutindia.com crafthubs.com
creditmoney.com.ua crazyprotocol.com
cupday.com custom-electric-guitar.com
cxpromote.com customchocolate.business-for-home.com
cyprusbuyproperties.com customergrowthsystems.com
czat.wp.pl czeshop.info
d2jsp.org dame-ns.kz
dangphoto.trade daneshetabiat.com
daretodonate.co data.vtc.pw
darkages.info dcj-nn.ru
davebestdeals.com deadlinkchecker.com
davidovic.info degerlund.net
dellalimov.com deniven.1bb.ru
deplim.com
design-ideas.info
designdevise.com designdevise.com
dev-seo.blog detmebel.su
deutschland123.de
devochki.top
dfwu1019.info
dikx.gdn dikx.gdn
dispo.de dildofotzen.net
dividendo.ru diusyawiga.tk
djihispano.com dojki365.online
domoysshop.ru dominateforex.ml
downtuptv.gq dorratex.tn
drivotracker.com dostavka-v-ukrainu.ru
drpornogratisx.xxx dptaughtme.com
drugspowerstore.com
druzhininevgeniy63.blogspot.com
dscaas.website dscaas.website
eager-nash.188-93-233-196.plesk.page dtnlyss.com
earl-brown.info e-collantes.com
easyshoppermac.com
ecxtracking.com ecxtracking.com
educontest.net ege-russian.ru
eduinfosite.com egovaleo.it
ekspertmed.com
election.interferencer.ru
elektrischeziga.livejournal.com elektrischeziga.livejournal.com
epicbrogaming.com elektrischezigaretten1.blogse.nl
eshop.md energy-ua.com
eurosamodelki.ru enginebay.ru
euroskat.ru erotiktreff24.info
everytalk.tv euromasterclass.ru
exdocsfiles.com everypony.ru
expertnaya-ocenka.ru exchanges-bet.com
extlinks.com exmasters.com
eyes-on-you.ga extener.org
eyessurgery.ru extractorandburner.com
ezigaretteshop2.vefblog.net eyelike.com.ua
fashion.stellaconstance.co fatfasts-4tmz.com
fashionavenuegame.com fatmaelgarny.com
fastcrawl.com feel-planet.com
fighrofacciufreesig.ga filefilter.weebly.com
figuringmoneyout.com filmgo.ru
filesdatabase.net filunika.com.ru
filmetricsasia.com finansov.info
finuse.com findacheaplawyers.com
floating-share-buttons.com fiverr.com
flowwwers.com food.dtu.dk
folowsite.com forum.poker4life.ru
forminecrafters.ru forums.d2jsp.org
fotzen-ficken.com frighteningremain.cf
franch.info fullgirl.ru
free-share-buttons.blogspot.com fun2cell.net
freenom.link gallictures.com
froggytube.com gambarxkata.co
funponsel.com gamblingnerd.com
gallery.rennlist.com gamedayassist.com
gamerscorps.com gameprimary.com
gamingspark.com gaz-voshod.ru
gdcentre.ru gemgrab-a.akamaihd.net
ge0ip.org genericlowlatencyasiodriverhq.aircus.com
gentamicineyedrops.blogspot.com genericviagrasildenafiled.net
getprismatic.com generousdeal-a.akamaihd.net
girls-ufa.ru getdot.ru
gloverid.site getlaid-xxxhookupdirect.com
goforexvps.com glall.ru
goldpanningtools.com globalscam.ga
golmau.host.sk gnuetella.com
gonextmedia.com goodnightjournal.com
goodhousekeeping.com goodwinmetals.co
goodhumor24.com googst2.ru
goodprotein.ru gosreg.amchs.ru
goroda-vsego-mira.ru grand-chlen.ru
gourcy.altervista.org greatidea.marketing
graphicwe.org gsasearchengineranker.top
green-tea.tv hasshe.com
growshop.es
guidefs.ru
handsandlegs.ru
hatedriveapart.com hatedriveapart.com
hdpixion.com havepussy.com
hdseriale.pl headpress.ru
helpmymacfaster.trade hifidesign.ru
hem.passagen.se hjaoopoa.top
histock.info houston-vikings.com
hunterboots.online hoverboard360.de
ibmdatamanagement.co howlongdoestizanidinestayinyoursystem.blogspot.com
idc.com.ua howopen.ru
igrovyeavtomaty777.ru hunthillfarmtrust.org
imagecoolpub.com idealtits.net
images-graphics-pics.com igadgetsworld.com
imguramx.pw igru-xbox.net
infospot.pt imgarit.pw
infostatsvc.com imk.com.ua
int.search.mywebsearch.com import-sales.com
internetproviderstucson.com inboxdollars.com
investzalog.ru infoupdate.org
iomoio.net inspiring-desperate.tk
irunfar.com insta-add.pro
ixora.pro intimshop-fantasy.ru
jackwolfskinoutlet.online ip-guide.com
james13prix.info irkutsk.online-podarki.com
jav-idol.com it-max.com.ua
javidol.site itch.io
jerseyssportsshop.com itsdp3.com
jerseywholesalebizchina.com jav-fetish.site
jimmychoosale.online jerseysbizwholesalecheap.com
job.icivil.ir joingames.org
jrpmakati.com kaac.ru
juliadiets.com
kamorel.com kamorel.com
kartiniresto.com kanimage.com
kasino-money.pw karusel-market.ru
kemerovo.zrus.org kashubadesign.ru
ketoanhanoi.info keyword-suggestions.com
keywordblocks.com keywordsking.com
keywordspay.com kinky-fetishes.com
keywordteam.net kino-fun.ru
kimcurlrvsms.com kinofak.net
kino2018.cc kiprinform.com
kinogonew.ru kiwi237au.tk
kosova.de kmd-pto.ru
krasnodar.zrus.org kmgamex.cf
kskjco.club koopilka.com
ktotut.net krasivye-devushki.net
kumuk.info krynica.info
kupit-adenu.ru lacapilla.info
lancheck.net ladsblue.com
lalalove.ru
landoftracking.com
landreferat.ru
laparfumotec.com laparfumotec.com
laptop-4-less.com lapitec.eu
laxob.com lashstudia.ru
lechenie-gemorroya.com lawyersinfo.org
letmacworkfaster.world lenpipet.ru
lidoradio.com letslowbefast.site
lightinghomes.net lifebyleese.com
lineshops.biz liky.co.ua
lion.bolegapakistan.com locimge.pw
liveinternet.ro lomb.co
lkbennettoutlet.online lotto6888.com
lostfilm-online.ru mackeeper-center.club
lyrster.com mainlinehobby.net
mac-shield.com makenahartlin.com
mapquestz.us manyresultshub-a.akamaihd.net
markjaybeefractal.com marcowebonyodziezowe.pl
marmitaco.cat
martlinker.com martlinker.com
masterseek.com matsdale.com
matchpal-a.akamaihd.net mcar.in.ua
mearns-tractors.co.uk meds-online24.com
mebel-ekb.com medtherapy.ru
meet-flirt-dating.com mega-bony2017.pl
michaelkorssaleoutletonline.net mejoresfotos.eu
millionare.com methodsmarketing.com
minyetki.ru milblueprint.com
mir-betting.ru
misswell.net
mitrasound.ru mitrasound.ru
mixed-wrestling.ru mmstat.com
mobicover.com.ua mnogabukaff.net
montredemarque.nl mobile-appster.ru
morepoweronmac.trade modabutik.ru
moskva.nodup.ru moncleroutletonline.pw
moyakuhnia.ru monsterdivx.com
mp3ritm.top monthlywinners.com
mriyadh.com moonci.ru
mttwtrack.com morlat.altervista.org
mturkcontent.com mp3ringtone.info
mydownloadengine.com mtmtv.info
mydownlodablefiles.com musicvidz.ru
myliveblog.ru mustwineblog.com
myseoconsultant.com muzaporn.com
narosty.com mxgetcode.com
negral.pluto.ro mycouponizemac.com
neks.info myfreemp3.eu
netallergy.ru myiptest.com
netanalytics.xyz myphotopipe.com
nhl17coinsforps3.gratisblog.biz na-telefon.biz
niki-mlt.ru narutonaruto.ru
noclegonline.info nationalbreakdown.com
nowtorrents.com naughtyconnect.com
o.light.d0t.ru net-profits.xyz
obiavo.su netvouz.com
odoratus.net networkcheck.xyz
offtopic.biz no-rx.info
okuos.com nordstar.pro
old-rock.com notasprensa.info
olgacvetmet.com nsatc.net
onemantrip.com nudejapan.net
onlainbesplatno.ru nvssf.com
online-sbank.ru oakleyglassesonline.us
online247.ml of-ireland.info
onlinebay.ru office2web.com
onlinefilmz.net okeinfo.online
onload.pw oksrv.com
onlyforemont.ru one-gear.com
onstrapon.purplesphere.in opinionreelle.com
onthemarch.co outpersonals.com
optikremont.ru p-business.ru
oranga.host.sk papasdelivery.ru
ourville.info
ovirus.ru
paccohichetoti.ml
panchro.xyz
paperwritingservice17.blogspot.ru paperwritingservice17.blogspot.ru
paradontozanet.ru partybunny.ru
pelfind.me pawli.eu
photorepair.ru pcads.ru
photostudiolightings.com pestomou.info
petedrummond.com
petsblogroll.com
peugeot-club.org
photokitchendesign.com
physfunc.ru physfunc.ru
pinsdaddy.com pic-re.blogspot.com
pisanieprac.info picphotos.net
piuminiita.com picturesmania.com
pladform.ru piluli.info
plastgranar.nu pinkduck.ga
platezhka.net pivka.xyz
platinumdeals.gr pix24x7.com
pobeiranie.pl pkr1hand.com
podshipniki-nsk.ru plaff-go.ru
poiskzakona.ru ploenjitmedia.azurewebsites.net
polcin.de plohaya-kreditnaya-istoriya.ru
poligon.com pony-business.com
pornoinn.com poppen-nw.net
posible.net popunder.net
pozdravleniya-c.ru porno.simple-image.com.ua
pravoholding.ru pornosee.info
prchecker.info powc.r.ca.d.sendibm2.com
privatefx-in.ru prefersurvey.net
profolan.pl preparevideosafesystem4unow.site
progress-upakovka.ru pricheski-video.com
promoforum.ru privatamateure.com
pro-okis.ru
profitkode.com
promover.org promover.org
prosperent.com pron.pro
prostitutki-novgoroda.org prostitutki-yaroslavlya.org
quickbuck.com pskcijdc.bloger.index.hr
razleton.com purplesphere.in
rebuildermedical.com pussysaga.com
redirectingat.com qcstrtvt.bloger.index.hr
rednise.com qualitymarketzone.com
remorcicomerciale.ro quizzitch.net
radiodigital.co
ragecash.com
rangjued.com
rankscanner.com
rebrand.ly
recordpage-a.akamaihd.net
registrationdomainsite.com
registry-cleaner.net
reimageplus.com
relayblog.com
rent2spb.ru rent2spb.ru
resellerclub.com replica-watch.ru
rockingclicks.com replicaclub.ru
royalvegascasino.com retreatia.com
ruclicks.com reversing.cc
sabaapress.com revistaindustria.com
salutmontreal.com ric.info
sammyweaver.com riders.ro
savingsslider-a.akamaihd.net rollercoin.com
sayyoethe.blogspot.co.za rossanasaavedra.net
scanner-jessica.top runstocks.com
scanner-walt.top ruscopybook.com
scanner-walter.top s-forum.biz
scanspyware.net santechnik.jimdo.com
schlampen-treffen.com sape.top
scottbywater.com sarm.tk
sdelatmebel.ru scanner-alexa.top
seansonline24.pl scanner-don.top
search-goo.com scanner-elena.top
seksotur.ru scanner-mark.top
semaltmedia.com searchimpression.com
seoboxes.com searchlock.com
service.adtech.us secret.xn--oogle-wmc.com
servingnotice.com securityallianceservices.com
sex-znakomstva.online semprofile.com
sexpartygirls.net seoholding.com
sexpornotales.com seomarketings.online
sexy-screen-savers.com sex-tracker.com
sharebutton.net sexiporno.net
ship-marvel.co.ua sexkontakte-seite.com
sexkrasivo.net
sexsaoy.com
sexyteens.hol.es
shopperifymac.com
similardeals.net similardeals.net
sims-sims.ru
site3.free-share-buttons.com site3.free-share-buttons.com
siteimprove.com sklad-24.ru
sitevalued.com skladvaz.ru
slutloadlive.com snip.to
snomer1.ru snjack.info
sobecjvuwa.com.ru sofit-dmd.ru
soblaznu.net softlinesolutions.me
social-s-iii.xyz
socialseet.ru
softomix.com softomix.com
solution4u.com
sonata-arctica.wz.cz
sotkal.lark.ru sotkal.lark.ru
spabali.org sovinsteel.ru
spy-app.info space4update.pw
spyfu.com spravka-medosmotr.ru
spywarebegone.com srdrvp.com
stats-collector.org stackthatbucks.com
stillmiracle.com stpolice.com
strigkaomsk.ru stretchmate.net
stroimajor.ru stroilka.info
strongsignal-a.akamaihd.net
stroydetali.ru
sundrugstore.com sundrugstore.com
superfish.com svensk-poesi.com
supermesta.ru tarad.com
superstarfloraluk.com telefonsexsofort.tk
surcentro.com telegramdownload10.com
svarkagid.com
sygraem.com
taranerymagesswa.blogspot.com
taxi-v-eisk.ru
td-l-market.ru
telsis.com telsis.com
templates.franklinfire.co
terraclicks.com terraclicks.com
tgtclick.com thefotosgratis.eu
thecoolimages.net
thecoral.com.br
theguardlan.com
thenetinfo.com thenetinfo.com
thetoiletpaper.com thepantonpractice.co.uk
tiandeural.ru thesmartsearch.net
tizanidine4mgtablets.blogspot.com tiens2010.ru
timer4web.com
tizanidinefibromyalgia.blogspot.com tizanidinefibromyalgia.blogspot.com
tizanidinehcl2mgsideeffects.blogspot.com tjkckpytpnje.com
tizanidinepills.blogspot.com token-lab.org
tizanidinesideeffectsweightloss.blogspot.com tongkatmadura.info
tizanidinewithdrawalsymptoms.blogspot.com topflownews.com
toolsky.com touchmods.fr
top1-seo-service.com tradedeals.biz
topsy.com tradgardspartner.se
torrents.life trafaret74.ru
trackmedia101.com trafficmonetize.org
trafficnetzwerk.de trahvid.com
treasuretrack-a.akamaihd.net treasuretrack-a.akamaihd.net
tri-slona.org tsan.net
trudogolik.net tula.howotorg.ru
uk-zheu20.ru uhodzalijami.ru
undergroundcityphoto.com ukrtextbook.com
unece.org urengoy.pro
unionmarkt.de us-america.ru
unitygame3d.com vandrie-ict.nl
urzedowski.eu
vacuumscleaner.com
vadimkravtcov.ru
vavilone.com
vegan-foods.us vegan-foods.us
versaut.xxx-cam.webcam vepad.com
via-energy-acquistare.com
viagra.pp.ua
videochat.bz
videochat.life videochat.life
videosbox.ru videosbox.ru
vikistars.com vinylvault.co.uk
visa-pasport.ru vitoriacabos.com
visitcambridge.org viven.host.sk
vizag.kharkov.ua vod.com.ua
vkak.ru volgograd.xrus.org
vmnmvzsmn.over-blog.com voprosotvet24.ru
vote-up.ru
vozbujdenie.com
vpnmouse.com
vremya.eu
vut.com.ru vut.com.ru
wallpaperaccess.com vykup-avto-krasnodar.ru
watchinf.com wallpapersdesk.info
we-are-gamers.com wallpaperstock.net
webjam.com watchdogs-2.ru
webnode.me watracker.net
website-audit.com.ua web.cvut.cz
websites-reviews.com webpromotion.ae
weekes.biz.tc webscouter.net
welcomeauto.ru wesharepics.info
whatsappbot.flyland.ru westsextube.com
wholesalejerseys-cheapest.com whereiskentoday.com
wholesalejerseysgaa.com wholesalejerseychinashop.com
williamrobsonproperty.com williamrobsonproperty.com
woodyguthrie.se wineitudes.wordpress.com
workle.website wingsofrefuge.net
world-mmo.com wladimirpayen.com
woweb.com.ua womens-journal.net
www.bookmaker-bets.com worldmusicfests.com
www.wohnkabinen-shop.de wpthemedetector.co.uk
wzgyyq.com wstroika.ru
xev.ru wygraj-teraz.com
xlolitka.com x-diesel.biz
xlovecam.com xn----7sbahjd3btneuw1joc.xn--p1ai
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf xn----9sbubg3ambdfl1j.xn--p1ai
xn--80aafb2a.xn--p1ai xn--80ajbshivpvn2i.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai xn--80atua3d.xn--p1ai
xn--80aodinpgi.xn--p1ai xnxx-n.com
xnxxandxvideos.com xnxxandxvideos.com
xxxkaz.org xxx-treker.ru
y8games-free.com xxxasianporn.net
yoshkarola.zrus.org xxxtube69.com
yun56.co yginekologa.com
za-music.mymobiplanet.com yoluxuryevents.com
zaim-pod-zalog-krasnodar.ru youraticles.pl
zakonobosago.ru yourporn.com
zaobao.com.sg zaimite.ru
zawyna.ua zastroyka.org
zdesformula.ru zeroredirect.com
zeroredirect8.com zigzog.ru
zmoda.hostreo.com
zojirushi-products.ru zojirushi-products.ru
zootoplist.com zona-aqua.ru
zrus.org zoogdinsney.com
zx6.ru zs2vm.top
zygophyceous.womanstars.site zvezdagedon.ru

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
03p.info 1kinobig.ru
0912701309f8ce.com 45en.ru
0lovespells0.blogspot.com
1pamm.ru
1stat.ru
214jaluwobapef.cf
4kpics.rocks 4kpics.rocks
aaa38852e886ac4af1a3cff9b47cab6272.com 5forex.ru
abcguru.xyz 7makemoneyonline.com
addblueoff.com.ua 7zap.com
allnews.md aa8780bb28a1de4eb5bff33c28a218a930.com
anonymizeme.pro abwa.tk
anonymousfox.co adconscious.com
antons-transporte.de admanaerofoil.com
apibrowsesmartne-a.akamaihd.net adultgalls.com
aramaicmedia.org adverster.com
audiobangout.com ageofclones.com
australia-opening-times.com albuteroli.com
autobrennero.it allnews24.in
aviav.ru.com alphaforum.ru
ayerbo.xhost.ro ample-awards-today.us
azadnegar.com apelsinnik.website
b-styles.xyz arate.ru
babespcs.com arclk.net
baditri.com asrv-a.akamoihd.net
barbourjackets.online audiofree.ru
bestcurs.org auto-zapchasti.org
bigpenisguide.com autorn.ru
bio-optomarket.ru autoua.top
availit.weebly.com
ayeartoforget.com
babs.com.ua
bamo.xsl.pt
basisches-wasser.net
beauty-bracelet.com
bensbargains.net
best-seo-software.xyz
bezcmexa.ru
bgviagrachrx.com
bird1.ru
bjgugu.net.ua bjgugu.net.ua
bloggedporn.com buchananshardware.com
bloglag.com c-english.ru
bmwhighperformers.com candycrushshop.com
boanonihaca.tk cartechnic.ru
bosman.pluto.ro cartierbraceletsreplica.pw
breastaugmentation.co.za cartierlove2u.com
btnativenav.com
buttons-for-website.com
bwlx.prepedu.cn
cam-kontakte.org
candytiner.com
carsplate.com
cashonads.com
cdncash.com cdncash.com
chastnoeporno.com cete.ru
chatroulette.world chelnytruck.ru
cheapjerseysbizwholesale.us chocoslim.pro
ciceros.co choosecuisine.com
clips.ua.ac.be cl.s7.exct.net
coderstate.com clarithromycin500mg.com
compliance-brian.top claytransformations.info
compliance-julia.top clmforexeu.com
conboy.us conboy.us
confib.ifmo.ru creativehutindia.com
custom-product-labels.com customboxes4less.com
damedingel.ya.ru customcatchcan.com
dating-app.ru dalavia.ru
dbmkfhqk.bloger.index.hr datract.com
depositfiles-porn.ga dengi-pod-zalog-nedvizhimosti.ru
dev-seo.blog documentsite.net
diarioaconcagua.com dodlive.mil
diegolopezcastan.com domain-submit.info
digitalfaq.com domain-tracker.com
dikx.gdn doxyporno.com
dispo.de dptaughtme.com
downloads-whatsapp.com
drunkenstepfather.com
dverimegapolis.ru dverimegapolis.ru
eandsgallery.com e2click.com
egovaleo.it elektronischezigarettekaufen2.tumblr.com
elexies.info eloxal.ru
elvenar.com emergencyneeds.org
eredijovon.com escortplius.com
es5.com farmprofi.net
eshop.md fashion-mk.net
ezigarettenkaufen1.pagina.gr fba-mexico.com
ezofest.sk fetlifeblog.com
figuringmoneyout.com flavors.me
filmgo.ru flirt4free.com
findpics.pw florida-tourism.net
finteks.ru forodvd.com
firstdrugmall.ru free-deals.faith
firstsiteguide.com freewebs.com
flytourisme.org fxtips.ru
forum.tvmir.org galeon.com
foto-basa.com gamedayhouse.com
freefoto.ca gaming-journal.com
freejabs.com gate5.co.za
freemaintenancesysforpcandmac.top gaytube.com
freevpn.space gazporno.com
ftns.ru global-ics.co.za
fullgirl.ru goodhousekeeping.com
geile-lelly.eu googst2.ru
girlsfuckdick.com gopro-online.info
glastecfilms.com.my grizzlysgrill.com
growmyfunds.ca
guardlink.com
hatdc.org hatdc.org
hccoder.info heatpower.ru
hundejo.com hitcpm.com
incanto.in.ua hotloans.ru
indiakino.net hybrid.ru
inzn.ru hydropump.su
isoveti.ru ie.57883.net
istanbulit.com igadgetsworld.com
janavibekken.no imagui.eu
jongose.ninja imperiafilm.ru
kaliningrad.zrus.org indo-export.ru
kazan.xrus.org interesnie-faktu.ru
kinohall.ru izhevsk.xrus.org
kletkimehan.ru jerseychinabizwholesale.us
lafourmiliaire.com jerseysbizwholesalecheap.com
lampokrat.ws jerseywholesalechinabiz.com
landmania.ru jrpmakati.com
languagecode.com kaz.kz
lawyers.cafe kolotiloff.ru
le-clos-des-alouettes.com ks1234.com
lenpipet.ru kurbappeal.info
lietaer.com labelwater.se
likesdesign.com labplus.ru
lineavabit.it lacasamorett.com
listiseltemournlan.gq lalalove.ru
lkbennettoutlet.online libertybilisim.com
lonely-mature.com likesuccess.com
lonfon.xyz linkarena.com
lyrics.home-task.com mafa-free.com
m4ever.net magento-crew.net
macdamaged.tech makemoneyonline.com
madot.onlinewebshop.net mansparskats.com
manifestyourmillion.com manve.info
manipulyator-peterburg.ru map028.com
marsgatan.com maxthon.com
mazda-roadsters.com mecash.ru
mcnamaratech.com minneapoliscopiers.com
medicinacom.ru misandesign.se
methodsmarketing.com missclub.info
mir-business-24.ru mobicover.com.ua
monclerboots.xyz mobioffertrck.com
moncleronline.xyz muabancantho.info
monclervests.xyz mulberryoutletonlineeu.com
moz.com multgo.ru
mural.co mural.co
musicvidz.ru musicas.baixar-musicas-gratis.com
myperiod.club mustat.com
myseoconsultant.com naphukete.ru
nash-krym.info nash-krym.info
natali-forex.com net-profits.xyz
netallergy.ru newsperuse.com
nfljerseys.online nhl09.ru
no-fuel.org nw-servis.ru
novosti-hi-tech.ru o-o-6-o-o.com
nrjmobile.fr ochistka-stokov.ru
obesidadealgarve.com oconto.ru
oil-td.ru okonich.com.ua
onblastblog.online openlibrary.org
online247.ml paparazzistudios.com.au
openfrost.net paperwritingservice17.blogspot.ru
optibuymac.com paradontozanet.ru
outclicks.net pc-services.ru
oyster-green.com pendelprognos.se
partner-stop.men pinwallpaper.top
pinapchik.com pivka.xyz
pochemychka.net pixell.club
popads.net poiskzakona.ru
porno-play.net popunder.ru
predominant-invent.tk porno.simple-image.com.ua
producm.ru poshiv-chehol.ru
prohoster.info puserving.com
projectforte.ru qualitymarketzone.com
promover.org rank-checker.online
prostitutki-odessa.org razamicroelectronics.com
ragecash.com registrationdomainsite.com
rainbowice.ru remybutler.fr
rapidokbrain.com replicaclub.ru
royalcar-ufa.ru rubanners.com
russian-postindex.ru runtnc.net
satoristudio.net ruscopybook.com
sax-sex.com russia-today-video.ru
sbdl.no rutor.vip
sc-specialhost.com rybalka-opt.ru
scanner-john.top samo-soznanie.ru
schoolfiles.net sanatorrii.ru
screpy.com sarafangel.ru
sdelatmebel.ru savefrom.com
seeingmeerkat.com scanner-alexa.top
senger.atspace.co.uk score-ads.men
seoboxes.com seo-traffic-ranking.info
sexflirtbook.com sex-sex-sex5.com
sexphoto.site sexkontakteao.info
sexs-foto.top shopperifymac.com
site-analyzer.com simple-image.com.ua
site-speed-checker.site simple-share-buttons.com
sitiz.club site-auditor.online
sivs.ru smartadserver.com
slayerlife.com socialbookmarksubmission.org
socialbuttons.xyz socialseet.ru
sovetogorod.ru soft-terminal.ru
srecorder.com soserfis.com
stard.shop soviet-portal.do.am
spabali.org
spbchampionat.ru
spitfiremusic.com
spylog.com
spymac.net
stariy-baku.com
start.myplaycity.com
stroilka.info stroilka.info
sucsesofinspiration.com suture.co
sukarame.net swinon.site
tacbibirfa.tk taihouse.ru
teesdaleflyballclub.co.uk tam-gde-more.ru
thecoolimages.net tampabaywatch.org
thelottosecrets.com teenporn18.net
thepornsex.org teslathemes.com
thruport.com theprofitsmaker.net
time-japan.ru tizanidineinteractions.blogspot.com
timer4web.com torontoplumbinggroup.com
tizanidine4mgstreetvalue.blogspot.com tt-ipd.info
tizanidineinfo.blogspot.com ttrraacckkrr.com
topquality.cf tube8.com
torrents-tracker.com tvand.ru
toy-shop.top twiclub.in
track-rankings.online unlimitdocs.net
trackmedia101.com unmaroll.ya.ru
uasb.ru uogonline.com
uchebavchehii.ru
undergroundcityphoto.com
unece.org
ussearche.cf
vabasa.inwtrade.com vabasa.inwtrade.com
valoresito.com valueclick.com
videospornogratisx.net veerotech.com
vipcallsgirls.com videositename.com
villacoloniale.com
vip-file.com
viveresaniesnelli.it
vkgaleria.com vkgaleria.com
vladhistory.com vmnmvzsmn.over-blog.com
vladimir.zrus.org vzlom-na-zakaz.com
wahicbefa31.soup.io wapsite.me
webextract.profound.net warningwar.ru
weclipart.com wcb.su
wejdz-tu.pl websites-reviews.com
wholesalenfljerseys.us.com websitevaluebot.com
worst-sites.online wesharepics.site
www.arenda-yeisk.ru westvilletowingservices.co.za
www.kabbalah-red-bracelets.com whatistizanidinehclusedfor.blogspot.com
xn----ctbigni3aj4h.xn--p1ai whereiskentoday.com
xn--90acenikpebbdd4f6d.xn--p1ai white-truck.life
xtube.com wstroika.ru
xwatt.ru wyniki-lista.pl
yoshkarola.zrus.org xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
youresponsive.com xn----7sbahjd3btneuw1joc.xn--p1ai
zapatosenventa.info xn--b1ag5cfn.xn--p1ai
zeroredirect11.com yellowfootprints.com
zetmaster.ru youghbould.wordpress.com
zona-aqua.ru zdesoboi.com
zoosexart.com zscaler.net
zootravel.com
zscalerone.net

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.6 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.08.3278 ### Version: V4.2022.08.3279
### Updated: Mon Aug 8 22:01:17 UTC 2022 ### Updated: Tue Aug 9 10:03:38 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.08.3278 ### Version: V4.2022.08.3279
### Updated: Mon Aug 8 22:01:17 UTC 2022 ### Updated: Tue Aug 9 10:03:38 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##