vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.02.3573 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-02-10 22:07:32 +00:00
parent daf19565b0
commit 8e276305cf
46 changed files with 99133 additions and 99133 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.02.3572 #### Version: V4.2023.02.3573
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.02.3572 #### Version: V4.2023.02.3573
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.02.3572 #### Version: V4.2023.02.3573
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12256
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.02.3572 #### Version: V4.2023.02.3573
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12260
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12260
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12260
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12260
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.52 0;
127.0.0.1 0; 127.0.0.1 0;

12260
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.52 0;
127.0.0.1 0; 127.0.0.1 0;

12260
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.52 0;
127.0.0.1 0; 127.0.0.1 0;

12260
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3572 3573

12260
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12260
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
23.98.136.33 0; 13.83.3.160 0;

12260
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.52 0;
127.0.0.1 0; 127.0.0.1 0;

12260
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.52 0;
127.0.0.1 0; 127.0.0.1 0;

12260
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.52 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.52 0;
127.0.0.1 0; 127.0.0.1 0;

12260
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
23.98.136.33 0; 13.83.3.160 0;

12260
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
23.98.136.33 0; 13.83.3.160 0;

12260
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 1; 10.1.0.52 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Anarchie Bitacle
Crawling at Home Project DSearch
Discoverybot EirGrabber
HEADMasterSEO FlashGet
IstellaBot Lanshanbot
Pandalytics Leap
PeoplePal Masscan
Rankivabot Semrush
SpankBot WPScan
com.plumanalytics clark-crawler

320
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider
80legs 80legs
AIBOT AIBOT
ASPSeek Abonti
Aboundexbot Aboundexbot
AfD-Verbotsverfahren
AhrefsBot
AiHitBot AiHitBot
Alexibot Aipbot
Alligator AlphaBot
Anarchie Anarchy
Ankit Ankit
Anthill Aspiegel
Asterias Atomseobot
Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot
BBBike
BDCbot
BackDoorBot
BackStreet BackStreet
BacklinkCrawler BackWeb
Badass
Bandit Bandit
Battleztar Bazinga Battleztar Bazinga
Bigfoot Blackboard
Black Hole Boardreader
Brandprotect Bolt
Brandwatch
Buck
BuiltBotTough
Bullseye Bullseye
BunnySlippers
BuzzSumo
CATExplorador
CODE87 CODE87
CSHttp
Calculon
CazoodleBot CazoodleBot
ChinaClaw Cegbfeieh
Citoid CensysInspect
Claritybot CheeseBot
Cliqzbot CherryPicker
Cogentbot Cogentbot
CopyRightCheck Copier
Copyscape
Craftbot Craftbot
Custo DSearch
DTS Agent DataCha0s
DatabaseDriverMysqli Disco
DigitalPebble
Discoverybot
Dispatch Dispatch
DomainAppender DnBCrawler-Analytics
DomainCrawler Download Wonder
Domains Project
Dragonfly Dragonfly
Drip EMail Siphon
ECCP/1.0 EirGrabber
EasyDL Exabot
Evil Extractor
Express WebPictures
ExtractorPro ExtractorPro
FDM EyeNetIE
FHscan FemtosearchBot
Fuzz Fimap
Genieo Flunky
GetRight Freeuploader
Gotit G-i-g-a-b-o-t
GT::WWW
GetWeb
GrabNet GrabNet
GrapeshotCrawler Grabber
HMView Grafula
HTMLparser Haansoft
HaosouSpider
Harvest
Havij
Heritrix Heritrix
Hloader
Humanlinks Humanlinks
IDBTE4M IDBTE4M
IRLbot IRLbot
Id-search Id-search
IlseBot IlseBot
Image Fetch
Image Sucker Image Sucker
InfoNaviRobot Intelliseek
InterGET
Internet Ninja
Iskanie
IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT JamesBOT
JennyBot JennyBot
JetCar JikeSpider
JustView Joomla
Jyxobot Keybot Translation-Search-Machine
Kenjin Spider Kinza
LWP::Simple
Larbin
Lftp Lftp
LieBaoFast Libwhisker
LinkWalker Lightspeedsystems
Linkbot Linkbot
LinkextractorPro LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider Lipperhey Spider
Lmspider Litemage_walker
MQQBrowser MFC_Tear_Sample
MTRobot MIDown tool
MSIECrawler
Mag-Net
Mail.RU_Bot Mail.RU_Bot
MarkMonitor Majestic SEO
MarkWatch Mass Downloader
Masscan
Mata Hari
Mb2345Browser
MeanPath Bot MeanPath Bot
MegaIndex.ru MicroMessenger
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Moblie Safari
Mister PiX Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Musobot
Mozlila
Mr.4x3
Msrabot
Name Intelligence Name Intelligence
Navroad Navroad
NearSite
Needle
Nessus Nessus
Net Vampire Net Vampire
NetMechanic NetAnts
NetZIP NetSpider
Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler Nibbler
Niki-bot Niki-bot
NimbleCrawler Nimbostratus
Ninja Ninja
Nmap Not
Nuclei Nuclei
Nutch
Offline Navigator
OnCrawl OnCrawl
OpenVAS OpenLinkProfiler
Openfind Openfind
Openvas OutclicksBot
OrangeBot PHPCrawl
OrangeSpider POE-Component-Client-HTTP
PECL::HTTP
Page Analyzer Page Analyzer
PageGrabber PageThing.com
Pandalytics Pandalytics
Pi-Monster Panscient
Picsearch Pavuk
PictureFinder Petalbot
Pimonster Pimonster
Pixray Pockey
PleaseCrawl ProWebWalker
ProPowerBot
Probethenet
Pu_iN
Pump Pump
PxBroker QueryN Metasearch
RankActive RankActiveLinkBot
RankFlex
RankingBot RankingBot
Rankivabot RankingBot2
Re-re
RedesScrapy RedesScrapy
RepoMonkey RepoMonkey
Rogerbot Rogerbot
SBIder
SEOkicks SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler SEOprofiler
SISTRIX
SMTBot SMTBot
Scanbot SalesIntelligent
Semrush SearchmetricsBot
SemrushBot SemanticJuice
SentiBot SeobilityBot
SeoSiteCheckup Siphon
Shodan
SiteSnagger SiteSnagger
SiteSucker SiteSucker
Sitebeam
Sitevigil
SlySearch SlySearch
SmartDownload
Snake
Snapbot Snapbot
Snoopy
SocialRankIOBot SocialRankIOBot
Sogou web spider Sociscraper
Sosospider
Sottopop Sottopop
Spammen SpaceBison
SpankBot SpankBot
Sqlworm Spanner
Sucker Spbot
Sucuri Stripper
SuperBot SuperBot
SurveyBot Surfbot
Swiftbot
Szukacz Szukacz
T8Abot
Teleport Teleport
TeleportPro
Telesoft
Telesphoreo
TheNomad
Titan Titan
Toata
Toweyabot
Tracemyfile Tracemyfile
True_Robot Trendiction
Turnitin Trendictionbot
TurnitinBot TurnitinBot
Twice
Typhoeus
URLy Warning
URLy.Warning URLy.Warning
V-BOT Upflow
VB Project VCI
Vacuum Vacuum
VelenPublicWebCrawler Vagabondo
VidibleScraper VeriCiteCrawler
Virusdie
VoidEYE VoidEYE
WASALive-Bot Voltron
WBSearchBot WBSearchBot
WWW-Mechanize WEBDAV
Web Sauger WPScan
Web Sucker WWW-Collector-E
WebImageCollector WWW::Mechanize
WWWOFFLE
WallpapersHD
Web Collage
Web Fuck
WebFuck
WebPix WebPix
WebReaper
WebStripper
WebZIP WebZIP
Webalta Website Quester
WebmasterWorldForumBot
Whack Whack
Whatweb WinHTTrack
Wonderbot WiseGuys Robot
Woobot Xaldon_WebSpider
Wprecon YoudaoBot
Xaldon WebSpider ZoominfoBot
Zermelo ZyBorg
ZoomBot
ZumBot
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo-web-crawler
autoemailspider
backlink-check
clark-crawler
coccocbot
cognitiveseo cognitiveseo
com.plumanalytics
crawl.sogou.com crawl.sogou.com
crawler.feedback
crawler4j
domainsproject.org
eCatch eCatch
facebookscraper evc-batch
gopher
instabid
internetVista monitor
isitwp.com isitwp.com
iubenda-radar linkdexbot
lwp-trivial magpie-crawler
meanpathbot mediawords
oBot netEstate NE Crawler
pcBrowser
plumanalytics plumanalytics
ripz ripz
seocompany.store satoristudio.net
sexsearcher scan.lol
sitechecker.pro seostar
sp_auditbot siteripz
sysscan sogouspider
trendiction.com spyfu
voyagerx.com ubermetrics-technologies.com
webgains-bot
webpros.com
webprosbot
x09Mozilla x09Mozilla
xpymep1.exe
zauba.io

298
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404enemy 360Spider
AIBOT Abonti
Aboundex Acunetix
AhrefsBot Aipbot
Alexibot
AllSubmitter AllSubmitter
Alligator
Anarchy Anarchy
Anthill Ankit
Asterias Asterias
AwarioRssBot Backlink-Ceck
BBBike
BDCbot
BLEXBot
BackDoorBot
BackWeb
BacklinkCrawler BacklinkCrawler
Badass Badass
Battleztar Bazinga BetaBot
Bitacle
BlackWidow
Blackboard Blackboard
BotALot Boardreader
Bolt
Brandprotect Brandprotect
Brandwatch
Buck
Buddy Buddy
Bullseye Bullseye
BuzzSumo CCBot
CATExplorador CazoodleBot
Calculon
Cegbfeieh Cegbfeieh
CensysInspect CensysInspect
CheTeam CheTeam
CherryPicker CheeseBot
ChinaClaw
Citoid Citoid
Cliqzbot Cliqzbot
Cloud mapping
Cocolyzebot
Cogentbot Cogentbot
Copier Collector
Craftbot CopyRightCheck
Copyscape
Cosmos
CrazyWebCrawler
Crescent Crescent
CrunchBot Curious
CyotekWebCopy
DSearch
DatabaseDriverMysqli
Demon
Devil Devil
Digincore
DigitalPebble DigitalPebble
Disco
Discobot Discobot
Discoverybot
Dispatch Dispatch
DittoSpyder DnBCrawler-Analytics
DnyzBot
DomainAppender DomainAppender
DomainSigmaCrawler Dragonfly
Download Wonder Ecxi
EMail Wolf EroCrawler
Evil
ExtLinksBot ExtLinksBot
Extractor EyeNetIE
ExtractorPro
Extreme Picture Finder
Ezooms Ezooms
Firefox/7.0 Fimap
FlashGet Flunky
Foobot
Freeuploader
FrontPage
FyberSpider FyberSpider
Fyrebot GalaxyBot
Genieo
GermCrawler GermCrawler
GetRight GrabNet
GridBot GrapeshotCrawler
HMView
HTTrack HTTrack
Haansoft
HaosouSpider
Havij Havij
HonoluluBot IRLbot
HybridBot Image Sucker
IDBTE4M Indy Library
Id-search InfoNaviRobot
Image Fetch Intelliseek
Iskanie InterGET
InternetSeer
IstellaBot
JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz Jbrofuzz
Jorgee Jorgee
Jyxobot Kenjin Spider
Keyword Density
Kinza
Kozmosbot Kozmosbot
Lanshanbot LNSpiderguy
LWP::Simple
Larbin Larbin
LeechGet
LibWeb
Libwhisker Libwhisker
LieBaoFast Likse
Lightspeedsystems
LinkScan LinkScan
Linkbot Linkbot
LinkextractorPro LinkpadBot
LinqiaMetadataDownloaderBot LinksManager
Litemage_walker Lipperhey Spider
Lmspider Lmspider
Ltx71 Ltx71
MFC_Tear_Sample Mag-Net
MIDown tool Majestic12
MQQBrowser
Magnet
Mail.RU_Bot
Mass Downloader Mass Downloader
Masscan
Mb2345Browser
MeanPath Bot
Meanpathbot Meanpathbot
MegaIndex.ru
Metauri
MicroMessenger MicroMessenger
Microsoft URL Control Microsoft Data Access
Minefield Minefield
Mojeek Mojeek
MolokaiBot
Msrabot Msrabot
NPbot NICErsPRO
Nameprotect Nameprotect
NearSite NearSite
Net Vampire Needle
NetAnts Nessus
NetLyzer NetLyzer
NetMechanic
NetSpider NetSpider
NetZIP Netcraft
Netvibes Nettrack
NextGenSearchBot NextGenSearchBot
Niki-bot Niki-bot
Ninja Nikto
Not Not
Offline Explorer Octopus
OnCrawl Offline Navigator
OpenVAS
Openfind
Openvas Openvas
OutclicksBot OutclicksBot
OutfoxBot
PHPCrawl PHPCrawl
PageAnalyzer POE-Component-Client-HTTP
PageScorer Page Analyzer
PageThing.com PageThing.com
Pandalytics Pandalytics
Pavuk Pi-Monster
Petalbot Picscout
Piepmatz Picsearch
Pimonster Pu_iN
Pixray PxBroker
ProWebWalker Quick-Crawler
PyCurl
QueryN Metasearch
RankActiveLinkBot RankActiveLinkBot
Reaper RankFlex
Rankivabot
ReGet
RealDownload
RebelMouse
Recorder
RedesScrapy
RocketCrawler
Rogerbot Rogerbot
SBIder SBIder
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOstats
SalesIntelligent
ScanAlert ScanAlert
Scanbot Scrapy
ScreenerBot ScreenerBot
Searchestate
Semrush Semrush
SemrushBot SemrushBot
SeoSiteCheckup SeoSiteCheckup
Seomoz Seomoz
Shodan Siphon
SiteCheckerBotCrawler SiteLockSpider
SiteSnagger SiteSnagger
Siteimprove SmartDownload
Sitevigil Snoopy
Snake SocialRankIOBot
Snapbot Sogou web spider
Sociscraper
Sosospider
Spammen Spammen
Spanner Spanner
Spbot Stripper
Sucker
Sucuri Sucuri
SurveyBot SuperBot
Suzuran
T0PHackTeam
T8Abot
TeleportPro
Telesoft Telesoft
Telesphoreo Telesphorep
The Intraformant Thumbor
TheNomad Titan
TightTwatBot Toata
Tracemyfile
Trendiction
Trendictionbot
True_Robot
Turingos
Turnitin Turnitin
TurnitinBot Twice
TwengaBot URLy Warning
VB Project URLy.Warning
VelenPublicWebCrawler UnisterBot
Upflow
VCI
VeriCiteCrawler VeriCiteCrawler
Virusdie Virusdie
VoidEYE VoidEYE
Voil
Voltron
WASALive-Bot
WEBDAV WEBDAV
WPScan
WWW-Collector-E
WWWOFFLE WWWOFFLE
Web Collage Web Auto
WebAuto Web Enhancer
WebBandit Web Fuck
WebCollage Web Pix
Web Sauger
Web Sucker
WebCopier
WebEnhancer WebEnhancer
WebGo IS
WebImageCollector WebImageCollector
WebSauger WebPix
WebStripper
WebWhacker WebWhacker
WebZIP WebZIP
Webshag Webshag
WebsiteQuester WebsiteQuester
Webster Whack
Whatweb
WinHTTrack
Wonderbot
Wotbox Wotbox
Wprecon Wprecon
Xaldon WebSpider YoudaoBot
Zermelo Zauba
Zeus ZmEu
ZoomBot ZoomBot
ZyBorg ZoominfoBot
adscanner adscanner
backlink-check arquivo-web-crawler
cah.io.community cah.io.community
cognitiveseo coccocbot
crawler.feedback com.plumanalytics
dataforseo.com
domainsproject.org domainsproject.org
gopher facebookscraper
instabid
internetVista monitor internetVista monitor
isitwp.com
linkdexbot linkdexbot
meanpathbot
muhstik-scan muhstik-scan
netEstate NE Crawler
oBot
page scorer page scorer
pcBrowser plumanalytics
probe-image-size ripz
s1z.ru seoscanners
scan.lol serpstatbot
seocompany.store
sexsearcher sexsearcher
sitechecker.pro sitechecker.pro
sp_auditbot spyfu
sysscan sysscan
tAkeOut tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com voyagerx.com
webmeup-crawler webpros.com
webprosbot x09Mozilla
zauba.io xpymep1.exe
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
1314dh.com brainxs.us
adtech.de e-buyeasy.com
cottageofgrace.com figuringmoneyout.com
elektronischezigarettekaufen1.beeplog.com find1friend.com
espaceinventoristes.com gosmeb.ru
islamtoday.co.za guruofcasino.com
jackwolfskinoutlet.online mydeathspace.com
keywords-monitoring-success.com powc.r.ca.d.sendibm2.com
twelvevisionspartyofcolorado.com realting-moscow.ru
zaimhelp.ru remont-mobile-phones.ru

920
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1flag.co.za 0daymusic.org
1kdailyprofit.me 1001desert.com
1kinobig.ru 1008.su
1webmaster.ml 1adult.com
21h2o.com 2rich4bitches.com
2728fb936f0.com 2x2fan.ru
3w24.com
4senses.co 4senses.co
4webmasters.com
51.la 51.la
7fon.ru 7zap.com
7yue.org abcdeg.xyz
a.frcls.fr abcguru.xyz
aa625d84f1587749c1ab011d6f269f7d64.com absolutelyanalog.com
aad1f4acb0a373420d9b0c4202d38d94fa.com adclickthru.net
abclauncher.com addtoadd.men
abwa.tk admitad.com
adexprts.com aero2.ru
adloads.com agapovdg.ru
admanaerofoil.com agysya.ru
ads.gold akita.kz
adsland.men akusherok.ru
adultwebhosting.info albumsuper.info
advertex.info alert.scansafe.net
adzerg.com alfapro.ru
alert-hgd.xyz aliexpresscashback.club
alice.it allnews24.in
alkoravto.ru alloysteel.ru
alot.com ananumous.ru
anaksma.info
anal-acrobats.com
androids-store.com androids-store.com
apifasterlightin-a.akamaihd.net anjalika.co.in
apispringsmartne-a.akamaihd.net aosheng-tech.com
apple.com-webbrowsing-security.review apiholdingmypage-a.akamaihd.net
appsecurityr.com apioutoboxnet-a.akamaihd.net
aquarium-pro.ru apxeo.info
arcadepages.com asacopaco.tk
asociatia-tipografilor-transilvania.ro asdfz.pro
asscenihotosidea.blogspot.co.za asia-forum.ru
automobile-spec.com asscenihotosidea.blogspot.com
autoua.top asseenontv.ru
avon-severozapad.ru atmagroup.ru
axbocz.net audiobangout.com
ayanaspa.com auspolice.com
azazu.ru autoblogger24.info
autobrennero.it
autoplate.info
aviapanda.ru
awaybird.ru
azbukafree.com azbukafree.com
babyguns.ru backlink4u.net
baixar-musicas-gratis.com baltgem.com
balitouroffice.com banan.tv
bannerpower.com
bannerspace.com bannerspace.com
bard-real.com.ua best-games.today
beachpics.com best-seo-software.xyz
belsetirehafi.tk best-way.men
best-deals-products.com bestchoice.cf
best-seo-offer.com biglistofwebsites.com
bestbrainprod.win blackcurranthumidifiers.site
bet-prognoz.com blockety.co
betterscooter.com blockworld.ru
bezsporno.ru blog.koreadaily.com
bikini-image.com blog.remote-computer.de
bkgr.se blogqpot.com
blue-square.biz bloke.com
bluejays-jerseys.us boazpower.com
bluesman.nu bombla.org
bmw-ark.ru boostmyppc.com
boleznikogi.com brainsandeggs.blogspot.com
bolezniorganov.ru
bolitgorlo.net
bonux.nextview.ru
bookmaker-bet.com
brandedleadgeneration.com brandedleadgeneration.com
brandehk.dk bravegirlsclub.com
brewdom.ru brillianty.info
brianhenry.co bristolhotel.com
bukleteg.xyz bronzeaid-a.akamaihd.net
buscarfoto.com brus.city
buyfriend.ru burger-tycoon.com
cam-kontakte.org buyessay3.blogspot.ru
candlehandmade.com buypuppies.ca
candypeople.se buytizanidineonlinenoprescription.blogspot.com
cartierlovebracelet.xyz buzz-porno.info
cartierreplica.win bvlgaribracelet.xyz
casinorewards.info cacheimages.com
cdncash.org calvet.altervista.org
centraletermice.us candiceloves.us
cerev.info canrole.com
cfacarrosserie74.com carabela.com.do
cheap-trusted-backlinks.com carson.getenjoyment.net
cheapbelstaff.online carsplate.com
cheapmarmot.online cartechnic.ru
check-this-out-now.online cartierbracelet.xyz
cherrypointplace.ca cartierlove2u.xyz
chimiver.info cartierloveringreplica.xyz
chinese-amezon.com cartujano-pre.de
centre-indigo.org.ua
certifywebsite.win
chatroulette.si
cheapcigarettesc.info
chidporn.com
cinemaenergy-hd.ru cinemaenergy-hd.ru
clean-virus-mac.com citetick.com
cleanmypc.com clickbanksites.info
collegeessay19.blogspot.ru clippingphotoindia.com
compliance-andrew.top clknsee.com
compliance-julia.top clmforexeu.com
compliance-mark.top compliance-alexa.top
confib.ifmo.ru compliance-viktor.top
cordstrap.cc conocer-sanabria.com
counterbot.com cosmeticswomens-womensports.rhcloud.com
covadhosting.biz counciltally.com
credit.co.ua counter.bloke.com
creditnation.ru craftburg.ru
creditwell.ru craftinsta.ru
custom-product-labels.com crest3d.ru
dairyindia.in cubook.supernew.org
data1.scopich.com culturevie.info
date-withme.com d0t.ru
dayibiao.com dangphoto.trade
dekorkeramik.ru day-news.info
demian.kz daydream-studio.ru
design-ideas.info de.zapmeta.com
detki-opt.ru dealighted.com
detmebel.su dellalimov.com
detoxmed24.ru deniven.1bb.ru
dichvuvesinhhanoi.com diesel-parts28.ru
dickssportinggoods.com dikqlyremy.info
discountliv.com dimkino.ru
diusyawiga.tk diy-handmade-ideas.com
doska-vsem.ru djihispano.com
dogsrun.net
dostavka-v-krym.com dostavka-v-krym.com
downloadmefiranaratb1972.xpg.com.br downloadeer.net
doxysexy.com drivotracker.com
drupa.com easytuningshop.ru
dstroy.su educontest.net
dvr.biz.ua eduinfosite.com
ebonyporn.site ee77ee.com
ekn-art.se egvar.net
elektrischezigarette2.devhub.com ekaterinburg.xrus.org
elektroniksigaraankara.info ekto.ee
elektrischezigarette1.blog.pl
elektrozigaretten1.tumblr.com
elektrozigaretten2.yn.lt elektrozigaretten2.yn.lt
elitesportsadvisor.com elstal.com.pl
enge-fotzen.info elvenar.com
engines-usa.com enternet.ee
erotiktreff24.info envaseslotusama.com
estibot.com erot.co
escort-russian.com
essay-writing.work
eu-cookie-law.blogspot.com
euroskat.ru euroskat.ru
exci.ru evaashop.ru
eyes-on-you.ga evening-dating-club.info
ezigarettenshop2.myblog.de ewebarticle.info
facialporntube.com exportshop.us
fantasticpixcool.com ezigarettenshop2.postbit.com
fba-mexico.com f0815.de
fdzone.org
fearcrow.com
fefo.gdn fefo.gdn
femdom.twiclub.in fil.ru
fetlifeblog.com filesmonster.porn
fickblock18.com filmi-onlain.info
findpik.com financepoints.eu
findsexguide.com flex4launch.ru
fireads.men flipper.top
flagcounter.me flirt4free.com
foodcrafts.website free-social-buttons.com
foodgid.net fres-news.com
formseo.com froggytube.com
foxtechfpv.com funnel.co.za
franch.info galleryawesome.com
freenom.link gaming-journal.com
frustrated-favorable.gq gavuer.ru
funnypica.com gazoblok.net.ua
fxtips.ru gearsadspromo.club
galaxy-family.ru gemgrab-a.akamaihd.net
gelezki.com getyourimage.club
ghazel.ru
girlsatgames.ru girlsatgames.ru
glall.ru glicol.kz
gnuetella.com goodhumor24.com
goatse.ru gorabagrata.ru
gobongo.info gotcher.us
goldadpremium.com greenzaim.ru
goldandcard.ru growboxbl.ru
gombita.info grtyi.com
goo.ne.jp gsasearchengineranker.site
google-liar.ru
gosarhivrt.ru
green-tea.tv
gta-club.ru gta-club.ru
guitar-master.org guardlink.com
h2monline.com guge.io
hermesreplica.win guod.me
himazin.info gwagka.com
home-task.com gwhwpxbw.bloger.index.hr
homedecoguide.info hanink.biz.ly
homemature.net hard-porn.mobi
homik.pw hdseriale.pl
hoporno.com hdxnxxtube.mobi
hotchatdate.com hentai-manga.porn
hotshoppymac.com herehloadibs.cf
houtings.xyz hermesbelts.xyz
hoverboard360.se highland-homes.com
howlongdoestizanidinestayinyoursystem.blogspot.com hostsshop.ru
howopen.ru houston-vikings.com
hptwaakw.blog.fc2.com husky-shop.cz
humanelydrew.com i-service.kz
hyiphunter.org ibb.com.ua
iboss.com idealtits.net
ilte.info ilmexico.com
imabase.com imgata.com
incep.imagine-ex.co iminent.com
infobabki.ru importchinacoach-teach.com
infospot.pt in-tandem.co
intervsem.ru incomekey.net
inzn.ru indetiske.ya.ru
istanbulit.com infazavr.ru
j33x.com infogame.name
javlibrary.cc infowarcraft.ru
journalhome.com invest-pamm.ru
kabbalah-red-bracelets.com iopeninghours.co.uk
kakadu-interior.com.ua iqoption-bin.com
kalb.ru istizanidineanarcoticdrug.blogspot.com
kambasoft.com ivoiretechnocom.ci
kchaxton.com jbl-charge.info
ketoanhanoi.info jerseysbizwholesalecheap.com
keywordhouse.com jobgirl24.ru
kihi.gdn jumptap.com
kiinomaniak.pl kevblog.top
kmgamex.cf keywordglobal.co.za
keywordsuggest.org
kino-ecran.ru
kino2018.cc
kinopolet.net
kirov.zrus.org
kladrus.ru
kongoultry.net kongoultry.net
konkursowo-24.pl kopihijau.info
konturkrasoty.ru krafte.ru
kosmetyki.tm.pl kurbappeal.info
kvartiry-remont.ucoz.ru kustanay.kz
labplus.ru lawyersinfo.org
lashstudia.ru laxdrills.com
lazy-z.com lernur.net
lechtaczka.net lescinq.com
legalrc.biz levaquin750mg.blogspot.com
lemon-ade.site life-instyle.com
letmacworkfaster.world
letsart.ru
linkdebrideur.xyz linkdebrideur.xyz
linksharingt.com lion.snapmap.us
lion.misba.us liran-locks.com
lionking-1994.blogspot.com liveu.infoteka.hu
liquimondo.com locationdesaison.com
live-sexcam.tk lsp-awak-perikanan.com
livejasmin.com
lol-smurfs.com
lostfilm-online.ru
luchshie-topcasino.ru luchshie-topcasino.ru
luckyshop.net.ua m4ever.net
luxmagazine.cf mac-shield.com
m5home.ru mafcards.ru
magda-gadalka.ru
maglid.ru
mailemedicinals.com
matematikus.info matematikus.info
mebel-ekb.com mcnamaratech.com
member-quiz.com medicinacom.ru
memberty.com medictube.ru
metallosajding.ru megakino.net
metaxalonevstizanidine.blogspot.com mere.host.sk
methodsmarketing.com mesto-x.com
michellblog.online metafilter.com
middlerush-a.akamaihd.net mhi-systems.ru
milblueprint.com mhtr.be
minyetki.ru mielec.pl
missclub.info mindbox.co.za
misswell.net miniads.ca
mixx.com mirobuvi.com.ua
moesonce.com misandesign.se
monclerjacketsoutlet.pw missvietnam.org
monclerjacketsoutlet.win mobplayer.ru
moneyzzz.ru monclercoats.xyz
mp3ritm.top monetizationking.net
musicktab.com morepoweronmac.trade
mygameplus.ru morf.snn.gr
myhealthcare.com muabancantho.info
myonigroup.com musicpro.monster
naj-filmy24.pl myperiod.club
naturalbreakthroughsresearch.com na-telefon.biz
naturalpharm.com.ua nakrutka.cc
news-speaker.com narco24.me
newtechspb.ru nastroyke.net
nfljerseysforsalewholesaler.com naval.jislaaik.com
niki-mlt.ru netvouz.com
niroo.info noumeda.com
nmrk.ru novatech.vn
no-rx.info nrjmobile.fr
nomuos.it
norththeface.store
novgorod.xrus.org
nvformula.ru nvformula.ru
o-o-6-o-o.com o00.in
oddamzadarmo.eu ochistka-stokov.ru
offer.camp ofanda.com
offergroup.info offtime.ru
offtopic.biz okout.ru
ok-ua.info
okroshki.ru
oksrv.com
omgtnc.com
onlinewritingjobs17.blogspot.ru
onlyforemont.ru
onlywoman.org onlywoman.org
onthemarch.co
open-odyssey.org
opinionreelle.com opinionreelle.com
optom-deshevo.ru
orenburg-gsm.ru
orgasmus-virtual.com
otbelivanie-zubov.com otbelivanie-zubov.com
oyster-green.com oyster-green.com
paccohichetoti.ml palvira.com.ua
paphoselectricianandplumber.com panchro.co.uk
parlament.biz partner-stop.men
partner-cdn.men pc-virus-d0l92j2.pw
photosaga.info pcgroup.com.uy
phuketscreen.com pekori.to
picturesboss.com perfectpracticeweb.com
pila.pl picture-group.com
pisanieprac.info pintattoos.com
piulatte.cz piski.top
pizza-tycoon.com pix-hd.com
pk-pomosch.ru pizda.lol
plastgranar.nu pizza-imperia.com
pogosh.com plugingeorgia.com
pogruztehnik.ru
popmarker.com
poppen-nw.net
popugaychiki.com popugaychiki.com
popup-hgd.xyz porn-w.org
porndairy.in porno-dojki.net
porno-home365.com pornobrazzers.biz
pornonik.com pornotubexxx.name
pornotubs.com
portside.cc
postclass.com
powitania.pl
pravoholding.ru
prchecker.info prchecker.info
predmety.in.ua privacylocationforloc.com
preparevideosafesystem4unow.space prize44.com
primedice.com prizesbook.online
printdirectforless.com prodvigator.ua
prizeestates.cricket profitsport.club
prohoster.info prom23.ru
promodj.com
pronorm.fr pronorm.fr
proposal-engine.com prostitutki-kiev.org
prostitutki-rostova.org proxyelite.biz
puzzleweb.ru puserving.com
queerspace.com pussyspace.net
qwarckoine.com quebec-bin.com
radiogambling.com randalljhoward.com
ranksignals.com realtytimes.com
raschtextil.com.ua rednise.com
razorweb-a.akamaihd.net regdefense.com
redhotfreebies.co.uk registratciya-v-moskve.ru
relayblog.com
remont-ustanovka-tehniki.ru remont-ustanovka-tehniki.ru
remontbiz.ru renhacklids.tk
renecaovillasale.online resant.ru
rfd-split.hr research.ifmo.ru
rfid-locker.co rockingclicks.com
ribieiendom.no rozalli.com
ringporno.com rvi.biz
rixpix.ru salmonfishingsacramentoriver.com
romhacking.ru sarafangel.ru
russian-postindex.ru sarahmilne.top
rvtv.ru sashagreyblog.ga
rvzr-a.akamaihd.net satellite.maps.ilovevitaly.com
saitevpatorii.com saugatuck.com
savetubevideo.com savefrom.com
sawin.beth.webd.pl sax-sex.com
scanner-elena.top scanhub.ru
scanner-mary.top scanner-marwin.top
score-ads.men search.pch.com
search.alot.com searchinterneat-a.akamaihd.net
searchlock.com senger.atspace.co.uk
seasonvar.ru seo-2-0.com
seo-tools-optimizing.com
seo18.su seo18.su
seocdvig.ru seorussian.ru
servicecenter.co.ua seotoolsagency.com
seryeznie-znakomstva.ru serviporno.com
sex-sex-sex5.com sex-tracker.com
sex-watch.com sexcamamateurchat.com
sexad.net sexkontakte-seite.com
sexfreepornoxxx.com sexkontakteao.info
sexreliz.com sexs-foto.top
sexyteens.hol.es
sexytrend.ru
shlyahten.ru
shmetall.com.ua shmetall.com.ua
shop.xz618.com shodanhq.com
sigmund-freud.co.uk sicfor.bcu.cc
sitebeam.net signal03.ru
sitechecker.pro signx.info
siteexpress.co.il similardeals.net
sixcooler.de site5.com
snomer1.ru sitiz.club
soblaznu.net skytraf.xyz
social-buttons.xyz slavia.info
sovinsteel.ru slavkokacunko.de
spasswelt.net slomm.ru
spidtest.org small-games.biz
spl63.fr snegozaderzhatel.ru
sprttrack.com sockshare.net
soft-terminal.ru
softomix.net
softonicads.com
sotechco.co
spaceshipad.com
spbchampionat.ru
spy-app.info
spywareit.com spywareit.com
star61.de stal-rulon.ru
stardevine.com startwp.org
sterva.cc statustroll.com
stmassage.ru stefanbakosab.se
stevemonsen.com
stoki.ru
stroilka.info stroilka.info
stroiminsk.com stroymonolit.su
supermama.top studentguide.ru
support.nopeas.sk suggest-keywords.com
surflinksmedical.com sukirgenk.dvrlists.com
suture.co sumo.com
swiped.su supermodni.com.ua
tagil.zrus.org suralink.com
talant-factory.ru sureone.pro
tattomedia.com surintech.ac.th
technika-remont.ru taqywu51.soup.io
tecnoteakviareggio.it teguh.info
teenforporn.com telefonsexi.com
terraclicks.com the-usa-games.blogspot.com
thedownloadfromwarez.blogspot.com thecoral.com.br
themeforest.net thegalerie.eu
tizanidineforopiatewithdrawal.blogspot.com thepornsex.org
tizanidineonline.blogspot.com theprofitsmaker.net
tizanidineusedfor.blogspot.com tivolibasket.it
tizanidinevscyclobenzaprine.blogspot.com tizanidinehcl4mgisitanarcotic.blogspot.com
topclickguru.com tizanidineingredients.blogspot.com
tovaroboom.vast.ru tizanidinestreetprice.blogspot.com
tracfone.com toolsky.com
trackzapper.com toondinsey.com
trafficfactory.biz torrent-to-magnet.com
traxdom.ru torrentred.games
triplepanda.xyz torture.ml
trubywriting.com tozup.com
tt-ipd.info track-rankings.online
tubo360.com track112.site
tula.mdverey.ru trafficjunky.net
turizmus.us traffictrade.life
twittruth.com trailer.cinemaflix.website
tsyndicate.com
tuberkulezanet.ru
tupper-shop.ru
turbabitload.weebly.com
tvand.ru
twittrading.com
tyumen.xrus.org tyumen.xrus.org
unrealcommander.com uasb.ru
upproar.com universals.com.ua
uptime-delta.net
uptimebot.net uptimebot.net
urldelivery.com urccvfmc.bloger.index.hr
usadacha.net urdoot.win
uzporno.mobi uzungil.com
validccseller.com vacuumscleaner.com
vapeface.club vasileostrovsky-rayon.ru
vavilone.com
vbtracker.net vbtracker.net
vchulkah.net
vclicks.net vclicks.net
vereo.eu viberdownload10.com
vertaform.com video-chat.cn
viagra.pp.ua videochat.ph
videochat.bz vipromoffers.com
videonsk.com virus-schutzmasken.de
vikistars.com viteonlusarezzo.it
vizag.kharkov.ua vkak.ru
vk-mus.ru voloo.ru
vksaver-all.ru vpnmouse.com
vltai.com vrotike.ru
vodaodessa.com vysigy.su
vut.com.ru wahicbefa31.soup.io
w-journal.ru webcamtalk.net
walpaperlist.com website-stealer.nufaq.com
webshoppermac.com websitevaluebot.com
website-datenbank.de webstatsdomain.org
wetgames.ru weddingdresses.xyz
where-toget.com wesharepics.info
wildworld.site windowssearch-exp.com
witherrom55.eklablog.fr winner7777.net
wladimirpayen.com
womensterritory.ru
worldlovers.ru worldlovers.ru
writingservices17.blogspot.ru www.labves.ru
ws.ampower.me wygraj-teraz.com
xblognetwork.com x-diesel.org
xgames-04.com xchangetrak.com
xgftnlrt.bloger.index.hr xmlinde.com
xn----dtbndd4ae7eub.top xn----7sbifcamovvfggw9d.xn--p1ai
xtubeporno.net xn--1-8sbcpb0bdm8k6a.xn--p1ai
xxlargepop.com xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xxxhdvideo.site xnxx699.com
yeartwit.com xportvusbdriver8i.snack.ws
yellowads.men xvideospornoru.com
yhit.press xxxrus.org
xzlive.com
yougotanewdomain.com
youradulthosting.com youradulthosting.com
youresponsive.com yoursite.com
yourmovies.pl ytmnd.com
yugo-star.ru yugk.net
zarabotat-na-sajte.ru zarabotok--doma.ru
zarajbuilders.com zero1.it
zdesformula.ru
zdesoboi.com
zed21.net
zeikopay.com
zeroredirect.com
zeroredirect2.com zeroredirect2.com
zhongwenlink.com zhongwenlink.com
zhuravlev.info
zigarettenonlinekaufen.tumblr.com zigarettenonlinekaufen.tumblr.com
zigarettenonlinekaufen1.bloog.pl zigarettenonlinekaufen1.bloog.pl
zigarettenonlinekaufen2.bloog.pl zynax.ua
zojirushi-products.ru zytpirwai.net
zolotoy-lis.ru
zoompegs.com

480
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
007angels.com 03p.info
0lovespells0.blogspot.com 4rent.ru
1adult.com 72-news.com
2nt.ru 7fon.ru
5u.com 883zy.com
6hopping.com aad1f4acb0a373420d9b0c4202d38d94fa.com
8gold.com abovetherivernc.com
ads-seo.men adminshop.com
algarveglobal.com ads.gold
alldubai.biz adventureparkcostarica.com
allnews24.in afftrack001.com
alpinism.ru alfa-medosmotr.ru
ampower.me all-streaming-media.com
arendovalka.xyz animenime.ru
artel116.ru anti-virus-removal.info
asscenihotosidea.blogspot.com apidiamondatanet-a.akamaihd.net
avtointeres.ru arenda-avtoprokat-krasnodar.ru
azlex.uz asrvvv-a.akamaihd.net
beatifullandscape.co atlant-auto.info
bestcalovebracelet.cn bangla-chat-uk-paralud.ga
bestofferswalkmydogouteveryday.gq berrymall.ru
bet-prognoz.com best-way.men
bjetjt.com bestfortraders.com
blogporn.in bestwaystogetpaid.us
browsepulse-a.akamaihd.net bio-market.kz
bscodecs.com biteg.xyz
buyfriend.ru c1.onedmp.com
buzzsumo.com cactussoft.biz
candypeople.se carson.getenjoyment.net
chatroulette.world cartierjusteunclou.xyz
cheapestjerseysonwholesale.com cascadelink.org
check-this-out-now.online cayado.snn.gr
chorus.terakeet.com cheapjerseysfootballshop.com
citysecurity.nu chocolateslim-en-espana.com
clean-start.net cialischstgerts.com
cleanersoft.com club-musics.ru
companies-catalog.com compliance-irvin.top
compliance-checker-7.info compliance-jane.top
compliance-fred.xyz copypaste.traderzplanet.in
compliance-jess.top costablue.xyz
construmac.com.mx counciltally.com
coolwallpapers-hd.com countbertwistdisp26.soup.io
cottageofgrace.com
covadhosting.biz
customboxes4less.com
customchocolate.business-for-home.com customchocolate.business-for-home.com
daretodonate.co customergrowthsystems.com
diminishedvalueoforegon.com damasarenai.info
dipstar.org daneshetabiat.com
disruptingdinnerparties.com data-ox.com
dkvorota.ru dengi-pod-zalog-nedvizhimosti.ru
doxyporno.com design-ideas.info
e-poker-2005.com digilander.libero.it
easyshoppermac.com digitalassetmanagement.site
eloconcream.blogspot.com dostavka-v-ukrainu.ru
drillsaw.ru
earthmagic.info
easync.io
ebooknovel.club
elektrozigarette2.wordpressy.pl
elvenar.com elvenar.com
empirepoker.com en.home-task.com
eorogo.top euronis-free.com
ereko.ru excitacao.com
erotische-geschichten-xxl.com extremepornos.net
espaceinventoristes.com ezofest.sk
etotupo.ru faithe.top
eyessurgery.ru farmprofi.net
ezrvrentals.com fetishinside.com
falcoware.com filesclub.net
fanoboi.com fodelsedagspresenter.nu
fashion.stellaconstance.co forum.tvmir.org
fastcrawl.com foto-weinberger.at
fenoyl.batcave.net fotoxxxru.com
fil.ru foxweber.com
firesub.pl free-stock-illustration.com
firma-legion.ru fyl.com.ru
fitness-video.net geckoandfly.com
forums.d2jsp.org get-seo-domain.com
frankofficial.ru getmiro.com
free-today.com given2.com
fres-news.com graphicwe.org
freshdz.com gsmtlf.ru
getlaid-xxxhookupdirect.com gwhwpxbw.bloger.index.hr
getoutofdebtfree.org hausfrauensex18.com
getridofstretchmarks.org hccoder.info
gratuitxblcodes.com hd-filmy.net
hackers-crackers.tk hd720kino.ru
hdwallpapers-free.com helicalpile.us
headpharmacy.com hermesbracelets.xyz
hentaiheroes.com hit-men.men
hermesbirkinhandbagoutlets.com holidaypics.org
homedo.fabpage.com homedo.fabpage.com
homeinns.com homelygarden.com
homemade.gq homemature.net
hosting-tracker.com hothor.se
huimin764128.com hully.altervista.org
hustoon.over-blog.com hydropump.su
i-midias.net.br ilikevitaly.com
igrovyeavtomaty777.ru impresagaia.it
iklysha.ml incolors.club
ilmexico.com indetiske.ya.ru
iloveitaly.ru indiakino.net
jav-way.site instabid.tech
jaxcube.info invitefashion.com
kabbalah-red-bracelets.com ip-guide.com
kiwi237au.tk isitpaleo.info
kutikomi.net istripper.com
lacloop.info kakablog.net
katjimej.blog.fc2.com
keywordsuggests.com
kino-ecran.ru
kinogolos.ru
kinotorka.ru
kiprinform.com
kletkimehan.ru
krasula.pp.ua
laexotic.com
lampokrat.ws
laptop-4-less.com laptop-4-less.com
larger.io laudit.ru
le-clos-des-alouettes.com linkredirect.biz
leadwayau.com linkrr.com
likrot.com louboutinreplica.pw
lineavabit.it luciddiagnostics.in
livetsomudvekslingstudent.bloggersdelight.dk macotool.com
loginduepunti.it magiadinamibia.blogspot.com
lomb.co makemoneyonline.com
lonerangergames.com makenahartlin.com
louboutinreplica.xyz mamasuper.prom.ua
love-baby.cz manuscript.su
m5home.ru matchpal-a.akamaihd.net
machicon-ueno.info medicinacom.ru
mackeeper-land-672695126.us-east-1.elb.amazonaws.com mensandals.xyz
markjaybeefractal.com metalonly.info
meget.co.za mielec.pl
mesto-x.com mobilierland.com
metabar.ru monclercoats.xyz
mexicosleevegastrectomy.com monetizer.com-01.site
mikrobiologies.ru moscow.xrus.org
misandesign.se movies-in-theaters.net
modabutik.ru mybackgroundlandscape.blogspot.com
modnie-futbolki.net nash-krym.info
mountainstream.ms new-post.tk
mulberryoutletonlineeu.com nuit-artisanale.com
musicspire.online oddamzadarmo.eu
mylesosibirsk.ru offtime.ru
nalogovyykodeks.ru ooomeru.ru
nastydollars.com opinionreelle.com
nlfjjunb5.ru origin-my.ru
norththeface.store ossmalta.com
notasprensa.info pendelprognos.se
nudepatch.net pensplan.com
offer.wpsecurity.website peugeot-club.org
officedocuments.net phormchina.com
online-podarki.com phrcialiled.com
openmediasoft.com pic-re.blogspot.com
outrageousdeal-a.akamaihd.net pinapchik.com
partner-high.men pinkduck.ga
pateaswing.com pixell.club
pc-virus-d0l92j2.pw plastgran.com
pcads.ru pogruztehnik.ru
pharmacyincity.com popcash.net
photo.houseofgaga.ru poptool.net
picurams.pw popup.matchmaker.com
pihl.se pornoblood.com
pixelrz.com pornonik.com
plastgranar.nu
ploenjitmedia.azurewebsites.net
pogodnyyeavarii.gq
polytopesexempt.com
pornophoto.xyz pornophoto.xyz
preventheadacheguide.info pozdravleniya-c.ru
priscilarodrigues.com.br projefrio.com.br
privhosting.com prostitutki-rostova.org
promalp-universal.ru py100.ru
queerspace.com questionmarque.ch
quickloanbank.com refudiatethissarah.info
rangoman.date relax.ru
rcb101.ru
rebelmouse.com
regionshop.biz
reimageplus.com
remont-fridge-tv.ru remont-fridge-tv.ru
rfd-split.hr renecaovilla.online
ringtonepartner.com reyel1985.webnode.fr
rocis.site sa-live.com
rock-cafe.info sabaapress.com
royal-betting.net salut-camp.ru
roznica.com.ua satoristudio.net
rucrypt.com schoolfiles.net
rusprostitute.com searchkut.com
rvtv.ru senger.atspace.co.uk
s-iwantyou.com serptehnika.ru
sashagreyblog.ga service-core.ru
scanner-brian.top sevendays.com.ua
scanner-olga.top sex-watch.com
scripted.com sex-znakomstva.online
semxiu.com sexobzor.info
servingnotice.com sexyali.com
seryeznie-znakomstva.ru sexyebonyteen.com
sexy-pings.com shopfishing.com.ua
share-buttons-for-free.com sibtest.ru
sisiynas.ru sisi-go.ru
smailik.org skladvaz.ru
smartadserver.com smichovbike.cz
smartshoppymac.com snegozaderzhatel.ru
soc-econom-problems.ru sobecjvuwa.com.ru
soietvousmaime.fr special-porn.com
sortthemesitesby.com stats-public.grammarly.io
sports-supplements.us staynplay.net
stairliftsarea.com stefanbakosab.se
stickers-market.ru
strangeduckfilms.com strangeduckfilms.com
survival.betteroffers.review surcentro.com
svbur.ru symphonyintegratedhealthcare.com
tarad.com takeflyte.com
teesdaleflyballclub.co.uk talant-factory.ru
theallgirlarcade.com taqplayer.info
theendivechronicles.com teguh.info
titelhelden.eu templates.franklinfire.co
titslove.yopoint.in thegameriders.com
tizanidinewithdrawal.blogspot.com thegioixekhach.com
torrent-newgames.com tizanidineinfo.blogspot.com
totu.us tizanidineoral.blogspot.com
toys.erolove.in tv-spoty.info
tracksurf.daooda.com uasb.ru
traxdom.ru unionmarkt.de
trubywriting.com universals.com.ua
tubo360.com upupa.net
tvoystartup.ru urlopener.com
ukrobstep.com urzedowski.eu
ultramart.biz vacuumcleanerguru.com
uptimebot.net vchulkah.net
urldelivery.com versaut.xxx-cam.webcam
viagroid.ru vipps.com.my
videotuber.ru vvon.co.uk
vip-file.com vzubah.com
viveresaniesnelli.it vzube.com
vriel.batcave.net wardreapptokone.tk
w3data.co watchinf.com
wanker.us wesharepics.site
we-are-gamers.com
webpromotion.ae
weightatraining.com
welck.octopis.com
whatsupinfoley.com whatsupinfoley.com
witclub.info wholesalejerseysgaa.com
wrona.it wufak.com
wwwadultcheck.com www.pinnacle-bets.com
xitjw.info x-rates.ru
xn--80aafb2a.xn--p1ai x5market.ru
xn--80ahvj9e.xn--p1ai xmnb.net
xn--90acenikpebbdd4f6d.xn--p1ai xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
xn--oogle-wmc.com
xxxhdvideo.site
ynymnwbm.bloger.index.hr ynymnwbm.bloger.index.hr
yoopsie.com youporn-ru.com
youradexchange.com yourtemplatefinder.com
zaimhelp.ru yuarra.pluto.ro
zajm-pod-zalog-nedvizhimosti.ru zarabiaj-dzis.pl
zarabotat-na-sajte.ru zigarettenonlinekaufen2.bloog.pl
zazagames.org
zojirushi-products.ru zojirushi-products.ru
zoosexart.com zona-aqua.ru
zrelaya.pw zverokruh-shop.cz
zrizvtrnpale.tk zytpirwai.net

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.128 0; 10.1.0.52 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.02.3572 ### Version: V4.2023.02.3573
### Updated: Thu Feb 9 22:04:58 UTC 2023 ### Updated: Fri Feb 10 22:01:31 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.02.3572 ### Version: V4.2023.02.3573
### Updated: Thu Feb 9 22:04:58 UTC 2023 ### Updated: Fri Feb 10 22:01:31 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##