vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 04:00:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.04.3074 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-04-29 10:09:19 +00:00
parent dfc5c82b1a
commit e39e5525df
46 changed files with 91828 additions and 91828 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3073 #### Version: V4.2022.04.3074
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3073 #### Version: V4.2022.04.3074
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.04.3073 #### Version: V4.2022.04.3074
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11344
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.04.3073 #### Version: V4.2022.04.3074
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11348
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11348
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11348
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11348
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 1; 10.1.0.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 0; 10.1.0.217 0;
127.0.0.1 0; 127.0.0.1 0;

11348
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 1; 10.1.0.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 0; 10.1.0.217 0;
127.0.0.1 0; 127.0.0.1 0;

11348
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 1; 10.1.0.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 0; 10.1.0.217 0;
127.0.0.1 0; 127.0.0.1 0;

11348
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3073 3074

11348
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11348
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.83.242.98 0; 40.78.153.135 0;

11348
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 1; 10.1.0.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 0; 10.1.0.217 0;
127.0.0.1 0; 127.0.0.1 0;

11348
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 1; 10.1.0.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 0; 10.1.0.217 0;
127.0.0.1 0; 127.0.0.1 0;

11348
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 1; 10.1.0.217 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 0; 10.1.0.217 0;
127.0.0.1 0; 127.0.0.1 0;

11348
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.83.242.98 0; 40.78.153.135 0;

11348
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.83.242.98 0; 40.78.153.135 0;

11348
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 1; 10.1.0.217 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AiHitBot DomCopBot
AwarioSmartBot ExtLinksBot
BackStreet Lipperhey
Foobot ReGet
JOC Web Spider SiteSnagger
JikeSpider Stripper
Keybot Translation-Search-Machine Sucker
Minefield T8Abot
Sqlworm Twice
WISENutbot WebGo IS

300
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
404checker 404checker
80legs ADmantX
AIBOT AIBOT
Aboundexbot ASPSeek
Abonti
AfD-Verbotsverfahren
AiHitBot
Aipbot Aipbot
Alexibot Alexibot
Anarchy Alligator
AlphaBot
Anarchy99 Anarchy99
Asterias Ankit
Attach Anthill
AwarioSmartBot Apexoo
BBBike Aspiegel
BDCbot Badass
BackStreet
Backlink-Ceck
Bandit Bandit
BatchFTP Barkrowler
Battleztar Bazinga Battleztar Bazinga
Bitacle
Black Hole
BlackWidow
Boardreader
Bolt Bolt
Bullseye BotALot
CSHttp Buck
CheTeam Buddy
CheeseBot CCBot
CherryPicker CazoodleBot
Chlooe Cegbfeieh
Citoid
Claritybot
Copyscape Copyscape
Cosmos Cosmos
CrazyWebCrawler Curious
Crescent
Custo
CyotekWebCopy CyotekWebCopy
DBLBot DBLBot
DIIbot
DSearch
DTS Agent
Demon Demon
Deusu Deusu
DigitalPebble
Dirbuster
Discobot Discobot
Dispatch Dispatch
DittoSpyder DnBCrawler-Analytics
DomainAppender DomCopBot
DomainSigmaCrawler
Domains Project Domains Project
Dotbot Dotbot
Drip
ECCP/1.0 ECCP/1.0
EasyDL
Ecxi Ecxi
Exabot Evil
Express WebPictures
Extractor Extractor
ExtractorPro EyeNetIE
Extreme Picture Finder
Ezooms
FDM
FHscan
FemtosearchBot
Fimap Fimap
FlashGet
Flunky Flunky
Foobot Foobot
FrontPage Freeuploader
Fuzz Fuzz
FyberSpider FyberSpider
Fyrebot GalaxyBot
GT::WWW
Genieo Genieo
Go!Zilla GetWeb
GrabNet Go-Ahead-Got-It
Grabber GoZilla
GrapeFX Gotit
GrapeshotCrawler
GridBot GridBot
HEADMasterSEO HEADMasterSEO
Haansoft Haansoft
HaosouSpider HonoluluBot
Havij Humanlinks
Heritrix
Hloader
HybridBot
IDBTE4M
Iblog Iblog
Id-search
Image Fetch Image Fetch
IndeedBot Image Sucker
Indy Library InfoNaviRobot
InfoTekies Iskanie
InterGET
InternetSeer
JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz JennyBot
Joomla Jetty
JikeSpider
Jorgee Jorgee
JustView Jyxobot
Kenjin Spider LWP::Simple
LNSpiderguy
Larbin Larbin
LeechFTP Leap
LexiBot LeechGet
Lftp Libwhisker
Lightspeedsystems LinkScan
Likse LinksManager
LinkWalker MFC_Tear_Sample
LinkextractorPro MSFrontPage
LinqiaRSSBot MSIECrawler
LinqiaScrapeBot Mag-Net
Lipperhey
MIDown tool
MJ12bot
Magnet Magnet
Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic12 Mass Downloader
Masscan Mb2345Browser
MeanPath Bot Mediatoolkitbot
Meanpathbot Minefield
Metauri Moblie Safari
Mojeek Mojeek
Mojolicious Mojolicious
MolokaiBot Msrabot
Mr.4x3 NPbot
Name Intelligence Name Intelligence
NearSite
Nessus
Net Vampire Net Vampire
NetLyzer NetLyzer
NetSpider NetMechanic
Netcraft NetZIP
Nettrack NextGenSearchBot
Niki-bot Niki-bot
Nikto NimbleCrawler
Octopus Octopus
Offline Explorer Offline Explorer
Offline Navigator
OpenVAS OpenVAS
Openfind
PECL::HTTP PECL::HTTP
PHPCrawl POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer PageAnalyzer
Pandalytics
Panscient
Papa Foto Papa Foto
PeoplePal
Picsearch Picsearch
Piepmatz PleaseCrawl
Pimonster
Pockey Pockey
ProPowerBot ProPowerBot
ProWebWalker Psbot
Probethenet PyCurl
Pu_iN
PxBroker
QueryN Metasearch QueryN Metasearch
Quick-Crawler
RankActiveLinkBot RankActiveLinkBot
RankingBot RankingBot
Rankivabot
RankurBot RankurBot
Re-re
ReGet ReGet
RedesScrapy RealDownload
RepoMonkey RebelMouse
Rogerbot RocketCrawler
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SISTRIX
SEOstats
SalesIntelligent SalesIntelligent
ScoutJet ScoutJet
Scrapy Scrapy
Screaming Screaming
ScreenerBot
ScrepyBot
Searchestate Searchestate
SearchmetricsBot SearchmetricsBot
Semrush Seekport
SemanticJuice
SentiBot
Seomoz Seomoz
Shodan SiteCheckerBotCrawler
Siphon SiteExplorer
Site Sucker Sitevigil
SiteSucker Snapbot
Sitebeam Snoopy
Sociscraper SocialRankIOBot
Sqlmap Sosospider
Sottopop
SpankBot
Spinn3r
Stripper
Sucuri
SuperBot SuperBot
SurveyBot Swiftbot
T0PHackTeam
Teleport Teleport
TeleportPro TeleportPro
Telesoft
Telesphorep
TheNomad
Thumbor
Toata Toata
Toweyabot Toweyabot
Tracemyfile Trendiction
Trendictionbot
True_Robot
TurnitinBot TurnitinBot
TwengaBot
Twice Twice
Typhoeus Typhoeus
UnisterBot URLy.Warning
Upflow V-BOT
Vacuum
Vagabondo Vagabondo
VeriCiteCrawler
Virusdie
Voltron Voltron
WASALive-Bot
WPScan WPScan
WWW-Collector-E WWW-Mechanize
WWW::Mechanize
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD
WeSEE
Web Auto
Web Enhancer Web Enhancer
Web Sauger Web Pix
Web Sucker Web Sucker
WebGo IS WebAuto
WebBandit
WebEnhancer
WebImageCollector WebImageCollector
WebLeacher WebStripper
WebReaper
WebSucker WebSucker
WebmasterWorldForumBot Webshag
WebsiteExtractor
WebsiteQuester
Webster
Whack Whack
Who.is Bot Whatweb
Widow Widow
WiseGuys Robot WiseGuys Robot
Woobot Woobot
Wotbox
Wprecon Wprecon
Xaldon WebSpider
YoudaoBot
Zade
Zermelo Zermelo
ZmEu Zeus
ZoomBot ZoomBot
ZumBot ZumBot
autoemailspider adscanner
backlink-check archive.org_bot
arquivo-web-crawler
coccocbot
cognitiveseo cognitiveseo
gopher com.plumanalytics
heritrix demandbase-bot
internetVista monitor evc-batch
linkdexbot facebookscraper
lwp-request lwp-request
lwp-trivial
netEstate NE Crawler netEstate NE Crawler
oBot
pcBrowser pcBrowser
probe-image-size plumanalytics
ripz polaris version
s1z.ru satoristudio.net
seobility scalaj-http
scan.lol
seocompany.store
seoscanners seoscanners
sexsearcher seostar
siteripz serpstatbot
sogouspider sogouspider
sp_auditbot sp_auditbot
spyfu spyfu
sysscan
trendiction.de trendiction.de
ubermetrics-technologies.com
webgains-bot webgains-bot
webmeup-crawler webpros.com
webprosbot
x09Mozilla
zauba.io zauba.io
zgrab

306
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 80legs
ADmantX
AIBOT
ASPSeek ASPSeek
AfD-Verbotsverfahren Abonti
AhrefsBot Aboundex
AiHitBot AiHitBot
Alexibot Alexibot
Alligator
AlphaBot
Anarchy Anarchy
Ankit
Anthill
Aspiegel Aspiegel
AwarioRssBot Attach
BBBike BBBike
BDCbot BDCbot
BDFetch BDFetch
BLEXBot BackStreet
BackWeb BackWeb
Backlink-Ceck BacklinkCrawler
BatchFTP Badass
Bitacle Bandit
Black Hole Barkrowler
Bigfoot
Blackboard Blackboard
BlowFish BlowFish
Boardreader
Buck Buck
Buddy Bullseye
BuiltWith BuzzSumo
CCBot CODE87
CensysInspect
CheeseBot
CherryPicker CherryPicker
Chlooe Chlooe
Citoid
Cogentbot Cogentbot
Collector CopyRightCheck
Craftbot Copyscape
Crawling at Home Project Crescent
CrunchBot CrunchBot
Curious
CyotekWebCopy
DBLBot DBLBot
Demon DatabaseDriverMysqli
Devil Devil
Dispatch DigitalPebble
DittoSpyder DnBCrawler-Analytics
DnyzBot DnyzBot
DomainAppender
DomainStatsBot
Dotbot Dotbot
Download Wonder
Dragonfly Dragonfly
Drip
ECCP/1.0
EMail Wolf EMail Wolf
EasyDL
Ebingbong Ebingbong
Ecxi EirGrabber
Exabot EroCrawler
Express WebPictures
ExtLinksBot
Extractor
ExtractorPro ExtractorPro
FemtosearchBot
Fimap Fimap
Flunky Firefox/7.0
Foobot Foobot
GT::WWW Freeuploader
Go!Zilla Genieo
GermCrawler
GetRight
GetWeb
Getintent
Gigabot
GoZilla
Gotit Gotit
GrabNet
Grabber Grabber
Grafula GrapeshotCrawler
GrapeFX HTTrack
HEADMasterSEO Haansoft
HTTP::Lite HaosouSpider
Havij Harvest
Heritrix
Hloader
Humanlinks Humanlinks
IRLbot HybridBot
Id-search Id-search
Image Fetch IlseBot
Indy Library Image Sucker
InfoNaviRobot InterGET
InfoTekies
Internet Ninja
InternetSeer InternetSeer
JOC Web Spider Iria
Iskanie
IstellaBot
Jbrofuzz Jbrofuzz
JennyBot JennyBot
JikeSpider
Joomla Joomla
Keybot Translation-Search-Machine Jorgee
Keyword Density Keyword Density
Kinza
Kozmosbot Kozmosbot
Leap Leap
LeechFTP LeechFTP
LexiBot LexiBot
Lftp Lftp
LieBaoFast
Lightspeedsystems Lightspeedsystems
Likse Linkbot
LinkScan LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Litemage_walker
MIIxpc
MJ12bot
MQQBrowser MQQBrowser
MTRobot MTRobot
Majestic-SEO Mag-Net
Mail.RU_Bot
Majestic SEO
Majestic12 Majestic12
MarkMonitor MarkWatch
Mass Downloader
Masscan Masscan
Mata Hari
Mb2345Browser Mb2345Browser
MeanPath Bot
Meanpathbot Meanpathbot
MegaIndex.ru MegaIndex.ru
Metauri
MicroMessenger
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Minefield
Mister PiX
Moblie Safari Moblie Safari
Mojeek Morfeus Fucking Scanner
Mojolicious
MolokaiBot
Mozlila Mozlila
Msrabot Mr.4x3
Name Intelligence NICErsPRO
Needle Nessus
Net Vampire Net Vampire
NetLyzer NetMechanic
NetSpider NetZIP
Netcraft Niki-bot
NextGenSearchBot Nikto
NimbleCrawler
Nimbostratus
Nmap
Offline Navigator
OnCrawl
Openfind Openfind
OrangeBot Openvas
OrangeSpider
OutfoxBot OutfoxBot
PageScorer PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer
PageGrabber
Pandalytics Pandalytics
Panscient
Papa Foto Papa Foto
Picsearch Petalbot
PictureFinder Pi-Monster
Pimonster Picscout
ProPowerBot Piepmatz
Pixray
PleaseCrawl
Pu_iN Pu_iN
PyCurl
QueryN Metasearch
Quick-Crawler
RSSingBot
RankActive RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex Rankivabot
RankingBot RankurBot
RankingBot2 Re-re
RealDownload
Reaper Reaper
RebelMouse Recorder
RedesScrapy RedesScrapy
Ripper Ripper
Rogerbot RocketCrawler
SBIder SEOkicks
SEOkicks-Robot SEOlyticsCrawler
SISTRIX SalesIntelligent
ScanAlert ScoutJet
Scrapy Scrapy
Screaming Screaming
ScreenerBot
Searchestate
Seekport
SemanticJuice
Semrush Semrush
SemrushBot SemrushBot
SeobilityBot Shodan
SiteCheckerBotCrawler Siphon
SiteExplorer SiteExplorer
SiteSnagger SiteSnagger
SiteSucker
Sitebeam
Sitevigil
Snapbot
Snoopy Snoopy
SocialRankIOBot SocialRankIOBot
Sociscraper
Sogou web spider Sogou web spider
SpaceBison
Spbot Spbot
Spinn3r
SputnikBot
Sqlmap Sqlmap
Steeler Steeler
Stripper Stripper
Sucker
Sucuri Sucuri
Surfbot SuperHTTP
Thumbor Swiftbot
Turingos The Intraformant
Turnitin TightTwatBot
TurnitinBot Trendictionbot
URLy.Warning True_Robot
UnisterBot TwengaBot
V-BOT Twice
Typhoeus
URLy Warning
VB Project
VCI VCI
Vacuum Vacuum
Vagabondo Vagabondo
WPScan VelenPublicWebCrawler
VeriCiteCrawler
Virusdie
VoidEYE
Voltron
WBSearchBot
WEBDAV
WWW-Collector-E
WWW-Mechanize WWW-Mechanize
Web Enhancer WWW::Mechanize
Web Sauger WWWOFFLE
WebCollage WeSEE
WebFetch Web Collage
Web Pix
WebBandit
WebEnhancer
WebFuck WebFuck
WebGo IS
WebImageCollector WebImageCollector
WebPix
WebReaper
WebSauger
WebStripper WebStripper
WebZIP Webalta
WebmasterWorldForumBot
WebsiteQuester WebsiteQuester
Whatweb Widow
WinHTTrack
Woobot Woobot
Xaldon WebSpider YoudaoBot
Xaldon_WebSpider Zade
Zauba Zauba
Zeus Zermelo
ZoominfoBot ZmEu
ZyBorg ZumBot
archive.org_bot clark-crawler
arquivo.pt cognitiveseo
autoemailspider
com.plumanalytics
crawler.feedback
demandbase-bot
domainsproject.org domainsproject.org
evc-batch
facebookscraper
gopher gopher
instabid iubenda-radar
ips-agent linkdexbot
isitwp.com lwp-request
lwp-trivial lwp-trivial
meanpathbot mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler netEstate NE Crawler
pcBrowser pcBrowser
plumanalytics
polaris version polaris version
scan.lol ripz
seobility satoristudio.net
serpstatbot serpstatbot
sexsearcher sp_auditbot
sogouspider
spyfu spyfu
sysscan
tAkeOut tAkeOut
trendiction.de webpros.com
webgains-bot
x09Mozilla
x22Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
8xv8.com adsfresh.men
balook.com ap.senai.br
continent-e.tv backgroundpictures.net
delfin-aqua.com.ua europages.com.ru
enbersoft.com intermesh.net
greatgrace.ru net-profits.xyz
hchha.com pozdravleniya-c.ru
igru-xbox.net swimpool.ca
importchinacoach-teach.com topflownews.com
online7777.com vashsvet.com

912
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
007angels.com 08800.top
03e.info 0n-line.tv
11pikachu.ru 12bet.com
123cha.com
12u.info 12u.info
1314dh.com 1j7740kd.website
1kinobig.ru 1pamm.ru
1qingdao.com 1xbet4you.com
2345.com 24x7-server-support.site
2drittel.de 2drittel.de
3dracergames.com 4istoshop.com
3w24.com 5kstore.com
7makemoneyonline.com 76brighton.co.uk
aa8780bb28a1de4eb5bff33c28a218a930.com 8lufu.com
abcguru.xyz 8xv8.com
abclauncher.com aa8b68101d388c446389283820863176e7.com
accessoires-mode.in aa9bd78f328a6a41279d0fad0a88df1901.com
acgs.tk aaa38852e886ac4af1a3cff9b47cab6272.com
adamoads.com aasoldes.fr
adbetclickin.pink adanih.com
adman.gr adclickservice.com
adultwebhosting.info adconscious.com
agroeconom.kz adinterax.com
agysya.ru adloads.net
alert-jdh.xyz adman.se
alkoravto.ru ads-cool.pro
adsref.men
adtech.fr
adultactioncam.com
ahrefs.com
alborzan.com
alldownload.pw alldownload.pw
alldubai.biz alphahoverboards.com
alphavisions.net amazon-adsystem.com
alvtank.se amung.us
ameblo.jp an-donut.com
anaseracresar.tk ananumous.ru
android-style.com angigreene.com
animali.deagostinipassion.it animal-drawings.com
ap.senai.br animal36.com
apccargo.com animalia-life.club
apioasisspacenet-a.akamaihd.net apiallgeniusinfo-a.akamaihd.net
apispringsmartne-a.akamaihd.net apimountainbikei-a.akamaihd.net
asscenihotosidea.blogspot.com apiwebwebgetcom-a.akamaihd.net
atlant-auto.info applyneedy.xyz
atley.eu.pn approvedlocal.co.za
auto-moto-elektronika.cz arabsexxxtube.com
aviav.co arcadeplayhouse.com
avkzarabotok.info arendakvartir.kz
artdestshop.eu
artparquet.ru
asiengirls.net
asrv-a.akamaihd.net
atmovs.com
autonew.biz
awency.com awency.com
axbocz.net ayanaspa.com
b-styles.xyz azazu.ru
bannerconnect.net azlex.uz
beachfix.co backlinkwatch.com
beget.tech backuperwebcam.weebly.com
best-seo-software.xyz balitouroffice.com
bestofupload.info bard-real.com.ua
bestserials.com bbsoldes.fr
besttorrentknifta.weebly.com bcmp.org
bibys.com belinvestdom.by
biketank.ga bestchoice.cf
biographiya.com bestempresas.es
bioscorp.ru bestwaystogetpaid.us
bitcoins-live.ru bg6s0.com
bjgugu.net.ua bimatoprost-careprost.com
bjorkbacken.nu bioinnovate.co
blockworld.ru blog-hits.com
bluesalt.co boazpower.com
bonzybuddy.com brateg.xyz
bookmark4you.com.biz bristolhostel.com
bradleylive.xyz brothers-smaller.ru
breget74.ru buscarfoto.com
browserprotecter.com bvlgaribracelet.xyz
buchananshardware.com bvlgariring.xyz
budilneg.xyz byme.se
builtwith.com
businesn.men
buytizanidineonline.blogspot.com
buytizanidineonlinenoprescription.blogspot.com
cablecar.us cablecar.us
callawaygolfoutlet.online camdenmemorials.com
callmd5map.com candiceloves.us
candlewooddentalcentre.com.au candlewooddentalcentre.com.au
canvas.gloverid.site
capsderfudd.tk capsderfudd.tk
cbb1smartlist12.click carmuffler.net
chastnoeporno.com cartierlovebraceletreplica.xyz
cartierreplica.xyz
centraletermice.us
centrumcoachingu.com
chatroulette.life
chatroulette.online
cheap-pills-norx.com
cheapestjerseys-wholesale.com cheapestjerseys-wholesale.com
cheapjerseysbizwholesale.us chelnytruck.ru
cheapmoncler.xyz choosecuisine.com
christianlouboutinshoes.xyz chorus.terakeet.com
cindymatches.com chuckguilford.com
cleanpcnow.com ciekawinki.pl
clickaider.com cienum.fr
clknsee.com citysecurity.nu
clmforexeu.com commoncrawl.org
cobaltpro.ru compliance-alexa.xyz
cocolyze.com compliance-julia.top
coindirect.io connectingsingles.com
colehaanoutlet.store conocer-sanabria.com
collegerentals.com consorzioilmosaico.org
community.allhiphop.com customcatchcan.com
compliance-nelson.top cutt.us
contextualyield.com dafatiri.com
coolwallpapers4k.info damasarenai.info
cornomase.win dame-ns.kz
crafthubs.com daooda.com
crd.clan.su date-withme.com
creativehutindia.com
culad.com
davebestdeals.com davebestdeals.com
ddpills.com day-news.info
descargar-musica-gratis.net dcj-nn.ru
dgfitness.co deda-moroza-zakaz.ru
dichvuvesinhhanoi.com degerlund.net
dickssportinggoods.com dekorkeramik.ru
dipstar.org depositfiles-porn.ga
discovertreasure-a.akamaihd.net detoxmed24.ru
distver.ru deutschland123.de
digitalfaq.com
dktr.ru
dnepr-avtospar.com.ua
documentsite.net documentsite.net
domainsphoto.com domainroam.win
dosugrostov.site download-walpaperhd.blogspot.com
druzhbany.ru doxyporno.com
dscaas.website e-avon.ru
e-collantes.com e-biznes.info
ekosmetyki.net.pl ec-file.info
ekto.ee edudocs.net
electricwheelchairsarea.com ee77ee.com
elidelcream.weebly.com ege-russian.ru
electronicadirect.com
elektroniksigaraankara.info
elektrozigaretten1.webs.com
elmifarhangi.com
emailaccountlogin.co emailaccountlogin.co
engines-usa.com encodable.com
eralph.tk eng-lyrics.com
es5.com ero-advertising.com
essay-edu.biz estibot.com
eugenevaultstorage.com etenininrade.ga
evening-dating-club.info europeanwatches.ru
excitacao.com ezofest.sk
excitacion.info facialporntube.com
exportshop.us fantasticpixcool.com
f-loaded.de fashion.stellaconstance.co
f012.de fbdownloader.com
fanpagerobot.com felizporno.com
faracontrol.ir fil.ru
farm26.ru filmi-v.online
fashion-stickers.ru financepoints.eu
fashionindeed.ml findclan.org
feargames.ru finejewelryshop.ru
figuringmoneyout.com focalink.com
filefilter.weebly.com fok.nl
filerockstar298.weebly.com foodgid.net
financetip.eu forexgb.ru
findacheaplawyers.com forexunion.net
findpik.com forminecrafters.ru
firstdrugmall.ru forum20.smailik.org
fix-website-errors.com forumprofi.de
flash4fun.com fotoxxxru.com
food.dtu.dk fr-bearings.ru
footbalive.org free-video-tool.com
frameimage.org freelinkbuilding.website.tk
free-fbook-traffic.com freelotto.com
free-floating-buttons.com
free-stock-illustration.com
freefoto.ca
freemaintenancesysforpcandmac.top
fres-news.com
freshdz.com
frighteningremain.cf
fugarif.ga fugarif.ga
funponsel.com funponsel.com
galaxys6manual.info fyl.com.ru
gamesprite.me game300.ru
gamezblox.com games.kolossale.ru
gavuer.ru gazobeton-p.com.ua
gaytube.com gdzkurokam.ru
gazoblok.net.ua get-seo-domain.com
gembird.com
gemgrab-a.akamaihd.net
generalporn.org
getlaid-xxxhookupdirect.com
getlamborghini.ga getlamborghini.ga
getprismatic.com getridofstretchmarks.org
gfaq.ru gk170.ru
glall.ru glall.ru
goatse.ru glicol.kz
good-mummy.ru gogps.me
goroda-vsego-mira.ru goldadpremium.com
graphicwe.org goldenggames.com
greenidesign.co gosarhivrt.ru
gotomontenegro.net
greatfind-a.akamaihd.net
greatidea.marketing
greenshop.su
gsasearchengineranker.pw
gta-club.ru
guardlink.org
guigyverpo.cf
handicapvantoday.com
hanwei.us hanwei.us
hausfrauensex18.com havepussy.com
hdimagelib.com hchha.com
healthcarestore.info hdpixion.com
hdwallpapers-free.com
heatpower.ru
heliko.no
hermesreplica.pw hermesreplica.pw
histock.info hit-men.men
hmmm.cz hitcpm.com
home.myplaycity.com hledejvshopech.cz
honyaku.yahoofs.jp hostnow.men
hothot.ru hotblognetwork.com
houseofrose.com hotel-mkad.ru
houtings.xyz hoterika.com
houston-vikings.com
hscsscotland.com
hystersister.com hystersister.com
icaseclub.ru
ico.re
idc.com.ua
ideashome.id
igtools.club igtools.club
ihtec2019.org
ilovevitaly.ru ilovevitaly.ru
imagez.co imicrovision.com
in-tandem.co impotentik.com
inbabes.sexushost.com
infokonkurs.ru infokonkurs.ru
innodgfdriverhm.aircus.com
instasexyblog.com instasexyblog.com
iptool.xyz interfucks.net
iqs.biz.ua invest-pamm.ru
iradiology.ru invitefashion.com
ix20.ru iplogger.org
izismile.com istock-mebel.ru
j-times.ru ixora.pro
jagg.info j33x.com
jerseywholesalechinabiz.com jav-idol.com
jetsli.de javatex.co.id
jikoman.info javxxx18.com
jobgirl24.ru jaxcube.info
jofucipiku.tk je7.us
kandidos.com justkillingti.me
kazan.zrus.org kareliatobacco.ru
kinocccp.net kemerovo.zrus.org
kladrus.ru kenaba.su
konpax.com kino-key.info
kosova.de kinobest.pl
knogg.net
kolotiloff.ru
kostenlos-sexvideos.com
krasnodar-avtolombards.ru krasnodar-avtolombards.ru
ks1234.com krasnodar.zrus.org
kunstaktien.info krasula.pp.ua
kupit-adenu.ru kredit-blog.ru
kw21.org kurwa.win
laexotic.com lakomka.com.ua
lanadelreyfans.us lasvegaslockandsafe.com
law-check-seven.xyz
lawyersinfo.org lawyersinfo.org
ldrtrack.com
leadn.pl leadn.pl
legionalpha.com leadwayau.com
lesbian.xyz letmacworkfaster.site
letsart.ru letslowbefast.site
lezbiyanki.net li-er.ru
linkrr.com live-sexchat.ru
liran-locks.com lob.com.ru
littlecity.ch loginduepunti.it
livesex-amateure.info lookingglassemb.com
lomb.co louboutinreplica.pw
louisvuittonoutletstore.net louboutinshoes.xyz
machicon-akihabara.info low-format.ru
magazin-pics.ru lowephotos.info
mactechinfo.info
magicdiet.gq
maglid.ru
managerpak204.weebly.com managerpak204.weebly.com
map028.com manve.info
marketingtechniques.info marketingvici.com
marktforschung-stuttgart.com
maximpartnerspr.com maximpartnerspr.com
maxxtor.eu medicineseasybuy.com
mb140.ru medizinreisen.de
mecash.ru metallosajding.ru
mega-bony2017.pl metascephe.com
members.ghanaweb.com minneapoliscopiers.com
microsoftportal.net mixtapetorrent.com
mil-stak.com moesen-ficken.com
mmgq.ru moneymaster.ru
mmoguider.ru moneyteam24.com
mojowhois.com moscow-region.ru
mokrayakiska.com moskva.nodup.ru
moncleronline.xyz mp3ringtone.info
monclervests.xyz
musicas.baixar-musicas-gratis.com musicas.baixar-musicas-gratis.com
muz-baza.net myplaycity.com
muz-shoes.ru netoil.no
muznachas-service.ru nodding-passion.tk
mygameplus.com noscrapleftbehind.co
mypornfree.ru o-o-8-o-o.com
mystats.xyz o.light.d0t.ru
narkologiya-orel.ru offside2.5v.pl
needtosellmyhousefast.com oneshotdate.com
negral.pluto.ro onetravelguides.com
networkad.net online247.ml
newstaffadsshop.club onlineslotmaschine.com
noclegonline.info ooo-gotovie.ru
nofreezingmac.work openfrost.com
nordstar.pro opinionreelle.com
northfacestore.online ourtherapy.ru
notebook-pro.ru paclitor.com
nottyu.xyz panamaforbeginners.com
noumeda.com partnerads.men
novosti-hi-tech.ru partners-ship.pro
ntic.fr
nudejapan.net
nw-servis.ru
oecnhs.info
oklogistic.ru
okmedia.sk
olvanto.ru
onefilms.net
oneminutesite.it
online-templatestore.com
onlinedomains.ru
onlinefilmz.net
onlineporno.site
open-odyssey.org
oranga.host.sk
osoznanie-narkotikam.net
otbelivanie-zubov.com
ownshop.cf
oxotl.com
page2rss.com
partner-high.men
pc-virus-d0l92j2.pw
pdn-4.com pdn-4.com
pdns.cz pensplan.com
perfection-pleasure.ru phpdevops.com
picswe.com picsfair.com
picturesmania.com pinstake.com
pihl.se pliks.pl
pizdopletka.club pojdelo.weebly.com
pkr1hand.com polcin.de
pl-vouchers.com polimga.pw
platinumdeals.gr polybuild.ru
play.leadzupc.com
poem-paying.gq
poisk-zakona.ru
popup.matchmaker.com popup.matchmaker.com
porndl.org pornogratisdiario.com
pornoelita.info pornohub.me
pornokorol.com pornoplen.com
pornolook.net pornosee.info
pornophoto.xyz pornozhara.com
pp-budpostach.com.ua pospr.waw.pl
preg.marketingvici.com privatamateure.com
priceg.com prizesbook.online
prizefestival.mobi profitsport.club
productarium.com projectforte.ru
prohoster.info prosperent.com
promoforum.ru pskcijdc.bloger.index.hr
proposal-engine.com pts163.ru
prostitutki-tyumeni.org quangcaons.com
prpops.com rabotaetvse.ru
pshare.biz rapidgator-porn.ga
purplestats.com rarbg.to
qwesa.ru rasteniya-vs-zombi.ru
rank-checker.online razorweb-a.akamaihd.net
ranksonic.info
razvratnoe.org
realtytimes.com
rebuildermedical.com rebuildermedical.com
rednise.com reelheroes.net
regionshop.biz referencemoi.com
registratciya-v-moskve.ru refudiatethissarah.info
remontvsamare.su registry-cleaner.net
research.ifmo.ru registrydomainservices.com
responsive-test.net remont-mobile-phones.ru
rightenergysolutions.com.au rep-am.com
robertefuller.com rezeptiblud.ru
rfserial.net
rialp.getenjoyment.net
ringporno.com
risparmiocasa.bz.it
rosbalt.com.ua
rospromtest.ru rospromtest.ru
rozalli.com
rubanners.com
rucrypt.com rucrypt.com
ruex.org.ua russia-today-video.ru
rusenvironmental.net rutor.vip
rvzr-a.akamaihd.net rvzr-a.akamaihd.net
s1z.ru saneyes.com
sarm.tk sax-sex.com
sbtdesign.co.uk scanner-don.top
scanner-julia.top searchimpression.com
scanner-margo.top
scanner-mary.top
scenicmissouri.us
schalke04fc.info
scrapinghub.com
sdrescher.net
search.pch.com
searchkut.com searchkut.com
seo-2-0.com security60-e.com
seorankinglinks.xyz semaltmedia.com
seozoom.it service.adtech.fr
serw.clicksor.com sevenstars7.com
sex.hotblog.top sextracker.be
sexblog.pw sexuria.net
sexkrasivo.net
sexsearch.com
sextracker.com
sexyteens.hol.es
shakhtar-doneck.ru shakhtar-doneck.ru
sharebutton.net sharebutton.net
sharebutton.org shell-pmr.ru
shemalegalls.blogporn.in shisha-swag.de
shtaketniki.kz shitmovs.com
sibdevice.ru shoesonlinebuy.xyz
sigmund-freud.co.uk shop.acim.org
signal03.ru sitiz.club
silvercash.com slomm.ru
simon3.ru smartshoppymac.com
siteimprove.com smsactivator.ru
skypasss.com snip.to
sloopyjoes.com soc-proof.su
smartadserver.com socialmediasuggest.com
socialsignals24.com sohoindia.net
sockshare.net solinf.co
sockshares.tv sophang8.com
sosdepotdebilan.com spidtest.org
srgwebmail.nl spinnerco.ca
stackthatbucks.com sports-supplements.us
stard.shop stairliftstrue.com
stathat.com stats-public.grammarly.io
steamoff.net stevemonsen.com
stroiminsk.com sticken.co
sucsesofinspiration.com stroydetali.ru
supermesta.ru strv.se
subj.ukr-lit.com
superstarfloraluk.com superstarfloraluk.com
svensk-poesi.com surffoundation.nl
swagbucks.com susanholtphotography.com
swingerseiten.com tdsing.ru
syndicate.fun teenporn18.net
t-bygg.com telefonsexi.com
tackletarts.co telegraf.by
tamada69.com tetracsaudi.com
telegramdownload10.com theautoprofit.ml
testbotprocessor44.com
textads.men
tfxiq.com
the-usa-games.blogspot.com
thebestweightlosspills.ovh thebestweightlosspills.ovh
thefotosgratis.eu thegolfclub.info
thelottosecrets.com themestotal.com
thepokertimer.com thenews-today.info
timhost.ru thewomenlife.com
titelhelden.eu tizanidine4mgprice.blogspot.com
tizanidinewithdrawal.blogspot.com tizanidine4mgstreetprice.blogspot.com
tk-assortiment.ru tizanidine4mgstreetvalue.blogspot.com
tobeyouday.win tizanidinedosage.blogspot.com
toonfamilies.net tizanidinehcl2mg.blogspot.com
top250movies.ru tizanidinehcl4mgtabinfo.blogspot.com
tradgardspartner.se tizanidinehcl4mgtablet.blogspot.com
traffixer.com tizanidineinteractions.blogspot.com
tri-slona.org tizanidineshowupondrugtest.blogspot.com
truck-land.life top10-way.com
truemfilelj.gq topads.men
trustedhealthtips.com topbestgames.com
tversvet.ru torrentdownloadhub.com
tvorozhnaja-zapekanka-recept.ru tourismvictoria.com
typer.one track112.site
u17795.netangels.ru trafficmonetize.org
uac.net.au trafficstars.com
uamtrk.com traffpartners.com
uhodzalijami.ru trainoffend.ml
ukrtvir.com.ua tula.mdverey.ru
ultimateclassicrock.com turbabitload.weebly.com
turizm.bz
turvgori.ru
tvand.ru
ucoz.ru
uggbootsoutletsale.us
ukr-lit.com
unacittaconte.org unacittaconte.org
unitygame3d.com underthesite.com
uptime.com unimodemhalfduplefw.pen.io
url-extractor.xyz unionmarkt.de
url-img.link unpredictable.ga
urll.eu ural-buldozer.ru
us-america.ru urdoot.win
vadimkravtcov.ru usadacha.net
varikozdok.ru vasileostrovsky-rayon.ru
vegan-foods.us vchulkax.com
venta-prom.ru via-energy-cumpara.com
viagra-soft.ru via-energy-order.com
viagra.pp.ua videochat.bz
videooko.weebly.com videojam.tv
videotuber.ru videokrik.net
vietimgy.pw villacoloniale.com
vip-file.com vintontech.info
virtuagirl.com vip.51.la
vitalads.net vkontaktemusic.ru
vksaver-all.ru voltrknc1.com
vpnhowto.info vonradio.com
vsesubwaysurfers.com vseigry.fun
wareseeker.com vykupavto-krasnodar.ru
wdrake.com webcamtalk.net
webads.co.nz website-datenbank.de
webkeyit.com weightatraining.com
website-analytics.online whatistizanidinehclusedfor.blogspot.com
website-speed-up.top whatzmyip.net
weburlopener.com wholesalecheapjerseysfree.com
wellcome2slovenia.ru wnhjavlhezp.gq
wesharepics.site works.if.ua
whiteproduct.com writingservices17.blogspot.ru
whosonmyserver.com wtsindia.in
winner7777.net www.ehscloud.cn
withstandingheartwarming.com wygraj-skiny.win
worldoffiles.ru xchangetrak.com
www.wohnkabinen-shop.de
xblog.in
xdoza.com xdoza.com
xkaz.org xfire.com
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai xlolitka.com
xmronta.com
xn----7sbho2agebbhlivy.xn--p1ai
xn--80ahdheogk5l.xn--p1ai xn--80ahdheogk5l.xn--p1ai
xn--80ak6aa92e.com xolodremont.ru
xn--b1agm2d.net xxx-treker.ru
xn--l1aengat.xn--p1ai
xn--oogle-wmc.com
xtube.com
xxart.ru
xz618.com xz618.com
yaaknaa.info yaaknaa.info
yellowads.men yogamatsexpert.com
yougetsignal.com youdao.com
yurgorod.ru youporn-forum.ga
yuweng.info youradulthosting.com
zeroredirect2.com ypmuseum.ru
zionstar.net zaapplesales.blogspot.com
zojirushi-products.ru zakazvzloma.com
zvuker.net zarabotok--doma.ru
zatjmuzu.info
zhorapankratov7.blogspot.com
zkjovpdgxivg.ga
zmoda.hostreo.com
zoogdiesney.com
zooggames.com
zscaler.net
zx6.ru
zynax.ua

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
100dollars-seo.com 24videos.tv
2nt.ru 72-news.com
45en.ru 83net.jp
4rent.ru aa9bd78f328a6a41279d0fad0a88df1901.com
8jn.dating-app.ru abctoppictures.net
aa81bf391151884adfa3dd677e41f94be1.com above.com
abwa.tk adnotbad.com
adanih.com ads-cool.pro
addblueoff.com.ua advancedmassagebysara.com
adktrailmap.com adzpower.com
advokat-grodno.by akuhni.by
advokateg.xyz akusherok.ru
alessandraleone.com apartmentratings.com
alphacarolinas.com apccargo.com
amateurgalls.com apiitsthirteende-a.akamaihd.net
anal-acrobats.com apilinkswiftco-a.akamaihd.net
android-vsem.org apioasisspacenet-a.akamaihd.net
animalia-life.club
animenime.ru
anonymous-redirect.com
apiallgeniusinfo-a.akamaihd.net
apiappenableinfo-a.akamaihd.net
apiwebwebgetcom-a.akamaihd.net apiwebwebgetcom-a.akamaihd.net
applyneedy.xyz apple.com-cleaner.systems
artdeko.info aproposde.com
asophoto.com arkkivoltti.net
arvut.org
asseenontvonline.ru asseenontvonline.ru
axbocz.net auspolice.com
backgroundpictures.net autobudpostach.club
berrymall.ru automobile-spec.com
best-seo-solution.com avtoarenda.by
bestcalovebracelet.cn avtolombard-krasnodar.ru
bestcurs.org azartmix.com
bestplacetobuyeriacta.jw.lt backiee.com
betshuckclean.com balitouroffice.com
bezcmexa.ru beachpics.com
bingoporno.com beauty-bracelet.com
bloggers.nl belstaffstore.online
blogqpot.com bestbrainprod.win
blue-square.biz bestdooz.com
bookhome.info beztuberkuleza.ru
bronzeaid-a.akamaihd.net bjgugu.net.ua
buqyxa.rincian.info blackbabesporn.com
buydissertation.net blacktwhite.com
buyessayonline19.blogspot.ru bonzybuddy.com
candycrushshop.com brillianty.info
cartierlovebracelet.xyz bristolhotel.com
chocolateslim-u-srbiji.com btnativenav.com
cialischstgerts.com burn-fat.ga
cialisndbrx.com buron.pw
clickhype.com buy-cheap-online.info
clknsee.com callmd5map.com
cmd.kz canacopegdl.com
compliance-jessica.top carson.getenjoyment.net
compliance-nelson.top cerev.info
compliance-willy.top cheapjerseysbizwholesale.us
confib.ifmo.ru compliance-alexa.xyz
connectionstrenth.com compliance-checker.info
construmac.com.mx concealthyself.com
country-chic.ru cookielawblog.wordpress.com
creditnation.ru coolyarddecorations.com
custom-electric-guitar.com coprofam.org
customcollegeessays.net covetnica.com
domain.webkeyit.com cupday.com
domainsatcost.com cyse.tk
donvito.unas.cz dame-ns.kz
duplicashapp.com designdevise.com
dverimegapolis.ru detoxmed24.ru
dyt.net diamond-necklace.info
e2click.com djihispano.com
ednorxmed.com dpihatinh.gov.vn
ege-russian.ru drivotracker.com
ekspertmed.com dvr.biz.ua
elektrischezi.canalblog.com eager-nash.188-93-233-196.plesk.page
elektrischeziga.livejournal.com earncash.com.ua
elektronischezigarettekaufen1.myblog.de eduinfosite.com
elektrozigaretten1.webs.com ege-essay.ru
elmacho.xyz electricwheelchairsarea.com
escort-russian.com elvenar.com
euromasterclass.ru eropho.com
ewebarticle.info evaashop.ru
ezofest.sk exportshop.us
fastcrawl.com expresstoplivo.ru
fil.ru ezigarettenkaufen1.pagina.gr
forms-mtm.ru farm26.ru
fortevidyoze.net feeriaclub.ru
franklinfire.co filmetricsasia.com
free-gluten.ru fivedwld.cf
freefoto.ca frankofficial.ru
freewebs.com free-social-buttons.xyz
frequiry.com free-video-tool.com
fullgirl.ru fungirlsgames.net
funponsel.com
fxund.us
gabeshop.ru gabeshop.ru
gallery.rennlist.com gamezblox.com
gambarkataku.co ge0ip.net
gelezki.com
gerhardhealer.com gerhardhealer.com
ghernnqr.skyrock.com gg-arena.ru
gigapeta.com ghostvisitor.com
gne8.com gidonline.one
gombita.info goldandcard.ru
gosmeb.ru goosefishpost.bid
guge.io gourcy.altervista.org
guigyverpo.cf growshop.es
handsandlegs.ru gstatey.net
hasfun.com hard-porn.mobi
hawaiielectriclight.com hchha.com
headpress.ru homegardenlova.com
hledejvshopech.cz iminent.com
hobbyhuren24.net istanbulit.com
holidaypics.org jaxcube.info
hothot.ru jrcigars.com
huhn.altervista.org katjimej.blog.fc2.com
hundejo.com kazan.xrus.org
iamsport.org keywords-monitoring-success.com
iccornacircri.cf keywordteam.net
ido3.com kichenaid.ru
ilo134uloh.com kinobest.pl
images.gyffu.com kit-opt.ru
infokonkurs.ru komp-pomosch.ru
interesnie-faktu.ru komputernaya-pomosh-moscow.ru
istizanidineanarcoticdrug.blogspot.com koptims.tiu.ru
izismile.com kupit-adenu.ru
jackwolfskinoutlet.online la-fa.ru
japan-bearings.ru labplus.ru
jerseyssportsshop.com ledpolice.ru
jerseyswholesalechinalimited.com lightinghomes.net
jo24news.com lonely-mature.com
jumptap.com lonerangergames.com
karusel-market.ru mackeeper-land-672695126.us-east-1.elb.amazonaws.com
kerei.ru mafa-free.com
keywordbasket.com magazintiande.ru
kiinomaniak.pl mistr-x.org
labvis.host.sk
lanasshop.ru
laulini.soclog.se
law-check-nine.xyz
liveu.infoteka.hu
luciddiagnostics.in
m0r0zk0-krava.ru
mackeeper-center.club
marmotstore.online
maslenka.kz
message-warning.net
missis.top
mmog-play.ru mmog-play.ru
multgo.ru mp3ritm.top
myanyone.net muizre.ru
myprintscreen.com musicdaddy.net
nastydollars.com mylida.org
netallergy.ru mystats.xyz
newstraveller.ru na-telefon.biz
nextrent-crimea.ru nacap.ru
nina.az namecrumilchlet.tk
november-lax.com naphukete.ru
nw-servis.ru netanalytics.xyz
okel.co nmrk.ru
onstrapon.purplesphere.in nomuos.it
onthemarch.co obiavo.by
openmultipleurl.com obiavo.com
ordernorxx.com offer.camp
outpersonals.com online-hd.pl
outrageousdeal-a.akamaihd.net online.ktc45.ru
parfusale.se onlinemeetingnow.com
partner-high.men onlyforemont.ru
pastaleads.com openlibrary.org
pc-services.ru openmediasoft.com
pcgroup.com.uy oren-cats.ru
picsforkeywordsuggestion.com pacificair.com
pierrehardysale.online perm.xrus.org
pirateday.ru pestomou.info
plugingeorgia.com plaff-go.ru
pogodnyyeavarii.gq polska-poezja.com
pontiacsolstice.info poppen-nw.net
porndl.org porn-w.org
predominant-invent.tk pornhive.org
prefersurvey.net pornoreino.com
pornozhara.com
priscilarodrigues.com.br priscilarodrigues.com.br
qor360.com privetsochi.ru
realtytimes.com proekt-gaz.ru
refads.pro profitkode.com
remont-comp-pomosh.ru prombudpostach.com.ua
remote-dba.de pushdata.sendpulse.com
rixpix.ru pyrodesigns.com.au
rusvideos.su reeyanaturopathy.com
rutor.vip reklama-i-rabota.ru
s.lollypopgaming.com repeatlogo.co.uk
sady-urala.ru restaurantlescampi.com
runtnc.net
salutmontreal.com
sanidumps.com sanidumps.com
savetubevideo.com scanner-margo.top
scalerite.co.za school-diplomat.ru
scrnet.biz.ua seo-traffic-ranking.info
search.alot.com seojokes.net
securityallianceservices.com seorussian.ru
sharebutton.net serviporno.com
shodanhq.com sexobzor.info
shop.xz618.com sextracker.de
shopsellcardsdumps.com shoesonlinebuy.cn
snsdeainavi.info similardeals.net
space-worry.ml sklad-24.ru
spbchampionat.ru sluganarodu.ru
spl63.fr socialmediasuggest.com
stuffpride.com soheavyblog.com
svbur.ru sport-video-obzor.ru
takethatad.com sptslmtrafms.com
teslathemes.com surintech.ac.th
theheroes.ru svnuppsalaorebro.se
tizanidinehcl4mgtabinfo.blogspot.com syvertsen-da.no
tizanidinemusclerelaxant.blogspot.com tako3.com
tizanidinerestlesslegsyndrome.blogspot.com tattooreligion.ru
tizanidinestreetprice.blogspot.com tds-advert005.info
token-lab.org teastory.co
tomck.com technika-remont.ru
top-study.work tengohydar.tk
thefarmergame.com
tilido.com
time-japan.ru
tizanidinecost.blogspot.com
tizanidinewithdrawal.blogspot.com
tk-assortiment.ru
tnctrx.com
todohr.com
topanasex.com topanasex.com
torrentred.games torrents.cd
tuckermktg.com tovaroboom.vast.ru
tvteleport.ru try-rx.com
twelvevisionspartyofcolorado.com tuberkuleznik.ru
ua-company.ru tvoystartup.ru
uginekologa.com typer.one
undergroundcityphoto.com uhodzalijami.ru
vekzdorov.ru ukrobstep.com
upupa.net
vandrie-ict.nl
vektorpress.ru
viandpet.com
videochat.tv.br videochat.tv.br
videokrik.net videojam.tv
vip2ch.com viven.host.sk
vipms.ru vizag.kharkov.ua
vrotike.ru volgograd.xrus.org
vzglyadriv.kg vriel.batcave.net
websiteexplorer.info webads.co.nz
weprik.ru website-analytics.online
wesharepics.info websites-reviews.com
word-vorlagen.net weddingdresses.xyz
ww2awards.info whatsupinfoley.com
xfluro.com wholinkstome.com
xn----ctbbcjd3dbsehgi.xn--p1ai wma-x.com
xn--b1ag5cfn.xn--p1ai www.wohnkabinen-shop.de
xsion.net wzgyyq.com
xxx-treker.ru x-diesel.com
xxxasianporn.net x-lime.com
xzlive.com xboxster.ru
youradulthosting.com xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
ytmnd.com xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
yun56.co xn--80aafb2a.xn--p1ai
zetgie.com.pl xn--hxazdsfy.blogspot.com
zlatnajesen.com xxxkaz.org
znakomstva-moskva77.ru yougetsignal.com
zoominfo.com zaapplesales.blogspot.com
zzbroya.com.ua zigarettenonlinekaufen.tumblr.com
znakomstva-piter78.ru
znakomstvaonlain.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.241 0; 10.1.0.217 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3073 ### Version: V4.2022.04.3074
### Updated: Thu Apr 28 22:01:24 UTC 2022 ### Updated: Fri Apr 29 10:03:24 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3073 ### Version: V4.2022.04.3074
### Updated: Thu Apr 28 22:01:24 UTC 2022 ### Updated: Fri Apr 29 10:03:24 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##