vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 15:34:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2933 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-20 11:07:55 +00:00
parent 8458c19b86
commit b2182f8e55
55 changed files with 92354 additions and 92333 deletions
Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2932 #### Version: V4.2022.02.2933
#### Bad Referrers Blocked: [7091](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2932 #### Version: V4.2022.02.2933
#### Bad Referrers Blocked: [7091](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -13,8 +13,8 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2932 #### Version: V4.2022.02.2933
#### Bad Referrers Blocked: [7091](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

11402
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_generator_lists/bad-referrers.list
View file

@ -1,4 +1,3 @@
trafficcentr.xyz
000free.us 000free.us
007angels.com 007angels.com
00author.com 00author.com
@ -6122,6 +6121,7 @@ traffic-club.info
traffic100.com traffic100.com
traffic2cash.org traffic2cash.org
traffic2money.com traffic2money.com
trafficcentr.xyz
trafficfactory.biz trafficfactory.biz
trafficgenius.xyz trafficgenius.xyz
trafficjunky.com trafficjunky.com

4
_google_analytics_ghost_spam/README.md
View file

@ -3,8 +3,8 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2932 #### Version: V4.2022.02.2933
#### Bad Referrers Blocked: [7091](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

2
_google_analytics_ghost_spam/google-exclude-04.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-05.txt
View file

File diff suppressed because one or more lines are too long

1
_google_webmaster_disavow_links/google-disavow.txt
View file

@ -6121,6 +6121,7 @@ domain:traffic-club.info
domain:traffic100.com domain:traffic100.com
domain:traffic2cash.org domain:traffic2cash.org
domain:traffic2money.com domain:traffic2money.com
domain:trafficcentr.xyz
domain:trafficfactory.biz domain:trafficfactory.biz
domain:trafficgenius.xyz domain:trafficgenius.xyz
domain:trafficjunky.com domain:trafficjunky.com

11409
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11409
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11409
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11409
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 1; 10.1.1.98 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 0; 10.1.1.98 0;
127.0.0.1 0; 127.0.0.1 0;

11409
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 1; 10.1.1.98 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 0; 10.1.1.98 0;
127.0.0.1 0; 127.0.0.1 0;

11409
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 1; 10.1.1.98 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/beta_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -6168,6 +6168,7 @@
"~*(?:\b)traffic100\.com(?:\b)" 0; "~*(?:\b)traffic100\.com(?:\b)" 0;
"~*(?:\b)traffic2cash\.org(?:\b)" 0; "~*(?:\b)traffic2cash\.org(?:\b)" 0;
"~*(?:\b)traffic2money\.com(?:\b)" 0; "~*(?:\b)traffic2money\.com(?:\b)" 0;
"~*(?:\b)trafficcentr\.xyz(?:\b)" 0;
"~*(?:\b)trafficfactory\.biz(?:\b)" 0; "~*(?:\b)trafficfactory\.biz(?:\b)" 0;
"~*(?:\b)trafficgenius\.xyz(?:\b)" 0; "~*(?:\b)trafficgenius\.xyz(?:\b)" 0;
"~*(?:\b)trafficjunky\.com(?:\b)" 0; "~*(?:\b)trafficjunky\.com(?:\b)" 0;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 0; 10.1.1.98 0;
127.0.0.1 0; 127.0.0.1 0;

11409
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2932 2933

11409
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11409
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

1
dev-tools/referrers-regex-format-whitelist-test.txt
View file

@ -6121,6 +6121,7 @@
"~*(?:\b)traffic100\.com(?:\b)" 0; "~*(?:\b)traffic100\.com(?:\b)" 0;
"~*(?:\b)traffic2cash\.org(?:\b)" 0; "~*(?:\b)traffic2cash\.org(?:\b)" 0;
"~*(?:\b)traffic2money\.com(?:\b)" 0; "~*(?:\b)traffic2money\.com(?:\b)" 0;
"~*(?:\b)trafficcentr\.xyz(?:\b)" 0;
"~*(?:\b)trafficfactory\.biz(?:\b)" 0; "~*(?:\b)trafficfactory\.biz(?:\b)" 0;
"~*(?:\b)trafficgenius\.xyz(?:\b)" 0; "~*(?:\b)trafficgenius\.xyz(?:\b)" 0;
"~*(?:\b)trafficjunky\.com(?:\b)" 0; "~*(?:\b)trafficjunky\.com(?:\b)" 0;

1
dev-tools/referrers-regex-format.txt
View file

@ -6121,6 +6121,7 @@
"~*(?:\b)traffic2cash\.org(?:\b)" 1; "~*(?:\b)traffic2cash\.org(?:\b)" 1;
"~*(?:\b)traffic2money\.com(?:\b)" 1; "~*(?:\b)traffic2money\.com(?:\b)" 1;
"~*(?:\b)traffic\-club\.info(?:\b)" 1; "~*(?:\b)traffic\-club\.info(?:\b)" 1;
"~*(?:\b)trafficcentr\.xyz(?:\b)" 1;
"~*(?:\b)trafficfactory\.biz(?:\b)" 1; "~*(?:\b)trafficfactory\.biz(?:\b)" 1;
"~*(?:\b)trafficgenius\.xyz(?:\b)" 1; "~*(?:\b)trafficgenius\.xyz(?:\b)" 1;
"~*(?:\b)trafficjunky\.com(?:\b)" 1; "~*(?:\b)trafficjunky\.com(?:\b)" 1;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.89.96.249 0; 40.121.255.106 0;

11409
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 1; 10.1.1.98 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 0; 10.1.1.98 0;
127.0.0.1 0; 127.0.0.1 0;

11409
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 1; 10.1.1.98 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 0; 10.1.1.98 0;
127.0.0.1 0; 127.0.0.1 0;

11409
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 1; 10.1.1.98 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test1_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -6168,6 +6168,7 @@
"~*(?:\b)traffic100\.com(?:\b)" 0; "~*(?:\b)traffic100\.com(?:\b)" 0;
"~*(?:\b)traffic2cash\.org(?:\b)" 0; "~*(?:\b)traffic2cash\.org(?:\b)" 0;
"~*(?:\b)traffic2money\.com(?:\b)" 0; "~*(?:\b)traffic2money\.com(?:\b)" 0;
"~*(?:\b)trafficcentr\.xyz(?:\b)" 0;
"~*(?:\b)trafficfactory\.biz(?:\b)" 0; "~*(?:\b)trafficfactory\.biz(?:\b)" 0;
"~*(?:\b)trafficgenius\.xyz(?:\b)" 0; "~*(?:\b)trafficgenius\.xyz(?:\b)" 0;
"~*(?:\b)trafficjunky\.com(?:\b)" 0; "~*(?:\b)trafficjunky\.com(?:\b)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 0; 10.1.1.98 0;
127.0.0.1 0; 127.0.0.1 0;

11409
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.89.96.249 0; 40.121.255.106 0;

11409
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.89.96.249 0; 40.121.255.106 0;

11409
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 1; 10.1.1.98 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test_units/custom-bad-referrers.conf
View file

@ -6168,6 +6168,7 @@
"~*(?:\b)traffic100\.com(?:\b)" 0; "~*(?:\b)traffic100\.com(?:\b)" 0;
"~*(?:\b)traffic2cash\.org(?:\b)" 0; "~*(?:\b)traffic2cash\.org(?:\b)" 0;
"~*(?:\b)traffic2money\.com(?:\b)" 0; "~*(?:\b)traffic2money\.com(?:\b)" 0;
"~*(?:\b)trafficcentr\.xyz(?:\b)" 0;
"~*(?:\b)trafficfactory\.biz(?:\b)" 0; "~*(?:\b)trafficfactory\.biz(?:\b)" 0;
"~*(?:\b)trafficgenius\.xyz(?:\b)" 0; "~*(?:\b)trafficgenius\.xyz(?:\b)" 0;
"~*(?:\b)trafficjunky\.com(?:\b)" 0; "~*(?:\b)trafficjunky\.com(?:\b)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
GoZilla AwarioRssBot
Gotit CrunchBot
LeechFTP Grafula
Re-re RankActive
Snake SlySearch
Steeler Spanner
Swiftbot Stripper
Trendiction VCI
Vacuum Zeus
WebCopier x22Mozilla

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 360Spider
404enemy 404checker
ALittle Client 80legs
AfD-Verbotsverfahren Abonti
AiHitBot Aboundexbot
Aipbot AhrefsBot
Alexibot Alexibot
AllSubmitter
Alligator Alligator
AlphaBot AlphaBot
Ankit Ankit
Aspiegel Apexoo
Asterias
Atomseobot
Attach Attach
AwarioSmartBot
BDCbot
BackStreet BackStreet
BackWeb Backlink-Ceck
Bandit BacklinkCrawler
Badass
Barkrowler Barkrowler
BatchFTP
Battleztar Bazinga
BetaBot
Bigfoot
Bitacle
BlackWidow BlackWidow
Blackboard Blow
BlowFish
Boardreader
Bolt
Brandprotect Brandprotect
Brandwatch
Buck Buck
Buddy Buddy
Bullseye
BunnySlippers BunnySlippers
BuzzSumo
CATExplorador CATExplorador
CODE87 CSHttp
Cegbfeieh Cegbfeieh
CensysInspect Chlooe
CheTeam
Citoid Citoid
Cloud mapping Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot Crawling at Home Project
Copier
CopyRightCheck
Copyscape
CrazyWebCrawler CrazyWebCrawler
Crescent Curious
DBLBot Custo
DIIbot CyotekWebCopy
DataCha0s DatabaseDriverMysqli
Demon Demon
Discobot Dirbuster
Disco
Discoverybot Discoverybot
Dispatch Dispatch
DnBCrawler-Analytics DittoSpyder
DnyzBot DomCopBot
DomainCrawler DomainAppender
DomainSigmaCrawler Drip
DomainStatsBot
ECCP/1.0 ECCP/1.0
EirGrabber EMail Siphon
Exabot EMail Wolf
ExtLinksBot Ebingbong
ExtractorPro ExtractorPro
Ezooms
FDM
Fimap Fimap
Firefox/7.0 Flunky
Foobot Foobot
Freeuploader Freeuploader
FrontPage FrontPage
GalaxyBot Fuzz
Genieo FyberSpider
GermCrawler G-i-g-a-b-o-t
Go!Zilla GT::WWW
Getintent
Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
GrabNet
Grabber
Grafula
GrapeFX GrapeFX
HEADMasterSEO GrapeshotCrawler
Havij HTTrack
Haansoft
HaosouSpider
Hloader
HonoluluBot HonoluluBot
Humanlinks Humanlinks
IlseBot HybridBot
Iblog
Image Fetch Image Fetch
Image Sucker Image Sucker
IndeedBot
Indy Library Indy Library
InterGET InfoNaviRobot
Internet Ninja Intelliseek
Jbrofuzz InternetSeer
Iria
JamesBOT
JennyBot JennyBot
JetCar
Jetty Jetty
Joomla Jorgee
JustView Kinza
Kenjin Spider Lanshanbot
Keyword Density Leap
Kozmosbot
Lftp Lftp
Libwhisker LibWeb
LieBaoFast LieBaoFast
LinkWalker LinkWalker
Linkdexbot
LinkpadBot LinkpadBot
LinksManager
LinqiaRSSBot LinqiaRSSBot
Lipperhey Spider Lipperhey
Litemage_walker Lmspider
MSFrontPage Ltx71
MQQBrowser
MSIECrawler
MTRobot
Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Masscan
MarkWatch Microsoft URL Control
Mb2345Browser Minefield
Meanpathbot Mister PiX
Metauri
Moblie Safari Moblie Safari
Mojeek Mr.4x3
Morfeus Fucking Scanner Navroad
NICErsPRO
NPbot
Name Intelligence
Nameprotect
Needle Needle
Nessus Nessus
NetAnts NetAnts
NetMechanic NetLyzer
NetSpider
NetZIP NetZIP
Nettrack Netcraft
Niki-bot Netvibes
Nikto NextGenSearchBot
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Not Ninja
Offline Navigator Nutch
Octopus
OnCrawl
OpenLinkProfiler OpenLinkProfiler
Openfind Openvas
OrangeBot
OrangeSpider OrangeSpider
POE-Component-Client-HTTP PECL::HTTP
Page Analyzer
PageAnalyzer PageAnalyzer
PageGrabber PageGrabber
PageScorer Panscient
Pavuk
PeoplePal
Pi-Monster
Picscout
Picsearch Picsearch
PictureFinder Probethenet
ProWebWalker
Pu_iN Pu_iN
Pump Pump
RSSingBot PxBroker
RankActiveLinkBot QueryN Metasearch
Quick-Crawler
RankingBot2
Rankivabot Rankivabot
ReGet Re-re
RebelMouse
RedesScrapy RedesScrapy
RepoMonkey RepoMonkey
Ripper
RocketCrawler RocketCrawler
SEOkicks-Robot Rogerbot
SEOprofiler SBIder
SMTBot SalesIntelligent
ScanAlert ScoutJet
Scrapy Scrapy
Screaming ScreenerBot
Searchestate
SearchmetricsBot SearchmetricsBot
Seekport Seekport
SemrushBot
SeoSiteCheckup SeoSiteCheckup
Seomoz
Shodan Shodan
Site Sucker
SiteCheckerBotCrawler
SiteLockSpider SiteLockSpider
Siteimprove SiteSucker
Snoopy Sitebeam
SmartDownload
Snake
Snapbot
SocialRankIOBot
Sottopop
SpaceBison
Spammen
Spanner
Spinn3r
Sqworm Sqworm
Steeler Steeler
SurveyBot Sucker
Suzuran SuperBot
Swiftbot SuperHTTP
Surfbot
Szukacz Szukacz
TeleportPro T8Abot
Telesphoreo Thumbor
Titan TightTwatBot
Toweyabot Toweyabot
Trendiction Turingos
Trendictionbot Turnitin
V-BOT Twice
VCI VB Project
VelenPublicWebCrawler VeriCiteCrawler
Virusdie VidibleScraper
VoidEYE
Voil Voil
Voltron
WBSearchBot
WISENutbot
WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW::Mechanize
WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD Web Collage
Web Auto Web Enhancer
WebCollage Web Sauger
WebAuto
WebBandit
WebCopier
WebGo IS WebGo IS
WebLeacher WebPix
WebSauger WebSauger
WebStripper WebZIP
WebSucker
Webalta
Webshag
Website Quester Website Quester
WebsiteQuester
Whacker
Whatweb
Who.is Bot Who.is Bot
Widow Widow
Wonderbot Wonderbot
Wotbox YoudaoBot
Wprecon Zade
Xenu ZoominfoBot
Zauba
Zitebot
ZmEu
ZoomBot
ZumBot
adscanner adscanner
archive.org_bot archive.org_bot
arquivo.pt
autoemailspider autoemailspider
backlink-check clark-crawler
check1.exe
coccocbot coccocbot
com.plumanalytics cognitiveseo
crawler.feedback
eCatch
evc-batch evc-batch
heritrix gopher
internetVista monitor internetVista monitor
lwp-request lwp-request
lwp-trivial
muhstik-scan muhstik-scan
netEstate NE Crawler
page scorer
polaris version polaris version
probe-image-size probe-image-size
ripz scan.lol
seocompany.store seobility
seostar
serpstatbot serpstatbot
sexsearcher
siteripz
sp_auditbot sp_auditbot
spyfu spyfu
sysscan
tAkeOut tAkeOut
voyagerx.com ubermetrics-technologies.com
webpros.com webpros.com
webprosbot
xpymep1.exe
zauba.io zauba.io
zgrab zgrab

298
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
404enemy 404enemy
80legs
AIBOT AIBOT
ASPSeek Aboundex
Aboundexbot Alexibot
AfD-Verbotsverfahren
AiHitBot
Aipbot
AllSubmitter AllSubmitter
Alligator AlphaBot
Anarchie
Anarchy99
Ankit Ankit
Apexoo Anthill
Aspiegel
Asterias Asterias
Atomseobot AwarioSmartBot
BDCbot
BDFetch BDFetch
BLEXBot Backlink-Ceck
BackDoorBot BetaBot
BackStreet
Bandit
Bigfoot Bigfoot
BlackWidow Black Hole
BlowFish
Boardreader
Bolt Bolt
BuiltWith BotALot
BunnySlippers Brandprotect
CCBot Brandwatch
Calculon Buck
BuiltBotTough
BuzzSumo
CSHttp
CazoodleBot CazoodleBot
Cegbfeieh CensysInspect
CheTeam CherryPicker
Citoid Citoid
Cloud mapping
Collector Collector
Cosmos CrazyWebCrawler
Crescent Crescent
Curious Curious
DBLBot DBLBot
DSearch
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli
Deusu
Devil Devil
Dirbuster DigitalPebble
Discobot
Discoverybot
DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project
Download Wonder Download Wonder
Drip EMail Wolf
EasyDL Ecxi
Ebingbong
Evil Evil
Exabot Exabot
Extractor Extreme Picture Finder
EyeNetIE EyeNetIE
FDM
FemtosearchBot
Firefox/7.0
FlashGet
Foobot Foobot
Freeuploader Freeuploader
Fuzz GT::WWW
Fyrebot
GermCrawler GermCrawler
GetRight GetRight
GetWeb Go!Zilla
GrabNet Gotit
Grafula Grafula
HEADMasterSEO
HMView
HTTP::Lite HTTP::Lite
HTTrack
Haansoft
Havij
Heritrix Heritrix
Iblog HonoluluBot
Image Sucker HybridBot
IndeedBot IDBTE4M
IDBot
IRLbot
IlseBot
Image Fetch
InfoNaviRobot InfoNaviRobot
Intelliseek InterGET
InternetSeer
Iria Iria
Iskanie Iskanie
Keyword Density IstellaBot
Lanshanbot Jbrofuzz
Jorgee
Jyxobot
Kozmosbot
LNSpiderguy
LWP::Simple
LeechFTP LeechFTP
LeechGet
LexiBot LexiBot
Lftp LibWeb
LieBaoFast LieBaoFast
Linkdexbot
LinkextractorPro
LinkpadBot LinkpadBot
Lipperhey Spider LinksManager
Lipperhey
Litemage_walker Litemage_walker
Lmspider
MIDown tool MIDown tool
MJ12bot MIIxpc
MQQBrowser
MSFrontPage MSFrontPage
MTRobot MTRobot
Mag-Net
Magnet Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic SEO
Majestic-SEO Majestic-SEO
MarkMonitor MarkMonitor
Mass Downloader Masscan
Mata Hari
MauiBot MauiBot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
MicroMessenger Moblie Safari
Microsoft Data Access
Mojeek
Mojolicious
MolokaiBot MolokaiBot
Mr.4x3 Msrabot
Nameprotect NICErsPRO
Navroad Name Intelligence
NearSite NearSite
Needle
Net Vampire Net Vampire
NetAnts NetAnts
NetSpider NetLyzer
NetZIP NetZIP
Nettrack Nibbler
Netvibes
NextGenSearchBot
Niki-bot Niki-bot
Nikto NimbleCrawler
Nimbostratus Nimbostratus
Ninja Nuclei
Nmap
Not
Octopus
Offline Explorer Offline Explorer
OnCrawl OpenLinkProfiler
Openfind Openvas
OrangeBot OrangeBot
OrangeSpider OutclicksBot
OutfoxBot POE-Component-Client-HTTP
Page Analyzer Panscient
PageGrabber Papa Foto
PageScorer Pavuk
Pandalytics Pi-Monster
PeoplePal Picscout
Picsearch
PictureFinder
Piepmatz Piepmatz
Pockey Pimonster
Pixray
ProPowerBot ProPowerBot
Psbot
Pu_iN Pu_iN
Pump PxBroker
Quick-Crawler
RSSingBot RSSingBot
RankActive RankingBot
RankActiveLinkBot RankingBot2
RealDownload RealDownload
RebelMouse RebelMouse
Recorder RedesScrapy
Ripper
RocketCrawler RocketCrawler
Rogerbot Rogerbot
SBIder
SEOkicks
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SISTRIX
SalesIntelligent SMTBot
ScanAlert Scrapy
Scanbot Screaming
ScreenerBot
Searchestate Searchestate
SearchmetricsBot
Seekport Seekport
SemanticJuice SemrushBot
Semrush SentiBot
Seomoz Seomoz
Site Sucker SiteLockSpider
SiteSucker SiteSucker
Sitebeam Sitebeam
Siteimprove Sogou web spider
Sitevigil Sosospider
Snake
Snapbot
Snoopy
Sociscraper
Sottopop Sottopop
SpaceBison
SpankBot
Spanner Spanner
Spinn3r Spbot
SputnikBot SputnikBot
Stripper Sqlmap
Sqlworm
Sucker Sucker
SuperBot
SuperHTTP
SurveyBot SurveyBot
Suzuran Swiftbot
Szukacz
T0PHackTeam T0PHackTeam
Teleport Teleport
Telesoft The Intraformant
Thumbor TheNomad
Titan TightTwatBot
Tracemyfile Toata
Toweyabot
Trendiction Trendiction
Trendictionbot
True_Robot
Turnitin Turnitin
TwengaBot TwengaBot
Typhoeus
URLy Warning URLy Warning
UnisterBot
V-BOT V-BOT
VeriCiteCrawler VB Project
Vacuum
VoidEYE VoidEYE
WEBDAV Voil
WISENutbot WASALive-Bot
WPScan WPScan
WeSEE WWW-Mechanize
Web Fuck WWWOFFLE
Web Pix Wallpapers
Wallpapers/3.0
Web Collage
Web Enhancer
Web Sucker Web Sucker
WebAuto WebAuto
WebFuck WebCollage
WebEnhancer
WebFetch
WebSucker
WebWhacker WebWhacker
WebsiteExtractor WebZIP
WebsiteQuester WebmasterWorldForumBot
Webster Webster
Whatweb
Widow
Wonderbot Wonderbot
Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Zade Zermelo
Zeus ZoominfoBot
Zitebot cah.io.community
ZumBot clark-crawler
arquivo-web-crawler cognitiveseo
check1.exe
crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j crawler4j
dataforseo.com dataforseo.com
demandbase-bot demandbase-bot
evc-batch domainsproject.org
facebookscraper
gopher gopher
heritrix
internetVista monitor internetVista monitor
isitwp.com iubenda-radar
magpie-crawler netEstate NE Crawler
page scorer pcBrowser
polaris version
probe-image-size
ripz ripz
seobility scan.lol
seoscanners seocompany.store
serpstatbot serpstatbot
sexsearcher sexsearcher
sitechecker.pro
sp_auditbot sp_auditbot
webgains-bot tAkeOut
trendiction.com
trendiction.de
voyagerx.com
webpros.com
webprosbot webprosbot
x22Mozilla x22Mozilla
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
bestpornuha.com 00webcams.com
dynainbox.com 5000-cotydzien.com
martlinker.com clarithromycin500mg.com
news-readers.ru elektrischezigarette2.devhub.com
nhl17coins.exblog.jp keywords-monitoring-your-success.com
okuos.com mundoaberrante.com
saitevpatorii.com platezhka.net
satellite.maps.ilovevitaly.com pogosh.com
seo-platform.com supermama.top
xportvusbdriver8i.snack.ws vip-file.com

952
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
08800.top
1000n1.ru 1000n1.ru
108shot.com 1001watch.com.ua
2728fb936f0.com 1bet.com
2daytrendingnews.com 1millionusd.xyz
4kplayer.pl 4senses.co
55wmz.ru 7makemoneyonline.com
57883.net 9icmzvn6.website
5i2.net a14download.com
76brighton.co.uk aa9d046aab36af4ff182f097f840430d51.com
7fon.ru abbanreddy.com
7yue.org abiente.ru
8si.ru adexprts.com
aad1f4acb0a373420d9b0c4202d38d94fa.com adhome.biz
addbin.men adnotbad.com
adtech.fr adobereader-free.ru
adtiger.tk
adult-shop.com.ua
adult3dgames.com
adultgalls.com
advancedmassagebysara.com advancedmassagebysara.com
advertisingtag.net adzerg.com
agreda.pluto.ro aibolita.com
agysya.ru alldezservice.kz
albuteroli.com alphaforum.ru
alevinefesleri.com amazon-adsystem.com
algarveglobal.com angigreene.com
all-streaming-media.com apimossnetinfo-a.akamaihd.net
allfinweb.com approved.su
alphavisions.net
android-systems.ru
android-vsem.org
animaltoplist.com
anime.dougasouko.com
anonymizeme.pro
anonymousfox.co
apilinkswiftco-a.akamaihd.net
apioasisspacenet-a.akamaihd.net
apxeo.info apxeo.info
arabgirls.adultgalls.com arraty.altervista.org
arabsexxxtube.com arvut.org
arenda-yeisk.ru atmagroup.ru
artdeko.info aussie-prizes.com
artpicso.com autowebmarket.com.ua
asophoto.com avon-severozapad.ru
asscenihotosidea.blogspot.co.za avtocredit-legko.ru
auto-moto-elektronika.cz avtovolop.ru
autogrep.ru azinoofficial777.ru
autotop.com.ua babieca.com
avtoarenda.by banki76.ru
azartmsl.com barnfurnituremart.com
back.dog beatifullandscape.co
bagcionderlab.com beauty-bracelet.com
balois.worldbreak.com becuo.com
bannerspace.com belinvestdom.by
barbourjackets.online best-businessman.ru
beachtoday.ru bestchoice.cf
best-seo-offer.com bestofpicture.com
bestwaystogetpaid.us bestwebsiteawards.com
bibys.com beta.hotkeys.com
bezsporno.ru
bg6s0.com
bikini-image.com bikini-image.com
binomo.kz bimatoprost-careprost.com.ua
bird1.ru bioca.org
blackcurranthumidifiers.site biocn.dx.am
bloggerads.net bizlist.com.de
bluerobot.info bizru.info
board.f00d.de blackbabesporn.com
boc.kz blog.axant.it
boleznikogi.com bloggedporn.com
bolezniorganov.ru bloke.com
bongacams.com bluesman.nu
bouda.kvalitne.cz bracketsmackdown.com
bpro1.top braintobe.win
brewdom.ru bugshoot.cn
brus.city
buketeg.xyz buketeg.xyz
butstrap.space bukleteg.xyz
buntube.net
buy-cheap-online.info
buyk.host.sk
buyscabiescream.com buyscabiescream.com
c1.onedmp.com c-english.ru
calvet.altervista.org cactussoft.biz
candy-glam-hp.com callmd5map.com
canoncdriverq3.pen.io canoncdriverq3.pen.io
carapics.com capsderfudd.tk
cardiosport.com.ua cars-modification.net
cartierbracelet.xyz cellfun.mobi
cartierlove2u.xyz cgi2.nintendo.co.jp
cartierreplica.win cheatcode-lita12.rhcloud.com
centrdebut.ru check-host.net
centre-indigo.org.ua chinese-amezon.com
cheapjerseysa.com christianlouboutinoutlet.win
cheapmarmot.online christianlouboutinreplica.pw
cheapmoncler.win cineacademy.ru
chidporn.com claytransformations.info
chocolateslim-original.com cocyq.inwtrade.com
chocolateslim-u-srbiji.com comeondog.info
ci.ua compliance-alex.xyz
cienum.fr compliance-barak.top
cjs.com.ru compliance-brian.top
clayvasedesigns.tk compliance-jack.top
click2pawn.com compliance-nelson.top
clickbank.net conciergegroup.org
clickhype.com converse.ddsoldes.fr
clicksotrk.com cookie-law-enforcement-ff.xyz
clubfashionista.com craftinsta.ru
compliance-checker.info cscwtalkto.site
compliance-elena.xyz culad.com
compliance-george.top dandiyabeats.in
compliance-viktor.top day-news.info
contentsexpress.com dbmkfhqk.bloger.index.hr
cookie-law-enforcement-aa.xyz ddospanel.com
copyrightclaims.org ddpills.com
cowlmash.com detmebel.su
cranly.net diamond-necklace.info
credit-online.ws discountbarbour.online
cubs-jerseys.us djonwatch.ru
cymbaltaandtizanidine.blogspot.com
cypernhuset.se
d-black.bz
datadepths.com
dentalpearls.com.au
dev-seo.blog
diffbot.com
dktr.ru
dms-sw.ru dms-sw.ru
documentbase.net domain.webkeyit.com
domain-submit.info domoysshop.ru
domaineaneblanc.com dumpsmania24.com
domination.ml dvd-famille.com
doreenblog.online eager-nash.188-93-233-196.plesk.page
dorratex.tn
drugspowerstore.com
drupa.com
dverimegapolis.ru
e90post.com
easyshoppermac.com easyshoppermac.com
econom.co editmedios.com
ege-essay.ru ekaterinburg.xrus.org
elektrischezi.canalblog.com elektroniksigaraankara.info
eralph.tk eluxer.net
erotikchat-24.com eng-lyrics.com
euroskat.ru englate.com
evaashop.ru epngo.bz
eropho.com
erotag.com
esnm.ru
essay-writing.work
este-line.com.ua
event-tracking.com event-tracking.com
exmasters.com exonline.info
ez8motelseaworldsandiego.com expdom.com
ezigarettekaufen1.hpage.com extremepornos.net
farm26.ru ezb.elvenmachine.com
farmingworm.com f1nder.org
farmprofi.net fakehandbags.xyz
famix.xyz
fashionavenuegame.com fashionavenuegame.com
favornews.com fdzone.org
filmci.pro fickblock18.com
firstdrugmall.ru filesvine.com
flex4launch.ru findsexguide.com
foodgid.net fireads.men
footbalive.org firesub.pl
forumprofi.de flagcounter.me
free-deals.faith foto-sisek.porngalleries.top
free-today.com freejabs.com
freefoto.ca freshberry.com.ua
freeuploader.ml fuck-paid-share-buttons.xyz
freshdz.com fungamelands.com
freshmac.space fym.com.ru
froggytube.com g7m.pl
ftns.ru galaxyflowers.ru
g.starmoe.xyz galeon.com
gamedayhouse.com gay-site.store
gate5.co.za gazobeton-p.com.ua
gaygalls.net gdebestkupit.ru
gayxperience.com
gdzkurokam.ru
geckoandfly.com
gemgrab-a.akamaihd.net
gepezz.info gepezz.info
get-your-social-buttons.info getlaid-xxxhookupdirect.com
getdot.ru getprismatic.com
ghostvisitor.com getridofstretchmarks.org
gigapeta.com ghernnqr.skyrock.com
glogow.pl gkvector.ru
glopages.ru glcomputers.ru
gloverid.site globetrotting-culture.ru
golden-catalog.pro golfresa.lucania.se
goodwriterssales.com googlemare.com
googlsucks.com gourcy.altervista.org
gq-catalog.gq gowreckdiving.com
graphid.com gribokstop.com
gratuitxblcodes.com gsmlab.pl
gsbs.com.ua
guardlink.com
guardlink.org guardlink.org
guruofcasino.com gymi.name
gz2.bbsoldes.fr hard-porn.mobi
hacktougroup.ru hayate.biz
handicapvantoday.com hdimagelib.com
hao123.com healmytrauma.info
hatdc.org health-medical-portal.info
hatedriveapart.com hentaiheroes.com
hdimagegallery.net hermesreplica.pw
hdpixent.com high-speed1.net
healgastro.com historichometeam.com
hermesbracelets.xyz hotblognetwork.com
hkdiiohi.skyrock.com hotchatdate.com
hkladys.com house.sieraddns.com
homegardenlova.com
honyaku.yahoofs.jp
horoshieokna.com
hothor.se
houtings.xyz
hoverboardforsaledirect.com
howlongdoestizanidinestayinyoursystem.blogspot.com
hscsscotland.com hscsscotland.com
hspline.com huimin764128.com
hyhj.info idegenvezeto.eu
iblogpress.xyz ideibiznesa2015.ru
ie.57883.net ido3.com
ifmo.ru ilo134uloh.com
igithab.com images-free.net
imperia31.ru
imperiafilm.ru
impresagaia.it
inbabes.sexushost.com inbabes.sexushost.com
internetproviderstucson.com incest-ru.com
iplusbit.blogspot.co.za insider.pro
itis4you.com iqoption.pro
jerseychinabizwholesale.us itronics.ca
jmat.cn ivanovo.zrus.org
johnnyhaley.top iwantedmoney.com
karting196.ru iweblist.info
keenoutlet.online jacago.com
kinogonew.ru jackpotchances.com
klikbonus.com jamiembrown.com
kmd-pto.ru jennyfire.ru
kndxbkdx.bloger.index.hr jrpmakati.com
koleso24.com.ua kazinogames.lv
konpax.com kerch.site
kosova.de kevblog.top
krasivye-devushki.net klosetkitten.com
labafydjxa.su knowsitall.info
landmania.ru kollesa.ru
kolotiloff.ru
konoplisemena.com
kostenlos-sexvideos.com
kriokomora.info
ks1234.com
kursy-ege.ru
kurwa.win
laparfumotec.com laparfumotec.com
laudit.ru layola.biz.tc
lecbter-relationships.ga leftofcentrist.com
lechtaczka.net lesbian.xyz
ledis.top letmacworkfaster.world
lego4x4.ru lexixxx.com
liky.co.ua libertybilisim.com
littleberry.ru life.biz.ua
littlecity.ch limads.men
livejasmin.com lincolntheatre.com
livingcanarias.com lineavabit.it
lmrauction.com livefixer.com
lombia.com liveshoppersmac.com
lonfon.xyz livetsomudvekslingstudent.bloggersdelight.dk
long-beach-air-conditioning.com locustdesign.co
lovelycraftyhome.com longadventure.com
lsex.xyz lunamedia.co
macdamaged.tech m3gadownload.pl
makemoneyonline.com machicon-ueno.info
makenahartlin.com maladot.com
manifestation.betteroffers.review malwareremovalcenter.com
manve.info mararoom.ru
marinetraffic.com marmitaco.cat
masthopehomes.com masserect.com
maxxtor.eu medkletki.ru
mebel-vstroika.ru mega-bony-2017.pl
meetlocalchicks.com megaapteka.ru
mielec.pl megavolt.net.ua
mirmedinfo.ru merryhouse.co.uk
mlvc4zzw.space migente.com
mmgq.ru mirtorrent.net
mixtapetorrent.com
mjchamonix.org
mmog-play.ru mmog-play.ru
mobicover.com.ua mmoguider.ru
mobot.site mobplayer.net
mobsfun.net moetomnenie.com
movpod.in mojaocena.com
moyaterapiya.ru moscow.online-podarki.com
mp3ringtone.info
mug-na-chas-moscow.ru mug-na-chas-moscow.ru
musicpro.monster musezone.su
mxgetcode.com musicspire.online
mybackgroundlandscape.blogspot.com mydirtystuff.com
myplaycity.com myfreetutorials.com
myshopmatemac.com myindospace.com
namenectar.com mymercy.info
naperehresti.info myphotopipe.com
nardulan.com myyour.eu
narkologiya-voronezh.ru na15.ru
neodownload.webcam nacap.ru
netoil.no nakrutka.cc
new-post.tk nalogovyy-kodeks.ru
newfilmsonline.ru narco24.me
nfljerseyscheapchinabiz.com nastydollars.com
nhl09.ru naughtyconnect.com
njkmznnb.ru needtosellmyhousefast.com
nonameread45.live negral.pluto.ro
nordstar.pro netallergy.ru
nosecret.com.ua networkcheck.xyz
nottyu.xyz news-readers.ru
noumeda.com nfvsz.com
nuker.com nl.netlog.com
obiavo.ru nlfjjunb5.ru
oecnhs.info nofreezingmac.click
ofanda.com novosti-hi-tech.ru
okonich.com.ua nsatc.net
o-o-8-o-o.ru
obiavo.com
obuv-kupit.ru
ochistka-stokov.ru
offergroup.info
okmusic.jp
okuos.com okuos.com
olgacvetmet.com olympescort.com
onblastblog.online onlainbesplatno.ru
onlineporno.site onlinadverts.com
ooo-olni.ru onlineserialy.ru
optitrade24.com onlinewritingjobs17.blogspot.ru
orsonet.ru onload.pw
page2rss.com ororodnik.goodbb.ru
paradontozanet.ru ozoz.it
parajumpersjakkesalgnorge.info pandroid.co
paramountmarble.co.uk partnerworkroom.men
pb-dv.ru pc-virus-d0l92j2.pw
pcboa.se peekyou.com
pdamods.ru perederni.net
penisvergrotendepillennl.ovh perso.wanadoo.es
picsearch.com petitions.whitehouse.gov
pinwallpaper.xyz petsblogroll.com
piratecams.com picsforkeywordsuggestion.com
pixgood.com pinapchik.com
plastweb.ru piter.xrus.org
poffet.net pix-hd.com
poquoson.org pl-top.pl
pornoreino.com play-movie.pl
pornozhara.com playtap.us
portadd.men poem-paying.gq
powitania.pl poiskzakona.ru
praisong.net popads.net
prchecker.info poppen-nw.net
presleycollectibles.com popup-hgd.xyz
prize44.com porn555.com
productarium.com pornmania.pl
profolan.pl pornohd1080.online
prohoster.info pornophoto.xyz
prointer.net.ua pozdravleniya-c.ru
prom23.ru pp-budpostach.com.ua
promover.org preparevideosafesystem4unow.site
pronorm.fr priceg.com
prostitutki-kharkova.org printie.com
privatamateure.com
prod2016.com
producm.ru
proekt-mos.ru
projefrio.com.br
prpops.com prpops.com
pssucai.info puzzleweb.ru
puteshestvennik.com questionmarque.ch
qor360.com qwesa.ru
quick-offer.com ranksignals.com
ranksays.com rating-casino2021.ru
ranksonic.info realnye-otzyvy.info
rasteniya-vs-zombi.ru rebelmouse.com
refererx.com rentaremotecomputer.com
remmling.de resant.ru
remont-komputerov-notebook.ru reyel1985.webnode.fr
rentehno.ru royalcar-ufa.ru
rezeptiblud.ru rp9.ru
rfd-split.hr rrutw.com
robotixix.com ruex.org.ua
roofers.org.uk rukino.org
royal-investments.net rus-teh.narod.ru
ruf777.com russia-tao.ru
russintv.fr sarahmilne.top
saugatuck.com savefrom.com
sbdl.no savememoney.co.za
scanner-jessica.top sbricur.com
search.1and1.com sbt-aqua.ru
sc-specialhost.com
scanmarine.info
scanner-alexa.top
scat.porn
scenarii-1-sentyabrya.uroki.org.ua
scrapinghub.com
searchaddis.com
searchengineranker.email
searchinquire.com searchinquire.com
sei80.com searchmywindow-a.akamaihd.net
senger.atspace.co.uk seasonvar.ru
seo-prof1.xyz secret.xn--oogle-wmc.com
seokicks.de see-your-website-here.com
sex-tracker.com seksotur.ru
sexblog.pw sendearnings.com
sexvporno.ru seoheap.com
sexytrend.ru seojokes.net
sfj-ror.no seolab.top
sherlock.se service-core.ru
ship-marvel.co.ua sextracker.com
shoesonlinebuy.cn shop.acim.org
shop-electron.ru silvercash.com
shopfishing.com.ua similardeals.net
shopperifymac.com simplepooltips.com
shtora66.ru simply.net
shymkent.xkaz.org sims-sims.ru
sibtest.ru sirpornogratis.xxx
sientalyric.co siteexpress.co.il
silktide.com skutecznetabletkinaporostwlosow.pl
site3.free-share-buttons.com smadihome.com
sitebeam.net smartbalanceworld.com
smichovbike.cz smokewithrabbits.com
snjack.info social-buttons.com
socialmonkee.com sockshares.tv
soda.media softomix.net
sotkal.lark.ru solinf.co
souvenir.cc spamnuker.com
soviet-portal.do.am stickers-market.ru
spabali.org survival.betteroffers.review
space-worry.ml suture.co
spb.afora.ru svetodiodoff.ru
spravka130.ru swagbucks.com
star61.de taihouse.ru
staticfs.host takeflyte.com
stefanbakosab.se tako3.com
strana-krasoty.ru tamada69.com
streetfire.net tedxrj.com
supernew.org teencastingporn.com
svetlotorg.ru telefonsex-ohne0900.net
swinon.site telefonsexsofort.tk
tacbelarus.ru testbotprocessor44.com
tackletarts.co thiegs.reco.ws
taraz.xkaz.org ticketsys.inetwd.com
td-33.ru tiens2010.ru
td-l-market.ru tizanidinemusclerelaxant.blogspot.com
tengohydar.tk tizanidineonline.blogspot.com
thefotosgratis.eu tizanidinesideeffects.blogspot.com
thepokertimer.com tizanidinestreetvalue.blogspot.com
thetattoohut.com tnaionline.org
tizanidine4mg.blogspot.com todohr.com
tizanidineandcipro.blogspot.com tongkatmadura.info
tizanidineandgabapentin.blogspot.com top10registrycleaners.com
tizanidinedosageforsleep.blogspot.com torrnada.ru
tizanidinehcl4mgtabinfo.blogspot.com tozup.com
tizanidineorflexeril.blogspot.com tpu.ru
tizanidineshowupondrugtest.blogspot.com tracfone.com
tn811.us trackmedia101.com
top-karkas.ru tracksz.co
top-l2.com treasuretrack-a.akamaihd.net
tourcroatia.co.uk triplepanda.xyz
trudogolik.net trustedmaccleaner.com
turbabitload.weebly.com tryrating.com
tversvet.ru tula.howotorg.ru
twitlinks.com turizmus.us
twodollarshows.com tvteleport.ru
uginekologa.com u555u.info
uk-zheu20.ru ublaze.ru
ukrtvir.com.ua uchetunet.su
umekana.ru ugogo.info
umityangin.net umaseh.com
undergroundcityphoto.com unblocksit.es
unrealcommander.biz uni.me
uptime-as.net upproar.com
uptime-gamma.net uptimebot.net
uptime.com ural-buldozer.ru
usdx.us urlopener.com
vadimkravtcov.ru uroffer.link
ventelnos.com usbggettwku.ga
veselokloun.ru vabasa.inwtrade.com
viagengrarx.com validccseller.com
videochat.ph valueclick.com
videogamesecrets.com vbabule.net
videospornogratisx.net venerologiya.com
vltai.com via-gra.webstarts.com
voloslove.ru viddyoze.com
voluumtrk.com video-camer.com
vostoktrade.info vintontech.info
vipsiterip.org
virus-schutzmasken.de
vozbujdenie.com vozbujdenie.com
vut.com.ru vpnmouse.com
vzlom-na-zakaz.com vroze.com
w3data.co vseigry.fun
wait3sec.org wait3sec.org
walkme.com warningwar.ru
watchdogs-2.ru web-analytics.date
we-are-gamers.com web-betting.ru
webextract.profound.net webalan.ru
webinstantservice.com website-audit.com.ua
webpromotion.ae websiteexplorer.info
webscouter.net weburlopener.com
website-analytics.online wechatdownload10.com
webtherapy.ru welcomeauto.ru
westsextube.com weprik.ru
where-toget.com wfb.hatedriveapart.com
whiteproduct.com wjgony.com
wholinkstome.com wosik-dach.service-for-web.de
whosonmyserver.com wrz0iuebwhp5fg.freeddns.com
williamrobsonproperty.com xbaboon.com
word-vorlagen.xyz xchangetrak.com
www.ehscloud.cn xmlinde.com
www.jbetting.com xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
x-diesel.com xn----btbdvdh4aafrfciljm6k.xn--p1ai
xep.info xn----dtbndd4ae7eub.top
xfire.com xn--80ab4aa2g.xn--p1ai
xgftnlrt.bloger.index.hr xn--80ajbshivpvn2i.xn--p1ai
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xpresscare.ru
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai xtubeporno.net
xn----9sbebi2bvzr7h.xn--p1ai
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xn--90acenikpebbdd4f6d.xn--p1ai
xn--d1abj0abs9d.in.ua
xn--l1aengat.xn--p1ai
xn--lifehacer-1rb.com
xxx-cam.webcam xxx-cam.webcam
xxxasianporn.net xxxtube69.com
xzlive.com yaoguangdj.com
yougetsignal.com yellowstonesafaritours.com
yousense.info yopoint.in
youtubedownload.org yourdesires.ru
yunque.pluto.ro yourtemplatefinder.com
zastenchivosti.net zaapplesales.blogspot.com
zed21.net zaimite.ru
zeikopay.com zdesformula.ru
zeroredirect12.com zeg-distribution.com
zeroredirect2.com zeleznobeton.ru
zoogdiesney.com zeroredirect6.com
zscaler.net zigarettenonlinekaufen1.blox.pl
zzbroya.com.ua zigarettenonlinekaufen2.drupalgardens.com
zoompegs.com
zscalertwo.net

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1-99seo.com 1qingdao.com
12masterov.com 1stat.ru
1j7740kd.website 7zap.com
2nt.ru 83net.jp
3w24.com a96527gi.beget.tech
4ureyesonly.com aa81bf391151884adfa3dd677e41f94be1.com
7minuteworkout.com aab94f698f36684c5a852a2ef272e031bb.com
a.frcls.fr abcdefh.xyz
acgs.tk activepr.ru
advocatemsk.ru adamoads.com
aero2.ru adinterax.com
affordablewebsitesandmobileapps.com alekseevec.ru
aibolita.com alfa-medosmotr.ru
allkrim.com alldubai.biz
allornamenti.com altamayoztourism.com
allpdfmags.net animal-drawings.com
anatomiy.com apiallgeniusinfo-a.akamaihd.net
apisecretsaucebi-a.akamaihd.net app-ready.xyz
arpelsreplica.xyz arabseyes.com
autoua.top articlesdirectoryme.info
avek.ru ausergrubhof.info
ayakino.net autonew.biz
batanga.net avon-novosib.ru
beremenyashka.com awency.com
bestimagecoollibrary.com baltgem.com
bestplacetobuyeriacta.jw.lt beachtoday.ru
bezsporno.ru bestmobilityscooterstoday.com
bidbuy.co.kr bet365.com
bildsuche.ru bg6s0.com
bimatoprost-careprost.com bkns.vn
bitcoinpile.com blog-hits.com
blackcurranthumidifiers.site bluesman.nu
board.f00d.de bonkers.name
boobsimge.com boom-celebs.com
bristolhotel.com brothers-smaller.ru
brus.city
buchananshardware.com
bugof.gq
buynorxx.com buynorxx.com
camel-beach.com bytimedance.ru
canrioloadm.gq callcafe.info
cartierlove2u.com callejondelpozo.es
cartujano-pre.de calstaterealty.us
centrdebut.ru carina-sy.de
chatroulette.si carsplate.com
cheapjerseysfootballshop.com cartierlovestore.xyz
chuckguilford.com cdncash.net
ci.ua chocoslim.pro
cienum.fr citizenclsdriveri7.pen.io
club-musics.ru cleanmypc.com
compliance-elena.top cocolyze.com
costablue.xyz compliance-andrew.xyz
crest3d.ru compliance-checker-7.info
cubs-jerseys.us compliance-irvin.top
compliance-olga.top
compliance-viktor.top
cookie-law-enforcement-hh.xyz
copyrightclaims.org
cottageofgrace.com
covadhosting.biz
covid-schutzmasken.de
crazyboost.pro
credit.co.ua
crystalslot.com
customcollegeessays.net
cymbaltaandtizanidine.blogspot.com
cyse.tk
d-black.bz
datract.com
davebestdeals.com
db.speedup.gdn db.speedup.gdn
delayreferat.ru dear-diary.ru
deniven.1bb.ru deutschland123.de
dev.citetick.com documentsite.net
djonwatch.ru doreenblog.online
doctormakarova.ru dorratex.tn
domaincdn.xyz druzhininevgeniy63.blogspot.com
doxyporno.com dyshagi.ru
easyshoppermac.com eachdayisagift.review
ecookna.com.ua ecommerce-seo.com
encodable.com edudocs.net
engines-usa.com en.altezza.travel
ero-advertising.com ertelecom.ru
erotik-kostenlos.net evehemming.blogspot.com.au
ezigarettekaufen2.blox.pl expresstoplivo.ru
ezigarettekaufen2.yolasite.com extremez.net
ezigarettenshop2.postbit.com f5mtrack.com
factorynightclub.com favoritemoney.ru
fanrto.com fba-mexico.com
farm26.ru feorina.ru
felizporno.com fetroshok.ru
forex.osobye.ru fettefrauen.net
forum.tvmir.org flipper.top
frequiry.com formularz-konkurs.tk
freshdz.com free-social-buttons.com
fullgirl.ru gamblingnerd.com
fym.com.ru gameonasia.com
g7m.pl gamerextra.com
gambarkataku.co gate5.co.za
genericlowlatencyasiodriverhq.aircus.com generalporn.org
gnuetella.com get-your-social-buttons.info
goodwriterssales.com gg-arena.ru
googlepositions.com girlspicsa.com
googlsucks.com greenzaim.ru
gopro-online.info grizzlysgrill.com
gosmeb.ru grupografico-pilar.com.ar
growboxbl.ru gsasearchenginerankerdiscount.com
gtopstats.com
hccoder.info
hd-filmy.net hd-filmy.net
hdwalls.xyz helpmymacfaster.trade
healthcarestore.info
hem.passagen.se
hentai-manga.porn
hjaoopoa.top
hostnow.men
hotchatdate.com hotchatdate.com
humanelydrew.com
hydropump.su
hyhj.info
ideoworld.org ideoworld.org
ikearugs.xyz iinstalll-fii1leis.jus0wil.pp.ua
investmac.com imitex-plus.ru
investsuccess.org import-sales.com
isoveti.ru inome.com.ua
itbc.kiev.ua interfucks.net
itrevolution.cf investpamm.ru
itsdp3.com iqupdatetmz.win
iwanttodeliver.com istripper.com
jav-idol.com jerseyswholesalechinalimited.com
jongose.ninja johnrobertsoninc.com
jyvopys.com kam-dom.ru
karpun-iris.ru kiinomaniak.pl
ketrzyn.pl kino-filmi.com
kicknights.gq kino-rf.ru
kit-opt.ru
kol-energo.ru kol-energo.ru
krasivoe-hd.com kung-fu-ru.com
kursy-ege.ru kutikomi.net
kurwa.win larger.io
lawyersinfo.org lechtaczka.net
lerporn.info lexiangwan.com
linkbux.ru lirunet.ru
lion.snapmap.us llastbuy.ru
lolnex.us lollypopgaming.com
lonely-mature.com lonfon.xyz
longadventure.com love-baby.cz
manipulyator-peterburg.ru ltvperf.com
mansiondelrio.co magicalfind-a.akamaihd.net
manuscript.su marketingvici.com
marmotstore.online metallosajding.ru
maslenka.kz mindeyegames.com
mediaoffers.click minet.club
merryhouse.co.uk moje-recenze.cz
milblueprint.com monarhs.info
mirzonru.net monclercoats.xyz
mompussy.net moscow.online-podarki.com
moneytop.ru muz-tracker.net
morocco-nomad-excursions.com narkologiya-peterburg.ru
moscow-clining.ru navalwiki.info
mp3s.club negral.pluto.ro
musicdaddy.net neodownload.webcam
musicktab.com netpics.org
musicpro.monster netvouz.com
mymobilemoneypages.com nfljerseyscheapbiz.us
mypets.by nufaq.com
naturtreenspicerx.pw online-zaymy.ru
neks.info online7777.com
nevansk.ru optibuymac.com
new-apps.ru
nyfinance.ml
o-o-8-o-o.ru
online247.ml
origin-my.ru
osnova3.ru
ostroike.org ostroike.org
ozas.net oxford-book.com.ua
partner-trustworthy.men parfusale.se
photo.houseofgaga.ru patol01.pw
picphotos.net pc-virus-d0l92j2.pw
picturesmania.com pdns.download
piski.top piski.top
pixell.club platinumdeals.gr
pizza-tycoon.com pokemon-go-play.online
pobeiranie.pl polska-poezja.com
poem-paying.gq polybuild.ru
porn-w.org pornofiljmi.com
porngalleries.top pornonik.com
pornoinn.com pornophoto.xyz
pp-budpostach.com.ua pr0fit-b0x.com
pro-poly.ru privacylocationforloc.com
prostitutki-tolyatti.org prizesbook.online
quickcashlimited.com prokotov.com
rabot.host.sk promgirldresses.xyz
prostitutki-yaroslavlya.org
qualitymarketzone.com
rank-checker.online
rankscanner.com rankscanner.com
rasteniya-vs-zombi.ru razorweb-a.akamaihd.net
rcpmda.ikan1080.xyz razyboard.com
renewablewealth.com reklama-i-rabota.ru
research.ifmo.ru remybutler.fr
ringporno.com rep-am.com
rniaeba.ga rogervivierforsale.com
robotixix.com rostov.xrus.org
rockma.se roznica.com.ua
roleforum.ru ru-dety.ru
rossmark.ru scmor.ilxc.cc
sack.net seasaltwithfood.com
saecsa.co serving.adbetclickin.pink
samolet.fr sexsaoy.com
scanner-fred.top shoesonlinebuy.xyz
schalke04fc.info signx.info
scoopquest.com sildenafilcitratemed.com
senger.atspace.co.uk sirpornogratis.xxx
seokicks.de skylta.com
seonetwizard.com skyway24.ru
sexkvartal.com slonechka.ru
sexpartygirls.net socialmonkee.com
sexpornotales.com solicita.info
sexsearch.com sortthemesitesby.com
shell-pmr.ru spidtest.space
sibecoprom.ru start.myplaycity.com
simpoed.ufop.br sweepstakes.rewardit.com
softwaretrend.net tizanidine6mg.blogspot.com
space2019.top tizanidineandgabapentin.blogspot.com
spy-app.info tizanidinemusclerelaxant.blogspot.com
standardchartered-forex.com umekana.ru
steebook.com unblocksit.es
strv.se unionmarkt.de
sukirgenk.dvrlists.com upproar.com
supernew.org uptime-as.net
swagbucks.com usadacha.net
swingerseiten.com vacuumscleaner.com
tastyfoodideas.com velpanex.ru
technika-remont.ru videochat.life
testbotprocessor44.com videochat.mx
tetracsaudi.com vinylvault.co.uk
theallgirlarcade.com vipps.com.my
time-japan.ru vipsexfinders.com
titan-ads.life virus-schutzmasken.de
torrnada.ru visa-china.ru
tracklead.net vizag.kharkov.ua
tuberkulezanet.ru vkmusics.ru
twincitiescarservice.com webmasterhome.cn
ultimateclassicrock.com webtherapy.ru
validdomain.xyz weburlopener.com
valsalud.com where-toget.com
vegascosmetics.ru white-truck.life
vellings.info wordkeyhelper.com
viberdownload10.com writingservices17.blogspot.ru
vkgaleria.com x-true.info
voprosotvet24.ru xn--80aaagvmjabrs1aoc9luc.xn--p1ai
wanker.us xn--b1adccaf1bzj.xn--p1ai
webenlace.com.ar yeartwit.com
webmonetizer.net youandcredit.ru
websitevaluebot.com youthreaders.com
welcomeauto.ru zarabiaj-dzis.pl
westen-z.life zastenchivosti.net
wheelchairliftsarea.com zazagames.org
wildnatureimages.com zigarettenonlinekaufen2.bloog.pl
wladimirpayen.com zionstar.net
wowcasinoonline.ooo zoodrawings.com
wrz0iuebwhp5fg.freeddns.com zygophyceous.womanstars.site
xlovecam.com
xn----ctbigni3aj4h.xn--p1ai
yhit.press
yottos.com
yousense.info
zawyna.ua
zeroredirect12.com
znakomstva-piter78.ru
znaturaloriginal.com
zojirushi-products.ru
zs2vm.top

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.49 0; 10.1.1.98 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2932 ### Version: V4.2022.02.2933
### Updated: Sun Feb 20 10:39:12 UTC 2022 ### Updated: Sun Feb 20 11:02:14 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2932 ### Version: V4.2022.02.2933
### Updated: Sun Feb 20 10:39:12 UTC 2022 ### Updated: Sun Feb 20 11:02:14 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##