vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.07.2463 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-07-09 07:13:44 +00:00
parent 93076abef4
commit f088fbfc60
56 changed files with 90908 additions and 88382 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.07.2462 #### Version: V4.2021.07.2463
#### Bad Referrers Blocked: [7066](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7067](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [615](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [615](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,8 +4,8 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.07.2462 #### Version: V4.2021.07.2463
#### Bad Referrers Blocked: [7066](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7067](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [615](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [615](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -11,8 +11,8 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.07.2462 #### Version: V4.2021.07.2463
#### Bad Referrers Blocked: [7066](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7067](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [615](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [615](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

10910
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

5
_generator_lists/bad-referrers.list
View file

@ -1,5 +1,3 @@
nina.az
pornoinn.com
000free.us 000free.us
007angels.com 007angels.com
00author.com 00author.com
@ -4139,6 +4137,7 @@ niki-mlt.ru
nikitabuch.com nikitabuch.com
nikitsyringedrivelg.pen.io nikitsyringedrivelg.pen.io
nikkiewart.ru nikkiewart.ru
nina.az
ningessaybe.me ningessaybe.me
nippon-bearings.ru nippon-bearings.ru
niroo.info niroo.info
@ -7066,4 +7065,4 @@ zygophyceous.womanstars.site
zynax.ua zynax.ua
zytpirwai.net zytpirwai.net
zzbroya.com.ua zzbroya.com.ua
zzlgxh.com zzlgxh.com

4
_google_analytics_ghost_spam/README.md
View file

@ -3,8 +3,8 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.07.2462 #### Version: V4.2021.07.2463
#### Bad Referrers Blocked: [7066](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7067](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [615](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [615](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

2
_google_analytics_ghost_spam/google-exclude-03.txt
View file

File diff suppressed because one or more lines are too long

2
_google_analytics_ghost_spam/google-exclude-04.txt
View file

File diff suppressed because one or more lines are too long

1
_google_webmaster_disavow_links/google-disavow.txt
View file

@ -4137,6 +4137,7 @@ domain:niki-mlt.ru
domain:nikitabuch.com domain:nikitabuch.com
domain:nikitsyringedrivelg.pen.io domain:nikitsyringedrivelg.pen.io
domain:nikkiewart.ru domain:nikkiewart.ru
domain:nina.az
domain:ningessaybe.me domain:ningessaybe.me
domain:nippon-bearings.ru domain:nippon-bearings.ru
domain:niroo.info domain:niroo.info

10917
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10917
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10917
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10917
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10917
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10917
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/beta_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -4184,6 +4184,7 @@
"~*(?:\b)nikitabuch\.com(?:\b)" 0; "~*(?:\b)nikitabuch\.com(?:\b)" 0;
"~*(?:\b)nikitsyringedrivelg\.pen\.io(?:\b)" 0; "~*(?:\b)nikitsyringedrivelg\.pen\.io(?:\b)" 0;
"~*(?:\b)nikkiewart\.ru(?:\b)" 0; "~*(?:\b)nikkiewart\.ru(?:\b)" 0;
"~*(?:\b)nina\.az(?:\b)" 0;
"~*(?:\b)ningessaybe\.me(?:\b)" 0; "~*(?:\b)ningessaybe\.me(?:\b)" 0;
"~*(?:\b)nippon\-bearings\.ru(?:\b)" 0; "~*(?:\b)nippon\-bearings\.ru(?:\b)" 0;
"~*(?:\b)niroo\.info(?:\b)" 0; "~*(?:\b)niroo\.info(?:\b)" 0;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10917
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2462 2463

10917
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10917
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

1
dev-tools/referrers-regex-format-whitelist-test.txt
View file

@ -4137,6 +4137,7 @@
"~*(?:\b)nikitabuch\.com(?:\b)" 0; "~*(?:\b)nikitabuch\.com(?:\b)" 0;
"~*(?:\b)nikitsyringedrivelg\.pen\.io(?:\b)" 0; "~*(?:\b)nikitsyringedrivelg\.pen\.io(?:\b)" 0;
"~*(?:\b)nikkiewart\.ru(?:\b)" 0; "~*(?:\b)nikkiewart\.ru(?:\b)" 0;
"~*(?:\b)nina\.az(?:\b)" 0;
"~*(?:\b)ningessaybe\.me(?:\b)" 0; "~*(?:\b)ningessaybe\.me(?:\b)" 0;
"~*(?:\b)nippon\-bearings\.ru(?:\b)" 0; "~*(?:\b)nippon\-bearings\.ru(?:\b)" 0;
"~*(?:\b)niroo\.info(?:\b)" 0; "~*(?:\b)niroo\.info(?:\b)" 0;

1
dev-tools/referrers-regex-format.txt
View file

@ -4137,6 +4137,7 @@
"~*(?:\b)nikitabuch\.com(?:\b)" 1; "~*(?:\b)nikitabuch\.com(?:\b)" 1;
"~*(?:\b)nikitsyringedrivelg\.pen\.io(?:\b)" 1; "~*(?:\b)nikitsyringedrivelg\.pen\.io(?:\b)" 1;
"~*(?:\b)nikkiewart\.ru(?:\b)" 1; "~*(?:\b)nikkiewart\.ru(?:\b)" 1;
"~*(?:\b)nina\.az(?:\b)" 1;
"~*(?:\b)ningessaybe\.me(?:\b)" 1; "~*(?:\b)ningessaybe\.me(?:\b)" 1;
"~*(?:\b)nippon\-bearings\.ru(?:\b)" 1; "~*(?:\b)nippon\-bearings\.ru(?:\b)" 1;
"~*(?:\b)niroo\.info(?:\b)" 1; "~*(?:\b)niroo\.info(?:\b)" 1;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.211.4.195 0; 52.158.247.210 0;

10917
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10917
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10917
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test1_conf_files_whitelist/bots.d/custom-bad-referrers.conf
View file

@ -4184,6 +4184,7 @@
"~*(?:\b)nikitabuch\.com(?:\b)" 0; "~*(?:\b)nikitabuch\.com(?:\b)" 0;
"~*(?:\b)nikitsyringedrivelg\.pen\.io(?:\b)" 0; "~*(?:\b)nikitsyringedrivelg\.pen\.io(?:\b)" 0;
"~*(?:\b)nikkiewart\.ru(?:\b)" 0; "~*(?:\b)nikkiewart\.ru(?:\b)" 0;
"~*(?:\b)nina\.az(?:\b)" 0;
"~*(?:\b)ningessaybe\.me(?:\b)" 0; "~*(?:\b)ningessaybe\.me(?:\b)" 0;
"~*(?:\b)nippon\-bearings\.ru(?:\b)" 0; "~*(?:\b)nippon\-bearings\.ru(?:\b)" 0;
"~*(?:\b)niroo\.info(?:\b)" 0; "~*(?:\b)niroo\.info(?:\b)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10917
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.211.4.195 0; 52.158.247.210 0;

10917
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.211.4.195 0; 52.158.247.210 0;

10917
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

1
dev-tools/test_units/custom-bad-referrers.conf
View file

@ -4184,6 +4184,7 @@
"~*(?:\b)nikitabuch\.com(?:\b)" 0; "~*(?:\b)nikitabuch\.com(?:\b)" 0;
"~*(?:\b)nikitsyringedrivelg\.pen\.io(?:\b)" 0; "~*(?:\b)nikitsyringedrivelg\.pen\.io(?:\b)" 0;
"~*(?:\b)nikkiewart\.ru(?:\b)" 0; "~*(?:\b)nikkiewart\.ru(?:\b)" 0;
"~*(?:\b)nina\.az(?:\b)" 0;
"~*(?:\b)ningessaybe\.me(?:\b)" 0; "~*(?:\b)ningessaybe\.me(?:\b)" 0;
"~*(?:\b)nippon\-bearings\.ru(?:\b)" 0; "~*(?:\b)nippon\-bearings\.ru(?:\b)" 0;
"~*(?:\b)niroo\.info(?:\b)" 0; "~*(?:\b)niroo\.info(?:\b)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
GetWeb Abonti
Lftp BDCbot
MQQBrowser BackDoorBot
Mag-Net FlashGet
Nibbler MarkMonitor
PageScorer Offline Explorer
Papa Foto Re-re
T8Abot Screaming
WebCollage Toweyabot
WebCopier zauba.io

312
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
ADmantX 360Spider
404enemy
80legs
AIBOT AIBOT
Aboundex Abonti
Aboundexbot
Acunetix
AfD-Verbotsverfahren
AhrefsBot
AllSubmitter AllSubmitter
Anarchie AlphaBot
Anarchy99 Ankit
Apexoo Anthill
Asterias Aspiegel
Attach
AwarioRssBot AwarioRssBot
BBBike
BDCbot BDCbot
BackDoorBot BackWeb
Badass Badass
Bandit
BatchFTP BatchFTP
Battleztar Bazinga Bigfoot
Bitacle BlackWidow
Blackboard
Blow Blow
BlowFish BlowFish
Boardreader Bolt
BotALot
Brandprotect Brandprotect
Brandwatch
Buddy
BuiltBotTough BuiltBotTough
BuiltWith Bullseye
CCBot BunnySlippers
CheeseBot Calculon
ChinaClaw
Chlooe
Cliqzbot
Cogentbot Cogentbot
Cosmos Copyscape
Craftbot
CrazyWebCrawler
Curious
CyotekWebCopy CyotekWebCopy
DBLBot DSearch
DataCha0s DTS Agent
DatabaseDriverMysqli
Devil Devil
Disco Digincore
Dispatch Dirbuster
Discobot
DnyzBot
DomainCrawler DomainCrawler
DomainSigmaCrawler DomainStatsBot
Dragonfly
Drip
EMail Siphon
EMail Wolf EMail Wolf
Ebingbong
Ecxi Ecxi
EirGrabber Express WebPictures
EroCrawler
ExtLinksBot ExtLinksBot
FHscan Extractor
Fimap ExtractorPro
Extreme Picture Finder
EyeNetIE
Firefox/7.0
FlashGet FlashGet
Flunky Foobot
FrontPage FrontPage
FyberSpider G-i-g-a-b-o-t
Fyrebot GT::WWW
GalaxyBot Genieo
Getintent GermCrawler
GoZilla GetRight
Gigablast
Go!Zilla
Gotit Gotit
GrabNet Grabber
GrapeFX GrapeshotCrawler
HEADMasterSEO HMView
HTTP::Lite HTTP::Lite
Havij HTTrack
Heritrix HaosouSpider
Humanlinks Harvest
HybridBot IDBTE4M
Iblog IRLbot
Id-search Id-search
IlseBot
Image Sucker
IndeedBot IndeedBot
Indy Library Indy Library
InfoTekies InfoTekies
InterGET Intelliseek
Iria
JamesBOT JamesBOT
Jbrofuzz Jbrofuzz
Jetty Jetty
Jorgee JikeSpider
Joomla
JustView JustView
Jyxobot Jyxobot
Keyword Density LexiBot
LNSpiderguy
Lftp
Libwhisker
LieBaoFast
Lightspeedsystems
Likse Likse
LinkScan
LinkWalker LinkWalker
Linkdexbot
LinkextractorPro
LinkpadBot LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaScrapeBot
Lipperhey Litemage_walker
Lipperhey Spider
Lmspider Lmspider
MJ12bot MIDown tool
MQQBrowser MIIxpc
MSFrontPage Mag-Net
MTRobot
Magnet
Majestic-SEO Majestic-SEO
Majestic12
MarkMonitor
MarkWatch
Mass Downloader Mass Downloader
Masscan
Mata Hari
MauiBot
Mb2345Browser Mb2345Browser
Metauri Microsoft Data Access
Mister PiX
Moblie Safari Moblie Safari
Mojolicious Morfeus Fucking Scanner
Musobot Msrabot
NICErsPRO
NPbot NPbot
Nameprotect
Navroad Navroad
NearSite Nessus
Net Vampire
NetAnts NetAnts
NetLyzer NetLyzer
NetSpider NetSpider
Netcraft NetZIP
Nettrack NextGenSearchBot
Netvibes Nibbler
Nikto
Ninja
Nuclei Nuclei
Nutch OPPO A33
Octopus
Offline Explorer Offline Explorer
Offline Navigator
OnCrawl OnCrawl
OpenVAS
Openfind
Openvas
OrangeBot OrangeBot
OrangeSpider OrangeSpider
OutclicksBot OutclicksBot
PHPCrawl OutfoxBot
Page Analyzer PECL::HTTP
PageAnalyzer POE-Component-Client-HTTP
PageGrabber Pandalytics
PageScorer Panscient
Papa Foto Pavuk
Pump PeoplePal
Pi-Monster
Piepmatz
PleaseCrawl
Pockey
Psbot
PxBroker PxBroker
PyCurl Quick-Crawler
RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
RankingBot
RankingBot2
Re-re
RealDownload
Reaper Reaper
Ripper RedesScrapy
Rogerbot RepoMonkey
SBIder Scanbot
SEOkicks-Robot
SISTRIX
SMTBot
SalesIntelligent
Scrapy Scrapy
Screaming
ScreenerBot ScreenerBot
SemrushBot ScrepyBot
Searchestate
SearchmetricsBot
SeoSiteCheckup
Seomoz Seomoz
Shodan SiteExplorer
SiteLockSpider
SiteSnagger SiteSnagger
Sitebeam
Siteimprove Siteimprove
Sitevigil
SmartDownload SmartDownload
Snake Snake
Snapbot Snapbot
SocialRankIOBot
Sociscraper
Sogou web spider Sogou web spider
Sosospider
Spammen Spammen
Spbot
SputnikBot SputnikBot
Sqworm Sqworm
Steeler Sucker
Sucuri Sucuri
SuperBot SuperBot
SurveyBot Surfbot
Suzuran
Swiftbot Swiftbot
Szukacz T8Abot
T0PHackTeam
Teleport Teleport
TeleportPro
Telesphoreo Telesphoreo
Telesphorep Telesphorep
The Intraformant TheNomad
Thumbor
Toata Toata
Tracemyfile Trendiction
Trendictionbot Turnitin
Twice
URLy Warning URLy Warning
URLy.Warning
UnisterBot
Upflow Upflow
VB Project V-BOT
VidibleScraper Vacuum
VoidEYE VelenPublicWebCrawler
WASALive-Bot VeriCiteCrawler
WBSearchBot Virusdie
WWW-Collector-E WeSEE
WWW-Mechanize Web Collage
WWWOFFLE
Web Fetch Web Fetch
Web Pix Web Fuck
Web Sucker WebAuto
WebCopier WebBandit
WebFuck WebFetch
WebLeacher WebGo IS
WebSucker WebSucker
WebWhacker WebZIP
Webshag WebsiteExtractor
Website Quester Webster
WebsiteQuester Whatweb
Whack Wonderbot
Who.is Bot
Wotbox
Wprecon Wprecon
Xaldon WebSpider Xenu
Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zauba
Zermelo
Zeus
Zitebot Zitebot
ZumBot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt autoemailspider
backlink-check backlink-check
coccocbot-web coccocbot-web
cognitiveseo cognitiveseo
com.plumanalytics
crawler.feedback crawler.feedback
crawler4j demandbase-bot
eCatch domainsproject.org
evc-batch gopher
lwp-request instabid
lwp-trivial ips-agent
magpie-crawler muhstik-scan
netEstate NE Crawler netEstate NE Crawler
oBot oBot
pcBrowser pcBrowser
probe-image-size polaris version
ripz ripz
satoristudio.net satoristudio.net
seoscanners scan.lol
seostar
serpstatbot serpstatbot
sysscan sexsearcher
tAkeOut sitechecker.pro
sogouspider
trendiction.de trendiction.de
voyagerx.com
webmeup-crawler webmeup-crawler
zauba.io x09Mozilla
zgrab

286
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
80legs 80legs
ADmantX
AIBOT AIBOT
ASPSeek
Aboundex
Aboundexbot Aboundexbot
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot
AiHitBot AiHitBot
Alligator
AlphaBot AlphaBot
Anarchy Anarchy
Anarchy99 Anthill
Aspiegel Apexoo
Attach Asterias
BBBike AwarioRssBot
BDCbot BDCbot
BackDoorBot BLEXBot
BackStreet BackStreet
BackWeb BackWeb
BacklinkCrawler BacklinkCrawler
Bandit
BatchFTP BatchFTP
Blackboard BetaBot
Bigfoot
BlowFish BlowFish
BotALot Bolt
Brandprotect Buck
Brandwatch BunnySlippers
BuiltBotTough BuzzSumo
Bullseye CCBot
ChinaClaw CODE87
Chlooe Calculon
CensysInspect
Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot Cogentbot
Copier
CopyRightCheck CopyRightCheck
Copyscape
Cosmos Cosmos
Craftbot
CrazyWebCrawler CrazyWebCrawler
Crescent Curious
Custo CyotekWebCopy
DSearch DIIbot
DTS Agent DTS Agent
DataCha0s
DatabaseDriverMysqli
Demon Demon
Deusu Deusu
DigitalPebble Devil
Digincore
Dirbuster
Disco
Discobot Discobot
Dispatch DittoSpyder
DomCopBot DomainCrawler
DomainAppender EMail Siphon
Domains Project
Dotbot
Download Wonder
ECCP/1.0
EMail Wolf EMail Wolf
EasyDL EasyDL
Ebingbong Ebingbong
EroCrawler Ecxi
Evil Evil
ExtLinksBot Express WebPictures
ExtractorPro Extractor
Extreme Picture Finder EyeNetIE
Ezooms FDM
FemtosearchBot FemtosearchBot
Fimap Fimap
Firefox/7.0 Fuzz
Flunky FyberSpider
Foobot
Freeuploader
Fyrebot
Genieo Genieo
GetRight GetWeb
Getintent
Gigabot
Go!Zilla Go!Zilla
Grafula Go-Ahead-Got-It
GrapeshotCrawler Gotit
Haansoft GrabNet
HaosouSpider GrapeFX
HybridBot GridBot
HMView
IDBot
IRLbot IRLbot
Id-search
IlseBot IlseBot
Image Fetch InfoTekies
Image Sucker InterGET
Indy Library
InfoNaviRobot
Intelliseek
InternetSeer
Iria Iria
Jbrofuzz Jetty
JennyBot
JetCar
JikeSpider JikeSpider
Jyxobot Jyxobot
Kenjin Spider Kenjin Spider
Kinza Kozmosbot
Lanshanbot LNSpiderguy
Larbin LeechFTP
LeechGet LexiBot
LibWeb Lightspeedsystems
LinksManager Likse
LinqiaMetadataDownloaderBot LinkpadBot
MFC_Tear_Sample Lipperhey Spider
Ltx71
MIDown tool MIDown tool
MIIxpc MIIxpc
MQQBrowser MQQBrowser
MSFrontPage
MSIECrawler MSIECrawler
Mail.RU_Bot MTRobot
Majestic SEO Mag-Net
Majestic12 Magnet
Mata Hari
MauiBot MauiBot
Mb2345Browser Mediatoolkitbot
MicroMessenger MegaIndex.ru
Microsoft Data Access
Mister PiX Mister PiX
Moblie Safari
Mojeek
Morfeus Fucking Scanner
Mr.4x3 Mr.4x3
Msrabot Name Intelligence
NPbot Net Vampire
Navroad NetLyzer
NetAnts NetSpider
NetZIP
Nettrack Nettrack
Nibbler Nibbler
Niki-bot Nikto
NimbleCrawler NimbleCrawler
Nimbostratus
Ninja Ninja
Nuclei Octopus
Nutch
Offline Explorer Offline Explorer
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler OpenLinkProfiler
Openvas Openvas
PHPCrawl OrangeSpider
PageGrabber PECL::HTTP
Page Analyzer
PageScorer PageScorer
Pandalytics PageThing.com
Panscient
PeoplePal
Petalbot
Pi-Monster Pi-Monster
Picsearch
Piepmatz
Pimonster Pimonster
Pixray Pixray
PleaseCrawl PleaseCrawl
Pockey
ProPowerBot
Probethenet Probethenet
Pu_iN Psbot
PyCurl PyCurl
QueryN Metasearch
Quick-Crawler
RSSingBot
RankActiveLinkBot
RankFlex RankFlex
RankingBot RankurBot
RankingBot2
Rankivabot
Reaper Reaper
RedesScrapy Recorder
RepoMonkey RepoMonkey
RocketCrawler RocketCrawler
SEOkicks
SEOprofiler
SEOstats
SalesIntelligent SalesIntelligent
ScanAlert Scanbot
ScoutJet
Scrapy
Screaming Screaming
ScreenerBot ScrepyBot
SearchmetricsBot SearchmetricsBot
SemrushBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Shodan Shodan
Siphon
Site Sucker Site Sucker
Siteimprove
Sitevigil Sitevigil
Snapbot SlySearch
Snake
Sociscraper Sociscraper
SpaceBison
Spammen
SpankBot SpankBot
Spbot SputnikBot
Spinn3r
Sqlworm Sqlworm
Sqworm Sqworm
Stripper SuperBot
SuperHTTP
Suzuran Suzuran
Teleport Teleport
TeleportPro
Telesphorep
The Intraformant The Intraformant
Thumbor TheNomad
Titan TightTwatBot
Toata
Toweyabot
Trendictionbot Trendictionbot
True_Robot
Turingos Turingos
Turnitin TwengaBot
TurnitinBot
Twice Twice
URLy.Warning Typhoeus
UnisterBot
Upflow
V-BOT V-BOT
Vacuum Vacuum
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VoidEYE VidibleScraper
Voil Virusdie
WISENutbot WEBDAV
WPScan
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWW::Mechanize
Web Auto Web Collage
Web Enhancer
Web Fetch Web Fetch
WebCollage WebAuto
WebCopier WebCopier
WebEnhancer
WebFetch
WebFuck
WebImageCollector WebImageCollector
WebStripper WebSauger
WebWhacker
WebZIP WebZIP
Website Quester
WebsiteExtractor WebsiteExtractor
Whack WebsiteQuester
Webster
Whacker Whacker
Wonderbot Widow
WinHTTrack
WiseGuys Robot
Wprecon Wprecon
Xaldon WebSpider
Xenu
Zade
Zermelo
Zeus
Zitebot Zitebot
ZumBot ZmEu
backlink-check ZoomBot
ZoominfoBot
check1.exe check1.exe
coccocbot-web
cognitiveseo cognitiveseo
crawl.sogou.com crawl.sogou.com
domainsproject.org crawler4j
eCatch eCatch
evc-batch evc-batch
instabid instabid
internetVista monitor internetVista monitor
ips-agent isitwp.com
magpie-crawler lwp-request
mediawords meanpathbot
netEstate NE Crawler oBot
page scorer
pcBrowser pcBrowser
plumanalytics plumanalytics
polaris version polaris version
satoristudio.net ripz
scalaj-http
seobility seobility
seostar seostar
siteripz serpstatbot
sogouspider
sp_auditbot sp_auditbot
spyfu
sysscan
tAkeOut tAkeOut
trendiction.de trendiction.com
webmeup-crawler x22Mozilla
x09Mozilla
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
7ooo.ru adtech.fr
demian.kz conboy.us
diamond-necklace.info dekoration.us
distver.ru freegamesplay.online
eluxer.net landscapebackgrounds.blogspot.com
hd-filmy.net offtime.ru
iminent.com onlyforemont.ru
privatefx.all4invest.info rightenergysolutions.com.au
saitevpatorii.com svnuppsalaorebro.se
spaceshipad.com videtubs.pl

924
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
123kuma.com 03p.info
4kpics.rocks 1-99seo.com
4rent.ru 1pamm.ru
4senses.co 24x7-server-support.site
4webmasters.org
5-steps-to-start-business.com 5-steps-to-start-business.com
5000-cotydzien.com 51unlim.ru
5kstore.com 7a2rnuey1tw9ar.ru
7fon.ru a96527gi.beget.tech
8558l.jobs.net abctoppictures.net
acc.eu.org acortarurl.es
activecampaign.dreamhosters.com actulite.com
adcash.com adsfresh.men
adclickservice.com adultactioncam.com
addbin.men advertex.info
adminshop.com afmuseum.com
adviceforum.info agreda.pluto.ro
advokateg.ru ahmedabadwebs.com
affrh2015.com alienwheels.de
afftrack001.com all-streaming-media.com
agysya.ru alt-servis.ru
alert-fjg.xyz androids-store.com
alfa-medosmotr.ru animal-drawings.com
alfa9.com animalia-life.club
alibestsale.com anjalika.co.in
aliexpresscashback.club anti-virus-removal.info
allkrim.com aosexkontakte.net
am-se.com apartment.ru
ameblo.top apifasterlightin-a.akamaihd.net
amung.us apimountainbikei-a.akamaihd.net
anatomiy.com apioasisspacenet-a.akamaihd.net
android-vsem.org apisecretsaucebi-a.akamaihd.net
animal36.com apple.com-webbrowsing-security.review
animaltoplist.com apple.com-webbrowsing-security.science
anticrawler.org arabseyes.com
aquarium-pro.ru as5000.com
ardimobileinfo.ml asdfg.pro
arenda-avtoprokat-krasnodar.ru asmxsatadriverin.aircus.com
artavenuegardenstudios.com asrv-a.akamoihd.net
asia-forum.ru asscenihotosidea.blogspot.co.za
asiengirls.net atout-energie-69.com
astana.xxxkaz.org atovh.local-finders.com
atlant-auto.info auto-moto-elektronika.cz
autolombard-krasnodar.ru
automobile-spec.com
autovideobroadcast.com autovideobroadcast.com
availit.weebly.com autowebmarket.com.ua
avtocredit-legko.ru aviav.eu
avtolombard-krasnodar.ru avtovolop.ru
azadnegar.com awaybird.ru
azartmsl.com ayakino.net
b00kmarks.com baltgem.com
bablonow.ru beauty-b0x.pl
bagcionderlab.com berlininsl.com
bankcrediti.ru bestwebsitesawards.com
bannerads.de
beta.hotkeys.com beta.hotkeys.com
big-cash.net bjgugu.net.ua
biglistofwebsites.com bloke.com
bio-market.kz bmsco.co
bio-optomarket.ru bonanza-fortune.men
bio.trade-jp.net
birzha-truda.eu
bjetjt.com
blackwitchcraft.ru
blog-hits.com
bloggedporn.com
bluerobot.info
bombla.org
bonkers.name
boo-arts.com boo-arts.com
brandedleadgeneration.com brendbutik.ru
brandimensions.com brianhenry.co
briomotor.co buchananshardware.com
bus-offer.com businesxxl.com
buzz-porno.info buyessayonline19.blogspot.ru
bvlgaribracelet.xyz cactussoft.biz
byme.se cam-kontakte.org
c1.onedmp.com candycrushshop.com
cablecar.us canoncdriverq3.pen.io
candelluxsklep.pl cartierring.xyz
candlehandmade.com casablancamanor.co.za
carsdined.org casinopinup-wins.com
cartierlove.xyz cbox.ws
cartierlovebracelet.xyz centrdebut.ru
cartierlovestore.com certifywebsite.win
cdncash.org cheapmoncler.xyz
cementaresearch.se
cenoval.ru
cezartabac.ro
chatroulette.life
chatseo.com
chelyabinsk.xrus.org chelyabinsk.xrus.org
chinacheapelitejerseys.com
chinajerseyswholesalecoupons.com chinajerseyswholesalecoupons.com
chlooe.com chiptuninger.com
christianlouboutinshoes.xyz chocolateslim-u-srbiji.com
cialisovercounteratwalmartusa.com christianlouboutinreplica.pw
ciekawinki.pl
cjmc.info
clayvasedesigns.tk clayvasedesigns.tk
clean-start.net coin-hive.com
cleanersoft.com com-supportcenter.website
clicksor.net cometorussia.net
clipartnew.com comissionka.net
club-musics.ru
coaching-netz.info
codq.info
companies-catalog.com companies-catalog.com
compliance-alex.xyz compliance-brian.xyz
compliance-alexa.xyz compliance-checker-7.info
compliance-don.xyz compliance-george.top
compliance-irvin.top consorzioilmosaico.org
compliance-john.top cordstrap.cc
compliance-viktor.top craftburg.ru
compucelunlock.net creditnation.ru
coolbar.pro cyclobenzaprinevstizanidine.blogspot.com
copenergo.ru dafatiri.com
courtshipgift.com ddsoldes.fr
crest3d.ru dealitright.click
cyse.tk decoratinghomes.ga
czat.wp.pl demon-tweeks.com
dating-app.ru detalizaciya-tut.biz
deadlinkchecker.com devochki.top
dentfidemountpreach.tk directivepub.com
deplim.com djonwatch.ru
derevesendeco.com
deutschland123.de
dev.citetick.com
dfwu1013.info
diegolopezcastan.com
divatraffic.com
dlya-android.org dlya-android.org
dojki-hd.com dnsrsearch.com
download-of-the-warez.blogspot.com domain2008.com
downloaddy.net domaincheck.io
downloadeer.net doreenblog.online
downloadkakaotalk.com dprtb.com
downloadmefiranaratb1972.xpg.com.br dptaughtme.com
draniki.org drunkenstepfather.com
dustyorate.com easycommerce.cf
dynainbox.com easync.io
edudocs.net easyshoppermac.com
eduserver.net easyukraine.com
eldiariodeguadalajara.com econom.co
elektrozigaretten1.tumblr.com electricwheelchairsarea.com
elektrischezigarettekaufen2.cowblog.fr
elektronischezigarettekaufen1.beeplog.com
elektrozigarette1.dreamwidth.org
elmacho.xyz elmacho.xyz
elmifarhangi.com eng-lyrics.com
eloconcream.blogspot.com
encodable.com
energydiet24.ru
englishgamer.com
enthuse.computernetworksonline.com enthuse.computernetworksonline.com
eredijovon.com eonpal.com
eropornosex.ru ero-advertising.com
es5.com erotikstories.ru
essay-writing.work erotische-geschichten-xxl.com
este-line.com.ua etotupo.ru
everypony.ru eupornstar.info
exchangeit.gq evehemming.blogspot.com.au
event-tracking.com
exoclick.com exoclick.com
ezigarettekaufen1.hpage.com extlabs.io
fashion.stellaconstance.co extremal-blog.com
ferretsoft.com extremepornos.net
financetip.eu eyelike.com.ua
finemanteam.com ezigaretteshop.webs.com
fitness-video.net f07.de
flytourisme.org f1nder.org
fm-upgrade.ru facebook-mobile.xyz
fok.nl failingmarriege.blogspot.com
forensicpsychiatry.ru fanoboi.com
forpostlock.ru fashionavenuegame.com
fortwosmartcar.pw fastfixing.tech
forum20.smailik.org filmi-onlain.info
fr-bearings.ru find1friend.com
franecki.net fodelsedagspresenter.nu
free-stock-illustration.com frankofficial.ru
freegamesplay.online freenode.info
freewareseek.com freeuploader.ml
freshmac.space frivgame250.com
freza-sverlo.ru fusoradio.info
fun2cell.net gambarkatabaru.com
g.starmoe.xyz gamedayassist.com
galeria-zdjec.com gamerextra.com
gallictures.com games.kolossale.ru
game-top.su
gamesprite.me
gamezblox.com
gate5.co.za
gaytube.com gaytube.com
gembird.com generic-pills-online.com
geoads.com geoads.com
getaclueamerica.com getmiro.com
getpopunder.com gheus.altervista.org
goldpanningtools.com ghostvisitor.com
golmau.host.sk globalsurfari.com
gonextmedia.com
goodhumor24.com goodhumor24.com
googlsucks.com gopixdatabase.com
gosreg.amchs.ru gosmeb.ru
gox.com.ua graphicwe.org
gribokstop.com greatgrace.ru
growboxbl.ru gtopstats.com
grupografico-pilar.com.ar hawaiisurf.com
gsmlab.pl healthcarestore.info
gta-top.ru highstairs-a.akamaihd.net
guod.me hmywwogw.bloger.index.hr
haikuware.com home-task.com
hamptonoaks.ca homedo.fabpage.com
hao123.com hoporno.com
hdimagegallery.net hotkeys.com
hebr.myddns-flir.com hotsocialz.com
hexpilot.com hoverboard360.nl
historichometeam.com howmuchdoestizanidinecost.blogspot.com
hledejvshopech.cz hptwaakw.blog.fc2.com
hol.es hully.altervista.org
homeinns.com iboss.com
hotcore.info id-forex.com
hoterika.com ido3.com
howlongdoestizanidinestayinyoursystem.blogspot.com ilikevitaly.com
hoztorg-opt.ru imgarcade.com
idealtits.net imk.com.ua
ilovevitaly.xyz impresagaia.it
imagecoolpub.com infokonkurs.ru
images-free.net insta-add.pro
imagui.eu intellekt21.ru
imicrovision.com ipornox.xxx
indiasourcemart.in irkutsk.zrus.org
infoupdate.org istizanidinelikexanax.blogspot.com
innoslicon.com iwantmyfreecash.com
istizanidineanarcoticdrug.blogspot.com jackpotchances.com
it-max.com.ua janavibekken.no
jav-fetish.com japan-bearings.ru
jerseywholesalechinabiz.com javcoast.com
jimmychoosale.online jewelryandfiligree.com
junglenet-a.akamaihd.net jikoman.info
kashubadesign.ru jjbabskoe.ru
kellyonline.xyz jrpmakati.com
ketoanhanoi.info juliadiets.com
kichenaid.ru kakablog.net
kino-key.info karusel-market.ru
kino2018.cc kazan.xrus.org
kinofak.net kazan.zrus.org
keki.info
keywordteam.net
kino-ecran.ru
kino-filmi.com
kiprinform.com kiprinform.com
kongoultry.net kit-opt.ru
kopihijau.info kliksaya.com
kovrenok.ru koronirealestate.gr
krafte.ru krasivoe-hd.com
krasivoe-hd.net krasivye-devushki.net
krynica.info krasula.pp.ua
kumuk.info ktotut.net
lapolis.it kupit-adenu.ru
laptoper.net kvartir-remont.biz
law-check-eight.xyz labelwater.se
lerporn.info lacapilla.info
lexixxx.com lakomka.com.ua
life-instyle.com law-check-nine.xyz
linkredirect.biz layola.biz.tc
liran-locks.com lidoradio.com
listiseltemournlan.gq liquimondo.com
loftdigital.eu littleberry.ru
longgreen.info littlesunraiser.com
lovelycraftyhome.com livingroomdecor.info
ltvperf.com ljusihus.se
lubetube.com louboutinreplica.pw
m-google.xyz louboutinreplica.xyz
macfix.life luckyshop.net.ua
magda-gadalka.ru machicon-ueno.info
magicdiet.gq mackeeper-land-672695126.us-east-1.elb.amazonaws.com
maridan.com.ua mahnwachen-helfen.info
masthopehomes.com makenahartlin.com
masturbate.co.uk maladot.com
mature.free-websites.com marmitaco.cat
mebeldekor.com.ua marmotstore.online
medi-fitt.hu mb140.ru
medicineseasybuy.com mecash.ru
medkritika.ru medispainstitute.com.au
medmajor.ru metaxalonevstizanidine.blogspot.com
megahdporno.net mhtr.be
megakino.net millionare.com
merryhouse.co.uk minharevisao.com
metascephe.com mirmedinfo.ru
mexicotravelnet.com mjchamonix.org
mhi-systems.ru mlf.hordo.win
michaelkorsoutletstore.net mobioffertrck.com
miniads.ca monclerjacketsoutlet.win
mir-limuzinov.ru monetizer.com-01.site
mobsfun.net monitorwebsitespeed.com
monclervests.xyz monsterdivx.tv
moneyteam24.com monthlywinners.com
more-letom.ru moroccosurfadventures.com
moviezbonkerssk.cf movpod.in
mrbitsandbytes.com mowser.com
msk-diplomat.com mrvideospornogratis.xxx
muz-baza.net mttwtrack.com
my-floor.in.ua musicspire.online
mwtpludn.review
mycouponizemac.com mycouponizemac.com
mydownloadengine.com myfreetutorials.com
mylida.org mykings.pw
myonigroup.com nalogovyy-kodeks.ru
na15.ru namenectar.com
nacap.ru nash-krym.info
narutonaruto.ru naturtreenspicerx.pw
nastydollars.com niki-mlt.ru
nfljerseys.online nl.netlog.com
nordvpn.com nomuos.it
novosti-hi-tech.ru nootrino.com
nsatc.net nowtorrents.com
ntic.fr nudo.ca
nucia.biz.ly nufaq.com
o-o-8-o-o.com nullrefer.com
oda.as o-o-6-o-o.com
obiavo.net
of-ireland.info of-ireland.info
offer.wpsecurity.website oivcvx.website
ohmyrings.com oktube.ru
ok-ua.info omgtnc.com
okmedia.sk
oksrv.com
omsk.xrus.org omsk.xrus.org
onemantrip.com online-podarki.com
onlineserialy.ru online7777.com
onlyforemont.ru onlinemegax.com
ooo-gotovie.ru optitrade24.com
open-odyssey.org orsonet.ru
openmediasoft.com ostroike.org
osoznanie-narkotikam.net outrageousdeal-a.akamaihd.net
oyster-green.com ozas.net
paparazzistudios.com.au panchro.xyz
pechikamini.ru papasdelivery.ru
petitions.whitehouse.gov pdns.cz
peugeot-club.org pensplan4u.com
photochki.com pewit.pw
photokitchendesign.com phrcialiled.com
phuketscreen.com phuketscreen.com
picphotos.net
pictures-and-images.net
pigrafix.at
pinapchik.com pinapchik.com
piski.top pizza-tycoon.com
pivka.xyz pladform.ru
pixell.club pogodnyyeavarii.gq
pizdopletka.club poisk-zakona.ru
poiskzakona.ru popmarker.com
pokerniydom.ru
polcin.de
poligon.com
polska-poezja.com
poptool.net
popup-hgd.xyz
pornokajf.com pornokajf.com
pornoslive.net portadd.men
potoideas.us postclass.com
pozvonim.com potolokelekor.ru
princevc.com pozdravleniya-c.ru
privatbank46.ru pozdrawleniya.ru
privat-girl.net
prizesbook.online prizesbook.online
profolan.pl professionalwritingservices15.blogspot.ru
pronorm.fr projefrio.com.br
prostitutki-kiev.org pron.pro
psa48.ru prostitutki-belgoroda.org
pts163.ru prostitutki-novgoroda.org
pxhdwsm.com prostitutki-tolyatti.org
quickchange.cc purplesphere.in
quit-smoking.ga qualitymarketzone.com
ranksonic.info quizzitch.net
rasteniya-vs-zombi.ru qwertty.net
rating-casino2021.ru radiodigital.co
redirect.trafficreceiver.club rednise.com
refererx.com refererx.com
registrationdomainsite.com remont-komputerov-notebook.ru
renecaovillasale.online reyel1985.webnode.fr
rfd-split.hr rialp.getenjoyment.net
ring4rhino.com ring4rhino.com
rique.host.sk ringporno.com
royal-betting.net rostov.xrus.org
royalads.net rubbed.us
ru-dety.ru s-forum.biz
russkie-gorki.ru saitevpatorii.com
saddiechoua.com samara.rosfirm.ru
sady-urala.ru santasgift.ml
scanmyphones.com savingsslider-a.akamaihd.net
scanner-donald.top sbricur.com
scanner-julia.top scanner-george.top
screpy.com scanner-irvin.top
search.1and1.com scrapy.org
searchlock.com scripted.com
seeresultshub-a.akamaihd.net seansonline24.pl
sel-hoz.com searchengineranker.email
sergiorossistore.online searchtooknow-a.akamaihd.net
searchwebknow-a.akamaihd.net
secretscook.ru
sei80.com
serpstat.com serpstat.com
servisural.ru service-core.ru
sevendays.com.ua sevendays.com.ua
sex-tracker.de sex-dating.co
sexpartygirls.net sexkontakteao.info
ship-marvel.co.ua sexreliz.com
shoplvlv.us sexs-foto.com
shtora66.ru sextracker.be
simon3.ru sfj-ror.no
sims-sims.ru shakhtar-doneck.ru
site3.free-share-buttons.com shtaketniki.ru
siteaero.com similardeals.net
siteripz.net site-speed-check.site
sitevaluation.com siteheart.net
sixcooler.de sizeplus.work
smart-scripts.com skladvaz.ru
snyatie-lomki-v-stacionare.ru sky-mine.ru
sobecjvuwa.com.ru slomm.ru
softlinesolutions.me slutloadlive.com
sneakyboy.com
social-vestnik.ru
socialsignals24.com
softtor.com
songplanet.ru songplanet.ru
spamnuker.com sotkal.lark.ru
spasswelt.xyz space-worry.ml
srgwebmail.nl space4updating.win
spbchampionat.ru
sport-video-obzor.ru
srdrvp.com
sstroy44.ru
starpages.net
stathat.com
stats-public.grammarly.io stats-public.grammarly.io
steelmaster.lv stpolice.com
stillmiracle.com strv.se
styro.ru superfish.com
supervesti.ru surfbuyermac.com
svensk-poesi.com sweepstakes.rewardit.com
taraz.xkaz.org szucs.ru
teenbbw.yopoint.in tattooreligion.ru
teenforporn.com taxi-v-eisk.ru
textads.men tedxrj.com
teletype.in
theallgirlarcade.com theallgirlarcade.com
thin.me.pn thebluffs.com
thruport.com thecoral.com.br
ticketsys.inetwd.com thegalerie.eu
tilido.com timeallnews.ru
time-japan.ru timer4web.com
tizanidinedruginteractions.blogspot.com tizanidinedosage.blogspot.com
tizanidinepills.blogspot.com tizanidineforopiatewithdrawal.blogspot.com
tizanidinevscyclobenzaprine.blogspot.com tizanidinehcl2mg.blogspot.com
top-study.work tmtrck.com
topkarkas.com tomatis.gospartner.com
torrentgamer.net toon-families.com
torrentred.games toondinsey.com
touchmods.fr tooplay.com
tourismvictoria.com top10-online-games.com
track2.shop torrentdownloadhub.com
trafficgenius.xyz torture.ml
trafficmania.com toxicwap.com
trailer.cinemaflix.website toys.erolove.in
tracxn.com
traffic2money.com
trafficmonetize.org
tsyndicate.com tsyndicate.com
turvgori.ru ugogo.info
tvorozhnaja-zapekanka-recept.ru
ucsol.ru
udayavani.com
uhod-za-sobakoj.ru
uk-zheu20.ru uk-zheu20.ru
ultimateclassicrock.com ukkelberg.no
umekana.ru ukr-lit.com
unitexindia.com ultimatesetnewfreeallsoftupgradesystems.pw
unlimitdocs.net unionmarkt.de
uprour.com uogonline.com
urzedowski.eu uptime-as.net
vavilone.com urengoy.pro
vedomstvo.net urll.eu
videojam.tv utrolive.ru
vigrx-original.ru velen.io
virtuagirl.com velobikestock.com
visa-pasport.ru viagengrarx.com
viteonlusarezzo.it viagra-soft.ru
vmnmvzsmn.over-blog.com viddyoze.com
voprosotvet24.ru video-chat.cn
voronezh.xrus.org videochat.life
vse-pesni.com videositename.com
vseigru.one vip-parfumeria.ru
vtcdns.com vksaver-all.ru
vucms.com vksex.ru
vzube.com vltai.com
web-analytics.date volume-pills.biz
wechatdownload10.com voluumtracker1.com
wetgames.ru wallpaperaccess.com
waterefficiency.co
webstatsdomain.org
welcomeauto.ru
westen-v.life
westen-z.life
whengirlsgowild.com whengirlsgowild.com
whereiskentoday.com wikes.20fr.com
wholesalejerseysgaa.com wnhjavlhezp.gq
winner7777.net woman-orgasm.ru
wladimirpayen.com word-vorlagen.net
woman-tampon.ru works.if.ua
wrona.it wtsindia.in
ww1943.ru wufak.com
wwwadultcheck.com www.event-tracking.com
x-lime.net www.pinnacle-bets.com
wzgyyq.com
x-musics.com
x-porno.video x-porno.video
xcc24.pl xcombear.ru
xchangetrak.com
xev.ru xev.ru
xjlottery.com xgftnlrt.bloger.index.hr
xjrul.com xlovecam.com
xn----7sbho2agebbhlivy.xn--p1ai xmlinde.com
xn----7sbifcamovvfggw9d.xn--p1ai xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf xn--80atua3d.xn--p1ai
xn--d1abj0abs9d.in.ua xnxx699.com
xn--h1ahbi.com.ua xxxguitars.com
xn--l1aengat.xn--p1ai yaaknaa.info
xportvusbdriver8i.snack.ws yaminecraft.ru
xtube.com yellowfootprints.com
xxxmania.top yougame.biz
xz618.com zaimite.ru
yougotanewdomain.com zapatosenventa.info
youporn-ru.com zastenchivosti.net
yourporn.com
zaapplesales.blogspot.com
zarepta.com
zastroyka.org zastroyka.org
zdesoboi.com
zeleznobeton.ru
zeroredirect12.com zeroredirect12.com
zeroredirect2.com zeroredirect7.com
zigarettenonlinekaufen1.blox.pl zionstar.net
zixizop.net.ru zixizop.net.ru
znakomstva-piter78.ru
zolotoy-lis.ru
zoolubimets.ru
zrizvtrnpale.tk
zverokruh-shop.cz

490
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01apple.com 1webmaster.ml
11pikachu.ru 24-ak.ru
1adult.com 2nt.ru
777-club.ru 4istoshop.com
7wind.ru 7zap.com
8558l.jobs.net a.pr-cy.ru
8jn.dating-app.ru adcash.com
aa8b68101d388c446389283820863176e7.com adclickthru.net
adman.gr addray.pro
adminshop.com adpremium.org
adtiger.tk adrenalin-stk.ru
adzerg.com affiliate-fr.com
agreda.pluto.ro agecheckadult.com
aihelen.net alert-fjg.xyz
alfabot.xyz all-streaming-media.com
alldezservice.kz allproblog.com
alldubai.biz aludecor.info
allforminecraft.ru am-se.com
allwidewallpapers.com anatomiy.com
amateurlivechat.org android4fun.org
anal-acrobats.hol.es apartment.ru
anjalika.co.in apiallgeniusinfo-a.akamaihd.net
api.stathat.com apifasterlightin-a.akamaihd.net
arpelsreplica.xyz asrv-a.akamaihd.net
asrvvv-a.akamaihd.net autodo.info
autobrennero.it autoloans.com
avramstroy.ru aviav.ru.com
baersaratov.ru avkzarabotok.com
balois.worldbreak.com azinoofficial777.ru
bannerswap.com azlex.uz
basisches-wasser.net babs.com.ua
battle.net balla-boo.se
bestbuy.ca begalka.xyz
book-bets.com beget.tech
bradleylive.xyz besttorrentknifta.weebly.com
brateg.xyz betteroffers.review
browsepulse-a.akamaihd.net bird1.ru
brus-vsem.ru bloggers.nl
bscodecs.com boobsimge.com
buqyxa.rincian.info bookhome.info
buy-loft.ru c1.onedmp.com
buypuppies.ca celejihad.info
cakemediahq.com.au cenoval.ru
cartujano-pre.de centrdebut.ru
cementaresearch.se chlooe.com
choice-credit.ru chocolateslim-u-srbiji.com
clevernt.com choosecuisine.com
co.lumb.co cialischstgerts.com
coin-hive.com cinemaenergy-hd.ru
com-secure.download click2pawn.com
cometorussia.net club-musics.ru
cookie-law-enforcement-hh.xyz computernetworksonline.com
crynet.cc comsysnet.com
crystalslot.com corta.co
damedingel.ya.ru cscwtalkto.site
dandiyabeats.in custom-product-labels.com
dangphoto.trade cutt.us
dayibiao.com dailyfinancefix.com
dfwu1013.info db.speedup.gdn
dgfitness.co ddsoldes.fr
discovertreasure-a.akamaihd.net detmebel.su
domain-tracker.com diamond-necklace.info
domainsphoto.com dmmspy.com
droidlook.net dojki365.online
drunkmoms.net domain-submit.info
dscaas.website domaineaneblanc.com
earncash.com.ua doyouknowtheword-flummox.ml
elektrischezigarette2.onsugar.com druzhbany.ru
eng-lyrics.com e705.net
elitesportsadvisor.com
englishdictionaryfree.com
enskedesquashclub.se
epngo.bz epngo.bz
este-line.com.ua exdocsfiles.com
everytalk.tv ezigarettenkaufen2.dreamwidth.org
evogarage.com farmprofi.net
executehosting.com fbdownloader.com
exonline.info fickkontaktehobbyhuren.com
fable.in.ua floating-share-buttons.com
fastcrawl.com foto-basa.com
feargames.ru froggytube.com
fidelityfunding.com fungirlsgames.net
financetip.eu gallerylisting.com
findacheaplawyers.com games.kolossale.ru
finuse.com gdzkurokam.ru
firesub.pl generalporn.org
fitness-video.net getpopunder.com
floridamobilebillboards.com gq-catalog.gq
forensicpsychiatry.ru green-tea.tv
formulaf1results.blogspot.com gsbs.com.ua
funnypica.com hacktougroup.ru
fxund.us hamacapty.com
gameplexcity.com hccoder.info
ge0ip.com heartofbeijing.blogspot.com
goblacked.com hesteel.pl
gratis-sexkontakte.com hjaoopoa.top
gsasearchengineranker.space home.myplaycity.com
gsasearchengineranker.top hoverboardforsaledirect.com
guitar-master.org hptwaakw.blog.fc2.com
happy.new.yeartwit.com ifirestarter.ru
harmonyglen.us ilovevitaly.com
heroz.fr imitex-plus.ru
hidemyass.com inet-traffic.com
highspeed5.net infodocsportal.com
imagui.eu internetartfair.com
imfamous.info jerseywholesalebizchina.com
img.wallpaperstock.net jikoman.info
imicrovision.com junketjuice.blogspot.com
incanto.in.ua jyrxd.com
intellego.info kchaxton.com
interior-stickers.ru kimcurlrvsms.com
internetproviderstucson.com kinoduh.ru
iplusbit.blogspot.co.za kolotiloff.ru
iusstf.org koptims.tiu.ru
jofucipiku.tk kw21.org
jrpmakati.com lescinq.com
kakadu-interior.com.ua lidoradio.com
karusel-market.ru lineavabit.it
kazrent.com lockerz.com
keywordsdoctor.com mahnwachen-helfen.info
kinobaks.com mapquestz.us
kinoplen.ru masterseek.com
kinopolet.net matsdale.com
kostenloser-sex.com mcadamssupplyco.com
kwzf.net militarysale.pro
laapp.com mnogolok.info
laptop-4-less.com monetizer.com-01.site
lawyersinfo.org morocco-nomad-excursions.com
likesuccess.com moyakuhnia.ru
littleberry.ru music7s.me
locationdesaison.com my-bc.ru
lovelycraftyhome.com myshopmatemac.com
m-google.xyz national-today-winning-winner.club
m1media.net naturtreenspicerx.pw
macdamaged.space news-readers.ru
makedo.ru no-fuel.org
manifestyourmillion.com notasprensa.info
marmitaco.cat novosti-hi-tech.ru
martlinker.com nowtorrents.com
masthopehomes.com nudejapan.net
mebel-ekb.com obiavo.kz
medicines-choice.com obnal.org
menhealed.net okayimage.com
mensandals.xyz oralsexfilme.net
microsearch.ru ourville.info
missvietnam.org oxotl.com
moetomnenie.com panicatack.com
moinozhki.com parajumpersstore.online
moroccosurfadventures.com partner-print.men
morpicert.pw partner-stop.men
mowser.com pearlisland.ru
muz-shoes.ru percin.biz.ly
naphukete.ru poem-paying.gq
nastydollars.com positive2b.ru
nmrk.ru priceg.com
notebook-pro.ru prize44.com
one.net.in purplesphere.in
openmultipleurl.com pxhdwsm.com
ownshop.cf queerspace.com
owohho.com razvratnoe.org
parajumpersoutlet.online real-time-analytics.com
partnersafe.men redbottomheels.xyz
partnerworkroom.men rednise.com
pateaswing.com reklamuss.ru
pensplan4u.com relax.ru
picscout.com remont-mobile-phones.ru
pl.aasoldes.fr renecaovillasale.online
play.leadzupc.com rentalcarnavi.info
plusnetwork.com riralmolamsaca.tk
prodvigator.ua
progress-upakovka.ru
promgirldresses.xyz
questionmarque.ch
radiodigital.co
rarbg.to
rcb101.ru
relayblog.com
responsive-test.net
ric.info
robot-forex.biz
royal-betting.net royal-betting.net
rukino.org rutor.group
russintv.fr salmonfishingsacramentoriver.com
sabaapress.com samoiedo.it
sack.net sanjosestartups.com
savefrom.com sbricur.com
scanner-donald.top scenarii-1-sentyabrya.uroki.org.ua
searchaddis.com see-your-website-here.com
searchinterneat-a.akamaihd.net service.adtech.fr
seminarygeorgia59.ga sexkontakteao.info
serialsway.ucoz.ru shareyards.com
servisural.ru shariki-zuma-lines.ru
sexkvartal.com shiksabd.com
shoppytoolmac.com shopvilleroyboch.com.ua
sim-service.net sigmund-freud.co.uk
siteimprove.com silvermature.net
sitevaluation.org simpoed.ufop.br
sixcooler.de sisiynas.ru
sky-mine.ru site-speed-checker.site
slonechka.ru slowmacfaster.trade
smallseotools.com smartpet.ru
sogimlecal.tk smartshoppymac.com
spidtest.space soft1.ru
sstroy44.ru speedup-my.site
stal-rulon.ru spin2016.cf
steamoff.net sr-rekneskap.no
streha-metalko.si sribno.net
stroyhelp-dv.ru strangeduckfilms.com
thefotosgratis.eu stroilka.info
ticketsys.inetwd.com sugarkun.com
time-japan.ru takeflyte.com
titelhelden.eu tako3.com
tizanidinestreetvalue.blogspot.com tdsing.ru
testbotprocessor44.com
thebitcoincode.com
thebluenoodle.com
thefarmergame.com
thegameriders.com
thewebsitetemplate.info
thfox.com
titan-ads.life
tizanidineandcipro.blogspot.com
tizanidineandhydrocodone.blogspot.com
tizanidinehcl2mgtablet.blogspot.com
tizanidinewithdrawal.blogspot.com
toon-families.com
top-deal.com.pl
torture.ml
toys.erolove.in
trkdf.com
tryrating.com tryrating.com
tsan.net twiclub.in
tt-ipd.info twittruth.com
tvorozhnaja-zapekanka-recept.ru
uasb.ru
udayavani.com
ukkala.xyz ukkala.xyz
unimodemhalfduplefw.pen.io unblocksit.es
unrealcommander.org v720hd.ru
uvozdeckych.info vacances-voyages.info
vacuumscleaner.com vchulkah.net
validdomain.xyz ventelnos.com
valueclick.com via-energy-cumpara.com
viagengrarx.com videosbox.ru
virtualbb.com vsesubwaysurfers.com
voluumtrk.com vseuznaem.com
vucms.com walkme.com
watchinf.com we-are-gamers.com
waycash.net webadvance.club
webads.co.nz
webinstantservice.com webinstantservice.com
webmasters.stackexchange.com wholesalecheapjerseysfree.com
wesharepics.com www.solartek.ru
wordkeyhelper.com xitjw.info
wstroika.ru xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
www.bookmaker-bets.com xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
x-mix.info xvideosbay.com
xbaboon.com xxxhdvideo.site
xlovecam.com yaaknaa.info
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai yaminecraft.ru
xn----ctbigni3aj4h.xn--p1ai yourads.website
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai znakom.sibtest.ru
xn--h1ahbi.com.ua zoodrawings.com
xxxnatelefon.ru zrelaya.pw
xxxtubesafari.com zvooq.eu
youresponsive.com
zakon-ob-obrazovanii.ru
zapiszto.pl
zarabotok--doma.ru
zeleznobeton.ru

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

1248
dev-tools/tmp/robots.txt
View file

File diff suppressed because it is too large Load diff

1248
robots.txt/robots.txt
View file

File diff suppressed because it is too large Load diff