vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2691 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-03 10:09:26 +00:00
parent 0ed3949793
commit 66b69eb801
47 changed files with 68720 additions and 68710 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2690 #### Version: V4.2021.11.2691
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2690 #### Version: V4.2021.11.2691
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2690 #### Version: V4.2021.11.2691
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8454
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2690 #### Version: V4.2021.11.2691
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8458
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8458
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8458
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8458
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.0.131 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.0.131 0;
127.0.0.1 0; 127.0.0.1 0;

8458
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.0.131 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.0.131 0;
127.0.0.1 0; 127.0.0.1 0;

8458
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.0.131 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.0.131 0;
127.0.0.1 0; 127.0.0.1 0;

8458
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2690 2691

8458
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8458
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.120.110.158 0; 13.67.182.243 0;

8458
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.0.131 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.0.131 0;
127.0.0.1 0; 127.0.0.1 0;

8458
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.0.131 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.0.131 0;
127.0.0.1 0; 127.0.0.1 0;

8458
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.0.131 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.0.131 0;
127.0.0.1 0; 127.0.0.1 0;

8458
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.120.110.158 0; 13.67.182.243 0;

8458
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.120.110.158 0; 13.67.182.243 0;

8458
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.0.131 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
CheTeam ASPSeek
Crescent AiHitBot
Disco BackDoorBot
GridBot Chlooe
Mojeek Ninja
PxBroker PxBroker
Reaper Spammen
WebSauger T8Abot
WiseGuys Robot Who.is Bot
lwp-request facebookscraper

292
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
404enemy ADmantX
80legs Aboundex
ASPSeek
Abonti
Aboundexbot Aboundexbot
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot AhrefsBot
Alexibot Aipbot
AlphaBot
Anarchie Anarchie
Anarchy Anarchy
Anarchy99 Aspiegel
Anthill Asterias
Apexoo
AwarioSmartBot
BBBike BBBike
BDCbot BDFetch
BLEXBot
BackDoorBot BackDoorBot
BackWeb Barkrowler
Backlink-Ceck BetaBot
Bandit
BatchFTP
Bigfoot Bigfoot
Bitacle Bitacle
Black Hole BlowFish
Blackboard
Boardreader Boardreader
Bolt Brandwatch
BuiltBotTough Buck
Bullseye BuiltWith
BuzzSumo BuzzSumo
CCBot CCBot
CODE87 CODE87
CSHttp Calculon
CazoodleBot
CheTeam CheTeam
CheeseBot
CherryPicker
ChinaClaw ChinaClaw
Chlooe
Cloud mapping Cloud mapping
CopyRightCheck Copier
Copyscape
Cosmos
Craftbot Craftbot
Curious CrazyWebCrawler
CyotekWebCopy Crescent
DSearch CrunchBot
DataCha0s DBLBot
DatabaseDriverMysqli Deusu
Dispatch DigitalPebble
DittoSpyder Discobot
DnBCrawler-Analytics DnBCrawler-Analytics
DomainCrawler DnyzBot
DomainSigmaCrawler DomainAppender
Dotbot ECCP/1.0
Drip EMail Siphon
Exabot EMail Wolf
Extractor Ecxi
Evil
Express WebPictures
ExtLinksBot
ExtractorPro ExtractorPro
Extreme Picture Finder
Ezooms
FHscan
FemtosearchBot
Fimap Fimap
Foobot Firefox/7.0
Genieo Freeuploader
GermCrawler FrontPage
Fuzz
FyberSpider
G-i-g-a-b-o-t
GetRight
Getintent Getintent
Gigabot Go!Zilla
Gotit
GrabNet GrabNet
Grabber Grabber
GrapeshotCrawler HTMLparser
HEADMasterSEO HTTP::Lite
HMView
HTTrack HTTrack
Havij
Heritrix Heritrix
IDBot Hloader
Humanlinks
HybridBot
IDBTE4M
Iblog Iblog
IlseBot
Image Fetch Image Fetch
Image Sucker Image Sucker
IndeedBot IndeedBot
InfoTekies Indy Library
Intelliseek
InterGET
InternetSeer
Iria Iria
Iskanie Iskanie
IstellaBot JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz Jbrofuzz
JetCar JikeSpider
Jetty
Joomla Joomla
Jorgee Keyword Density
JustView
Jyxobot
LNSpiderguy LNSpiderguy
Leap LWP::Simple
LeechGet LeechGet
LibWeb LexiBot
Libwhisker LieBaoFast
LinkScan
Linkdexbot Linkdexbot
LinkextractorPro
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider
Lmspider
Ltx71 Ltx71
MFC_Tear_Sample MSFrontPage
MIIxpc
MJ12bot
MSIECrawler
MTRobot MTRobot
Mag-Net
Magnet Magnet
Majestic SEO Majestic SEO
Majestic-SEO Majestic12
Mass Downloader
MauiBot
Mb2345Browser
Mediatoolkitbot Mediatoolkitbot
MicroMessenger Microsoft Data Access
Morfeus Fucking Scanner Mojolicious
Mozlila Mozlila
NICErsPRO NICErsPRO
NPbot NPbot
Name Intelligence
Nameprotect
NearSite NearSite
Nessus
NetAnts NetAnts
NetMechanic Nettrack
Netvibes Netvibes
Nibbler Niki-bot
NimbleCrawler
Nmap Nmap
Nuclei
Nutch
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler
OpenVAS
Openfind Openfind
OrangeSpider PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageAnalyzer
PageGrabber PageGrabber
Pandalytics
Pavuk
PeoplePal PeoplePal
Petalbot
Pi-Monster Pi-Monster
PictureFinder PictureFinder
Piepmatz Piepmatz
ProWebWalker ProPowerBot
Probethenet Psbot
QueryN Metasearch Pump
Quick-Crawler PxBroker
RankingBot RSSingBot
RealDownload RankActiveLinkBot
RebelMouse RankFlex
Rankivabot
RankurBot
ReGet
Reaper
RedesScrapy RedesScrapy
RepoMonkey RepoMonkey
SBIder SBIder
SEOkicks SEOkicks-Robot
ScoutJet SEOlyticsCrawler
SEOprofiler
SalesIntelligent
ScanAlert
Searchestate Searchestate
Semrush Semrush
SemrushBot SeoSiteCheckup
SeobilityBot
Seomoz Seomoz
Shodan Siphon
Site Sucker
SiteCheckerBotCrawler
SiteSucker
SlySearch SlySearch
Snake Snake
SocialRankIOBot
Sottopop Sottopop
Spanner SpaceBison
SputnikBot Sqlmap
Sqworm Steeler
Sucuri SuperBot
SuperHTTP
SurveyBot SurveyBot
Swiftbot Suzuran
Szukacz Szukacz
T0PHackTeam T0PHackTeam
Teleport Teleport
Telesphorep
The Intraformant
TheNomad TheNomad
Thumbor Thumbor
TightTwatBot Titan
Toata
Toweyabot
Tracemyfile
Trendiction Trendiction
Trendictionbot Trendictionbot
Turingos Turingos
Turnitin
TwengaBot TwengaBot
UnisterBot URLy.Warning
Upflow Upflow
V-BOT
VB Project
VCI VCI
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
Virusdie Virusdie
Voil
Voltron
WEBDAV
WISENutbot
WPScan WPScan
WWW-Mechanize
WWWOFFLE
WallpapersHD WallpapersHD
WeSEE
Web Auto
Web Collage
Web Enhancer Web Enhancer
Web Fetch
Web Fuck Web Fuck
Web Pix
Web Sauger
WebAuto
WebBandit WebBandit
WebEnhancer WebCollage
WebFuck WebFuck
WebLeacher WebGo IS
WebPix WebReaper
WebSauger
WebSucker WebSucker
WebZIP Website Quester
Webalta
WebsiteExtractor WebsiteExtractor
Who.is Bot Whatweb
Widow Widow
Woobot WiseGuys Robot
Xenu Wonderbot
Wotbox
Zade Zade
Zermelo
Zeus Zeus
Zitebot ZmEu
ZoomBot
ZoominfoBot
archive.org_bot archive.org_bot
cah.io.community arquivo-web-crawler
coccocbot-web crawl.sogou.com
crawler4j crawler.feedback
demandbase-bot
eCatch eCatch
evc-batch facebookscraper
instabid
internetVista monitor internetVista monitor
isitwp.com isitwp.com
lwp-trivial magpie-crawler
mediawords meanpathbot
muhstik-scan muhstik-scan
oBot oBot
page scorer
plumanalytics
polaris version polaris version
s1z.ru
scan.lol scan.lol
serpstatbot seoscanners
seostar
sexsearcher
sitechecker.pro
siteripz siteripz
sogouspider
sysscan
tAkeOut tAkeOut
webmeup-crawler trendiction.de
webpros.com ubermetrics-technologies.com
x09Mozilla voyagerx.com
zgrab xpymep1.exe

302
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404checker
360Spider
404enemy 404enemy
Abonti ADmantX
ALittle Client
ASPSeek
Aboundexbot
Acunetix
AfD-Verbotsverfahren
AhrefsBot
AiHitBot AiHitBot
Alligator Alexibot
AlphaBot AlphaBot
Anarchie Aspiegel
AwarioSmartBot AwarioSmartBot
BackDoorBot BBBike
BDCbot
BLEXBot
BackWeb BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler
Badass
Barkrowler
BetaBot
Bitacle Bitacle
Black Hole
BlackWidow BlackWidow
Blow Blow
Bolt
BotALot
Buck
Buddy
BuiltBotTough BuiltBotTough
Bullseye Bullseye
BuzzSumo BuzzSumo
CODE87 CATExplorador
CSHttp CSHttp
CheTeam Calculon
CheeseBot CazoodleBot
Chlooe Cegbfeieh
CherryPicker
ChinaClaw
Claritybot Claritybot
Cocolyzebot Cocolyzebot
Copyscape
Cosmos Cosmos
Craftbot
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler
CrunchBot
Custo Custo
DTS Agent
DataCha0s DataCha0s
Demon Devil
Digincore DigitalPebble
Discoverybot Dirbuster
Disco
Dispatch
DnBCrawler-Analytics
DnyzBot DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler Domains Project
DomainStatsBot
Dotbot Dotbot
Dragonfly Dragonfly
EMail Siphon Drip
Ecxi ECCP/1.0
EroCrawler
Exabot
ExtLinksBot ExtLinksBot
ExtractorPro ExtractorPro
FDM Extreme Picture Finder
EyeNetIE
Flunky
Foobot
Freeuploader
Fuzz
FyberSpider FyberSpider
G-i-g-a-b-o-t
GT::WWW
GalaxyBot
Genieo Genieo
GetRight GetWeb
Go!Zilla Gigabot
Gotit Gotit
Grabber
GrapeFX GrapeFX
GrapeshotCrawler GridBot
HEADMasterSEO
HTMLparser HTMLparser
HTTrack
HaosouSpider HaosouSpider
Harvest
Heritrix Heritrix
Hloader Humanlinks
IRLbot HybridBot
IDBTE4M
Iblog
IlseBot IlseBot
Indy Library Image Fetch
InfoTekies
InterGET
Internet Ninja Internet Ninja
Iria JOC Web Spider
Iskanie Kozmosbot
IstellaBot
JamesBOT
Jetty
Joomla
Lanshanbot
LeechFTP
LeechGet LeechGet
LexiBot Lftp
LibWeb
Libwhisker
LieBaoFast LieBaoFast
Lightspeedsystems
Likse Likse
LinkScan LinkScan
LinkWalker LinkWalker
Linkbot LinkpadBot
LinksManager Lipperhey Spider
LinqiaMetadataDownloaderBot Lmspider
Ltx71
MIDown tool MIDown tool
MJ12bot
MSIECrawler MSIECrawler
MTRobot
Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic12 Majestic-SEO
MarkWatch MarkMonitor
Masscan Mata Hari
MauiBot MauiBot
MeanPath Bot MeanPath Bot
Meanpathbot
Mediatoolkitbot
MegaIndex.ru
MicroMessenger MicroMessenger
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Mister PiX
Mojolicious Mojolicious
Msrabot
Musobot Musobot
Name Intelligence Needle
Navroad NetAnts
Nessus
NetLyzer NetLyzer
NetSpider NetMechanic
NetZIP Nibbler
Nettrack Niki-bot
NextGenSearchBot
Nikto Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Nmap Ninja
Offline Explorer Offline Explorer
OnCrawl OpenLinkProfiler
OpenVAS Openfind
OrangeBot Openvas
OrangeSpider
OutfoxBot OutfoxBot
POE-Component-Client-HTTP PECL::HTTP
PageGrabber
PageScorer PageScorer
PageThing.com Pandalytics
Panscient
Pavuk Pavuk
Petalbot
Pi-Monster Pi-Monster
Picscout Picsearch
Piepmatz Piepmatz
Pimonster PleaseCrawl
Pixray
Pockey
ProPowerBot
ProWebWalker
Probethenet
Psbot
Pu_iN
Pump Pump
PyCurl QueryN Metasearch
Quick-Crawler RankActiveLinkBot
RSSingBot RankFlex
RankActive
RankingBot2 RankingBot2
Rankivabot Re-re
Reaper RealDownload
RepoMonkey Rogerbot
SEOkicks-Robot SBIder
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOstats
SISTRIX SISTRIX
ScoutJet ScanAlert
Scrapy Scanbot
Screaming Screaming
ScreenerBot ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
Semrush SeobilityBot
SeoSiteCheckup Siphon
SiteExplorer SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker SiteSucker
Siteimprove Sitebeam
Sitevigil SmartDownload
Snapbot Snapbot
SocialRankIOBot Sociscraper
Sogou web spider
SpaceBison SpaceBison
SpankBot SpankBot
Spinn3r
Sqlmap
Sqworm
Sucker Sucker
Sucuri Sucuri
Szukacz SuperBot
Surfbot
Suzuran
T0PHackTeam T0PHackTeam
Teleport T8Abot
TeleportPro Telesoft
Telesphoreo
Telesphorep Telesphorep
Trendiction The Intraformant
TheNomad
Thumbor
True_Robot
Turnitin
TwengaBot
Twice Twice
Typhoeus URLy Warning
Upflow UnisterBot
V-BOT V-BOT
VB Project
VCI VCI
VelenPublicWebCrawler Vagabondo
VeriCiteCrawler VidibleScraper
Virusdie
Voil
Voltron Voltron
WISENutbot WBSearchBot
WWW-Collector-E WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 WallpapersHD
Web Auto
Web Collage
Web Enhancer Web Enhancer
WebAuto Web Fetch
WebCollage Web Sauger
WebCopier WebCopier
WebEnhancer WebFetch
WebReaper WebGo IS
WebSauger WebStripper
WebWhacker WebWhacker
WebmasterWorldForumBot WebmasterWorldForumBot
Whack Website Quester
Whacker Whacker
Whatweb Who.is Bot
Widow Widow
WiseGuys Robot
Wonderbot Wonderbot
Wotbox Wprecon
Zitebot Xaldon_WebSpider
ZoominfoBot Xenu
archive.org_bot YoudaoBot
ZmEu
arquivo.pt
autoemailspider autoemailspider
cah.io.community backlink-check
check1.exe check1.exe
coccocbot-web clark-crawler
cognitiveseo com.plumanalytics
crawl.sogou.com crawl.sogou.com
dataforseo.com
demandbase-bot demandbase-bot
eCatch
evc-batch evc-batch
heritrix
instabid
internetVista monitor internetVista monitor
ips-agent ips-agent
lwp-trivial isitwp.com
magpie-crawler
meanpathbot
muhstik-scan
netEstate NE Crawler
probe-image-size
ripz ripz
s1z.ru s1z.ru
scan.lol scan.lol
seobility seoscanners
seostar seostar
sexsearcher spyfu
sitechecker.pro sysscan
siteripz
sp_auditbot
tAkeOut tAkeOut
ubermetrics-technologies.com trendiction.com
webprosbot voyagerx.com
webpros.com
x09Mozilla x09Mozilla
x22Mozilla
xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
aac500b7a15b2646968f6bd8c6305869d7.com adloads.net
academia-nsk.org adman.gr
cartierlovebracelet.xyz alert-fjg.xyz
en.altezza.travel android-vsem.org
flprog.com bankcrediti.ru
investpamm.ru famix.xyz
iptool.xyz job.icivil.ir
priora-2.com mobioffertrck.com
seemoreresultshu-a.akamaihd.net pron.pro
tizanidine4mgprice.blogspot.com tedy.su

944
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0lovespells0.blogspot.com 03p.info
1-free-share-buttons.com 0n-line.tv
123movies.love 101flag.ru
178evakuator178.ru 14b.info
20pascals.nl 1kinobig.ru
273-fz.ru 24videos.tv
5elementov.ru 2girls1cup-free.com
66cpwgln.space 3rbseyes.com
7minuteworkout.com 3th.co.in
8si.ru 404.mn
91abcw.com 4rent.ru
aa9bd78f328a6a41279d0fad0a88df1901.com a.pr-cy.ru
addbin.men aa81bf391151884adfa3dd677e41f94be1.com
addblueoff.com.ua aa9d046aab36af4ff182f097f840430d51.com
addray.pro aaa38852e886ac4af1a3cff9b47cab6272.com
adinterax.com aasoldes.fr
adman.gr absolute-s.ru
acads.net
admatic.com.tr
admitad.com admitad.com
adpremium.org adnotbad.com
ads-cool.pro adultfullhd.com
adssafeprotected.com advertisingtag.net
adultnet.in alf-img.com
advancedsoftwaresupport.com alive-ua.com
afmuseum.com alkoravto.ru
agapovdg.ru allfinweb.com
ahmedabadwebs.com allnews.md
ahrntal.verymes.xyz alveris.ru
akvamaster.dp.ua amateurgalls.com
alfa-img.com ameblo.top
alfabot.xyz animalia-life.club
allcredits.su aosheng-tech.com
alldownload.pw apilinkswiftco-a.akamaihd.net
aludecor.info appfastplay.com
alyeskaresort.com aproposde.com
android-style.com arkartex.ru
androids-store.com asacopaco.tk
anjalika.co.in atlasvkusov.ru
apccargo.com aucoinhomes.com
approvedlocal.co.za ausergrubhof.info
aquarium-pro.ru autoblogger24.info
asrv-a.akamoihd.net autobudpostach.club
atlant-auto.info
auto-zapchasti.org
autonew.biz
avkzarabotok.info avkzarabotok.info
backiee.com avon-novosib.ru
badavit.com.ua ayerbo.xhost.ro
balans.shahterworld.org azbukafree.com
balitouroffice.com azte.ch
bannerswap.com b-styles.xyz
barbourjackets.online babieca.com
babs.com.ua
backuperwebcam.weebly.com
baersaratov.ru
baikaleminer.com
bank.uz
banksrf.ru
bcmp.org
beauty-clinic.ru
beauty-things.com beauty-things.com
beget.tech bedandbreakfast.com
beremenyashka.com bestimagecoollibrary.com
berrymall.ru bestmarriages.com
bestchoice.cf bestsexyblog.com
bestkfiledxd.cf beta.hotkeys.com
betterhdporn.com bimatoprost-careprost.com
bigames.online bimatoprost-careprost.com.ua
bioscorp.ru bioinnovate.co
blackhatworth.com bitcoin-ua.top
blog.f00kclan.de bjetjt.com
blog.yam.com blagovest-med.ru
bloggerads.net
bloggers.nl
bloke.com bloke.com
bluejays-jerseys.us braintobe.win
boleznikogi.com briomotor.co
bombla.org browsepulse-a.akamaihd.net
bongacams.com brus-vsem.ru
bonky.biz bum.com.ru
boobsimge.com bumble.cheapwebsitehoster.com
bosman.pluto.ro burkesales.com
braip.com.br bus-offer.com
brakehawk.com business-suggested.tk
breastaugmentation.co.za buyantiviralwp.com
brendbutik.ru buyessay3.blogspot.ru
buqayy0.livejournal.com buytizanidineonline.blogspot.com
canrioloadm.gq byme.se
canvas.gloverid.site
canzoni.ru canzoni.ru
carina-sy.de cartierwatch.xyz
cartierlovestore.xyz cashonads.com
cb.iphantom.com catalogs-parts.com
cbozhe.com cayado.snn.gr
cenokos.ru cdncash.com
chelyabinsk.xrus.org centre-indigo.org.ua
cindymatches.com ceotrk.com
civilwartheater.com certifywebsite.win
clean-start.net cherubinimobili.it
click2pawn.com chocoslim.pro
clickaider.com christianlouboutinsaleonline.us
clickprotects.com chuckguilford.com
clmforexeu.com cialiswithoutadoctor.net
cienum.fr
cineacademy.ru
circlesl.com
clipartnew.com
clodo.ru clodo.ru
coldfilm.ru cometorussia.net
comeondog.info compliance-fred.top
compliance-irvin.xyz compliance-george.top
contentsexpress.com compliance-viktor.top
contextualyield.com comsysnet.com
copblock.org cookie-law-enforcement-aa.xyz
coslab.club cookie-law-enforcement-hh.xyz
coolgramgoods.com
corta.co
countbertwistdisp26.soup.io countbertwistdisp26.soup.io
counterbot.com cowlmash.com
covadhosting.biz dafatiri.com
crackguru.tk data-mining.tk
customsua.com.ua datract.com
cypernhuset.se dayibiao.com
cyse.tk
czeshop.info
data.vtc.pw
ddpills.com
dealighted.com dealighted.com
decenttools.com decorationspcs.com
diamond-necklace.info delitime.info
digital-sale.su dengi-pod-zalog-nedvizhimosti.ru
digitalfaq.com div.as
dinosaurus.site
dipstar.org
directivepub.com
divatraffic.com divatraffic.com
dnepropetrovsk.mistr-x.org dodlive.mil
domainsphoto.com domainanalyzing.xyz
domoysshop.ru domcran.net
doska-vsem.ru donvito.unas.cz
dosugrostov.site downloadeer.net
doublepimp.com downloader12.ru
draniki.org drev.biz
e-biznes.info drunkenstepfather.com
e-poker-2005.com dvervmoskvu.ru
easycommerce.cf dvr.biz.ua
ec-file.info e-c.al
ekaterinburg.xrus.org econom.co
elektrischezi.canalblog.com educontest.net
elektrischezigarette2.onsugar.com el-nation.com
elektrischezigaretten2.beeplog.com eleimgo.pw
elektroniksigaraankara.info elektrozigarette1.dreamwidth.org
elektronischezigarettekaufen1.beeplog.com elstal.com.pl
emergencyneeds.org emergencyneeds.org
englate.com erectile.bid
enskedesquashclub.se eropho.net
ereko.ru erotik-kostenlos.net
eshop.md erotik0049.com
european-torches.ru eshop4u.jp
evrotekhservis.ru essaypro.com
extremepornos.net europages.com.ru
fashion-stickers.ru evogarage.com
fealq.com expolicenciaslatam.co
femdom.twiclub.in ezigarettekaufen1.hpage.com
fishtauto.ru ezigarettekaufen2.mpbloggar.se
flauntyoursite.com ezigarettenshop2.myblog.de
floridamobilebillboards.com ezigarettenshop2.postbit.com
forodvd.com f-loaded.de
fortevidyoze.net feorina.ru
fermersovet.ru
fetishinside.com
filunika.com.ru
findthe.pet
fitness-video.net
fm-upgrade.ru
fordsonmajbor.cf
forexgb.ru
forum.poker4life.ru forum.poker4life.ru
foto-telok.net forums.d2jsp.org
foxtechfpv.com foxydeal.com
free-laptop-reward.com frankofficial.ru
free-stock-illustration.com frcls.fr
freelinkbuilding.website.tk freakycheats.com
freshdz.com free-video-tool.com
freshmac.space free411games.com
frivgame250.com funcrushgames.com
fuckmill.com fyxabomiw.ru
gameonasia.com gael-s.ru
garciniaxt.us gamblingnerd.com
geileweiber.tk getoutofdebtfree.org
gloverid.site gilsonchiro.xyz
golden-praga.ru glavprofit.ru
gombita.info glcomputers.ru
goodwinmetals.co gojiberriess.apishops.ru
google-liar.ru google-liar.ru
goosefishpost.bid gosarhivrt.ru
gov.yanao.ru gratuitxblcodes.com
greenidesign.co gribokstop.com
grupografico-pilar.com.ar
gsasearchengineranker.pw
gstatey.net
guigyverpo.cf
guitar-master.org
habermetre.com habermetre.com
hamptonoaks.ca harmonyglen.us
hdwalls.xyz healthcarestore.info
heartofbeijing.blogspot.com hmywwogw.bloger.index.hr
historichometeam.com homedecoguide.info
hol.es homedo.fabpage.com
homeart.space houseofrose.com
homeinns.com hybrid.ru
homemature.net iframe-toloka.com
hotelcrocenzi.sm igtools.club
hundejo.com ilikevitaly.com
hydropump.su incanto.in.ua
i-hobot.ru incep.imagine-ex.co
igru-xbox.net infazavr.ru
ilo134uloh.com insta-add.pro
images-graphics-pics.com investzalog.ru
images.gyffu.com iphantom.com
impresagaia.it isotoner.com
incolors.club ispac.org
integritylandscapeservices.com istizanidineacontrolledsubstance.blogspot.com
interior-stickers.ru itag.pw
iqs.biz.ua janavibekken.no
istizanidineanopiate.blogspot.com jav-idol.com
istock-mebel.ru javlibrary.cc
ix20.ru jose.mulinohouse.co
janettabridal.com jovencitas.gratis
japfm.com juliadiets.com
je7.us juliaworld.net
jeremyeaton.co jwss.cc
jerseychinabizwholesale.us kchaxton.com
jerseywholesalechinabiz.com keyword-suggestions.com
joessmogtestonly.com kino-rating.ru
johnrobertsoninc.com kinobaks.com
joyceblog.top
jpcycles.com
jrcigars.com
jumptap.com
jurajskie.info
karachev-city.ru
kaz.kz
ketoanhanoi.info
keywordteam.net
kino-ecran.ru
kinogolos.ru
kinogonew.ru kinogonew.ru
kndxbkdx.bloger.index.hr konica.kz
kol-energo.ru konturkrasoty.ru
kolotiloff.ru kosmetyki.tm.pl
komp-pomosch.ru kotaku.com
kostenlos-sexvideos.com krasivoe-hd.com
kreativperlen.ch krasivye-devushki.net
ku6.com kumuk.info
lampokrat.ws kwzf.net
lashstudia.ru lakomka.com.ua
leboard.ru lanadelreyfans.us
ledpolice.ru law-check-eight.xyz
lennatin.info
lescinq.com
lflash.ru
liffa.ru
lignofix.ua lignofix.ua
lion.conboy.us limtu.ifmo.ru
liquimondo.com linkrr.com
liver-chintai.org live-sexcam.tk
liveu.infoteka.hu
livingcanarias.com
lkbennettoutlet.online lkbennettoutlet.online
loadingpages.me loadingpages.me
lol-smurfs.com localflirtbuddies.com
lrac.info localmatchbook.com
m3gadownload.pl ltvperf.com
maju.bluesalt.co mabdoola.blogspot.com
mall.uk mackeeper-land-672695126.us-east-1.elb.amazonaws.com
malwareremovalcenter.com magazintiande.ru
managerpak204.weebly.com magicplayer-s.acestream.net
manuscript.su makemoneyonline.com
manyresultshub-a.akamaihd.net map028.com
matematikus.info mastershef.club
mathgym.com.au masthopehomes.com
mbiologi.ru meendo-free-traffic.ga
medicine-4u.org mega-polis.biz.ua
medicovi.com megavolt.net.ua
mejoresfotos.eu mhtr.be
menstennisforums.com microsoftportal.net
metarip.ru mindeyegames.com
michellblog.online minegam.com
microstatic.pl mini.7zap.com
mil-stak.com mirmedinfo.ru
millionare.com mirtorrent.net
mistr-x.org mlf.hordo.win
mobile-appster.ru mnimmigrantrights.net
moivestiy.biz mobioffertrck.com
monarhs.info moscow.online-podarki.com
moncleronline.xyz mossmesi.com
most.gov.iq mrpornogratis.xxx
musicdaddy.net msk-diplomat.com
muztops.ru mtmtv.info
mydearest.co mvpicton.co.uk
mydeathspace.com my-big-family.com
mydirtystuff.com mydirtystuff.com
myfreecams.com
myfreemp3.eu
mymobilemoneypages.com
myonlinepayday.co
mysexpics.ru mysexpics.ru
myshopmatemac.com nacap.ru
mystats.xyz narco24.me
nalogovyykodeks.ru nastydollars.com
narkologiya-voronezh.ru newtechspb.ru
naval.jislaaik.com nextbackgroundcheck.gq
needtosellmyhousefast.com nextconseil.com
newsperuse.com nicovideo.jp
newstraveller.ru nmrk.ru
nhl17coins.exblog.jp nottyu.xyz
novosibirsk.xrus.org november-lax.com
nowtorrents.com nudejapan.net
nw-servis.ru oddamzadarmo.eu
o.light.d0t.ru
o00.in
oda.as
odesproperty.com
odywpjtw.bloger.index.hr odywpjtw.bloger.index.hr
oecnhs.info okout.ru
offers.bycontext.com oneclickfiles.com
okel.co online-podarki.com
onefilms.net online247.ml
onemactrckr.com onlinefilmz.net
oneminutesite.it ons-add.men
onescreen.cc ooo-olni.ru
online-x.ru openfrost.net
onlinemegax.com optitrade24.com
oranga.host.sk optom-deshevo.ru
orsonet.ru ororodnik.goodbb.ru
oyster-green.com osoznanie-narkotikam.net
p-business.ru page2rss.com
paclitor.com
paighambot.com paighambot.com
panicatack.com pammik.ru
partner-pop.men panouri-solare-acoperis.com
partnerads.men papasdelivery.ru
partnersafe.men parvezmia.xyz
patol01.pw pastaleads.com
paydayloanslocal.com pc-test.net
pestomou.info peekyou.com
phrcialiled.com perfectpracticeweb.com
pic-re.blogspot.com perm.xrus.org
piccdata.com photorepair.ru
pinwallpaper.top picturesboss.com
pixshark.com pizdeishn.com
pooleroadmedicalcentre.co.uk pl-top.pl
porno-transsexuals.ru pl.id-forex.com
porno-video-chati.ru platesauto.com
pornosee.info platezhka.net
pornoted.com play.leadzupc.com
predmety.in.ua podshipniki-nsk.ru
preg.marketingvici.com pokemon-go-play.online
presleycollectibles.com popcash.net
privatefx-in.ru popugaychiki.com
pro-tec.kz porndairy.in
proposal-engine.com pornokajf.com
propranolol40mg.blogspot.com pornonik.com
prostitutki-belgoroda.org pornosok.ru
prostitutki-rostova.org pourvous.info
puteshestvennik.com powenlite24.ru
qor360.com preventheadacheguide.info
rabotaetvse.ru privat-girl.net
rank-checker.online prizesbook.online
rankings-analytics.com professionaldieselcare.com
rankscanner.com profitsport.club
realmonte.net psvita.ru
rebelmouse.com pufip.com
pukaporn.com
pussysaga.com
qwesa.ru
randalljhoward.com
rangjued.com
ranksonic.com
ranksonic.org
ratemodels.net
razorweb-a.akamaihd.net
rczhan.com
recordpage-a.akamaihd.net recordpage-a.akamaihd.net
reeyanaturopathy.com redbottomheels.xyz
registrydomainservices.com redhotfreebies.co.uk
remorcicomerciale.ro regdefense.com
renhacklids.tk registry-cleaner.net
research.ifmo.ru remontvsamare.su
restorator-msk.ru rentehno.ru
rewardit.com rep-am.com
rfd-split.hr reversing.cc
rfid-locker.co reyel1985.webnode.fr
rfserial.net rezeptiblud.ru
rialp.getenjoyment.net rockma.se
ringporno.com rospromtest.ru
rogervivierforsale.com rrutw.com
rossmark.ru ruclicks.com
rucrypt.com rumamba.com
rusoft-zone.ru russkoe-zdorovie.ru
rusprostitute.com sabizonline.com
s-forum.biz sadaholding.com
samara.rosfirm.ru sajatvelemeny.com
sarahmilne.top salmonfishingsacramentoriver.com
sbdl.no sanatorrii.ru
sbtdesign.co.uk sarf3omlat.com
scanmarine.info satoristudio.net
scanner-jack.top saugatuck.com
score-ads.men scanner-viktor.top
searchaddis.com scanner-willy.top
searchengineranker.email screpy.com
searchimage.co seo-tools-optimizing.com
secretscook.ru seo18.su
seofirmreviewsus.info service-core.ru
seorank.info seryeznie-znakomstva.ru
seorankinglinks.xyz sexvideo-sex.com
sethrollins.net shinikiev.com.ua
sevendays.com.ua shitmovs.com
sexobzor.info shtora66.ru
sexspornotub.com sicfor.bcu.cc
sfj-ror.no signx.info
shell-pmr.ru sildenafil-tadalafil.info
site-analyzer.com simplesite.com
skatestick.bid sirpornogratis.xxx
slowmacfaster.trade slavkokacunko.de
slujbauborki.ru smadihome.com
smailik.org smartshoppymac.com
snyatie-lomki-v-stacionare.ru snworks.com
soaksoak.ru social-buttons.com
socialmonkee.com soft-program.com
spasswelt.net softlinesolutions.me
spidtest.space sortthemesitesby.com
sticken.co sotechco.co
space2019.top
special-porn.com
specialfinanceoffers.com
speechfoodie.com
sport-video-obzor.ru
sportbetfair.com
spravka-medosmotr.ru
stardevine.com
statoutlook.info
strana-solnca.ru
stretchingabuckblog.com
strongsignal-a.akamaihd.net
styro.ru styro.ru
sudexpert66.ru subj.ukr-lit.com
sugarlyflex.pw success-seo.com
sunflowerdrawingpaintings.blogspot.com sugarkun.com
supermama.top supermesta.ru
supers.com.ua suture.co
superstats.com taxi-v-eisk.ru
surflinksmedical.com teencastingporn.com
surgut.zrus.org telegramdownload10.com
survival.betteroffers.review
swinon.site
syvertsen-da.no
taaaak.com
tackletarts.co
tandvardshuset.net
tattooha.com
tattooreligion.ru
taximytishi.ru
technika-remont.ru
technopellet.gr
teenbbw.yopoint.in
teletype.in teletype.in
theendivechronicles.com textads.men
thetoiletpaper.com thaisamkok.com
thruport.com thebestphotos.eu
tiandeural.ru thebitcoincode.com
titan-ads.life thegioixekhach.com
tizanidine4mgtablets.blogspot.com thfox.com
tizanidinepills.blogspot.com ticketsys.inetwd.com
tnaionline.org tizanidine4mg.blogspot.com
tooplay.com tizanidine4mgstreetvalue.blogspot.com
top10-way.com tizanidineshowupondrugtest.blogspot.com
topanasex.com tizanidinesideeffects.blogspot.com
topquality.cf tk-assortiment.ru
torrentred.games tonivedu.it
toyota.7zap.com torrent-to-magnet.com
track-rankings.online torture.ml
trafficfactory.biz totu.info
trafficjunky.com traffmonster.info
trahic.ru trailer.cinemaflix.website
trkdf.com tramadolandtizanidine.blogspot.com
tube8.com traxdom.ru
tula.mdverey.ru treasuretrack-a.akamaihd.net
tupper-shop.ru tryrating.com
tvteleport.ru tsan.net
tubo360.com
tv-spoty.info
tvoystartup.ru
twincitiescarservice.com twincitiescarservice.com
twittruth.com u555u.info
txxx.com udsgame.online
unimodemhalfduplefw.pen.io ugguk.online
upupa.net ukrobstep.com
uroffer.link ultimatesetnewfreeallsoftupgradesystems.pw
vchulkax.com urldelivery.com
venerologiya.com usiad.net
viagengrarx.com
viberdownload10.com
video-chat.love
videochat.life videochat.life
videonsk.com videonsk.com
viktoria-center.ru videtubs.pl
vipms.ru vip.51.la
vitalads.net visionwell.com.cn
vizitki.net vltai.com
vkak.ru vriel.batcave.net
voloo.ru wahicbefa31.soup.io
voprosotvet24.ru wardreapptokone.tk
web-analytics.date wareseeker.com
web.cvut.cz warningzscaler.heraeus.com
website-datenbank.de webenlace.com.ar
website-speed-checker.site webshoppermac.com
website-speed-up.site websitevaluebot.com
webstatsdomain.org wellcome2slovenia.ru
weburlopener.com westermarkanjou.se
wholesalejerseychinaoutlet.com whengirlsgowild.com
wholesalenfljerseys.us.com winterclassichockeyjerseys.com
wholinkstome.com winwotgold.pl
wladimirpayen.com wordkeyhelper.com
womens-journal.net ws.ampower.me
womensterritory.ru wtsindia.in
wordpress-crew.net www.888.com
wpthemedetector.co.uk www.wohnkabinen-shop.de
writersgroup580.web.fc2.com x-diesel.biz
x69ty.ru x-musics.com
xblognetwork.com xitjw.info
xev.ru xjrul.com
xn--80ak6aa92e.com xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xn--b1agm2d.net xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xportvusbdriver8i.snack.ws xn--80aeahghtf8ac5i.xn--p1ai
yoshkarola.zrus.org xn--b1addnj3cah.xn--p1ai
youdesigner.kz xn--h1aakne2ba.xn--p1ai
yourdesires.ru xsion.net
yourmovies.pl xxxhdvideo.site
yuweng.info yougetsignal.com
zambini.ru youtubedownload.org
zeleznobeton.ru zaim-pod-zalog-krasnodar.ru
zeroredirect12.com zarenica.net
zeroredirect8.com zatjmuzu.info
zhongwenlink.com zetgie.com.pl
zixizop.net.ru zhuravlev.info
zigarettenonlinekaufen2.bloog.pl
zlatnajesen.com
zmoda.hostreo.com
zona-aqua.ru zona-aqua.ru
zoodrawings.com zoominfo.com
zscaler.net zrizvtrnpale.tk
zscalertwo.net zvetki.ru
zzlgxh.com

464
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00it.com 101raccoon.ru
100searchengines.com 123kuma.com
11pikachu.ru 2pxg8bcf.top
2kata.ru 3-letter-domains.net
4webmasters.org 5forex.ru
5tey7463.weebly.com 70casino.online
7xc4n.com 8kisses.eu
a.pr-cy.ru
aadroid.net
acmebtn.ml
adexprts.com
admitad.com
adult-shop.com.ua adult-shop.com.ua
adventureparkcostarica.com adultactioncam.com
advokateg.xyz adultfriendfinder.com
affordablewebsitesandmobileapps.com
agahinameh.com
alborzan.com alborzan.com
alibestsale.com
alice.it
allblogroll.com allblogroll.com
amateurmatch.com alldezservice.kz
andpolice.com amazon-seo-service.com
androids-store.com androids-store.com
apiluckyleapnet-a.akamaihd.net apessay.com
apiportalmorecom-a.akamaihd.net apidiamondatanet-a.akamaihd.net
appfixing.space apiqualitinknet-a.akamaihd.net
arate.ru arabsexxxtube.com
asscenihotosidea.blogspot.co.za aramaicmedia.org
arcteryxstore.online
arendakvartir.kz
artparquet.ru
ausergrubhof.info ausergrubhof.info
auto4style.ru automobile-spec.com
avon-ukraine.com autonew.biz
avtolombard-krasnodar.ru autoplate.info
backiee.com autorn.ru
backlink4u.net avto-oligarh.ru
bestbrainprod.win beachfix.co
bestbuy.ca beauty-b0x.pl
bestofferswalkmydogouteveryday.gq bestwebsitesawards.com
blpmovies.com betwinservice.com
braintobe.win bitcoin-ua.top
buy-meds24.com blavia.00author.com
blog.koreadaily.com
blogrankers.com
bmw.afora.ru
bobba.dzaba.com
brewdom.ru
bronzeaid-a.akamaihd.net
buigas.00it.com
bus-offer.com
butstrap.space
buyantiviralwp.com buyantiviralwp.com
carina-sy.de bwlx.prepedu.cn
cars-modification.net cardsharp1.ru
carson.getenjoyment.net chinese-amezon.com
casite-513049.cloudaccess.net ciekawinki.pl
caulderwoodkennels.com coldfilm.ru
cbox.ws
celebsopics.com
cenoval.ru
cheap-trusted-backlinks.com
cheapmoncler.win
cleanmypc.com
clicksor.com
clknsee.com
coderstate.com
com-supportcenter.website com-supportcenter.website
compliance-nelson.top compliance-barak.xyz
contentlook.co compliance-ivan.top
cookie-law-enforcement-aa.xyz cookie-law-enforcement-hh.xyz
cookingmeat.ru copenergo.ru
creditwell.ru data-mining.tk
cupday.com davebestdeals.com
dealitright.click deutschland123.de
decenttools.com dfwu1013.info
deplim.com dinkolove.ya.ru
detalizaciya-tut.biz dipstar.org
driving.kiev.ua doctissimo.fr
e-c.al dotnetdotcom.org
e90post.com download-walpaperhd.blogspot.com
ecookna.com.ua downloader12.ru
educontest.net downloadkakaotalk.com
electricwheelchairsarea.com ednorxmed.com
eluxer.net elmifarhangi.com
energy-ua.com en.home-task.com
europages.com.ru eralph.tk
ezigarettenshop2.postbit.com estibot.com
ezrvrentals.com fearcrow.com
familyphysician.ru femmesdenudees.com
fba-mexico.com ferretsoft.com
fefo.gdn fickenprivat.info
fifa-coins.online fifa-coins.online
fikasound.tk firesub.pl
flowwwers.com focalink.com
fr-bearings.ru forum.tvmir.org
freejabs.com genericlowlatencyasiodriverhq.aircus.com
freelotto.com getresponse.com
fungamelands.com gfaq.ru
funponsel.com gkvector.ru
gagrasector.ru glass-msk.ru
gallictures.com gratis-sexkontakte.com
goo.ne.jp gsasearchengineranker.pw
graphics8.info gsasearchenginerankerdiscount.com
grtyi.com gstatey.net
gsou.cf harmonyglen.us
guigyverpo.cf hermesreplica.win
hchha.com hmywwogw.bloger.index.hr
hebr.myddns-flir.com host-protection.com
hem.passagen.se hotkeys.com
hildinghr.se hreade.com
hitmuzik.ru hubbble.com
housediz.com hunthillfarmtrust.org
ibmdatamanagement.co hustoon.over-blog.com
ilo134uloh.com iamsport.org
images-graphics-pics.com infobabki.ru
iphantom.com inmate-locator.us
itch.io intimshop-fantasy.ru
jabimgo.pw investyb.com
kazrent.com ip-guide.com
kinohit1.ru javidol.site
kochanelli.com jpcycles.com
kol-energo.ru jyvopys.com
komputers-best.ru kerch.site
kosova.de knigonosha.net
lacapilla.info kollekcioner.ru
ladov.ru komp-pomosch.ru
lasvegaslockandsafe.com kupiproday.com.ua
laxob.com lacasamorett.com
livejasmin.com landmania.ru
livingroomdecor.info languagecode.com
locimge.pw larger.io
locksmith.jp laulini.soclog.se
lollypopgaming.com likesdesign.com
marcowebonyodziezowe.pl liky.co.ua
medtherapy.ru linkwithin.com
mere.host.sk lion.misba.us
metaxalonevstizanidine.blogspot.com lion.playtap.us
mielec.pl lonely-mature.com
mjchamonix.org macnewtech.com
maslenka.kz
mazda-roadsters.com
medicinacom.ru
medictube.ru
menhealed.net
metascephe.com
mir-business-24.ru
mncrftpcs.com mncrftpcs.com
mnogolok.info mobifunapp.weebly.com
moetomnenie.com moetomnenie.com
mosrif.ru monclerjacketsoutlet.pw
mouselink.co movpod.in
movies-in-theaters.net multgo.ru
mp3ritm.top musezone.ru
mvpicton.co.uk muz-tracker.net
news-speaker.com mxgetcode.com
ningessaybe.me my-big-family.com
nonews.co mydeathspace.com
nootrino.com mylovelibrabry.com
o-o-6-o-o.ru newfilmsonline.ru
obiavo.by nikhilbahl.com
offer.camp november-lax.com
oneclickfiles.com oconto.ru
oneminutesite.it odywpjtw.bloger.index.hr
onescreen.cc
onlainbesplatno.ru
onlineslotmaschine.com
ooomeru.ru ooomeru.ru
openstat.com openfrost.net
orel-reshka.net oyster-green.com
pendelprognos.se pandroid.co
percin.biz.ly parajumpersstore.online
pochtovyi-index.ru partnerads.men
popserve.adscpm.net pateaswing.com
piratecams.com
pistonclasico.com
pl.aasoldes.fr
popugaychiki.com
porno-home365.com porno-home365.com
pornoelita.info pornoblood.com
pornosemki.info pornoreino.com
powenlite24.ru pornosee.info
pretty-mart.com posible.net
printingpeach.com potolokelekor.ru
progonrumarket.ru proekt-gaz.ru
proposal-engine.com professionaldieselcare.com
proxyelite.biz prosmibank.ru
quickchange.cc
radiodigital.co radiodigital.co
raisedseo.com rankscanner.com
rangjued.com rcpmda.ikan1080.xyz
ranksonic.com remontbiz.ru
realitykings.com rentalcarnavi.info
relax.ru rethinkwasteni.info
remont-comp-pomosh.ru rootandroid.org
resant.ru rosbalt.com.ua
retreatia.com rp9.ru
rfid-locker.co sankt-peterburg.nodup.ru
rocketchange.ru
roma-kukareku.livejournal.com
russkie-gorki.ru
rustic-quiver.win
sale-japan.com
sawin.beth.webd.pl
sax-sex.com sax-sex.com
sbetodiodnye-lampy.ru scanner-irvin.top
seccioncontrabajo.com scanner-viktor.top
security60-e.com securityallianceservices.com
seemoreresultshu-a.akamaihd.net semalt.com
serpstat.com semrush.com
service-core.ru seo18.su
serviporno.com servicecenter.co.ua
sexreliz.com sex-pr.net
shohanb.com sexpornotales.net
simple-image.com.ua shakhtar-doneck.ru
slavia.info shmetall.com.ua
slim.sellany.ru sitechecker.pro
snts.shell-pmr.ru sivs.ru
socialbuttons.xyz sizeplus.work
sogimlecal.tk slowmac.tech
solution4u.com smartadserver.com
souvenir.cc social-s-iii.xyz
spinnerco.ca solinf.co
srdrvp.com spasswelt.net
start.myplaycity.com sportbetfair.com
statustroll.com statustroll.com
steebook.com streetfire.net
stockquotes.wooeb.com stroicol.net
sunflowerdrawingpaintings.blogspot.com stromerrealty.com
supernew.org telefonsexkostenlos.tk
superoboi.com.ua tengohydar.tk
svbur.ru thefotosgratis.eu
swinger-mobil.net thelottosecrets.com
swiped.su tizanidine4mgstreetprice.blogspot.com
szucs.ru tizanidinerecreationaluse.blogspot.com
tackletarts.co tonerbox.kz
technopellet.gr toonfamilies.net
telsis.com trahic.ru
thecoral.com.br trustedhealthtips.com
thegolfclub.info tuberkuleznik.ru
tizanidineandhydrocodone.blogspot.com
tizanidinehcl4mgtablet.blogspot.com
tizanidinevsvalium.blogspot.com
tonivedu.it
truebeauty.cc
tuckpointingmasonrysystems.com tuckpointingmasonrysystems.com
universals.com.ua turbabitload.weebly.com
usadacha.net twitlinks.com
usdx.us udayavani.com
usiad.net ultimatesetnewfreeallsoftupgradesystems.pw
uvozdeckych.info unacittaconte.org
vacuumscleaner.com vacances-voyages.info
venerologiya.com varikoz24.com
veselokloun.ru video-chat.in
vgoloveboli.net vykupavto-krasnodar.ru
vigrx-original.ru
vsexkontakte.net
w-journal.ru
waterpurifier.club
webinstantservice.com webinstantservice.com
weightatraining.com webix.me
wesharepics.site weddingdresses.xyz
whatistizanidinehclusedfor.blogspot.com westvilletowingservices.co.za
wladimirpayen.com wma-x.com
worldtraveler.world worldhistory.biz
wpsecurity.website wttavern.com
xerox-douglas.cf www.labves.ru
xn--80ahvj9e.xn--p1ai xlovecam.com
xn--b1ag5cfn.xn--p1ai xn----7sbahjd3btneuw1joc.xn--p1ai
xtubeporno.net xn--80aaagvmjabrs1aoc9luc.xn--p1ai
yellowads.men xn--80atua3d.xn--p1ai
yellowfootprints.com xn--d1aifoe0a9a.top
yhit.press
youghbould.wordpress.com
youradulthosting.com
zajm-pod-zalog-nedvizhimosti.ru
zarajbuilders.com zarajbuilders.com
zeroredirect5.com zastroyka.org
zhuravlev.info zhorapankratov7.blogspot.com
zophim.me zootravel.com
zscaler.net
zzlgxh.com

14
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -1,11 +1,21 @@
<html> <!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
ml><html> <html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.0.131 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2690 ### Version: V4.2021.11.2691
### Updated: Tue Nov 2 22:03:12 UTC 2021 ### Updated: Wed Nov 3 10:03:20 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2690 ### Version: V4.2021.11.2691
### Updated: Tue Nov 2 22:03:12 UTC 2021 ### Updated: Wed Nov 3 10:03:20 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##