vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.08.2555 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-08-28 22:07:16 +00:00
parent 657f09d36b
commit 7ebae02f38
47 changed files with 56763 additions and 56773 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2554 #### Version: V4.2021.08.2555
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2554 #### Version: V4.2021.08.2555
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.08.2554 #### Version: V4.2021.08.2555
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

6958
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.08.2554 #### Version: V4.2021.08.2555
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

6962
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

6962
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

6962
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

6962
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.129 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.129 0;
127.0.0.1 0; 127.0.0.1 0;

6962
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.129 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.129 0;
127.0.0.1 0; 127.0.0.1 0;

6962
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.129 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.129 0;
127.0.0.1 0; 127.0.0.1 0;

6962
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2554 2555

6962
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

6962
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.148.183.141 0; 20.106.167.166 0;

6962
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.129 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.129 0;
127.0.0.1 0; 127.0.0.1 0;

6962
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.129 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.129 0;
127.0.0.1 0; 127.0.0.1 0;

6962
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.129 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.129 0;
127.0.0.1 0; 127.0.0.1 0;

6962
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.148.183.141 0; 20.106.167.166 0;

6962
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.148.183.141 0; 20.106.167.166 0;

6962
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.129 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
CSHttp Aboundexbot
Craftbot Brandprotect
Intelliseek Ezooms
RepoMonkey InfoNaviRobot
SentiBot MSFrontPage
Surfbot MSIECrawler
Webster Semrush
ZumBot Zermelo
lwp-trivial check1.exe
trendiction.de instabid

328
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
80legs 01h4x.com
ADmantX 360Spider
AIBOT AIBOT
ALittle Client ALittle Client
ASPSeek Abonti
Aboundex Aboundex
AhrefsBot AllSubmitter
AiHitBot Anarchie
Aipbot Anarchy99
Anthill
Attach Attach
AwarioRssBot
AwarioSmartBot
BBBike BBBike
BDCbot
BDFetch BDFetch
BackDoorBot
BackWeb BackWeb
BacklinkCrawler
Badass
Barkrowler Barkrowler
BatchFTP Battleztar Bazinga
Black Hole
BlackWidow BlackWidow
Blackboard
BlowFish BlowFish
Brandwatch BotALot
BuiltWith
Bullseye Bullseye
BunnySlippers
BuzzSumo
CATExplorador CATExplorador
CODE87
CSHttp CSHttp
CazoodleBot
CensysInspect CensysInspect
CheTeam CheeseBot
Chlooe Claritybot
Cliqzbot CopyRightCheck
Cogentbot
Collector
Copier
Copyscape
Cosmos Cosmos
Craftbot
Crescent Crescent
Curious Custo
DSearch DBLBot
DatabaseDriverMysqli DatabaseDriverMysqli
Dirbuster Demon
Discoverybot Devil
Dispatch Disco
DomainAppender Discobot
DnyzBot
DomCopBot
DomainStatsBot DomainStatsBot
Dotbot Domains Project
Download Wonder
Dragonfly
ECCP/1.0
EMail Wolf
EasyDL EasyDL
Ebingbong
Ecxi Ecxi
EirGrabber Exabot
Express WebPictures ExtractorPro
Extractor Extreme Picture Finder
EyeNetIE EyeNetIE
Firefox/7.0 FHscan
FlashGet FemtosearchBot
Fimap
Flunky
FrontPage FrontPage
Fyrebot Fuzz
G-i-g-a-b-o-t FyberSpider
GalaxyBot GT::WWW
Genieo
GermCrawler
GetRight GetRight
Go!Zilla GetWeb
Getintent
Go-Ahead-Got-It
GoZilla
Gotit Gotit
GrabNet Grabber
GrapeFX
HEADMasterSEO HEADMasterSEO
HTMLparser HTMLparser
Haansoft HTTP::Lite
HaosouSpider HTTrack
Harvest Havij
Humanlinks Hloader
HybridBot HybridBot
IDBTE4M
IDBot IDBot
IRLbot IlseBot
Indy Library
InfoNaviRobot InfoNaviRobot
Iskanie InterGET
JOC Web Spider Internet Ninja
InternetSeer
IstellaBot
JamesBOT JamesBOT
Jbrofuzz JennyBot
JikeSpider JetCar
Jetty
Joomla Joomla
Kozmosbot Jorgee
JustView
Kenjin Spider
Keyword Density
LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
Larbin Larbin
Leap LeechFTP
LieBaoFast LeechGet
Likse Libwhisker
LinkWalker LinkWalker
Linkdexbot Linkdexbot
LinkextractorPro
LinkpadBot LinkpadBot
LinksManager
LinqiaRSSBot LinqiaRSSBot
Litemage_walker Lmspider
Ltx71
MQQBrowser MQQBrowser
MSFrontPage
MTRobot
Mag-Net
Magnet Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic SEO MeanPath Bot
Majestic-SEO
MarkWatch
Mass Downloader
Masscan
Mata Hari
Meanpathbot
MegaIndex.ru MegaIndex.ru
Microsoft Data Access MicroMessenger
Mojeek Microsoft URL Control
Moblie Safari
Mojolicious Mojolicious
Mr.4x3 Mozlila
NPbot Msrabot
NearSite Musobot
Needle Needle
Nessus
Net Vampire
NetAnts
NetMechanic NetMechanic
Netvibes Nikto
NextGenSearchBot Nimbostratus
NimbleCrawler
Nmap
Nutch
Octopus
Offline Explorer Offline Explorer
OrangeBot OnCrawl
OrangeSpider Openvas
OutclicksBot OutclicksBot
PECL::HTTP
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageScorer
PageThing.com PageThing.com
Pavuk Pandalytics
Picscout PeoplePal
Picsearch Petalbot
ProPowerBot PictureFinder
ProWebWalker ProWebWalker
Pump Probethenet
PxBroker Pu_iN
QueryN Metasearch
Quick-Crawler
RankActiveLinkBot RankActiveLinkBot
RankingBot RankingBot
Re-re RankingBot2
ReGet RealDownload
RebelMouse
Recorder
RepoMonkey RepoMonkey
Ripper Ripper
RocketCrawler SBIder
SEOkicks SEOstats
SEOlyticsCrawler SISTRIX
SEOprofiler SalesIntelligent
ScanAlert
ScoutJet
Screaming
ScreenerBot
ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
SemrushBot SentiBot
Site Sucker SeoSiteCheckup
Siphon
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteLockSpider SiteSnagger
SiteSucker
Sitebeam
Siteimprove Siteimprove
Sitevigil SlySearch
SmartDownload
Snake
Snapbot Snapbot
SocialRankIOBot
Sociscraper Sociscraper
Sogou web spider Sogou web spider
Sosospider Sosospider
Sottopop Sottopop
SpaceBison
Spammen
Spinn3r Spinn3r
Sqlmap SurveyBot
Sucuri T8Abot
Teleport
TeleportPro
Telesoft
Telesphoreo
Telesphorep Telesphorep
TightTwatBot The Intraformant
Trendiction TheNomad
Trendictionbot Toweyabot
True_Robot True_Robot
Turingos Turnitin
Typhoeus TurnitinBot
TwengaBot
Twice
URLy Warning URLy Warning
VelenPublicWebCrawler Upflow
VB Project
Vagabondo
VidibleScraper VidibleScraper
Virusdie Virusdie
VoidEYE
Voil
Voltron Voltron
WISENutbot WEBDAV
WPScan
WWW-Collector-E
WWW-Mechanize
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE WeSEE
Web Collage
Web Enhancer Web Enhancer
Web Fetch
Web Fuck
Web Sauger Web Sauger
Web Sucker
WebAuto
WebBandit WebBandit
WebCopier WebGo IS
WebEnhancer WebReaper
WebImageCollector WebWhacker
WebPix Webalta
WebStripper WebmasterWorldForumBot
Webshag
Website Quester Website Quester
WebsiteExtractor
WebsiteQuester
Webster
Whack
Whacker
Who.is Bot
Widow
WinHTTrack WinHTTrack
Wonderbot WiseGuys Robot
Woobot
Wotbox Wotbox
Wprecon
Xenu Xenu
YoudaoBot Zade
Zermelo Zeus
Zitebot ZmEu
ZoomBot ZoomBot
ZoominfoBot archive.org_bot
ZumBot autoemailspider
check1.exe backlink-check
clark-crawler clark-crawler
crawler.feedback crawl.sogou.com
eCatch crawler4j
demandbase-bot
evc-batch
facebookscraper
gopher gopher
heritrix
internetVista monitor
ips-agent
isitwp.com
magpie-crawler
meanpathbot meanpathbot
mediawords
muhstik-scan muhstik-scan
oBot oBot
page scorer pcBrowser
polaris version probe-image-size
ripz
satoristudio.net
scalaj-http
scan.lol scan.lol
seoscanners serpstatbot
seostar siteripz
sogouspider sp_auditbot
sysscan
trendiction.com
trendiction.de
ubermetrics-technologies.com
webmeup-crawler
webpros.com
webprosbot webprosbot
xpymep1.exe x22Mozilla
zgrab

290
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
ADmantX Acunetix
Aboundex
AllSubmitter AllSubmitter
Alligator Anarchy
Anarchy99 Ankit
Anthill Anthill
AwarioRssBot Apexoo
Asterias
Attach
AwarioSmartBot
BBBike BBBike
BDFetch BackWeb
BLEXBot
Backlink-Ceck
Badass Badass
Bandit Bandit
Barkrowler BatchFTP
BetaBot Battleztar Bazinga
Brandwatch Bigfoot
Buck Bitacle
BuiltBotTough Black Hole
BuiltWith Blow
Bolt
Buddy
Bullseye Bullseye
BuzzSumo
CCBot
CODE87
CSHttp CSHttp
CazoodleBot CazoodleBot
Cegbfeieh
CensysInspect CensysInspect
CherryPicker ChinaClaw
Cocolyzebot Claritybot
Crescent Collector
CyotekWebCopy Craftbot
DatabaseDriverMysqli Crawling at Home Project
Demon CrazyWebCrawler
Digincore Curious
Custo
Deusu
Devil
Dirbuster Dirbuster
Disco Dispatch
DnyzBot DnyzBot
DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
Domains Project Domains Project
Dotbot
Download Wonder
Dragonfly Dragonfly
Drip EMail Siphon
Ebingbong Ebingbong
Ecxi EroCrawler
EirGrabber Express WebPictures
Evil
Exabot
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms Ezooms
FHscan FDM
FemtosearchBot
Fimap
Flunky
Freeuploader Freeuploader
Fyrebot FrontPage
GT::WWW
Genieo
GetRight GetRight
GetWeb GetWeb
Gigablast
Gigabot
Go!Zilla Go!Zilla
Grabber GoZilla
GrapeshotCrawler Gotit
GrapeFX
GridBot GridBot
HEADMasterSEO HEADMasterSEO
HTMLparser HMView
HTTrack HaosouSpider
Haansoft
Havij Havij
Humanlinks Humanlinks
HybridBot IDBTE4M
IRLbot
Iblog
Id-search Id-search
IndeedBot IlseBot
Intelliseek Image Fetch
Iria InfoNaviRobot
InfoTekies
Internet Ninja
JamesBOT JamesBOT
JennyBot
Jetty Jetty
JikeSpider
Joomla Joomla
Jorgee
JustView JustView
Keyword Density Keyword Density
Lanshanbot Kinza
LNSpiderguy
Leap Leap
LeechFTP LexiBot
LeechGet Lftp
LibWeb LibWeb
Libwhisker Libwhisker
LieBaoFast Likse
LinkextractorPro Linkbot
LinkpadBot LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey MJ12bot
Litemage_walker MSFrontPage
MQQBrowser
MSIECrawler MSIECrawler
Mag-Net Mag-Net
Magnet
Mail.RU_Bot
Majestic SEO
Majestic-SEO Majestic-SEO
Majestic12
MarkMonitor MarkMonitor
MauiBot
MeanPath Bot MeanPath Bot
Meanpathbot MegaIndex.ru
Metauri
Mister PiX Mister PiX
Moblie Safari
Mojeek
Mojolicious
Mozlila Mozlila
Mr.4x3 Mr.4x3
NICErsPRO
Name Intelligence Name Intelligence
Navroad Nameprotect
Needle NearSite
Nessus
Net Vampire
NetZIP NetZIP
Netcraft Netcraft
Nettrack
NextGenSearchBot
Nibbler Nibbler
Niki-bot
NimbleCrawler NimbleCrawler
Nimbostratus
Ninja
Nmap Nmap
Nuclei
Nutch Nutch
Offline Explorer
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS
Openfind Openfind
Openvas
OrangeBot OrangeBot
OrangeSpider OrangeSpider
PECL::HTTP PECL::HTTP
PageAnalyzer Page Analyzer
PageScorer PageScorer
PageThing.com
Pandalytics Pandalytics
Panscient Panscient
Papa Foto Picsearch
PeoplePal PleaseCrawl
Petalbot
PictureFinder
Pimonster
Pockey Pockey
Probethenet ProWebWalker
Psbot
Pu_iN Pu_iN
RSSingBot PxBroker
RankFlex QueryN Metasearch
RankurBot Quick-Crawler
RankingBot
RankingBot2
Re-re Re-re
RealDownload
Reaper Reaper
Recorder Recorder
RedesScrapy RedesScrapy
RepoMonkey RepoMonkey
RocketCrawler Rogerbot
SBIder
SEOkicks SEOkicks
SEOprofiler SEOprofiler
SalesIntelligent SMTBot
Scanbot
Scrapy Scrapy
Screaming
ScreenerBot
ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
Semrush Semrush
SentiBot SemrushBot
Seomoz SeoSiteCheckup
Shodan
Siphon
Site Sucker Site Sucker
SiteExplorer SiteCheckerBotCrawler
SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker SiteSucker
Sitebeam Sitebeam
Siteimprove Sitevigil
Snake SocialRankIOBot
Sociscraper
Sogou web spider
Sottopop Sottopop
SpaceBison
Spammen
SpankBot SpankBot
Sucker Spbot
Steeler
Sucuri
SuperBot SuperBot
SuperHTTP
Surfbot Surfbot
SurveyBot Szukacz
Swiftbot
T0PHackTeam T0PHackTeam
TeleportPro T8Abot
Telesoft Telesoft
Telesphoreo
Telesphorep
The Intraformant The Intraformant
TheNomad TheNomad
Thumbor Tracemyfile
TightTwatBot Trendictionbot
Toata True_Robot
Trendiction Turingos
Turnitin
Typhoeus Typhoeus
Upflow URLy.Warning
V-BOT V-BOT
Vacuum
Vagabondo
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie Virusdie
VoidEYE Voil
Voltron
WASALive-Bot WASALive-Bot
WEBDAV WBSearchBot
WISENutbot WPScan
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WeSEE
Web Collage
Web Pix Web Pix
Web Sucker WebAuto
WebBandit WebCopier
WebFuck WebEnhancer
WebGo IS
WebImageCollector
WebLeacher WebLeacher
WebSucker WebPix
WebReaper
WebSauger
Webalta
Webshag Webshag
Website Quester
WebsiteQuester WebsiteQuester
Webster Webster
Whatweb Whack
Who.is Bot WiseGuys Robot
Widow Wonderbot
Woobot Woobot
Wprecon
Xenu
Zauba
Zitebot Zitebot
ZoomBot ZoomBot
arquivo-web-crawler archive.org_bot
arquivo.pt
autoemailspider autoemailspider
backlink-check
cah.io.community
check1.exe check1.exe
clark-crawler crawl.sogou.com
crawler4j demandbase-bot
domainsproject.org
eCatch eCatch
heritrix gopher
internetVista monitor magpie-crawler
lwp-request
lwp-trivial
meanpathbot
muhstik-scan muhstik-scan
serpstatbot netEstate NE Crawler
sp_auditbot ripz
trendiction.de scalaj-http
ubermetrics-technologies.com seoscanners
voyagerx.com sitechecker.pro
webmeup-crawler spyfu
sysscan
trendiction.com
webpros.com webpros.com
zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
adman.se adimmix.com
ameblo.jp boom-celebs.com
elektrozigarette2.wordpressy.pl cheapmoncler.pw
essay-edu.biz fermersovet.ru
hoverboard360.nl haveinc.xyz
november-lax.com oops-cinema.ru
omgtnc.com paintingplanet.ru
positive2b.ru rusvideos.su
spravka-medosmotr.ru vote-up.ru
xblog.in wedding-salon.net

938
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
03e.info 0daymusic.org
101billion.com 1-free-share-buttons.com
123movies.love 1001desert.com
13tabs.com 2nt.ru
14b.info 3xforum.ro
21h2o.com 8lufu.com
273-fz.ru a.frcls.fr
2girls1cup-free.com aa9bd78f328a6a41279d0fad0a88df1901.com
76brighton.co.uk adultnet.in
a14download.com advokateg.ru
aa08daf7e13b6345e09e92f771507fa5f4.com agahinameh.com
aac52006ec82a24e08b665f4db2b5013f7.com agecheckadult.com
aaronabel.com ahrefs.com
abbanreddy.com alphahoverboards.com
abovetherivernc.com alphavisions.net
acads.net am15.net
acheterviagrafr24.com amateurgalls.com
actices.com amoi.tn
activepr.ru apiallgeniusinfo-a.akamaihd.net
admanaerofoil.com apiappenableinfo-a.akamaihd.net
adminshop.com appiq.mobi
ads-cool.pro apple.com-webbrowsing-security.science
adserver-e7.com artel116.ru
adsfresh.men atmovs.com
adtech.de autoloans.com
advancedcleaner.com autotrends.today
adventureparkcostarica.com aviav.eu
advokat-grodno.by back.dog
aghanyna.com baikaleminer.com
agreda.pluto.ro baladur.ru
ahrntal.verymes.xyz bang-hotties.com
albumsuper.info battlecarnival.su
albuteroli.com bavsac.com
alert-jdh.xyz bbsoldes.fr
aliexpresscashback.club bcmp.org
alkoravto.ru bedcapdealers.com
all-streaming-media.com bestdooz.com
amateurlivechat.org bg6s0.com
amazingpic.net bidr.trellian.com
an-donut.com billiard-classic.com.ua
apimossnetinfo-a.akamaihd.net bimatoprost-careprost.com
apioutoboxnet-a.akamaihd.net blackplanet.com
apispringsmartne-a.akamaihd.net blacktwhite.com
appfastplay.com blog.axant.it
arabseyes.com blogs.rediff.com
arclk.net bmw.afora.ru
arcteryxsale.online boanonihaca.tk
arenanews.com.ua boazpower.com
arkkivoltti.net boleznikogi.com
ascat.porn bombla.org
asiengirls.net bookhome.info
atovh.local-finders.com bosefux.esy.es
audiofree.ru bumskontakte.org
auto.rusvile.lt
autobudpostach.club
avon-ukraine.com
babyfactory.fr
bagcionderlab.com
bankiem.pl
batanga.net
bazaronline24.ru
beauby.ru
beauty-bracelet.com
beauty-clinic.ru
begalka.xyz
benchmarkcommunications.co.uk
best-coupon-offer.com
bestmouthwash.club
bestofferswalkmydogouteveryday.gq
bestsexyblog.com
bestwebsitesawards.com
betterscooter.com
bif-ru.info
bikini-image.com
bitcoinremote.com
blackcurranthumidifiers.site
blog.koorg.ru
bloke.com
bosman.pluto.ro
bradleylive.xyz
brains2.biz
braintobe.win
brandov.ru
brillianty.info
bscodecs.com
buntube.net
buqayy0.livejournal.com
businesn.men businesn.men
buyessaynow.biz buy-cheap-online.info
bvlgariwallet.xyz buy-forum.ru
californianews.cf buybest1.biz
canoncdriverq3.pen.io buypillsorderonline.com
canyougethighofftizanidine.blogspot.com buzzsumo.com
cartierreplica.top candypeople.se
cartujano-pre.de canrioloadm.gq
casablancamanor.co.za cardul.ru
cctva.tv cartiertrinity.xyz
cheapjerseysbizwholesale.us centrdebut.ru
checkhit.com cheapestjerseys-wholesale.com
chlooe.com chocolateslim-u-srbiji.com
ciceros.co cialisndbrx.com
ciekawinki.pl
cinemaflix.website
cityadspix.com cityadspix.com
clash-clans.ru cognitiveseo.com
clicksor.com colehaanoutlet.store
clksite.com compliance-checker-7.info
coaching-netz.info compliance-don.xyz
collegeessay19.blogspot.ru compliance-elena.xyz
comissionka.net conboy.us
compliance-alexa.xyz contentsexpress.com
compliance-barak.top
compliance-irvin.top
compliance-ivan.top
cookie-law-enforcement-aa.xyz
cookielawblog.wordpress.com
coprofam.org
cornomase.win
cosmeticswomens-womensports.rhcloud.com
costablue.xyz costablue.xyz
cottageofgrace.com
cougfan.info cougfan.info
creams.makeforum.eu covid-schutzmasken.de
customergrowthsystems.com creditnation.ru
cypernhuset.se customerguru.in
d0t.ru cwetochki.ru
d2jsp.org dawlenie.com
dandiyabeats.in decorazilla.com
datadepths.com dekoration.us
dating-time-now.com delitime.info
de.zapmeta.com design-ideas.info
decoratinghomes.ga detmebel.su
demon-tweeks.com detskie-zabavi.ru
dengi-pod-zalog-nedvizhimosti.ru deutschehobbyhuren.net
derevesendeco.com digital-video-processing.com
diminishedvalueoforegon.com directivepub.com
dirtpics.pw discovertreasure-a.akamaihd.net
discountliv.com
dnepropetrovsk.mistr-x.org
dnsrsearch.com
dokumentalkino.net dokumentalkino.net
domain-submit.info domainanalyzing.xyz
driving.kiev.ua domini.cat
drpornogratisx.xxx dominterior.org
dtm-spain.com download-of-the-warez.blogspot.com
dyt.net download-walpaperhd.blogspot.com
e-c.al downloaddy.net
earl-brown.info downloadeer.net
earn-from-articles.com downvids.net
earncash.com.ua drugspowerstore.com
earthmagic.info e-collantes.com
editmedios.com e-commerce-seo1.com
election.interferencer.ru e-kwiaciarz.pl
elektrischezigaretten1.blogse.nl e705.net
elektrozigarette1.dreamwidth.org eandsgallery.com
eloxal.ru easyukraine.com
energydiet-info.ru edudocs.net
eng-lyrics.com ekobata.ru
epicbrogaming.com erectile.bid
eropho.com eropho.com
etm-consult.de europeanwatches.ru
evogarage.com everypony.ru
exdocsfiles.com exct.net
extremez.net executehosting.com
eyes-on-you.ga extener.org
family1st.ca f07.de
fandlr.com face.hostingx.eu
fastfixing.tech facialporntube.com
feeriaclub.ru faithe.top
falcon-images.blogspot.com
familienzahnaerzte.com
fashionindeed.ml
fefo.gdn
fellowshipoftheminds.com fellowshipoftheminds.com
fifa-coins.online fighrofacciufreesig.ga
fikasound.tk fikasound.tk
fil.ru firma-legion.ru
filmfanatic.com fodelsedagspresenter.nu
financeloan.us foodcrafts.website
finansov.info forum20.smailik.org
findpics.pw fotoxxxru.com
fitness-video.net fr-bearings.ru
flashlarevista.com frbizlist.com
floating-share-buttons.com free-social-buttons.com
florida-tourism.net freshberry.com.ua
formulaantiuban.com fusoradio.info
forums.toucharcade.com fz139.ttk.ru
franch.info gambarxkata.co
free-floating-buttons.com
free-laptop-reward.com
freelifetimefuckbook.com
freemags.cc
freevpn.space
front.ru
front.to
fyxabomiw.ru
gallery.rennlist.com
gamebackyard.com gamebackyard.com
gamedayhouse.com gameprimary.com
gamerscorps.com gamesprite.me
gamingspark.com gay.adultgalls.com
gazporno.com gaygalls.net
ge0ip.org gaypornmovie.net
generousdeal-a.akamaihd.net gcup.ru
globalscam.ga gdzkurokam.ru
germetiki.com.ua
get-your-social-buttons.info
getmyads24.com
gigapeta.com
global-ics.co.za
go2mike.ru
gonextmedia.com
goodhousekeeping.com
goodnightjournal.com
goodwriterssales.com goodwriterssales.com
googlemare.com
googst2.ru
gopro-online.info gopro-online.info
gorabagrata.ru goroda-vsego-mira.ru
gov.yanao.ru gox.com.ua
gowreckdiving.com gpms.org.my
grand-chlen.ru greatzip.com
greatdealshop.com growboxbl.ru
gsasearchengineranker.top growmyfunds.ca
gtopstats.com guardlink.com
hamptonoaks.ca hanwei.us
handsandlegs.ru hatedriveapart.com
hawaiielectriclight.com hausfrauensex18.com
hd-filmy.net hccoder.info
hdpixion.com hdhc.ru
hermesbirkinhandbagoutlets.com heatpower.ru
hermesreplica.pw helicalpile.us
hit-men.men helvetia.com.ua
hjaoopoa.top hermesbracelets.xyz
hidefiles.org
home-task.com
homeandhealth.ru
homedo.fabpage.com
host-tracker.com host-tracker.com
hotblog.top hotel-mkad.ru
huimin764128.com howtoclean.club
hundejo.com https-legalrc.biz
hunterboots.online humanelydrew.com
hunthillfarmtrust.org ie.57883.net
husky-shop.cz iideaidekonkatu.info
ibb.com.ua imagez.co
igadgetsworld.com immobiliaremassaro.com
iklysha.ml impotentik.com
img.wallpaperstock.net inmate-locator.us
imguramx.pw intellego.info
imitex-plus.ru intimshop-fantasy.ru
indetiske.ya.ru invivo.hu
infobabki.ru isotoner.com
innodgfdriverhm.aircus.com ispaniya-costa-blanca.ru
intelhdgraphicsgtdrive6w.metroblog.com istizanidinelikexanax.blogspot.com
iqbazar.ru iwanttodeliver.com
iron-age.info izoll.ru
jjbabskoe.ru jabimgo.pw
jpcycles.com janavibekken.no
jurajskie.info japfm.com
kazrent.com javidol.site
keywordhouse.com juliadiets.com
keywordsking.com justkillingti.me
klikbonus.com kaidalibor.de
klin3952.ru kalandranis.gr
kometars.xyz karadene.com
konkursov.net kazinogames.lv
konoplisemena.com kinofak.net
konturkrasoty.ru knowsitall.info
kotaku.com kollesa.ru
kskjco.club konpax.com
kvartira-sutochno.com kopihijau.info
laapp.com kvartiry-remont.ucoz.ru
lampokrat.ws lacapilla.info
landoftracking.com lasvegaslockandsafe.com
landscapebackgrounds.blogspot.com lechenie-gemorroya.com
ldrtrack.com lenpipet.ru
lechtaczka.net lenvred.org
leftofcentrist.com linkredirect.biz
letslowbefast.site lion.snapmap.us
li-er.ru littleberry.ru
liffa.ru lockerz.com
lignofix.ua lonerangergames.com
lion.misba.us long-beach-air-conditioning.com
littlecity.ch lookingglassemb.com
lottospring.com lyrster.com
lovelycraftyhome.com m3gadownload.pl
luciddiagnostics.in m5home.ru
lulea-auktionsverk.se
m-google.xyz
machicon-akihabara.info machicon-akihabara.info
mainhp.net magda-gadalka.ru
magiadinamibia.blogspot.com
maju.bluesalt.co
managerpak204.weebly.com
marmitaco.cat
masserect.com
matb3aa.com
matematikus.info
matpre.top
mature.free-websites.com mature.free-websites.com
medmajor.ru mazda-roadsters.com
medtherapy.ru mebel-alait.ru
megagrabber.ru meteocast.net
megavolt.net.ua miniads.ca
mesbuta.info minyetki.ru
mesto-x.com mitrasound.ru
metarip.ru
metascephe.com
mhi-systems.ru
mirobuvi.com.ua
misswell.net
mixx.com mixx.com
moesexy.com mjchamonix.org
moncleroutletonline.pw mlf.hordo.win
monthlywinners.com mockupui.com
moomi-daeri.com mokrayakiska.com
musezone.ru mompussy.net
muz-baza.net movies-in-theaters.net
myfreemp3.eu musflashtv.com
mygameplus.com myanyone.net
mykings.pw mydeathspace.com
mylesosibirsk.ru mydirtyhobby.com
myonigroup.com myra.top
naperehresti.info nac-bearings.ru
nardulan.com nakozhe.com
net-radar.com nakrutka.cc
netanalytics.xyz narco24.me
new-apps.ru narkologiya-peterburg.ru
newfilmsonline.ru nastroyke.net
nextbackgroundcheck.gq naughtyconnect.com
nmrk.ru needtosellmyhousefast.com
noscrapleftbehind.co neodownload.webcam
novgorod.xrus.org net-profits.xyz
ntic.fr news-readers.ru
officedocuments.net nhl09.ru
ok-ua.info nhl17coins.exblog.jp
okuos.com nhl17coinsforps3.gratisblog.biz
omoikiri-japan.ru nicovideo.jp
one-gear.com nofreezingmac.click
onemactrckr.com nufaq.com
online-zaymy.ru nuker.com
online247.ml office2web.com
online7777.com okeinfo.online
onstrapon.purplesphere.in okout.ru
optibuymac.com onemantrip.com
oranga.host.sk onko-24.com
palvira.com.ua online-templatestore.com
paradontozanet.ru onload.pw
park.above.com oralsexfilme.net
partnerline.men otbelivanie-zubov.com
perederni.net ourtherapy.ru
perfectpracticeweb.com outpersonals.com
pic2fly.com outrageousdeal-a.akamaihd.net
piccshare.com paidonlinesites.com
pixell.club paparazzistudios.com.au
pizdeishn.com pastaleads.com
pl.aasoldes.fr pc-services.ru
plohaya-kreditnaya-istoriya.ru pendelprognos.se
pochemychka.net pestomou.info
pogruztehnik.ru pflexads.com
pokemon-go-play.online picswe.com
pornhubforum.tk picture-group.com
pornodojd.ru pinstake.com
pinup-casino1.ru
pirateday.ru
pisanieprac.info
pistonclasico.com
pk-pomosch.ru
pl-vouchers.com
plastweb.ru
playfortuna-play.ru
pobeiranie.pl
poem-paying.gq
pokerniydom.ru
popads.net
popander.mobi
porn-w.org
porndl.org
pornoblood.com
pornoklad.net pornoklad.net
pornosok.ru pornokorol.com
portnoff.od.ua porodasobak.net
potolokelekor.ru portadd.men
prom23.ru portside.cc
pron.pro pospr.waw.pl
privatov-zapisi.ru
prlog.ru
progress-upakovka.ru
pronekut.com pronekut.com
proposal-engine.com propranolol40mg.blogspot.com
prostitutki-tyumeni.org prosperent.com
pts163.ru pskcijdc.bloger.index.hr
putevka24.ru puzo2arbuza.ru
queerspace.com quickcashlimited.com
qwesa.ru r-control.ru
ranksonic.com rabotaetvse.ru
ranksonic.info ragecash.com
razamicroelectronics.com real-time-analytics.com
realting-moscow.ru realitykings.com
recipedays.com remont-fridge-tv.ru
registratciya-v-moskve.ru rfserial.net
registrydomainservices.com rialp.getenjoyment.net
registrysweeper.com rocketchange.ru
reklama1.ru roleforum.ru
remmling.de rucrypt.com
renecaovilla.online rus-teh.narod.ru
responsinator.com
restaurantlescampi.com
robot-forex.biz
ruscoininvest.company ruscoininvest.company
russian-postindex.ru ruspdd.com
sack.net sady-urala.ru
saitevpatorii.com
sampleletters.net sampleletters.net
scanner-donald.top sarahmilne.top
scanner-walt.top scanner-alexa.top
schalke04fc.info scanner-andrew.top
schoolfiles.net scanner-ivan.top
semrush.com scanner-jack.top
seofied.com scanner-mark.top
seoheap.com schlampen-treffen.com
serialsway.ucoz.ru search.xtconnect.com
serw.clicksor.com secret.xn--oogle-wmc.com
sex.hotblog.top security60-e.com
sexpartygirls.net seminarygeorgia59.ga
sexpornotales.com seo-2-0.com
sexsearch.com seo-tools-optimizing.com
sexstream.pl seobility.net
sexvporno.ru seorankinglinks.us
sexy-screen-savers.com seorankinglinks.xyz
shell-pmr.ru seotoolsagency.com
shemale-sex.net sergiorossistore.online
shiksabd.com sex-dating.co
ship-marvel.co.ua sex-sex-sex5.com
shmetall.com.ua sexspornotub.com
shopcheermakeup.info sfd-chess.ru
shopsellcardsdumps.com shikiso.info
silktide.com shopfishing.com.ua
sinestesia.host.sk shtaketniki.kz
sitevalued.com signx.info
sivs.ru sims-sims.ru
sklad-24.ru sinel.info
slomm.ru site5.com
smichovbike.cz sitopreferito.it
snomer1.ru sledstvie-veli.net
snyatie-lomki-v-stacionare.ru small-game.com
sofit-dmd.ru smstraf.ru
soserfis.com snworks.com
sportbetfair.com socialseet.ru
softxaker.ru
sohoindia.net
space-worry.ml
spb.ru
spravka130.ru spravka130.ru
sprttrack.com sprttrack.com
store-rx.com sr-rekneskap.no
strigkaomsk.ru srecorder.com
sukarame.net ssconstruction.co
superoboi.com.ua stariy-baku.com
svbur.ru statistici.ro
svensk-poesi.com studworks.org
surffoundation.nl
svetlotorg.ru svetlotorg.ru
swinon.site tabakur77.com
syvertsen-da.no takethatad.com
td-l-market.ru tattomedia.com
tecspb.ru tecnoteakviareggio.it
teenbbw.yopoint.in
telefonsexkostenlos.tk telefonsexkostenlos.tk
telegraf.by
telesvoboda.ru
thaismartloan.com thaismartloan.com
thetattoohut.com the-torrent-tracker.blogspot.com
thewomenlife.com thepornsex.org
tizanidinedosage.blogspot.com tizanidine4mg.blogspot.com
tizanidine4mgstreetprice.blogspot.com
tizanidine6mg.blogspot.com
tizanidinefibromyalgia.blogspot.com tizanidinefibromyalgia.blogspot.com
tizanidinehcl4mgtabinfo.blogspot.com
tizanidinehcl4mgtablet.blogspot.com
tizanidineingredients.blogspot.com tizanidineingredients.blogspot.com
tizanidinemusclerelaxant.blogspot.com tizanidinerecreationaluse.blogspot.com
tizanidineoral.blogspot.com tizanidinevssoma.blogspot.com
tomck.com tmm-kurs.ru
top-deal.com.pl tnctrx.com
top10-online-games.com torrent-newgames.com
topanasex.com toursmaps.com
topappspro.com
tovaroboom.vast.ru tovaroboom.vast.ru
tracfone.com tracksurf.daooda.com
traffic-club.info trafficmania.com
trion.od.ua trichizobswiv.agddns.net
tripper.de truck-addzilla.life
tt-ipd.info trustaffs.com
turbabitload.weebly.com turist-strani.ru
twu.com.ua tytoona.com
tx41tclega.ru u555u.info
txxx.com ucoz.ru
tyumen.xrus.org uggsale.online
underthesite.com uhod-za-sobakoj.ru
unlimitdocs.net umekana.ru
uptime-as.net universals.com.ua
ural-buldozer.ru url2image.com
userequip.com uzporno.mobi
vapsy.com vabasa.inwtrade.com
velen.io
vesnatehno.com vesnatehno.com
vezuviy.su videochat.bz
video--production.com videochat.ph
videojam.tv
videonsk.com videonsk.com
videospornogratisx.net vielporno.net
videotuber.ru vip-file.com
vietimgy.pw vip-parfumeria.ru
vinsit.ru vip2ch.com
vinylvault.co.uk vitanail.ru
vksex.ru vksaver-all.ru
vseuznaem.com vodkoved.ru
voluumtracker1.com
vriel.batcave.net
vrotike.ru
vseigru.one
vzlomfb.com vzlomfb.com
wdfdocando.com walkme.com
webextract.profound.net webads.co.nz
weblibrary.win webnode.me
weblo.com webshoppermac.com
webradiology.ru webstatsdomain.org
westvilletowingservices.co.za wedding-salon.net
withstandingheartwarming.com weddingdresses.xyz
word-vorlagen.xyz westermarkanjou.se
wowas31.ucoz.ru westum.se
www.wohnkabinen-shop.de whatsupinfoley.com
wygraj-teraz.com wholesalenfljerseys.us.com
xfire.com wildcattube.com
xlolitka.com win-spy.com
xmladserver.com wingsofrefuge.net
xmronta.com winner7777.net
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai winx-play.ru
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf woman-h.ru
xn----btbdvdh4aafrfciljm6k.xn--p1ai wordkeyhelper.com
xn--80aeahghtf8ac5i.xn--p1ai worldis.me
xn--d1acah0c.xn--p1ai wrona.it
xpresscare.ru ww1943.ru
wyniki-lista.pl
x-diesel.biz
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn----7sbaphztdjeboffeiof6c.xn--p1ai
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xn----ctbbcjd3dbsehgi.xn--p1ai
xn----itbkqkfiq.xn--p1ai
xxxtubesafari.com
xzlive.com xzlive.com
yaoguangdj.com yaminecraft.ru
yellowads.men yatrk.xyz
yginekologa.com yellowproxy.net
zagadki.in.ua youbrainboost.asia
zazagames.org youdesigner.kz
zebradudka.com yoursearch.me
zerocash.msk.ru yoursite.com
zeroredirect10.com yuarra.pluto.ro
zeroredirect5.com yunque.pluto.ro
zona-aqua.ru zaimite.ru
zoolubimets.ru zebramart.ru
zrizvtrnpale.tk zero1.it
zscaler.net zigarettenonlinekaufen.tumblr.com
zvooq.eu zmoda.hostreo.com
zzbroya.com.ua zojirushi-products.ru
zvuker.net
zx6.ru

490
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
101flag.ru 00it.com
123cha.com 10bet.com
214jaluwobapef.cf 6128786.com
3rbseyes.com 7fon.ru
3th.co.in abbanreddy.com
3w24.com absugars.com
4webmasters.com actices.com
98oi.ru actulite.com
abtasty.com ad-words.ru
acheterviagrafr24.com addblueoff.com.ua
adultactioncam.com adman.se
afmuseum.com allornamenti.com
ahrntal.verymes.xyz animaltoplist.com
alarmobninsk.ru ap.senai.br
alert-fdm.xyz apptester.tk
all-number.com auto-zapchasti.org
amung.us
animal36.com
artefakct.com
asociatia-tipografilor-transilvania.ro
asrv-a.akamaihd.net
asscenihotosidea.blogspot.com
autolombard-krasnodar.ru autolombard-krasnodar.ru
autoseo-traffic.com avon-severozapad.ru
aviapanda.ru becuo.com
azartmsl.com bestcurs.org
babyguns.ru beyan.host.sk
bagcionderlab.com bidr.trellian.com
balois.worldbreak.com biocn.dx.am
bankiem.pl biographiya.com
bannerswap.com bjetjt.com
beachdriveblog.com blackwitchcraft.ru
bestfortraders.com blockety.co
bg6s0.com blogos.kz
bizzliving.com blogqpot.com
blackplanet.com
bobba.dzaba.com bobba.dzaba.com
bobinoz.com braindaily.xyz
bochemit.com.ua burger-tycoon.com
boeuklad.com bus-offer.com
boobsimge.com buytizanidineonlinenoprescription.blogspot.com
bosefux.esy.es buyviagraa.blogspot.com
brainboostingsupplements.org camdenmemorials.com
braip.com.br cementaresearch.se
btc4u.ru christianlouboutinsaleonline.us
canacopegdl.com citizenclsdriveri7.pen.io
canrioloadm.gq claytransformations.info
cartierlove.xyz clothesforcash.com
cdncash.org com-onlinesupport.host
cenokos.ru coolgramgoods.com
chatroulette.online crackzplanet.com
check-host.net crazyprotocol.com
check-this-out-now.online
cherrypointplace.ca
chiptuninger.com
clickaider.com
clippingphotoindia.com
codq.info
compliance-jessica.top
concordexoticrentals.com
cpajunkies.com
cwetochki.ru cwetochki.ru
dating-app.ru damedingel.ya.ru
deadlinkchecker.com db.speedup.gdn
decorazilla.com deanmoore.ie
delta-line.men deda-moroza-zakaz.ru
deluxedumps.com deutschland123.de
distver.ru docs4all.com
dkvorota.ru docsportal.net
dumpsmania24.com domaincrawler.com
ebooknovel.club drunkenstepfather.com
ed-shop01.ru dtnlyss.com
empathica.com econom.co
esfchat.tk elektroniksigaraankara.info
essay-edu.biz eluxer.net
etm-consult.de emediate.eu
euromasterclass.ru ero-advertising.com
exlarseva.webblog.es escortslet.net
extremepornos.net everytalk.tv
f-online.de ezaz.info
failingmarriege.blogspot.com ezigarettekaufen.myblog.de
farmingworm.com ezigarettekaufen2.blox.pl
finstroy.net ezigaretteshop2.mywapblog.com
flashahead.info f07.de
floridamobilebillboards.com f0815.de
forensicpsychiatry.ru falopicm.pw
free-stock-illustration.com fast-wordpress-start.com
freecamdollars.com fastcrawl.com
freeuploader.com femmesdenudees.com
freewareseek.com fighrofacciufreesig.ga
fungamelands.com franch.info
funtoonez.com free-laptop-reward.com
gael-s.ru free-share-buttons.top
gamewrath.com free-video-tool.com
gamingspark.com freenom.link
gdebestkupit.ru frequiry.com
gdzkurokam.ru front.ru
get-your-social-buttons.info g7m.pl
girlgamerdaily.com gambarkatabaru.com
girls-ufa.ru geile-lelly.eu
gktt.ru genericlowlatencyasiodriverhq.aircus.com
gotcher.us golden-praga.ru
greatzip.com gombita.info
greendream.com.ua goodly.pro
hayate.biz greatgrace.ru
hcate.com gstatey.net
hetmanship.xyz handsandlegs.ru
himazin.info healmytrauma.info
homemade.gq hitcpm.com
homemature.net hoholikik.club
ico.re hoztorg-opt.ru
imperia31.ru htmlcorner.com
infektsii.com hyhj.info
intermesh.net i-midias.net.br
intervsem.ru ictizanidinehcl4mg.blogspot.com
itis4you.com ideibiznesa2015.ru
iweblist.info imagefinder.site
jewelryandfiligree.com investyb.com
jikoman.info javxxx18.com
jjbabskoe.ru johnrobertsoninc.com
job.icivil.ir kambasoft.com
jpcycles.com karachev-city.ru
kadashihotel.com kazan.zrus.org
kamagragelusa.net kellyonline.xyz
kichenaid.ru kino-key.info
klumba55.ru kiwi237au.tk
kosova.de kliksaya.com
kostenloser-sex.com kraljeva-sutjeska.com
kvartira-sutochno.com kunstaktien.info
lakomka.com.ua laapp.com
ledis.top law-check-seven.xyz
lerporn.info lescinq.com
likesuccess.com letmacworkfaster.site
liquimondo.com linkpulse.com
lodki-pvh.dp.ua liver-chintai.org
lookingglassemb.com loadingpages.me
maglid.ru lostaruban.com
maju.bluesalt.co macdamaged.tech
maranbrinfo.com.br machicon-ueno.info
mcnamaratech.com manipulyator-peterburg.ru
medtherapy.ru marcogrup.com
message-warning.net med-zdorovie.com.ua
mexicotravelnet.com metaxalonevstizanidine.blogspot.com
mobplayer.ru microsoftportal.net
monsterdivx.com middlerush-a.akamaihd.net
monsterdivx.tv mielec.pl
more-letom.ru mojowhois.com
mp3films.ru morocco-nomad-excursions.com
musezone.ru mosdverka.ru
musicdaddy.net my-bc.ru
my-big-family.com myblogregistercm.tk
mydoctorok.ru mypets.by
narkologiya-penza.ru nalogovyy-kodeks.ru
natali-forex.com
nationalbreakdown.com nationalbreakdown.com
nelc.edu.eg neks.info
netvouz.com nero-us.com
newstudio.tv netoil.no
nfljerseyscheapbiz.us next-dentists.tk
nhl17coinsforps3.gratisblog.biz nonameread45.live
nibbler.silktide.com nuker.com
offtopic.biz o-o-6-o-o.ru
ok-ua.info office-windows.ru
onlinedomains.ru olgacvetmet.com
onstrapon.purplesphere.in orion-v.com
openmultipleurl.com outpersonals.com
optitrade24.com outrageousdeal-a.akamaihd.net
ozoz.it ownshop.win
painting-planet.com paparazzistudios.com.au
pdns.cz paradontozanet.ru
photochki.com paydayloanslocal.com
photorepair.ru photo.houseofgaga.ru
photosaga.info picture-group.com
pornoforadult.com pinapchik.com
pornophoto.xyz pk-pomosch.ru
prize44.com pornmania.pl
proprostatit.com porno-chaman.info
prosmibank.ru pornodojd.ru
pulse33.ru pornokajf.com
rankscanner.com pornosemki.info
rating-casino2021.ru positive2b.ru
razamicroelectronics.com pregnant.guru
registratciya-v-moskve.ru privhosting.com
reimageplus.com profitwithalex.info
remorcicomerciale.ro pukaporn.com
rep-am.com qwarckoine.com
replicaclub.ru ragecash.com
rfserial.net rapevideosmovies.com
rightenergysolutions.com.au restaurantlescampi.com
santasgift.ml rethinkwasteni.info
scanner-margo.top rialp.getenjoyment.net
scopich.com ringtonepartner.com
screpy.com risparmiocasa.bz.it
seo-traffic-ranking.info sa-live.com
seoholding.com sabaapress.com
seorankinglinks.us salutmontreal.com
sex-foto.pw sanjosestartups.com
shell-pmr.ru sbetodiodnye-lampy.ru
siha.de scanner-donald.top
simplesite.com scanner-viktor.top
sledstvie-veli.net sdi-pme.com
socialbookmarksubmission.org semxiu.com
soochi.co seocheki.net
speeddream.xyz seoexperimenty.ru
sports-supplements.us serdcenebolit.com
sprttrack.com servicecenter.co.ua
stroicol.net sibtest.ru
sudexpert66.ru simpoed.ufop.br
sundrugstore.com sirpornogratis.xxx
superkanpo.com slavia.info
supernew.org social-fun.ru
svensk-poesi.com social-s-hhh.xyz
sygraem.com solicita.info
td-l-market.ru spabali.org
teguh.info sport7777.net
tengohydar.tk spy-sts.com
textads.men subj.ukr-lit.com
the-usa-games.blogspot.com success-seo.com
theallgirlarcade.com swiped.su
thecoral.com.br tacbelarus.ru
tizanidineandhydrocodone.blogspot.com tarad.com
tizanidineinfo.blogspot.com template-kid.com
tizanidineusedfor.blogspot.com terrafootwear.us
top250movies.ru thefds.net
topkarkas.com theplacetoupdating.pw
trucri.me tizanidineforopiatewithdrawal.blogspot.com
ural-buldozer.ru tizanidinesideeffectsweightloss.blogspot.com
us-america.ru tn811.us
valsalud.com tnctrx.com
vduplo.ru tongkatmadura.info
vvpg.ru track112.site
webcamdevochka.com trafaret74.ru
webmasterhome.cn trk-4.net
webs.com trkur.com
whiteelephantwellington.com tuberkulezanet.ru
writersgroup580.web.fc2.com tupper-shop.ru
wygraj-teraz.com tvand.ru
xblognetwork.com tversvet.ru
xfluro.com twu.com.ua
xmronta.com uhdtv.website
vacances-voyages.info
vegascosmetics.ru
via-energy-cumpara.com
videochat.tv.br
vikistars.com
vladimir.zrus.org
voprosotvet24.ru
vote-up.ru
vpnhowto.info
wallpapersdesk.info
wildcattube.com
williamrobsonproperty.com
ww1943.ru
x-diesel.biz
xn--80ahvj9e.xn--p1ai xn--80ahvj9e.xn--p1ai
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai xn--b1addnj3cah.xn--p1ai
xn--80atua3d.xn--p1ai xnxxandxvideos.com
xn--90acenikpebbdd4f6d.xn--p1ai yatrk.xyz
xn--h1ahbi.com.ua ynymnwbm.bloger.index.hr
yaaknaa.info youtuhe.com
your-bearings.com zazagames.org
yubikk.info zhacker.net
zawyna.ua zigarettenonlinekaufen1.bloog.pl
zocaparj.kz zone-kev717.info
zoompegs.com zoogdisany.com
zs2vm.top

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.129 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2554 ### Version: V4.2021.08.2555
### Updated: Sat Aug 28 10:04:02 UTC 2021 ### Updated: Sat Aug 28 22:01:30 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2554 ### Version: V4.2021.08.2555
### Updated: Sat Aug 28 10:04:02 UTC 2021 ### Updated: Sat Aug 28 22:01:30 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##