vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 11:10:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.06.3177 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-06-19 22:07:10 +00:00
parent cedaf813dd
commit 9e75373dc5
47 changed files with 81299 additions and 81292 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.06.3176 #### Version: V4.2022.06.3177
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.06.3176 #### Version: V4.2022.06.3177
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.06.3176 #### Version: V4.2022.06.3177
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10016
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.06.3176 #### Version: V4.2022.06.3177
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10020
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10020
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10020
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10020
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.24 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.24 0;
127.0.0.1 0; 127.0.0.1 0;

10020
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.24 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.24 0;
127.0.0.1 0; 127.0.0.1 0;

10020
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.24 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.24 0;
127.0.0.1 0; 127.0.0.1 0;

10020
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3176 3177

10020
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10020
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.122.13.233 0; 20.38.6.136 0;

10020
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.24 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.24 0;
127.0.0.1 0; 127.0.0.1 0;

10020
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.24 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.24 0;
127.0.0.1 0; 127.0.0.1 0;

10020
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.24 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.24 0;
127.0.0.1 0; 127.0.0.1 0;

10020
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.122.13.233 0; 20.38.6.136 0;

10020
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.122.13.233 0; 20.38.6.136 0;

10020
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 1; 10.1.0.24 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
ExtLinksBot AllSubmitter
GalaxyBot BackWeb
LeechFTP Chlooe
PHPCrawl Jorgee
SBIder Kozmosbot
ScanAlert Musobot
Spammen Pixray
T0PHackTeam Trendictionbot
TheNomad VeriCiteCrawler
Thumbor satoristudio.net

310
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 80legs
404enemy ADmantX
Acunetix ALittle Client
AiHitBot AhrefsBot
Aipbot AllSubmitter
AlphaBot AlphaBot
Ankit Anthill
Aspiegel Aspiegel
Asterias Asterias
Attach Atomseobot
AwarioSmartBot AwarioSmartBot
BDFetch BLEXBot
BackStreet Badass
BackWeb
Barkrowler
BetaBot BetaBot
Bigfoot
Bitacle Bitacle
BlackWidow Black Hole
Blackboard
Blow
BlowFish
Bolt
BotALot
Brandprotect Brandprotect
BuiltBotTough Buck
CATExplorador BuiltWith
Bullseye
BuzzSumo
CCBot
CODE87
CSHttp
Cegbfeieh Cegbfeieh
CheTeam CheTeam
CherryPicker ChinaClaw
Chlooe Cloud mapping
Citoid Copier
Claritybot
Cogentbot
CopyRightCheck
Copyscape Copyscape
Cosmos
Craftbot
Curious Curious
CyotekWebCopy
DBLBot DBLBot
DIIbot
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Demon
Deusu
Devil
Disco Disco
Discobot Discoverybot
DittoSpyder DnBCrawler-Analytics
DomainAppender DomainAppender
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot Domains Project
ECCP/1.0 Download Wonder
Dragonfly
Drip
EMail Siphon EMail Siphon
EMail Wolf
Ebingbong Ebingbong
Ecxi Ecxi
EirGrabber EroCrawler
Extractor Exabot
ExtractorPro FDM
Extreme Picture Finder FHscan
Ezooms Fimap
FrontPage Flunky
FyberSpider Foobot
Fyrebot Freeuploader
G-i-g-a-b-o-t GT::WWW
GetRight Gigabot
Go!Zilla Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla
Gotit Gotit
Grabber Grabber
Grafula
GrapeFX GrapeFX
GridBot GridBot
Hloader HMView
HTMLparser
HaosouSpider
Harvest
HonoluluBot
Humanlinks Humanlinks
HybridBot
IDBTE4M
IDBot
Iblog Iblog
IlseBot IlseBot
Image Fetch Image Fetch
Image Sucker
Indy Library
Intelliseek Intelliseek
Iskanie Iskanie
JamesBOT IstellaBot
Jbrofuzz JOC Web Spider
JennyBot
JetCar JetCar
Jetty Jetty
Jyxobot Joomla
JustView
Kenjin Spider
Lanshanbot
Leap Leap
LeechGet
LexiBot LexiBot
Lftp LieBaoFast
Likse Likse
LinkScan
LinkWalker LinkWalker
LinkextractorPro LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey Lipperhey
Ltx71
MFC_Tear_Sample
MIIxpc MIIxpc
MJ12bot
MSIECrawler
MTRobot MTRobot
Magnet
Majestic-SEO
Majestic12
MarkMonitor
Mass Downloader
Mata Hari
Mb2345Browser Mb2345Browser
MeanPath Bot MeanPath Bot
Microsoft URL Control Mediatoolkitbot
Metauri
MicroMessenger
Mister PiX
Moblie Safari Moblie Safari
MolokaiBot
Morfeus Fucking Scanner
Mozlila Mozlila
NICErsPRO
Navroad Navroad
Needle Nessus
Net Vampire
NetLyzer
NetMechanic NetMechanic
NetSpider
Nettrack Nettrack
Netvibes Netvibes
Nibbler Nikto
Niki-bot
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Ninja Nmap
Octopus Not
Nutch
Offline Explorer
OpenVAS
Openfind Openfind
Openvas
OrangeBot OrangeBot
OrangeSpider PageScorer
OutclicksBot
Page Analyzer
PageGrabber
Pandalytics Pandalytics
Papa Foto Panscient
Pavuk
Petalbot
Pi-Monster Pi-Monster
Picsearch
PictureFinder
Pimonster
Pixray Pixray
ProPowerBot PleaseCrawl
Pockey
Psbot Psbot
Pump Pump
QueryN Metasearch
Quick-Crawler
RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankFlex
RankingBot2 RankingBot2
RealDownload Re-re
RebelMouse ReGet
Ripper Recorder
RedesScrapy
Rogerbot Rogerbot
SBIder
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOstats Scanbot
SISTRIX Scrapy
SalesIntelligent ScrepyBot
ScanAlert
Searchestate
SearchmetricsBot
Seekport Seekport
Semrush
SemrushBot
Seomoz Seomoz
Siphon
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
Snake SiteSnagger
Sitevigil
Snapbot Snapbot
Snoopy Sosospider
SocialRankIOBot Spammen
Sottopop
SpankBot
Sqlworm Sqlworm
Sqworm Steeler
Stripper Sucker
Sucuri Sucuri
SuperBot Surfbot
SurveyBot
Suzuran Suzuran
Szukacz Swiftbot
T0PHackTeam T8Abot
Telesoft Teleport
Telesphorep Telesphorep
TheNomad The Intraformant
Thumbor
Titan
Toata
Toweyabot Toweyabot
Trendictionbot Tracemyfile
True_Robot Turingos
Turnitin Twice
URLy Warning
URLy.Warning URLy.Warning
Vacuum UnisterBot
VeriCiteCrawler Vagabondo
VidibleScraper VelenPublicWebCrawler
VoidEYE
Voil Voil
Voltron WBSearchBot
WEBDAV
WISENutbot WISENutbot
WPScan
WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WallpapersHD WWW::Mechanize
Web Auto WWWOFFLE
Web Enhancer Wallpapers/3.0
Web Fetch
Web Fuck
Web Sauger
WebBandit WebBandit
WebCollage
WebCopier
WebEnhancer
WebFetch WebFetch
WebReaper WebGo IS
WebZIP WebLeacher
WebPix
WebSauger
WebSucker
WebmasterWorldForumBot
Webshag Webshag
WebsiteExtractor Website Quester
WebsiteQuester
Webster
Whack Whack
Whacker Whacker
Whatweb Who.is Bot
WiseGuys Robot Widow
Wonderbot
Woobot Woobot
Wprecon Wotbox
Xaldon WebSpider Xaldon WebSpider
Xenu Xenu
YoudaoBot
Zermelo Zermelo
Zeus adscanner
ZoominfoBot arquivo-web-crawler
ZyBorg
arquivo.pt arquivo.pt
autoemailspider autoemailspider
cognitiveseo backlink-check
com.plumanalytics crawler.feedback
crawl.sogou.com crawler4j
dataforseo.com demandbase-bot
domainsproject.org
eCatch eCatch
facebookscraper evc-batch
gopher
heritrix heritrix
instabid ips-agent
isitwp.com iubenda-radar
linkdexbot mediawords
oBot muhstik-scan
netEstate NE Crawler
page scorer page scorer
ripz pcBrowser
scan.lol polaris version
seobility probe-image-size
sexsearcher
sitechecker.pro sitechecker.pro
sp_auditbot sogouspider
sysscan sysscan
tAkeOut trendiction.com
trendiction.de
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webmeup-crawler
x09Mozilla x09Mozilla
xpymep1.exe

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 80legs
ADmantX ADmantX
AIBOT
ALittle Client ALittle Client
ASPSeek
Abonti Abonti
Aboundex
Aboundexbot
Acunetix Acunetix
Anarchy AiHitBot
Anthill AllSubmitter
Aspiegel AlphaBot
Ankit
Asterias Asterias
Attach Attach
AwarioSmartBot AwarioSmartBot
BBBike BBBike
BDFetch
BLEXBot
BackDoorBot
BackStreet BackStreet
BackWeb BacklinkCrawler
Badass Bandit
Barkrowler BatchFTP
Battleztar Bazinga BetaBot
Black Hole Black Hole
Blow
Bolt Bolt
Brandprotect BotALot
Buck
BuiltBotTough BuiltBotTough
BunnySlippers BuiltWith
BuzzSumo CATExplorador
CCBot CCBot
CODE87 CSHttp
CazoodleBot Calculon
Cegbfeieh
CensysInspect
CheeseBot CheeseBot
ChinaClaw ChinaClaw
Citoid Citoid
Cogentbot Claritybot
Cliqzbot
Copier Copier
CopyRightCheck Copyscape
Cosmos Cosmos
Crawling at Home Project Crawling at Home Project
CrunchBot
Curious
Custo Custo
CyotekWebCopy DIIbot
DBLBot DSearch
DatabaseDriverMysqli
Demon Demon
Deusu Deusu
Devil Devil
Digincore DittoSpyder
DigitalPebble DomainAppender
Dirbuster DomainSigmaCrawler
DnBCrawler-Analytics
DomainStatsBot
Domains Project
Drip Drip
Ebingbong ECCP/1.0
EirGrabber EasyDL
EroCrawler EroCrawler
Evil ExtLinksBot
Exabot Extreme Picture Finder
ExtractorPro
Ezooms Ezooms
FemtosearchBot FHscan
Flunky Firefox/7.0
FrontPage FlashGet
FyberSpider Fuzz
Fyrebot Fyrebot
G-i-g-a-b-o-t G-i-g-a-b-o-t
GermCrawler Getintent
Gigabot Gigabot
Go!Zilla Go!Zilla
Go-Ahead-Got-It
GoZilla
GrabNet GrabNet
GrapeshotCrawler Grabber
HTTP::Lite Grafula
HTTrack GrapeFX
GridBot
HEADMasterSEO
HMView
Haansoft Haansoft
HaosouSpider
Havij
HonoluluBot HonoluluBot
IDBTE4M IDBTE4M
IDBot IDBot
IlseBot Id-search
Image Fetch Image Fetch
Image Sucker
Indy Library
InfoNaviRobot
Intelliseek
InterGET InterGET
Iria
Iskanie Iskanie
JetCar IstellaBot
JamesBOT
JikeSpider
Joomla Joomla
JustView Jorgee
Lanshanbot Jyxobot
Kenjin Spider
Keybot Translation-Search-Machine
LWP::Simple
Larbin Larbin
Leap Leap
LeechFTP Lftp
LibWeb LibWeb
LieBaoFast
Lightspeedsystems Lightspeedsystems
Likse Likse
Linkbot LinkextractorPro
LinkpadBot LinksManager
LinqiaScrapeBot LinqiaMetadataDownloaderBot
Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
Ltx71 Ltx71
MFC_Tear_Sample MIDown tool
MJ12bot MJ12bot
MQQBrowser Mag-Net
MSFrontPage Magnet
MTRobot Mail.RU_Bot
Majestic-SEO MarkMonitor
Majestic12
Mass Downloader Mass Downloader
Masscan Meanpathbot
Mediatoolkitbot
Metauri Metauri
Minefield MicroMessenger
Mister PiX Microsoft Data Access
Moblie Safari Moblie Safari
Mojeek Mojeek
Mojolicious
MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mr.4x3 Msrabot
NICErsPRO
Nameprotect Nameprotect
NearSite
NetLyzer
NetMechanic NetMechanic
NetSpider NetSpider
Netcraft
Nettrack
Netvibes Netvibes
NextGenSearchBot Nikto
Nibbler
Niki-bot
Ninja Ninja
Nutch Not
Openvas Offline Explorer
OutclicksBot Offline Navigator
OutfoxBot OnCrawl
OpenVAS
OrangeSpider
PageAnalyzer PageAnalyzer
PageScorer
PageThing.com PageThing.com
Papa Foto Papa Foto
PeoplePal PleaseCrawl
Pi-Monster Pockey
Picscout Psbot
Picsearch
PictureFinder
Piepmatz
Pimonster
ProPowerBot
ProWebWalker
Pu_iN
PxBroker
Quick-Crawler Quick-Crawler
RankFlex RSSingBot
RankActive
RankingBot
RankingBot2 RankingBot2
RepoMonkey Rankivabot
Ripper Ripper
SEOkicks Rogerbot
SEOlyticsCrawler SBIder
SEOkicks-Robot
SEOstats SEOstats
SMTBot
ScoutJet ScoutJet
Screaming Screaming
Searchestate ScrepyBot
SearchmetricsBot
SemrushBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot SeobilityBot
Shodan SiteExplorer
Siphon
Site Sucker
Sitebeam Sitebeam
Sitevigil Sitevigil
SlySearch SmartDownload
Snake Snake
Snapbot SpankBot
Snoopy Spanner
Sociscraper Spinn3r
Sogou web spider
Sottopop
Spammen
SputnikBot
Sqworm Sqworm
SuperBot Steeler
Sucuri
SuperHTTP
Surfbot
SurveyBot SurveyBot
Suzuran Suzuran
Szukacz
T0PHackTeam
Teleport Teleport
TeleportPro TeleportPro
TheNomad Telesphoreo
Tracemyfile Thumbor
True_Robot Toweyabot
Trendictionbot
Turingos Turingos
Turnitin Turnitin
TurnitinBot TurnitinBot
URLy Warning
UnisterBot UnisterBot
Upflow
VCI
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
Virusdie
VoidEYE VoidEYE
WASALive-Bot
WEBDAV
WWWOFFLE WWWOFFLE
WeSEE Wallpapers/3.0
WallpapersHD
Web Auto
Web Collage Web Collage
Web Fuck Web Fetch
Web Pix
Web Sauger
WebAuto
WebBandit
WebCollage
WebCopier WebCopier
WebEnhancer WebEnhancer
WebFuck WebFuck
WebGo IS WebSucker
WebImageCollector
WebStripper
WebWhacker WebWhacker
WebZIP WebmasterWorldForumBot
Website Quester Website Quester
WebsiteQuester
Webster
Whack
Who.is Bot
Widow
WinHTTrack
WiseGuys Robot WiseGuys Robot
Wotbox Wotbox
Xenu Wprecon
YoudaoBot Xaldon WebSpider
Zermelo Zade
Zitebot
ZmEu
ZoominfoBot
adscanner adscanner
arquivo.pt
autoemailspider
check1.exe check1.exe
coccocbot
cognitiveseo cognitiveseo
crawler.feedback
crawler4j crawler4j
demandbase-bot dataforseo.com
evc-batch eCatch
heritrix gopher
instabid
ips-agent
isitwp.com
iubenda-radar
lwp-trivial lwp-trivial
muhstik-scan magpie-crawler
netEstate NE Crawler mediawords
plumanalytics page scorer
seobility pcBrowser
s1z.ru
seoscanners
sexsearcher sexsearcher
sitechecker.pro sitechecker.pro
sogouspider sogouspider
sysscan sp_auditbot
tAkeOut tAkeOut
trendiction.com
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com webgains-bot
webprosbot webmeup-crawler
zauba.io webpros.com
xpymep1.exe
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
adidas.frwebs.fr atmagroup.ru
crynet.cc christianlouboutinsaleoutletonline.us
drillsaw.ru easyshoppermac.com
edwinkonijn.com.au facecup.top
forumprofi.de hawaiisurf.com
konturkrasoty.ru invivo.hu
oktube.ru natali-forex.com
svolze.com onlyforemont.ru
tizanidineonline.blogspot.com piluli.info
topcasinoratings.ru ugguk.online

940
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0c47f8422d3f.com 1webmaster.ml
1-99seo.com 3-letter-domains.net
101billion.com 3rbseyes.com
101raccoon.ru
123any.com
1bet.com
1xbet4you.com
24x7-server-support.site
2girls1cup-free.com
2itech.ru
2kata.ru
2rich4bitches.com
3dracergames.com
4kepics.com 4kepics.com
4ureyesonly.com 55wmz.ru
5000-cotydzien.com 7ooo.ru
777-club.ru a.pr-cy.ru
7a2rnuey1tw9ar.ru abbanreddy.com
8kisses.eu absolute-s.ru
aad1f4acb0a373420d9b0c4202d38d94fa.com actices.com
aarbur.com
aasoldes.fr
activecampaign.dreamhosters.com activecampaign.dreamhosters.com
acunetix-referrer.com addbin.men
addray.pro adloads.com
adktrailmap.com admeasures.com
admatic.com.tr adrenalin-stk.ru
adminshop.com adsloads.com
adrunnr.com adultfriendfinder.com
adtology.com advokateg.xyz
advancedmassagebysara.com afftrack001.com
adviceforum.info agysya.ru
akama.com aliexpresscashback.club
alessandraleone.com all-streaming-media.com
alfa9.com allesohnegirls.net
alibestsale.com alloysteel.ru
alienwheels.de amyfoxfitness.com
allbizne.co.ua android-style.com
allornamenti.com androids-store.com
alphahoverboards.com animal-drawings.com
amateurgalls.com apartamentwroclaw.eu
amateurlivechat.org apimegabrowsebiz-a.akamaihd.net
anaksma.info apparel-offer.com
anaseracresar.tk applicationg29.com
animal36.com arcteryxstore.online
anjalika.co.in arkartex.ru
antons-transporte.de arquivo.pt
apartmentratings.com artel116.ru
apiallgeniusinfo-a.akamaihd.net asociatia-tipografilor-transilvania.ro
appartement-stumm.at audiofree.ru
ardimobileinfo.ml ausergrubhof.info
arenda-avtoprokat-krasnodar.ru autochoixspinelli.com
arkkivoltti.net autotrends.today
artparquet.ru aviav.ru.com
asscenihotosidea.blogspot.com avkzarabotok.info
autobrennero.it avtocredit-legko.ru
autolombard-krasnodar.ru azadnegar.com
avon-severozapad.ru babs.com.ua
avto-oligarh.ru
balla-boo.se
bankmib.ru
bankofthewext.com bankofthewext.com
banksrf.ru bear.gotcher.us
best-seo-solution.com beauty-things.com
bezlimitko.xyz belsetirehafi.tk
big-cash.net bestbrainprod.win
bioca.org bestpornuha.com
biocn.dx.am bfz.biz
biznesluxe.ru bidr.trellian.com
bjorkbacken.nu blackbabesporn.com
blogig.org blog.axant.it
blogos.kz bloggers.nl
blogrankers.com blogporn.in
bluesalt.co boc.kz
boazpower.com bocoarchives.org
bobinoz.com bodybuilding-shop.biz
boleznikogi.com bongacams.com
bolezniorganov.ru bookmaker-bet.com
brains2.biz boom-celebs.com
brainzpod.win brus.city
breget74.ru buqayy0.livejournal.com
btvn.ru buy-cheap-pills-order-online.com
cah.io.community buybest1.biz
callstevens.com buyessaynow.biz
candycrushshop.com buynorxx.com
cars-modification.net buzz-porno.info
cartierbraceletsreplica.pw buzzsumo.com
cartierlovestore.xyz bvlgariring.xyz
cartierring.xyz c-gainsbourg.com
callejondelpozo.es
carstrends2015.com
cartierreplica.win
cb.iphantom.com
cbb1smartlist12.click
cdncash.com
cfsrating.sonicwall.com
chatroulette.world
cheap-pills-norx.com
cheapmarmot.online cheapmarmot.online
cheapwebsitehoster.com cherubinimobili.it
cheatcode-lita12.rhcloud.com cialischstgerts.com
chinacheapelitejerseys.com cialisovercounteratwalmartusa.com
chocolateslim-original.com cialiswithoutadoctor.net
christianlouboutinreplica.win claytransformations.info
christianlouboutinsaleonline.us cleansearch.net
cjmc.info clickbanksites.info
clickzzs.nl clodo.ru
clothesforcash.com cntravelre.com
coderstate.com coindirect.io
codq.info collegerentals.com
compliance-checker.info compliance-alex.top
compliance-fred.top compliance-checker-7.info
compliance-fred.xyz compliance-jane.top
compliance-julianna.top
compliance-nelson.top
connectingsingles.com
continent-e.tv continent-e.tv
coolwallpapers4k.info cookie-law-enforcement-hh.xyz
cordstrap.cc coolgamechannel.com
cosmeticswomens-womensports.rhcloud.com copypaste.traderzplanet.in
covadhosting.biz counter.bloke.com
crafthubs.com counterbot.com
credit-respect.ru courtshipgift.com
creditwell.ru crackguru.tk
customerpromos-a.akamaihd.net crirussian.ru
cutt.us cryptoswap.biz
cv.wallhade.co cwetochki.ru
cyclobenzaprinevstizanidine.blogspot.com daretodonate.co
damasarenai.info darkages.info
data.vtc.pw datadepths.com
den-noch24.ru dating-time-now.com
designdevise.com davidovic.info
digilander.libero.it dealitright.click
digital-video-processing.com dev33.dioniqlabb.se
dinosaurus.site diarioaconcagua.com
dktr.ru dlya-android.org
dogbreedspicture.net dnepropetrovsk.mistr-x.org
domini.cat dodlive.mil
dotnetdotcom.org dokfilms.net
drugs-no-rx.info doktoronline.no
dscaas.website domain.webkeyit.com
dstroy.su domain2008.com
dynainbox.com dominterior.org
easyshoppermac.com dosug-lux.ru
ebooknovel.club dosugrostov.site
eets.net doyouknowtheword-flummox.ml
elektronischezigarette2.mex.tl drivotracker.com
elektronischezigarettekaufen1.beeplog.com dumuelave.xyz
embedle.com dvrlists.com
enbersoft.com e-collantes.com
eropho.com earl-brown.info
estibot.com editmedios.com
exportshop.us educatemetv.com
ekaterinburg.xrus.org
election.interferencer.ru
elektrischezi.canalblog.com
elektrozigarette2.wordpressy.pl
elektrozigarettekaufen1.devhub.com
elektrozigarettekaufen2.blogse.nl
elitesportsadvisor.com
enews.tech
englishgamer.com
eredijovon.com
eropho.net
erotiktreff24.info
essaypro.com
euronis-free.com
euroskat.ru
exci.ru
expresstoplivo.ru
extener.org
extlabs.io
extremez.net extremez.net
eywords-monitoring-your-success.com ezigarettekaufen2.yolasite.com
ezb.elvenmachine.com falopicm.pw
ezigarettekaufen2.mpbloggar.se fba-mexico.com
ezrvrentals.com fdzone.org
facialporntube.com
faithe.top
faracontrol.ir
favorcosmetics.com
feel-planet.com feel-planet.com
felizporno.com femmesdenudees.com
fil.ru feorina.ru
financetip.eu fickenprivat.info
floridahuntingfishingadventures.com fickkontakte.org
fotosfotos.eu fifa-coins.online
fototravel.eu filesvine.com
foxtechfpv.com findsexguide.com
freenode.info formularz-konkurs.tk
frequiry.com forpackningsutveckling.se
fres-news.com forsex.info
forums.d2jsp.org
frbizlist.com
free411games.com
freelinkbuilding.website.tk
freemags.cc
freewebs.com
freza-sverlo.ru
fsalas.com fsalas.com
fullfileaccess.com
fusoradio.info fusoradio.info
fxgallery.com gallery.rennlist.com
gagrasector.ru gardene.ru
galaxyflowers.ru gaytube.com
gambarxkata.co ge0ip.net
gelezki.com gearsadspromo.club
gemgrab-a.akamaihd.net geckoandfly.com
girlgamerdaily.com gentamicineyedrops.blogspot.com
goforexvps.com gilsonchiro.xyz
goodhousekeeping.com glass-msk.ru
googlemare.com glcomputers.ru
googleseo.com.tr gogalleryawesome.com
gomusix.com
gonextmedia.com
google-liar.ru
goosefishpost.bid
gosmeb.ru
gosreg.amchs.ru
graphics8.info graphics8.info
gwhwpxbw.bloger.index.hr greatfind-a.akamaihd.net
handicapvansarea.com green-tea.tv
hannasolution.ru grtyi.com
hanwei.us gsasearchenginerankerdiscount.com
hatedriveapart.com handicapbathtubarea.com
hdhc.ru havepussy.com
hdpixent.com hayate.biz
heartofbeijing.blogspot.com hdapp1008-a.akamaihd.net
heliko.no hdimagelib.com
hermesbirkinhandbagoutlets.com heartofpayne.xyz
hobbyhuren24.net hollywoodactress.info
homegardenlova.com host-protection.com
homemature.net hotdl.in
hostnow.men hothor.se
howtowhitenteethfast.xyz hotloans.ru
hut1.ru hoverboardforsaledirect.com
idegenvezeto.eu huhn.altervista.org
images-graphics-pics.com hundejo.com
img.wallpaperstock.net iblogpress.xyz
imperiafilm.ru ilmexico.com
infobabki.ru imagefinder.site
infobanks.ru imfamous.info
imicrovision.com
imk.com.ua
importchinacoach-teach.com
impotentik.com
infogame.name infogame.name
infostatsvc.com interesnie-faktu.ru
inmate-locator.us iradiology.ru
innoslicon.com javstock.com
insta-add.pro jjbabskoe.ru
istizanidineanopiate.blogspot.com jobgirl24.ru
istock-mebel.ru jofucipiku.tk
iwantmyfreecash.com joy-penguin.com
jagg.info kanimage.com
jaxcube.info keywordhouse.com
jrcigars.com keywordsdoctor.com
juliaworld.net
kam-dom.ru
karadene.com
kasino-money.pw
kazan.xrus.org
kchaxton.com
keywordsking.com keywordsking.com
kfon.eu keywordsuggest.org
kimcurlrvsms.com kicknights.gq
kinogolos.ru kiinomaniak.pl
kinohit1.ru kino-filmi.com
kinosed.net kinogonew.ru
kletkimehan.ru kmd-pto.ru
krasnodar.zrus.org konpax.com
kreativperlen.ch kostenlos-sexvideos.com
kredit-blog.ru kountrylife.com
kustanay.kz krasivoe-hd.com
laexotic.com kurwa.win
lanadelreyfans.us lacave.ntic.fr
landscapebackgrounds.blogspot.com lapitec.eu
laxob.com lechenie-gemorroya.com
liky.co.ua link.ac
linkbolic.com linkbux.ru
lion.conboy.us linkredirect.biz
lion.snapmap.us lisque.batcave.net
lionking-1994.blogspot.com littleberry.ru
lmrauction.com locksmith.jp
lomza.info lomza.info
lonely-mature.com lostfilm-online.ru
lotto6888.com
low-format.ru
lowephotos.info
lyngdalhudterapi.no lyngdalhudterapi.no
m0r0zk0-krava.ru macfix.life
m1media.net machicon-akihabara.info
mackeeper-center.club magicalfind-a.akamaihd.net
magicplayer-s.acestream.net make-money-online.com
maladot.com mamasuper.prom.ua
manyresultshub-a.akamaihd.net maridan.com.ua
maranbrinfo.com.br marketingtechniques.info
marblestyle.ru meds-online24.com
marmotstore.online meendo-free-traffic.ga
martlinker.com merryhouse.co.uk
matematikus.info methodsmarketing.com
mbiologi.ru michaelkorssaleoutletonline.net
mesbuta.info midst.eu
mex-annushka.ru mir-limuzinov.ru
mexicotravelnet.com misswell.net
minneapoliscopiers.com mmgq.ru
mlf.hordo.win mojaocena.com
mmoguider.ru mompussy.net
mobicover.com.ua monarchfind-a.akamaihd.net
mobsfun.net monitorwebsitespeed.com
mockupui.com most-kerch.org
money-for-placing-articles.com mp3films.ru
moneymaster.ru mrpornogratis.xxx
morf.snn.gr mttwtrack.com
moroccosurfadventures.com muizre.ru
moscow-region.ru mural.co
motherboard.vice.com musflashtv.com
musicvidz.ru mustwineblog.com
mwtpludn.review muycerdas.xxx
my-cash-bot.co muztops.ru
mydirtystuff.com mycouponizemac.com
mylesosibirsk.ru mykings.pw
mysexpics.ru mypets.by
naphukete.ru mywallpaper.top
narkologiya-voronezh.ru nakrutka.cc
neks.info narutonaruto.ru
neobux-bg.info naval.jislaaik.com
netvouz.com nerudlogistik.ru
newstraveller.ru nmrk.ru
newstudio.tv no-fuel.org
nfljerseysforsalewholesaler.com no-rx.info
nikitabuch.com novosti-hi-tech.ru
nippon-bearings.ru nucia.biz.ly
noumeda.com o-o-8-o-o.com
novatech.vn obiavo.kz
nudepatch.net obnal.org
oballergiya.ru
obiavo.uz
odywpjtw.bloger.index.hr
of-ireland.info
offf.info offf.info
okeinfo.online okonich.com.ua
okel.co olvanto.ru
omgtnc.com omoikiri-japan.ru
oneminutesite.it onclickpredictiv.com
onetravelguides.com online-templatestore.com
onlinebay.ru online-x.ru
onthemarch.co onlineslotmaschine.com
ooo-gotovie.ru openmultipleurl.com
optitrade24.com osagonline.ru
osnova3.ru panamaforbeginners.com
ostrovtaxi.ru panicatack.com
ownshop.cf par-fallen.ga
oz-offers.com parajumpersjakkesalgnorge.info
palocco.it paramountmarble.co.uk
pc-services.ru patol01.pw
pertlocogasilk.tk pay2me.pl
petitions.whitehouse.gov perm-profnastil.ru
photorepair.ru perosan.com
picturesfrom.com pillscheap24h.com
pihl.se pintattoos.com
pk-services.ru pizda.lol
pl.id-forex.com pl-vouchers.com
playlott.com poisk-zakona.ru
pony-business.com popcash.net
popserve.adscpm.net popup-fdm.xyz
poquoson.org poquoson.org
pornhubforum.tk pornhive.org
pornhub-ru.com
porno-dojki.net porno-dojki.net
porno-play.net porno-video-chati.ru
porno-raskazy.ru pornosexrolik.com
porno.simple-image.com.ua porodasobak.net
pornoplen.com pozdrawleniya.ru
pornpost.in
portnoff.od.ua
portside.xyz
poshiv-chehol.ru
pospr.waw.pl
pozdrawleniya.com
preg.marketingvici.com preg.marketingvici.com
pretty-mart.com pretty-mart.com
pricheski-video.com priceg.com
prizesbook.online princevc.com
produkto.net printie.com
professionalwritingservices15.blogspot.ru privatefx.all4invest.info
prom23.ru privatov-zapisi.ru
promgirldresses.xyz prod2016.com
proprostatit.com profit-opportunity.com
prostitutki-yaroslavlya.org puzo2arbuza.ru
pssucai.info pxhdwsm.com
qld10000.net r-e-f-e-r-e-r.com
qpypcx.com ranksignals.com
quangcaons.com razamicroelectronics.com
rapevideosmovies.com remybutler.fr
rapidgator-porn.ga replicaclub.ru
ratemodels.net responsinator.com
rebuildermedical.com restorator-msk.ru
registrationdomainsite.com rollercoin.com
reimageplus.com rossanasaavedra.net
reklama-i-rabota.ru rus-pornuha.com
remontgruzovik.ru russian-postindex.ru
rentaremotecomputer.com sale-japan.com
rewardpoll.com saneyes.com
rfd-split.hr sarm.tk
rockma.se sbricur.com
roma-kukareku.livejournal.com sdi-pme.com
rome2rio.com seadragonherbery.com
romhacking.ru search.pch.com
royalvegascasino.com seccioncontrabajo.com
rp9.ru seeresultshub-a.akamaihd.net
rrutw.com semalt.com
ru-mediaget.ru seo-tools-optimizing.com
runofilms.ru
runstocks.com
russia-today-video.ru
rustic-quiver.win
rusvideos.su
rutor.group
saneitconsulting.com
santechnik.jimdo.com
sape.top
sbdl.no
sbf441.com
scanner-brian.top
scanner-jane.top
scanner-julianna.top
scanner-margo.top
scanner-walter.top
scanner-willy.top
scmor.ilxc.cc
searchkut.com
seorank.info
seorussian.ru seorussian.ru
seotoolsagency.com serialsway.ucoz.ru
service-core.ru serving.adbetclickin.pink
sexpartygirls.net sexvporno.ru
shakhtar-doneck.ru shakhtar-doneck.ru
shell-pmr.ru shillyourcoins.com
sibvitr.ru shoppingjequiti.com.br
sigmund-freud.co.uk simple-image.com.ua
siteaero.com sitevalued.com
skatestick.bid sitopreferito.it
skanninge.se
sladkoevideo.com sladkoevideo.com
smart-scripts.com slowmacfaster.trade
smichovbike.cz snip.to
sneakyboy.com snomer1.ru
soaksoak.ru snts.shell-pmr.ru
socas.pluto.ro social-vestnik.ru
socialmadesimple.com sofit-dmd.ru
socialtrade.biz
soft-terminal.ru soft-terminal.ru
songoo.wz.cz softtor.com
sribno.net space-worry.ml
store-rx.com spammen.de
stroydetali.ru spasswelt.xyz
superkanpo.com srecorder.com
supermesta.ru statoutlook.info
support.nopeas.sk stmassage.ru
sureone.pro stockquotes.wooeb.com
survival.betteroffers.review streetfooduncovered.com
svtrd.com stretchingabuckblog.com
swimpool.ca studworks.org
sukirgenk.dvrlists.com
sunflowerdrawingpaintings.blogspot.com
supers.com.ua
svetlotorg.ru
swsociety.se swsociety.se
takeflyte.com syndicate.fun
tam-gde-more.ru teletype.in
tamada69.com thegameriders.com
taqplayer.info thelottosecrets.com
taranerymagesswa.blogspot.com timeallnews.ru
td-l-market.ru titangel-vietnam.com
teenbbw.yopoint.in tizanidine4mgstreetprice.blogspot.com
telefonsexkostenlos.tk tizanidinehydrochloride4mgstreetvalue.blogspot.com
templates.franklinfire.co tizanidinevssoma.blogspot.com
texbaza.by toonfamilies.net
theporndude.com top10registrycleaners.com
tizanidinecost.blogspot.com topbestgames.com
tizanidinehydrochloride2mg.blogspot.com topclickguru.com
tizanidinepills.blogspot.com topvidos.ru
tizanidinerecreationaluse.blogspot.com totu.us
topanasex.com toursmaps.com
topcar-krasnodar.ru toyota.7zap.com
topflownews.com track.deriv.com
torrents-tracker.com traderzplanet.in
trailer.cinemaflix.website trafficcentr.xyz
truck-land.life trafficgenius.xyz
tryrating.com traffique.net
tt-ipd.info trichizobswiv.agddns.net
tubeoffline.com trumpetedextremes.com
twu.com.ua tube8.com
ucoz.ru turizm.bz
usacasino.com tvorozhnaja-zapekanka-recept.ru
usbggettwku.ga tvteleport.ru
ustion.ru twinderbella.com
vacuumcleanerguru.com umnovocaminho.com
valuado.com unimodemhalfduplefw.pen.io
vellings.info usdx.us
velobikestock.com valmetrundan.se
vandrie-ict.nl
veselokloun.ru veselokloun.ru
video-chat.cn viddyoze.com
video--production.com
video-chat.in video-chat.in
video-hollywood.ru
videochat.world videochat.world
videonsk.com videos-for-your-business.com
visionwell.com.cn videositename.com
vitalads.net villakohlanta.nu
vksaver-all.ru vip-file.com
volume-pills.biz vipromoffers.com
vpnhowto.info visitcambridge.org
vse-pesni.com vodaodessa.com
vvpg.ru vseigry.fun
wallpapers-best.com
wallpapersdesk.info
wallpapersist.com
wallpaperstock.net wallpaperstock.net
wareseeker.com watracker.net
warning.or.kr webix.me
waysbetter.cn weblibrary.win
webcamdevochka.com website-analyzer.info
weblo.com webstatsdomain.org
webradiology.ru whatistizanidine2mg.blogspot.com
westvilletowingservices.co.za wholesalenfljerseys.us.com
wheelchairliftsarea.com wildnatureimages.com
wladimirpayen.com wineration.com
wnhjavlhezp.gq winx-play.ru
woman-orgasm.ru witclub.info
workle.website wnoz.de
worldmusicfests.com works.if.ua
woweb.com.ua worst-sites.online
wstroika.ru ww2awards.info
www.souvenirua.com x-diesel.biz
wygraj-teraz.com xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
x-lime.com xn----btbdvdh4aafrfciljm6k.xn--p1ai
xaijo.com xn----ctbigni3aj4h.xn--p1ai
xep.info xn----itbkqkfiq.xn--p1ai
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xn--d1acah0c.xn--p1ai
xn--80aafb2a.xn--p1ai xn--lifehacer-1rb.com
xn--80aanaardaperhcem4a6i.com xwatt.ru
xn--b1addnj3cah.xn--p1ai xxart.ru
xn--e1aggki3c.xn--80adxhks youbrainboost.asia
yaminecraft.ru youthreaders.com
yellowfootprints.com yunque.pluto.ro
yoopsie.com zagadki.in.ua
youbloodyripper.com zapiszto.pl
yoursearch.me zarajbuilders.com
yousense.info zarepta.com
zaimite.ru zebradudka.com
zakazvzloma.com zerocash.msk.ru
zeroredirect7.com
zeroredirect9.com zeroredirect9.com
zetgie.com.pl zetmaster.ru
zigarettenonlinekaufen.tumblr.com zigarettenonlinekaufen.tumblr.com
zoogdiesney.com zigarettenonlinekaufen1.bloog.pl
zrizvtrnpale.tk zirondelli.it
zskdla.site

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
0912701309f8ce.com 1hwy.com
1j7740kd.website 2345.com
2girls1cup-free.com 300richmond.co.nz
3weekdiet.com 4ur.click
5i2.net 57883.net
7yue.org 7search.com
8kisses.eu DomainStatsBot
abcguru.xyz aa08daf7e13b6345e09e92f771507fa5f4.com
absolutelyanalog.com acunetix-referrer.com
absugars.com advertisingtag.net
adman.se
adpremium.org
adult3dgames.com
adviceforum.info adviceforum.info
affrh2015.com algarveglobal.com
agadelha.com.br alkoravto.ru
alborzan.com analytics-ads.xyz
albuteroli.com ananumous.ru
alert-hgd.xyz aosexkontakte.net
allsilver925.co.il aosheng-tech.com
alot.com apimountainbikei-a.akamaihd.net
amateurmatch.com
android-style.com
angkortours.vn
animal36.com
anonymousfox.co
apisecretsaucebi-a.akamaihd.net
app-ready.xyz app-ready.xyz
apple.com-cleaner.systems artpicso.com
arvut.org
atout-energie-69.com
auto.rusvile.lt
autodo.info
aviav.co aviav.co
awency.com ayerbo.xhost.ro
best-deals-products.com azlex.uz
best-seo-software.xyz balla-boo.se
bestprofits-there.com belstaffstore.online
bio.trade-jp.net berrymall.ru
blagovest-med.ru bestdraws.com
blog.axant.it bestofferswalkmydogouteveryday.gq
blogos.kz beyan.host.sk
blogs.rediff.com bingo8888.com
blue-square.biz bitcoins-live.ru
bluesalt.co bmw-ark.ru
braip.com.br bolegapakistan.com
brandimensions.com bonkers.name
bukleteg.xyz boostmyppc.com
bumskontakte.org buntube.net
buybest1.biz call-of-duty.info
bwlx.prepedu.cn cars-modification.net
candcstuccoandstone.com cheapestjerseysonwholesale.com
candycrushshop.com children-learningreading.info
canrole.com clothesforcash.com
cardsharp1.ru codysbbq.com
cartierlove.xyz coin-hive.com
certifywebsite.win commoncrawl.org
cheapestjerseys-wholesale.com companies-catalog.com
check-this-out-now.online compliance-alexa.top
chorus.terakeet.com compliance-jessica.top
christianlouboutinoutlet.win compliance-margo.top
cialisndbrx.com crafthubs.com
compiko.info culturevie.info
compliance-mary.top
connectingsingles.com
darodar.com
day-news.info
daydream-studio.ru daydream-studio.ru
dbmkfhqk.bloger.index.hr dbmkfhqk.bloger.index.hr
dfiles.me dcj-nn.ru
dfwu1019.info dekoration.us
dimkino.ru diegolopezcastan.com
drivotracker.com digest-project.ru
dvr.biz.ua digitalassetmanagement.site
dvrlists.com dignitasdata.se
el-nation.com divatraffic.com
elektrozigarettekaufen2.blogse.nl domain-tracker.com
elitesportsadvisor.com domain.webkeyit.com
emailaccountlogin.co domaincheck.io
enpolis.ru domznaniy.ru
everypony.ru dorratex.tn
executehosting.com download-wallpaper.net
extlinks.com dwomlink.info
f-online.de e705.net
familienzahnaerzte.com elektrozigaretten1.tumblr.com
fickenprivat.info en.altezza.travel
filmania-x.ru engines-usa.com
filmidivx.com englishgamer.com
finstroy.net erotiktreff24.info
flipper.top ertelecom.ru
forex-procto.ru esnm.ru
forminecrafters.ru extener.org
freemags.cc eyessurgery.ru
fxgallery.com ezigarettekaufen.myblog.de
getaclueamerica.com ezigarettekaufen2.blox.pl
getpopunder.com face.hostingx.eu
gg-arena.ru fastfixing.tech
gheus.altervista.org filesclub.net
girlsatgames.ru firesub.pl
gogps.me flytourisme.org
goodhousekeeping.com footbalive.org
guardlink.org free-floating-buttons.com
hamptonoaks.ca free411games.com
havepussy.com freenom.link
health-medical-portal.info fun2cell.net
hkdiiohi.skyrock.com fyxabomiw.ru
hostcritique.com games.kolossale.ru
houston-vikings.com gateway.zscalerone.net
humanorightswatch.org generalporn.org
hundejo.com getmyads24.com
i-hobot.ru goldadpremium.com
images-graphics-pics.com goodprotein.ru
imgarit.pw gstatey.net
imicrovision.com gwagka.com
interferencer.ru happy.new.yeartwit.com
iqoption.com help.tpu.ru
jolic2.com hmywwogw.bloger.index.hr
kasino-money.pw home-task.com
keenoutlet.online horoshieokna.com
kellyonline.xyz hplaserjetpdriver8y.pen.io
keywords-monitoring-your-success.com ibb.com.ua
keywordsuggest.org ido3.com
kinogonew.ru igithab.com
kirov.zrus.org ikritikimou.gr
kollekcioner.ru ilo134uloh.com
koronirealestate.gr ilovevitaly.ru
krassh.ru indianmedicaltourismshop.com
kursy-ege.ru indo-export.ru
lacasamorett.com internet-apteka.ru
letmacwork.world itag.pw
letmacworkfaster.site itis4you.com
linkpulse.com izhevsk.zrus.org
live-sexcam.tk jerseyswholesalechinalimited.com
mabdoola.blogspot.com jo24news.com
mackeeper-land-672695126.us-east-1.elb.amazonaws.com joessmogtestonly.com
macnewtech.com karusel-market.ru
mainlinehobby.net kino-ecran.ru
maladot.com kino-fun.ru
marketingvici.com kiwi237au.tk
masserect.com kolotiloff.ru
meds-online24.com konkursowo-24.pl
mistr-x.org kvartir-remont.biz
mmstat.com lalalove.ru
mobilemedia.md lida-ru.com
moesonce.com localmatchbook.com
moivestiy.biz lockerz.com
mojowhois.com lulea-auktionsverk.se
mossmesi.com lyrics.home-task.com
mostantikor.ru m4ever.net
mototsikl.org magicdiet.gq
mountainstream.ms mansiondelrio.co
muz-tracker.net marcoislandvacations.net
negociosdasha.com marketland.ml
nordstar.pro mcadamssupplyco.com
mebel-alait.ru
mecash.ru
mediaoffers.click
meget.co.za
metabar.ru
migente.com
moetomnenie.com
moncleroutletonline.xyz
muabancantho.info
music.utrolive.ru
myanyone.net
mydoctorok.ru
myphotopipe.com
nrv.co.za
o00.in
obnal.org obnal.org
onlineporno.site oneminutesite.it
orion-v.com onlineslotmaschine.com
orsonet.ru paparazzistudios.com.au
pandarastore.top parfusale.se
pdn-4.com paydayloanslocal.com
physfunc.ru perfection-pleasure.ru
pio.polytopesexempt.com perfectpracticeweb.com
pizda.lol platinumdeals.gr
pomoc-drogowa.cba.pl polska-poezja.com
popmarker.com porndroids.com
porndairy.in pornobrazzers.biz
pornoforadult.com pornozhara.com
preconnubial.usuby.site preg.marketingvici.com
predmety.in.ua pro-tec.kz
printingpeach.com profitkode.com
prlog.ru prokotov.com
pts163.ru pronekut.com
quickcashlimited.com propranolol40mg.blogspot.com
qwarckoine.com proprostatit.com
raisedseo.com pussysaga.com
rankscanner.com quick-offer.com
ranksonic.com radiodigital.co
runofilms.ru randalljhoward.com
sampleletters.net raschtextil.com.ua
scanner-josh.top razorweb-a.akamaihd.net
seasaltwithfood.com realizmobi.com
semp.net redhotfreebies.co.uk
seo-prof1.xyz remont-comp-pomosh.ru
seolab.top reward-survey.net
serdcenebolit.com rialp.getenjoyment.net
service.adtech.us rixpix.ru
sexy-pings.com rniaeba.ga
sexytrend.ru ruf777.com
sibvitr.ru rutor.vip
sigmund-freud.co.uk saitevpatorii.com
siteonomy.com sanidumps.com
sky-mine.ru savememoney.co.za
slutloadlive.com scanner-barak.top
smartbalanceworld.com scanner-marwin.top
social-buttons.com scansafe.net
solmarket.by secretscook.ru
sonnikforme.ru securityallianceservices.com
speechfoodie.com seeingmeerkat.com
spidtest.space sei80.com
stop-zavisimost.com serving.adbetclickin.pink
streamin.to sexphoto.site
streetfooduncovered.com sexpornotales.net
stroimajor.ru shitmovs.com
subj.ukr-lit.com shopwme.ru
superlist.biz signoredom.com
supervesti.ru siteaero.com
support.nopeas.sk sitechecker.pro
swsociety.se small-game.com
symbaloo.com snsdeainavi.info
tecspb.ru spywarespy.com
sukarame.net
superfish.com
teenbbw.yopoint.in teenbbw.yopoint.in
templates.franklinfire.co thefotosgratis.eu
thecoral.com.br theprofitsmaker.net
thegamerznetwork.com toproadrunner5.info
theguardlan.com topsy.com
theporndude.com torrents.life
tizanidine4mguses.blogspot.com tsyndicate.com
tizanidinenarcotic.blogspot.com tula.howotorg.ru
tonivedu.it unacittaconte.org
toon-families.com unece.org
topads.men uptimebot.net
trafficshaper.com uptimechecker.com
traffpartners.com url-img.link
trustl.life velobikestock.com
tvteleport.ru vepad.com
uggbootsoutletsale.us versaut.xxx-cam.webcam
usswrite.com vipromoffers.com
vbtracker.net warezkeeper.com
veles.shop waterpurifier.club
viagra-soft.ru webextract.profound.net
videochat.cafe welck.octopis.com
videochat.mx whengirlsgowild.com
videokrik.net wholinkstome.com
vod.com.ua womens-journal.net
voronezh.xrus.org wpsecurity.website
wanker.us writingservices17.blogspot.ru
web-analytics.date www.souvenirua.com
where-toget.com
wordpress-crew.net
www.timer4web.com
wwwadultcheck.com
x-diesel.info x-diesel.info
xn--80ab4aa2g.xn--p1ai xtraffic.plus
xpresscare.ru xtubeporno.net
yadro.ru xxxasianporn.net
youthreaders.com xxxmania.top
zajm-zalog-krasnodar.ru yatrk.xyz
zarenica.net yes-com.com
zeleznobeton.ru youdao.com
zx6.ru your-bearings.com
youtubedownload.org
zaimite.ru
zeg-distribution.com
zhuravlev.info

153
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,79 +8,6 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -494,6 +421,76 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -584,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.134 0; 10.1.0.24 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.06.3176 ### Version: V4.2022.06.3177
### Updated: Sun Jun 19 10:03:19 UTC 2022 ### Updated: Sun Jun 19 22:01:22 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.06.3176 ### Version: V4.2022.06.3177
### Updated: Sun Jun 19 10:03:19 UTC 2022 ### Updated: Sun Jun 19 22:01:22 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##