vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-06 20:50:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.10.2649 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-10-13 22:09:39 +00:00
parent b086278b48
commit 9ec996bbbe
47 changed files with 67870 additions and 67860 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2648 #### Version: V4.2021.10.2649
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2648 #### Version: V4.2021.10.2649
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2648 #### Version: V4.2021.10.2649
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8348
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2648 #### Version: V4.2021.10.2649
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8352
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8352
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8352
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8352
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 1; 10.1.1.135 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 0; 10.1.1.135 0;
127.0.0.1 0; 127.0.0.1 0;

8352
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 1; 10.1.1.135 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 0; 10.1.1.135 0;
127.0.0.1 0; 127.0.0.1 0;

8352
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 1; 10.1.1.135 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 0; 10.1.1.135 0;
127.0.0.1 0; 127.0.0.1 0;

8352
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2648 2649

8352
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8352
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.69.94.2 0; 13.67.200.226 0;

8352
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 1; 10.1.1.135 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 0; 10.1.1.135 0;
127.0.0.1 0; 127.0.0.1 0;

8352
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 1; 10.1.1.135 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 0; 10.1.1.135 0;
127.0.0.1 0; 127.0.0.1 0;

8352
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 1; 10.1.1.135 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 0; 10.1.1.135 0;
127.0.0.1 0; 127.0.0.1 0;

8352
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.69.94.2 0; 13.67.200.226 0;

8352
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.69.94.2 0; 13.67.200.226 0;

8352
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 1; 10.1.1.135 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AfD-Verbotsverfahren Alexibot
AwarioSmartBot BuzzSumo
Custo DigitalPebble
DIIbot Go!Zilla
IlseBot GoZilla
Iskanie LNSpiderguy
IstellaBot LinqiaRSSBot
JOC Web Spider Ltx71
LinkScan Wotbox
MarkWatch magpie-crawler

308
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 80legs
AIBOT ASPSeek
ALittle Client Aboundexbot
Aboundex
AhrefsBot AhrefsBot
AiHitBot AiHitBot
Alexibot Aipbot
Anarchie Alligator
AlphaBot
Anarchy Anarchy
Anarchy99 Anarchy99
Ankit
Anthill Anthill
Aspiegel
Asterias Asterias
AwarioSmartBot AwarioRssBot
BBBike BLEXBot
BDFetch BackDoorBot
BackStreet BackStreet
BackWeb BackWeb
Bandit BacklinkCrawler
Badass
Barkrowler Barkrowler
Battleztar Bazinga Battleztar Bazinga
Black Hole Bigfoot
Blackboard
Blow Blow
BlowFish BlowFish
BotALot Bolt
Brandwatch Brandprotect
BuiltBotTough Buck
BuzzSumo BuiltWith
Bullseye
CODE87
CensysInspect CensysInspect
CheeseBot CheeseBot
CherryPicker CherryPicker
Chlooe Chlooe
Cloud mapping Collector
Cocolyzebot
Cogentbot
Copier Copier
Cosmos CopyRightCheck
Crawling at Home Project Copyscape
CrazyWebCrawler Craftbot
CrunchBot CrunchBot
Curious
Custo Custo
DBLBot CyotekWebCopy
DIIbot
DSearch
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Demon Deusu
DnyzBot Digincore
DomCopBot DigitalPebble
DomainAppender Dirbuster
Discoverybot
Domains Project Domains Project
Dotbot Dotbot
Dragonfly Dragonfly
EMail Wolf EMail Siphon
EasyDL
Ebingbong Ebingbong
EroCrawler Ecxi
Evil
Exabot Exabot
ExtLinksBot ExtLinksBot
Extractor Extractor
ExtractorPro
Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms Ezooms
FemtosearchBot FHscan
Fimap Fimap
Firefox/7.0 Freeuploader
FlashGet
Foobot
Fuzz Fuzz
FyberSpider FyberSpider
G-i-g-a-b-o-t GT::WWW
GetWeb GalaxyBot
Getintent Genieo
Go!Zilla Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GrabNet GrabNet
Grabber Haansoft
GrapeFX
GrapeshotCrawler
HMView
HTMLparser
HTTrack
Harvest
Hloader
Humanlinks Humanlinks
HybridBot IRLbot
IDBot
Iblog Iblog
Image Sucker Id-search
InfoTekies Image Fetch
JOC Web Spider InfoNaviRobot
Intelliseek
InterGET
Internet Ninja
InternetSeer
IstellaBot
Jbrofuzz
Jetty
JikeSpider JikeSpider
Jorgee
JustView JustView
Jyxobot
Kenjin Spider
Keyword Density
Kozmosbot Kozmosbot
LNSpiderguy
LeechFTP LeechFTP
LexiBot LibWeb
LinkScan LieBaoFast
Linkbot Lightspeedsystems
LinkpadBot LinkWalker
LinqiaRSSBot LinqiaRSSBot
Lipperhey Spider LinqiaScrapeBot
Litemage_walker Lipperhey
Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MQQBrowser MIDown tool
MJ12bot
MSFrontPage MSFrontPage
MTRobot
Mag-Net Mag-Net
Magnet
Mail.RU_Bot
Majestic-SEO Majestic-SEO
Majestic12
MarkMonitor
MarkWatch MarkWatch
Mata Hari Masscan
Mb2345Browser MeanPath Bot
MegaIndex.ru Meanpathbot
Microsoft Data Access Mediatoolkitbot
Mister PiX MicroMessenger
Mojolicious Mojeek
NPbot Msrabot
Name Intelligence NICErsPRO
NearSite Needle
NetSpider NetMechanic
Netcraft
Nettrack
Netvibes Netvibes
NextGenSearchBot
Nibbler Nibbler
Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Ninja Ninja
Nmap Nmap
Nuclei
Nutch Nutch
Offline Explorer Octopus
OpenVAS OpenLinkProfiler
OrangeBot OrangeBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP Panscient
Page Analyzer
PageAnalyzer
PageGrabber
Papa Foto
Pavuk Pavuk
Petalbot Picscout
Picsearch Picsearch
PictureFinder Piepmatz
Pixray ProPowerBot
PxBroker Probethenet
Pu_iN
Pump
QueryN Metasearch QueryN Metasearch
Quick-Crawler RankActive
Re-re RankActiveLinkBot
RankingBot2
ReGet ReGet
Recorder Reaper
RepoMonkey RepoMonkey
Ripper RocketCrawler
Rogerbot SBIder
SEOkicks-Robot SEOlyticsCrawler
SEOprofiler
SEOstats
SISTRIX
SMTBot SMTBot
SalesIntelligent SalesIntelligent
ScanAlert ScanAlert
Scanbot Scanbot
ScoutJet ScoutJet
Screaming Scrapy
Searchestate Semrush
SeobilityBot SentiBot
Siphon Seomoz
SiteSnagger Site Sucker
SiteLockSpider
SiteSucker
Sitebeam Sitebeam
Siteimprove Snake
SmartDownload
Snapbot
Snoopy Snoopy
SocialRankIOBot Spammen
Sogou web spider
Sottopop
SpaceBison
SpankBot SpankBot
Spbot
SputnikBot
Sqlmap Sqlmap
Sqworm Sqlworm
Steeler
Sucker
Sucuri Sucuri
Surfbot
Suzuran Suzuran
Swiftbot Szukacz
T8Abot TeleportPro
Telesoft
Telesphoreo Telesphoreo
Telesphorep
TheNomad TheNomad
Toata Titan
Toweyabot
Trendiction Trendiction
TwengaBot TurnitinBot
VeriCiteCrawler Twice
Virusdie UnisterBot
V-BOT
Vacuum
Voil Voil
WISENutbot Voltron
WPScan WEBDAV
WWW-Collector-E WWW-Mechanize
WWW::Mechanize Wallpapers
WallpapersHD Web Auto
Web Pix Web Enhancer
Web Sauger
WebAuto
WebBandit WebBandit
WebCollage WebCollage
WebEnhancer WebEnhancer
WebReaper WebLeacher
WebPix
WebStripper WebStripper
WebZIP Webshag
Webalta Website Quester
WebmasterWorldForumBot
WebsiteExtractor WebsiteExtractor
Webster
Whack
Whatweb Whatweb
Widow Widow
WinHTTrack WinHTTrack
Wonderbot
Woobot Woobot
Wprecon
Xaldon WebSpider Xaldon WebSpider
YoudaoBot Xenu
Zauba Zade
Zitebot Zitebot
ZumBot ZoomBot
archive.org_bot ZyBorg
arquivo-web-crawler cah.io.community
autoemailspider
clark-crawler clark-crawler
demandbase-bot com.plumanalytics
evc-batch crawl.sogou.com
instabid eCatch
ips-agent heritrix
internetVista monitor
lwp-request lwp-request
magpie-crawler lwp-trivial
meanpathbot
mediawords
muhstik-scan muhstik-scan
oBot netEstate NE Crawler
plumanalytics satoristudio.net
probe-image-size scan.lol
s1z.ru
scalaj-http
seobility
seoscanners seoscanners
serpstatbot
sexsearcher
sitechecker.pro sitechecker.pro
sogouspider sp_auditbot
spyfu
sysscan sysscan
tAkeOut
trendiction.com trendiction.com
trendiction.de trendiction.de
webpros.com webpros.com
xpymep1.exe x09Mozilla
zauba.io

278
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 404enemy
80legs 80legs
ADmantX ALittle Client
AIBOT
Abonti Abonti
Acunetix Acunetix
AiHitBot
Alexibot
Anarchie Anarchie
Anarchy99 Anarchy99
Anthill Ankit
Aspiegel Aspiegel
AwarioRssBot AwarioRssBot
AwarioSmartBot
BDFetch BDFetch
BLEXBot
BackDoorBot BackDoorBot
BackWeb BackStreet
Backlink-Ceck
BacklinkCrawler
Badass Badass
Bandit
Barkrowler
BatchFTP
Black Hole Black Hole
Buck Blackboard
Blow
Boardreader
BotALot
Buddy
BuiltBotTough BuiltBotTough
BuiltWith
BunnySlippers BunnySlippers
BuzzSumo CODE87
CATExplorador CSHttp
CCBot Calculon
CazoodleBot Cegbfeieh
ChinaClaw CensysInspect
CheeseBot
CherryPicker
Chlooe Chlooe
Cliqzbot Claritybot
Cloud mapping
Cocolyzebot
Collector Collector
CopyRightCheck Copyscape
Craftbot
Crawling at Home Project Crawling at Home Project
Curious DatabaseDriverMysqli
Custo
DIIbot
DSearch
DataCha0s
Devil Devil
Digincore DigitalPebble
Dirbuster
Discoverybot
Dispatch Dispatch
DittoSpyder DittoSpyder
DnyzBot DnyzBot
DomCopBot
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Dotbot EasyDL
Download Wonder Ecxi
Drip
EMail Siphon
EMail Wolf
Ebingbong
EirGrabber EirGrabber
Evil Evil
Express WebPictures
ExtLinksBot ExtLinksBot
ExtractorPro Extractor
Ezooms Ezooms
FDM FDM
FHscan Fimap
Firefox/7.0
Flunky Flunky
Foobot Foobot
Freeuploader Freeuploader
FrontPage FrontPage
FyberSpider FyberSpider
Fyrebot G-i-g-a-b-o-t
GT::WWW
Genieo Genieo
GetWeb GetWeb
Gigabot
GrabNet GrabNet
GrapeFX HEADMasterSEO
GrapeshotCrawler
Haansoft Haansoft
HaosouSpider
Harvest Harvest
Heritrix
Humanlinks Humanlinks
Iblog Iblog
Id-search Id-search
Image Fetch Image Sucker
Indy Library Indy Library
InfoNaviRobot
InfoTekies InfoTekies
Intelliseek Intelliseek
InterGET
Internet Ninja
InternetSeer InternetSeer
Iria
JamesBOT JamesBOT
Jbrofuzz
JennyBot JennyBot
JetCar Jetty
Jorgee Jorgee
Kenjin Spider JustView
Keyword Density
Kinza Kinza
Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot Lanshanbot
Larbin Larbin
Leap
LeechGet LeechGet
Lftp Libwhisker
Lightspeedsystems
LinkScan LinkScan
Linkdexbot Linkdexbot
LinkextractorPro
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Litemage_walker Lipperhey Spider
Lmspider Ltx71
MIIxpc MFC_Tear_Sample
MSFrontPage MIDown tool
MSIECrawler MSIECrawler
MTRobot Mag-Net
Magnet
Mail.RU_Bot Mail.RU_Bot
MarkMonitor
Mass Downloader Mass Downloader
Mata Hari Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot MegaIndex.ru
Mediatoolkitbot Metauri
MicroMessenger
Mojolicious Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila
Mr.4x3 Mr.4x3
Musobot NPbot
NICErsPRO Name Intelligence
Net Vampire Nameprotect
NetZIP NetZIP
NextGenSearchBot
Nibbler
Nikto Nikto
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Nmap Nmap
Nuclei
Offline Explorer Offline Explorer
Offline Navigator OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS OpenVAS
OutclicksBot
OutfoxBot
PECL::HTTP
PHPCrawl PHPCrawl
PageScorer PageScorer
Pandalytics Panscient
Pavuk Petalbot
PeoplePal Pi-Monster
Picsearch Piepmatz
PictureFinder Psbot
PleaseCrawl
Pockey
ProPowerBot
ProWebWalker
Probethenet
Pu_iN Pu_iN
PxBroker Pump
PyCurl Quick-Crawler
QueryN Metasearch RSSingBot
RankActive RankActive
RankActiveLinkBot
RankingBot RankingBot
Re-re
ReGet
RealDownload RealDownload
Recorder RebelMouse
RedesScrapy Rogerbot
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler
SEOstats SEOstats
SISTRIX ScanAlert
SMTBot
SalesIntelligent
Scanbot Scanbot
Scrapy
ScreenerBot ScreenerBot
Searchestate ScrepyBot
SearchmetricsBot
SentiBot
SeoSiteCheckup
SeobilityBot SeobilityBot
Siphon Seomoz
Site Sucker
Sitebeam
Siteimprove Siteimprove
SmartDownload SmartDownload
Snoopy Snapbot
SocialRankIOBot
Sociscraper Sociscraper
Sosospider
Sottopop Sottopop
Spammen SpaceBison
SpankBot
Spbot Spbot
Spinn3r
Sqlmap
Sqlworm Sqlworm
Steeler Sqworm
Stripper
Sucuri
SuperBot
Surfbot
SurveyBot
Swiftbot
Szukacz Szukacz
T0PHackTeam T0PHackTeam
T8Abot
Telesoft Telesoft
Telesphorep TheNomad
Turnitin Titan
URLy Warning Toata
UnisterBot Toweyabot
Vacuum Tracemyfile
Vagabondo Trendiction
VidibleScraper Twice
VoidEYE Typhoeus
Voil VB Project
Voltron VCI
Virusdie
WEBDAV WEBDAV
WISENutbot
WPScan WPScan
WWW-Mechanize
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers
Web Auto Web Auto
Web Fetch Web Collage
Web Enhancer
Web Pix Web Pix
WebGo IS Web Sauger
WebLeacher WebFuck
WebImageCollector
WebReaper
WebStripper WebStripper
WebSucker WebSucker
WebmasterWorldForumBot WebWhacker
Webshag Webalta
Website Quester
WebsiteExtractor
Whack
WinHTTrack
Wonderbot
Woobot
Wprecon Wprecon
Xenu YoudaoBot
Zermelo ZyBorg
ZoominfoBot
ZumBot
autoemailspider autoemailspider
backlink-check backlink-check
cah.io.community
cognitiveseo
com.plumanalytics com.plumanalytics
dataforseo.com crawl.sogou.com
heritrix crawler4j
gopher
instabid instabid
netEstate NE Crawler lwp-request
oBot lwp-trivial
magpie-crawler
mediawords
page scorer page scorer
pcBrowser pcBrowser
plumanalytics
polaris version polaris version
probe-image-size
ripz ripz
s1z.ru
satoristudio.net satoristudio.net
scalaj-http seobility
scan.lol seostar
serpstatbot
sexsearcher sexsearcher
sogouspider sitechecker.pro
sysscan siteripz
trendiction.com sp_auditbot
webmeup-crawler tAkeOut
x09Mozilla trendiction.de
xpymep1.exe webprosbot
zauba.io zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
budilneg.xyz absugars.com
citizenclsdriveri7.pen.io bestmouthwash.club
dataloading.net bird1.ru
invitefashion.com edwinkonijn.com.au
iwanttodeliver.com forminecrafters.ru
nsatc.net investmac.com
replicaclub.ru mrpornogratis.xxx
turkeyreport.tk porodasobak.net
winwotgold.pl social-fun.ru
yogamatsexpert.com tizanidinepain.blogspot.com

928
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 0lovespells0.blogspot.com
00webcams.com 1001desert.com
1000n1.ru
100dollars-seo.com 100dollars-seo.com
178evakuator178.ru 101lesbian.xyz
1adult.com 11235813.webzdarma.cz
5tey7463.weebly.com 1pamm.ru
7a2rnuey1tw9ar.ru 23kw.ru
7wind.ru 2728fb936f0.com
8xv8.com 4k-player.pl
absolutelyanalog.com 91abcw.com
a.frcls.fr
a1security.com.ua
aad1f4acb0a373420d9b0c4202d38d94fa.com
aaronabel.com
academia-nsk.org academia-nsk.org
actulite.com adconscious.com
adman.gr
adminshop.com
adrunnr.com adrunnr.com
adtech.us adsloads.com
afmuseum.com adultwebhosting.info
all4invest.ru advancedcleaner.com
allcredits.su adverster.com
alpinism.ru afftrack001.com
akita.kz
albamargroup.com
alert-fdm.xyz
all4wap.ru
am-se.com am-se.com
android-vsem.org ameblo.top
angigreene.com ampower.me
angry-fermi-7633.arukascloud.io
animali.deagostinipassion.it
animalia-life.club animalia-life.club
anjalika.co.in
apiappenableinfo-a.akamaihd.net
apibrowsesmartne-a.akamaihd.net
apiitsthirteende-a.akamaihd.net apiitsthirteende-a.akamaihd.net
appfastplay.com apishops.ru
apple.com-webbrowsing-security.science apper.de
applyneedy.xyz
appsaurus.com
appsecurityr.com
arendas.net arendas.net
articlesdirectoryme.info artel116.ru
ascat.porn as5000.com
aucoinhomes.com atley.eu.pn
ausmepa.org.au
autoseo-traffic.com
autovideobroadcast.com autovideobroadcast.com
aviapanda.ru aviav.ru
avitocars.ru avtovolop.ru
avon-novosib.ru azbukafree.com
azbaseimages.net babyguns.ru
backiee.com bag77.ru
baditri.com banan.tv
balkanfarma.ru beauty-b0x.pl
battleforupdating.site bensbargains.net
beachfix.co bestempresas.es
beachtoday.ru bestfortraders.com
belinvestdom.by bestssaker.com
best-games.today bestwebsitesawards.com
bestimagecoollibrary.com bigpictures.club
betteroffers.review bildsuche.ru
beyan.host.sk
bidr.trellian.com
biketank.ga
billiard-classic.com.ua
bitnote.co bitnote.co
board.f00d.de blavia.00author.com
bobba.dzaba.com bobinoz.com
boo-arts.com brainsandeggs.blogspot.com
boobsimge.com browserprotecter.com
bookmaker-bet.com brus.city
boole.onlinewebshop.net bscodecs.com
boom-celebs.com buy-cheap-online.info
bristolhostel.com buyessaynow.biz
buckspar.gq buytizanidineonline.blogspot.com
burger-tycoon.com buytizanidineonlinenoprescription.blogspot.com
businesxxl.com cah.io.community
buttons-for-your-website.com candiceloves.us
buyantiviralwp.com capacitacionyaprendizaje.com
buybest1.biz cartierloveringreplica.xyz
buzzonclick.com cartierreplica.top
bvlgariring.xyz cartiertrinity.xyz
callcafe.info centrumcoachingu.com
candycrushshop.com
canyougethighofftizanidine.blogspot.com
capsderfudd.tk
carrauterie.be
cartierlove2u.xyz
cartierlovebracelet.xyz
cartierreplica.xyz
casinopinup-wins.com
casinorewards.info
catalogs-parts.com
changde.58.com changde.58.com
cheap-pills-norx.com chatroulette.life
cheapbelstaff.online chatroulette.video
cheatcode-lita12.rhcloud.com cheapestjerseys-wholesale.com
chlooe.com cheapmoncler.win
christianlouboutinreplica.win chinese-amezon.com
claytransformations.info chocolateslim-original.com
cleansearch.net cialiswithoutadoctor.net
clodo.ru ciceros.co
cmd.kz classicakuhni.ru
cntravelre.com cleanersoft.com
cocyq.inwtrade.com cleanpcnow.com
codq.info cocolyze.com
cometorussia.net com-secure.download
compliance-alexa.top compliance-fred.top
compliance-barak.xyz compliance-john.top
conocer-sanabria.com compliance-margo.top
coolwallpapers4k.info compliance-mary.top
cornomase.win constantaservice.net
cosmetic.donna7753191.ru contextualyield.com
cosmeticswomens-womensports.rhcloud.com converse.ddsoldes.fr
cpajunkies.com cookie-law-enforcement-aa.xyz
crackguru.tk creditmoney.com.ua
creativehutindia.com czeshop.info
creditnation.ru d-black.bz
crest-poloski.ru d2jsp.org
cryptoswap.biz dalavia.ru
customerguru.in damedingel.ya.ru
dame-ns.kz
danashop.ru
data1.scopich.com data1.scopich.com
davebestdeals.com ddospanel.com
dawlenie.com deanmoore.ie
daydream-studio.ru dekoration.us
deda-moroza-zakaz.ru dentfidemountpreach.tk
delayreferat.ru dev33.dioniqlabb.se
derevesendeco.com dev78.dioniqlabb.se
diegolopezcastan.com dfwu1013.info
discovertreasure-a.akamaihd.net digest-project.ru
disruptingdinnerparties.com digilander.libero.it
dkvorota.ru digital-sale.su
dnepr-avtospar.com.ua dinkolove.ya.ru
documentsite.net div.as
dokumentalkino.net divci-hry.info
donna7753191.ru divku.ru
dood.live diy-handmade-ideas.com
doreenblog.online documentbase.net
doxyporno.com doeco.ru
duawitchrarato.tk dojki-hd.com
e-commerce-seo.com dollartree.info
domaincheck.io
domaincrawler.com
domini.cat
doxysexy.com
druzhininevgeniy63.blogspot.com
dtnlyss.com
eandsgallery.com eandsgallery.com
ebooknovel.club ecommerce-seo.org
educatemetv.com edelstahlschornstein-123.de
educontest.net ekobata.ru
eets.net electricwheelchairsarea.com
ekaterinburg.xrus.org elektir.ru
electronicadirect.com elektronischezigarettekaufen1.beeplog.com
elektrischezigarettekaufen2.cowblog.fr emerson-rus.ru
elektronischezigarettekaufen2.tumblr.com energydiet24.ru
elmifarhangi.com enews.tech
en.home-task.com epicbrogaming.com
enskedesquashclub.se erotik0049.com
erot.co erotikstories.ru
eshop4u.jp eshop.md
essay-edu.biz essayservicewriting.org
estelight.ru everytalk.tv
evening-dating-club.info extener.org
ezaz.info extlabs.io
ezigarettekaufen1.hpage.com ezigarettenkaufen2.dreamwidth.org
ezigarettekaufen2.blox.pl f07.de
ezigarettekaufen2.yolasite.com f0815.de
faithe.top face.hostingx.eu
fanoboi.com facialporntube.com
farmprofi.net farmprofi.net
fashionavenuegame.com fatfasts-4tmz.com
felizporno.com
fellowshipoftheminds.com fellowshipoftheminds.com
fertilitetsradgivningen.se fickkontaktehobbyhuren.com
fickblock18.com filefilter.weebly.com
fishtauto.ru filesclub.net
fitness-video.net filesdatabase.net
flavors.me filmi-onlain.info
formularz-konkurs.tk forpackningsutveckling.se
forum69.info frankofficial.ru
fotzen-ficken.com free-gluten.ru
foxinsocks.ru free-share-buttons.blogspot.com
free-share-buttons.com
free-video-tool.com free-video-tool.com
freelotto.com freegamesplay.online
gambarkataku.co freeuploader.com
gamevalue7.weebly.com fres-news.com
gaming-journal.com frighteningremain.cf
genericviagrasildenafiled.net ftns.ru
genetworx.com funcrushgames.com
gentamicineyedrops.blogspot.com g.starmoe.xyz
getoutofdebtfree.org galaxy-family.ru
glicol.kz gallictures.com
global-ics.co.za gamesprite.me
get-your-social-buttons.info
getaclueamerica.com
getridofstretchmarks.org
ghazel.ru
girlgamerdaily.com
gloverid.site
goodly.pro goodly.pro
googglet.com googst2.ru
googlepositions.com goroda-vsego-mira.ru
gotorussia.com
gourcy.altervista.org gourcy.altervista.org
greatzip.com green-tea.tv
grtyi.com
gsasearchengineranker.site
gsasearchenginerankerhelp.com
gta-club.ru gta-club.ru
guigyverpo.cf guardlink.org
guod.me
gwebtools.com gwebtools.com
h2monline.com
hamilton.ca hamilton.ca
hccoder.info hawaiielectriclight.com
hd-film.pl hazardky.net
hdwallpapers-free.com hdpixent.com
headpress.ru healmytrauma.info
help.tpu.ru help.tpu.ru
hentaiheroes.com hermesbelts.xyz
hidefiles.org hetmanship.xyz
highstairs-a.akamaihd.net home.myplaycity.com
hobbyhuren-datenbank.com homelygarden.com
homegardenlova.com horoshieokna.com
homemature.net howopen.ru
hopeonthestreet.co.uk huhn.altervista.org
hostingclub.lk huimin764128.com
hotenergy.ru hunthillfarmtrust.org
hoterika.com hydropump.su
howtoclean.club hystersister.com
hustoon.over-blog.com i-midias.net.br
ideibiznesa2015.ru ictizanidinehcl4mg.blogspot.com
ikritikimou.gr ifirestarter.ru
imagine-ex.co igru-xbox.net
inet-traffic.com imgarcade.com
insider.pro imperia31.ru
instasexyblog.com inboundlinks.win
invest-pamm.ru inspiring-desperate.tk
integritylandscapeservices.com
investsuccess.org
invivo.hu invivo.hu
iomoio.net inzn.ru
iopeninghours.co.uk iqbazar.ru
iradiology.ru irunfar.com
irkutsk.online-podarki.com itservicesthatworkforyou.com
iron-age.info izoll.ru
iskalko.ru jackwolfskinoutlet.online
islamtoday.co.za jamiembrown.com
it-max.com.ua janerikholst.se
itis4you.com japfm.com
iusstf.org javitas.info
ivearchenceinflu.cf javspace.net
iwanttodeliver.com jobmarket.com.ua
izhevsk.xrus.org joingames.org
jackpotchances.com
japan-bearings.ru
jongose.ninja jongose.ninja
joy-penguin.com jose.mulinohouse.co
jrcigars.com journalhome.com
juliadiets.com karadene.com
jurajskie.info kchaxton.com
jwcialislrt.com ketrzyn.pl
kazka.ru keywordhut.com
kerei.ru kino-rating.ru
khadastoafarde.tk kino2018.club
kicknights.gq kndxbkdx.bloger.index.hr
kinobest.pl koral.se
kinomaniatv.pl krasivye-devushki.net
kletkimehan.ru kvartir-remont.biz
klosetkitten.com
kmd-pto.ru
koronirealestate.gr
kosynka-games.ru
ku6.com
kw21.org
lafriore.ru lafriore.ru
landoftracking.com landinez.co
larose.jb2c.me landreferat.ru
legionalpha.com laptop-4-less.com
lion.misba.us lawyersinfo.org
littlecity.ch layola.biz.tc
livefixer.com lecbter-relationships.ga
llastbuy.ru lionking-1994.blogspot.com
loadingpages.me liran-locks.com
locatellicorretor.com.br lirunet.ru
loftdigital.eu listiseltemournlan.gq
lolnex.us live-sexchat.ru
lostfilm-online.ru liveu.infoteka.hu
lumb.co lockerz.com
lunchrock.co lulea-auktionsverk.se
luxmagazine.cf macrotek.ru
m292.info makis.nu
madot.onlinewebshop.net manyresultshub-a.akamaihd.net
makemoneyonline.com marcogrup.com
manuscript.su marketingvici.com
masterseek.com marketland.ml
medanestesia.ru max-p.men
medicineseasybuy.com medicines-choice.com
migente.com mixtapetorrent.com
miniature.io mnimmigrantrights.net
mirzonru.net moneyteam24.com
moscow.online-podarki.com morf.snn.gr
moz.com moskva.nodup.ru
mttwtrack.com moviemail-online.co.uk
mug-na-chas-moscow.ru mrlmedia.net
mulberryoutletonlineeu.com musicktab.com
musezone.ru mydeathspace.com
music7s.me myfreetutorials.com
mustat.com mypets.by
mwtpludn.review mysex21.com
myanyone.net na15.ru
mylesosibirsk.ru namecrumilchlet.tk
mymobilemoneypages.com naval.jislaaik.com
naj-filmy24.pl needtosellmyhousefast.com
narkologiya-penza.ru net-profits.xyz
nash-krym.info newsperuse.com
net-radar.com newstudio.tv
nhl17coins.exblog.jp nextrent-crimea.ru
nibbler.silktide.com nofreezingmac.click
nisuturnetdgu.tk notebook-pro.ru
o-o-8-o-o.ru novgorod.xrus.org
oddamzadarmo.eu oakridgemo.com
office-windows.ru oconto.ru
office2web.com oda.as
ohmyrings.com offergroup.info
ok.ru okeinfo.online
online-hit.info old-rock.com
onlinefilmz.net omgtnc.com
onlinemegax.com onclkads.com
onlineporno.site oneshotdate.com
owohho.com onlinemeetingnow.com
paramountmarble.co.uk orgasmus-virtual.com
partnerline.men orsonet.ru
pcboa.se outrageousdeal-a.akamaihd.net
pdns.download paighambot.com
pendelprognos.se panouri-solare-acoperis.com
perm.xrus.org partner-high.men
perosan.com partners-ship.pro
photosaga.info pateaswing.com
piccdata.com pathwhelp.org
picphotos.net pb-dv.ru
pihl.se percin.biz.ly
pinwallpaper.top perso.wanadoo.es
peugeot-club.org
phormchina.com
photo-clip.ru
photochki.com
phrcialiled.com
piccshare.com
picsforkeywordsuggestion.com
pisanieprac.info
piter.xrus.org
piuminiita.com piuminiita.com
pixelrz.com
pixgood.com
pizza-imperia.com pizza-imperia.com
placid-rounded-coast.glitch.me
plastgranar.nu plastgranar.nu
plastjulgranar.se pojdelo.weebly.com
poems.com.ua popserve.adscpm.net
pokemongooo.ml popup-hgd.xyz
pony-business.com popup.matchmaker.com
pornhive.org pornofeuer.com
porno-chaman.info pornokorol.com
pornogig.com pozvonim.com
pornoklad.net preparevideosafesystem4unow.space
posible.net pretty-mart.com
pp-budpostach.com.ua producm.ru
pr0fit-b0x.com proftests.net
preconnubial.usuby.site proposal-engine.com
predominant-invent.tk putitin.me
priora-2.com qld10000.net
productarium.com rankings-analytics.com
proekt-mos.ru realting-moscow.ru
prointer.net.ua realtytimes.com
prostitutki-kharkova.org registratciya-v-moskve.ru
psoriasis-file.trade
pufip.com
r-control.ru
rada.ru
raisedseo.com
rapidgator-porn.ga
raschtextil.com.ua
rating-bestcasino.com
rebelmouse.com
redhotfreebies.co.uk
registrysweeper.com
reklama1.ru reklama1.ru
renecaovilla.online rep-am.com
renewablewealth.com rollercoin.com
responsinator.com rp9.ru
reyel1985.webnode.fr ruclicks.com
rezeptiblud.ru ruscopybook.com
rn-to-bsn.com saecsa.co
rockma.se sammyweaver.com
rockprogblog.com santechnik.jimdo.com
ruex.org.ua sanyuprojects.com
running-line.ru sarahmilne.top
runofilms.ru saveindex.xyz
rvzr-a.akamaihd.net savingsslider-a.akamaihd.net
sakhboard.ru
sawin.beth.webd.pl sawin.beth.webd.pl
sbricur.com scanner-brian.top
sbt-aqua.ru securityallianceservices.com
scanner-alex.top sendearnings.com
scanner-elena.top seoexperimenty.ru
scanner-willy.top sexreliz.com
searchinquire.com
securesmrt-dt.com
sel-hoz.com
seminarygeorgia59.ga
seolab.top
servicecenter.co.ua
serving.adbetclickin.pink
servingnotice.com
sex-watch.com
sexkontakte-seite.com
sexkvartal.com
sexy.babes.frontend-stack.top
sexystrippe.info sexystrippe.info
sexyteens.hol.es shisha-swag.de
shariki-zuma-lines.ru shlyahten.ru
shikiso.info shoesonlinebuy.cn
signoredom.com siteonomy.com
site-speed-check.site
sixcooler.de
sladkoevideo.com sladkoevideo.com
slim.sellany.ru slim.sellany.ru
slujbauborki.ru sloopyjoes.com
small-games.biz sneakyboy.com
snts.shell-pmr.ru sobecjvuwa.com.ru
sogimlecal.tk social-buttons.xyz
spb.ru softonicads.com
sr-rekneskap.no softtor.com
stefanbakosab.se soochi.co
stickers-market.ru spb.afora.ru
stop-zavisimost.com splendorsearch-a.akamaihd.net
storehouse.ua squidoo.com
strag-invest.ru stop-gepatit.te.ua
stretchmate.net strana-krasoty.ru
strongholdsb.ru strfls.com
stroyhelp-dv.ru stroydetali.ru
stroymonolit.su strv.se
studiofmp.com sunflowerdrawingpaintings.blogspot.com
superinterstitial.com supermodni.com.ua
supermama.top surflinksmedical.com
svnuppsalaorebro.se svetodiodoff.ru
swinginwithme.ru sweepstakes.rewardit.com
syndicate.fun szamponrevita.pl
tandvardshuset.net taihouse.ru
tattoo33.ru taqywu51.soup.io
taxi-v-eisk.ru taranerymagesswa.blogspot.com
teastory.co tds-advert005.info
tedy.su technopellet.gr
templates.radiodigital.co telemetryverification.net
terrafootwear.us telsis.com
testingads.pro texbaza.by
tfxiq.com tfxiq.com
thaismartloan.com tgtclick.com
thebestphotos.eu thebitcoincode.com
thedownloadfromwarez.blogspot.com thegolfclub.info
theporndude.com thetardistimes.ovh
titangel-vietnam.com thewomenlife.com
tizanidineandhydrocodone.blogspot.com thexart.club
toolsky.com time-japan.ru
trafficgenius.xyz timetorelax.biz
tsan.net tizanidineformigraines.blogspot.com
tubeline.biz tizanidineingredients.blogspot.com
turbo-suslik.org tizanidinerecreationaluse.blogspot.com
turizm.bz torrents.cd
tvorozhnaja-zapekanka-recept.ru torrnada.ru
twelvevisionspartyofcolorado.com tourcroatia.co.uk
typimga.pw toy-shop.top
ufa.xrus.org tpu.ru
uggbootsoutletsale.us trackzapper.com
uhdtv.website trahvid.com
unitexindia.com tri-slona.org
usadacha.net triplepanda.xyz
uvozdeckych.info try-rx.com
vadimkravtcov.ru tuberkuleznik.ru
validccseller.com turist-strani.ru
valmetrundan.se turkeyreport.tk
vclicks.net udayavani.com
veerotech.com ukkala.xyz
ventelnos.com ukrtextbook.com
unblocksit.es
unece.org
universals.com.ua
unrealcommander.biz
url-img.link
urlopener.blogspot.com.au
usiad.net
varikoz24.com
vashsvet.com
vereo.eu
versaut.xxx-cam.webcam
verymes.xyz verymes.xyz
video-production.com vezuviy.su
vinylvault.co.uk videochat.tv.br
vkak.ru vielporno.net
vod.com.ua vikistars.com
voprosotvet24.ru vipsexfinders.com
vseigru.one vipsiterip.org
vseigry.fun visionwell.com.cn
vzlomsn.org vkontaktemusic.ru
w7s.ru vksex.ru
walpaperlist.com vostoktrade.info
wanker.us wait3sec.org
watchmyfb.pl walkme.com
web-betting.ru wapsite.me
web.cvut.cz watch-movies.ru
webix.biz weblibrary.win
webscouter.net westsextube.com
westen-v.life whatistizanidine2mg.blogspot.com
wheelchairliftsarea.com whiteelephantwellington.com
winner7777.net wholesalejerseys-cheapest.com
winterclassichockeyjerseys.com wholinkstome.com
withstandingheartwarming.com winx-play.ru
womama.ru woman-tampon.ru
works.if.ua worldlovers.ru
world-mmo.com worldoffiles.ru
wtsindia.in www.jbetting.com
www.timer4web.com wzgyyq.com
wygraj-teraz.com x5market.ru
xmladserver.com xblog.in
xn----9sbubg3ambdfl1j.xn--p1ai xboxster.ru
xn--80aanaardaperhcem4a6i.com xchat26.myfreecams.com
xn--80aodinpgi.xn--p1ai xkaz.org
xn--hxazdsfy.blogspot.com xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn--oogle-wmc.com xn----ctbbcjd3dbsehgi.xn--p1ai
xolodremont.ru xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
xpresscare.ru xn--80aikhbrhr.net
xxlargepop.com xn--e1afanlbnfckd7c3d.xn--p1ai
xxx-cam.webcam
xxx-treker.ru
xxxasianporn.net
xxxguitars.com
yaaknaa.info yaaknaa.info
yadro.ru yellocloud.be
ytmnd.com yellowstonesafaritours.com
youbloodyripper.com
youghbould.wordpress.com
youraticles.pl
yourothersite.com
yourporn.com
youtube-downloader.savetubevideo.com
zaim-pod-zalog-krasnodar.ru
zaloadi.ru zaloadi.ru
zebramart.ru zarajbuilders.com
zeroredirect12.com zarenica.net
zeroredirect9.com zeleznobeton.ru
zhacker.net zero1.it
zoompegs.com zeroredirect11.com
zygophyceous.womanstars.site zootoplist.com
zynax.ua zrizvtrnpale.tk
zvuker.net

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01apple.com 4kpics.rocks
123any.com 76brighton.co.uk
51unlim.ru aa81bf391151884adfa3dd677e41f94be1.com
7zap.com aab94f698f36684c5a852a2ef272e031bb.com
abusora.com abcguru.xyz
acads.net abiente.ru
adloads.com adcash.com
advokat-grodno.by adminshop.com
affordablewebsitesandmobileapps.com ads-cool.pro
affrh2015.com adultfullhd.com
agroeconom.kz adverster.com
alert.scansafe.net akama.com
alkoravto.ru alive-ua.com
allboard.xobor.de
amyfoxfitness.com amyfoxfitness.com
apibetweenlinesn-a.akamaihd.net android4fun.org
app-ready.xyz animal36.com
appfixing.space apartmentratings.com
appleid-verification.com arate.ru
applicationg29.com arcadeplayhouse.com
apxeo.info ascat.porn
arendovalka.xyz asiavirtualsolutions.com
as5000.com asrv-a.akamoihd.net
asseenontv.ru auspolice.com
auto.rusvile.lt autochoixspinelli.com
automobile-spec.com bestplacetobuyeriacta.jw.lt
aviav.eu bestsexyblog.com
bannerswap.com bgviagrachrx.com
basedecor.ru bharatdefencekavach.com
bbsoldes.fr binomo.kz
beauty-b0x.pl bioscorp.ru
berlininsl.com bloglag.com
bestprofits-there.com blogs.rediff.com
betshuckclean.com boostmyppc.com
bg6s0.com bracketsmackdown.com
bget.ru brainsandeggs.blogspot.com
biznesluxe.ru brainxs.us
bluejays-jerseys.us businesn.men
briomotor.co cacheimages.com
burger-tycoon.com candcstuccoandstone.com
businescoop.men canyougethighofftizanidine.blogspot.com
bvlgaribracelet.xyz cartierloveringreplica.xyz
callawaygolfstore.online casinos4dummies.co.uk
canrioloadm.gq cdncash.com
cartierlovestore.com cdncash.org
certifywebsite.win chastnoeporno.com
cheapmoncler.win chocolateslim-en-france.com
chiptuninger.com cleanersoft.com
claytransformations.info cmd.kz
club-samodelkin.ru coindirect.io
coolgramgoods.com compliance-barak.xyz
cowblog.fr compliance-john.top
culad.com computernetworksonline.com
ddpills.com concealthyself.com
construmac.com.mx
credit-respect.ru
customchocolate.business-for-home.com
cypernhuset.se
daneshetabiat.com
datract.com
dentalpearls.com.au dentalpearls.com.au
djihispano.com detmebel.su
docs4all.com devochki.top
domain-tracker.com discovertreasurenow.com
doreenblog.online disruptingdinnerparties.com
drugs-no-rx.info dogsrun.net
edelstahlschornstein-123.de downloadme.life
educontest.net downloads-whatsapp.com
egovaleo.it doxysexy.com
elektronischezigarettekaufen1.myblog.de doyouknowtheword-flummox.ml
european-torches.ru dvd-famille.com
exchanges-bet.com easytuningshop.ru
f-online.de efkt.jp
fapgon.com elektrischezigarette2.onsugar.com
fast-wordpress-start.com empathica.com
femdom.twiclub.in ertelecom.ru
flowersbazar.com expertnaya-ocenka.ru
flprog.com faptitans.com
fok.nl fettefrauen.net
forex.osobye.ru findclan.org
free-social-buttons.com finuse.com
freegamesplay.online flauntyoursite.com
freenom.link fm-upgrade.ru
freesitetest.com food.dtu.dk
freetangodownload.com free-floating-buttons.com
fuel-gas.com free-stock-illustration.com
gamebackyard.com girlspicsa.com
geile-lelly.eu gonextmedia.com
gentamicineyedrops.blogspot.com grizzlysgrill.com
getoutofdebtfree.org gsasearchenginerankerhelp.com
gg.zzyjxs.com haveinc.xyz
glicol.kz hchha.com
globalscam.ga hetmanship.xyz
goldenggames.com himgaws.pw
goodnightjournal.com histock.info
gotwebsite1.com hledejvshopech.cz
guardlink.com homeart.space
highland-homes.com hop.clickbank.net
hkladys.com hotel-mkad.ru
home.myplaycity.com housediz.com
hully.altervista.org infobabki.ru
husky-shop.cz instabid.tech
hut1.ru interesnie-faktu.ru
hydropump.su iomoio.net
idegenvezeto.eu ipostroika.ru
ilo134uloh.com ivearchenceinflu.cf
images-graphics-pics.com jewelryandfiligree.com
imfamous.info jofucipiku.tk
imgarit.pw johannesburgsingles.co.za
imitex-plus.ru jurajskie.info
infodocsportal.com
intim-uslugi.info
io9.com
iopeninghours.co.uk
iqbazar.ru
istizanidineanarcoticdrug.blogspot.com
jacago.com
javlibrary.cc
jbl-charge.info
juliaworld.net
karpun-iris.ru
katadhin.co
katjimej.blog.fc2.com katjimej.blog.fc2.com
kerwinandcariza.com kchaxton.com
keywordblocks.com keyhantercume.com
kimcurlrvsms.com klitimg.pw
konkursowo-24.pl larose.jb2c.me
labvis.host.sk legionalpha.com
lechenie-gemorroya.com letmacworkfaster.world
live-cam6.info linkdebrideur.xyz
lollypopgaming.com longgreen.info
louboutinreplica.xyz louboutinbooties.xyz
lyrics.home-task.com lunamedia.co
mabdoola.blogspot.com m3gadownload.pl
mac-shield.com machicon-akihabara.info
magnetpress.sk mailemedicinals.com
marcoislandvacations.net makemoneyonline.com
mega-bony-2017.pl manifestyourmillion.com
memberty.com marketingvici.com
moesen-ficken.com maximpartnerspr.com
moetomnenie.com medispainstitute.com.au
mowser.com meteocast.net
moxo.com mirobuvi.com.ua
mriyadh.com misterjtbarbers.com
mnogolok.info
mobilemedia.md
mojowhois.com
monclercheap.xyz
moncleroutletonline.win
moz.com
mrmoneymustache.com
mundoaberrante.com
musezone.su musezone.su
musirc.com music.utrolive.ru
muztops.ru musicpro.monster
nalogovyy-kodeks.ru my-aladin.com
naphukete.ru myphotopipe.com
neodownload.webcam netallergy.ru
newstaffadsshop.club newstaffadsshop.club
oda.as obiavo.su
olympescort.com old-rock.com
online7777.com online-templatestore.com
oralsexfilme.net ovirus.ru
paclitor.com partner-cdn.men
paparazzistudios.com.au partner-trustworthy.men
paradontozanet.ru peekyou.com
patterntrader-en.com perfectplanned.com
perl.dp.ua picurams.pw
picscout.com pillscheap24h.com
pixshark.com pizdeishn.com
playlott.com pogosh.com
polimga.pw pojdelo.weebly.com
popunder.ru pomoc-drogowa.cba.pl
pornosemki.info popup-hgd.xyz
pornozhara.com porndroids.com
porodasobak.net pornotubs.com
powenlite24.ru pro-okis.ru
prizestohandle.club prospekt-st.ru
productarium.com purplesphere.in
professionaldieselcare.com putevka24.ru
profitfx.online puzzleweb.ru
pulse33.ru quangcaons.com
realmonte.net quizzitch.net
rednise.com razvratnoe.org
reversing.cc rebuildermedical.com
sad-torg.com.ua relayblog.com
sadaholding.com remorcicomerciale.ro
satoristudio.net research.ifmo.ru
sayyoethe.blogspot.co.za rewardit.com
sbt-aqua.ru rfserial.net
scanner-don.top rumamba.com
scanner-fred.top sa-rewards.co.za
sabizonline.com
saltspray.ru
samo-soznanie.ru
santasgift.ml
sbricur.com
scanner-julianna.top scanner-julianna.top
schalke04fc.info scanner-willy.top
scripted.com scanspyware.net
searchinterneat-a.akamaihd.net searchwebknow-a.akamaihd.net
security60-e.com seo-2-0.com
seminarygeorgia59.ga seozoom.it
seofirmreviewsus.info
seorank.info
servethis.com servethis.com
serving.adbetclickin.pink sexobzor.info
servisural.ru sextracker.be
sexy-pings.com sexyteens.hol.es
slomm.ru sharebutton.to
smartbalanceworld.com shopfishing.com.ua
snworks.com siteaero.com
social-button.xyz skincrate.net
sptslmtrafms.com softlinesolutions.me
staticfs.host sohoindia.net
supermesta.ru space2019.top
supermodni.com.ua sports-supplements.us
sweepstakes.rewardit.com spy-app.info
syvertsen-da.no stathat.com
szamponrevita.pl stockquotes.wooeb.com
tech4master.com streha-metalko.si
thebestweightlosspills.ovh superiends.org
theendivechronicles.com superlist.biz
tizanidine4mgstreetvalue.blogspot.com supers.com.ua
tizanidinenarcotic.blogspot.com surintech.ac.th
tk-assortiment.ru takeflyte.com
totu.info taqplayer.info
tovaroboom.vast.ru template-kid.com
tradgardspartner.se thegamerznetwork.com
trion.od.ua tizanidinehcl2mgsideeffects.blogspot.com
trucri.me todohr.com
truebeauty.cc traffmonster.info
ttsq.fr traffpartners.com
tupper-posuda.ru tri-slona.org
uasb.ru turizm.bz
unblocksit.es tx41tclega.ru
unimodemhalfduplefw.pen.io uac.net.au
unrealcommander.com uchebavchehii.ru
utrolive.ru ufa.xrus.org
vasileostrovsky-rayon.ru urlopener.blogspot.com.au
via-energy-acquistare.com uzungil.com
viagra.pp.ua v720hd.ru
videotuber.ru vapomnoncri.tk
vip.51.la vduplo.ru
vitalads.net vipsiterip.org
vkmusics.ru virtualbb.com
webadvance.club voloomoney.com
webjam.com watchinf.com
webmonetizer.net webextract.profound.net
wesharepics.info weblibrary.win
williamrobsonproperty.com webstatsdomain.org
win-spy.com weddingdresses.xyz
wtsindia.in wesharepics.com
www.wohnkabinen-shop.de whipme.yopoint.in
wygraj-skiny.win wikes.20fr.com
x-lime.com works.if.ua
xev.ru wsgames.ru
xn--80aikhbrhr.net ww1943.ru
xn--80aodinpgi.xn--p1ai x-diesel.org
xtremeeagles.net x-stars.ru
xxxguitars.com xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
yaaknaa.info youbrainboost.asia
yopoint.in zakazfutbolki.com
youradulthosting.com zoominfo.com
zarabotat-na-sajte.ru zverokruh-shop.cz
zigzog.ru
znakomstvaonlain.ru
zophim.me
zvezdagedon.ru

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.181 0; 10.1.1.135 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2648 ### Version: V4.2021.10.2649
### Updated: Wed Oct 13 10:12:13 UTC 2021 ### Updated: Wed Oct 13 22:03:37 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2648 ### Version: V4.2021.10.2649
### Updated: Wed Oct 13 10:12:13 UTC 2021 ### Updated: Wed Oct 13 22:03:37 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##