vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 03:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.05.3115 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-05-19 22:07:27 +00:00
parent deac1539c9
commit 4d53f7180a
47 changed files with 104839 additions and 104829 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3114 #### Version: V4.2022.05.3115
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3114 #### Version: V4.2022.05.3115
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.05.3114 #### Version: V4.2022.05.3115
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12970
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.05.3114 #### Version: V4.2022.05.3115
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12974
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12974
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12974
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12974
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 1; 10.1.0.18 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 0; 10.1.0.18 0;
127.0.0.1 0; 127.0.0.1 0;

12974
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 1; 10.1.0.18 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 0; 10.1.0.18 0;
127.0.0.1 0; 127.0.0.1 0;

12974
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 1; 10.1.0.18 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 0; 10.1.0.18 0;
127.0.0.1 0; 127.0.0.1 0;

12974
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3114 3115

12974
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12974
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.255.181 0; 40.86.112.56 0;

12974
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 1; 10.1.0.18 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 0; 10.1.0.18 0;
127.0.0.1 0; 127.0.0.1 0;

12974
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 1; 10.1.0.18 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 0; 10.1.0.18 0;
127.0.0.1 0; 127.0.0.1 0;

12974
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 1; 10.1.0.18 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 0; 10.1.0.18 0;
127.0.0.1 0; 127.0.0.1 0;

12974
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.255.181 0; 40.86.112.56 0;

12974
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.255.181 0; 40.86.112.56 0;

12974
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 1; 10.1.0.18 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BLEXBot Cliqzbot
DIIbot Flunky
FHscan Mata Hari
Id-search NICErsPRO
Image Fetch PECL::HTTP
LinkWalker PxBroker
OpenLinkProfiler VB Project
SEOlyticsCrawler WEBDAV
arquivo.pt WebsiteExtractor
sysscan scan.lol

284
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 01h4x.com
ADmantX
AIBOT
ASPSeek ASPSeek
Aboundex
Aboundexbot
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AiHitBot
AllSubmitter
Alligator Alligator
AlphaBot Anarchy
Anarchie
Anarchy99
Ankit Ankit
Anthill
Asterias Asterias
Attach AwarioRssBot
BBBike BBBike
BLEXBot BLEXBot
BackStreet BackStreet
Badass BackWeb
Battleztar Bazinga Backlink-Ceck
BacklinkCrawler
Barkrowler
BatchFTP
BetaBot BetaBot
Bitacle
Black Hole
Blow Blow
BlowFish Boardreader
Bolt BotALot
Buck
Buddy Buddy
BuiltBotTough BuiltBotTough
BunnySlippers
CATExplorador CATExplorador
CODE87
CSHttp CSHttp
CensysInspect Cegbfeieh
CheTeam ChinaClaw
CherryPicker
Chlooe
Citoid Citoid
Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot
Collector Collector
Copier Copier
Copyscape CopyRightCheck
Craftbot
CrunchBot CrunchBot
Custo
CyotekWebCopy CyotekWebCopy
DBLBot DSearch
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Devil Deusu
Digincore DigitalPebble
Dirbuster
Disco
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot
DomainAppender DomainAppender
Domains Project DomainCrawler
Download Wonder DomainSigmaCrawler
Dotbot
Dragonfly Dragonfly
EMail Wolf Drip
EroCrawler Evil
Express WebPictures Extractor
Extreme Picture Finder EyeNetIE
FDM FemtosearchBot
Firefox/7.0 Firefox/7.0
Freeuploader Flunky
FrontPage
FyberSpider
Fyrebot
G-i-g-a-b-o-t
GT::WWW
GalaxyBot GalaxyBot
GermCrawler
GetRight GetRight
GetWeb Go!Zilla
Getintent Go-Ahead-Got-It
Gigabot GrabNet
GoZilla
Grafula Grafula
GrapeFX
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO
HMView HMView
HTMLparser
HTTrack
Haansoft
Havij
Humanlinks Humanlinks
IDBTE4M
IRLbot IRLbot
Iblog Iblog
Image Fetch InfoNaviRobot
IndeedBot
Indy Library
Intelliseek Intelliseek
InterGET
Internet Ninja
InternetSeer InternetSeer
IstellaBot IstellaBot
JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
JennyBot LWP::Simple
Jorgee Lanshanbot
Kenjin Spider LeechGet
Keyword Density
Larbin
Libwhisker
LieBaoFast LieBaoFast
LinkScan LinkextractorPro
Linkbot
LinksManager LinksManager
Lipperhey Litemage_walker
Ltx71 Ltx71
MFC_Tear_Sample MIIxpc
MSIECrawler MQQBrowser
MTRobot MSFrontPage
Magnet Majestic12
MarkMonitor Masscan
Mass Downloader
Mata Hari
MauiBot
Mb2345Browser Mb2345Browser
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri
MicroMessenger
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Minefield
Mister PiX
Moblie Safari Moblie Safari
Mojeek
MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mr.4x3
Msrabot
NICErsPRO NICErsPRO
NPbot Nameprotect
NearSite NearSite
Needle Needle
NetAnts NetAnts
NetLyzer
NetMechanic
NetSpider
Netcraft
Nettrack Nettrack
NextGenSearchBot NextGenSearchBot
Nibbler
Niki-bot Niki-bot
Nikto
NimbleCrawler NimbleCrawler
Not
Nuclei Nuclei
Nutch Nutch
Octopus Octopus
Offline Navigator OpenVAS
OnCrawl
Openfind Openfind
OrangeSpider Openvas
OutfoxBot OutfoxBot
POE-Component-Client-HTTP PECL::HTTP
Page Analyzer Page Analyzer
PageScorer PageAnalyzer
PageGrabber
PageThing.com PageThing.com
Papa Foto Petalbot
Pavuk
Pi-Monster Pi-Monster
Picscout
Picsearch
PictureFinder PictureFinder
Piepmatz Piepmatz
Pixray
Pockey
ProPowerBot ProPowerBot
ProWebWalker
Probethenet Probethenet
Psbot Psbot
PxBroker
QueryN Metasearch QueryN Metasearch
RankActiveLinkBot RankActive
Rankivabot RankFlex
RankingBot2
ReGet
Reaper Reaper
Recorder RebelMouse
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SISTRIX SMTBot
SalesIntelligent SalesIntelligent
Scanbot
Scrapy Scrapy
ScreenerBot
ScrepyBot ScrepyBot
Seekport Searchestate
SearchmetricsBot
SemanticJuice SemanticJuice
Semrush Semrush
SemrushBot
SeoSiteCheckup SeoSiteCheckup
Seomoz Siphon
Shodan SiteSnagger
Site Sucker Sitebeam
SiteSucker SlySearch
Siteimprove
SmartDownload SmartDownload
Snake Snake
Snapbot SocialRankIOBot
Sociscraper Sociscraper
Sogou web spider Spammen
SpaceBison SpankBot
SputnikBot Spbot
Sqlmap
Sqlworm Sqlworm
Stripper Sqworm
Sucker Sucker
Sucuri
SuperBot
SuperHTTP
SurveyBot SurveyBot
Suzuran Swiftbot
Szukacz T8Abot
Teleport Teleport
TeleportPro TeleportPro
Telesoft
Telesphoreo Telesphoreo
Telesphorep TheNomad
Titan TightTwatBot
Toweyabot Toweyabot
Trendiction Trendiction
True_Robot Turnitin
TurnitinBot TurnitinBot
Twice Twice
Typhoeus
URLy.Warning URLy.Warning
UnisterBot
V-BOT V-BOT
VB Project
VCI
Vacuum
Vagabondo
VoidEYE VoidEYE
WISENutbot
WPScan WPScan
WeSEE WWWOFFLE
Web Auto Wallpapers/3.0
Web Collage
Web Fetch
Web Fuck
Web Sucker Web Sucker
WebAuto
WebBandit WebBandit
WebFetch WebCopier
WebEnhancer
WebImageCollector WebImageCollector
WebPix
WebReaper WebReaper
WebStripper
WebWhacker WebWhacker
Webalta WebsiteQuester
Webshag Widow
Whatweb Xaldon WebSpider
WinHTTrack YoudaoBot
Wonderbot
Xenu
Zade
Zermelo
Zeus
Zitebot Zitebot
ZmEu ZoominfoBot
cah.io.community ZumBot
check1.exe arquivo.pt
crawler.feedback coccocbot
com.plumanalytics
crawler4j crawler4j
internetVista monitor eCatch
isitwp.com isitwp.com
iubenda-radar iubenda-radar
linkdexbot
magpie-crawler magpie-crawler
meanpathbot meanpathbot
mediawords mediawords
muhstik-scan plumanalytics
pcBrowser polaris version
s1z.ru satoristudio.net
scan.lol
seocompany.store
seoscanners seoscanners
seostar seostar
serpstatbot sitechecker.pro
sogouspider sp_auditbot
tAkeOut trendiction.com
webprosbot trendiction.de
webgains-bot
webpros.com
x22Mozilla x22Mozilla
zauba.io xpymep1.exe
zgrab

300
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
AIBOT 01h4x.com
ASPSeek 360Spider
Aboundex 404checker
Acunetix
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot
Aipbot Aipbot
Alexibot
Alligator
AlphaBot AlphaBot
Anarchie Anarchie
Ankit Apexoo
Aspiegel
Asterias Asterias
Attach Atomseobot
BDFetch AwarioSmartBot
BackStreet
Backlink-Ceck Backlink-Ceck
BacklinkCrawler
Bandit
BatchFTP
Battleztar Bazinga Battleztar Bazinga
Bigfoot
BlackWidow BlackWidow
Blackboard Blackboard
Blow
BlowFish
Boardreader Boardreader
BotALot BotALot
Buddy Buddy
BunnySlippers BuiltBotTough
BuiltWith
CATExplorador
CCBot
CSHttp
CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect
CheTeam CheTeam
CherryPicker CheeseBot
ChinaClaw
Chlooe Chlooe
Citoid Claritybot
Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Collector Collector
Copier Craftbot
CopyRightCheck CrazyWebCrawler
Copyscape Crescent
Crawling at Home Project Custo
Curious
CyotekWebCopy CyotekWebCopy
DTS Agent DSearch
Demon Deusu
DigitalPebble
Dirbuster Dirbuster
Disco DnBCrawler-Analytics
Discobot
Discoverybot
DnyzBot DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Dragonfly DomainStatsBot
EMail Wolf Domains Project
Dotbot
EasyDL EasyDL
Express WebPictures Express WebPictures
Extractor ExtLinksBot
FDM
FHscan
FemtosearchBot FemtosearchBot
FlashGet Firefox/7.0
Foobot
Fuzz
Fyrebot
GalaxyBot GalaxyBot
GermCrawler GermCrawler
Go-Ahead-Got-It GetRight
GrabNet Gotit
HEADMasterSEO Grabber
GrapeshotCrawler
HTMLparser HTMLparser
Harvest HTTrack
Humanlinks Haansoft
IDBot HaosouSpider
IRLbot Havij
IndeedBot Heritrix
Indy Library IDBTE4M
InfoTekies IlseBot
Image Sucker
Intelliseek Intelliseek
Internet Ninja
IstellaBot IstellaBot
JOC Web Spider JamesBOT
JennyBot JetCar
JikeSpider Joomla
Jorgee Jyxobot
Keybot Translation-Search-Machine
Kinza Kinza
LNSpiderguy
Larbin Larbin
Lftp LeechGet
LexiBot
LibWeb LibWeb
LieBaoFast LieBaoFast
Likse Likse
Linkbot LinkScan
LinqiaMetadataDownloaderBot LinkWalker
Lmspider LinkpadBot
LinksManager
LinqiaRSSBot
Lipperhey Spider
Litemage_walker
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MIIxpc
MSFrontPage
MSIECrawler MSIECrawler
Mail.RU_Bot MTRobot
Majestic12 Magnet
MarkWatch
Masscan Masscan
Meanpathbot MauiBot
Metauri Metauri
Microsoft Data Access MicroMessenger
Microsoft URL Control
Moblie Safari Moblie Safari
Mojeek
Mojolicious Mojolicious
Mr.4x3 MolokaiBot
Msrabot Morfeus Fucking Scanner
Musobot Musobot
NPbot
Nameprotect Nameprotect
Navroad Navroad
Nessus
Net Vampire
NetAnts NetAnts
NetLyzer NetLyzer
NetSpider Netcraft
Netvibes NextGenSearchBot
Nibbler Niki-bot
Nikto Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Ninja Ninja
Nmap
Nuclei
Nutch Nutch
OpenVAS Offline Explorer
OnCrawl
Openfind Openfind
OrangeBot
OrangeSpider OrangeSpider
OutfoxBot OutfoxBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP PageScorer
PageGrabber PageThing.com
Papa Foto Pavuk
Picsearch Picsearch
Piepmatz PictureFinder
PleaseCrawl Pimonster
Probethenet ProPowerBot
ProWebWalker
Psbot
Pu_iN Pu_iN
PyCurl PxBroker
Quick-Crawler QueryN Metasearch
RankActive RankingBot
Rankivabot RankurBot
Re-re
Reaper Reaper
RebelMouse
Ripper
RocketCrawler RocketCrawler
SEOkicks SalesIntelligent
SEOkicks-Robot
SEOprofiler
SEOstats
SISTRIX
SMTBot
Scanbot Scanbot
ScoutJet Scrapy
ScrepyBot SearchmetricsBot
SemrushBot Seekport
Semrush
SentiBot SentiBot
SeoSiteCheckup SeoSiteCheckup
Shodan SeobilityBot
SiteCheckerBotCrawler
SiteLockSpider
SiteSnagger
SiteSucker SiteSucker
Sitebeam Sitebeam
Sitevigil SmartDownload
SlySearch
Snake Snake
Snoopy Snapbot
Sociscraper Sosospider
Sogou web spider SpaceBison
Sottopop
Spammen Spammen
SpankBot SpankBot
Spinn3r
SputnikBot
Steeler Steeler
Stripper Stripper
Sucuri
SuperBot SuperBot
SuperHTTP SuperHTTP
T0PHackTeam SurveyBot
T8Abot Swiftbot
Telesoft Telesoft
TheNomad
Thumbor Thumbor
Toata Toweyabot
Tracemyfile
Trendiction Trendiction
Trendictionbot
True_Robot True_Robot
UnisterBot Turingos
TurnitinBot
URLy Warning
URLy.Warning
Upflow
VB Project VB Project
Vacuum Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
Virusdie Virusdie
VoidEYE VoidEYE
WEBDAV WASALive-Bot
WISENutbot
WWW-Collector-E WWW-Collector-E
WWW::Mechanize WWW-Mechanize
WWWOFFLE Web Collage
Wallpapers
WallpapersHD
Web Enhancer Web Enhancer
Web Sauger Web Pix
WebAuto
WebCollage
WebCopier WebCopier
WebEnhancer WebFuck
WebFetch
WebImageCollector WebImageCollector
WebLeacher
WebReaper WebReaper
WebSauger WebStripper
WebWhacker WebsiteExtractor
WebZIP WebsiteQuester
Whack Whacker
WiseGuys Robot WinHTTrack
Wprecon Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zade
Zauba ZoomBot
Zitebot ZumBot
archive.org_bot archive.org_bot
arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider autoemailspider
backlink-check cah.io.community
coccocbot clark-crawler
com.plumanalytics cognitiveseo
crawl.sogou.com crawl.sogou.com
crawler4j eCatch
evc-batch
facebookscraper
heritrix heritrix
instabid instabid
internetVista monitor internetVista monitor
ips-agent linkdexbot
lwp-request
magpie-crawler magpie-crawler
mediawords
muhstik-scan
netEstate NE Crawler
oBot oBot
page scorer
pcBrowser pcBrowser
plumanalytics polaris version
ripz s1z.ru
scan.lol seobility
seocompany.store sexsearcher
seoscanners
spyfu spyfu
sysscan sysscan
tAkeOut tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com trendiction.de
voyagerx.com webpros.com
webgains-bot xpymep1.exe
webmeup-crawler zauba.io
x09Mozilla
x22Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
alessandraleone.com himgaws.pw
amehdaily.com lidoradio.com
aosheng-tech.com michellblog.online
dichvuvesinhhanoi.com pronekut.com
map028.com prospekt-st.ru
mowser.com scat.porn
net-radar.com splendorsearch-a.akamaihd.net
sampleletters.net sr-rekneskap.no
scanner-don.top tracklead.net
vrotike.ru zagadki.in.ua

924
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
000free.us 0daymusic.org
03e.info 1flag.co.za
0c47f8422d3f.com 1hwy.com
12bet.com 20pascals.nl
1688.com 2345.com
1millionusd.xyz 300richmond.co.nz
1stat.ru
28n2gl3wfyb0.ru
2ads.co.uk
4senses.co
4ur.click
4ureyesonly.com
4webmasters.org
5elementov.ru
5u.com
7fon.ru 7fon.ru
acc.eu.org 7xc4n.com
actices.com aa81bf391151884adfa3dd677e41f94be1.com
ad-words.ru aaa38852e886ac4af1a3cff9b47cab6272.com
addblueoff.com.ua abcdeg.xyz
ads-seo.men abclauncher.com
adsland.men adloads.com
adsloads.com adultfriendfinder.com
adultwebhosting.info adultfullhd.com
advancedcleaner.com adultgalls.com
adventureparkcostarica.com advertex.info
agahinameh.com afftrack001.com
aihelen.net afora.ru
aktivator-windows10.blogspot.com agadelha.com.br
all-number.com akvamaster.dp.ua
alldezservice.kz alldownload.pw
alloysteel.ru allfinweb.com
allsilver925.co.il allsilver925.co.il
altamayoztourism.com am15.net
alveris.ru amehdaily.com
android4fun.org amung.us
ap.senai.br anaksma.info
appsaurus.com angkortours.vn
apptester.tk api.stathat.com
aquarium-pro.ru apiallgeniusinfo-a.akamaihd.net
arabgirls.adultgalls.com apiholdingmypage-a.akamaihd.net
arenanews.com.ua apple.com-webbrowsing-security.science
artavenuegardenstudios.com appmsr.org
aruplighting.com approvedlocal.co.za
asdfg.pro asdfz.pro
atelielembrancaqueencanta.com.br asiengirls.net
atlasvkusov.ru asscenihotosidea.blogspot.com
atlant-auto.info
atmagroup.ru atmagroup.ru
aussie-prizes.com australia-opening-times.com
auto-moto-elektronika.cz autoplate.info
avkzarabotok.com avkzarabotok.com
azartniy-bonus.com avon-severozapad.ru
azbukafree.com avon-ukraine.com
bablonow.ru avtovolop.ru
badavit.com.ua azartmix.com
baditri.com backlink4u.net
balitouroffice.com balkanfarma.ru
banksrf.ru bannerbank.ru
bannerspace.com baoxaydung.com.vn
beauty-things.com barbourjackets.online
belinka.com.ua bard-real.com.ua
belinvestdom.by
belsetirehafi.tk belsetirehafi.tk
bensbargains.net bestdraws.com
bezlimitko.xyz bestkfiledxd.cf
bfz.biz bestofupload.info
bidbuy.co.kr besttorrentknifta.weebly.com
binomo.com bezcmexa.ru
binomo.kz bigcities.org
bio-market.kz bitcoinpile.com
biznesluxe.ru blackcurranthumidifiers.site
blue-square.biz blagovest-med.ru
bobinoz.com blog.axant.it
bodybuilding-shop.biz blogqpot.com
bongacams.com bolegapakistan.com
book-bets.com boleznikogi.com
bookhome.info boole.onlinewebshop.net
bracketsmackdown.com brakehawk.com
break-the-chains.com brateg.xyz
brewdom.ru buckspar.gq
brothers-smaller.ru bumascloset.com
btvn.ru buntube.net
builtwith.com buyclomidonlaine.com
burkesales.com buytizanidineonline.blogspot.com
buscarfoto.com bvlgariring.xyz
buynorxx.com canacopegdl.com
buzz-porno.info capsderfudd.tk
callcafe.info carson.getenjoyment.net
callmd5map.com cartierjusteunclou.xyz
capacitacionyaprendizaje.com cartierlovestore.xyz
capture-room.com cartierreplica.top
cartierlovebraceletreplica.xyz chatroulette.life
cashkitten-a.akamaihd.net christianlouboutinsaleoutletonline.us
ccjp.eu cialischmrx.com
cdncash.com clean-start.net
cdncash.net clickaider.com
centraletermice.us clickhype.com
cete.ru clicksor.net
cgi2.nintendo.co.jp clicksotrk.com
cheapbelstaff.online compliance-alexa.top
cheapmoncler.pw compliance-checker.info
chinaelitecheapjerseys.com compliance-don.top
chorus.terakeet.com compliance-jess.top
cl.s7.exct.net compliance-viktor.top
cleanpcnow.com consorzioilmosaico.org
clickprotects.com construmac.com.mx
compliance-barak.top copyrightclaims.org
coolbar.pro crazyprotocol.com
coolgamechannel.com crd.clan.su
corta.co credit.co.ua
counciltally.com crystalslot.com
crackzplanet.com data-mining.tk
crazyboost.pro dawlenie.com
creditmoney.com.ua ddpills.com
customsua.com.ua ddsoldes.fr
cymbaltaandtizanidine.blogspot.com
cyse.tk
d0t.ru
datsun-do.com
dayibiao.com
db.speedup.gdn
deadlinkchecker.com deadlinkchecker.com
demenageur.com dealitright.click
detalizaciya-tut.biz decenttools.com
dev-seo.blog defenderxtactical.com
diesel-parts28.ru delayreferat.ru
distver.ru dentfidemountpreach.tk
design-ideas.info
detki-opt.ru
detsky-nabytek.info
dfwu1019.info
dichvuvesinhhanoi.com
dikqlyremy.info
dimfour.com
dipstar.org
discovertreasure-a.akamaihd.net
divisioncore.com divisioncore.com
dktr.ru dnsrsearch.com
dkvorota.ru documentbase.net
domainsphoto.com domain2008.com
domoysshop.ru
dotmass.top dotmass.top
ekosmetyki.net.pl downtuptv.gq
ekto.ee dvr.biz.ua
elektrischezigarettekaufen2.cowblog.fr eandsgallery.com
elitepcgames.com earl-brown.info
easyukraine.com
ed-shop01.ru
ekaterinburg.xrus.org
ekspresihati.info
elektrozigaretten1.tumblr.com
embedle.com embedle.com
emediate.eu
energydiet24.ru energydiet24.ru
enge-fotzen.info enhand.se
eropornosex.ru enthuse.computernetworksonline.com
eralph.tk
escortplius.com
escortslet.net escortslet.net
eshop.md esnm.ru
espaceinventoristes.com este-line.com.ua
evogarage.com estelight.ru
extlinks.com estibot.com
extractorandburner.com etur.ru
eyelike.com.ua european-torches.ru
eywords-monitoring-your-success.com event-tracking.com
ezigarettenkaufen1.pagina.gr ewebarticle.info
facebook-mobile.xyz exhibitionplus.eu
fashion-mk.net exoclick.com
fashion-stickers.ru f-online.de
fearcrow.com f00kclan.de
femdom.twiclub.in facecup.top
finteks.ru fashionindeed.ml
flipper.top fefo.gdn
fm-upgrade.ru ferretsoft.com
footbalive.org filunika.com.ru
frameimage.org financetip.eu
free-traffic.xyz fitness-video.net
fivedwld.cf
flytourisme.org
folowsite.com
forum69.info
freejabs.com
freelifetimefuckbook.com
freeseedsonline.com
freeuploader.ml
freshberry.com.ua
front.to front.to
fuckbuddybestgilf.info funnypica.com
fxgallery.com
fyl.com.ru
g7m.pl
gabeshop.ru gabeshop.ru
gallery.rennlist.com gamblingnerd.com
gerhardhealer.com germetiki.com.ua
gettpromos.com getmiro.com
getyourimage.club
gidonline.one gidonline.one
girls-ufa.ru
glavprofit.ru
globalsurfari.com globalsurfari.com
gnuetella.com gojiberriess.apishops.ru
goatse.ru gopro-online.info
gogps.me gosmeb.ru
gourcy.altervista.org gourcy.altervista.org
graphicwe.org gq-catalog.gq
guitar-master.org greatdealshop.com
gunsvicceadadebt.tk gsasearchengineranker.top
haikuware.com gsou.cf
handsandlegs.ru gyffu.com
hanwei.us gymi.name
happy.new.yeartwit.com handicapvantoday.com
haveinc.xyz hausfrauensex18.com
hawaiielectriclight.com help.tpu.ru
heartofpayne.xyz herehloadibs.cf
hexpilot.com
highspeed5.net highspeed5.net
histock.info himgaws.pw
hobbyhuren24.net hiwibyh.bugs3.com
homeandhealth.ru hjaoopoa.top
homeart.space hostingclub.lk
homedo.fabpage.com hotblognetwork.com
host-tracker.com hothor.se
hotkeys.com hscsscotland.com
i-service.kz husky-shop.cz
iklysha.ml idealtits.net
imadedinner.net igadgetsworld.com
imagez.co imagecoolpub.com
imicrovision.com images.gyffu.com
imediadesk.com
imgata.com
impresagaia.it impresagaia.it
inboxdollars.com inbabes.sexushost.com
indetiske.ya.ru inspiring-desperate.tk
indiakino.net intelhdgraphicsgtdrive6w.metroblog.com
infobanks.ru intermesh.net
infowarcraft.ru iron-age.info
innodgfdriverhm.aircus.com iscblog.info
insider.pro
insurple.com
internetartfair.com
ip-guide.com
ipchicken.com
islamtoday.co.za islamtoday.co.za
ispac.org istanbulit.com
itrevolution.cf istripper.com
ivearchenceinflu.cf it-max.com.ua
ivoiretechnocom.ci itronics.ca
jasonpartington.com iyasimasennka.com
javstock.com janavibekken.no
jerseyssportsshop.com japan-bearings.ru
jmat.cn jerseywholesalebizchina.com
jongose.ninja junglenet-a.akamaihd.net
kadashihotel.com jus0wil.pp.ua
katadhin.co justdating.online
kazinogames.lv kamalsinha.com
kchaxton.com kashubadesign.ru
keywordblocks.com keywordsking.com
keywordspay.com khadastoafarde.tk
keywordsuggests.com kimcurlrvsms.com
kinocccp.net kino-rf.ru
kliksaya.com kinobest.pl
kometars.xyz kinoduh.ru
komp-pomosch.ru knogg.net
komputers-best.ru knowyournextmove.com
kozhniebolezni.com kozhniebolezni.com
kwzf.net krasivoe-hd.com
ladov.ru krassh.ru
ladsup.com la-fa.ru
lafriore.ru ladsblue.com
lapolis.it letmacworkfaster.site
lazymanyoga.com libertybilisim.com
ledpolice.ru life.biz.ua
lego4x4.ru lineshops.biz
lenvred.org linkrr.com
letolove.ru linksharingt.com
lider-zhaluzi.kiev.ua littleberry.ru
lidoradio.com localmatchbook.com
lifeinsurancekb.com lonely-mature.com
linerdrilling.com lumb.co
lion.conboy.us mackeeper-center.club
lunamedia.co macotool.com
luxmagazine.cf maju.bluesalt.co
magazin-pics.ru mansparskats.com
manifestation.betteroffers.review marcowebonyodziezowe.pl
marcoislandvacations.net mastershef.club
marketland.ml mebel-vstroika.ru
markjaybeefractal.com mecash.ru
matsdale.com medispainstitute.com.au
mearns-tractors.co.uk megaapteka.ru
medicovi.com megahdporno.net
medictube.ru meteostate.com
mega-polis.biz.ua mex-annushka.ru
megakino.net mexicotravelnet.com
michellblog.online minet.club
mindbox.co.za misslike.ru
minecraft-rus.org mlvc4zzw.space
minyetki.ru
mirmedinfo.ru
mixtapetorrent.com
mnogabukaff.net
mobsfun.net mobsfun.net
modnie-futbolki.net monarchfind-a.akamaihd.net
moivestiy.biz monclerjacketsoutlet.win
mole.pluto.ro moncleroutletonline.pw
monclercoats.xyz moncleroutletonline.xyz
montazhnic.ru moneyteam24.com
moscow-region.ru more-letom.ru
moscow.xrus.org morefastermac.trade
movies-in-theaters.net mowser.com
mturkcontent.com mrcsa.com.au
muabancantho.info mural.co
my-bc.ru musicvidz.ru
myhealthcare.com mycouponizemac.com
myplaycity.com mydearest.co
narkologiya-peterburg.ru mydirtystuff.com
narosty.com mykings.pw
naughtyconnect.com myliveblog.ru
negociosdasha.com myshopmatemac.com
newstaffadsshop.club nakozhe.com
next-dentists.tk nalogovyy-kodeks.ru
nicefloor.co.uk nalogovyykodeks.ru
nisuturnetdgu.tk nardulan.com
nl.netlog.com nevansk.ru
norththeface.store nippon-bearings.ru
november-lax.com nmrk.ru
novodigs.com nosecret.com.ua
novosti-hi-tech.ru notfastfood.ru
nrjmobile.fr novgorod.xrus.org
nudejapan.net novosibirsk.xrus.org
nudo.ca nuker.com
nuup.info offers.bycontext.com
nzfilecloud.weebly.com oivcvx.website
obiavo.in olvanto.ru
obiavo.ru onemactrckr.com
okmusic.jp onlinemegax.com
online-x.ru opinionreelle.com
onlineslotmaschine.com optitrade24.com
onlythegames.com optom-deshevo.ru
ooo-gotovie.ru owohho.com
oralsexfilme.net oxotl.com
origin-my.ru pacificair.com
osb.se11.ru pammik.ru
ovirus.ru pateaswing.com
oyster-green.com pay2me.pl
panicatack.com pcboa.se
paperwritingservice17.blogspot.ru perfectplanned.com
paradontozanet.ru perfectpracticeweb.com
partners-ship.pro photochki.com
pastaleads.com phuketscreen.com
pb-dv.ru picmoonco.pw
peekyou.com picturesify.com
pensplan.com pihl.se
picquery.com pinkduck.ga
pics-group.com pizda.lol
pirateday.ru playlott.com
pistonclasico.com podshipniki-ntn.ru
pl.aasoldes.fr poquoson.org
pl.id-forex.com porn555.com
plaff-go.ru
plastweb.ru
pliks.pl
polytopesexempt.com
popunder.net
popup-fdm.xyz
popup.matchmaker.com
pornhive.org
pornodojd.ru pornodojd.ru
pornofeuer.com
pornogad.com pornogad.com
pornoklad.net
pornowarp.info
pornzone.tv pornzone.tv
pozdrawleniya.com postclass.com
privatefx-in.ru powc.r.ca.d.sendibm2.com
profitkode.com pozdravleniya-c.ru
proxyelite.biz prefersurvey.net
pyrodesigns.com.au pricheskaonline.ru
qitt.ru prizesbook.online
quebec-bin.com professionalwritingservices15.blogspot.ru
r-control.ru prostitutki-odessa.org
rainbowice.ru prostitutki-rostova.org
rankscanner.com pskcijdc.bloger.index.hr
ranksonic.info push-ad.com
pussyfleet.com
puzzleweb.ru
q-moto.ru
qld10000.net
qor360.com
quelle.ru
ratemodels.net
razamicroelectronics.com razamicroelectronics.com
razyboard.com realizmobi.com
realitykings.com realnye-otzyvy.info
redirectme.net recipedays.ru
reeyanaturopathy.com relatodelpresente.com.ar
refudiatethissarah.info remont-fridge-tv.ru
regionshop.biz renecaovilla.online
reklamuss.ru renecaovillasale.online
remont-comp-pomosh.ru
rennlist.com rennlist.com
retailwith.com
revistaindustria.com
rewardit.com
rezeptiblud.ru rezeptiblud.ru
riders.ro rock-cafe.info
rightenergysolutions.com.au rockma.se
roofers.org.uk rumamba.com
sarafangel.ru rus-teh.narod.ru
sardinie.us rusenvironmental.net
scanmarine.info russia-tao.ru
russia-today-video.ru
rutor.group
s-forum.biz
salmonfishingsacramentoriver.com
sammlungfotos.online
sandhillsonline.com
santechnik.jimdo.com
sape.top
saratov.xrus.org
savefrom.com
saveriopiazza.it
scanner-jack.top scanner-jack.top
scanner-john.top scat.porn
scanner-willy.top scenarii-1-sentyabrya.uroki.org.ua
scrapinghub.com searchwebknow-a.akamaihd.net
search-goo.com semrush.com
searchinquire.com senger.atspace.co.uk
seinterface.com seo18.su
seo-tools-optimizing.com seofied.com
seomarketings.online seofirmreviewsus.info
sex-foto.pw seolab.top
sex-pr.net sex-pr.net
sex-tracker.com sexs-foto.top
sexy-pings.com shell-pmr.ru
sheerseo.com shymkent.xkaz.org
shohanb.com simon3.ru
shopperifymac.com
shoppingjequiti.com.br
sibvitr.ru
sildenafil-tadalafil.info
site-speed-checker.site
sizeplus.work
skutecznetabletkinaporostwlosow.pl skutecznetabletkinaporostwlosow.pl
sky-mine.ru sladkoevideo.com
smart-scripts.com slavia.info
smsactivator.ru slavic-magic.ru
snsdeainavi.info smallseotools.com
soc-econom-problems.ru smartpet.ru
soc-proof.su snip.tw
socas.pluto.ro snomer1.ru
soft-terminal.ru snow.nvr163.com
solnplast.ru social-buttons.xyz
sonata-arctica.wz.cz softxaker.ru
sophang8.com solicita.info
songplanet.ru
spb-plitka.ru spb-plitka.ru
special-porn.com spin2016.cf
stard.shop spy-app.info
stardevine.com spymac.net
starwars.wikia.com stair.registrydomainservices.com
stockquotes.wooeb.com staynplay.net
strv.se success-seo.com
studentguide.ru suggest-keywords.com
sundrugstore.com sundrugstore.com
superstats.com sureone.pro
surcentro.com syndicate.fun
surfbuyermac.com syvertsen-da.no
svarkagid.com theendivechronicles.com
svensk-poesi.com thejournal.ru
swimpool.ca thenews-today.info
swinginwithme.ru theporndude.com
teencastingporn.com tizanidineandgabapentin.blogspot.com
telefonsexsofort.tk tizanidinemusclerelaxant.blogspot.com
thebestphotos.eu tizanidinesideeffectsweightloss.blogspot.com
thetardistimes.ovh tnctrx.com
timhost.ru todohr.com
tizanidineduringpregnancy.blogspot.com token-lab.org
tizanidineformigraines.blogspot.com tonerbox.kz
tizanidinevssoma.blogspot.com tongkatmadura.info
tjkckpytpnje.com toolsky.com
top-l2.com topbestgames.com
topdownloads.ru torrent-newgames.com
tracxn.com totu.us
trafficmania.com touchmods.fr
tourcroatia.co.uk
tracksurf.daooda.com
trafficshaper.com
traffique.net traffique.net
trahic.ru traffixer.com
try-rx.com tsyndicate.com
tsan.net
tt-ipd.info tt-ipd.info
turn-up-life.life twincitiescarservice.com
udayavani.com typimga.pw
unisexjewelry.org tzritel.tk
upupa.net ugguk.online
urdoot.win uhodzalijami.ru
url-extractor.xyz uk-zheu20.ru
valmetrundan.se ukkelberg.no
vapeface.club unimodemhalfduplefw.pen.io
vavilone.com universals.com.ua
viagra.pp.ua url-img.link
viandpet.com userequip.com
videonsk.com vasileostrovsky-rayon.ru
vids18.site vipps.com.my
viel.su viven.host.sk
villacoloniale.com vkontarkte.com
vip2ch.com vostoktrade.info
vipromoffers.com vucms.com
vipsexfinders.com warezkeeper.com
virtualbb.com watchinf.com
visitcambridge.org waysbetter.cn
vkontaktemusic.ru webscouter.net
vote-up.ru website-datenbank.de
vrnelectro.ru wedding0venues.tk
vroze.com winx-play.ru
w-journal.ru witclub.info
webnode.me
website-analyzer.info
website-speed-checker.site
weburok.com
welcomeauto.ru
weprik.ru
worldlovers.ru worldlovers.ru
writingservices17.blogspot.ru www.ehscloud.cn
wttavern.com wyniki-lista.pl
wufak.com x-stars.ru
wurr.voila.net xlolitka.com
x-lime.com xn--80aaagvmjabrs1aoc9luc.xn--p1ai
x-porno.video xn--80aodinpgi.xn--p1ai
xchangetrak.com
xdoza.com
xgftnlrt.bloger.index.hr
xmnb.net
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xn----7sbahjd3btneuw1joc.xn--p1ai
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn--24-glceagatoq7c2a6ioc.xn--p1ai
xn--d1abj0abs9d.in.ua
xn--h1aakne2ba.xn--p1ai
xnxx-n.com xnxx-n.com
xxx-cam.webcam xtubeporno.net
xxxdatinglocal.us xxxkaz.org
xxxhdvideo.site xzlive.com
xxxmania.top yaderenergy.ru
xz618.com yellowfootprints.com
yellowads.men yellowstonesafaritours.com
yginekologa.com
you-shall-not-pass.is74.ru
yougame.biz yougame.biz
youjizz.vc
youporn-forum.ga youporn-forum.ga
yourads.website youthreaders.com
yun56.co yuweng.info
zajm-zalog-krasnodar.ru zaimite.ru
zeroredirect2.com zajm-pod-zalog-nedvizhimosti.ru
zrelaya.pw zerocash.msk.ru
zeroredirect11.com
zeroredirect12.com
zeroredirect5.com
zocaparj.kz
zog.link
zoominfo.com
zverokruh-shop.cz

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
03p.info 00author.com
6128786.com 1-99seo.com
6hopping.com 1pamm.ru
7a2rnuey1tw9ar.ru 2drittel.de
7search.com 4rent.ru
7wind.ru 7zap.com
8xv8.com a14download.com
aa81bf391151884adfa3dd677e41f94be1.com aac52006ec82a24e08b665f4db2b5013f7.com
aaa38852e886ac4af1a3cff9b47cab6272.com acheterviagrafr24.com
abwa.tk adconscious.com
adserver-e7.com adman.se
adultactioncam.com akama.com
advocatemsk.ru all-streaming-media.com
afora.ru alloysteel.ru
agardomains.com animalia-life.club
aksonural.ru apibatbrowsecom-a.akamaihd.net
alfa-img.com appleid-verification.com
algarveglobal.com appsaurus.com
allkrim.com arendakvartir.kz
allornamenti.com aviav.ru.com
anlimebel.kiev.ua azbuka-mo.ru
anniemation.com bad-stars.net
apiqualitinknet-a.akamaihd.net balook.com
aproposde.com bayareaaeroclub.org
arenda-yeisk.ru best-seo-solution.com
artparquet.ru birzha-truda.eu
asrv-a.akamoihd.net blackcurranthumidifiers.site
autoplate.info blacktwhite.com
aviav.co blackwitchcraft.ru
awency.com blockworld.ru
azartmix.com blog.koorg.ru
azbukadiets.ru bluejays-jerseys.us
backgroundpictures.net bobinoz.com
biglistofwebsites.com brandimensions.com
bioscorp.ru browserprotecter.com
brokergid.ru btvn.ru
bronzeaid-a.akamaihd.net buchananshardware.com
bumskontakte.org bukleteg.xyz
bumble.cheapwebsitehoster.com
buron.pw buron.pw
business-suggested.tk buyessaynow.biz
buytizanidineonlinenoprescription.blogspot.com buyscabiescream.com
cam-kontakte.org calstaterealty.us
cartechnic.ru camdenmemorials.com
cartierlove2u.xyz camsex.xxx-cam.webcam
cartujano-pre.de canrole.com
catalogs-parts.com cardul.ru
caulderwoodkennels.com cartierreplica.pw
cayado.snn.gr casinorewards.info
ccjp.eu cheapmoncler.pw
civilwartheater.com cheapmoncler.win
collegerentals.com checkhit.com
comissionka.net cialiswithoutadoctor.net
community.allhiphop.com club-samodelkin.ru
compliance-ivan.top coderstate.com
creditwell.ru companies-catalog.com
cubs-jerseys.us compliance-mark.top
d-black.bz cottageofgrace.com
davebestdeals.com cryptoswap.biz
detalizaciya-tut.biz cutt.us
detoxmed24.ru cyse.tk
directivepub.com czat.wp.pl
domain-tracker.com delitime.info
dostavka-v-krym.com demian.kz
elektrischezi.canalblog.com dfiles.me
elektrischeziga.livejournal.com dollartree.info
elektrischezigarette2.onsugar.com doxysexy.com
elektrischezigaretten2.beeplog.com dscaas.website
eloconcream.blogspot.com eaplay.ru
educatemetv.com
efnor-ac.com
egovaleo.it
election.interferencer.ru
elexies.info
eluxer.net
elvenar.com
emediate.eu emediate.eu
engines-usa.com empathica.com
englishdictionaryfree.com enternet.ee
erotag.com
erotikchat-24.com
essay-edu.biz essay-edu.biz
evrotekhservis.ru extstat.com
ezigaretteshop2.vefblog.net ezofest.sk
farmingworm.com fatmaelgarny.com
fickkontaktehobbyhuren.com findanysex.com
fil.ru findpik.com
flash4fun.com firstsiteguide.com
flipper.top flavors.me
flirt4free.com frankofficial.ru
fotzen-ficken.com freefoto.ca
frameimage.org freevpn.space
freakycheats.com germetiki.com.ua
freenom.link getaclueamerica.com
fun2cell.net getmiro.com
fusoradio.info goodly.pro
fxund.us gsasearchenginerankerdiscount.com
game-mmorpg.net gunsvicceadadebt.tk
games.kolossale.ru hdwallpapers-free.com
get-free-traffic-now.com health-medical-portal.info
getdot.ru help.tpu.ru
goforexvps.com highland-homes.com
hahashka.ru hop.clickbank.net
hatedriveapart.com hoverboard360.nl
hidemyass.com hustoon.over-blog.com
hikesearch.net ilo134uloh.com
hitsbox.info images-graphics-pics.com
honyaku.yahoofs.jp imagine-ex.co
hotgirlhdwallpaper.com imediadesk.com
housekuba.org impisr.ru
houseofgaga.ru investpamm.ru
htmlcorner.com j33x.com
humanorightswatch.org jeremyeaton.co
iblogpress.xyz jus0wil.pp.ua
iplusbit.blogspot.co.za kareliatobacco.ru
istizanidineanopiate.blogspot.com kashubadesign.ru
kinky-fetishes.com kinky-fetishes.com
kinosed.net kinofak.net
klin3952.ru kinohall.ru
kosynka-games.ru kozhniebolezni.com
languagecode.com krasivye-devushki.net
lankarns.com krasnodar-avtolombards.ru
laptop-4-less.com laptoper.net
lashstudia.ru livejasmin.com
letmacworkfaster.site liver-chintai.org
letmacworkfaster.world m4ever.net
linkbux.ru mailemedicinals.com
livingcanarias.com malls.com
lomza.info max-p.men
louboutinreplica.pw micasainvest.com
lsp-awak-perikanan.com microsearch.ru
lyrster.com mobicover.com.ua
mabdoola.blogspot.com moskva.nodup.ru
mantramusic.ru mp3downloadhq.com
manve.info mybackgroundlandscape.blogspot.com
marcogrup.com mycouponizemac.com
marketingvici.com mylesosibirsk.ru
medkritika.ru netpics.org
meds-online24.com nfljerseyscheapbiz.us
meet-flirt-dating.com njpalletremoval.com
metafilter.com norththeface.store
mini.7zap.com nsatc.net
modenamebel.ru nuit-artisanale.com
my-bc.ru nvssf.com
mycaf.it oda.as
mygameplus.com olgacvetmet.com
myyour.eu omoikiri-japan.ru
nalogovyy-kodeks.ru onemactrckr.com
napalm51.nut.cc online247.ml
netcheckcdn.xyz onlinetvseries.me
nicefloor.co.uk openfrost.com
nikitabuch.com openmultipleurl.com
nmrk.ru otbelivanie-zubov.com
nrv.co.za partnersafe.men
offtopic.biz pb-dv.ru
oivcvx.website pdns.download
online-podarki.com pensplan4u.com
onload.pw perm.xrus.org
optitrade24.com
paccohichetoti.ml
pacificair.com
parvezmia.xyz
pastaleads.com
pharmacyincity.com pharmacyincity.com
picquery.com piccdata.com
pinapchik.com pizdeishn.com
playtap.us poffet.net
poems.com.ua poiskzakona.ru
polimga.pw popup-hgd.xyz
popserve.adscpm.net pornobrazzers.biz
pornosexrolik.com
pornoted.com
pornowarp.info pornowarp.info
pourvous.info portal-eu.ru
powitania.pl powc.r.ca.d.sendibm2.com
prombudpostach.com.ua productarium.com
proprostatit.com psoriasis-file.trade
pts163.ru qualitymarketzone.com
puserving.com quizzitch.net
quangcaons.com recordpage-a.akamaihd.net
reklamuss.ru ricorsogiustizia.org
rock-cafe.info ritlweb.com
rocketchange.ru
rossmark.ru rossmark.ru
rutor.vip rvzr-a.akamaihd.net
s8-nowy-wygraj.comli.com samolet.fr
sarf3omlat.com sayyoethe.blogspot.co.za
savetubevideo.com sbricur.com
sbetodiodnye-lampy.ru sbtdesign.co.uk
scanner-jane.top scanner-elena.top
scanner-john.top seksvideoonlain.com
scanner-margo.top serving.adbetclickin.pink
scanner-olga.top sexobzor.info
scanner-willy.top sexvporno.ru
search-goo.com shareyards.com
secretscook.ru shoplvlv.us
segol.tv shtaketniki.kz
selectads.men shtora66.ru
seo-smm.kz simple-image.com.ua
serptehnika.ru sitopreferito.it
sextracker.de smartpet.ru
sfd-chess.ru smartshoppymac.com
sharebutton.net sportbetfair.com
sherlock.se storehouse.ua
shoesonlinebuy.cn stroyplus.ru
simplepooltips.com studentguide.ru
site-analyzer.com studiofaca.com
sivs.ru support.nopeas.sk
skylta.com sureone.pro
socialmadesimple.com susanholtphotography.com
sohoindia.net svetlotorg.ru
solnplast.ru szucs.ru
speeddream.xyz tandvardshuset.net
spinazdrav.ru taxi-v-eisk.ru
stuff-about-money.com
swagbucks.com
taximytishi.ru
td-l-market.ru
tdsing.ru
teencastingporn.com teencastingporn.com
threecolumnblogger.com testbotprocessor44.com
tizanidineandhydrocodone.blogspot.com texbaza.by
torrent-newgames.com thewomenlife.com
treasuretrack-a.akamaihd.net tizanidinehcl4mgtab.blogspot.com
trustedhealthtips.com tizanidineinteractions.blogspot.com
tuberkulezanet.ru torrentred.games
tuckpointingmasonrysystems.com toursmaps.com
turizmus.us truck-land.life
turvgori.ru tubeline.biz
udsgame.online uhdtv.website
ukrtvir.com.ua uhodzalijami.ru
validdomain.xyz unpredictable.ga
vbtracker.net vapsy.com
ventelnos.com vezuviy.su
vereo.eu villakohlanta.nu
video-hollywood.ru virtualbb.com
videochat.life virus-respirators.com
videochaty.ru vizitki.net
vkak.ru
vksaver-all.ru vksaver-all.ru
w-journal.ru vzlomfb.com
wahicbefa31.soup.io wahicbefa31.soup.io
watchmyfb.pl watchmygf.net
wheelchairliftsarea.com waterefficiency.co
wholesalecheapjerseysfree.com weblo.com
windowssearch-exp.com webmasterhome.cn
womens-journal.net williamrobsonproperty.com
wordpresscore.com winner7777.net
worldlovers.ru ws.ampower.me
wowcasinoonline.ooo www.pinnacle-bets.com
www.get-free-traffic-now.com www.wohnkabinen-shop.de
x-stars.ru
xdoza.com xdoza.com
xjlottery.com xkaz.org
xn--oogle-wmc.com xmnb.net
xvideosbay.com xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
yadro.ru xuki.us
youradulthosting.com xxxtubesafari.com
yousense.info yur-p.ru
zdesformula.ru zajm-pod-zalog-nedvizhimosti.ru
zhacker.net zdesoboi.com
zhorapankratov7.blogspot.com zerocash.msk.ru
zolotoy-lis.ru

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.221 0; 10.1.0.18 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3114 ### Version: V4.2022.05.3115
### Updated: Thu May 19 10:03:51 UTC 2022 ### Updated: Thu May 19 22:01:32 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3114 ### Version: V4.2022.05.3115
### Updated: Thu May 19 10:03:51 UTC 2022 ### Updated: Thu May 19 22:01:32 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##