vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 02:29:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2917 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-14 10:09:10 +00:00
parent 0524e1620e
commit 459c82be14
48 changed files with 87490 additions and 87490 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2916 #### Version: V4.2022.02.2917
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2916 #### Version: V4.2022.02.2917
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2916 #### Version: V4.2022.02.2917
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10800
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2916 #### Version: V4.2022.02.2917
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10804
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10804
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10804
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10804
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10804
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10804
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10804
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2916 2917

10804
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10804
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.127.56.26 0; 13.89.38.15 0;

10804
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10804
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10804
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10804
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.127.56.26 0; 13.89.38.15 0;

10804
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.127.56.26 0; 13.89.38.15 0;

10804
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Aboundexbot Crescent
Anthill Freeuploader
BBBike GermCrawler
CheTeam Haansoft
GetWeb HonoluluBot
GrapeFX MauiBot
PHPCrawl Mojeek
Seomoz Shodan
SiteCheckerBotCrawler Telesphorep
iubenda-radar Zade

312
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 01h4x.com
404checker
404enemy 404enemy
80legs 80legs
ADmantX
AIBOT
Aboundex
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot
AllSubmitter
Alligator
AlphaBot AlphaBot
Anarchie
Anarchy
Anarchy99 Anarchy99
Ankit Ankit
Aspiegel Anthill
Apexoo
Asterias
Atomseobot
Attach Attach
BDCbot AwarioSmartBot
BDFetch BDFetch
BackDoorBot BLEXBot
BackWeb BackStreet
Bandit BacklinkCrawler
BatchFTP Badass
Battleztar Bazinga Battleztar Bazinga
Blow BetaBot
Buck Black Hole
Buddy Boardreader
BuiltBotTough BuiltWith
Bullseye
BunnySlippers BunnySlippers
BuzzSumo CSHttp
CATExplorador Cegbfeieh
CODE87 CensysInspect
CheTeam CheTeam
CheeseBot Chlooe
CherryPicker Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot
Copier
Copyscape Copyscape
CrazyWebCrawler Cosmos
CrunchBot Craftbot
Crawling at Home Project
Custo Custo
DBLBot
DIIbot DIIbot
Deusu DSearch
DigitalPebble Dirbuster
Discoverybot
Dispatch Dispatch
DnyzBot
DomCopBot DomCopBot
Domains Project Domains Project
Download Wonder Download Wonder
Drip Dragonfly
Ecxi Ecxi
Express WebPictures EirGrabber
EroCrawler
Extractor Extractor
Extreme Picture Finder ExtractorPro
Ezooms EyeNetIE
FHscan
FemtosearchBot FemtosearchBot
Fimap
FlashGet FlashGet
Flunky Flunky
Foobot Fuzz
FrontPage Fyrebot
GT::WWW GT::WWW
GalaxyBot
Genieo Genieo
GermCrawler Getintent
GetRight Gigabot
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla Grabber
Grafula Grafula
GrapeFX GrapeshotCrawler
GridBot
HMView
HTMLparser
HTTrack HTTrack
Haansoft Haansoft
HaosouSpider
Heritrix
Hloader Hloader
HonoluluBot
IDBTE4M IDBTE4M
IRLbot IDBot
Image Fetch Id-search
IndeedBot Image Sucker
InfoTekies Intelliseek
InterGET Iskanie
Iria JOC Web Spider
JamesBOT
Jetty
JikeSpider JikeSpider
Kinza Joomla
Kozmosbot Kozmosbot
Larbin LNSpiderguy
LWP::Simple
Lanshanbot
Leap Leap
LeechFTP LeechFTP
LexiBot Lftp
LibWeb
Libwhisker
Lightspeedsystems Lightspeedsystems
LinkWalker Likse
LinksManager LinkScan
LinqiaMetadataDownloaderBot Linkbot
LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider Lipperhey Spider
Lmspider Lmspider
Ltx71
MFC_Tear_Sample
MJ12bot MJ12bot
MSFrontPage MSFrontPage
MSIECrawler Mag-Net
Magnet Magnet
Majestic-SEO
MarkMonitor MarkMonitor
Mass Downloader
MauiBot MauiBot
MeanPath Bot Minefield
Meanpathbot Mister PiX
Metauri Moblie Safari
Mojolicious
MolokaiBot MolokaiBot
Mozlila Mr.4x3
Musobot Musobot
NPbot NICErsPRO
Name Intelligence
Nameprotect Nameprotect
NearSite Net Vampire
NetAnts
NetLyzer NetLyzer
Netcraft NetZIP
Nettrack
Netvibes Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler
Niki-bot Niki-bot
NimbleCrawler
Nmap Nmap
OpenLinkProfiler Not
Openvas Nuclei
OrangeBot Octopus
PHPCrawl Offline Navigator
POE-Component-Client-HTTP OnCrawl
PageAnalyzer PageScorer
PageThing.com
Panscient Panscient
Papa Foto Papa Foto
PeoplePal PeoplePal
Petalbot Picscout
Picsearch Piepmatz
Pockey Pixray
ProPowerBot PleaseCrawl
Probethenet
PxBroker PxBroker
PyCurl
Quick-Crawler Quick-Crawler
RankActive RankActive
RankActiveLinkBot ReGet
RankingBot RepoMonkey
RankingBot2
RankurBot
Re-re
RealDownload
RebelMouse
RedesScrapy
Ripper
Rogerbot Rogerbot
SBIder
SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats Scanbot
SISTRIX ScrepyBot
SalesIntelligent Searchestate
ScanAlert
ScreenerBot
SearchmetricsBot
Seekport Seekport
SemanticJuice SemanticJuice
Semrush SemrushBot
SentiBot SiteSucker
SeobilityBot Sitevigil
Seomoz
Shodan
Siphon
Site Sucker
SiteExplorer
SmartDownload SmartDownload
Snapbot
Snoopy Snoopy
Sosospider Sogou web spider
SpaceBison
Spammen Spammen
SpankBot Spanner
Spbot Spbot
SputnikBot
Sqlmap Sqlmap
Sqlworm Sqlworm
Sqworm Sqworm
Stripper Sucker
Sucuri
SuperBot SuperBot
Surfbot Surfbot
SurveyBot
Suzuran
Swiftbot
Telesphoreo
Telesphorep
The Intraformant
TightTwatBot
Titan Titan
Toata Tracemyfile
Toweyabot Trendiction
Trendictionbot
TurnitinBot TurnitinBot
TwengaBot
Twice Twice
URLy Warning URLy.Warning
UnisterBot
VB Project
VeriCiteCrawler VeriCiteCrawler
Virusdie
Voil Voil
Voltron Voltron
WEBDAV WISENutbot
WPScan WWW::Mechanize
WWWOFFLE
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD WeSEE
Web Pix Web Collage
Web Fetch
Web Sauger Web Sauger
Web Sucker Web Sucker
WebAuto WebBandit
WebFuck WebEnhancer
WebImageCollector WebWhacker
WebReaper Webalta
WebStripper
WebSucker
WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
Whack WebsiteQuester
Whatweb Widow
Who.is Bot Wprecon
WinHTTrack Xaldon WebSpider
Wotbox YoudaoBot
Xaldon_WebSpider Zeus
Xenu ZmEu
Zitebot ZoomBot
ZoominfoBot
ZumBot
adscanner adscanner
arquivo.pt arquivo.pt
coccocbot-web autoemailspider
cognitiveseo backlink-check
cah.io.community
clark-crawler
com.plumanalytics
crawler.feedback
crawler4j crawler4j
domainsproject.org demandbase-bot
internetVista monitor evc-batch
magpie-crawler facebookscraper
meanpathbot gopher
heritrix
instabid
lwp-trivial
muhstik-scan muhstik-scan
netEstate NE Crawler
page scorer page scorer
pcBrowser pcBrowser
s1z.ru plumanalytics
scan.lol
seocompany.store
seoscanners seoscanners
seostar
serpstatbot serpstatbot
sp_auditbot sexsearcher
spyfu sitechecker.pro
sogouspider
sysscan
tAkeOut tAkeOut
trendiction.com webprosbot
webgains-bot x22Mozilla
zauba.io xpymep1.exe
zgrab zgrab

286
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404checker ADmantX
AIBOT AIBOT
ALittle Client
ASPSeek ASPSeek
Abonti AhrefsBot
Aboundexbot
AiHitBot AiHitBot
AllSubmitter Alexibot
AlphaBot Anarchy
Asterias Anarchy99
Atomseobot Anthill
Attach Aspiegel
AwarioSmartBot BDCbot
BLEXBot BackWeb
BackDoorBot Badass
BackStreet
BacklinkCrawler
Bandit Bandit
Battleztar Bazinga Barkrowler
Bigfoot Bigfoot
Bitacle Bitacle
BlackWidow BlackWidow
Blackboard Brandprotect
Blow Brandwatch
BlowFish
Buck Buck
Buddy
Bullseye Bullseye
CODE87 CODE87
Calculon ChinaClaw
CazoodleBot
CheTeam
CherryPicker
Citoid Citoid
Claritybot
Cliqzbot Cliqzbot
Cocolyzebot
Cogentbot Cogentbot
Copier CopyRightCheck
Craftbot
Crawling at Home Project
CrazyWebCrawler
Crescent
Curious
Custo Custo
DBLBot DBLBot
DIIbot DIIbot
DSearch Demon
Devil
DigitalPebble
Dispatch
DittoSpyder
DnyzBot DnyzBot
DomainAppender DomCopBot
DomainSigmaCrawler
Domains Project
Dotbot Dotbot
Download Wonder Download Wonder
Dragonfly
Drip Drip
ECCP/1.0 EMail Siphon
Ebingbong EasyDL
Ecxi
EirGrabber EirGrabber
Extractor EroCrawler
Evil
Exabot
ExtractorPro
EyeNetIE
Ezooms Ezooms
FHscan Firefox/7.0
Flunky
Freeuploader
Fuzz
FyberSpider
G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot
Genieo
GermCrawler GermCrawler
Gigabot GetRight
Go!Zilla
Go-Ahead-Got-It
Gotit
GrabNet
Grabber
Grafula Grafula
HEADMasterSEO HEADMasterSEO
HTMLparser HTMLparser
HTTrack
HaosouSpider
Harvest Harvest
Heritrix IDBTE4M
Hloader
IDBot
IRLbot IRLbot
Image Fetch Id-search
IndeedBot IlseBot
Intelliseek
Iskanie Iskanie
IstellaBot IstellaBot
Jbrofuzz JOC Web Spider
Jorgee JetCar
Kinza Jyxobot
Kozmosbot Kenjin Spider
Lanshanbot Keyword Density
Leap LWP::Simple
LeechFTP Larbin
LibWeb LexiBot
Lftp
Libwhisker Libwhisker
LieBaoFast Likse
LinkScan LinkextractorPro
Linkbot
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker
Lmspider Lmspider
MFC_Tear_Sample Ltx71
MQQBrowser MQQBrowser
MSFrontPage MSFrontPage
MSIECrawler Mag-Net
Majestic SEO Magnet
Mail.RU_Bot
Majestic12
MarkWatch MarkWatch
Mass Downloader Mass Downloader
Mata Hari
Mb2345Browser Mb2345Browser
Mediatoolkitbot
Metauri Metauri
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Mister PiX Mister PiX
Mojeek Moblie Safari
Mojolicious Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner
Mozlila
Mr.4x3
Musobot
NICErsPRO NICErsPRO
Name Intelligence Nameprotect
Navroad
NearSite
Needle Needle
Nessus
Net Vampire Net Vampire
NetAnts
NetLyzer NetLyzer
NetMechanic
NetSpider
Netcraft Netcraft
Nettrack Nettrack
Netvibes
NextGenSearchBot NextGenSearchBot
Niki-bot
Nimbostratus Nimbostratus
Nmap Nmap
Offline Explorer Octopus
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler OpenVAS
Openfind PageAnalyzer
OrangeBot
OutclicksBot
PECL::HTTP
PageGrabber
Panscient
Papa Foto Papa Foto
Pi-Monster
Picscout
Picsearch Picsearch
Piepmatz Piepmatz
Pockey PleaseCrawl
ProWebWalker
Probethenet
Pump Pump
RSSingBot
RankActive RankActive
RankFlex
RankingBot RankingBot
RankingBot2
RealDownload RealDownload
Reaper
RebelMouse RebelMouse
Recorder RedesScrapy
RepoMonkey
Ripper
Rogerbot Rogerbot
SBIder SBIder
SEOkicks-Robot
SEOlyticsCrawler
SEOstats SEOstats
SalesIntelligent SMTBot
Seekport Scrapy
SemanticJuice Searchestate
SearchmetricsBot
Semrush Semrush
SentiBot SentiBot
SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz
Shodan
Siphon
Site Sucker Site Sucker
SiteExplorer
Sitebeam Sitebeam
Siteimprove
Sitevigil Sitevigil
SlySearch Sosospider
SmartDownload SpaceBison
Snapbot Spanner
SpankBot
Spbot Spbot
Spinn3r Spinn3r
SputnikBot Sqlworm
Steeler Steeler
Stripper Sucker
Sucuri
SuperHTTP
Surfbot Surfbot
Suzuran Suzuran
Szukacz
T0PHackTeam T0PHackTeam
T8Abot Teleport
TeleportPro TeleportPro
Telesoft
Telesphoreo Telesphoreo
The Intraformant TheNomad
Thumbor TightTwatBot
Titan
Toweyabot
Tracemyfile Tracemyfile
Trendiction Trendiction
Turingos Trendictionbot
Typhoeus Typhoeus
UnisterBot V-BOT
Upflow
VB Project VB Project
VCI VelenPublicWebCrawler
Vagabondo
VeriCiteCrawler VeriCiteCrawler
Voltron VidibleScraper
WBSearchBot
WEBDAV WEBDAV
Wallpapers/3.0 WWW-Mechanize
WallpapersHD
Web Auto
Web Collage Web Collage
WebCollage Web Enhancer
WebBandit
WebCopier
WebFetch WebFetch
WebImageCollector
WebLeacher WebLeacher
WebStripper WebPix
WebSucker
WebWhacker WebWhacker
WebZIP WebZIP
WebmasterWorldForumBot Webalta
Webshag
Website Quester Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster Widow
Whack
WiseGuys Robot
Wonderbot Wonderbot
Wotbox
Xaldon_WebSpider
Xenu Xenu
YoudaoBot YoudaoBot
Zade Zade
ZoomBot ZmEu
ZyBorg ZoominfoBot
arquivo-web-crawler arquivo-web-crawler
cah.io.community arquivo.pt
check1.exe autoemailspider
cognitiveseo cognitiveseo
crawl.sogou.com com.plumanalytics
dataforseo.com crawler4j
demandbase-bot
domainsproject.org
evc-batch
gopher gopher
internetVista monitor internetVista monitor
ips-agent isitwp.com
lwp-request iubenda-radar
muhstik-scan lwp-trivial
mediawords
netEstate NE Crawler netEstate NE Crawler
probe-image-size page scorer
ripz s1z.ru
scan.lol scalaj-http
seobility siteripz
seocompany.store
seoscanners
seostar
sitechecker.pro
sogouspider
sysscan sysscan
tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com
voyagerx.com
webgains-bot
webprosbot webprosbot
zgrab x22Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
7zap.com fatfasts-4tmz.com
erectile.bid ideibiznesa2015.ru
evehemming.blogspot.com.au meteostate.com
goodhousekeeping.com ningessaybe.me
listiseltemournlan.gq pokerniydom.ru
manuscript.su sexyali.com
subj.ukr-lit.com sloopyjoes.com
track-rankings.online uogonline.com
unrealcommander.biz velobikestock.com
youdesigner.kz xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai

926
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 123cha.com
00it.com
00webcams.com
178evakuator178.ru
1millionusd.xyz 1millionusd.xyz
1pamm.ru
1stat.ru 1stat.ru
2728fb936f0.com 2itech.ru
2pxg8bcf.top
2x2fan.ru
34.gs 34.gs
45en.ru 3th.co.in
4webmasters.org 4inn.ru
66cpwgln.space 4kpics.rocks
70casino.online 7minuteworkout.com
72-news.com 7yue.org
7search.com 8lufu.com
98oi.ru
aa625d84f1587749c1ab011d6f269f7d64.com
aaa38852e886ac4af1a3cff9b47cab6272.com
abcdefh.xyz
absolutelyanalog.com absolutelyanalog.com
actionnooz.com abusora.com
adbetclickin.pink adktrailmap.com
adloads.net
adssafeprotected.com
adtech.de
adtech.us adtech.us
ageofclones.com adtology.com
alert-fjg.xyz adverster.com
alfa9.com agreda.pluto.ro
allnews.md albumsuper.info
ananumous.ru alert-fdm.xyz
android-systems.ru alert.scansafe.net
androids-store.com all4wap.ru
apifasterlightin-a.akamaihd.net amazon-adsystem.com
apispringsmartne-a.akamaihd.net anaksma.info
appfixing.space anti-virus-removal.info
arabseyes.com apelsinnik.website
artdestshop.eu apimegabrowsebiz-a.akamaihd.net
asacopaco.tk apiportalmorecom-a.akamaihd.net
asmxsatadriverin.aircus.com appmsr.org
approved.su
appsecurityr.com
artdeko.info
as5000.com
asseenontv.ru asseenontv.ru
autogrep.ru atelielembrancaqueencanta.com.br
aviav.co aussie-prizes.com
autoblogger24.info
autoloans.com
aviva-limoux.com
avon-severozapad.ru
azazu.ru azazu.ru
backlinkwatch.com azbukadiets.ru
balla-boo.se babyguns.ru
bannerbank.ru backlink4u.net
barnfurnituremart.com balitouroffice.com
bcmp.org balkanfarma.org
beachtoday.ru balkanfarma.ru
besthomemadepornsites.com basedecor.ru
bestsexyblog.com belinvestdom.by
betterscooter.com berdasovivan.ru
beztuberkuleza.ru berrymall.ru
bharatdefencekavach.com best-seo-software.xyz
bimmerpost.com bestchoice.cf
bizcheapjerseyswholesalechina.com bestempresas.es
bkns.vn bestfortraders.com
bnt-team.com bfz.biz
bookmark4you.com bigcareer.info
bouda.kvalitne.cz biketank.ga
braintobe.win bird1.ru
brandbucket.com bitcoinpile.com
brandedleadgeneration.com blogporn.in
brandehk.dk blogrankers.com
brateg.xyz boeuklad.com
bristolhotel.com.ua brainxs.us
bronzeaid-a.akamaihd.net briomotor.co
browserprotecter.com budilneg.xyz
btc4u.ru bukleteg.xyz
btnativenav.com bumskontakte.org
bum.com.ru burger-tycoon.com
buypillsorderonline.com buy-meds24.com
buyviagraa.blogspot.com buyhoverboard.com
buzzonclick.com buypanicdisorderpill.com
cactussoft.biz buzzzg.men
camel-beach.com byme.se
candelluxsklep.pl callstevens.com
cam-kontakte.org
candycrushshop.com
canrole.com canrole.com
cartechnic.ru cardsharp1.ru
caveavins.fr carina-sy.de
cbb1smartlist12.click cartierreplica.pw
cbcseward.com cbcseward.com
cbox.ws cdncash.org
cementaresearch.se cenokos.ru
centrumcoachingu.com cfjump.com
cfcl.co.uk cheapestjerseys-wholesale.com
chatroulette.world chinaelitecheapjerseys.com
chcu.net
cheapcigarettesc.info
chocolateslim-original.com chocolateslim-original.com
christianlouboutinsaleoutletonline.us chocoslim.pro
cialgenisrx.com choice-credit.ru
cialisndbrx.com christianlouboutinoutlet.win
cleanersoft.com ci.ua
clickbank.net cialischmrx.com
clips.ua.ac.be cityadspix.com
coaching-netz.info clarithromycin500mg.com
cocyq.inwtrade.com clmforexeu.com
com-secure.download colehaanoutlet.store
compliance-barak.xyz comissionka.net
compucelunlock.net community.allhiphop.com
confib.ifmo.ru compiko.info
contentlook.co compliance-alexa.xyz
cookie-law-enforcement-aa.xyz compliance-ivan.xyz
cordstrap.cc conciergegroup.org
crafthubs.com continent-e.tv
crazyprotocol.com coprofam.org
custom-product-labels.com counter.yadro.ru
dangphoto.trade crynet.cc
darkages.info cubs-jerseys.us
dellalimov.com custodieva.ru
descargar-musica-gratis.net customchocolate.business-for-home.com
dev-seo.blog cwetochki.ru
diegolopezcastan.com damedingel.ya.ru
diesel-parts28.ru danashop.ru
dieswaene.com data-mining.tk
dimaka.info dating-time-now.com
dimfour.com datract.com
dinkolove.ya.ru dawlenie.com
directivepub.com designdevise.com
discovertreasurenow.com dev33.dioniqlabb.se
divatraffic.com dildofotzen.net
dividendo.ru discountliv.com
documentbase.net diusyawiga.tk
domainsphoto.com dkvorota.ru
dominterior.org dojki365.online
dosug-lux.ru dollartree.info
doxysexy.com domainroam.win
dptaughtme.com dprtb.com
drev.biz draniki.org
drpornogratisx.xxx dynainbox.com
drugs-no-rx.info
drunkenstepfather.com
e-commerce-seo.com
e-poker-2005.com
eaplay.ru
earn-from-articles.com
earncash.com.ua earncash.com.ua
ekobata.ru efnor-ac.com
electronicadirect.com ekn-art.se
elektrischezigaretten1.blogse.nl electrik-avenue.com
elektrozigarette2.webs.com elektrozigaretten1.tumblr.com
elektrozigarettekaufen2.blogse.nl elkacentr.ru
empis.magix.net eluxer.net
en.home-task.com embedle.com
enpolis.ru emergencyneeds.org
englate.com
eonpal.com
ereko.ru ereko.ru
ero-advertising.com eropho.com
etur.ru everflora.ru
eu-cookie-law.info executehosting.com
evidencecleanergold.com exportshop.us
exmasters.com extads.net
exoneration-project.us ez8motelseaworldsandiego.com
ezigarettenkaufen1.deviantart.com f07.de
facialporntube.com fapgon.com
fashion-stickers.ru fashion.stellaconstance.co
fenoyl.batcave.net favoritemoney.ru
ferretsoft.com fefo.gdn
filkhbr.com fettefrauen.net
florida-tourism.net fickkontakte.org
formseo.com filesvine.com
forum.doctissimo.fr filmi-v.online
forum.tvmir.org fitnesspiks.com
franch.info flagcounter.me
free-fbook-traffic.com flashlarevista.com
freelinkbuilding.website.tk flex4launch.ru
freeuploader.com flipper.top
freevpn.space folowsite.com
fuckbuddybestgilf.info formularz-konkurs.tk
fugarif.ga forpackningsutveckling.se
fx-brokers-review.com forum-engineering.ru
freefoto.ca
freeuploader.ml
freshmac.space
frighteningremain.cf
funnel.co.za
funnypica.com
funtoonez.com
galeon.com galeon.com
galeria-zdjec.com gallerily.com
games.kolossale.ru galleryawesome.com
geckoandfly.com ge0ip.com
gembird.com get-free-social-traffic.com
getlaid-xxxhookupdirect.com get-seo-domain.com
gg-arena.ru girls-ufa.ru
gifspics.com girlsatgames.ru
glass-msk.ru gogps.me
goldenggames.com goodhousekeeping.com
gsasearchenginerankerhelp.com gopixdatabase.com
guod.me gpms.org.my
handicapvantoday.com gratuitxblcodes.com
handsandlegs.ru greatfind-a.akamaihd.net
hcate.com green-tea.tv
hd-film.pl gsasearchengineranker.top
hdapp1008-a.akamaihd.net gsbs.com.ua
heroz.fr gsmlab.pl
hikesearch.net gyffu.com
home-task.com happy.new.yeartwit.com
homeart.space hayate.biz
homeinns.com hd720kino.ru
hostcritique.com hdpixent.com
hostsshop.ru healbio.ru
hoterika.com helicalpile.us
huimin764128.com hermesbirkinhandbagoutlets.com
humanelydrew.com hmywwogw.bloger.index.hr
i4track.net hobild.net
ico.re hollywoodactress.info
ilmexico.com homemade.gq
imabase.com hopeonthestreet.co.uk
imagui.eu howlongdoestizanidinestayinyoursystem.blogspot.com
img.wallpaperstock.net husky-shop.cz
importchinacoach-teach.com i-service.kz
infospot.pt ie.57883.net
intermesh.net igru-xbox.net
ip-guide.com ilte.info
itservicesthatworkforyou.com imagecoolpub.com
jacago.com imgata.com
jackpotchances.com import-sales.com
incolors.club
incomekey.net
indetiske.ya.ru
instabid.tech
interferencer.ru
interior-stickers.ru
internetartfair.com
iqoption-bin.com
isitpaleo.info
istmira.ru
iyasimasennka.com
j-times.ru
janerikholst.se janerikholst.se
janettabridal.com jav-fetish.com
jestr.org jav-idol.com
jimmychoosale.online jeremyeaton.co
jerseysbizwholesalecheap.com
jongose.ninja jongose.ninja
junglenet-a.akamaihd.net jyvopys.com
kalandranis.gr kam-dom.ru
kamalsinha.com kamalsinha.com
katadhin.co kazinogames.lv
keywords-monitoring-success.com keywords-monitoring-your-success.com
keywordsdoctor.com keywordspay.com
kihi.gdn kmd-pto.ru
kimcurlrvsms.com kovrenok.ru
kinomaniatv.pl krasivoe-hd.net
kredit-blog.ru kredit-blog.ru
kretpicf.pw kurbappeal.info
ktotut.net landreferat.ru
kupit-adenu.ru lechenie-gemorroya.com
kutikomi.net lennatin.info
lakomka.com.ua lexiangwan.com
landmania.ru lider-zhaluzi.kiev.ua
landoftracking.com life.biz.ua
landscaping.center lignofix.ua
layola.biz.tc lineshops.biz
lego4x4.ru livefixer.com
letmacworkfaster.world locksmith.jp
letslowbefast.site longlifelomilomi.info
lflash.ru lottospring.com
li-er.ru louboutinreplica.xyz
lineavabit.it
liquimondo.com
littleberry.ru
liver-chintai.org
liverpool.gsofootball.com
livesex-amateure.info
loadopia.com
lombia.co
louboutinshoes.xyz louboutinshoes.xyz
luckyshop.net.ua
lyrster.com
mackeeper-land-672695126.us-east-1.elb.amazonaws.com mackeeper-land-672695126.us-east-1.elb.amazonaws.com
macrotek.ru madot.onlinewebshop.net
magicalfind-a.akamaihd.net maridan.com.ua
mail.allnews24.in marketingvici.com
map028.com master-muznachas.ru
marcowebonyodziezowe.pl masturbate.co.uk
marmotstore.online meccadumps.net
marwer.info medicovi.com
matpre.top megagrabber.ru
mcadamssupplyco.com mejoresfotos.eu
me-ke.com metarip.ru
mediaoffers.click meteostate.com
medizinreisen.de mini-modus.ru
meendo-free-traffic.ga mixtapetorrent.com
mesto-x.com mmostrike.ru
mir-limuzinov.ru mobot.site
missclub.info monarchfind-a.akamaihd.net
missvietnam.org morepoweronmac.trade
mobile.ok.ru morf.snn.gr
mobilemedia.md most-kerch.org
moesexy.com moyaterapiya.ru
monclerjacketsoutlet.pw
monetizer.com-01.site
monitorwebsitespeed.com
moscow-region.ru
mrbitsandbytes.com
mriyadh.com mriyadh.com
mug-na-chas-moscow.ru
musicdaddy.net musicdaddy.net
muz-tracker.net muztops.ru
mydirtyhobby.com my-big-family.com
myindospace.com mydoctorok.ru
myonlinepayday.co myhealthcare.com
myhitmp3.club
myliveblog.ru
myseoconsultant.com myseoconsultant.com
mystats.xyz mywallpaper.top
myxdate.info narkologiya-belgorod.ru
myyour.eu national-today-winning-winner.club
namecrumilchlet.tk negociosdasha.com
namenectar.com
narkologiya-voronezh.ru
neks.info neks.info
neodownload.webcam nerudlogistik.ru
newhairstylesformen2014.com nfvsz.com
nextbackgroundcheck.gq nightvision746.weebly.com
nextlnk12.com norththeface.store
nikhilbahl.com nrjmobile.fr
ningessaybe.me
noclegonline.info
nodding-passion.tk
nordstar.pro
novodigs.com
novosti-hi-tech.ru
nyfinance.ml nyfinance.ml
o333o.com obiavo.in
obiavo.net officedocuments.net
obiavo.su offside2.5v.pl
ochistka-stokov.ru offtime.ru
offergroup.info onemactrckr.com
ons-add.men onlinedomains.ru
onthemarch.co onlineserialy.ru
ooomeru.ru onlywoman.org
orgasmatrix.com onstrapon.purplesphere.in
openstat.com
opinionreelle.com
ops.picscout.com
osoznanie-narkotikam.net osoznanie-narkotikam.net
page2rss.com oynat.info
partner-stop.men partnerads.men
pathwhelp.org paydayonlinecom.com
pekori.to pc-virus-d0l92j2.pw
perfectplanned.com pcimforum.com
petsblogroll.com pendelprognos.se
photosaga.info perfection-pleasure.ru
picmoonco.pw petitions.whitehouse.gov
picture-group.com pharmacyincity.com
pipki.r.acdnpro.com pihl.se
pizdopletka.club pkr1hand.com
placid-rounded-coast.glitch.me plastgranar.nu
popmarker.com plastweb.ru
popup-fdm.xyz play.leadzupc.com
pornhub-ru.com podshipniki-ntn.ru
porno-play.net pornhub-forum.ga
porno-video-chati.ru pornogratisdiario.com
pornokorol.com portadd.men
pozdravleniya-c.ru portnoff.od.ua
pozdrawleniya.com poshiv-chehol.ru
pr-ten.de potoideas.us
praisong.net pozvonim.com
presleycollectibles.com predominant-invent.tk
primedice.com printingpeach.com
privatefx-in.ru
privatov-zapisi.ru
privhosting.com
pro-tec.kz pro-tec.kz
produkto.net prodess.ru
profolan.pl projefrio.com.br
prom23.ru prostitutki-novgoroda.org
promalp-universal.ru purplestats.com
promgirldresses.xyz pyramidlitho.webs.com
pronekut.com qpypcx.com
proposal-engine.com randalljhoward.com
prostitutki-kharkova.org rankingchart.de
psvita.ru rapevideosmovies.com
py100.ru rebrand.ly
queerspace.com redirectme.net
rainbowice.ru registrysweeper.com
registry-cleaner.net relatodelpresente.com.ar
remont-comp-pomosh.ru remontvsamare.su
resellerclub.com respublica-otel.ru
restorator-msk.ru restaurantlescampi.com
richinvestmonitor.com rewardit.com
rostov.xrus.org reyel1985.webnode.fr
ru-mediaget.ru rocis.site
rukino.org ruclicks.com
rusoft-zone.ru rvtv.ru
rusvideos.su saecsa.co
rutor.group santechnik.jimdo.com
sad-torg.com.ua savefrom.com
sbdl.no scanner-olga.top
scanner-nelson.top score-ads.men
scripted.com screpy.com
se.bnt-team.com se.bnt-team.com
searchimpression.com search.xtconnect.com
seeingmeerkat.com secretscook.ru
sell-fb-group-here.com security60-e.com
semalt.com seo-prof1.xyz
seoprofiler.com seoboxes.com
seozoom.it seofirmreviewsus.info
sergiorossistore.online sergiorossistore.online
serialsway.ucoz.ru serialsway.ucoz.ru
sex-dating.co serpstat.com
serw.clicksor.com
sex-pr.net sex-pr.net
sexs-foto.com sex-tracker.de
sexystrippe.info sexsearch.com
sexspornotub.com
sextracker.com
sexyali.com
shakhtar-doneck.ru shakhtar-doneck.ru
share-buttons-for-free.com shell-pmr.ru
sharebutton.to ship-marvel.co.ua
shijian.ac.cn shoppingjequiti.com.br
shiksabd.com signx.info
shitting.pro siteexpress.co.il
shmetall.com.ua
shohanb.com
shopvilleroyboch.com.ua
si-unique.com
sindragosa.comxa.com
sitebeam.net
siteonomy.com
skanninge.se skanninge.se
sladkoevideo.com skuteczna-dieta.co.pl
slavic-magic.ru skyway24.ru
smart-scripts.com slujbauborki.ru
smartadserver.com small-game.com
smsactivator.ru smichovbike.cz
social-vestnik.ru snjack.info
sockshare.net snow.nvr163.com
softomix.com snworks.com
sohoindia.net sodexo.com
spitfiremusic.com softomix.net
splendorsearch-a.akamaihd.net soviet-portal.do.am
spravka130.ru stal-rulon.ru
spywarenuker.com stanthonyscatholicchurch.org
srdrvp.com stopnarco.ru
srecorder.com stroydetali.ru
staff.prairiesouth.ca svensk-poesi.com
startwp.org t3chtonic.com
stathat.com tacbelarus.ru
staynplay.net tacbibirfa.tk
stroyplus.ru takeflyte.com
success-seo.com tako3.com
supermama.top taqywu51.soup.io
svarkagid.com td-l-market.ru
taaaak.com tdsing.ru
tabletkinaodchudzanie.com.pl telefonsex-ohne0900.net
takethatad.com telegraf.by
tandvardshuset.net templates.radiodigital.co
taqplayer.info texbaza.by
taranerymagesswa.blogspot.com the-trader.net
taximytishi.ru theautoprofit.ml
tech4master.com thebestweightlosspills.ovh
testbotprocessor44.com tilido.com
thetattoohut.com tizanidine4mgprice.blogspot.com
tiandeural.ru tizanidinecapsules.blogspot.com
tizanidine4mgtablets.blogspot.com tizanidinesleepaid.blogspot.com
tizanidine6mg.blogspot.com tizanidinestreetprice.blogspot.com
tizanidinehcl4mgtab.blogspot.com tnaionline.org
tizanidinehydrochloride2mg.blogspot.com
tizanidineinfo.blogspot.com
tizanidinemusclerelaxant.blogspot.com
tmm-kurs.ru
tooplay.com tooplay.com
topkarkas.com topanasex.com
topshef.ru trackmedia101.com
tourcroatia.co.uk tradedeals.biz
toxicwap.com trafficreceiver.club
toys.erolove.in trafficshaper.com
track.deriv.com truck-addzilla.life
traffic2money.com truebeauty.cc
traffique.net turvgori.ru
treasuretrack-a.akamaihd.net twinderbella.com
umaseh.com twittrading.com
urll.eu txxx.com
uzporno.mobi uchil.net
vacances-voyages.info ukrobstep.com
velen.io ukrtextbook.com
viagra.pp.ua ukrtvir.com.ua
umekana.ru
url-extractor.xyz
url2image.com
uroki.net
vegascosmetics.ru
vesnatehno.ru
video--production.com video--production.com
video-woman.com video-production.com
videotuber.ru videtubs.pl
vinylvault.co.uk vipms.ru
vip-parfumeria.ru
vipsiterip.org vipsiterip.org
virus-schutzmasken.de vkgaleria.com
vitalads.net vostoktrade.info
vkmusics.ru vseigry.fun
vkonche.com vtcdns.com
vladimir.xrus.org wallpapers-best.com
vsdshnik.com wallpaperstock.net
vykupavto-krasnodar.ru wanker.us
wait3sec.org wapsite.me
walkme.com waycash.net
warningzscaler.heraeus.com web-analytics.date
waysbetter.cn wedding0venues.tk
wdrake.com westermarkanjou.se
webix.me westum.se
webradiology.ru whatsupinfoley.com
website-stealer.nufaq.com wheelchairliftsarea.com
wesharepics.info whereverdesperate.gq
westen-v.life woman-h.ru
white-truck.life womens-journal.net
wholesalejerseychinashop.com world-mmo.com
wildcattube.com wufak.com
windowssearch-exp.com wwwadultcheck.com
wineration.com wzgyyq.com
winwotgold.pl x5market.ru
womensplay.net
wordpresscore.com
wstroika.ru
xaijo.com
xbaboon.com
xblognetwork.com
xcc24.pl xcc24.pl
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai xingzi-vision.com
xn--hxazdsfy.blogspot.com xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xrus.org xnxx699.com
xtrafficplus.com xtremeeagles.net
y8games-free.com xz618.com
yachts-cruise.info yellowfootprints.com
yebocasino.co.za youdao.com
yourmovies.pl youjizz.com
youtube-downloader.savetubevideo.com yourdesires.ru
ytmnd.com yourporngay.com
z-master.ru yuarra.pluto.ro
zarajbuilders.com zapiszto.pl
zebradudka.com zarenica.net
zeroredirect1.com zeroredirect1.com
zigarettenonlinekaufen1.bloog.pl zhorapankratov7.blogspot.com
zoolubimets.ru zoompegs.com
zskdla.site zophim.me
zvooq.eu

480
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
03e.info 7yue.org
100dollars-seo.com
1bet.com
28n2gl3wfyb0.ru
883zy.com 883zy.com
91abcw.com aa8780bb28a1de4eb5bff33c28a218a930.com
aa14ab57a3339c4064bd9ae6fad7495b5f.com aa9bd78f328a6a41279d0fad0a88df1901.com
adnotbad.com abiente.ru
ads-seo.men ad-words.ru
adserver-e7.com adamoads.com
adsfresh.men adrunnr.com
adtology.com adsref.men
afftrack001.com afmuseum.com
akuhni.by all4wap.ru
allfinweb.com ameblo.top
alpharma.net android4fun.org
amateurmatch.com apilinkswiftco-a.akamaihd.net
anime.dougasouko.com apple.com-cleaner.systems
arate.ru artel116.ru
arenda-avtoprokat-krasnodar.ru arvut.org
arendatora.ru ascat.porn
aviav.org asscenihotosidea.blogspot.co.za
aviav.ru.com auto-moto-elektronika.cz
avirasecureserver.com autobudpostach.club
azbaseimages.net avkzarabotok.info
baersaratov.ru azlex.uz
bamo.xsl.pt backuperwebcam.weebly.com
bazaronline24.ru bagcionderlab.com
best-way.men baltgem.com
bestofupload.info bank.uz
big-boards.info bdsmgalls.net
bjetjt.com beauty-things.com
blackwitchcraft.ru beetpics.pw
bouda.kvalitne.cz best-seo-software.xyz
brandehk.dk best-seo-solution.com
brillianty.info bidbuy.co.kr
buy-cheap-online.info bioscorp.ru
cacheimages.com bizfly.info
calcularpagerank.com blpmovies.com
cannibalcheerleader.com bobinoz.com
cartechnic.ru brothers-smaller.ru
cartierlovebracelet.xyz btvn.ru
cartierring.xyz bugshoot.cn
cdncash.net calstaterealty.us
cheapjerseysa.com candcstuccoandstone.com
cialis-samples.com canrole.com
clodo.ru cardul.ru
cascadelink.org
cercacamion.it
chuckguilford.com
citizenclsdriveri7.pen.io
coaching-netz.info
cocolyze.com cocolyze.com
com-secure.download coindirect.io
comeondog.info com-onlinesupport.site
commoncrawl.org cometorussia.net
compliance-alex.xyz compliance-alex.top
construmac.com.mx compliance-elena.xyz
copro.pw compliance-olga.top
countbertwistdisp26.soup.io cosmeticswomens-womensports.rhcloud.com
creams.makeforum.eu cypernhuset.se
creativehutindia.com data-mining.tk
cutt.us defenderxtactical.com
cwetochki.ru degerlund.net
dandiyabeats.in dengi-pod-zalog-nedvizhimosti.ru
dickssportinggoods.com dimfour.com
djstools.com diusyawiga.tk
donvito.unas.cz djonwatch.ru
drugstoreforyou.com doeco.ru
easycommerce.cf domaincdn.xyz
easyukraine.com doyouknowtheword-flummox.ml
education-cz.ru drev.biz
elektrischezigarette2.onsugar.com ednorxmed.com
emergencyneeds.org ekaterinburg.xrus.org
englishgamer.com elektronischezigarettekaufen1.myblog.de
erotik-kostenlos.net eropornosex.ru
europeanwatches.ru euromasterclass.ru
everflora.ru exmasters.com
expdom.com extremez.net
facecup.top facialporntube.com
falcon-images.blogspot.com falcon-images.blogspot.com
falcoware.com fickkontaktehobbyhuren.com
fast-wordpress-start.com filesvine.com
fastfixing.tech flashahead.info
filesclub.net freelinkbuilding.website.tk
filunika.com.ru funponsel.com
firstdrugmall.ru gaytube.com
flashlarevista.com ge0ip.com
florida-tourism.net gemara.com
g7m.pl getyourimage.club
gallerily.com goblacked.com
go2jump.org gungamesz.com
gq-catalog.gq haveinc.xyz
gratuitbaise.com headpress.ru
greenzaim.ru hebr.myddns-flir.com
gsmlab.pl hem.passagen.se
guigyverpo.cf hop.clickbank.net
heatpower.ru hotdl.in
hikesearch.net hotshoppymac.com
hkladys.com houdom.net
hollywoodactress.info i-hobot.ru
horoshieokna.com igithab.com
host-tracker.com ihtec2019.org
ido3.com iideaidekonkatu.info
iflycapetown.co.za imperiafilm.ru
igrovyeavtomaty777.ru intelhdgraphicsgtdrive6w.metroblog.com
infospot.pt io9.com
instabid.tech iomoio.net
integritylandscapeservices.com
irkutsk.zrus.org irkutsk.zrus.org
it-max.com.ua
itronics.ca
iyasimasennka.com iyasimasennka.com
jabimgo.pw japan-bearings.ru
jamiembrown.com junglenet-a.akamaihd.net
jasonpartington.com kalandranis.gr
javatex.co.id kino2018.club
jo24news.com klitimg.pw
kellyonline.xyz kollesa.ru
kerei.ru komp-pomosch.ru
keywordbasket.com koronirealestate.gr
kiwi237au.tk kredit-pod-zalog-krasnodar.ru
kliksaya.com kupit-adenu.ru
knigonosha.net lacasamorett.com
koleso24.com.ua letsrepair.in
ks1234.com liffa.ru
kskjco.club liky.co.ua
labvis.host.sk linksharingt.com
ladsblue.com listiseltemournlan.gq
landinez.co littlesunraiser.com
landreferat.ru liveinternet.ro
lecbter-relationships.ga long-beach-air-conditioning.com
lion.misba.us louboutinreplica.pw
livingroomdecor.info lunchrock.co
lollypopgaming.com mabdoola.blogspot.com
lrac.info mailemedicinals.com
lsp-awak-perikanan.com malls.com
lyrics.home-task.com matpre.top
marketingvici.com medizinreisen.de
matb3aa.com megagrabber.ru
mbiologi.ru megakino.net
meccadumps.net michaelkorsoutletstore.net
medtherapy.ru misswell.net
mirmedinfo.ru mmstat.com
mixtapetorrent.com mnogabukaff.net
mjchamonix.org monarchfind-a.akamaihd.net
mobioffertrck.com mossmesi.com
moneyzzz.ru mrbitsandbytes.com
musicktab.com musicdaddy.net
nastroyke.net muznachas-service.ru
needtosellmyhousefast.com my-big-family.com
nordstar.pro narkologiya-orel.ru
nucia.biz.ly narkologiya-penza.ru
office-windows.ru njpalletremoval.com
offtime.ru november-lax.com
oivcvx.website o-o-6-o-o.com
onlinewritingjobs17.blogspot.ru office2web.com
optom-deshevo.ru olgacvetmet.com
oxotl.com onemantrip.com
panchro.xyz onlineserialy.ru
partner-trustworthy.men ororodnik.goodbb.ru
pila.pl partner-print.men
pornohub.me patol01.pw
pornokajf.com piratecams.com
pornonik.com pizdeishn.com
pornosee.info plusnetwork.com
pornowarp.info pornokorol.com
pourvous.info privacyassistant.net
privat-girl.net privhosting.com
pro-poly.ru prod2016.com
prokotov.com pskcijdc.bloger.index.hr
promoheads.com purplestats.com
promover.org pussysaga.com
psvita.ru quality-traffic.com
pyramidlitho.webs.com razleton.com
r-e-f-e-r-e-r.com reklama1.ru
rapidsites.pro rezeptiblud.ru
razvratnoe.org rialp.getenjoyment.net
realmonte.net ribieiendom.no
redhotfreebies.co.uk rubanners.com
rff-cfal.info rucrypt.com
rome2rio.com
romhacking.ru
royalads.net
rrutw.com
rumamba.com
rusenvironmental.net rusenvironmental.net
sarahmilne.top salmonfishingsacramentoriver.com
scanner-alex.top sampleletters.net
scanner-brian.top scenarii-1-sentyabrya.uroki.org.ua
scanner-don.top secretscook.ru
scanner-fred.top seeresultshub-a.akamaihd.net
schalke04fc.info serviporno.com
screpy.com shijian.ac.cn
seasaltwithfood.com ship-marvel.co.ua
sexs-foto.top silvermature.net
sharebutton.to sispe.com.br
skypasss.com sitechecker.pro
sloopyjoes.com siteheart.net
smadihome.com smadihome.com
smartbalanceworld.com softonicads.com
softlinesolutions.me space4updating.win
spidtest.space spravka130.ru
steelmaster.lv starwars.wikia.com
streetfire.net steebook.com
stuff-about-money.com stillmiracle.com
sucsesofinspiration.com subj.ukr-lit.com
svetodiodoff.ru sugarkun.com
szucs.ru susanholtphotography.com
telegramdownload10.com svbur.ru
textads.men swiped.su
the-trader.net swsociety.se
thecounter.com taqywu51.soup.io
thenews-today.info tattooreligion.ru
thepokertimer.com terrafootwear.us
testbotprocessor44.com
timer4web.com
timetorelax.biz
tivolibasket.it tivolibasket.it
tizanidineformigraines.blogspot.com topflownews.com
tizanidinehcl4mgisitanarcotic.blogspot.com toproadrunner5.info
todohr.com topshef.ru
torrents.life trafficshaper.com
tozup.com trainoffend.ml
trion.od.ua tramadolandtizanidine.blogspot.com
trkdf.com
tsyndicate.com tsyndicate.com
turbabitload.weebly.com ttrraacckkrr.com
tvorozhnaja-zapekanka-recept.ru ttsq.fr
tvory.predmety.in.ua tytoona.com
twelvevisionspartyofcolorado.com ultimateclassicrock.com
typimga.pw unrealcommander.com
ukrup.com
us-america.ru
utrolive.ru
uzporno.mobi
vacuumcleanerguru.com vacuumcleanerguru.com
vasileostrovsky-rayon.ru vancleefreplica.pw
vektorpress.ru vashsvet.com
venta-prom.ru vezuviy.su
via-energy-acquistare.com via-energy-order.com
videochaty.ru viktoria-center.ru
videospornogratisx.net visitcambridge.org
vkontaktemusic.ru vykup-avto-krasnodar.ru
vonradio.com wallpapers-best.com
vtc.pw watchinf.com
vzubah.com webcamdevochka.com
wait3sec.org webradiology.ru
watch-movies.ru weprik.ru
wnhjavlhezp.gq woman-tampon.ru
worldmusicfests.com wurr.voila.net
x-rates.ru xerox-douglas.cf
xxxhdvideo.site xingzi-vision.com
xxxmania.top xn----itbeirbjbi7bc6bh2d.xn--p1ai
yachts-cruise.info xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
yur-p.ru xn--80atua3d.xn--p1ai
zawyna.ua xn--d1abj0abs9d.in.ua
zero1.it youdao.com
zryydi.com yousense.info
zs2vm.top yun56.co
zscalertwo.net zajm-pod-zalog-nedvizhimosti.ru
zeroredirect10.com
zoodrawings.com

14
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -1,11 +1,21 @@
<html> <!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
ml><html> <html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2916 ### Version: V4.2022.02.2917
### Updated: Sun Feb 13 22:01:28 UTC 2022 ### Updated: Mon Feb 14 10:03:13 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2916 ### Version: V4.2022.02.2917
### Updated: Sun Feb 13 22:01:28 UTC 2022 ### Updated: Mon Feb 14 10:03:13 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##