vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.05.3134 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-05-29 10:09:10 +00:00
parent c4d004a1dd
commit f90079fbda
47 changed files with 99013 additions and 99003 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3133 #### Version: V4.2022.05.3134
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3133 #### Version: V4.2022.05.3134
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.05.3133 #### Version: V4.2022.05.3134
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12240
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.05.3133 #### Version: V4.2022.05.3134
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12244
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12244
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12244
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12244
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12244
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12244
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12244
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3133 3134

12244
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12244
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.225.143.173 0; 20.230.46.22 0;

12244
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12244
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12244
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12244
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.225.143.173 0; 20.230.46.22 0;

12244
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.225.143.173 0; 20.230.46.22 0;

12244
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
360Spider CODE87
Brandprotect CrazyWebCrawler
Brandwatch G-i-g-a-b-o-t
EroCrawler Gigabot
Lanshanbot HybridBot
Litemage_walker IlseBot
NICErsPRO Ninja
OpenVAS Picscout
seostar ips-agent
sogouspider scalaj-http

292
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker 404checker
ASPSeek AIBOT
Aboundex Aboundex
Aboundexbot Aboundexbot
AiHitBot
Aipbot Aipbot
Alexibot Alexibot
Alligator Anarchy99
AlphaBot Ankit
Anarchie
Anarchy
Apexoo
Asterias
Atomseobot Atomseobot
Attach AwarioRssBot
BBBike
BDCbot
BLEXBot BLEXBot
BackDoorBot BackWeb
BackStreet BacklinkCrawler
Badass Badass
BatchFTP BatchFTP
BetaBot
Bigfoot
Bitacle
Black Hole
Blow
BlowFish
Boardreader
BotALot BotALot
Buck Buck
Buddy Buddy
BuiltWith BuiltWith
Bullseye
BuzzSumo BuzzSumo
CazoodleBot CATExplorador
CensysInspect Cegbfeieh
Claritybot ChinaClaw
Cliqzbot Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot Cocolyzebot
Collector
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Crescent
Curious Curious
DTS Agent CyotekWebCopy
Digincore DSearch
Disco Demon
Discobot Devil
Discoverybot DigitalPebble
Dirbuster
Dispatch
DnBCrawler-Analytics DnBCrawler-Analytics
DomCopBot DomCopBot
DomainAppender
DomainCrawler DomainCrawler
Domains Project
Download Wonder
Drip
ECCP/1.0 ECCP/1.0
EMail Siphon EMail Siphon
Ebingbong
Ecxi Ecxi
EirGrabber EirGrabber
Express WebPictures Exabot
ExtractorPro
EyeNetIE EyeNetIE
Ezooms Ezooms
FDM FHscan
FemtosearchBot
Fimap
Firefox/7.0
FlashGet FlashGet
Foobot Freeuploader
FyberSpider
Fyrebot
GT::WWW GT::WWW
GermCrawler GalaxyBot
GetRight
Gigabot Gigabot
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla Grabber
GrabNet Grafula
GrapeFX
GrapeshotCrawler GrapeshotCrawler
GridBot
HEADMasterSEO HEADMasterSEO
HMView HTMLparser
HTTrack HTTrack
Haansoft
Harvest Harvest
Havij Havij
Heritrix
Hloader Hloader
HonoluluBot HonoluluBot
Humanlinks
IDBTE4M IDBTE4M
IDBot
IRLbot IRLbot
Iblog
Id-search Id-search
IlseBot
Image Fetch Image Fetch
IndeedBot Indy Library
InfoNaviRobot
InfoTekies InfoTekies
InterGET Intelliseek
InternetSeer InternetSeer
Iskanie Iria
Jbrofuzz IstellaBot
JennyBot Joomla
Jetty
Jorgee Jorgee
JustView JustView
Keybot Translation-Search-Machine
Keyword Density
Kozmosbot
LNSpiderguy LNSpiderguy
Lanshanbot Larbin
Leap Leap
LexiBot LeechFTP
LinkScan Lightspeedsystems
Linkbot Linkbot
LinkextractorPro LinksManager
LinkpadBot LinqiaMetadataDownloaderBot
Lipperhey LinqiaRSSBot
Ltx71 LinqiaScrapeBot
MFC_Tear_Sample Lmspider
MJ12bot MJ12bot
MQQBrowser
MSFrontPage
MSIECrawler MSIECrawler
Mail.RU_Bot MTRobot
MarkMonitor Magnet
Mass Downloader Majestic-SEO
Majestic12
Masscan Masscan
Mata Hari
MauiBot
Meanpathbot
Metauri
Microsoft URL Control
Minefield
Mister PiX Mister PiX
Mojeek Moblie Safari
Mojolicious Mozlila
Mr.4x3
Musobot Musobot
NPbot NICErsPRO
Name Intelligence Nameprotect
NearSite Navroad
Needle Needle
Netcraft Nessus
Net Vampire
Nettrack
NextGenSearchBot
Nibbler
Nikto
Ninja Ninja
Not Nmap
OpenLinkProfiler OpenVAS
Openfind Openfind
Openvas OrangeSpider
OrangeBot
OutclicksBot
OutfoxBot
PECL::HTTP PECL::HTTP
Page Analyzer PHPCrawl
PageAnalyzer Pandalytics
PageScorer Papa Foto
Pavuk PeoplePal
Piepmatz Pimonster
ProPowerBot Pixray
PleaseCrawl
Probethenet Probethenet
Psbot Psbot
Pu_iN
Pump
PyCurl
RankActive RankActive
RankActiveLinkBot
RankingBot2 RankingBot2
Rankivabot
RankurBot
Re-re
Reaper Reaper
RebelMouse
Recorder
RedesScrapy RedesScrapy
SBIder SEOprofiler
SEOkicks
SEOlyticsCrawler
SISTRIX SISTRIX
SMTBot Scrapy
ScanAlert SemanticJuice
ScoutJet
Screaming
ScrepyBot
Seekport
Seomoz
Shodan
Siphon Siphon
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteLockSpider
SiteSnagger
Sitebeam Sitebeam
Siteimprove
Sitevigil Sitevigil
Sottopop Snoopy
SpaceBison Spammen
SpankBot SpankBot
Sqlmap Spanner
Sqlworm Spinn3r
Sqworm
Sucuri Sucuri
SuperBot Suzuran
SurveyBot Swiftbot
Szukacz
T8Abot
TeleportPro
Telesoft Telesoft
Telesphorep Telesphorep
The Intraformant The Intraformant
Thumbor TightTwatBot
Titan Toata
Toweyabot Tracemyfile
TwengaBot Trendiction
True_Robot
Twice Twice
UnisterBot Typhoeus
VB Project
VCI
Vagabondo Vagabondo
Virusdie VeriCiteCrawler
Voil WASALive-Bot
Voltron
WISENutbot WISENutbot
WWW-Collector-E Wallpapers/3.0
WWW-Mechanize WeSEE
Wallpapers Web Fetch
Web Enhancer
Web Pix Web Pix
WebBandit Web Sauger
WebEnhancer Web Sucker
WebFuck
WebGo IS WebGo IS
WebImageCollector WebImageCollector
WebPix WebReaper
WebSauger WebSauger
WebStripper
WebSucker
WebZIP WebZIP
Webalta WebmasterWorldForumBot
Webshag Website Quester
WebsiteExtractor WebsiteExtractor
Webster Webster
Whack Whacker
Whatweb Whatweb
WinHTTrack
WiseGuys Robot WiseGuys Robot
Wonderbot
Woobot Woobot
Wotbox
Wprecon Wprecon
Zade
Zauba Zauba
Zeus Zeus
Zitebot ZyBorg
ZmEu
ZoomBot
adscanner adscanner
arquivo.pt arquivo.pt
backlink-check backlink-check
coccocbot cah.io.community
demandbase-bot check1.exe
evc-batch com.plumanalytics
crawler.feedback
eCatch
facebookscraper facebookscraper
gopher
internetVista monitor internetVista monitor
ips-agent
iubenda-radar iubenda-radar
lwp-request lwp-request
lwp-trivial lwp-trivial
page scorer netEstate NE Crawler
s1z.ru plumanalytics
ripz
satoristudio.net satoristudio.net
scalaj-http seocompany.store
seobility
seoscanners seoscanners
seostar seostar
sogouspider serpstatbot
tAkeOut siteripz
sp_auditbot
spyfu
sysscan
trendiction.de trendiction.de
x09Mozilla ubermetrics-technologies.com
zauba.io voyagerx.com
zgrab x22Mozilla

308
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404checker
404enemy 404enemy
80legs 80legs
ADmantX AIBOT
ALittle Client
Abonti
Aboundex
Aboundexbot
Acunetix
AhrefsBot
AiHitBot AiHitBot
Aipbot Aipbot
Anarchie
Anarchy Anarchy
Ankit Anarchy99
Anthill
Apexoo Apexoo
AwarioSmartBot Attach
AwarioRssBot
BBBike
BDCbot
BDFetch
BLEXBot BLEXBot
BatchFTP BackDoorBot
Bitacle Badass
Black Hole Battleztar Bazinga
Bigfoot
Blackboard
BlowFish BlowFish
Boardreader
Bolt Bolt
BotALot BotALot
Brandwatch
Buck
Buddy Buddy
CATExplorador BuiltWith
BunnySlippers
BuzzSumo
CSHttp CSHttp
CheeseBot Calculon
Chlooe CazoodleBot
Citoid CheTeam
Cliqzbot Cocolyzebot
Cloud mapping Collector
Copier CopyRightCheck
Copyscape
Craftbot Craftbot
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Custo Crescent
CyotekWebCopy
DBLBot
DIIbot DIIbot
DSearch DataCha0s
Demon
Deusu Deusu
Digincore Devil
Dirbuster DigitalPebble
Discoverybot Discobot
Dispatch Dispatch
DnBCrawler-Analytics
DomCopBot DomCopBot
DomainAppender
DomainCrawler
DomainStatsBot
Dragonfly
Drip Drip
EasyDL EroCrawler
Exabot
ExtLinksBot
Extractor
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE FDM
Ezooms Fimap
Flunky FlashGet
Genieo FyberSpider
GT::WWW
GalaxyBot
GermCrawler GermCrawler
GetRight GetRight
GetWeb
Gigabot
GoZilla GoZilla
HEADMasterSEO Gotit
HMView Grabber
HTMLparser HTMLparser
HTTP::Lite HTTP::Lite
Haansoft Haansoft
HaosouSpider Harvest
Havij
Heritrix
Hloader
Humanlinks
HybridBot
IDBTE4M
IDBot
IRLbot
Id-search Id-search
InterGET InfoNaviRobot
InfoTekies
Intelliseek
Internet Ninja Internet Ninja
InternetSeer JOC Web Spider
Iria
JetCar JetCar
Jetty
Joomla
Jorgee Jorgee
JustView
Jyxobot
Kenjin Spider
Keybot Translation-Search-Machine
Keyword Density Keyword Density
Kozmosbot
LNSpiderguy
Lanshanbot Lanshanbot
Larbin
Leap Leap
LeechFTP LeechGet
LexiBot LexiBot
Libwhisker
Lightspeedsystems Lightspeedsystems
Likse
LinkWalker
Linkbot
LinkpadBot LinkpadBot
LinqiaScrapeBot LinksManager
Lipperhey LinqiaMetadataDownloaderBot
Lipperhey Spider LinqiaRSSBot
Litemage_walker
Lmspider Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool
MQQBrowser
MSFrontPage MSFrontPage
MSIECrawler Magnet
Majestic SEO Majestic-SEO
Majestic12
MarkWatch MarkWatch
Mass Downloader Masscan
Mb2345Browser
MeanPath Bot
MegaIndex.ru MegaIndex.ru
Microsoft URL Control Microsoft URL Control
Minefield Minefield
Mister PiX Mister PiX
Mojeek
MolokaiBot MolokaiBot
Msrabot Mr.4x3
NPbot
Name Intelligence Name Intelligence
Navroad Nameprotect
Needle NetSpider
NetMechanic
NetZIP
Nettrack Nettrack
Nikto Netvibes
NimbleCrawler Nibbler
Not Nimbostratus
Nuclei
Nutch
OnCrawl OnCrawl
OpenLinkProfiler Openvas
PHPCrawl PECL::HTTP
Page Analyzer
PageAnalyzer PageAnalyzer
PageThing.com PageScorer
Papa Foto
Pavuk
PeoplePal
Pi-Monster Pi-Monster
PictureFinder Picscout
Pixray Piepmatz
PleaseCrawl ProPowerBot
ProWebWalker ProWebWalker
Pu_iN Pu_iN
PxBroker
PyCurl PyCurl
Quick-Crawler QueryN Metasearch
RankActiveLinkBot RankingBot
RankingBot2
RankurBot RankurBot
Reaper Re-re
RedesScrapy ReGet
SBIder Recorder
Ripper
SEOkicks SEOkicks
SISTRIX SISTRIX
SMTBot SMTBot
Screaming ScanAlert
ScoutJet
Searchestate Searchestate
SearchmetricsBot
SemanticJuice SemanticJuice
Semrush
SemrushBot
SentiBot
SeoSiteCheckup
SeobilityBot
Seomoz
Shodan Shodan
Siphon
Site Sucker Site Sucker
SiteExplorer SiteExplorer
SiteLockSpider
SiteSnagger SiteSnagger
Sitebeam
Siteimprove Siteimprove
Sitevigil Sitevigil
Snoopy SlySearch
Sociscraper SmartDownload
Sogou web spider Snake
Sottopop SocialRankIOBot
Sqlworm Spammen
SpankBot
Spanner
Spbot
Spinn3r
SputnikBot
Stripper Stripper
Sucuri
SuperHTTP
Surfbot Surfbot
SurveyBot SurveyBot
Suzuran Szukacz
TeleportPro T8Abot
TightTwatBot Teleport
Titan Telesphorep
Toweyabot Toweyabot
Trendiction Trendiction
Trendictionbot
True_Robot True_Robot
Upflow Turnitin
VB Project Typhoeus
URLy.Warning
UnisterBot
V-BOT
VCI VCI
Vacuum
VidibleScraper
Voil Voil
WBSearchBot Voltron
WISENutbot
WPScan WPScan
WWW-Mechanize
WWW::Mechanize
WWWOFFLE WWWOFFLE
Web Collage WeSEE
Web Enhancer Web Auto
Web Fetch Web Sauger
Web Pix
Web Sucker Web Sucker
WebAuto WebCopier
WebBandit WebEnhancer
WebGo IS WebLeacher
WebImageCollector WebPix
WebSauger WebSauger
Webalta WebStripper
Webshag WebWhacker
Webster Webster
Whacker Whack
Whatweb
WinHTTrack WinHTTrack
WiseGuys Robot WiseGuys Robot
Wonderbot Wonderbot
Woobot Wotbox
Wprecon
Xaldon_WebSpider Xaldon_WebSpider
Xenu Xenu
Zermelo Zauba
Zeus
ZoominfoBot ZoominfoBot
ZumBot adscanner
ZyBorg arquivo.pt
arquivo-web-crawler check1.exe
autoemailspider clark-crawler
backlink-check cognitiveseo
coccocbot crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j
demandbase-bot
domainsproject.org
eCatch
evc-batch
facebookscraper facebookscraper
heritrix heritrix
ips-agent instabid
lwp-request isitwp.com
lwp-trivial
netEstate NE Crawler
oBot oBot
probe-image-size probe-image-size
ripz
s1z.ru
satoristudio.net satoristudio.net
scalaj-http
seobility
seocompany.store
seoscanners seoscanners
sexsearcher seostar
sitechecker.pro sogouspider
spyfu spyfu
sysscan sysscan
tAkeOut
trendiction.com
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webgains-bot webgains-bot
webmeup-crawler
webpros.com
webprosbot webprosbot
x09Mozilla
zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
best-businessman.ru 1001desert.com
chastnoeporno.com academiacsmendoza.org
ero-advertising.com ihc2015.info
hubbble.com monclerboots.xyz
makenahartlin.com moroccosurfadventures.com
meteocast.net na-telefon.biz
obsessionphrases.com nsatc.net
sispe.com.br openlibrary.org
soundfrost.org oxotl.com
theheroes.ru xxx-cam.webcam

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0912701309f8ce.com 0n-line.tv
0c47f8422d3f.com 1008.su
11235813.webzdarma.cz 10bet.com
12u.info 12bet.com
18ps.ru 1314dh.com
1adult.com 1bet.com
1millionusd.xyz 273-fz.ru
1webmaster.ml 34.gs
24-ak.ru 3rbseyes.com
24x7-server-support.site 4webmasters.org
2girls1cup-free.com 51unlim.ru
2itech.ru 7wind.ru
5elementov.ru 8kisses.eu
5tey7463.weebly.com 8si.ru
777-club.ru aa8b68101d388c446389283820863176e7.com
91abcw.com abcdefh.xyz
a96527gi.beget.tech abclauncher.com
aanapa.ru adelly.bg
abcdeg.xyz adidas.frwebs.fr
adcash.com adman.se
adclickservice.com
adobereader-free.ru
adsland.men
adsloads.com adsloads.com
adsref.men adult-shop.com.ua
adultmeetonline.info agahinameh.com
advancedsoftwaresupport.com aghanyna.com
aimayangzhi.com ahrntal.verymes.xyz
all-streaming-media.com amazingpic.net
allblogroll.com anatomiy.com
amazon-adsystem.com
amoi.tn
anabolics.shop
anime.dougasouko.com anime.dougasouko.com
animenime.ru apptester.tk
apibatbrowsecom-a.akamaihd.net arclk.net
appleid-verification.com as5000.com
appmsr.org asociatia-tipografilor-transilvania.ro
approvedlocal.co.za asscenihotosidea.blogspot.com
arenda-yeisk.ru auto-moto-elektronika.cz
arewater.com autobrennero.it
asia-forum.ru autoplate.info
asrv-a.akamaihd.net autotop.com.ua
asseenontvonline.ru aviav.eu
atovh.local-finders.com aviav.ru
autoblogger24.info awency.com
autonew.biz ayeartoforget.com
autotrends.today b-styles.xyz
avkzarabotok.com babosas.com
azarttoto.com
azbuka-mo.ru
back.dog
backgroundpictures.net backgroundpictures.net
balla-boo.se
bamo.xsl.pt bamo.xsl.pt
battle.net barbourjackets.online
bayareaaeroclub.org bayareaaeroclub.org
besthomemadepornsites.com bestcurs.org
bestkfiledxd.cf bestdooz.com
bestplacetobuyeriacta.jw.lt bestfortraders.com
bibys.com bestmobilityscooterstoday.com
biocn.dx.am bestpornuha.com
bitcoin-ua.top bestsexyblog.com
biznesluxe.ru beta.hotkeys.com
biznesrealnost.ru betune.onlinewebshop.net
bildsuche.ru
bimatoprost-careprost.com
bio-japan.net
bio-market.kz
bioscorp.ru
bizfly.info
bizlist.com.de
bizzliving.com
bkgr.se
bkns.vn bkns.vn
blackhatworth.com blacktwhite.com
blackle.com blog.koreadaily.com
blackplanet.com bloggerads.net
blavia.00author.com blogig.org
blog.remote-computer.de blogs.rediff.com
bochemit.com.ua bonkers.name
bocoarchives.org boo-arts.com
boeuklad.com bracketsmackdown.com
bookmark4you.com brendbutik.ru
bradleylive.xyz
brainsandeggs.blogspot.com
bristolhostel.com
bryansk.zrus.org
budilneg.xyz budilneg.xyz
buynorxx.com bugshoot.cn
buyscabiescream.com business-made-fun.com
calc-for-credit.ru buttons-for-your-website.com
cand.jp buyantiviralwp.com
cardiosport.com.ua buyk.host.sk
castingbank.ru buzzurl.jp
cejewelry.xyz callawaygolfoutlet.online
celebsopics.com candiceloves.us
cenoval.ru candytiner.com
cheapjerseysap.com capsderfudd.tk
christianlouboutinoutlet.win carabela.com.do
cindymatches.com carson.getenjoyment.net
clickbank.net cartierjusteunclou.xyz
clickhype.com casablancamanor.co.za
clknsee.com cezartabac.ro
co.lumb.co cfcl.co.uk
compliance-alexa.top chatroulette.world
copro.pw cheapbelstaff.online
cottageofgrace.com cheapmoncler.pw
counterbot.com checktext.org
craftinsta.ru chee-by.biz
customsua.com.ua chocolateslim-original.com
cxpromote.com christianlouboutinsaleoutletonline.us
czeshop.info chuckguilford.com
dailyrank.net cialischstgerts.com
cicaki.net
circlesl.com
cognitiveseo.com
compliance-checker-7.info
compliance-george.top
compliance-ivan.top
compliance-nelson.top
cookie-law-enforcement-hh.xyz
covetnica.com
creditnation.ru
culad.com
culmimg.pw
cv.wallhade.co
cymbaltaandtizanidine.blogspot.com
d0t.ru
dafatiri.com
daisye.top daisye.top
damedingel.ya.ru delayreferat.ru
darkages.info
dbmkfhqk.bloger.index.hr
ddospanel.com
defenderxtactical.com
deluxedumps.com deluxedumps.com
destinationrealestate.com
dev.citetick.com
dfiles.me
dieta-personalna.pl
dinkolove.ya.ru dinkolove.ya.ru
disruptingdinnerparties.com
div.as div.as
dmmspy.com divisioncore.com
dolohen.com divku.ru
domini.cat djekxa.ru
domznaniy.ru dkvorota.ru
driving.kiev.ua dlya-android.org
droidlook.net domain2008.com
e-avon.ru downloadeer.net
e-commerce-seo.com dtm-spain.com
easyukraine.com e2click.com
editors.choice6912650.hulfingtonpost.com e90post.com
efkt.jp easytuningshop.ru
ege-russian.ru ecommerce-seo.org
egvar.net ed-shop01.ru
ekaterinburg.xrus.org electricwheelchairsarea.com
elektrischeziga.livejournal.com elektrischeziga.livejournal.com
elektrozigarette2.wordpressy.pl elektrozigarette2.webs.com
elektrozigarettekaufen2.blogse.nl eloconcream.blogspot.com
elvel.com.ua emerson-rus.ru
epngo.bz enews.tech
erectile.bid etenininrade.ga
escortslet.net eupornstar.info
essayservicewriting.org everypony.ru
eu-cookie-law.info extener.org
eugenevaultstorage.com extlinks.com
exmasters.com extstat.com
extlabs.io eyelike.com.ua
ezaz.info eyes-on-you.ga
ezigarettekaufen1.hpage.com ezb.elvenmachine.com
ezigaretteshop2.vefblog.net ezofest.sk
face.hostingx.eu fable.in.ua
falllow.gq family1st.ca
familyholiday.ml fantasticpixcool.com
faracontrol.ir fastfixing.tech
fatmaelgarny.com feorina.ru
fearcrow.com
fifa-coins.online fifa-coins.online
filefilter.weebly.com filmi-onlain.info
filesclub.net
filesvine.com
filmidivx.com filmidivx.com
findthe.pet forexunion.net
finejewelryshop.ru
finemanteam.com
fiverr.com
flytourisme.org
footballfarrago.com
forms-mtm.ru forms-mtm.ru
fortwosmartcar.pw free-floating-buttons.com
forums.d2jsp.org free-video-tool.com
fotopop.club freesitetest.com
foxinsocks.ru freetangodownload.com
free-social-buttons.com fullgirl.ru
frequiry.com fym.com.ru
front.ru g33.org
ftns.ru gael-s.ru
fungirlsgames.net galaxy-family.ru
fxund.us gamesprite.me
g.starmoe.xyz gaming-journal.com
gamedayassist.com gardene.ru
gamezblox.com gcup.ru
gelezki.com gembird.com
genericviagrasildenafiled.net getpopunder.com
ggiaro.com
girlgamerdaily.com
glall.ru glall.ru
gobongo.info glastecfilms.com.my
gogps.me go2mike.ru
goldadpremium.com goblacked.com
goodhumor24.com gq-catalog.gq
goodwriterssales.com graphid.com
gosarhivrt.ru gratis-sexkontakte.com
graphics8.info gsmtlf.ru
greenzaim.ru habermetre.com
gsasearchengineranker.top hd720kino.ru
guildebzh.info hentai-manga.porn
gwebtools.com.br herokuapp.com
hamptonoaks.ca hobild.net
hasfun.com homeandhealth.ru
hatdc.org homeart.space
hdxnxxtube.mobi hosting-tracker.com
hitsbox.info hut1.ru
hkladys.com i-hobot.ru
honyaku.yahoofs.jp icaseclub.ru
hotcore.info ideashome.id
hotelcrocenzi.sm ilikevitaly.com
hothot.ru ilmexico.com
iboss.com
igtools.club
ihtec2019.org
imagui.eu imagui.eu
img.wallpaperstock.net impisr.ru
imgarit.pw innodgfdriverhm.aircus.com
incest-ru.com insta-add.pro
incolors.club interesnie-faktu.ru
increasewwwtraffic.info interior-stickers.ru
indianmedicaltourismshop.com intervsem.ru
int.search.mywebsearch.com ipostroika.ru
intellekt21.ru iron-age.info
interferencer.ru
internetproviderstucson.com
isitpaleo.info
istizanidinelikexanax.blogspot.com
istripper.com
it-max.com.ua
ivoiretechnocom.ci ivoiretechnocom.ci
iwantedmoney.com iwantmyfreecash.com
janavibekken.no javatex.co.id
jav-way.site javspace.net
javrip.net jeremyeaton.co
javstock.com jerseychinabizwholesale.us
jerseywholesaleelitestore.com jongose.ninja
junketjuice.blogspot.com joy-penguin.com
jyrxd.com kaidalibor.de
kam-dom.ru kambasoft.com
kazan.zrus.org kashubadesign.ru
katushka.net
keki.info
keywordblocks.com keywordblocks.com
kinobaks.com keywords-monitoring-success.com
kinoflux.net keywordteam.net
kinomaniatv.pl khafre.us
kmgamex.cf
knigonosha.net knigonosha.net
konturkrasoty.ru kometars.xyz
kosmetyki.tm.pl konkursowo-24.pl
kraljeva-sutjeska.com koronirealestate.gr
krasnodar.zrus.org kotaku.com
kredit-pod-zalog-krasnodar.ru kozhniebolezni.com
kretpicf.pw ks1234.com
ktotut.net kwzf.net
ku6.com laexotic.com
landmania.ru laparfumotec.com
landscaping.center larchik.net
laserpen.club lernur.net
lasvegaslockandsafe.com letmacwork.world
law-check-nine.xyz lexixxx.com
letmacworkfaster.world lietaer.com
lezbiyanki.net liffa.ru
lockerz.com limads.men
lion.conboy.us
lordzfilmz.me
lost-alpha.ru
louboutinshoes.xyz
luciddiagnostics.in
luckyday.world
lyrster.com lyrster.com
m.b00kmarks.com magazintiande.ru
m3gadownload.pl maglid.ru
magento-crew.net magnetic-bracelets.ru
mahnwachen-helfen.info mahnwachen-helfen.info
mall.uk mathgym.com.au
manualterap.roleforum.ru matsdale.com
mararoom.ru mature.free-websites.com
marinetraffic.com mega-bony-2017.pl
mcnamaratech.com meltwater.com
mebel-alait.ru menetie.ru
meet-flirt-dating.com metafilter.com
metascephe.com meteostate.com
midst.eu mir-business-24.ru
militarysale.pro mistr-x.org
mindbox.co.za mixtapetorrent.com
mini-modus.ru mixx.com
minneapoliscopiers.com mobplayer.ru
mir-limuzinov.ru moesexy.com
mirtorrent.net mole.pluto.ro
missclub.info mompussy.net
mmog-play.ru monclercoats.xyz
mobicover.com.ua moneymaster.ru
moesonce.com more-letom.ru
moje-recenze.cz mossmesi.com
morlat.altervista.org moviemail-online.co.uk
mp3films.ru mp3films.ru
mrlmedia.net mtmtv.info
mundoaberrante.com mturkcontent.com
muycerdas.xxx muz-shoes.ru
muz-tracker.net my-cash-bot.co
mwtpludn.review mylida.org
myblogregistercm.tk myonlinepayday.co
mycaf.it mypornfree.ru
myftpupload.com narkologiya-peterburg.ru
myhealthcare.com narkologiya-voronezh.ru
myshopmatemac.com
napalm51.nut.cc
narutonaruto.ru narutonaruto.ru
networkcheck.xyz nationalbreakdown.com
next-dentists.tk netfacet.net
nfljerseys.online news-readers.ru
ngps1.ru news-speaker.com
ningessaybe.me newsperuse.com
nootrino.com nicefloor.co.uk
nordstar.pro niki-mlt.ru
notfastfood.ru niroo.info
novosti-hi-tech.ru nmrk.ru
nvformula.ru no-fuel.org
oakridgemo.com o-o-8-o-o.com
odoratus.net o.light.d0t.ru
oivcvx.website obiavo.net
okuos.com obsessionphrases.com
omgtnc.com offer.camp
omsk.xrus.org olympescort.com
onefilms.net omoikiri-japan.ru
onlainbesplatno.ru oneclickfiles.com
onlineserialy.ru onlineslotmaschine.com
oops-cinema.ru onlywoman.org
openfrost.com
optibuymac.com optibuymac.com
optikremont.ru ownshop.cf
ostroike.org panicatack.com
outpersonals.com pay2me.pl
outshop.ru pearlisland.ru
panouri-solare-acoperis.com perm-profnastil.ru
paparazzistudios.com.au petedrummond.com
partner-trustworthy.men petsblogroll.com
pdns.cz picswe.com
perfectpracticeweb.com
perosan.com
perso.wanadoo.es
petitions.whitehouse.gov
pflexads.com
picsearch.com
picture-group.com
piluli.info
piratecams.com
piski.top
pl.aasoldes.fr pl.aasoldes.fr
playboyfiles.xblog.in plastgran.com
ploenjitmedia.azurewebsites.net play-mp3.com
pony-business.com pontiacsolstice.info
poppen-nw.net popup-jdh.xyz
poquoson.org pornhub-forum.ga
pornohub.me pornonik.com
pornstartits.xblog.in pornotubs.com
pozdravleniya-c.ru potoideas.us
prizeestates.cricket pr0fit-b0x.com
presleycollectibles.com
priceg.com
professionaldieselcare.com
profitfx.online
progress-upakovka.ru progress-upakovka.ru
promover.org pron.pro
pulseonclick.com prostitutki-astana.org
puteshestvennik.com prostitutki-belgoroda.org
qld10000.net prostitutki-rostova.org
questionmarque.ch puzzleweb.ru
ranksignals.com q-moto.ru
rarbg.to quickchange.cc
razorweb-a.akamaihd.net raschtextil.com.ua
rczhan.com rcb101.ru
reckonstat.info realitykings.com
remontbiz.ru redirectingat.com
refads.pro
reklama1.ru
remont-ustanovka-tehniki.ru
replicaclub.ru replicaclub.ru
restorator-msk.ru rixpix.ru
ring4rhino.com runstocks.com
roofers.org.uk rvzr-a.akamaihd.net
ruscoininvest.company ryetaw.com
s-iwantyou.com s-iwantyou.com
s1z.ru
sack.net sack.net
sale-japan.com sajatvelemeny.com
samoiedo.it saletool.ru
sardinie.us savememoney.co.za
satellite.maps.ilovevitaly.com sbwealthsolutions.ca
sax-sex.com scanner-julia.top
scanner-don.top scottbywater.com
scanner-josh.top
scat.porn
scenicmissouri.us
school-diplomat.ru
scrapinghub.com scrapinghub.com
se.bnt-team.com search.alot.com
search.pch.com searchwebknow-a.akamaihd.net
securityallianceservices.com seo-traffic-ranking.info
semalt.com seorankinglinks.xyz
semrush.com service-core.ru
semxiu.com serw.clicksor.com
seo18.su sethrollins.net
seoboxes.com sexgalleries.top
seokicks.de sexkrasivo.net
serptehnika.ru
servethis.com
servicecenter.co.ua
sexstream.pl
sexy-screen-savers.com sexy-screen-savers.com
shopsellcardsdumps.com sfd-chess.ru
sicfor.bcu.cc sheerseo.com
siha.de shemalegalls.blogporn.in
shop-electron.ru
shoppingjequiti.com.br
sibecoprom.ru
sildenafilcitratemed.com sildenafilcitratemed.com
simon3.ru simul.co
simple-share-buttons.com site3.free-share-buttons.com
sindragosa.comxa.com sitevaluation.org
sitebeam.net skanninge.se
siteheart.net skypasss.com
sitevaluation.com sloopyjoes.com
sk.golden-praga.ru slujbauborki.ru
slowmac.tech smallseotools.com
snip.tw smartshoppymac.com
socialbuttons.xyz social-s-iii.xyz
sockshares.tv softonicads.com
sodexo.com solmarket.by
sogimlecal.tk solnplast.ru
sonata-arctica.wz.cz stairliftstrue.com
spammen.de starwars.wikia.com
spbchampionat.ru stoki.ru
spywarebegone.com stop-zavisimost.com
startufa.ru stopnarco.ru
stathat.com sudexpert66.ru
staticfs.host support.nopeas.sk
stmassage.ru surcentro.com
stop-gepatit.te.ua svolze.com
storehouse.ua swsociety.se
studiofaca.com taaaak.com
sucsesofinspiration.com tam-gde-more.ru
survival.betteroffers.review tattoo33.ru
svnuppsalaorebro.se tech4master.com
swiped.su teguh.info
syndicate.fun telegramdownload10.com
t3chtonic.com ticketsys.inetwd.com
tandvardshuset.net tizanidine6mg.blogspot.com
tasteidea.com tizanidineingredients.blogspot.com
tdsing.ru tizanidinesideeffectsweightloss.blogspot.com
techart24.com tksn.ru
teencastingporn.com top-deal.com.pl
telefonsexkostenlos.tk torrentgamer.net
testbotprocessor44.com
thaisamkok.com
thebestweightlosspills.ovh
thebitcoincode.com
thexart.club
time-japan.ru
timhost.ru
tivolibasket.it
tizanidineduringpregnancy.blogspot.com
tizanidinehcl4mgtab.blogspot.com
tonerbox.kz
top-karkas.ru
topcar-krasnodar.ru
topflownews.com
topkarkas.com
torture.ml
tracfone.com tracfone.com
trafficshaper.com track2.shop
tryrating.com trion.od.ua
tsstcorpcddvdwshbbdriverfb.aircus.com trumpetedextremes.com
tupper-shop.ru tversvet.ru
tvnewsclips.info tvorozhnaja-zapekanka-recept.ru
twiclub.in tx41tclega.ru
twitlinks.com u555u.info
ugguk.online ultramart.biz
unece.org unmaroll.ya.ru
uptime-delta.net unrealcommander.com
vacances-voyages.info uptime-alpha.net
vacuumscleaner.com uptimebot.net
viel.su urll.eu
vikistars.com userequip.com
villacoloniale.com vacuumcleanerguru.com
vita.com.hr vancleefreplica.pw
vapeface.club
veopornogratis.xxx
viagengrarx.com
viagraneggrx.com
viandpet.com
videochat.bz
videochat.tv.br
videochaty.ru
vilingstore.net
villakohlanta.nu
vinylvault.co.uk
vkgaleria.com vkgaleria.com
vkontaktemusic.ru vkontaktemusic.ru
vpnhowto.info vltai.com
vremya.eu voloo.ru
vriel.batcave.net vsexkontakte.net
vsesubwaysurfers.com vzlomsn.org
vzubah.com wait3sec.org
w7s.ru warningzscaler.heraeus.com
waterpurifier.club
webalan.ru webalan.ru
webkeyit.com webnode.me
webpromotion.ae
website-speed-check.site
wedding-salon.net wedding-salon.net
wedding0venues.tk
weekes.biz.tc
wesharepics.site
westen-v.life
westermarkanjou.se
whatistizanidine2mg.blogspot.com
whereverdesperate.gq whereverdesperate.gq
wikes.20fr.com whipme.yopoint.in
wladimirpayen.com wildnatureimages.com
womama.ru wildworld.site
woman-h.ru windowssearch-exp.com
winwotgold.pl
wiosenny-bon-1500.pl
woman-orgasm.ru woman-orgasm.ru
womensplay.net woweb.com.ua
wonderfulflowers.biz wpsecurity.website
wordpresscore.com wpthemedetector.co.uk
x5market.ru xblognetwork.com
xaylapdiendanang.com xjlottery.com
xep.info xn--b1adccaf1bzj.xn--p1ai
xgftnlrt.bloger.index.hr xn--b1addnj3cah.xn--p1ai
xingzi-vision.com xn--h1aakne2ba.xn--p1ai
xkaz.org xn--hxazdsfy.blogspot.com
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai xolodremont.ru
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf xvideospornoru.com
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai xxxrus.org
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai xxxtube69.com
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai yhit.press
xxx-cam.webcam ynymnwbm.bloger.index.hr
yginekologa.com yourporngay.com
yourads.website yourtemplatefinder.com
zatjmuzu.info yun56.co
zigarettenonlinekaufen.tumblr.com zahvat.ru
zigzog.ru zaloadi.ru
zoogdisany.com zaloro.com
zoominfo.com zapiszto.pl
zarepta.com
zastenchivosti.net
zebradudka.com
zero1.it
zeroredirect11.com
zeroredirect7.com
znakomstvaonlain.ru
zoomovies.org zoomovies.org
zvetki.ru zvooq.eu
zx6.ru

496
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00go.com 00webcams.com
1000n1.ru 1001watch.com.ua
1008.su 1adult.com
55wmz.ru 202ch.com
7fon.ru 76brighton.co.uk
DomainStatsBot 7yue.org
ads.tremorhub.com aaa38852e886ac4af1a3cff9b47cab6272.com
adserver-e7.com aarbur.com
adtech.fr adcash.com
aerogo.com adrunnr.com
agahinameh.com advertisingtag.net
ageofclones.com afftrack001.com
aihelen.net ahrntal.verymes.xyz
akusherok.ru alldezservice.kz
albuteroli.com alpinism.ru
allesohnegirls.net anaseracresar.tk
altamayoztourism.com app-ready.xyz
amateurlivechat.org apple.com-webbrowsing-security.review
amateurmatch.com arcteryxsale.online
android-vsem.org artdestshop.eu
angkortours.vn aruplighting.com
aosexkontakte.net auto-moto-elektronika.cz
apiportalmorecom-a.akamaihd.net avtolombard-krasnodar.ru
appfastplay.com azbukadiets.ru
aramaicmedia.org balla-boo.se
arewater.com bestcurs.org
artavenuegardenstudios.com bestdraws.com
asscenihotosidea.blogspot.com bestpriceninja.com
atovh.local-finders.com beta.hotkeys.com
avitocars.ru betune.onlinewebshop.net
avon-ukraine.com bingo8888.com
azazaporn.com bobinoz.com
backgroundpictures.net bolegapakistan.com
bad-stars.net bongiornos.info
baditri.com
bala.getenjoyment.net
balans.shahterworld.org
bannertesting.com
batanga.net
battle.net
battlecarnival.su
bausparen.bz.it
bayareaaeroclub.org
beget.tech
bezlimitko.xyz
bmw.afora.ru
bmwhighperformers.com
brandov.ru brandov.ru
bupropion-sr-150-mg.us brus.city
buy-loft.ru buy-meds24.com
buyhoverboard.com callawaygolfoutlet.online
buypuppies.ca carrauterie.be
buzz-porno.info cartierbraceletsreplica.pw
bytimedance.ru cartujano-pre.de
cacheimages.com celebsopics.com
carmuffler.net cheapbarbour.online
cartierlove.xyz check-this-out-now.online
cartierring.xyz christianlouboutinsaleoutletonline.us
cascadelink.org clean-start.net
caulderwoodkennels.com cntravelre.com
ccbill.com cookie-law-enforcement-aa.xyz
checktext.org courtshipgift.com
chlooe.com crest-poloski.ru
cl.s7.exct.net crystalslot.com
classicakuhni.ru culmimg.pw
clickintext.net customboxes4less.com
clicksor.net customerguru.in
clips.ua.ac.be cutt.us
cometorussia.net danschawbel.com
compliance-walter.top dikqlyremy.info
cookie-law-enforcement-ff.xyz dikx.gdn
cpajunkies.com divisioncore.com
crazyboost.pro domoysshop.ru
custodieva.ru downvids.net
customergrowthsystems.com draniki.org
danashop.ru drhomes.biz
dating-time-now.com dvd-famille.com
dayibiao.com eager-nash.188-93-233-196.plesk.page
decorazilla.com eduserver.net
derevesendeco.com ekto.ee
dirtpics.pw eleimgo.pw
djonwatch.ru elektroniksigaraankara.info
dominterior.org erot.co
download-walpaperhd.blogspot.com exlarseva.webblog.es
ecommerce-seo.org extrabot.com
encodable.com ezigarettenshop1.yolasite.com
enthuse.computernetworksonline.com fandlr.com
eshop.md fashion-mk.net
esnm.ru figuringmoneyout.com
expdom.com financeloan.us
felizporno.com financetip.eu
flauntyoursite.com forum.doctissimo.fr
flprog.com free-laptop-reward.com
forum-engineering.ru free-video-tool.com
foxydeal.com freegamesplay.online
freshberry.com.ua frighteningremain.cf
fugarif.ga frivgame250.com
fun2cell.net galleryawesome.com
gallerylisting.com gamezblox.com
getlamborghini.ga girlsatgames.ru
gkvector.ru glall.ru
goforexvps.com gox.com.ua
gojiberriess.apishops.ru gsou.cf
googlepositions.com gwebtools.com.br
grand-chlen.ru gwhwpxbw.bloger.index.hr
gsasearchengineranker.xyz hentaiheroes.com
guitar-master.org homemature.net
hanink.biz.ly hotcore.info
headpress.ru houtings.xyz
hem.passagen.se ictizanidinehcl4mg.blogspot.com
hesteel.pl ilo134uloh.com
homegardenlova.com inboundlinks.win
hotgirlhdwallpaper.com innoslicon.com
houseofgaga.ru insomniagamingfestival.com
howtoclean.club ipostroika.ru
husky-shop.cz irkutsk.zrus.org
igru-xbox.net istizanidineanopiate.blogspot.com
ikearugs.xyz j33x.com
incep.imagine-ex.co javlibrary.cc
incomekey.net jwss.cc
internetproviderstucson.com kalandranis.gr
investsuccess.org kinosed.net
irunfar.com kinozapas.org
iweblist.info kostenlos-sexvideos.com
james13prix.info kriokomora.info
japan-bearings.ru ktotut.net
kakablog.net labelwater.se
kareliatobacco.ru labvis.host.sk
landmania.ru landinez.co
lechenie-gemorroya.com languagecode.com
louisvuittonoutletstore.net letmacwork.world
manuscript.su lexaprogeneric.link
marcoislandvacations.net linkbux.ru
michaelkorsoutlet.store lmrauction.com
mir-business-24.ru lostfilm-online.ru
mirmedinfo.ru lovasszovetseg.hu
mojaocena.com ltvperf.com
moneymaster.ru luxmagazine.cf
moneyviking-a.akamaihd.net m4ever.net
monsterdivx.com matpre.top
mriyadh.com mazda-roadsters.com
mygameplus.ru meet-flirt-dating.com
mylovelibrabry.com megaindex.ru
myphotopipe.com mmstat.com
myseoconsultant.com morocco-nomad-excursions.com
nacap.ru mossmesi.com
national-today-winning-winner.club my-floor.in.ua
nationalbreakdown.com myblogregistercm.tk
naturalshair.site mycouponizemac.com
navalwiki.info myhydros.org
neodownload.webcam mysex21.com
new-apps.ru netvouz.com
offergroup.info nhl17coins.exblog.jp
online7777.com nhl17coinsforps3.gratisblog.biz
onlinefilmz.net njkmznnb.ru
onlinewritingjobs17.blogspot.ru northfacestore.online
ororodnik.goodbb.ru o-o-8-o-o.ru
partner-cdn.men obiavo.by
parvezmia.xyz online-templatestore.com
pcimforum.com online-x.ru
pdn-4.com onlyforemont.ru
pensplan.com orion-v.com
phormchina.com ozoz.it
pl.id-forex.com paccohichetoti.ml
placid-rounded-coast.glitch.me palocco.it
podshipniki-ntn.ru par-fallen.ga
pooleroadmedicalcentre.co.uk partners-ship.pro
popunder.net partnerworkroom.men
pornhub-ru.com pb-dv.ru
porno-chaman.info pdns.cz
porno-transsexuals.ru perfection-pleasure.ru
pornofiljmi.com pics-group.com
pornoreino.com pk-services.ru
porto.abuilder.net pony-business.com
printdirectforless.com popup-jdh.xyz
pro-okis.ru pornobrazzers.biz
prostitutki-novgoroda.org pornoklad.net
pulseonclick.com postclass.com
rankscanner.com propranolol40mg.blogspot.com
redirectme.net prostitutki-almata.org
referencemoi.com proxyradar.com
registrationdomainsite.com push-ad.com
replicaclub.ru qpypcx.com
ribieiendom.no rank-checker.online
rixpix.ru razorweb-a.akamaihd.net
rostov.xrus.org real-time-analytics.com
runstocks.com reklama-i-rabota.ru
saveindex.xyz remontgruzovik.ru
sbornik-zakonov.ru rep-am.com
scanner-viktor.top reyel1985.webnode.fr
searchimpression.com rockma.se
searchkut.com rozalli.com
seasaltwithfood.com ruscoininvest.company
rusoft-zone.ru
scanner-walt.top
search-error.com
searchlock.com
securityallianceservices.com securityallianceservices.com
seokicks.de seo-tools-optimizing.com
sex-foto.pw seoanalyses.com
sexuria.net seojokes.net
sexyebonyteen.com sexkvartal.com
sfd-chess.ru sexy.babes.frontend-stack.top
sharebutton.to sexyali.com
shareyards.com simply.net
ship-marvel.co.ua site3.free-share-buttons.com
shivafurnishings.com slowmacfaster.trade
sibvitr.ru smadihome.com
sims-sims.ru smallseotools.com
skylta.com socialbuttons.xyz
socialbookmarksubmission.org spywareit.com
socialbutton.xyz stard.shop
socialseet.ru stats-collector.org
softomix.net strangeduckfilms.com
srecorder.com stroydetali.ru
stillmiracle.com subj.ukr-lit.com
sumo.com success-seo.com
susanholtphotography.com superstarfloraluk.com
taihouse.ru swingerseiten.com
taqplayer.info tagil.zrus.org
telesvoboda.ru tam-gde-more.ru
tourismvictoria.com tattomedia.com
trafficshaper.com thedownloadfreeonlinegames.blogspot.com
traffpartners.com thetoiletpaper.com
try-rx.com titelhelden.eu
tversvet.ru tizanidinefibromyalgia.blogspot.com
twelvevisionspartyofcolorado.com tizanidinemusclerelaxant.blogspot.com
udsgame.online tizanidineorflexeril.blogspot.com
uptimechecker.com tizanidinevscyclobenzaprine.blogspot.com
uzporno.mobi top10registrycleaners.com
vod.com.ua tracxn.com
vseigru.one trafficstars.com
wait3sec.org tupper-posuda.ru
wallpaperaccess.com tv-spoty.info
warezkeeper.com u-cheats.ru
watchinf.com usiad.net
webcamtalk.net vesnatehno.com
webradiology.ru viddyoze.com
websitevaluebot.com vipromoffers.com
webstatsdomain.org vipsiterip.org
wemarketing.se virus-schutzmasken.de
wineration.com vkgaleria.com
wleuaprpxuvr.ga voluumtrk.com
x-diesel.com vseuznaem.com
x-porno.video watsonrealtycorp.com
xaijo.com waysbetter.cn
xn--80aafb2a.xn--p1ai webkeyit.com
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai wordpress-crew.net
xn--80ajbshivpvn2i.xn--p1ai wosik-dach.service-for-web.de
xxx-cam.webcam wufak.com
zaim-pod-zalog-krasnodar.ru www.888.com
zaloro.com x-lime.com
zebramart.ru x-mix.info
zed21.net xn----ctbigni3aj4h.xn--p1ai
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xportvusbdriver8i.snack.ws
yaaknaa.info
zaapplesales.blogspot.com
zazagames.org
zeikopay.com
zero1.it
zoompegs.com
zzlgxh.com

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.17 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3133 ### Version: V4.2022.05.3134
### Updated: Sat May 28 22:01:27 UTC 2022 ### Updated: Sun May 29 10:03:23 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3133 ### Version: V4.2022.05.3134
### Updated: Sat May 28 22:01:27 UTC 2022 ### Updated: Sun May 29 10:03:23 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##