vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 10:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.01.2838 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-01-08 10:09:18 +00:00
parent 934b49726a
commit 44bcb9eb95
47 changed files with 74588 additions and 74578 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2837 #### Version: V4.2022.01.2838
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2837 #### Version: V4.2022.01.2838
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.01.2837 #### Version: V4.2022.01.2838
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9188
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.01.2837 #### Version: V4.2022.01.2838
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9192
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9192
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

9192
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9192
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

9192
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

9192
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

9192
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2837 2838

9192
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

9192
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.65.103.147 0; 20.83.109.36 0;

9192
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

9192
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

9192
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

9192
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.65.103.147 0; 20.83.109.36 0;

9192
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.65.103.147 0; 20.83.109.36 0;

9192
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
ADmantX Crawling at Home Project
Apexoo HTTrack
GermCrawler HybridBot
JustView InfoTekies
MarkMonitor LinqiaRSSBot
SearchmetricsBot Magnet
Vagabondo Rogerbot
WWW-Collector-E Suzuran
adscanner TheNomad
scalaj-http Tracemyfile

312
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 404checker
404enemy ADmantX
Aboundex AIBOT
ALittle Client
ASPSeek
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
Anarchie Aipbot
Alexibot
Alligator
Anarchy
Apexoo
Asterias Asterias
Attach AwarioSmartBot
AwarioRssBot BBBike
BLEXBot BDCbot
BackDoorBot BDFetch
Backlink-Ceck BackWeb
Bandit BacklinkCrawler
Barkrowler BatchFTP
BetaBot Bigfoot
BlackWidow Bitacle
Black Hole
Blackboard Blackboard
Blow Blow
BlowFish Boardreader
BuiltWith Brandwatch
CATExplorador Buddy
CCBot Bullseye
CODE87
CSHttp CSHttp
Calculon CazoodleBot
CheeseBot CensysInspect
CherryPicker CheTeam
Chlooe
Citoid
Claritybot
Cliqzbot Cliqzbot
Cocolyzebot Copier
Collector CrazyWebCrawler
CopyRightCheck CyotekWebCopy
Copyscape DIIbot
Cosmos DTS Agent
Crawling at Home Project DatabaseDriverMysqli
Custo Devil
DSearch DigitalPebble
Deusu Disco
Digincore Discoverybot
Discobot DomCopBot
DittoSpyder DomainAppender
DnBCrawler-Analytics DomainStatsBot
DnyzBot Download Wonder
DomainCrawler Dragonfly
Drip Drip
ECCP/1.0
EasyDL EasyDL
Ebingbong
Ecxi Ecxi
EirGrabber EyeNetIE
Express WebPictures FHscan
Extreme Picture Finder
Ezooms
FDM
Fuzz Fuzz
FyberSpider FyberSpider
Fyrebot Genieo
GetWeb Go!Zilla
GoZilla Gotit
GrabNet
Grafula Grafula
GrapeshotCrawler GridBot
HMView HMView
HTMLparser HTMLparser
HTTP::Lite HTTrack
Haansoft Haansoft
HaosouSpider Harvest
Havij Havij
Heritrix
Hloader Hloader
IDBTE4M
Iblog
Id-search Id-search
Image Fetch IlseBot
Indy Library
InfoTekies InfoTekies
Iskanie
JOC Web Spider JOC Web Spider
JamesBOT JetCar
Joomla Keyword Density
JustView Kozmosbot
Jyxobot LNSpiderguy
Kenjin Spider Lanshanbot
LWP::Simple LexiBot
Larbin Lftp
Leap Libwhisker
LeechFTP
LeechGet
LieBaoFast
Lightspeedsystems Lightspeedsystems
Likse
LinkScan LinkScan
LinkWalker LinkWalker
LinkpadBot LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey Lipperhey
Litemage_walker Lipperhey Spider
Lmspider
MFC_Tear_Sample
MQQBrowser MQQBrowser
MSFrontPage MSIECrawler
MTRobot
Mag-Net Mag-Net
Majestic SEO Mail.RU_Bot
Majestic-SEO Majestic12
MarkWatch MauiBot
Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot MegaIndex.ru
Metauri
MicroMessenger
Minefield
Mojeek Mojeek
NICErsPRO Morfeus Fucking Scanner
Name Intelligence Msrabot
Nameprotect Navroad
NearSite NearSite
Nettrack NetLyzer
Netvibes NextGenSearchBot
Niki-bot Niki-bot
NimbleCrawler NimbleCrawler
Nmap Ninja
Nuclei Nuclei
OnCrawl Nutch
Octopus
Offline Navigator
OpenLinkProfiler OpenLinkProfiler
OpenVAS OpenVAS
Openfind Openfind
Openvas Openvas
OrangeSpider OrangeBot
OutclicksBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageGrabber
PageThing.com PageThing.com
Pandalytics Panscient
PeoplePal Pi-Monster
Petalbot
Picscout Picscout
Picsearch Picsearch
Pimonster
PleaseCrawl PleaseCrawl
Pockey Pu_iN
ProWebWalker
Pump Pump
QueryN Metasearch PxBroker
Quick-Crawler
RSSingBot RSSingBot
RankActive
RankingBot RankingBot
RankingBot2
Rankivabot
RankurBot RankurBot
Re-re Re-re
ReGet ReGet
RealDownload RealDownload
RedesScrapy RedesScrapy
RepoMonkey
Ripper
RocketCrawler RocketCrawler
SEOprofiler Rogerbot
SISTRIX SEOkicks
SEOlyticsCrawler
SEOstats
ScanAlert ScanAlert
ScoutJet Scanbot
Screaming Screaming
ScreenerBot ScreenerBot
ScrepyBot Semrush
Searchestate SemrushBot
SearchmetricsBot
SemanticJuice
SentiBot SentiBot
SeoSiteCheckup SeoSiteCheckup
Seomoz Seomoz
Shodan Site Sucker
SiteExplorer SiteCheckerBotCrawler
SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker SiteSucker
Sitebeam
Siteimprove
SlySearch SlySearch
Sociscraper SmartDownload
Sogou web spider SocialRankIOBot
Sosospider Sqlmap
Sottopop
Spanner
Sqlworm Sqlworm
Sucuri Steeler
SuperBot SuperBot
Surfbot
SurveyBot SurveyBot
Suzuran
Teleport Teleport
Telesphoreo
The Intraformant The Intraformant
TightTwatBot TheNomad
Toata Toata
Toweyabot
Tracemyfile
Trendiction
True_Robot
Turingos Turingos
Turnitin
TwengaBot TwengaBot
Twice Twice
URLy Warning Typhoeus
URLy.Warning
UnisterBot
Upflow Upflow
VB Project
VCI
Vagabondo Vagabondo
VelenPublicWebCrawler WBSearchBot
VeriCiteCrawler WEBDAV
Voil WPScan
Voltron WWWOFFLE
WallpapersHD Wallpapers/3.0
WeSEE
Web Collage
Web Fetch
Web Sauger Web Sauger
Web Sucker Web Sucker
WebAuto WebEnhancer
WebCopier WebFuck
WebGo IS
WebImageCollector
WebLeacher
WebSauger WebSauger
WebStripper WebStripper
Webshag WebZIP
Whatweb WebmasterWorldForumBot
Website Quester
Webster
Whack
Whacker
Who.is Bot
Widow Widow
WinHTTrack WinHTTrack
WiseGuys Robot
Wonderbot
Woobot Woobot
Wotbox Wotbox
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider
Xenu
Zade Zade
Zauba adscanner
Zermelo
Zeus
Zitebot
arquivo-web-crawler
arquivo.pt
backlink-check backlink-check
check1.exe check1.exe
clark-crawler clark-crawler
com.plumanalytics
crawler.feedback crawler.feedback
crawler4j crawler4j
demandbase-bot
eCatch
evc-batch
facebookscraper
gopher
heritrix heritrix
ips-agent
isitwp.com isitwp.com
magpie-crawler meanpathbot
mediawords
netEstate NE Crawler netEstate NE Crawler
oBot pcBrowser
page scorer plumanalytics
ripz
s1z.ru
satoristudio.net satoristudio.net
scalaj-http seocompany.store
scan.lol seoscanners
serpstatbot seostar
sitechecker.pro sitechecker.pro
siteripz
sogouspider
sp_auditbot
tAkeOut
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webgains-bot webgains-bot
webmeup-crawler
webpros.com
webprosbot
zgrab

292
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404enemy 404enemy
ADmantX AIBOT
ALittle Client ALittle Client
ASPSeek Acunetix
AhrefsBot AhrefsBot
Aipbot
AllSubmitter
Alligator Alligator
Anarchie Anarchie
Ankit Ankit
Apexoo
Aspiegel Aspiegel
Asterias AwarioSmartBot
BBBike
BDCbot BDCbot
BacklinkCrawler BackDoorBot
BatchFTP BatchFTP
BetaBot BlackWidow
Bigfoot
Bitacle
Blow
BlowFish BlowFish
Boardreader Boardreader
Bolt Bolt
BotALot
Brandwatch
Buddy
BuiltBotTough BuiltBotTough
BuzzSumo BuzzSumo
CATExplorador
CCBot CCBot
CSHttp CSHttp
CazoodleBot
CensysInspect CensysInspect
ChinaClaw ChinaClaw
Chlooe Chlooe
Cliqzbot Citoid
Collector Cocolyzebot
CopyRightCheck Cosmos
Copyscape
Craftbot Craftbot
Crawling at Home Project
CrazyWebCrawler
CrunchBot CrunchBot
Curious
CyotekWebCopy
DBLBot
DTS Agent DTS Agent
DataCha0s DatabaseDriverMysqli
Devil Deusu
Discoverybot Digincore
DigitalPebble
Dirbuster
Disco
Discobot
Dispatch Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnyzBot
DomCopBot DomCopBot
DomainStatsBot Domains Project
Download Wonder Dotbot
Drip
EMail Siphon
EMail Wolf
EasyDL
Ebingbong Ebingbong
ExtLinksBot EroCrawler
Express WebPictures
Extractor Extractor
ExtractorPro Extreme Picture Finder
Firefox/7.0 FDM
FHscan
FemtosearchBot
FlashGet FlashGet
Foobot Freeuploader
GT::WWW FyberSpider
GalaxyBot Genieo
Getintent GetRight
Gigabot Go-Ahead-Got-It
Go!Zilla GoZilla
Gotit
GrabNet
Grafula Grafula
GrapeFX GrapeFX
GrapeshotCrawler
GridBot GridBot
HEADMasterSEO HEADMasterSEO
HTTrack HTTrack
Haansoft
HaosouSpider
Havij
HybridBot
IDBot
IRLbot IRLbot
Iblog Iblog
IlseBot Id-search
Indy Library Image Fetch
InfoNaviRobot InfoNaviRobot
Intelliseek IstellaBot
InternetSeer JennyBot
Iria JetCar
Iskanie Jetty
JOC Web Spider JustView
JamesBOT Jyxobot
JikeSpider
Keyword Density Keyword Density
Kinza
Kozmosbot
LWP::Simple
Leap Leap
LeechGet
LexiBot LexiBot
Lftp
LieBaoFast LieBaoFast
LinkScan LinkWalker
LinkextractorPro Linkdexbot
LinkpadBot
LinksManager LinksManager
Lipperhey Spider LinqiaMetadataDownloaderBot
Lmspider LinqiaRSSBot
MFC_Tear_Sample LinqiaScrapeBot
MIDown tool MJ12bot
MQQBrowser
MSFrontPage MSFrontPage
MSIECrawler
MTRobot
Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic12 MegaIndex.ru
MarkMonitor
Mata Hari
Mb2345Browser
Meanpathbot
MicroMessenger
Minefield Minefield
Moblie Safari Moblie Safari
Mojeek
Mojolicious Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
Msrabot Navroad
Musobot Netcraft
NICErsPRO Netvibes
Nameprotect
NearSite
Needle
Nessus
NetLyzer
NetSpider
Nettrack
NextGenSearchBot NextGenSearchBot
Ninja Nibbler
NimbleCrawler
Nmap Nmap
Not Not
Nuclei Nuclei
Nutch Nutch
Openvas OpenVAS
OrangeSpider OrangeSpider
OutclicksBot
PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer
PageGrabber PageGrabber
PageScorer Pandalytics
Panscient PeoplePal
Piepmatz Petalbot
Pimonster Picscout
Picsearch
Pixray Pixray
PleaseCrawl
Pockey
ProPowerBot ProPowerBot
PxBroker Probethenet
RankFlex RankActive
RankingBot RealDownload
RankingBot2 Ripper
Reaper RocketCrawler
RepoMonkey
Rogerbot Rogerbot
SBIder
SEOkicks
SEOlyticsCrawler SEOlyticsCrawler
SEOstats
SMTBot
Scanbot Scanbot
ScoutJet ScoutJet
ScreenerBot Scrapy
SearchmetricsBot Searchestate
SemanticJuice SemanticJuice
Semrush SemrushBot
SeoSiteCheckup
SeobilityBot
Seomoz
Siphon Siphon
Site Sucker SiteSnagger
SiteCheckerBotCrawler
SiteSucker
Siteimprove
SlySearch SlySearch
Snoopy
SocialRankIOBot
Sosospider Sosospider
Sottopop Sottopop
Spammen Spammen
SpankBot Spanner
Spbot
Spinn3r
Sqlmap Sqlmap
Sqworm
Steeler Steeler
Stripper SuperBot
Sucuri SuperHTTP
Surfbot SurveyBot
Suzuran
Swiftbot Swiftbot
Szukacz
T0PHackTeam
Teleport
TeleportPro TeleportPro
Telesoft Telesoft
Telesphoreo Telesphoreo
TheNomad
TightTwatBot TightTwatBot
Titan Titan
Trendictionbot Trendiction
Turnitin
TwengaBot
Twice Twice
Upflow Typhoeus
VCI URLy Warning
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VoidEYE Virusdie
Voil
WASALive-Bot WASALive-Bot
WEBDAV WEBDAV
WISENutbot
WWW::Mechanize WWW::Mechanize
WWWOFFLE WWWOFFLE
WallpapersHD
WeSEE WeSEE
Web Auto Web Auto
Web Collage Web Collage
WebAuto Web Fetch
WebBandit Web Fuck
WebCollage
WebCopier WebCopier
WebEnhancer WebImageCollector
WebFetch
WebLeacher
WebPix WebPix
WebWhacker WebStripper
WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Whacker Webster
Whack
Whatweb
Who.is Bot Who.is Bot
WinHTTrack Widow
Woobot WiseGuys Robot
Wonderbot
Wotbox
Wprecon Wprecon
Xenu Xaldon WebSpider
Zeus Zauba
ZoomBot ZoomBot
ZumBot ZoominfoBot
adscanner adscanner
arquivo-web-crawler autoemailspider
backlink-check cah.io.community
check1.exe
clark-crawler clark-crawler
cognitiveseo com.plumanalytics
domainsproject.org crawl.sogou.com
eCatch crawler4j
gopher dataforseo.com
evc-batch
heritrix
isitwp.com isitwp.com
meanpathbot magpie-crawler
mediawords
netEstate NE Crawler
page scorer page scorer
plumanalytics
probe-image-size probe-image-size
s1z.ru
satoristudio.net
scalaj-http scalaj-http
scan.lol seocompany.store
seobility seostar
seoscanners
sexsearcher
sitechecker.pro sitechecker.pro
sogouspider siteripz
sp_auditbot sp_auditbot
spyfu sysscan
trendiction.com trendiction.de
voyagerx.com ubermetrics-technologies.com
webmeup-crawler webmeup-crawler
webpros.com webpros.com
x22Mozilla webprosbot
x09Mozilla
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
101flag.ru customergrowthsystems.com
amyfoxfitness.com extremepornos.net
autoloans.com filkhbr.com
clicksor.com fivedwld.cf
dorratex.tn gemara.com
escortslet.net howmuchdoestizanidinecost.blogspot.com
iomoio.net kometars.xyz
kosmetyki.tm.pl monthlywinners.com
perfection-pleasure.ru scanner-irvin.top
proekt-gaz.ru video-chat.in

914
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
101lesbian.xyz 20pascals.nl
123locker.com 4istoshop.com
24x7-server-support.site 6128786.com
2girls1cup-free.com 6hopping.com
2pxg8bcf.top 7ooo.ru
404.mn
4kpics.rocks
4ur.click
5ws.dating-app.ru
70casino.online
7search.com 7search.com
98oi.ru adconscious.com
a96527gi.beget.tech addbin.men
aa625d84f1587749c1ab011d6f269f7d64.com adminshop.com
aa8780bb28a1de4eb5bff33c28a218a930.com adsfresh.men
abovetherivernc.com adssafeprotected.com
adbetclickin.pink adultactioncam.com
admatic.com.tr advancedcleaner.com
adserver-e7.com
adultmeetonline.info
advertex.info
advertisingtag.net advertisingtag.net
airmaxshoes-2016.com agroeconom.kz
ahmedabadwebs.com
alfa-medosmotr.ru alfa-medosmotr.ru
algerianembassy.co.in alfabot.xyz
all4invest.info alice.it
allesohnegirls.net alive-ua.com
alta-realestate.com alldownload.pw
altamayoztourism.com allkrim.com
amanda-porn.ga am-se.com
amateurmatch.com anapa-inns.ru
amazingpic.net apartmentratings.com
androids-store.com apiadanaknet-a.akamaihd.net
angkortours.vn apibatbrowsecom-a.akamaihd.net
animali.deagostinipassion.it apiluckyleapnet-a.akamaihd.net
apessay.com approvedlocal.co.za
apimegabrowsebiz-a.akamaihd.net apps-analytics.net
apimossnetinfo-a.akamaihd.net arewater.com
applyneedy.xyz arkkivoltti.net
apxeo.info asociatia-tipografilor-transilvania.ro
asdfg.pro asrv-a.akamoihd.net
autobudpostach.club asrvrep-a.akamaihd.net
aviav.ru.com asscenihotosidea.blogspot.com
avon-ukraine.com atdedinotuho.tk
azadnegar.com ausergrubhof.info
bagsonsale.online avek.ru
baikaleminer.com avtolombard-krasnodar.com
balans.shahterworld.org avtolombard-krasnodar.ru
balkanfarma.org babosas.com
bagcionderlab.com
baixar-musicas-gratis.com
balois.worldbreak.com balois.worldbreak.com
balook.com bambi.ck.ua
banan.tv banki76.ru
bannerconnect.net bankiem.pl
barnfurnituremart.com banksrf.ru
battlecarnival.su bedcapdealers.com
beetpics.pw beget.tech
belsetirehafi.tk bfz.biz
bensbargains.net bjanshee.ru
bestcurs.org blackle.com
bestdraws.com blavia.00author.com
bestserials.com bloggerads.net
beyan.host.sk blogrankers.com
biglistofwebsites.com boeuklad.com
bingoporno.com bonvillan.com
biznesrealnost.ru bookhome.info
biznesschat.net
bloggers.nl
bolezniorganov.ru
boobsimge.com
bracketsmackdown.com
brandov.ru brandov.ru
browsepulse-a.akamaihd.net brothers-smaller.ru
brus-vsem.ru browserprotecter.com
btnativenav.com budpost.com.ua
bumble.cheapwebsitehoster.com
buscarfoto.com
business-made-fun.com
butstrap.space
buyessaynow.biz
buytizanidineonline.blogspot.com buytizanidineonline.blogspot.com
byme.se buyviagraa.blogspot.com
calvet.altervista.org bvlgariring.xyz
carapics.com calc-for-credit.ru
cartierbraceletsreplica.pw cam-kontakte.org
candytiner.com
canrole.com
cartierloveringreplica.xyz
cctva.tv
cdn.walkme.com cdn.walkme.com
cheapmarmot.online changde.58.com
cheapwebsitehoster.com chcu.net
cheapcigarettesc.info
cheapsergiorossi.online
chee-by.biz
chelyabinsk.xrus.org chelyabinsk.xrus.org
christianlouboutinsaleoutletonline.us cherrypointplace.ca
ciceros.co chinacheapelitejerseys.com
christianlouboutinoutlet.win
christianlouboutinreplica.win
civilwartheater.com
classiquebijoux.ru classiquebijoux.ru
cleanmypc.com claytransformations.info
clickso.com cleanallspyware.com
codysbbq.com clippingphotoindia.com
com-cleaner.systems cobaltpro.ru
compliance-alex.top coindirect.io
compliance-barak.xyz colehaanoutlet.store
compliance-checker-7.info communisave.co.za
compliance-ivan.top compliance-mark.top
contentlook.co conciergegroup.org
contextualyield.com cookie-law-enforcement-ff.xyz
continent-e.tv coolgamechannel.com
cookie-law-enforcement-aa.xyz costablue.xyz
cookielawblog.wordpress.com counciltally.com
cool-wedding.net
coolgramgoods.com
coolingoods.com
copenergo.ru
cornomase.win
country-chic.ru country-chic.ru
cowblog.fr covadhosting.biz
craftinsta.ru craftinsta.ru
crd.clan.su credit-respect.ru
custom-product-labels.com customerguru.in
danashop.ru cyse.tk
dangphoto.trade db.speedup.gdn
daooda.com ddsoldes.fr
data1.scopich.com
ddpills.com
deluxedumps.com
depositfiles-porn.ga
detoxmed24.ru detoxmed24.ru
detsky-nabytek.info dfwu1013.info
digitalassetmanagement.site
digitalfaq.com digitalfaq.com
dignitasdata.se dinosaurus.site
dikqlyremy.info dms-sw.ru
dostavka-v-ukrainu.ru dnepropetrovsk.mistr-x.org
doxyporno.com documentbase.net
doxysexy.com doeco.ru
drunkmoms.net domcran.net
druzhbany.ru domznaniy.ru
dvr.biz.ua dpihatinh.gov.vn
dwomlink.info drunkenstepfather.com
eaplay.ru e-buyeasy.com
easycommerce.cf e-poker-2005.com
ecxtracking.com e2click.com
eets.net election.interferencer.ru
egvar.net eleimgo.pw
electro-prom.com elektrischezigarette1.blog.pl
elektronischezigarettekaufen1.beeplog.com en.home-task.com
elektronischezigarettekaufen1.myblog.de energy-ua.com
elektrozigaretten1.postbit.com eng-lyrics.com
elektrozigaretten1.tumblr.com envaseslotusama.com
eloconcream.blogspot.com etur.ru
eredijovon.com europages.com.ru
erotag.com evidencecleanergold.com
erotikchat-24.com expertblog.info
everytalk.tv extads.net
evrotekhservis.ru extstat.com
excaliburfilms.com eyes-on-you.ga
extlinks.com ezigaretteshop2.vefblog.net
ezigarettekaufen2.blox.pl fable.in.ua
ezofest.sk
ezrvrentals.com
f0815.de
fakehandbags.xyz fakehandbags.xyz
fanoboi.com familyholiday.ml
fanpagerobot.com fast-wordpress-start.com
felizporno.com fetishinside.com
fighrofacciufreesig.ga fifa-coins.online
firstsiteguide.com filesclub.net
fitfloponline.store filmbokep69.com
fitnesspiks.com filmetricsasia.com
finemanteam.com
fiuxy.com
fivedwld.cf fivedwld.cf
fm-upgrade.ru flipper.top
flooringinstallation-edmonton.com
formseo.com formseo.com
forpostlock.ru
forum.doctissimo.fr forum.doctissimo.fr
forum.tvmir.org forum69.info
foto-sisek.porngalleries.top freesitetest.com
fr.netlog.com freshsuperbloop.com
freakycheats.com fuckmill.com
friendflnder.com fyl.com.ru
fungirlsgames.net gambarkatabaru.com
funnymama.com gamblingnerd.com
futbolkisales.ru gayxperience.com
galeon.com gazobeton-p.com.ua
galeria-zdjec.com gdebestkupit.ru
gamedayhouse.com gilbertbanda.net
gamewrath.com glavprofit.ru
gay.adultgalls.com globalsurfari.com
gemgrab-a.akamaihd.net goldadpremium.com
generalporn.org gombita.info
generic-pills-online.com goodly.pro
getoutofdebtfree.org gox.com.ua
ghernnqr.skyrock.com greatzip.com
girlsatgames.ru gsasearchengineranker.xyz
gk170.ru guge.io
global-ics.co.za gwhwpxbw.bloger.index.hr
glogow.pl
golden-catalog.pro
goldpanningtools.com
good-mummy.ru
googlepositions.com
greenzaim.ru
gsasearchengineranker.pw
guardlink.org
guildebzh.info
hamptonoaks.ca hamptonoaks.ca
haveinc.xyz hatdc.org
hausfrauensex18.com
hchha.com
hd720kino.ru
hdapp1008-a.akamaihd.net hdapp1008-a.akamaihd.net
hdimagelib.com
heartofbeijing.blogspot.com heartofbeijing.blogspot.com
heatpower.ru
helpmymacfaster.trade
hermesbelts.xyz hermesbelts.xyz
hermesbirkinhandbagoutlets.com hermesreplica.pw
hostsshop.ru heroz.fr
hoverboard360.se histock.info
howmuchdoestizanidinecost.blogspot.com home-task.com
howtowhitenteethfast.xyz homeandhealth.ru
ibb.com.ua hotblog.top
iblogpress.xyz houtings.xyz
idegenvezeto.eu hyhj.info
ifirestarter.ru hyiphunter.org
igadgetsworld.com ictizanidinehcl4mg.blogspot.com
ilte.info igru-xbox.net
images.gyffu.com igtools.club
indo-export.ru ihtec2019.org
infoupdate.org ikearugs.xyz
innodgfdriverhm.aircus.com ilbe.club
io9.com ilikevitaly.com
ipchicken.com ilovevitaly.ru
isistaylorporn.info iminent.com
itservicesthatworkforyou.com in-tandem.co
iwantmyfreecash.com inome.com.ua
janavibekken.no investmac.com
japan-bearings.ru iplogger.org
javstock.com iqupdatetmz.win
jerseychinabizwholesale.com itrevolution.cf
job.icivil.ir ivearchenceinflu.cf
jongose.ninja iweblist.info
jav-fetish.com
javrip.net
jennyfire.ru
jofucipiku.tk
jose.mulinohouse.co
justdating.online justdating.online
jwcialislrt.com kaac.ru
jyvopys.com keywordblocks.com
kalb.ru keywordsuggests.com
kartiniresto.com kinobaks.com
kerei.ru kleine-titten.biz
keywordbasket.com
kino-ecran.ru
kinomaniatv.pl
kinozapas.com
klosetkitten.com klosetkitten.com
klumba55.ru koversite.info
knigonosha.net krasivye-devushki.net
kollekcioner.ru krasnodar.xrus.org
komp-pomosch.ru kretpicf.pw
kongruan.com kurbappeal.info
konteiner24.com laexotic.com
kraljeva-sutjeska.com landmania.ru
krasnodar-avtolombards.ru landscaping.center
kriokomora.info laptop-4-less.com
krynica.info lenvred.org
kvartir-remont.biz light.ifmo.ru
labvis.host.sk liky.co.ua
landreferat.ru lineavabit.it
lankarns.com linksharingt.com
larutti.ru littleberry.ru
laxdrills.com
leftofcentrist.com
lexaprogeneric.link
libertybilisim.com
lider-zhaluzi.kiev.ua
likrot.com
limads.men
lkbennettoutlet.online lkbennettoutlet.online
lmrauction.com llastbuy.ru
lonerangergames.com loadingpages.me
lost-alpha.ru louboutinreplica.pw
macnewtech.com louisvuittonoutletstore.net
maglid.ru luckyday.world
macresource.co.uk
mactechinfo.info
mailemedicinals.com mailemedicinals.com
malls.com manifestyourmillion.com
mapquestz.us marcowebonyodziezowe.pl
matpre.top maslenka.kz
mbiologi.ru masserect.com
medkritika.ru mebeldekor.com.ua
meduza-consult.ru medi-fitt.hu
mega-bony2017.pl member-quiz.com
megagrabber.ru merryhouse.co.uk
meteostate.com mmstat.com
methodsmarketing.com monclercoats.xyz
mezaruk.info
mhi-systems.ru
michellblog.online
migente.com
mindeyegames.com
mirmedinfo.ru
moncleroutletonline.pw
moneyviking-a.akamaihd.net moneyviking-a.akamaihd.net
monthlywinners.com moscow.xrus.org
moscow-region.ru musirc.com
mp3ringtone.info my-bc.ru
mug-na-chas-moscow.ru my-big-family.com
mustat.com my-cash-bot.co
mustwineblog.com
myblogregistercm.tk myblogregistercm.tk
myplaycity.com mydeathspace.com
narkologiya-orel.ru myfreemp3.eu
narkologiya-penza.ru mymobilemoneypages.com
narosty.com mypornfree.ru
newfilmsonline.ru myyour.eu
news-readers.ru nac-bearings.ru
newstraveller.ru naj-filmy24.pl
newstudio.tv nalogovyykodeks.ru
nonews.co narco24.me
novosibirsk.xrus.org nardulan.com
novosti-hi-tech.ru natali-forex.com
nrjmobile.fr net-profits.xyz
nuit-artisanale.com nlfjjunb5.ru
oballergiya.ru no-fuel.org
obiavo.net nomuos.it
offtime.ru nudepatch.net
okmedia.sk nufaq.com
okuos.com oakleyglassesonline.us
olgacvetmet.com oconto.ru
onlinebay.ru oddamzadarmo.eu
oralsexfilme.net ofanda.com
paceform.com offers.bycontext.com
palocco.it okonich.com.ua
pammik.ru omsk.xrus.org
pdns.download oneclickfiles.com
penisvergrotendepillennl.ovh ontargetseo.us
pepperstyle.ru ooo-olni.ru
perfection-pleasure.ru open-odyssey.org
php-market.ru ostrovtaxi.ru
pic-re.blogspot.com ozas.net
picsfair.com panchro.co.uk
picsforkeywordsuggestion.com paphoselectricianandplumber.com
picture-group.com pastaleads.com
pinkduck.ga pawli.eu
pl-vouchers.com paydayonlinecom.com
play.leadzupc.com pdn-4.com
playboyfiles.xblog.in photo-clip.ru
pliks.pl phuketscreen.com
pillscheap24h.com
pio.polytopesexempt.com
pisanieprac.info
pizza-imperia.com
pkr1hand.com
pl-top.pl
pl.id-forex.com
pod-muzyku.club pod-muzyku.club
poisk-zakona.ru podshipniki-nsk.ru
popup-fdm.xyz pony-business.com
pornosmola.info porndairy.in
pornotubs.com porndl.org
pornzone.tv porno-transsexuals.ru
portnoff.od.ua porno-video-chati.ru
pricheskaonline.ru pornoelita.info
pornohd1080.online
pornoklad.net
pornoxxx.com.mx
porodasobak.net
pozdrawleniya.com
prchecker.info
preconnubial.usuby.site
printingpeach.com
privatbank46.ru
producm.ru producm.ru
proekt-mos.ru profitsport.club
professionalwritingservices15.blogspot.ru pronekut.com
prom23.ru
pshare.biz pshare.biz
puserving.com psoriasis-file.trade
pushdata.sendpulse.com pwwysydh.com
pussysaga.com raavidesigns.com
pussyspace.net rada.ru
rapidsites.pro rangoman.date
rebuildermedical.com ranking2017.ga
redirect.trafficreceiver.club rapidgator-porn.ga
reklama-i-rabota.ru realting-moscow.ru
reklama1.ru
relax.ru
remedyotc.com
remont-ustanovka-tehniki.ru
remybutler.fr remybutler.fr
renewablewealth.com restaurantlescampi.com
rennlist.com reversing.cc
rentehno.ru
reward-survey.net reward-survey.net
rewardpoll.com ringporno.com
rfd-split.hr rome2rio.com
ribieiendom.no ruscopybook.com
riders.ro rusenvironmental.net
riralmolamsaca.tk rutor.group
rogervivierforsale.com sa-live.com
ros-ctm.ru salmonfishingsacramentoriver.com
runofilms.ru samo-soznanie.ru
russian-postindex.ru sampleletters.net
rutor.vip santasgift.ml
s.lollypopgaming.com sarahmilne.top
sadaholding.com sardinie.us
sale-japan.com saveriopiazza.it
sammyweaver.com savetubevideo.com
samolet.fr sbt-aqua.ru
sax-sex.com scanner-jack.top
scanner-viktor.top scanner-julianna.top
schoolfiles.net school-diplomat.ru
seadragonherbery.com searchinterneat-a.akamaihd.net
searchaddis.com searchlock.com
seksvideoonlain.com seeingmeerkat.com
semrush.com seo-tools-optimizing.com
semxiu.com service-core.ru
seo-prof1.xyz serving.adbetclickin.pink
seocheki.net sevendays.com.ua
seoholding.com sex.hotblog.top
seorankinglinks.com sexspornotub.com
sex-watch.com sharpchallenge.com
sexiporno.net
sexpornotales.net
sexvideo-sex.com
sexy-screen-savers.com
shakhtar-doneck.ru
sharebutton.net
sherlock.se sherlock.se
siha.de shmetall.com.ua
silvermature.net signoredom.com
simple-image.com.ua simplesite.com
skladvaz.ru skatestick.bid
smadihome.com sklad-24.ru
smstraf.ru smallseotools.com
socialbookmarksubmission.org smartbalanceworld.com
spaceshipad.com snomer1.ru
spasswelt.net social-button.xyz
softomix.com
softtor.com
sovinsteel.ru
spb.afora.ru
spin2016.cf
spitfiremusic.com
sports-supplements.us sports-supplements.us
srgwebmail.nl stal-rulon.ru
stackthatbucks.com start.myplaycity.com
staff.prairiesouth.ca startwp.org
stickers-market.ru stathat.com
stoki.ru stats-public.grammarly.io
sudexpert66.ru stauga.altervista.org
superlist.biz sterva.cc
sureone.pro stillmiracle.com
surffoundation.nl storehouse.ua
susanholtphotography.com streamin.to
swimpool.ca stretchingabuckblog.com
taboola.com styro.ru
tamada69.com supernew.org
tds-advert005.info sweepstakes.rewardit.com
technika-remont.ru syndicate.fun
tehngr.ru talant-factory.ru
telefonsexi.com tarad.com
thefds.net teastory.co
tizanidine4mgtablets.blogspot.com technopellet.gr
tizanidine6mg.blogspot.com teencastingporn.com
tizanidinevssoma.blogspot.com thetoiletpaper.com
tizanidinewithdrawal.blogspot.com tiens2010.ru
timer4web.com
tizanidineforopiatewithdrawal.blogspot.com
tizanidinestreetprice.blogspot.com
toolsky.com
toon-families.com
top10registrycleaners.com
torrents-tracker.com torrents-tracker.com
toy-shop.top torture.ml
tracksurf.daooda.com tracksurf.daooda.com
trackzapper.com trackzapper.com
trafficreceiver.club traffic2money.com
treasuretrack-a.akamaihd.net tsyndicate.com
truemfilelj.gq tt-ipd.info
tuberkuleznik.ru tvnewsclips.info
tula.mdverey.ru
turist-strani.ru
tvand.ru
tvorozhnaja-zapekanka-recept.ru tvorozhnaja-zapekanka-recept.ru
umaseh.com tvory.predmety.in.ua
unitexindia.com twittruth.com
upproar.com ucsol.ru
uhodzalijami.ru
uptime-alpha.net uptime-alpha.net
uptimechecker.com urzedowski.eu
urccvfmc.bloger.index.hr valkiria-tk.ru
url-img.link valuado.com
urlopener.blogspot.com.au vduplo.ru
utiblog.fr videochat.life
v-doc.co videochaty.ru
vashsvet.com videosbox.ru
vavilone.com
vegan-foods.us
vesnatehno.com
video-chat.love
videojam.tv
videooko.weebly.com
videospornogratisx.net
vids18.site vids18.site
vietimgy.pw
vintontech.info vintontech.info
vipms.ru vip-parfumeria.ru
virus-respirators.com vkmusics.ru
vkontarkte.com vod.com.ua
vladhistory.com volgograd.xrus.org
voprosotvet24.ru voloo.ru
vrotike.ru
vvon.co.uk
vzlomsn.org vzlomsn.org
warezaccess.com walkme.com
web.cvut.cz warningwar.ru
webextract.profound.net we-are-gamers.com
webmonetizer.net webads.co.nz
webradiology.ru webmasterhome.cn
website-speed-up.site webtherapy.ru
websitevaluebot.com weburok.com
wemedinc.com wheelchairliftsarea.com
wesharepics.com whereiskentoday.com
whengirlsgowild.com whereverdesperate.gq
whipme.yopoint.in windowssearch-exp.com
win-spy.com wineration.com
witclub.info wnoz.de
witherrom55.eklablog.fr wordkeyhelper.com
womensplay.net wtsindia.in
womensterritory.ru www.solartek.ru
workle.website wygraj-teraz.com
world-mmo.com x-mix.info
wurr.voila.net xchat26.myfreecams.com
www.get-free-traffic-now.com xclicks.net
xblog.in xedserver.com
xcombear.ru xn----9sbebi2bvzr7h.xn--p1ai
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai xn--80ab4aa2g.xn--p1ai
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
xn----7sbifcamovvfggw9d.xn--p1ai xn--80ahvj9e.xn--p1ai
xn--80ahdheogk5l.xn--p1ai xn--hxazdsfy.blogspot.com
xn--b1ag5cfn.xn--p1ai xxlargepop.com
xn--h1ahbi.com.ua xxx-cam.webcam
xn--l1aengat.xn--p1ai xxxasianporn.net
xtremeeagles.net xxxdatinglocal.us
xxxguitars.com
you-shall-not-pass.is74.ru you-shall-not-pass.is74.ru
yougame.biz yougotanewdomain.com
youghbould.wordpress.com zaobao.com.sg
za-fun-offer.com zastenchivosti.net
zajm-zalog-krasnodar.ru zazagames.org
zakazfutbolki.com zigarettenonlinekaufen1.bloog.pl
zambini.ru zigarettenonlinekaufen1.blox.pl
zarabiaj-dzis.pl zirondelli.it
zeroredirect7.com znakomstvaonlain.ru
zionstar.net zona-aqua.ru
zskdla.site zone-kev717.info
zoogdinsney.com
zvetki.ru

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
000free.us 1-free-share-buttons.com
101lesbian.xyz 1001watch.com.ua
2345.com 2girls1cup-free.com
3weekdiet.com 3rbseyes.com
7wind.ru 5000-cotydzien.com
8gold.com 5kstore.com
abcguru.xyz aa81bf391151884adfa3dd677e41f94be1.com
adhome.biz addbin.men
ads.gold ads.gold
adzpower.com adserver-e7.com
affrh2015.com adservme.com
albamargroup.com adtech.us
alert-jdh.xyz aktivator-windows10.blogspot.com
alive-ua.com albumsuper.info
alveris.ru all4invest.ru
amazingninja.com apartamentwroclaw.eu
anonymousfox.co api.stathat.com
apartmentbay.ru arendovalka.xyz
arabgirls.adultgalls.com artdeko.info
astana.xxxkaz.org asrvrep-a.akamaihd.net
aversis.net aviav.org
balkanfarma.org ayanaspa.com
bear.gotcher.us azbukadiets.ru
beauty-b0x.pl b-styles.xyz
berrymall.ru bag77.ru
bestssaker.com baltgem.com
big-cash.net bard-real.com.ua
binomo.kz battle.net
blockety.co bestbuy.ca
blog.yam.com bestkfiledxd.cf
boom-celebs.com betterhdporn.com
brendbutik.ru betwinservice.com
buyantiviralwp.com bioscorp.ru
buzzzg.men blog.koorg.ru
carapics.com bmsco.co
ci.ua bolegapakistan.com
cialisndbrx.com bongacams.com
ciceros.co brainboostingsupplements.org
clayvasedesigns.tk brains2.biz
cornerstone-countertops.com buybest1.biz
cowlmash.com bvlgaribracelet.xyz
customerguru.in cakesplus.com.au
d2jsp.org calcularpagerank.com
dangphoto.trade cardul.ru
dayibiao.com cartierring.xyz
ddospanel.com caveavins.fr
deanmoore.ie cheatcode-lita12.rhcloud.com
dengi-pod-zalog-nedvizhimosti.ru ciekawinki.pl
diegolopezcastan.com coindirect.io
djstools.com confib.ifmo.ru
dneprsvet.com.ua counter.yadro.ru
dotmass.top darmebel.com.ua
download-of-the-warez.blogspot.com dataloading.net
download-wallpaper.net date-withme.com
dpihatinh.gov.vn delitime.info
drillsaw.ru deniven.1bb.ru
droidlook.net dev.citetick.com
dwomlink.info dipstar.org
dyt.net dosugrostov.site
e-buyeasy.com dotnetdotcom.org
eaplay.ru downloadmefiranaratb1972.xpg.com.br
eas-seo.com dprtb.com
edge.sharethis.com dscaas.website
elmifarhangi.com ekspertmed.com
ereko.ru englate.com
ertelecom.ru eyes-on-you.ga
exdocsfiles.com ezigarettenkaufen2.dreamwidth.org
f-loaded.de falllow.gq
family1st.ca famix.xyz
fefo.gdn fantasticpixcool.com
flashahead.info fearcrow.com
forums.toucharcade.com fetroshok.ru
free-fbook-traffic.com fickkontakte.org
freetangodownload.com filkhbr.com
freeuploader.com financetip.eu
freevpn.space forex-procto.ru
funponsel.com forsex.info
gaming-journal.com forzeronly.com
gearcraft.us foto-weinberger.at
gg.zzyjxs.com frbizlist.com
gilbertbanda.net freshwallpapers.info
gk-atlant.info funtoonez.com
goforexvps.com gameplexcity.com
gelezki.com
germes-trans.com
getmyads24.com
gifspics.com
globatur.ru
globetrotting-culture.ru
goldandcard.ru
google-liar.ru google-liar.ru
greendream.com.ua hackers-crackers.tk
gsasearchenginerankerdiscount.com haikuware.com
hanink.biz.ly hao123.com
hdimagegallery.net hausfrauensex18.com
hdseriale.pl headpress.ru
hentai-manga.porn hermesbirkinhandbagoutlets.com
historichometeam.com
hmywwogw.bloger.index.hr
homeandhealth.ru homeandhealth.ru
hotshoppymac.com hostsshop.ru
imagecoolpub.com hotxnights.info
infodocsportal.com i4track.net
iplusbit.blogspot.co.za ictizanidinehcl4mg.blogspot.com
irkutsk.online-podarki.com ifmo.ru
isistaylorporn.info ilmexico.com
istanbulit.com images-free.net
izoll.ru imitex-plus.ru
jovencitas.gratis insta-add.pro
kamalsinha.com investzalog.ru
kamen-e.ru iqoption-bin.com
keywordsdoctor.com iskalko.ru
kinobest.pl ispac.org
knowsitall.info ixora.pro
kung-fu-ru.com jagg.info
kutikomi.net jikoman.info
laulini.soclog.se johnnyhaley.top
listiseltemournlan.gq kazka.ru
liverpool.gsofootball.com kino-key.info
ljusihus.se klitimg.pw
mackeeper-land-672695126.us-east-1.elb.amazonaws.com klosetkitten.com
mebeldekor.com.ua knigonosha.net
medanestesia.ru koronirealestate.gr
mediaoffers.click krasula.pp.ua
mere.host.sk kredit-blog.ru
mex-annushka.ru kunstaktien.info
law-check-seven.xyz
lechtaczka.net
levaquin750mg.blogspot.com
limtu.ifmo.ru
lostaruban.com
lrac.info
macdamaged.tech
magda-gadalka.ru
magicdiet.gq
makemoneyonline.com
marketingtechniques.info
marmotstore.online
me-ke.com
medkritika.ru
meltwater.com
mexicotravelnet.com mexicotravelnet.com
midst.eu
mlf.hordo.win
mmofreegames.online mmofreegames.online
montredemarque.nl mobilemedia.md
mosdverka.ru mobplayer.ru
mostantikor.ru mockupui.com
monarhs.info
monclerjacketsoutlet.win
mowser.com
moyakuhnia.ru
mp3s.club
msk.afora.ru msk.afora.ru
mulberryoutletonlineeu.com mybackgroundlandscape.blogspot.com
muycerdas.xxx mydownlodablefiles.com
my-cash-bot.co mylovelibrabry.com
naphukete.ru myshopmatemac.com
narco24.me naj-filmy24.pl
nardulan.com national-today-winning-winner.club
naturtreenspicerx.pw nevansk.ru
networkcheck.xyz nicovideo.jp
newfilmsonline.ru nvformula.ru
nofreezingmac.work obesidadealgarve.com
nootrino.com opinionreelle.com
northfacestore.online optom-deshevo.ru
nowtorrents.com ourville.info
npoet.ru panchro.co.uk
nzfilecloud.weebly.com popugauka.ru
oil-td.ru pornodojd.ru
okmedia.sk pornofeuer.com
okuos.com pornonik.com
oneminutesite.it portside.xyz
oneshotdate.com
online-zaymy.ru
pandarastore.top
partnerworkroom.men
pekori.to
phrcialiled.com
picsearch.com
porndl.org
porno-transsexuals.ru
porodasobak.net
potolokelekor.ru
predominant-invent.tk
privat-girl.net privat-girl.net
producm.ru
profitsport.club
promoforum.ru promoforum.ru
prostitutki-kiev.org
prostitutki-novgoroda.org
proxyradar.com
pulse33.ru pulse33.ru
purchasepillsnorx.com py100.ru
qwertty.net r-control.ru
rabotaetvse.ru rankscanner.com
rankexperience.com rasteniya-vs-zombi.ru
remont-comp-pomosh.ru rednise.com
rethinkwasteni.info reimageplus.com
rfid-locker.co remedyotc.com
rfserial.net remont-fridge-tv.ru
rialp.getenjoyment.net rent2spb.ru
roll123.com rep-am.com
rvi.biz reward-survey.net
samolet.fr rp9.ru
scanner-jess.top rrutw.com
seadragonherbery.com running-line.ru
seeingmeerkat.com sarafangel.ru
seinterface.com scanner-barak.top
seokicks.de searchimage.co
searchkut.com
service.adtech.us service.adtech.us
serw.clicksor.com sexphoto.site
sevenstars7.com shop.xz618.com
sexflirtbook.com sideeffectsoftizanidine.blogspot.com
sexobzor.info site-speed-check.site
sexs-foto.com sluganarodu.ru
sexsearch.com smart-balancewheel.com
shijian.ac.cn snyatie-lomki-v-stacionare.ru
shikiso.info sockshare.net
shinikiev.com.ua softomix.net
site.ru songplanet.ru
sitevalued.com soundfrost.org
skladvaz.ru splendorsearch-a.akamaihd.net
skyway24.ru stackthatbucks.com
smartadserver.com stopnarco.ru
softlinesolutions.me stroiminsk.org
souvenirua.com stroymonolit.su
spb.ru students-cheapskate.ml
spravka-medosmotr.ru studiofmp.com
statistici.ro syndicate.fun
steame.ru t-bygg.com
stillmiracle.com tiens2010.ru
swingerseiten.com tizanidinehcl4mgtablet.blogspot.com
tech4master.com tizanidineonline.blogspot.com
telefonsexsofort.tk tk-assortiment.ru
testbotprocessor44.com tmearegion26.com
textads.men tooplay.com
tgtclick.com topcasinoratings.ru
thecoolimages.net toys.erolove.in
thedownloadfreeonlinegames.blogspot.com tracksz.co
thegioixekhach.com trahic.ru
thelottosecrets.com treasuretrack-a.akamaihd.net
thfox.com trubywriting.com
thiegs.reco.ws um-razum.ru
tizanidine4mgstreetvalue.blogspot.com unlimitdocs.net
tizanidineandcipro.blogspot.com uprour.com
top-deal.com.pl uytmaster.ru
toproadrunner5.info veerotech.com
triumf-realty.ru vgoloveboli.net
tt-ipd.info video-woman.com
tubeoffline.com vip-parfumeria.ru
turvgori.ru
udsgame.online
ukrtvir.com.ua
ultimatesetnewfreeallsoftupgradesystems.pw
usiad.net
v-doc.co
valoresito.com
videositename.com
vids18.site
vksex.ru
vladimir.xrus.org
voloo.ru
vseigru.one vseigru.one
vykupavto-krasnodar.ru works.if.ua
wapsite.me wovis.site
watch-movies.ru www.jbetting.com
webadvance.club xmronta.com
websiteaccountant.de xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
whos.amung.us xn----7sbifcamovvfggw9d.xn--p1ai
woman-orgasm.ru xn----9sbubg3ambdfl1j.xn--p1ai
xn----7sbaphztdjeboffeiof6c.xn--p1ai xolodremont.ru
xn----btbdvdh4aafrfciljm6k.xn--p1ai yellowproxy.net
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai zaloadi.ru
xn--80aikhbrhr.net zapatosenventa.info
xtubeporno.net zastroyka.org
xxart.ru zazagames.org
xxxasianporn.net zigarettenonl.canalblog.com
yellowfootprints.com
youradulthosting.com
zhacker.net

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.86 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2837 ### Version: V4.2022.01.2838
### Updated: Fri Jan 7 22:01:33 UTC 2022 ### Updated: Sat Jan 8 10:03:26 UTC 2022
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2837 ### Version: V4.2022.01.2838
### Updated: Fri Jan 7 22:01:33 UTC 2022 ### Updated: Sat Jan 8 10:03:26 UTC 2022
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##