vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2725 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-19 22:07:07 +00:00
parent 958b61eca7
commit 63fc69ed62
47 changed files with 61325 additions and 61335 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2724 #### Version: V4.2021.11.2725
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2724 #### Version: V4.2021.11.2725
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2724 #### Version: V4.2021.11.2725
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7532
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2724 #### Version: V4.2021.11.2725
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7536
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7536
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

7536
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7536
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 1; 10.1.0.248 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 0; 10.1.0.248 0;
127.0.0.1 0; 127.0.0.1 0;

7536
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 1; 10.1.0.248 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 0; 10.1.0.248 0;
127.0.0.1 0; 127.0.0.1 0;

7536
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 1; 10.1.0.248 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 0; 10.1.0.248 0;
127.0.0.1 0; 127.0.0.1 0;

7536
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2724 2725

7536
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

7536
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
51.143.41.154 0; 20.115.165.187 0;

7536
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 1; 10.1.0.248 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 0; 10.1.0.248 0;
127.0.0.1 0; 127.0.0.1 0;

7536
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 1; 10.1.0.248 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 0; 10.1.0.248 0;
127.0.0.1 0; 127.0.0.1 0;

7536
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 1; 10.1.0.248 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 0; 10.1.0.248 0;
127.0.0.1 0; 127.0.0.1 0;

7536
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
51.143.41.154 0; 20.115.165.187 0;

7536
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
51.143.41.154 0; 20.115.165.187 0;

7536
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 1; 10.1.0.248 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Brandprotect 404enemy
Disco Dispatch
Download Wonder Magnet
Fyrebot ProPowerBot
Nessus SEOlyticsCrawler
Octopus ScrepyBot
PxBroker SemrushBot
SalesIntelligent SurveyBot
Web Sucker TwengaBot
tAkeOut Xaldon WebSpider

290
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider
404enemy
ADmantX ADmantX
AIBOT AIBOT
ALittle Client
Abonti Abonti
Aboundex Acunetix
Aboundexbot AhrefsBot
Aipbot AiHitBot
Alligator
AlphaBot AlphaBot
Anarchy Anarchy99
Ankit
Apexoo Apexoo
Asterias Aspiegel
Attach AwarioSmartBot
BDCbot
BDFetch BDFetch
BLEXBot Backlink-Ceck
BackStreet
BacklinkCrawler BacklinkCrawler
Battleztar Bazinga Badass
Bandit
BatchFTP
Bigfoot Bigfoot
Bitacle
Black Hole
BlackWidow BlackWidow
Blackboard
Boardreader Boardreader
Bolt Bolt
Brandprotect BotALot
Buck BuiltWith
BunnySlippers
BuzzSumo BuzzSumo
CODE87 CATExplorador
CCBot
Calculon Calculon
CensysInspect CazoodleBot
ChinaClaw CheTeam
Chlooe Chlooe
Claritybot
Cogentbot Cogentbot
Copier
CopyRightCheck
Copyscape
Cosmos Cosmos
CrazyWebCrawler Crawling at Home Project
Crescent
Curious
DBLBot DBLBot
DIIbot DIIbot
DSearch DSearch
DTS Agent DTS Agent
DataCha0s Demon
Disco Devil
DomainStatsBot Digincore
Dotbot DigitalPebble
Download Wonder Dirbuster
Discoverybot
Dispatch
DittoSpyder
DnBCrawler-Analytics
DomainAppender
Domains Project
Dragonfly
Drip Drip
ECCP/1.0 ECCP/1.0
Ebingbong Exabot
Ecxi Express WebPictures
EirGrabber
EroCrawler
ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms FDM
FHscan FHscan
FemtosearchBot Fimap
Firefox/7.0
FlashGet
Flunky Flunky
Foobot
FrontPage FrontPage
GermCrawler Fuzz
GetRight Fyrebot
GT::WWW
Genieo
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla
GrabNet GrabNet
GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
HMView
HTTP::Lite
HTTrack HTTrack
Haansoft HaosouSpider
Heritrix Havij
Hloader IDBTE4M
Humanlinks
HybridBot
IRLbot
Iblog Iblog
Id-search
IlseBot IlseBot
InfoNaviRobot InfoNaviRobot
InterGET
Internet Ninja Internet Ninja
IstellaBot InternetSeer
Iria
JOC Web Spider JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
JetCar
Jetty
JikeSpider
Joomla Joomla
Jorgee Jorgee
JustView
Keyword Density Keyword Density
Kozmosbot Kozmosbot
LNSpiderguy LWP::Simple
Lanshanbot
Larbin Larbin
LeechFTP
LeechGet LeechGet
Libwhisker Libwhisker
LieBaoFast
Likse Likse
LinkScan LinkWalker
Linkbot
Linkdexbot Linkdexbot
LinkextractorPro Lipperhey
LinkpadBot Lmspider
LinksManager
LinqiaRSSBot
MFC_Tear_Sample
MIDown tool
MQQBrowser MQQBrowser
MSFrontPage
MSIECrawler
Mag-Net Mag-Net
Magnet
Mail.RU_Bot
Majestic SEO Majestic SEO
MarkMonitor Majestic-SEO
Majestic12
Masscan
Mata Hari Mata Hari
Mb2345Browser MegaIndex.ru
MeanPath Bot
Mediatoolkitbot
Metauri
MicroMessenger MicroMessenger
Microsoft Data Access Microsoft URL Control
Mister PiX
Mojeek Mojeek
Mr.4x3 Mr.4x3
NetLyzer NearSite
NextGenSearchBot Needle
Nessus
NetMechanic
NetSpider
NetZIP
Nibbler Nibbler
Niki-bot Niki-bot
Nikto
Ninja Ninja
Nmap Nmap
Nuclei
Nutch Nutch
OnCrawl Octopus
Offline Navigator
OpenLinkProfiler OpenLinkProfiler
OutfoxBot OrangeBot
PECL::HTTP PECL::HTTP
Page Analyzer Page Analyzer
PageScorer PageGrabber
PageThing.com
Pandalytics Pandalytics
Papa Foto
PeoplePal PeoplePal
Pi-Monster Picsearch
PictureFinder
Piepmatz
Pimonster Pimonster
Pixray
PleaseCrawl
ProPowerBot ProPowerBot
Probethenet ProWebWalker
Psbot
Pu_iN Pu_iN
PyCurl Pump
Quick-Crawler RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankFlex RankingBot
RankingBot2 RankingBot2
Re-re Rankivabot
RankurBot
ReGet ReGet
RealDownload Reaper
RedesScrapy RedesScrapy
RepoMonkey RepoMonkey
Ripper Ripper
SBIder
SEOkicks SEOkicks
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler
SMTBot
SalesIntelligent SalesIntelligent
ScanAlert
ScoutJet ScoutJet
Scrapy Scrapy
ScreenerBot
ScrepyBot ScrepyBot
SearchmetricsBot SemanticJuice
SentiBot SentiBot
SeobilityBot SeoSiteCheckup
Siphon
SiteCheckerBotCrawler
SiteSucker SiteSucker
Sitebeam
Sitevigil Sitevigil
SlySearch Snake
SocialRankIOBot Snoopy
Sottopop Sosospider
SpaceBison
Spbot Spbot
Spinn3r Sqlworm
Sqworm SuperHTTP
Steeler
Sucuri
Surfbot
Suzuran Suzuran
Szukacz T0PHackTeam
Thumbor T8Abot
TightTwatBot TheNomad
Toweyabot Tracemyfile
Trendiction Trendiction
Trendictionbot True_Robot
TwengaBot TwengaBot
Twice
Typhoeus
URLy Warning URLy Warning
URLy.Warning
UnisterBot
Upflow Upflow
VB Project
Vacuum
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
Virusdie Virusdie
Voil Voil
WBSearchBot Voltron
WISENutbot
WWW-Collector-E WWW-Collector-E
WallpapersHD WWW-Mechanize
WWWOFFLE
WeSEE
Web Collage Web Collage
Web Enhancer WebAuto
Web Fetch WebCopier
WebBandit WebFetch
WebCollage WebGo IS
WebFuck WebPix
WebStripper WebReaper
WebSauger
WebSucker
WebWhacker
WebmasterWorldForumBot WebmasterWorldForumBot
WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster Whacker
Whack
Whatweb Whatweb
Widow
Wonderbot Wonderbot
Wprecon Xaldon_WebSpider
Xaldon WebSpider
Zeus
Zitebot Zitebot
ZmEu ZoominfoBot
backlink-check ZumBot
coccocbot-web arquivo-web-crawler
autoemailspider
cah.io.community
check1.exe
com.plumanalytics com.plumanalytics
dataforseo.com
demandbase-bot demandbase-bot
domainsproject.org eCatch
evc-batch
facebookscraper facebookscraper
instabid instabid
lwp-trivial internetVista monitor
meanpathbot lwp-request
mediawords magpie-crawler
muhstik-scan muhstik-scan
netEstate NE Crawler plumanalytics
probe-image-size seoscanners
scalaj-http serpstatbot
scan.lol sexsearcher
seocompany.store
seostar
sitechecker.pro sitechecker.pro
sp_auditbot trendiction.com
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webpros.com webpros.com
webprosbot xpymep1.exe
x09Mozilla
x22Mozilla
zgrab

300
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
404checker 404checker
ADmantX 80legs
AIBOT
ALittle Client ALittle Client
Abonti
Aboundex
Aboundexbot Aboundexbot
Acunetix AfD-Verbotsverfahren
Alexibot AhrefsBot
Alligator
AlphaBot AlphaBot
Anarchie
Anarchy
Anarchy99 Anarchy99
Ankit Ankit
Apexoo Aspiegel
Asterias Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot BDFetch
BLEXBot
BackDoorBot
BackStreet BackStreet
Battleztar Bazinga BacklinkCrawler
Bigfoot Bandit
Black Hole
BlackWidow
Blackboard Blackboard
Bolt BlowFish
Buddy Boardreader
Bullseye BotALot
Brandwatch
BuiltWith
BunnySlippers
BuzzSumo BuzzSumo
CATExplorador CATExplorador
CCBot
CODE87 CODE87
Calculon CSHttp
Cegbfeieh CazoodleBot
Copier CensysInspect
Craftbot CheTeam
CheeseBot
Collector
Cosmos
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Crescent
CrunchBot CrunchBot
Curious Curious
Custo CyotekWebCopy
DataCha0s DIIbot
DTS Agent
DatabaseDriverMysqli
Demon Demon
Discobot Digincore
Discoverybot Dirbuster
Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics
DomCopBot DomCopBot
DomainAppender DomainStatsBot
DomainSigmaCrawler EMail Siphon
Dotbot
Download Wonder
ECCP/1.0
EMail Wolf
EasyDL EasyDL
Ebingbong Ebingbong
Ecxi Ecxi
Evil
Express WebPictures Express WebPictures
Extractor
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
Ezooms Freeuploader
FDM
Fimap
FrontPage FrontPage
Fuzz Fuzz
GalaxyBot GalaxyBot
Genieo Gigabot
Go!Zilla Go!Zilla
GoZilla GoZilla
Gotit Gotit
Grabber
Grafula
GrapeshotCrawler GrapeshotCrawler
GridBot HMView
HaosouSpider Haansoft
Harvest
Heritrix Heritrix
Hloader
HybridBot
IDBTE4M IDBTE4M
IRLbot
Id-search Id-search
IlseBot IlseBot
Image Sucker
InfoNaviRobot
Intelliseek
Internet Ninja
InternetSeer
JamesBOT JamesBOT
Jbrofuzz
JennyBot JennyBot
Jetty JetCar
JikeSpider Jorgee
JustView
Jyxobot
Kenjin Spider
Kozmosbot Kozmosbot
LNSpiderguy
Lanshanbot Lanshanbot
Larbin Leap
LeechFTP LeechFTP
LeechGet LexiBot
LieBaoFast LieBaoFast
Likse Likse
LinkextractorPro LinkScan
LinksManager LinkpadBot
LinqiaScrapeBot LinqiaRSSBot
Lipperhey
Lipperhey Spider Lipperhey Spider
MFC_Tear_Sample Lmspider
MIIxpc MIIxpc
MSFrontPage MJ12bot
MSIECrawler
MTRobot
Mag-Net Mag-Net
Mail.RU_Bot Magnet
Majestic SEO
Majestic-SEO Majestic-SEO
MarkMonitor Majestic12
MarkWatch
Mass Downloader Mass Downloader
Masscan Mb2345Browser
MegaIndex.ru MeanPath Bot
Metauri
Microsoft Data Access Microsoft Data Access
Mojeek Morfeus Fucking Scanner
Mojolicious
Mr.4x3 Mr.4x3
Msrabot
Musobot Musobot
NearSite NICErsPRO
NPbot
Nameprotect
Navroad
Needle Needle
Nessus
NetLyzer
NetMechanic NetMechanic
NetSpider NetSpider
Netcraft Netcraft
Nettrack Netvibes
NimbleCrawler Niki-bot
Nimbostratus Nimbostratus
Ninja
Nmap
Nuclei Nuclei
OnCrawl OnCrawl
OpenLinkProfiler OpenLinkProfiler
Openfind OpenVAS
PHPCrawl OrangeBot
OutclicksBot
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageThing.com PageGrabber
Pandalytics PageScorer
Petalbot Pimonster
Picsearch Pixray
PictureFinder
PleaseCrawl
Pockey
ProPowerBot ProPowerBot
ProWebWalker ProWebWalker
Psbot Psbot
PxBroker
PyCurl PyCurl
QueryN Metasearch
RSSingBot RSSingBot
Rankivabot RankingBot2
RankurBot
Re-re Re-re
ReGet
RealDownload
RebelMouse
Ripper
RocketCrawler RocketCrawler
Rogerbot
SBIder SBIder
SEOkicks-Robot SEOkicks
SEOlyticsCrawler SEOlyticsCrawler
SalesIntelligent SEOprofiler
Scanbot SEOstats
SISTRIX
SMTBot
ScoutJet ScoutJet
Scrapy ScrepyBot
Screaming
Searchestate
SearchmetricsBot SearchmetricsBot
Semrush SemrushBot
SentiBot
SeoSiteCheckup SeoSiteCheckup
Siphon
SiteLockSpider
SiteSnagger
SiteSucker
Sitebeam Sitebeam
SlySearch Sitevigil
Sosospider SmartDownload
SpankBot Snake
Spinn3r Snapbot
Sqlmap Snoopy
Sociscraper
Sogou web spider
Sottopop
SpaceBison
Spammen
Spanner
SputnikBot
Stripper Stripper
Surfbot Sucker
Swiftbot Suzuran
Szukacz T0PHackTeam
TeleportPro
Telesoft Telesoft
Telesphorep Telesphorep
Thumbor The Intraformant
TightTwatBot Toata
Titan Tracemyfile
Toweyabot Turingos
Turnitin TurnitinBot
Twice Twice
Upflow URLy.Warning
V-BOT V-BOT
VB Project
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
Virusdie Virusdie
VoidEYE Voil
WEBDAV WASALive-Bot
WISENutbot WPScan
WWWOFFLE
Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE
Web Auto
Web Collage Web Collage
Web Enhancer Web Enhancer
Web Fuck Web Fuck
Web Pix
WebBandit
WebCollage WebCollage
WebFuck WebPix
WebGo IS
WebSauger
WebStripper
WebSucker WebSucker
Website Quester WebsiteQuester
Webster Webster
Widow Whack
WinHTTrack Whacker
Wprecon Whatweb
Who.is Bot
Wonderbot
Wotbox
Xaldon WebSpider Xaldon WebSpider
YoudaoBot Xaldon_WebSpider
Zade Zeus
ZyBorg ZyBorg
arquivo.pt autoemailspider
cah.io.community crawler.feedback
coccocbot-web evc-batch
com.plumanalytics
crawler4j
eCatch
facebookscraper facebookscraper
gopher gopher
heritrix instabid
isitwp.com magpie-crawler
lwp-request mediawords
lwp-trivial
meanpathbot
muhstik-scan muhstik-scan
netEstate NE Crawler netEstate NE Crawler
pcBrowser pcBrowser
plumanalytics plumanalytics
polaris version ripz
s1z.ru s1z.ru
scan.lol
seocompany.store seocompany.store
seoscanners
seostar seostar
serpstatbot trendiction.com
siteripz trendiction.de
sogouspider
spyfu
sysscan
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com voyagerx.com
webpros.com webmeup-crawler
x09Mozilla webprosbot
x22Mozilla
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
advocatemsk.ru 00it.com
ahrefs.com apiqualitinknet-a.akamaihd.net
bizru.info aviva-limoux.com
cialisndbrx.com elektrischezigarette1.onsugar.com
dptaughtme.com lennatin.info
gateway.zscalerone.net lodki-pvh.dp.ua
sildenafil-tadalafil.info magicalfind-a.akamaihd.net
softtor.com michaelkorssaleoutletonline.net
voltrknc1.com mostantikor.ru
xn----ctbbcjd3dbsehgi.xn--p1ai summerlinhomes411.info

928
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0daymusic.org 1001watch.com.ua
101billion.com 123kuma.com
12masterov.com 23kw.ru
1bet.com 3dracergames.com
1j7740kd.website 404.mn
1millionusd.xyz 4kplayer.pl
2daytrendingnews.com 4pp13.com
3-letter-domains.net 5ws.dating-app.ru
45en.ru 72-news.com
4kepics.com 8jn.dating-app.ru
5000-cotydzien.com aa8b68101d388c446389283820863176e7.com
7a2rnuey1tw9ar.ru aa9bd78f328a6a41279d0fad0a88df1901.com
888.com abwa.tk
8kisses.eu accessoires-mode.in
991mostfm.co.id ad-words.ru
a.pr-cy.ru adsloads.com
aa08daf7e13b6345e09e92f771507fa5f4.com advancedmassagebysara.com
aac500b7a15b2646968f6bd8c6305869d7.com afmuseum.com
abctoppictures.net airlandsea.info
adamoads.com aliexpresscashback.club
adexprts.com all-number.com
adrunnr.com amt-k.ru
ads.gold apartmentbay.ru
adserver-e7.com apiholdingmypage-a.akamaihd.net
adtech.us apimegabrowsebiz-a.akamaihd.net
adultgalls.com arabsexxxtube.com
akusherok.ru architecturebest.com
alldezservice.kz arcteryxstore.online
allproblog.com arewater.com
alphacarolinas.com asiengirls.net
amanda-porn.ga
amoi.tn
an-donut.com
apioasisspacenet-a.akamaihd.net
apioutoboxnet-a.akamaihd.net
apiwoodensealcom-a.akamaihd.net
arenda-yeisk.ru
artdestshop.eu
asdfz.pro
asrv-a.akamoihd.net asrv-a.akamoihd.net
autoloans.com asseenontvonline.ru
avkzarabotok.info ausergrubhof.info
avtoarenda.by avtoarenda.by
backuperwebcam.weebly.com avtolombard-krasnodar.com
bannerconnect.net avtolombard-krasnodar.ru
bazaronline24.ru azbaseimages.net
beachtoday.ru babs.com.ua
beremenyashka.com bagcionderlab.com
bestprofits-there.com bbtec.net
bestwebsiteawards.com bear.gotcher.us
bestwrinklecreamnow.com bedandbreakfast.com
bezsporno.ru belinka.com.ua
bgviagrachrx.com benchmarkcommunications.co.uk
bidbuy.co.kr bestmobilityscooterstoday.com
biographiya.com betteroffers.review
bigpictures.club
bio-japan.net
bitcoin-ua.top bitcoin-ua.top
biznesluxe.ru bkgr.se
blue-square.biz bonzbuddy.com
bmwhighperformers.com bosman.pluto.ro
bobinoz.com bpro1.top
boostmyppc.com brainzpod.win
brainxs.us brandbucket.com
brakehawk.com brandehk.dk
brandedleadgeneration.com brimstonehillfortress.org
bravegirlsclub.com bristolhotel.com.ua
brokergid.ru bronzeaid-a.akamaihd.net
buqyxa.rincian.info bryansk.zrus.org
buttons-for-your-website.com btc4u.ru
buybest1.biz burger-tycoon.com
buyessay3.blogspot.ru burn-fat.ga
candiceloves.us businesn.men
candlewooddentalcentre.com.au business-suggested.tk
buy-meds24.com
buyantiviralwp.com
canacopegdl.com
candytiner.com candytiner.com
canoncdriverq3.pen.io capacitacionyaprendizaje.com
cardsharp1.ru cardul.ru
cars-modification.net carrauterie.be
cartechnic.ru casablancamanor.co.za
cartierring.xyz cb.iphantom.com
cayado.snn.gr cbozhe.com
cejewelry.xyz
centraletermice.us
centre-indigo.org.ua
centrumcoachingu.com centrumcoachingu.com
chatseo.com cheapestjerseysonwholesale.com
cheapestjerseys-wholesale.com check-host.net
chocolateslim-en-france.com chocolateslim-u-srbiji.com
choice-credit.ru choice-credit.ru
cialischsrx.com christianlouboutinsaleoutletonline.us
cialischstgerts.com christianlouboutinshoes.xyz
cialisndbrx.com civilwartheater.com
cigarpass.com cjmc.info
cinemaflix.website clayvasedesigns.tk
citysecurity.nu codq.info
cl.s7.exct.net companies-catalog.com
clarithromycin500mg.com compliance-fred.top
clean-virus-mac.com compliance-irvin.top
clickso.com compliance-ivan.xyz
coderstate.com conocer-sanabria.com
commoncrawl.org coolgamechannel.com
compliance-donald.xyz coop-gamers.ru
confib.ifmo.ru crackguru.tk
cookielawblog.wordpress.com cranly.net
coolyarddecorations.com crd.clan.su
copypaste.traderzplanet.in customcollegeessays.net
cordstrap.cc customerpromos-a.akamaihd.net
cougfan.info cwetochki.ru
countercrazy.com dafatiri.com
crackzplanet.com dalavia.ru
craftinsta.ru damedingel.ya.ru
credit-online.ws dandiyabeats.in
crystalslot.com daneshetabiat.com
customerguru.in davidovic.info
czat.wp.pl dealighted.com
dame-ns.kz dekoration.us
darkages.info delayreferat.ru
data-mining.tk den-noch24.ru
de.zapmeta.com dentfidemountpreach.tk
design-lands.ru detskie-konstruktory.ru
dikx.gdn dev78.dioniqlabb.se
directrev.com digilander.libero.it
djstools.com dildofotzen.net
dnepropetrovsk.mistr-x.org dipstar.org
dominterior.org dirtpics.pw
dostavka-v-krym.com dokfilms.net
downloads-whatsapp.com doktoronline.no
doxysexy.com dollartree.info
doyouknowtheword-flummox.ml download-walpaperhd.blogspot.com
dptaughtme.com
dverimegapolis.ru dverimegapolis.ru
dynainbox.com ebooknovel.club
e2click.com efkt.jp
easytuningshop.ru election.interferencer.ru
ednorxmed.com
eduinfosite.com
eleimgo.pw eleimgo.pw
elektrischezigaretten1.blogse.nl elektrischezigarette1.blog.pl
elektrozigaretten2.yn.lt elektroniksigaraankara.info
elitedollars.com elitepcgames.com
eloconcream.blogspot.com elstal.com.pl
emailaccountlogin.co eshop.md
emediate.eu essayassist.com
esnm.ru etm-consult.de
evening-dating-club.info eupornstar.info
excaliburfilms.com evogarage.com
exoneration-project.us exportshop.us
exonline.info fashion-stickers.ru
failingmarriege.blogspot.com fashionavenuegame.com
fakehandbags.xyz fertilitetsradgivningen.se
fanrto.com filmi-v.online
farm26.ru filmidivx.com
fast-wordpress-start.com findpik.com
faz99.com floating-share-buttons.com
fickkontaktehobbyhuren.com flowwwers.com
fickluder69.com forexunion.net
fikasound.tk formulaf1results.blogspot.com
filesclub.net fortunejack.com
finteks.ru fotosfotos.eu
flash4fun.com foxydeal.com
forms-mtm.ru frameimage.org
foxinsocks.ru frbizlist.com
foxtechfpv.com free-share-buttons.top
free-deals.faith freefoto.ca
free-floating-buttons.com freemaintenancesysforpcandmac.top
fun-mobi.pl fuck-paid-share-buttons.xyz
fx-brokers-review.com fuckbuddybestgilf.info
gabeshop.ru fun2cell.net
gagrasector.ru fyl.com.ru
galaxys6manual.info g.starmoe.xyz
gambarkatabaru.com gamebackyard.com
gaming-journal.com gamerscorps.com
gateway.zscalertwo.net getlamborghini.ga
gay-site.store ghazel.ru
gaytube.com ghostvisitor.com
gazporno.com gktt.ru
gentamicineyedrops.blogspot.com gojiberriess.apishops.ru
gifspics.com gonextmedia.com
gigapeta.com goodhousekeeping.com
gloverid.site gopro-online.info
gobongo.info greatgrace.ru
gogps.me green-tea.tv
goldadpremium.com greenzaim.ru
googst2.ru
gosarhivrt.ru
gotwebsite1.com
gsbs.com.ua gsbs.com.ua
guiadeserraazul.com gungamesz.com
guigyverpo.cf
gwebtools.com.br gwebtools.com.br
hackers-crackers.tk hazardky.net
hamilton.ca
handicapvansarea.com
hdfreeporno.net
helvetia.com.ua helvetia.com.ua
hermesbirkinhandbagoutlets.com herehloadibs.cf
historichometeam.com hifidesign.ru
hitcpm.com highland-homes.com
holidaypics.org himazin.info
hotel-mkad.ru histock.info
hotelcrocenzi.sm homedecoguide.info
hothor.se hop.clickbank.net
hreade.com hotchatdate.com
humanelydrew.com hotkeys.com
humbmosquina.tk hotloans.ru
hoverboard360.de
hoverboard360.es
hubbble.com
huimin764128.com
idegenvezeto.eu
ido3.com
ilovevitaly.ru ilovevitaly.ru
imagui.eu imadedinner.net
imfamous.info imagefinder.site
img.wallpaperstock.net images.gyffu.com
imguramx.pw incanto.in.ua
imk.com.ua indiakino.net
imperia31.ru infektsii.com
impisr.edunsk.ru investingclub.ru
impresagaia.it invitefashion.com
indianmedicaltourismshop.com iomoio.net
infobabki.ru iradiology.ru
infokonkurs.ru it-max.com.ua
infostatsvc.com karusel-market.ru
infowarcraft.ru ketrzyn.pl
interferencer.ru keywesthideaways.co
iphantom.com keywordteam.net
iplogger.org
iqoption.pro
irkutsk.online-podarki.com
irunfar.com
itservicesthatworkforyou.com
ix20.ru
jackpotchances.com
janerikholst.se
japan-bearings.ru
jillepille.com
jimmychoosale.online
jongose.ninja
junketjuice.blogspot.com
jus0wil.pp.ua
kabbalah-red-bracelets.com
kalb.ru
kambasoft.com
kinoflux.net kinoflux.net
kochanelli.com kirov.zrus.org
kiskinhouse.com
kongruan.com
koptims.tiu.ru koptims.tiu.ru
koronirealestate.gr kountrylife.com
kotaku.com ku6.com
krafte.ru kutikomi.net
kvartiry-remont.ucoz.ru lacave.ntic.fr
lacasamorett.com laexotic.com
lalalove.ru lafourmiliaire.com
landinez.co lafriore.ru
landscapebackgrounds.blogspot.com lampokrat.ws
laptoper.net lapitec.eu
letto.by lawyers.cafe
life-instyle.com laxdrills.com
likesdesign.com legionalpha.com
liveshoppersmac.com lemon-ade.site
lizace.com lflash.ru
linkbux.ru
lion.playtap.us
lionking-1994.blogspot.com
live-cam6.info
livetsomudvekslingstudent.bloggersdelight.dk
loadopia.com loadopia.com
lollypopgaming.com lonely-mature.com
lonerangergames.com louisvuittonoutletstore.net
lookingglassemb.com lowephotos.info
macfix.life macdamaged.space
magda-gadalka.ru macrotek.ru
magiadinamibia.blogspot.com mafcards.ru
mansparskats.com mail.allnews24.in
make-money-online.com
marcogrup.com
marktforschung-stuttgart.com marktforschung-stuttgart.com
masterseek.com maxxtor.eu
mcar.in.ua mb140.ru
meccadumps.net mebel-alait.ru
medi-fitt.hu meduza-consult.ru
meget.co.za meget.co.za
metarip.ru metallosajding.ru
metaxalonevstizanidine.blogspot.com metascephe.com
michellblog.online mexicotravelnet.com
mikozstop.com middlerush-a.akamaihd.net
minet.club mirmedinfo.ru
mixed-wrestling.ru moesexy.com
mmofreegames.online monsterdivx.tv
mmog-play.ru morepoweronmac.trade
mncrftpcs.com moscow.xrus.org
mnogabukaff.net moyaterapiya.ru
mobile.ok.ru musicstock.me
moesen-ficken.com mwtpludn.review
monarhs.info mydoctorok.ru
monclerboots.xyz najaden.no
monclercheap.xyz narkologiya-penza.ru
moneyteam24.com
morpicert.pw
moscow-clining.ru
moskva.nodup.ru
moz.com
mrbitsandbytes.com
muizre.ru
muz-shoes.ru
myliveblog.ru
myonlinepayday.co
nagdak.ru
narkologiya-peterburg.ru narkologiya-peterburg.ru
nastroyke.net nash-krym.info
needtosellmyhousefast.com net-radar.com
neks.info netanalytics.xyz
networkad.net newstaffadsshop.club
new-post.tk nfljerseys.online
next-dentists.tk nibbler.silktide.com
nicovideo.jp no-fuel.org
nippon-bearings.ru
nodup.ru nodup.ru
nordvpn.com normalegal.ru
northfacestore.online noscrapleftbehind.co
notfastfood.ru notasprensa.info
novodigs.com novosibirsk.xrus.org
ntic.fr npoet.ru
nudejapan.net nuit-artisanale.com
nyfinance.ml
o-dachnik.ru
o-o-6-o-o.com
o00.in
obiavo.kz
odywpjtw.bloger.index.hr
offf.info
oivcvx.website
olvanto.ru olvanto.ru
openfrost.net onclickpredictiv.com
optom-deshevo.ru onefilms.net
orion-v.com onemactrckr.com
page2rss.com oneshotdate.com
paighambot.com online-podarki.com
openstat.com
oz-offers.com
ozoz.it
painting-planet.com
pateaswing.com
patterntrader-en.com patterntrader-en.com
pawli.eu pc-services.ru
pcads.ru peugeot-club.org
pestomou.info pharmacyincity.com
phuketscreen.com photo-clip.ru
picsforkeywordsuggestion.com picsearch.com
picturesfrom.com picturesify.com
picurams.pw pierrehardysale.online
pintattoos.com pijoto.net
pinwallpaper.top piluli.info
pistonclasico.com plastgranar.nu
pixell.club platinumdeals.gr
pizza-imperia.com playtap.us
play.leadzupc.com pokerniydom.ru
pons-presse.com polcin.de
poptool.net porndl.org
pornohd1080.online pornhubforum.tk
pornoinn.com porno-transsexuals.ru
pornosok.ru pornofeuer.com
pornowarp.info pornokorol.com
potolokelekor.ru pornstartits.xblog.in
pp-budpostach.com.ua portnoff.od.ua
privat-girl.net pr-ten.de
proekt-gaz.ru pr0fit-b0x.com
profitfx.online preventheadacheguide.info
progress-upakovka.ru privacyassistant.net
prokotov.com pro-tec.kz
prombudpostach.com.ua
propranolol40mg.blogspot.com
proprostatit.com proprostatit.com
prosmibank.ru prostitutki-kharkova.org
pussyspace.net purplestats.com
qualitymarketzone.com push-ad.com
queerspace.com pyrodesigns.com.au
quit-smoking.ga q-moto.ru
r-e-f-e-r-e-r.com qcstrtvt.bloger.index.hr
randki-sex.com quebec-bin.com
quick-offer.com
raavidesigns.com
rada.ru
rangjued.com rangjued.com
rczhan.com rangoman.date
rcb101.ru
rebuildermedical.com
recinziireale.com recinziireale.com
reelheroes.net reeyanaturopathy.com
regdefense.com refudiatethissarah.info
remont-comp-pomosh.ru regionshop.biz
remont-fridge-tv.ru registratciya-v-moskve.ru
remontgruzovik.ru remontgruzovik.ru
research.ifmo.ru renewablewealth.com
rezeptiblud.ru rniaeba.ga
riralmolamsaca.tk rollercoin.com
royalcar-ufa.ru rome2rio.com
runofilms.ru roofers.org.uk
rusenvironmental.net rootandroid.org
rutor.vip rossanasaavedra.net
s1z.ru royal-betting.net
sabaapress.com rucrypt.com
sady-urala.ru running-line.ru
samo-soznanie.ru russian-postindex.ru
samolet.fr s-forum.biz
sbprabooks.com s-iwantyou.com
sbricur.com sack.net
scanner-fred.top satoristudio.net
scanner-josh.top saveriopiazza.it
scrapinghub.com savetubevideo.com
scripted.com scanner-brian.top
search.pch.com scanner-margo.top
searchimage.co scanner-willy.top
scat.porn
scenicmissouri.us
school-diplomat.ru
sdelai-prosto.ru
sdsjweb.com
searchmywindow-a.akamaihd.net
sell-fb-group-here.com sell-fb-group-here.com
seo-smm.kz seogadget.ru
seozoom.it seoholding.com
servethis.com seomarketings.online
sexcamamateurchat.com sergiorossistore.online
sexkrasivo.net servingnotice.com
sextracker.be sex-znakomstva.online
sexxdate.net sextracker.com
sexystrippe.info sexyali.com
sharebutton.net sfd-chess.ru
sharebutton.org sharebutton.to
shinikiev.com.ua shariki-zuma-lines.ru
shitmovs.com
shivafurnishings.com
shoppytoolmac.com
shtaketniki.kz
shtora66.ru shtora66.ru
sisiynas.ru siteaero.com
site-speed-check.site siteheart.net
skincrate.net siteripz.net
skylta.com skanninge.se
skypasss.com slavkokacunko.de
sledstvie-veli.net
social-fun.ru
social-search.me social-search.me
socialsignals24.com solnplast.ru
soserfis.com sonata-arctica.wz.cz
sovinsteel.ru spb.afora.ru
space2019.top spbchampionat.ru
speeddream.xyz spitfiremusic.com
sports-supplements.us sprttrack.com
sr-rekneskap.no srgwebmail.nl
sribno.net statoutlook.info
stackthatbucks.com steame.ru
start.myplaycity.com
stauga.altervista.org
store-rx.com store-rx.com
stroicol.net stpicks.com
success-seo.com streetfooduncovered.com
sunflowerdrawingpaintings.blogspot.com streha-metalko.si
supermesta.ru
supernew.org supernew.org
taaaak.com superoboi.com.ua
tacbibirfa.tk superstats.com
taximytishi.ru surcentro.com
telegraf.by tackletarts.co
terraclicks.com tampabaywatch.org
teslathemes.com taqywu51.soup.io
techart24.com
telefonsexsofort.tk
terrafootwear.us
thaismartloan.com thaismartloan.com
the-trader.net thegamerznetwork.com
thebestweightlosspills.ovh thesmartsearch.net
thecoolimages.net thexart.club
thedownloadfromwarez.blogspot.com time-japan.ru
thegameriders.com tizanidine4mguses.blogspot.com
thepantonpractice.co.uk tizanidinefibromyalgia.blogspot.com
tilido.com tizanidineorflexeril.blogspot.com
tizanidinehydrochloride2mg.blogspot.com tobeyouday.win
tizanidinenarcotic.blogspot.com
tnctrx.com
tonerbox.kz tonerbox.kz
top10-online-games.com topcasinoratings.ru
topmira.com torontoplumbinggroup.com
torrnada.ru tour-line.net
totu.us toy-shop.top
tourcroatia.co.uk tradedeals.biz
track.deriv.com trafaret74.ru
tracksurf.daooda.com trafficshaper.com
traffic2cash.org truebeauty.cc
trafficmania.com
traffictrade.life
trustaffs.com
tryrating.com
tsstcorpcddvdwshbbdriverfb.aircus.com tsstcorpcddvdwshbbdriverfb.aircus.com
turizmus.us tula.howotorg.ru
twittrading.com twittruth.com
typer.one typimga.pw
ua.tc unionmarkt.de
ukrup.com uroki.net
ultimatesetnewfreeallsoftupgradesystems.pw uvozdeckych.info
underthesite.com valueclick.com
unitexindia.com vchulkah.net
unitygame3d.com venta-prom.ru
unmaroll.ya.ru vereo.eu
uptimechecker.com video-hollywood.ru
urlcut.ru videochat.ph
vegan-foods.us villakohlanta.nu
velen.io vip.51.la
vepad.com
vezuviy.su
videochat.cafe
vipms.ru
virtuagirl.com
virus-respirators.com virus-respirators.com
visa-pasport.ru vozbujdenie.com
voloomoney.com
vpnmouse.com
vse-pesni.com
vtcdns.com
vucms.com
vzlomsn.org
watsonrealtycorp.com watsonrealtycorp.com
webshoppermac.com webkeyit.com
wesharepics.info website-audit.com.ua
westen-v.life weprik.ru
wieseversa.no wholinkstome.com
wikes.20fr.com wingsofrefuge.net
wildworld.site wma-x.com
windowssearch-exp.com woman-orgasm.ru
wowas31.ucoz.ru wormix-cheats.ru
wurr.voila.net wosik-dach.service-for-web.de
www.bookmaker-bets.com wpsecurity.website
wyniki-lista.pl wpthemedetector.co.uk
xn----dtbndd4ae7eub.top www.timer4web.com
xn--80aikhbrhr.net wwwadultcheck.com
x-diesel.org
x5market.ru
xaijo.com
xboxster.ru
xerox-douglas.cf
xjlottery.com
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
xn--80aanaardaperhcem4a6i.com
xn--b1addnj3cah.xn--p1ai
xn--e1aggki3c.xn--80adxhks
xolodremont.ru xolodremont.ru
xpresscare.ru xuki.us
xsion.net youresponsive.com
yaderenergy.ru yourothersite.com
yellocloud.be yoursearch.me
yogamatsexpert.com yoursite.com
yopoint.in youtubologia.it
zaim-pod-zalog-krasnodar.ru ypmuseum.ru
zarepta.com yuweng.info
zastenchivosti.net zarabotok--doma.ru
zastroyka.org zebradudka.com
zhongwenlink.com zeroredirect12.com
zigarettenonlinekaufen2.bloog.pl zigarettenonlinekaufen1.bloog.pl
zirondelli.it zoompegs.com
zoogdinsney.com
zoomovies.org
zophim.me
zscalerone.net

468
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
0c47f8422d3f.com 123cha.com
1001desert.com 123movies.love
273-fz.ru
3th.co.in 3th.co.in
5i2.net admitad.com
5ws.dating-app.ru all-streaming-media.com
999webdesign.com allesohnegirls.net
aa625d84f1587749c1ab011d6f269f7d64.com allnews.md
absolute-s.ru alveris.ru
adbetclickin.pink amazingpic.net
adtech.us andpolice.com
agapovdg.ru angkortours.vn
air-edem.ru anti-virus-removal.info
allforminecraft.ru aosexkontakte.net
animali.deagostinipassion.it approvedlocal.co.za
anniemation.com
apessay.com
api.stathat.com
apibrowsesmartne-a.akamaihd.net
arendakvartir.kz
artdeko.info
artel116.ru artel116.ru
atlant-auto.info aruplighting.com
aviav.ru.com asdfg.pro
azazaporn.com asophoto.com
backgroundpictures.net astrochicks.com
bankofthewext.com autobudpostach.club
bestmarriages.com autochoixspinelli.com
bioca.org autotop.com.ua
biteg.xyz autovideobroadcast.com
biznesschat.net azbaseimages.net
blog-hits.com azbukafree.com
bloggerads.net bangla-chat-uk-paralud.ga
blpmovies.com bannerbank.ru
boeuklad.com batanga.net
bonzybuddy.com bausparen.bz.it
brk-rti.ru best-games.today
brothers-smaller.ru bestssaker.com
browserprotecter.com betwinservice.com
buckspar.gq bio-market.kz
bjorkbacken.nu
bkns.vn
buqayy0.livejournal.com
businesn.men businesn.men
buyclomidonlaine.com callawaygolfoutlet.online
buyfriend.ru callawaygolfstore.online
canrole.com calstaterealty.us
carina-sy.de caveavins.fr
carsplate.com
cartujano-pre.de
cascadelink.org
celejihad.info celejihad.info
christianlouboutinsaleonline.us cenokos.ru
cialisndbrx.com changde.58.com
civilwartheater.com cheapmoncler.xyz
classicakuhni.ru citysecurity.nu
cleanmypc.com clean-virus-mac.com
com-secure.download compliance-irvin.top
compliance-checker.info compliance-irvin.xyz
compliance-olga.top compliance-olga.top
conboy.us corta.co
country-chic.ru cosmetic.donna7753191.ru
covadhosting.biz
crackguru.tk crackguru.tk
cymbaltaandtizanidine.blogspot.com cubook.supernew.org
dame-ns.kz darodar.com
davidovic.info ddlmega.net
demian.kz deanmoore.ie
dignitasdata.se delitime.info
dogbreedspicture.net demon-tweeks.com
ecxtracking.com deutschehobbyhuren.net
elitepcgames.com digitalfaq.com
eluxer.net dimaka.info
enews.tech discovertreasurenow.com
essaypro.com download-wallpaper.net
etotupo.ru druzhbany.ru
facebook-mobile.xyz dvrlists.com
farmprofi.net e-buyeasy.com
fettefrauen.net easytuningshop.ru
fickkontaktehobbyhuren.com efnor-ac.com
filesdatabase.net ege-russian.ru
finteks.ru elektrozigarette2.webs.com
fishtauto.ru energy-ua.com
flowwwers.com enskedesquashclub.se
franklinfire.co errorfixing.space
fym.com.ru eu-cookie-law.blogspot.com
gambarkataku.co eugenevaultstorage.com
getprismatic.com european-torches.ru
getyourimage.club eurosamodelki.ru
gg.zzyjxs.com ezigarettekaufen2.blox.pl
ggiaro.com fbdownloader.com
golaya.pw feargames.ru
gomusix.com fetlifeblog.com
gsasearchenginerankerhelp.com findanysex.com
gwebtools.com findpik.com
hasfun.com forum69.info
hdseriale.pl foto-sisek.porngalleries.top
healing-dysplasia.ru freegamesplay.online
hem.passagen.se freetangodownload.com
hermesbirkinhandbagoutlets.com froggytube.com
hostnow.men front.to
hotchatdate.com fuel-gas.com
houdom.net fun-mobi.pl
hoverboard360.nl gambarxkata.co
hybrid.ru gamerextra.com
iamsport.org getmiro.com
ilovevitaly.ru gidonline.one
imguramx.pw gourcy.altervista.org
inome.com.ua gpirate.com
interior-stickers.ru gsasearchengineranker.xyz
handicapbathtubarea.com
hd720kino.ru
headpharmacy.com
healthcarestore.info
hosting-tracker.com
hotsocialz.com
howlongdoestizanidinestayinyoursystem.blogspot.com
ideawheel.com
ie.57883.net
images.gyffu.com
incep.imagine-ex.co
incolors.club
indiakino.net
infodocsportal.com
intellego.info
intermesh.net
iphantom.com
irkutsk.online-podarki.com irkutsk.online-podarki.com
islandminingsupply.wordpress.com isitpaleo.info
itronics.ca isitwp.com
jetsli.de itis4you.com
johnrobertsoninc.com janavibekken.no
kareliatobacco.ru jav-way.site
karusel-market.ru jaxcube.info
kichenaid.ru jongose.ninja
kinohall.ru journeydownthescale.info
klin3952.ru karachev-city.ru
koptims.tiu.ru kchaxton.com
kursy-ege.ru keki.info
ladsblue.com ketoanhanoi.info
ladsup.com kotaku.com
landreferat.ru kvartira-sutochno.com
laptoper.net lexaprogeneric.link
ledpolice.ru linkdebrideur.xyz
legalrc.biz liverpool.gsofootball.com
limads.men lkbennettoutlet.online
lineavabit.it locationdesaison.com
linkbolic.com lostfilm-online.ru
linkbux.ru mac-shield.com
lol-smurfs.com max-p.men
lombia.com
lonfon.xyz
longgreen.info
lsp-awak-perikanan.com
manve.info
maximilitary.ru
members.ghanaweb.com members.ghanaweb.com
micasainvest.com mere.host.sk
mini.7zap.com metabar.ru
modabutik.ru midst.eu
mirmedinfo.ru
mojowhois.com mojowhois.com
monclercheap.xyz
moncleroutletonline.win
monetizer.com-01.site monetizer.com-01.site
moz.com mypets.by
mrwhite.biz nagdak.ru
musicpro.monster nakozhe.com
myphotopipe.com national-today-winning-winner.club
na-telefon.biz neks.info
net-radar.com neodownload.webcam
nmrk.ru nextconseil.com
o333o.com njpalletremoval.com
obiavo.uz no-rx.info
online-hit.info nudo.ca
online-templatestore.com o-o-6-o-o.com
ooo-gotovie.ru o.light.d0t.ru
optikremont.ru oda.as
paccohichetoti.ml onlainbesplatno.ru
paydayonlinecom.com onlinemeetingnow.com
picsearch.com picsfair.com
picurams.pw picturesfrom.com
piski.top pkr1hand.com
play-movie.pl
pobeiranie.pl
pokemon-go-play.online pokemon-go-play.online
pornhubforum.tk popander.mobi
pornoreino.com popugaychiki.com
powitania.pl porndl.org
primedice.com porno-chaman.info
prosmibank.ru pregnant.guru
ranksignals.com pretty-mart.com
reining.lovasszovetseg.hu printdirectforless.com
remmling.de pro-poly.ru
rewardpoll.com prostitutki-yaroslavlya.org
rff-cfal.info quangcaons.com
rfid-locker.co rankexperience.com
rrutw.com rebrand.ly
rubbed.us regionshop.biz
ruf777.com reward-survey.net
ruspdd.com runstocks.com
rustic-quiver.win rvtv.ru
ryetaw.com s.lollypopgaming.com
sa-rewards.co.za saletool.ru
sad-torg.com.ua saneitconsulting.com
sbf441.com sanyuprojects.com
screpy.com scanner-olga.top
search.pch.com scanner-willy.top
seccioncontrabajo.com search-goo.com
sexkontakteao.info secretscook.ru
sexvideo-sex.com seeresultshub-a.akamaihd.net
sexxdate.net service.adtech.fr
sexy-screen-savers.com sex-pr.net
sfd-chess.ru shiksabd.com
sharebutton.to shop.xz618.com
shareyards.com site-speed-checker.site
shopcheermakeup.info sitechecker.pro
site-speed-check.site skyway24.ru
sitebeam.net slimcdn.com
smstraf.ru sloopyjoes.com
soserfis.com smartadserver.com
speechfoodie.com soaksoak.ru
spylog.com sports-supplements.us
spywarenuker.com spywarenuker.com
starpages.net sr-rekneskap.no
stefanbakosab.se steamoff.net
strana-solnca.ru sterva.cc
studiokamyk.com.pl storehouse.ua
suchenindeutschland.com stroy-portal22.ru
superstats.com summerlinhomes411.info
sygraem.com susanholtphotography.com
tattooreligion.ru tdsing.ru
teenforporn.com techart24.com
telemetryverification.net telemetryverification.net
theheroes.ru template-kid.com
thejournal.ru theprofitsmaker.net
timdreby.com timetorelax.biz
titelhelden.eu tizanidinehcl2mg.blogspot.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com todohr.com
tootoo.to top-study.work
top-l2.com trichizobswiv.agddns.net
tracfone.com tula.mdverey.ru
ttsq.fr turn-up-life.life
turbabitload.weebly.com uytmaster.ru
twittrading.com
u17795.netangels.ru
umaseh.com
unece.org
v24s.net v24s.net
videonsk.com vashsvet.com
vip-parfumeria.ru vbabule.net
virus-schutzmasken.de vesnatehno.ru
viveresaniesnelli.it viddyoze.com
vkak.ru vipps.com.my
vse-pesni.com vkontarkte.com
waycash.net webalan.ru
westvilletowingservices.co.za webpromotion.ae
winterclassichockeyjerseys.com xn--24-glceagatoq7c2a6ioc.xn--p1ai
worldis.me xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
x-diesel.com xn--80aanaardaperhcem4a6i.com
x5market.ru xn--b1agm2d.net
xchangetrak.com xn--d1aifoe0a9a.top
xn--1-8sbcpb0bdm8k6a.xn--p1ai xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
xn--lifehacer-1rb.com xpresscare.ru
xsion.net
xxxtube69.com
yaminecraft.ru
yebocasino.co.za
yellowstonevisitortours.com yellowstonevisitortours.com
yes-com.com
youdao.com youdao.com
youradexchange.com zajm-pod-zalog-nedvizhimosti.ru
zeg-distribution.com zambini.ru
zeikopay.com zytpirwai.net
zigarettenonlinekaufen1.bloog.pl
zone-kev717.info
zscaler.net

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.249 0; 10.1.0.248 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2724 ### Version: V4.2021.11.2725
### Updated: Fri Nov 19 10:03:12 UTC 2021 ### Updated: Fri Nov 19 22:01:14 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2724 ### Version: V4.2021.11.2725
### Updated: Fri Nov 19 10:03:12 UTC 2021 ### Updated: Fri Nov 19 22:01:14 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##