vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.12.2785 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-12-16 10:09:31 +00:00
parent 9d06fa7262
commit 545a3a02e7
47 changed files with 67936 additions and 67946 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2784 #### Version: V4.2021.12.2785
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2784 #### Version: V4.2021.12.2785
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -9,7 +9,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.12.2784 #### Version: V4.2021.12.2785
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8356
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.12.2784 #### Version: V4.2021.12.2785
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8360
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8360
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8360
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8360
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 1; 10.1.0.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 0; 10.1.0.48 0;
127.0.0.1 0; 127.0.0.1 0;

8360
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 1; 10.1.0.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 0; 10.1.0.48 0;
127.0.0.1 0; 127.0.0.1 0;

8360
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 1; 10.1.0.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 0; 10.1.0.48 0;
127.0.0.1 0; 127.0.0.1 0;

8360
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2784 2785

8360
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8360
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.112.89.132 0; 20.69.178.66 0;

8360
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 1; 10.1.0.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 0; 10.1.0.48 0;
127.0.0.1 0; 127.0.0.1 0;

8360
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 1; 10.1.0.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 0; 10.1.0.48 0;
127.0.0.1 0; 127.0.0.1 0;

8360
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 1; 10.1.0.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 0; 10.1.0.48 0;
127.0.0.1 0; 127.0.0.1 0;

8360
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.112.89.132 0; 20.69.178.66 0;

8360
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.112.89.132 0; 20.69.178.66 0;

8360
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 1; 10.1.0.48 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Disco Barkrowler
EirGrabber Citoid
Mojeek ExtractorPro
Sqlmap Jetty
Sqworm Kinza
Trendictionbot Pandalytics
Webshag RankFlex
arquivo.pt RedesScrapy
eCatch Whatweb
xpymep1.exe seocompany.store

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404enemy
404checker ADmantX
80legs AIBOT
ASPSeek ALittle Client
Aboundex AhrefsBot
Acunetix
AfD-Verbotsverfahren
AllSubmitter
AlphaBot
Anarchie
Anarchy Anarchy
Apexoo Anarchy99
Anthill
Asterias
AwarioSmartBot AwarioSmartBot
BBBike BDCbot
BLEXBot
BackDoorBot
BackStreet
BackWeb
Bandit Bandit
Barkrowler Battleztar Bazinga
BetaBot Black Hole
Bolt BlackWidow
BotALot Blow
BlowFish
Boardreader
Brandprotect
Buck Buck
BuiltBotTough BuiltWith
CATExplorador BunnySlippers
CCBot BuzzSumo
CODE87
CSHttp CSHttp
Calculon
CazoodleBot CazoodleBot
Cegbfeieh Cegbfeieh
CheTeam
CheeseBot CheeseBot
Cliqzbot CherryPicker
Cocolyzebot ChinaClaw
Cosmos Citoid
Claritybot
Cloud mapping
Copier
Craftbot Craftbot
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
DBLBot Crescent
Custo
DTS Agent DTS Agent
DataCha0s DataCha0s
Demon Deusu
Devil Devil
Digincore
DigitalPebble
Dirbuster Dirbuster
Discobot Discobot
Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics
DnyzBot DnyzBot
DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler
DomainStatsBot
Domains Project
Download Wonder
Dragonfly Dragonfly
EasyDL Drip
Ebingbong ECCP/1.0
EirGrabber Ecxi
Exabot Evil
Express WebPictures Fimap
ExtractorPro
EyeNetIE
Ezooms
FDM
Foobot Foobot
FyberSpider Fyrebot
GT::WWW G-i-g-a-b-o-t
Genieo Genieo
GermCrawler
GetRight
GetWeb GetWeb
Getintent Grabber
Gigabot
Go!Zilla
GoZilla
Grafula Grafula
GrapeFX
GrapeshotCrawler
Haansoft Haansoft
Harvest Heritrix
Id-search Hloader
IlseBot Humanlinks
IDBot
Iblog
Image Fetch
Image Sucker
InfoNaviRobot InfoNaviRobot
Internet Ninja
Iria Iria
Iskanie JennyBot
JamesBOT JetCar
Jbrofuzz Joomla
Jorgee
Jyxobot Jyxobot
Kinza
Kozmosbot Kozmosbot
LNSpiderguy LNSpiderguy
LexiBot
Lftp Lftp
LibWeb Libwhisker
Lightspeedsystems Lightspeedsystems
Linkbot
LinkextractorPro LinkextractorPro
LinkpadBot LinksManager
LinqiaMetadataDownloaderBot LinqiaScrapeBot
LinqiaRSSBot
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
MIIxpc Lmspider
MJ12bot MSIECrawler
MSFrontPage
Mag-Net Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic-SEO
Masscan MarkWatch
Mata Hari Mass Downloader
Microsoft Data Access MegaIndex.ru
Microsoft URL Control Minefield
Mojeek Moblie Safari
Mojolicious Mojolicious
NICErsPRO Morfeus Fucking Scanner
NearSite Msrabot
Needle Navroad
Nessus Net Vampire
NetLyzer NetAnts
Niki-bot Netcraft
Nettrack
NextGenSearchBot
Nibbler
Nikto Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Ninja Ninja
Nmap
Not Not
Nuclei Nutch
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler Openfind
Openvas
OrangeBot
OutclicksBot OutclicksBot
PECL::HTTP OutfoxBot
PHPCrawl PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageGrabber
Pandalytics Pandalytics
Panscient
PeoplePal PeoplePal
Pi-Monster Petalbot
Picsearch Picsearch
Piepmatz PictureFinder
Pimonster
Pixray Pixray
Pockey
ProPowerBot ProPowerBot
ProWebWalker
Probethenet Probethenet
Psbot Psbot
Pu_iN
Pump
PyCurl PyCurl
RSSingBot Quick-Crawler
RankActive RankFlex
RankingBot2
RankurBot
Re-re
ReGet ReGet
RealDownload RebelMouse
RedesScrapy Ripper
Rogerbot RocketCrawler
SEOkicks SBIder
SEOlyticsCrawler SEOprofiler
SEOstats SISTRIX
SMTBot
SalesIntelligent SalesIntelligent
Scanbot
ScoutJet
ScrepyBot ScrepyBot
SemanticJuice Semrush
SentiBot Seomoz
Shodan Shodan
Siphon Siphon
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker SiteSucker
Sitebeam
SlySearch SlySearch
SmartDownload
Snapbot
Sociscraper Sociscraper
Sogou web spider
Sosospider Sosospider
Sqlworm Sottopop
SpankBot
Sqworm Sqworm
Steeler Stripper
Sucker
Sucuri
SuperHTTP SuperHTTP
SurveyBot T8Abot
Szukacz
T0PHackTeam
Telesoft Telesoft
Telesphoreo
Telesphorep
The Intraformant
Thumbor
TightTwatBot
Titan
Toata Toata
Toweyabot
True_Robot
Turnitin Turnitin
TurnitinBot TwengaBot
Twice Twice
URLy Warning
URLy.Warning URLy.Warning
V-BOT
VB Project VB Project
VCI VCI
Vacuum
Vagabondo Vagabondo
VeriCiteCrawler Virusdie
VidibleScraper WWW-Mechanize
VoidEYE WWW::Mechanize
WBSearchBot
WISENutbot
WWWOFFLE
Wallpapers
Wallpapers/3.0
WallpapersHD
Web Auto Web Auto
Web Fuck Web Fuck
Web Sucker Web Pix
WebFuck Web Sauger
WebImageCollector WebImageCollector
WebLeacher WebLeacher
WebPix WebPix
WebSauger
WebStripper WebStripper
WebSucker
WebWhacker WebWhacker
Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
WebsiteExtractor WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster Whack
Whacker WinHTTrack
Xaldon WebSpider WiseGuys Robot
Xaldon_WebSpider Wonderbot
Zade
Zitebot
ZmEu
ZoomBot ZoomBot
ZoominfoBot ZoominfoBot
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt
backlink-check backlink-check
cah.io.community
clark-crawler clark-crawler
coccocbot-web
cognitiveseo cognitiveseo
com.plumanalytics crawler4j
crawl.sogou.com
crawler.feedback
dataforseo.com dataforseo.com
domainsproject.org domainsproject.org
facebookscraper eCatch
evc-batch
gopher gopher
instabid heritrix
ips-agent internetVista monitor
lwp-request
lwp-trivial lwp-trivial
magpie-crawler oBot
meanpathbot page scorer
plumanalytics ripz
polaris version s1z.ru
scalaj-http
seobility
seocompany.store seocompany.store
seoscanners sexsearcher
seostar sitechecker.pro
sysscan sogouspider
sp_auditbot
spyfu
trendiction.de
ubermetrics-technologies.com
voyagerx.com
webgains-bot
webmeup-crawler webmeup-crawler
webprosbot
xpymep1.exe xpymep1.exe
zauba.io
zgrab zgrab

306
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
ALittle Client 404enemy
ASPSeek 80legs
Aboundex AhrefsBot
Aboundexbot
AfD-Verbotsverfahren
Aipbot Aipbot
AllSubmitter
Alligator Alligator
AlphaBot
Anarchy Anarchy
Ankit Apexoo
Anthill
Aspiegel Aspiegel
Asterias
AwarioSmartBot AwarioSmartBot
BBBike BLEXBot
BackWeb BackDoorBot
Backlink-Ceck BackStreet
BacklinkCrawler BacklinkCrawler
Battleztar Bazinga Badass
Bigfoot BetaBot
Blackboard
BlowFish BlowFish
Boardreader Boardreader
BotALot BuiltBotTough
Buck Bullseye
BunnySlippers CATExplorador
BuzzSumo CCBot
CSHttp CODE87
Calculon Calculon
Cegbfeieh Cegbfeieh
CensysInspect CheTeam
CherryPicker CheeseBot
Chlooe Cloud mapping
Citoid Cocolyzebot
Cogentbot
Collector Collector
Copier
Copyscape
Cosmos
Craftbot Craftbot
Crawling at Home Project CrunchBot
Crescent
Curious Curious
DBLBot DBLBot
DatabaseDriverMysqli DSearch
DTS Agent
DataCha0s
Demon
Deusu
Devil Devil
Digincore Digincore
Dirbuster DnBCrawler-Analytics
Disco
Discoverybot
Dispatch
DittoSpyder
DnyzBot DnyzBot
DomCopBot
DomainCrawler
DomainSigmaCrawler
Download Wonder Download Wonder
Drip Drip
ECCP/1.0 ECCP/1.0
EMail Siphon EasyDL
Ebingbong Ebingbong
EirGrabber
Exabot Exabot
ExtLinksBot ExtractorPro
EyeNetIE
Ezooms
FemtosearchBot
Fimap Fimap
FlashGet Foobot
Flunky
Freeuploader Freeuploader
Fuzz FrontPage
G-i-g-a-b-o-t FyberSpider
Fyrebot
GT::WWW
Genieo Genieo
GetWeb GermCrawler
GoZilla GetRight
Getintent
Go-Ahead-Got-It
Gotit
Grabber
Grafula
GrapeFX
GrapeshotCrawler GrapeshotCrawler
HMView HMView
HTTP::Lite HTMLparser
Haansoft Haansoft
Hloader HaosouSpider
IDBTE4M Havij
Humanlinks
HybridBot
IDBot
IRLbot IRLbot
Id-search Id-search
IlseBot Iria
InterGET JOC Web Spider
IstellaBot Jbrofuzz
Jetty JennyBot
JustView JikeSpider
Kenjin Spider Jorgee
Keyword Density Jyxobot
Kinza LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot
Larbin
Leap Leap
LeechGet
LexiBot LexiBot
Libwhisker
LieBaoFast LieBaoFast
Likse Likse
Linkbot LinkWalker
Linkdexbot LinksManager
Lipperhey Litemage_walker
Lmspider MIDown tool
MFC_Tear_Sample MIIxpc
Mag-Net MQQBrowser
Magnet
Mail.RU_Bot
Majestic SEO
MarkMonitor
MarkWatch
Mass Downloader Mass Downloader
MauiBot MauiBot
Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Mojeek Mojeek
Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3
Msrabot
Musobot
NICErsPRO NICErsPRO
Name Intelligence NPbot
Nameprotect
Navroad
NetAnts
NetLyzer NetLyzer
NetSpider NetMechanic
Nettrack NetZIP
Netvibes Netvibes
NextGenSearchBot NextGenSearchBot
Niki-bot Ninja
NimbleCrawler Nuclei
Not Octopus
Offline Navigator Offline Explorer
OrangeBot OrangeBot
POE-Component-Client-HTTP OutfoxBot
Page Analyzer Page Analyzer
PageScorer
PageThing.com
Panscient Panscient
Pavuk Pimonster
PeoplePal
Petalbot
Pi-Monster
Picscout
PictureFinder
Piepmatz
Pixray
Pockey
ProPowerBot
ProWebWalker ProWebWalker
Pu_iN Psbot
PyCurl PxBroker
QueryN Metasearch RSSingBot
RankActive
RankActiveLinkBot RankActiveLinkBot
Rankivabot Rankivabot
RankurBot RebelMouse
RealDownload Ripper
Reaper SBIder
RepoMonkey SEOkicks-Robot
SEOprofiler SEOlyticsCrawler
SEOstats SEOstats
SISTRIX
SMTBot SMTBot
SalesIntelligent ScanAlert
Scanbot ScoutJet
Screaming Scrapy
Searchestate ScreenerBot
SearchmetricsBot SearchmetricsBot
Semrush SemrushBot
SentiBot
SeobilityBot
Siphon Siphon
Site Sucker
SiteExplorer
SiteLockSpider
Sitevigil Sitevigil
SlySearch
Snake Snake
Snapbot
Snoopy
Sociscraper
Sosospider
Sottopop Sottopop
SpankBot
Spinn3r
SputnikBot SputnikBot
Sqlworm Sqlmap
Sqworm
Steeler Steeler
Stripper
Sucker Sucker
Sucuri
SuperBot SuperBot
Surfbot Surfbot
Suzuran Suzuran
Swiftbot Szukacz
T0PHackTeam
T8Abot
Teleport
Telesoft Telesoft
Telesphorep TheNomad
Thumbor Thumbor
Toweyabot TightTwatBot
Trendiction Trendictionbot
True_Robot URLy Warning
Turingos
Turnitin
UnisterBot UnisterBot
Upflow Upflow
VeriCiteCrawler VeriCiteCrawler
VidibleScraper Virusdie
VoidEYE
Voil Voil
Voltron Voltron
WBSearchBot WASALive-Bot
Wallpapers WWW-Collector-E
WWWOFFLE
Wallpapers/3.0
WeSEE WeSEE
Web Fetch Web Auto
Web Enhancer
Web Fuck
Web Pix
Web Sucker Web Sucker
WebAuto WebAuto
WebBandit WebBandit
WebCopier WebGo IS
WebLeacher
WebPix WebPix
WebReaper
WebSauger WebSauger
WebStripper
WebSucker
WebWhacker
Webalta Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Webshag
WebsiteQuester
Webster Webster
Whack Whacker
Whatweb
Who.is Bot
Widow Widow
WinHTTrack WinHTTrack
Wonderbot Wonderbot
Woobot Wotbox
Xaldon WebSpider Xaldon WebSpider
Xenu
YoudaoBot
Zade
Zauba
Zermelo Zermelo
Zitebot Zitebot
arquivo.pt ZoomBot
autoemailspider ZyBorg
coccocbot-web archive.org_bot
arquivo-web-crawler
crawler.feedback crawler.feedback
crawler4j
dataforseo.com dataforseo.com
eCatch domainsproject.org
facebookscraper facebookscraper
instabid gopher
lwp-request internetVista monitor
polaris version meanpathbot
mediawords
probe-image-size probe-image-size
s1z.ru scan.lol
seostar seobility
sexsearcher seocompany.store
siteripz seoscanners
sitechecker.pro
sp_auditbot
spyfu spyfu
sysscan
tAkeOut
trendiction.com trendiction.com
trendiction.de x22Mozilla
ubermetrics-technologies.com
voyagerx.com
webgains-bot
webmeup-crawler
webpros.com
x09Mozilla
xpymep1.exe xpymep1.exe
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
bimatoprost-careprost.com.ua alekseevec.ru
leboard.ru beauby.ru
moi-glazki.ru compliance-barak.top
mtmtv.info compliance-donald.xyz
nero-us.com elektroniksigaraankara.info
ownshop.cf investzalog.ru
plugingeorgia.com online247.ml
rusprostitute.com predmety.in.ua
unrealcommander.biz service-core.ru
yoopsie.com xsion.net

924
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00it.com 007angels.com
1-99seo.com 123movies.love
123kuma.com 13tabs.com
1688.com 1stat.ru
2000k.ru 21h2o.com
2girls1cup-free.com 23kw.ru
4senses.co 3dracergames.com
4ur.click 4pp13.com
55wmz.ru 4rent.ru
7minuteworkout.com 70casino.online
aa9d046aab36af4ff182f097f840430d51.com 7makemoneyonline.com
aarbur.com 7search.com
abiente.ru 9icmzvn6.website
aadroid.net
actices.com actices.com
actionnooz.com adelly.bg
actulite.com
adloads.com adloads.com
admanaerofoil.com
adservme.com
adtech.us
adviceforum.info
affordablewebsitesandmobileapps.com affordablewebsitesandmobileapps.com
aihelen.net afftrack001.com
alfa9.com agadelha.com.br
allwomen.info airlandsea.info
altamayoztourism.com akita.kz
aktobe.xkaz.org
alert-fjg.xyz
allfinweb.com
alphacarolinas.com
alyeskaresort.com alyeskaresort.com
am-se.com amazingninja.com
am15.net apelsinnik.website
ameblo.jp
amospalla.es
ample-awards-today.us
apartmentbay.ru
api.stathat.com api.stathat.com
apibrowsesmartne-a.akamaihd.net apimountainbikei-a.akamaihd.net
apimegabrowsebiz-a.akamaihd.net appmsr.org
apparel-offer.com approved.su
approvedlocal.co.za arendas.net
arcteryxsale.online asrvvv-a.akamaihd.net
arewater.com asscenihotosidea.blogspot.co.za
arvut.org astana.xxxkaz.org
asseenontv.ru atlant-auto.info
australia-opening-times.com atlasvkusov.ru
autorn.ru atmovs.com
availit.weebly.com autonew.biz
avitocars.ru aviav.eu
aviav.org
avtolombard-krasnodar.ru
ayakino.net
azartmsl.com azartmsl.com
azte.ch
babespcs.com
back.dog back.dog
backiee.com
bad-stars.net
badavit.com.ua badavit.com.ua
bala.getenjoyment.net bank.uz
balkanfarma.org bannerpower.com
batanga.net barnfurnituremart.com
bayareaaeroclub.org belinka.com.ua
bestmobilityscooterstoday.com belsetirehafi.tk
bezcmexa.ru best-games.today
bg6s0.com bestbrainprod.win
big-boards.info bestchoice.cf
bio-market.kz betteroffers.review
billiard-classic.com.ua
bimatoprost-careprost.com
binomo.com
bio-japan.net
bioca.org bioca.org
bioscorp.ru bitnote.co
bitcoins-live.ru blackcurranthumidifiers.site
bochemit.com.ua bloggen.be
bookmark4you.com blogsfunky672.weebly.com
brains2.biz blpmovies.com
brk-rti.ru bluesman.nu
btnativenav.com bmsco.co
bnt-team.com
bodybuilding-shop.biz
bombla.org
bookmark4you.com.biz
boole.onlinewebshop.net
bracketsmackdown.com
brainboostingsupplements.org
brandimensions.com
bronzeaid-a.akamaihd.net
bugof.gq
buron.pw buron.pw
businesn.men buyhoverboard.com
buyviagraa.blogspot.com c-english.ru
calc-for-credit.ru
callejondelpozo.es
callmd5map.com
candiceloves.us candiceloves.us
cardsharp1.ru carabela.com.do
cardul.ru
carloans.com carloans.com
carrauterie.be casinos4dummies.co.uk
carstrends2015.com ceotrk.com
cartierlove2u.com cfacarrosserie74.com
cartierlovestore.xyz chatroulette.online
cbcseward.com cheap-trusted-backlinks.com
cctva.tv cheapmoncler.win
cdn.walkme.com chocolateslim-original.com
cdnanalytics.xyz christianlouboutinsaleonline.us
centraletermice.us cindymatches.com
cfcl.co.uk ciproandtizanidine.blogspot.com
cgi2.nintendo.co.jp cleanersoft.com
changde.58.com clickbanksites.info
chastnoeporno.com cocyq.inwtrade.com
chatroulette.life colehaanoutlet.store
cheapjerseysa.com collegerentals.com
cheapjerseysfootballshop.com community.allhiphop.com
cheapmoncler.pw compliance-barak.xyz
cheatcode-lita12.rhcloud.com compliance-george.xyz
children-learningreading.info compliance-irvin.top
clarithromycin500mg.com compliance-jack.top
clean-start.net compliance-mark.top
coldfilm.ru construmac.com.mx
conocer-sanabria.com cookie-law-enforcement-ff.xyz
converse.ddsoldes.fr copblock.org
cookingmeat.ru cosmetic.donna7753191.ru
credit.co.ua creatives.adbetclickin.pink
cutt.us
cv.wallhade.co cv.wallhade.co
d2jsp.org data-mining.tk
dairyindia.in ddlmega.net
data1.scopich.com ddsoldes.fr
dayibiao.com dear-diary.ru
dcj-nn.ru decorationspcs.com
degerlund.net delitime.info
demenageur.com delta-line.men
deplim.com dengi-pod-zalog-nedvizhimosti.ru
depositfiles-porn.ga depositfiles-porn.ga
detki-opt.ru detmebel.su
dfiles.me deutschehobbyhuren.net
diarioaconcagua.com dev.citetick.com
divisioncore.com dickssportinggoods.com
djonwatch.ru discovertreasure-a.akamaihd.net
documentsite.net diusyawiga.tk
dollartree.info dividendo.ru
domaincrawler.com divku.ru
domination.ml dogclothing.store
dorratex.tn dosug-lux.ru
downloadmefiranaratb1972.xpg.com.br doyouknowtheword-flummox.ml
e-collantes.com dtm-spain.com
eaplay.ru dustyorate.com
easyukraine.com earl-brown.info
ekosmetyki.net.pl ebooknovel.club
elektrozigaretten1.webs.com econom.co
empirepoker.com ecookna.com.ua
eredijovon.com editmedios.com
espaceinventoristes.com efnor-ac.com
excaliburfilms.com eldiariodeguadalajara.com
exchangeit.gq elektir.ru
f-online.de elitepcgames.com
fable.in.ua epicbrogaming.com
faithe.top erot.co
fanoboi.com etur.ru
fatfasts-4tmz.com evehemming.blogspot.com.au
fertilitetsradgivningen.se exchanges-bet.com
fighrofacciufreesig.ga exoclick.com
filmci.pro exoneration-project.us
financehint.eu expertnaya-ocenka.ru
financeloan.us f5mtrack.com
flagcounter.me favoritemoney.ru
flashbannernow.com feel-planet.com
floridamobilebillboards.com feorina.ru
fok.nl fettefrauen.net
foxinsocks.ru fikasound.tk
franch.info financepoints.eu
fuck-paid-share-buttons.xyz find1friend.com
fxtips.ru firstdrugmall.ru
fym.com.ru flashahead.info
game-top.su flprog.com
gameprimary.com fodelsedagspresenter.nu
gamerextra.com foodgid.net
gamevalue7.weebly.com forexgb.ru
gateway.zscalertwo.net free-gluten.ru
gdzkurokam.ru free-video-tool.com
gentamicineyedrops.blogspot.com freewareseek.com
getridofstretchmarks.org freshsuperbloop.com
girlgamerdaily.com fsalas.com
girlsatgames.ru gabeshop.ru
glavprofit.ru gameonasia.com
glcomputers.ru gardene.ru
gobongo.info gazobeton-p.com.ua
gojiberriess.apishops.ru generic-pills-online.com
goodprotein.ru getprismatic.com
googlepositions.com gg-arena.ru
grand-chlen.ru globatur.ru
gta-top.ru gonextmedia.com
hacktougroup.ru googlefeud.com
hdxnxxtube.mobi googlemare.com
hermesreplica.pw googlsucks.com
hesteel.pl googst2.ru
hexpilot.com gosarhivrt.ru
highland-homes.com gq-catalog.gq
gratuitxblcodes.com
green-tea.tv
grizzlysgrill.com
guigyverpo.cf
guod.me
hanink.biz.ly
hawaiisurf.com
heartofbeijing.blogspot.com
helvetia.com.ua
herehloadibs.cf
heroz.fr
highstairs-a.akamaihd.net highstairs-a.akamaihd.net
homeart.space hildinghr.se
homedecoguide.info himazin.info
hop.clickbank.net hitsbox.info
houdom.net hkdiiohi.skyrock.com
hoverboard360.de hledejvshopech.cz
hybrid.ru homemature.net
hystersister.com hopeonthestreet.co.uk
ihc2015.info hostingclub.lk
ilovevitaly.xyz hotcore.info
imagine-ex.co hoterika.com
imgarit.pw hotshoppymac.com
incolors.club hotsocialz.com
infazavr.ru houseofgaga.ru
infostatsvc.com houston-vikings.com
int.search.mywebsearch.com hully.altervista.org
invest-pamm.ru ideashome.id
investsuccess.org ie.57883.net
indiasourcemart.in
infospot.pt
investpamm.ru
invivo.hu invivo.hu
iplogger.org izoll.ru
ivanovo.zrus.org janettabridal.com
izismile.com jaxcube.info
jav-way.site jewelryandfiligree.com
jerseychinabizwholesale.us jose.mulinohouse.co
jerseyssportsshop.com kaac.ru
jillepille.com karaganda.xkaz.org
job.icivil.ir kareliatobacco.ru
joy-penguin.com karting196.ru
junglenet-a.akamaihd.net kazan.zrus.org
justdating.online kellyonline.xyz
jwss.cc kerei.ru
kabbalah-red-bracelets.com kerwinandcariza.com
karachev-city.ru kicknights.gq
kazan.xrus.org kliksaya.com
kerch.site kongruan.com
kino-fun.ru kosova.de
kino2018.club krassh.ru
kinobest.pl krasula.pp.ua
kinoplen.ru kumuk.info
kvartira-sutochno.com kupit-adenu.ru
kvartiry-remont.ucoz.ru kurbappeal.info
ladov.ru laapp.com
lanadelreyfans.us lafriore.ru
landinez.co
laptoper.net
larger.io
law-check-eight.xyz law-check-eight.xyz
laxob.com lazymanyoga.com
lazy-z.com letsrepair.in
lecbter-relationships.ga lexaprogeneric.link
legionalpha.com light.ifmo.ru
lidoradio.com lignofix.ua
life-instyle.com littleberry.ru
livesex-amateure.info locksmith.jp
louboutinreplica.xyz locustdesign.co
luckyday.world losangeles-ads.com
m1media.net luchshie-topcasino.ru
m3gadownload.pl
macdamaged.space
mackeeper-land-672695126.us-east-1.elb.amazonaws.com
macnewtech.com macnewtech.com
magicalfind-a.akamaihd.net maladot.com
magicplayer-s.acestream.net manipulyator-peterburg.ru
mail.allnews24.in matrixalchemy.com
mainhp.net medictube.ru
mall.uk meduza-consult.ru
mapquestz.us merryhouse.co.uk
mararoom.ru mesbuta.info
markjaybeefractal.com metalonly.info
mature.free-websites.com meteocast.net
me-ke.com mhtr.be
medicine-4u.org
medmajor.ru
meds-online24.com
megavolt.net.ua
message-warning.net
mexicotravelnet.com
mielec.pl
minegam.com minegam.com
minneapoliscopiers.com mini.7zap.com
mir-business-24.ru miniads.ca
misswell.net mixx.com
misterjtbarbers.com mmog-play.ru
mjchamonix.org moi-glazki.ru
mokrayakiska.com mompussy.net
money-every-day.com monclercoats.xyz
msk.afora.ru monclerjacketsoutlet.pw
music.utrolive.ru morefastermac.trade
mvpicton.co.uk mosdverka.ru
my-aladin.com mosrif.ru
mydoctorok.ru muz-shoes.ru
myindospace.com muznachas-service.ru
mykings.pw
mylesosibirsk.ru
mymercy.info
myseoconsultant.com
myshopmatemac.com
myxdate.info
nacap.ru nacap.ru
nailsimg.com najaden.no
nalogovyy-kodeks.ru natali-forex.com
nardulan.com net-profits.xyz
nash-krym.info netcheckcdn.xyz
naturalbreakthroughsresearch.com netoil.no
needtosellmyhousefast.com newfilmsonline.ru
netanalytics.xyz newhairstylesformen2014.com
nextlnk12.com nikitsyringedrivelg.pen.io
nfljerseys.online njkmznnb.ru
nfljerseyscheapchinabiz.com njpalletremoval.com
nightvision746.weebly.com nodding-passion.tk
nmrk.ru nodup.ru
nomuos.it
nonameread45.live
noscrapleftbehind.co noscrapleftbehind.co
nudo.ca
nw-servis.ru nw-servis.ru
of-ireland.info oakleyglassesonline.us
okmusic.jp offer.wpsecurity.website
okroshki.ru offers.bycontext.com
oktube.ru okayimage.com
omgtnc.com onlyforemont.ru
online-templatestore.com oranga.host.sk
ooo-gotovie.ru orion-code-access.net
oops-cinema.ru palocco.it
ordernorxx.com
orel-reshka.net
ossmalta.com
paccohichetoti.ml
palvira.com.ua palvira.com.ua
partners-ship.pro panamaforbeginners.com
pay2me.pl pdn-4.com
pelfind.me pechikamini.ru
phpdevops.com perfectpracticeweb.com
picquery.com physfunc.ru
pills24h.com pic2fly.com
pisanieprac.info picsearch.com
polska-poezja.com picsfair.com
popander.mobi piuminiita.com
poppen-nw.net pixell.club
popup-jdh.xyz pizdeishn.com
pornhub-forum.ga pizdopletka.club
porno-play.net playboyfiles.xblog.in
pornobrazzers.biz plugingeorgia.com
pornolook.net pogruztehnik.ru
pornophoto.xyz porn-w.org
pornosee.info pornotubs.com
pornotubexxx.name potoideas.us
pornstartits.xblog.in potolokelekor.ru
portside.xyz preconnubial.usuby.site
pr0fit-b0x.com predominant-invent.tk
preg.marketingvici.com pricheski-video.com
privacylocationforloc.com priora-2.com
prostitutki-tyumeni.org privat-girl.net
prpops.com prlog.ru
pussyfleet.com progonrumarket.ru
raavidesigns.com prokotov.com
rapidgator-porn.ga promodj.com
razleton.com proprostatit.com
razvratnoe.org prostitutki-astana.org
real-time-analytics.com purplestats.com
rebuildermedical.com puteshestvennik.com
reckonstat.info quit-smoking.ga
rialp.getenjoyment.net rankexperience.com
ric.info rankia.com
rankingchart.de
rapevideosmovies.com
rczhan.com
recinziireale.com
redhotfreebies.co.uk
reelheroes.net
refads.pro
referencemoi.com
refererx.com
registrationdomainsite.com
relax.ru
remedyotc.com
renecaovilla.online
rennlist.com
rep-am.com
risparmiocasa.bz.it risparmiocasa.bz.it
robotixix.com roofers.org.uk
rocis.site
rock-cafe.info
rockingclicks.com
rospromtest.ru rospromtest.ru
saneitconsulting.com royalvegascasino.com
sardinie.us rubanners.com
runtnc.net
rusenvironmental.net
russia-today-video.ru
rustic-quiver.win
s-iwantyou.com
sajatvelemeny.com
sawin.beth.webd.pl sawin.beth.webd.pl
sayyoethe.blogspot.co.za scalerite.co.za
sbt-aqua.ru
scanmyphones.com
scanner-john.top scanner-john.top
scanner-julia.top scanner-julianna.top
searchinterneat-a.akamaihd.net scanner-olga.top
secret.xn--oogle-wmc.com scottbywater.com
security60-e.com searchwebknow-a.akamaihd.net
sel-hoz.com seasonvar.ru
seo-smm.kz semaltmedia.com
seobility.net seokicks.de
seoexperimenty.ru seonetwizard.com
seofirmreviewsus.info seorankinglinks.us
servicecenter.co.ua service.adtech.fr
servingnotice.com
serviporno.com
sethrollins.net sethrollins.net
sexad.net sexs-foto.top
sexblog.pw sexvporno.ru
sexpornotales.com sfj-ror.no
sexytrend.ru shareyards.com
sharebutton.org sharpchallenge.com
shivafurnishings.com shoesonlinebuy.xyz
shoesonlinebuy.cn shtora66.ru
shop.xz618.com simple-share-buttons.com
sierraapps.com sindragosa.comxa.com
silktide.com sixcooler.de
silverage.ru skypasss.com
site-speed-checker.site smart-scripts.com
siteimprove.com socialmadesimple.com
sitiz.club
skatestick.bid
skutecznetabletkinaporostwlosow.pl
sluganarodu.ru
smailik.org
snjack.info
soc-econom-problems.ru
soc-proof.su
social-button.xyz
socialbutton.xyz
sockshares.tv
sofit-dmd.ru
softlinesolutions.me
soserfis.com soserfis.com
spamnuker.com sotechco.co
starpages.net souvenir.cc
starwars.wikia.com spabali.org
steame.ru spasswelt.net
steebook.com special-porn.com
splendorsearch-a.akamaihd.net
spyfu.com
srdrvp.com
sribno.net
ssconstruction.co
steamoff.net
stillmiracle.com stillmiracle.com
strana-solnca.ru stmassage.ru
strangeduckfilms.com storehouse.ua
streha-metalko.si stroiminsk.org
strongholdsb.ru studworks.org
strv.se stuffpride.com
styro.ru sumo.com
sukarame.net superfish.com
suralink.com superinterstitial.com
surcentro.com superoboi.com.ua
survival.betteroffers.review superstats.com
susanholtphotography.com svetodiodoff.ru
swinger-mobil.net swinger-mobil.net
swingerseiten.com tam-gde-more.ru
sygraem.com taraz.xkaz.org
tacbibirfa.tk
talant-factory.ru
tattomedia.com
tds-advert005.info tds-advert005.info
tedxrj.com tehngr.ru
teenforporn.com terraclicks.com
terrafootwear.us textads.men
testingads.pro the-trader.net
thecoolimages.net the-usa-games.blogspot.com
thegamerznetwork.com thecoral.com.br
themestotal.com thecounter.com
thin.me.pn thefotosgratis.eu
time-japan.ru theporndude.com
titan-ads.life tizanidineandhydrocodone.blogspot.com
tizanidine4mgstreetvalue.blogspot.com tizanidinedosage.blogspot.com
tizanidineduringpregnancy.blogspot.com tizanidinehcl2mgtablet.blogspot.com
tizanidinehcl2mgsideeffects.blogspot.com
tizanidinehcl4mgtablet.blogspot.com tizanidinehcl4mgtablet.blogspot.com
tizanidineoral.blogspot.com tizanidinehydrochloride4mgstreetvalue.blogspot.com
topcar-krasnodar.ru tizanidineinteractions.blogspot.com
topsy.com tizanidinevssoma.blogspot.com
tour-line.net tksn.ru
toy-shop.top tooplay.com
trackzapper.com top1-seo-service.com
trafficmania.com topads.men
traffmonster.info topappspro.com
tuberkuleznik.ru track112.site
tuckpointingmasonrysystems.com trafficreceiver.club
tuckermktg.com
tula.mdverey.ru tula.mdverey.ru
turist-strani.ru
twittrading.com
twojebook.pl
txxx.com
u17795.netangels.ru u17795.netangels.ru
ugogo.info ucoz.ru
ultimateclassicrock.com ukrtextbook.com
um-razum.ru unisexjewelry.org
universals.com.ua univerfiles.com
urlopener.blogspot.com.au ural-buldozer.ru
vapeface.club velen.io
vashsvet.com video-chat.cn
vchulkax.com videochat.mx
venerologiya.com videojam.tv
videonsk.com videokrik.net
videotuber.ru vsdshnik.com
vigrx-original.ru vzlomtw.com
vipsiterip.org w-journal.ru
visionwell.com.cn wallpapersist.com
vkmusics.ru wapsite.me
vozbujdenie.com website-audit.com.ua
vsesubwaysurfers.com website-stealer.nufaq.com
vsexkontakte.net wesharepics.site
vzubah.com whatistizanidine2mg.blogspot.com
warezkeeper.com wholesalecheapjerseysfree.com
we-are-gamers.com wholesalejerseychinaoutlet.com
webradiology.ru win-spy.com
weburlopener.com womens-journal.net
wedding-salon.net worldmusicfests.com
wemedinc.com xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
westvilletowingservices.co.za
whatsappbot.flyland.ru
whos.amung.us
williamrobsonproperty.com
womama.ru
woweb.com.ua
www.bookmaker-bets.com
www.solartek.ru
wygraj-skiny.win
xchat26.myfreecams.com
xitjw.info
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks xuki.us
xn--80aeahghtf8ac5i.xn--p1ai yaoguangdj.com
xn--d1abj0abs9d.in.ua youjizz.com
xn--e1aggki3c.xn--80adxhks youporn-ru.com
xxxguitars.com youradexchange.com
xxxhdvideo.site
yaaknaa.info
yachts-cruise.info
yoluxuryevents.com
yottos.com
youtube-downloader.savetubevideo.com
youtubologia.it
yur-p.ru yur-p.ru
zaimite.ru zahvat.ru
zajm-zalog-krasnodar.ru zaimhelp.ru
zetmaster.ru zdesoboi.com
zigarettenonlinekaufen2.bloog.pl zeikopay.com
znakom.sibtest.ru zeroredirect9.com
zigzog.ru
zocaparj.kz zocaparj.kz
zoolubimets.ru
zootravel.com
zrelaya.pw
zreloeporno.tv
zrizvtrnpale.tk
zvuker.net
zzlgxh.com

490
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
300richmond.co.nz 28n2gl3wfyb0.ru
4ureyesonly.com 2kata.ru
9med.net 2nt.ru
abcdefh.xyz 4rent.ru
abclauncher.com 8558l.jobs.net
adbetclickin.pink abusora.com
adf.ly actulite.com
admanaerofoil.com adtech.de
adultfullhd.com adtech.fr
adultgalls.com afora.ru
advancedcleaner.com algerianembassy.co.in
animaltoplist.com alt-servis.ru
apelsinnik.website amazingpic.net
apisecretsaucebi-a.akamaihd.net amigobulls.com
apple.com-webbrowsing-security.science apilinkswiftco-a.akamaihd.net
aruplighting.com apimountainbikei-a.akamaihd.net
asia-forum.ru apispringsmartne-a.akamaihd.net
ausergrubhof.info appearance-cool.com
auto.rusvile.lt appmsr.org
autoblogger24.info aquarium-pro.ru
autobudpostach.club asiavirtualsolutions.com
autorn.ru asseenontv.ru
azartclub.org atmovs.com
azartmsl.com avramstroy.ru
ayanaspa.com
bausparen.bz.it
beauby.ru beauby.ru
bestcalovebracelet.cn beauty-clinic.ru
binaryoptionscops.info benchmarkcommunications.co.uk
blogig.org big-cash.net
bluesman.nu biocn.dx.am
bobba.dzaba.com bloggers.nl
bonvillan.com boole.onlinewebshop.net
brg8.com brandedleadgeneration.com
bryansk.zrus.org break-the-chains.com
bscodecs.com brk-rti.ru
buckspar.gq burkesales.com
buy-cheap-pills-order-online.com cablecar.us
calstaterealty.us callmd5map.com
carabela.com.do cand.jp
caveavins.fr capacitacionyaprendizaje.com
chatroulette.life cbb1smartlist12.click
chatroulette.online cheap-trusted-backlinks.com
cheapjerseysfootballshop.com cheatcode-lita12.rhcloud.com
chelyabinsk.xrus.org checkhit.com
chocolateslim-en-espana.com coindirect.io
christianlouboutinreplica.win companies-catalog.com
clickcash.com copyrightclaims.org
cocolyze.com cpajunkies.com
comeondog.info crazyboost.pro
cometorussia.net crystalslot.com
compliance-viktor.top cyse.tk
craftinsta.ru de.zapmeta.com
cscwtalkto.site decoratinghomes.ga
danceuniverse.ru delta-line.men
day-news.info discovertreasure-a.akamaihd.net
daydream-studio.ru divatraffic.com
daymusam.com dogsrun.net
design-lands.ru dolohen.com
dev-seo.blog duawitchrarato.tk
dieswaene.com earthmagic.info
digilander.libero.it eleimgo.pw
dollartree.info elidelcream.weebly.com
doska-vsem.ru erotikstories.ru
dynainbox.com eshop.md
eachdayisagift.review evaashop.ru
easyukraine.com extads.net
eluxer.net ezigarettekaufen1.hpage.com
enge-fotzen.info fashion-mk.net
ertelecom.ru feargames.ru
expdom.com film-one.ru
ezigarettekaufen2.mpbloggar.se financeloan.us
familyholiday.ml findpics.pw
fapgon.com flex4launch.ru
farmprofi.net footballfarrago.com
fealq.com forex21.ru
fertilitetsradgivningen.se foxweber.com
fidelityfunding.com free-share-buttons.blogspot.com
finansov.info freenom.link
focalink.com
foxjuegos.com
freelotto.com
freemags.cc
front.ru
funnel.co.za funnel.co.za
goldandcard.ru gambarxkata.co
golden-catalog.pro game-mmorpg.net
gotcher.us ge0ip.com
gratuitbaise.com gemgrab-a.akamaihd.net
guitar-master.org get-free-social-traffic.com
gunsvicceadadebt.tk getprismatic.com
hacktougroup.ru gogalleryawesome.com
haikuware.com greatfind-a.akamaihd.net
heartofpayne.xyz gsasearchengineranker.top
houston-vikings.com hard-porn.mobi
hully.altervista.org hd-film.pl
hyhj.info hd720kino.ru
ilovevitaly.com hdpixion.com
inome.com.ua healing-dysplasia.ru
interfucks.net hidefiles.org
isitwp.com hitmuzik.ru
javatex.co.id hobild.net
justbcause.com homelygarden.com
kamagragelusa.net hothor.se
kamen-e.ru hotkeys.com
kino-rating.ru humanelydrew.com
kinobaks.com hyipmanager.in
koptims.tiu.ru ie.57883.net
kskjco.club incanto.in.ua
landreferat.ru innoslicon.com
leadwayau.com iron-age.info
ledpolice.ru jerseywholesalebizchina.com
life.biz.ua kiwi237au.tk
littlesexdolls.com kollekcioner.ru
lomza.info ks1234.com
lost-alpha.ru la-fa.ru
lottospring.com landinez.co
lovi-moment.com.ua letmacwork.world
lyrster.com littlesunraiser.com
macotool.com liveshoppersmac.com
magicdiet.gq livingroomdecor.info
make-money-online.com loadopia.com
meduza-consult.ru luxmagazine.cf
megahdporno.net lyngdalhudterapi.no
mikrobiologies.ru m0r0zk0-krava.ru
mil-stak.com mailemedicinals.com
mirtorrent.net malls.com
moesexy.com mansiondelrio.co
monthlywinners.com map028.com
morf.snn.gr maxthon.com
musicstock.me maxxtor.eu
mydoctorok.ru medanestesia.ru
megavolt.net.ua
mezaruk.info
minegam.com
mompussy.net
morlat.altervista.org
movpod.in
music.utrolive.ru
myfreemp3.eu myfreemp3.eu
myhitmp3.club mykings.pw
myonigroup.com mymobilemoneypages.com
mywallpaper.top nardulan.com
naj-filmy24.pl naval.jislaaik.com
nelc.edu.eg nmrk.ru
netvouz.com obiavo.kz
niki-mlt.ru oecnhs.info
nikkiewart.ru ofanda.com
nordvpn.com omsk.xrus.org
o-o-6-o-o.com onlinebay.ru
oddamzadarmo.eu onlinemeetingnow.com
odesproperty.com ooomeru.ru
ontargetseo.us orgasmatrix.com
ooo-olni.ru osb.se11.ru
osnova3.ru ossmalta.com
oynat.info parlament.biz
pammik.ru penisvergrotendepillennl.ovh
piccshare.com physfunc.ru
picphotos.net pihl.se
picquery.com pinstake.com
pics-group.com pokemongooo.ml
picture-group.com pornofeuer.com
pintattoos.com pornoinn.com
placid-rounded-coast.glitch.me powenlite24.ru
plastweb.ru primedice.com
pobeiranie.pl privacyassistant.net
poems.com.ua prohoster.info
pogodnyyeavarii.gq prom23.ru
pokemon-go-play.online promoheads.com
polska-poezja.com purplesphere.in
pontiacsolstice.info pushdata.sendpulse.com
popserve.adscpm.net qpypcx.com
pornosemki.info ragecash.com
pornoslive.net restorator-msk.ru
pornotubs.com rock-cafe.info
preventheadacheguide.info rospromtest.ru
privatov-zapisi.ru rubbed.us
privetsochi.ru rusenvironmental.net
pulseonclick.com sarm.tk
qcstrtvt.bloger.index.hr sayyoethe.blogspot.co.za
qor360.com scanner-ivan.top
quizzitch.net scanner-mark.top
rarbg.to screpy.com
razorweb-a.akamaihd.net sdi-pme.com
recordpage-a.akamaihd.net sdrescher.net
referencemoi.com seadragonherbery.com
remont-komputerov-notebook.ru seorankinglinks.com
revistaindustria.com serving.adbetclickin.pink
rfd-split.hr sevendays.com.ua
ricorsogiustizia.org sexkontakte-seite.com
roznica.com.ua sharebutton.org
rusexy.xyz sibecoprom.ru
scrnet.biz.ua site.ru
seomarketings.online slomm.ru
seonetwizard.com sloopyjoes.com
seozoom.it smailik.org
sergiorossistore.online snworks.com
sexsaoy.com space4update.pw
sexxdate.net spamnuker.com
shlyahten.ru stard.shop
shop.xz618.com startwp.org
sicfor.bcu.cc
sideeffectsoftizanidine.blogspot.com
sildenafil-tadalafil.info
skytraf.xyz
snsdeainavi.info
socas.pluto.ro
sockshare.net
sophang8.com
spidtest.space
spin2016.cf
srdrvp.com
staticfs.host staticfs.host
stop-zavisimost.com stevemonsen.com
streetfooduncovered.com stroilka.info
stretchingabuckblog.com suchenindeutschland.com
studiofmp.com sudexpert66.ru
superlist.biz superstats.com
survival.betteroffers.review surintech.ac.th
teencastingporn.com swsociety.se
telefonsexi.com telegramdownload10.com
telesvoboda.ru thedownloadfreeonlinegames.blogspot.com
template-kid.com thetoiletpaper.com
thesmartsearch.net tiens2010.ru
tizanidinewithdrawalsymptoms.blogspot.com timetorelax.biz
tomck.com tizanidine6mg.blogspot.com
tpu.ru tizanidinevsvalium.blogspot.com
treasuretrack-a.akamaihd.net tonerbox.kz
trumpetedextremes.com topappspro.com
twittrading.com toys.erolove.in
umekana.ru traffixer.com
uzungil.com tula.howotorg.ru
vabasa.inwtrade.com tvand.ru
valkiria-tk.ru ua.tc
unblocksit.es
uptime-delta.net
urdoot.win
urlopener.blogspot.com.au
valoresito.com
vbikse.com vbikse.com
velen.io veles.shop
viktoria-center.ru vezuviy.su
vip2ch.com vsdshnik.com
visa-pasport.ru wallpaperaccess.com
voditeltrezviy.ru waycash.net
vysigy.su webix.biz
vzlomtw.com webscouter.net
w7s.ru westum.se
warezaccess.com whatistizanidinehclusedfor.blogspot.com
watracker.net ws.ampower.me
webalan.ru x-diesel.com
wineration.com x-lime.net
wingsoffury2.com xaylapdiendanang.com
wurr.voila.net xbaboon.com
x-mix.info xn--e1afanlbnfckd7c3d.xn--p1ai
xchat26.myfreecams.com xxxmania.top
xev.ru yginekologa.com
xlolitka.com yhit.press
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai youdesigner.kz
xn----ctbigni3aj4h.xn--p1ai yougame.biz
xn--b1adccaf1bzj.xn--p1ai yousense.info
xn--b1agm2d.net zarabotok--doma.ru
yottos.com zdesformula.ru
zaapplesales.blogspot.com zeroredirect1.com
zeroredirect6.com
znaniyapolza.ru

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.193 0; 10.1.0.48 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2784 ### Version: V4.2021.12.2785
### Updated: Wed Dec 15 22:01:40 UTC 2021 ### Updated: Thu Dec 16 10:03:36 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2784 ### Version: V4.2021.12.2785
### Updated: Wed Dec 15 22:01:40 UTC 2021 ### Updated: Thu Dec 16 10:03:36 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##