vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-06 20:50:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.10.2678 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-10-27 22:09:05 +00:00
parent 31ce9f7d29
commit ae93a81ec5
48 changed files with 71002 additions and 71022 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2677 #### Version: V4.2021.10.2678
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2677 #### Version: V4.2021.10.2678
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2677 #### Version: V4.2021.10.2678
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8738
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2677 #### Version: V4.2021.10.2678
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8742
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8742
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8742
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8742
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

8742
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

8742
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

8742
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2677 2678

8742
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8742
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.211.46.112 0; 20.185.255.187 0;

8742
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

8742
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

8742
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

8742
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.211.46.112 0; 20.185.255.187 0;

8742
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.211.46.112 0; 20.185.255.187 0;

8742
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
CazoodleBot Drip
Lanshanbot EirGrabber
Pavuk Express WebPictures
Piepmatz Jbrofuzz
RankActiveLinkBot JustView
SalesIntelligent Nuclei
Trendiction Semrush
WallpapersHD Swiftbot
pcBrowser Vacuum
s1z.ru Zitebot

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider
404checker
80legs
AIBOT AIBOT
ALittle Client
Abonti
Aboundex Aboundex
Aboundexbot Aboundexbot
AfD-Verbotsverfahren AfD-Verbotsverfahren
AiHitBot Aipbot
AllSubmitter Alexibot
Anarchy Anarchie
Ankit Apexoo
Anthill Asterias
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike BackDoorBot
BackWeb
BacklinkCrawler
Badass
Barkrowler Barkrowler
BatchFTP BlackWidow
Battleztar Bazinga Bolt
BetaBot Brandprotect
Black Hole Buddy
Blow BuiltWith
BotALot
Buck
BuiltBotTough
Bullseye
BunnySlippers
CATExplorador CATExplorador
CODE87
CazoodleBot
CensysInspect
CheTeam
CheeseBot CheeseBot
ChinaClaw ChinaClaw
Claritybot
Cliqzbot Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot
Collector Collector
CopyRightCheck
Copyscape Copyscape
Crescent
Custo Custo
CyotekWebCopy CyotekWebCopy
DBLBot
DIIbot
DSearch
DatabaseDriverMysqli DatabaseDriverMysqli
Deusu Demon
Digincore Digincore
DigitalPebble
Dirbuster Dirbuster
Discobot Disco
Discoverybot Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnyzBot
DomCopBot DomCopBot
DomainCrawler DomainAppender
Domains Project DomainSigmaCrawler
DomainStatsBot
Dotbot Dotbot
Dragonfly
Drip
EMail Siphon
EasyDL EasyDL
Ebingbong Ebingbong
EirGrabber EirGrabber
EroCrawler Exabot
Evil
ExtLinksBot
Extractor
ExtractorPro ExtractorPro
Extreme Picture Finder
Ezooms
FDM
FemtosearchBot FemtosearchBot
Fimap Fimap
FlashGet Flunky
Freeuploader Fyrebot
Fuzz
FyberSpider
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW
GalaxyBot GalaxyBot
Genieo
GermCrawler GermCrawler
GetWeb
Getintent Getintent
GoZilla Gigabot
Go-Ahead-Got-It
Gotit Gotit
GrabNet
Grabber Grabber
Grafula
GrapeFX GrapeFX
GridBot GridBot
HEADMasterSEO HTMLparser
HTTP::Lite
HTTrack HTTrack
HaosouSpider
Harvest
Havij Havij
Heritrix Humanlinks
IDBot IDBot
Id-search Id-search
IlseBot
Image Fetch
Image Sucker
InfoNaviRobot InfoNaviRobot
Intelliseek Intelliseek
InterGET
Internet Ninja
InternetSeer InternetSeer
Iria IstellaBot
JikeSpider JOC Web Spider
Jbrofuzz
Jetty
Joomla Joomla
Jorgee
Jyxobot Jyxobot
Kinza Keyword Density
Kozmosbot
Lanshanbot Lanshanbot
LexiBot Lftp
Lightspeedsystems LieBaoFast
Likse Likse
LinkWalker Linkbot
LinkextractorPro LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
MIDown tool Ltx71
MIIxpc MFC_Tear_Sample
MQQBrowser MJ12bot
MSFrontPage MSFrontPage
MSIECrawler
Majestic-SEO
MarkMonitor MarkMonitor
MarkWatch Masscan
Mata Hari Mb2345Browser
MeanPath Bot Microsoft URL Control
Moblie Safari Mister PiX
Mojeek Morfeus Fucking Scanner
Musobot Mozlila
NICErsPRO
NPbot NPbot
Name Intelligence
Nameprotect
NearSite NearSite
Needle Nessus
NetMechanic NetMechanic
NetSpider
Netcraft
Nettrack Nettrack
NextGenSearchBot NextGenSearchBot
Nibbler NimbleCrawler
Nuclei Nimbostratus
Ninja
Nmap
Nutch Nutch
Openfind Offline Explorer
PHPCrawl OrangeSpider
PageThing.com OutclicksBot
OutfoxBot
PECL::HTTP
PageAnalyzer
PageScorer
Panscient
Papa Foto Papa Foto
Pavuk Pavuk
PeoplePal PeoplePal
Petalbot Petalbot
Pi-Monster Pi-Monster
Picscout
Picsearch Picsearch
PictureFinder
PleaseCrawl
Pockey Pockey
ProWebWalker Psbot
Probethenet Pu_iN
Pump PyCurl
PxBroker Quick-Crawler
QueryN Metasearch Rankivabot
RankActiveLinkBot SEOkicks
RankFlex SEOkicks-Robot
RebelMouse SEOprofiler
Ripper
Rogerbot
SBIder
SEOlyticsCrawler
SMTBot SMTBot
ScanAlert SalesIntelligent
Scanbot
Scrapy Scrapy
ScreenerBot Screaming
Searchestate ScrepyBot
SearchmetricsBot
Semrush Semrush
SentiBot SentiBot
SeoSiteCheckup
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteSucker
SiteLockSpider Siteimprove
Sitebeam
SlySearch
SmartDownload SmartDownload
Snake
Snapbot Snapbot
Snoopy Sogou web spider
SpankBot Sosospider
SpaceBison
Spanner
SputnikBot SputnikBot
Stripper Stripper
Sucuri Sucker
SuperBot
Surfbot
SurveyBot SurveyBot
Szukacz Suzuran
Telesphoreo Swiftbot
T0PHackTeam
T8Abot
Teleport
TeleportPro
The Intraformant
Thumbor Thumbor
Toata TightTwatBot
Titan
Toweyabot Toweyabot
Tracemyfile
Trendiction Trendiction
Trendictionbot
True_Robot True_Robot
Turingos Turingos
Turnitin Typhoeus
Twice
UnisterBot UnisterBot
Upflow Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
Virusdie Virusdie
WPScan VoidEYE
Voltron
WBSearchBot
WWW-Collector-E WWW-Collector-E
WWW::Mechanize WWWOFFLE
Wallpapers Wallpapers/3.0
WeSEE
Web Auto Web Auto
Web Enhancer Web Collage
Web Fetch
Web Fuck Web Fuck
Web Pix Web Sauger
WebAuto WebCollage
WebBandit WebPix
WebGo IS
WebImageCollector
WebReaper WebReaper
WebSauger WebSauger
WebSucker WebWhacker
WebZIP
WebsiteQuester WebsiteQuester
Webster Webster
Who.is Bot Whatweb
Widow Widow
Wprecon Wotbox
Xaldon WebSpider YoudaoBot
Zade
Zauba
Zeus
Zitebot
ZmEu
ZoomBot ZoomBot
autoemailspider ZumBot
archive.org_bot
arquivo.pt
backlink-check backlink-check
check1.exe check1.exe
clark-crawler crawl.sogou.com
com.plumanalytics
crawler4j crawler4j
demandbase-bot dataforseo.com
eCatch eCatch
instabid evc-batch
facebookscraper
heritrix
internetVista monitor internetVista monitor
muhstik-scan isitwp.com
magpie-crawler
netEstate NE Crawler
oBot
page scorer
pcBrowser pcBrowser
polaris version probe-image-size
ripz satoristudio.net
seoscanners scalaj-http
seostar
serpstatbot
sitechecker.pro
siteripz siteripz
sp_auditbot sogouspider
tAkeOut spyfu
webpros.com webpros.com
webprosbot zauba.io
x09Mozilla
xpymep1.exe
zgrab

334
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
80legs
ADmantX
AIBOT AIBOT
ASPSeek ASPSeek
Abonti Abonti
Aboundex Aboundexbot
Acunetix
AfD-Verbotsverfahren
AiHitBot AiHitBot
AlphaBot Alexibot
AllSubmitter
Anarchy Anarchy
Ankit
Apexoo Apexoo
Aspiegel Attach
Asterias
AwarioRssBot AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike BackStreet
BDCbot Backlink-Ceck
BDFetch BacklinkCrawler
BackWeb Badass
BetaBot Bandit
Bigfoot
Black Hole Black Hole
BlackWidow
Blackboard Blackboard
Blow
Boardreader
Brandprotect Brandprotect
BuiltWith Brandwatch
BunnySlippers Buck
BuzzSumo CATExplorador
CODE87 CCBot
CSHttp Calculon
CazoodleBot CazoodleBot
Cegbfeieh
CensysInspect CensysInspect
CheeseBot CheeseBot
CherryPicker ChinaClaw
Chlooe Chlooe
Cliqzbot Claritybot
Cloud mapping Cocolyzebot
Cogentbot
Collector Collector
CopyRightCheck Copyscape
CrunchBot Cosmos
Custo Craftbot
CyotekWebCopy Crawling at Home Project
CrazyWebCrawler
Crescent
DBLBot DBLBot
DSearch DIIbot
DTS Agent
DatabaseDriverMysqli DatabaseDriverMysqli
Demon Demon
Devil Devil
Dirbuster DittoSpyder
Disco DomainAppender
Dispatch
DnBCrawler-Analytics
DnyzBot
DomainCrawler
DomainSigmaCrawler
DomainStatsBot
Domains Project
Dotbot Dotbot
Download Wonder Dragonfly
EasyDL Drip
Extractor ECCP/1.0
ExtractorPro EMail Siphon
Ebingbong
EroCrawler
Express WebPictures
ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
FDM Fimap
Firefox/7.0 Firefox/7.0
Fyrebot Foobot
GalaxyBot GT::WWW
Genieo GetWeb
Go!Zilla Gigabot
Go-Ahead-Got-It GoZilla
Gotit
Grabber Grabber
Grafula Grafula
HEADMasterSEO HTMLparser
HTTrack HTTrack
Harvest Haansoft
HybridBot Heritrix
IDBTE4M
IDBot IDBot
IlseBot IRLbot
Iblog
Id-search
Image Fetch
Image Sucker Image Sucker
Intelliseek
InterGET
Internet Ninja Internet Ninja
Iskanie InternetSeer
IstellaBot
JOC Web Spider JOC Web Spider
JetCar JamesBOT
Jetty
JikeSpider JikeSpider
JustView JustView
Kozmosbot Jyxobot
LeechGet Kenjin Spider
LibWeb Kinza
LNSpiderguy
Lanshanbot
LeechFTP
Lftp
Libwhisker
LieBaoFast LieBaoFast
Lightspeedsystems
Linkdexbot Linkdexbot
LinkextractorPro LinkextractorPro
LinkpadBot
LinksManager LinksManager
LinqiaRSSBot LinqiaScrapeBot
Lipperhey Spider MFC_Tear_Sample
Litemage_walker
Lmspider
MIIxpc MIIxpc
MJ12bot Majestic12
MSIECrawler Masscan
MTRobot MegaIndex.ru
Mag-Net
Magnet
Mail.RU_Bot
MarkMonitor
Mass Downloader
Mata Hari
MauiBot
Meanpathbot
Mediatoolkitbot
MicroMessenger MicroMessenger
Microsoft URL Control
Moblie Safari Moblie Safari
Mojeek
Mojolicious Mojolicious
Mozlila Morfeus Fucking Scanner
Msrabot NICErsPRO
Needle
Nessus Nessus
Net Vampire
NetLyzer
NetZIP
Netcraft Netcraft
Nettrack Netvibes
Niki-bot NextGenSearchBot
Nikto Nimbostratus
Ninja Nmap
Nuclei Nutch
Offline Explorer Octopus
OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
OrangeBot Openfind
OutclicksBot
OutfoxBot OutfoxBot
PECL::HTTP
PHPCrawl PHPCrawl
PageGrabber PageAnalyzer
Pandalytics PageScorer
Panscient Pavuk
PeoplePal
Petalbot Petalbot
Picsearch Pi-Monster
Piepmatz Picscout
Pixray PictureFinder
Pimonster
Pockey Pockey
Pu_iN
Pump
PyCurl PyCurl
Quick-Crawler
RankActive RankActive
RankingBot RankActiveLinkBot
Re-re RankFlex
ReGet ReGet
RealDownload Reaper
Recorder RedesScrapy
SBIder RepoMonkey
SEOkicks Ripper
Rogerbot
SEOstats
SISTRIX SISTRIX
SMTBot
SalesIntelligent SalesIntelligent
ScanAlert
ScoutJet
Scrapy
ScreenerBot
ScrepyBot
Searchestate Searchestate
SearchmetricsBot SearchmetricsBot
SentiBot SemrushBot
Siphon SeoSiteCheckup
Site Sucker SiteSucker
SiteLockSpider Siteimprove
Sitebeam Sitevigil
SlySearch Snoopy
SmartDownload SocialRankIOBot
Snake
Sociscraper Sociscraper
Sosospider Sogou web spider
Spammen SpaceBison
Sqlmap SpankBot
Spbot
Sqlworm Sqlworm
Sqworm
Steeler Steeler
Stripper Stripper
Sucuri
SuperBot
SuperHTTP SuperHTTP
Suzuran Surfbot
Swiftbot T0PHackTeam
T8Abot
Teleport Teleport
TeleportPro TeleportPro
Telesoft
Telesphoreo
The Intraformant The Intraformant
Thumbor TheNomad
Toata TightTwatBot
Toweyabot Tracemyfile
Trendictionbot Trendiction
True_Robot
Turingos Turingos
TwengaBot Turnitin
UnisterBot Twice
Upflow Typhoeus
VCI V-BOT
VeriCiteCrawler Vacuum
Virusdie VidibleScraper
Voil
WASALive-Bot
WBSearchBot
WEBDAV WEBDAV
WISENutbot
WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
Web Fetch WWWOFFLE
Web Pix Wallpapers
Wallpapers/3.0
WallpapersHD
WeSEE
Web Auto
Web Collage
Web Fuck
Web Sauger Web Sauger
WebImageCollector WebFetch
WebLeacher
WebWhacker WebWhacker
WebmasterWorldForumBot Webalta
Webshag
WebsiteQuester
Webster Webster
Whack Whack
Who.is Bot Whatweb
Widow WinHTTrack
WiseGuys Robot Wonderbot
Wotbox Wotbox
Xaldon WebSpider Zade
Xenu Zeus
YoudaoBot
ZmEu
ZoomBot ZoomBot
ZyBorg ZoominfoBot
ZumBot
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo.pt
backlink-check autoemailspider
check1.exe cah.io.community
cognitiveseo
crawl.sogou.com
dataforseo.com dataforseo.com
demandbase-bot demandbase-bot
domainsproject.org
eCatch eCatch
evc-batch evc-batch
instabid
isitwp.com
meanpathbot
mediawords
netEstate NE Crawler netEstate NE Crawler
plumanalytics
s1z.ru s1z.ru
scalaj-http
seoscanners
seostar seostar
sexsearcher siteripz
sogouspider spyfu
sp_auditbot tAkeOut
trendiction.com
trendiction.de trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com webmeup-crawler
webpros.com
webprosbot webprosbot
x22Mozilla x09Mozilla
zgrab xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
bbsoldes.fr egvar.net
betterscooter.com hopeonthestreet.co.uk
braintobe.win kuzinsp.ru
fishtauto.ru matsdale.com
onlinemegax.com resultshub-a.akamaihd.net
plastweb.ru sdrescher.net
pornogad.com slavia.info
top10registrycleaners.com tecnoteakviareggio.it
xmlinde.com topsiteminecraft.com
yopoint.in zytpirwai.net

924
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 00go.com
00it.com
1001watch.com.ua
123movies.love
13tabs.com 13tabs.com
14b.info 178evakuator178.ru
1qingdao.com 1millionusd.xyz
1stat.ru 2020iscoming.info
4inn.ru 214jaluwobapef.cf
7makemoneyonline.com 2rich4bitches.com
991mostfm.co.id 2x2fan.ru
a1security.com.ua 4kepics.com
abiente.ru 7a2rnuey1tw9ar.ru
91abcw.com
absugars.com absugars.com
abwa.tk adclickthru.net
actices.com addblueoff.com.ua
adf.ly addray.pro
adidas.frwebs.fr admanaerofoil.com
adloads.com adsland.men
adsfresh.men adsloads.com
adtech.fr adult3dgames.com
adultmeetonline.info
advancedsoftwaresupport.com advancedsoftwaresupport.com
adverster.com
advertisingtag.net
adviceforum.info adviceforum.info
afora.ru adzerg.com
agecheckadult.com adzpower.com
akusherok.ru agroeconom.kz
allboard.xobor.de ahmedabadwebs.com
allmarketsnewdayli.gdn air-edem.ru
alphaforum.ru allforminecraft.ru
am-se.com allornamenti.com
amateurgalls.com altamayoztourism.com
anaksma.info ameblo.jp
analnoeporno.tv analytics-ads.xyz
ananumous.ru apidigidocketnet-a.akamaihd.net
anapa-inns.ru appiq.mobi
animalrank.com applyneedy.xyz
anmysite.com approved.su
apiwoodensealcom-a.akamaihd.net aproposde.com
asdfz.pro artel116.ru
asrv-a.akamaihd.net arvut.org
atley.eu.pn autowebmarket.com.ua
auspolice.com
autotrends.today
avkzarabotok.com avkzarabotok.com
azazaporn.com azarttoto.com
backiee.com baikaleminer.com
balook.com banki76.ru
battleforupdating.site bannerpower.com
bazaronline24.ru bannerswap.com
belstaffstore.online beachfix.co
best-mam.ru beachtoday.ru
bestcalovebracelet.cn becuo.com
bestdooz.com bestadbid.com
bestserials.com bestfortraders.com
bestsexyblog.com bestofupload.info
besttorrentknifta.weebly.com besttorrentknifta.weebly.com
bestwaystogetpaid.us bet-prognoz.com
bestwrinklecreamnow.com bezlimitko.xyz
beta.hotkeys.com beztuberkuleza.ru
bimatoprost-careprost.com bibys.com
biznesrealnost.ru bigcities.org
bizru.info bildsuche.ru
bloggen.be bingoporno.com
blogos.kz bizfly.info
bkns.vn
boanonihaca.tk
board.f00d.de board.f00d.de
boazpower.com bookmark4you.com
bombla.org braintobe.win
bonzybuddy.com bravegirlsclub.com
boo-arts.com brg8.com
brandov.ru bristolhotel.com.ua
brateg.xyz
break-the-chains.com
bronzeaid-a.akamaihd.net
brus-vsem.ru brus-vsem.ru
bscodecs.com burger-tycoon.com
btnativenav.com business-made-fun.com
buehne-fuer-menschenrechte.de buttons-for-website.com
bugshoot.cn
burn-fat.ga
buypuppies.ca buypuppies.ca
buyscabiescream.com buzzurl.jp
bycontext.com bvlgariwallet.xyz
callawaygolfoutlet.online byme.se
canrole.com cartierbraceletsreplica.pw
cartierreplica.xyz cartierreplica.xyz
catalogs-parts.com cdn.walkme.com
cdncash.com cdncash.com
centraletermice.us cfacarrosserie74.com
cerev.info cheapjerseysa.com
cheapcigarettesc.info cheapwebsitehoster.com
chiblackhawks-jerseys.com chimiver.info
chocoslim.pro chocolateslim-original.com
cialischsrx.com cleanersoft.com
clayvasedesigns.tk clickhype.com
click2pawn.com clickso.com
clknsee.com
cmhomestayagency.com cmhomestayagency.com
codysbbq.com cobaltpro.ru
compliance-jane.top coldfilm.ru
compliance-jessica.top com-onlinesupport.site
compliance-john.top companies-catalog.com
computernetworksonline.com compliance-ivan.top
cookie-law-enforcement-ee.xyz compliance-josh.top
coolgamechannel.com compliance-olga.top
costablue.xyz compliance-walter.top
cougfan.info compliance-willy.top
covetnica.com converse.ddsoldes.fr
cracksplanet.com credit-online.ws
cupday.com czeshop.info
customchocolate.business-for-home.com danceuniverse.ru
customcollegeessays.net darmebel.com.ua
dairyindia.in dating-time-now.com
daneshetabiat.com deanmoore.ie
data-mining.tk delayreferat.ru
ddpills.com delfin-aqua.com.ua
deluxedumps.com delta-line.men
demon-tweeks.com diarioaconcagua.com
designdevise.com dickssportinggoods.com
detskie-zabavi.ru
diminishedvalueoforegon.com diminishedvalueoforegon.com
distver.ru dimkino.ru
dividendo.ru divku.ru
docstoc.com documentbase.net
doreenblog.online documentsite.net
dostavka-v-ukrainu.ru domcran.net
download-of-the-warez.blogspot.com downloadmefiranaratb1972.xpg.com.br
downloads-whatsapp.com dprtb.com
drpornogratisx.xxx drugspowerstore.com
drunkenstepfather.com duplicashapp.com
drupa.com ed-shop01.ru
dstroy.su edudocs.net
e-poker-2005.com elektrischezigaretten1.blogse.nl
eaplay.ru elektronischezi.livejournal.com
earthmagic.info elektronischezigarettekaufen2.tumblr.com
editmedios.com elektrozigarette2.wordpressy.pl
eduserver.net elexies.info
ekosmetyki.net.pl elidelcream.weebly.com
el-nation.com embedle.com
elektrischezigarette1.blog.pl energydiet24.ru
eng-lyrics.com
enge-fotzen.info enge-fotzen.info
ereko.ru erotik0049.com
eropornosex.ru escortslet.net
estibot.com esfchat.tk
everypony.ru eu-cookie-law.info
exchanges-bet.com executehosting.com
ez8motelseaworldsandiego.com expertnaya-ocenka.ru
ezigarettekaufen.myblog.de eyelike.com.ua
ezigaretteshop.webs.com eyessurgery.ru
facecup.top ezigarettenkaufen2.dreamwidth.org
ezigaretteshop2.vefblog.net
ezrvrentals.com
f1nder.org
failingmarriege.blogspot.com failingmarriege.blogspot.com
falcon-images.blogspot.com familienzahnaerzte.com
famix.xyz fatmaelgarny.com
farmprofi.net feargames.ru
fatfasts-4tmz.com femdom.twiclub.in
fdzone.org
fealq.com
fertilitetsradgivningen.se
ff30236ddef1465f88547e760973d70a.com ff30236ddef1465f88547e760973d70a.com
fickenbumsen.net filesdatabase.net
fickluder69.com finansov.info
filesclub.net findanysex.com
filmetricsasia.com flytourisme.org
filunika.com.ru food.dtu.dk
firma-legion.ru fordsonmajbor.cf
focalink.com
forex.osobye.ru
forexgb.ru
forexunion.net forexunion.net
forminecrafters.ru forum20.smailik.org
forums.toucharcade.com
foxinsocks.ru foxinsocks.ru
fr.netlog.com foxtechfpv.com
freetangodownload.com frankofficial.ru
freeuploader.ml free-video-tool.com
freevpn.space freenom.link
fres-news.com freewebs.com
fuckingawesome.com freshsuperbloop.com
fullfileaccess.com frighteningremain.cf
galeon.com frvo.alptandem.ru
gamevalue7.weebly.com fsakhalin.ru
gaypornmovie.net fuel-gas.com
gaytube.com funnymama.com
geilehausfrauen.net fxgallery.com
gentamicineyedrops.blogspot.com gazobeton-p.com.ua
getlamborghini.ga ge0ip.org
gilbertbanda.net geile-lelly.eu
global-ics.co.za gemara.com
globalscam.ga gettpromos.com
gogps.me getyourimage.club
gotorussia.com gg-arena.ru
gratuitbaise.com gigapeta.com
gribkovye-zabolevaniya.com glavtral.ru
gribokstop.com glicol.kz
guardlink.org goatse.ru
hasshe.com gojiberriess.apishops.ru
hdfreeporno.net gomusix.com
hdseriale.pl goodnightjournal.com
healbio.ru gov.yanao.ru
hobbyhuren24.net greenzaim.ru
homeandhealth.ru gsasearchengineranker.xyz
hornymatches.com gta-top.ru
hothor.se gymi.name
hoverboard360.se habermetre.com
hplaserjetpdriver8y.pen.io handicapbathtubarea.com
hreade.com hao123.com
hspline.com hatdc.org
ideashome.id hatedriveapart.com
ihtec2019.org havepussy.com
imagui.eu hchha.com
img.wallpaperstock.net hd-film.pl
impresagaia.it hermesreplica.win
in-tandem.co heroz.fr
intervsem.ru hexpilot.com
invest-pamm.ru himazin.info
ip-guide.com histats.com
hjaoopoa.top
hledejvshopech.cz
horoshieokna.com
hosting-tracker.com
hostsshop.ru
hotcore.info
hotenergy.ru
housediz.com
houseofrose.com
i-service.kz
ideawheel.com
ideibiznesa2015.ru
igru-xbox.net
igtools.club
impisr.ru
inboundlinks.win
inet-traffic.com
infodocsportal.com
interesnie-faktu.ru
interior-stickers.ru
invitefashion.com
iqoption.com
istanbulit.com istanbulit.com
izoll.ru itis4you.com
jacago.com ixora.pro
james13prix.info izhevsk.zrus.org
jamiembrown.com izismile.com
jmat.cn janettabridal.com
jolic2.com japfm.com
journalhome.com jav-way.site
jovencitas.gratis javstock.com
juliaworld.net jerseyswholesalechinalimited.com
jumptap.com jestr.org
jwcialislrt.com jetsli.de
kandidos.com jpcycles.com
karachev-city.ru jwss.cc
karusel-market.ru karadene.com
keywordbasket.com karpun-iris.ru
klin3952.ru kasino-money.pw
knowsitall.info katadhin.co
koronirealestate.gr kazan.zrus.org
krynica.info keywordsdoctor.com
keywordsuggest.org
keywordsuggests.com
kihi.gdn
kinoduh.ru
kinohit1.ru
kinoplen.ru
kliksaya.com
kmd-pto.ru
kol-energo.ru
konpax.com
kotaku.com
kovesszucs.atw.hu
krasivye-devushki.net
ku6.com ku6.com
labafydjxa.su kvartiry-remont.ucoz.ru
lacapilla.info
lacasamorett.com lacasamorett.com
lafriore.ru ladsup.com
lalalove.ru lanadelreyfans.us
landscaping.center languagecode.com
laserpen.club lapitec.eu
lazy-z.com larose.jb2c.me
ldrtrack.com lawyers.cafe
lechtaczka.net lechtaczka.net
letsrepair.in leftofcentrist.com
lider-zhaluzi.kiev.ua letolove.ru
lidoradio.com life.biz.ua
linerdrilling.com liffa.ru
linkpulse.com lion.misba.us
linkredirect.biz littlecity.ch
linkrr.com live-sexcam.tk
liquimondo.com love-baby.cz
lisque.batcave.net lsp-awak-perikanan.com
liupis.com lulea-auktionsverk.se
livejasmin.com lumb.co
lkbennettstore.online macfix.life
localflirtbuddies.com makenahartlin.com
locustdesign.co malwareremovalcenter.com
lomb.co mansparskats.com
lovelycraftyhome.com marcoislandvacations.net
low-format.ru marketingvici.com
ltvperf.com massage-info.nl
m292.info masterseek.com
magnetpress.sk masturbate.co.uk
mahnwachen-helfen.info matematikus.info
mail.allnews24.in medanestesia.ru
make-money-online.com meds-online24.com
marinetraffic.com mega-polis.biz.ua
marketland.ml menhealed.net
matsdale.com mere.host.sk
maximilitary.ru merryhouse.co.uk
mebeldekor.com.ua
med-zdorovie.com.ua
meddesk.ru
mediaoffers.click
medictube.ru
mejoresfotos.eu
metarip.ru metarip.ru
minyetki.ru milblueprint.com
misandesign.se mindbox.co.za
mlvc4zzw.space minneapoliscopiers.com
mmog-play.ru missis.top
mobplayer.net mlf.hordo.win
moje-recenze.cz mmostrike.ru
moscow-clining.ru mnogolok.info
moesen-ficken.com
mompussy.net
moomi-daeri.com
morpicert.pw
mouselink.co
movpod.in movpod.in
mp3ritm.top moyakuhnia.ru
mriyadh.com msk-diplomat.com
mrwhite.biz mydirtystuff.com
mttwtrack.com nailsimg.com
my-big-family.com nash-krym.info
myfreetutorials.com nastydollars.com
myplaycity.com
myxdate.info
naval.jislaaik.com
netoil.no netoil.no
nisuturnetdgu.tk netpics.org
notebook-pro.ru newsperuse.com
nvssf.com nfvsz.com
o-o-8-o-o.ru nhl17coins.exblog.jp
o00.in oakridgemo.com
oakleyglassesonline.us obsessionphrases.com
obiavo.su ochistka-stokov.ru
obnal.org oivcvx.website
oil-td.ru ok-ua.info
okuos.com okuos.com
online-hd.pl olvanto.ru
onlinebay.ru olympescort.com
onlinemeetingnow.com one-gear.com
onemantrip.com
onlinefilmz.net
ops.picscout.com
oralsexfilme.net
orel-reshka.net
orhonit.com
ownshop.cf
oynat.info oynat.info
paceform.com paparazzistudios.com.au
panicatack.com pavlodar.xkaz.org
par-fallen.ga
partnerworkroom.men
patterntrader-en.com
pay2me.pl pay2me.pl
pdamods.ru pcads.ru
petedrummond.com penisvergrotendepillennl.ovh
picturesboss.com perl.dp.ua
picurams.pw phrcialiled.com
pisanieprac.info picscout.com
platezhka.net picture-group.com
play-movie.pl pijoto.net
poptool.net piulatte.cz
poquoson.org piuminiita.com
porndairy.in pl-top.pl
pornobrazzers.biz pl-vouchers.com
pornogratisdiario.com plugingeorgia.com
pornohub.me polybuild.ru
pornoklad.ru pooleroadmedicalcentre.co.uk
pornokorol.com porno-raskazy.ru
pornosemki.info pornokajf.com
pornosexrolik.com poshiv-chehol.ru
porto.abuilder.net
postclass.com postclass.com
powenlite24.ru
presleycollectibles.com
pretty-mart.com pretty-mart.com
professionaldieselcare.com printingpeach.com
profitkode.com prizestohandle.club
progonrumarket.ru prod2016.com
promalp-universal.ru profolan.pl
promgirldresses.xyz ptr.ruvds.com
prostitutki-kharkova.org pyramidlitho.webs.com
proxyelite.biz qitt.ru
pufip.com qld10000.net
pussyspace.net ragecash.com
qexyfu.bugs3.com refads.pro
quelle.ru
quickbuck.com
radiodigital.co
ranksays.com
rapevideosmovies.com
reckonstat.info
redirectme.net
redirlock.com
reelheroes.net
refererx.com
regdefense.com
registry-cleaner.net registry-cleaner.net
registrydomainservices.com remorcicomerciale.ro
relatodelpresente.com.ar rff-cfal.info
remont-ustanovka-tehniki.ru rixpix.ru
resellerclub.com rn-to-bsn.com
riders.ro roleforum.ru
ringtonepartner.com rubanners.com
risparmiocasa.bz.it runofilms.ru
ru-dety.ru runstocks.com
rumamba.com saratov.xrus.org
s.lollypopgaming.com sarm.tk
sad-torg.com.ua sashagreyblog.ga
sanatorrii.ru sc-specialhost.com
scanmyphones.com scanner-jess.top
scanner-jane.top scanner-josh.top
schalke04fc.info scanner-julianna.top
school-diplomat.ru scrapy.org
sdrescher.net sdelai-prosto.ru
searchimage.co securesmrt-dt.com
seo-2-0.com seeingmeerkat.com
sergiorossistore.online servethis.com
sevendays.com.ua service-core.ru
servicecenter.co.ua
seryeznie-znakomstva.ru
sex-pr.net
sexad.net sexad.net
sexpornotales.net sexblog.pw
sherlock.se share-buttons-for-free.com
simplesite.com skanninge.se
siteaero.com skypasss.com
sitebeam.net social-buttons.com
sitechecker.pro sohoindia.net
siteheart.net sonata-arctica.wz.cz
sitiz.club spacash.com
slowmac.tech space4updating.win
small-game.com speeddream.xyz
smartpet.ru srgwebmail.nl
soblaznu.net stard.shop
social-s-iii.xyz statoutlook.info
soft1.ru stauga.altervista.org
sotkal.lark.ru steamoff.net
spammen.de stmassage.ru
spidtest.space stpicks.com
stockquotes.wooeb.com stroy-portal22.ru
strana-krasoty.ru stroymonolit.su
stroicol.net studworks.org
stroiminsk.com svtrd.com
sucsesofinspiration.com sweepstakes.rewardit.com
sudexpert66.ru szamponrevita.pl
svensk-poesi.com t-bygg.com
svetodiodoff.ru talant-factory.ru
tacbelarus.ru tasteidea.com
tako3.com tastyfoodideas.com
tattooha.com tdsing.ru
thecoolimages.net teastory.co
thedownloadfromwarez.blogspot.com theporndude.com
thegamerznetwork.com thewomenlife.com
theguardlan.com thexart.club
thenews-today.info thin.me.pn
thesmartsearch.net timdreby.com
tiandeural.ru
timetorelax.biz
titelhelden.eu
titslove.yopoint.in titslove.yopoint.in
tizanidinesideeffectsweightloss.blogspot.com tizanidinehclsideeffects.blogspot.com
top10-way.com tizanidinerestlesslegsyndrome.blogspot.com
topcasinoratings.ru tmtrck.com
topsiteminecraft.com tobeyouday.win
totu.us traffic2cash.org
trackzapper.com traffictrade.life
traderzplanet.in trion.od.ua
traffic2money.com trumpetedextremes.com
trafficjunky.com trustedmaccleaner.com
trafficjunky.net turizmus.us
trafficshaper.com uchetunet.su
trustl.life ugguk.online
tsstcorpcddvdwshbbdriverfb.aircus.com ukrtextbook.com
tx41tclega.ru uptime-delta.net
u17795.netangels.ru uroki.net
uggsale.online usadacha.net
ukrtvory.in.ua v-doc.co
universals.com.ua valmetrundan.se
unlimitdocs.net
uogonline.com
upstore.me
urccvfmc.bloger.index.hr
usacasino.com
valsalud.com valsalud.com
vandrie-ict.nl vapsy.com
vashsvet.com velen.io
vereo.eu veles.shop
versaut.xxx-cam.webcam vesnatehno.ru
vezuviy.su viagraneggrx.com
videos-for-your-business.com vip-dom.in
videositename.com vip.51.la
vipps.com.my virtualbb.com
vita.com.hr vladhistory.com
vitanail.ru vmnmvzsmn.over-blog.com
vod.com.ua vodaodessa.com
voloslove.ru vseigry.fun
vseuznaem.com vzlomsn.org
w3javascript.com watracker.net
walpaperlist.com webalan.ru
web-betting.ru weblibrary.win
website-speed-up.site website-speed-up.site
websitevaluebot.com wemarketing.se
weburok.com wildworld.site
wineitudes.wordpress.com wineration.com
winterclassichockeyjerseys.com winwotgold.pl
worldis.me woman-orgasm.ru
wurr.voila.net wowas31.ucoz.ru
wyniki-lista.pl wufak.com
x-musics.com www.labves.ru
xerox-douglas.cf xchangetrak.com
xev.ru xjrul.com
xfire.com xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xn--80ahdheogk5l.xn--p1ai
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai xn--80aodinpgi.xn--p1ai
xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xn--80ahvj9e.xn--p1ai
xn--b1ag5cfn.xn--p1ai xn--b1ag5cfn.xn--p1ai
xtrafficplus.com xn--d1acah0c.xn--p1ai
yachts-cruise.info xn--h1ahbi.com.ua
xrus.org
xxxguitars.com
xxxhdvideo.site
yadro.ru
yaoguangdj.com
yogamatsexpert.com
yopoint.in yopoint.in
youraticles.pl youandcredit.ru
youresponsive.com youghbould.wordpress.com
yourporngay.com youradulthosting.com
yuarra.pluto.ro zarabotat-na-sajte.ru
zakazvzloma.com zdesoboi.com
zarenica.net zeroredirect10.com
zazagames.org zeroredirect11.com
zerocash.msk.ru zhongwenlink.com
znaturaloriginal.com zhuravlev.info
zog.link zmoda.hostreo.com
zoogdinsney.com zygophyceous.womanstars.site
zrizvtrnpale.tk
zskdla.site
zx6.ru

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1flag.co.za 12bet.com
1stat.ru 178evakuator178.ru
2daytrendingnews.com 1hwy.com
4kplayer.pl 1xbet4you.com
a14download.com 24x7-server-support.site
aadroid.net 55wmz.ru
aaronabel.com 76brighton.co.uk
addblueoff.com.ua 999webdesign.com
akita.kz aasoldes.fr
alarmobninsk.ru aghanyna.com
albuteroli.com allboard.xobor.de
alta-realestate.com amateurgalls.com
angel1777.kz animal36.com
animaltoplist.com appiq.mobi
anticrawler.org apps-analytics.net
apiadanaknet-a.akamaihd.net architecturebest.com
approvedlocal.co.za arendakvartir.kz
arewater.com arkkivoltti.net
asscenihotosidea.blogspot.com arraty.altervista.org
ausergrubhof.info autorn.ru
auto-moto-elektronika.cz availit.weebly.com
autotop.com.ua aviav.eu
avirasecureserver.com azartmsl.com
avtochehli.by
azbaseimages.net
babosas.com babosas.com
babs.com.ua balois.worldbreak.com
bannerbank.ru betteroffers.review
barbourjackets.online bigames.online
beauty-bracelet.com bioca.org
bedcapdealers.com bluejays-jerseys.us
best-seo-offer.com bobba.dzaba.com
bet-prognoz.com
blog-hits.com
boanonihaca.tk
bobinoz.com bobinoz.com
boc.kz bocoarchives.org
book-bets.com bookhome.info
brainboosting.club business-suggested.tk
brains2.biz bvlgariring.xyz
braintobe.win cartierlove2u.com
buqayy0.livejournal.com castingbank.ru
buyk.host.sk cbox.ws
buypanicdisorderpill.com cdncash.net
callejondelpozo.es certifywebsite.win
candlewooddentalcentre.com.au cfsrating.sonicwall.com
cardsharp1.ru cgi2.nintendo.co.jp
cdncash.org cheap-trusted-backlinks.com
ceotrk.com cheapjerseysap.com
chocolateslim-u-srbiji.com checktext.org
club-musics.ru clickzzs.nl
corta.co comissionka.net
custom-electric-guitar.com compliance-jack.top
customchocolate.business-for-home.com compliance-jess.top
detective01.ru connectingsingles.com
dildofotzen.net cpabegins.ru
creditmoney.com.ua
cypernhuset.se
danashop.ru
date-withme.com
decorazilla.com
delfin-aqua.com.ua
descargar-musica-gratis.net
directrev.com directrev.com
domaincheck.io diy-handmade-ideas.com
download-wallpaper.net doctormakarova.ru
dprtb.com dostavimvdom.ru
drhomes.biz downloader12.ru
dtm-spain.com dyshagi.ru
e-buyeasy.com ekspertmed.com
educontest.net elektroniksigaraankara.info
empirepoker.com elite-sex-finders.com
erotische-geschichten-xxl.com erotikstories.ru
eyelike.com.ua esopini.com
facialporntube.com excitacao.com
fanpagerobot.com eywords-monitoring-your-success.com
farmprofi.net ezaz.info
ferieboligkbh.dk fastcrawl.com
filesmonster.porn fickenprivat.info
filkhbr.com flirt4free.com
fillmewithhappiness.com flytourisme.org
firma-legion.ru focalink.com
flauntyoursite.com forensicpsychiatry.ru
freza-sverlo.ru forumprofi.de
fyl.com.ru free-deals.faith
gameonasia.com free-traffic.xyz
getlamborghini.ga fullfileaccess.com
getyourimage.club gallerily.com
ghazel.ru gamerextra.com
gheus.altervista.org gaygalls.net
girlgamerdaily.com ghernnqr.skyrock.com
given2.com gidonline.one
glcomputers.ru goatse.ru
good-mummy.ru golden-praga.ru
goodprotein.ru goodwinmetals.co
gourcy.altervista.org grupografico-pilar.com.ar
happy.new.yeartwit.com hamacapty.com
home-task.com hchha.com
hopeonthestreet.co.uk hebr.myddns-flir.com
horoshieokna.com hem.passagen.se
hotkeys.com hobbyhuren24.net
hreade.com hut1.ru
hubbble.com hyipmanager.in
iinstalll-fii1leis.jus0wil.pp.ua ideashome.id
images-graphics-pics.com ilovevitaly.com
incanto.in.ua imagui.eu
indetiske.ya.ru indiasourcemart.in
internetartfair.com infobanks.ru
istripper.com infospot.pt
javstock.com infostatsvc.com
jolic2.com investingclub.ru
kazan.xrus.org istizanidineanarcoticdrug.blogspot.com
ketoanhanoi.info jabimgo.pw
keywordglobal.co.za janavibekken.no
keywords-monitoring-your-success.com jav-fetish.site
keywordsuggest.org jestr.org
klosetkitten.com jillepille.com
joessmogtestonly.com
joy-penguin.com
kaac.ru
kabbalah-red-bracelets.com
karpun-iris.ru
kinky-fetishes.com
kinostorm.net
kiwi237au.tk
kollekcioner.ru
kosova.de
koversite.info koversite.info
kumuk.info krasnodar.ru
kvartiry-remont.ucoz.ru kredit-blog.ru
linksharingt.com ktotut.net
liupis.com kunstaktien.info
longgreen.info kw21.org
m.b00kmarks.com lakomka.com.ua
mabdoola.blogspot.com larutti.ru
macdamaged.tech libertybilisim.com
macotool.com life.biz.ua
marwer.info livejasmin.com
maxthon.com liveshoppersmac.com
medicine-4u.org louisvuittonoutletstore.net
meetlocalchicks.com luciddiagnostics.in
mnogabukaff.net lyrics.home-task.com
macrotek.ru
magento-crew.net
mansparskats.com
manuscript.su
mature.free-websites.com
microsoftportal.net
mielec.pl
miniature.io
mnogolok.info
mobicover.com.ua
modabutik.ru
moivestiy.biz moivestiy.biz
mokrayakiska.com moncleroutletonline.win
mompussy.net moneyzzz.ru
mountainstream.ms mossmesi.com
muz-tracker.net mrcsa.com.au
mydearest.co mttwtrack.com
naj-filmy24.pl myanyone.net
mylesosibirsk.ru
mzdish.site
najaden.no
nalogovyy-kodeks.ru
narkologiya-orel.ru narkologiya-orel.ru
nastydollars.com nash-krym.info
news-readers.ru netoil.no
nextconseil.com netvouz.com
nightvision746.weebly.com nhl09.ru
niki-mlt.ru obiavo.su
niroo.info omgtnc.com
nodup.ru onblastblog.online
o-o-6-o-o.com onload.pw
oneminutesite.it owathemes.com
online-templatestore.com ownshop.cf
oops-cinema.ru pammik.ru
outpersonals.com partner-print.men
pechikamini.ru partner-trustworthy.men
pertlocogasilk.tk partnerads.men
petrovka-online.com pics-group.com
photo-clip.ru pihl.se
pic-re.blogspot.com pinsdaddy.com
pio.polytopesexempt.com pogodnyyeavarii.gq
pornhub-forum.ga pornolook.net
porno-home365.com pourvous.info
pornogratisdiario.com preparevideosafesystem4unow.space
pornpost.in prlog.ru
positive2b.ru professionalwritingservices15.blogspot.ru
pozdravleniya-c.ru pskcijdc.bloger.index.hr
privatefx-in.ru rabotaetvse.ru
prodess.ru reelheroes.net
professionaldieselcare.com remont-comp-pomosh.ru
promover.org renecaovillasale.online
rainbowice.ru rentehno.ru
ranksignals.com rospromtest.ru
rarbg.to rucrypt.com
rebelmouse.com rvi.biz
recordpage-a.akamaihd.net sa-rewards.co.za
registrysweeper.com sack.net
reimageplus.com safe-app.net
remont-fridge-tv.ru sajatvelemeny.com
riralmolamsaca.tk saletool.ru
robot-forex.biz santasgift.ml
rukino.org scanner-alex.top
russian-postindex.ru scanner-julia.top
sankt-peterburg.nodup.ru scanner-willy.top
sanyuprojects.com scanspyware.net
sbricur.com sdelai-prosto.ru
seasonvar.ru seccioncontrabajo.com
seemoreresultshu-a.akamaihd.net service-core.ru
seinterface.com sexxdate.net
seksotur.ru shemale-sex.net
semalt.com shikiso.info
semrush.com sientalyric.co
seotoolsagency.com site3.free-share-buttons.com
serviporno.com skyway24.ru
sexy-pings.com snts.shell-pmr.ru
site-speed-checker.site spb.afora.ru
skylta.com stauga.altervista.org
smart-balancewheel.com stmassage.ru
smartadserver.com stopnarco.ru
smsactivator.ru stpicks.com
snjack.info tizanidinestreetprice.blogspot.com
social-fun.ru tongkatmadura.info
socialtrade.biz tula.mdverey.ru
soda.media turist-strani.ru
sortthemesitesby.com txxx.com
spamnuker.com ussearche.cf
speechfoodie.com v24s.net
spitfiremusic.com vchulkah.net
spy-sts.com viagra-soft.ru
spywarenuker.com videochat.cafe
starpages.net videochat.mx
startwp.org videochat.ph
steamoff.net vizitki.net
sugarkun.com
suhanpacktech.com
supernew.org
superstarfloraluk.com
superstats.com
supervesti.ru
tacbibirfa.tk
tamada69.com
templates.radiodigital.co
tetracsaudi.com
thebluenoodle.com
thefotosgratis.eu
thexart.club
topbestgames.com
tozup.com
tripper.de
tupper-shop.ru
tx41tclega.ru
uac.net.au
ultimateclassicrock.com
unionmarkt.de
vabasa.inwtrade.com
viel.su
vielporno.net
vinsit.ru
vkontaktemusic.ru vkontaktemusic.ru
vodkoved.ru voprosotvet24.ru
vriel.batcave.net warning.or.kr
websiteexplorer.info webalan.ru
wellcome2slovenia.ru weprik.ru
whereverdesperate.gq westen-v.life
wholesalejerseysgaa.com whatistizanidinehclusedfor.blogspot.com
wieseversa.no
wovis.site wovis.site
wpsecurity.website ww2awards.info
writingservices17.blogspot.ru www.bookmaker-bets.com
www.solartek.ru x-mix.info
xmronta.com xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn--80aikhbrhr.net xtube.com
xn--oogle-wmc.com yourporngay.com
xxxkaz.org ytmnd.com
youjizz.com yugk.net
yourmovies.pl zarenica.net
youtube-downloader.savetubevideo.com
yuweng.info
zdesoboi.com zdesoboi.com
zscalerone.net zigarettenonlinekaufen2.bloog.pl
zvuker.net

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2677 ### Version: V4.2021.10.2678
### Updated: Wed Oct 27 10:12:33 UTC 2021 ### Updated: Wed Oct 27 22:03:11 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2677 ### Version: V4.2021.10.2678
### Updated: Wed Oct 27 10:12:33 UTC 2021 ### Updated: Wed Oct 27 22:03:11 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##