vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 15:34:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.03.2991 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-03-20 22:07:07 +00:00
parent 3407b5c051
commit 91440f448c
47 changed files with 95371 additions and 95381 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.03.2990 #### Version: V4.2022.03.2991
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.03.2990 #### Version: V4.2022.03.2991
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.03.2990 #### Version: V4.2022.03.2991
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11784
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.03.2990 #### Version: V4.2022.03.2991
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11788
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11788
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11788
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11788
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.71 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.71 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.71 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.71 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.71 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.71 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2990 2991

11788
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11788
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.102.122.225 0; 40.112.145.61 0;

11788
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.71 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.71 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.71 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.71 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.71 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.71 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.102.122.225 0; 40.112.145.61 0;

11788
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.102.122.225 0; 40.112.145.61 0;

11788
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 1; 10.1.0.71 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Alexibot BuzzSumo
Demon CyotekWebCopy
HybridBot Dirbuster
Image Fetch Offline Explorer
InfoTekies PageAnalyzer
SEOkicks SemanticJuice
Screaming Sitebeam
SputnikBot URLy Warning
VelenPublicWebCrawler WebEnhancer
ZyBorg YoudaoBot

288
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404enemy 360Spider
ALittle Client 404checker
Abonti ADmantX
ASPSeek
Aboundex
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot
Alexibot Alexibot
AllSubmitter AllSubmitter
Anthill Anarchy99
Apexoo Aspiegel
Atomseobot BDCbot
BackDoorBot BLEXBot
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler Badass
Barkrowler Bandit
Battleztar Bazinga Battleztar Bazinga
Bitacle Blow
BlackWidow
BlowFish BlowFish
Boardreader
BotALot BotALot
BuiltBotTough BuiltBotTough
CODE87 BunnySlippers
Calculon CCBot
CheTeam
CherryPicker CherryPicker
ChinaClaw
Chlooe Chlooe
Citoid Collector
Cliqzbot Craftbot
Copier Crawling at Home Project
Cosmos CrunchBot
CrazyWebCrawler Curious
Crescent Custo
DIIbot
DSearch DSearch
DataCha0s DTS Agent
DatabaseDriverMysqli
Demon Demon
Disco Discobot
Discoverybot
DomCopBot
DomainAppender
DomainCrawler DomainCrawler
Dotbot DomainSigmaCrawler
Ebingbong
EroCrawler
Evil Evil
Exabot Exabot
Express WebPictures
ExtractorPro ExtractorPro
EyeNetIE EyeNetIE
Ezooms FDM
FlashGet FHscan
Flunky GT::WWW
Foobot GoZilla
FrontPage
FyberSpider
GalaxyBot
GermCrawler
GetRight
Getintent
Go-Ahead-Got-It
Gotit
GrabNet GrabNet
Grabber Grabber
GrapeFX GrapeFX
GrapeshotCrawler
GridBot GridBot
HMView
HTMLparser HTMLparser
HTTP::Lite
HaosouSpider
Havij
Heritrix Heritrix
Hloader HonoluluBot
HybridBot HybridBot
IDBot
IRLbot
Iblog Iblog
Id-search Id-search
IlseBot Image Fetch
IndeedBot Image Sucker
InterGET
Internet Ninja
InternetSeer InternetSeer
Iria IstellaBot
Jbrofuzz JennyBot
JetCar
Joomla Joomla
Jorgee Jorgee
JustView JustView
Jyxobot Keyword Density
Kinza Kinza
LNSpiderguy
Leap Leap
Lftp LeechFTP
LibWeb LeechGet
LexiBot
LieBaoFast
Likse Likse
LinkScan LinkScan
LinkWalker LinkextractorPro
Linkbot
Linkdexbot
LinkpadBot
LinksManager LinksManager
LinqiaRSSBot Lipperhey
LinqiaScrapeBot Litemage_walker
Lipperhey Spider Lmspider
MIDown tool
MIIxpc
MJ12bot
MSFrontPage MSFrontPage
MSIECrawler MSIECrawler
Mail.RU_Bot
Majestic SEO
Majestic-SEO
Majestic12
MarkMonitor
Mass Downloader Mass Downloader
Masscan Mb2345Browser
Meanpathbot
Metauri
Minefield
Moblie Safari Moblie Safari
MolokaiBot Mojeek
NPbot Mr.4x3
Name Intelligence Msrabot
Navroad Nameprotect
NearSite
Needle Needle
Net Vampire Net Vampire
NetAnts NetAnts
NetLyzer NetMechanic
NetSpider
Netcraft
Netvibes Netvibes
NextGenSearchBot Nibbler
Nimbostratus Nikto
Octopus Nmap
Nuclei
Offline Explorer Offline Explorer
Offline Navigator
Openfind Openfind
OrangeBot OrangeSpider
OutclicksBot
PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageGrabber
PageScorer
Pandalytics Pandalytics
Panscient
Papa Foto Papa Foto
Petalbot PeoplePal
Pi-Monster Pi-Monster
Picscout
Picsearch
Piepmatz Piepmatz
Pimonster
Pixray Pixray
PleaseCrawl
ProPowerBot
Probethenet Probethenet
Pu_iN PxBroker
Pump
PyCurl PyCurl
Quick-Crawler
RankActive
RankActiveLinkBot
RankFlex RankFlex
Rankivabot RankingBot
RankurBot RankingBot2
RealDownload Reaper
RebelMouse
Recorder Recorder
RedesScrapy RedesScrapy
Ripper Ripper
Rogerbot Rogerbot
SEOkicks SEOkicks
SEOlyticsCrawler
SEOstats SEOstats
SISTRIX SMTBot
ScanAlert ScanAlert
Scanbot Scanbot
ScoutJet
Scrapy
Screaming Screaming
ScrepyBot
Seekport
Semrush Semrush
SemrushBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz
Shodan Shodan
SiteExplorer Site Sucker
SiteLockSpider SiteCheckerBotCrawler
Sitebeam SiteSnagger
SiteSucker
Siteimprove Siteimprove
SlySearch Sitevigil
Snake SocialRankIOBot
Spammen
SpankBot SpankBot
Spanner
SputnikBot SputnikBot
Sqworm Stripper
Sucker Sucuri
SuperBot SuperHTTP
Surfbot
Suzuran
Swiftbot
Szukacz Szukacz
The Intraformant T0PHackTeam
Teleport
TeleportPro
TheNomad
Thumbor Thumbor
TightTwatBot TightTwatBot
Titan Titan
Toata Trendiction
Turingos Trendictionbot
Turnitin Turnitin
TurnitinBot TurnitinBot
TwengaBot
Typhoeus
URLy Warning
URLy.Warning URLy.Warning
UnisterBot UnisterBot
V-BOT
VB Project VB Project
Virusdie VCI
Vacuum
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
VoidEYE
WBSearchBot
WEBDAV
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
Wallpapers/3.0 Wallpapers/3.0
Web Collage Web Auto
Web Enhancer Web Sauger
Web Fetch
Web Pix
WebAuto WebAuto
WebBandit WebBandit
WebCopier WebCollage
WebEnhancer WebEnhancer
WebFetch WebFetch
WebLeacher WebSauger
WebSucker WebSucker
WebWhacker
WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
WebsiteExtractor Webster
WebsiteQuester
Whack
Whacker
Who.is Bot
WinHTTrack WinHTTrack
WiseGuys Robot WiseGuys Robot
Wotbox Wonderbot
Xaldon WebSpider Wprecon
Xaldon_WebSpider Xaldon_WebSpider
Xenu
YoudaoBot YoudaoBot
Zermelo Zade
Zitebot
ZmEu
ZumBot ZumBot
archive.org_bot
arquivo-web-crawler
arquivo.pt arquivo.pt
cah.io.community autoemailspider
check1.exe clark-crawler
com.plumanalytics
crawl.sogou.com crawl.sogou.com
crawler4j crawler4j
dataforseo.com
evc-batch evc-batch
facebookscraper heritrix
instabid
internetVista monitor
isitwp.com isitwp.com
iubenda-radar magpie-crawler
lwp-trivial
meanpathbot meanpathbot
mediawords mediawords
netEstate NE Crawler netEstate NE Crawler
page scorer oBot
pcBrowser pcBrowser
probe-image-size s1z.ru
scalaj-http scalaj-http
seocompany.store seocompany.store
seoscanners
sitechecker.pro
siteripz
sp_auditbot
sysscan
trendiction.com trendiction.com
trendiction.de
voyagerx.com
webmeup-crawler
webpros.com
x22Mozilla x22Mozilla

334
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 360Spider
AIBOT 80legs
Abonti ADmantX
Aboundex Acunetix
AfD-Verbotsverfahren
AiHitBot
Aipbot Aipbot
AllSubmitter Alligator
AlphaBot
Anarchie
Anarchy99
Anthill Anthill
Asterias Asterias
Atomseobot Attach
AwarioRssBot
AwarioSmartBot
BDCbot BDCbot
BDFetch
BLEXBot BLEXBot
BackDoorBot BackDoorBot
BackStreet BackStreet
BackWeb
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Bandit Bandit
Barkrowler BatchFTP
Battleztar Bazinga
BetaBot BetaBot
BlackWidow Bigfoot
BotALot Bitacle
Brandprotect Black Hole
Blackboard
BlowFish
Boardreader
Buddy
BuiltBotTough BuiltBotTough
BunnySlippers BunnySlippers
BuzzSumo CODE87
CSHttp Calculon
CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect
CheeseBot CheeseBot
CherryPicker CherryPicker
ChinaClaw Cliqzbot
Chlooe Copier
Citoid Cosmos
Claritybot
Cogentbot
Collector
CrazyWebCrawler CrazyWebCrawler
Crescent CrunchBot
CyotekWebCopy CyotekWebCopy
DIIbot
DSearch DSearch
DTS Agent DTS Agent
DataCha0s Digincore
DatabaseDriverMysqli Dirbuster
Demon
Deusu
DigitalPebble
Dispatch Dispatch
DnBCrawler-Analytics DittoSpyder
DomainCrawler DnyzBot
DomainSigmaCrawler DomainSigmaCrawler
Domains Project
Dotbot Dotbot
Dragonfly ECCP/1.0
Drip EMail Siphon
EMail Wolf Ecxi
EirGrabber EroCrawler
Express WebPictures Evil
ExtLinksBot
Extractor Extractor
FlashGet Extreme Picture Finder
Freeuploader FHscan
FyberSpider Fimap
FrontPage
Fuzz
Fyrebot Fyrebot
Genieo GT::WWW
GermCrawler GermCrawler
GetRight
Gigabot Gigabot
Go!Zilla Go!Zilla
GoZilla GoZilla
Grafula Gotit
HTTP::Lite GrapeshotCrawler
HMView
HTMLparser
HTTrack
HaosouSpider HaosouSpider
Harvest Harvest
Hloader Heritrix
Humanlinks Humanlinks
IDBTE4M IDBTE4M
IDBot Id-search
Iblog Image Sucker
Indy Library Indy Library
InfoNaviRobot
Iskanie
JamesBOT JamesBOT
Jbrofuzz JetCar
Joomla Jorgee
Keyword Density JustView
Kozmosbot Keybot Translation-Search-Machine
LNSpiderguy
Lanshanbot
Larbin Larbin
Leap Lftp
LeechFTP Libwhisker
LexiBot Linkdexbot
LibWeb
Lightspeedsystems
LinkWalker
Linkbot
LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MSFrontPage
MSIECrawler
MTRobot MTRobot
Mag-Net Majestic-SEO
Mail.RU_Bot
Majestic12 Majestic12
MarkMonitor
Masscan
MauiBot MauiBot
Meanpathbot Meanpathbot
MegaIndex.ru
Metauri Metauri
MicroMessenger MicroMessenger
Mister PiX Microsoft Data Access
MolokaiBot Minefield
Mojeek
Mojolicious
Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3
Msrabot Msrabot
NICErsPRO Musobot
NPbot NPbot
Name Intelligence Name Intelligence
NearSite NearSite
Nessus Net Vampire
NetZIP NetLyzer
NetMechanic
Nettrack
Netvibes
NextGenSearchBot
Niki-bot Niki-bot
Nikto
Ninja Ninja
Nmap Nuclei
Not Octopus
Offline Explorer Offline Navigator
OnCrawl
OpenLinkProfiler OpenLinkProfiler
Openvas OpenVAS
OutclicksBot
OutfoxBot
PHPCrawl PHPCrawl
POE-Component-Client-HTTP PageGrabber
PageThing.com Pandalytics
Panscient Papa Foto
Pavuk
Petalbot Petalbot
Piepmatz Picscout
Pu_iN Pimonster
Pump ProPowerBot
PxBroker ProWebWalker
QueryN Metasearch
Quick-Crawler
RankActive RankActive
Rankivabot RankActiveLinkBot
RebelMouse RankFlex
Recorder RankingBot
RankurBot
ReGet
Reaper
RedesScrapy RedesScrapy
RepoMonkey RepoMonkey
Rogerbot Ripper
SEOprofiler RocketCrawler
SISTRIX SEOkicks
SalesIntelligent SEOkicks-Robot
ScanAlert SEOlyticsCrawler
SMTBot
ScoutJet ScoutJet
Scrapy ScrepyBot
Screaming
Searchestate Searchestate
Seekport SemanticJuice
Semrush SentiBot
SeoSiteCheckup SiteExplorer
Seomoz SiteLockSpider
Shodan
Siphon
SiteCheckerBotCrawler
SiteSucker
Sitebeam
Siteimprove Siteimprove
SmartDownload
Snake Snake
SocialRankIOBot Snapbot
Sociscraper
Sogou web spider
SpaceBison
SpankBot SpankBot
Spanner Spanner
Spbot Spbot
Sqlmap Spinn3r
Sqlworm Sqworm
Sucker Stripper
SuperBot SuperHTTP
Suzuran Surfbot
SurveyBot
Swiftbot Swiftbot
Szukacz T0PHackTeam
Teleport Telesoft
TeleportPro
Titan
Toata Toata
Trendiction Tracemyfile
Turingos Turnitin
TwengaBot TurnitinBot
Twice UnisterBot
Typhoeus
URLy Warning
Upflow Upflow
VCI VCI
Vacuum Vacuum
VeriCiteCrawler Voltron
Virusdie WASALive-Bot
Voil
WBSearchBot WBSearchBot
WISENutbot WWW-Collector-E
Web Auto WWW-Mechanize
Web Enhancer WWWOFFLE
Wallpapers
WeSEE
Web Collage
Web Fuck Web Fuck
Web Sauger WebBandit
WebCollage WebCopier
WebImageCollector WebFetch
WebSauger WebFuck
WebStripper
WebSucker
WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Website Quester
WebsiteExtractor
Webster
Whack Whack
Whacker Whacker
Who.is Bot Who.is Bot
Wonderbot Widow
Wotbox WinHTTrack
Wprecon WiseGuys Robot
Xenu Xaldon WebSpider
YoudaoBot Zermelo
Zauba
Zeus
Zitebot
ZmEu ZmEu
ZumBot ZumBot
arquivo-web-crawler adscanner
autoemailspider archive.org_bot
check1.exe arquivo.pt
clark-crawler cah.io.community
coccocbot cognitiveseo
com.plumanalytics domainsproject.org
crawl.sogou.com facebookscraper
crawler.feedback gopher
demandbase-bot instabid
eCatch ips-agent
internetVista monitor
isitwp.com isitwp.com
iubenda-radar
lwp-trivial lwp-trivial
mediawords mediawords
oBot muhstik-scan
seoscanners page scorer
serpstatbot polaris version
ripz
scalaj-http
seobility
sexsearcher sexsearcher
sitechecker.pro
siteripz siteripz
sysscan sp_auditbot
tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com trendiction.de
x09Mozilla voyagerx.com
zauba.io webgains-bot
webpros.com
webprosbot
xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
dikqlyremy.info 2020iscoming.info
dwomlink.info jikoman.info
gembird.com kongoultry.net
jose.mulinohouse.co labafydjxa.su
norththeface.store luxup.ru
poshiv-chehol.ru photo.houseofgaga.ru
proxyelite.biz sady-urala.ru
rospromtest.ru semaltmedia.com
shoplvlv.us sharebutton.org
tvnewsclips.info xrus.org

948
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
10bet.com 00author.com
123locker.com 03p.info
0c47f8422d3f.com
0daymusic.org
1001watch.com.ua
101billion.com
101lesbian.xyz
101raccoon.ru
1kinobig.ru
2015god.org 2015god.org
202ch.com 24x7-server-support.site
2345.com 34.gs
3dracergames.com 3dracergames.com
4pp13.com
4senses.co
4ur.click 4ur.click
5forex.ru 5forex.ru
5kstore.com 8gold.com
9icmzvn6.website aasoldes.fr
aa14ab57a3339c4064bd9ae6fad7495b5f.com activecampaign.dreamhosters.com
aa8780bb28a1de4eb5bff33c28a218a930.com adclickservice.com
acunetix-referrer.com adidas.frwebs.fr
ad-words.ru admeasures.com
adinterax.com ads.tremorhub.com
ads.gold adultactioncam.com
adservme.com adultgalls.com
adsref.men advancedmassagebysara.com
adtech.us
adviceforum.info
adzpower.com
aero2.ru
aerogo.com
affiliate-fr.com
affordablewebsitesandmobileapps.com affordablewebsitesandmobileapps.com
agardomains.com agroeconom.kz
ahmedabadwebs.com aibolita.com
albumsuper.info aktivator-windows10.blogspot.com
alert-fjg.xyz alfapro.ru
alldubai.biz allblogroll.com
allpdfmags.net
alphaforum.ru
alphavisions.net
alpinism.ru alpinism.ru
amazingpic.net altamayoztourism.com
amazon-seo-service.com amazon-adsystem.com
animenime.ru amyfoxfitness.com
apccargo.com ananumous.ru
angkortours.vn
aosheng-tech.com
apartmentbay.ru
apiholdingmypage-a.akamaihd.net apiholdingmypage-a.akamaihd.net
apple.com-webbrowsing-security.science apishops.ru
approvedlocal.co.za apper.de
arabseyes.com appmsr.org
arcadeplayhouse.com appsaurus.com
arcteryxsale.online arabsexxxtube.com
artavenuegardenstudios.com arenanews.com.ua
articlesdirectoryme.info arendakvartir.kz
arkartex.ru
arquapetrarca.info
arvut.org arvut.org
autochoixspinelli.com asscenihotosidea.blogspot.co.za
autorn.ru asseenontv.ru
avirasecureserver.com atmagroup.ru
avtolombard-krasnodar.com australia-opening-times.com
awaybird.ru awency.com
azbukafree.com baditri.com
azte.ch bagsonsale.online
babespcs.com
backlink4u.net
backlinkwatch.com
bad-stars.net
baixar-musicas-gratis.com
bamo.xsl.pt
bannertesting.com bannertesting.com
barnfurnituremart.com baoxaydung.com.vn
beachdriveblog.com barbourjackets.online
bestwrinklecreamnow.com bdsmgalls.net
billiard-classic.com.ua beatifullandscape.co
binomo.com beauty-things.com
biznesluxe.ru berrymall.ru
biznesschat.net bestbuy.ca
bjanshee.ru betterhdporn.com
blacktwhite.com bidr.trellian.com
blog-hits.com bigpictures.club
bloggedporn.com biketank.ga
bloggerads.net bildsuche.ru
blpmovies.com bimatoprost-careprost.com.ua
bookmark4you.com binaryoptionscops.info
brainzpod.win binomo.kz
bum.com.ru blackcurranthumidifiers.site
cakemediahq.com.au blackhatworth.com
callejondelpozo.es blog.koorg.ru
candelluxsklep.pl bloglag.com
blogrankers.com
bolitgorlo.net
boo-arts.com
bosman.pluto.ro
brainboosting.club
brendbutik.ru
burn-fat.ga
bus-offer.com
buy-cheap-online.info
buyparajumpers.online
buypuppies.ca
cah.io.community
callawaygolfstore.online
camdolls.com
canacopegdl.com
carloans.com
carstrends2015.com carstrends2015.com
cartierlove.xyz casinos4dummies.co.uk
cartierlove2u.com casite-513049.cloudaccess.net
cartierlovebraceletreplica.xyz cdn.walkme.com
ccbill.com chinajerseyswholesalecoupons.com
cheapsergiorossi.online chlooe.com
chelyabinsk.xrus.org christianlouboutinsaleonline.us
chinaelitecheapjerseys.com cialischstgerts.com
christianlouboutinreplica.pw cialisndbrx.com
christianlouboutinreplica.win citetick.com
cialis-samples.com cleanpcnow.com
clickhype.com clicksor.com
com-onlinesupport.host clknsee.com
compliance-alex.top clmforexeu.com
compliance-fred.top cntravelre.com
compliance-jessica.top collegeessay19.blogspot.ru
concealthyself.com com-secure.download
constantaservice.net cometorussia.net
cordstrap.cc compliance-brian.top
cowblog.fr compliance-mary.top
cranly.net crackzplanet.com
crest-poloski.ru cymbaltaandtizanidine.blogspot.com
crystalslot.com czeshop.info
d0t.ru dandiyabeats.in
dailystrength.org dangphoto.trade
dame-ns.kz day-news.info
daneshetabiat.com dealwifi.com
dcj-nn.ru decorationspcs.com
decoratinghomes.ga dentfidemountpreach.tk
delitime.info destinationrealestate.com
deluxedumps.com directrev.com
deplim.com discountbarbour.online
dev78.dioniqlabb.se div.as
devochki.top divisioncore.com
diminishedvalueoforegon.com divku.ru
directivepub.com dkvorota.ru
distver.ru dmmspy.com
djihispano.com dodlive.mil
docsportal.net domoysshop.ru
dogbreedspicture.net donvito.unas.cz
dolohen.com draniki.org
domaincheck.io drugstoreforyou.com
domaincrawler.com drupa.com
dostavimvdom.ru
dostavka-v-ukrainu.ru
download-wallpaper.net
doxysexy.com
drev.biz
drhomes.biz
duawitchrarato.tk
dvrlists.com dvrlists.com
e-avon.ru e-collantes.com
e-commerce-seo1.com easycommerce.cf
earn-from-articles.com ebonyporn.site
earncash.com.ua ecommerce-seo.com
ecxtracking.com
editmedios.com editmedios.com
efnor-ac.com eduserver.net
egvar.net ee77ee.com
ekaterinburg.xrus.org ege-essay.ru
electrik-avenue.com ekn-art.se
elexies.info ekspresihati.info
energydiet-info.ru electro-prom.com
ereko.ru elektrischezigaretten2.beeplog.com
erotikchat-24.com eluxer.net
evening-dating-club.info empirepoker.com
everytalk.tv en.home-task.com
exlarseva.webblog.es enbersoft.com
exonline.info enternet.ee
expresstoplivo.ru enthuse.computernetworksonline.com
extremal-blog.com eonpal.com
eyes-on-you.ga esopini.com
eywords-monitoring-your-success.com essay-edu.biz
fanpagerobot.com essaypro.com
fashion-stickers.ru euronis-free.com
fba-mexico.com europeanwatches.ru
fellowshipoftheminds.com evrotekhservis.ru
findpik.com exci.ru
fishingwholesale.us exportshop.us
fishtauto.ru extremepornos.net
folowsite.com ezigarettenkaufen1.pagina.gr
food.dtu.dk f-loaded.de
formulaantiuban.com fatfasts-4tmz.com
forpackningsutveckling.se ferretsoft.com
fortunejack.com filesvine.com
forum.doctissimo.fr filmania-x.ru
forum.poker4life.ru
forums.d2jsp.org
fr-bearings.ru
franch.info
free-social-buttons.xyz free-social-buttons.xyz
free-today.com fyxabomiw.ru
freelotto.com fz139.ttk.ru
freeuploader.com gamedayhouse.com
freewlan.info gate5.co.za
froggytube.com gateway.zscalerone.net
fugarif.ga gembird.com
fullgirl.ru gemgrab-a.akamaihd.net
g7m.pl germetiki.com.ua
galaxys6manual.info girlporn.ru
galleryawesome.com girlspicsa.com
gavuer.ru gogalleryawesome.com
gcup.ru gpms.org.my
get-your-social-buttons.info greendream.com.ua
getlaid-xxxhookupdirect.com grupografico-pilar.com.ar
getmyads24.com gsasearchengineranker.site
gktt.ru
gombita.info
goo.ne.jp
googlefeud.com
goosefishpost.bid
gsasearchengineranker.top
gstatey.net gstatey.net
gunsvicceadadebt.tk gta-top.ru
gwagka.com hahashka.ru
harmonyglen.us handicapvantoday.com
hdpixion.com hannasolution.ru
herokuapp.com hao123.com
highspeed5.net hawaiisurf.com
hjaoopoa.top headpharmacy.com
hotel-mkad.ru hermesbracelets.xyz
hoterika.com homegardenlova.com
housediz.com homik.pw
https-legalrc.biz hoporno.com
ideashome.id host-tracker.com
igithab.com hostcritique.com
igtools.club hotdl.in
hotgirlhdwallpaper.com
hothot.ru
houseofgaga.ru
howopen.ru
hulfingtonpost.com
hunthillfarmtrust.org
hut1.ru
iccornacircri.cf
igru-xbox.net
ihc2015.info ihc2015.info
ikearugs.xyz ikritikimou.gr
ilovevitaly.ru iminent.com
imadedinner.net import-sales.com
imfamous.info interfucks.net
imgarit.pw internetproviderstucson.com
indianmedicaltourismshop.com iomoio.net
infektsii.com iopeninghours.co.uk
inzn.ru ipostroika.ru
iplusbit.blogspot.co.za irkutsk.zrus.org
iqs.biz.ua isitwp.com
istizanidinelikexanax.blogspot.com iskalko.ru
istripper.com ivanovo.zrus.org
ivanstroi.ru
iyasimasennka.com iyasimasennka.com
jerseywholesaleelitestore.com janerikholst.se
jrcigars.com jav-idol.com
jwcialislrt.com jaxcube.info
kchaxton.com joingames.org
keenoutlet.online juliadiets.com
kerei.ru justbcause.com
keywesthideaways.co kareliatobacco.ru
keywords-monitoring-success.com kevblog.top
keywordsking.com
kichenaid.ru kichenaid.ru
kino-fun.ru kidskunst.info
kinozapas.com kiinomaniak.pl
kleine-titten.biz klejonka.info
konkursowo-24.pl knogg.net
konturkrasoty.ru kovrenok.ru
koral.se ks1234.com
kskjco.club lanasshop.ru
kupit-adenu.ru lancheck.net
labplus.ru lashstudia.ru
larger.io law-check-nine.xyz
law-check-two.xyz lennatin.info
ledpolice.ru letolove.ru
linerdrilling.com lida-ru.com
linkarena.com lider-zhaluzi.kiev.ua
lizace.com lion.misba.us
lolnex.us locustdesign.co
lonerangergames.com ltvperf.com
longlifelomilomi.info luxup.ru
lost-alpha.ru macresource.co.uk
louboutinbooties.xyz maju.bluesalt.co
louboutinreplica.xyz mansiondelrio.co
low-format.ru manualterap.roleforum.ru
luckyshop.net.ua manyresultshub-a.akamaihd.net
lulea-auktionsverk.se marketland.ml
m-google.xyz masthopehomes.com
m3gadownload.pl mebel-alait.ru
mac-shield.com mebelcomplekt.ru
magnetpress.sk meccadumps.net
make-money-online.com mega-polis.biz.ua
marcoislandvacations.net megavolt.net.ua
masturbate.co.uk metascephe.com
megaapteka.ru mezaruk.info
megaindex.ru mirobuvi.com.ua
menhealed.net missvietnam.org
mesbuta.info modabutik.ru
meteocast.net mojaocena.com
mikozstop.com mole.pluto.ro
misandesign.se moncleronline.xyz
mixtapetorrent.com monsterdivx.com
mmgq.ru moscow.xrus.org
mobile-appster.ru mosdverka.ru
moesen-ficken.com mosrif.ru
monarhs.info mustat.com
monsterdivx.tv muznachas-service.ru
morf.snn.gr mydownlodablefiles.com
motherboard.vice.com myhealthcare.com
muschisexbilder.com myiptest.com
musicas.baixar-musicas-gratis.com mykings.pw
muycerdas.xxx myxdate.info
my-big-family.com needtosellmyhousefast.com
mycaf.it newstaffadsshop.club
mydeathspace.com nightvision746.weebly.com
myonlinepayday.co nisuturnetdgu.tk
namenectar.com njkmznnb.ru
narkologiya-voronezh.ru nomuos.it
navalwiki.info nudejapan.net
newhairstylesformen2014.com oakridgemo.com
newyorkhotelsmotels.info obiavo.kz
next-dentists.tk
nikitabuch.com
novosti-hi-tech.ru
office-windows.ru office-windows.ru
onlineporno.site offtopic.biz
okel.co
omoikiri-japan.ru
onclkads.com
one.net.in
online-sbank.ru
openfrost.net
optibuymac.com
optitrade24.com optitrade24.com
orhonit.com oralsexfilme.net
osb.se11.ru owathemes.com
osnova3.ru oyster-green.com
ownshop.cf ozoz.it
owohho.com pagesense.com
paleohub.info pandroid.co
parajumpersstore.online pawli.eu
pcboa.se peekyou.com
pdns.download pendelprognos.se
pekori.to perfectplanned.com
pelfind.me photostudiolightings.com
perl.dp.ua phpdevops.com
peugeot-club.org
photosaga.info
picmoonco.pw
picquery.com
picture-group.com
picturesfrom.com
picturesmania.com
pills24h.com
pixell.club
pizdopletka.club pizdopletka.club
pl-top.pl pl-vouchers.com
platezhka.net plaff-go.ru
pobeiranie.pl play.leadzupc.com
pontiacsolstice.info plohaya-kreditnaya-istoriya.ru
poptool.net polimga.pw
pornoplen.com poppen-nw.net
pornoslive.net popugauka.ru
portal-eu.ru pornhub-ru.com
porno-transsexuals.ru
pornoklad.ru
pornpost.in
positive2b.ru positive2b.ru
postclass.com potolokelekor.ru
pravoholding.ru priceg.com
preparevideosafesystem4unow.site printie.com
privatamateure.com privat-girl.net
privatbank46.ru privatov-zapisi.ru
prizestohandle.club privhosting.com
prod2016.com prointer.net.ua
profitsport.club propranolol40mg.blogspot.com
prosmibank.ru prostitutki-kharkova.org
prostitutki-rostova.org prostitutki-novgoroda.org
pshare.biz purplesphere.in
pskcijdc.bloger.index.hr pyrodesigns.com.au
puserving.com qpypcx.com
qld10000.net qualitymarketzone.com
quality-traffic.com qwertty.net
quangcaons.com r-e-f-e-r-e-r.com
radiodigital.co rank-checker.online
radiogambling.com rapidgator-porn.ga
ranksignals.com razvratnoe.org
rapidokbrain.com realting-moscow.ru
rating-bestcasino.com redirlock.com
refererx.com relax.ru
reklama-i-rabota.ru remont-fridge-tv.ru
rent2spb.ru remont-mobile-phones.ru
reyel1985.webnode.fr riders.ro
rfd-split.hr ring4rhino.com
rfid-locker.co roofers.org.uk
roll123.com ru-dety.ru
ros-ctm.ru rubanners.com
ru-mediaget.ru rutor.group
ruf777.com rvtv.ru
russian-postindex.ru s-forum.biz
sabizonline.com sajatvelemeny.com
savingsslider-a.akamaihd.net satellite.maps.ilovevitaly.com
scanhub.ru sbwealthsolutions.ca
scanner-irvin.top scanner-alex.top
schalke04fc.info scanner-josh.top
school-diplomat.ru scanner-mary.top
scanner-walter.top
scopich.com scopich.com
searchinquire.com score-ads.men
seccioncontrabajo.com scrapinghub.com
secret.xn--oogle-wmc.com screentoolkit.com
security60-e.com seansonline24.pl
securityallianceservices.com search.pch.com
seinterface.com searchlock.com
senger.atspace.co.uk segol.tv
seoholding.com semalt.com
seotoolsagency.com seolab.top
serdcenebolit.com seoprofiler.com
serpstat.com sethrollins.net
seryeznie-znakomstva.ru sex-pr.net
sex-dating.co sex-tracker.com
sexyebonyteen.com sex-znakomstva.online
sexytrend.ru shijian.ac.cn
sfj-ror.no ship-marvel.co.ua
shohanb.com shoesonlinebuy.xyz
shop-electron.ru sibdevice.ru
shop.xz618.com sideeffectsoftizanidine.blogspot.com
sibecoprom.ru site-speed-check.site
silvercash.com
sitebeam.net
sitevaluation.org
sky-mine.ru sky-mine.ru
sladkoevideo.com slavia.info
slayerlife.com slavkokacunko.de
slonechka.ru smsactivator.ru
smokewithrabbits.com snegozaderzhatel.ru
social-search.me sobecjvuwa.com.ru
softomix.com socialmonkee.com
soda.media
sodexo.com
softonicads.com softonicads.com
soheavyblog.com solnplast.ru
souvenirua.com sortthemesitesby.com
sps-shop.com spabali.org
spy-sts.com space2019.top
ssconstruction.co spin2016.cf
stard.shop spl63.fr
statustroll.com splendorsearch-a.akamaihd.net
stevemonsen.com sr-rekneskap.no
stop-gepatit.te.ua srecorder.com
storehouse.ua stairliftstrue.com
streetfire.net steamoff.net
success-seo.com strana-krasoty.ru
sukarame.net streamin.to
sumo.com stuff-about-money.com
svarbit.com superstarfloraluk.com
tds-advert005.info svensk-poesi.com
tedy.su swimpool.ca
terrafootwear.us tattoo33.ru
tfxiq.com td-33.ru
thedownloadfromwarez.blogspot.com techart24.com
thegioixekhach.com teesdaleflyballclub.co.uk
theguardlan.com terraclicks.com
thenetinfo.com thefarmergame.com
thetoiletpaper.com thefotosgratis.eu
tiandeural.ru thegamerznetwork.com
timetorelax.biz threecolumnblogger.com
tizanidinedosage.blogspot.com timeallnews.ru
tizanidinehcl4mgtabinfo.blogspot.com tizanidinefibromyalgia.blogspot.com
tizanidineinfo.blogspot.com tizanidinehydrochloride2mg.blogspot.com
tizanidineingredients.blogspot.com tizanidineinteractions.blogspot.com
tmtrck.com torrent-to-magnet.com
top250movies.ru torrentred.games
torrents.life tracksurf.daooda.com
torrnada.ru trackzapper.com
touchmods.fr tradgardspartner.se
tourcroatia.co.uk trafaret74.ru
toursmaps.com treasuretrack-a.akamaihd.net
tovaroboom.vast.ru tri-slona.org
tracxn.com tube8.com
trafficshaper.com turizm.bz
trk-4.net turizmus.us
truebeauty.cc turn-up-life.life
tsyndicate.com twittrading.com
twelvevisionspartyofcolorado.com ugguk.online
u-cheats.ru umaseh.com
uchil.net umekana.ru
ultimateclassicrock.com unmaroll.ya.ru
unimodemhalfduplefw.pen.io uogonline.com
unitexindia.com
urccvfmc.bloger.index.hr
urdoot.win urdoot.win
uroki.net urlopener.com
validccseller.com urzedowski.eu
valkiria-tk.ru vacuumcleanerguru.com
vbtracker.net vandrie-ict.nl
velpanex.ru velen.io
venta-prom.ru veles.shop
ventelnos.com vertaform.com
vesnatehno.ru veselokloun.ru
via-energy-acquistare.com viagra.pp.ua
via-energy-order.com vip.51.la
villakohlanta.nu virtuagirl.com
vinsit.ru
vintontech.info
vinylvault.co.uk
vipsiterip.org
visa-china.ru visa-china.ru
vkmusics.ru vk-mus.ru
vod.com.ua vsesubwaysurfers.com
vostoktrade.info vseuznaem.com
vpnmouse.com vtc.pw
warning.or.kr vvon.co.uk
warningzscaler.heraeus.com vzube.com
waterefficiency.co walpaperlist.com
wdfdocando.com webadvance.club
weboptimizes.com webalan.ru
website-datenbank.de weblo.com
website-stealer.nufaq.com websitevaluebot.com
weddingdresses.xyz wetgames.ru
weekes.biz.tc where-toget.com
welck.octopis.com wildcattube.com
weprik.ru williamrobsonproperty.com
wheelchairliftsarea.com witclub.info
whiteelephantwellington.com wleuaprpxuvr.ga
wnoz.de wowcasinoonline.ooo
womensplay.net xgftnlrt.bloger.index.hr
wormix-cheats.ru xkaz.org
wowas31.ucoz.ru xmladserver.com
www.ehscloud.cn xn--80ab4aa2g.xn--p1ai
wzgyyq.com xn--80ahvj9e.xn--p1ai
xblognetwork.com xn--80ak6aa92e.com
xgames-04.com xn--b1addnj3cah.xn--p1ai
xitjw.info xn--d1aifoe0a9a.top
xn--80aikhbrhr.net xxxrus.org
xn--e1aggki3c.xn--80adxhks y8games-free.com
xn--oogle-wmc.com ynymnwbm.bloger.index.hr
xtremeeagles.net zaim-pod-zalog-krasnodar.ru
xxlargepop.com zeroredirect9.com
xxxtube69.com zreloeporno.tv
xz618.com zscaler.net
yourtemplatefinder.com zvetki.ru
yubikk.info zytpirwai.net
za-music.mymobiplanet.com
zeroredirect8.com
znakomstva-moskva77.ru
zojirushi-products.ru
zooggames.com
zryydi.com

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01apple.com 007angels.com
1flag.co.za 03p.info
3w24.com 1001watch.com.ua
7xc4n.com 2kata.ru
aa9d046aab36af4ff182f097f840430d51.com 4kepics.com
aadroid.net 4pp13.com
acgs.tk aab94f698f36684c5a852a2ef272e031bb.com
adultgalls.com adobereader-free.ru
adultmeetonline.info adultfriendfinder.com
ageofclones.com
aihelen.net aihelen.net
all4wap.ru alibestsale.com
allornamenti.com all4invest.ru
allproblog.com allpdfmags.net
amazingpic.net alltheviews.com
ananumous.ru appsaurus.com
anatomiy.com apptester.tk
android-systems.ru autoblogger24.info
animalia-life.club autochoixspinelli.com
apartamentwroclaw.eu autowebmarket.com.ua
apiwebwebgetcom-a.akamaihd.net avek.ru
apple.com-cleaner.systems aversis.net
asdfz.pro avtocredit-legko.ru
asophoto.com avtointeres.ru
atout-energie-69.com bala.getenjoyment.net
atovh.local-finders.com bannerswap.com
azarttoto.com beatifullandscape.co
best-mam.ru belstaffstore.online
besthoro.ru bestbrainprod.win
bestwrinklecreamnow.com betterscooter.com
betterhdporn.com bildsuche.ru
binaryoptionscops.info boanonihaca.tk
bioinnovate.co bolezniorganov.ru
biznesschat.net brk-rti.ru
bizzliving.com bumble.cheapwebsitehoster.com
blogrankers.com buqyxa.rincian.info
bolegapakistan.com buyclomidonlaine.com
bonzybuddy.com buypanicdisorderpill.com
boobsimge.com byme.se
bracketsmackdown.com bytimedance.ru
brakehawk.com callstevens.com
bristolhostel.com cartierlove.xyz
browserprotecter.com cartierreplica.top
buy-cheap-pills-order-online.com casinorewards.info
bvlgariring.xyz centre-indigo.org.ua
call-of-duty.info cheap-pills-norx.com
candlehandmade.com
candycrushshop.com
cardsharp1.ru
carrauterie.be
cartechnic.ru
caveavins.fr
cheapbarbour.online
cheapmoncler.pw cheapmoncler.pw
cherubinimobili.it chiptuninger.com
ci.ua cienum.fr
citetick.com ciproandtizanidine.blogspot.com
citysecurity.nu cleanmypc.com
cleanallspyware.com compliance-alex.top
cleanersoft.com compliance-jane.top
clickso.com confib.ifmo.ru
coin-hive.com conocer-sanabria.com
com-cleaner.systems cosmeticswomens-womensports.rhcloud.com
compliance-alex.xyz cxpromote.com
compliance-andrew.top danashop.ru
compliance-jack.top decenttools.com
contentlook.co deplim.com
covetnica.com detki-opt.ru
culmimg.pw
customboxes4less.com
customcollegeessays.net
date-withme.com
discountliv.com
divku.ru divku.ru
dnepropetrovsk.mistr-x.org
docs4all.com docs4all.com
domain-submit.info doctormakarova.ru
domain.webkeyit.com dojki-hd.com
dtm-spain.com downloadeer.net
dvd-famille.com draniki.org
e-poker-2005.com drugstoreforyou.com
easycommerce.cf drunkmoms.net
editors.choice6912650.hulfingtonpost.com dwomlink.info
eldiariodeguadalajara.com ed-shop01.ru
electrik-avenue.com election.interferencer.ru
elektrischezi.canalblog.com eloxal.ru
elektronischezigarettekaufen1.beeplog.com enge-fotzen.info
empirepoker.com ezigarettenkaufen1.deviantart.com
eorogo.top fast-wordpress-start.com
etenininrade.ga filesdatabase.net
evening-dating-club.info findanysex.com
fashion.stellaconstance.co floating-share-buttons.com
fatmaelgarny.com fotosfotos.eu
ferrotodo.com fr-bearings.ru
fickenprivat.info free-video-tool.com
finansov.info freelotto.com
frighteningremain.cf frivgame250.com
front.to fsakhalin.ru
gamesprite.me funtoonez.com
gamezblox.com gamedayassist.com
gaming-journal.com gameprimary.com
gazporno.com gamerscorps.com
gettpromos.com gateway.zscalertwo.net
greenidesign.co gay.adultgalls.com
gsou.cf geoads.com
guardlink.com germetiki.com.ua
gunsvicceadadebt.tk global-ics.co.za
hd720kino.ru goldpanningtools.com
googst2.ru
gosmeb.ru
gotomontenegro.net
gwagka.com
hchha.com
hdfreeporno.net hdfreeporno.net
hexpilot.com hdxnxxtube.mobi
hitcpm.com hesteel.pl
ibb.com.ua hidefiles.org
idc.com.ua home-task.com
ihtec2019.org iblogpress.xyz
innodgfdriverhm.aircus.com igadgetsworld.com
investpamm.ru ilikevitaly.com
iphantom.com incolors.club
irkutsk.online-podarki.com interferencer.ru
irunfar.com ipchicken.com
j33x.com iron-age.info
javidol.site isitpaleo.info
jrpmakati.com istanbulit.com
kaz.kz ix20.ru
kchaxton.com japan-bearings.ru
kerwinandcariza.com jaxcube.info
kollekcioner.ru job.icivil.ir
kosynka-games.ru johannesburgsingles.co.za
ku6.com jrcigars.com
lalalove.ru karaganda.xkaz.org
lennatin.info keywordteam.net
letmacworkfaster.site kichenaid.ru
likesdesign.com kiinomaniak.pl
luckyshop.net.ua kimcurlrvsms.com
mactechinfo.info kinobaks.com
markjaybeefractal.com klosetkitten.com
master-muznachas.ru kmgamex.cf
medanestesia.ru kndxbkdx.bloger.index.hr
medictube.ru konica.kz
mmoguider.ru konteiner24.com
mnimmigrantrights.net kvartira-sutochno.com
mobplayer.net ladov.ru
most-kerch.org laserpen.club
msk.afora.ru livejasmin.com
natali-forex.com localflirtbuddies.com
naturalpharm.com.ua locimge.pw
nerudlogistik.ru masserect.com
o-o-6-o-o.com masthopehomes.com
obiavo.kz matsdale.com
oktube.ru mebel-alait.ru
olvanto.ru metallosajding.ru
onko-24.com metascephe.com
onthemarch.co mini.7zap.com
openfrost.com mitrasound.ru
park.above.com mlf.hordo.win
parlament.biz mmog-play.ru
partnersafe.men mompussy.net
pornobrazzers.biz moomi-daeri.com
pornoelita.info morpicert.pw
portnoff.od.ua mtmtv.info
posible.net mylesosibirsk.ru
myonigroup.com
nac-bearings.ru
nakrutka.cc
netanalytics.xyz
nfljerseysforsalewholesaler.com
novatech.vn
onlinefilmz.net
ororodnik.goodbb.ru
pacificair.com
pagesense.com
paperwritingservice17.blogspot.ru
paramountmarble.co.uk
perfection-pleasure.ru
pinstake.com
pornohub.me
pornpost.in
portside.xyz
pozvonim.com
prchecker.info prchecker.info
pregnant.guru producm.ru
pricheskaonline.ru psoriasis-file.trade
privatbank46.ru quickloanbank.com
produkto.net realitykings.com
prospekt-st.ru remybutler.fr
raisedseo.com restorator-msk.ru
rankings-analytics.com reward-survey.net
redbottomheels.xyz rfd-split.hr
refudiatethissarah.info rumamba.com
rentehno.ru russkoe-zdorovie.ru
responsive-test.net rusvideos.su
russia-tao.ru safe-app.net
sabizonline.com sammlungfotos.online
saletool.ru scalerite.co.za
sarf3omlat.com scanner-andrew.top
satellite.maps.ilovevitaly.com scenarii-1-sentyabrya.uroki.org.ua
scanner-alex.top seoholding.com
scanner-donald.top sex-foto.pw
scansafe.net sex-tracker.com
scenicmissouri.us
schlampen-treffen.com
seo-traffic-ranking.info
sexy-pings.com sexy-pings.com
sexyali.com shitmovs.com
shariki-zuma-lines.ru shmetall.com.ua
shoppytoolmac.com sispe.com.br
simplesite.com site3.free-share-buttons.com
sixcooler.de sitebeam.net
skyway24.ru sklad-24.ru
slimcdn.com small-games.biz
snjatie-geroinovoy-lomki.ru smart-balancewheel.com
social-s-ggg.xyz sockshares.tv
softlinesolutions.me solmarket.by
spasswelt.xyz sophang8.com
streamin.to sortthemesitesby.com
streetfire.net spamnuker.com
stroydetali.ru spidtest.org
studentguide.ru stairliftstrue.com
summerlinhomes411.info stoki.ru
surcentro.com svnuppsalaorebro.se
svetlotorg.ru
swagbucks.com
swingerseiten.com
szucs.ru szucs.ru
tanieaukcje.com.pl telefonsexi.com
tattomedia.com telegramdownload10.com
taximytishi.ru thecoral.com.br
thebitcoincode.com tizanidine4mguses.blogspot.com
tizanidinesideeffectsweightloss.blogspot.com tizanidineoral.blogspot.com
tk-assortiment.ru tksn.ru
tooplay.com tongkatmadura.info
tour-line.net tonivedu.it
truemfilelj.gq torture.ml
tuberkuleznik.ru trackzapper.com
turbodsp.com trafficstars.com
uhdtv.website traffpartners.com
triumf-realty.ru
turkeyreport.tk
twitlinks.com
typimga.pw
um-razum.ru
uroki.net uroki.net
urzedowski.eu ussearche.cf
valkiria-tk.ru uytmaster.ru
vashsvet.com veerotech.com
vavilone.com vrnelectro.ru
vedomstvo.net
videokrik.net
vladimir.zrus.org
vodkoved.ru
wanker.us
warezkeeper.com
webscouter.net
website-speed-checker.site
wejdz-tu.pl
whengirlsgowild.com whengirlsgowild.com
while.cheapwebsitehoster.com world-mmo.com
writingservices17.blogspot.ru
wttavern.com wttavern.com
x-musics.com wwwadultcheck.com
xboxster.ru xgames-04.com
xn----7sbho2agebbhlivy.xn--p1ai xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xsion.net xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
yaderenergy.ru xn--24-glceagatoq7c2a6ioc.xn--p1ai
yopoint.in xn--80ajbshivpvn2i.xn--p1ai
your-bearings.com xn--90acenikpebbdd4f6d.xn--p1ai
youraticles.pl xn--d1acah0c.xn--p1ai
youtubologia.it xn--lifehacer-1rb.com
yun56.co xxxasianporn.net
yadro.ru
yourporn.com
zacreditom.ru
zajm-zalog-krasnodar.ru zajm-zalog-krasnodar.ru
zerocash.msk.ru zhuravlev.info
zeroredirect10.com zoomovies.org
zscalertwo.net

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.18 0; 10.1.0.71 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.03.2990 ### Version: V4.2022.03.2991
### Updated: Sun Mar 20 10:03:43 UTC 2022 ### Updated: Sun Mar 20 22:01:20 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.03.2990 ### Version: V4.2022.03.2991
### Updated: Sun Mar 20 10:03:43 UTC 2022 ### Updated: Sun Mar 20 22:01:20 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##