vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-07 13:10:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.01.3547 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-01-15 10:09:20 +00:00
parent 48eb7edb62
commit 482e335258
47 changed files with 98637 additions and 98644 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3546 #### Version: V4.2023.01.3547
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3546 #### Version: V4.2023.01.3547
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.01.3546 #### Version: V4.2023.01.3547
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12186
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.01.3546 #### Version: V4.2023.01.3547
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12190
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12190
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12190
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12190
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

12190
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

12190
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

12190
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3546 3547

12190
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12190
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.168.158.208 0; 172.176.137.176 0;

12190
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

12190
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

12190
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

12190
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.168.158.208 0; 172.176.137.176 0;

12190
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.168.158.208 0; 172.176.137.176 0;

12190
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BDFetch AwarioSmartBot
Backlink-Ceck BotALot
BlowFish HybridBot
CheeseBot Moblie Safari
Dispatch OnCrawl
MarkWatch Rankivabot
Masscan Teleport
ProWebWalker Vacuum
WebCollage com.plumanalytics
WebSucker webgains-bot

290
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404enemy
80legs
AIBOT AIBOT
ALittle Client ALittle Client
Abonti Aboundex
Aboundexbot Aboundexbot
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot AiHitBot
Aipbot
AllSubmitter AllSubmitter
Alligator Alligator
Anarchie Anarchie
Ankit Anarchy
Anthill Anthill
Apexoo Apexoo
Atomseobot Asterias
Attach Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot BDCbot
BLEXBot BackDoorBot
BacklinkCrawler
Badass Badass
Bitacle BatchFTP
Blow
BlowFish
Boardreader Boardreader
Bolt Bolt
BotALot
Brandprotect Brandprotect
Brandwatch BuiltWith
BuiltBotTough BunnySlippers
Bullseye CazoodleBot
CCBot
Calculon
CensysInspect
CheTeam CheTeam
CheeseBot CheeseBot
CherryPicker CherryPicker
Chlooe Chlooe
Cliqzbot Citoid
Collector Cogentbot
CopyRightCheck
Copyscape Copyscape
Cosmos
CrazyWebCrawler
CrunchBot CrunchBot
Curious
CyotekWebCopy CyotekWebCopy
DBLBot DBLBot
DIIbot
DSearch
DTS Agent DTS Agent
DataCha0s
DatabaseDriverMysqli
Demon Demon
Digincore Deusu
Discobot Dirbuster
Discoverybot
Dispatch
DnyzBot
DomCopBot
DomainCrawler DomainCrawler
DomainSigmaCrawler
Domains Project Domains Project
Dragonfly Download Wonder
Drip Drip
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
Express WebPictures EirGrabber
Exabot
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE FDM
Ezooms
FemtosearchBot
Fimap
Firefox/7.0
Flunky
Freeuploader Freeuploader
Fuzz Fuzz
FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
Genieo GermCrawler
GetRight
Gigabot Gigabot
Gotit GrabNet
Grabber
GrapeFX
GrapeshotCrawler GrapeshotCrawler
GridBot
HMView HMView
HTTP::Lite
HTTrack
HaosouSpider HaosouSpider
Heritrix Harvest
Hloader HonoluluBot
Iblog Humanlinks
IndeedBot Id-search
Image Fetch
Indy Library Indy Library
InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
InterGET InterGET
Internet Ninja Internet Ninja
IstellaBot Jbrofuzz
JOC Web Spider Joomla
JetCar
JustView
Jyxobot
Kenjin Spider
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Keyword Density Leap
Kozmosbot
Larbin
LeechFTP LeechFTP
LexiBot LeechGet
Likse Libwhisker
LinkWalker
Linkbot
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot LinkpadBot
LinqiaRSSBot LinksManager
LinqiaScrapeBot Lipperhey Spider
Ltx71 Lmspider
MJ12bot MIDown tool
Majestic SEO MIIxpc
MQQBrowser
MSIECrawler
MTRobot
Magnet
Mail.RU_Bot
Majestic12 Majestic12
Mass Downloader MarkMonitor
Masscan
Mata Hari
Mb2345Browser Mb2345Browser
Meanpathbot MegaIndex.ru
Mediatoolkitbot Metauri
Microsoft Data Access Moblie Safari
Mojeek MolokaiBot
Mojolicious
Morfeus Fucking Scanner
Mr.4x3
Msrabot Msrabot
NICErsPRO NICErsPRO
Needle
Nessus Nessus
Net Vampire
NetAnts
NetMechanic
Nettrack
Netvibes Netvibes
NextGenSearchBot Niki-bot
NimbleCrawler Nikto
Nimbostratus Nimbostratus
Ninja
Nmap
Offline Explorer
Offline Navigator Offline Navigator
OrangeBot OpenLinkProfiler
OrangeSpider OpenVAS
OutfoxBot Openfind
PHPCrawl Page Analyzer
PageGrabber PageScorer
PageThing.com
Panscient Panscient
Papa Foto Papa Foto
PictureFinder Petalbot
Pimonster Picsearch
Pixray PleaseCrawl
Pockey ProPowerBot
ProWebWalker Probethenet
Psbot Psbot
Pump PxBroker
PyCurl PyCurl
RankActive Quick-Crawler
RankFlex RankingBot
Rankivabot Re-re
RankurBot SBIder
RealDownload
RepoMonkey
Ripper
RocketCrawler
Rogerbot
SEOkicks SEOkicks
SISTRIX SEOkicks-Robot
SMTBot SEOlyticsCrawler
SalesIntelligent SalesIntelligent
ScanAlert
ScoutJet
Screaming Screaming
ScreenerBot SemanticJuice
Searchestate Shodan
SeoSiteCheckup
Seomoz
Siphon
Site Sucker
SiteLockSpider
SiteSnagger
Site Sucker Site Sucker
SiteExplorer
Sitebeam Sitebeam
Sitevigil Sitevigil
SocialRankIOBot
Sociscraper Sociscraper
Sogou web spider
Sosospider
SpaceBison SpaceBison
Spammen Spammen
Spbot Spbot
Sqlmap
Steeler Steeler
Sucker Stripper
Suzuran Sucuri
Teleport SuperBot
Telesphoreo Swiftbot
T0PHackTeam
The Intraformant The Intraformant
TheNomad
TightTwatBot
Toata Toata
Toweyabot
Tracemyfile
Trendiction
True_Robot True_Robot
Turnitin Turnitin
TurnitinBot
TwengaBot TwengaBot
Typhoeus VB Project
VCI Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
VoidEYE
Voil Voil
Voltron
WBSearchBot WBSearchBot
WEBDAV
WISENutbot WISENutbot
WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
Web Collage WeSEE
WebAuto Web Enhancer
WebCollage
WebCopier
Web Fetch Web Fetch
Web Fuck Web Fuck
WebStripper Web Pix
WebCollage
WebEnhancer
WebImageCollector
WebLeacher
WebSucker WebSucker
WebWhacker
WebZIP WebZIP
Webshag Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster Webster
Whack Whacker
Whatweb
Who.is Bot Who.is Bot
Widow
WiseGuys Robot
Wonderbot
Woobot
Wotbox
Xaldon_WebSpider Xaldon_WebSpider
Zauba Xenu
Zeus YoudaoBot
ZoominfoBot ZmEu
ZoomBot
ZumBot ZumBot
ZyBorg
adscanner adscanner
arquivo.pt arquivo.pt
clark-crawler backlink-check
coccocbot coccocbot
com.plumanalytics com.plumanalytics
crawl.sogou.com crawl.sogou.com
crawler.feedback crawler.feedback
demandbase-bot crawler4j
eCatch
evc-batch
facebookscraper
gopher gopher
heritrix
ips-agent ips-agent
iubenda-radar iubenda-radar
magpie-crawler linkdexbot
mediawords
muhstik-scan muhstik-scan
oBot pcBrowser
page scorer
plumanalytics plumanalytics
probe-image-size sexsearcher
scan.lol
seocompany.store
sitechecker.pro
siteripz siteripz
trendiction.de sogouspider
ubermetrics-technologies.com sysscan
zgrab trendiction.com
voyagerx.com
webmeup-crawler
xpymep1.exe
zauba.io

308
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404enemy
Aboundex 80legs
ADmantX
AIBOT
Abonti
Aboundexbot
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AiHitBot AiHitBot
Alexibot Aipbot
Alligator
AlphaBot
Anarchie Anarchie
Atomseobot Aspiegel
Attach Attach
AwarioSmartBot AwarioSmartBot
BBBike BDFetch
BLEXBot BackWeb
BackDoorBot Backlink-Ceck
Bandit Barkrowler
BetaBot Black Hole
Blow BlackWidow
Blackboard
BlowFish
Bolt Bolt
BotALot Brandprotect
Brandwatch
Buck
BuiltBotTough BuiltBotTough
BuiltWith BuiltWith
CATExplorador Bullseye
CCBot CCBot
CODE87 CSHttp
Cegbfeieh
CheTeam
ChinaClaw
Chlooe Chlooe
Citoid
Claritybot Claritybot
Cloud mapping Cliqzbot
Collector Cocolyzebot
Copier Copier
Copyscape Cosmos
Craftbot CrazyWebCrawler
Crescent Crescent
CrunchBot CrunchBot
Custo Custo
DataCha0s DIIbot
Demon DSearch
DTS Agent
Devil Devil
Digincore
Dirbuster
Disco Disco
Discoverybot Discobot
DomCopBot Dispatch
Domains Project Domains Project
Download Wonder Download Wonder
Dragonfly Dragonfly
Drip ECCP/1.0
EMail Siphon EMail Siphon
EasyDL
Ecxi
EirGrabber EirGrabber
EroCrawler EroCrawler
Evil
Express WebPictures
ExtLinksBot
Extractor Extractor
EyeNetIE Extreme Picture Finder
Ezooms FHscan
Firefox/7.0 FyberSpider
Freeuploader GalaxyBot
G-i-g-a-b-o-t
GT::WWW
GermCrawler GermCrawler
GetRight GoZilla
GetWeb Gotit
Go!Zilla
Grabber Grabber
GrapeFX GrapeFX
HEADMasterSEO HTTP::Lite
HMView HTTrack
Harvest
Havij Havij
Heritrix Heritrix
Hloader
Iblog Iblog
Id-search Id-search
IlseBot
Image Fetch Image Fetch
InfoTekies Image Sucker
Intelliseek
InterGET
Internet Ninja
InternetSeer
Iskanie
IstellaBot
JOC Web Spider
JamesBOT
JennyBot JennyBot
JikeSpider Jyxobot
Jorgee Kenjin Spider
JustView Keybot Translation-Search-Machine
Keyword Density Kinza
LNSpiderguy LWP::Simple
Larbin LeechGet
Leap LexiBot
LibWeb LibWeb
Lightspeedsystems LieBaoFast
LinkScan LinkScan
LinkpadBot Linkbot
LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaRSSBot
Lipperhey MJ12bot
Lipperhey Spider MSFrontPage
MIIxpc
MTRobot
Magnet
Majestic-SEO Majestic-SEO
Majestic12 MauiBot
MarkMonitor Meanpathbot
MarkWatch
Mass Downloader
Mata Hari
Mb2345Browser
MeanPath Bot
Mediatoolkitbot
Metauri Metauri
Msrabot MicroMessenger
Musobot Mister PiX
NICErsPRO Moblie Safari
NPbot Mojeek
Morfeus Fucking Scanner
Mozlila
Name Intelligence Name Intelligence
Nameprotect
Navroad Navroad
Nessus Nessus
Net Vampire
NetAnts NetAnts
NetLyzer NetLyzer
Netvibes Netcraft
NextGenSearchBot Niki-bot
Nimbostratus NimbleCrawler
Ninja
Nmap
Nuclei Nuclei
Octopus Nutch
OnCrawl OnCrawl
OpenVAS
Openvas Openvas
OutclicksBot OrangeBot
PECL::HTTP OrangeSpider
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer
PageGrabber
PageScorer PageScorer
PageThing.com Pavuk
PeoplePal
Petalbot Petalbot
Pi-Monster
Picsearch Picsearch
ProPowerBot
ProWebWalker
Probethenet Probethenet
Pu_iN Pu_iN
Pump
PxBroker
PyCurl PyCurl
RankActiveLinkBot QueryN Metasearch
RankActive
RankingBot
RankingBot2 RankingBot2
Rankivabot Rankivabot
RankurBot RankurBot
Re-re RealDownload
ReGet
RedesScrapy RedesScrapy
RepoMonkey Ripper
RocketCrawler
Rogerbot Rogerbot
SEOkicks SBIder
SEOlyticsCrawler SEOstats
SISTRIX SISTRIX
ScanAlert SalesIntelligent
Scanbot Scanbot
Scrapy
ScreenerBot ScreenerBot
SemanticJuice Searchestate
SemrushBot SearchmetricsBot
SeoSiteCheckup SentiBot
Siphon Siphon
Site Sucker SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker SiteSucker
Siteimprove Sitebeam
Sitevigil Sitevigil
SmartDownload SmartDownload
Snake SocialRankIOBot
Snoopy Sogou web spider
Spammen Sosospider
SpaceBison
SpankBot SpankBot
Spanner Spanner
Spinn3r
Sqlmap
Sqlworm Sqlworm
Sucker Sqworm
Surfbot Stripper
Thumbor Sucuri
SuperHTTP
SurveyBot
T0PHackTeam
T8Abot
Teleport
Telesoft
Telesphorep
The Intraformant
TheNomad
TightTwatBot TightTwatBot
Titan Trendictionbot
Toata
Trendiction
True_Robot True_Robot
Turnitin Typhoeus
TurnitinBot
URLy.Warning
VCI VCI
Vacuum Vacuum
Vagabondo Vagabondo
Virusdie VeriCiteCrawler
Voil VoidEYE
Voltron Voltron
WBSearchBot WBSearchBot
WEBDAV WISENutbot
WWW-Collector-E
WWW::Mechanize WWW::Mechanize
Web Auto Web Auto
Web Pix
WebCopier
WebEnhancer
Web Fetch Web Fetch
WebWhacker WebCopier
WebmasterWorldForumBot WebFetch
Webshag WebFuck
WebImageCollector
WebLeacher
WebPix
WebSauger
WebStripper
WebSucker
Webalta
Website Quester Website Quester
Webster WebsiteQuester
Whack
Whatweb Whatweb
WinHTTrack Widow
WiseGuys Robot
Wonderbot Wonderbot
Wotbox Xaldon_WebSpider
Xenu Zermelo
Zade Zeus
Zauba ZoomBot
Zitebot
ZmEu
ZoominfoBot ZoominfoBot
ZumBot archive.org_bot
ZyBorg autoemailspider
backlink-check cah.io.community
check1.exe check1.exe
coccocbot clark-crawler
crawler4j cognitiveseo
com.plumanalytics
crawl.sogou.com
demandbase-bot demandbase-bot
domainsproject.org eCatch
facebookscraper facebookscraper
heritrix gopher
instabid instabid
ips-agent ips-agent
iubenda-radar
linkdexbot linkdexbot
lwp-trivial
meanpathbot meanpathbot
mediawords mediawords
muhstik-scan muhstik-scan
page scorer netEstate NE Crawler
plumanalytics pcBrowser
polaris version polaris version
probe-image-size probe-image-size
s1z.ru s1z.ru
satoristudio.net sogouspider
scan.lol sp_auditbot
seoscanners ubermetrics-technologies.com
spyfu
tAkeOut
trendiction.com
trendiction.de
webmeup-crawler
webpros.com
webprosbot webprosbot
x22Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
chatroulette.world boc.kz
clickaider.com firma-legion.ru
hut1.ru geilehausfrauen.net
monetizationking.net japfm.com
saugatuck.com maladot.com
simplesite.com popunder.ru
stroiminsk.com real-time-analytics.com
tengohydar.tk servisural.ru
thetattoohut.com yoopsie.com
yopoint.in zeroredirect8.com

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
000free.us 00go.com
08800.top 1001watch.com.ua
123kuma.com 123movies.love
1kdailyprofit.me 273-fz.ru
2x2fan.ru 2x2fan.ru
3rbseyes.com 300richmond.co.nz
4kplayer.pl 3xforum.ro
7makemoneyonline.com
7search.com
883zy.com 883zy.com
a.frcls.fr 8lufu.com
aa8b68101d388c446389283820863176e7.com 8si.ru
aa9bd78f328a6a41279d0fad0a88df1901.com aa08daf7e13b6345e09e92f771507fa5f4.com
aa9d046aab36af4ff182f097f840430d51.com abclauncher.com
abcdeg.xyz academia-nsk.org
adcash.com ad-words.ru
adrenalin-stk.ru addblueoff.com.ua
adserver-e7.com addray.pro
adtech.us adservme.com
adtiger.tk advancedcleaner.com
adultactioncam.com adventureparkcostarica.com
afmuseum.com adzpower.com
afora.ru ageofclones.com
agapovdg.ru aktivator-windows10.blogspot.com
ahrefs.com alert-hgd.xyz
alborzan.com allmarketsnewdayli.gdn
alif-ba-ta.com alot.com
alphahoverboards.com alphahoverboards.com
alphavisions.net aludecor.info
amateurmatch.com
anaksma.info anaksma.info
anal-acrobats.com antons-transporte.de
ananumous.ru apifasterlightin-a.akamaihd.net
anapa-inns.ru appmsr.org
angigreene.com apptester.tk
aosheng-tech.com arabsexxxtube.com
apessay.com arenanews.com.ua
apiwebwebgetcom-a.akamaihd.net asrvvv-a.akamaihd.net
appiq.mobi ausmepa.org.au
apple.com-webbrowsing-security.science auto.rusvile.lt
arendakvartir.kz avtovolop.ru
as5000.com ayanaspa.com
autotop.com.ua azartmsl.com
aviva-limoux.com azbaseimages.net
awency.com babyfactory.fr
ayakino.net babyguns.ru
ayeartoforget.com baixar-musicas-gratis.com
azartmix.com baladur.ru
babieca.com bambi.ck.ua
balook.com battle.net
bangla-chat-uk-paralud.ga battlecarnival.su
bannerads.de bausparen.bz.it
bear.gotcher.us bayareaaeroclub.org
belsetirehafi.tk becuo.com
bestadbid.com benchmarkcommunications.co.uk
bet-prognoz.com bestfortraders.com
bezcmexa.ru bestmouthwash.club
bestwaystogetpaid.us
beta.hotkeys.com
betonka.pro
bgviagrachrx.com bgviagrachrx.com
bidr.trellian.com bigames.online
bigpenisguide.com bigcareer.info
bildsuche.ru biketank.ga
bimatoprost-careprost.com binomo.com
bio.trade-jp.net bitcoinpile.com
bjgugu.net.ua bjgugu.net.ua
bongacams.com blackhatworth.com
bonky.biz bloggers.nl
boo-arts.com blogsfunky672.weebly.com
bookmaker-bet.com blpmovies.com
bosefux.esy.es blue-square.biz
bristolhostel.com bmw-ark.ru
buyviagraa.blogspot.com bonkers.name
buzzurl.jp boobsimge.com
bookmark4you.com.biz
brandov.ru
bryansk.zrus.org
buron.pw
buttons-for-your-website.com
buy-cheap-pills-order-online.com
buyfriend.ru
c-english.ru c-english.ru
cacheimages.com
camdenmemorials.com camdenmemorials.com
capacitacionyaprendizaje.com camdolls.com
capture-room.com camsex.xxx-cam.webcam
cartierwatch.xyz carrauterie.be
casinorewards.info carsplate.com
casite-513049.cloudaccess.net catalogs-parts.com
celebsopics.com cayado.snn.gr
cenoval.ru cdn.walkme.com
chocolateslim-original.com cheap-pills-norx.com
cialis-samples.com cheapjerseysbizwholesale.us
citetick.com cialisovercounteratwalmartusa.com
clarithromycin500mg.com cigarpass.com
club-musics.ru clash-clans.ru
codq.info classiquebijoux.ru
coinsspb.com claytransformations.info
colehaanoutlet.store clickbanksites.info
com-supportcenter.website clknsee.com
compliance-donald.xyz clodo.ru
com-cleaner.systems
com-onlinesupport.host
compliance-mark.top compliance-mark.top
constantaservice.net concealthyself.com
contentsexpress.com concordexoticrentals.com
cookie-law-enforcement-ff.xyz cordstrap.cc
coolgamechannel.com corta.co
coolingoods.com counterbot.com
copyrightclaims.org crd.clan.su
cosmeticswomens-womensports.rhcloud.com creatives.adbetclickin.pink
counter.bloke.com crest3d.ru
country-chic.ru culturevie.info
creditwell.ru cupday.com
crystalslot.com cvety24.by
datract.com danschawbel.com
daymusam.com daooda.com
delayreferat.ru data1.scopich.com
deluxedumps.com dating-app.ru
demenageur.com de.zapmeta.com
derevesendeco.com dentfidemountpreach.tk
destinationrealestate.com dikx.gdn
digital-sale.su disruptingdinnerparties.com
dimaka.info dojki365.online
directrev.com domaincheck.io
discountliv.com dorratex.tn
divci-hry.info dvrlists.com
dnepr-avtospar.com.ua
dojki-hd.com
dominateforex.ml
drivotracker.com
dwomlink.info
eachdayisagift.review eachdayisagift.review
eager-nash.188-93-233-196.plesk.page eas-seo.com
earncash.com.ua easytuningshop.ru
easync.io ecxtracking.com
ecookna.com.ua electro-prom.com
elektrischezigaretten2.beeplog.com elektronischezigarettekaufen2.tumblr.com
elektronischezi.livejournal.com elektrozigarettekaufen1.devhub.com
elektrozigaretten1.tumblr.com elite-sex-finders.com
elidelcream.weebly.com elvel.com.ua
elitesportsadvisor.com essaypro.com
eorogo.top europages.com.ru
eropornosex.ru european-torches.ru
escortslet.net everflora.ru
este-line.com.ua exoneration-project.us
excitacion.info extlinks.com
exlarseva.webblog.es ezigarettenshop2.postbit.com
exmasters.com farm26.ru
ezb.elvenmachine.com
ezigarettekaufen2.blox.pl
ezigarettenshop2.myblog.de
ezrvrentals.com
farmingworm.com
fashion-mk.net fashion-mk.net
fetlifeblog.com fba-mexico.com
feargames.ru
fermersovet.ru
fickluder69.com fickluder69.com
financepoints.eu filkhbr.com
findpics.pw fillmewithhappiness.com
finemanteam.com fishtauto.ru
fiuxy.com flytourisme.org
fivedwld.cf
fok.nl
food.dtu.dk
footbalive.org
forensicpsychiatry.ru forensicpsychiatry.ru
forms-mtm.ru
formulaf1results.blogspot.com formulaf1results.blogspot.com
fortunejack.com formularz-konkurs.tk
forum.poker4life.ru foto-weinberger.at
forum.tvmir.org foxweber.com
foto-basa.com free-deals.faith
foto-sisek.porngalleries.top freejabs.com
fr-bearings.ru freemags.cc
frbizlist.com freewebs.com
freelifetimefuckbook.com freshdz.com
freemaintenancesysforpcandmac.top freshmac.space
froggytube.com freshwallpapers.info
front.to friendflnder.com
funnel.co.za fullfileaccess.com
g.starmoe.xyz fun-mobi.pl
g7m.pl fxgallery.com
gallerylisting.com g33.org
gamblingnerd.com gabeshop.ru
gameonasia.com galaxy-family.ru
game300.ru
gamebackyard.com
gamerscorps.com gamerscorps.com
gateway.zscalerone.net gate5.co.za
gerhardhealer.com gavuer.ru
get-your-social-buttons.info gayxperience.com
gkvector.ru geile-lelly.eu
goodprotein.ru generousdeal-a.akamaihd.net
gotomontenegro.net getmyads24.com
graphicwe.org gidonline.one
gsasearchengineranker.site girlsatgames.ru
handicapvantoday.com given2.com
hanwei.us gojiberriess.apishops.ru
havepussy.com goldandcard.ru
hawaiisurf.com gtopstats.com
hdseriale.pl handicapvansarea.com
herokuapp.com handsandlegs.ru
heygidday.biz hdapp1008-a.akamaihd.net
hikesearch.net hdfreeporno.net
histock.info hdimagelib.com
historichometeam.com hdwallpapers-free.com
hobbyhuren24.net healbio.ru
honyaku.yahoofs.jp hermesbelts.xyz
hotelcrocenzi.sm heroz.fr
hotxnights.info hifidesign.ru
highland-homes.com
home.myplaycity.com
hotsocialz.com
houston-vikings.com
hoverboard360.nl hoverboard360.nl
hoztorg-opt.ru i4track.net
hptwaakw.blog.fc2.com ilikevitaly.com
hscsscotland.com incest-ru.com
humanelydrew.com inet-traffic.com
ifirestarter.ru infowarcraft.ru
ilo134uloh.com intellektmedia.at
imagine-ex.co investpamm.ru
immobiliaremassaro.com it-max.com.ua
impisr.edunsk.ru
impisr.ru
increasewwwtraffic.info
insurple.com
int.search.tb.ask.com
integritylandscapeservices.com
intim-uslugi.info
isistaylorporn.info
isitpaleo.info
istizanidineanopiate.blogspot.com
istizanidinelikexanax.blogspot.com
iusstf.org iusstf.org
izoll.ru
jackpotchances.com jackpotchances.com
javspace.net janavibekken.no
jerseyssportsshop.com jerseywholesalechinabiz.com
jerseyswholesalechinalimited.com
jo24news.com
jobgirl24.ru
jofucipiku.tk
jose.mulinohouse.co
jumptap.com
kamagragelusa.net
kazan.xrus.org
kchaxton.com
kerch.site
keywordblocks.com keywordblocks.com
kfon.eu keywordhouse.com
kihi.gdn keywordhut.com
kinoduh.ru kicknights.gq
kinozapas.com kimcurlrvsms.com
kmgamex.cf kino2018.club
konturkrasoty.ru kinocccp.net
kountrylife.com kinoflux.net
krasnodar.ru kinopolet.net
kirov.zrus.org
krafte.ru
kreativperlen.ch
ktotut.net ktotut.net
labplus.ru laapp.com
leftofcentrist.com lenpipet.ru
letmacwork.world lerporn.info
lidoradio.com lesbian.xyz
linkdebrideur.xyz letslowbefast.site
lion.misba.us life-instyle.com
liran-locks.com lifeinsurancekb.com
listiseltemournlan.gq link.ac
livejasmin.com lionking-1994.blogspot.com
lomb.co longadventure.com
louboutinshoes.xyz luckyday.world
lrac.info mactechinfo.info
lubetube.com mailemedicinals.com
m5home.ru mararoom.ru
madot.onlinewebshop.net marinetraffic.com
mafcards.ru marsgatan.com
mainlinehobby.net maslenka.kz
mature.free-websites.com mastershef.club
me-ke.com matchpal-a.akamaihd.net
mebel-alait.ru mb140.ru
message-warning.net mbiologi.ru
metascephe.com mebelcomplekt.ru
mmgq.ru megagrabber.ru
mobot.site meltwater.com
mobplayer.net mere.host.sk
mobplayer.ru metallosajding.ru
modnie-futbolki.net metaxalonevstizanidine.blogspot.com
missis.top
mobilemedia.md
modabutik.ru
moi-glazki.ru moi-glazki.ru
moncleroutletonline.win mojpregled.com
moncleroutletonline.xyz monclerboots.xyz
monclervests.xyz more-letom.ru
monthlywinners.com mtmtv.info
moscow.online-podarki.com mttwtrack.com
moscow.xrus.org musezone.ru
mountainstream.ms musicas.baixar-musicas-gratis.com
mp3downloadhq.com myanyone.net
msfsaar.de mybackgroundlandscape.blogspot.com
muschisexbilder.com myblogregistercm.tk
mustwineblog.com mydownlodablefiles.com
mvpicton.co.uk mysexpics.ru
mycouponizemac.com nalogovyy-kodeks.ru
myfreetutorials.com national-today-winning-winner.club
myonigroup.com navalwiki.info
mzdish.site netallergy.ru
najaden.no netpics.org
narosty.com newsperuse.com
narutonaruto.ru nfljerseyscheapbiz.us
nash-krym.info npoet.ru
neobux-bg.info nvssf.com
netcheckcdn.xyz nzfilecloud.weebly.com
nextrent-crimea.ru offside2.5v.pl
northfacestore.online ok-ua.info
notaria-desalas.com olympescort.com
notasprensa.info omgtnc.com
notebook-pro.ru onload.pw
nucia.biz.ly onlythegames.com
nullrefer.com openfrost.net
offf.info openstat.com
ok.ru opinionreelle.com
okayimage.com origin-my.ru
oneminutesite.it outclicks.net
online-sbank.ru paramountmarble.co.uk
online.ktc45.ru peekyou.com
onstrapon.purplesphere.in penisvergrotendepillennl.ovh
ops.picscout.com percin.biz.ly
ordernorxx.com petsblogroll.com
orel-reshka.net photokitchendesign.com
orsonet.ru pic2fly.com
osnova3.ru
oxotl.com
ozoz.it
pammik.ru
par-fallen.ga
pathwhelp.org
pb-dv.ru
pdn-4.com
picanalyzer.data-ox.com
picsearch.com picsearch.com
picsforkeywordsuggestion.com picturesmania.com
pio.polytopesexempt.com pizdeishn.com
pl-vouchers.com pkr1hand.com
platesauto.com pl.id-forex.com
playtap.us pobeiranie.pl
pokemongooo.ml pochtovyi-index.ru
poligon.com
polimga.pw
polytopesexempt.com
popcash.net popcash.net
pornhub-ru.com popugaychiki.com
pornoted.com popunder.net
portnoff.od.ua popup.matchmaker.com
powc.r.ca.d.sendibm2.com pornmania.pl
powitania.pl pornobrazzers.biz
pozdravleniya-c.ru pornohub.me
pricheskaonline.ru pornokajf.com
produkto.net pornonik.com
professionaldieselcare.com pornowarp.info
profitfx.online pp-budpostach.com.ua
projefrio.com.br primedice.com
priscilarodrigues.com.br
privatefx-in.ru
prizeestates.cricket
prlog.ru
promoheads.com promoheads.com
proposal-engine.com proposal-engine.com
pulseonclick.com prpops.com
pulse33.ru
purplesphere.in
purplestats.com
pussyfleet.com pussyfleet.com
puteshestvennik.com
puzo2arbuza.ru
pxhdwsm.com
questionmarque.ch questionmarque.ch
r-control.ru quick-seeker.com
r-e-f-e-r-e-r.com
rabot.host.sk rabot.host.sk
radiodigital.co rankexperience.com
ranking2017.ga rapidgator-porn.ga
rankscanner.com realmonte.net
razorweb-a.akamaihd.net redbottomheels.xyz
razyboard.com rednise.com
rcb101.ru refads.pro
referencemoi.com registry-clean-up.net
refudiatethissarah.info reklama-i-rabota.ru
relatodelpresente.com.ar remedyotc.com
reversing.cc remont-comp-pomosh.ru
rossmark.ru remont-fridge-tv.ru
ruclicks.com rent2spb.ru
ruex.org.ua rentaremotecomputer.com
russintv.fr rewardpoll.com
sady-urala.ru ringporno.com
sarafangel.ru romhacking.ru
scanmarine.info rucrypt.com
scanner-alex.top rukino.org
scanner-andrew.top ruspdd.com
scanner-jack.top rusvideos.su
scanner-josh.top samara.rosfirm.ru
scanner-julianna.top santasgift.ml
school-diplomat.ru sape.top
scoopquest.com sawin.beth.webd.pl
score-ads.men sbt-aqua.ru
search.pch.com scanner-donald.top
searchmywindow-a.akamaihd.net scanner-jess.top
scanner-marwin.top
scansafe.net
scrapy.org
scripted.com
sdelatmebel.ru
search-goo.com
seccioncontrabajo.com
seksotur.ru
sel-hoz.com sel-hoz.com
seo-tools-optimizing.com
seocheckupx.com
seoheap.com seoheap.com
sexfreepornoxxx.com seorankinglinks.xyz
sexs-foto.top servethis.com
service-core.ru
sethrollins.net
sex-tracker.de
sexkontakte-seite.com
sexpornotales.net
sexsearch.com sexsearch.com
sexy-pings.com sexyali.com
sexyebonyteen.com sharebutton.org
shakhtar-doneck.ru shikiso.info
shillyourcoins.com shillyourcoins.com
shmetall.com.ua shoppingmiracles.co.uk
shodanhq.com shopwme.ru
sicfor.bcu.cc sibtest.ru
sierraapps.com silvercash.com
siteimprove.com simul.co
skanninge.se slowmacfaster.trade
skylta.com snip.tw
smailik.org soc-proof.su
smart-balancewheel.com socialsignals24.com
smart-scripts.com soft-program.com
sneakyboy.com softomix.net
snjatie-geroinovoy-lomki.ru space2019.top
snworks.com spasswelt.xyz
snyatie-lomki-v-stacionare.ru spb-plitka.ru
social-buttons.xyz sports-supplements.us
social-s-hhh.xyz spravka130.ru
sovinsteel.ru stal-rulon.ru
spanking.to statistici.ro
spywarenuker.com steame.ru
starpages.net
stats-public.grammarly.io
sticken.co
stockquotes.wooeb.com
stoki.ru
stop-zavisimost.com
streetfire.net
streha-metalko.si
stroiminsk.org
strongholdsb.ru
sugarlyflex.pw sugarlyflex.pw
symphonyintegratedhealthcare.com suhanpacktech.com
tandvardshuset.net superiends.org
telefonsex-ohne0900.net supermama.top
teletype.in suralink.com
thecounter.com surflinksmedical.com
thedownloadfreeonlinegames.blogspot.com svarbit.com
thefotosgratis.eu svtrd.com
thegalerie.eu taboola.com
thegameriders.com taximytishi.ru
themestotal.com tds-advert005.info
teastory.co
technika-remont.ru
tecnoteakviareggio.it
terrafootwear.us
thenetinfo.com thenetinfo.com
tizanidine4mgtablets.blogspot.com tiandeural.ru
tizanidine6mg.blogspot.com time-japan.ru
tizanidinehclsideeffects.blogspot.com titan-ads.life
tizanidinemusclerelaxant.blogspot.com tizanidineandgabapentin.blogspot.com
tizanidinerestlesslegsyndrome.blogspot.com tizanidinecapsules.blogspot.com
toolsky.com tizanidinedosageforsleep.blogspot.com
top-deal.com.pl tizanidinehcl4mgtablet.blogspot.com
top10-online-games.com tonivedu.it
topshef.ru top1-seo-service.com
torrent-newgames.com totu.us
toy-shop.top toxicwap.com
trafficmania.com track112.site
trafficshaper.com trafficmp.com
traffictrade.life
tripper.de
tryrating.com
tsstcorpcddvdwshbbdriverfb.aircus.com
turbo-suslik.org
tvnewsclips.info tvnewsclips.info
twojebook.pl u17795.netangels.ru
typer.one ukrtextbook.com
uggbootsoutletsale.us ukrtvory.in.ua
ugogo.info um-razum.ru
underthesite.com uprour.com
unionmarkt.de urzedowski.eu
unpredictable.ga usacasino.com
uptime-delta.net usadacha.net
urlcut.ru
usiad.net usiad.net
uytmaster.ru vegan-foods.us
uzungil.com vereo.eu
vchulkah.net vip2ch.com
vellings.info vipms.ru
verymes.xyz
via-energy-cumpara.com
viagra-soft.ru
videochat.tv.br
videochat.world
vielporno.net
vipcallsgirls.com
vipsiterip.org
vitoriacabos.com vitoriacabos.com
vkonche.com voronezh.xrus.org
vrotike.ru vrnelectro.ru
wardreapptokone.tk vysigy.su
warningzscaler.heraeus.com w7s.ru
watracker.net wapsite.me
wedding-salon.net web.cvut.cz
welck.octopis.com webads.co.nz
welcomeauto.ru webextract.profound.net
witherrom55.eklablog.fr website-stealer.nufaq.com
withstandingheartwarming.com websiteexplorer.info
wttavern.com wemedinc.com
wufak.com westum.se
www.kabbalah-red-bracelets.com whatsappdownload10.com
wzgyyq.com whatsupinfoley.com
x-musics.com wieseversa.no
xep.info wleuaprpxuvr.ga
xfluro.com worldhistory.biz
xkaz.org wormix-cheats.ru
xlovecam.com wovis.site
xn----9sbebi2bvzr7h.xn--p1ai writersgroup580.web.fc2.com
xn----itbkqkfiq.xn--p1ai writingservices17.blogspot.ru
xn--80aanaardaperhcem4a6i.com www.wohnkabinen-shop.de
x-porno.video
xaijo.com
xev.ru
xmladserver.com
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn----7sbifcamovvfggw9d.xn--p1ai
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn--80abgj3a5acid6ghs.top xn--80abgj3a5acid6ghs.top
xn--e1afanlbnfckd7c3d.xn--p1ai xsion.net
xn--h1ahbi.com.ua
xn--oogle-wmc.com
xxxkaz.org xxxkaz.org
xxxrus.org
xxxsiterips.xyz xxxsiterips.xyz
yeartwit.com
yebocasino.co.za
yes-com.com yes-com.com
yoshkarola.zrus.org youradulthosting.com
you-shall-not-pass.is74.ru zeroredirect9.com
youdao.com zigarettenonlinekaufen2.bloog.pl
yoursearch.me zmoda.hostreo.com
yugo-star.ru znakomstvaonlain.ru
zahvat.ru zoolubimets.ru
zarabotat-na-sajte.ru zvetki.ru
zarabotok--doma.ru zzbroya.com.ua
zeikopay.com
zhacker.net
zog.link
zoodrawings.com
zoompegs.com

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1688.com 1000n1.ru
3-letter-domains.net 1001desert.com
aa8780bb28a1de4eb5bff33c28a218a930.com 1j7740kd.website
ad-words.ru 1qingdao.com
addtoadd.men 3xforum.ro
adidas.frwebs.fr 4webmasters.com
adultactioncam.com 5000-cotydzien.com
advokateg.ru 6hopping.com
aadroid.net
abcdefh.xyz
actulite.com
adamoads.com
addray.pro
ads-seo.men
adsref.men
adultnet.in
advancedmassagebysara.com
albuteroli.com
alert-fjg.xyz
alert.scansafe.net alert.scansafe.net
alevinefesleri.com alive-ua.com
alfabot.xyz allcredits.su
amazingninja.com alpharma.net
ananumous.ru anapa-inns.ru
androids-store.com animenime.ru
apessay.com anti-virus-removal.info
arabseyes.com apimossnetinfo-a.akamaihd.net
arkartex.ru atmagroup.ru
asacopaco.tk autoplate.info
atlasvkusov.ru avek.ru
autotrends.today bangla-chat-uk-paralud.ga
avtoarenda.by beauty-things.com
bablonow.ru best-games.today
bang-hotties.com bestcurs.org
bedandbreakfast.com bestdooz.com
bestsexyblog.com betterhdporn.com
bestwebsitesawards.com betterscooter.com
blackplanet.com binomo.kz
bloggers.nl blackbabesporn.com
blacktwhite.com
brainzpod.win
brakehawk.com
brk-rti.ru brk-rti.ru
burn-fat.ga bupropion-sr-150-mg.us
buscarfoto.com callcafe.info
businescoop.men cheapjerseysfootballshop.com
candytiner.com check-this-out-now.online
cartierlove2u.xyz chocolateslim-u-srbiji.com
cartierring.xyz ciproandtizanidine.blogspot.com
cdncash.com cleanallvirus.com
certifywebsite.win coolwallpapers4k.info
chatroulette.online credit-online.ws
cl.s7.exct.net
com-secure.download
community.allhiphop.com
compliance-alex.top
compliance-don.xyz
connectionstrenth.com
crd.clan.su
custodieva.ru
cymbaltaandtizanidine.blogspot.com
czeshop.info czeshop.info
data1.scopich.com dame-ns.kz
dentalpearls.com.au darkages.info
dev33.dioniqlabb.se datadepths.com
digest-project.ru dimkino.ru
dildofotzen.net
dimaka.info
dimfour.com
dktr.ru
documentbase.net documentbase.net
dogbreedspicture.net domain-submit.info
dojki365.online domainsatcost.com
dollartree.info doska-vsem.ru
domain-tracker.com downloaddy.net
domainanalyzing.xyz drupa.com
elektrischezi.canalblog.com dtm-spain.com
elektrischezigarettekaufen2.cowblog.fr dumuelave.xyz
elektronischezigarettekaufen2.tumblr.com e-collantes.com
epngo.bz efkt.jp
escortslet.net electronicadirect.com
este-line.com.ua elektrischeziga.livejournal.com
extrabot.com erotikchat-24.com
ferieboligkbh.dk evrotekhservis.ru
flooringinstallation-edmonton.com exonline.info
freewareseek.com extlinks.com
gamingspark.com ezigaretteshop2.mywapblog.com
gate5.co.za fable.in.ua
gateway.zscalerone.net fashion-stickers.ru
getoutofdebtfree.org fearcrow.com
gettpromos.com femmesdenudees.com
gheus.altervista.org fertilitetsradgivningen.se
girls-ufa.ru ff30236ddef1465f88547e760973d70a.com
gogalleryawesome.com findpik.com
gosarhivrt.ru finuse.com
growmyfunds.ca fishtauto.ru
homeandhealth.ru forminecrafters.ru
hptwaakw.blog.fc2.com formseo.com
idealtits.net fr-bearings.ru
ideoworld.org freefoto.ca
ifmo.ru gabeshop.ru
igrovyeavtomaty777.ru genericlowlatencyasiodriverhq.aircus.com
iideaidekonkatu.info getlaid-xxxhookupdirect.com
ilovevitaly.ru glavprofit.ru
images.gyffu.com globalscam.ga
imguramx.pw gotomontenegro.net
imicrovision.com gta-club.ru
infobanks.ru guge.io
inmate-locator.us hamacapty.com
int.search.mywebsearch.com headpress.ru
interesnie-faktu.ru heroz.fr
iscblog.info highspeed5.net
itservicesthatworkforyou.com host-tracker.com
ivanstroi.ru houseofgaga.ru
jamiembrown.com hyiphunter.org
jav-way.site ihc2015.info
javrip.net internetproviderstucson.com
jerseywholesaleelitestore.com investpamm.ru
jongose.ninja invivo.hu
journeydownthescale.info ivoiretechnocom.ci
jovencitas.gratis j33x.com
jyrxd.com jackpotchances.com
kaac.ru jaxcube.info
kino-filmi.com kakadu-interior.com.ua
kino-fun.ru kemerovo.zrus.org
klumba55.ru keyhantercume.com
kountrylife.com kostenlos-sexvideos.com
koversite.info krasivoe-hd.net
ladsblue.com kuzinsp.ru
lexixxx.com landinez.co
likrot.com landmania.ru
lion.conboy.us laptoper.net
live-sexcam.tk leadn.pl
live-sexchat.ru lerporn.info
localflirtbuddies.com letmacwork.world
louboutinbooties.xyz lezbiyanki.net
lubetube.com linkbolic.com
martlinker.com listiseltemournlan.gq
maslenka.kz lost-alpha.ru
master-muznachas.ru lottospring.com
me-ke.com mailemedicinals.com
mebel-vstroika.ru maju.bluesalt.co
mega-bony2017.pl matrixalchemy.com
medanestesia.ru
mediaoffers.click
mediawhirl.net
mhi-systems.ru mhi-systems.ru
michaelkorssaleoutletonline.net miniature.io
mnogabukaff.net minneapoliscopiers.com
mobstarr.com moneyviking-a.akamaihd.net
monclercheap.xyz monthlywinners.com
morlat.altervista.org most-kerch.org
mossmesi.com muzaporn.com
mrinsidesales.com mwtpludn.review
muizre.ru my-floor.in.ua
muznachas-service.ru mybackgroundlandscape.blogspot.com
nacap.ru mydirtystuff.com
net-profits.xyz myhealthcare.com
mymobilemoneypages.com
nalogovyy-kodeks.ru
netcheckcdn.xyz
newfilmsonline.ru newfilmsonline.ru
newtechspb.ru newsperuse.com
nicovideo.jp normalegal.ru
nikitsyringedrivelg.pen.io notebook-pro.ru
noclegonline.info npoet.ru
notaria-desalas.com nuup.info
notfastfood.ru obsessionphrases.com
obiavo.by olgacvetmet.com
oddamzadarmo.eu onetravelguides.com
office-windows.ru onlinadverts.com
onlinemeetingnow.com onlinedomains.ru
oops-cinema.ru onlyforemont.ru
orgasmatrix.com ordernorxx.com
p-business.ru orel-reshka.net
page2rss.com orgasmus-virtual.com
parfusale.se osoznanie-narkotikam.net
parvezmia.xyz paighambot.com
pepperstyle.ru partner-high.men
petedrummond.com partner-host.men
petitions.whitehouse.gov partner-stop.men
petrovka-online.com paydayloanslocal.com
picurams.pw picsearch.com
play-movie.pl pokemongooo.ml
ploenjitmedia.azurewebsites.net porn-w.org
podshipniki-nsk.ru pornoklad.ru
popugauka.ru pornosexrolik.com
pornhub-forum.ga pornstartits.xblog.in
pufip.com presleycollectibles.com
pxhdwsm.com pricheski-video.com
quizzitch.net quebec-bin.com
rankingchart.de r-control.ru
redirlock.com resant.ru
rightenergysolutions.com.au rfserial.net
rocis.site santechnik.jimdo.com
roma-kukareku.livejournal.com sardinie.us
rutor.group scanner-andrew.top
saneyes.com scanner-barak.top
scanmarine.info scanner-don.top
searchinquire.com scenicmissouri.us
sel-hoz.com search-error.com
sell-fb-group-here.com search.1and1.com
seo-2-0.com searchtooknow-a.akamaihd.net
seorankinglinks.xyz sex.hotblog.top
service-core.ru shopcheermakeup.info
servisural.ru simplesite.com
sexobzor.info site5.com
sexpornotales.com sitechecker.pro
sexreliz.com sklad-24.ru
sharpchallenge.com snegozaderzhatel.ru
sitebeam.net snts.shell-pmr.ru
siteonomy.com socialbuttons.xyz
skypasss.com spinnerco.ca
sledstvie-veli.net srdrvp.com
soc-econom-problems.ru
solnplast.ru
solution4u.com
ssconstruction.co
staticfs.host
statistici.ro
steelmaster.lv steelmaster.lv
stop-zavisimost.com strana-krasoty.ru
supermodni.com.ua stroimajor.ru
surfbuyermac.com supernew.org
swingerseiten.com superoboi.com.ua
tacbelarus.ru surgut.zrus.org
talant-factory.ru survival.betteroffers.review
telefonsexi.com tgtclick.com
testbotprocessor44.com thebluenoodle.com
thecoral.com.br theguardlan.com
thefarmergame.com time-japan.ru
thfox.com tizanidinezanaflex.blogspot.com
tizanidineingredients.blogspot.com tn811.us
trafficmania.com topappspro.com
trafficmonetize.org track-rankings.online
trafficstars.com traffic-club.info
tramadolandtizanidine.blogspot.com trafficshaper.com
truck-addzilla.life tt-ipd.info
ultimatesetnewfreeallsoftupgradesystems.pw tv-spoty.info
uptime-alpha.net u-cheats.ru
valueclick.com uchil.net
vbtracker.net ukkelberg.no
vellings.info uptimebot.net
vertaform.com urldelivery.com
video--production.com usiad.net
viel.su vancleefreplica.pw
vipromoffers.com velpanex.ru
visa-china.ru verymes.xyz
vvon.co.uk viddyoze.com
vzlom-na-zakaz.com vitalads.net
warningwar.ru voditeltrezviy.ru
waterefficiency.co volgograd.xrus.org
westen-v.life vroze.com
wetgames.ru vucms.com
whatsappdownload10.com vzlomfb.com
white-truck.life wareseeker.com
woman-orgasm.ru webinstantservice.com
wovis.site website-speed-check.site
wsgames.ru wellcome2slovenia.ru
x-diesel.org westvilletowingservices.co.za
x-musics.com xaijo.com
x-true.info xn--80ahvj9e.xn--p1ai
xxxnatelefon.ru xn--l1aengat.xn--p1ai
you-shall-not-pass.is74.ru xsfetish.org
your-bearings.com xtrafficplus.com
yun56.co yaoguangdj.com
zaloadi.ru yebocasino.com
zeroredirect12.com zeroredirect12.com
zootravel.com zoogdinsney.com
zvezdagedon.ru

153
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,6 +8,79 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -421,76 +494,6 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -581,14 +584,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.30 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3546 ### Version: V4.2023.01.3547
### Updated: Sat Jan 14 22:01:12 UTC 2023 ### Updated: Sun Jan 15 10:03:27 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3546 ### Version: V4.2023.01.3547
### Updated: Sat Jan 14 22:01:12 UTC 2023 ### Updated: Sun Jan 15 10:03:27 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##