vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.01.3548 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-01-15 22:07:11 +00:00
parent 482e335258
commit 043145f416
47 changed files with 104636 additions and 104629 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3547 #### Version: V4.2023.01.3548
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3547 #### Version: V4.2023.01.3548
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.01.3547 #### Version: V4.2023.01.3548
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12934
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.01.3547 #### Version: V4.2023.01.3548
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12938
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12938
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12938
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12938
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12938
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12938
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12938
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3547 3548

12938
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12938
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.176.137.176 0; 13.91.160.241 0;

12938
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12938
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12938
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

12938
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.176.137.176 0; 13.91.160.241 0;

12938
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.176.137.176 0; 13.91.160.241 0;

12938
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AwarioSmartBot ADmantX
BotALot Dotbot
HybridBot Image Fetch
Moblie Safari Kozmosbot
OnCrawl LieBaoFast
Rankivabot LinqiaMetadataDownloaderBot
Teleport Quick-Crawler
Vacuum Reaper
com.plumanalytics SISTRIX
webgains-bot Suzuran

318
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
80legs
AIBOT AIBOT
ALittle Client Abonti
Aboundex Aboundex
Aboundexbot Aboundexbot
AhrefsBot AhrefsBot
AiHitBot Aipbot
AllSubmitter Alexibot
Alligator
Anarchie Anarchie
Anarchy Anarchy
Anthill
Apexoo Apexoo
Aspiegel
Asterias Asterias
Attach Atomseobot
AwarioRssBot BDFetch
BDCbot
BackDoorBot BackDoorBot
Backlink-Ceck
BacklinkCrawler
Badass Badass
BatchFTP BatchFTP
Battleztar Bazinga
Bitacle
BlackWidow
BlowFish
Boardreader Boardreader
Bolt
BotALot BotALot
Brandprotect Brandwatch
BuiltWith Buddy
BuiltBotTough
Bullseye
BunnySlippers BunnySlippers
CCBot
Calculon
CazoodleBot CazoodleBot
CheTeam Cegbfeieh
CheeseBot CheeseBot
CherryPicker CherryPicker
Chlooe ChinaClaw
Citoid Cliqzbot
Cogentbot Collector
CopyRightCheck Crescent
Copyscape Curious
Cosmos Custo
CrazyWebCrawler
CrunchBot
CyotekWebCopy CyotekWebCopy
DBLBot DatabaseDriverMysqli
DIIbot
DSearch
DTS Agent
Demon
Deusu Deusu
Dirbuster DigitalPebble
Discoverybot Disco
Dispatch Discobot
DnyzBot DittoSpyder
DomCopBot DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler ECCP/1.0
Domains Project
Download Wonder
Drip
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
EirGrabber Ebingbong
Exabot Ecxi
ExtractorPro EroCrawler
Extreme Picture Finder ExtLinksBot
FDM Extractor
Ezooms
FemtosearchBot
Fimap
FlashGet
Foobot
Freeuploader Freeuploader
Fuzz Fuzz
FyberSpider
Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot
GermCrawler GermCrawler
Gigabot GetRight
Go-Ahead-Got-It
Gotit
GrabNet GrabNet
Grabber Grafula
GrapeFX HEADMasterSEO
GrapeshotCrawler
HMView
HaosouSpider
Harvest Harvest
HonoluluBot Havij
Hloader
Humanlinks Humanlinks
IDBot
IRLbot
Id-search Id-search
Image Fetch IlseBot
Indy Library Image Sucker
Intelliseek IndeedBot
InterGET InfoNaviRobot
Internet Ninja Internet Ninja
IstellaBot
Jbrofuzz Jbrofuzz
Joomla JetCar
Keybot Translation-Search-Machine Jetty
Leap JikeSpider
LeechFTP Jyxobot
LeechGet Keyword Density
Libwhisker LNSpiderguy
LinkWalker Larbin
Linkbot LexiBot
Lftp
LinkScan
LinkextractorPro LinkextractorPro
LinkpadBot
LinksManager
Lipperhey Spider
Lmspider Lmspider
MIDown tool MJ12bot
MIIxpc
MQQBrowser MQQBrowser
MSIECrawler MSIECrawler
MTRobot Majestic SEO
Magnet Majestic-SEO
Mail.RU_Bot
Majestic12
MarkMonitor MarkMonitor
Mass Downloader
Masscan Masscan
Mata Hari Meanpathbot
Mb2345Browser
MegaIndex.ru
Metauri Metauri
Moblie Safari Microsoft Data Access
MolokaiBot Microsoft URL Control
Mister PiX
Mojolicious
Msrabot Msrabot
NICErsPRO Musobot
NPbot
Nessus Nessus
Net Vampire Net Vampire
NetAnts NetZIP
NetMechanic
Nettrack
Netvibes Netvibes
Niki-bot Nibbler
Nikto
Nimbostratus
Ninja
Nmap Nmap
Nuclei
Octopus
Offline Explorer Offline Explorer
Offline Navigator Offline Navigator
OpenLinkProfiler
OpenVAS OpenVAS
Openfind OrangeBot
OrangeSpider
PECL::HTTP
Page Analyzer
PageAnalyzer PageAnalyzer
PageScorer PageScorer
PageThing.com Pandalytics
Panscient
Papa Foto Papa Foto
Petalbot PeoplePal
Picsearch PictureFinder
Piepmatz
Pimonster
Pixray
PleaseCrawl PleaseCrawl
Pockey
ProPowerBot ProPowerBot
Probethenet
Psbot
PxBroker PxBroker
PyCurl PyCurl
Quick-Crawler Quick-Crawler
RankingBot RankActiveLinkBot
Re-re RankingBot2
SBIder Rankivabot
SEOkicks Reaper
SEOkicks-Robot RedesScrapy
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler
SEOstats
SMTBot
SalesIntelligent SalesIntelligent
Screaming ScoutJet
SemanticJuice SearchmetricsBot
Shodan Seekport
SemrushBot
SeoSiteCheckup
Site Sucker Site Sucker
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteLockSpider
SiteSucker
Sitebeam Sitebeam
Sitevigil Siteimprove
Sociscraper SmartDownload
SpaceBison Snake
Snapbot
SocialRankIOBot
Spammen Spammen
Spbot Spanner
Steeler SputnikBot
Stripper Sqlworm
Sucker
Sucuri Sucuri
SuperBot SuperBot
Suzuran
Swiftbot Swiftbot
T0PHackTeam Telesoft
Telesphoreo
The Intraformant The Intraformant
TheNomad TheNomad
TightTwatBot Titan
Toata
Toweyabot Toweyabot
Tracemyfile Tracemyfile
Trendiction Trendiction
Trendictionbot
True_Robot True_Robot
Turnitin URLy Warning
TurnitinBot UnisterBot
TwengaBot VidibleScraper
VB Project VoidEYE
Vagabondo
VeriCiteCrawler
Voil
Voltron
WBSearchBot WBSearchBot
WISENutbot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWWOFFLE
Wallpapers/3.0
WeSEE
Web Enhancer
Web Fetch
Web Fuck Web Fuck
Web Pix Web Sauger
WebCollage
WebEnhancer
WebImageCollector
WebLeacher
Web Sucker Web Sucker
WebBandit
WebLeacher
WebPix
WebReaper
WebStripper
WebSucker
WebWhacker
WebZIP WebZIP
Webshag Website Quester
WebsiteExtractor WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster
Whacker Whacker
Whatweb
Who.is Bot Who.is Bot
Widow WinHTTrack
WiseGuys Robot
Wonderbot
Woobot
Wotbox Wotbox
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Xenu Zermelo
YoudaoBot
ZmEu
ZoomBot ZoomBot
ZumBot ZyBorg
adscanner adscanner
arquivo.pt arquivo-web-crawler
backlink-check check1.exe
coccocbot cognitiveseo
com.plumanalytics com.plumanalytics
crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j
eCatch
evc-batch evc-batch
facebookscraper facebookscraper
gopher isitwp.com
heritrix lwp-trivial
ips-agent magpie-crawler
iubenda-radar
linkdexbot
mediawords
muhstik-scan muhstik-scan
pcBrowser netEstate NE Crawler
plumanalytics scan.lol
seobility
seoscanners
sexsearcher sexsearcher
siteripz sitechecker.pro
sogouspider sp_auditbot
sysscan webgains-bot
trendiction.com
voyagerx.com
webmeup-crawler webmeup-crawler
webprosbot
x22Mozilla
xpymep1.exe xpymep1.exe
zauba.io zgrab

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404enemy
80legs
ADmantX
AIBOT AIBOT
Abonti Abonti
Aboundexbot
Acunetix Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AiHitBot AiHitBot
Aipbot Alligator
AlphaBot
Anarchie Anarchie
Anarchy99
Apexoo
Aspiegel Aspiegel
Attach Attach
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike
BDFetch BDFetch
BackWeb
Backlink-Ceck
Barkrowler Barkrowler
Black Hole BetaBot
BlackWidow Bigfoot
Blackboard Blackboard
BlowFish BlowFish
Bolt Bolt
Brandprotect BotALot
Buck
BuiltBotTough BuiltBotTough
BuiltWith BuiltWith
Bullseye BunnySlippers
CCBot BuzzSumo
CSHttp Calculon
Cegbfeieh Cegbfeieh
CheTeam CherryPicker
ChinaClaw ChinaClaw
Chlooe Chlooe
Citoid Citoid
Claritybot Cloud mapping
Cliqzbot CopyRightCheck
Cocolyzebot
Copier
Cosmos Cosmos
CrazyWebCrawler
Crescent Crescent
CrunchBot CrunchBot
Custo Custo
CyotekWebCopy
DIIbot DIIbot
DSearch Demon
DTS Agent Dirbuster
Devil
Disco
Discobot Discobot
Discoverybot
Dispatch Dispatch
Domains Project DittoSpyder
DnBCrawler-Analytics
DomainAppender
DomainStatsBot
Download Wonder Download Wonder
Dragonfly Drip
ECCP/1.0
EMail Siphon
EasyDL EasyDL
Ebingbong
Ecxi Ecxi
EirGrabber
EroCrawler EroCrawler
Extractor ExtractorPro
Extreme Picture Finder Extreme Picture Finder
FHscan EyeNetIE
Ezooms
Flunky
Freeuploader
Fuzz
FyberSpider FyberSpider
GalaxyBot Fyrebot
GermCrawler G-i-g-a-b-o-t
GoZilla GT::WWW
Gigabot
Gotit Gotit
Grabber GrabNet
Grafula
GrapeFX GrapeFX
GrapeshotCrawler
HMView
HTMLparser
HTTP::Lite HTTP::Lite
HTTrack HaosouSpider
Havij Harvest
Heritrix
Hloader Hloader
HybridBot
Iblog Iblog
Id-search
Image Fetch Image Fetch
Image Sucker Image Sucker
Indy Library
InternetSeer
Iria
Iskanie
JOC Web Spider
Jbrofuzz
JennyBot JennyBot
Jyxobot Jetty
Kenjin Spider JikeSpider
Keybot Translation-Search-Machine Jorgee
Kinza Kinza
Kozmosbot
LWP::Simple LWP::Simple
Lanshanbot
LeechGet LeechGet
LexiBot Lftp
LibWeb LibWeb
LieBaoFast LieBaoFast
LinkScan LinkScan
LinkWalker
Linkbot Linkbot
LinkextractorPro LinkextractorPro
LinksManager LinkpadBot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
MJ12bot Lipperhey
MSFrontPage MFC_Tear_Sample
Majestic-SEO MSIECrawler
MauiBot MTRobot
Mag-Net
Majestic12
MarkWatch
Mass Downloader
Meanpathbot Meanpathbot
MegaIndex.ru
Metauri Metauri
MicroMessenger Microsoft URL Control
Mister PiX Minefield
Moblie Safari
Mojeek Mojeek
Morfeus Fucking Scanner Mr.4x3
Mozlila Msrabot
Name Intelligence NICErsPRO
Navroad NPbot
Nessus Nessus
Net Vampire Net Vampire
NetAnts
NetLyzer NetLyzer
NetZIP
Netcraft Netcraft
Niki-bot Nimbostratus
NimbleCrawler Not
Ninja Octopus
Nmap OpenLinkProfiler
Nuclei
Nutch
OnCrawl
OpenVAS
Openvas Openvas
OrangeBot OutclicksBot
OrangeSpider OutfoxBot
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageGrabber Panscient
PageScorer Papa Foto
Pavuk
PeoplePal PeoplePal
Petalbot Picscout
Pi-Monster
Picsearch Picsearch
Probethenet PictureFinder
Pu_iN Pimonster
PyCurl Psbot
PxBroker
QueryN Metasearch QueryN Metasearch
RankActive Quick-Crawler
RankingBot RankActiveLinkBot
RankingBot2 RankingBot2
Rankivabot Re-re
RankurBot
RealDownload RealDownload
RedesScrapy Recorder
Ripper RepoMonkey
RocketCrawler RocketCrawler
Rogerbot
SBIder SBIder
SEOlyticsCrawler
SEOstats SEOstats
SISTRIX ScanAlert
SalesIntelligent Scrapy
Scanbot Screaming
ScreenerBot ScrepyBot
Searchestate Seekport
SearchmetricsBot SemanticJuice
SemrushBot
SentiBot SentiBot
SeoSiteCheckup
Siphon Siphon
SiteLockSpider
SiteSnagger
Site Sucker Site Sucker
SiteCheckerBotCrawler
SiteSnagger
Sitebeam Sitebeam
Sitevigil
SmartDownload SmartDownload
Snake
Snoopy
SocialRankIOBot SocialRankIOBot
Sogou web spider Sogou web spider
Sosospider Sosospider
SpaceBison
SpankBot SpankBot
Spanner Spanner
Spinn3r SputnikBot
Sqlmap Sqlmap
Sqlworm Sqlworm
Sqworm Steeler
Stripper Suzuran
Sucuri
SuperHTTP
SurveyBot
T0PHackTeam
T8Abot
Teleport
Telesoft
Telesphorep Telesphorep
The Intraformant The Intraformant
TheNomad TheNomad
TightTwatBot Thumbor
Trendictionbot Toata
True_Robot Toweyabot
Typhoeus Turingos
VCI TurnitinBot
Vacuum UnisterBot
Vagabondo
VeriCiteCrawler
VoidEYE
Voltron
WBSearchBot
WISENutbot WISENutbot
WWW-Collector-E WeSEE
WWW::Mechanize Web Collage
Web Auto
Web Fetch
WebCopier
WebFetch
Web Fuck Web Fuck
WebImageCollector
WebLeacher
Web Pix Web Pix
WebSauger WebAuto
WebCollage
WebEnhancer
WebFuck
WebGo IS
WebReaper
WebStripper WebStripper
WebSucker WebSucker
WebZIP
Webalta Webalta
Website Quester Woobot
WebsiteQuester Wotbox
Whack Wprecon
Whatweb
Widow
Wonderbot
Xaldon_WebSpider Xaldon_WebSpider
Zermelo Xenu
Zeus YoudaoBot
Zade
ZmEu
ZoomBot ZoomBot
ZoominfoBot ZumBot
adscanner
archive.org_bot archive.org_bot
autoemailspider autoemailspider
cah.io.community cah.io.community
check1.exe
clark-crawler
cognitiveseo cognitiveseo
com.plumanalytics com.plumanalytics
crawl.sogou.com crawler.feedback
crawler4j
demandbase-bot demandbase-bot
eCatch
facebookscraper facebookscraper
gopher gopher
instabid heritrix
ips-agent internetVista monitor
iubenda-radar lwp-request
linkdexbot lwp-trivial
meanpathbot meanpathbot
mediawords oBot
muhstik-scan
netEstate NE Crawler
pcBrowser
polaris version
probe-image-size
s1z.ru s1z.ru
sogouspider seoscanners
sp_auditbot seostar
ubermetrics-technologies.com serpstatbot
webprosbot sitechecker.pro
x22Mozilla trendiction.com
voyagerx.com
webmeup-crawler
webpros.com
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
boc.kz 7xc4n.com
firma-legion.ru bmw.afora.ru
geilehausfrauen.net bnt-team.com
japfm.com bouda.kvalitne.cz
maladot.com discountliv.com
popunder.ru olympescort.com
real-time-analytics.com sexspornotub.com
servisural.ru thewebsitetemplate.info
yoopsie.com vapomnoncri.tk
zeroredirect8.com yougotanewdomain.com

914
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 007angels.com
1001watch.com.ua 1-free-share-buttons.com
123movies.love 214jaluwobapef.cf
273-fz.ru 24videos.tv
2x2fan.ru 24x7-server-support.site
300richmond.co.nz 28n2gl3wfyb0.ru
3xforum.ro 3dracergames.com
883zy.com 3rbseyes.com
8lufu.com 4istoshop.com
8si.ru 4webmasters.org
aa08daf7e13b6345e09e92f771507fa5f4.com 51unlim.ru
abclauncher.com 5kstore.com
academia-nsk.org 7minuteworkout.com
ad-words.ru 7wind.ru
addblueoff.com.ua 7zap.com
addray.pro 8gold.com
adservme.com 999webdesign.com
advancedcleaner.com aaa38852e886ac4af1a3cff9b47cab6272.com
adventureparkcostarica.com aad1f4acb0a373420d9b0c4202d38d94fa.com
adzpower.com adktrailmap.com
ageofclones.com adnotbad.com
aktivator-windows10.blogspot.com ads-seo.men
alert-hgd.xyz adtech.fr
allmarketsnewdayli.gdn adultgalls.com
ahhjf.com
aimayangzhi.com
alarmobninsk.ru
alfa-img.com
alfa9.com
algerianembassy.co.in
alif-ba-ta.com
allboard.xobor.de
allcredits.su
allsilver925.co.il
alot.com alot.com
alphahoverboards.com alta-realestate.com
aludecor.info altamayoztourism.com
amateurmatch.com amospalla.es
anaksma.info anaksma.info
animalrank.com
animenime.ru
antons-transporte.de antons-transporte.de
apifasterlightin-a.akamaihd.net ap.senai.br
appmsr.org apccargo.com
apptester.tk apibetweenlinesn-a.akamaihd.net
arabsexxxtube.com apioasisspacenet-a.akamaihd.net
arenanews.com.ua apper.de
asrvvv-a.akamaihd.net arate.ru
ausmepa.org.au arcadeplayhouse.com
auto.rusvile.lt asdfz.pro
avtovolop.ru asia-forum.ru
ayanaspa.com atlant-auto.info
azartmsl.com autotrends.today
azbaseimages.net autovideobroadcast.com
babyfactory.fr availit.weebly.com
babyguns.ru aviapanda.ru
baixar-musicas-gratis.com aviva-limoux.com
avtointeres.ru
azartniy-bonus.com
bablonow.ru
baladur.ru baladur.ru
bambi.ck.ua bank.uz
battle.net beauty-clinic.ru
battlecarnival.su bestchoice.cf
bausparen.bz.it bestprofits-there.com
bayareaaeroclub.org bidr.trellian.com
becuo.com bif-ru.info
benchmarkcommunications.co.uk bimatoprost-careprost.com.ua
bestfortraders.com bizcheapjerseyswholesalechina.com
bestmouthwash.club bizlist.com.de
bestwaystogetpaid.us blackplanet.com
beta.hotkeys.com boc.kz
betonka.pro bombla.org
bgviagrachrx.com brandedleadgeneration.com
bigames.online brimstonehillfortress.org
bigcareer.info browserprotecter.com
biketank.ga bupropion-sr-150-mg.us
binomo.com buynorxx.com
bitcoinpile.com bvlgaribracelet.xyz
bjgugu.net.ua bvlgariring.xyz
blackhatworth.com bvlgariwallet.xyz
bloggers.nl cactussoft.biz
blogsfunky672.weebly.com
blpmovies.com
blue-square.biz
bmw-ark.ru
bonkers.name
boobsimge.com
bookmark4you.com.biz
brandov.ru
bryansk.zrus.org
buron.pw
buttons-for-your-website.com
buy-cheap-pills-order-online.com
buyfriend.ru
c-english.ru
cacheimages.com
camdenmemorials.com
camdolls.com camdolls.com
camsex.xxx-cam.webcam candcstuccoandstone.com
carrauterie.be cannibalcheerleader.com
carsplate.com carabela.com.do
catalogs-parts.com cartierlove2u.com
cayado.snn.gr cartierring.xyz
cdn.walkme.com cartujano-pre.de
cheap-pills-norx.com cashonads.com
castingbank.ru
caulderwoodkennels.com
cbcseward.com
cheapjerseysbizwholesale.us cheapjerseysbizwholesale.us
cialisovercounteratwalmartusa.com choice-credit.ru
cigarpass.com cialischstgerts.com
clash-clans.ru cleanersoft.com
classiquebijoux.ru cleanpcnow.com
claytransformations.info clickbank.net
clickbanksites.info clickprotects.com
clknsee.com clickso.com
clodo.ru cobaltpro.ru
com-cleaner.systems colehaanoutlet.store
com-onlinesupport.host compliance-fred.top
compliance-mark.top compliance-jessica.top
concealthyself.com compliance-willy.top
concordexoticrentals.com concordexoticrentals.com
cordstrap.cc cosmeticswomens-womensports.rhcloud.com
corta.co courtshipgift.com
counterbot.com
crd.clan.su crd.clan.su
creatives.adbetclickin.pink creativehutindia.com
creditnation.ru
crest-poloski.ru
crest3d.ru crest3d.ru
culturevie.info cryptoswap.biz
cupday.com cubs-jerseys.us
cvety24.by culmimg.pw
danschawbel.com custom-electric-guitar.com
cutalltheshit.com
daooda.com daooda.com
daptravel.com
data-mining.tk
data1.scopich.com data1.scopich.com
dating-app.ru dating-time-now.com
de.zapmeta.com daymusam.com
dentfidemountpreach.tk db.speedup.gdn
dikx.gdn deda-moroza-zakaz.ru
disruptingdinnerparties.com dekoration.us
dojki365.online derevesendeco.com
domaincheck.io designdevise.com
dorratex.tn dev.citetick.com
dgfitness.co
dimkino.ru
dinosaurus.site
divci-hry.info
dms-sw.ru
domaineaneblanc.com
domainroam.win
download-walpaperhd.blogspot.com
doyouknowtheword-flummox.ml
dpihatinh.gov.vn
dvrlists.com dvrlists.com
eachdayisagift.review e-buyeasy.com
eas-seo.com ebooknovel.club
easytuningshop.ru ege-essay.ru
ecxtracking.com eksprognoz.ru
electro-prom.com electro-prom.com
elektronischezigarettekaufen2.tumblr.com elektrischezigarette1.onsugar.com
elektrozigarettekaufen1.devhub.com elektrozigarette2.wordpressy.pl
elite-sex-finders.com elektrozigaretten1.tumblr.com
elvel.com.ua elkacentr.ru
eropho.net
errorfixing.space
essay-writing.work
essaypro.com essaypro.com
europages.com.ru everytalk.tv
european-torches.ru exdocsfiles.com
everflora.ru exlarseva.webblog.es
exoneration-project.us exportshop.us
extlinks.com extener.org
ezigarettenshop2.postbit.com ezigarettekaufen2.yolasite.com
farm26.ru f07.de
fashion-mk.net familienzahnaerzte.com
fba-mexico.com fanoboi.com
feargames.ru fapgon.com
fermersovet.ru favoritemoney.ru
fickluder69.com feorina.ru
filkhbr.com fickenbumsen.net
fillmewithhappiness.com fickkontaktehobbyhuren.com
fishtauto.ru filefilter.weebly.com
flytourisme.org filesdatabase.net
forensicpsychiatry.ru finansov.info
forms-mtm.ru findsexguide.com
formulaf1results.blogspot.com finemanteam.com
formularz-konkurs.tk fingerprintjs.com
foto-weinberger.at finstroy.net
firstsiteguide.com
flashbannernow.com
flashlarevista.com
florida-tourism.net
flprog.com
footbalive.org
forminecrafters.ru
formseo.com
foxweber.com foxweber.com
free-deals.faith frcls.fr
free-share-buttons.com
freejabs.com freejabs.com
freemags.cc freewlan.info
freewebs.com frequiry.com
freshdz.com galaxys6manual.info
freshmac.space gallerily.com
freshwallpapers.info gamewrath.com
friendflnder.com gamingspark.com
fullfileaccess.com gaz-voshod.ru
fun-mobi.pl ge0ip.org
fxgallery.com gerhardhealer.com
g33.org girls-ufa.ru
gabeshop.ru globetrotting-culture.ru
galaxy-family.ru goforexvps.com
game300.ru golden-catalog.pro
gamebackyard.com golmau.host.sk
gamerscorps.com gosarhivrt.ru
gate5.co.za graphid.com
gavuer.ru greatzip.com
gayxperience.com gstatey.net
geile-lelly.eu gwebtools.com
generousdeal-a.akamaihd.net gymi.name
getmyads24.com headpress.ru
gidonline.one helpmymacfaster.trade
girlsatgames.ru high-speed1.net
given2.com
gojiberriess.apishops.ru
goldandcard.ru
gtopstats.com
handicapvansarea.com
handsandlegs.ru
hdapp1008-a.akamaihd.net
hdfreeporno.net
hdimagelib.com
hdwallpapers-free.com
healbio.ru
hermesbelts.xyz
heroz.fr
hifidesign.ru
highland-homes.com highland-homes.com
home.myplaycity.com highstairs-a.akamaihd.net
hotsocialz.com histats.com
houston-vikings.com hit-men.men
hoverboard360.nl hitmuzik.ru
hmmm.cz
hobild.net
hopeonthestreet.co.uk
hostcritique.com
hotblognetwork.com
hoterika.com
hptwaakw.blog.fc2.com
i4track.net i4track.net
ilikevitaly.com iboss.com
incest-ru.com idc.com.ua
inet-traffic.com ido3.com
infowarcraft.ru iflycapetown.co.za
intellektmedia.at impresagaia.it
investpamm.ru increasewwwtraffic.info
it-max.com.ua instasexyblog.com
iusstf.org iskalko.ru
izoll.ru iwanttodeliver.com
jackpotchances.com iweblist.info
janavibekken.no jacago.com
jerseywholesalechinabiz.com janettabridal.com
keywordblocks.com jasonpartington.com
keywordhouse.com javrip.net
keywordhut.com jbl-charge.info
kicknights.gq jjbabskoe.ru
kimcurlrvsms.com jolic2.com
kino2018.club jongose.ninja
kinocccp.net junketjuice.blogspot.com
kinoflux.net kamalsinha.com
kazinogames.lv
kchaxton.com
kino-ecran.ru
kinobaks.com
kinopolet.net kinopolet.net
kirov.zrus.org kmgamex.cf
krafte.ru komp-pomosch.ru
kreativperlen.ch komputers-best.ru
ktotut.net konkursov.net
laapp.com konkursowo-24.pl
lenpipet.ru kopihijau.info
lerporn.info kozhniebolezni.com
lesbian.xyz krasnodar.zrus.org
letslowbefast.site kupiproday.com.ua
kvartira-sutochno.com
kwzf.net
landmania.ru
landscaping.center
larger.io
letmacworkfaster.site
life-instyle.com life-instyle.com
lifeinsurancekb.com lion.bolegapakistan.com
link.ac liran-locks.com
lionking-1994.blogspot.com liveshoppersmac.com
longadventure.com lollypopgaming.com
lombia.com
luckyday.world luckyday.world
mactechinfo.info luxup.ru
mailemedicinals.com m0r0zk0-krava.ru
mararoom.ru mabdoola.blogspot.com
marinetraffic.com magda-gadalka.ru
marsgatan.com malls.com
maridan.com.ua
maslenka.kz maslenka.kz
mastershef.club mastershef.club
matchpal-a.akamaihd.net matrixalchemy.com
mb140.ru max-eclat.men
mbiologi.ru maximilitary.ru
mebelcomplekt.ru maxxtor.eu
megagrabber.ru mearns-tractors.co.uk
meltwater.com mebel-alait.ru
mere.host.sk microsoftportal.net
metallosajding.ru misswell.net
metaxalonevstizanidine.blogspot.com mncrftpcs.com
missis.top moesexy.com
mobilemedia.md
modabutik.ru
moi-glazki.ru moi-glazki.ru
mojpregled.com mojaocena.com
monclerboots.xyz montredemarque.nl
more-letom.ru moroccosurfadventures.com
mtmtv.info mossmesi.com
mttwtrack.com moz.com
musezone.ru mp3ritm.top
musicas.baixar-musicas-gratis.com mug-na-chas-moscow.ru
myanyone.net music.utrolive.ru
mybackgroundlandscape.blogspot.com muycerdas.xxx
myblogregistercm.tk mxgetcode.com
mydownlodablefiles.com mykings.pw
mysexpics.ru myprintscreen.com
nalogovyy-kodeks.ru mysex21.com
national-today-winning-winner.club na-telefon.biz
navalwiki.info navalwiki.info
netallergy.ru neodownload.webcam
netpics.org nerudlogistik.ru
newsperuse.com netanalytics.xyz
nfljerseyscheapbiz.us newfilmsonline.ru
npoet.ru next-dentists.tk
nvssf.com nfljerseys.online
nzfilecloud.weebly.com nisuturnetdgu.tk
offside2.5v.pl njpalletremoval.com
ok-ua.info noumeda.com
olympescort.com novosti-hi-tech.ru
omgtnc.com nullrefer.com
odesproperty.com
oil-td.ru
okeinfo.online
onload.pw onload.pw
onlythegames.com
openfrost.net
openstat.com openstat.com
opinionreelle.com ordernorxx.com
origin-my.ru ororodnik.goodbb.ru
outclicks.net panicatack.com
paramountmarble.co.uk partnerads.men
peekyou.com perm.xrus.org
penisvergrotendepillennl.ovh photo-clip.ru
percin.biz.ly picscout.com
petsblogroll.com pillscheap24h.com
photokitchendesign.com piuminiita.com
pic2fly.com pixelrz.com
picsearch.com pk-pomosch.ru
picturesmania.com pl.aasoldes.fr
pizdeishn.com platesauto.com
pkr1hand.com play-movie.pl
pl.id-forex.com pogosh.com
pobeiranie.pl pokemongooo.ml
pochtovyi-index.ru polcin.de
poligon.com pornophoto.xyz
polimga.pw pornosexrolik.com
polytopesexempt.com pornoted.com
popcash.net pornoxxx.com.mx
popugaychiki.com pornozhara.com
popunder.net portadd.men
popup.matchmaker.com portside.cc
pornmania.pl potolokelekor.ru
pornobrazzers.biz pozdravleniya-c.ru
pornohub.me privat-girl.net
pornokajf.com
pornonik.com
pornowarp.info
pp-budpostach.com.ua
primedice.com
priscilarodrigues.com.br
privatefx-in.ru
prizeestates.cricket
prlog.ru prlog.ru
promoheads.com prodess.ru
profitfx.online
prohoster.info
prom23.ru
promodj.com
proposal-engine.com proposal-engine.com
prpops.com proxyelite.biz
pulse33.ru pukaporn.com
pulseonclick.com
purplesphere.in purplesphere.in
purplestats.com quangcaons.com
pussyfleet.com queerspace.com
puteshestvennik.com ranking2017.ga
puzo2arbuza.ru rating-bestcasino.com
pxhdwsm.com rating-casino2021.ru
questionmarque.ch razleton.com
quick-seeker.com realting-moscow.ru
rabot.host.sk rebelmouse.com
rankexperience.com recinziireale.com
rapidgator-porn.ga
realmonte.net
redbottomheels.xyz
rednise.com
refads.pro
registry-clean-up.net
reklama-i-rabota.ru reklama-i-rabota.ru
remedyotc.com remontbiz.ru
remont-comp-pomosh.ru remorcicomerciale.ro
remont-fridge-tv.ru remybutler.fr
rent2spb.ru rewardit.com
rentaremotecomputer.com richinvestmonitor.com
rewardpoll.com rp9.ru
ringporno.com rvi.biz
romhacking.ru s-forum.biz
rucrypt.com sa-rewards.co.za
rukino.org sajatvelemeny.com
ruspdd.com sammlungfotos.online
rusvideos.su samoiedo.it
samara.rosfirm.ru satoristudio.net
santasgift.ml sbdl.no
sape.top scanner-don.top
sawin.beth.webd.pl scanner-irvin.top
sbt-aqua.ru scoopquest.com
scanner-donald.top secretscook.ru
scanner-jess.top semxiu.com
scanner-marwin.top seo-tools-optimizing.com
scansafe.net seo-traffic-ranking.info
scrapy.org seocdvig.ru
scripted.com seokicks.de
sdelatmebel.ru seonetwizard.com
search-goo.com sexpartygirls.net
seccioncontrabajo.com sexphoto.site
seksotur.ru sibecoprom.ru
sel-hoz.com site3.free-share-buttons.com
seoheap.com slavkokacunko.de
seorankinglinks.xyz
servethis.com
service-core.ru
sethrollins.net
sex-tracker.de
sexkontakte-seite.com
sexpornotales.net
sexsearch.com
sexyali.com
sharebutton.org
shikiso.info
shillyourcoins.com
shoppingmiracles.co.uk
shopwme.ru
sibtest.ru
silvercash.com
simul.co
slowmacfaster.trade
snip.tw snip.tw
snjatie-geroinovoy-lomki.ru
snts.shell-pmr.ru
snworks.com
sobecjvuwa.com.ru
soc-proof.su soc-proof.su
socialsignals24.com socialmonkee.com
soft-program.com soft-program.com
softomix.net spaceshipad.com
space2019.top sportbetfair.com
spasswelt.xyz stair.registrydomainservices.com
spb-plitka.ru strana-solnca.ru
sports-supplements.us stroicol.net
spravka130.ru studiofmp.com
stal-rulon.ru studworks.org
statistici.ro sukirgenk.dvrlists.com
steame.ru superoboi.com.ua
sugarlyflex.pw svnuppsalaorebro.se
suhanpacktech.com
superiends.org
supermama.top
suralink.com
surflinksmedical.com
svarbit.com
svtrd.com svtrd.com
taboola.com tam-gde-more.ru
taximytishi.ru taximytishi.ru
tds-advert005.info td-l-market.ru
teastory.co telegraf.by
technika-remont.ru thedownloadfromwarez.blogspot.com
tecnoteakviareggio.it theendivechronicles.com
terrafootwear.us titslove.yopoint.in
thenetinfo.com
tiandeural.ru
time-japan.ru
titan-ads.life
tizanidineandgabapentin.blogspot.com
tizanidinecapsules.blogspot.com tizanidinecapsules.blogspot.com
tizanidinedosageforsleep.blogspot.com tizanidineformigraines.blogspot.com
tizanidinehcl4mgtablet.blogspot.com tizanidineingredients.blogspot.com
tonivedu.it tmtrck.com
top1-seo-service.com toondinsey.com
totu.us tooplay.com
toxicwap.com trafficjunky.com
track112.site traxdom.ru
trafficmp.com trkdf.com
traffictrade.life truck-addzilla.life
tripper.de trumpetedextremes.com
tryrating.com tupper-shop.ru
tsstcorpcddvdwshbbdriverfb.aircus.com u-cheats.ru
turbo-suslik.org
tvnewsclips.info
u17795.netangels.ru u17795.netangels.ru
ukrtextbook.com uchebavchehii.ru
ukrtvory.in.ua ucoz.ru
um-razum.ru uggbootsoutletsale.us
uprour.com ultimateclassicrock.com
urzedowski.eu unitygame3d.com
usacasino.com unrealcommander.com
usadacha.net urdoot.win
urlcut.ru
usiad.net usiad.net
vegan-foods.us uvozdeckych.info
vereo.eu uzporno.mobi
vip2ch.com vbikse.com
vipms.ru vekzdorov.ru
vitoriacabos.com ventelnos.com
voronezh.xrus.org video-chat.love
vrnelectro.ru vids18.site
vysigy.su vielporno.net
viveresaniesnelli.it
vkak.ru
vkontarkte.com
vsexkontakte.net
vut.com.ru
w7s.ru w7s.ru
wapsite.me wahicbefa31.soup.io
web.cvut.cz wallpaperstock.net
webads.co.nz watchinf.com
webextract.profound.net watchmyfb.pl
website-stealer.nufaq.com web-analytics.date
websiteexplorer.info website-speed-checker.site
wemedinc.com weightatraining.com
westum.se welck.octopis.com
whatsappdownload10.com whatsappbot.flyland.ru
whatsupinfoley.com www.kabbalah-red-bracelets.com
wieseversa.no xjlottery.com
wleuaprpxuvr.ga xkaz.org
worldhistory.biz xlolitka.com
wormix-cheats.ru xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
wovis.site xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
writersgroup580.web.fc2.com xn--1-8sbcpb0bdm8k6a.xn--p1ai
writingservices17.blogspot.ru xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
www.wohnkabinen-shop.de xn--80aikhbrhr.net
x-porno.video xn--80ak6aa92e.com
xaijo.com xn--oogle-wmc.com
xev.ru yachts-cruise.info
xmladserver.com yoluxuryevents.com
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai yoshkarola.zrus.org
xn----7sbifcamovvfggw9d.xn--p1ai youbrainboost.asia
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai youporn-forum.ga
xn--80abgj3a5acid6ghs.top
xsion.net
xxxkaz.org
xxxrus.org
xxxsiterips.xyz
yeartwit.com
yebocasino.co.za
yes-com.com
youradulthosting.com youradulthosting.com
youresponsive.com
youtube-downloader.savetubevideo.com
yur-p.ru
zaapplesales.blogspot.com
zakazfutbolki.com
zazagames.org
zeroredirect12.com
zeroredirect2.com
zeroredirect7.com
zeroredirect9.com zeroredirect9.com
zigarettenonlinekaufen2.bloog.pl
zmoda.hostreo.com zmoda.hostreo.com
znakomstvaonlain.ru zytpirwai.net
zoolubimets.ru zzlgxh.com
zvetki.ru
zzbroya.com.ua

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1000n1.ru 2020iscoming.info
1001desert.com 2nt.ru
1j7740kd.website 4ureyesonly.com
1qingdao.com 5kstore.com
3xforum.ro 7minuteworkout.com
4webmasters.com aa08daf7e13b6345e09e92f771507fa5f4.com
5000-cotydzien.com actices.com
6hopping.com alf-img.com
aadroid.net alfa-img.com
abcdefh.xyz altamayoztourism.com
actulite.com ap.senai.br
adamoads.com arendakvartir.kz
addray.pro arraty.altervista.org
ads-seo.men articlesdirectoryme.info
adsref.men asophoto.com
adultnet.in astana.xxxkaz.org
advancedmassagebysara.com autonew.biz
albuteroli.com aviapanda.ru
alert-fjg.xyz avtointeres.ru
alert.scansafe.net badavit.com.ua
alive-ua.com bank.uz
allcredits.su bankofthewext.com
alpharma.net batanga.net
anapa-inns.ru bestadbid.com
animenime.ru besthoro.ru
anti-virus-removal.info
apimossnetinfo-a.akamaihd.net
atmagroup.ru
autoplate.info
avek.ru
bangla-chat-uk-paralud.ga
beauty-things.com
best-games.today
bestcurs.org
bestdooz.com
betterhdporn.com betterhdporn.com
betterscooter.com bird1.ru
binomo.kz biznesschat.net
blackbabesporn.com blackplanet.com
blacktwhite.com blog.koreadaily.com
brainzpod.win bmw-ark.ru
brakehawk.com boleznikogi.com
brk-rti.ru bonzybuddy.com
bupropion-sr-150-mg.us brendbutik.ru
callcafe.info brianhenry.co
cheapjerseysfootballshop.com buchananshardware.com
check-this-out-now.online bukleteg.xyz
chocolateslim-u-srbiji.com businescoop.men
ciproandtizanidine.blogspot.com butstrap.space
cleanallvirus.com bvlgaribracelet.xyz
coolwallpapers4k.info cacheimages.com
credit-online.ws cascadelink.org
czeshop.info ceotrk.com
dame-ns.kz cinemaenergy-hd.ru
darkages.info citizenclsdriveri7.pen.io
datadepths.com classicakuhni.ru
dimkino.ru clipartnew.com
documentbase.net collegeessay19.blogspot.ru
domain-submit.info compliance-donald.xyz
domainsatcost.com connectingsingles.com
doska-vsem.ru coslab.club
downloaddy.net crafthubs.com
drupa.com customboxes4less.com
dtm-spain.com customcollegeessays.net
dumuelave.xyz darmebel.com.ua
e-collantes.com dataloading.net
efkt.jp dengi-pod-zalog-nedvizhimosti.ru
electronicadirect.com dgfitness.co
elektrischeziga.livejournal.com dirtpics.pw
erotikchat-24.com drugstoreforyou.com
evrotekhservis.ru eduserver.net
exonline.info el-nation.com
extlinks.com elektronischezigarette2.mex.tl
ezigaretteshop2.mywapblog.com elitedollars.com
fable.in.ua elvenar.com
fashion-stickers.ru enbersoft.com
fearcrow.com engines-usa.com
femmesdenudees.com erotiktreff24.info
fertilitetsradgivningen.se estelight.ru
ff30236ddef1465f88547e760973d70a.com eupornstar.info
findpik.com euroskat.ru
finuse.com evidencecleanergold.com
fishtauto.ru exmasters.com
forminecrafters.ru eyelike.com.ua
formseo.com faithe.top
fr-bearings.ru falcon-images.blogspot.com
freefoto.ca familienzahnaerzte.com
gabeshop.ru faracontrol.ir
genericlowlatencyasiodriverhq.aircus.com feel-planet.com
getlaid-xxxhookupdirect.com feeriaclub.ru
glavprofit.ru fickenbumsen.net
globalscam.ga fickkontaktehobbyhuren.com
gotomontenegro.net flytourisme.org
gta-club.ru forexgb.ru
guge.io forum69.info
hamacapty.com freesitetest.com
headpress.ru fres-news.com
heroz.fr front.ru
highspeed5.net fungirlsgames.net
host-tracker.com g.starmoe.xyz
houseofgaga.ru gateway.zscalerone.net
hyiphunter.org gayxperience.com
ihc2015.info gdcentre.ru
internetproviderstucson.com get-free-social-traffic.com
investpamm.ru gne8.com
invivo.hu googleseo.com.tr
ivoiretechnocom.ci gowreckdiving.com
j33x.com greendream.com.ua
jackpotchances.com hd-filmy.net
jaxcube.info hmmm.cz
kakadu-interior.com.ua hotgirlhdwallpaper.com
kemerovo.zrus.org idegenvezeto.eu
keyhantercume.com imagecoolpub.com
kostenlos-sexvideos.com intervsem.ru
krasivoe-hd.net istizanidineacontrolledsubstance.blogspot.com
kuzinsp.ru iwanttodeliver.com
landinez.co journeydownthescale.info
landmania.ru jyrxd.com
laptoper.net kamen-e.ru
leadn.pl karpun-iris.ru
lerporn.info kartiniresto.com
letmacwork.world kazka.ru
lezbiyanki.net keenoutlet.online
linkbolic.com keywords-monitoring-success.com
koopilka.com
krasivoe-hd.com
lankarns.com
lazymanyoga.com
lflash.ru
libertybilisim.com
listiseltemournlan.gq listiseltemournlan.gq
lost-alpha.ru liveu.infoteka.hu
lottospring.com lodki-pvh.dp.ua
mailemedicinals.com lovasszovetseg.hu
maju.bluesalt.co lrac.info
matrixalchemy.com lsex.xyz
medanestesia.ru macnewtech.com
mediaoffers.click makemoneyonline.com
mediawhirl.net mapquestz.us
mhi-systems.ru mecash.ru
miniature.io medicines-choice.com
minneapoliscopiers.com megaindex.ru
moneyviking-a.akamaihd.net menetie.ru
monthlywinners.com methodsmarketing.com
most-kerch.org mex-annushka.ru
muzaporn.com minet.club
mwtpludn.review mobplayer.ru
my-floor.in.ua mobstarr.com
mybackgroundlandscape.blogspot.com morlat.altervista.org
mydirtystuff.com musflashtv.com
myhealthcare.com myfreemp3.eu
mymobilemoneypages.com myhydros.org
nalogovyy-kodeks.ru nalogovyy-kodeks.ru
netcheckcdn.xyz narkologiya-voronezh.ru
newfilmsonline.ru naturalshair.site
newsperuse.com new-post.tk
normalegal.ru newstraveller.ru
notebook-pro.ru next-dentists.tk
npoet.ru niroo.info
nuup.info nofreezingmac.work
obsessionphrases.com norththeface.store
olgacvetmet.com nw-servis.ru
onetravelguides.com oneshotdate.com
onlinadverts.com online-x.ru
onlinedomains.ru online7777.com
onlyforemont.ru
ordernorxx.com ordernorxx.com
orel-reshka.net paceform.com
orgasmus-virtual.com paintingplanet.ru
osoznanie-narkotikam.net panchro.co.uk
paighambot.com parajumpersoutlet.online
partner-high.men park.above.com
partner-host.men physfunc.ru
partner-stop.men pinapchik.com
paydayloanslocal.com play.leadzupc.com
picsearch.com pod-muzyku.club
pokemongooo.ml popunder.net
porn-w.org
pornoklad.ru pornoklad.ru
pornosexrolik.com profit-opportunity.com
pornstartits.xblog.in psoriasis-file.trade
presleycollectibles.com rennlist.com
pricheski-video.com reversing.cc
quebec-bin.com rezeptiblud.ru
r-control.ru rospromtest.ru
resant.ru rp9.ru
rfserial.net sad-torg.com.ua
santechnik.jimdo.com salutmontreal.com
sardinie.us savefrom.com
scanner-andrew.top scanner-irvin.top
scanner-barak.top scottbywater.com
scanner-don.top sdi-pme.com
scenicmissouri.us searchaddis.com
search-error.com
search.1and1.com
searchtooknow-a.akamaihd.net searchtooknow-a.akamaihd.net
sex.hotblog.top seasaltwithfood.com
shopcheermakeup.info semrush.com
simplesite.com seoboxes.com
site5.com seoheap.com
sitechecker.pro servingnotice.com
sklad-24.ru sherlock.se
snegozaderzhatel.ru ship-marvel.co.ua
snts.shell-pmr.ru simul.co
socialbuttons.xyz smartshoppymac.com
spinnerco.ca sofit-dmd.ru
srdrvp.com soft1.ru
steelmaster.lv sptslmtrafms.com
strana-krasoty.ru stariy-baku.com
stroimajor.ru stats-collector.org
supernew.org studiofmp.com
superoboi.com.ua sumo.com
surgut.zrus.org t-bygg.com
survival.betteroffers.review taxi-v-eisk.ru
tgtclick.com tedxrj.com
thebluenoodle.com teletype.in
theguardlan.com teslathemes.com
time-japan.ru thecounter.com
tizanidinezanaflex.blogspot.com thefds.net
tn811.us thesmartsearch.net
topappspro.com thiegs.reco.ws
track-rankings.online tizanidineinfo.blogspot.com
traffic-club.info tkanorganizma.ru
trafficshaper.com tmtrck.com
tt-ipd.info toursmaps.com
tv-spoty.info toy-shop.top
u-cheats.ru track2.shop
uchil.net tuberkulezanet.ru
ukkelberg.no uggbootsoutletsale.us
uptimebot.net ugogo.info
urldelivery.com ukr-lit.com
usiad.net ukrobstep.com
vancleefreplica.pw unblocksit.es
velpanex.ru v720hd.ru
verymes.xyz videotuber.ru
viddyoze.com vipsiterip.org
vitalads.net vod.com.ua
voditeltrezviy.ru vostoktrade.info
volgograd.xrus.org vseigru.one
vroze.com vzglyadriv.kg
vucms.com wardreapptokone.tk
vzlomfb.com
wareseeker.com
webinstantservice.com webinstantservice.com
website-speed-check.site weblo.com
wellcome2slovenia.ru website-speed-up.top
westvilletowingservices.co.za wholesalecheapjerseysfree.com
xaijo.com wleuaprpxuvr.ga
xn--80ahvj9e.xn--p1ai woodyguthrie.se
xn--l1aengat.xn--p1ai www.ehscloud.cn
xsfetish.org xboxster.ru
xtrafficplus.com xgftnlrt.bloger.index.hr
yaoguangdj.com xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
yebocasino.com xn----itbkqkfiq.xn--p1ai
zeroredirect12.com xn--80ak6aa92e.com
zoogdinsney.com xn--b1addnj3cah.xn--p1ai
xn--d1abj0abs9d.in.ua
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
xnxx-n.com
xtubeporno.net
yaderenergy.ru
zacreditom.ru
zatjmuzu.info
zebramart.ru
znaturaloriginal.com
zojirushi-products.ru

153
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,79 +8,6 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -494,6 +421,76 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -584,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.69 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3547 ### Version: V4.2023.01.3548
### Updated: Sun Jan 15 10:03:27 UTC 2023 ### Updated: Sun Jan 15 22:01:13 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3547 ### Version: V4.2023.01.3548
### Updated: Sun Jan 15 10:03:27 UTC 2023 ### Updated: Sun Jan 15 22:01:13 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##