vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 10:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.01.2865 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-01-20 10:09:22 +00:00
parent 03df1de1c8
commit 08817ef8f4
47 changed files with 94000 additions and 94007 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2864 #### Version: V4.2022.01.2865
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [633](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [633](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2864 #### Version: V4.2022.01.2865
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [633](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [633](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.01.2864 #### Version: V4.2022.01.2865
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [633](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [633](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11606
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.01.2864 #### Version: V4.2022.01.2865
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [633](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [633](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11610
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11610
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11610
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11610
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 1; 10.1.0.156 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 0; 10.1.0.156 0;
127.0.0.1 0; 127.0.0.1 0;

11610
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 1; 10.1.0.156 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 0; 10.1.0.156 0;
127.0.0.1 0; 127.0.0.1 0;

11610
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 1; 10.1.0.156 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 0; 10.1.0.156 0;
127.0.0.1 0; 127.0.0.1 0;

11610
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2864 2865

11610
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11610
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.83.246.59 0; 40.114.52.122 0;

11610
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 1; 10.1.0.156 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 0; 10.1.0.156 0;
127.0.0.1 0; 127.0.0.1 0;

11610
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 1; 10.1.0.156 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 0; 10.1.0.156 0;
127.0.0.1 0; 127.0.0.1 0;

11610
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 1; 10.1.0.156 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 0; 10.1.0.156 0;
127.0.0.1 0; 127.0.0.1 0;

11610
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.83.246.59 0; 40.114.52.122 0;

11610
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.83.246.59 0; 40.114.52.122 0;

11610
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 1; 10.1.0.156 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Calculon AfD-Verbotsverfahren
FemtosearchBot ChinaClaw
FyberSpider
GridBot
Iblog Iblog
MauiBot Seekport
WallpapersHD Whatweb
Web Fuck ZyBorg
lwp-request dataforseo.com
scalaj-http eCatch
facebookscraper
iubenda-radar

294
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
80legs 01h4x.com
ALittle Client 360Spider
Acunetix ADmantX
Alligator AIBOT
AlphaBot AfD-Verbotsverfahren
Anarchy99 AhrefsBot
Ankit AiHitBot
Alexibot
Aspiegel
Attach Attach
AwarioRssBot
AwarioSmartBot
BDFetch BDFetch
BackDoorBot BackWeb
Bandit BacklinkCrawler
BatchFTP Barkrowler
Battleztar Bazinga Black Hole
BlackWidow Boardreader
Bolt Brandprotect
Brandwatch
Buck Buck
BuiltWith BuzzSumo
Cegbfeieh
CensysInspect CensysInspect
ChinaClaw ChinaClaw
Citoid Chlooe
Cliqzbot Cliqzbot
Cocolyzebot
Cogentbot
Copier
CopyRightCheck
Cosmos Cosmos
CrunchBot Crawling at Home Project
Custo
CyotekWebCopy CyotekWebCopy
DBLBot DBLBot
DIIbot DSearch
Demon Demon
Deusu
Digincore Digincore
DigitalPebble
Discobot
Discoverybot
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot Dotbot
DomCopBot
Download Wonder Download Wonder
Dragonfly Dragonfly
Drip ECCP/1.0
EMail Siphon
EMail Wolf
Ecxi
EirGrabber EirGrabber
EroCrawler
Exabot Exabot
Express WebPictures
Extractor
ExtractorPro ExtractorPro
Extreme Picture Finder EyeNetIE
Ezooms
FHscan
FemtosearchBot FemtosearchBot
Firefox/7.0
Flunky
Foobot
Freeuploader Freeuploader
FrontPage
Fuzz Fuzz
FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t G-i-g-a-b-o-t
Genieo
GermCrawler GermCrawler
GetWeb GetWeb
Getintent
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
Gotit GoZilla
GrabNet
Grabber
GrapeshotCrawler GrapeshotCrawler
GridBot
HEADMasterSEO HEADMasterSEO
HTMLparser HMView
HTTrack
Haansoft Haansoft
HaosouSpider
Harvest
Havij
Heritrix
Hloader Hloader
IDBTE4M IDBot
Iblog
Id-search
IlseBot IlseBot
Image Sucker Image Fetch
Intelliseek Indy Library
Jbrofuzz InfoNaviRobot
InfoTekies
InterGET
Internet Ninja
IstellaBot
JamesBOT
JennyBot JennyBot
Joomla Joomla
Jyxobot JustView
Kenjin Spider
Keyword Density Keyword Density
Kinza Kinza
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot
Larbin Larbin
Leap Leap
LexiBot LexiBot
Lftp
LieBaoFast LieBaoFast
Lightspeedsystems
LinkScan LinkScan
LinkpadBot Linkdexbot
LinksManager LinksManager
LinqiaRSSBot Lipperhey
Lmspider Lipperhey Spider
Litemage_walker
MIDown tool MIDown tool
MTRobot MIIxpc
MJ12bot
MSIECrawler
Mag-Net Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic-SEO Majestic-SEO
MarkMonitor Majestic12
Mb2345Browser Masscan
MeanPath Bot Mata Hari
Meanpathbot Meanpathbot
Metauri Mediatoolkitbot
MegaIndex.ru
MicroMessenger
Moblie Safari Moblie Safari
Mojeek
Mr.4x3 Mr.4x3
Msrabot Msrabot
NPbot NICErsPRO
Navroad Nameprotect
NearSite Nessus
Needle Net Vampire
NetAnts
NetLyzer
NetMechanic NetMechanic
NetSpider
NetZIP
Netcraft
Nettrack Nettrack
Netvibes Netvibes
Niki-bot Niki-bot
Nikto
NimbleCrawler
Nimbostratus
Nutch Nutch
Offline Explorer
OpenLinkProfiler OpenLinkProfiler
OpenVAS OpenVAS
Openvas Openvas
OrangeBot
OutfoxBot OutfoxBot
POE-Component-Client-HTTP Page Analyzer
PageAnalyzer PageScorer
PageGrabber Papa Foto
Pandalytics
Panscient
PeoplePal
Pi-Monster Pi-Monster
Piepmatz Picscout
Pixray Pixray
PleaseCrawl Psbot
ProPowerBot
Probethenet
Pu_iN
Pump Pump
QueryN Metasearch PxBroker
Quick-Crawler PyCurl
Rankivabot RankActiveLinkBot
RankingBot2
RankurBot
Re-re Re-re
ReGet ReGet
RebelMouse RebelMouse
Recorder
RepoMonkey
RocketCrawler RocketCrawler
Rogerbot Rogerbot
SEOkicks
SEOprofiler
SISTRIX SISTRIX
SMTBot SMTBot
SalesIntelligent SalesIntelligent
Scanbot ScanAlert
ScoutJet ScreenerBot
Scrapy ScrepyBot
Searchestate SemanticJuice
SearchmetricsBot
Seekport
Semrush
SentiBot SentiBot
SeobilityBot SeoSiteCheckup
Shodan Seomoz
SiteSnagger
SiteSucker SiteSucker
Siteimprove Siteimprove
Sitevigil Sitevigil
Snapbot
Snoopy Snoopy
SocialRankIOBot SocialRankIOBot
Sogou web spider Sociscraper
SpankBot SpaceBison
Spbot Spanner
Spinn3r Spinn3r
Sqlworm Sqworm
Stripper
Sucuri Sucuri
Surfbot SuperBot
Swiftbot Teleport
Szukacz
T8Abot
TeleportPro
Telesoft Telesoft
Telesphorep Telesphorep
TheNomad Thumbor
Titan TightTwatBot
Toata Tracemyfile
Toweyabot Turingos
Trendiction
TurnitinBot TurnitinBot
TwengaBot
Twice
UnisterBot UnisterBot
VCI Vacuum
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Virusdie Virusdie
VoidEYE
Voil
Voltron
WASALive-Bot
WEBDAV WEBDAV
WISENutbot
WPScan
WWW::Mechanize WWW::Mechanize
Wallpapers/3.0 Wallpapers
WallpapersHD WebAuto
Web Collage WebFetch
Web Enhancer WebFuck
Web Sucker WebGo IS
WebImageCollector WebImageCollector
WebPix
WebReaper
WebStripper WebStripper
WebSucker
WebZIP WebZIP
Webalta Webalta
WebmasterWorldForumBot
Website Quester
WebsiteExtractor WebsiteExtractor
Whack Webster
Who.is Bot Widow
WinHTTrack Xenu
WiseGuys Robot
Wonderbot
Woobot
Wotbox
Xaldon WebSpider
YoudaoBot YoudaoBot
Zauba Zauba
Zitebot
ZmEu ZmEu
ZoominfoBot ZoomBot
adscanner adscanner
arquivo.pt arquivo-web-crawler
backlink-check autoemailspider
cah.io.community cah.io.community
coccocbot-web cognitiveseo
crawl.sogou.com com.plumanalytics
eCatch crawler.feedback
evc-batch demandbase-bot
instabid domainsproject.org
internetVista monitor heritrix
iubenda-radar isitwp.com
magpie-crawler lwp-trivial
mediawords netEstate NE Crawler
muhstik-scan oBot
pcBrowser
probe-image-size probe-image-size
seobility ripz
scan.lol
seoscanners seoscanners
serpstatbot siteripz
sogouspider sogouspider
sysscan
tAkeOut tAkeOut
trendiction.com trendiction.com
voyagerx.com
webgains-bot webgains-bot
webprosbot webmeup-crawler
x09Mozilla webpros.com
zauba.io zauba.io
zgrab

300
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker 404checker
80legs 404enemy
ADmantX Abonti
AIBOT
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot AhrefsBot
Aipbot Alexibot
AllSubmitter
AlphaBot
Anarchie Anarchie
Anarchy
Anarchy99 Anarchy99
Apexoo
Asterias Asterias
AwarioSmartBot
BBBike BBBike
BDCbot BDCbot
BackDoorBot BackStreet
BackWeb
Backlink-Ceck
Barkrowler Barkrowler
Battleztar Bazinga
BetaBot BetaBot
Black Hole BlackWidow
Blackboard
Blow Blow
BlowFish Boardreader
Buck Buck
Buddy BunnySlippers
Bullseye CATExplorador
BuzzSumo Chlooe
CCBot Citoid
CODE87 Cloud mapping
CazoodleBot
Cegbfeieh
CheeseBot
Claritybot
Cogentbot Cogentbot
Copier Copier
Cosmos CopyRightCheck
CrazyWebCrawler Copyscape
Crescent Crescent
CrunchBot Curious
Custo
CyotekWebCopy CyotekWebCopy
DIIbot DTS Agent
DatabaseDriverMysqli DatabaseDriverMysqli
Demon Demon
Deusu
Devil Devil
Dirbuster Digincore
Discobot Dispatch
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot
DomainCrawler
DomainStatsBot
Domains Project
Dotbot Dotbot
Download Wonder Drip
Dragonfly ECCP/1.0
EirGrabber EMail Siphon
Evil Ebingbong
Ecxi
Exabot
Express WebPictures Express WebPictures
EyeNetIE Extractor
ExtractorPro
Ezooms Ezooms
FHscan FemtosearchBot
Fimap
FlashGet FlashGet
Flunky Foobot
Fuzz
FyberSpider
GT::WWW
GalaxyBot
GermCrawler GermCrawler
Getintent GetRight
GetWeb
Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
GrabNet Grafula
GridBot GrapeFX
Haansoft GrapeshotCrawler
Hloader HEADMasterSEO
HTMLparser
HaosouSpider
Harvest
Havij
Heritrix
Humanlinks Humanlinks
HybridBot IDBot
IRLbot Id-search
IlseBot IlseBot
Image Fetch
Image Sucker
IndeedBot
InfoNaviRobot InfoNaviRobot
Intelliseek Intelliseek
IstellaBot
JOC Web Spider
JamesBOT
Jbrofuzz
JennyBot JennyBot
Jetty JustView
Joomla Jyxobot
Jorgee Kenjin Spider
Keyword Density
Kinza Kinza
Kozmosbot Kozmosbot
LNSpiderguy
LeechFTP LeechFTP
LeechGet
LexiBot
LibWeb LibWeb
Libwhisker Lightspeedsystems
LieBaoFast Linkdexbot
LinkScan
LinkWalker
LinkextractorPro LinkextractorPro
LinkpadBot
LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
Ltx71
MIDown tool MIDown tool
MIIxpc MIIxpc
MJ12bot
MSFrontPage
MSIECrawler MSIECrawler
MTRobot MTRobot
Mag-Net
Magnet
Mail.RU_Bot
Majestic-SEO
Majestic12 Majestic12
Mass Downloader Mass Downloader
Masscan Masscan
Mata Hari MauiBot
MeanPath Bot Mb2345Browser
Meanpathbot MegaIndex.ru
Metauri
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Morfeus Fucking Scanner Moblie Safari
Mr.4x3 Mojolicious
NPbot Mozlila
Musobot
NICErsPRO
Nameprotect Nameprotect
Needle
Nessus Nessus
Net Vampire Net Vampire
Netcraft NetAnts
NetMechanic
Netvibes
Nibbler Nibbler
Niki-bot
Nikto Nikto
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Nuclei Nuclei
Nutch Offline Explorer
Octopus
OpenLinkProfiler OpenLinkProfiler
OutclicksBot OpenVAS
Openfind
Openvas
OrangeBot
OrangeSpider
OutfoxBot
POE-Component-Client-HTTP POE-Component-Client-HTTP
Picsearch Page Analyzer
PageScorer
PageThing.com
Panscient
Petalbot
Pimonster Pimonster
Pixray
PleaseCrawl
ProPowerBot
RSSingBot RSSingBot
RankActiveLinkBot RankActive
RankingBot2 RankingBot
RankurBot
Reaper Reaper
Recorder Recorder
RedesScrapy
RepoMonkey RepoMonkey
Ripper Ripper
Rogerbot SEOlyticsCrawler
SEOstats SEOprofiler
SalesIntelligent ScanAlert
ScoutJet Scanbot
Scrapy ScrepyBot
Screaming SearchmetricsBot
ScreenerBot SemanticJuice
SemrushBot SemrushBot
SeobilityBot SentiBot
Shodan Shodan
Siphon
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteSnagger SiteSnagger
SiteSucker Sociscraper
Sitevigil
SlySearch
Snake
Sogou web spider Sogou web spider
Sosospider
Sottopop
SpaceBison SpaceBison
Spammen
SpankBot
Spanner
Spinn3r Spinn3r
Sucker Steeler
SuperBot Sucuri
SuperHTTP SurveyBot
TeleportPro Swiftbot
Toata Szukacz
T8Abot
Thumbor
TightTwatBot
Tracemyfile
Trendiction Trendiction
Turnitin
Twice Twice
URLy.Warning UnisterBot
Upflow
V-BOT
VB Project VB Project
Vacuum VeriCiteCrawler
VidibleScraper Virusdie
VoidEYE VoidEYE
Voltron
WASALive-Bot
WEBDAV WEBDAV
WISENutbot WISENutbot
WPScan WPScan
WWW::Mechanize WWW-Mechanize
WWWOFFLE Wallpapers
Web Pix Wallpapers/3.0
Web Sucker WallpapersHD
WebCopier Web Auto
WebEnhancer WebAuto
WebFuck WebBandit
WebImageCollector
WebPix WebPix
WebReaper
WebSauger
WebStripper
WebSucker WebSucker
WebWhacker WebWhacker
WebZIP WebmasterWorldForumBot
Whacker WebsiteQuester
Whatweb
Widow Widow
WinHTTrack
Wonderbot Wonderbot
Xaldon WebSpider Woobot
Xaldon_WebSpider Xenu
Zade Zauba
Zermelo Zeus
Zitebot
ZmEu ZmEu
ZoomBot
ZumBot ZumBot
adscanner adscanner
arquivo.pt arquivo-web-crawler
backlink-check backlink-check
cah.io.community cah.io.community
check1.exe
clark-crawler clark-crawler
cognitiveseo coccocbot-web
com.plumanalytics
crawl.sogou.com
crawler.feedback
crawler4j
dataforseo.com dataforseo.com
demandbase-bot
eCatch eCatch
facebookscraper
gopher gopher
internetVista monitor internetVista monitor
ips-agent iubenda-radar
isitwp.com lwp-request
lwp-trivial
magpie-crawler magpie-crawler
mediawords mediawords
muhstik-scan page scorer
netEstate NE Crawler s1z.ru
oBot scalaj-http
probe-image-size scan.lol
seobility
serpstatbot
sexsearcher
sogouspider sogouspider
sp_auditbot
spyfu
sysscan sysscan
trendiction.com tAkeOut
ubermetrics-technologies.com
webmeup-crawler webmeup-crawler
webpros.com webpros.com
zauba.io webprosbot

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
apiitsthirteende-a.akamaihd.net 08800.top
arabsexxxtube.com carfax.com.ua
finejewelryshop.ru cymbaltaandtizanidine.blogspot.com
jasonpartington.com gonextmedia.com
nippon-bearings.ru naturalshair.site
novgorod.xrus.org pornogratisdiario.com
savememoney.co.za tvoystartup.ru
silverage.ru vchulkax.com
tattooha.com visa-china.ru
zynax.ua x-lime.net

942
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
000free.us 000free.us
0c47f8422d3f.com
0n-line.tv 0n-line.tv
18ps.ru 101lesbian.xyz
1millionusd.xyz 123locker.com
24-ak.ru 2345.com
2drittel.de 256bit.by
2nt.ru 2kata.ru
45en.ru 2rich4bitches.com
51unlim.ru 3th.co.in
57883.net 3weekdiet.com
5kstore.com 4kpics.rocks
777-club.ru 55wmz.ru
7zap.com 5u.com
98oi.ru 7fon.ru
DomainStatsBot aa9d046aab36af4ff182f097f840430d51.com
aa08daf7e13b6345e09e92f771507fa5f4.com aanapa.ru
aa9bd78f328a6a41279d0fad0a88df1901.com absolute-s.ru
aaa38852e886ac4af1a3cff9b47cab6272.com add-add.men
aasoldes.fr ads.gold
above.com adultfullhd.com
activepr.ru adultwebhosting.info
adamoads.com advokat-grodno.by
addtoadd.men ahhjf.com
admanaerofoil.com aimayangzhi.com
adprotect.net alert.scansafe.net
ads-cool.pro alessandraleone.com
adssafeprotected.com algarveglobal.com
adultactioncam.com allknow.info
agahinameh.com alloysteel.ru
airmaxshoes-2016.com alta-realestate.com
alborzan.com
alfa9.com
allkrim.com
allnews.md
alpharma.net
amazingpic.net
ameblo.jp ameblo.jp
analnoeporno.tv amyfoxfitness.com
angigreene.com
animali.deagostinipassion.it animali.deagostinipassion.it
anime.dougasouko.com animenime.ru
apibrowsesmartne-a.akamaihd.net anonymousfox.co
applicationg29.com apiappenableinfo-a.akamaihd.net
arenanews.com.ua arcteryxsale.online
asiavirtualsolutions.com ardimobileinfo.ml
asrvvv-a.akamaihd.net arvut.org
aucoinhomes.com as5000.com
autoblogger24.info asacopaco.tk
avtovolop.ru asrvrep-a.akamaihd.net
azarttoto.com avon-ukraine.com
babyguns.ru avtoarenda.by
baditri.com ayakino.net
bamo.xsl.pt azte.ch
bannertesting.com babieca.com
bedcapdealers.com backlink4u.net
belinvestdom.by backlinkwatch.com
best-seo-offer.com baersaratov.ru
bestpornuha.com balans.shahterworld.org
bestwrinklecreamnow.com bang-hotties.com
bin-brokers.com bannerbank.ru
bingo8888.com battlecarnival.su
biocn.dx.am beauty-b0x.pl
bestbuy.ca
bestimagecoollibrary.com
bestofferswalkmydogouteveryday.gq
biteg.xyz
biznesschat.net
blagovest-med.ru
blog.koreadaily.com
blog.yam.com
blogig.org blogig.org
bobba.dzaba.com bmwhighperformers.com
bonkers.name boanonihaca.tk
boole.onlinewebshop.net bolegapakistan.com
bradleylive.xyz bookmark4you.com.biz
brandbucket.com bosman.pluto.ro
brandov.ru braindaily.xyz
bronzeaid-a.akamaihd.net breget74.ru
buqayy0.livejournal.com brus-vsem.ru
buypillsorderonline.com bscodecs.com
c-gainsbourg.com burger-imperia.com
cactussoft.biz butstrap.space
caulderwoodkennels.com buytizanidineonlinenoprescription.blogspot.com
cementaresearch.se byme.se
camel-beach.com
candiceloves.us
canrioloadm.gq
cardsharp1.ru
catalogs-parts.com
cdncash.org
celejihad.info
cenoval.ru cenoval.ru
cheapjerseysap.com centraletermice.us
cheapmoncler.xyz cezartabac.ro
checkhit.com cheapcigarettesc.info
cleanallspyware.com checkm8.com
comissionka.net cherrypointplace.ca
compliance-andrew.top chocoslim.pro
chorus.terakeet.com
christianlouboutinreplica.win
cialischmrx.com
ciekawinki.pl
cleansearch.net
clicksotrk.com
clksite.com
club-lukojl.ru
cometorussia.net
compliance-checker.info compliance-checker.info
compliance-fred.top compliance-irvin.top
compliance-irvin.xyz
compliance-jessica.top
compliance-mark.top
compliance-nelson.top compliance-nelson.top
compliance-viktor.top
conboy.us
concordexoticrentals.com concordexoticrentals.com
contentlook.co courtshipgift.com
coolgramgoods.com crackzplanet.com
coolyarddecorations.com creams.makeforum.eu
cougfan.info cupday.com
counciltally.com dailyrank.net
country-chic.ru danceuniverse.ru
covid-schutzmasken.de data.vtc.pw
crystalslot.com data1.scopich.com
csgo4.win daydream-studio.ru
culmimg.pw decenttools.com
cutalltheshit.com demian.kz
cv.wallhade.co demon-tweeks.com
cvety24.by designdevise.com
cwetochki.ru
daisye.top
danschawbel.com
darkages.info
dcj-nn.ru
dear-diary.ru
dekorkeramik.ru
detki-opt.ru detki-opt.ru
dickssportinggoods.com diamond-necklace.info
digital-sale.su dimfour.com
dildofotzen.net dipstar.org
djstools.com discountbarbour.online
dlya-android.org dividendo.ru
dms-sw.ru dollartree.info
dnepr-avtospar.com.ua domoysshop.ru
downloadkakaotalk.com domznaniy.ru
dostavka-v-krym.com
dotnetdotcom.org
doublepimp.com
downtuptv.gq
downvids.net downvids.net
doxysexy.com dptaughtme.com
doyouknowtheword-flummox.ml drupa.com
dpihatinh.gov.vn dvd-famille.com
duawitchrarato.tk e-commerce-seo.com
dumpsmania24.com e90post.com
ekosmetyki.net.pl ebonyporn.site
elektrischezigarette1.blog.pl ecookna.com.ua
elkacentr.ru elektrischezigaretten1.blogse.nl
eloconcream.blogspot.com elektronischezi.livejournal.com
elvenar.com emediate.eu
elvenmachine.com enge-fotzen.info
englate.com enpolis.ru
eonpal.com erotische-geschichten-xxl.com
erot.co eshop4u.jp
escortslet.net essaypro.com
essayassist.com etotupo.ru
estelight.ru etur.ru
eurosamodelki.ru eu-cookie-law.info
excitacion.info evening-dating-club.info
ez8motelseaworldsandiego.com evogarage.com
facialporntube.com exct.net
fapgon.com exdocsfiles.com
fatmaelgarny.com executehosting.com
feorina.ru expertblog.info
finejewelryshop.ru f00kclan.de
fiuxy.com factorynightclub.com
flipper.top fashion-stickers.ru
flytourisme.org fdzone.org
foodcrafts.website filmania-x.ru
fortevidyoze.net filmidivx.com
fotoxxxru.com findpik.com
foxinsocks.ru findthe.pet
firesub.pl
flagcounter.me
flavors.me
flooringinstallation-edmonton.com
flprog.com
fm-upgrade.ru
fr.netlog.com
frankofficial.ru
freejabs.com freejabs.com
freemaintenancesysforpcandmac.top freelifetimefuckbook.com
freenom.link freemags.cc
fz139.ttk.ru freza-sverlo.ru
gabeshop.ru g33.org
gagrasector.ru galleryawesome.com
gambarkatabaru.com gamedayassist.com
gamedayhouse.com gateway.zscalerone.net
gazoblok.net.ua gazporno.com
gearcraft.us generic-pills-online.com
geileweiber.tk
gepezz.info
get-free-traffic-now.com get-free-traffic-now.com
getmiro.com getlamborghini.ga
gigixo.com gettpromos.com
globetrotting-culture.ru gilsonchiro.xyz
glopages.ru go2album.com
go2jump.org goforexvps.com
gogps.me greatfind-a.akamaihd.net
gopro-online.info
gotomontenegro.net
gox.com.ua
gq-catalog.gq
greatdealshop.com
greenzaim.ru greenzaim.ru
gtopstats.com gsasearchengineranker.xyz
guiadeserraazul.com gsasearchenginerankerhelp.com
haikuware.com gwhwpxbw.bloger.index.hr
hamacapty.com habermetre.com
hatedriveapart.com handicapvansarea.com
hawaiielectriclight.com hanwei.us
hdseriale.pl hd-filmy.net
hdwallpapers-free.com headpharmacy.com
hdxnxxtube.mobi healthcarestore.info
healmytrauma.info helpmymacfaster.trade
help.tpu.ru helvetia.com.ua
high-speed1.net hikesearch.net
histats.com hjaoopoa.top
hitsbox.info homedecoguide.info
hollywoodactress.info homelygarden.com
hotchatdate.com homemature.net
hotelcrocenzi.sm hosting-tracker.com
hoverboard360.at hotblog.top
hoverboard360.de
hoverboardforsaledirect.com
howopen.ru howopen.ru
hplaserjetpdriver8y.pen.io hplaserjetpdriver8y.pen.io
huimin764128.com huhn.altervista.org
hyiphunter.org humanelydrew.com
ideawheel.com hut1.ru
immigrational.info hydropump.su
importchinacoach-teach.com ideibiznesa2015.ru
increasewwwtraffic.info ido3.com
infobanks.ru ifmo.ru
innodgfdriverhm.aircus.com ilmen.net
internetproviderstucson.com ilovevitaly.xyz
intim-uslugi.info impotentik.com
ip-guide.com intermesh.net
iqupdatetmz.win investmac.com
iron-age.info iplusbit.blogspot.co.za
islamtoday.co.za isoveti.ru
ispac.org itrevolution.cf
ix20.ru izhevsk.zrus.org
iyasimasennka.com jacago.com
japan-bearings.ru
jasonpartington.com
jav-fetish.site
jav-idol.com jav-idol.com
jimmychoosale.online javitas.info
jjbabskoe.ru javxxx18.com
jrcigars.com jerseywholesaleelitestore.com
juliadiets.com jetsli.de
jyrxd.com joingames.org
kaidalibor.de joyceblog.top
kamagragelusa.net kemerovo.zrus.org
karting196.ru kinoflux.net
kerch.site kiprinform.com
keywordbasket.com komputers-best.ru
keywords-monitoring-your-success.com konica.kz
keywordsking.com kraljeva-sutjeska.com
kicknights.gq kredit-pod-zalog-krasnodar.ru
kinobest.pl kvartir-remont.biz
kit-opt.ru lafourmiliaire.com
kol-energo.ru lankarns.com
konkursowo-24.pl larchik.net
kostenlos-sexvideos.com lechenie-gemorroya.com
kountrylife.com lion.snapmap.us
krassh.ru liumimgx.pw
ktotut.net livefixer.com
kustanay.kz livejasmin.com
laserpen.club lomb.co
letolove.ru m3gadownload.pl
letto.by mac-shield.com
likesdesign.com machicon-ueno.info
liveu.infoteka.hu magnetpress.sk
lombia.co marblestyle.ru
lomza.info marktforschung-stuttgart.com
lsp-awak-perikanan.com matematikus.info
lunchrock.co mebel-alait.ru
luxmagazine.cf mebel-ekb.com
macdamaged.tech meble-bogart.info
mackeeper-center.club meccadumps.net
madot.onlinewebshop.net meddesk.ru
magicdiet.gq mediaoffers.click
mainhp.net medicine-4u.org
make-money-online.com medosmotr-ufa.ru
mall.uk memberty.com
managerpak204.weebly.com merryhouse.co.uk
maranbrinfo.com.br mesbuta.info
marketingvici.com microsearch.ru
marsgatan.com mir-business-24.ru
matpre.top mmgq.ru
mebeldekor.com.ua moje-recenze.cz
medizinreisen.de mojowhois.com
meet-flirt-dating.com mojpregled.com
mega-polis.biz.ua monclercoats.xyz
megagrabber.ru monclervests.xyz
mexicotravelnet.com mowser.com
mezaruk.info mustwineblog.com
minecraft-neo.ru muztops.ru
mirtorrent.net myblogregistercm.tk
missis.top mydirtyhobby.com
mobsfun.net myftpupload.com
mockupui.com mygameplus.com
moesen-ficken.com myseoconsultant.com
monclerjacketsoutlet.win namecrumilchlet.tk
most.gov.iq netallergy.ru
movpod.in netfacet.net
moxo.com newtechspb.ru
mrpornogratis.xxx njpalletremoval.com
muabancantho.info nonews.co
mundoaberrante.com npoet.ru
my-aladin.com nvformula.ru
mypets.by oballergiya.ru
mysexpics.ru obiavo.com
nacap.ru offer.wpsecurity.website
narco24.me onemactrckr.com
national-today-winning-winner.club onlinemeetingnow.com
naturtreenspicerx.pw onlineslotmaschine.com
needtosellmyhousefast.com onlyforemont.ru
news-readers.ru oren-cats.ru
nikitabuch.com osb.se11.ru
niroo.info ourtherapy.ru
nl.netlog.com
nlfjjunb5.ru
nordvpn.com
norththeface.store
notebook-pro.ru
novatech.vn
novosibirsk.xrus.org
nrv.co.za
nullrefer.com
nuup.info
offers.bycontext.com
office-windows.ru
olympescort.com
openlibrary.org
osnova3.ru
owathemes.com owathemes.com
page2rss.com page2rss.com
palvira.com.ua
paperwritingservice17.blogspot.ru
partner-high.men partner-high.men
partnerline.men partner-trustworthy.men
partners-ship.pro pastaleads.com
pateaswing.com pavlodar.xkaz.org
pdamods.ru pawli.eu
pc-virus-d0l92j2.pw
pekori.to
pensplan.com pensplan.com
petitions.whitehouse.gov petrovka-online.com
piccdata.com phobia.us
picsfair.com php-market.ru
pipki.r.acdnpro.com picmoonco.pw
pirateday.ru pijoto.net
piter.xrus.org pizza-imperia.com
pl-top.pl pk-services.ru
platinumdeals.gr pl.aasoldes.fr
playfortuna-play.ru plastgran.com
pogruztehnik.ru plastweb.ru
pornhive.org platezhka.net
pornhubforum.tk pobeiranie.pl
pornoted.com pochtovyi-index.ru
pornstartits.xblog.in pogodnyyeavarii.gq
portal-eu.ru pojdelo.weebly.com
pozvonim.com pomoc-drogowa.cba.pl
poptool.net
pornhub-ru.com
pornosemki.info
pornoslive.net
pornotubs.com
pospr.waw.pl
pourvous.info
pozdrawleniya.com
pozdrawleniya.ru
pr0fit-b0x.com
presleycollectibles.com presleycollectibles.com
princevc.com princevc.com
privatamateure.com privacylocationforloc.com
pronekut.com prize44.com
pronorm.fr profolan.pl
proprostatit.com proposal-engine.com
prostitutki-yaroslavlya.org putevka24.ru
push-ad.com qcstrtvt.bloger.index.hr
pussyfleet.com ranksignals.com
puteshestvennik.com
puzo2arbuza.ru
puzzleweb.ru
qualitymarketzone.com
quelle.ru
quick-seeker.com
quickbuck.com
ranksonic.org ranksonic.org
reelheroes.net razvratnoe.org
regdefense.com recordpage-a.akamaihd.net
registrydomainservices.com redirectme.net
remote-dba.de refads.pro
renewablewealth.com registry-clean-up.net
rentaremotecomputer.com remontvsamare.su
rentehno.ru retreatia.com
rep-am.com rezeptiblud.ru
rethinkwasteni.info ros-ctm.ru
rimedia.org russkie-gorki.ru
rique.host.sk sale-japan.com
rockprogblog.com scanmyphones.com
rustic-quiver.win scanner-jack.top
sardinie.us scanner-nelson.top
sayyoethe.blogspot.co.za scanner-olga.top
sbprabooks.com scanner-viktor.top
scanner-brian.top sdrescher.net
scanner-john.top se-welding.ru
scansafe.net se.bnt-team.com
scanspyware.net search-goo.com
searchaddis.com seasonvar.ru
searchinterneat-a.akamaihd.net seinterface.com
searchtooknow-a.akamaihd.net selectads.men
security60-e.com semrush.com
seminarygeorgia59.ga seoexperimenty.ru
seorankinglinks.us seorank.info
seozoom.it seotoolsagency.com
servethis.com sex-sex-sex5.com
sexgalleries.top sexgalleries.top
sexpornotales.com sexiporno.net
sexs-foto.com sexkrasivo.net
sexyali.com sexobzor.info
shariki-zuma-lines.ru sexsaoy.com
sharpchallenge.com sextracker.com
sherlock.se sexyteens.hol.es
shop-electron.ru shareyards.com
sibdevice.ru shopvilleroyboch.com.ua
simon3.ru sideeffectsoftizanidine.blogspot.com
siteonomy.com sigmund-freud.co.uk
skutecznetabletkinaporostwlosow.pl sildenafilcitratemed.com
slavia.info silvercash.com
smadihome.com simpoed.ufop.br
smartbalanceworld.com sinestesia.host.sk
smartshoppymac.com sirpornogratis.xxx
sneakyboy.com siteaero.com
sophang8.com smart-balancewheel.com
sortthemesitesby.com smichovbike.cz
sosdepotdebilan.com smsactivator.ru
spravka130.ru socialmonkee.com
spywarebegone.com solmarket.by
sonnikforme.ru
souvenir.cc
soviet-portal.do.am
spammen.de
specialfinanceoffers.com
spidtest.org
spidtest.space
splendorsearch-a.akamaihd.net
srdrvp.com srdrvp.com
staff.prairiesouth.ca srgwebmail.nl
starpages.net startwp.org
stmassage.ru statistici.ro
streetfire.net sterva.cc
stretchmate.net sticken.co
stroy-portal22.ru stroiminsk.com
stuffpride.com sundrugstore.com
success-seo.com superstarfloraluk.com
suggest-keywords.com svarkagid.com
sunflowerdrawingpaintings.blogspot.com svnuppsalaorebro.se
superkanpo.com tacbibirfa.tk
svensk-poesi.com taqywu51.soup.io
tanieaukcje.com.pl td-l-market.ru
tarad.com teenbbw.yopoint.in
tds-advert002.info tehngr.ru
telsis.com telemetryverification.net
texbaza.by thaismartloan.com
themeforest.net the-usa-games.blogspot.com
thetattoohut.com thebitcoincode.com
timdreby.com themestotal.com
titslove.yopoint.in thepornsex.org
tizanidinedruginteractions.blogspot.com theprofitsmaker.net
tizanidinefibromyalgia.blogspot.com tizanidine4mg.blogspot.com
tizanidinehcl4mgisitanarcotic.blogspot.com todohr.com
tizanidinehclsideeffects.blogspot.com top-study.work
tizanidineinteractions.blogspot.com topflownews.com
tizanidinevsvalium.blogspot.com tourismvictoria.com
tkanorganizma.ru tri-slona.org
toondinsey.com trubywriting.com
toonfamilies.net trustl.life
torrent-newgames.com ttsq.fr
trion.od.ua tvnewsclips.info
trkur.com ua-company.ru
truck-land.life ukrobstep.com
try-rx.com url-img.link
tvand.ru usbggettwku.ga
tvteleport.ru ussearche.cf
ugogo.info vapomnoncri.tk
ukr-lit.com vekzdorov.ru
unpredictable.ga velen.io
unrealcommander.org vereo.eu
uprour.com video-camer.com
urdoot.win videokrik.net
url2image.com videos-for-your-business.com
uvozdeckych.info vip-parfumeria.ru
v720hd.ru vip.51.la
vabasa.inwtrade.com vipromoffers.com
vacuumcleanerguru.com volgograd.xrus.org
vapsy.com voprosotvet24.ru
videochat.life vzlomfb.com
videosbox.ru walkme.com
vipms.ru wareseeker.com
visionwell.com.cn watchinf.com
vksex.ru web.cvut.cz
vremya.eu webadvance.club
vroze.com webalan.ru
vvpg.ru wedding0venues.tk
vzglyadriv.kg wesharepics.com
web-analytics.date
webix.me
webmasterhome.cn
webtherapy.ru
welcomeauto.ru
wesharepics.site
whengirlsgowild.com whengirlsgowild.com
williamrobsonproperty.com whiteelephantwellington.com
wingsofrefuge.net winner7777.net
winx-play.ru withstandingheartwarming.com
wonderfulflowers.biz wonderfulflowers.biz
wrz0iuebwhp5fg.freeddns.com woodyguthrie.se
word-vorlagen.net
worldmusicfests.com
wosik-dach.service-for-web.de
www.event-tracking.com www.event-tracking.com
www.souvenirua.com x-true.info
xchat26.myfreecams.com xcombear.ru
xjrul.com xitjw.info
xmladserver.com xkaz.org
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai xn--80ahvj9e.xn--p1ai
xn----7sbho2agebbhlivy.xn--p1ai xolodremont.ru
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai xportvusbdriver8i.snack.ws
xn--80aafb2a.xn--p1ai xxxkaz.org
xn--e1afanlbnfckd7c3d.xn--p1ai yaaknaa.info
xn--l1aengat.xn--p1ai ynymnwbm.bloger.index.hr
xn--lifehacer-1rb.com youdesigner.kz
xsfetish.org youjizz.vc
xvideosbay.com yourserverisdown.com
xvideospornoru.com zahvat.ru
yougame.biz zaim-pod-zalog-krasnodar.ru
yougetsignal.com zaimite.ru
yubikk.info zaloadi.ru
yurgorod.ru zeroredirect.com
yuweng.info zoogdiesney.com
za-music.mymobiplanet.com zygophyceous.womanstars.site
zapiszto.pl
zebramart.ru
zeroredirect6.com
zigarettenonlinekaufen2.bloog.pl
zzlgxh.com

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123movies.love 2020iscoming.info
202ch.com 21h2o.com
777-club.ru 273-fz.ru
7fon.ru 2kata.ru
98oi.ru 2x2fan.ru
aa14ab57a3339c4064bd9ae6fad7495b5f.com 34.gs
ad-words.ru 3rbseyes.com
adf.ly 55wmz.ru
agecheckadult.com 76brighton.co.uk
alert-jdh.xyz 7a2rnuey1tw9ar.ru
all-streaming-media.com a.frcls.fr
alta-realestate.com adanih.com
animali.deagostinipassion.it adktrailmap.com
apartamentwroclaw.eu adrunnr.com
apiholdingmypage-a.akamaihd.net adult3dgames.com
arabseyes.com adultfullhd.com
articlesdirectoryme.info agahinameh.com
asiavirtualsolutions.com agroeconom.kz
astana.xxxkaz.org alborzan.com
atlasvkusov.ru amazon-adsystem.com
aucoinhomes.com amehdaily.com
auto-zapchasti.org analnoeporno.tv
aversis.net anonymous-redirect.com
b-styles.xyz apiqualitinknet-a.akamaihd.net
baladur.ru autoseo-traffic.com
barbourjackets.online aviav.eu
basisches-wasser.net aviav.org
beachtoday.ru awaybird.ru
beauby.ru azarttoto.com
babs.com.ua
bag77.ru
balkanfarma.org
baltgem.com
bank.uz
bannerspace.com
bestdraws.com bestdraws.com
bestofpicture.com bestimagecoollibrary.com
bestserials.com betonka.pro
besttorrentknifta.weebly.com bidbuy.co.kr
bibys.com bjanshee.ru
bio-optomarket.ru blog.axant.it
bmsco.co bmw-ark.ru
bochemit.com.ua bracketsmackdown.com
boleznikogi.com
boobsimge.com
braindaily.xyz braindaily.xyz
buyessaynow.biz businesn.men
bycontext.com candytiner.com
cartierlove.xyz canoncdriverq3.pen.io
cartierlovestore.xyz cashkitten-a.akamaihd.net
cdn.walkme.com cenokos.ru
cementaresearch.se centrdebut.ru
centraletermice.us
ceotrk.com ceotrk.com
cgi2.nintendo.co.jp chcu.net
chorus.terakeet.com cheapmoncler.pw
christianlouboutinreplica.win cheapwebsitehoster.com
ciceros.co christianlouboutinsaleonline.us
cleanpcnow.com cialgenisrx.com
cmhomestayagency.com cityadspix.com
com-secure.download citysecurity.nu
compliance-don.top clickzzs.nl
compliance-julia.top continent-e.tv
comsysnet.com cornomase.win
cookie-law-enforcement-hh.xyz counciltally.com
crazyboost.pro
cyclobenzaprinevstizanidine.blogspot.com cyclobenzaprinevstizanidine.blogspot.com
czeshop.info d0t.ru
date-withme.com darmebel.com.ua
descargar-musica-gratis.net davebestdeals.com
digitalassetmanagement.site dealwifi.com
downvids.net delfin-aqua.com.ua
doxyporno.com depositfiles-porn.ga
dvr.biz.ua detmebel.su
e-biznes.info detskie-zabavi.ru
elidelcream.weebly.com digital-video-processing.com
eng-lyrics.com dimkino.ru
exoneration-project.us diusyawiga.tk
extrabot.com donvito.unas.cz
ezigarettekaufen1.hpage.com dotmass.top
ezigarettenkaufen1.deviantart.com dstroy.su
fastcrawl.com eaplay.ru
ferretsoft.com ecookna.com.ua
ferrotodo.com ed-shop01.ru
fighrofacciufreesig.ga enbersoft.com
findanysex.com encodable.com
get-free-social-traffic.com englate.com
getaclueamerica.com estibot.com
getyourimage.club everypony.ru
goodwinmetals.co excaliburfilms.com
gopro-online.info fbdownloader.com
gotwebsite1.com fdzone.org
greendream.com.ua fertilitetsradgivningen.se
greenidesign.co flagcounter.me
handsandlegs.ru flipper.top
hanwei.us floating-share-buttons.com
hcate.com forumprofi.de
hdpixion.com forums.toucharcade.com
hildinghr.se free-gluten.ru
himazin.info free-traffic.xyz
hotblog.top gameonasia.com
hotloans.ru get-free-traffic-now.com
ifirestarter.ru gilbertbanda.net
igithab.com gk170.ru
ilmexico.com gnuetella.com
innoslicon.com goldadpremium.com
inome.com.ua goodhousekeeping.com
insta-add.pro gymi.name
internetproviderstucson.com hdapp1008-a.akamaihd.net
isitwp.com honyaku.yahoofs.jp
itronics.ca howmuchdoestizanidinecost.blogspot.com
jamiembrown.com hybrid.ru
joy-penguin.com i-midias.net.br
jrpmakati.com images-free.net
kakadu-interior.com.ua imgarit.pw
ketoanhanoi.info imguramx.pw
keywordsking.com importchinacoach-teach.com
keywordspay.com infogame.name
konica.kz infoupdate.org
koronirealestate.gr investyb.com
kotaku.com iomoio.net
ks1234.com istizanidinelikexanax.blogspot.com
larutti.ru itag.pw
ledpolice.ru izhevsk.xrus.org
link.ac juliaworld.net
lion.misba.us kleine-titten.biz
littlecity.ch kmd-pto.ru
lombardfinder.ru labelwater.se
lazymanyoga.com
liky.co.ua
louboutinreplica.xyz
m1media.net
mactechinfo.info
magda-gadalka.ru
mahnwachen-helfen.info
manyresultshub-a.akamaihd.net
mararoom.ru mararoom.ru
maslenka.kz marsgatan.com
matrixalchemy.com mb140.ru
mcadamssupplyco.com medkritika.ru
members.ghanaweb.com minecraft-neo.ru
menstennisforums.com mobilemedia.md
mezaruk.info mockupui.com
mmog-play.ru mojaocena.com
mmoguider.ru monarchfind-a.akamaihd.net
moesexy.com monsterdivx.com
moivestiy.biz mundoaberrante.com
montredemarque.nl mustat.com
morefastermac.trade muz-baza.net
moscow.xrus.org my-big-family.com
mp3s.club mysex21.com
mrbojikobi4.biz newyorkhotelsmotels.info
mrvideospornogratis.xxx nhl09.ru
musflashtv.com norththeface.store
muznachas-service.ru obesidadealgarve.com
nextbackgroundcheck.gq oil-td.ru
nextrent-crimea.ru omoikiri-japan.ru
nhl17coinsforps3.gratisblog.biz one.net.in
nikhilbahl.com online-hd.pl
nootrino.com onlinebay.ru
noumeda.com oyster-green.com
nvformula.ru pastaleads.com
oconto.ru pc-test.net
offtime.ru pdamods.ru
oneclickfiles.com pila.pl
onlainbesplatno.ru pix24x7.com
optikremont.ru pooleroadmedicalcentre.co.uk
ostrovtaxi.ru poppen-nw.net
paperwritingservice17.blogspot.ru pornobrazzers.biz
parajumpersoutlet.online pornolook.net
parlament.biz pornosmola.info
partner-stop.men pourvous.info
parvezmia.xyz pricheskaonline.ru
pc-services.ru privatamateure.com
pdn-4.com proposal-engine.com
pekori.to prostitutki-tyumeni.org
pensplan.com purchasepillsnorx.com
pizza-tycoon.com quebec-bin.com
plaff-go.ru rabot.host.sk
plastweb.ru realitykings.com
play.leadzupc.com redbottomheels.xyz
portnoff.od.ua regionshop.biz
pregnant.guru rospromtest.ru
printie.com russia-tao.ru
privacylocationforloc.com sajatvelemeny.com
prospekt-st.ru scanner-walt.top
ragecash.com seccioncontrabajo.com
rankia.com sei80.com
ranksonic.org seoexperimenty.ru
rcb101.ru sex-tracker.com
reining.lovasszovetseg.hu sexy-pings.com
reklamuss.ru sexy-screen-savers.com
rique.host.sk sexy.babes.frontend-stack.top
rozalli.com shinikiev.com.ua
rusprostitute.com shlyahten.ru
russia-today-video.ru sispe.com.br
russkie-gorki.ru skachat-besplatno-obrazcy.ru
scanner-alexa.top slavia.info
scanner-irvin.top smart-balancewheel.com
scanner-josh.top socialbookmarksubmission.org
secret.xn--oogle-wmc.com socialbutton.xyz
see-your-website-here.com sonata-arctica.wz.cz
sevenstars7.com sovinsteel.ru
sexiporno.net spy-sts.com
sexkvartal.com staynplay.net
sheerseo.com strana-krasoty.ru
shijian.ac.cn sukirgenk.dvrlists.com
sizeplus.work svolze.com
soc-econom-problems.ru tamada69.com
social-search.me taranerymagesswa.blogspot.com
soft1.ru teguh.info
softonicads.com timetorelax.biz
sosdepotdebilan.com tizanidineinfo.blogspot.com
souvenir.cc tizanidinemusclerelaxant.blogspot.com
star61.de tizanidineoral.blogspot.com
stauga.altervista.org tizanidineshowupondrugtest.blogspot.com
stretchingabuckblog.com tjkckpytpnje.com
surflinksmedical.com topflownews.com
tedy.su
textads.men
thepantonpractice.co.uk
thewebsitetemplate.info
tiens2010.ru
titan-cloud.life
tizanidine4mgstreetprice.blogspot.com
tizanidineandgabapentin.blogspot.com
tizanidinestreetvalue.blogspot.com
tomatis.gospartner.com
toonfamilies.net
topbestgames.com
torrents.cd torrents.cd
toy-shop.top totu.info
txxx.com toxicwap.com
udayavani.com trkur.com
universals.com.ua trustl.life
viberdownload10.com tula.howotorg.ru
vielporno.net twojebook.pl
vseigry.fun uamtrk.com
vsesubwaysurfers.com uhod-za-sobakoj.ru
uk-zheu20.ru
undergroundcityphoto.com
uprour.com
upupa.net
url2image.com
urldelivery.com
vduplo.ru
villacoloniale.com
vinsit.ru
vseigru.one
vykup-avto-krasnodar.ru vykup-avto-krasnodar.ru
vzube.com wapsite.me
wallpapersdesk.info watchmygf.net
wardreapptokone.tk westermarkanjou.se
warningwar.ru wholesalecheapjerseysfree.com
webix.me wildcattube.com
webjam.com winner7777.net
wedding0venues.tk www.arenda-yeisk.ru
wetgames.ru www.labves.ru
wholesalejerseychinashop.com x-diesel.org
wineration.com xedserver.com
wovis.site yopoint.in
www.get-free-traffic-now.com za-fun-offer.com
wygraj-teraz.com znakomstva-moskva77.ru
xep.info znakomstvaonlain.ru
xev.ru zzbroya.com.ua
xn----7sbaphztdjeboffeiof6c.xn--p1ai
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn----9sbebi2bvzr7h.xn--p1ai
xn--80ajbshivpvn2i.xn--p1ai
xn--90acenikpebbdd4f6d.xn--p1ai
xn--b1ag5cfn.xn--p1ai
xuki.us
yaaknaa.info
youjizz.vc
zhongwenlink.com

139
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,6 +8,72 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -428,69 +494,6 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -581,14 +584,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.250 0; 10.1.0.156 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2864 ### Version: V4.2022.01.2865
### Updated: Wed Jan 19 22:01:22 UTC 2022 ### Updated: Thu Jan 20 10:03:30 UTC 2022
### Bad Bot Count: 633 ### Bad Bot Count: 633
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2864 ### Version: V4.2022.01.2865
### Updated: Wed Jan 19 22:01:22 UTC 2022 ### Updated: Thu Jan 20 10:03:30 UTC 2022
### Bad Bot Count: 633 ### Bad Bot Count: 633
################################################### ###################################################
### Version Information ## ### Version Information ##