vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.04.3031 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-04-08 10:09:30 +00:00
parent 94b45eb965
commit faea82cf1d
47 changed files with 83859 additions and 83856 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3030 #### Version: V4.2022.04.3031
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3030 #### Version: V4.2022.04.3031
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.04.3030 #### Version: V4.2022.04.3031
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10348
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.04.3030 #### Version: V4.2022.04.3031
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10352
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10352
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10352
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10352
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 1; 10.1.0.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 0; 10.1.0.68 0;
127.0.0.1 0; 127.0.0.1 0;

10352
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 1; 10.1.0.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 0; 10.1.0.68 0;
127.0.0.1 0; 127.0.0.1 0;

10352
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 1; 10.1.0.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 0; 10.1.0.68 0;
127.0.0.1 0; 127.0.0.1 0;

10352
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3030 3031

10352
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10352
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.230.233.150 0; 13.83.234.98 0;

10352
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 1; 10.1.0.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 0; 10.1.0.68 0;
127.0.0.1 0; 127.0.0.1 0;

10352
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 1; 10.1.0.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 0; 10.1.0.68 0;
127.0.0.1 0; 127.0.0.1 0;

10352
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 1; 10.1.0.68 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 0; 10.1.0.68 0;
127.0.0.1 0; 127.0.0.1 0;

10352
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.230.233.150 0; 13.83.234.98 0;

10352
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.230.233.150 0; 13.83.234.98 0;

10352
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 1; 10.1.0.68 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Black Hole Bitacle
CherryPicker Craftbot
Discoverybot Crescent
ECCP/1.0 Haansoft
Pi-Monster LinksManager
RankFlex MIIxpc
Twice Pavuk
VeriCiteCrawler Piepmatz
WebCollage SemrushBot
WebImageCollector sogouspider

282
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker
404enemy 404enemy
ASPSeek ASPSeek
Abonti
Acunetix Acunetix
AhrefsBot
AiHitBot AiHitBot
Alexibot
AlphaBot AlphaBot
Anarchie
Anarchy99 Anarchy99
Ankit
Aspiegel Aspiegel
Atomseobot Atomseobot
BBBike AwarioRssBot
BDFetch BDCbot
BackDoorBot
BackStreet
BackWeb BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler
Badass Badass
Bandit
BatchFTP BatchFTP
Bigfoot
Black Hole Black Hole
Blow Blow
BlowFish
Boardreader Boardreader
Bolt Brandprotect
BotALot
Brandwatch
Buck Buck
Buddy Buddy
Bullseye CODE87
BuzzSumo CensysInspect
CazoodleBot CheeseBot
Cegbfeieh CherryPicker
CheTeam Chlooe
Claritybot Claritybot
Cliqzbot Cliqzbot
Cocolyzebot Cocolyzebot
Collector
Copier
Cosmos
Crawling at Home Project Crawling at Home Project
Curious Curious
DBLBot DBLBot
DSearch Demon
Deusu
DigitalPebble DigitalPebble
Dirbuster
Disco
Discobot Discobot
Discoverybot Dispatch
DittoSpyder DittoSpyder
DomainAppender DnBCrawler-Analytics
Download Wonder DnyzBot
DomainSigmaCrawler
DomainStatsBot
Dragonfly Dragonfly
EMail Siphon Drip
EroCrawler Express WebPictures
ExtLinksBot Extreme Picture Finder
ExtractorPro Ezooms
EyeNetIE Fimap
FHscan FlashGet
FemtosearchBot Flunky
Freeuploader
FrontPage FrontPage
Genieo GalaxyBot
GetRight GetWeb
Getintent
Gigabot Gigabot
Go-Ahead-Got-It Go!Zilla
GoZilla GoZilla
GrabNet
Grabber Grabber
Grafula Grafula
GrapeFX HMView
GridBot
HTTP::Lite
Haansoft
HaosouSpider HaosouSpider
Harvest Havij
Heritrix Hloader
IDBot Humanlinks
IRLbot IRLbot
Id-search
Image Fetch
Image Sucker Image Sucker
IndeedBot IndeedBot
Indy Library
InfoNaviRobot InfoNaviRobot
Intelliseek InfoTekies
InterGET InterGET
Internet Ninja
InternetSeer InternetSeer
IstellaBot JOC Web Spider
JetCar Jbrofuzz
Jetty Jetty
Jorgee
JustView
Kenjin Spider
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Keyword Density
Kinza Kinza
Kozmosbot
LWP::Simple LWP::Simple
Larbin Larbin
Leap Leap
Lftp LeechFTP
Likse LexiBot
LinkWalker LinkWalker
Linkbot LinksManager
LinqiaScrapeBot
Lipperhey Spider
Litemage_walker
Ltx71 Ltx71
MIDown tool
MIIxpc MIIxpc
MQQBrowser MSIECrawler
MTRobot MTRobot
Mail.RU_Bot Majestic-SEO
Majestic SEO Majestic12
MarkMonitor
MarkWatch MarkWatch
Masscan
Mata Hari
Mb2345Browser Mb2345Browser
Meanpathbot Microsoft Data Access
MegaIndex.ru Microsoft URL Control
Minefield Minefield
Morfeus Fucking Scanner Mojolicious
Name Intelligence Mozlila
Musobot
NICErsPRO
Nameprotect Nameprotect
Navroad NearSite
Net Vampire Net Vampire
NetAnts NetMechanic
Nettrack NetSpider
NetZIP
Netcraft
Nibbler Nibbler
Not Niki-bot
Nuclei Nimbostratus
Offline Explorer Nmap
OpenLinkProfiler Octopus
OnCrawl
Openfind Openfind
Openvas
OrangeBot OrangeBot
OutclicksBot OutclicksBot
OutfoxBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP Page Analyzer
PageGrabber
PageScorer PageScorer
PageThing.com PageThing.com
Pandalytics
Papa Foto Papa Foto
Pavuk
Petalbot
Pi-Monster Pi-Monster
Picsearch
PictureFinder PictureFinder
PleaseCrawl Piepmatz
Pockey
ProPowerBot
ProWebWalker ProWebWalker
Probethenet Probethenet
Pump Psbot
PxBroker PxBroker
QueryN Metasearch
RankActive
RankFlex RankFlex
RankingBot RankingBot2
Rankivabot
Re-re
ReGet ReGet
Recorder RebelMouse
RedesScrapy Ripper
RepoMonkey SEOkicks-Robot
Rogerbot SEOlyticsCrawler
SEOkicks
SEOstats SEOstats
SMTBot
SalesIntelligent SalesIntelligent
Scanbot
ScoutJet ScoutJet
Screaming Scrapy
ScreenerBot ScreenerBot
SemanticJuice ScrepyBot
SearchmetricsBot
Seekport
Semrush Semrush
SeobilityBot SentiBot
SiteCheckerBotCrawler SeoSiteCheckup
SiteExplorer Shodan
SiteSnagger SiteSucker
SlySearch Sitebeam
Sociscraper Sociscraper
Sogou web spider Spammen
SpankBot SpankBot
Spanner
Sqworm
SuperBot SuperBot
SuperHTTP
SurveyBot
Swiftbot
Szukacz Szukacz
T0PHackTeam Teleport
T8Abot
TeleportPro
Telesphoreo
Thumbor Thumbor
Toata Titan
Toweyabot True_Robot
Tracemyfile URLy Warning
URLy.Warning UnisterBot
Upflow V-BOT
VB Project VB Project
Vacuum VCI
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper VidibleScraper
Virusdie Virusdie
VoidEYE
Voltron Voltron
WASALive-Bot WASALive-Bot
WISENutbot WBSearchBot
WPScan WWW-Collector-E
WWWOFFLE Wallpapers/3.0
WallpapersHD Web Auto
Web Collage Web Collage
Web Fetch Web Fetch
Web Pix Web Fuck
Web Sauger WebCopier
WebBandit WebFuck
WebCollage WebGo IS
WebImageCollector WebImageCollector
WebPix WebPix
WebSauger WebZIP
WebStripper Webalta
WebWhacker
Website Quester
WebsiteQuester
Webster
Whack Whack
Widow
Wonderbot
Wotbox Wotbox
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Zauba
Zermelo Zermelo
Zeus
ZmEu ZmEu
ZoomBot ZoomBot
ZoominfoBot
ZumBot ZumBot
ZyBorg
adscanner
archive.org_bot
arquivo.pt arquivo.pt
autoemailspider backlink-check
clark-crawler
coccocbot coccocbot
crawl.sogou.com com.plumanalytics
dataforseo.com
domainsproject.org
eCatch
facebookscraper
heritrix heritrix
internetVista monitor linkdexbot
iubenda-radar lwp-trivial
lwp-request magpie-crawler
mediawords netEstate NE Crawler
oBot
page scorer page scorer
s1z.ru plumanalytics
satoristudio.net polaris version
scan.lol ripz
scalaj-http
seobility seobility
seoscanners seoscanners
serpstatbot serpstatbot
sexsearcher sitechecker.pro
siteripz
sogouspider sogouspider
sp_auditbot
spyfu spyfu
sysscan sysscan
trendiction.de ubermetrics-technologies.com
webpros.com voyagerx.com
xpymep1.exe webgains-bot
x22Mozilla
zgrab zgrab

302
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
80legs 360Spider
AIBOT 404enemy
ASPSeek
Abonti Abonti
Aboundex Aboundex
Aboundexbot Aboundexbot
AfD-Verbotsverfahren
AiHitBot
Alexibot Alexibot
Alligator Alligator
AlphaBot
Anarchie
Anarchy Anarchy
Aspiegel Ankit
Asterias
AwarioSmartBot AwarioSmartBot
BDFetch BDCbot
BackDoorBot
BackWeb BackWeb
Backlink-Ceck BatchFTP
BacklinkCrawler Bitacle
Bandit BlowFish
Battleztar Bazinga
Black Hole
BlackWidow
Blow
Boardreader Boardreader
Bolt Bolt
Brandprotect
Brandwatch Brandwatch
Buddy CCBot
BuiltWith CSHttp
CazoodleBot Cegbfeieh
CherryPicker CheeseBot
CopyRightCheck Citoid
Claritybot
Cogentbot
Copier
Copyscape Copyscape
Custo CrunchBot
CyotekWebCopy CyotekWebCopy
DSearch DataCha0s
DTS Agent Devil
Demon
DigitalPebble
Dirbuster
Disco
Discobot
Dispatch
DnBCrawler-Analytics DnBCrawler-Analytics
DomainCrawler DnyzBot
DomCopBot
DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
Dotbot Dragonfly
Drip Drip
ECCP/1.0 EMail Wolf
EMail Siphon Ebingbong
Ecxi
EirGrabber EirGrabber
EroCrawler Exabot
Express WebPictures Extractor
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms FDM
FHscan
Flunky
Foobot Foobot
FyberSpider Freeuploader
G-i-g-a-b-o-t G-i-g-a-b-o-t
GalaxyBot GT::WWW
GermCrawler GermCrawler
GetRight GetRight
GetWeb
Gigabot
Go!Zilla Go!Zilla
GoZilla GoZilla
GrabNet
Grabber Grabber
Grafula GrapeshotCrawler
GridBot GridBot
Haansoft HEADMasterSEO
HaosouSpider HMView
HTTrack
Havij Havij
Hloader Heritrix
HonoluluBot HonoluluBot
IDBTE4M IDBTE4M
IRLbot IlseBot
Iblog IndeedBot
Image Fetch
InfoNaviRobot
InfoTekies InfoTekies
Internet Ninja
InternetSeer InternetSeer
Iria
Jbrofuzz Jbrofuzz
JennyBot JennyBot
JikeSpider Keyword Density
Keybot Translation-Search-Machine
Kinza
Kozmosbot Kozmosbot
LWP::Simple LNSpiderguy
Larbin Lanshanbot
Leap LeechFTP
Lftp LinkScan
Libwhisker LinkWalker
Likse Linkbot
LinkextractorPro LinkextractorPro
LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey Lipperhey
Lipperhey Spider Litemage_walker
MFC_Tear_Sample Lmspider
MIDown tool MIDown tool
Mail.RU_Bot MIIxpc
Mass Downloader MSFrontPage
Mata Hari Majestic12
MauiBot
Mb2345Browser Mb2345Browser
MeanPath Bot MegaIndex.ru
Meanpathbot
Mediatoolkitbot
Metauri
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Mister PiX
Mojolicious Mojolicious
MolokaiBot MolokaiBot
Mozlila
Msrabot
Musobot Musobot
NICErsPRO NearSite
Nessus
Net Vampire Net Vampire
NetAnts NetLyzer
NetMechanic
NetZIP
Nettrack
Netvibes Netvibes
Niki-bot NextGenSearchBot
Nikto Nikto
NimbleCrawler NimbleCrawler
Not Not
Nuclei Nutch
Octopus
Offline Explorer Offline Explorer
Offline Navigator Offline Navigator
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS OpenVAS
Openvas
OrangeBot OrangeBot
OutfoxBot PHPCrawl
PECL::HTTP PageAnalyzer
Page Analyzer
PageScorer PageScorer
PageThing.com PageThing.com
Pandalytics Pandalytics
PeoplePal Petalbot
Pi-Monster Pimonster
PictureFinder Pixray
Piepmatz PleaseCrawl
ProPowerBot Pockey
Pump Pu_iN
PxBroker RankActiveLinkBot
PyCurl RankFlex
Quick-Crawler RankingBot
RankingBot2
Rankivabot Rankivabot
RankurBot RankurBot
RealDownload Re-re
Reaper
RebelMouse RebelMouse
Recorder Recorder
RedesScrapy RedesScrapy
RocketCrawler RocketCrawler
Rogerbot SBIder
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOstats SEOprofiler
SearchmetricsBot SISTRIX
SalesIntelligent
Scanbot
ScoutJet
Searchestate
SemanticJuice
Semrush Semrush
SemrushBot Seomoz
SeobilityBot Siphon
Site Sucker SiteCheckerBotCrawler
SiteLockSpider
SiteSucker SiteSucker
SlySearch Siteimprove
SmartDownload Sitevigil
Snake
Snapbot Snapbot
Snoopy Snoopy
SocialRankIOBot SocialRankIOBot
Sociscraper
Sosospider Sosospider
SpankBot Sottopop
SpaceBison
Spammen
Spbot Spbot
Sqworm Spinn3r
Sucker SputnikBot
Stripper
SuperBot SuperBot
Surfbot Surfbot
SurveyBot T0PHackTeam
Swiftbot Teleport
T8Abot TeleportPro
Telesphoreo Telesoft
Thumbor TightTwatBot
Toata Tracemyfile
Turingos Turnitin
Typhoeus URLy Warning
UnisterBot
V-BOT
Vacuum Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
Virusdie
VoidEYE
Voil Voil
WBSearchBot Voltron
WPScan WISENutbot
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0 Web Collage
WeSEE
Web Auto
Web Fetch
Web Fuck Web Fuck
Web Sucker
WebAuto WebAuto
WebBandit WebBandit
WebCopier
WebFuck
WebGo IS WebGo IS
WebImageCollector WebLeacher
WebSauger WebPix
WebReaper
WebStripper
WebWhacker WebWhacker
Website Quester
WebsiteExtractor WebsiteExtractor
Whack WebsiteQuester
Webster
Whatweb
Who.is Bot Who.is Bot
WinHTTrack Wotbox
Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Xenu
Zade
Zauba
Zeus Zeus
Zitebot ZmEu
ZoomBot
ZoominfoBot ZoominfoBot
ZumBot
ZyBorg ZyBorg
archive.org_bot
backlink-check
check1.exe check1.exe
clark-crawler coccocbot
crawl.sogou.com crawl.sogou.com
domainsproject.org crawler.feedback
eCatch crawler4j
gopher internetVista monitor
ips-agent ips-agent
lwp-request isitwp.com
iubenda-radar
linkdexbot
lwp-trivial
meanpathbot meanpathbot
muhstik-scan netEstate NE Crawler
s1z.ru pcBrowser
polaris version
probe-image-size
ripz
scalaj-http scalaj-http
seobility
seocompany.store
seoscanners seoscanners
seostar
sitechecker.pro sitechecker.pro
siteripz
sp_auditbot sp_auditbot
spyfu sysscan
trendiction.de tAkeOut
ubermetrics-technologies.com
voyagerx.com voyagerx.com
x09Mozilla webpros.com
webprosbot
x22Mozilla
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
4senses.co compiko.info
ezigarettekaufen1.hpage.com dstroy.su
hiwibyh.bugs3.com konkursov.net
localmatchbook.com lenvred.org
manimpotence.com magazin-pics.ru
rainbowice.ru ooo-olni.ru
riralmolamsaca.tk qualitymarketzone.com
sex-tracker.de similardeals.net
spyfu.com whatsappbot.flyland.ru
testingads.pro ww1943.ru

914
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
000free.us 1001watch.com.ua
123cha.com 20pascals.nl
1millionusd.xyz 23kw.ru
1xbet4you.com 256bit.by
214jaluwobapef.cf 4istoshop.com
3rbseyes.com 4webmasters.org
3weekdiet.com 8gold.com
4k-player.pl 8si.ru
5kstore.com 98oi.ru
a.frcls.fr
aa08daf7e13b6345e09e92f771507fa5f4.com aa08daf7e13b6345e09e92f771507fa5f4.com
aa625d84f1587749c1ab011d6f269f7d64.com aa8b68101d388c446389283820863176e7.com
abcdeg.xyz actionnooz.com
absolute-s.ru adhome.biz
accessoires-mode.in adsref.men
activecampaign.dreamhosters.com adtiger.tk
actulite.com adultfriendfinder.com
adrunnr.com alarmobninsk.ru
adtech.us alert-fdm.xyz
advokateg.ru alert.scansafe.net
aktivator-windows10.blogspot.com allboard.xobor.de
alekseevec.ru allnews24.in
alfapro.ru
all-number.com
allbizne.co.ua
alphacarolinas.com
alphavisions.net alphavisions.net
aludecor.info altamayoztourism.com
alveris.ru alveris.ru
am-se.com amt-k.ru
amazingninja.com an-donut.com
ample-awards-today.us ananumous.ru
andpolice.com anatomiy.com
android-style.com angigreene.com
android4fun.org anonymous-redirect.com
antons-transporte.de aosexkontakte.net
apartamentwroclaw.eu
apartmentbay.ru apartmentbay.ru
apiitsthirteende-a.akamaihd.net apibatbrowsecom-a.akamaihd.net
asrv-a.akamaihd.net appiq.mobi
asscenihotosidea.blogspot.com apple.com-webbrowsing-security.review
autodo.info arclk.net
autoplate.info ardimobileinfo.ml
arquapetrarca.info
artel116.ru
asiavirtualsolutions.com
asociatia-tipografilor-transilvania.ro
asrvvv-a.akamaihd.net
automatic-seo.com
autowebmarket.com.ua
aversis.net aversis.net
avirasecureserver.com avtointeres.ru
azartclub.org avtolombard-krasnodar.ru
bangla-chat-uk-paralud.ga awency.com
bank.uz azartmix.com
bazaronline24.ru azbaseimages.net
beachdriveblog.com baikaleminer.com
beget.tech baladur.ru
best-coupon-offer.com balitouroffice.com
best-seo-offer.com balois.worldbreak.com
beta.hotkeys.com bankiem.pl
basisches-wasser.net
belinka.com.ua
bezsporno.ru
bg6s0.com
bidr.trellian.com
bigcareer.info
bigpenisguide.com
bigpictures.club
bin-brokers.com bin-brokers.com
bitnote.co bingo8888.com
bloggedporn.com bingoporno.com
bluesman.nu bioscorp.ru
bmwhighperformers.com biteg.xyz
boltalko.xyz bizcheapjerseyswholesalechina.com
blackhatworth.com
blackwitchcraft.ru
blavia.00author.com
blockworld.ru
bocoarchives.org
bonanza-fortune.men bonanza-fortune.men
braindaily.xyz bonkers.name
brothers-smaller.ru bosman.pluto.ro
bryansk.zrus.org brains2.biz
bumble.cheapwebsitehoster.com brandbucket.com
buqyxa.rincian.info breastaugmentation.co.za
cablecar.us briomotor.co
calvet.altervista.org bristolhotel.com.ua
cardsharp1.ru browsepulse-a.akamaihd.net
cartierlove2u.xyz buketeg.xyz
castingbank.ru business-suggested.tk
caveavins.fr buyessayonline19.blogspot.ru
cbb1smartlist12.click buyviagraa.blogspot.com
cementaresearch.se buzzzg.men
chastnoeporno.com byme.se
chelnytruck.ru cakemediahq.com.au
callawaygolfoutlet.online
callawaygolfstore.online
canacopegdl.com
candcstuccoandstone.com
carsdined.org
cartierlovestore.xyz
casablancamanor.co.za
casite-513049.cloudaccess.net
cercacamion.it
chatroulette.video
chcu.net
cheapestjerseys-wholesale.com
check-this-out-now.online
checkpagerank.net
cherubinimobili.it cherubinimobili.it
christianlouboutinreplica.win chinese-amezon.com
christianlouboutinsaleonline.us cialisndbrx.com
cialisovercounteratwalmartusa.com
cialiswithoutadoctor.net
citysecurity.nu
cleanallspyware.com
clickcash.com clickcash.com
co.lumb.co cntravelre.com
cocyq.inwtrade.com cocyq.inwtrade.com
com-onlinesupport.host coin-hive.com
compliance-alex.xyz coldfilm.ru
compliance-donald.xyz collegeessay19.blogspot.ru
compliance-ivan.top cool-wedding.net
compliance-jack.top cordstrap.cc
compliance-julianna.top counter.bloke.com
conboy.us cpabegins.ru
continent-e.tv
coolgamechannel.com
coolgramgoods.com
coolwallpapers-hd.com
copyrightclaims.org
craftinsta.ru
crazyboost.pro crazyboost.pro
cyclobenzaprinevstizanidine.blogspot.com crd.clan.su
datract.com crest3d.ru
dawlenie.com culturevie.info
custom-electric-guitar.com
customcollegeessays.net
cv.wallhade.co
dailyfinancefix.com
data.vtc.pw
dataloading.net
day-news.info day-news.info
deadlinkchecker.com decoratinghomes.ga
dealighted.com deda-moroza-zakaz.ru
decorazilla.com detki-opt.ru
deshevo-nedorogo.ru dfiles.me
design-ideas.info dieta-personalna.pl
deutschland123.de digitalfaq.com
diesel-parts28.ru distver.ru
dieswaene.com
dikqlyremy.info
dispo.de
dnepropetrovsk.mistr-x.org dnepropetrovsk.mistr-x.org
doktoronline.no
dokumentalkino.net
domain.webkeyit.com domain.webkeyit.com
draniki.org domashneeruporno.com
downloadeer.net
drivotracker.com
drugs-no-rx.info drugs-no-rx.info
e-c.al
e-poker-2005.com e-poker-2005.com
e705.net earthmagic.info
e90post.com eas-seo.com
ebonyporn.site eets.net
ecookna.com.ua ekobata.ru
ekspertmed.com
elektrischezigarette2.onsugar.com elektrischezigarette2.onsugar.com
elektrischezigaretten1.blogse.nl elvenmachine.com
elektronischezi.livejournal.com envaseslotusama.com
en.altezza.travel escort-russian.com
energydiet24.ru evaashop.ru
eng-lyrics.com evidencecleanergold.com
eralph.tk excitacion.info
esfchat.tk exonline.info
esopini.com ezaz.info
etotupo.ru ezigarettekaufen2.blox.pl
etur.ru f-online.de
euronis-free.com family1st.ca
euroskat.ru fandlr.com
exdocsfiles.com fapgon.com
executehosting.com fatmaelgarny.com
fable.in.ua favorcosmetics.com
face.hostingx.eu fearcrow.com
fastfixing.tech findacheaplawyers.com
ff30236ddef1465f88547e760973d70a.com fiverr.com
filefilter.weebly.com floridamobilebillboards.com
film-one.ru forodvd.com
filmfanatic.com fototravel.eu
filmi-onlain.info free-stock-illustration.com
financehint.eu
finemanteam.com
finstroy.net
flashbannernow.com
flipper.top
florida-tourism.net
forumprofi.de
fr-bearings.ru
free-laptop-reward.com
freeuploader.com freeuploader.com
fres-news.com freeuploader.ml
freshberry.com.ua friendflnder.com
fxund.us fuckbuddybestgilf.info
gallerily.com fungamelands.com
gallictures.com
game-mmorpg.net
gardene.ru
gaygalls.net
gaypornmovie.net gaypornmovie.net
gaytube.com gazobeton-p.com.ua
gayxperience.com gdzkurokam.ru
gaz-voshod.ru ge0ip.com
generalporn.org getmiro.com
getlaid-xxxhookupdirect.com
getpopunder.com getpopunder.com
getresponse.com glcomputers.ru
girlgamerdaily.com
glass-msk.ru
glicol.kz glicol.kz
goldadpremium.com globetrotting-culture.ru
gombita.info gobongo.info
googlemare.com golaya.pw
gopixdatabase.com greatdealshop.com
gosarhivrt.ru greatgrace.ru
gowreckdiving.com greatidea.marketing
gpirate.com haikuware.com
green-tea.tv handicapbathtubarea.com
groupmoney.ru hd720kino.ru
gsasearchengineranker.space hermesbracelets.xyz
hentaiheroes.com histats.com
heygidday.biz histock.info
hotsocialz.com hiwibyh.bugs3.com
hotxnights.info hkdiiohi.skyrock.com
hoverboard360.es hkladys.com
husky-shop.cz homedecoguide.info
hustoon.over-blog.com hopeonthestreet.co.uk
hydropump.su hoste.octopis.com
i-hobot.ru hotelcrocenzi.sm
iccornacircri.cf hothot.ru
ico.re housediz.com
ihc2015.info hulfingtonpost.com
images.gyffu.com hunthillfarmtrust.org
imitex-plus.ru hvd-store.com
increasewwwtraffic.info hystersister.com
innodgfdriverhm.aircus.com ifmo.ru
innoslicon.com imagerydatabase.com
insider.pro inet-traffic.com
inspiring-desperate.tk investzalog.ru
interior-stickers.ru iomoio.net
intimshop-fantasy.ru
investsuccess.org
ip-guide.com ip-guide.com
iqoption.pro isitwp.com
iqs.biz.ua istizanidineanopiate.blogspot.com
ispac.org istmira.ru
ivanovo.zrus.org jav-idol.com
iwantmyfreecash.com journeydownthescale.info
iweblist.info kabbalah-red-bracelets.com
jagg.info kaidalibor.de
jav-fetish.com kalandranis.gr
javcoast.com kanimage.com
jerseyswholesalechinalimited.com kashubadesign.ru
jimmychoosale.online keywordblocks.com
joessmogtestonly.com keywordteam.net
johnnyhaley.top kinohit1.ru
johnrobertsoninc.com kmgamex.cf
jwss.cc kovesszucs.atw.hu
keywordhut.com krasnodar.ru
kino-rating.ru krasnodar.xrus.org
kinogolos.ru
kleine-titten.biz
klejonka.info
kndxbkdx.bloger.index.hr
kometars.xyz
komputers-best.ru
krasnodar-avtolombards.ru
krassh.ru
kredit-blog.ru kredit-blog.ru
kretpicf.pw kriokomora.info
kunstaktien.info ks1234.com
kurbappeal.info
kurwa.win
la-fa.ru la-fa.ru
ladsblue.com labafydjxa.su
landinez.co lanadelreyfans.us
lapitec.eu lanasshop.ru
larger.io landscapebackgrounds.blogspot.com
lashstudia.ru laxdrills.com
law-check-nine.xyz lazymanyoga.com
leftofcentrist.com leadwayau.com
lesbian.xyz ledpolice.ru
letsrepair.in lerporn.info
letto.by letmacworkfaster.site
lexiangwan.com letslowbefast.site
likrot.com lexaprogeneric.link
littlecity.ch libertybilisim.com
livetsomudvekslingstudent.bloggersdelight.dk liffa.ru
lkbennettstore.online lineavabit.it
link.ac
lion.snapmap.us
liquimondo.com
liupis.com
livingcanarias.com
lizace.com
loginduepunti.it loginduepunti.it
lowephotos.info lombardfinder.ru
luxmagazine.cf lombia.co
louisvuittonoutletstore.net
lovasszovetseg.hu
luciddiagnostics.in
lyngdalhudterapi.no lyngdalhudterapi.no
m3gadownload.pl lyrster.com
magicdiet.gq mafcards.ru
magnetpress.sk magnetpress.sk
makenahartlin.com marcogrup.com
manualterap.roleforum.ru marwer.info
marketland.ml mastershef.club
matb3aa.com
matematikus.info matematikus.info
matsdale.com mbiologi.ru
mazda-roadsters.com medicines-choice.com
mebelcomplekt.ru medosmotr-ufa.ru
mediafresh.online megakino.net
medkritika.ru mejoresfotos.eu
medmajor.ru menstennisforums.com
meetingrainstorm.bid metabar.ru
menetie.ru micasainvest.com
menhealed.net milblueprint.com
metalonly.info
microstatic.pl
militarysale.pro
millionare.com millionare.com
mindbox.co.za mindbox.co.za
minecraft-rus.org miniature.io
mobifunapp.weebly.com mir-betting.ru
moivestiy.biz mlvc4zzw.space
monclerboots.xyz mmstat.com
moncleroutletonline.pw mobile-appster.ru
morocco-nomad-excursions.com mobplayer.net
moscow-clining.ru monarchfind-a.akamaihd.net
mosdverka.ru monarhs.info
mouselink.co monclercoats.xyz
moyakuhnia.ru moyaterapiya.ru
mrbojikobi4.biz mug-na-chas-moscow.ru
music7s.me musezone.su
musicpro.monster musicspire.online
mybinaryoptionsrobot.com myhealthcare.com
myfreetutorials.com myprintscreen.com
mygameplus.ru mystats.xyz
myperiod.club na-telefon.biz
naperehresti.info nac-bearings.ru
narkologiya-penza.ru neks.info
narkologiya-voronezh.ru newstraveller.ru
national-today-winning-winner.club nextlnk12.com
naturalbreakthroughsresearch.com nextrent-crimea.ru
netvouz.com nhl17coins.exblog.jp
new-post.tk niki-mlt.ru
news-speaker.com novgorod.xrus.org
next-dentists.tk nuit-artisanale.com
nina.az
nonameread45.live
norththeface.store
notaria-desalas.com
novatech.vn
nudo.ca
ofanda.com
offside2.5v.pl
okmedia.sk okmedia.sk
online-hit.info oktube.ru
online-podarki.com
online.ktc45.ru online.ktc45.ru
onlineslotmaschine.com online247.ml
orenburg-gsm.ru onlythegames.com
orgasmus-virtual.com oops-cinema.ru
ostroike.org ops.picscout.com
otbelivanie-zubov.com orhonit.com
outpersonals.com
outshop.ru
owohho.com
oxotl.com
ozas.net ozas.net
ozoz.it
pandroid.co
paramountmarble.co.uk paramountmarble.co.uk
patterntrader-en.com pdn-4.com
pepperstyle.ru phrcialiled.com
phelissota.xyz pinsdaddy.com
phormchina.com pizza-tycoon.com
photosaga.info popunder.net
piski.top popup-jdh.xyz
pl.id-forex.com pornhive.org
platezhka.net porno-raskazy.ru
platinumdeals.gr pornobrazzers.biz
poem-paying.gq pornoklad.ru
polska-poezja.com pornosok.ru
popads.net potoideas.us
popserve.adscpm.net
porndroids.com
pornoted.com
powenlite24.ru
prizesbook.online
progonrumarket.ru progonrumarket.ru
prointer.net.ua prointer.net.ua
promoheads.com prostitutki-yaroslavlya.org
promover.org pyrodesigns.com.au
purchasepillsnorx.com quick-seeker.com
purplesphere.in quickchange.cc
puteshestvennik.com quickloanbank.com
pwwysydh.com rada.ru
py100.ru rapidokbrain.com
quebec-bin.com rcpmda.ikan1080.xyz
quit-smoking.ga remontbiz.ru
qwertty.net renhacklids.tk
raavidesigns.com
razleton.com
registratciya-v-moskve.ru
replicalouboutin.xyz replicalouboutin.xyz
royal-betting.net rosbalt.com.ua
rp9.ru
rubanners.com rubanners.com
running-line.ru ruex.org.ua
runstocks.com saddiechoua.com
s-forum.biz salutmontreal.com
sack.net saugatuck.com
saecsa.co scanner-julia.top
salmonfishingsacramentoriver.com scopich.com
sammyweaver.com scrnet.biz.ua
sashagreyblog.ga secret.xn--oogle-wmc.com
saveriopiazza.it seksvideoonlain.com
sbornik-zakonov.ru semaltmedia.com
sbprabooks.com seminarykansas904.ml
sbricur.com seobility.net
scanner-marwin.top seofied.com
scanner-willy.top servicecenter.co.ua
scansafe.net sex-foto.pw
scenarii-1-sentyabrya.uroki.org.ua sex-sex-sex5.com
schlampen-treffen.com sexflirtbook.com
score-ads.men
screentoolkit.com
searchaddis.com
searchmywindow-a.akamaihd.net
searchwebknow-a.akamaihd.net
selectads.men
sendearnings.com
seocheki.net
seomarketings.online
serdcenebolit.com
sergiorossistore.online
sexad.net
sexs-foto.com
sexy.babes.frontend-stack.top sexy.babes.frontend-stack.top
shiksabd.com shariki-zuma-lines.ru
signx.info ship-marvel.co.ua
sklad-24.ru shitmovs.com
slomm.ru shivafurnishings.com
smailik.org shopfishing.com.ua
smartshoppymac.com shopperifymac.com
softomix.com shtora66.ru
sibecoprom.ru
silktide.com
site-speed-check.site
siteonomy.com
sitopreferito.it
sluganarodu.ru
smsactivator.ru
socialsignals24.com
soda.media
space4update.pw
spammen.de
special-porn.com
sport7777.net
srecorder.com
sribno.net
stairliftsarea.com
stanthonyscatholicchurch.org
stard.shop stard.shop
starpages.net stefanbakosab.se
streetfire.net stoki.ru
streetfooduncovered.com strag-invest.ru
stroimajor.ru strfls.com
stroyhelp-dv.ru strigkaomsk.ru
stroymonolit.su stroiminsk.com
stromerrealty.com
students-cheapskate.ml
studiofaca.com
studiokamyk.com.pl
sugarkun.com sugarkun.com
superstarfloraluk.com surfbuyermac.com
support.nopeas.sk svbur.ru
susanholtphotography.com swinger-mobil.net
svtrd.com swiped.su
t3chtonic.com sygraem.com
taaaak.com tako3.com
taqywu51.soup.io tecnoteakviareggio.it
technopellet.gr theallgirlarcade.com
tehngr.ru theautoprofit.ml
templates.radiodigital.co thecoolimages.net
testingads.pro thefotosgratis.eu
thefarmergame.com thegioixekhach.com
thejournal.ru thegolfclub.info
thetattoohut.com theplacetoupdating.pw
thfox.com threecolumnblogger.com
timdreby.com tiandeural.ru
time-japan.ru timeallnews.ru
titan-cloud.life tizanidine4mgstreetprice.blogspot.com
tizanidinedruginteractions.blogspot.com tizanidineandcipro.blogspot.com
tizanidineduringpregnancy.blogspot.com tizanidinedosageforsleep.blogspot.com
tizanidinehcl2mg.blogspot.com tizanidinehcl4mgtabinfo.blogspot.com
tizanidinehydrochloride2mg.blogspot.com
tmtrck.com
token-lab.org
toonfamilies.net toonfamilies.net
topflownews.com top1-seo-service.com
torrentgamer.net topcar-krasnodar.ru
totu.us topdownloads.ru
tourismvictoria.com torrentdownloadhub.com
toy-shop.top torture.ml
traffic-club.info totu.info
trafficgenius.xyz toursmaps.com
trafficshaper.com toyota.7zap.com
trahic.ru tracklead.net
tubo360.com trion.od.ua
twittruth.com truebeauty.cc
ucoz.ru tsan.net
uhod-za-sobakoj.ru tsyndicate.com
ukrup.com ttsq.fr
umnovocaminho.com turizmus.us
unacittaconte.org ua-company.ru
unrealcommander.com uac.net.au
upstore.me uchil.net
url-img.link udsgame.online
urlopener.com uggbootsoutletsale.us
usadacha.net ukkala.xyz
uytmaster.ru ultramart.biz
viddyoze.com unimodemhalfduplefw.pen.io
videochat.tv.br vacuumcleanerguru.com
videtubs.pl vapomnoncri.tk
vitanail.ru velen.io
vkontaktemusic.ru venta-prom.ru
vodaodessa.com vertaform.com
voloo.ru viagra-soft.ru
voloomoney.com video-production.com
vpnmouse.com videochat.life
vremya.eu vilingstore.net
w-journal.ru villakohlanta.nu
w3data.co vizitki.net
w7s.ru volgograd.xrus.org
waterefficiency.co wardreapptokone.tk
wdfdocando.com warezkeeper.com
webs.com warningzscaler.heraeus.com
whereverdesperate.gq we-are-gamers.com
wholinkstome.com webenlace.com.ar
wikes.20fr.com weblo.com
wildnatureimages.com webnode.me
wineration.com westen-v.life
wetgames.ru
woman-orgasm.ru woman-orgasm.ru
wonderfulflowers.biz wordpress-crew.net
works.if.ua worldhistory.biz
worldmusicfests.com wormix-cheats.ru
wrona.it wowcasinoonline.ooo
wufak.com www.jbetting.com
www.bookmaker-bets.com www.wohnkabinen-shop.de
x5market.ru x-diesel.info
xchangetrak.com xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
xfluro.com xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xjlottery.com youbloodyripper.com
xlovecam.com yoursearch.me
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai zagadki.in.ua
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai zajm-zalog-krasnodar.ru
xn----7sbho2agebbhlivy.xn--p1ai zapatosenventa.info
xn----9sbebi2bvzr7h.xn--p1ai zarabotat-na-sajte.ru
xportvusbdriver8i.snack.ws zawyna.ua
xxx-cam.webcam zebradudka.com
yoursite.com zeroredirect5.com
yuarra.pluto.ro zeroredirect6.com
zacreditom.ru zreloeporno.tv
zaimite.ru zscalertwo.net
zarajbuilders.com zskdla.site
zazagames.org zvooq.eu
zed21.net
zigarettenonlinekaufen2.drupalgardens.com
zoosexart.com
zygophyceous.womanstars.site

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
101lesbian.xyz 2345.com
11pikachu.ru 24-ak.ru
2020iscoming.info 2ads.co.uk
2drittel.de 2rich4bitches.com
4webmasters.org 3w24.com
883zy.com 4replicawatch.net
adhome.biz 5-steps-to-start-business.com
adimmix.com 5kstore.com
adprotect.net 5tey7463.weebly.com
agroeconom.kz 8gold.com
ahhjf.com 8lufu.com
ahrefs.com 8xv8.com
aab94f698f36684c5a852a2ef272e031bb.com
accessoires-mode.in
adexprts.com
admitad.com
adult-shop.com.ua
advancedmassagebysara.com
akvamaster.dp.ua akvamaster.dp.ua
aliexpresscashback.club altamayoztourism.com
alif-ba-ta.com anniemation.com
anatomiy.com apiadanaknet-a.akamaihd.net
appsaurus.com apimegabrowsebiz-a.akamaihd.net
artel116.ru apioutoboxnet-a.akamaihd.net
ayeartoforget.com applicationg29.com
azbukafree.com arabsexxxtube.com
bablonow.ru arabseyes.com
baladur.ru asia-forum.ru
balkanfarma.ru asrvrep-a.akamaihd.net
bamo.xsl.pt aviav.ru.com
bannerpower.com avon-novosib.ru
battlecarnival.su bear.gotcher.us
beclean-nn.ru beclean-nn.ru
berdasovivan.ru best-way.men
bestbuy.ca betwinservice.com
bimmerpost.com bioscorp.ru
biocn.dx.am bird1.ru
bizzliving.com birzha-truda.eu
blog.koorg.ru bmw-ark.ru
brainxs.us boleznikogi.com
brandbucket.com braintobe.win
brendbutik.ru brateg.xyz
carsdined.org browserprotecter.com
cartierbracelet.xyz brus.city
cartujano-pre.de business-made-fun.com
cdncash.org buy-meds24.com
cheapmoncler.pw bvlgariwallet.xyz
check-host.net callstevens.com
children-learningreading.info candiceloves.us
cardsdumps.com
cartierlove2u.com
cartierlovestore.com
chatroulette.online
chatroulette.world
chocolateslim-en-france.com chocolateslim-en-france.com
community.allhiphop.com cialiswithoutadoctor.net
compliance-nelson.top ciekawinki.pl
cookie-law-enforcement-aa.xyz circlesl.com
cl.s7.exct.net
clash-clans.ru
clickaider.com
compliance-barak.top
compliance-don.top
construmac.com.mx
coolwallpapers4k.info coolwallpapers4k.info
corta.co copenergo.ru
cottageofgrace.com copro.pw
counter.yadro.ru cornerstone-countertops.com
courtshipgift.com
crackzplanet.com
cranly.net
credit.co.ua
crest3d.ru
cubook.supernew.org
customergrowthsystems.com customergrowthsystems.com
danashop.ru danashop.ru
deshevo-nedorogo.ru dayibiao.com
digital-sale.su ddospanel.com
dlya-android.org decorazilla.com
doctormakarova.ru dentfidemountpreach.tk
dominterior.org digitalassetmanagement.site
doreenblog.online dinosaurus.site
dotnetdotcom.org dmmspy.com
ekn-art.se docs4all.com
eropho.com documentbase.net
ez8motelseaworldsandiego.com dyshagi.ru
favornews.com edwinkonijn.com.au
fickblock18.com election.interferencer.ru
fikasound.tk elektir.ru
elektrozigarette1.dreamwidth.org
elektrozigarettekaufen1.devhub.com
elitepcgames.com
eng-lyrics.com
eropho.net
erotik-kostenlos.net
ertelecom.ru
etm-consult.de
eugenevaultstorage.com
european-torches.ru
ewebarticle.info
excitacion.info
falcoware.com
fba-mexico.com
filmci.pro
financehint.eu financehint.eu
fitfloponline.store flowwwers.com
fitnesspiks.com fotoxxxru.com
fm-upgrade.ru free-deals.faith
forex-procto.ru free-share-buttons.blogspot.com
forumprofi.de free-stock-illustration.com
franklinfire.co fuckbuddybestgilf.info
friendflnder.com gamedayhouse.com
game-mmorpg.net get-your-social-buttons.info
gaypornmovie.net girls-ufa.ru
gdzkurokam.ru goblacked.com
gerhardhealer.com gomusix.com
gg.zzyjxs.com good-mummy.ru
goatse.ru goodly.pro
growmyfunds.ca guitar-master.org
hd-film.pl gwebtools.com.br
heartofpayne.xyz hentai-manga.porn
herehloadibs.cf hkdiiohi.skyrock.com
hmmm.cz hornymatches.com
hoste.octopis.com ilikevitaly.com
hoverboardforsaledirect.com ilovevitaly.ru
humanorightswatch.org imadedinner.net
ikearugs.xyz images.gyffu.com
inclk.com imguramx.pw
interesnie-faktu.ru infektsii.com
iskalko.ru innodgfdriverhm.aircus.com
ispac.org ispaniya-costa-blanca.ru
ivearchenceinflu.cf jav-idol.com
iwantmyfreecash.com je7.us
jerseywholesalebizchina.com journeydownthescale.info
kabbalah-red-bracelets.com jurajskie.info
ketoanhanoi.info kadashihotel.com
koleso24.com.ua kaliningrad.zrus.org
kosova.de klikbonus.com
krassh.ru
kredit-pod-zalog-krasnodar.ru
ladsup.com ladsup.com
larchik.net landoftracking.com
lashstudia.ru laulini.soclog.se
lawyersinfo.org letolove.ru
lietaer.com lidoradio.com
lifehacĸer.com livetsomudvekslingstudent.bloggersdelight.dk
lifeinsurancekb.com louboutinshoes.xyz
locatellicorretor.com.br lsp-awak-perikanan.com
lsex.xyz
ltvperf.com ltvperf.com
mainlinehobby.net mackeeper-land-672695126.us-east-1.elb.amazonaws.com
matematikus.info macresource.co.uk
medosmotr-ufa.ru mall.uk
meduza-consult.ru manipulyator-peterburg.ru
mere.host.sk markjaybeefractal.com
michaelkorsoutletstore.net medi-fitt.hu
mncrftpcs.com metaxalonevstizanidine.blogspot.com
mobplayer.ru mobicover.com.ua
moneyviking-a.akamaihd.net mobsfun.net
montazhnic.ru moncleronline.xyz
mountainstream.ms morlat.altervista.org
musicvidz.ru moviemail-online.co.uk
mwtpludn.review msfsaar.de
myhitmp3.club myblogregistercm.tk
mylovelibrabry.com naperehresti.info
myra.top narco24.me
naphukete.ru national-today-winning-winner.club
naughtyconnect.com nfvsz.com
netcheckcdn.xyz niroo.info
notfastfood.ru njkmznnb.ru
novodigs.com noscrapleftbehind.co
oktube.ru obiavo.su
okuos.com odoratus.net
online-zaymy.ru oivcvx.website
ourtherapy.ru onlinedomains.ru
p-business.ru onlineporno.site
papasdelivery.ru ozoz.it
par-fallen.ga pateaswing.com
parajumpersstore.online pawli.eu
partner-high.men penisvergrotendepillennl.ovh
partnersafe.men pic-re.blogspot.com
pdn-4.com pinstake.com
pensplan4u.com pipki.r.acdnpro.com
perfectpracticeweb.com pl-vouchers.com
petedrummond.com pliks.pl
picsforkeywordsuggestion.com poptool.net
platesauto.com
pony-business.com
popup-fdm.xyz popup-fdm.xyz
pornoinn.com pornhub-ru.com
portnoff.od.ua positive2b.ru
pospr.waw.pl prohoster.info
powenlite24.ru prom23.ru
profit-opportunity.com purplesphere.in
profitsport.club quick-seeker.com
proftests.net quickbuck.com
promoforum.ru rangjued.com
pukaporn.com realresultslist.com
rank-checker.online relayblog.com
ranksays.com remmling.de
ric.info remontvsamare.su
ritlweb.com ringtonepartner.com
s-iwantyou.com rospromtest.ru
saltspray.ru rp9.ru
savefrom.com rukino.org
sayyoethe.blogspot.co.za sabizonline.com
scanner-alex.top saecsa.co
scanner-josh.top scanner-josh.top
scanner-mark.top scrapinghub.com
seansonline24.pl seadragonherbery.com
securityallianceservices.com seinterface.com
sell-fb-group-here.com semxiu.com
seogadget.ru seorank.info
seomarketings.online seozoom.it
seorankinglinks.com sexgalleries.top
sex-dating.co sexy-screen-savers.com
sexphoto.site shakhtar-doneck.ru
shopwme.ru shareyards.com
shtaketniki.kz shlyahten.ru
sims-sims.ru sindragosa.comxa.com
site-speed-checker.site sispe.com.br
siteexpress.co.il smokewithrabbits.com
siteheart.net soc-econom-problems.ru
slavkokacunko.de spb-plitka.ru
socialbutton.xyz spinnerco.ca
solmarket.by stpolice.com
souvenirua.com surfbuyermac.com
soviet-portal.do.am symphonyintegratedhealthcare.com
starpages.net szamponrevita.pl
studiofaca.com teletype.in
sugvant.ru tizanidinenarcotic.blogspot.com
superlist.biz tizanidinesideeffectsweightloss.blogspot.com
swimpool.ca tnctrx.com
tampabaywatch.org top10-online-games.com
td-l-market.ru top10registrycleaners.com
teastory.co track112.site
telesvoboda.ru tripper.de
terrafootwear.us veerotech.com
thebitcoincode.com villacoloniale.com
thewebsitetemplate.info vintontech.info
tilido.com vizitki.net
timetorelax.biz vsdshnik.com
tizanidinefibromyalgia.blogspot.com web.cvut.cz
todohr.com wesharepics.com
track-rankings.online wnhjavlhezp.gq
trackzapper.com wufak.com
tula.howotorg.ru ww1943.ru
tvnewsclips.info www.arenda-yeisk.ru
ucsol.ru wyniki-lista.pl
uhodzalijami.ru xcombear.ru
unitygame3d.com xfluro.com
usacasino.com xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
ustion.ru xn--lifehacer-1rb.com
velobikestock.com yogamatsexpert.com
via-energy-cumpara.com you-shall-not-pass.is74.ru
viagra-soft.ru yourads.website
vinylvault.co.uk zaim-pod-zalog-krasnodar.ru
virus-schutzmasken.de zakazfutbolki.com
vod.com.ua zaloadi.ru
vodkoved.ru zebramart.ru
voronezh.xrus.org zhuravlev.info
wheelchairliftsarea.com
wma-x.com
womens-journal.net
word-vorlagen.net
wordpresscore.com
wstroika.ru
wygraj-teraz.com
xbaboon.com
xingzi-vision.com
xlovecam.com
xn--80ab4aa2g.xn--p1ai
xn--c1acygb.xn--p1ai
yaminecraft.ru
yellowstonevisitortours.com
youghbould.wordpress.com
your-bearings.com
zajm-zalog-krasnodar.ru
zebradudka.com
zeroredirect1.com
znakomstva-moskva77.ru
zytpirwai.net

19
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -8,14 +8,7 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><html> </html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>
@ -581,4 +574,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.6 0; 10.1.0.68 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3030 ### Version: V4.2022.04.3031
### Updated: Thu Apr 7 22:01:43 UTC 2022 ### Updated: Fri Apr 8 10:03:41 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3030 ### Version: V4.2022.04.3031
### Updated: Thu Apr 7 22:01:43 UTC 2022 ### Updated: Fri Apr 8 10:03:41 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##