vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.07.3221 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-07-11 10:09:27 +00:00
parent 67f20b10fc
commit 8bfba19346
48 changed files with 102152 additions and 102135 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.07.3220 #### Version: V4.2022.07.3221
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.07.3220 #### Version: V4.2022.07.3221
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.07.3220 #### Version: V4.2022.07.3221
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12634
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.07.3220 #### Version: V4.2022.07.3221
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12638
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12638
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12638
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12638
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 1; 10.1.1.80 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 0; 10.1.1.80 0;
127.0.0.1 0; 127.0.0.1 0;

12638
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 1; 10.1.1.80 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 0; 10.1.1.80 0;
127.0.0.1 0; 127.0.0.1 0;

12638
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 1; 10.1.1.80 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 0; 10.1.1.80 0;
127.0.0.1 0; 127.0.0.1 0;

12638
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3220 3221

12638
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12638
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
23.98.150.56 0; 137.117.91.45 0;

12638
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 1; 10.1.1.80 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 0; 10.1.1.80 0;
127.0.0.1 0; 127.0.0.1 0;

12638
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 1; 10.1.1.80 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 0; 10.1.1.80 0;
127.0.0.1 0; 127.0.0.1 0;

12638
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 1; 10.1.1.80 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 0; 10.1.1.80 0;
127.0.0.1 0; 127.0.0.1 0;

12638
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
23.98.150.56 0; 137.117.91.45 0;

12638
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
23.98.150.56 0; 137.117.91.45 0;

12638
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 1; 10.1.1.80 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Go-Ahead-Got-It Domains Project
Msrabot EirGrabber
Musobot Grafula
Niki-bot Joomla
Nutch LeechGet
Semrush Mag-Net
Spinn3r PECL::HTTP
Web Sucker Page Analyzer
dataforseo.com isitwp.com
probe-image-size probe-image-size

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 01h4x.com
404checker 404enemy
ADmantX ADmantX
AIBOT AIBOT
ALittle Client
Abonti
Aboundex Aboundex
Aboundexbot Aboundexbot
Acunetix
AhrefsBot
Aipbot
Alexibot Alexibot
Alligator Anarchie
Apexoo Apexoo
Aspiegel Asterias
AwarioRssBot Atomseobot
BBBike
BDCbot BDCbot
BDFetch
BLEXBot BLEXBot
BackDoorBot
BacklinkCrawler BacklinkCrawler
BatchFTP Badass
BetaBot Bitacle
BlowFish Buddy
Brandprotect
BuiltBotTough
BuiltWith BuiltWith
BuzzSumo BuzzSumo
Calculon CSHttp
CazoodleBot
CensysInspect CensysInspect
Chlooe CheeseBot
CherryPicker
ChinaClaw
Citoid Citoid
Claritybot Cliqzbot
Cloud mapping
Cogentbot Cogentbot
Collector
Copier
Copyscape Copyscape
Crescent Crawling at Home Project
Custo CrazyWebCrawler
Curious
CyotekWebCopy CyotekWebCopy
DIIbot
DataCha0s DataCha0s
DatabaseDriverMysqli
Deusu Deusu
Devil Digincore
DigitalPebble DigitalPebble
Dirbuster Disco
Discoverybot DnyzBot
DomainCrawler DomCopBot
DomainStatsBot Domains Project
Dotbot
Download Wonder Download Wonder
Drip
EasyDL EasyDL
EroCrawler Ebingbong
Ecxi
EirGrabber
ExtLinksBot ExtLinksBot
ExtractorPro Extractor
Extreme Picture Finder Extreme Picture Finder
FDM FDM
FlashGet Firefox/7.0
Flunky Freeuploader
Foobot
FrontPage
FyberSpider
Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
GermCrawler GalaxyBot
Genieo
GetRight GetRight
GetWeb
Getintent
Gigabot
Go-Ahead-Got-It
GoZilla GoZilla
Grafula GrabNet
GrapeshotCrawler GrapeFX
GridBot
HEADMasterSEO
HTTP::Lite HTTP::Lite
HTTrack HTTrack
Havij Haansoft
Humanlinks Hloader
HybridBot
IDBTE4M IDBTE4M
IRLbot IDBot
Iblog Id-search
IlseBot
Image Fetch Image Fetch
Indy Library Image Sucker
InfoTekies
Intelliseek
Internet Ninja
InternetSeer InternetSeer
JOC Web Spider Iskanie
JennyBot IstellaBot
JetCar JamesBOT
Jetty Jetty
Joomla Jorgee
Kenjin Spider Kenjin Spider
Keybot Translation-Search-Machine LWP::Simple
Keyword Density
Lanshanbot
LeechFTP LeechFTP
LeechGet
LexiBot
Lftp Lftp
Likse LibWeb
LinkScan
LinkWalker LinkWalker
LinkpadBot Linkbot
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
Lmspider Ltx71
MIDown tool
MIIxpc
MQQBrowser MQQBrowser
MSFrontPage Mag-Net
MTRobot Magnet
Mail.RU_Bot Majestic SEO
Majestic-SEO Majestic-SEO
MarkMonitor
Mass Downloader
Masscan Masscan
Mata Hari
Mb2345Browser
Mediatoolkitbot
MegaIndex.ru
Metauri
Microsoft URL Control Microsoft URL Control
Minefield Minefield
Mister PiX
Moblie Safari
Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mr.4x3 Mozlila
Musobot NICErsPRO
NearSite Navroad
Nessus
NetMechanic NetMechanic
NetSpider
NetZIP NetZIP
Netcraft Netvibes
Nettrack
Niki-bot Niki-bot
NimbleCrawler Octopus
Nutch
Offline Explorer Offline Explorer
Offline Navigator Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
Openvas PECL::HTTP
OrangeSpider POE-Component-Client-HTTP
OutclicksBot
PHPCrawl
PageAnalyzer
PageScorer PageScorer
PageThing.com
Pandalytics Pandalytics
Panscient Panscient
Pi-Monster Pi-Monster
Picsearch
PictureFinder
Pimonster Pimonster
Pixray
PleaseCrawl PleaseCrawl
ProWebWalker ProWebWalker
Probethenet Psbot
Pu_iN
Pump
PxBroker
PyCurl PyCurl
Quick-Crawler RSSingBot
RankActive
RankActiveLinkBot
RankingBot RankingBot
Re-re RankingBot2
RealDownload
Recorder
RepoMonkey RepoMonkey
RocketCrawler Ripper
Rogerbot SBIder
SISTRIX SEOkicks
ScoutJet SEOkicks-Robot
Searchestate SEOstats
SearchmetricsBot SMTBot
ScanAlert
SemanticJuice SemanticJuice
SemrushBot SeobilityBot
SeoSiteCheckup
Seomoz Seomoz
Shodan Shodan
Site Sucker
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteSucker SiteSucker
Sitevigil SmartDownload
Snake
Snapbot
Snoopy
SocialRankIOBot
Sociscraper
Sogou web spider Sogou web spider
Sosospider
Sottopop Sottopop
SpaceBison Spammen
Spanner
Spinn3r Spinn3r
Sucuri Sqlmap
Surfbot Steeler
Szukacz
T8Abot T8Abot
TeleportPro Teleport
Telesphoreo Telesoft
Telesphorep TheNomad
Toweyabot Thumbor
Trendiction TightTwatBot
True_Robot Tracemyfile
Trendictionbot
Turingos
Twice Twice
Typhoeus
URLy Warning URLy Warning
URLy.Warning
Upflow
VB Project
Vacuum
Vagabondo Vagabondo
VeriCiteCrawler
Virusdie Virusdie
VoidEYE VoidEYE
WEBDAV
WPScan WPScan
Wallpapers/3.0 WWW::Mechanize
WallpapersHD Wallpapers
WeSEE
Web Fetch
Web Fuck Web Fuck
Web Sucker Web Sucker
WebAuto WebAuto
WebCopier WebBandit
WebCollage
WebEnhancer WebEnhancer
WebFetch WebFetch
WebGo IS WebGo IS
WebLeacher WebImageCollector
WebPix
WebSauger WebSauger
WebWhacker WebZIP
Website Quester Webshag
WebsiteExtractor
WebsiteQuester
Whacker Whacker
Widow Whatweb
Wonderbot WinHTTrack
Wotbox Woobot
Xaldon_WebSpider Xaldon WebSpider
Xenu
Zermelo Zermelo
Zeus Zitebot
ZmEu
ZoominfoBot ZoominfoBot
ZumBot archive.org_bot
ZyBorg backlink-check
autoemailspider
cah.io.community cah.io.community
check1.exe clark-crawler
cognitiveseo
crawl.sogou.com crawl.sogou.com
crawler.feedback crawler4j
domainsproject.org
eCatch
evc-batch evc-batch
internetVista monitor facebookscraper
ips-agent
linkdexbot linkdexbot
lwp-trivial
meanpathbot
mediawords mediawords
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
oBot polaris version
page scorer probe-image-size
s1z.ru ripz
scalaj-http
scan.lol scan.lol
seostar seoscanners
sitechecker.pro sexsearcher
siteripz
sogouspider
spyfu spyfu
ubermetrics-technologies.com sysscan
tAkeOut
trendiction.com
trendiction.de
webgains-bot
webpros.com webpros.com
webprosbot webprosbot
x09Mozilla

282
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404enemy
80legs 80legs
ADmantX
ASPSeek
Abonti Abonti
Aboundexbot Aboundexbot
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
Aipbot Alligator
AllSubmitter
Anarchie Anarchie
Anarchy Anarchy
Anarchy99 Apexoo
Ankit Attach
Anthill
Aspiegel
AwarioRssBot
AwarioSmartBot
BBBike BBBike
BDFetch BDFetch
BLEXBot
BackDoorBot
BackStreet BackStreet
BackWeb Backlink-Ceck
BacklinkCrawler Barkrowler
Bandit
BatchFTP BatchFTP
Battleztar Bazinga Battleztar Bazinga
Bitacle Bitacle
Black Hole Black Hole
Blow BlackWidow
BlowFish Boardreader
BunnySlippers BotALot
CATExplorador Brandwatch
CODE87 CCBot
Calculon Calculon
Cegbfeieh
CensysInspect
CheTeam CheTeam
CherryPicker CherryPicker
ChinaClaw
Chlooe Chlooe
Cliqzbot Claritybot
Cloud mapping Copyscape
Cocolyzebot Craftbot
Collector Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Curious Curious
DBLBot DBLBot
DataCha0s DTS Agent
DatabaseDriverMysqli
Demon Demon
Deusu Deusu
Digincore Devil
DigitalPebble DigitalPebble
Dirbuster
Disco Disco
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot
DomainAppender
DomainCrawler
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project
Download Wonder
Dragonfly Dragonfly
Drip Drip
EroCrawler ECCP/1.0
Evil EMail Siphon
Express WebPictures EMail Wolf
Extractor EasyDL
ExtractorPro EirGrabber
Exabot
EyeNetIE EyeNetIE
Ezooms
FHscan FHscan
FemtosearchBot
FlashGet
Flunky
Freeuploader
FrontPage FrontPage
GT::WWW GT::WWW
Genieo GetRight
GoZilla GetWeb
Go-Ahead-Got-It
Gotit Gotit
HTTP::Lite Grabber
Grafula
GrapeshotCrawler
GridBot
Haansoft Haansoft
Heritrix HaosouSpider
Harvest
Hloader
Humanlinks
IDBTE4M
Id-search Id-search
IlseBot IlseBot
Image Sucker
IndeedBot
Intelliseek
Internet Ninja
Iria Iria
JamesBOT JOC Web Spider
JetCar
JikeSpider JikeSpider
Kenjin Spider Jorgee
JustView
LWP::Simple LWP::Simple
Lanshanbot
LeechFTP
LeechGet
Lftp Lftp
LibWeb Likse
Linkbot Ltx71
LinkextractorPro MJ12bot
LinqiaMetadataDownloaderBot
LinqiaRSSBot
MIDown tool
MIIxpc
MQQBrowser MQQBrowser
MSIECrawler Mail.RU_Bot
MTRobot Mass Downloader
MarkMonitor Masscan
MauiBot Mata Hari
MegaIndex.ru Mediatoolkitbot
Metauri Metauri
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Minefield
Mister PiX Mister PiX
Moblie Safari Mojolicious
Mojeek Mozlila
MolokaiBot Nameprotect
Morfeus Fucking Scanner
Mr.4x3
NPbot
Nessus Nessus
Net Vampire NetSpider
NetAnts
NextGenSearchBot NextGenSearchBot
Nibbler Nibbler
Niki-bot Niki-bot
NimbleCrawler NimbleCrawler
Offline Navigator Octopus
OpenVAS Offline Explorer
OrangeBot Openfind
OrangeSpider OutclicksBot
PHPCrawl PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageGrabber Page Analyzer
PageScorer
PageThing.com PageThing.com
Pandalytics Papa Foto
Pavuk Pi-Monster
PeoplePal
Picscout Picscout
Picsearch
PictureFinder PictureFinder
Piepmatz Pimonster
PleaseCrawl ProPowerBot
Pump Psbot
PyCurl PyCurl
RankActive
RankFlex RankFlex
RankingBot RankingBot
RankingBot2
Rankivabot
RankurBot RankurBot
SBIder Re-re
SEOkicks-Robot RealDownload
SEOprofiler RedesScrapy
SEOstats Ripper
SISTRIX RocketCrawler
SMTBot SMTBot
SalesIntelligent SalesIntelligent
ScanAlert
Scanbot
Scrapy
Screaming Screaming
Searchestate ScrepyBot
SearchmetricsBot
SemanticJuice
Semrush
SemrushBot SemrushBot
SentiBot
SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz Seomoz
Shodan Shodan
Site Sucker
SiteSnagger SiteSnagger
SiteSucker SiteSucker
SmartDownload Sitebeam
Snake Snapbot
Snoopy Snoopy
Sogou web spider SocialRankIOBot
Sosospider Sociscraper
SpaceBison SpaceBison
Spammen Spammen
Sqlmap SpankBot
Sqlworm Sqlworm
Sucuri Sucker
SuperBot
SuperHTTP SuperHTTP
Surfbot Surfbot
Suzuran Suzuran
Swiftbot
Teleport Teleport
TeleportPro
Telesoft Telesoft
Telesphoreo The Intraformant
Telesphorep TheNomad
Thumbor Thumbor
Titan TightTwatBot
Trendiction Toweyabot
Tracemyfile
Trendictionbot Trendictionbot
True_Robot True_Robot
Turingos Turingos
TwengaBot Turnitin
Twice
Typhoeus Typhoeus
Vacuum URLy Warning
VidibleScraper URLy.Warning
WEBDAV UnisterBot
Upflow
VCI
Vagabondo
VoidEYE
Voil
WISENutbot
WPScan WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWW::Mechanize
Wallpapers/3.0 Wallpapers
WallpapersHD WeSEE
Web Enhancer Web Auto
Web Fuck Web Fetch
Web Pix Web Pix
Web Sauger Web Sucker
WebBandit WebImageCollector
WebEnhancer
WebFetch
WebGo IS
WebLeacher WebLeacher
WebReaper WebReaper
WebSauger
WebWhacker WebWhacker
Webshag
Website Quester Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster
Whack
Whacker
Whatweb
Widow Widow
Wotbox WiseGuys Robot
Zade Woobot
Zauba Xenu
Zermelo Zermelo
Zeus Zeus
ZumBot ZmEu
adscanner ZoomBot
ZoominfoBot
arquivo-web-crawler
arquivo.pt arquivo.pt
cah.io.community cah.io.community
com.plumanalytics check1.exe
crawler.feedback cognitiveseo
crawler4j
dataforseo.com dataforseo.com
eCatch demandbase-bot
domainsproject.org
evc-batch
facebookscraper facebookscraper
heritrix meanpathbot
internetVista monitor
iubenda-radar
mediawords mediawords
muhstik-scan
netEstate NE Crawler
oBot oBot
polaris version pcBrowser
probe-image-size
ripz ripz
s1z.ru s1z.ru
scan.lol
seobility seobility
seocompany.store
seostar
serpstatbot serpstatbot
siteripz siteripz
sogouspider
sp_auditbot sp_auditbot
spyfu
tAkeOut tAkeOut
ubermetrics-technologies.com trendiction.de
webmeup-crawler webgains-bot
webpros.com
webprosbot webprosbot

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
0daymusic.org 100searchengines.com
adloads.net amehdaily.com
anniemation.com brimstonehillfortress.org
flooringinstallation-edmonton.com clubfashionista.com
gg.zzyjxs.com enginebay.ru
glass-msk.ru javrip.net
member-quiz.com mobile.ok.ru
nippon-bearings.ru offtime.ru
onstrapon.purplesphere.in piccshare.com
tsyndicate.com xtremeeagles.net

906
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
101flag.ru 1-99seo.com
101lesbian.xyz
123movies.love 123movies.love
12masterov.com
1bet.com
1j7740kd.website 1j7740kd.website
2000k.ru 1kdailyprofit.me
5ws.dating-app.ru 28n2gl3wfyb0.ru
888.com 2ads.co.uk
91abcw.com 4kepics.com
4rent.ru
7yue.org
999webdesign.com
DomainStatsBot
a.pr-cy.ru a.pr-cy.ru
a1security.com.ua a14download.com
ad-words.ru aarbur.com
adamoads.com adanih.com
adconscious.com
add-add.men add-add.men
adsloads.com addtoadd.men
adult3dgames.com adelly.bg
advancedcleaner.com adidas.frwebs.fr
advokat-grodno.by adminshop.com
afslankpillen2017nl.eu adrunnr.com
agecheckadult.com adtiger.tk
albuteroli.com airmaxshoes-2016.com
alekseevec.ru all4invest.info
alfabot.xyz
allbizne.co.ua allbizne.co.ua
allblogroll.com allkrim.com
allnews.md alloysteel.ru
allwidewallpapers.com amazon-seo-service.com
alphaforum.ru analytics-ads.xyz
alvtank.se anti-virus-removal.info
alyeskaresort.com apibetweenlinesn-a.akamaihd.net
ameblo.top apiwebwebgetcom-a.akamaihd.net
ample-awards-today.us apple.com-webbrowsing-security.review
anabolics.shop appmsr.org
ananumous.ru aquarium-pro.ru
anmysite.com arenanews.com.ua
anonymizeme.pro arpelsreplica.xyz
aosheng-tech.com arraty.altervista.org
apartamentwroclaw.eu artpicso.com
apccargo.com asiengirls.net
apioasisspacenet-a.akamaihd.net asmxsatadriverin.aircus.com
appfastplay.com asseenontvonline.ru
apple.com-webbrowsing-security.science automatic-seo.com
arabseyes.com automobile-spec.com
ardimobileinfo.ml autoplate.info
arenda-avtoprokat-krasnodar.ru autovideobroadcast.com
artdestshop.eu availit.weebly.com
azartclub.org avtochehli.by
babyguns.ru ayakino.net
bad-stars.net azbaseimages.net
bala.getenjoyment.net balkanfarma.org
balook.com bankofthewext.com
bangla-chat-uk-paralud.ga baoxaydung.com.vn
bannerspace.com bdsmgalls.net
bannerswap.com beachtoday.ru
bannertesting.com beauty-clinic.ru
bard-real.com.ua beauty-things.com
bbtec.net becuo.com
beetpics.pw
berdasovivan.ru
best-games.today best-games.today
bestadbid.com
bestfortraders.com bestfortraders.com
bestwaystogetpaid.us bestpriceninja.com
bezcmexa.ru bestssaker.com
biketank.ga bget.ru
bimatoprost-careprost.com.ua bgviagrachrx.com
bingoporno.com bio-japan.net
binomo.kz bio.trade-jp.net
bluesalt.co birzha-truda.eu
boltalko.xyz bizfly.info
brandehk.dk biznesluxe.ru
btnativenav.com biznesrealnost.ru
bumble.cheapwebsitehoster.com blackcurranthumidifiers.site
buyclomidonlaine.com bookmark4you.com
call-of-duty.info braintobe.win
canyougethighofftizanidine.blogspot.com brateg.xyz
carabela.com.do breastaugmentation.co.za
carloans.com bristolhostel.com
cartierlovestore.com budilneg.xyz
bugshoot.cn
burn-fat.ga
bus-offer.com
buscarfoto.com
buttons-for-website.com
buzzonclick.com
buzzzg.men
bvlgariring.xyz
bzero1jewelry.net
cand.jp
candlehandmade.com
candy-glam-hp.com
candycrushshop.com
canrioloadm.gq
cartierreplica.pw cartierreplica.pw
censys.io casablancamanor.co.za
chatseo.com castingbank.ru
cheap-pills-norx.com cbcseward.com
checkhit.com chatroulette.online
checkpagerank.net
children-learningreading.info
chocolateslim-en-france.com
choosecuisine.com
cialgenisrx.com cialgenisrx.com
ciekawinki.pl cicaki.net
clickbank.net
clickso.com
clippingphotoindia.com clippingphotoindia.com
clmforexeu.com clmforexeu.com
club-musics.ru club-lukojl.ru
compliance-donald.xyz club-samodelkin.ru
compliance-julia.top co.lumb.co
conboy.us com-cleaner.systems
continent-e.tv comeondog.info
coolingoods.com compliance-brian.top
coolwallpapers-hd.com compliance-don.top
copypaste.traderzplanet.in compliance-mary.top
cornomase.win concealthyself.com
corta.co contentsexpress.com
cougfan.info cookielawblog.wordpress.com
countercrazy.com crackzplanet.com
craftinsta.ru credit-respect.ru
creatives.adbetclickin.pink creditmoney.com.ua
crirussian.ru
culmimg.pw culmimg.pw
customergrowthsystems.com cyse.tk
customerpromos-a.akamaihd.net czeshop.info
cymbaltaandtizanidine.blogspot.com dailyfinancefix.com
danschawbel.com de.zapmeta.com
dbmkfhqk.bloger.index.hr dikqlyremy.info
decorazilla.com dktr.ru
deplim.com domination.ml
dev33.dioniqlabb.se dostavimvdom.ru
devochki.top dosugrostov.site
dfwu1019.info doublepimp.com
diminishedvalueoforegon.com downloadmefiranaratb1972.xpg.com.br
dipstar.org doxyporno.com
dispo.de druzhininevgeniy63.blogspot.com
djonwatch.ru dverimegapolis.ru
domaincrawler.com e-c.al
downloadkakaotalk.com
dvrlists.com
dyt.net
e-collantes.com
e2click.com e2click.com
ebonyporn.site easync.io
ekn-art.se ednorxmed.com
elektronischezigarettekaufen1.beeplog.com eets.net
elektrozigaretten2.yn.lt ege-essay.ru
elite-sex-finders.com ekto.ee
en.home-task.com elektir.ru
enskedesquashclub.se elektrischezi.canalblog.com
ereko.ru empathica.com
erotikchat-24.com empis.magix.net
escortslet.net enthuse.computernetworksonline.com
euromasterclass.ru eonpal.com
european-torches.ru essay-edu.biz
event-tracking.com excitacion.info
evidencecleanergold.com ezofest.sk
extlinks.com face.hostingx.eu
ezigaretteshop2.vefblog.net falopicm.pw
f-loaded.de fast-torrent.ru
f07.de feel-planet.com
falcoware.com findpics.pw
farmingworm.com flashahead.info
fba-mexico.com flipper.top
fefo.gdn floridamhca.org
fickblock18.com formulaantiuban.com
fifa-coins.online forpostlock.ru
filmgo.ru forsex.info
flash4fun.com
fotosfotos.eu
fotoxxxru.com
freefoto.ca freefoto.ca
friendflnder.com gael-s.ru
fugarif.ga galleryawesome.com
fyl.com.ru gameonasia.com
gaz-voshod.ru geile-lelly.eu
gerhardhealer.com glavtral.ru
get-seo-domain.com goblacked.com
given2.com goodprotein.ru
gosarhivrt.ru gopixdatabase.com
graphid.com
gratis-sexkontakte.com gratis-sexkontakte.com
gribkovye-zabolevaniya.com gribkovye-zabolevaniya.com
grizzlysgrill.com gribokstop.com
growmyfunds.ca groupmoney.ru
growshop.es gunsvicceadadebt.tk
hahashka.ru hahashka.ru
hao123.com
hasfun.com hasfun.com
hauleddes.com hauleddes.com
hd-film.pl hdpixion.com
hd-filmy.net headpharmacy.com
hdfreeporno.net healthcarestore.info
hdseriale.pl homik.pw
heartofpayne.xyz hotgirlhdwallpaper.com
hostsshop.ru houdom.net
hotelcrocenzi.sm hptwaakw.blog.fc2.com
houtings.xyz husky-shop.cz
hybrid.ru hyipmanager.in
hystersister.com hystersister.com
i4track.net i4track.net
ico.re iamsport.org
ideibiznesa2015.ru imgata.com
igadgetsworld.com
igrovyeavtomaty777.ru
iideaidekonkatu.info
imediadesk.com
imitex-plus.ru
in-tandem.co
incolors.club
indiasourcemart.in indiasourcemart.in
inet-traffic.com innodgfdriverhm.aircus.com
infowarcraft.ru ipchicken.com
int.search.tb.ask.com ipornox.xxx
intellekt21.ru iqoption-bin.com
internetartfair.com iskalko.ru
iptool.xyz iwantmyfreecash.com
isitpaleo.info
itch.io
iusstf.org
iwanttodeliver.com iwanttodeliver.com
izhevsk.zrus.org izhevsk.zrus.org
j-times.ru jacago.com
jasonpartington.com javatex.co.id
jerseywholesalebizchina.com jerseywholesalebizchina.com
jmat.cn jobmarket.com.ua
jovencitas.gratis kaliningrad.zrus.org
jpcycles.com kamagragelusa.net
juliadiets.com kaz.kz
jus0wil.pp.ua
karpun-iris.ru
keywordspay.com keywordspay.com
khadastoafarde.tk keywordteam.net
kino-rf.ru kinobaks.com
kinoduh.ru kinotorka.ru
kinogolos.ru kirov.zrus.org
kinogonew.ru kletkimehan.ru
klin3952.ru kol-energo.ru
klitimg.pw konpax.com
kongoultry.net koral.se
kreativperlen.ch kovrenok.ru
kriokomora.info krasnodar.ru
kung-fu-ru.com kw21.org
kupiproday.com.ua
labafydjxa.su
labplus.ru labplus.ru
lacave.ntic.fr labvis.host.sk
laexotic.com ladsblue.com
laptoper.net lancheck.net
laudit.ru landscaping.center
law-check-nine.xyz larchik.net
laxob.com larger.io
lazymanyoga.com lashstudia.ru
ldrtrack.com lasvegaslockandsafe.com
lecbter-relationships.ga law-check-eight.xyz
lezbiyanki.net lawyers.cafe
lignofix.ua lechenie-gemorroya.com
linksharingt.com lida-ru.com
liveinternet.ro life-instyle.com
lmrauction.com linkpulse.com
locksmith.jp lion.bolegapakistan.com
lodki-pvh.dp.ua livejasmin.com
long-beach-air-conditioning.com lkbennettstore.online
longgreen.info lolnex.us
lottospring.com losangeles-ads.com
lost-alpha.ru
lostaruban.com
lsex.xyz
lsp-awak-perikanan.com lsp-awak-perikanan.com
luciddiagnostics.in luckyday.world
luxmagazine.cf luckyshop.net.ua
macdamaged.tech m-google.xyz
magnetic-bracelets.ru mabdoola.blogspot.com
mamasuper.prom.ua mainhp.net
manifestation.betteroffers.review malwareremovalcenter.com
marketingvici.com manifestyourmillion.com
marmitaco.cat marketland.ml
matpre.top matchpal-a.akamaihd.net
maxthon.com mathgym.com.au
mearns-tractors.co.uk mebel-vstroika.ru
medicineseasybuy.com medmajor.ru
medkletki.ru megaindex.ru
meltwater.com meltwater.com
mensandals.xyz metalonly.info
menstennisforums.com metarip.ru
metaxalonevstizanidine.blogspot.com
methodsmarketing.com
micasainvest.com micasainvest.com
michaelkorsoutlet.store minharevisao.com
mikrobiologies.ru minyetki.ru
mini.7zap.com mobile-appster.ru
misterjtbarbers.com monclerboots.xyz
mixtapetorrent.com money-every-day.com
mixx.com montazhnic.ru
mole.pluto.ro moonci.ru
monclercoats.xyz moscow.xrus.org
monetizer.com-01.site mossmesi.com
moneyteam24.com mowser.com
more-letom.ru moyaterapiya.ru
moxo.com mrlmedia.net
mp3ringtone.info mtmtv.info
mrbitsandbytes.com muabancantho.info
mriyadh.com mydearest.co
musicdaddy.net mylida.org
musicstock.me mysexpics.ru
muz-baza.net nacap.ru
myanyone.net nalogovyykodeks.ru
myfreemp3.eu narkologiya-peterburg.ru
myhealthcare.com networkad.net
myphotopipe.com newhairstylesformen2014.com
myxdate.info news-speaker.com
nac-bearings.ru newstudio.tv
narkologiya-voronezh.ru nfljerseys.online
national-today-winning-winner.club nicefloor.co.uk
nationalbreakdown.com nsatc.net
news-readers.ru o333o.com
newstaffadsshop.club obiavo.in
ningessaybe.me officedocuments.net
nippon-bearings.ru old-rock.com
november-lax.com olympescort.com
nullrefer.com onemantrip.com
ofanda.com oneshotdate.com
offer.wpsecurity.website ons-add.men
office2web.com ooo-olni.ru
okonich.com.ua oops-cinema.ru
online-zaymy.ru
ontargetseo.us
origin-my.ru origin-my.ru
ororodnik.goodbb.ru ostrovtaxi.ru
osagonline.ru ourville.info
outshop.ru outshop.ru
paclitor.com ownshop.win
panchro.xyz oxotl.com
partner-host.men panicatack.com
partner-print.men panouri-solare-acoperis.com
pcgroup.com.uy partner-cdn.men
pcimforum.com partner-high.men
pdamods.ru paydayloanslocal.com
pdfprof.com peekyou.com
perfection-pleasure.ru penisvergrotendepillennl.ovh
picture-group.com phobia.us
pierrehardysale.online photo-clip.ru
pics-group.com
picscout.com
picsfair.com
pila.pl pila.pl
pinkduck.ga piluli.info
pistonclasico.com pio.polytopesexempt.com
piter.xrus.org pix24x7.com
piulatte.cz pl.id-forex.com
plastgran.com play-mp3.com
platezhka.net podshipniki-nsk.ru
pochtovyi-index.ru pogodnyyeavarii.gq
pooleroadmedicalcentre.co.uk pons-presse.com
poptool.net porngalleries.top
popugauka.ru
popugaychiki.com
popunder.ru
pornhub-ru.com
porno-raskazy.ru porno-raskazy.ru
porno.simple-image.com.ua pornogad.com
pornobrazzers.biz pornoplen.com
pornofeuer.com pornzone.tv
pornoklad.net potoideas.us
pornokorol.com preparevideosafesystem4unow.site
portside.cc progress-upakovka.ru
poshiv-chehol.ru prokotov.com
pretty-mart.com promoforum.ru
preventheadacheguide.info pwwysydh.com
pricheski-video.com pxhdwsm.com
printingpeach.com quelle.ru
productarium.com quick-seeker.com
profolan.pl quizzitch.net
proprostatit.com qwesa.ru
prospekt-st.ru realting-moscow.ru
prostitutki-yaroslavlya.org
pulse33.ru
pussyspace.net
puzo2arbuza.ru
quick-offer.com
rabot.host.sk
rcpmda.ikan1080.xyz
realtytimes.com
rebrand.ly rebrand.ly
redirectingat.com rebuildermedical.com
refads.pro refads.pro
remont-comp-pomosh.ru refudiatethissarah.info
remorcicomerciale.ro registrysweeper.com
renhacklids.tk reimageplus.com
rewardit.com reklamuss.ru
ringporno.com rentehno.ru
rootandroid.org resellerclub.com
rusprostitute.com restaurantlescampi.com
rusvideos.su richinvestmonitor.com
saitevpatorii.com rniaeba.ga
saletool.ru rockprogblog.com
saneitconsulting.com rossmark.ru
sanidumps.com roznica.com.ua
satoristudio.net rucrypt.com
saveriopiazza.it rvtv.ru
sawin.beth.webd.pl s.lollypopgaming.com
sbricur.com samo-soznanie.ru
scanner-don.top santasgift.ml
scanner-jack.top savememoney.co.za
screentoolkit.com savetubevideo.com
seadragonherbery.com scanner-alex.top
searchaddis.com scanner-elena.top
searchinquire.com scansafe.net
searchlock.com scripted.com
seksvideoonlain.com search.xtconnect.com
semprofile.com searchkut.com
seocheckupx.com securityallianceservices.com
serdcenebolit.com seminarygeorgia59.ga
sexsaoy.com serialsway.ucoz.ru
share-buttons-for-free.com sex-watch.com
shoesonlinebuy.cn sexpornotales.net
sibdevice.ru sexvporno.ru
silverage.ru sexytrend.ru
sinestesia.host.sk sharebutton.net
siteripz.net shariki-zuma-lines.ru
sluganarodu.ru sheerseo.com
small-game.com ship-marvel.co.ua
smichovbike.cz shitting.pro
snjack.info signal03.ru
snsdeainavi.info sisiynas.ru
sobecjvuwa.com.ru site-speed-checker.site
social-s-ggg.xyz siteonomy.com
sockshare.net smadihome.com
soheavyblog.com snegozaderzhatel.ru
solitaire-game.ru snjatie-geroinovoy-lomki.ru
soserfis.com social-fun.ru
space4update.pw social-s-hhh.xyz
spravka-medosmotr.ru socialmadesimple.com
stopnarco.ru softomix.net
strana-solnca.ru spidtest.org
stroilka.info sr-rekneskap.no
stromerrealty.com stariy-baku.com
stmassage.ru
stop-gepatit.te.ua
store-rx.com
stroicol.net
stroymonolit.su stroymonolit.su
strv.se sucsesofinspiration.com
studentguide.ru sugarkun.com
stuff-about-money.com sunflowerdrawingpaintings.blogspot.com
sukarame.net superfish.com
support.nopeas.sk surcentro.com
survival.betteroffers.review swsociety.se
swimpool.ca symphonyintegratedhealthcare.com
syvertsen-da.no tabletkinaodchudzanie.com.pl
szamponrevita.pl taraz.xkaz.org
takethatad.com tasteidea.com
tarad.com td-33.ru
teesdaleflyballclub.co.uk teencastingporn.com
telefonsexkostenlos.tk the-torrent-tracker.blogspot.com
the-usa-games.blogspot.com thedownloadfromwarez.blogspot.com
themestotal.com thegolfclub.info
thewebsitetemplate.info thenetinfo.com
ticketsys.inetwd.com thetoiletpaper.com
tizanidine6mg.blogspot.com titan-ads.life
tizanidinehydrochloride2mg.blogspot.com titan-cloud.life
topads.men tizanidinehcl2mgtablet.blogspot.com
tizanidinenarcotic.blogspot.com
tizanidineusedfor.blogspot.com
tjkckpytpnje.com
tmearegion26.com
tnctrx.com
topclickguru.com topclickguru.com
torture.ml toproadrunner5.info
tourismvictoria.com torrentdownloadhub.com
torrnada.ru
toursmaps.com toursmaps.com
toy-shop.top trafficjunky.net
treasuretrack-a.akamaihd.net traffpartners.com
trkdf.com tramadolandtizanidine.blogspot.com
trichizobswiv.agddns.net
trubywriting.com trubywriting.com
tsyndicate.com truemfilelj.gq
ttrraacckkrr.com
tubeoffline.com tubeoffline.com
tuckermktg.com
tupper-posuda.ru tupper-posuda.ru
turbodsp.com ua-company.ru
u555u.info
ublaze.ru ublaze.ru
udayavani.com uchetunet.su
uogonline.com uchil.net
uptime-alpha.net udsgame.online
uroki.net ukkala.xyz
usdx.us ukrtvir.com.ua
uzporno.mobi unitexindia.com
vacuumscleaner.com ural-buldozer.ru
vancleefreplica.pw uvozdeckych.info
vbabule.net velobikestock.com
vektorpress.ru vereo.eu
villakohlanta.nu versaut.xxx-cam.webcam
vip-dom.in viagengrarx.com
vkgaleria.com viagroid.ru
voluumtrk.com videochat.mx
wahicbefa31.soup.io videochaty.ru
wardreapptokone.tk videtubs.pl
wesharepics.com vietimgy.pw
westum.se vilingstore.net
wholesalejerseys-cheapest.com virtuagirl.com
whos.amung.us virus-schutzmasken.de
whosonmyserver.com vitoriacabos.com
windowssearch-exp.com volgograd.xrus.org
winner7777.net w7s.ru
wallpaperaccess.com
wallpapersdesk.info
watracker.net
webshoppermac.com
website-audit.com.ua
webtherapy.ru
welcomeauto.ru
wma-x.com wma-x.com
womens-journal.net wnhjavlhezp.gq
wordpresscore.com worst-sites.online
works.if.ua writingservices17.blogspot.ru
www.888.com www.arenda-yeisk.ru
www.bookmaker-bets.com www.solartek.ru
x-mix.info xev.ru
x-musics.com xgames-04.com
xlolitka.com xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xmlinde.com xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai xn--h1aakne2ba.xn--p1ai
xtubeporno.net xtubeporno.net
xwatt.ru xvideosbay.com
yellowproxy.net xxxasianporn.net
yoopsie.com
youjizz.com
yourads.website yourads.website
yourdesires.ru zahvat.ru
youtubologia.it zapiszto.pl
yuarra.pluto.ro zastenchivosti.net
yunque.pluto.ro zazagames.org
zagadki.in.ua zerocash.msk.ru
zarabiaj-dzis.pl zetmaster.ru
zigarettenonlinekaufen.tumblr.com zog.link
znakomstvaonlain.ru zoosexart.com
zs2vm.top
zvezdagedon.ru
zygophyceous.womanstars.site

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
24videos.tv 08800.top
2drittel.de 4kpics.rocks
3weekdiet.com 51.la
4k-player.pl 883zy.com
aab94f698f36684c5a852a2ef272e031bb.com 8si.ru
abcdeg.xyz adnotbad.com
actionnooz.com ads.tremorhub.com
adelly.bg adult3dgames.com
adtech.us advertisingtag.net
adviceforum.info agysya.ru
affrh2015.com alive-ua.com
agahinameh.com allstatesugarbowl.org
agardomains.com alphacarolinas.com
alevinefesleri.com alphahoverboards.com
all4invest.ru angigreene.com
anime.dougasouko.com animenime.ru
app-ready.xyz apispringsmartne-a.akamaihd.net
arkartex.ru applyneedy.xyz
ascat.porn asiengirls.net
asscenihotosidea.blogspot.co.za atley.eu.pn
asseenontv.ru autobudpostach.club
autogrep.ru autochoixspinelli.com
avitocars.ru autotrends.today
babosas.com azbukafree.com
backgroundpictures.net battlecarnival.su
bbtec.net bayareaaeroclub.org
beachdriveblog.com bg6s0.com
bedandbreakfast.com biteg.xyz
belinka.com.ua btvn.ru
best-games.today bupropion-sr-150-mg.us
bestmobilityscooterstoday.com buyscabiescream.com
bestofferswalkmydogouteveryday.gq buyviagraa.blogspot.com
bin-brokers.com calvet.altervista.org
binomo.kz canzoni.ru
bjanshee.ru carina-sy.de
bluesalt.co cartierreplica.top
boom-celebs.com cashkitten-a.akamaihd.net
brandedleadgeneration.com ceotrk.com
bronzeaid-a.akamaihd.net cerev.info
builtwith.com chatroulette.world
buzzurl.jp
cakesplus.com.au
carsplate.com
cartechnic.ru
cete.ru
cheapbarbour.online
cheapestjerseys-wholesale.com
cheapestjerseysonwholesale.com cheapestjerseysonwholesale.com
cibpenonptib.flu.cc chidporn.com
cindymatches.com
clean-start.net
cleanpcnow.com cleanpcnow.com
clickzzs.nl clicksor.com
crazyprotocol.com compliance-julia.top
cookingmeat.ru
cyse.tk
d-black.bz d-black.bz
daisye.top dafatiri.com
data-ox.com decorationspcs.com
detsky-nabytek.info demon-tweeks.com
dimaka.info design-lands.ru
djonwatch.ru diamond-necklace.info
donna7753191.ru dikx.gdn
dorratex.tn docstoc.com
download-wallpaper.net doctissimo.fr
eachdayisagift.review domainsatcost.com
eandsgallery.com downloadmefiranaratb1972.xpg.com.br
edudocs.net dwomlink.info
elektrischezigarette2.devhub.com efnor-ac.com
elektrozigaretten2.yn.lt ekspertmed.com
en.altezza.travel erotiktreff24.info
ero-advertising.com escortplius.com
eurosamodelki.ru essay-writing.work
expdom.com etotupo.ru
f5mtrack.com europeanwatches.ru
falcoware.com extractorandburner.com
flashlarevista.com eyes-on-you.ga
fm-upgrade.ru ez8motelseaworldsandiego.com
forodvd.com ezigarettekaufen.myblog.de
foto-basa.com ezigarettekaufen2.mpbloggar.se
frcls.fr faracontrol.ir
freetangodownload.com fashion-stickers.ru
futbolkisales.ru feorina.ru
gheus.altervista.org fetishinside.com
gifspics.com filesvine.com
glass-msk.ru fishtauto.ru
glogow.pl flowersbazar.com
go2jump.org freewhatsappload.com
guiadeserraazul.com frvo.alptandem.ru
hasshe.com fugarif.ga
hdhc.ru g.starmoe.xyz
hdpixion.com gallery.rennlist.com
helpmymacfaster.trade galleryawesome.com
home-task.com gameonasia.com
horoshieokna.com gazporno.com
hothot.ru geckoandfly.com
hotxnights.info getoutofdebtfree.org
houseofgaga.ru gg-arena.ru
hyipmanager.in goldadpremium.com
iboss.com goldenggames.com
id-forex.com gunsvicceadadebt.tk
idc.com.ua handsandlegs.ru
ilovevitaly.ru hasfun.com
ilte.info hchha.com
hdwalls.xyz
hmmm.cz
hobild.net
hully.altervista.org
ideawheel.com
imediadesk.com
imgata.com
indo-export.ru
insider.pro
int.search.mywebsearch.com int.search.mywebsearch.com
investzalog.ru iqbazar.ru
iskalko.ru iqs.biz.ua
jacago.com isitpaleo.info
journeydownthescale.info isitwp.com
kartiniresto.com istock-mebel.ru
koptims.tiu.ru ivoiretechnocom.ci
kosmetyki.tm.pl ix20.ru
kutikomi.net izoll.ru
live-cam6.info jackwolfskinoutlet.online
livefixer.com javidol.site
loftdigital.eu jolic2.com
louboutinbooties.xyz jwcialislrt.com
lovasszovetseg.hu karusel-market.ru
love-baby.cz kazrent.com
lovi-moment.com.ua kino-filmi.com
ltvperf.com kinopolet.net
macrotek.ru knogg.net
mainhp.net kvartir-remont.biz
managerpak204.weebly.com laapp.com
manifestyourmillion.com letslowbefast.today
manualterap.roleforum.ru lion.misba.us
maranbrinfo.com.br lion.playtap.us
massage-info.nl lubetube.com
mcnamaratech.com m292.info
medanestesia.ru mamasuper.prom.ua
mindbox.co.za meddesk.ru
mister-shop.com medi-fitt.hu
mompussy.net mega-polis.biz.ua
monthlywinners.com megahdporno.net
megaindex.ru
mesbuta.info
mielec.pl
militarysale.pro
misslike.ru
mlvc4zzw.space
moesen-ficken.com
moncleroutletonline.xyz
moscow-region.ru moscow-region.ru
moviezbonkerssk.cf moxo.com
mp3ringtone.info musicktab.com
myiptest.com myftpupload.com
myliveblog.ru myghillie.info
mypornfree.ru
myseoconsultant.com
nailsimg.com nailsimg.com
namecrumilchlet.tk nardulan.com
naturtreenspicerx.pw nastroyke.net
netfacet.net netpics.org
nonameread45.live new-apps.ru
notfastfood.ru nikhilbahl.com
november-lax.com njpalletremoval.com
o.light.d0t.ru nl.netlog.com
olvanto.ru ntic.fr
ooo-gotovie.ru obiavo.com
obuv-kupit.ru
offf.info
okuos.com
online-hit.info
ostrovtaxi.ru ostrovtaxi.ru
oyster-green.com otbelivanie-zubov.com
photochki.com ovirus.ru
photokitchendesign.com physfunc.ru
picsforkeywordsuggestion.com pictures-and-images.net
pinwallpaper.xyz piluli.info
piski.top pk-pomosch.ru
plastweb.ru pons-presse.com
pornoklad.net prefersurvey.net
potoideas.us pregnant.guru
pr-ten.de prostitutki-rostova.org
princevc.com ranksignals.com
privatbank46.ru ranksonic.com
profitkode.com ranksonic.org
r-control.ru rebrand.ly
rada.ru rff-cfal.info
rcb101.ru rique.host.sk
redhotfreebies.co.uk rrutw.com
royalcar-ufa.ru scanner-jess.top
ruscoininvest.company scansafe.net
russintv.fr scanspyware.net
sanjosestartups.com sdi-pme.com
scanner-george.top seobility.net
scanner-jane.top seoboxes.com
scopich.com serw.clicksor.com
searchkut.com sexflirtbook.com
seinterface.com sexvideo-sex.com
semaltmedia.com shama-rc.net
seo-smm.kz signal03.ru
seo-traffic-ranking.info site-auditor.online
seokicks.de sitiz.club
serialsway.ucoz.ru smartpet.ru
sexiporno.net snegozaderzhatel.ru
sexkontakteao.info snworks.com
sexkvartal.com sobecjvuwa.com.ru
sexsaoy.com soc-proof.su
shisha-swag.de soserfis.com
shtaketniki.kz spravka-medosmotr.ru
skutecznetabletkinaporostwlosow.pl spy-app.info
snts.shell-pmr.ru
spamnuker.com
spy-sts.com spy-sts.com
spywareit.com stard.shop
sr-rekneskap.no sukarame.net
staticfs.host supermama.top
stats-public.grammarly.io tanieaukcje.com.pl
sudexpert66.ru
symbaloo.com
takeflyte.com
tattooha.com tattooha.com
techart24.com textads.men
teesdaleflyballclub.co.uk timeallnews.ru
thebestweightlosspills.ovh tizanidineandcipro.blogspot.com
thepantonpractice.co.uk tizanidinedosageforsleep.blogspot.com
titan-ads.life tizanidinedruginteractions.blogspot.com
tizanidineforopiatewithdrawal.blogspot.com tizanidinevscyclobenzaprine.blogspot.com
tizanidinezanaflex.blogspot.com topbestgames.com
tkanorganizma.ru totu.info
toproadrunner5.info toy-shop.top
traffictrade.life track112.site
trkur.com tracksurf.daooda.com
trustedmaccleaner.com traffpartners.com
tubo360.com traxdom.ru
twittrading.com tversvet.ru
tytoona.com typer.one
uggsale.online ukkala.xyz
underthesite.com ultramart.biz
uptimebot.net unrealcommander.biz
urlcut.ru vbtracker.net
ussearche.cf vchulkah.net
valmetrundan.se veopornogratis.xxx
vbikse.com via-energy-acquistare.com
viddyoze.com viagra.pp.ua
videogamesecrets.com vintontech.info
visa-china.ru vse-pesni.com
vita.com.hr website-speed-checker.site
vksex.ru wetgames.ru
vladimir.zrus.org wholesalejerseys-cheapest.com
vltai.com wholesalejerseyscheapjerseys.us.com
vodaodessa.com wildworld.site
wanker.us womensplay.net
warning.or.kr wordpresscore.com
web.cvut.cz world-mmo.com
webpromotion.ae ww2awards.info
webshoppermac.com www.arenda-yeisk.ru
westen-z.life www.kabbalah-red-bracelets.com
where-toget.com xlolitka.com
whosonmyserver.com xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
worldhistory.biz xn--80aanaardaperhcem4a6i.com
worldlovers.ru xn--h1ahbi.com.ua
www.timer4web.com yourmovies.pl
xn----ctbbcjd3dbsehgi.xn--p1ai zeroredirect12.com
xn--d1acah0c.xn--p1ai zocaparj.kz
xn--h1aakne2ba.xn--p1ai zytpirwai.net
xrus.org
xtraffic.plus
xz618.com
yoursite.com
zaidia.xhost.ro
zastroyka.org
zeroredirect5.com

9
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -8,7 +8,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.57 0; 10.1.1.80 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.07.3220 ### Version: V4.2022.07.3221
### Updated: Sun Jul 10 22:01:15 UTC 2022 ### Updated: Mon Jul 11 10:03:39 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.07.3220 ### Version: V4.2022.07.3221
### Updated: Sun Jul 10 22:01:15 UTC 2022 ### Updated: Mon Jul 11 10:03:39 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##