vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2907 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-09 10:09:32 +00:00
parent b9791bd791
commit 7f291f67c4
47 changed files with 84300 additions and 84310 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2906 #### Version: V4.2022.02.2907
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2906 #### Version: V4.2022.02.2907
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2906 #### Version: V4.2022.02.2907
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10402
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2906 #### Version: V4.2022.02.2907
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10406
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10406
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10406
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10406
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 1; 10.1.0.53 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 0; 10.1.0.53 0;
127.0.0.1 0; 127.0.0.1 0;

10406
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 1; 10.1.0.53 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 0; 10.1.0.53 0;
127.0.0.1 0; 127.0.0.1 0;

10406
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 1; 10.1.0.53 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 0; 10.1.0.53 0;
127.0.0.1 0; 127.0.0.1 0;

10406
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2906 2907

10406
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10406
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.66.244.148 0; 20.187.3.24 0;

10406
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 1; 10.1.0.53 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 0; 10.1.0.53 0;
127.0.0.1 0; 127.0.0.1 0;

10406
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 1; 10.1.0.53 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 0; 10.1.0.53 0;
127.0.0.1 0; 127.0.0.1 0;

10406
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 1; 10.1.0.53 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 0; 10.1.0.53 0;
127.0.0.1 0; 127.0.0.1 0;

10406
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.66.244.148 0; 20.187.3.24 0;

10406
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.66.244.148 0; 20.187.3.24 0;

10406
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 1; 10.1.0.53 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
01h4x.com Black Hole
Alexibot Collector
ChinaClaw DIIbot
Lipperhey Digincore
NetZIP DittoSpyder
Snapbot G-i-g-a-b-o-t
Spammen JustView
WASALive-Bot Spanner
WWWOFFLE VidibleScraper
heritrix Wprecon

310
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404enemy 80legs
AIBOT Aboundex
Acunetix Alexibot
Anarchie AlphaBot
Aspiegel Anarchy
Ankit
Anthill
Asterias
Atomseobot Atomseobot
Attach
AwarioRssBot AwarioRssBot
BDCbot AwarioSmartBot
BackStreet BBBike
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler BotALot
Badass
Bandit
BatchFTP
BlowFish
Boardreader
Bolt
Brandprotect Brandprotect
Buck Brandwatch
Buddy BunnySlippers
BuiltBotTough BuzzSumo
Bullseye Calculon
CazoodleBot CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect CensysInspect
CheTeam
CheeseBot CheeseBot
ChinaClaw
Citoid
Claritybot
Cloud mapping Cloud mapping
Cocolyzebot Collector
Copyscape CopyRightCheck
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
DSearch Crescent
DIIbot
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli Devil
Digincore DigitalPebble
Dirbuster
Disco Disco
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DnyzBot
DomainCrawler DomainAppender
DomainSigmaCrawler
Download Wonder Download Wonder
EMail Siphon ECCP/1.0
EroCrawler
Evil
Express WebPictures
ExtractorPro ExtractorPro
Extreme Picture Finder FHscan
Ezooms
FemtosearchBot
Fimap Fimap
Firefox/7.0 Firefox/7.0
Foobot
Freeuploader
FrontPage FrontPage
Fuzz
FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot
Genieo Genieo
GermCrawler GetRight
GetWeb Getintent
Gigabot Go!Zilla
GoZilla Go-Ahead-Got-It
Gotit GrabNet
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO
HMView HMView
HTTP::Lite HTMLparser
HTTrack
Haansoft Haansoft
Heritrix HaosouSpider
Hloader Harvest
Humanlinks HonoluluBot
IDBTE4M
IDBot IDBot
IRLbot
Iblog
Id-search Id-search
IlseBot IlseBot
Image Sucker
Indy Library Indy Library
InfoNaviRobot
InfoTekies InfoTekies
Internet Ninja InterGET
InternetSeer
Iskanie
IstellaBot IstellaBot
JamesBOT Jetty
Jbrofuzz Joomla
JetCar Jyxobot
Jorgee Kenjin Spider
JustView
Kozmosbot Kozmosbot
LNSpiderguy LWP::Simple
Lanshanbot LeechFTP
LeechGet LibWeb
LexiBot Libwhisker
Lftp
LieBaoFast LieBaoFast
Linkbot Likse
LinkWalker
Linkdexbot
LinkextractorPro
LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Litemage_walker Litemage_walker
Lmspider Lmspider
Ltx71 MFC_Tear_Sample
MIIxpc
MJ12bot MJ12bot
MSFrontPage MSFrontPage
MSIECrawler MTRobot
MarkWatch Mag-Net
Mass Downloader Mail.RU_Bot
MarkMonitor
Masscan Masscan
MauiBot MauiBot
Mediatoolkitbot Microsoft URL Control
MegaIndex.ru
Minefield
Mojeek
Mojolicious Mojolicious
Mozlila MolokaiBot
Morfeus Fucking Scanner
Mr.4x3 Mr.4x3
Msrabot
Musobot
Nameprotect Nameprotect
Navroad Navroad
NearSite Needle
Nessus Net Vampire
NetAnts NetAnts
NetLyzer NetLyzer
Nettrack NetZIP
NextGenSearchBot
Niki-bot
Nikto
Nimbostratus
Ninja Ninja
Nmap Nutch
Not
Nuclei
Offline Explorer
OnCrawl OnCrawl
OpenLinkProfiler
Openfind Openfind
PageAnalyzer OrangeBot
OrangeSpider
OutclicksBot
OutfoxBot
PECL::HTTP
POE-Component-Client-HTTP
PageGrabber PageGrabber
PageThing.com Pandalytics
PeoplePal Panscient
Pavuk
Petalbot Petalbot
Pi-Monster Pi-Monster
Picsearch
PictureFinder PictureFinder
Pimonster Pixray
PleaseCrawl ProWebWalker
ProPowerBot Pu_iN
PyCurl QueryN Metasearch
Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot
RankFlex RankFlex
RankingBot RankurBot
RankingBot2
Re-re Re-re
ReGet
RealDownload
Reaper Reaper
Ripper RebelMouse
Recorder
SBIder
SEOkicks SEOkicks
SISTRIX SEOprofiler
ScanAlert SEOstats
ScoutJet
Searchestate
SemanticJuice
SemrushBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot Shodan
Site Sucker Site Sucker
SiteSucker SiteCheckerBotCrawler
Siteimprove Siteimprove
SlySearch
Snake Snake
Sogou web spider Snapbot
Sosospider SpaceBison
Sottopop SpankBot
Spbot Spinn3r
SputnikBot SputnikBot
Sqlworm SuperHTTP
Stripper Suzuran
Surfbot Swiftbot
Szukacz Szukacz
T0PHackTeam TeleportPro
T8Abot
Telesoft Telesoft
Telesphoreo
The Intraformant
TightTwatBot
Titan
Toata Toata
Trendiction Toweyabot
True_Robot
Turingos
TurnitinBot TurnitinBot
Twice TwengaBot
Typhoeus
URLy Warning
URLy.Warning URLy.Warning
VB Project UnisterBot
VidibleScraper Vacuum
VeriCiteCrawler
Voil Voil
Voltron
WASALive-Bot
WBSearchBot
WISENutbot
WPScan WPScan
WWW::Mechanize WWW-Collector-E
Web Sauger WWW-Mechanize
Web Fetch
WebCollage WebCollage
WebCopier
WebEnhancer WebEnhancer
WebFetch
WebFuck WebFuck
WebLeacher WebImageCollector
WebPix WebPix
WebZIP WebReaper
WebSauger
WebSucker
Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag
WebsiteExtractor WebsiteExtractor
WebsiteQuester WebsiteQuester
Whack Whacker
Whatweb WinHTTrack
Widow Wprecon
WiseGuys Robot Xaldon WebSpider
Wotbox Xenu
Zade
Zauba Zauba
Zitebot Zermelo
ZmEu ZmEu
adscanner adscanner
archive.org_bot
arquivo-web-crawler
arquivo.pt
backlink-check backlink-check
cah.io.community cah.io.community
com.plumanalytics dataforseo.com
crawler.feedback demandbase-bot
domainsproject.org domainsproject.org
eCatch facebookscraper
gopher
instabid instabid
internetVista monitor ips-agent
isitwp.com iubenda-radar
lwp-trivial lwp-request
meanpathbot
mediawords mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler
oBot oBot
plumanalytics plumanalytics
polaris version
probe-image-size probe-image-size
ripz
satoristudio.net satoristudio.net
seobility scalaj-http
scan.lol
seocompany.store seocompany.store
seoscanners seostar
sexsearcher serpstatbot
siteripz sitechecker.pro
sogouspider sogouspider
sp_auditbot
sysscan
tAkeOut
trendiction.de
webgains-bot webgains-bot
webpros.com
x22Mozilla x22Mozilla
zauba.io

306
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 01h4x.com
80legs 404enemy
ADmantX
ALittle Client ALittle Client
ASPSeek ASPSeek
Abonti Abonti
Aboundexbot
Acunetix Acunetix
AiHitBot AfD-Verbotsverfahren
Aipbot Aipbot
Ankit Alexibot
Alligator
Anarchie
Anthill Anthill
Apexoo Aspiegel
Asterias
AwarioRssBot AwarioRssBot
AwarioSmartBot BBBike
BLEXBot BDFetch
BackDoorBot BackDoorBot
BackStreet Backlink-Ceck
Bandit BacklinkCrawler
Barkrowler BetaBot
Bitacle
Black Hole Black Hole
BlowFish BlowFish
Bolt Boardreader
BotALot
Brandwatch Brandwatch
BuiltWith BuiltWith
BunnySlippers
CATExplorador CATExplorador
CODE87
CSHttp
Calculon
CazoodleBot CazoodleBot
Cegbfeieh CheeseBot
CensysInspect CherryPicker
Citoid ChinaClaw
CopyRightCheck Chlooe
Claritybot
Cocolyzebot
Collector
Copier
Copyscape Copyscape
Cosmos
Craftbot Craftbot
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
CrunchBot CrunchBot
DBLBot DIIbot
DSearch DTS Agent
DatabaseDriverMysqli DataCha0s
Deusu Deusu
DigitalPebble Devil
Dirbuster Disco
DnyzBot Dispatch
DomainAppender DomainAppender
DomainCrawler DomainStatsBot
Domains Project Dotbot
Download Wonder Download Wonder
Dragonfly
ECCP/1.0
EMail Siphon
EMail Wolf EMail Wolf
EirGrabber
EroCrawler EroCrawler
Exabot Evil
Express WebPictures
ExtLinksBot ExtLinksBot
EyeNetIE
Ezooms Ezooms
FDM Fimap
Firefox/7.0 Flunky
Foobot
Freeuploader Freeuploader
FrontPage FrontPage
Fuzz Fyrebot
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW GalaxyBot
Genieo
Gigabot Gigabot
Go!Zilla Go!Zilla
Go-Ahead-Got-It Gotit
Grabber
Grafula Grafula
GrapeFX HEADMasterSEO
HMView
HTMLparser
HTTP::Lite HTTP::Lite
Humanlinks HTTrack
IDBot Haansoft
Havij
Iblog
IndeedBot
Indy Library Indy Library
InfoTekies InfoTekies
InterGET
InternetSeer InternetSeer
JamesBOT Iskanie
Jbrofuzz
JennyBot JennyBot
JetCar Jetty
Jyxobot JikeSpider
Kinza Jorgee
JustView
Kenjin Spider
Keyword Density
LNSpiderguy LNSpiderguy
Larbin Leap
LeechGet LeechGet
Lftp Libwhisker
LieBaoFast
Lightspeedsystems Lightspeedsystems
Likse
LinkScan LinkScan
LinkpadBot LinkWalker
LinksManager Linkdexbot
Litemage_walker LinkextractorPro
Lmspider LinqiaMetadataDownloaderBot
Ltx71 LinqiaRSSBot
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool
MIIxpc
MJ12bot MJ12bot
Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic-SEO Majestic-SEO
MarkMonitor
Mass Downloader
Masscan Masscan
Mata Hari
MauiBot MauiBot
Meanpathbot Mb2345Browser
MegaIndex.ru MeanPath Bot
MicroMessenger Metauri
Microsoft Data Access Minefield
Mister PiX
Moblie Safari Moblie Safari
Mojolicious Mojolicious
MolokaiBot Mr.4x3
Mozlila Msrabot
NICErsPRO NPbot
Name Intelligence Name Intelligence
NearSite
Needle
Nessus Nessus
NetMechanic NetMechanic
NetZIP
Netcraft
Nettrack Nettrack
Nimbostratus Nibbler
Nuclei Nmap
Not
Nutch
Octopus Octopus
Offline Explorer Offline Explorer
OpenLinkProfiler OpenLinkProfiler
OpenVAS Openfind
OrangeBot
OrangeSpider
OutfoxBot OutfoxBot
PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageScorer
Panscient Panscient
PeoplePal Picsearch
Pi-Monster Piepmatz
PictureFinder
PleaseCrawl
ProWebWalker
Probethenet Probethenet
Pu_iN PxBroker
Pump PyCurl
QueryN Metasearch QueryN Metasearch
RSSingBot RSSingBot
Rankivabot RankActiveLinkBot
ReGet RankingBot2
RealDownload Re-re
RebelMouse RebelMouse
Recorder RedesScrapy
RepoMonkey RepoMonkey
Ripper Ripper
Rogerbot
SBIder
SEOkicks-Robot SEOkicks-Robot
SEOprofiler SEOlyticsCrawler
ScanAlert Scanbot
ScreenerBot Scrapy
Searchestate ScrepyBot
SearchmetricsBot SearchmetricsBot
Seekport
SemanticJuice SemanticJuice
Semrush Semrush
SentiBot Shodan
Siphon Siphon
Site Sucker Site Sucker
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger Sitebeam
Siteimprove Siteimprove
SlySearch Snapbot
Snoopy Sogou web spider
SocialRankIOBot Spammen
Sosospider
SpankBot SpankBot
Spanner
SputnikBot
Sqworm
Steeler
Sucker Sucker
SuperBot SuperBot
SuperHTTP SurveyBot
Suzuran Suzuran
Szukacz
T0PHackTeam T0PHackTeam
T8Abot Telesoft
TeleportPro
Telesphoreo Telesphoreo
Telesphorep
The Intraformant The Intraformant
Thumbor Toata
TightTwatBot
Titan
Toweyabot
Tracemyfile
Trendiction
Trendictionbot Trendictionbot
Turingos
Turnitin
TwengaBot
Typhoeus Typhoeus
V-BOT URLy Warning
UnisterBot
Upflow
VCI VCI
Vagabondo Virusdie
VidibleScraper Voil
WEBDAV
WISENutbot WISENutbot
WPScan WWW-Collector-E
WWW::Mechanize Wallpapers/3.0
Wallpapers WallpapersHD
WeSEE Web Fetch
WebBandit Web Sauger
WebAuto
WebCollage WebCollage
WebCopier WebCopier
WebFetch WebLeacher
WebImageCollector WebSucker
WebPix Webalta
WebWhacker Webshag
WebZIP
WebmasterWorldForumBot
Website Quester Website Quester
Webster WebsiteExtractor
Whack WebsiteQuester
Whatweb Who.is Bot
WinHTTrack Wonderbot
Xenu Woobot
Zermelo Wotbox
ZoominfoBot Xaldon WebSpider
YoudaoBot
Zade
Zeus
ZoomBot
adscanner adscanner
archive.org_bot
autoemailspider autoemailspider
backlink-check clark-crawler
check1.exe crawl.sogou.com
com.plumanalytics crawler.feedback
dataforseo.com crawler4j
demandbase-bot eCatch
heritrix instabid
ips-agent
lwp-request
magpie-crawler magpie-crawler
muhstik-scan mediawords
netEstate NE Crawler oBot
pcBrowser
probe-image-size probe-image-size
ripz ripz
s1z.ru
satoristudio.net satoristudio.net
seoscanners serpstatbot
sexsearcher sexsearcher
siteripz
sogouspider sogouspider
sysscan sp_auditbot
voyagerx.com voyagerx.com
webpros.com webgains-bot
x09Mozilla webmeup-crawler
zgrab x22Mozilla
xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
abusora.com cartierreplica.top
bannerconnect.net christianlouboutinoutlet.win
bard-real.com.ua cienum.fr
eduserver.net heygidday.biz
jav-fetish.com hobbyhuren24.net
mini.7zap.com lolnex.us
piuminiita.com pufip.com
search.xtconnect.com reklama-i-rabota.ru
tizanidine4mgstreetvalue.blogspot.com scrnet.biz.ua
wejdz-tu.pl website-datenbank.de

912
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00it.com 00webcams.com
12u.info 0lovespells0.blogspot.com
101raccoon.ru
123locker.com
14b.info 14b.info
1kinobig.ru 404.mn
1stat.ru 66cpwgln.space
2pxg8bcf.top 7a2rnuey1tw9ar.ru
2rich4bitches.com 7makemoneyonline.com
40cg.com 7xc4n.com
4kepics.com a.pr-cy.ru
8xv8.com
a14download.com
absugars.com
abtasty.com
acc.eu.org
adclickservice.com adclickservice.com
adinterax.com adclickthru.net
adman.se adconscious.com
adserver-e7.com adrunnr.com
advertisingtag.net adtech.fr
agecheckadult.com advokateg.ru
alevinefesleri.com afftrack001.com
alfapro.ru aihelen.net
all-streaming-media.com aimayangzhi.com
all4invest.ru air-edem.ru
allblogroll.com akvamaster.dp.ua
allwomen.info amazon-seo-service.com
aloofly.com ameblo.top
amoi.tn amoi.tn
ample-awards-today.us amung.us
anonymizeme.pro angigreene.com
apibrowsesmartne-a.akamaihd.net animal36.com
app-ready.xyz anmysite.com
appiq.mobi apilinkswiftco-a.akamaihd.net
arkkivoltti.net apispringsmartne-a.akamaihd.net
ascat.porn apple.com-cleaner.systems
asiengirls.net appmsr.org
aversis.net atmovs.com
ayerbo.xhost.ro auto-moto-elektronika.cz
azlex.uz autodo.info
babosas.com aviav.ru
babyfactory.fr ayodhya.co
azadnegar.com
babespcs.com
babieca.com
bala.getenjoyment.net
batanga.net batanga.net
beachdriveblog.com beachdriveblog.com
beachpics.com beachfix.co
beachtoday.ru berlininsl.com
beremenyashka.com best-way.men
berrymall.ru bestcalovebracelet.cn
best-coupon-offer.com bestserials.com
best-games.today bestsexyblog.com
bestfortraders.com bharatdefencekavach.com
bestpriceninja.com biglistofwebsites.com
betune.onlinewebshop.net bitcoin-ua.top
betwinservice.com blackbabesporn.com
bin-brokers.com blog.axant.it
bio-market.kz bloglag.com
blackle.com blogos.kz
blacktwhite.com blogs.rediff.com
bonkers.name bnt-team.com
brainxs.us boazpower.com
bristolhotel.com.ua bookmark4you.com
bronzeaid-a.akamaihd.net bronzeaid-a.akamaihd.net
browserprotecter.com businesxxl.com
brus.city buypuppies.ca
buchananshardware.com buzz-porno.info
buyclomidonlaine.com buzzurl.jp
byme.se call-of-duty.info
calcularpagerank.com
callmd5map.com callmd5map.com
callstevens.com camsex.xxx-cam.webcam
candelluxsklep.pl candycrushshop.com
canrole.com canrioloadm.gq
canvas.gloverid.site
carabela.com.do carabela.com.do
cardsdumps.com cartierlove.xyz
cartierjusteunclou.xyz cdncash.com
catalogs-parts.com cdncash.net
cellfun.mobi cfacarrosserie74.com
centrdebut.ru changde.58.com
cete.ru cheapjerseysa.com
cezartabac.ro cheatcode-lita12.rhcloud.com
chcu.net chimiver.info
cherrypointplace.ca chocolateslim-en-espana.com
ciekawinki.pl chocoslim.pro
cienum.fr ci.ua
cityadspix.com cigarpass.com
cl.s7.exct.net citetick.com
clickbanksites.info clean-virus-mac.com
clickintext.net clickhype.com
clmforexeu.com collegerentals.com
club-lukojl.ru community.allhiphop.com
cntravelre.com compliance-barak.xyz
co.lumb.co concordexoticrentals.com
colehaanoutlet.store construmac.com.mx
com-onlinesupport.site cookie-law-enforcement-ee.xyz
compliance-irvin.xyz copenergo.ru
compliance-jane.top cosmeticswomens-womensports.rhcloud.com
compliance-mary.top
confib.ifmo.ru
contentlook.co
coolwallpapers-hd.com
costablue.xyz costablue.xyz
cougfan.info country-chic.ru
credit-online.ws covadhosting.biz
crystalslot.com crazyboost.pro
customchocolate.business-for-home.com customerguru.in
cutt.us cypernhuset.se
d2jsp.org dame-ns.kz
darkages.info daptravel.com
dcj-nn.ru
deda-moroza-zakaz.ru
defenderxtactical.com defenderxtactical.com
demenageur.com
dentalpearls.com.au dentalpearls.com.au
dentfidemountpreach.tk design-lands.ru
detskie-zabavi.ru designdevise.com
digitalassetmanagement.site detskie-konstruktory.ru
dichvuvesinhhanoi.com
diffbot.com
dignitasdata.se dignitasdata.se
dimaka.info discovertreasurenow.com
discountbarbour.online distver.ru
doeco.ru djihispano.com
dokumentalkino.net djstools.com
dolohen.com domaineaneblanc.com
domaincdn.xyz dpihatinh.gov.vn
domcran.net drupa.com
dosugrostov.site dvr.biz.ua
download-walpaperhd.blogspot.com dvrlists.com
downloadmefiranaratb1972.xpg.com.br
dprtb.com
e-biznes.info
e2click.com e2click.com
eager-nash.188-93-233-196.plesk.page earthmagic.info
ebooknovel.club easyukraine.com
educatemetv.com ecookna.com.ua
elektrischezigarette2.devhub.com efkt.jp
elektrozigaretten1.webs.com electricwheelchairsarea.com
elexies.info elektrozigarette1.dreamwidth.org
elidelcream.weebly.com englishdictionaryfree.com
elkacentr.ru
ereko.ru
erotag.com erotag.com
errorfixing.space estelight.ru
escortplius.com
eshop.md
essay-writing.work
essaypro.com
eu-cookie-law.info
eugenevaultstorage.com eugenevaultstorage.com
evidencecleanergold.com europeanwatches.ru
evogarage.com ezigarettenshop2.myblog.de
exchanges-bet.com fable.in.ua
ezigarettenkaufen2.dreamwidth.org faithe.top
f1nder.org
f5mtrack.com
face.hostingx.eu
facialporntube.com
factorynightclub.com
failingmarriege.blogspot.com
family1st.ca
fanrto.com fanrto.com
fantasticpixcool.com fapgon.com
fashion.stellaconstance.co fickblock18.com
fastcrawl.com filesmonster.porn
favorcosmetics.com filesvine.com
favornews.com find1friend.com
femdom.twiclub.in findanysex.com
fickenbumsen.net findthe.pet
fil.ru
filmbokep69.com
filmi-onlain.info
financeloan.us
findacheaplawyers.com
findpik.com
finstroy.net finstroy.net
forensicpsychiatry.ru flipper.top
forex-procto.ru flowersbazar.com
forms-mtm.ru footbalive.org
formulaf1results.blogspot.com forminecrafters.ru
forums.d2jsp.org fotzen-ficken.com
franch.info franecki.net
free-fbook-traffic.com free-gluten.ru
free-share-buttons.blogspot.com free-share-buttons.blogspot.com
free-share-buttons.top free-share-buttons.top
free-traffic.xyz free-social-buttons.xyz
freejabs.com freevpn.space
freelinkbuilding.website.tk freshdz.com
funnypica.com front.ru
galleryawesome.com fullfileaccess.com
gambarkataku.co fxtips.ru
gamedayhouse.com gambarkatabaru.com
gameplexcity.com game-mmorpg.net
gazporno.com gameprimary.com
ggiaro.com gate5.co.za
glass-msk.ru generic-pills-online.com
gne8.com getresponse.com
ghernnqr.skyrock.com
gilsonchiro.xyz
glall.ru
globetrotting-culture.ru
goatse.ru goatse.ru
golaya.pw
golden-catalog.pro golden-catalog.pro
golden-praga.ru goodly.pro
greatdealshop.com goodnightjournal.com
green-tea.tv gotomontenegro.net
hd-filmy.net gpms.org.my
hdhc.ru growboxbl.ru
healmytrauma.info habermetre.com
heatpower.ru handsandlegs.ru
herokuapp.com hatedriveapart.com
heroz.fr haveinc.xyz
himgaws.pw hchha.com
hoverboardforsaledirect.com helvetia.com.ua
hoztorg-opt.ru hidemyass.com
husky-shop.cz highspeed5.net
ictizanidinehcl4mg.blogspot.com historichometeam.com
igadgetsworld.com hitmuzik.ru
ikritikimou.gr homedecoguide.info
imfamous.info hostcritique.com
imicrovision.com hosting-tracker.com
imitex-plus.ru hotcore.info
imk.com.ua hotelcrocenzi.sm
incest-ru.com houdom.net
investyb.com hoverboard360.at
https-legalrc.biz
iloveitaly.ru
ilte.info
importchinacoach-teach.com
in-tandem.co
incomekey.net
integritylandscapeservices.com
interfucks.net
intermesh.net
iplogger.org
iplusbit.blogspot.co.za iplusbit.blogspot.co.za
iqoption.com ipostroika.ru
iqs.biz.ua iradiology.ru
irkutsk.zrus.org isitpaleo.info
istanbulit.com istmira.ru
istizanidineanopiate.blogspot.com itbc.kiev.ua
it-max.com.ua itrevolution.cf
izismile.com iwanttodeliver.com
jackpotchances.com jav-way.site
jeremyeaton.co javcoast.com
jerseywholesalechinabiz.com je7.us
kamalsinha.com johannesburgsingles.co.za
karusel-market.ru johnrobertsoninc.com
keywordsdoctor.com junketjuice.blogspot.com
justdating.online
kellyonline.xyz
kerch.site
keyhantercume.com
keywordspay.com keywordspay.com
kinopolet.net kinohit1.ru
kinozapas.org kladrus.ru
kiwe-analytics.com klosetkitten.com
klumba55.ru kolotiloff.ru
kmd-pto.ru krafte.ru
kosmetyki.tm.pl kupit-adenu.ru
kunstaktien.info kursy-ege.ru
kw21.org kw21.org
law-check-eight.xyz landscaping.center
lechtaczka.net letslowbefast.site
leftofcentrist.com limtu.ifmo.ru
legalrc.biz linkbolic.com
legionalpha.com linkrr.com
lesbian.xyz lion.misba.us
letolove.ru lisque.batcave.net
letsrepair.in live-cam6.info
life.biz.ua live-sexcam.tk
lineshops.biz lmrauction.com
linkdebrideur.xyz locustdesign.co
linksharingt.com lyrster.com
lirunet.ru m.b00kmarks.com
littlesexdolls.com m5home.ru
livejasmin.com mabdoola.blogspot.com
lonely-mature.com mac-shield.com
lookingglassemb.com mackeeper-land-672695126.us-east-1.elb.amazonaws.com
macnewtech.com magiadinamibia.blogspot.com
magento-crew.net mamasuper.prom.ua
mailemedicinals.com map028.com
manyresultshub-a.akamaihd.net marketingvici.com
me-ke.com marmotstore.online
mebel-ekb.com mebelcomplekt.ru
medicinacom.ru medicinacom.ru
medicineseasybuy.com merryhouse.co.uk
medicovi.com metarip.ru
meetlocalchicks.com mielec.pl
megahdporno.net mirmedinfo.ru
meteostate.com mmstat.com
microsoftportal.net mobifunapp.weebly.com
minecraft-neo.ru mobplayer.net
mirtorrent.net modnie-futbolki.net
misandesign.se moneymaster.ru
misslike.ru moneytop.ru
mixed-wrestling.ru motherboard.vice.com
mobile.ok.ru mrsdalloways.com
mobstarr.com mrvideospornogratis.xxx
moncleronline.xyz mttwtrack.com
monitorwebsitespeed.com mug-na-chas-moscow.ru
montazhnic.ru mwtpludn.review
morepoweronmac.trade mybackgroundlandscape.blogspot.com
morocco-nomad-excursions.com
moscow-region.ru
mountainstream.ms
mrpornogratis.xxx
musicktab.com
mybinaryoptionsrobot.com mybinaryoptionsrobot.com
mydearest.co myhydros.org
myfreemp3.eu myperiod.club
mykings.pw mysexpics.ru
mymercy.info navalwiki.info
mzdish.site netallergy.ru
nailsimg.com newsperuse.com
narco24.me nhl17coinsforps3.gratisblog.biz
naturtreenspicerx.pw nina.az
new-apps.ru nippon-bearings.ru
news-readers.ru nosecret.com.ua
nextrent-crimea.ru notfastfood.ru
ningessaybe.me nudepatch.net
nodup.ru nullrefer.com
nonameread45.live o333o.com
normalegal.ru obiavo.ru
norththeface.store oktube.ru
nsatc.net onblastblog.online
obiavo.uz onlinedomains.ru
oecnhs.info optitrade24.com
ofanda.com orion-code-access.net
offer.wpsecurity.website ossmalta.com
onetravelguides.com ostroike.org
online.ktc45.ru ostrovtaxi.ru
onlinemeetingnow.com
onlinetvseries.me
ontargetseo.us
oops-cinema.ru
openmultipleurl.com
oyster-green.com oyster-green.com
pacificair.com pacificair.com
painting-planet.com pagesense.com
paparazzistudios.com.au palocco.it
paradontozanet.ru partners-ship.pro
partybunny.ru partybunny.ru
pastaleads.com paydayloanslocal.com
pcimforum.com pekori.to
pic2fly.com perfectplanned.com
picmoonco.pw pestomou.info
pics-group.com phelissota.xyz
picsforkeywordsuggestion.com picsearch.com
picturesify.com pisanieprac.info
pkr1hand.com pistonclasico.com
platesauto.com plastgranar.nu
platezhka.net playboyfiles.xblog.in
pooleroadmedicalcentre.co.uk playtap.us
popup-jdh.xyz pogodnyyeavarii.gq
pornmania.pl pons-presse.com
popup.matchmaker.com
pornophoto.xyz pornophoto.xyz
pornoreino.com pornotubs.com
pornoslive.net powc.r.ca.d.sendibm2.com
pornoted.com pozvonim.com
pro-okis.ru praisong.net
pronekut.com pravoholding.ru
pufip.com privatamateure.com
pulse33.ru privatbank46.ru
purplesphere.in prizeestates.cricket
quelle.ru prod2016.com
quickloanbank.com prostitutki-astana.org
rankia.com proxyelite.biz
rankscanner.com pukaporn.com
ranksonic.info putevka24.ru
realnye-otzyvy.info puzzleweb.ru
quickchange.cc
rangjued.com
ratemodels.net
rating-casino2021.ru
razleton.com
razorweb-a.akamaihd.net
rcb101.ru
redbottomheels.xyz redbottomheels.xyz
rednise.com redirectme.net
remont-fridge-tv.ru registratciya-v-moskve.ru
resultshub-a.akamaihd.net relayblog.com
restaurantlescampi.com
rezeptiblud.ru rezeptiblud.ru
rff-cfal.info riders.ro
robot-forex.biz ring4rhino.com
ros-ctm.ru robotixix.com
rucrypt.com roll123.com
rutor.group royalvegascasino.com
s8-nowy-wygraj.comli.com rozalli.com
sa-rewards.co.za ru-mediaget.ru
salmonfishingsacramentoriver.com rubbed.us
samoiedo.it runstocks.com
sc-specialhost.com ruscoininvest.company
scanner-brian.top rutor.vip
scanner-julia.top saltspray.ru
scanner-margo.top sanatorrii.ru
se-welding.ru sbdl.no
seasonvar.ru scanner-mary.top
scanner-willy.top
school-diplomat.ru
searchengineranker.email
searchimpression.com
seccioncontrabajo.com
securesmrt-dt.com
segol.tv segol.tv
semaltmedia.com semalt.com
semprofile.com seocheckupx.com
semrush.com servethis.com
seo-2-0.com service.adtech.fr
seobility.net servisural.ru
seofied.com sex-tracker.com
seofirmreviewsus.info sexkontakteao.info
seonetwizard.com
sexcamamateurchat.com
sexpartygirls.net
sexpornotales.com
sexy-pings.com
sexyali.com sexyali.com
sharebutton.org sexyebonyteen.com
sharebutton.to sexytrend.ru
shariki-zuma-lines.ru sfj-ror.no
sharpchallenge.com shijian.ac.cn
shitmovs.com shoesonlinebuy.cn
shopcheermakeup.info shop.acim.org
shoplvlv.us shop.xz618.com
simon3.ru shopfishing.com.ua
sinestesia.host.sk sientalyric.co
sispe.com.br sildenafil-tadalafil.info
skladvaz.ru sims-sims.ru
smartadserver.com sinel.info
social-search.me sitevaluation.org
socialbuttons.xyz sitevalued.com
socialmediasuggest.com sneakyboy.com
socialseet.ru snowplanes.com
sophang8.com soft-terminal.ru
softxaker.ru
solmarket.by
space2019.top space2019.top
space4updating.win special-porn.com
spamnuker.com spidtest.space
spitfiremusic.com stard.shop
sport-video-obzor.ru staynplay.net
sports-supplements.us stefanbakosab.se
starpages.net strigkaomsk.ru
stop-gepatit.te.ua stroydetali.ru
success-seo.com stroyhelp-dv.ru
suggest-keywords.com strv.se
svensk-poesi.com sudexpert66.ru
sweepstakes.rewardit.com suhanpacktech.com
sukarame.net
superstats.com
swingerseiten.com swingerseiten.com
tandvardshuset.net taaaak.com
taqplayer.info taqplayer.info
telefonsexsofort.tk tecnoteakviareggio.it
telsis.com teencastingporn.com
the-usa-games.blogspot.com teenporn18.net
theplacetoupdating.pw telefonsexi.com
thetattoohut.com telegraf.by
timeallnews.ru tfxiq.com
tizanidineforopiatewithdrawal.blogspot.com the-trader.net
tizanidineingredients.blogspot.com theheroes.ru
tizanidinemusclerelaxant.blogspot.com themestotal.com
tizanidineoral.blogspot.com thepornsex.org
tizanidinerecreationaluse.blogspot.com titan-cloud.life
tivolibasket.it
tizanidinenarcotic.blogspot.com
tizanidinepain.blogspot.com
tizanidinerestlesslegsyndrome.blogspot.com tizanidinerestlesslegsyndrome.blogspot.com
tizanidinevsvalium.blogspot.com tizanidinewithdrawal.blogspot.com
torrent-newgames.com tn811.us
torrentdownloadhub.com top-study.work
toursmaps.com top250movies.ru
toy-shop.top topads.men
tozup.com topdownloads.ru
track112.site tourismvictoria.com
trackmedia101.com tradedeals.biz
traderzplanet.in trkur.com
triumf-realty.ru trucri.me
truck-land.life trumpetedextremes.com
tube8.com tube8.com
tyumen.xrus.org tuberkulezanet.ru
uac.net.au turizm.bz
uchetunet.su turn-up-life.life
ussearche.cf twitlinks.com
utiblog.fr ua.tc
uzungil.com udayavani.com
valkiria-tk.ru ukkala.xyz
vclicks.net upupa.net
versaut.xxx-cam.webcam url-img.link
vertaform.com v720hd.ru
valmetrundan.se
velobikestock.com
venerologiya.com
veselokloun.ru veselokloun.ru
vgoloveboli.net vgoloveboli.net
viagengrarx.com videochaty.ru
viagraneggrx.com vip-dom.in
viberdownload10.com vipps.com.my
vip-file.com vodaodessa.com
vkonche.com voloomoney.com
vseigru.one
vzglyadriv.kg
vzlom-na-zakaz.com
vzlomfb.com
vzlomsn.org
vzube.com
wallpaperaccess.com
wallpapers-best.com
wallpapersist.com
watchmyfb.pl
waterefficiency.co
wdfdocando.com
webadvance.club
webcamdevochka.com webcamdevochka.com
webenlace.com.ar webscouter.net
website-speed-check.site webshoppermac.com
website-speed-checker.site weprik.ru
webtherapy.ru wleuaprpxuvr.ga
weekes.biz.tc woman-h.ru
westen-v.life worldinternetauthority.com
wholesalecheapjerseysfree.com worldtraveler.world
wholesalejerseychinaoutlet.com xn----9sbebi2bvzr7h.xn--p1ai
whosonmyserver.com xn----ctbigni3aj4h.xn--p1ai
wikes.20fr.com
wildworld.site
withstandingheartwarming.com
wordpress-crew.net
workle.website
woweb.com.ua
ws.ampower.me
wufak.com
ww1943.ru
www.jbetting.com
www.souvenirua.com
x-diesel.biz
x-musics.com
x-stars.ru
xchat26.myfreecams.com
xep.info
xfire.com
xn----itbkqkfiq.xn--p1ai
xn--24-glceagatoq7c2a6ioc.xn--p1ai
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xn--80aodinpgi.xn--p1ai xn--80aodinpgi.xn--p1ai
xnxx-n.com xn--oogle-wmc.com
xtube.com xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
xxx-cam.webcam xportvusbdriver8i.snack.ws
xxxkaz.org xtremeeagles.net
yatrk.xyz xvideosbay.com
you-shall-not-pass.is74.ru xxxnatelefon.ru
yourads.website yachts-cruise.info
z-master.ru yebocasino.com
za-fun-offer.com youradexchange.com
zaloro.com youtubedownload.org
zebradudka.com yuarra.pluto.ro
zahvat.ru
zeikopay.com zeikopay.com
zhuravlev.info zigarettenonlinekaufen.tumblr.com
zrizvtrnpale.tk znakom.sibtest.ru
znakomstva-moskva77.ru
znakomstva-piter78.ru

488
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1-free-share-buttons.com 00author.com
1xbet4you.com 108shot.com
2pxg8bcf.top 11235813.webzdarma.cz
8jn.dating-app.ru 1j7740kd.website
aa9d046aab36af4ff182f097f840430d51.com 2020iscoming.info
adamoads.com 21h2o.com
addray.pro 24x7-server-support.site
advertex.info 2kata.ru
ageofclones.com 4kepics.com
allcredits.su 57883.net
allkrim.com adanih.com
amazingninja.com adktrailmap.com
ameblo.jp adrunnr.com
analnoeporno.tv alibestsale.com
anapa-inns.ru amehdaily.com
appmsr.org amoi.tn
apps-analytics.net anonymous-redirect.com
arkkivoltti.net apartamentwroclaw.eu
aucoinhomes.com apidigidocketnet-a.akamaihd.net
avtolombard-krasnodar.ru apilinkswiftco-a.akamaihd.net
balook.com app-ready.xyz
beauby.ru appfastplay.com
bedcapdealers.com appfixing.space
best-mam.ru baksman.com
bestdraws.com bala.getenjoyment.net
bio-japan.net balla-boo.se
biographiya.com bannerspace.com
blog.yam.com bbsoldes.fr
bloglag.com besthomemadepornsites.com
bookmark4you.com beyan.host.sk
brainboosting.club bfz.biz
brandbucket.com bibys.com
brandedleadgeneration.com bjgugu.net.ua
brillianty.info bkns.vn
brus.city blagovest-med.ru
buqayy0.livejournal.com blogrankers.com
burkesales.com boleznikogi.com
business-made-fun.com bonzbuddy.com
c1.onedmp.com buynorxx.com
calc-for-credit.ru carina-sy.de
cardiosport.com.ua
cartierring.xyz
casablancamanor.co.za casablancamanor.co.za
cezartabac.ro cascadelink.org
cheapmoncler.win catalogs-parts.com
chocoslim.pro cementaresearch.se
christianlouboutinsaleoutletonline.us cfsrating.sonicwall.com
cialis-samples.com chcu.net
citetick.com checkpagerank.net
clickso.com cookingmeat.ru
clicksotrk.com cottageofgrace.com
clubfashionista.com cpabegins.ru
coldfilm.ru creditnation.ru
colobit.biz dailyrank.net
consorzioilmosaico.org dbmkfhqk.bloger.index.hr
countbertwistdisp26.soup.io decorazilla.com
crirussian.ru delayreferat.ru
cv.wallhade.co derevesendeco.com
cxpromote.com dfwu1019.info
dalavia.ru djihispano.com
daneshetabiat.com doctissimo.fr
davidovic.info domaincrawler.com
ddlmega.net dorratex.tn
ddospanel.com duawitchrarato.tk
depositfiles-porn.ga enternet.ee
designdevise.com eropho.com
dev-seo.blog eropho.net
dogbreedspicture.net erotik-kostenlos.net
dollartree.info estibot.com
domain.webkeyit.com evidencecleanergold.com
drev.biz extstat.com
dscaas.website eyes-on-you.ga
dvr.biz.ua ezigarettenshop1.yolasite.com
ecommerce-seo.com f0815.de
ee77ee.com fetlifeblog.com
elektronischezigarettekaufen1.beeplog.com findclan.org
elektrozigarettekaufen2.blogse.nl flauntyoursite.com
eloconcream.blogspot.com floridahuntingfishingadventures.com
estelight.ru flowersbazar.com
extractorandburner.com frameimage.org
ezb.elvenmachine.com free411games.com
ezigarettekaufen.myblog.de freenode.info
fastfixing.tech game-mmorpg.net
floridamhca.org gazobeton-p.com.ua
focalink.com generalporn.org
forums.d2jsp.org getyourimage.club
forzeronly.com goldandcard.ru
frcls.fr googlefeud.com
fyxabomiw.ru guardlink.com
gallerily.com gz2.bbsoldes.fr
girlporn.ru handicapbathtubarea.com
gk170.ru hasshe.com
golaya.pw hd-filmy.net
goodwriterssales.com help.tpu.ru
handicapvansarea.com hobbyhuren24.net
headpress.ru hotblognetwork.com
helpmymacfaster.trade hoverboard360.es
hol.es husky-shop.cz
homedecorpicture.us idealtits.net
hotshoppymac.com ido3.com
hoverboard360.de imicrovision.com
iccornacircri.cf impisr.edunsk.ru
ilikevitaly.com interferencer.ru
imfamous.info ipostroika.ru
impresagaia.it iqupdatetmz.win
indiasourcemart.in izhevsk.xrus.org
istmira.ru javspace.net
istripper.com kamalsinha.com
iweblist.info kamen-e.ru
ixora.pro keyword-suggestions.com
javatex.co.id keywordspay.com
jerseychinabizwholesale.us
jerseywholesaleelitestore.com
jmat.cn
jo24news.com
kaac.ru
kino-key.info
kinocccp.net
kinozapas.org kinozapas.org
klosetkitten.com klikbonus.com
kosova.de krasnodar.ru
kosynka-games.ru krynica.info
landscapebackgrounds.blogspot.com lacloop.info
lapolis.it lapolis.it
larose.jb2c.me lida-ru.com
leadwayau.com link.ac
ledpolice.ru machicon-akihabara.info
lodki-pvh.dp.ua makedo.ru
lookingglassemb.com manifestation.betteroffers.review
lubetube.com marktforschung-stuttgart.com
macresource.co.uk martlinker.com
madot.onlinewebshop.net matb3aa.com
meble-bogart.info maximilitary.ru
medkletki.ru meendo-free-traffic.ga
middlerush-a.akamaihd.net message-warning.net
misslike.ru midst.eu
mmog-play.ru missclub.info
mockupui.com mobifunapp.weebly.com
mokrayakiska.com mojowhois.com
mrbitsandbytes.com monclerjacketsoutlet.win
mundoaberrante.com mowser.com
mxgetcode.com mrsdalloways.com
myftpupload.com musicas.baixar-musicas-gratis.com
myhydros.org mustat.com
nacap.ru muznachas-service.ru
nagdak.ru my-bc.ru
najaden.no mymobilemoneypages.com
narutonaruto.ru na-telefon.biz
nash-krym.info narkologiya-peterburg.ru
obiavo.in narkologiya-voronezh.ru
obiavo.ru naturalpharm.com.ua
offers.bycontext.com nofreezingmac.click
ok-ua.info nomuos.it
okel.co obiavo.kz
online7777.com offf.info
onlinewritingjobs17.blogspot.ru office-windows.ru
ooo-gotovie.ru oklogistic.ru
oranga.host.sk omgtnc.com
oren-cats.ru onlineserialy.ru
parlament.biz ooo-olni.ru
pavlodar.xkaz.org ourtherapy.ru
pdns.cz pawli.eu
perederni.net pay2me.pl
pharmacyincity.com perm-profnastil.ru
pigrafix.at photokitchendesign.com
pio.polytopesexempt.com picquery.com
platesauto.com pirateday.ru
pliks.pl pk-services.ru
polytopesexempt.com play-movie.pl
popads.net playtap.us
porno-video-chati.ru porto.abuilder.net
pornosmola.info portside.cc
powenlite24.ru poshiv-chehol.ru
prizefestival.mobi privhosting.com
promgirldresses.xyz pro-tec.kz
pronekut.com prointer.net.ua
pulseonclick.com proprostatit.com
qitt.ru rank-checker.online
quangcaons.com ranksonic.com
rasteniya-vs-zombi.ru rapevideosmovies.com
realtytimes.com rarbg.to
referencemoi.com realizmobi.com
regdefense.com reklama-i-rabota.ru
reklamuss.ru ru-mediaget.ru
rethinkwasteni.info ruf777.com
ribieiendom.no runstocks.com
rogervivierforsale.com screentoolkit.com
roleforum.ru scrnet.biz.ua
rubbed.us seemoreresultshu-a.akamaihd.net
russia-today-video.ru seo-platform.com
salutmontreal.com sex-pr.net
scanner-brian.top shopwme.ru
scanner-ivan.top si-unique.com
scenicmissouri.us
schalke04fc.info
sdelatmebel.ru
se-welding.ru
searchencrypt.com
searchmywindow-a.akamaihd.net
searchtooknow-a.akamaihd.net
secret.xn--oogle-wmc.com
seorussian.ru
sexpornotales.net
sexsearch.com
shell-pmr.ru
shemalegalls.blogporn.in
shymkent.xkaz.org
similardeals.net
site-speed-check.site site-speed-check.site
siteimprove.com siteimprove.com
social-s-ggg.xyz smailik.org
soft1.ru soaksoak.ru
spacash.com soc-proof.su
stair.registrydomainservices.com solinf.co
stefanbakosab.se songoo.wz.cz
streetfire.net spinnerco.ca
susanholtphotography.com spy-sts.com
tehngr.ru staff.prairiesouth.ca
telemetryverification.net start.myplaycity.com
telesvoboda.ru statistici.ro
thewomenlife.com strag-invest.ru
tizanidineshowupondrugtest.blogspot.com stretchmate.net
tjkckpytpnje.com stuff-about-money.com
trafficmania.com summerlinhomes411.info
tecnoteakviareggio.it
thebluffs.com
thesmartsearch.net
thin.me.pn
threecolumnblogger.com
tizanidinecapsules.blogspot.com
tizanidinedruginteractions.blogspot.com
toloka.hurtom.com
tongkatmadura.info
topads.men
topcasinoratings.ru
tracfone.com
trafficgenius.xyz
trafficmonetize.org
trafficstars.com trafficstars.com
tupper-posuda.ru tuberkuleznik.ru
underthesite.com tubo360.com
us-america.ru tvnewsclips.info
v24s.net unitygame3d.com
vashsvet.com varbergsvind.se
vvon.co.uk vasileostrovsky-rayon.ru
watchmyfb.pl vesnatehno.com
web.cvut.cz vezuviy.su
weblibrary.win vkonche.com
webmasters.stackexchange.com vladimir.zrus.org
webnode.me vltai.com
whatsappbot.flyland.ru vuclip.com
wholesalejerseychinaoutlet.com vzlomtw.com
wildnatureimages.com wallpapersdesk.info
wineration.com wcb.su
wpthemedetector.co.uk web-betting.ru
xcc24.pl webcamdevochka.com
yourtemplatefinder.com webenlace.com.ar
z-master.ru webinstantservice.com
zeikopay.com worldoffiles.ru
zigzog.ru x-diesel.biz
znakomstva-piter78.ru x-lime.net
zrus.org x-rates.ru
zscaler.net x-true.info
zverokruh-shop.cz xerox-douglas.cf
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xn--d1aifoe0a9a.top
xtube.com
xxxtubesafari.com
youtube-downloader.savetubevideo.com
youtubologia.it
zastenchivosti.net
zeroredirect5.com
zscalertwo.net

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.183 0; 10.1.0.53 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2906 ### Version: V4.2022.02.2907
### Updated: Tue Feb 8 22:01:30 UTC 2022 ### Updated: Wed Feb 9 10:03:33 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2906 ### Version: V4.2022.02.2907
### Updated: Tue Feb 8 22:01:30 UTC 2022 ### Updated: Wed Feb 9 10:03:33 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##