vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.05.3114 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-05-19 10:09:43 +00:00
parent 230f568bcc
commit deac1539c9
47 changed files with 97992 additions and 98002 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3113 #### Version: V4.2022.05.3114
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3113 #### Version: V4.2022.05.3114
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.05.3113 #### Version: V4.2022.05.3114
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12114
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.05.3113 #### Version: V4.2022.05.3114
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12118
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12118
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12118
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12118
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.0.221 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.0.221 0;
127.0.0.1 0; 127.0.0.1 0;

12118
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.0.221 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.0.221 0;
127.0.0.1 0; 127.0.0.1 0;

12118
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.0.221 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.0.221 0;
127.0.0.1 0; 127.0.0.1 0;

12118
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3113 3114

12118
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12118
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.165.255.125 0; 20.245.255.181 0;

12118
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.0.221 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.0.221 0;
127.0.0.1 0; 127.0.0.1 0;

12118
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.0.221 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.0.221 0;
127.0.0.1 0; 127.0.0.1 0;

12118
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.0.221 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.0.221 0;
127.0.0.1 0; 127.0.0.1 0;

12118
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.165.255.125 0; 20.245.255.181 0;

12118
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.165.255.125 0; 20.245.255.181 0;

12118
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 1; 10.1.0.221 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Asterias BLEXBot
Chlooe DIIbot
Copyscape FHscan
Ebingbong Id-search
Image Fetch Image Fetch
Nessus LinkWalker
Quick-Crawler OpenLinkProfiler
Spanner SEOlyticsCrawler
Zitebot arquivo.pt
seocompany.store sysscan

308
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
404enemy
ADmantX ADmantX
Aboundexbot AIBOT
ASPSeek
AfD-Verbotsverfahren AfD-Verbotsverfahren
AllSubmitter
Alligator Alligator
Anarchy AlphaBot
Apexoo Anarchie
Atomseobot Anarchy99
Ankit
Asterias
Attach Attach
AwarioRssBot BBBike
AwarioSmartBot BLEXBot
BDCbot BackStreet
BackWeb Badass
Backlink-Ceck Battleztar Bazinga
Bandit
Barkrowler
BetaBot BetaBot
Bitacle
Black Hole Black Hole
BlackWidow Blow
Blackboard BlowFish
Boardreader Bolt
Buck Buck
Buddy Buddy
Bullseye BuiltBotTough
BunnySlippers
CATExplorador
CODE87
CSHttp CSHttp
CensysInspect CensysInspect
CheTeam
CherryPicker CherryPicker
ChinaClaw Chlooe
Claritybot Citoid
Cocolyzebot Cocolyzebot
Craftbot Collector
Crawling at Home Project Copier
Crescent Copyscape
Curious CrunchBot
Custo CyotekWebCopy
DigitalPebble DBLBot
Dirbuster DataCha0s
Discobot DatabaseDriverMysqli
Devil
Digincore
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot
DomainAppender DomainAppender
DomainStatsBot
Domains Project Domains Project
EMail Siphon Download Wonder
EasyDL Dragonfly
EirGrabber EMail Wolf
Evil EroCrawler
Exabot Express WebPictures
ExtLinksBot Extreme Picture Finder
EyeNetIE
FDM FDM
Flunky Firefox/7.0
Foobot Freeuploader
FrontPage FrontPage
FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t
GT::WWW
GalaxyBot
GermCrawler
GetRight GetRight
Gotit GetWeb
Grabber Getintent
Gigabot
GoZilla
Grafula Grafula
GrapeshotCrawler GrapeshotCrawler
GridBot HMView
HEADMasterSEO Humanlinks
HTTP::Lite IDBTE4M
HTTrack
HaosouSpider
Harvest
IDBot
IRLbot IRLbot
IlseBot Iblog
Image Fetch
IndeedBot
Indy Library Indy Library
InfoTekies Intelliseek
InternetSeer InternetSeer
Joomla IstellaBot
JOC Web Spider
JamesBOT
Jbrofuzz
JennyBot
Jorgee Jorgee
Jyxobot
Kenjin Spider Kenjin Spider
Keybot Translation-Search-Machine Keyword Density
Kinza
Kozmosbot
LWP::Simple
Larbin Larbin
LexiBot
Lftp
Libwhisker Libwhisker
Likse LieBaoFast
LinkScan LinkScan
LinkWalker Linkbot
LinksManager LinksManager
LinqiaMetadataDownloaderBot Lipperhey
LinqiaScrapeBot
Litemage_walker
Ltx71 Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MSIECrawler
Mag-Net MTRobot
Majestic-SEO Magnet
Majestic12 MarkMonitor
Mass Downloader
Mata Hari Mata Hari
MauiBot MauiBot
Mb2345Browser
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru
Metauri Metauri
Minefield MicroMessenger
Mojolicious Microsoft Data Access
MolokaiBot Microsoft URL Control
Moblie Safari
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
Musobot NICErsPRO
NPbot
NearSite NearSite
Nessus Needle
NetLyzer NetAnts
NetZIP Nettrack
Netvibes NextGenSearchBot
Nibbler
Niki-bot
Nikto
NimbleCrawler NimbleCrawler
Ninja
Nmap
Nuclei Nuclei
Nutch
Octopus Octopus
Offline Explorer Offline Navigator
OpenLinkProfiler OnCrawl
OpenVAS
Openfind Openfind
OrangeBot
OrangeSpider OrangeSpider
Pandalytics OutfoxBot
PeoplePal POE-Component-Client-HTTP
Petalbot Page Analyzer
PageScorer
PageThing.com
Papa Foto
Pavuk
Pi-Monster Pi-Monster
Picsearch PictureFinder
Piepmatz Piepmatz
PleaseCrawl ProPowerBot
Pockey Probethenet
ProWebWalker Psbot
Pu_iN
Pump
QueryN Metasearch QueryN Metasearch
Quick-Crawler
RSSingBot
RankActiveLinkBot RankActiveLinkBot
Re-re Rankivabot
ReGet Reaper
RebelMouse
Recorder Recorder
RedesScrapy
Ripper
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SISTRIX
SEOstats SalesIntelligent
ScanAlert Scrapy
Screaming
ScreenerBot ScreenerBot
SearchmetricsBot ScrepyBot
Seekport Seekport
SemanticJuice SemanticJuice
Semrush
SemrushBot SemrushBot
SeoSiteCheckup
Seomoz Seomoz
Shodan Shodan
Site Sucker Site Sucker
SiteExplorer SiteSucker
SiteLockSpider
Sitebeam
Siteimprove Siteimprove
SlySearch SmartDownload
Snake
Snapbot Snapbot
SocialRankIOBot Sociscraper
Sogou web spider Sogou web spider
Sottopop
SpaceBison SpaceBison
Spammen
SputnikBot SputnikBot
Sqlmap Sqlmap
Sqlworm Sqlworm
Steeler Stripper
Sucuri Sucker
SuperBot SurveyBot
SuperHTTP
Suzuran Suzuran
Szukacz Szukacz
T0PHackTeam Teleport
TeleportPro
Telesoft
Telesphoreo
Telesphorep Telesphorep
TheNomad Titan
Toweyabot
Trendiction
True_Robot
TurnitinBot
Twice
Typhoeus Typhoeus
URLy.Warning URLy.Warning
UnisterBot
V-BOT
VB Project
VCI VCI
Vacuum Vacuum
VeriCiteCrawler Vagabondo
VoidEYE VoidEYE
Voltron WISENutbot
WPScan WPScan
WWW-Mechanize
WWW::Mechanize
Wallpapers/3.0
WeSEE WeSEE
Web Auto Web Auto
Web Collage
Web Fetch
Web Fuck Web Fuck
Web Sauger Web Sucker
WebBandit WebBandit
WebCopier WebFetch
WebFuck
WebImageCollector WebImageCollector
WebLeacher
WebPix WebPix
WebReaper WebReaper
WebSauger WebStripper
WebWhacker
Webalta Webalta
Website Quester Webshag
WebsiteExtractor
WebsiteQuester
Whacker
Whatweb Whatweb
Widow
WinHTTrack WinHTTrack
WiseGuys Robot Wonderbot
Woobot Xenu
Wotbox Zade
Xaldon_WebSpider Zermelo
Zeus
Zitebot Zitebot
ZmEu ZmEu
ZoomBot cah.io.community
arquivo-web-crawler
autoemailspider
backlink-check
check1.exe check1.exe
cognitiveseo
com.plumanalytics
crawler.feedback crawler.feedback
crawler4j crawler4j
dataforseo.com internetVista monitor
demandbase-bot isitwp.com
gopher
ips-agent
iubenda-radar iubenda-radar
linkdexbot linkdexbot
magpie-crawler magpie-crawler
meanpathbot meanpathbot
mediawords mediawords
muhstik-scan
pcBrowser
s1z.ru s1z.ru
satoristudio.net
seobility
seoscanners seoscanners
sexsearcher seostar
siteripz serpstatbot
sogouspider
tAkeOut tAkeOut
ubermetrics-technologies.com webprosbot
voyagerx.com x22Mozilla
x09Mozilla zauba.io
xpymep1.exe
zgrab

300
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404enemy AIBOT
ASPSeek ASPSeek
Abonti Aboundex
Aboundexbot AhrefsBot
Acunetix AiHitBot
Alexibot Aipbot
Alligator
AlphaBot AlphaBot
Anarchie Anarchie
Anarchy Ankit
Apexoo Aspiegel
AwarioRssBot Asterias
AwarioSmartBot Attach
BLEXBot BDFetch
BackStreet BackStreet
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass Bandit
BetaBot BatchFTP
Bitacle Battleztar Bazinga
Bigfoot
BlackWidow BlackWidow
Blackboard
Blow Blow
BlowFish BlowFish
Boardreader Boardreader
Bolt BotALot
Brandwatch Buddy
BuiltBotTough BunnySlippers
Bullseye Cegbfeieh
BuzzSumo
CCBot
CODE87
CensysInspect CensysInspect
CheTeam
CherryPicker CherryPicker
ChinaClaw
Chlooe
Citoid Citoid
Claritybot
Cocolyzebot Cocolyzebot
Collector Collector
Copier
CopyRightCheck CopyRightCheck
Copyscape Copyscape
Crescent Crawling at Home Project
CrunchBot
Curious Curious
CyotekWebCopy CyotekWebCopy
DBLBot DTS Agent
DatabaseDriverMysqli Demon
Devil DigitalPebble
Dirbuster
Disco
Discobot Discobot
Discoverybot Discoverybot
Dispatch DnyzBot
DomainStatsBot DomCopBot
Dotbot DomainAppender
Download Wonder DomainSigmaCrawler
Dragonfly Dragonfly
EMail Wolf EMail Wolf
EasyDL EasyDL
Ecxi Express WebPictures
Evil
Exabot
ExtLinksBot
Extractor Extractor
ExtractorPro
Extreme Picture Finder
Ezooms
FDM FDM
FHscan
FemtosearchBot FemtosearchBot
FlashGet
Foobot Foobot
FyberSpider Fuzz
Fyrebot Fyrebot
GalaxyBot GalaxyBot
GermCrawler GermCrawler
Go!Zilla Go-Ahead-Got-It
Gotit GrabNet
HEADMasterSEO
HTMLparser HTMLparser
Haansoft Harvest
HaosouSpider Humanlinks
Havij IDBot
Heritrix IRLbot
HybridBot IndeedBot
IDBTE4M Indy Library
Id-search
InfoTekies InfoTekies
Intelliseek Intelliseek
Internet Ninja IstellaBot
InternetSeer JOC Web Spider
JamesBOT
JennyBot JennyBot
JikeSpider JikeSpider
JustView Jorgee
Jyxobot
Kinza Kinza
Kozmosbot
LNSpiderguy LNSpiderguy
Lanshanbot
Larbin Larbin
LeechGet
Lftp Lftp
LibWeb LibWeb
Libwhisker LieBaoFast
Likse Likse
LinkScan Linkbot
LinkWalker
LinkextractorPro
LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
Lipperhey
Lipperhey Spider
Lmspider Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool
MSFrontPage
MSIECrawler MSIECrawler
MTRobot Mail.RU_Bot
Mag-Net
Majestic SEO
Majestic12 Majestic12
MarkMonitor
MarkWatch MarkWatch
Mass Downloader Masscan
MauiBot Meanpathbot
Mb2345Browser Metauri
MolokaiBot Microsoft Data Access
Mozlila Microsoft URL Control
Moblie Safari
Mojolicious
Mr.4x3 Mr.4x3
Msrabot
Musobot
Nameprotect
Navroad Navroad
NetAnts NetAnts
NetMechanic NetLyzer
NetSpider
Netvibes
Nibbler Nibbler
Niki-bot Nikto
Nmap Nimbostratus
Not Ninja
Nutch Nutch
OpenLinkProfiler OpenVAS
Openvas Openfind
OrangeBot OrangeSpider
OutfoxBot
PECL::HTTP PECL::HTTP
PageScorer POE-Component-Client-HTTP
Pandalytics PageGrabber
Panscient
Papa Foto Papa Foto
Petalbot Picsearch
Pi-Monster Piepmatz
PleaseCrawl PleaseCrawl
ProWebWalker
Probethenet Probethenet
Pump Pu_iN
RSSingBot PyCurl
Quick-Crawler
RankActive RankActive
Rankivabot
Re-re Re-re
RealDownload
Reaper Reaper
RebelMouse RocketCrawler
Recorder SEOkicks
RepoMonkey SEOkicks-Robot
Rogerbot SEOprofiler
SEOlyticsCrawler SEOstats
SISTRIX SISTRIX
SMTBot
Scanbot Scanbot
ScoutJet
ScrepyBot ScrepyBot
Seekport
SemrushBot SemrushBot
SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Seomoz
Shodan Shodan
Siphon
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteLockSpider
SiteSnagger
SiteSucker
Sitebeam Sitebeam
Siteimprove Sitevigil
SmartDownload SlySearch
Snapbot Snake
Snoopy
Sociscraper Sociscraper
Sogou web spider Sogou web spider
Sosospider Sottopop
Spammen Spammen
SpankBot SpankBot
Spbot Steeler
Spinn3r Stripper
SputnikBot SuperBot
Sqlworm
Sqworm
Sucker
SuperHTTP SuperHTTP
Swiftbot T0PHackTeam
Teleport T8Abot
Telesoft Telesoft
Telesphoreo TheNomad
Telesphorep Thumbor
TightTwatBot
Toata Toata
Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot True_Robot
TurnitinBot
TwengaBot
URLy Warning
UnisterBot UnisterBot
V-BOT VB Project
Vacuum
VelenPublicWebCrawler
VeriCiteCrawler
Virusdie Virusdie
VoidEYE VoidEYE
WASALive-Bot WEBDAV
WBSearchBot
WISENutbot WISENutbot
WPScan WWW-Collector-E
WWW-Mechanize WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0 Wallpapers
WallpapersHD WallpapersHD
WeSEE
Web Collage
Web Enhancer Web Enhancer
Web Fetch Web Sauger
Web Pix WebCopier
WebBandit
WebEnhancer WebEnhancer
WebFetch
WebImageCollector
WebLeacher
WebReaper
WebSauger
WebWhacker
WebZIP WebZIP
WebmasterWorldForumBot Whack
WebsiteQuester WiseGuys Robot
Whacker Wprecon
Widow Xaldon_WebSpider
Xaldon WebSpider
YoudaoBot YoudaoBot
ZmEu Zade
ZoominfoBot Zauba
ZyBorg Zitebot
adscanner archive.org_bot
arquivo-web-crawler arquivo.pt
autoemailspider
backlink-check
coccocbot coccocbot
com.plumanalytics
crawl.sogou.com
crawler4j crawler4j
dataforseo.com
domainsproject.org
eCatch
evc-batch evc-batch
lwp-trivial facebookscraper
heritrix
instabid
internetVista monitor
ips-agent
lwp-request
magpie-crawler magpie-crawler
mediawords
netEstate NE Crawler
oBot oBot
pcBrowser
plumanalytics plumanalytics
s1z.ru ripz
scalaj-http
scan.lol scan.lol
seocompany.store
seoscanners
spyfu spyfu
sysscan
tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webgains-bot webgains-bot
webmeup-crawler webmeup-crawler
webpros.com
x09Mozilla x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
autochoixspinelli.com alessandraleone.com
bupropion-sr-150-mg.us amehdaily.com
cbox.ws aosheng-tech.com
exoclick.com dichvuvesinhhanoi.com
i-hobot.ru map028.com
kladrus.ru mowser.com
searchwebknow-a.akamaihd.net net-radar.com
sexyteens.hol.es sampleletters.net
slowmacfaster.trade scanner-don.top
xn----ctbigni3aj4h.xn--p1ai vrotike.ru

920
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
000free.us 000free.us
123any.com 03e.info
123kuma.com 0c47f8422d3f.com
18ps.ru 12bet.com
1hwy.com 1688.com
3dracergames.com 1millionusd.xyz
4webmasters.com 1stat.ru
28n2gl3wfyb0.ru
2ads.co.uk
4senses.co
4ur.click
4ureyesonly.com
4webmasters.org
5elementov.ru
5u.com 5u.com
aa625d84f1587749c1ab011d6f269f7d64.com 7fon.ru
aa81bf391151884adfa3dd677e41f94be1.com acc.eu.org
aa9d046aab36af4ff182f097f840430d51.com actices.com
accessoires-mode.in ad-words.ru
adelly.bg addblueoff.com.ua
adrenalin-stk.ru
ads-seo.men ads-seo.men
adtech.de adsland.men
adultmeetonline.info adsloads.com
adverster.com adultwebhosting.info
advokat-grodno.by advancedcleaner.com
adzerg.com adventureparkcostarica.com
akama.com agahinameh.com
alekseevec.ru aihelen.net
alert-fdm.xyz aktivator-windows10.blogspot.com
alf-img.com all-number.com
alfa-medosmotr.ru alldezservice.kz
ameblo.top alloysteel.ru
animaltoplist.com allsilver925.co.il
anonymizeme.pro altamayoztourism.com
aosheng-tech.com alveris.ru
apiappenableinfo-a.akamaihd.net android4fun.org
apibatbrowsecom-a.akamaihd.net ap.senai.br
apimossnetinfo-a.akamaihd.net appsaurus.com
apparel-offer.com apptester.tk
appleid-verification.com
aquarium-pro.ru aquarium-pro.ru
arabgirls.adultgalls.com
arenanews.com.ua
artavenuegardenstudios.com
aruplighting.com aruplighting.com
asociatia-tipografilor-transilvania.ro asdfg.pro
atleticpharm.org atelielembrancaqueencanta.com.br
atlasvkusov.ru
atmagroup.ru
aussie-prizes.com
auto-moto-elektronika.cz auto-moto-elektronika.cz
autobrennero.it avkzarabotok.com
aviav.org azartniy-bonus.com
avitocars.ru azbukafree.com
azlex.uz bablonow.ru
babieca.com
badavit.com.ua badavit.com.ua
baltgem.com baditri.com
balitouroffice.com
banksrf.ru
bannerspace.com bannerspace.com
bbtec.net beauty-things.com
beachfix.co belinka.com.ua
begalka.xyz
belsetirehafi.tk belsetirehafi.tk
bestdooz.com bensbargains.net
bestimagecoollibrary.com bezlimitko.xyz
bgviagrachrx.com bfz.biz
bif-ru.info bidbuy.co.kr
bird1.ru binomo.com
bitporno.sx binomo.kz
biznesschat.net bio-market.kz
blog.koorg.ru biznesluxe.ru
board.f00d.de blue-square.biz
bolegapakistan.com bobinoz.com
bongiornos.info bodybuilding-shop.biz
boom-celebs.com bongacams.com
brendbutik.ru book-bets.com
buron.pw bookhome.info
buy-cheap-online.info bracketsmackdown.com
break-the-chains.com
brewdom.ru
brothers-smaller.ru
btvn.ru
builtwith.com
burkesales.com
buscarfoto.com
buynorxx.com buynorxx.com
buzzurl.jp buzz-porno.info
bvlgariring.xyz callcafe.info
cakemediahq.com.au callmd5map.com
callawaygolfoutlet.online capacitacionyaprendizaje.com
carfax.com.ua capture-room.com
cartierlovestore.com cartierlovebraceletreplica.xyz
caveavins.fr cashkitten-a.akamaihd.net
cerev.info ccjp.eu
cfacarrosserie74.com cdncash.com
chastnoeporno.com cdncash.net
chatroulette.life centraletermice.us
chatroulette.si cete.ru
cheap-pills-norx.com cgi2.nintendo.co.jp
chinajerseyswholesalecoupons.com cheapbelstaff.online
chlooe.com cheapmoncler.pw
choice-credit.ru chinaelitecheapjerseys.com
cigarpass.com chorus.terakeet.com
cinemaenergy-hd.ru cl.s7.exct.net
clevernt.com cleanpcnow.com
clicksor.net clickprotects.com
cmd.kz compliance-barak.top
compliance-brian.xyz coolbar.pro
compliance-viktor.top coolgamechannel.com
comsysnet.com corta.co
coolgramgoods.com counciltally.com
copenergo.ru crackzplanet.com
copro.pw
cottageofgrace.com
counter.yadro.ru
counterbot.com
craftburg.ru
crazyboost.pro crazyboost.pro
crazyprotocol.com creditmoney.com.ua
creditwell.ru customsua.com.ua
cubs-jerseys.us cymbaltaandtizanidine.blogspot.com
culturevie.info cyse.tk
cupday.com d0t.ru
cutt.us datsun-do.com
dafatiri.com dayibiao.com
daneshetabiat.com db.speedup.gdn
deda-moroza-zakaz.ru deadlinkchecker.com
demian.kz demenageur.com
dentalpearls.com.au detalizaciya-tut.biz
dfwu1019.info dev-seo.blog
diamond-necklace.info diesel-parts28.ru
diegolopezcastan.com distver.ru
djonwatch.ru divisioncore.com
dlya-android.org dktr.ru
dms-sw.ru dkvorota.ru
dnepropetrovsk.mistr-x.org domainsphoto.com
documentsite.net dotmass.top
donna7753191.ru ekosmetyki.net.pl
dood.live
dosugrostov.site
downloads-whatsapp.com
draniki.org
droidlook.net
drugs-no-rx.info
dvd-famille.com
eager-nash.188-93-233-196.plesk.page
ec-file.info
edge.sharethis.com
ekspresihati.info
ekto.ee ekto.ee
elektrischezi.canalblog.com elektrischezigarettekaufen2.cowblog.fr
elektrischezigarette1.onsugar.com elitepcgames.com
elektronischezi.livejournal.com embedle.com
elektrozigarettekaufen1.devhub.com energydiet24.ru
elektrozigaretten1.tumblr.com enge-fotzen.info
elstal.com.pl eropornosex.ru
elvel.com.ua escortslet.net
eredijovon.com eshop.md
evaashop.ru espaceinventoristes.com
exchangeit.gq evogarage.com
expdom.com extlinks.com
extremal-blog.com extractorandburner.com
ezrvrentals.com eyelike.com.ua
f1nder.org eywords-monitoring-your-success.com
f5mtrack.com ezigarettenkaufen1.pagina.gr
familyphysician.ru facebook-mobile.xyz
fandlr.com fashion-mk.net
fdzone.org fashion-stickers.ru
feel-planet.com fearcrow.com
ferieboligkbh.dk femdom.twiclub.in
ff30236ddef1465f88547e760973d70a.com finteks.ru
fickenprivat.info flipper.top
fifa-coins.online
figuringmoneyout.com
financeloan.us
find1friend.com
firstdrugmall.ru
fishingwholesale.us
fm-upgrade.ru fm-upgrade.ru
forpostlock.ru footbalive.org
foxydeal.com frameimage.org
frankofficial.ru free-traffic.xyz
frcls.fr front.to
free-share-buttons.com fuckbuddybestgilf.info
freshwallpapers.info fxgallery.com
fun-mobi.pl fyl.com.ru
gael-s.ru g7m.pl
gabeshop.ru
gallery.rennlist.com gallery.rennlist.com
gambarkataku.co gerhardhealer.com
gameplexcity.com gettpromos.com
gaming-journal.com gidonline.one
garciniaxt.us globalsurfari.com
gearcraft.us gnuetella.com
gentamicineyedrops.blogspot.com goatse.ru
gilbertbanda.net gogps.me
globetrotting-culture.ru gourcy.altervista.org
google-liar.ru graphicwe.org
gorabagrata.ru guitar-master.org
gowreckdiving.com gunsvicceadadebt.tk
graphics8.info haikuware.com
gratuitbaise.com handsandlegs.ru
growshop.es hanwei.us
gungamesz.com happy.new.yeartwit.com
hackers-crackers.tk haveinc.xyz
handicapvansarea.com hawaiielectriclight.com
hao123.com heartofpayne.xyz
hatdc.org hexpilot.com
health-medical-portal.info highspeed5.net
heroz.fr histock.info
hetmanship.xyz hobbyhuren24.net
hkladys.com homeandhealth.ru
hothot.ru homeart.space
hoverboard360.es homedo.fabpage.com
iamsport.org host-tracker.com
ico.re hotkeys.com
ilikevitaly.com i-service.kz
ilte.info iklysha.ml
imediadesk.com imadedinner.net
inboundlinks.win imagez.co
inclk.com imicrovision.com
infogame.name impresagaia.it
intellektmedia.at inboxdollars.com
iradiology.ru indetiske.ya.ru
irkutsk.zrus.org indiakino.net
ixora.pro infobanks.ru
iyasimasennka.com infowarcraft.ru
jaxcube.info innodgfdriverhm.aircus.com
jerseysbizwholesalecheap.com insider.pro
jerseyswholesalechinalimited.com insurple.com
jerseywholesalechinabiz.com internetartfair.com
jewelryandfiligree.com ip-guide.com
jobmarket.com.ua ipchicken.com
jumptap.com islamtoday.co.za
justdating.online ispac.org
kamalsinha.com itrevolution.cf
kartiniresto.com ivearchenceinflu.cf
kashubadesign.ru ivoiretechnocom.ci
kaz.kz jasonpartington.com
kerwinandcariza.com javstock.com
ketoanhanoi.info jerseyssportsshop.com
keywordsuggest.org jmat.cn
kfon.eu jongose.ninja
kino-fun.ru kadashihotel.com
kino-key.info katadhin.co
kinogonew.ru kazinogames.lv
kinostorm.net kchaxton.com
klin3952.ru keywordblocks.com
knigonosha.net keywordspay.com
koopilka.com keywordsuggests.com
koptims.tiu.ru kinocccp.net
kovrenok.ru kliksaya.com
kometars.xyz
komp-pomosch.ru
komputers-best.ru
kozhniebolezni.com kozhniebolezni.com
kupiproday.com.ua kwzf.net
kupit-adenu.ru ladov.ru
lacave.ntic.fr
lacloop.info
ladsup.com ladsup.com
lanasshop.ru lafriore.ru
languagecode.com lapolis.it
larutti.ru lazymanyoga.com
lazy-z.com ledpolice.ru
legalrc.biz
lego4x4.ru lego4x4.ru
lennatin.info lenvred.org
lignofix.ua letolove.ru
lineavabit.it lider-zhaluzi.kiev.ua
lidoradio.com
lifeinsurancekb.com
linerdrilling.com
lion.conboy.us lion.conboy.us
lirunet.ru lunamedia.co
liveinternet.ru luxmagazine.cf
ljusihus.se
lockerz.com
lombia.co
lotto6888.com
lyrster.com
magazin-pics.ru magazin-pics.ru
marketingvici.com manifestation.betteroffers.review
martlinker.com marcoislandvacations.net
mathgym.com.au marketland.ml
markjaybeefractal.com
matsdale.com
mearns-tractors.co.uk mearns-tractors.co.uk
meble-bogart.info medicovi.com
medanestesia.ru medictube.ru
medi-fitt.hu mega-polis.biz.ua
meetingrainstorm.bid megakino.net
mega-bony-2017.pl michellblog.online
megagrabber.ru mindbox.co.za
megavolt.net.ua
meltwater.com
metaxalonevstizanidine.blogspot.com
mhi-systems.ru
microsoftportal.net
minecraft-rus.org minecraft-rus.org
mirobuvi.com.ua minyetki.ru
mixed-wrestling.ru mirmedinfo.ru
mjchamonix.org mixtapetorrent.com
mmofreegames.online mnogabukaff.net
mobplayer.net mobsfun.net
moneymaster.ru modnie-futbolki.net
moz.com moivestiy.biz
mrpornogratis.xxx mole.pluto.ro
mrvideospornogratis.xxx monclercoats.xyz
mundoaberrante.com montazhnic.ru
muzaporn.com moscow-region.ru
myliveblog.ru moscow.xrus.org
myprintscreen.com movies-in-theaters.net
najaden.no mturkcontent.com
narkologiya-penza.ru muabancantho.info
natali-forex.com my-bc.ru
naturalbreakthroughsresearch.com myhealthcare.com
nbsproject.ru myplaycity.com
narkologiya-peterburg.ru
narosty.com
naughtyconnect.com
negociosdasha.com
newstaffadsshop.club
next-dentists.tk next-dentists.tk
nosecret.com.ua nicefloor.co.uk
notasprensa.info nisuturnetdgu.tk
nl.netlog.com
norththeface.store
november-lax.com november-lax.com
novgorod.xrus.org novodigs.com
obiavo.su novosti-hi-tech.ru
oddamzadarmo.eu nrjmobile.fr
oklogistic.ru nudejapan.net
one-gear.com nudo.ca
online-sbank.ru nuup.info
online-zaymy.ru nzfilecloud.weebly.com
obiavo.in
obiavo.ru
okmusic.jp
online-x.ru
onlineslotmaschine.com onlineslotmaschine.com
onstrapon.purplesphere.in onlythegames.com
optom-deshevo.ru ooo-gotovie.ru
owohho.com oralsexfilme.net
pandroid.co origin-my.ru
pdn-4.com osb.se11.ru
pdns.cz ovirus.ru
oyster-green.com
panicatack.com
paperwritingservice17.blogspot.ru
paradontozanet.ru
partners-ship.pro
pastaleads.com
pb-dv.ru
peekyou.com
pensplan.com pensplan.com
picture-group.com picquery.com
picturesboss.com pics-group.com
pijoto.net pirateday.ru
pintattoos.com pistonclasico.com
piter.xrus.org
pizdopletka.club
pl-vouchers.com
pl.aasoldes.fr pl.aasoldes.fr
plastjulgranar.se pl.id-forex.com
platinumdeals.gr plaff-go.ru
pokemon-go-play.online plastweb.ru
popup-jdh.xyz pliks.pl
porndroids.com polytopesexempt.com
pornmania.pl popunder.net
porno-transsexuals.ru popup-fdm.xyz
pornoelita.info popup.matchmaker.com
pornosemki.info pornhive.org
portadd.men pornodojd.ru
pr0fit-b0x.com pornogad.com
priora-2.com pornoklad.net
pornowarp.info
pornzone.tv
pozdrawleniya.com
privatefx-in.ru privatefx-in.ru
prizeestates.cricket profitkode.com
pro-poly.ru proxyelite.biz
producm.ru pyrodesigns.com.au
professionaldieselcare.com qitt.ru
profit-opportunity.com quebec-bin.com
prohoster.info r-control.ru
promodj.com rainbowice.ru
promoheads.com rankscanner.com
prospekt-st.ru ranksonic.info
prostitutki-tolyatti.org razamicroelectronics.com
pst2017.onlinewebshop.net razyboard.com
pulse33.ru realitykings.com
pussyfleet.com redirectme.net
qexyfu.bugs3.com reeyanaturopathy.com
raschtextil.com.ua refudiatethissarah.info
ratemodels.net regionshop.biz
razleton.com
razvratnoe.org
registrationdomainsite.com
registry-cleaner.net
reimageplus.com
reklamuss.ru reklamuss.ru
relatodelpresente.com.ar remont-comp-pomosh.ru
remontbiz.ru rennlist.com
renhacklids.tk retailwith.com
rentalcarnavi.info revistaindustria.com
rentehno.ru rewardit.com
rfd-split.hr rezeptiblud.ru
riders.ro
rightenergysolutions.com.au rightenergysolutions.com.au
rumamba.com roofers.org.uk
runofilms.ru
russia-today-video.ru
s.lollypopgaming.com
sadaholding.com
sarafangel.ru sarafangel.ru
sardinie.us sardinie.us
savetubevideo.com scanmarine.info
scanner-andrew.top scanner-jack.top
scanner-barak.top scanner-john.top
scanner-elena.top scanner-willy.top
scanner-nelson.top scrapinghub.com
scanspyware.net search-goo.com
schalke04fc.info searchinquire.com
scrapy.org seinterface.com
seadragonherbery.com seo-tools-optimizing.com
search-error.com seomarketings.online
searchkut.com sex-foto.pw
seasaltwithfood.com sex-pr.net
seminarygeorgia59.ga sex-tracker.com
seorankinglinks.com sexy-pings.com
seorankinglinks.us sheerseo.com
sex-dating.co shohanb.com
sex-tracker.de shopperifymac.com
sex.hotblog.top shoppingjequiti.com.br
sexkrasivo.net sibvitr.ru
sexreliz.com sildenafil-tadalafil.info
sexs-foto.top site-speed-checker.site
sexyali.com sizeplus.work
sharebutton.to skutecznetabletkinaporostwlosow.pl
shisha-swag.de sky-mine.ru
shoesonlinebuy.xyz smart-scripts.com
sirpornogratis.xxx smsactivator.ru
siteaero.com snsdeainavi.info
siteonomy.com soc-econom-problems.ru
sobecjvuwa.com.ru soc-proof.su
sofit-dmd.ru socas.pluto.ro
soft-terminal.ru
solnplast.ru
sonata-arctica.wz.cz sonata-arctica.wz.cz
sosdepotdebilan.com sophang8.com
space4updating.win spb-plitka.ru
spbchampionat.ru special-porn.com
sprttrack.com stard.shop
ssconstruction.co stardevine.com
star61.de starwars.wikia.com
statoutlook.info stockquotes.wooeb.com
strana-krasoty.ru strv.se
stretchmate.net studentguide.ru
strfls.com sundrugstore.com
stroiminsk.com superstats.com
stromerrealty.com surcentro.com
strongsignal-a.akamaihd.net surfbuyermac.com
sudexpert66.ru svarkagid.com
syvertsen-da.no svensk-poesi.com
tandvardshuset.net swimpool.ca
taximytishi.ru swinginwithme.ru
tds-advert005.info teencastingporn.com
telefonsexi.com telefonsexsofort.tk
theallgirlarcade.com thebestphotos.eu
thedownloadfreeonlinegames.blogspot.com thetardistimes.ovh
theendivechronicles.com timhost.ru
themestotal.com tizanidineduringpregnancy.blogspot.com
thfox.com tizanidineformigraines.blogspot.com
timeallnews.ru tizanidinevssoma.blogspot.com
tizanidinehcl2mgtablet.blogspot.com tjkckpytpnje.com
tizanidinehclsideeffects.blogspot.com top-l2.com
tizanidineoral.blogspot.com topdownloads.ru
tizanidineorflexeril.blogspot.com tracxn.com
todohr.com trafficmania.com
topbestgames.com traffique.net
topclickguru.com trahic.ru
totu.info try-rx.com
totu.us tsan.net
trichizobswiv.agddns.net tt-ipd.info
twittruth.com turn-up-life.life
uchebavchehii.ru
udayavani.com udayavani.com
ufa.xrus.org unisexjewelry.org
ukrtvir.com.ua upupa.net
ukrtvory.in.ua urdoot.win
unrealcommander.com url-extractor.xyz
vacuumscleaner.com valmetrundan.se
vclicks.net vapeface.club
vegascosmetics.ru vavilone.com
vekzdorov.ru viagra.pp.ua
velpanex.ru viandpet.com
videochat.world videonsk.com
vids18.site
viel.su
villacoloniale.com
vip2ch.com
vipromoffers.com vipromoffers.com
virus-respirators.com vipsexfinders.com
vizag.kharkov.ua virtualbb.com
vkak.ru visitcambridge.org
vkmusics.ru vkontaktemusic.ru
vmnmvzsmn.over-blog.com vote-up.ru
vpnmouse.com vrnelectro.ru
vseigry.fun vroze.com
vzlomtw.com w-journal.ru
wallpaperaccess.com webnode.me
webscouter.net website-analyzer.info
webshoppermac.com website-speed-checker.site
website-speed-check.site weburok.com
websiteaccountant.de
welcomeauto.ru welcomeauto.ru
wemarketing.se weprik.ru
wetgames.ru worldlovers.ru
wfb.hatedriveapart.com writingservices17.blogspot.ru
wholesalenfljerseys.us.com
wildworld.site
windowssearch-exp.com
wingsofrefuge.net
wjgony.com
wma-x.com
womens-journal.net
wosik-dach.service-for-web.de
wttavern.com wttavern.com
x-diesel.biz wufak.com
x-rates.ru wurr.voila.net
xaylapdiendanang.com x-lime.com
xep.info x-porno.video
xerox-douglas.cf xchangetrak.com
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai xdoza.com
xgftnlrt.bloger.index.hr
xmnb.net
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xn----7sbahjd3btneuw1joc.xn--p1ai
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn----dtbndd4ae7eub.top xn--24-glceagatoq7c2a6ioc.xn--p1ai
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai xn--d1abj0abs9d.in.ua
xn--80ajbshivpvn2i.xn--p1ai xn--h1aakne2ba.xn--p1ai
xn--90acenikpebbdd4f6d.xn--p1ai xnxx-n.com
xtrafficplus.com xxx-cam.webcam
xxxdatinglocal.us
xxxhdvideo.site xxxhdvideo.site
xxxmania.top
xz618.com
yellowads.men yellowads.men
yougotanewdomain.com yougame.biz
youresponsive.com youjizz.vc
yugk.net youporn-forum.ga
z-master.ru yourads.website
zahvat.ru yun56.co
zajm-zalog-krasnodar.ru zajm-zalog-krasnodar.ru
zakon-ob-obrazovanii.ru zeroredirect2.com
zakonobosago.ru zrelaya.pw
zarepta.com
zdesoboi.com
zebradudka.com
zeleznobeton.ru
zhongwenlink.com
zionstar.net

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
007angels.com 03p.info
00author.com 6128786.com
1008.su 6hopping.com
214jaluwobapef.cf 7a2rnuey1tw9ar.ru
2pxg8bcf.top 7search.com
5-steps-to-start-business.com 7wind.ru
DomainStatsBot 8xv8.com
aab94f698f36684c5a852a2ef272e031bb.com aa81bf391151884adfa3dd677e41f94be1.com
abtasty.com aaa38852e886ac4af1a3cff9b47cab6272.com
acads.net abwa.tk
acheterviagrafr24.com adserver-e7.com
addblueoff.com.ua adultactioncam.com
adhome.biz advocatemsk.ru
adktrailmap.com afora.ru
aimayangzhi.com agardomains.com
air-edem.ru aksonural.ru
aktivator-windows10.blogspot.com alfa-img.com
alert-jdh.xyz algarveglobal.com
alpharma.net allkrim.com
am15.net allornamenti.com
angkortours.vn anlimebel.kiev.ua
animali.deagostinipassion.it anniemation.com
asrvvv-a.akamaihd.net apiqualitinknet-a.akamaihd.net
auto4style.ru aproposde.com
autobudpostach.club arenda-yeisk.ru
aversis.net artparquet.ru
awaybird.ru asrv-a.akamoihd.net
axbocz.net autoplate.info
aviav.co
awency.com
azartmix.com azartmix.com
backiee.com azbukadiets.ru
biznesschat.net backgroundpictures.net
blacktwhite.com biglistofwebsites.com
bluejays-jerseys.us bioscorp.ru
carabela.com.do brokergid.ru
carloans.com bronzeaid-a.akamaihd.net
cartierreplica.top bumskontakte.org
cartierreplica.xyz buron.pw
casinorewards.info business-suggested.tk
centre-indigo.org.ua buytizanidineonlinenoprescription.blogspot.com
changde.58.com cam-kontakte.org
charmstroy.info cartechnic.ru
checkpagerank.net cartierlove2u.xyz
cleanpcnow.com cartujano-pre.de
clipartnew.com catalogs-parts.com
clothesforcash.com caulderwoodkennels.com
colehaanoutlet.store cayado.snn.gr
com-onlinesupport.site ccjp.eu
coolgramgoods.com civilwartheater.com
copyrightclaims.org collegerentals.com
cutalltheshit.com comissionka.net
dalavia.ru community.allhiphop.com
db.speedup.gdn compliance-ivan.top
delayreferat.ru creditwell.ru
dfwu1013.info cubs-jerseys.us
doxyporno.com d-black.bz
drunkenstepfather.com davebestdeals.com
elektir.ru detalizaciya-tut.biz
elektrozigarette2.webs.com detoxmed24.ru
eloxal.ru directivepub.com
erotik0049.com domain-tracker.com
erotische-geschichten-xxl.com dostavka-v-krym.com
european-torches.ru elektrischezi.canalblog.com
europeanwatches.ru elektrischeziga.livejournal.com
extlinks.com elektrischezigarette2.onsugar.com
eyes-on-you.ga elektrischezigaretten2.beeplog.com
factorynightclub.com eloconcream.blogspot.com
faithe.top emediate.eu
fakehandbags.xyz engines-usa.com
fashion.stellaconstance.co englishdictionaryfree.com
fba-mexico.com essay-edu.biz
filesclub.net evrotekhservis.ru
forex-procto.ru ezigaretteshop2.vefblog.net
forum.doctissimo.fr farmingworm.com
franklinfire.co fickkontaktehobbyhuren.com
free-stock-illustration.com fil.ru
freeseedsonline.com flash4fun.com
freshberry.com.ua flipper.top
fuckmill.com flirt4free.com
fullfileaccess.com fotzen-ficken.com
gabeshop.ru frameimage.org
galaxyflowers.ru freakycheats.com
gobongo.info freenom.link
goldandcard.ru fun2cell.net
gosmeb.ru fusoradio.info
guiadeserraazul.com fxund.us
guod.me game-mmorpg.net
games.kolossale.ru
get-free-traffic-now.com
getdot.ru
goforexvps.com
hahashka.ru
hatedriveapart.com hatedriveapart.com
hdapp1008-a.akamaihd.net hidemyass.com
heygidday.biz hikesearch.net
histock.info hitsbox.info
hollywoodactress.info honyaku.yahoofs.jp
homedecorpicture.us hotgirlhdwallpaper.com
hop.clickbank.net housekuba.org
hotsocialz.com houseofgaga.ru
htmlcorner.com htmlcorner.com
hully.altervista.org humanorightswatch.org
ideoworld.org iblogpress.xyz
iklysha.ml iplusbit.blogspot.co.za
imgata.com istizanidineanopiate.blogspot.com
iminent.com kinky-fetishes.com
in-tandem.co kinosed.net
infostatsvc.com klin3952.ru
innoslicon.com kosynka-games.ru
insider.pro languagecode.com
intellekt21.ru lankarns.com
investsuccess.org laptop-4-less.com
jagg.info lashstudia.ru
janerikholst.se letmacworkfaster.site
je7.us letmacworkfaster.world
jovencitas.gratis linkbux.ru
kakablog.net livingcanarias.com
keyhantercume.com lomza.info
kino-filmi.com louboutinreplica.pw
kinoplen.ru lsp-awak-perikanan.com
laptoper.net lyrster.com
letolove.ru mabdoola.blogspot.com
lifeinsurancekb.com mantramusic.ru
mackeeper-land-672695126.us-east-1.elb.amazonaws.com manve.info
makedo.ru marcogrup.com
maximpartnerspr.com marketingvici.com
medizinreisen.de medkritika.ru
menstennisforums.com meds-online24.com
mex-annushka.ru meet-flirt-dating.com
missvietnam.org metafilter.com
mncrftpcs.com mini.7zap.com
mnogabukaff.net modenamebel.ru
monclerjacketsoutlet.pw my-bc.ru
moncleroutletonline.win
mossmesi.com
mrinsidesales.com
mulberryoutletonlineeu.com
mustat.com
my-aladin.com
mycaf.it mycaf.it
mydirtyhobby.com
mygameplus.com mygameplus.com
myghillie.info myyour.eu
myhitmp3.club nalogovyy-kodeks.ru
mylesosibirsk.ru napalm51.nut.cc
mymobilemoneypages.com netcheckcdn.xyz
mypornfree.ru nicefloor.co.uk
myseoconsultant.com nikitabuch.com
narutonaruto.ru nmrk.ru
navalwiki.info nrv.co.za
nfvsz.com offtopic.biz
no-rx.info oivcvx.website
normalegal.ru online-podarki.com
nucia.biz.ly onload.pw
obsessionphrases.com optitrade24.com
offer.camp paccohichetoti.ml
okmusic.jp pacificair.com
online-zaymy.ru parvezmia.xyz
ooo-olni.ru pastaleads.com
paperwritingservice17.blogspot.ru pharmacyincity.com
paradontozanet.ru picquery.com
park.above.com pinapchik.com
paydayloanslocal.com playtap.us
pearlisland.ru poems.com.ua
pelfind.me polimga.pw
perm-profnastil.ru popserve.adscpm.net
phpdevops.com pornosexrolik.com
picturesify.com
pinkduck.ga
pivka.xyz
placid-rounded-coast.glitch.me
pomoc-drogowa.cba.pl
pornmania.pl
pornoblood.com
pornoted.com pornoted.com
pro-tec.kz pornowarp.info
pxhdwsm.com pourvous.info
py100.ru powitania.pl
ranking2017.ga prombudpostach.com.ua
rating-bestcasino.com proprostatit.com
razyboard.com pts163.ru
realtytimes.com puserving.com
rebelmouse.com quangcaons.com
repeatlogo.co.uk reklamuss.ru
rogervivierforsale.com rock-cafe.info
rozalli.com rossmark.ru
russian-postindex.ru rutor.vip
sabaapress.com s8-nowy-wygraj.comli.com
saddiechoua.com sarf3omlat.com
sbtdesign.co.uk savetubevideo.com
scanner-jack.top sbetodiodnye-lampy.ru
seoexperimenty.ru scanner-jane.top
service.adtech.fr scanner-john.top
sexuria.net scanner-margo.top
scanner-olga.top
scanner-willy.top
search-goo.com
secretscook.ru
segol.tv
selectads.men
seo-smm.kz
serptehnika.ru
sextracker.de
sfd-chess.ru sfd-chess.ru
shopperifymac.com sharebutton.net
sigmund-freud.co.uk sherlock.se
shoesonlinebuy.cn
simplepooltips.com simplepooltips.com
skladvaz.ru site-analyzer.com
sluganarodu.ru sivs.ru
slutloadlive.com skylta.com
snow.nvr163.com socialmadesimple.com
socialbuttons.xyz sohoindia.net
soda.media solnplast.ru
spaceshipad.com speeddream.xyz
statustroll.com spinazdrav.ru
sygraem.com stuff-about-money.com
tarad.com swagbucks.com
tds-advert002.info taximytishi.ru
telefonsex-ohne0900.net td-l-market.ru
templates.radiodigital.co tdsing.ru
terrafootwear.us teencastingporn.com
tfxiq.com threecolumnblogger.com
thegalerie.eu tizanidineandhydrocodone.blogspot.com
thenews-today.info torrent-newgames.com
thetoiletpaper.com treasuretrack-a.akamaihd.net
tizanidinefibromyalgia.blogspot.com trustedhealthtips.com
tizanidineonline.blogspot.com
tksn.ru
tn811.us
tomck.com
top10-way.com
topflownews.com
torrents-tracker.com
tour-line.net
trackzapper.com
truemfilelj.gq
tuberkulezanet.ru tuberkulezanet.ru
twiclub.in tuckpointingmasonrysystems.com
twincitiescarservice.com turizmus.us
unrealcommander.org turvgori.ru
utiblog.fr udsgame.online
vegascosmetics.ru ukrtvir.com.ua
vektorpress.ru validdomain.xyz
vmnmvzsmn.over-blog.com vbtracker.net
vrotike.ru ventelnos.com
vzlomtw.com vereo.eu
wapsite.me video-hollywood.ru
watchmygf.net videochat.life
webmonetizer.net videochaty.ru
wineitudes.wordpress.com vkak.ru
worst-sites.online vksaver-all.ru
ww2awards.info w-journal.ru
x-lime.com wahicbefa31.soup.io
xaijo.com watchmyfb.pl
xfluro.com wheelchairliftsarea.com
xn----ctbigni3aj4h.xn--p1ai wholesalecheapjerseysfree.com
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai windowssearch-exp.com
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai womens-journal.net
xnxx699.com wordpresscore.com
xvideospornoru.com worldlovers.ru
xxlargepop.com wowcasinoonline.ooo
zaim-pod-zalog-krasnodar.ru www.get-free-traffic-now.com
zambini.ru xdoza.com
zeleznobeton.ru xjlottery.com
zootoplist.com xn--oogle-wmc.com
xvideosbay.com
yadro.ru
youradulthosting.com
yousense.info
zdesformula.ru
zhacker.net
zhorapankratov7.blogspot.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.19 0; 10.1.0.221 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3113 ### Version: V4.2022.05.3114
### Updated: Wed May 18 22:01:23 UTC 2022 ### Updated: Thu May 19 10:03:51 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3113 ### Version: V4.2022.05.3114
### Updated: Wed May 18 22:01:23 UTC 2022 ### Updated: Thu May 19 10:03:51 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##