vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 11:40:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.05.3088 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-05-06 10:09:30 +00:00
parent d01cb91a78
commit 759c0e8e00
47 changed files with 96204 additions and 96214 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3087 #### Version: V4.2022.05.3088
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3087 #### Version: V4.2022.05.3088
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.05.3087 #### Version: V4.2022.05.3088
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11890
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.05.3087 #### Version: V4.2022.05.3088
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11894
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11894
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11894
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11894
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.200 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.200 0;
127.0.0.1 0; 127.0.0.1 0;

11894
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.200 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.200 0;
127.0.0.1 0; 127.0.0.1 0;

11894
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.200 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.200 0;
127.0.0.1 0; 127.0.0.1 0;

11894
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3087 3088

11894
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11894
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.230.50.60 0; 20.66.89.32 0;

11894
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.200 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.200 0;
127.0.0.1 0; 127.0.0.1 0;

11894
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.200 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.200 0;
127.0.0.1 0; 127.0.0.1 0;

11894
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.200 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.200 0;
127.0.0.1 0; 127.0.0.1 0;

11894
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.230.50.60 0; 20.66.89.32 0;

11894
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.230.50.60 0; 20.66.89.32 0;

11894
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.200 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Aspiegel Asterias
Extractor BackWeb
Indy Library Bolt
Keyword Density CopyRightCheck
Majestic-SEO Freeuploader
PleaseCrawl MQQBrowser
RepoMonkey RankFlex
SalesIntelligent Surfbot
Web Fetch Thumbor
internetVista monitor sp_auditbot

308
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404checker
360Spider ASPSeek
404enemy Aboundex
80legs
ALittle Client
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
Anarchy99 AhrefsBot
Asterias AiHitBot
Attach Anarchy
AwarioRssBot Apexoo
AwarioSmartBot Atomseobot
BLEXBot BDFetch
BackDoorBot BacklinkCrawler
BackWeb Bitacle
Badass BlackWidow
Bandit Blow
BatchFTP Buck
Battleztar Bazinga CATExplorador
BetaBot
Bigfoot
Black Hole
BotALot
Brandwatch
Bullseye
CSHttp
Calculon Calculon
CazoodleBot CheTeam
CherryPicker CheeseBot
Chlooe ChinaClaw
Citoid
Claritybot Claritybot
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot
Collector Collector
CopyRightCheck Copier
Copyscape
Cosmos
CrazyWebCrawler CrazyWebCrawler
Crescent
Curious
Custo
CyotekWebCopy CyotekWebCopy
DataCha0s DIIbot
DSearch
DTS Agent
Demon
Devil Devil
Digincore Dirbuster
Dispatch Discoverybot
DittoSpyder DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project Domains Project
ECCP/1.0 Dragonfly
EMail Wolf EMail Wolf
EasyDL EasyDL
Ebingbong
Ecxi
EirGrabber
Exabot
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms FHscan
FlashGet Fimap
Flunky Foobot
Fuzz Freeuploader
Fyrebot FrontPage
FyberSpider
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW Genieo
Gigabot Go-Ahead-Got-It
Grabber Gotit
GrapeFX Grafula
HEADMasterSEO
HMView
HTMLparser HTMLparser
HTTrack HTTP::Lite
Haansoft HaosouSpider
Harvest
Heritrix
HybridBot HybridBot
IRLbot IDBTE4M
Iblog IDBot
Id-search
IlseBot IlseBot
Image Sucker
IndeedBot
Indy Library Indy Library
Iria InfoNaviRobot
Iskanie
IstellaBot IstellaBot
JOC Web Spider JetCar
Jbrofuzz JustView
Jetty Kenjin Spider
Jorgee Kinza
Keybot Translation-Search-Machine Kozmosbot
Keyword Density
LWP::Simple LWP::Simple
Lanshanbot Leap
LibWeb LeechFTP
Libwhisker Lftp
LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkWalker LinkWalker
Linkbot LinkextractorPro
LinkpadBot LinkpadBot
LinksManager LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider Litemage_walker
Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MSFrontPage
MQQBrowser
MTRobot MTRobot
Mag-Net Magnet
Mail.RU_Bot Majestic SEO
MarkMonitor
MarkWatch
Mass Downloader
Masscan
Mata Hari Mata Hari
Mediatoolkitbot MeanPath Bot
Microsoft Data Access MegaIndex.ru
Microsoft URL Control
Moblie Safari Moblie Safari
Mr.4x3 Mr.4x3
Msrabot Msrabot
Name Intelligence
Nameprotect Nameprotect
Navroad NearSite
Needle
Nessus
Net Vampire Net Vampire
NetMechanic Netcraft
NetSpider Nettrack
Nibbler Netvibes
Nmap Nikto
Not
Nuclei Nuclei
Octopus
Offline Explorer Offline Explorer
Offline Navigator OnCrawl
OpenVAS OpenLinkProfiler
OrangeBot Openvas
OrangeSpider
OutclicksBot OutclicksBot
OutfoxBot PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageScorer PageGrabber
Panscient PageThing.com
PeoplePal PeoplePal
Petalbot
Picsearch Picsearch
Piepmatz Piepmatz
Pixray
PleaseCrawl PleaseCrawl
ProWebWalker ProWebWalker
PxBroker Probethenet
Psbot
PyCurl PyCurl
RankFlex QueryN Metasearch
RankingBot RSSingBot
RankActiveLinkBot
RankingBot2
Rankivabot
Re-re
ReGet ReGet
Reaper
RebelMouse RebelMouse
RedesScrapy
RepoMonkey RepoMonkey
Ripper Ripper
RocketCrawler
Rogerbot Rogerbot
SEOkicks-Robot
SEOlyticsCrawler
SEOstats SEOstats
SalesIntelligent SalesIntelligent
ScanAlert ScanAlert
ScoutJet
Scrapy
Screaming Screaming
ScrepyBot
Searchestate Searchestate
Semrush SearchmetricsBot
SentiBot SemanticJuice
Seomoz Seomoz
Shodan
Site Sucker Site Sucker
SiteExplorer SiteCheckerBotCrawler
SiteSnagger
Siteimprove Siteimprove
Sitevigil
SlySearch
SmartDownload SmartDownload
Snapbot Snake
Snoopy
SocialRankIOBot SocialRankIOBot
Sottopop Sogou web spider
Stripper Sosospider
SuperBot SpankBot
SputnikBot
Sqlworm
SuperHTTP SuperHTTP
SurveyBot
Swiftbot
Szukacz Szukacz
T8Abot
Teleport Teleport
Telesphorep TeleportPro
The Intraformant Telesoft
TheNomad
Thumbor Thumbor
Toweyabot
Trendiction Trendiction
Trendictionbot
Turingos Turingos
Turnitin
TwengaBot TwengaBot
URLy Warning Typhoeus
URLy.Warning
UnisterBot UnisterBot
VB Project Upflow
VeriCiteCrawler V-BOT
Vagabondo
Virusdie Virusdie
VoidEYE
Voltron Voltron
WBSearchBot WBSearchBot
WEBDAV WPScan
WWW-Mechanize WWW::Mechanize
WWWOFFLE Wallpapers
Wallpapers/3.0 Web Collage
WallpapersHD
WeSEE
Web Enhancer Web Enhancer
Web Fetch WebBandit
WebCollage WebFetch
WebCopier WebWhacker
WebEnhancer
WebReaper
WebZIP WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Website Quester
Webster WebsiteQuester
Whack
Whacker Whacker
Whatweb
Who.is Bot Who.is Bot
WiseGuys Robot WiseGuys Robot
Wotbox Wotbox
Wprecon Wprecon
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot
Zade Zade
Zermelo Zitebot
Zeus
ZoomBot ZoomBot
ZoominfoBot
ZumBot ZumBot
ZyBorg archive.org_bot
adscanner arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider
backlink-check backlink-check
cah.io.community cah.io.community
check1.exe
coccocbot
com.plumanalytics com.plumanalytics
crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j crawler4j
eCatch demandbase-bot
domainsproject.org
facebookscraper facebookscraper
gopher gopher
heritrix
instabid instabid
magpie-crawler magpie-crawler
page scorer mediawords
polaris version muhstik-scan
probe-image-size netEstate NE Crawler
ripz oBot
satoristudio.net plumanalytics
scan.lol
seobility
sexsearcher
sitechecker.pro sitechecker.pro
siteripz sp_auditbot
sogouspider tAkeOut
sysscan
trendiction.com trendiction.com
webmeup-crawler webgains-bot
webpros.com webpros.com
zauba.io zgrab

286
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 80legs
360Spider ADmantX
404enemy ASPSeek
Aboundex Abonti
Aboundexbot Acunetix
AhrefsBot AfD-Verbotsverfahren
Alexibot AiHitBot
AllSubmitter Alligator
AlphaBot AlphaBot
Anarchie
Anarchy Anarchy
Ankit Anarchy99
Anthill Aspiegel
Asterias Asterias
Atomseobot Atomseobot
Attach BDFetch
BDCbot
BLEXBot BLEXBot
BackDoorBot
BackWeb
Backlink-Ceck Backlink-Ceck
Badass
Barkrowler
Battleztar Bazinga Battleztar Bazinga
BetaBot BetaBot
Bitacle
Blackboard Blackboard
BlowFish BlowFish
Boardreader
Bolt
Brandprotect Brandprotect
Bullseye Brandwatch
CATExplorador Buck
Buddy
BuzzSumo
CCBot
CODE87 CODE87
Calculon CensysInspect
CheeseBot CherryPicker
Chlooe Citoid
Claritybot Cliqzbot
Cocolyzebot
Copier Copier
Craftbot Craftbot
CrazyWebCrawler
Crescent
CrunchBot CrunchBot
Custo Custo
CyotekWebCopy
DIIbot
DSearch
DTS Agent
Demon
Deusu
Devil Devil
Discobot Dirbuster
Discoverybot Disco
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DomainCrawler
DomainSigmaCrawler DomainStatsBot
Dragonfly
Drip Drip
EMail Siphon EasyDL
EirGrabber Exabot
EroCrawler
Express WebPictures
ExtLinksBot
Extractor Extractor
ExtractorPro ExtractorPro
EyeNetIE EyeNetIE
Ezooms
FHscan
Fimap
FlashGet
Flunky Flunky
Foobot
FyberSpider FyberSpider
Fyrebot
GT::WWW
Genieo Genieo
GermCrawler Getintent
Gigabot Gigabot
Go!Zilla
GoZilla GoZilla
Gotit GrabNet
Grabber
GrapeFX
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO HTMLparser
HTTrack
Haansoft Haansoft
HonoluluBot HybridBot
IRLbot
Iblog Iblog
Id-search IlseBot
Image Fetch Image Fetch
Image Sucker Image Sucker
IndeedBot
InterGET InterGET
IstellaBot Internet Ninja
JamesBOT Iskanie
JennyBot JennyBot
Jetty Jetty
JikeSpider JikeSpider
JustView Joomla
Jorgee
Kinza Kinza
Kozmosbot
LNSpiderguy LNSpiderguy
Lanshanbot LWP::Simple
Larbin Larbin
LeechFTP Leap
LeechGet
Lftp Lftp
LieBaoFast Libwhisker
Lightspeedsystems Likse
LinkScan LinkpadBot
LinkextractorPro LinksManager
LinqiaRSSBot LinqiaRSSBot
Lipperhey Lipperhey Spider
Lmspider Litemage_walker
MQQBrowser MFC_Tear_Sample
MIDown tool
MSIECrawler MSIECrawler
MTRobot MTRobot
Magnet
Majestic SEO Majestic SEO
Majestic12
MarkWatch MarkWatch
Mass Downloader
Mata Hari Mata Hari
Meanpathbot
MegaIndex.ru MegaIndex.ru
MicroMessenger Microsoft Data Access
Microsoft URL Control
Mister PiX Mister PiX
Mojeek Moblie Safari
MolokaiBot Morfeus Fucking Scanner
Mr.4x3
Musobot Musobot
Nameprotect NearSite
NetAnts Needle
Nessus
NetMechanic
NetSpider NetSpider
NimbleCrawler NextGenSearchBot
Ninja Nikto
Nmap
Not Not
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler OrangeBot
Openfind
Openvas
OutfoxBot OutfoxBot
PECL::HTTP PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageScorer PageAnalyzer
Pandalytics PageGrabber
Petalbot Pavuk
PeoplePal
Pi-Monster
Picscout Picscout
Picsearch
PictureFinder
Pimonster Pimonster
ProPowerBot Pixray
ProWebWalker ProWebWalker
Pump Psbot
PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler
RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankingBot RankingBot
Rankivabot Re-re
RankurBot RealDownload
Reaper
RebelMouse RebelMouse
Ripper RedesScrapy
SBIder RepoMonkey
RocketCrawler
SEOlyticsCrawler SEOlyticsCrawler
SMTBot SISTRIX
SalesIntelligent
Scanbot Scanbot
ScoutJet ScoutJet
Screaming ScreenerBot
Searchestate SearchmetricsBot
SemanticJuice
Semrush Semrush
SeoSiteCheckup SemrushBot
SeobilityBot
Seomoz Seomoz
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer SiteSnagger
Sitevigil Siteimprove
SmartDownload SlySearch
Snake
Snapbot Snapbot
SputnikBot SocialRankIOBot
Sociscraper
Spammen
Spbot
Sqlmap Sqlmap
Sqlworm Sqlworm
Sqworm Steeler
Stripper Stripper
Sucker
SuperBot SuperBot
SuperHTTP SuperHTTP
Surfbot
SurveyBot SurveyBot
Suzuran T8Abot
Teleport
Telesphoreo Telesphoreo
Thumbor Telesphorep
TheNomad
Toweyabot Toweyabot
Tracemyfile Tracemyfile
Trendictionbot Trendictionbot
True_Robot True_Robot
Turingos
TurnitinBot TurnitinBot
Typhoeus Typhoeus
URLy Warning URLy.Warning
UnisterBot VB Project
V-BOT
VCI VCI
Vagabondo Vagabondo
VoidEYE VelenPublicWebCrawler
VidibleScraper
Voltron Voltron
WASALive-Bot WASALive-Bot
WBSearchBot
WISENutbot WISENutbot
WPScan
WWW-Collector-E
WWW::Mechanize WWW::Mechanize
WallpapersHD WWWOFFLE
Web Auto
Web Fetch Web Fetch
Web Fuck Web Fuck
Web Sauger
WebBandit
WebFetch WebFetch
WebFuck WebFuck
WebGo IS WebImageCollector
WebReaper WebSauger
WebSucker WebSucker
WebZIP WebWhacker
WebsiteQuester Website Quester
Whacker WebsiteExtractor
WinHTTrack Webster
Whatweb
Who.is Bot
Wonderbot Wonderbot
Wprecon
Xaldon WebSpider Xaldon WebSpider
Zermelo
Zitebot Zitebot
ZmEu ZmEu
ZumBot
adscanner adscanner
archive.org_bot
arquivo.pt
backlink-check backlink-check
cah.io.community
check1.exe check1.exe
coccocbot cognitiveseo
demandbase-bot crawler.feedback
evc-batch crawler4j
dataforseo.com
domainsproject.org
gopher gopher
instabid heritrix
internetVista monitor mediawords
isitwp.com muhstik-scan
iubenda-radar
lwp-request
meanpathbot
netEstate NE Crawler netEstate NE Crawler
probe-image-size oBot
pcBrowser
ripz
s1z.ru
satoristudio.net satoristudio.net
scan.lol seocompany.store
seoscanners seoscanners
sitechecker.pro
siteripz siteripz
sogouspider spyfu
sp_auditbot sysscan
trendiction.com tAkeOut
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webgains-bot
webmeup-crawler
xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
allstatesugarbowl.org acheterviagrafr24.com
boom-celebs.com biznesluxe.ru
extremepornos.net bumskontakte.org
gsasearchengineranker.top kit-opt.ru
lyrics.home-task.com raschtextil.com.ua
sexad.net stevemonsen.com
tracfone.com telesvoboda.ru
wfb.hatedriveapart.com templates.franklinfire.co
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai topmira.com
zeroredirect8.com yourads.website

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0lovespells0.blogspot.com 0daymusic.org
1001watch.com.ua 1flag.co.za
11pikachu.ru 24-ak.ru
123any.com 24videos.tv
1bet.com 24x7-server-support.site
2000k.ru 2rich4bitches.com
2728fb936f0.com 3-letter-domains.net
2pxg8bcf.top 4inn.ru
4ur.click 4pp13.com
5-steps-to-start-business.com
5000-cotydzien.com 5000-cotydzien.com
6hopping.com
777-club.ru 777-club.ru
7search.com 883zy.com
8si.ru 91abcw.com
above.com aa08daf7e13b6345e09e92f771507fa5f4.com
adconscious.com aad1f4acb0a373420d9b0c4202d38d94fa.com
ads-seo.men abcdeg.xyz
adtech.us acunetix-referrer.com
advancedsoftwaresupport.com adanih.com
advertex.info advokateg.ru
advertisingtag.net affrh2015.com
akita.kz aktivator-windows10.blogspot.com
all4invest.ru alfapro.ru
alltheviews.com all-streaming-media.com
amazon-seo-service.com all4wap.ru
analnoeporno.tv allcredits.su
appfixing.space alldezservice.kz
apple.com-webbrowsing-security.review alldubai.biz
approvedlocal.co.za allnews24.in
arabsexxxtube.com allwidewallpapers.com
arcadepages.com alphavisions.net
arewater.com ampower.me
artel116.ru anonymousfox.co
asiengirls.net apiappenableinfo-a.akamaihd.net
asophoto.com apibrowsesmartne-a.akamaihd.net
asrvvv-a.akamaihd.net apidiamondatanet-a.akamaihd.net
atlasvkusov.ru apple.com-cleaner.systems
auto.rusvile.lt applyneedy.xyz
avto-oligarh.ru arate.ru
ayanaspa.com arenanews.com.ua
azartniy-bonus.com artavenuegardenstudios.com
aruplighting.com
arvut.org
as5000.com
asociatia-tipografilor-transilvania.ro
asrvrep-a.akamaihd.net
auto-moto-elektronika.cz
autobudpostach.club
autodo.info
avek.ru
azte.ch azte.ch
babyguns.ru b2b-lounge.com
baditri.com backlinkwatch.com
baladur.ru
baltgem.com baltgem.com
bannerpower.com bazaronline24.ru
battleforupdating.site beachpics.com
bbsoldes.fr belinvestdom.by
beachdriveblog.com bestmarriages.com
beatifullandscape.co bestsexyblog.com
beauby.ru bestwebsiteawards.com
beauty-b0x.pl bestwrinklecreamnow.com
beauty-bracelet.com bharatdefencekavach.com
bestmouthwash.club binomo.kz
bestplacetobuyeriacta.jw.lt bjgugu.net.ua
bestpriceninja.com blogs.rediff.com
bibys.com bluesman.nu
big-boards.info bmw-ark.ru
birzha-truda.eu
bodybuilding-shop.biz
bolegapakistan.com
bombla.org bombla.org
brainxs.us bongiornos.info
braip.com.br boo-arts.com
brianhenry.co bpro1.top
bristolhotel.com bradleylive.xyz
brokergid.ru brandedleadgeneration.com
btnativenav.com bravegirlsclub.com
buehne-fuer-menschenrechte.de brewdom.ru
bus-offer.com browserprotecter.com
business-suggested.tk buckspar.gq
buyessaynow.biz burger-imperia.com
buyviagraa.blogspot.com business-made-fun.com
c-gainsbourg.com bytimedance.ru
cah.io.community c-english.ru
californianews.cf cam-kontakte.org
candelluxsklep.pl carapics.com
candycrushshop.com cartierlovestore.com
canvas.gloverid.site cartiertrinity.xyz
cars-modification.net cbcseward.com
carsplate.com cdnnetwok.xyz
cb.iphantom.com cenoval.ru
cfsrating.sonicwall.com cheapjerseysfootballshop.com
cheapjerseysa.com cheapsergiorossi.online
chee-by.biz cheapwebsitehoster.com
cialis-samples.com cheatcode-lita12.rhcloud.com
cialischsrx.com cialischstgerts.com
ciceros.co cicaki.net
cjmc.info clickbank.net
cocyq.inwtrade.com clips.ua.ac.be
com-supportcenter.website coaching-netz.info
communisave.co.za coinsspb.com
compliance-brian.top cometorussia.net
compliance-john.top compliance-don.top
compliance-fred.top
compliance-ivan.top
conboy.us
conocer-sanabria.com
construmac.com.mx
converse.ddsoldes.fr converse.ddsoldes.fr
cookingmeat.ru cool-wedding.net
coop-gamers.ru cordstrap.cc
covetnica.com counter.bloke.com
creatives.adbetclickin.pink counter.yadro.ru
crynet.cc creams.makeforum.eu
cs-passion.pl
customchocolate.business-for-home.com
cutt.us
cvety24.by cvety24.by
d-black.bz dairyindia.in
d2jsp.org daretodonate.co
danceuniverse.ru darmebel.com.ua
deplim.com data-ox.com
detalizaciya-tut.biz depositfiles-porn.ga
devochki.top dirtpics.pw
dimfour.com divku.ru
dipstar.org docs4all.com
dmmspy.com dojki365.online
domaincheck.io domain-submit.info
dorratex.tn downvids.net
druzhininevgeniy63.blogspot.com e-biznes.info
e-c.al eandsgallery.com
eachdayisagift.review earl-brown.info
ed-shop01.ru ekobata.ru
egvar.net emergencyneeds.org
eldiariodeguadalajara.com en.altezza.travel
elektrischezigarette2.onsugar.com eshop4u.jp
elektrischezigaretten1.blogse.nl espaceinventoristes.com
empirepoker.com evogarage.com
erectile.bid executehosting.com
erotische-geschichten-xxl.com extremez.net
essayassist.com ezigarettekaufen2.yolasite.com
etm-consult.de faithe.top
event-tracking.com fandlr.com
exhibitionplus.eu faptitans.com
exoneration-project.us farmprofi.net
ezigarettenkaufen1.deviantart.com ferieboligkbh.dk
falopicm.pw financeloan.us
fashionavenuegame.com financetip.eu
fast-torrent.ru findacheaplawyers.com
fastfixing.tech finemanteam.com
fefo.gdn flowwwers.com
fenoyl.batcave.net folowsite.com
film-one.ru
findpik.com
finstroy.net
firesub.pl
fordsonmajbor.cf
forex21.ru
forminecrafters.ru forminecrafters.ru
formseo.com franecki.net
forpackningsutveckling.se free-floating-buttons.com
forum.tvmir.org free-laptop-reward.com
foxweber.com freejabs.com
franklinfire.co freeuploader.ml
free-share-buttons.blogspot.com freevpn.space
freewlan.info frequiry.com
freshdz.com futbolkisales.ru
freshmac.space fxtips.ru
freshsuperbloop.com g33.org
fsakhalin.ru gameonasia.com
fsalas.com gaytube.com
fullfileaccess.com gazoblok.net.ua
funtoonez.com geckoandfly.com
fx-brokers-review.com gelezki.com
fyxabomiw.ru genetworx.com
gay-file.com gepezz.info
gaz-voshod.ru gidonline.one
geile-lelly.eu gigapeta.com
getlaid-xxxhookupdirect.com girlgamerdaily.com
gigixo.com girls-ufa.ru
globalsurfari.com gkvector.ru
goforexvps.com glall.ru
gok-kasten.net globatur.ru
go2jump.org
goldandcard.ru
golden-catalog.pro golden-catalog.pro
googlemare.com goodwriterssales.com
gotcher.us gosreg.amchs.ru
greatdealshop.com gotwebsite1.com
gsasearchenginerankerhelp.com gowreckdiving.com
gta-top.ru gz2.bbsoldes.fr
gungamesz.com hitcpm.com
gwebtools.com holidaypics.org
gwhwpxbw.bloger.index.hr homeart.space
hanwei.us hotel-mkad.ru
hasshe.com hotenergy.ru
hdpixent.com houseofgaga.ru
hdseriale.pl hoverboard360.nl
hexpilot.com howtowhitenteethfast.xyz
histock.info hoztorg-opt.ru
hitmuzik.ru hundejo.com
homedecoguide.info
hop.clickbank.net
hotblog.top
hotgirlhdwallpaper.com
hothot.ru
hptwaakw.blog.fc2.com
htmlcorner.com
hubbble.com
humanorightswatch.org
hybrid.ru hybrid.ru
hyiphunter.org hystersister.com
ideawheel.com iklysha.ml
ihtec2019.org imgarit.pw
iloveitaly.ru imperiafilm.ru
ilovevitaly.xyz inboxdollars.com
imagefinder.site infobanks.ru
imagui.eu integritylandscapeservices.com
img.wallpaperstock.net io9.com
indo-export.ru iomoio.net
infogame.name iplogger.org
infoupdate.org istizanidinelikexanax.blogspot.com
invivo.hu
iqoption.pro
istizanidineanopiate.blogspot.com
itis4you.com itis4you.com
itsdp3.com itronics.ca
jackwolfskinoutlet.online ixora.pro
javrip.net jagg.info
jjbabskoe.ru jav-fetish.com
johnnyhaley.top jerseysbizwholesalecheap.com
joingames.org jerseyschinabizwholesale.us
jurajskie.info jerseywholesaleelitestore.com
justkillingti.me jofucipiku.tk
kadashihotel.com johannesburgsingles.co.za
karpun-iris.ru jose.mulinohouse.co
keki.info joyceblog.top
keywordbasket.com junglenet-a.akamaihd.net
kiinomaniak.pl kamen-e.ru
kinotorka.ru kartiniresto.com
konica.kz kazrent.com
koopilka.com kimcurlrvsms.com
koptims.tiu.ru kino-filmi.com
kw21.org kino-rating.ru
kwzf.net kinohall.ru
ladov.ru kupiproday.com.ua
landscapebackgrounds.blogspot.com kvartira-sutochno.com
laptoper.net kvartiry-remont.ucoz.ru
layola.biz.tc landreferat.ru
le-clos-des-alouettes.com lankarns.com
lesbian.xyz larger.io
lifeinsurancekb.com leboard.ru
livefixer.com legalrc.biz
livingcanarias.com letolove.ru
lordzfilmz.me lezbiyanki.net
louboutinreplica.pw lflash.ru
lowephotos.info livesex-amateure.info
lubetube.com loftdigital.eu
lol-smurfs.com
lonerangergames.com
lostaruban.com
lsp-awak-perikanan.com
luckyshop.net.ua luckyshop.net.ua
lutherstable.org m3gadownload.pl
madot.onlinewebshop.net mabdoola.blogspot.com
magazintiande.ru mackeeper-center.club
magicalfind-a.akamaihd.net mafa-free.com
makemoneyonline.com makis.nu
malwareremovalcenter.com
manifestyourmillion.com
mansparskats.com
marcowebonyodziezowe.pl marcowebonyodziezowe.pl
marmotstore.online martlinker.com
mcar.in.ua mathgym.com.au
meble-bogart.info med-zdorovie.com.ua
meccadumps.net medmajor.ru
medispainstitute.com.au medtherapy.ru
meetingrainstorm.bid meduza-consult.ru
michaelkorsoutlet.store mhtr.be
microsoftportal.net michellblog.online
midst.eu microstatic.pl
minecraft-neo.ru militarysale.pro
mini.7zap.com mindeyegames.com
mirzonru.net mirobuvi.com.ua
misterjtbarbers.com mister-shop.com
mixx.com mistr-x.org
mjchamonix.org mmofreegames.online
moje-recenze.cz mmog-play.ru
monclerboots.xyz mobile-appster.ru
monclercheap.xyz mobile.ok.ru
monsterdivx.tv mobilemedia.md
moonci.ru mojpreskumanie.com
moscow-clining.ru monclerjacketsoutlet.win
moviemail-online.co.uk monetizationking.net
moyakuhnia.ru moneyteam24.com
mp3ritm.top morlat.altervista.org
mrsdalloways.com mriyadh.com
musicdaddy.net mrpornogratis.xxx
musicpro.monster msk-diplomat.com
mustwineblog.com mulberryoutletonlineeu.com
muycerdas.xxx
mxgetcode.com
my-aladin.com my-aladin.com
my-bc.ru my-floor.in.ua
myanyone.net mycouponizemac.com
mybinaryoptionsrobot.com mydownloadengine.com
myiptest.com myindospace.com
mymercy.info mylesosibirsk.ru
mypornfree.ru myshopmatemac.com
naperehresti.info mzdish.site
narkologiya-orel.ru nakozhe.com
nastydollars.com namecrumilchlet.tk
negociosdasha.com narkologiya-peterburg.ru
nero-us.com neks.info
netallergy.ru ningessaybe.me
new-apps.ru
ngps1.ru
niroo.info niroo.info
novodigs.com novatech.vn
nvssf.com o-o-11-o-o.com
o-o-6-o-o.com oballergiya.ru
oconto.ru obiavo.su
ofanda.com obuv-kupit.ru
officedocuments.net okout.ru
onclkads.com olympescort.com
one-gear.com onescreen.cc
one.net.in online-zaymy.ru
onemantrip.com onlinewritingjobs17.blogspot.ru
onetravelguides.com onload.pw
ons-add.men onthemarch.co
oren-cats.ru orion-code-access.net
p-business.ru ororodnik.goodbb.ru
pacificair.com ovirus.ru
paighambot.com oz-offers.com
paintingplanet.ru ozas.net
palvira.com.ua
paperwritingservice17.blogspot.ru paperwritingservice17.blogspot.ru
parajumpersoutlet.online par-fallen.ga
pdfprof.com pastaleads.com
pepperstyle.ru pavlodar.xkaz.org
pertlocogasilk.tk paydayonlinecom.com
petedrummond.com pcboa.se
pflexads.com petsblogroll.com
phelissota.xyz
photostudiolightings.com
pipki.r.acdnpro.com
pirateday.ru pirateday.ru
piter.xrus.org pk-pomosch.ru
playtap.us plastjulgranar.se
pogosh.com platesauto.com
pojdelo.weebly.com
polcin.de polcin.de
pornoelita.info pomoc-drogowa.cba.pl
poshiv-chehol.ru poppen-nw.net
powc.r.ca.d.sendibm2.com porn-w.org
porn555.com
pornobrazzers.biz
pornogad.com
pornosee.info
pornosmola.info
pozdrawleniya.ru pozdrawleniya.ru
predmety.in.ua predmety.in.ua
prod2016.com predominant-invent.tk
pro-poly.ru
producm.ru
prohoster.info
projefrio.com.br
promalp-universal.ru
promgirldresses.xyz
prostitutki-kharkova.org
prostitutki-odessa.org
prpops.com
pts163.ru
pulse33.ru
purplesphere.in purplesphere.in
purplestats.com pxhdwsm.com
pussysaga.com raisedseo.com
quick-seeker.com realting-moscow.ru
real-time-analytics.com recipedays.com
realtytimes.com recordpage-a.akamaihd.net
redirlock.com reklamuss.ru
refudiatethissarah.info relayblog.com
remont-fridge-tv.ru
remote-dba.de
replica-watch.ru replica-watch.ru
replicalouboutin.xyz replicalouboutin.xyz
revistaindustria.com retreatia.com
ritlweb.com rewardit.com
rockma.se richinvestmonitor.com
roleforum.ru robotixix.com
rome2rio.com rockprogblog.com
ru-dety.ru roofers.org.uk
rubbed.us rootandroid.org
ruscopybook.com rosbalt.com.ua
russia-tao.ru rossmark.ru
s.lollypopgaming.com runofilms.ru
saitevpatorii.com rvi.biz
sarm.tk sajatvelemeny.com
satoristudio.net sammlungfotos.online
sc-specialhost.com samo-soznanie.ru
scanner-brian.top samoiedo.it
scanner-jane.top samolet.fr
scanner-josh.top sbprabooks.com
scanhub.ru
scanner-barak.top
scanner-willy.top scanner-willy.top
searchengineranker.email searchimage.co
seo-tools-optimizing.com searchmywindow-a.akamaihd.net
seo18.su secret.xn--oogle-wmc.com
seojokes.net semprofile.com
serdcenebolit.com semxiu.com
serpstat.com senger.atspace.co.uk
servisural.ru seotoolsagency.com
serw.clicksor.com servicecenter.co.ua
sexflirtbook.com sevendays.com.ua
sexpartygirls.net sevenstars7.com
sexstream.pl sextracker.com
shama-rc.net silvermature.net
shoplvlv.us simple-share-buttons.com
shoppytoolmac.com sisiynas.ru
sibtest.ru sispe.com.br
sim-service.net site-speed-check.site
skuteczna-dieta.co.pl
slimcdn.com
smailik.org
smallseotools.com
smartbalanceworld.com
smichovbike.cz
soblaznu.net soblaznu.net
soc-proof.su social-button.xyz
social-search.me softonicads.com
socialbutton.xyz sophang8.com
socialseet.ru sotechco.co
soft-program.com souvenirua.com
sonata-arctica.wz.cz space-worry.ml
sprttrack.com spb-plitka.ru
spyfu.com spb.ru
standardchartered-forex.com specialfinanceoffers.com
stanthonyscatholicchurch.org sportbetfair.com
stardevine.com srecorder.com
stariy-baku.com srgwebmail.nl
startwp.org stard.shop
statistici.ro starwars.wikia.com
statustroll.com statoutlook.info
stmassage.ru steame.ru
stop-zavisimost.com stromerrealty.com
stroyplus.ru suggest-keywords.com
sugarlyflex.pw sugvant.ru
sundrugstore.com sunflowerdrawingpaintings.blogspot.com
superkanpo.com superoboi.com.ua
supers.com.ua supers.com.ua
survival.betteroffers.review svtrd.com
swagbucks.com swimpool.ca
syvertsen-da.no swinger-mobil.net
szucs.ru swinon.site
taqywu51.soup.io symbaloo.com
tastyfoodideas.com taboola.com
teastory.co tam-gde-more.ru
teesdaleflyballclub.co.uk taranerymagesswa.blogspot.com
tgtclick.com tech4master.com
thaismartloan.com tedy.su
the-trader.net testingads.pro
theheroes.ru thegolfclub.info
thenews-today.info theguardlan.com
thetardistimes.ovh theprofitsmaker.net
timer4web.com tizanidineandhydrocodone.blogspot.com
titelhelden.eu tizanidinecost.blogspot.com
tizanidinedosage.blogspot.com tizanidinenarcotic.blogspot.com
tomatis.gospartner.com tizanidineoral.blogspot.com
tooplay.com tizanidinevsvalium.blogspot.com
tootoo.to tmearegion26.com
topappspro.com tmm-kurs.ru
topclickguru.com todohr.com
topflownews.com token-lab.org
topquality.cf top-karkas.ru
topvidos.ru top250movies.ru
torrentred.games topads.men
tozup.com tpu.ru
tracksurf.daooda.com
trackzapper.com trackzapper.com
traderzplanet.in tracxn.com
traffixer.com traffic2cash.org
triumf-realty.ru
trk-4.net
truck-addzilla.life
truck-rece.life truck-rece.life
trudogolik.net trucri.me
tupper-posuda.ru tubeoffline.com
tuberkuleznik.ru
turist-strani.ru
turizm.bz turizm.bz
uk-zheu20.ru twittrading.com
undergroundcityphoto.com uamtrk.com
unionmarkt.de ugogo.info
unitygame3d.com unimodemhalfduplefw.pen.io
uogonline.com urll.eu
usacasino.com uroki.net
vandrie-ict.nl v24s.net
varikozdok.ru vadimkravtcov.ru
vavilone.com vepad.com
vbtracker.net via-gra.webstarts.com
veerotech.com video-hollywood.ru
velen.io videosbox.ru
video-chat.in videositename.com
videochat.tv.br vitalads.net
vigrx-original.ru viteonlusarezzo.it
vikistars.com vk-mus.ru
vipms.ru vkontarkte.com
vkontaktemusic.ru
vladimir.xrus.org
voloo.ru
vpnmouse.com
vremya.eu vremya.eu
vse-pesni.com
vseigry.fun vseigry.fun
vzube.com vtc.pw
webcamdevochka.com vut.com.ru
webmonetizer.net vzubah.com
webtherapy.ru wallpaperstock.net
weburok.com wdrake.com
wemedinc.com webmasters.stackexchange.com
white-truck.life wejdz-tu.pl
wikes.20fr.com wetgames.ru
williamrobsonproperty.com whatsupinfoley.com
wrz0iuebwhp5fg.freeddns.com wineration.com
xaijo.com witclub.info
xfluro.com xjrul.com
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai xmronta.com
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn----9sbubg3ambdfl1j.xn--p1ai xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--e1aggki3c.xn--80adxhks xn--80ajbshivpvn2i.xn--p1ai
xnxx699.com xn--e1afanlbnfckd7c3d.xn--p1ai
xvideosbay.com xolodremont.ru
ynymnwbm.bloger.index.hr xportvusbdriver8i.snack.ws
yougetsignal.com yeartwit.com
youjizz.vc youradexchange.com
youtuhe.com yourdesires.ru
yugk.net youtube-downloader.savetubevideo.com
zarajbuilders.com yugo-star.ru
zatjmuzu.info zapiszto.pl
zazagames.org zarenica.net
zigzog.ru zeroredirect8.com
zionstar.net zigarettenonlinekaufen1.bloog.pl
zixizop.net.ru
znakom.sibtest.ru znakom.sibtest.ru
zoogdiesney.com
zrus.org
zscaler.net

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
2drittel.de 007angels.com
3rbseyes.com 100dollars-seo.com
abbanreddy.com 123locker.com
abwa.tk 3-letter-domains.net
aerogo.com 404.mn
ahhjf.com 4inn.ru
alphaforum.ru abtasty.com
alveris.ru admitad.com
an-donut.com adverster.com
bannerpower.com ageofclones.com
battlecarnival.su alvtank.se
beauty-things.com apartmentratings.com
best-way.men arquapetrarca.info
bet365.com artel116.ru
bingoporno.com asiengirls.net
bombla.org ausmepa.org.au
bongacams.com autolombard-krasnodar.ru
book-bets.com automobile-spec.com
brainboosting.club avon-severozapad.ru
bravegirlsclub.com backlinkwatch.com
brimstonehillfortress.org berdasovivan.ru
brus.city betterscooter.com
cacheimages.com bin-brokers.com
canacopegdl.com bio-japan.net
cartierring.xyz bio.trade-jp.net
cb.iphantom.com blackplanet.com
celebsopics.com blagovest-med.ru
cezartabac.ro bolitgorlo.net
cfsrating.sonicwall.com bonzbuddy.com
cheapestjerseys-wholesale.com bookhome.info
chocolateslim-en-espana.com bradleylive.xyz
cigarpass.com browserprotecter.com
classicakuhni.ru businesxxl.com
clickbank.net buyantiviralwp.com
clickpapa.com buypanicdisorderpill.com
cocolyze.com buzz-porno.info
cocyq.inwtrade.com buzzonclick.com
bvlgariring.xyz
calc-for-credit.ru
calvet.altervista.org
carfax.com.ua
cartujano-pre.de
censys.io
chatroulette.life
cheapbarbour.online
cheapmoncler.pw
chee-by.biz
cherrypointplace.ca
citysecurity.nu
classiquebijoux.ru
clickzzs.nl
clippingphotoindia.com
coinsspb.com coinsspb.com
com-secure.download coldfilm.ru
communisave.co.za compliance-checker-7.info
compliance-george.xyz compliance-fred.xyz
consorzioilmosaico.org compliance-willy.top
coolingoods.com coolgamechannel.com
credit-online.ws counter.bloke.com
credit-respect.ru creams.makeforum.eu
delta-line.men creditwell.ru
dev.citetick.com customcatchcan.com
digest-project.ru dellalimov.com
dlya-android.org depositfiles-porn.ga
dnepropetrovsk.mistr-x.org deshevo-nedorogo.ru
dollartree.info dgfitness.co
domain.webkeyit.com dividendo.ru
domik-derevne.ru dogoimage.com
eaplay.ru domaincdn.xyz
ekspertmed.com dvervmoskvu.ru
elektir.ru easyshoppermac.com
enews.tech educontest.net
englate.com ekosmetyki.net.pl
enthuse.computernetworksonline.com elektrozigarettekaufen1.devhub.com
eropho.net erectile.bid
escort-russian.com estelight.ru
eshop.md etm-consult.de
este-line.com.ua exct.net
evidencecleanergold.com executehosting.com
ezigarettenkaufen2.dreamwidth.org ezigarettekaufen2.blox.pl
faithe.top ezigarettenkaufen1.pagina.gr
fertilitetsradgivningen.se ezigarettenshop2.postbit.com
filmidivx.com fba-mexico.com
forum20.smailik.org fickluder69.com
frameimage.org filerockstar298.weebly.com
free-video-tool.com fishingwholesale.us
freshmac.space fm-upgrade.ru
frivgame250.com fok.nl
fsakhalin.ru forum.doctissimo.fr
fun2cell.net foxinsocks.ru
gaz-voshod.ru gamerscorps.com
germes-trans.com gok-kasten.net
get-free-traffic-now.com gorodservis.ru
ggiaro.com greenshop.su
girlsfuckdick.com growmyfunds.ca
goblacked.com hamacapty.com
guidefs.ru hawaiielectriclight.com
guod.me hchha.com
hacktougroup.ru hdseriale.pl
hcate.com heartofpayne.xyz
hothot.ru hebr.myddns-flir.com
hyiphunter.org helicalpile.us
inboxdollars.com hobild.net
ip-guide.com hostnow.men
iqupdatetmz.win hreade.com
irkutsk.online-podarki.com huimin764128.com
javcoast.com hully.altervista.org
jennyfire.ru hybrid.ru
joessmogtestonly.com i4track.net
iccornacircri.cf
ico.re
innodgfdriverhm.aircus.com
insomniagamingfestival.com
instasexyblog.com
isitpaleo.info
istizanidinelikexanax.blogspot.com
iwanttodeliver.com
jasonpartington.com
joy-penguin.com joy-penguin.com
jrcigars.com junketjuice.blogspot.com
kazan.zrus.org justkillingti.me
khafre.us karachev-city.ru
kirov.zrus.org karusel-market.ru
kiwe-analytics.com keki.info
knigonosha.net keywordblocks.com
krassh.ru kino-fun.ru
kvartiry-remont.ucoz.ru krafte.ru
krasnodar.xrus.org
laapp.com
lafourmiliaire.com lafourmiliaire.com
lafriore.ru larose.jb2c.me
lanasshop.ru letslowbefast.site
ldrtrack.com lineavabit.it
letmacworkfaster.site linkbuddies.com
lexaprogeneric.link liupis.com
link.ac longgreen.info
linkpulse.com longlifelomilomi.info
locationdesaison.com losangeles-ads.com
loginduepunti.it m.b00kmarks.com
love-baby.cz marcowebonyodziezowe.pl
machicon-akihabara.info marsgatan.com
magnetpress.sk mcnamaratech.com
makedo.ru mesbuta.info
maximilitary.ru mir-business-24.ru
meendo-free-traffic.ga mirmedinfo.ru
members.ghanaweb.com mirobuvi.com.ua
misandesign.se mobplayer.ru
mojaocena.com muztops.ru
money-every-day.com mxgetcode.com
morf.snn.gr my-bc.ru
muz-shoes.ru nac-bearings.ru
myfreetutorials.com namenectar.com
novgorod.xrus.org newhairstylesformen2014.com
obiavo.by normalegal.ru
obsessionphrases.com noscrapleftbehind.co
oconto.ru notaria-desalas.com
ok.ru notasprensa.info
omgtnc.com o-o-11-o-o.com
onlinebay.ru of-ireland.info
onlinemeetingnow.com online.ktc45.ru
openlibrary.org ons-add.men
openmediasoft.com orion-code-access.net
optikremont.ru
ossmalta.com
otbelivanie-zubov.com
parfusale.se
pathwhelp.org
photochki.com photochki.com
pillscheap24h.com pinsdaddy.com
piski.top
plastgran.com plastgran.com
polybuild.ru platesauto.com
pons-presse.com pogodnyyeavarii.gq
pornonik.com pomoc-drogowa.cba.pl
pornotubexxx.name pornoklad.net
preparevideosafesystem4unow.space poshiv-chehol.ru
prod2016.com pozdravleniya-c.ru
prostitutki-belgoroda.org profitwithalex.info
pshare.biz prombudpostach.com.ua
psvita.ru propranolol40mg.blogspot.com
purplesphere.in putevka24.ru
pussyspace.net razamicroelectronics.com
qor360.com registry-cleaner.net
quebec-bin.com rucrypt.com
quickloanbank.com saddiechoua.com
quit-smoking.ga saitevpatorii.com
radiogambling.com sampleletters.net
rankia.com satellite.maps.ilovevitaly.com
rankscanner.com sayyoethe.blogspot.co.za
rapidsites.pro
ratemodels.net
research.ifmo.ru
robotixix.com
rockma.se
rozalli.com
runtnc.net
sanatorrii.ru
sbornik-zakonov.ru sbornik-zakonov.ru
search-error.com seolab.top
searchaddis.com sevenstars7.com
seksotur.ru sex-znakomstva.online
selectads.men shodanhq.com
seminarykansas904.ml shopsellcardsdumps.com
seo-prof1.xyz site-speed-check.site
seonetwizard.com
serptehnika.ru
sexsearch.com
sexspornotub.com
sharebutton.org
sherlock.se
shiksabd.com
shohanb.com
site.ru site.ru
skuteczna-dieta.co.pl siteaero.com
smart-scripts.com sitebeam.net
sneakyboy.com sivs.ru
soaksoak.ru sledstvie-veli.net
soc-proof.su smartpet.ru
social-s-ggg.xyz snsdeainavi.info
socialseet.ru social-buttons.com
socialsignals24.com space4updating.win
solicita.info spb-plitka.ru
spacash.com sr-rekneskap.no
spywarenuker.com stackthatbucks.com
squidoo.com stpicks.com
stariy-baku.com success-seo.com
strv.se tabakur77.com
survival.betteroffers.review telefonsexsofort.tk
symbaloo.com tetracsaudi.com
tacbelarus.ru the-torrent-tracker.blogspot.com
tfxiq.com theallgirlarcade.com
theendivechronicles.com thebitcoincode.com
thenews-today.info ticketsys.inetwd.com
thetardistimes.ovh tivolibasket.it
timdreby.com tizanidinedosage.blogspot.com
tizanidine4mg.blogspot.com tizanidinehcl4mgtablet.blogspot.com
tizanidinecost.blogspot.com tizanidinerestlesslegsyndrome.blogspot.com
tizanidinedrugtest.blogspot.com toloka.hurtom.com
tizanidinepain.blogspot.com torrents-tracker.com
topclickguru.com track2.shop
torrentgamer.net tracxn.com
trafficmonetize.org traffpartners.com
truck-land.life trainoffend.ml
tuckpointingmasonrysystems.com triumf-realty.ru
uchebavchehii.ru twu.com.ua
udayavani.com txxx.com
urengoy.pro uchil.net
v720hd.ru ucoz.ru
valueclick.com ufa.xrus.org
video-production.com uptime-gamma.net
vintontech.info urccvfmc.bloger.index.hr
vitanail.ru usbggettwku.ga
viveresaniesnelli.it vadimkravtcov.ru
vk-mus.ru varbergsvind.se
vroze.com vellings.info
vtc.pw viddyoze.com
watsonrealtycorp.com vksaver-all.ru
webmasterhome.cn wedding0venues.tk
websiteexplorer.info wineration.com
wechatdownload10.com winx-play.ru
wesharepics.site wpsecurity.website
wingsoffury2.com www.wohnkabinen-shop.de
witherrom55.eklablog.fr x-stars.ru
xn--90acenikpebbdd4f6d.xn--p1ai xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn--oogle-wmc.com zapiszto.pl
xvideospornoru.com zebradudka.com
xwatt.ru zeleznobeton.ru
xxart.ru zigarettenonlinekaufen1.blox.pl
yourmovies.pl zooggames.com
yourporn.com zrelaya.pw
yunque.pluto.ro zytpirwai.net
zaimhelp.ru
zarenica.net
zeroredirect7.com
zreloeporno.tv
zs2vm.top
zvuker.net

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.200 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3087 ### Version: V4.2022.05.3088
### Updated: Thu May 5 22:01:27 UTC 2022 ### Updated: Fri May 6 10:03:39 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3087 ### Version: V4.2022.05.3088
### Updated: Thu May 5 22:01:27 UTC 2022 ### Updated: Fri May 6 10:03:39 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##