vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 04:00:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.09.2569 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-09-04 10:09:29 +00:00
parent 41a85e3260
commit d4e878c885
47 changed files with 83827 additions and 83837 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2568 #### Version: V4.2021.09.2569
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2568 #### Version: V4.2021.09.2569
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.09.2568 #### Version: V4.2021.09.2569
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10342
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.09.2568 #### Version: V4.2021.09.2569
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10346
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10346
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10346
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10346
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10346
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10346
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10346
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2568 2569

10346
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10346
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.173.143.232 0; 52.156.75.68 0;

10346
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10346
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10346
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10346
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.173.143.232 0; 52.156.75.68 0;

10346
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.173.143.232 0; 52.156.75.68 0;

10346
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Cosmos ADmantX
Deusu DnyzBot
HybridBot Ezooms
Linkbot Mister PiX
OutclicksBot NetZIP
PageGrabber ScoutJet
QueryN Metasearch Searchestate
WISENutbot WiseGuys Robot
seoscanners meanpathbot
sogouspider trendiction.com

296
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
AIBOT 01h4x.com
Abonti 360Spider
404checker
ALittle Client
Aboundex Aboundex
Aboundexbot Aboundexbot
Acunetix Acunetix
AhrefsBot
AiHitBot
Aipbot Aipbot
AlphaBot AlphaBot
Anarchie Anarchie
Anarchy Anarchy
Anarchy99 Ankit
Aspiegel Anthill
Apexoo
Asterias Asterias
BDFetch Attach
BackDoorBot AwarioSmartBot
BackStreet BLEXBot
BackWeb BackWeb
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass Badass
Barkrowler Barkrowler
Bigfoot BatchFTP
Bitacle BetaBot
Black Hole Black Hole
Blow Blackboard
BlowFish
Bolt Bolt
Brandprotect Buck
Brandwatch BuiltBotTough
Buddy BuiltWith
BunnySlippers Bullseye
CCBot BuzzSumo
CODE87 CATExplorador
CSHttp CSHttp
Calculon Calculon
CazoodleBot Cegbfeieh
CensysInspect
CheTeam CheTeam
CherryPicker
ChinaClaw ChinaClaw
Cloud mapping Collector
Cocolyzebot
Cogentbot
CopyRightCheck CopyRightCheck
Copyscape Copyscape
Cosmos Cosmos
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Crescent Curious
CyotekWebCopy Custo
DIIbot DBLBot
DSearch
DTS Agent DTS Agent
Demon DatabaseDriverMysqli
Digincore
DigitalPebble DigitalPebble
Dirbuster Dispatch
Discobot
DittoSpyder
DomainStatsBot
Dotbot
Download Wonder Download Wonder
Dragonfly
ECCP/1.0 ECCP/1.0
EMail Siphon
Ebingbong Ebingbong
Ecxi EroCrawler
EirGrabber Exabot
FDM FDM
FHscan Foobot
Firefox/7.0
FlashGet
Freeuploader Freeuploader
Fuzz
FyberSpider FyberSpider
Genieo Fyrebot
G-i-g-a-b-o-t
GalaxyBot
GetWeb GetWeb
Getintent Getintent
Gigablast Gigablast
Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla
Grafula Grafula
GrapeFX HEADMasterSEO
GrapeshotCrawler
HMView HMView
HTMLparser
Harvest
Hloader Hloader
Humanlinks
IDBTE4M IDBTE4M
Id-search IlseBot
IndeedBot IndeedBot
Indy Library
InfoNaviRobot InfoNaviRobot
InterGET InternetSeer
Iria Iria
JOC Web Spider Iskanie
JamesBOT JamesBOT
Jetty JikeSpider
Jyxobot Jorgee
Kenjin Spider Kenjin Spider
Keyword Density Keyword Density
Kozmosbot
LNSpiderguy LNSpiderguy
Larbin LWP::Simple
LeechFTP Lanshanbot
LexiBot LeechGet
LibWeb Libwhisker
LieBaoFast
Likse Likse
LinkScan LinkScan
LinkWalker Linkbot
Linkdexbot
LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey Spider Lipperhey Spider
Litemage_walker Ltx71
MFC_Tear_Sample MSIECrawler
MIDown tool MTRobot
MQQBrowser Mag-Net
Magnet Mail.RU_Bot
MauiBot Majestic SEO
Mediatoolkitbot Majestic12
Metauri MarkMonitor
Microsoft URL Control Masscan
Mister PiX Mata Hari
MeanPath Bot
MicroMessenger
Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3
NPbot NPbot
Nameprotect
Nessus Nessus
Net Vampire Net Vampire
NetSpider
Nettrack Nettrack
Netvibes
Nibbler Nibbler
Niki-bot Niki-bot
Nimbostratus
Nuclei
Nutch Nutch
Octopus Octopus
OpenLinkProfiler OpenLinkProfiler
Openvas
OrangeBot OrangeBot
OrangeSpider OutfoxBot
PageGrabber PECL::HTTP
PHPCrawl
PageAnalyzer
Pandalytics
Panscient Panscient
Papa Foto
Pavuk
PeoplePal PeoplePal
Petalbot Petalbot
Pi-Monster
Picscout Picscout
Pockey PleaseCrawl
ProWebWalker
Probethenet Probethenet
Pump Psbot
PxBroker
QueryN Metasearch
Quick-Crawler Quick-Crawler
RankActiveLinkBot RankActive
RankFlex
RankingBot2 RankingBot2
Rankivabot Re-re
RankurBot Reaper
RedesScrapy RepoMonkey
RocketCrawler Ripper
Rogerbot
SBIder SBIder
SEOkicks SEOkicks-Robot
SEOstats SEOprofiler
SMTBot Screaming
Scanbot SearchmetricsBot
ScreenerBot
ScrepyBot
SemrushBot SemrushBot
Siphon SentiBot
Seomoz
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer Sitebeam
SiteLockSpider SmartDownload
SiteSucker SocialRankIOBot
Sitevigil Sosospider
SlySearch Sottopop
Sociscraper Spammen
Sogou web spider
SpaceBison
SpankBot
Spbot
Spinn3r Spinn3r
SputnikBot SputnikBot
Steeler Sqlmap
Stripper Sqlworm
Sucker
Sucuri Sucuri
SuperBot SuperBot
SuperHTTP
Swiftbot Swiftbot
Szukacz
T8Abot T8Abot
TeleportPro Teleport
Telesphorep The Intraformant
TheNomad TheNomad
Thumbor
Toata Toata
Toweyabot Toweyabot
Tracemyfile Trendiction
True_Robot
Turingos
Turnitin
TurnitinBot TurnitinBot
Typhoeus Typhoeus
URLy Warning
URLy.Warning URLy.Warning
UnisterBot V-BOT
Upflow VB Project
VCI
Vacuum Vacuum
VeriCiteCrawler VidibleScraper
VoidEYE VoidEYE
WASALive-Bot Voil
WBSearchBot WWW-Collector-E
WISENutbot Wallpapers/3.0
WWW-Mechanize Web Fetch
Web Auto
Web Collage
Web Enhancer
Web Pix Web Pix
Web Sucker WebCopier
WebBandit
WebEnhancer WebEnhancer
WebSauger WebFuck
WebLeacher
WebPix
WebReaper
WebStripper WebStripper
WebSucker
WebWhacker WebWhacker
WebZIP Webalta
Whack Website Quester
Whacker WebsiteExtractor
Who.is Bot Whatweb
WinHTTrack
WiseGuys Robot WiseGuys Robot
Wotbox
Wprecon Wprecon
Xenu Xaldon WebSpider
Zermelo Zeus
ZmEu Zitebot
ZoominfoBot
ZumBot ZumBot
archive.org_bot
autoemailspider
check1.exe
clark-crawler
coccocbot-web coccocbot-web
cognitiveseo
com.plumanalytics com.plumanalytics
crawl.sogou.com gopher
crawler4j heritrix
dataforseo.com instabid
demandbase-bot
domainsproject.org
evc-batch
internetVista monitor
ips-agent ips-agent
lwp-request isitwp.com
meanpathbot lwp-trivial
magpie-crawler
mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler netEstate NE Crawler
plumanalytics plumanalytics
probe-image-size scalaj-http
scan.lol
seobility seobility
seostar seostar
sp_auditbot serpstatbot
trendiction.com sysscan
voyagerx.com tAkeOut
webmeup-crawler webmeup-crawler
webprosbot
x22Mozilla x22Mozilla
xpymep1.exe

324
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
404enemy
ADmantX
ALittle Client
Abonti
Aboundexbot Aboundexbot
Acunetix Acunetix
AfD-Verbotsverfahren AiHitBot
AhrefsBot AllSubmitter
Aipbot Alligator
Alexibot
AlphaBot AlphaBot
Anarchie Anarchy99
Anthill BLEXBot
Apexoo BackStreet
Aspiegel
AwarioRssBot
AwarioSmartBot
BBBike
BackWeb BackWeb
Backlink-Ceck Badass
BetaBot Bandit
Boardreader Barkrowler
BotALot BatchFTP
Bitacle
BlowFish
Brandprotect Brandprotect
Brandwatch Buddy
Buck BuiltBotTough
BuiltWith
Bullseye Bullseye
BunnySlippers BunnySlippers
CCBot CATExplorador
CheTeam Calculon
CheeseBot CazoodleBot
Cegbfeieh
Chlooe Chlooe
Cloud mapping Cliqzbot
Cocolyzebot Cocolyzebot
Cogentbot
Collector Collector
Copyscape Copier
Craftbot Craftbot
Curious CrazyWebCrawler
Crescent
CrunchBot
Custo Custo
DSearch
DatabaseDriverMysqli DatabaseDriverMysqli
Demon Demon
Digincore DomainAppender
DigitalPebble
Dirbuster
Disco
Discoverybot
Dispatch
DittoSpyder
DomainSigmaCrawler
DomainStatsBot
Domains Project
Dotbot Dotbot
Download Wonder Download Wonder
Ebingbong Dragonfly
Ecxi ECCP/1.0
EMail Siphon
EMail Wolf
EirGrabber EirGrabber
Exabot Evil
Express WebPictures Express WebPictures
ExtLinksBot ExtLinksBot
EyeNetIE EyeNetIE
FDM FHscan
Freeuploader FlashGet
Fuzz FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t
GalaxyBot GalaxyBot
Genieo
GermCrawler GermCrawler
GetWeb
Getintent
Gigablast
Gigabot Gigabot
Go!Zilla Go-Ahead-Got-It
GoZilla GoZilla
GrabNet Grabber
GrapeshotCrawler GrapeFX
HMView HMView
HTMLparser
HTTP::Lite HTTP::Lite
HTTrack
Haansoft Haansoft
Hloader Hloader
HybridBot HybridBot
IDBTE4M IDBot
Iblog IRLbot
Image Fetch IlseBot
Indy Library IndeedBot
Iria InfoTekies
IstellaBot Internet Ninja
JamesBOT InternetSeer
JetCar JOC Web Spider
Jetty Jetty
JikeSpider JikeSpider
Joomla Joomla
JustView Jorgee
Keyword Density
Kinza Kinza
Kozmosbot
LNSpiderguy LNSpiderguy
LexiBot LWP::Simple
LieBaoFast LibWeb
Lightspeedsystems Linkdexbot
LinkpadBot LinkpadBot
LinksManager LinksManager
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot Lmspider
Lipperhey MIDown tool
Lipperhey Spider
MFC_Tear_Sample
MIIxpc
MQQBrowser MQQBrowser
MSIECrawler
Mag-Net Mag-Net
Mass Downloader Magnet
MarkMonitor
Mata Hari Mata Hari
Meanpathbot MauiBot
MegaIndex.ru Mb2345Browser
Metauri Microsoft URL Control
Microsoft Data Access Moblie Safari
Mister PiX Mr.4x3
Mojolicious
Musobot Musobot
NPbot Name Intelligence
Navroad Navroad
Nessus Nessus
NetLyzer Net Vampire
NetSpider
NetZIP NetZIP
Netcraft Netvibes
Nettrack Nibbler
Niki-bot
NimbleCrawler
Ninja
Nuclei Nuclei
Nutch OpenVAS
Octopus
OpenLinkProfiler
Openvas
OutclicksBot
OutfoxBot OutfoxBot
Page Analyzer POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
Pandalytics PageGrabber
Panscient PageScorer
Pavuk
Petalbot Petalbot
Pi-Monster
Picscout
Picsearch Picsearch
Piepmatz
Pimonster
Pockey Pockey
Pump ProWebWalker
Probethenet
Psbot
Pu_iN
PxBroker PxBroker
PyCurl PyCurl
Quick-Crawler QueryN Metasearch
RSSingBot RSSingBot
RankActive RankurBot
RankingBot
RankingBot2
Rankivabot
Re-re Re-re
RebelMouse RealDownload
Recorder
RedesScrapy
RepoMonkey RepoMonkey
Rogerbot RocketCrawler
SBIder
SEOkicks-Robot SEOkicks-Robot
SEOprofiler SEOlyticsCrawler
SEOstats SEOstats
Screaming SISTRIX
SMTBot
ScanAlert
Scrapy
ScreenerBot
ScrepyBot ScrepyBot
Searchestate Semrush
SemrushBot SemrushBot
SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Seomoz
Shodan Shodan
Siphon Sitebeam
Site Sucker Siteimprove
SiteExplorer Sitevigil
SiteLockSpider SmartDownload
SiteSucker Snapbot
SlySearch
Snoopy
SocialRankIOBot SocialRankIOBot
Sociscraper Sosospider
SpaceBison Sottopop
Spammen
SpankBot
Spanner Spanner
Spbot
SputnikBot
Sqlworm Sqlworm
Sqworm Steeler
Sucker
Sucuri Sucuri
SurveyBot Surfbot
T0PHackTeam T0PHackTeam
Teleport The Intraformant
TeleportPro
Telesphoreo
TheNomad TheNomad
Titan TightTwatBot
Toata Toata
Toweyabot
Trendiction Trendiction
Trendictionbot Trendictionbot
Twice True_Robot
Typhoeus Turnitin
URLy.Warning UnisterBot
Upflow
V-BOT
VB Project VB Project
VCI Vacuum
Virusdie VeriCiteCrawler
WASALive-Bot
WBSearchBot
WEBDAV WEBDAV
WISENutbot WISENutbot
WWW-Collector-E WWWOFFLE
WWW::Mechanize Wallpapers
Wallpapers/3.0
WeSEE WeSEE
Web Auto Web Auto
Web Enhancer Web Enhancer
Web Fetch
Web Fuck Web Fuck
WebAuto Web Pix
WebBandit Web Sucker
WebCollage
WebEnhancer
WebFetch
WebGo IS
WebLeacher
WebReaper
WebSauger WebSauger
WebStripper
WebSucker
WebWhacker
WebZIP
Webalta
WebmasterWorldForumBot
Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
Whack Webster
Whatweb Whatweb
Widow Who.is Bot
WinHTTrack WiseGuys Robot
Woobot
Wprecon
Xaldon WebSpider Xaldon WebSpider
Zauba Xaldon_WebSpider
Zermelo Xenu
Zitebot Zade
ZumBot ZoomBot
arquivo-web-crawler ZoominfoBot
autoemailspider ZyBorg
backlink-check
clark-crawler clark-crawler
coccocbot-web
crawler.feedback crawler.feedback
domainsproject.org demandbase-bot
evc-batch facebookscraper
internetVista monitor instabid
isitwp.com lwp-request
magpie-crawler
meanpathbot meanpathbot
probe-image-size mediawords
muhstik-scan
oBot
pcBrowser
ripz ripz
satoristudio.net s1z.ru
seoscanners
serpstatbot serpstatbot
spyfu siteripz
sysscan sp_auditbot
trendiction.de tAkeOut
voyagerx.com trendiction.com
webmeup-crawler x22Mozilla
webprosbot xpymep1.exe
zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
autowebmarket.com.ua adtiger.tk
bitporno.sx biketank.ga
check-this-out-now.online candlewooddentalcentre.com.au
christianlouboutinsaleonline.us compliance-george.top
compliance-john.top pdn-4.com
educontest.net proxyelite.biz
iqs.biz.ua sbt-aqua.ru
pro-okis.ru svolze.com
sexy-screen-savers.com tizanidineandgabapentin.blogspot.com
spylog.com xn----btbdvdh4aafrfciljm6k.xn--p1ai

924
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
03p.info 0c47f8422d3f.com
123cha.com 1-free-share-buttons.com
1314dh.com 13tabs.com
14b.info 14b.info
4replicawatch.net 1kdailyprofit.me
5elementov.ru 404.mn
6128786.com 4kplayer.pl
66cpwgln.space 5forex.ru
70casino.online 777-club.ru
7fon.ru 7wind.ru
7xc4n.com 83net.jp
aa8780bb28a1de4eb5bff33c28a218a930.com 8si.ru
aaa38852e886ac4af1a3cff9b47cab6272.com 999webdesign.com
abclauncher.com a.frcls.fr
abiente.ru absolute-s.ru
adserver-e7.com absugars.com
academia-nsk.org
acortarurl.es
ad-words.ru
adelly.bg
adexprts.com
adloads.com
adloads.net
adsref.men adsref.men
adssafeprotected.com adtech.us
adult-shop.com.ua afslankpillen2017nl.eu
agahinameh.com agardomains.com
agroeconom.kz alarmobninsk.ru
ald2014.org alfabot.xyz
alevinefesleri.com allbizne.co.ua
algarveglobal.com alldownload.pw
alloysteel.ru allforminecraft.ru
alyeskaresort.com allmarketsnewdayli.gdn
amospalla.es alltheviews.com
anal-acrobats.com alpharma.net
anaseracresar.tk amazingninja.com
andpolice.com
anti-virus-removal.info anti-virus-removal.info
appmsr.org apartmentratings.com
approvedlocal.co.za apiadanaknet-a.akamaihd.net
appsecurityr.com appartement-stumm.at
arewater.com apxeo.info
arkkivoltti.net arclk.net
arquapetrarca.info arcteryxstore.online
asseenontvonline.ru asdfg.pro
autochoixspinelli.com asophoto.com
autogrep.ru asrv-a.akamoihd.net
autotop.com.ua atmagroup.ru
avitocars.ru automatic-seo.com
automobile-spec.com
autorn.ru
avtocredit-legko.ru
ayakino.net ayakino.net
bag77.ru b2b-lounge.com
baixar-musicas-gratis.com bankiem.pl
bard-real.com.ua bankofthewext.com
beauty-b0x.pl baoxaydung.com.vn
beetpics.pw bayareaaeroclub.org
best-mam.ru beauty-things.com
best-way.men bedandbreakfast.com
betteroffers.review bestdooz.com
bidr.trellian.com bestmobilityscooterstoday.com
billiard-classic.com.ua betterhdporn.com
bio-optomarket.ru bigcities.org
bkgr.se biznesrealnost.ru
blog.koreadaily.com bizru.info
blog.yam.com blackcurranthumidifiers.site
bloggen.be blogos.kz
boostmyppc.com bmwhighperformers.com
bpro1.top bolegapakistan.com
bracketsmackdown.com bongiornos.info
brainzpod.win boo-arts.com
breget74.ru bookhome.info
bristolhostel.com bosefux.esy.es
bristolhotel.com
bukleteg.xyz
buqayy0.livejournal.com
buy-meds24.com
buyessay3.blogspot.ru buyessay3.blogspot.ru
byme.se buzzsumo.com
californianews.cf cacheimages.com
callejondelpozo.es cakemediahq.com.au
callmd5map.com callawaygolfstore.online
calstaterealty.us candlewooddentalcentre.com.au
candypeople.se candytiner.com
canvas.gloverid.site canzoni.ru
cartierlovebracelet.xyz cartierjusteunclou.xyz
cartierlovebraceletreplica.xyz cartierlove2u.xyz
catalogs-parts.com cercacamion.it
cdncash.net cgi2.nintendo.co.jp
cementaresearch.se chatroulette.world
chatroulette.video cherubinimobili.it
chinajerseyswholesalecoupons.com citizenclsdriveri7.pen.io
cialiswithoutadoctor.net classiquebijoux.ru
ciekawinki.pl cntravelre.com
clarithromycin500mg.com cocyq.inwtrade.com
cmhomestayagency.com compliance-alexa.top
cocolyze.com compliance-don.top
communisave.co.za compliance-irvin.top
community.allhiphop.com compliance-willy.top
compliance-nelson.top connectingsingles.com
cookingmeat.ru constantaservice.net
cool-wedding.net contentlook.co
coolgamechannel.com coolbar.pro
counciltally.com coop-gamers.ru
cpajunkies.com credit-online.ws
cracksplanet.com creditwell.ru
crazyboost.pro cyse.tk
creams.makeforum.eu daptravel.com
crirussian.ru darkbooks.org
cscwtalkto.site datadepths.com
customerguru.in dealighted.com
customerpromos-a.akamaihd.net
cyclobenzaprinevstizanidine.blogspot.com
cyprusbuyproperties.com
day-news.info
ddsoldes.fr
decorazilla.com decorazilla.com
derevesendeco.com deda-moroza-zakaz.ru
dfiles.me dellalimov.com
dfwu1019.info delta-line.men
dieta-personalna.pl detki-opt.ru
digital-video-processing.com dev-seo.blog
directivepub.com dktr.ru
dispo.de dnepropetrovsk.mistr-x.org
djonwatch.ru dogclothing.store
dlya-android.org domik-derevne.ru
dollartree.info dominateforex.ml
domain-submit.info dptaughtme.com
donna7753191.ru eaplay.ru
e90post.com egvar.net
ecommerce-seo.org elmacho.xyz
ed-shop01.ru espaceinventoristes.com
elektrozigarettekaufen1.devhub.com essaypro.com
elitepcgames.com event-tracking.com
eloxal.ru evrotekhservis.ru
empathica.com excaliburfilms.com
englate.com exchanges-bet.com
eorogo.top exoneration-project.us
escortslet.net extlinks.com
estibot.com extremal-blog.com
euronis-free.com
evehemming.blogspot.com.au
ezb.elvenmachine.com
ezigarettenshop2.postbit.com ezigarettenshop2.postbit.com
f-loaded.de fable.in.ua
fanrto.com faithe.top
fashionavenuegame.com familyholiday.ml
fashionindeed.ml
favornews.com favornews.com
fenoyl.batcave.net feeriaclub.ru
ferretsoft.com fellowshipoftheminds.com
ferrotodo.com fertilitetsradgivningen.se
filkhbr.com fetishinside.com
filunika.com.ru financehint.eu
findclan.org find1friend.com
fishingwholesale.us
flauntyoursite.com
floridamhca.org floridamhca.org
fodelsedagspresenter.nu folowsite.com
food.dtu.dk
footbalive.org footbalive.org
forex.osobye.ru formulaf1results.blogspot.com
forexunion.net
forum69.info forum69.info
foto-sisek.porngalleries.top fotopop.club
free-deals.faith foxjuegos.com
free-gluten.ru foxydeal.com
free-today.com franklinfire.co
free-share-buttons.top
freecamdollars.com freecamdollars.com
freelinkbuilding.website.tk freeseedsonline.com
front.ru freshdz.com
front.to freshmac.space
game-top.su g.starmoe.xyz
gamebackyard.com games.kolossale.ru
gameonasia.com gcup.ru
gdcentre.ru
ge0ip.net
gearcraft.us
gelezki.com gelezki.com
germes-trans.com germetiki.com.ua
ghazel.ru getresponse.com
ghostvisitor.com gettpromos.com
gk170.ru ggiaro.com
glasof.es girlsfuckdick.com
globalscam.ga glastecfilms.com.my
gloverid.site goblacked.com
gnuetella.com golden-catalog.pro
goatse.ru goldpanningtools.com
gorabagrata.ru gratuitxblcodes.com
gotwebsite1.com hccoder.info
gsasearchengineranker.pw headpress.ru
guardlink.org heartofbeijing.blogspot.com
guod.me heatpower.ru
gwebtools.com.br hebr.myddns-flir.com
haveinc.xyz help.tpu.ru
hawaiisurf.com herehloadibs.cf
heartofpayne.xyz
helvetia.com.ua
hentaiheroes.com
hermesreplica.pw hermesreplica.pw
hildinghr.se hermesreplica.win
hit-kino.ru heygidday.biz
hitmuzik.ru histats.com
hitsbox.info homeart.space
hobbyhuren24.net homedo.fabpage.com
hoholikik.club host-protection.com
holidaypics.org hoste.octopis.com
homegardenlova.com hotchatdate.com
horoshieokna.com howtowhitenteethfast.xyz
hostcritique.com hreade.com
hotxnights.info hulfingtonpost.com
housekuba.org iboss.com
husky-shop.cz iideaidekonkatu.info
ilovevitaly.xyz inboxdollars.com
imagecoolpub.com infogame.name
imgarcade.com inspiring-desperate.tk
iminent.com iqoption.com
inboundlinks.win it-max.com.ua
incanto.in.ua itch.io
indetiske.ya.ru itis4you.com
indianmedicaltourismshop.com javlibrary.cc
infowarcraft.ru jennyfire.ru
instakink.com jolic2.com
intellektmedia.at jwcialislrt.com
internetproviderstucson.com kalandranis.gr
invest-pamm.ru
inzn.ru
iopeninghours.co.uk
iphantom.com
iqoption-bin.com
isotoner.com
janerikholst.se
jav-way.site
javspace.net
jbl-charge.info
jerseysbizwholesalecheap.com
jo24news.com
jobmarket.com.ua
justkillingti.me
kareliatobacco.ru kareliatobacco.ru
karting196.ru kartiniresto.com
konkursowo-24.pl katjimej.blog.fc2.com
koversite.info kaz.kz
kovesszucs.atw.hu keywesthideaways.co
kfon.eu
kidskunst.info
kino-key.info
kinobaks.com
kliksaya.com
klitimg.pw
komputers-best.ru
konkursov.net
kovrenok.ru kovrenok.ru
kozhniebolezni.com krasnodar.ru
krafte.ru kunstaktien.info
ks1234.com kutikomi.net
ktotut.net kvartira-sutochno.com
leboard.ru languagecode.com
lernur.net lapolis.it
letmacworkfaster.world laptop-4-less.com
lezbiyanki.net larose.jb2c.me
li-er.ru laserpen.club
lidoradio.com law-check-nine.xyz
lifebyleese.com layola.biz.tc
lifeinsurancekb.com le-clos-des-alouettes.com
legionalpha.com
lenvred.org
levaquin750mg.blogspot.com
lexaprogeneric.link
liky.co.ua liky.co.ua
lion.playtap.us linkbuddies.com
locustdesign.co linkbux.ru
longadventure.com liran-locks.com
louboutinbooties.xyz littlesunraiser.com
ltvperf.com liupis.com
machicon-ueno.info live-sexcam.tk
liver-chintai.org
lol-smurfs.com
louboutinreplica.xyz
love-baby.cz
lowephotos.info
luchshie-topcasino.ru
m0r0zk0-krava.ru
m292.info
magento-crew.net magento-crew.net
magicdiet.gq mainlinehobby.net
makemoneyonline.com marcoislandvacations.net
malls.com
manuscript.su
marwer.info marwer.info
mastershef.club matsdale.com
mediaoffers.click maximpartnerspr.com
medizinreisen.de mb140.ru
megaindex.ru medtherapy.ru
mere.host.sk meduza-consult.ru
mesbuta.info mesto-x.com
message-warning.net metalonly.info
mex-annushka.ru mexicosleevegastrectomy.com
michaelkorssaleoutletonline.net migente.com
midst.eu militarysale.pro
missclub.info minecraft-neo.ru
mlf.hordo.win mini-modus.ru
money-every-day.com minneapoliscopiers.com
most-kerch.org mir-business-24.ru
moviemail-online.co.uk mir-limuzinov.ru
misandesign.se
mobilemedia.md
mobioffertrck.com
mobstarr.com
monetizer.com-01.site
monitorwebsitespeed.com
monsterdivx.com
moyakuhnia.ru moyakuhnia.ru
mrsdalloways.com mp3downloadhq.com
mulberryoutletonlineeu.com musezone.ru
mustwineblog.com musicspire.online
muz-tracker.net mybackgroundlandscape.blogspot.com
mwtpludn.review mydirtystuff.com
mydirtyhobby.com mydoctorok.ru
myghillie.info mymobilemoneypages.com
mywallpaper.top napalm51.nut.cc
nakozhe.com narkologiya-voronezh.ru
naphukete.ru national-today-winning-winner.club
narkologiya-penza.ru naturalbreakthroughsresearch.com
netanalytics.xyz naturalshair.site
netfacet.net news-speaker.com
newtechspb.ru nisuturnetdgu.tk
nhl09.ru oddamzadarmo.eu
nikkiewart.ru
normalegal.ru
nosecret.com.ua
novosibirsk.xrus.org
nucia.biz.ly
nudepatch.net
oconto.ru
odywpjtw.bloger.index.hr odywpjtw.bloger.index.hr
okayimage.com ofanda.com
okout.ru omsk.xrus.org
oktube.ru onemantrip.com
onlinemeetingnow.com onlinebay.ru
onlythegames.com ons-add.men
ooomeru.ru oops-cinema.ru
ossmalta.com open-odyssey.org
ovirus.ru orion-code-access.net
oynat.info osoznanie-narkotikam.net
oz-offers.com outrageousdeal-a.akamaihd.net
pandroid.co paramountmarble.co.uk
partner-cdn.men paydayloanslocal.com
partner-print.men paydayonlinecom.com
pathwhelp.org pdn-4.com
petrovka-online.com pdns.download
photokitchendesign.com pensplan4u.com
physfunc.ru phormchina.com
picturesmania.com pinapchik.com
pinstake.com
pipki.r.acdnpro.com
piratecams.com piratecams.com
piuminiita.com pistonclasico.com
pl-vouchers.com
plaff-go.ru
plastgranar.nu plastgranar.nu
play-movie.pl playmsn.com
pobeiranie.pl poems.com.ua
podshipniki-nsk.ru poppen-nw.net
polytopesexempt.com pornmania.pl
porndairy.in pornoforadult.com
pornosexrolik.com pornosmola.info
pornotubexxx.name pornozhara.com
predmety.in.ua pozdrawleniya.ru
privacylocationforloc.com princeadvantagesales.com
privatamateure.com privatefx-in.ru
pro-poly.ru privhosting.com
profitfx.online prize44.com
profitkode.com prizeestates.cricket
profitwithalex.info prostitutki-belgoroda.org
promalp-universal.ru quick-offer.com
promoforum.ru quit-smoking.ga
prosmibank.ru radiogambling.com
proxyradar.com ragecash.com
purplesphere.in randalljhoward.com
push-ad.com raschtextil.com.ua
pushdata.sendpulse.com realting-moscow.ru
putitin.me remote-dba.de
quickchange.cc renewablewealth.com
rankings-analytics.com
razleton.com
realnye-otzyvy.info
reckonstat.info
registrysweeper.com
rennlist.com
respublica-otel.ru respublica-otel.ru
retreatia.com rezeptiblud.ru
reversing.cc ringtonepartner.com
robot-forex.biz sakhboard.ru
rostov.xrus.org sandhillsonline.com
rusoft-zone.ru satoristudio.net
saecsa.co
sankt-peterburg.nodup.ru
sayyoethe.blogspot.co.za sayyoethe.blogspot.co.za
sbetodiodnye-lampy.ru sbf441.com
scanmarine.info scanner-jack.top
scanner-julia.top score-ads.men
scanner-nelson.top search.1and1.com
scrnet.biz.ua search.alot.com
searchencrypt.com seasonvar.ru
searchengineranker.email securesmrt-dt.com
seminarykansas904.ml semalt.com
semxiu.com semaltmedia.com
sendearnings.com seo18.su
seo-prof1.xyz seofied.com
seocheki.net service-core.ru
seorankinglinks.com sexkontakte-seite.com
sevendays.com.ua sexreliz.com
sexspornotub.com sexs-foto.com
sexy-pings.com
sexyebonyteen.com
sexystrippe.info
sharebutton.net
shikiso.info
shohanb.com shohanb.com
sicfor.bcu.cc shop.acim.org
sigmund-freud.co.uk shopcheermakeup.info
shopsellcardsdumps.com
shopwme.ru
sildenafilcitratemed.com
similardeals.net similardeals.net
simple-share-buttons.com sinestesia.host.sk
sivs.ru siteimprove.com
sk.golden-praga.ru skutecznetabletkinaporostwlosow.pl
skatestick.bid sky-mine.ru
skladvaz.ru slutloadlive.com
sledstvie-veli.net smailik.org
smichovbike.cz small-games.biz
sms2x2.ru sneakyboy.com
social-button.xyz snjack.info
songplanet.ru snworks.com
sonnikforme.ru sobecjvuwa.com.ru
sovetogorod.ru socialbutton.xyz
socialseet.ru
sofit-dmd.ru
softtor.com
sortthemesitesby.com
sotkal.lark.ru
soundfrost.org
spamnuker.com spamnuker.com
spb.ru sportbetfair.com
speechfoodie.com spy-app.info
sribno.net stanthonyscatholicchurch.org
stariy-baku.com staynplay.net
steame.ru stpicks.com
streamin.to strag-invest.ru
streetfire.net strana-krasoty.ru
streha-metalko.si strangeduckfilms.com
strigkaomsk.ru stretchmate.net
stroyhelp-dv.ru stroyplus.ru
sukirgenk.dvrlists.com sukirgenk.dvrlists.com
surfbuyermac.com summerlinhomes411.info
surintech.ac.th superstats.com
svensk-poesi.com susanholtphotography.com
sweepstakes.rewardit.com swinginwithme.ru
swingerseiten.com szamponrevita.pl
taihouse.ru t-bygg.com
takeprofitsystem.com tabakur77.com
takeflyte.com
tamada69.com
tasteidea.com tasteidea.com
td-33.ru taximytishi.ru
techart24.com tecnoteakviareggio.it
tedxrj.com telsis.com
telegramdownload10.com template-kid.com
teletype.in thin.me.pn
templates.franklinfire.co tizanidine6mg.blogspot.com
thebestphotos.eu tizanidinedruginteractions.blogspot.com
thedownloadfromwarez.blogspot.com tizanidineonline.blogspot.com
theendivechronicles.com tizanidinestreetprice.blogspot.com
thefotosgratis.eu
thepantonpractice.co.uk
theporndude.com
thetardistimes.ovh
tiens2010.ru
titangel-vietnam.com
tizanidinehcl4mgisitanarcotic.blogspot.com
tizanidinepills.blogspot.com
token-lab.org token-lab.org
top-deal.com.pl tootoo.to
trafficreceiver.club tpu.ru
truebeauty.cc trafficshaper.com
tt-ipd.info trailer.cinemaflix.website
tyumen.xrus.org try-rx.com
ua.tc twincitiescarservice.com
ugguk.online twojebook.pl
umg-stroy.ru uhod-za-sobakoj.ru
unrealcommander.org undergroundcityphoto.com
upproar.com uptimechecker.com
url-img.link vacances-voyages.info
uzungil.com validdomain.xyz
v24s.net vancleefreplica.pw
vadimkravtcov.ru vandrie-ict.nl
vapsy.com
vashsvet.com
velen.io
veles.shop veles.shop
vereo.eu velobikestock.com
verymes.xyz venerologiya.com
veselokloun.ru viberdownload10.com
viagroid.ru video-woman.com
videochat.life videokrik.net
videochat.world villakohlanta.nu
vip-dom.in vip.51.la
vodaodessa.com vipms.ru
visitcambridge.org
vod.com.ua
vseigru.one vseigru.one
vtcdns.com
vvpg.ru
vykup-avto-krasnodar.ru vykup-avto-krasnodar.ru
walkme.com wahicbefa31.soup.io
watchmyfb.pl watchmyfb.pl
watsonrealtycorp.com webalan.ru
wdrake.com weclipart.com
webenlace.com.ar wedding0venues.tk
website-analytics.online wetgames.ru
websiteexplorer.info
weburok.com
welck.octopis.com
wesharepics.info
westvilletowingservices.co.za
wfb.hatedriveapart.com wfb.hatedriveapart.com
where-toget.com whatsappbot.flyland.ru
wholesalejerseysgaa.com womama.ru
wineration.com woman-h.ru
wleuaprpxuvr.ga woman-tampon.ru
womensterritory.ru wonderfulflowers.biz
wordpress-crew.net worldoffiles.ru
wordpresscore.com wowcasinoonline.ooo
wosik-dach.service-for-web.de wufak.com
wowas31.ucoz.ru ww2awards.info
writersgroup580.web.fc2.com x-diesel.info
wrz0iuebwhp5fg.freeddns.com xcc24.pl
www.event-tracking.com
x-mix.info
x-stars.ru
x-true.info
xlolitka.com
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xn--80ajbshivpvn2i.xn--p1ai xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn--80ak6aa92e.com xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xn--d1acah0c.xn--p1ai xn----itbkqkfiq.xn--p1ai
xn--e1aggki3c.xn--80adxhks xn--1-8sbcpb0bdm8k6a.xn--p1ai
xn--l1aengat.xn--p1ai xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xtremeeagles.net xn--80aodinpgi.xn--p1ai
xxxsiterips.xyz xxxguitars.com
yatrk.xyz yebocasino.co.za
yhit.press youbrainboost.asia
yougotanewdomain.com youjizz.com
yourothersite.com yourporngay.com
yourserverisdown.com yourserverisdown.com
yugk.net youtuhe.com
yuweng.info zaim-pod-zalog-krasnodar.ru
zeikopay.com zaimite.ru
zero1.it zeroredirect.com
zeroredirect5.com zigarettenonlinekaufen1.blox.pl
znakomstvaonlain.ru zlatnajesen.com
zryydi.com zona-aqua.ru
zvetki.ru zoodrawings.com
zoogdiesney.com
zverokruh-shop.cz
zynax.ua
zytpirwai.net

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1bet.com
1flag.co.za 1flag.co.za
1hwy.com 2x2fan.ru
51unlim.ru aa9bd78f328a6a41279d0fad0a88df1901.com
70casino.online aasoldes.fr
8558l.jobs.net abcdefh.xyz
aa8780bb28a1de4eb5bff33c28a218a930.com adobereader-free.ru
aad1f4acb0a373420d9b0c4202d38d94fa.com adsloads.com
accessoires-mode.in adtech.de
activecampaign.dreamhosters.com adzpower.com
adpremium.org alevinefesleri.com
ads-cool.pro allnews.md
adultactioncam.com allproblog.com
advocatemsk.ru aloofly.com
afora.ru amigobulls.com
akita.kz appartement-stumm.at
akusherok.ru artefakct.com
alive-ua.com autorn.ru
all4wap.ru balla-boo.se
allfinweb.com bangla-chat-uk-paralud.ga
ampower.me bin-brokers.com
apisecretsaucebi-a.akamaihd.net binomo.kz
artpicso.com blagovest-med.ru
avtoarenda.by blockworld.ru
ayakino.net bluesalt.co
ayanaspa.com bosman.pluto.ro
b2b-lounge.com brus-vsem.ru
baditri.com bwlx.prepedu.cn
bankcrediti.ru candcstuccoandstone.com
best-mam.ru canrole.com
bestdooz.com caveavins.fr
besthomemadepornsites.com cheapsergiorossi.online
bimatoprost-careprost.com checkm8.com
bitcoins-live.ru checkpagerank.net
biznesschat.net chocolateslim-en-espana.com
blackwitchcraft.ru cigarpass.com
bloggers.nl cinemaenergy-hd.ru
boltalko.xyz cleanmypc.com
brainzpod.win click2pawn.com
burkesales.com cocolyze.com
callejondelpozo.es coinsspb.com
callmd5map.com compliance-brian.top
capacitacionyaprendizaje.com
chlooe.com
clarithromycin500mg.com
clickbanksites.info
clicksotrk.com
compliance-jessica.top
compliance-walter.top
cookie-law-enforcement-ff.xyz cookie-law-enforcement-ff.xyz
cpabegins.ru cookielawblog.wordpress.com
cracksplanet.com country-chic.ru
craftburg.ru craftinsta.ru
cubs-jerseys.us crd.clan.su
creditnation.ru
customsua.com.ua
cvety24.by cvety24.by
dailyfinancefix.com datsun-do.com
digest-project.ru dentalpearls.com.au
dikx.gdn discountliv.com
dinosaurus.site domain-submit.info
diusyawiga.tk domashneeruporno.com
dogoimage.com drugstoreforyou.com
dosugrostov.site
download-wallpaper.net
download-walpaperhd.blogspot.com
e-avon.ru
e705.net
eas-seo.com eas-seo.com
eduinfosite.com ege-russian.ru
egovaleo.it ekosmetyki.net.pl
ekn-art.se elektrischeziga.livejournal.com
elektrischezi.canalblog.com elektrischezigarette1.onsugar.com
eng-lyrics.com elektronischezigarettekaufen1.beeplog.com
erotik0049.com euronis-free.com
erotische-geschichten-xxl.com eyessurgery.ru
ez8motelseaworldsandiego.com ezigarettekaufen2.blox.pl
ezigarettekaufen2.mpbloggar.se ezigarettenshop2.myblog.de
factorynightclub.com filmgo.ru
fanrto.com findclan.org
faptitans.com free-floating-buttons.com
fashionindeed.ml free-share-buttons.top
fellowshipoftheminds.com freza-sverlo.ru
feorina.ru fuckingawesome.com
filesdatabase.net g7m.pl
financepoints.eu gaming-journal.com
forex-procto.ru gemara.com
free-gluten.ru globalscam.ga
free-share-buttons.com goforexvps.com
free-video-tool.com gotorussia.com
fsakhalin.ru greatzip.com
fz139.ttk.ru guardlink.org
gamewrath.com handicapvansarea.com
gateway.zscalerone.net handsandlegs.ru
gdebestkupit.ru hawaiielectriclight.com
gentamicineyedrops.blogspot.com hit-kino.ru
given2.com hotchatdate.com
goodhumor24.com hotcore.info
grand-chlen.ru hotel-mkad.ru
gsasearchengineranker.site iframe-toloka.com
gtopstats.com images.gyffu.com
guge.io indetiske.ya.ru
gwebtools.com.br indiasourcemart.in
gz2.bbsoldes.fr infoupdate.org
hackers-crackers.tk iphantom.com
hdwallpapers-free.com irkutsk.online-podarki.com
healbio.ru it-max.com.ua
hosting-tracker.com ixora.pro
hotblognetwork.com jav-idol.com
hotenergy.ru jobgirl24.ru
hoverboard360.nl jongose.ninja
i-hobot.ru jose.mulinohouse.co
ideashome.id karting196.ru
ikritikimou.gr keywordsdoctor.com
ilbe.club kollekcioner.ru
infazavr.ru kupiproday.com.ua
innoslicon.com lacave.ntic.fr
iusstf.org landoftracking.com
jurajskie.info leadn.pl
kasino-money.pw lietaer.com
keyhantercume.com littlecity.ch
kino2018.club loadopia.com
kredit-pod-zalog-krasnodar.ru longgreen.info
kursy-ege.ru m292.info
kuzinsp.ru macdamaged.tech
kvartir-remont.biz marketland.ml
lalalove.ru meetingrainstorm.bid
laxob.com mejoresfotos.eu
lenvred.org milblueprint.com
letto.by minecraft-rus.org
louisvuittonoutletstore.net mini-modus.ru
mamasuper.prom.ua mirtorrent.net
members.ghanaweb.com
menhealed.net
mesbuta.info
mikozstop.com
mobsfun.net mobsfun.net
mojaocena.com moetomnenie.com
monarhs.info moneyzzz.ru
moomi-daeri.com mttwtrack.com
moyakuhnia.ru muizre.ru
mylesosibirsk.ru myblogregistercm.tk
na-telefon.biz myghillie.info
naturalshair.site nac-bearings.ru
netfacet.net naughtyconnect.com
new-apps.ru naval.jislaaik.com
nodup.ru netvouz.com
noscrapleftbehind.co networkad.net
obsessionphrases.com nhl17coinsforps3.gratisblog.biz
oktube.ru nicefloor.co.uk
outclicks.net nootrino.com
parfusale.se norththeface.store
park.above.com notebook-pro.ru
partner-high.men office2web.com
partner-print.men okmusic.jp
pastaleads.com onlinadverts.com
pcimforum.com openfrost.com
piter.xrus.org optom-deshevo.ru
pix24x7.com painting-planet.com
pixelrz.com picsforkeywordsuggestion.com
pizza-imperia.com pills24h.com
placid-rounded-coast.glitch.me playtap.us
pobeiranie.pl pod-muzyku.club
pooleroadmedicalcentre.co.uk poffet.net
pornhub-forum.ga popads.net
pornmania.pl popunder.ru
pornosexrolik.com porno.simple-image.com.ua
pr-ten.de pornofeuer.com
privatefx.all4invest.info pornoforadult.com
prodess.ru pornpost.in
professionalwritingservices15.blogspot.ru powc.r.ca.d.sendibm2.com
prointer.net.ua prizestohandle.club
pronorm.fr producm.ru
prostitutki-almata.org pronekut.com
qld10000.net pts163.ru
radiogambling.com q-moto.ru
ranksonic.info rank-checker.online
rcpmda.ikan1080.xyz raschtextil.com.ua
recinziireale.com redirect.trafficreceiver.club
rep-am.com reeyanaturopathy.com
ringtonepartner.com reklama-i-rabota.ru
robotixix.com remedyotc.com
russian-postindex.ru responsive-test.net
russkoe-zdorovie.ru rethinkwasteni.info
samoiedo.it riders.ro
savefrom.com rospromtest.ru
scanner-josh.top samolet.fr
scanner-mary.top saveriopiazza.it
schlampen-treffen.com scripted.com
schoolfiles.net seansonline24.pl
seminarygeorgia59.ga searchencrypt.com
sergiorossistore.online seasonvar.ru
sex-znakomstva.online sei80.com
sfd-chess.ru seocdvig.ru
shitting.pro seolab.top
shohanb.com sexy-screen-savers.com
silvermature.net shitmovs.com
sim-service.net simon3.ru
siteheart.net simplepooltips.com
skatestick.bid simul.co
smartshoppymac.com skincrate.net
socialmonkee.com skyway24.ru
sohoindia.net smailik.org
sotechco.co small-games.biz
spbchampionat.ru smokewithrabbits.com
snjack.info
socialsignals24.com
softxaker.ru
spl63.fr
spylog.com spylog.com
srecorder.com srgwebmail.nl
stevemonsen.com stariy-baku.com
streha-metalko.si sticken.co
stroymonolit.su stopnarco.ru
stroyplus.ru sugarkun.com
strv.se superiends.org
studworks.org supermama.top
tds-advert002.info svnuppsalaorebro.se
tizanidinehcl4mgtabinfo.blogspot.com telefonsexi.com
tizanidinestreetprice.blogspot.com telemetryverification.net
toon-families.com tomatis.gospartner.com
topclickguru.com totu.info
tpu.ru trafficmania.com
trafficjunky.com tula.howotorg.ru
trafficjunky.net tupper-posuda.ru
traffictrade.life twittruth.com
turbodsp.com tytoona.com
tvorozhnaja-zapekanka-recept.ru u17795.netangels.ru
twojebook.pl uggsale.online
unpredictable.ga unmaroll.ya.ru
unrealcommander.biz upstore.me
url-extractor.xyz uptime-gamma.net
us-america.ru vabasa.inwtrade.com
uytmaster.ru vacances-voyages.info
via-energy-acquistare.com veerotech.com
video-chat.in vepad.com
villacoloniale.com videonsk.com
visa-china.ru vonradio.com
visa-pasport.ru vsexkontakte.net
visitcambridge.org vvpg.ru
vitalads.net warezkeeper.com
wallpapersdesk.info watracker.net
walpaperlist.com weburlopener.com
waterefficiency.co whatsappdownload10.com
website-analyzer.info wholesalejerseys-cheapest.com
websiteaccountant.de williamrobsonproperty.com
whengirlsgowild.com winx-play.ru
womama.ru wonderfulflowers.biz
x-musics.com wordpress-crew.net
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai www.bookmaker-bets.com
x-porno.video
xaijo.com
xaylapdiendanang.com
xerox-douglas.cf
xgftnlrt.bloger.index.hr
xn----7sbaphztdjeboffeiof6c.xn--p1ai
xn----7sbho2agebbhlivy.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xxxtubesafari.com xn--lifehacer-1rb.com
yeartwit.com youjizz.com
youthreaders.com yourmovies.pl
zaapplesales.blogspot.com yubikk.info
zastenchivosti.net zetmaster.ru
zeroredirect1.com zigarettenonlinekaufen1.bloog.pl
zootoplist.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.104 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2568 ### Version: V4.2021.09.2569
### Updated: Fri Sep 3 22:01:12 UTC 2021 ### Updated: Sat Sep 4 10:03:39 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2568 ### Version: V4.2021.09.2569
### Updated: Fri Sep 3 22:01:12 UTC 2021 ### Updated: Sat Sep 4 10:03:39 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##