vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.08.3297 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-08-18 10:09:20 +00:00
parent 70c121079c
commit 7607e448fa
47 changed files with 84737 additions and 84727 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.08.3296 #### Version: V4.2022.08.3297
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.08.3296 #### Version: V4.2022.08.3297
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.08.3296 #### Version: V4.2022.08.3297
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10456
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.08.3296 #### Version: V4.2022.08.3297
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10460
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10460
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10460
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10460
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10460
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10460
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10460
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3296 3297

10460
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10460
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.143.24 0; 20.124.8.134 0;

10460
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10460
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10460
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

10460
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.143.24 0; 20.124.8.134 0;

10460
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.143.24 0; 20.124.8.134 0;

10460
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 1; 10.1.0.244 1;
127.0.0.1 1; 127.0.0.1 1;

16
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Aboundexbot
BLEXBot
BackStreet BackStreet
Crawling at Home Project CheeseBot
DBLBot Ebingbong
Digincore InterGET
Image Fetch
MIDown tool
Mozlila Mozlila
seobility SMTBot
SiteExplorer
V-BOT
Whack
x09Mozilla

312
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
Abonti 404checker
ALittle Client
Aboundexbot Aboundexbot
Acunetix AiHitBot
AhrefsBot Alexibot
Alligator Alligator
AlphaBot Anarchie
Anarchy99 Anarchy
Anthill Apexoo
Aspiegel Asterias
Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot BDCbot
BBBike
BDFetch
BLEXBot
BackDoorBot BackDoorBot
BackStreet BackStreet
BacklinkCrawler
Bandit Bandit
Barkrowler
BatchFTP
BetaBot BetaBot
Bigfoot Blow
Black Hole
BlowFish BlowFish
Boardreader BotALot
Brandwatch
Buck Buck
BuiltWith Buddy
BuzzSumo Bullseye
CATExplorador BunnySlippers
CODE87 CCBot
ChinaClaw Calculon
Cocolyzebot CazoodleBot
Cegbfeieh
CensysInspect
CheeseBot
Cliqzbot
Copier Copier
CopyRightCheck CopyRightCheck
Copyscape Cosmos
Craftbot CrazyWebCrawler
Crawling at Home Project
CrunchBot CrunchBot
DBLBot Curious
DTS Agent Custo
DataCha0s CyotekWebCopy
DatabaseDriverMysqli DSearch
Demon
Deusu Deusu
Devil Devil
Digincore
DigitalPebble
Disco Disco
Discoverybot Discobot
DittoSpyder
DnBCrawler-Analytics
DnyzBot DnyzBot
DomCopBot
DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler
Domains Project Domains Project
Download Wonder Dotbot
Drip Dragonfly
EMail Siphon EMail Siphon
EMail Wolf Ebingbong
EirGrabber EroCrawler
Exabot Exabot
ExtLinksBot Express WebPictures
Extractor
FHscan FHscan
FemtosearchBot
FlashGet
Freeuploader Freeuploader
FrontPage
Fuzz Fuzz
FyberSpider FyberSpider
Fyrebot Fyrebot
GalaxyBot
Genieo Genieo
GetRight
GetWeb GetWeb
Gigabot Gigabot
Go!Zilla
Gotit
GrabNet GrabNet
Grabber Grafula
GrapeFX
HEADMasterSEO HEADMasterSEO
HTMLparser HTMLparser
HTTP::Lite HTTP::Lite
HTTrack
Haansoft
HaosouSpider
Harvest
Heritrix Heritrix
Hloader Humanlinks
IRLbot
Id-search Id-search
IlseBot IlseBot
Image Fetch Indy Library
Image Sucker InterGET
InfoTekies
Intelliseek
Iskanie Iskanie
Jbrofuzz IstellaBot
JennyBot JOC Web Spider
JamesBOT
Jetty Jetty
JikeSpider
JustView
Jyxobot Jyxobot
Kenjin Spider
Keyword Density Keyword Density
Kinza
LNSpiderguy LNSpiderguy
LibWeb Leap
LinkWalker LeechGet
Linkbot Lftp
LinkScan
LinkpadBot LinkpadBot
LinksManager LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey Lipperhey
MIDown tool Lipperhey Spider
Ltx71
MFC_Tear_Sample
MQQBrowser MQQBrowser
MSFrontPage MSFrontPage
Mag-Net MSIECrawler
Mail.RU_Bot Magnet
Majestic12
MarkMonitor
MarkWatch MarkWatch
Mass Downloader
Masscan Masscan
Mata Hari
MeanPath Bot MeanPath Bot
Meanpathbot Mediatoolkitbot
Metauri
MicroMessenger
Microsoft Data Access Microsoft Data Access
Microsoft URL Control
Minefield Minefield
Mister PiX Mister PiX
Mojeek Mozlila
Morfeus Fucking Scanner Msrabot
Musobot NPbot
Name Intelligence Needle
Navroad Net Vampire
NearSite
NetLyzer NetLyzer
NetMechanic NetMechanic
NetSpider
NetZIP NetZIP
Netvibes Netvibes
Nibbler NextGenSearchBot
Niki-bot Ninja
Nmap Nuclei
Offline Navigator Offline Navigator
OpenVAS OnCrawl
OrangeSpider Openfind
OutfoxBot OutclicksBot
PageAnalyzer PECL::HTTP
POE-Component-Client-HTTP
Page Analyzer
PageGrabber
PageScorer PageScorer
PageThing.com Pavuk
Pandalytics Pi-Monster
Petalbot Pimonster
Pockey Pockey
ProWebWalker ProPowerBot
Pu_iN Probethenet
Psbot
Pump Pump
PxBroker
PyCurl PyCurl
Quick-Crawler Quick-Crawler
RSSingBot RSSingBot
RankActiveLinkBot RankFlex
RankingBot
RankingBot2 RankingBot2
Reaper Rankivabot
Recorder RealDownload
RedesScrapy RebelMouse
RocketCrawler RocketCrawler
SalesIntelligent SEOlyticsCrawler
SEOprofiler
ScanAlert
Scanbot Scanbot
ScreenerBot
ScrepyBot ScrepyBot
Searchestate Searchestate
SemanticJuice
Semrush
SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Seomoz Seomoz
Shodan Shodan
Siphon SiteLockSpider
SiteSnagger SiteSucker
Sitebeam SlySearch
Siteimprove Snapbot
Sitevigil Snoopy
Snake
SocialRankIOBot SocialRankIOBot
Sosospider Sociscraper
SpankBot SpankBot
Spbot Spinn3r
SurveyBot SputnikBot
Stripper
Sucker
Sucuri
SuperHTTP
Suzuran Suzuran
Swiftbot
Szukacz Szukacz
T0PHackTeam Telesphoreo
Teleport
Telesphorep Telesphorep
TightTwatBot The Intraformant
Titan Thumbor
Trendiction Toweyabot
True_Robot True_Robot
URLy.Warning Turnitin
UnisterBot Twice
VB Project URLy Warning
VelenPublicWebCrawler V-BOT
VCI
VeriCiteCrawler VeriCiteCrawler
Virusdie VidibleScraper
WBSearchBot WPScan
WEBDAV WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WallpapersHD Web Fuck
Web Auto Web Sucker
Web Enhancer WebBandit
Web Pix WebEnhancer
WebFetch
WebFuck WebFuck
WebGo IS WebSauger
WebPix WebsiteQuester
WebWhacker Whacker
WebZIP WinHTTrack
WebmasterWorldForumBot Wonderbot
WebsiteExtractor Wprecon
Whatweb Xaldon_WebSpider
Woobot
Xenu Xenu
YoudaoBot Zeus
Zauba Zitebot
Zermelo
ZoomBot ZoomBot
ZumBot ZoominfoBot
ZyBorg adscanner
arquivo.pt arquivo-web-crawler
autoemailspider backlink-check
coccocbot coccocbot
dataforseo.com cognitiveseo
eCatch crawl.sogou.com
gopher crawler.feedback
instabid crawler4j
domainsproject.org
facebookscraper
ips-agent
isitwp.com isitwp.com
iubenda-radar iubenda-radar
linkdexbot muhstik-scan
magpie-crawler
netEstate NE Crawler netEstate NE Crawler
oBot page scorer
pcBrowser polaris version
plumanalytics
probe-image-size probe-image-size
ripz
satoristudio.net
scalaj-http scalaj-http
seobility
seocompany.store
seoscanners seoscanners
sogouspider sitechecker.pro
sp_auditbot sp_auditbot
spyfu
sysscan sysscan
trendiction.com trendiction.com
voyagerx.com trendiction.de
webgains-bot webgains-bot
webmeup-crawler webmeup-crawler
webprosbot x09Mozilla
zauba.io zauba.io

298
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com ADmantX
404enemy
AIBOT
ASPSeek ASPSeek
Abonti Abonti
AfD-Verbotsverfahren Aboundex
Aipbot Aipbot
Alexibot AllSubmitter
Alligator Alligator
AlphaBot
Anarchie Anarchie
Anthill
Apexoo Apexoo
Aspiegel Aspiegel
Asterias Attach
BDCbot
BDFetch BDFetch
BLEXBot
BackStreet BackStreet
Badass
Bandit Bandit
Barkrowler
BatchFTP BatchFTP
Battleztar Bazinga
BetaBot BetaBot
Bigfoot
Bitacle Bitacle
Black Hole Black Hole
Boardreader Blackboard
BotALot Blow
Bolt
Brandprotect
Buck
Buddy Buddy
BunnySlippers CSHttp
CCBot
Calculon Calculon
ChinaClaw CheTeam
Cogentbot Cocolyzebot
Copier Craftbot
Copyscape Crawling at Home Project
Curious CrazyWebCrawler
Custo Crescent
CyotekWebCopy DBLBot
DIIbot DIIbot
DSearch
DTS Agent
DatabaseDriverMysqli
Demon Demon
Deusu Devil
DigitalPebble
Dirbuster
Disco
Discobot
DittoSpyder DittoSpyder
DomCopBot DnBCrawler-Analytics
DomainAppender DomainCrawler
DomainStatsBot Domains Project
Download Wonder Dragonfly
ECCP/1.0 ECCP/1.0
EMail Siphon EMail Siphon
EasyDL EMail Wolf
EroCrawler EroCrawler
Evil
Exabot
ExtLinksBot ExtLinksBot
FHscan FHscan
FemtosearchBot FemtosearchBot
Fimap Fimap
FlashGet FlashGet
Fuzz Flunky
Foobot
Fyrebot
G-i-g-a-b-o-t
GalaxyBot GalaxyBot
GermCrawler Genieo
GetRight
GetWeb
Go!Zilla Go!Zilla
Go-Ahead-Got-It
Grabber
GrapeFX GrapeFX
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO HTTrack
HMView HaosouSpider
HTTP::Lite
Harvest
Havij
Heritrix Heritrix
Hloader HonoluluBot
Image Fetch Image Sucker
IndeedBot
Indy Library Indy Library
InfoNaviRobot Intelliseek
InfoTekies Internet Ninja
InterGET IstellaBot
InternetSeer JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz
JennyBot JennyBot
JetCar JikeSpider
Joomla JustView
Jyxobot
Kenjin Spider Kenjin Spider
Keyword Density Keyword Density
Kozmosbot Kozmosbot
LWP::Simple LNSpiderguy
LexiBot Lanshanbot
LibWeb LeechGet
LinkWalker LieBaoFast
LinkextractorPro
LinksManager
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
MFC_Tear_Sample Lmspider
MJ12bot MIIxpc
MQQBrowser
MTRobot
Mail.RU_Bot Mail.RU_Bot
Majestic12 Majestic-SEO
Masscan
Mata Hari
MauiBot
Mb2345Browser Mb2345Browser
Mediatoolkitbot
Metauri Metauri
MicroMessenger Microsoft URL Control
Minefield Mister PiX
Mojeek Mozlila
Mr.4x3
Msrabot Msrabot
Musobot Musobot
NPbot Name Intelligence
Nameprotect NetAnts
NearSite NetLyzer
NetSpider NetMechanic
NetZIP Netcraft
Nettrack
Netvibes Netvibes
Nibbler Nibbler
Niki-bot NimbleCrawler
Ninja Nimbostratus
Nutch Not
Octopus Offline Explorer
Offline Navigator Offline Navigator
OnCrawl OnCrawl
OpenVAS
Openfind
Openvas Openvas
OrangeBot OrangeBot
OrangeSpider PECL::HTTP
OutclicksBot Page Analyzer
OutfoxBot PageAnalyzer
POE-Component-Client-HTTP
PageGrabber PageGrabber
PageScorer
PageThing.com
Papa Foto Papa Foto
PeoplePal PeoplePal
Picsearch Petalbot
PictureFinder Picscout
Piepmatz
Pimonster Pimonster
Pixray ProPowerBot
Pockey
ProWebWalker
Pu_iN
Pump Pump
QueryN Metasearch PyCurl
RSSingBot
RankActive RankActive
RankActiveLinkBot RankingBot
RankFlex
RankurBot
Re-re Re-re
ReGet ReGet
RepoMonkey
Ripper Ripper
RocketCrawler
Rogerbot Rogerbot
SBIder
SEOkicks
SEOlyticsCrawler SEOlyticsCrawler
SISTRIX Scanbot
ScoutJet ScoutJet
Scrapy
ScrepyBot
SearchmetricsBot
Seekport Seekport
SentiBot SeoSiteCheckup
Seomoz SiteExplorer
Siphon
Site Sucker
SiteSnagger
SiteSucker SiteSucker
Sitebeam
Siteimprove
Sitevigil Sitevigil
SmartDownload Snake
Sociscraper
Sogou web spider Sogou web spider
Sosospider Spammen
SpaceBison
SpankBot
Spbot Spbot
Spinn3r
SputnikBot
Sqlworm Sqlworm
Sqworm Surfbot
Stripper Swiftbot
Telesphoreo Teleport
TeleportPro
The Intraformant
TheNomad
Thumbor
Tracemyfile Tracemyfile
Trendiction Trendiction
Turnitin True_Robot
Turingos
TurnitinBot TurnitinBot
Typhoeus Typhoeus
URLy Warning URLy Warning
UnisterBot UnisterBot
Upflow V-BOT
VB Project VB Project
VCI VelenPublicWebCrawler
VoidEYE VidibleScraper
Voil WASALive-Bot
WBSearchBot
WEBDAV
WISENutbot
Wallpapers Wallpapers
Web Auto Wallpapers/3.0
Web Enhancer WallpapersHD
Web Sauger WeSEE
Web Collage
Web Sucker
WebBandit WebBandit
WebCollage
WebCopier WebCopier
WebFetch
WebFuck
WebGo IS
WebImageCollector WebImageCollector
WebStripper WebReaper
WebSucker WebWhacker
Webalta WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Website Quester WebsiteQuester
Whatweb Webster
Who.is Bot
Widow Widow
WinHTTrack WiseGuys Robot
Wonderbot Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Xenu Zauba
YoudaoBot
Zade
Zermelo Zermelo
ZmEu Zeus
ZumBot ZoomBot
adscanner ZoominfoBot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt
autoemailspider
backlink-check backlink-check
cah.io.community check1.exe
com.plumanalytics coccocbot
cognitiveseo
crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j
dataforseo.com dataforseo.com
evc-batch demandbase-bot
heritrix domainsproject.org
eCatch
facebookscraper
gopher
instabid instabid
ips-agent
isitwp.com
lwp-trivial lwp-trivial
magpie-crawler
meanpathbot meanpathbot
mediawords mediawords
muhstik-scan
oBot
pcBrowser pcBrowser
polaris version polaris version
ripz
s1z.ru
scalaj-http scalaj-http
scan.lol seostar
seocompany.store
seoscanners
serpstatbot serpstatbot
siteripz siteripz
spyfu
sysscan
trendiction.com
webpros.com webpros.com
webprosbot webprosbot
x09Mozilla x09Mozilla
zgrab zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
7search.com azinoofficial777.ru
audiobangout.com copypaste.traderzplanet.in
bus-offer.com dosug-lux.ru
cardiosport.com.ua enthuse.computernetworksonline.com
carloans.com eyemagination.com
electricwheelchairsarea.com host-tracker.com
intellektmedia.at johnnyhaley.top
pogruztehnik.ru kerei.ru
shtaketniki.kz lesbian.xyz
taihouse.ru thepokertimer.com

928
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00it.com 000free.us
0daymusic.org 00go.com
1-free-share-buttons.com 101lesbian.xyz
100dollars-seo.com 11pikachu.ru
202ch.com 123movies.love
256bit.by 12bet.com
2728fb936f0.com 12masterov.com
40cg.com 1kinobig.ru
4ureyesonly.com 1webmaster.ml
999webdesign.com 23kw.ru
aa8b68101d388c446389283820863176e7.com 2drittel.de
4kpics.rocks
5elementov.ru
7xc4n.com
888.com
aa8780bb28a1de4eb5bff33c28a218a930.com
aa9d046aab36af4ff182f097f840430d51.com
aad1f4acb0a373420d9b0c4202d38d94fa.com
aarbur.com aarbur.com
absolutelyanalog.com abcdeg.xyz
adconscious.com acortarurl.es
addbin.men activepr.ru
admeasures.com admeasures.com
adminshop.com adminshop.com
adsfresh.men adrunnr.com
adssafeprotected.com adtology.com
adtech.us adult3dgames.com
adult-shop.com.ua advancedmassagebysara.com
affiliate-fr.com alarmobninsk.ru
afora.ru alert.scansafe.net
alert-fdm.xyz alfa9.com
algarveglobal.com algarveglobal.com
alive-ua.com alienwheel.es
allblogroll.com all4wap.ru
allknow.info
allkrim.com allkrim.com
allornamenti.com alphacarolinas.com
alpharma.net alyeskaresort.com
android-style.com amateurlivechat.org
anime.dougasouko.com anonymousfox.co
anlimebel.kiev.ua aosheng-tech.com
apartmentratings.com apartamentwroclaw.eu
apibrowsesmartne-a.akamaihd.net apartment.ru
apimegabrowsebiz-a.akamaihd.net apartmentbay.ru
approved.su apiitsthirteende-a.akamaihd.net
arcteryxstore.online appsaurus.com
arius.tech arcadepages.com
artavenuegardenstudios.com aruplighting.com
ascat.porn asrv-a.akamoihd.net
asia-forum.ru ausmepa.org.au
asiavirtualsolutions.com autoseo-traffic.com
atley.eu.pn autovideobroadcast.com
australia-opening-times.com avtointeres.ru
autolombard-krasnodar.ru
aviav.co
avon-severozapad.ru
avtolombard-krasnodar.com
avtovolop.ru
ayakino.net
babespcs.com
backlinkwatch.com backlinkwatch.com
bala.getenjoyment.net backuperwebcam.weebly.com
bambi.ck.ua balook.com
bannertesting.com bear.gotcher.us
basedecor.ru beauty-b0x.pl
bestplacetobuyeriacta.jw.lt beauty-things.com
bet-prognoz.com beclean-nn.ru
bget.ru becuo.com
belstaffstore.online
bestempresas.es
bestmobilityscooterstoday.com
bestofferswalkmydogouteveryday.gq
besttorrentknifta.weebly.com
beztuberkuleza.ru
bif-ru.info
big-boards.info big-boards.info
bio-optomarket.ru bigpenisguide.com
bizlist.com.de bildsuche.ru
bjorkbacken.nu bioca.org
blog-hits.com biznesschat.net
blog.koreadaily.com bizzliving.com
boc.kz bloggerads.net
blogporn.in
bluesman.nu
bmw-ark.ru
bolitgorlo.net bolitgorlo.net
bonvillan.com
bookmaker-bet.com bookmaker-bet.com
bookmark4you.com bookmark4you.com.biz
boom-celebs.com braindaily.xyz
bosefux.esy.es
braintobe.win braintobe.win
buigas.00it.com breget74.ru
buqayy0.livejournal.com brillianty.info
bus-offer.com brokergid.ru
buketeg.xyz
buqyxa.rincian.info
business-made-fun.com business-made-fun.com
buy-cheap-pills-order-online.com buyparajumpers.online
buyclomidonlaine.com bytimedance.ru
buyessaynow.biz calstaterealty.us
buypillsorderonline.com cand.jp
bwlx.prepedu.cn cartierlovebracelet.xyz
bycontext.com cartujano-pre.de
c-gainsbourg.com cashkitten-a.akamaihd.net
candypeople.se cdncash.com
candytiner.com chastnoeporno.com
canvas.gloverid.site cheap-trusted-backlinks.com
cartierwatch.xyz checkhit.com
casinopinup-wins.com
casite-513049.cloudaccess.net
cellfun.mobi
certifywebsite.win
cheapjerseysfootballshop.com
cheapwebsitehoster.com
check-host.net
checkm8.com
chidporn.com
chocolateslim-u-srbiji.com chocolateslim-u-srbiji.com
ci.ua cityadspix.com
cjmc.info citysecurity.nu
clean-start.net cleansearch.net
coinsspb.com clicksor.com
companies-catalog.com clipartnew.com
compliance-brian.xyz cocolyze.com
compliance-ivan.top com-supportcenter.website
compliance-jessica.top compliance-alex.top
coolgramgoods.com compliance-julianna.top
courtshipgift.com conboy.us
covadhosting.biz concordexoticrentals.com
crackzplanet.com constantaservice.net
cookielawblog.wordpress.com
coolbar.pro
copenergo.ru
crackguru.tk
crafthubs.com
craftinsta.ru
crynet.cc
custodieva.ru
cv.wallhade.co cv.wallhade.co
cyprusbuyproperties.com cvety24.by
d0t.ru cwetochki.ru
damasarenai.info dailyrank.net
dame-ns.kz data-mining.tk
daooda.com data-ox.com
datadepths.com daydream-studio.ru
ddpills.com ddospanel.com
ddsoldes.fr de.zapmeta.com
delitime.info deanmoore.ie
demon-tweeks.com decorazilla.com
design-lands.ru demenageur.com
detskie-zabavi.ru dieswaene.com
detsky-nabytek.info diminishedvalueoforegon.com
discovertreasurenow.com dmmspy.com
djstools.com doeco.ru
dlya-android.org dojki365.online
dms-sw.ru doktoronline.no
dneprsvet.com.ua dollartree.info
dodlive.mil dosugrostov.site
dogclothing.store doxysexy.com
dogoimage.com dscaas.website
dojki-hd.com dyt.net
dokumentalkino.net ebooknovel.club
domaincrawler.com ecommerce-seo.org
domik-derevne.ru ekspresihati.info
download-of-the-warez.blogspot.com elektrischezigarette1.onsugar.com
downvids.net elektrozigarettekaufen1.devhub.com
drupa.com eloxal.ru
e-avon.ru empirepoker.com
earl-brown.info empis.magix.net
edwinkonijn.com.au en.altezza.travel
egovaleo.it
eleimgo.pw
energydiet-info.ru energydiet-info.ru
enskedesquashclub.se erotik-kostenlos.net
envaseslotusama.com excitacao.com
erectile.bid extlinks.com
erotische-geschichten-xxl.com ezaz.info
errorfixing.space falcoware.com
etm-consult.de fanpagerobot.com
exoneration-project.us fbdownloader.com
eyemagination.com
eywords-monitoring-your-success.com
ezofest.sk
famix.xyz
favorcosmetics.com
fdzone.org
feargames.ru
feel-planet.com
felizporno.com
fenoyl.batcave.net
fermersovet.ru fermersovet.ru
fickkontaktehobbyhuren.com filmbokep69.com
fidelityfunding.com financeloan.us
filesmonster.porn findclan.org
finuse.com finejewelryshop.ru
floridamhca.org fishingwholesale.us
forexgb.ru focalink.com
formulaf1results.blogspot.com forpostlock.ru
free-laptop-reward.com foxjuegos.com
frustrated-favorable.gq franch.info
gabeshop.ru frbizlist.com
game-top.su freeuploader.com
gameonasia.com frequiry.com
ge0ip.org friendflnder.com
gearcraft.us funcrushgames.com
geile-lelly.eu funnypica.com
generalporn.org funponsel.com
genetworx.com fusoradio.info
glall.ru galaxyflowers.ru
glavprofit.ru gallery.rennlist.com
goldenggames.com galleryawesome.com
gombita.info gamerscorps.com
guildebzh.info gateway.zscalerone.net
gayxperience.com
getdot.ru
getprismatic.com
gigapeta.com
global-ics.co.za
goosefishpost.bid
gorabagrata.ru
gorodservis.ru
gosarhivrt.ru
gourcy.altervista.org
green-tea.tv
gungamesz.com gungamesz.com
hanwei.us habermetre.com
harmonyglen.us hao123.com
havepussy.com hd-film.pl
hildinghr.se helvetia.com.ua
hmywwogw.bloger.index.hr highstairs-a.akamaihd.net
hobild.net histats.com
hostcritique.com hit-men.men
hoste.octopis.com hitcpm.com
hoterika.com home.myplaycity.com
humanelydrew.com hornymatches.com
hunthillfarmtrust.org hotblognetwork.com
hyhj.info hotgirlhdwallpaper.com
hothor.se
hotloans.ru
hotxnights.info
houston-vikings.com
houtings.xyz
hplaserjetpdriver8y.pen.io
hybrid.ru
iboss.com iboss.com
idealtits.net icaseclub.ru
ideawheel.com ideibiznesa2015.ru
iideaidekonkatu.info
ilbe.club
ilovevitaly.xyz ilovevitaly.xyz
ilte.info img.wallpaperstock.net
imadedinner.net imgarcade.com
imgata.com imgata.com
imicrovision.com inclk.com
interferencer.ru indo-export.ru
invitefashion.com infobabki.ru
iopeninghours.co.uk instakink.com
isoveti.ru iscblog.info
itrevolution.cf istripper.com
itronics.ca ivoiretechnocom.ci
ivearchenceinflu.cf izhevsk.zrus.org
ix20.ru izoll.ru
ixora.pro james13prix.info
javatex.co.id janerikholst.se
je7.us jav-way.site
johnrobertsoninc.com jerseysbizwholesalecheap.com
jolic2.com jo24news.com
juliaworld.net kambasoft.com
karaganda.xkaz.org karachev-city.ru
kaz.kz katushka.net
kino-fun.ru kenaba.su
kirov.zrus.org kfon.eu
komputers-best.ru kicknights.gq
kostenloser-sex.com kihi.gdn
krynica.info kinky-fetishes.com
ks1234.com kinogolos.ru
lapitec.eu konkursowo-24.pl
lapolis.it kozhniebolezni.com
lechtaczka.net krasula.pp.ua
lernur.net kretpicf.pw
lescinq.com kriokomora.info
livetsomudvekslingstudent.bloggersdelight.dk kskjco.club
ljusihus.se kupiproday.com.ua
lolnex.us ledpolice.ru
lombia.co letmacwork.world
lordzfilmz.me letsart.ru
lostaruban.com lietaer.com
lsp-awak-perikanan.com limtu.ifmo.ru
luchshie-topcasino.ru linerdrilling.com
link.ac
lirunet.ru
liveinternet.ro
loftdigital.eu
lost-alpha.ru
lostfilm-online.ru
louboutinbooties.xyz
lovi-moment.com.ua
lrac.info
lunamedia.co lunamedia.co
m5home.ru luxmagazine.cf
mailemedicinals.com mackeeper-center.club
manifestyourmillion.com maglid.ru
manve.info maju.bluesalt.co
mapquestz.us maladot.com
massage-info.nl maranbrinfo.com.br
maximpartnerspr.com masturbate.co.uk
mcnamaratech.com matchpal-a.akamaihd.net
mebelcomplekt.ru mathgym.com.au
mecash.ru mazda-roadsters.com
medanestesia.ru mcadamssupplyco.com
medkletki.ru mebel-vstroika.ru
meet-flirt-dating.com medizinreisen.de
megaapteka.ru mega-bony-2017.pl
menhealed.net megakino.net
minecraft-neo.ru members.ghanaweb.com
mixtapetorrent.com menetie.ru
monclercheap.xyz michaelkorsoutletstore.net
more-letom.ru mmog-play.ru
mouselink.co mobicover.com.ua
msk.afora.ru mobsfun.net
musicdaddy.net mockupui.com
musicspire.online moivestiy.biz
muycerdas.xxx monclerjacketsoutlet.win
mycouponizemac.com moneymaster.ru
myfreetutorials.com moneyviking-a.akamaihd.net
myliveblog.ru morpicert.pw
myphotopipe.com mostantikor.ru
mywallpaper.top moxo.com
myxdate.info moyakuhnia.ru
naj-filmy24.pl mustat.com
nakozhe.com mykings.pw
narkologiya-belgorod.ru mypornfree.ru
narosty.com mystats.xyz
naturtreenspicerx.pw naperehresti.info
net-radar.com neks.info
netcheckcdn.xyz new-apps.ru
newhairstylesformen2014.com nibbler.silktide.com
newstraveller.ru nonews.co
nextrent-crimea.ru northfacestore.online
nhl17coinsforps3.gratisblog.biz norththeface.store
nicovideo.jp obiavo.by
niki-mlt.ru obiavo.kz
normalegal.ru obiavo.ru
nudepatch.net odywpjtw.bloger.index.hr
nullrefer.com omgtnc.com
nyfinance.ml online-x.ru
oakridgemo.com oralsexfilme.net
obiavo.net
ochistka-stokov.ru
oecnhs.info
offtime.ru
oivcvx.website
ok.ru
okeinfo.online
onclickpredictiv.com
onemantrip.com
ontargetseo.us
ooo-gotovie.ru
oops-cinema.ru
ordernorxx.com ordernorxx.com
orgasmatrix.com ozoz.it
orgasmus-virtual.com pacificair.com
osnova3.ru
ostroike.org
oxford-book.com.ua
oz-offers.com
painting-planet.com painting-planet.com
pastaleads.com par-fallen.ga
pavlodar.xkaz.org paradontozanet.ru
pdamods.ru parajumpersoutlet.online
perosan.com partners-ship.pro
picanalyzer.data-ox.com perfectpracticeweb.com
pictures-and-images.com pixelrz.com
piski.top poems.com.ua
piuminiita.com poisk-zakona.ru
pixgood.com porngalleries.top
pixshark.com porno-video-chati.ru
platesauto.com pornohd1080.online
pogosh.com pornohub.me
polytopesexempt.com pornoted.com
pornoinn.com positive2b.ru
pornoklad.net pourvous.info
pornophoto.xyz
pp-budpostach.com.ua pp-budpostach.com.ua
prizesbook.online praisong.net
profitkode.com preparevideosafesystem4unow.site
progress-upakovka.ru princevc.com
promalp-universal.ru printie.com
pronekut.com printingpeach.com
proposal-engine.com privetsochi.ru
prosmibank.ru prizefestival.mobi
prosperent.com prohoster.info
prostitutki-tyumeni.org projefrio.com.br
pufip.com prokotov.com
pukaporn.com prostitutki-odessa.org
qcstrtvt.bloger.index.hr psvita.ru
pts163.ru
putevka24.ru
putitin.me
qor360.com qor360.com
rangjued.com quick-offer.com
rankia.com quickloanbank.com
ranksays.com ranksonic.info
redirectingat.com realnye-otzyvy.info
rednise.com reckonstat.info
regdefense.com recordpage-a.akamaihd.net
registry-cleaner.net registratciya-v-moskve.ru
relatodelpresente.com.ar remontvsamare.su
responsinator.com remote-dba.de
rialp.getenjoyment.net rep-am.com
riders.ro repeatlogo.co.uk
rn-to-bsn.com respublica-otel.ru
rumamba.com resultshub-a.akamaihd.net
runstocks.com revistaindustria.com
rus-pornuha.com rubanners.com
saletool.ru rucrypt.com
saltspray.ru sa-live.com
santechnik.jimdo.com
sardinie.us sardinie.us
savetubevideo.com sbprabooks.com
savingsslider-a.akamaihd.net scanner-mark.top
scanner-jess.top
scanner-jessica.top
scanner-john.top
scanner-margo.top
scanner-olga.top
scenarii-1-sentyabrya.uroki.org.ua
schlampen-treffen.com schlampen-treffen.com
scrapinghub.com scopich.com
sdi-pme.com scripted.com
search-goo.com searchaddis.com
searchinterneat-a.akamaihd.net searchkut.com
seasaltwithfood.com seksvideoonlain.com
seorankinglinks.xyz sendearnings.com
sex-znakomstva.online seo-tools-optimizing.com
sexkontakteao.info seo18.su
sexkrasivo.net sexblog.pw
sexphoto.site sexkvartal.com
sexuria.net sextracker.be
sexy-screen-savers.com sextracker.de
sexyebonyteen.com
shakhtar-doneck.ru
share-buttons-for-free.com share-buttons-for-free.com
sharebutton.org sharpchallenge.com
shopsellcardsdumps.com shiksabd.com
sibecoprom.ru shoppingjequiti.com.br
simple-share-buttons.com siha.de
simul.co simple-image.com.ua
site-speed-checker.site sitechecker.pro
skatestick.bid sk.golden-praga.ru
skylta.com skylta.com
sledstvie-veli.net social-vestnik.ru
slimcdn.com socialseet.ru
small-game.com softomix.com
socialmadesimple.com sogimlecal.tk
solmarket.by
sophang8.com sophang8.com
spasswelt.net souvenir.cc
spbchampionat.ru spb.ru
splendorsearch-a.akamaihd.net
sports-supplements.us
spravka-medosmotr.ru
spravka130.ru spravka130.ru
stanthonyscatholicchurch.org spy-app.info
starwars.wikia.com spywarebegone.com
statoutlook.info start.myplaycity.com
statustroll.com storehouse.ua
stefanbakosab.se strfls.com
stpolice.com stroiminsk.com
strana-krasoty.ru stromerrealty.com
stroicol.net strongholdsb.ru
stroyplus.ru strv.se
swagbucks.com studiofmp.com
sygraem.com studiokamyk.com.pl
tastyfoodideas.com sugvant.ru
teguh.info supermodni.com.ua
textads.men supervesti.ru
thebluenoodle.com suture.co
theendivechronicles.com svbur.ru
thegolfclub.info svetodiodoff.ru
theporndude.com szucs.ru
thetoiletpaper.com tackletarts.co
thfox.com telefonsexi.com
tiandeural.ru thebestweightlosspills.ovh
time-japan.ru thegalerie.eu
timecrimea.ru themeforest.net
tizanidine4mgtablets.blogspot.com theplacetoupdating.pw
tizanidinecost.blogspot.com thepornsex.org
tizanidineinteractions.blogspot.com threecolumnblogger.com
tomatis.gospartner.com titan-cloud.life
topvidos.ru tizanidineandhydrocodone.blogspot.com
toys.erolove.in tizanidinemusclerelaxant.blogspot.com
tradgardspartner.se tizanidinenarcotic.blogspot.com
trafficjunky.net tizanidinerestlesslegsyndrome.blogspot.com
treasuretrack-a.akamaihd.net toondinsey.com
trustedmaccleaner.com tootoo.to
tupper-shop.ru topads.men
turizmus.us tour-line.net
tvteleport.ru tozup.com
uasb.ru trafficshaper.com
ublaze.ru tramadolandtizanidine.blogspot.com
uchebavchehii.ru turist-strani.ru
ukrobstep.com turvgori.ru
unblocksit.es tvand.ru
u555u.info
uchetunet.su
ugguk.online
uhdtv.website
ukkelberg.no
umekana.ru
underthesite.com underthesite.com
url-img.link uytmaster.ru
usiad.net
usswrite.com
valmetrundan.se
valsalud.com
vandrie-ict.nl vandrie-ict.nl
vegan-foods.us varikozdok.ru
vertaform.com vbabule.net
viandpet.com veerotech.com
video-chat.in vekzdorov.ru
videochaty.ru velen.io
videonsk.com vesnatehno.ru
visa-china.ru via-energy-acquistare.com
vpnhowto.info videogamesecrets.com
vsexkontakte.net videotuber.ru
webextract.profound.net vipcallsgirls.com
weblibrary.win vipps.com.my
website-analytics.online viveresaniesnelli.it
wedding-salon.net vkgaleria.com
wesharepics.com vkontaktemusic.ru
westermarkanjou.se vladhistory.com
where-toget.com vonradio.com
wholesalecheapjerseysfree.com vroze.com
wholesalejerseys-cheapest.com w3javascript.com
wildcattube.com webix.biz
webshoppermac.com
websiteaccountant.de
websiteexplorer.info
wejdz-tu.pl
wineration.com
wingsoffury2.com wingsoffury2.com
winner7777.net wpsecurity.website
wladimirpayen.com
wnoz.de
wrz0iuebwhp5fg.freeddns.com wrz0iuebwhp5fg.freeddns.com
www.labves.ru xboxster.ru
x-lime.com xedserver.com
x-stars.ru xjlottery.com
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn--80aeahghtf8ac5i.xn--p1ai xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xn--80aikhbrhr.net xn----7sbho2agebbhlivy.xn--p1ai
xn--lifehacer-1rb.com xn----dtbndd4ae7eub.top
xnxx-n.com xn--e1aggki3c.xn--80adxhks
xnxx699.com xn--l1aengat.xn--p1ai
xsion.net xvideosbay.com
xxxmania.top xxxtube69.com
xz618.com yellowfootprints.com
yebocasino.com
youbloodyripper.com youbloodyripper.com
youghbould.wordpress.com yubikk.info
youtube-downloader.savetubevideo.com zigarettenonl.canalblog.com
zastenchivosti.net zone-kev717.info
zeroredirect1.com
zeroredirect10.com
zeroredirect7.com
zhorapankratov7.blogspot.com
znaturaloriginal.com
zoodrawings.com
zoomovies.org zoomovies.org
zryydi.com
zzlgxh.com

472
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
24x7-server-support.site 00author.com
273-fz.ru 202ch.com
4replicawatch.net 23kw.ru
2nt.ru
8jn.dating-app.ru 8jn.dating-app.ru
abctoppictures.net 8lufu.com
adminshop.com aadroid.net
adzpower.com advancedmassagebysara.com
aihelen.net ageofclones.com
alldezservice.kz amazingpic.net
amazon-seo-service.com anaseracresar.tk
amung.us anniemation.com
androids-store.com anonymous-redirect.com
anime.dougasouko.com ascat.porn
apifasterlightin-a.akamaihd.net autoloans.com
appfastplay.com avtolombard-krasnodar.com
arenda-avtoprokat-krasnodar.ru ayerbo.xhost.ro
asrv-a.akamaihd.net azbukafree.com
atelielembrancaqueencanta.com.br balook.com
azlex.uz bannerswap.com
bank.uz beauby.ru
bannerbank.ru best-coupon-offer.com
beachdriveblog.com bestadbid.com
becuo.com bestdraws.com
bestimagecoollibrary.com bestmobilityscooterstoday.com
betshuckclean.com beta.hotkeys.com
bif-ru.info biglistofwebsites.com
binomo.kz binomo.com
biznesrealnost.ru bizlist.com.de
bonanza-fortune.men blog.yam.com
bonky.biz bochemit.com.ua
book-bets.com bombla.org
buttons-for-website.com browsepulse-a.akamaihd.net
buyviagraa.blogspot.com buy-forum.ru
c1.onedmp.com c1.onedmp.com
calcularpagerank.com callawaygolfstore.online
callstevens.com calstaterealty.us
cam-kontakte.org candytiner.com
carrauterie.be cartierbraceletsreplica.pw
cheapmoncler.win cartierlovebracelet.xyz
casablancamanor.co.za
cdncash.org
cheapmoncler.xyz cheapmoncler.xyz
cialisovercounteratwalmartusa.com checkm8.com
clarithromycin500mg.com choice-credit.ru
compliance-alexa.top cl.s7.exct.net
compliance-donald.xyz classiquebijoux.ru
compliance-george.top clickpapa.com
compliance-mark.top clickso.com
contentsexpress.com clicksotrk.com
copypaste.traderzplanet.in club-samodelkin.ru
courtshipgift.com colehaanoutlet.store
cubs-jerseys.us compliance-barak.xyz
customcollegeessays.net compliance-elena.top
datadepths.com concordexoticrentals.com
decenttools.com continent-e.tv
dellalimov.com cornerstone-countertops.com
den-noch24.ru covetnica.com
devochki.top cubook.supernew.org
discountliv.com ddlmega.net
discovertreasure-a.akamaihd.net deadlinkchecker.com
docsportal.net dealitright.click
download-of-the-warez.blogspot.com directrev.com
druzhininevgeniy63.blogspot.com diusyawiga.tk
e-commerce-seo1.com dogbreedspicture.net
easyshoppermac.com domcran.net
ecommerce-seo.com drugstoreforyou.com
elektrischezigarettekaufen2.cowblog.fr drupa.com
elektrozigaretten1.webs.com dscaas.website
embedle.com dyt.net
engines-usa.com easync.io
expdom.com education-cz.ru
ezigarettenshop2.postbit.com elektrischezigarette1.onsugar.com
facebook-mobile.xyz elektronischezigarettekaufen1.myblog.de
fealq.com emergencyneeds.org
fellowshipoftheminds.com excitacion.info
filunika.com.ru expertblog.info
forum-engineering.ru ezigarettekaufen.myblog.de
franecki.net ezigarettekaufen2.blox.pl
free-share-buttons.top familyholiday.ml
freelinkbuilding.website.tk fastcrawl.com
freesitetest.com ff30236ddef1465f88547e760973d70a.com
freeuploader.com fil.ru
fsakhalin.ru floating-share-buttons.com
gallictures.com focalink.com
gay.adultgalls.com forpackningsutveckling.se
geilehausfrauen.net forsex.info
germes-trans.com freegamesplay.online
getmyads24.com fuckingawesome.com
getprismatic.com gateway.zscalerone.net
ghazel.ru gidonline.one
gifspics.com gkvector.ru
greamimgo.pw globalscam.ga
growshop.es gombita.info
guiadeserraazul.com gox.com.ua
headpharmacy.com gwebtools.com.br
heartofpayne.xyz hamptonoaks.ca
helicalpile.us hdimagelib.com
herokuapp.com hdpixion.com
hledejvshopech.cz hem.passagen.se
homeandhealth.ru hentaiheroes.com
homelygarden.com highland-homes.com
howlongdoestizanidinestayinyoursystem.blogspot.com hobild.net
hustoon.over-blog.com homedecorpicture.us
ilovevitaly.com howtoclean.club
iminent.com iframe-toloka.com
impisr.ru ihc2015.info
ilmen.net
imagerydatabase.com
impresagaia.it
indianmedicaltourismshop.com
indo-export.ru
infobanks.ru infobanks.ru
infowarcraft.ru kamagragelusa.net
instakink.com karadene.com
intermesh.net karpun-iris.ru
iomoio.net kartiniresto.com
iopeninghours.co.uk kerwinandcariza.com
iron-age.info kino-ecran.ru
jackwolfskinoutlet.online krasivoe-hd.net
jolic2.com kreativperlen.ch
jpcycles.com ks1234.com
jurajskie.info ku6.com
kadashihotel.com lampokrat.ws
kamorel.com landreferat.ru
kerch.site laptop-4-less.com
khadastoafarde.tk lazymanyoga.com
klosetkitten.com leadn.pl
komputernaya-pomosh-moscow.ru lionking-1994.blogspot.com
konteiner24.com liquimondo.com
labvis.host.sk
lenvred.org
linkbux.ru
littlesexdolls.com littlesexdolls.com
lkbennettstore.online locksmith.jp
lolnex.us m292.info
lombia.co make-money-online.com
louboutinreplica.pw maxthon.com
louisvuittonoutletstore.net mearns-tractors.co.uk
lyrster.com mesbuta.info
magnetic-bracelets.ru mp3ritm.top
mazda-roadsters.com mttwtrack.com
menetie.ru muzaporn.com
mmog-play.ru mydeathspace.com
mojpreskumanie.com
moomi-daeri.com
morf.snn.gr
moscow-region.ru
moz.com
musicpro.monster
muz-baza.net
myperiod.club myperiod.club
myyour.eu nastroyke.net
netcheckcdn.xyz net-profits.xyz
nfljerseys.online new-apps.ru
oklogistic.ru nomuos.it
oranga.host.sk odoratus.net
ordernorxx.com offtopic.biz
osoznanie-narkotikam.net onefilms.net
page2rss.com outpersonals.com
paparazzistudios.com.au oxotl.com
partner-host.men paphoselectricianandplumber.com
partnerline.men par-fallen.ga
physfunc.ru parajumpersjakkesalgnorge.info
picphotos.net pc-services.ru
picturesfrom.com phelissota.xyz
pk-pomosch.ru picanalyzer.data-ox.com
pl.aasoldes.fr picturesmania.com
plastweb.ru pinwallpaper.top
pomoc-drogowa.cba.pl pl-top.pl
porn555.com placid-rounded-coast.glitch.me
porndairy.in play-movie.pl
pornmania.pl pochemychka.net
pornotubexxx.name poisk-zakona.ru
preventheadacheguide.info polcin.de
privetsochi.ru pornogad.com
prlog.ru pornpost.in
producm.ru princeadvantagesales.com
profitwithalex.info princevc.com
prosmibank.ru profit-opportunity.com
pxhdwsm.com pron.pro
radiodigital.co pts163.ru
razamicroelectronics.com purplesphere.in
regdefense.com putitin.me
reimageplus.com qexyfu.bugs3.com
relatodelpresente.com.ar qld10000.net
remont-comp-pomosh.ru quangcaons.com
reversing.cc quickloanbank.com
richinvestmonitor.com quizzitch.net
rockma.se rankingchart.de
russia-tao.ru rentaremotecomputer.com
saletool.ru rome2rio.com
rossanasaavedra.net
rostov.xrus.org
rusenvironmental.net
sad-torg.com.ua
saveindex.xyz
sax-sex.com sax-sex.com
scanner-marwin.top scanner-john.top
scanner-nelson.top scanner-julia.top
search.pch.com
senger.atspace.co.uk senger.atspace.co.uk
seolab.top sexs-foto.com
sexflirtbook.com
sexpornotales.net
sexvideo-sex.com sexvideo-sex.com
share-buttons-for-free.com sexyebonyteen.com
sideeffectsoftizanidine.blogspot.com sinel.info
sientalyric.co smart-balancewheel.com
signal03.ru social-s-hhh.xyz
simplesite.com sofit-dmd.ru
slim.sellany.ru softtor.com
slowmac.tech soviet-portal.do.am
smichovbike.cz speechfoodie.com
smokewithrabbits.com stroicol.net
snomer1.ru stroiminsk.com
social-buttons.com stroy-portal22.ru
sovinsteel.ru sucsesofinspiration.com
space4update.pw suture.co
spymac.net tacbibirfa.tk
stairliftsarea.com tattooreligion.ru
stariy-baku.com teletype.in
sticken.co theautoprofit.ml
stroyhelp-dv.ru thegioixekhach.com
thebitcoincode.com timecrimea.ru
thecounter.com tizanidine4mgstreetprice.blogspot.com
theprofitsmaker.net tizanidine4mgstreetvalue.blogspot.com
tizanidineduringpregnancy.blogspot.com tizanidinecost.blogspot.com
tizanidineusedfor.blogspot.com tonivedu.it
toproadrunner5.info top1-seo-service.com
toys.erolove.in
trafficshaper.com trafficshaper.com
traffictrade.life turbabitload.weebly.com
trkur.com umityangin.net
tuberkuleznik.ru undergroundcityphoto.com
tuckermktg.com urdoot.win
tx41tclega.ru usswrite.com
u-cheats.ru utiblog.fr
ukr-lit.com vapomnoncri.tk
ultimateclassicrock.com vapsy.com
unrealcommander.com viagengrarx.com
veerotech.com
videochaty.ru videochaty.ru
videos-for-your-business.com vigrx-original.ru
vvpg.ru viteonlusarezzo.it
w7s.ru warning.or.kr
watchdogs-2.ru watchdogs-2.ru
webalan.ru websiteexplorer.info
website-datenbank.de websites-reviews.com
wemedinc.com wemedinc.com
wtsindia.in westvilletowingservices.co.za
www.888.com wholinkstome.com
x-lime.com word-vorlagen.xyz
xaylapdiendanang.com workle.website
xmronta.com worldinternetauthority.com
xn----7sbho2agebbhlivy.xn--p1ai wzgyyq.com
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai x-diesel.biz
xnxxandxvideos.com xcombear.ru
you-shall-not-pass.is74.ru xn--d1acah0c.xn--p1ai
youtubologia.it xz618.com
zeroredirect.com youtubedownload.org
zarabotat-na-sajte.ru
zeroredirect6.com zeroredirect6.com
zog.link zkjovpdgxivg.ga
zootoplist.com

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.145 0; 10.1.0.244 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.08.3296 ### Version: V4.2022.08.3297
### Updated: Wed Aug 17 22:01:49 UTC 2022 ### Updated: Thu Aug 18 10:03:35 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.08.3296 ### Version: V4.2022.08.3297
### Updated: Wed Aug 17 22:01:49 UTC 2022 ### Updated: Thu Aug 18 10:03:35 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##