vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 11:40:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.10.2630 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-10-04 10:09:03 +00:00
parent bf38b15cf8
commit 14cd7ffc5f
48 changed files with 86011 additions and 86011 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2629 #### Version: V4.2021.10.2630
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2629 #### Version: V4.2021.10.2630
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2629 #### Version: V4.2021.10.2630
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10614
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2629 #### Version: V4.2021.10.2630
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10618
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10618
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10618
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10618
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 1; 10.1.0.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 0; 10.1.0.19 0;
127.0.0.1 0; 127.0.0.1 0;

10618
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 1; 10.1.0.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 0; 10.1.0.19 0;
127.0.0.1 0; 127.0.0.1 0;

10618
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 1; 10.1.0.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 0; 10.1.0.19 0;
127.0.0.1 0; 127.0.0.1 0;

10618
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2629 2630

10618
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10618
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.183.121.91 0; 20.185.57.29 0;

10618
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 1; 10.1.0.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 0; 10.1.0.19 0;
127.0.0.1 0; 127.0.0.1 0;

10618
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 1; 10.1.0.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 0; 10.1.0.19 0;
127.0.0.1 0; 127.0.0.1 0;

10618
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 1; 10.1.0.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 0; 10.1.0.19 0;
127.0.0.1 0; 127.0.0.1 0;

10618
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.183.121.91 0; 20.185.57.29 0;

10618
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.183.121.91 0; 20.185.57.29 0;

10618
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 1; 10.1.0.19 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
ChinaClaw CATExplorador
DomainStatsBot CopyRightCheck
InfoNaviRobot Exabot
JOC Web Spider LinqiaScrapeBot
Nameprotect MTRobot
PyCurl NetSpider
Swiftbot Petalbot
URLy.Warning ScoutJet
lwp-request Snake
trendiction.de VoidEYE

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404enemy
360Spider
404checker
80legs 80legs
ADmantX AIBOT
ALittle Client Aboundex
AiHitBot Acunetix
AllSubmitter AhrefsBot
Aipbot
Alexibot
AlphaBot AlphaBot
Anarchie
Anarchy Anarchy
Anarchy99
Ankit Ankit
Anthill
Asterias Asterias
AwarioRssBot BBBike
AwarioSmartBot BLEXBot
BDCbot BackStreet
BDFetch Badass
Backlink-Ceck
Barkrowler Barkrowler
Bigfoot Battleztar Bazinga
BetaBot
Black Hole
Blackboard Blackboard
Blow Blow
Bolt
Brandprotect Brandprotect
Bullseye Brandwatch
BunnySlippers Buck
BuiltWith
BuzzSumo BuzzSumo
CATExplorador CheTeam
CODE87 CheeseBot
CazoodleBot ChinaClaw
Cegbfeieh Cloud mapping
Crescent Cocolyzebot
Custo Collector
CopyRightCheck
Copyscape
Crawling at Home Project
CrazyWebCrawler
Curious
DIIbot DIIbot
Devil DataCha0s
DatabaseDriverMysqli
Deusu
Digincore Digincore
Disco Dirbuster
Discobot Discobot
Dispatch DittoSpyder
DomCopBot
DomainAppender DomainAppender
DomainSigmaCrawler DomainCrawler
EMail Siphon Domains Project
EMail Wolf Download Wonder
Dragonfly
Drip
EasyDL
Ecxi Ecxi
EroCrawler EirGrabber
Exabot Evil
Extractor Extractor
ExtractorPro ExtractorPro
Extreme Picture Finder EyeNetIE
Ezooms Ezooms
FHscan
FemtosearchBot
Fimap Fimap
FlashGet Firefox/7.0
FrontPage Foobot
G-i-g-a-b-o-t Fuzz
Fyrebot
GT::WWW
GermCrawler GermCrawler
GetRight
GetWeb
GoZilla
Gotit Gotit
GrabNet
Grabber Grabber
Grafula
GrapeFX GrapeFX
GrapeshotCrawler GrapeshotCrawler
HMView HEADMasterSEO
HTMLparser HaosouSpider
HTTP::Lite
HTTrack
Harvest Harvest
Havij Havij
Hloader
Humanlinks Humanlinks
IDBot HybridBot
IRLbot
IlseBot
Image Fetch Image Fetch
Image Sucker Indy Library
InfoNaviRobot InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
Internet Ninja
InternetSeer InternetSeer
IstellaBot Jbrofuzz
JOC Web Spider JennyBot
Joomla
Jorgee
Kenjin Spider Kenjin Spider
Keyword Density Keyword Density
Kozmosbot Kinza
LWP::Simple Larbin
Lanshanbot LeechFTP
LeechGet LeechGet
LexiBot LinqiaRSSBot
Lftp MIIxpc
Libwhisker
LieBaoFast
Likse
LinkScan
LinkWalker
Linkbot
LinkextractorPro
LinksManager
LinqiaScrapeBot
Lmspider
Ltx71
MFC_Tear_Sample
MSFrontPage MSFrontPage
MSIECrawler MSIECrawler
MTRobot MTRobot
Mag-Net Mag-Net
MarkWatch Majestic-SEO
MeanPath Bot Mata Hari
Meanpathbot Mb2345Browser
MegaIndex.ru Mediatoolkitbot
Metauri Metauri
MicroMessenger
Mister PiX Mister PiX
Moblie Safari Moblie Safari
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mr.4x3 Mozlila
Msrabot Msrabot
Navroad Nessus
Net Vampire
NetAnts NetAnts
NetLyzer NetSpider
NetMechanic NetZIP
Netcraft Netcraft
NextGenSearchBot Nettrack
Nikto Netvibes
Niki-bot
Nimbostratus
Nuclei Nuclei
Nutch Offline Explorer
Octopus
Offline Navigator Offline Navigator
OrangeBot OpenLinkProfiler
OutclicksBot OutclicksBot
OutfoxBot PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer PageScorer
Pandalytics PageThing.com
PeoplePal PeoplePal
Piepmatz Petalbot
Pimonster Pimonster
Pixray
PleaseCrawl PleaseCrawl
Probethenet Pockey
Psbot Psbot
Pu_iN
PxBroker PxBroker
Quick-Crawler QueryN Metasearch
RSSingBot RankingBot2
RankFlex Rankivabot
Recorder Re-re
RedesScrapy Reaper
RepoMonkey RebelMouse
Ripper
RocketCrawler RocketCrawler
SBIder
SEOkicks SEOkicks
SEOkicks-Robot
SEOlyticsCrawler
SMTBot
SalesIntelligent
ScanAlert ScanAlert
SearchmetricsBot ScoutJet
Semrush Scrapy
Seomoz ScreenerBot
Site Sucker ScrepyBot
Searchestate
SeobilityBot
SiteCheckerBotCrawler
SiteExplorer
SiteSnagger SiteSnagger
SiteSucker
Siteimprove Siteimprove
SmartDownload Sitevigil
SlySearch
Snake Snake
Snoopy
SocialRankIOBot
Sociscraper Sociscraper
Sogou web spider Sogou web spider
Sosospider Spammen
Spbot SpankBot
Sqworm Spinn3r
SputnikBot
Sqlmap
Steeler Steeler
Stripper Sucker
SuperBot
SuperHTTP
Surfbot
SurveyBot SurveyBot
Suzuran
Szukacz Szukacz
T8Abot T0PHackTeam
TeleportPro TeleportPro
Telesphoreo Telesphoreo
The Intraformant
TheNomad TheNomad
Thumbor
Toweyabot
Trendiction Trendiction
Turingos True_Robot
TurnitinBot TurnitinBot
TwengaBot TwengaBot
URLy Warning
V-BOT
VB Project VB Project
Vagabondo VCI
VelenPublicWebCrawler VelenPublicWebCrawler
VoidEYE VeriCiteCrawler
Voltron Virusdie
Voil
WBSearchBot WBSearchBot
WEBDAV
WWW::Mechanize
WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD
Web Auto
Web Collage Web Collage
Web Enhancer
Web Fuck Web Fuck
Web Pix Web Pix
Web Sucker WebBandit
WebCollage WebCollage
WebGo IS WebFetch
WebImageCollector WebFuck
WebReaper
WebSauger
WebStripper WebStripper
WebZIP
Website Quester Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster
Whack
Whacker Whacker
Whatweb Who.is Bot
Woobot WiseGuys Robot
Wonderbot
Wprecon Wprecon
Zade Xaldon_WebSpider
Zauba Zauba
Zeus
Zitebot Zitebot
ZmEu
ZoomBot ZoomBot
ZoominfoBot
ZumBot ZumBot
ZyBorg ZyBorg
autoemailspider arquivo-web-crawler
arquivo.pt
cah.io.community cah.io.community
clark-crawler crawler4j
coccocbot-web
cognitiveseo
com.plumanalytics
crawl.sogou.com
crawler.feedback
dataforseo.com
eCatch eCatch
evc-batch facebookscraper
heritrix heritrix
ips-agent isitwp.com
lwp-request lwp-request
lwp-trivial
magpie-crawler
muhstik-scan muhstik-scan
oBot netEstate NE Crawler
pcBrowser pcBrowser
plumanalytics satoristudio.net
s1z.ru scalaj-http
seostar sitechecker.pro
siteripz
sogouspider
spyfu
tAkeOut tAkeOut
trendiction.com trendiction.com
trendiction.de voyagerx.com
webmeup-crawler webmeup-crawler
webprosbot
zauba.io
zgrab

316
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 80legs
404checker ADmantX
AIBOT ALittle Client
ASPSeek Aboundex
Acunetix Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot
Aipbot Aipbot
Alexibot
AllSubmitter AllSubmitter
AlphaBot
Anarchie Anarchie
Anarchy Anthill
Anarchy99 AwarioSmartBot
Ankit
Aspiegel
Asterias
AwarioRssBot
BDCbot
BDFetch BDFetch
BackWeb
Badass
Bandit Bandit
Barkrowler
BatchFTP BatchFTP
Battleztar Bazinga Battleztar Bazinga
Black Hole Blackboard
Brandprotect Blow
BuiltBotTough BlowFish
Bullseye Brandwatch
BunnySlippers BunnySlippers
CCBot
CODE87 CODE87
CazoodleBot Calculon
CherryPicker Cegbfeieh
Chlooe CheTeam
ChinaClaw
Claritybot Claritybot
Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot Cogentbot
Collector
CopyRightCheck CopyRightCheck
Copyscape
Cosmos Cosmos
Crawling at Home Project Craftbot
DBLBot CrazyWebCrawler
DIIbot CrunchBot
DSearch Curious
DataCha0s
DatabaseDriverMysqli
Dirbuster
Disco
Dispatch
DomainAppender DomainAppender
Download Wonder DomainSigmaCrawler
Dragonfly DomainStatsBot
EMail Siphon Domains Project
ECCP/1.0
EMail Wolf
EasyDL EasyDL
Ebingbong Ebingbong
Ecxi Express WebPictures
EroCrawler ExtLinksBot
Evil Extractor
EyeNetIE EyeNetIE
FDM FDM
FHscan FemtosearchBot
Fimap
FlashGet FlashGet
FyberSpider Foobot
GetRight Fuzz
GetWeb Fyrebot
Genieo
GermCrawler
Getintent Getintent
Gigabot GoZilla
Go!Zilla
Go-Ahead-Got-It
GrabNet
Grabber Grabber
Grafula GrapeFX
GridBot GrapeshotCrawler
HTTrack HEADMasterSEO
Heritrix HTTP::Lite
Hloader Haansoft
Iblog Harvest
Havij
IDBTE4M
IlseBot
Image Fetch
Image Sucker Image Sucker
IndeedBot InfoNaviRobot
Indy Library Intelliseek
InterGET
Internet Ninja Internet Ninja
Iskanie InternetSeer
IstellaBot Iria
JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz Jbrofuzz
JetCar JikeSpider
Jetty
Joomla
Jorgee Jorgee
JustView
Kenjin Spider
Keyword Density Keyword Density
Kinza
Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple Larbin
Lanshanbot Leap
LeechFTP LeechGet
LexiBot
LibWeb LibWeb
Libwhisker
LieBaoFast LieBaoFast
Lightspeedsystems LinkWalker
LinkScan
Linkbot
Linkdexbot
LinkpadBot LinkpadBot
LinqiaScrapeBot LinksManager
Lipperhey Spider
Litemage_walker Litemage_walker
MIDown tool MIDown tool
MJ12bot MIIxpc
MSFrontPage MSFrontPage
MSIECrawler MSIECrawler
MTRobot
Mag-Net Mag-Net
Magnet
Majestic SEO Majestic SEO
Majestic-SEO Majestic12
MauiBot MarkWatch
Mb2345Browser
Meanpathbot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri MicroMessenger
Mister PiX Microsoft Data Access
Mojeek Microsoft URL Control
Mojolicious Mr.4x3
Morfeus Fucking Scanner Musobot
Mozlila NPbot
Name Intelligence Name Intelligence
Nameprotect Navroad
NearSite
Nessus Nessus
NetAnts
NetLyzer
NetMechanic
NetSpider
NetZIP NetZIP
Netcraft Nettrack
Netvibes Netvibes
Nibbler Nibbler
Niki-bot
Nikto
Nuclei
Nutch Nutch
Offline Explorer Octopus
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS
Openfind Openfind
OutclicksBot OutfoxBot
PageScorer PECL::HTTP
PageThing.com Page Analyzer
PeoplePal PageGrabber
Pandalytics
Picsearch
Pimonster Pimonster
PleaseCrawl
Probethenet
Psbot
Pu_iN Pu_iN
QueryN Metasearch PyCurl
Quick-Crawler RSSingBot
RankActive
RankActiveLinkBot
RankFlex RankFlex
RankingBot2
Rankivabot Rankivabot
RankurBot RankurBot
Re-re
ReGet ReGet
Reaper
RebelMouse RebelMouse
RepoMonkey Recorder
SEOkicks RedesScrapy
Ripper
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats
SMTBot SMTBot
SalesIntelligent ScanAlert
Scanbot
ScoutJet ScoutJet
Scrapy Scrapy
Screaming Searchestate
ScreenerBot
SearchmetricsBot
SemrushBot
SentiBot
SeoSiteCheckup
SeobilityBot
Seomoz Seomoz
SiteLockSpider Shodan
Siteimprove SiteCheckerBotCrawler
Snake SiteExplorer
Sottopop SlySearch
SmartDownload
Snoopy
SocialRankIOBot
SpaceBison SpaceBison
Spinn3r SpankBot
Spbot
SputnikBot
Sqlmap Sqlmap
Stripper
Sucuri
SuperHTTP SuperHTTP
SurveyBot T0PHackTeam
Suzuran TheNomad
Swiftbot Thumbor
Telesphoreo
Telesphorep
The Intraformant
Titan
Toata Toata
Trendictionbot Tracemyfile
Trendiction
Turingos Turingos
Turnitin Turnitin
TurnitinBot TurnitinBot
TwengaBot URLy Warning
Twice VCI
Typhoeus VelenPublicWebCrawler
URLy.Warning VeriCiteCrawler
VidibleScraper
Virusdie
VoidEYE VoidEYE
Voltron Voltron
WASALive-Bot
WBSearchBot
WISENutbot WISENutbot
WWW-Collector-E Wallpapers
WallpapersHD Web Auto
WeSEE Web Fuck
Web Collage WebAuto
WebBandit WebBandit
WebFetch WebGo IS
WebFuck
WebImageCollector WebImageCollector
WebLeacher WebReaper
WebSauger WebSauger
WebStripper WebStripper
WebZIP WebWhacker
Webshag Webalta
Webster Webster
Whack Widow
Whacker WinHTTrack
Who.is Bot Wonderbot
Woobot Woobot
Wotbox Xenu
Xaldon WebSpider
YoudaoBot YoudaoBot
Zade ZoominfoBot
Zauba
Zermelo
Zeus
Zitebot
ZmEu
archive.org_bot archive.org_bot
arquivo.pt arquivo.pt
cognitiveseo backlink-check
clark-crawler
com.plumanalytics
crawl.sogou.com
crawler.feedback
crawler4j crawler4j
domainsproject.org domainsproject.org
facebookscraper evc-batch
gopher
isitwp.com isitwp.com
lwp-request
magpie-crawler magpie-crawler
netEstate NE Crawler meanpathbot
oBot oBot
page scorer page scorer
plumanalytics pcBrowser
polaris version
probe-image-size probe-image-size
satoristudio.net
scalaj-http scalaj-http
seostar
sitechecker.pro
spyfu
sysscan sysscan
tAkeOut tAkeOut
trendiction.de
ubermetrics-technologies.com
voyagerx.com
webmeup-crawler webmeup-crawler
webpros.com x22Mozilla
webprosbot

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
ahmedabadwebs.com adtech.de
atleticpharm.org adviceforum.info
hahashka.ru arvut.org
investyb.com manyresultshub-a.akamaihd.net
maslenka.kz mex-annushka.ru
mouselink.co paceform.com
my-aladin.com pornotubexxx.name
mymobilemoneypages.com socialbookmarksubmission.org
thewebsitetemplate.info witclub.info
uptimebot.net zirondelli.it

916
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
2drittel.de 0c47f8422d3f.com
2pxg8bcf.top 12u.info
1hwy.com
2728fb936f0.com
3xforum.ro 3xforum.ro
4istoshop.com 4replicawatch.net
4kplayer.pl 6hopping.com
4ur.click 7fon.ru
57883.net 8gold.com
5i2.net a.frcls.fr
7search.com aa8b68101d388c446389283820863176e7.com
888.com actionnooz.com
999webdesign.com ad-words.ru
aa14ab57a3339c4064bd9ae6fad7495b5f.com ads-cool.pro
aad1f4acb0a373420d9b0c4202d38d94fa.com adventureparkcostarica.com
aadroid.net advocatemsk.ru
aaronabel.com agapovdg.ru
aasoldes.fr ageofclones.com
activecampaign.dreamhosters.com aimayangzhi.com
adanih.com akita.kz
adelly.bg aksonural.ru
adinterax.com akvamaster.dp.ua
adult3dgames.com albuteroli.com
adviceforum.info algerianembassy.co.in
agahinameh.com all-number.com
akuhni.by all-streaming-media.com
alienwheels.de allfinweb.com
allforminecraft.ru allforminecraft.ru
allmarketsnewdayli.gdn alloysteel.ru
allornamenti.com allproblog.com
allpdfmags.net allwidewallpapers.com
alt-servis.ru aloofly.com
alveris.ru alphahoverboards.com
animali.deagostinipassion.it alpharma.net
apartamentwroclaw.eu amazingninja.com
api.stathat.com ampower.me
apiadanaknet-a.akamaihd.net animal36.com
appfastplay.com animenime.ru
aproposde.com apiportalmorecom-a.akamaihd.net
artdeko.info appartement-stumm.at
atofilms.com apple.com-cleaner.systems
autogrep.ru apple.com-webbrowsing-security.science
appmsr.org
apps-analytics.net
architecturebest.com
arkartex.ru
arquapetrarca.info
asrv-a.akamaihd.net
australia-opening-times.com
autodo.info
avkzarabotok.info avkzarabotok.info
avtocredit-legko.ru avtovolop.ru
azbukadiets.ru azartmix.com
babyguns.ru azartmsl.com
backuperwebcam.weebly.com azbaseimages.net
baersaratov.ru azinoofficial777.ru
beauty-things.com babieca.com
beetpics.pw bablonow.ru
belinka.com.ua babosas.com
bestadbid.com balla-boo.se
betterhdporn.com bambi.ck.ua
bang-hotties.com
bankcrediti.ru
bankmib.ru
bbsoldes.fr
beauby.ru
bestmarriages.com
bestsexyblog.com
betshuckclean.com
binomo.kz binomo.kz
bio.trade-jp.net biocn.dx.am
bioinnovate.co biznesschat.net
bitnote.co bkns.vn
blobar.org
blockety.co
blockworld.ru blockworld.ru
blog.f00kclan.de blog.remote-computer.de
blog.yam.com
blogig.org
board.f00d.de board.f00d.de
bolezniorganov.ru briomotor.co
bonkers.name bumskontakte.org
bonzibuddi.com burn-fat.ga
bpro1.top buyhoverboard.com
brainboostingsupplements.org candiceloves.us
brains2.biz candy-glam-hp.com
brainsandeggs.blogspot.com casinopinup-wins.com
bravegirlsclub.com celejihad.info
break-the-chains.com
brianhenry.co
browserprotecter.com
btnativenav.com
buigas.00it.com
buy-cheap-pills-order-online.com
buyfriend.ru
buytizanidineonlinenoprescription.blogspot.com
buyviagraa.blogspot.com
bvlgariring.xyz
cacheimages.com
cakesplus.com.au
callawaygolfoutlet.online
callejondelpozo.es
cam-kontakte.org
candypeople.se
cartierreplica.top
cayado.snn.gr
cdn.walkme.com
censys.io censys.io
charmstroy.info chatroulette.si
chatseo.com chatroulette.world
cheapjerseysbizwholesale.us chimiver.info
choosecuisine.com chorus.terakeet.com
cineacademy.ru classicakuhni.ru
cinemaflix.website click2pawn.com
cjs.com.ru clickbanksites.info
cl.s7.exct.net
clickcash.com clickcash.com
clksite.com clickintext.net
companies-catalog.com coderstate.com
compliance-alex.top com-onlinesupport.site
compliance-checker.info compliance-george.top
compliance-jess.top compliance-irvin.top
compliance-margo.top compliance-walt.top
compliance-mark.top coolbar.pro
coolwallpapers4k.info coprofam.org
coolyarddecorations.com corta.co
cosmeticswomens-womensports.rhcloud.com costablue.xyz
cottageofgrace.com crackguru.tk
crynet.cc crazyboost.pro
customsua.com.ua custom-product-labels.com
daisye.top cxpromote.com
daptravel.com d2jsp.org
daretodonate.co dataloading.net
dengi-pod-zalog-nedvizhimosti.ru decorazilla.com
detalizaciya-tut.biz dev33.dioniqlabb.se
detki-opt.ru dieswaene.com
detoxmed24.ru docsportal.net
dev78.dioniqlabb.se domini.cat
digilander.libero.it donvito.unas.cz
divisioncore.com doublepimp.com
dnepr-avtospar.com.ua drev.biz
domznaniy.ru druzhbany.ru
downloadkakaotalk.com dumuelave.xyz
drupa.com e705.net
duawitchrarato.tk eandsgallery.com
dvervmoskvu.ru easyukraine.com
e2click.com editmedios.com
e90post.com educatemetv.com
easyshoppermac.com efnor-ac.com
ec-file.info eleimgo.pw
ednorxmed.com elektrischezigarettekaufen2.cowblog.fr
edudocs.net elektronischezigarettekaufen1.beeplog.com
electro-prom.com elexies.info
elektir.ru elitesportsadvisor.com
emediate.eu energy-ua.com
energydiet-info.ru enpolis.ru
engines-usa.com
eropho.com
erotik-kostenlos.net
espaceinventoristes.com
eupornstar.info
europeanwatches.ru europeanwatches.ru
evehemming.blogspot.com.au
everflora.ru everflora.ru
exchanges-bet.com excitacao.com
facebook-mobile.xyz expresstoplivo.ru
famix.xyz ez8motelseaworldsandiego.com
fandlr.com ezigarettekaufen2.mpbloggar.se
fashionavenuegame.com ezigarettenkaufen2.dreamwidth.org
femmesdenudees.com f-online.de
falopicm.pw
fertilitetsradgivningen.se
fickkontakte.org fickkontakte.org
firesub.pl fikasound.tk
floridamobilebillboards.com film-one.ru
finejewelryshop.ru
finuse.com
fitness-video.net
flprog.com flprog.com
forminecrafters.ru folowsite.com
foto-sisek.porngalleries.top fordsonmajbor.cf
forsex.info
fotosfotos.eu fotosfotos.eu
frbizlist.com fotoxxxru.com
free-share-buttons.top foxydeal.com
freelotto.com free-deals.faith
freewareseek.com free-laptop-reward.com
front.to freemags.cc
frvo.alptandem.ru freeuploader.com
fuckingawesome.com freshwallpapers.info
funnypica.com fullgirl.ru
galaxyflowers.ru funcrushgames.com
gallerily.com gallerily.com
gazoblok.net.ua gdebestkupit.ru
genericlowlatencyasiodriverhq.aircus.com geilehausfrauen.net
germes-trans.com generousdeal-a.akamaihd.net
gilsonchiro.xyz ghernnqr.skyrock.com
girlsfuckdick.com gheus.altervista.org
glasof.es gifspics.com
glavtral.ru gilbertbanda.net
glopages.ru girlsatgames.ru
gobongo.info gktt.ru
gojiberriess.apishops.ru gogalleryawesome.com
googst2.ru golmau.host.sk
gsbs.com.ua gomusix.com
gsmlab.pl gowreckdiving.com
health-medical-portal.info gribokstop.com
highland-homes.com grupografico-pilar.com.ar
gsasearchengineranker.site
guod.me
hacktougroup.ru
hahashka.ru
hdfreeporno.net
hentai-manga.porn
herehloadibs.cf
histock.info
hit-kino.ru hit-kino.ru
hit-men.men hobild.net
home-task.com
homedecorpicture.us homedecorpicture.us
homegardenlova.com homegardenlova.com
homelygarden.com
honyaku.yahoofs.jp honyaku.yahoofs.jp
hostsshop.ru hostsshop.ru
hotshoppymac.com husky-shop.cz
hotsocialz.com hustoon.over-blog.com
housekuba.org hydropump.su
houseofgaga.ru ibmdatamanagement.co
hoztorg-opt.ru ico.re
iflycapetown.co.za ictizanidinehcl4mg.blogspot.com
igithab.com idealtits.net
ilovevitaly.com iframe-toloka.com
imabase.com iideaidekonkatu.info
inbabes.sexushost.com ilmexico.com
incest-ru.com iloveitaly.ru
infogame.name imagez.co
imguramx.pw
imicrovision.com
inet-traffic.com
informatiecentro.be informatiecentro.be
infostatsvc.com infostatsvc.com
infoupdate.org inome.com.ua
inmate-locator.us intellekt21.ru
insider.pro investmac.com
int.search.mywebsearch.com ipchicken.com
interesnie-faktu.ru iphantom.com
internet-apteka.ru iron-age.info
itag.pw isistaylorporn.info
itrevolution.cf isitpaleo.info
itsdp3.com itsdp3.com
iwantmyfreecash.com jaxcube.info
jagg.info jerseywholesalechinabiz.com
jeremyeaton.co jobgirl24.ru
jestr.org juliadiets.com
jolic2.com kambasoft.com
jongose.ninja kazka.ru
joyceblog.top kenaba.su
juliaworld.net keywordbasket.com
jurajskie.info
karaganda.xkaz.org
karusel-market.ru
kerch.site
ketoanhanoi.info
keywords-monitoring-success.com keywords-monitoring-success.com
kfon.eu kinocccp.net
kimcurlrvsms.com kinoplen.ru
kino-key.info kinotorka.ru
kino2018.cc kinozapas.com
kino2018.club kiwi237au.tk
kinobaks.com kladrus.ru
kndxbkdx.bloger.index.hr klitimg.pw
kollekcioner.ru koleso24.com.ua
konkursowo-24.pl komputernaya-pomosh-moscow.ru
krafte.ru kosmetyki.tm.pl
krasnodar.xrus.org ks1234.com
ktotut.net kunstaktien.info
kvartira-sutochno.com lechenie-gemorroya.com
kvartiry-remont.ucoz.ru leftofcentrist.com
ladov.ru
lapitec.eu
layola.biz.tc
lerporn.info lerporn.info
letolove.ru lexixxx.com
lexaprogeneric.link limtu.ifmo.ru
liran-locks.com liveu.infoteka.hu
livefixer.com lonely-mature.com
livetsomudvekslingstudent.bloggersdelight.dk longgreen.info
lolnex.us lostfilm-online.ru
lookingglassemb.com louboutinreplica.pw
lostaruban.com luciddiagnostics.in
luxup.ru lumb.co
macresource.co.uk lyrster.com
mail.allnews24.in macnewtech.com
maju.bluesalt.co macotool.com
magazin-pics.ru
makis.nu
manifestation.betteroffers.review
manimpotence.com
manyresultshub-a.akamaihd.net manyresultshub-a.akamaihd.net
marblestyle.ru marketingvici.com
marinetraffic.com medicines-choice.com
massage-info.nl medkritika.ru
mastershef.club
masturbate.co.uk
mature.free-websites.com
mediaoffers.click
medizinreisen.de
meds-online24.com meds-online24.com
medtherapy.ru mega-bony-2017.pl
megahdporno.net mega-polis.biz.ua
mere.host.sk mexicosleevegastrectomy.com
meteocast.net migente.com
minharevisao.com minecraft-rus.org
mir-betting.ru mmog-play.ru
missclub.info mncrftpcs.com
mlvc4zzw.space mobplayer.ru
mmstat.com mockupui.com
moetomnenie.com moetomnenie.com
monclerjacketsoutlet.win morepoweronmac.trade
moncleronline.xyz most.gov.iq
moncleroutletonline.pw mp3downloadhq.com
monclervests.xyz mp3ritm.top
moneyviking-a.akamaihd.net mug-na-chas-moscow.ru
more-letom.ru music.utrolive.ru
morefastermac.trade myblogregistercm.tk
moroccosurfadventures.com mystats.xyz
moscow-clining.ru
moscow.xrus.org
musicdaddy.net
my-cash-bot.co
myanyone.net
mycaf.it
myliveblog.ru
mypornfree.ru
na-telefon.biz
nakozhe.com
nakrutka.cc nakrutka.cc
narkologiya-belgorod.ru narkologiya-orel.ru
narkologiya-penza.ru narkologiya-peterburg.ru
nationalbreakdown.com narosty.com
nbsproject.ru naturalbreakthroughsresearch.com
negral.pluto.ro naval.jislaaik.com
nextbackgroundcheck.gq needtosellmyhousefast.com
nfljerseyscheapbiz.us neodownload.webcam
nhl09.ru netcheckcdn.xyz
netoil.no
newfilmsonline.ru
newtechspb.ru
nikhilbahl.com nikhilbahl.com
nootrino.com nikkiewart.ru
novgorod.xrus.org nodding-passion.tk
nudepatch.net nottyu.xyz
nufaq.com novodigs.com
obiavo.kz nzfilecloud.weebly.com
obiavo.ru obiavo.uz
obiavo.su office2web.com
one-gear.com
onescreen.cc onescreen.cc
onlinefilmz.net onlainbesplatno.ru
openmultipleurl.com ooo-gotovie.ru
opinionreelle.com opinionreelle.com
otbelivanie-zubov.com oranga.host.sk
oynat.info ordernorxx.com
paydayloanslocal.com pandroid.co
paydayonlinecom.com paramountmarble.co.uk
perm.xrus.org partner-stop.men
pestomou.info pc-test.net
photochki.com pendelprognos.se
picphotos.net perso.wanadoo.es
pierrehardysale.online pharmacyincity.com
phpdevops.com
picturesify.com
pihl.se
pijoto.net pijoto.net
pinapchik.com piluli.info
pizdeishn.com plastgranar.nu
plastgran.com playlott.com
podshipniki-nsk.ru pochemychka.net
pogodnyyeavarii.gq polimga.pw
pogosh.com polska-poezja.com
pokerniydom.ru polytopesexempt.com
popup-fdm.xyz porno-raskazy.ru
pornhub-ru.com
pornoforadult.com pornoforadult.com
pornokorol.com potoideas.us
pornosok.ru
pornotubexxx.name
portadd.men
pr-ten.de
praisong.net
preparevideosafesystem4unow.space preparevideosafesystem4unow.space
pricheski-video.com priora-2.com
priscilarodrigues.com.br progress-upakovka.ru
privacylocationforloc.com prostitutki-novgoroda.org
prospekt-st.ru purchasepillsnorx.com
prostitutki-rostova.org puteshestvennik.com
pshare.biz qualitymarketzone.com
pulse33.ru qwarckoine.com
pwwysydh.com rankia.com
pyrodesigns.com.au rarbg.to
qor360.com rasteniya-vs-zombi.ru
rada.ru rcb101.ru
redirectingat.com rebrand.ly
refererx.com referencemoi.com
relax.ru registratciya-v-moskve.ru
remmling.de reklama1.ru
replica-watch.ru remedyotc.com
rfd-split.hr remont-komputerov-notebook.ru
rn-to-bsn.com rentehno.ru
roofers.org.uk repeatlogo.co.uk
ringporno.com
robertefuller.com
rockprogblog.com
royal-betting.net royal-betting.net
royalcar-ufa.ru runstocks.com
ruex.org.ua rus-pornuha.com
ruscoininvest.company ruscopybook.com
rybalka-opt.ru rutor.vip
sammyweaver.com sa-live.com
sbt-aqua.ru sajatvelemeny.com
scanner-alex.top samara.rosfirm.ru
scanner-mary.top samolet.fr
searchaddis.com saratov.xrus.org
seo-prof1.xyz saveriopiazza.it
seogadget.ru sayyoethe.blogspot.co.za
scanner-julianna.top
scansafe.net
scanspyware.net
scenarii-1-sentyabrya.uroki.org.ua
scenicmissouri.us
scottbywater.com
sdsjweb.com
searchimage.co
searchkut.com
seeingmeerkat.com
seinterface.com
semaltmedia.com
semprofile.com
seo-2-0.com
seofirmreviewsus.info
seorankinglinks.xyz seorankinglinks.xyz
serdcenebolit.com serviporno.com
serialsway.ucoz.ru sethrollins.net
servisural.ru sexfreepornoxxx.com
sex-watch.com sexkvartal.com
sexblog.pw sextracker.de
sexreliz.com shareyards.com
sexy-screen-savers.com shemalegalls.blogporn.in
sexy.babes.frontend-stack.top shiksabd.com
sharpchallenge.com shoesonlinebuy.xyz
sheerseo.com sibecoprom.ru
shemale-sex.net sideeffectsoftizanidine.blogspot.com
simplesite.com sigmund-freud.co.uk
site-speed-check.site signal03.ru
slonechka.ru site.ru
smokewithrabbits.com sitevaluation.org
snjack.info skylta.com
soaksoak.ru skyway24.ru
socialmediasuggest.com slujbauborki.ru
sonnikforme.ru smailik.org
souvenirua.com social-s-hhh.xyz
space4update.pw soft-terminal.ru
spasswelt.xyz softlinesolutions.me
spidtest.org sogimlecal.tk
spravka130.ru soheavyblog.com
spyfu.com solmarket.by
steamoff.net spb-plitka.ru
stroilka.info special-porn.com
sugarlyflex.pw sportbetfair.com
supervesti.ru sribno.net
surfbuyermac.com stackthatbucks.com
suture.co stats-collector.org
tastyfoodideas.com stevemonsen.com
stroimajor.ru
sukarame.net
sundrugstore.com
svtrd.com
symbaloo.com
szamponrevita.pl
tagil.zrus.org
talant-factory.ru
tam-gde-more.ru
tanieaukcje.com.pl
tedxrj.com tedxrj.com
teletype.in tehngr.ru
template-kid.com thaisamkok.com
templates.franklinfire.co the-usa-games.blogspot.com
theguardlan.com thebitcoincode.com
thenetinfo.com thejournal.ru
thesmartsearch.net theporndude.com
thetoiletpaper.com tizanidinedrugtest.blogspot.com
thin.me.pn tizanidinehcl4mgisitanarcotic.blogspot.com
ticketsys.inetwd.com tizanidinepills.blogspot.com
tizanidinehcl4mgtab.blogspot.com top-deal.com.pl
tizanidinewithdrawal.blogspot.com torrents.life
topclickguru.com track-rankings.online
topsiteminecraft.com track2.shop
torontoplumbinggroup.com trackmedia101.com
track.deriv.com trafficfactory.biz
trafficshaper.com trafficjunky.net
traffixer.com truck-rece.life
traffpartners.com tvorozhnaja-zapekanka-recept.ru
trailer.cinemaflix.website twincitiescarservice.com
trainoffend.ml tyumen.xrus.org
tsan.net uchebavchehii.ru
tsyndicate.com udsgame.online
twittruth.com ukrtvory.in.ua
twojebook.pl uptimebot.net
tx41tclega.ru urlopener.blogspot.com.au
uginekologa.com uroffer.link
um-razum.ru vabasa.inwtrade.com
umekana.ru valkiria-tk.ru
umityangin.net vduplo.ru
uogonline.com
uptime-gamma.net
urengoy.pro
urlopener.com
uzungil.com
vacances-voyages.info
vashsvet.com
venta-prom.ru
via-energy-cumpara.com
virtuagirl.com virtuagirl.com
visitcambridge.org vitanail.ru
vltai.com vitoriacabos.com
vmnmvzsmn.over-blog.com vk-mus.ru
vodkoved.ru volgograd.xrus.org
vsexkontakte.net vzube.com
wallpaperaccess.com warning.or.kr
waysbetter.cn waterpurifier.club
webalan.ru weightatraining.com
webpromotion.ae wildnatureimages.com
website-speed-check.site wnoz.de
websites-reviews.com womens-journal.net
weburok.com word-vorlagen.net
wesharepics.info wordpresscore.com
westermarkanjou.se worldis.me
westvilletowingservices.co.za wtsindia.in
whiteelephantwellington.com www.timer4web.com
withstandingheartwarming.com
writingservices17.blogspot.ru
wrz0iuebwhp5fg.freeddns.com
www.labves.ru
www.solartek.ru
www.wohnkabinen-shop.de
xaijo.com
xdoza.com
xerox-douglas.cf
xev.ru xev.ru
xmnb.net xkaz.org
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf xn----dtbndd4ae7eub.top
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai xn--d1abj0abs9d.in.ua
xn----9sbubg3ambdfl1j.xn--p1ai xn--h1ahbi.com.ua
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn--80ab4aa2g.xn--p1ai
xn--80ak6aa92e.com
xn--90acenikpebbdd4f6d.xn--p1ai
xn--hxazdsfy.blogspot.com xn--hxazdsfy.blogspot.com
xrus.org xn--oogle-wmc.com
xsion.net xxxdatinglocal.us
xtube.com yoopsie.com
xxlargepop.com youdesigner.kz
yebocasino.co.za
yebocasino.com
yottos.com
youandcredit.ru
your-bearings.com your-bearings.com
yourads.website yourothersite.com
yourmovies.pl yourporn.com
youtuhe.com
yugo-star.ru
yuweng.info
z-master.ru
zacreditom.ru zacreditom.ru
zaim-pod-zalog-krasnodar.ru zajm-pod-zalog-nedvizhimosti.ru
zakazvzloma.com zapiszto.pl
zeroredirect7.com zeroredirect1.com
zona-aqua.ru zeroredirect8.com
zoogdiesney.com zigarettenonl.canalblog.com
zoomovies.org zigzog.ru
zojirushi-products.ru
zoolubimets.ru
zs2vm.top
zscalertwo.net
zx6.ru

490
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
8xv8.com 000free.us
a14download.com 0912701309f8ce.com
abusora.com 10bet.com
ad-words.ru 123movies.love
affiliate-fr.com 273-fz.ru
airlandsea.info 4kplayer.pl
albuteroli.com 91abcw.com
alibestsale.com aa8b68101d388c446389283820863176e7.com
allforminecraft.ru acunetix-referrer.com
alta-realestate.com adimmix.com
alyeskaresort.com aibolita.com
amanda-porn.ga alessandraleone.com
apiallgeniusinfo-a.akamaihd.net alienwheel.es
appearance-cool.com all4invest.ru
aramaicmedia.org alldubai.biz
arenda-avtoprokat-krasnodar.ru anniemation.com
atovh.local-finders.com anonymousfox.co
autochoixspinelli.com aosexkontakte.net
babyfactory.fr applyneedy.xyz
bear.gotcher.us ardimobileinfo.ml
bestprofits-there.com arendas.net
betshuckclean.com autobudpostach.club
bezlimitko.xyz aviva-limoux.com
biglistofwebsites.com baoxaydung.com.vn
bitcoinremote.com beatifullandscape.co
blog.yam.com belinvestdom.by
bodybuilding-shop.biz belsetirehafi.tk
bolegapakistan.com betterscooter.com
bongiornos.info biketank.ga
btc4u.ru bimatoprost-careprost.com
bumble.cheapwebsitehoster.com bimmerpost.com
buntube.net bio-japan.net
businescoop.men boazpower.com
buyclomidonlaine.com brains2.biz
buynorxx.com brianhenry.co
buzz-porno.info brillianty.info
carabela.com.do buzzurl.jp
carrauterie.be bvlgariwallet.xyz
cdn.walkme.com callstevens.com
celebsopics.com candy-glam-hp.com
cfsrating.sonicwall.com cars-modification.net
chocolateslim-original.com charmstroy.info
chorus.terakeet.com cheap-pills-norx.com
cmd.kz christianlouboutinoutlet.win
compiko.info ciekawinki.pl
coop-gamers.ru cigarpass.com
counciltally.com citysecurity.nu
crazyprotocol.com clickso.com
creditwell.ru comissionka.net
cubs-jerseys.us compliance-elena.xyz
customerpromos-a.akamaihd.net compliance-fred.xyz
data-mining.tk cookingmeat.ru
datract.com cracksplanet.com
dawlenie.com crackzplanet.com
demenageur.com creditnation.ru
diy-handmade-ideas.com daooda.com
dnsrsearch.com darkages.info
domik-derevne.ru data.vtc.pw
dataloading.net
db.speedup.gdn
dipstar.org
dispo.de
dms-sw.ru
domini.cat
download-of-the-warez.blogspot.com download-of-the-warez.blogspot.com
dustyorate.com dptaughtme.com
e705.net dyshagi.ru
earthmagic.info
econom.co econom.co
editors.choice6912650.hulfingtonpost.com elmifarhangi.com
eduinfosite.com engines-usa.com
egvar.net ero-advertising.com
electro-prom.com errorfixing.space
elektrischezigarette2.onsugar.com everflora.ru
erotik0049.com exchanges-bet.com
europages.com.ru expertnaya-ocenka.ru
evogarage.com extads.net
excaliburfilms.com extlinks.com
executehosting.com eyelike.com.ua
ezaz.info eyes-on-you.ga
ezigarettenshop2.postbit.com fanrto.com
failingmarriege.blogspot.com fashionavenuegame.com
fashion.stellaconstance.co faz99.com
fdzone.org feel-planet.com
financetip.eu fettefrauen.net
fitness-video.net fickenprivat.info
flex4launch.ru fil.ru
freewebs.com financehint.eu
front.to findanysex.com
finuse.com
firstsiteguide.com
flashahead.info
fok.nl
forexgb.ru
forum69.info
foto-telok.net
freemaintenancesysforpcandmac.top
fsakhalin.ru fsakhalin.ru
galaxyflowers.ru gentamicineyedrops.blogspot.com
gallerily.com girlsatgames.ru
gamerscorps.com goodly.pro
gavuer.ru gorabagrata.ru
gaygalls.net graphicwe.org
geilehausfrauen.net gratis-sexkontakte.com
gemara.com gratuitbaise.com
gg.zzyjxs.com grtyi.com
girlspicsa.com guge.io
goldpanningtools.com gymi.name
goodwriterssales.com hd-filmy.net
gox.com.ua hdapp1008-a.akamaihd.net
greendream.com.ua hdxnxxtube.mobi
hdfreeporno.net hesteel.pl
healing-dysplasia.ru hidefiles.org
highland-homes.com huimin764128.com
hothor.se ie.57883.net
i-midias.net.br ikearugs.xyz
ibmdatamanagement.co incomekey.net
ideashome.id internetartfair.com
ideibiznesa2015.ru investyb.com
imagecoolpub.com iopeninghours.co.uk
insurple.com it-max.com.ua
ipchicken.com ivanstroi.ru
jewelryandfiligree.com jabimgo.pw
jmat.cn javlibrary.cc
kamalsinha.com javspace.net
kambasoft.com jerseywholesalebizchina.com
kazan.xrus.org jikoman.info
keki.info jimmychoosale.online
keyword-suggestions.com junglenet-a.akamaihd.net
kinosed.net jwss.cc
konica.kz jyvopys.com
krasnodar-avtolombards.ru kamorel.com
landreferat.ru keenoutlet.online
leadwayau.com kino-rating.ru
lechenie-gemorroya.com kosmetyki.tm.pl
legionalpha.com libertybilisim.com
lizace.com lincolntheatre.com
lonfon.xyz linkwithin.com
lulea-auktionsverk.se littleberry.ru
manve.info littlecity.ch
marinetraffic.com liveinternet.ro
medkletki.ru liverpool.gsofootball.com
meendo-free-traffic.ga lockerz.com
member-quiz.com lombardfinder.ru
mirtorrent.net louboutinreplica.pw
mokrayakiska.com lyngdalhudterapi.no
moonci.ru lyrster.com
moscow-clining.ru m4ever.net
moscow.xrus.org mahnwachen-helfen.info
moz.com mesbuta.info
mp3s.club microsoftportal.net
msk.afora.ru mlvc4zzw.space
muabancantho.info mnimmigrantrights.net
musezone.ru mnogabukaff.net
myanyone.net mulberryoutletonlineeu.com
mywallpaper.top musicktab.com
myxdate.info my-big-family.com
narosty.com mypets.by
nfljerseysforsalewholesaler.com nevansk.ru
nufaq.com normalegal.ru
nuup.info noscrapleftbehind.co
okuos.com novodigs.com
openstat.com oddamzadarmo.eu
optibuymac.com okmedia.sk
outshop.ru orgasmus-virtual.com
partner-cdn.men osb.se11.ru
photosaga.info outrageousdeal-a.akamaihd.net
photostudiolightings.com oz-offers.com
pinsdaddy.com partner-host.men
pixelrz.com partnerworkroom.men
pl.id-forex.com pewit.pw
pladform.ru picsforkeywordsuggestion.com
porndairy.in pigrafix.at
porno.simple-image.com.ua pizdopletka.club
pornonik.com platezhka.net
pornoted.com playlott.com
positive2b.ru pony-business.com
printie.com porno-play.net
quickchange.cc pornoslive.net
realting-moscow.ru porodasobak.net
rebrand.ly pro-okis.ru
redbottomheels.xyz pro-tec.kz
redirect.trafficreceiver.club prostitutki-novgoroda.org
redirlock.com psa48.ru
rightenergysolutions.com.au quickcashlimited.com
roll123.com recordpage-a.akamaihd.net
roznica.com.ua reining.lovasszovetseg.hu
russia-today-video.ru reklamuss.ru
s-iwantyou.com relatodelpresente.com.ar
sabaapress.com renhacklids.tk
sadaholding.com rethinkwasteni.info
saveriopiazza.it rezeptiblud.ru
sbt-aqua.ru rfserial.net
scanner-ivan.top ribieiendom.no
screentoolkit.com sanidumps.com
seo-platform.com scanner-elena.top
seokicks.de seeresultshub-a.akamaihd.net
sexpornotales.com sexpartygirls.net
sexsearch.com signoredom.com
sexstream.pl siha.de
sherlock.se simplepooltips.com
shivafurnishings.com site3.free-share-buttons.com
shopperifymac.com skuteczna-dieta.co.pl
shopvilleroyboch.com.ua snip.to
sicfor.bcu.cc snsdeainavi.info
sim-service.net snworks.com
sladkoevideo.com soblaznu.net
slim.sellany.ru social-buttons.xyz
slutloadlive.com softxaker.ru
smsactivator.ru spidtest.space
snip.tw spin2016.cf
snts.shell-pmr.ru srecorder.com
sogimlecal.tk storehouse.ua
songoo.wz.cz streha-metalko.si
spb-plitka.ru sureone.pro
sports-supplements.us surffoundation.nl
stair.registrydomainservices.com susanholtphotography.com
steelmaster.lv tds-advert005.info
stromerrealty.com technika-remont.ru
sudexpert66.ru tecnoteakviareggio.it
sukarame.net teenbbw.yopoint.in
teenfuck.tv teesdaleflyballclub.co.uk
testingads.pro testingads.pro
thebluffs.com thesmartsearch.net
tizanidinesideeffects.blogspot.com thewebsitetemplate.info
tmearegion26.com tizanidinenarcotic.blogspot.com
tmtrck.com tizanidinerecreationaluse.blogspot.com
top10registrycleaners.com tracksz.co
topvidos.ru trafficmonetize.org
toyota.7zap.com trafficstars.com
track112.site trkur.com
traffpartners.com uac.net.au
tvory.predmety.in.ua umekana.ru
unblocksit.es vbabule.net
ventelnos.com ventelnos.com
vertaform.com via-gra.webstarts.com
video-chat.love visionwell.com.cn
videooko.weebly.com vonradio.com
vip2ch.com vvon.co.uk
vkonche.com vvpg.ru
vote-up.ru wardreapptokone.tk
vremya.eu webalan.ru
wallpapers-best.com xev.ru
watracker.net xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
webkeyit.com yginekologa.com
wesharepics.info yun56.co
whiteproduct.com zarenica.net
wholesalenfljerseys.us.com zixizop.net.ru
wineitudes.wordpress.com zootravel.com
wttavern.com zx6.ru
x69ty.ru
xlolitka.com
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn----btbdvdh4aafrfciljm6k.xn--p1ai
xn----itbeirbjbi7bc6bh2d.xn--p1ai
xvideosbay.com
yourmovies.pl
yourothersite.com
yousense.info
zakazvzloma.com
zastenchivosti.net
zeroredirect12.com
zocaparj.kz
zzlgxh.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.243 0; 10.1.0.19 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2629 ### Version: V4.2021.10.2630
### Updated: Sun Oct 3 22:01:31 UTC 2021 ### Updated: Mon Oct 4 10:03:15 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2629 ### Version: V4.2021.10.2630
### Updated: Sun Oct 3 22:01:31 UTC 2021 ### Updated: Mon Oct 4 10:03:15 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##