vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.07.3236 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-07-19 10:09:31 +00:00
parent 085d4cff22
commit 7207bdea8d
48 changed files with 103299 additions and 103306 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.07.3235 #### Version: V4.2022.07.3236
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.07.3235 #### Version: V4.2022.07.3236
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.07.3235 #### Version: V4.2022.07.3236
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12776
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.07.3235 #### Version: V4.2022.07.3236
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12780
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12780
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12780
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12780
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 1; 10.1.1.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 0; 10.1.1.96 0;
127.0.0.1 0; 127.0.0.1 0;

12780
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 1; 10.1.1.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 0; 10.1.1.96 0;
127.0.0.1 0; 127.0.0.1 0;

12780
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 1; 10.1.1.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 0; 10.1.1.96 0;
127.0.0.1 0; 127.0.0.1 0;

12780
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3235 3236

12780
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12780
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.82.102.13 0; 13.82.17.236 0;

12780
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 1; 10.1.1.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 0; 10.1.1.96 0;
127.0.0.1 0; 127.0.0.1 0;

12780
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 1; 10.1.1.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 0; 10.1.1.96 0;
127.0.0.1 0; 127.0.0.1 0;

12780
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 1; 10.1.1.96 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 0; 10.1.1.96 0;
127.0.0.1 0; 127.0.0.1 0;

12780
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.82.102.13 0; 13.82.17.236 0;

12780
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.82.102.13 0; 13.82.17.236 0;

12780
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 1; 10.1.1.96 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Aspiegel Apexoo
JennyBot Bitacle
MeanPath Bot Claritybot
Nmap Image Sucker
Openvas Intelliseek
PeoplePal POE-Component-Client-HTTP
WebBandit Pandalytics
arquivo-web-crawler Pimonster
serpstatbot WEBDAV
sogouspider WallpapersHD

288
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker AIBOT
ADmantX
ALittle Client ALittle Client
Abonti
Aboundex Aboundex
Aboundexbot Aboundexbot
Acunetix Acunetix
AhrefsBot
Aipbot Aipbot
AllSubmitter Alligator
AlphaBot Anarchie
Anarchy Anarchy99
Ankit Ankit
Asterias Aspiegel
Atomseobot AwarioSmartBot
Attach
BBBike BBBike
BDCbot BDFetch
BLEXBot
Backlink-Ceck
Badass Badass
Battleztar Bazinga Bigfoot
Bitacle Bitacle
Blow Blackboard
Bolt Brandprotect
Brandwatch Brandwatch
Buck Buddy
BuiltWith BuzzSumo
CODE87 CODE87
CSHttp
CheTeam
Chlooe
Citoid Citoid
Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Collector
Copyscape Copyscape
Craftbot Cosmos
Crescent
CrunchBot CrunchBot
Curious Curious
Custo Custo
CyotekWebCopy
DTS Agent DTS Agent
DataCha0s DataCha0s
Deusu DatabaseDriverMysqli
Dirbuster Digincore
Dispatch Disco
DomainAppender DnBCrawler-Analytics
DomainSigmaCrawler DnyzBot
Domains Project DomainCrawler
Dotbot Dotbot
Download Wonder Dragonfly
Drip Drip
EMail Siphon EMail Siphon
EMail Wolf
EasyDL
Ebingbong Ebingbong
ExtLinksBot Ecxi
ExtractorPro EirGrabber
EyeNetIE Evil
Exabot
Express WebPictures
Extreme Picture Finder
Ezooms Ezooms
FHscan FHscan
Flunky Firefox/7.0
FrontPage Foobot
FyberSpider G-i-g-a-b-o-t
Getintent Genieo
Gigabot
Go!Zilla Go!Zilla
Go-Ahead-Got-It GrabNet
GrapeFX Grabber
GrapeshotCrawler
GridBot
HMView HMView
HTMLparser HTMLparser
HTTP::Lite
Haansoft Haansoft
HaosouSpider HaosouSpider
Harvest Harvest
Havij Havij
Heritrix Heritrix
Hloader
HonoluluBot HonoluluBot
IDBot Humanlinks
IRLbot Iblog
Id-search Id-search
Image Fetch InfoNaviRobot
Image Sucker
IndeedBot
Indy Library
InfoTekies InfoTekies
InternetSeer Intelliseek
InterGET
Internet Ninja
Iria
Iskanie Iskanie
JOC Web Spider JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz JustView
JennyBot Keyword Density
JetCar Kinza
Jorgee
Kenjin Spider
Keybot Translation-Search-Machine
Kozmosbot Kozmosbot
LWP::Simple
Lanshanbot
Larbin Larbin
LeechFTP Leap
LeechGet
Lftp
LibWeb LibWeb
Lightspeedsystems LieBaoFast
Litemage_walker LinkWalker
Ltx71 Linkbot
MQQBrowser LinksManager
MTRobot LinqiaRSSBot
Mag-Net MIIxpc
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic SEO
MarkMonitor Majestic-SEO
MarkWatch MeanPath Bot
Mediatoolkitbot Mediatoolkitbot
Microsoft URL Control Microsoft Data Access
Minefield Mister PiX
Moblie Safari
Mojeek Mojeek
Mojolicious Mojolicious
MolokaiBot
Mozlila
Msrabot Msrabot
Musobot Name Intelligence
NICErsPRO
NPbot
Navroad Navroad
Needle Needle
Nessus Nessus
Net Vampire NetMechanic
Nettrack Nettrack
NextGenSearchBot Nibbler
Nimbostratus Ninja
Nmap Nuclei
Nutch Offline Navigator
OutfoxBot OnCrawl
OpenLinkProfiler
OpenVAS
OrangeBot
OutclicksBot
PECL::HTTP
PageAnalyzer PageAnalyzer
PeoplePal PageGrabber
Petalbot PageThing.com
Pandalytics
Pavuk
Picscout Picscout
Pimonster PictureFinder
Pixray
Pockey Pockey
ProWebWalker
Probethenet
Psbot Psbot
QueryN Metasearch Pump
PxBroker
RSSingBot RSSingBot
RankingBot RankActiveLinkBot
Rankivabot
ReGet
RealDownload RealDownload
Reaper
Recorder
RedesScrapy RedesScrapy
Ripper
RocketCrawler RocketCrawler
Rogerbot
SBIder SBIder
SEOlyticsCrawler
SMTBot
ScanAlert ScanAlert
Scrapy Scrapy
ScrepyBot Screaming
SearchmetricsBot ScreenerBot
SemanticJuice SemanticJuice
SemrushBot SemrushBot
SeoSiteCheckup SeoSiteCheckup
Shodan SeobilityBot
Siphon
Site Sucker Site Sucker
SiteExplorer SiteExplorer
SiteSnagger SiteLockSpider
Siteimprove Sitebeam
SmartDownload
Snake
Snoopy Snoopy
Sociscraper SocialRankIOBot
Sogou web spider
Sosospider Sosospider
Sottopop SpaceBison
SpankBot SpankBot
Spbot Spbot
Sqlworm SputnikBot
SuperBot Sqlmap
Teleport Sqworm
Telesoft Stripper
TheNomad Sucker
Surfbot
SurveyBot
Swiftbot
Szukacz
TeleportPro
Thumbor Thumbor
TightTwatBot TightTwatBot
Tracemyfile Titan
Trendictionbot Toweyabot
True_Robot
Turnitin Turnitin
TurnitinBot
URLy.Warning URLy.Warning
V-BOT Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler Virusdie
VoidEYE VoidEYE
Voil Voil
WBSearchBot WWW-Mechanize
WISENutbot WWWOFFLE
WPScan Wallpapers
Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE WeSEE
Web Auto
Web Collage Web Collage
Web Fuck Web Enhancer
Web Fetch
Web Pix Web Pix
Web Sauger Web Sucker
WebBandit
WebEnhancer WebEnhancer
WebFuck WebFetch
WebGo IS WebGo IS
WebImageCollector
WebReaper
WebStripper WebStripper
WebSucker Webalta
WebmasterWorldForumBot Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
Webster Whacker
Whack WiseGuys Robot
Whatweb
Widow
Wonderbot
Wotbox
Wprecon Wprecon
Xaldon_WebSpider Xenu
YoudaoBot YoudaoBot
Zade
Zauba Zauba
Zermelo Zermelo
ZmEu ZmEu
ZoomBot ZyBorg
arquivo-web-crawler arquivo-web-crawler
arquivo.pt
autoemailspider
backlink-check backlink-check
check1.exe
clark-crawler clark-crawler
coccocbot cognitiveseo
crawler.feedback crawler.feedback
dataforseo.com dataforseo.com
domainsproject.org domainsproject.org
eCatch eCatch
facebookscraper instabid
heritrix iubenda-radar
ips-agent linkdexbot
lwp-trivial
mediawords
muhstik-scan muhstik-scan
polaris version oBot
s1z.ru page scorer
ripz
satoristudio.net
scalaj-http
scan.lol scan.lol
seostar seoscanners
serpstatbot
sitechecker.pro sitechecker.pro
sp_auditbot siteripz
sysscan tAkeOut
ubermetrics-technologies.com ubermetrics-technologies.com
webgains-bot
webmeup-crawler webmeup-crawler
webprosbot
xpymep1.exe
zauba.io zauba.io

300
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404enemy ASPSeek
ADmantX AllSubmitter
Abonti
Aboundex
AhrefsBot
Alexibot
Alligator
Anarchie Anarchie
Anarchy Attach
Ankit
AwarioSmartBot AwarioSmartBot
BDFetch
BackStreet BackStreet
Badass BackWeb
Black Hole Backlink-Ceck
BlackWidow Bandit
Blow Barkrowler
Boardreader BetaBot
Brandprotect Bitacle
Brandwatch BotALot
Buck
Buddy Buddy
CATExplorador BuiltBotTough
BuiltWith
CCBot
CODE87
CSHttp CSHttp
Calculon
CazoodleBot CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect CensysInspect
CheTeam CheTeam
CheeseBot CheeseBot
CherryPicker
ChinaClaw ChinaClaw
Chlooe Citoid
Claritybot Claritybot
Cocolyzebot
Cogentbot
Collector Collector
CopyRightCheck Copier
Cosmos CrazyWebCrawler
Craftbot Curious
Crescent Custo
CrunchBot CyotekWebCopy
DBLBot DBLBot
DIIbot
DSearch DSearch
Deusu DTS Agent
Digincore DatabaseDriverMysqli
DnBCrawler-Analytics Demon
Devil
Disco
Discobot
Dispatch
DittoSpyder
DnyzBot
DomainStatsBot DomainStatsBot
Dragonfly
ECCP/1.0
EMail Siphon EMail Siphon
EMail Wolf EasyDL
Ecxi EirGrabber
Evil
ExtLinksBot
ExtractorPro ExtractorPro
EyeNetIE EyeNetIE
Ezooms
Fimap Fimap
FlashGet Firefox/7.0
Flunky Flunky
Freeuploader
FyberSpider FyberSpider
Fyrebot Fyrebot
GT::WWW G-i-g-a-b-o-t
Genieo Go!Zilla
GetRight
GetWeb
Getintent
Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GoZilla
GrapeFX Gotit
GridBot GrabNet
Grabber
GrapeshotCrawler
HaosouSpider HaosouSpider
Harvest Harvest
Humanlinks Havij
HybridBot
Iblog Iblog
Image Fetch
InfoNaviRobot
InfoTekies InfoTekies
Intelliseek
InterGET InterGET
Internet Ninja
InternetSeer InternetSeer
Iskanie Iria
JOC Web Spider JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
JennyBot
JetCar JetCar
Jetty
JikeSpider JikeSpider
JustView Joomla
Jorgee
Jyxobot Jyxobot
Keybot Translation-Search-Machine
Keyword Density Keyword Density
Kozmosbot
LWP::Simple LWP::Simple
LeechFTP Lanshanbot
Lftp LexiBot
Linkbot LibWeb
LinkextractorPro LieBaoFast
LinkWalker
LinksManager LinksManager
LinqiaRSSBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey
Lipperhey Spider
Lmspider Lmspider
Ltx71 Ltx71
MFC_Tear_Sample MIDown tool
MIIxpc MQQBrowser
MTRobot Mail.RU_Bot
MarkWatch Majestic-SEO
Mata Hari MarkMonitor
Mb2345Browser
MeanPath Bot
Meanpathbot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Minefield
Mister PiX
Mojeek
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
NPbot Nameprotect
NearSite NearSite
Net Vampire NetAnts
NetLyzer NetMechanic
NetSpider NetSpider
NetZIP Netcraft
Nettrack Nettrack
Netvibes NextGenSearchBot
Nibbler Niki-bot
NimbleCrawler Nikto
Nutch Nimbostratus
OpenLinkProfiler Nmap
Openfind Not
Nuclei
Offline Explorer
OpenVAS
OrangeBot OrangeBot
OrangeSpider OrangeSpider
OutfoxBot
Page Analyzer Page Analyzer
PageScorer PageGrabber
Pandalytics
Panscient
Pavuk
PeoplePal
Petalbot Petalbot
Pockey Picsearch
ProWebWalker PictureFinder
Piepmatz
Pu_iN Pu_iN
Pump
PxBroker
Quick-Crawler Quick-Crawler
RankActive RSSingBot
RankActiveLinkBot RankingBot2
RankFlex Rankivabot
RankurBot
Re-re
ReGet ReGet
RebelMouse RocketCrawler
Recorder Rogerbot
Ripper
SBIder SBIder
SEOprofiler SEOprofiler
SISTRIX
ScanAlert ScanAlert
Scanbot
Scrapy
Screaming Screaming
Searchestate ScreenerBot
SemanticJuice
Semrush Semrush
SentiBot SentiBot
SeoSiteCheckup
SeobilityBot
Seomoz
Siphon
Site Sucker Site Sucker
Sitevigil Snake
SmartDownload Snapbot
Snoopy
SocialRankIOBot
Sociscraper
Sosospider
Sottopop
SpankBot SpankBot
SuperHTTP Spbot
Surfbot Spinn3r
Sqlmap
Stripper
Sucuri
SurveyBot
Swiftbot Swiftbot
Szukacz
T0PHackTeam T0PHackTeam
T8Abot
Teleport
TeleportPro TeleportPro
Telesphoreo Telesphoreo
TheNomad TheNomad
Thumbor
Toata
Toweyabot
Tracemyfile Tracemyfile
Trendiction
Turnitin Turnitin
TurnitinBot TurnitinBot
TwengaBot Typhoeus
Twice
URLy.Warning URLy.Warning
UnisterBot UnisterBot
VB Project
Vacuum Vacuum
VidibleScraper Vagabondo
Virusdie Virusdie
Voil VoidEYE
Voltron
WASALive-Bot
WBSearchBot
WPScan
WWW-Collector-E WWW-Collector-E
WeSEE WallpapersHD
Web Enhancer Web Enhancer
Web Fuck Web Fuck
Web Sauger Web Sauger
WebAuto WebBandit
WebCopier WebEnhancer
WebFuck WebGo IS
WebLeacher WebImageCollector
WebPix WebReaper
WebZIP WebSauger
WebSucker
WebWhacker
Webshag
WebsiteExtractor WebsiteExtractor
WebsiteQuester
Webster Webster
Whacker Whack
Who.is Bot Whatweb
Widow
WinHTTrack WinHTTrack
Woobot Wonderbot
Wotbox
Wprecon Wprecon
Xaldon_WebSpider Xaldon_WebSpider
Xenu YoudaoBot
Zauba Zade
Zeus Zeus
ZoomBot ZmEu
ZoominfoBot
ZumBot ZumBot
ZyBorg adscanner
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
backlink-check arquivo.pt
cah.io.community
check1.exe check1.exe
clark-crawler
coccocbot
cognitiveseo cognitiveseo
crawl.sogou.com
crawler.feedback
crawler4j crawler4j
dataforseo.com
demandbase-bot
eCatch eCatch
evc-batch evc-batch
facebookscraper facebookscraper
heritrix heritrix
instabid ips-agent
iubenda-radar iubenda-radar
linkdexbot meanpathbot
magpie-crawler mediawords
muhstik-scan muhstik-scan
oBot
page scorer page scorer
pcBrowser
ripz ripz
scan.lol scalaj-http
sitechecker.pro
sp_auditbot
spyfu spyfu
trendiction.com trendiction.de
ubermetrics-technologies.com webgains-bot
voyagerx.com x09Mozilla
webmeup-crawler x22Mozilla
webpros.com
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
cablecar.us alveris.ru
canoncdriverq3.pen.io gosreg.amchs.ru
cfjump.com moneytop.ru
m292.info orel-reshka.net
plastgranar.nu popads.net
pskcijdc.bloger.index.hr pulse33.ru
s-forum.biz rfserial.net
shodanhq.com sack.net
timer4web.com vabasa.inwtrade.com
wildcattube.com wfb.hatedriveapart.com

940
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
01apple.com 00go.com
101billion.com 0n-line.tv
202ch.com 1000n1.ru
2daytrendingnews.com 1001desert.com
3weekdiet.com 6hopping.com
76brighton.co.uk 8si.ru
7fon.ru aa14ab57a3339c4064bd9ae6fad7495b5f.com
7wind.ru above.com
8558l.jobs.net absolutelyanalog.com
991mostfm.co.id abusora.com
999webdesign.com acunetix-referrer.com
aac500b7a15b2646968f6bd8c6305869d7.com aerogo.com
abbanreddy.com ageofclones.com
acmebtn.ml aksonural.ru
adclickthru.net alert-fdm.xyz
adman.se allbizne.co.ua
adminshop.com alldubai.biz
adobereader-free.ru allwomen.info
adrenalin-stk.ru alvtank.se
adservme.com amateurgalls.com
adsloads.com amateurlivechat.org
agadelha.com.br amazon-adsystem.com
agardomains.com android-vsem.org
agecheckadult.com angry-fermi-7633.arukascloud.io
alfabot.xyz anjalika.co.in
alibestsale.com anonymizeme.pro
alienwheels.de apispringsmartne-a.akamaihd.net
alldezservice.kz arendas.net
allknow.info artel116.ru
allsilver925.co.il asscenihotosidea.blogspot.com
ameblo.top atelielembrancaqueencanta.com.br
anatomiy.com ausmepa.org.au
angigreene.com auspolice.com
animal-drawings.com autochoixspinelli.com
animali.deagostinipassion.it autoloans.com
ap.senai.br avtolombard-krasnodar.ru
api.stathat.com azadnegar.com
apiwoodensealcom-a.akamaihd.net azartmix.com
artdestshop.eu azarttoto.com
artpicso.com babyfactory.fr
asacopaco.tk balkanfarma.ru
auto-zapchasti.org bankmib.ru
autoseo-traffic.com
aversis.net
avirasecureserver.com
azartclub.org
bablonow.ru
bala.getenjoyment.net
bambi.ck.ua
bang-hotties.com
basisches-wasser.net basisches-wasser.net
bausparen.bz.it bear.gotcher.us
bavsac.com belinvestdom.by
bcmp.org best-seo-solution.com
beachpics.com beta.hotkeys.com
beauty-clinic.ru bigames.online
berlininsl.com
bestmouthwash.club
besttorrentknifta.weebly.com
bioca.org bioca.org
bitcoin-ua.top bkns.vn
bjetjt.com blacktwhite.com
bjgugu.net.ua bloke.com
bloggen.be bluesman.nu
bmw.afora.ru boltalko.xyz
bongacams.com brandimensions.com
bookhome.info bravegirlsclub.com
bookmark4you.com brendbutik.ru
brainboostingsupplements.org briomotor.co
braindaily.xyz brokergid.ru
brainxs.us bronzeaid-a.akamaihd.net
brakehawk.com btvn.ru
brianhenry.co buehne-fuer-menschenrechte.de
brillianty.info bytimedance.ru
brothers-smaller.ru candy-glam-hp.com
buttons-for-your-website.com carrauterie.be
buybest1.biz
buyessay3.blogspot.ru
buytizanidineonline.blogspot.com
buzzurl.jp
calc-for-credit.ru
callcafe.info
camdenmemorials.com
canoncdriverq3.pen.io
carmuffler.net
cars-modification.net
carsplate.com
cartierlovestore.com cartierlovestore.com
cartierreplica.win cartierlovestore.xyz
casite-513049.cloudaccess.net cbox.ws
cbcseward.com censys.io
cfsrating.sonicwall.com centraletermice.us
chcu.net cgi2.nintendo.co.jp
cheapcigarettesc.info cheapjerseysfootballshop.com
cheapwebsitehoster.com
cherrypointplace.ca
chocolateslim-u-srbiji.com
chorus.terakeet.com chorus.terakeet.com
cialis-samples.com cialisovercounteratwalmartusa.com
cialisndbrx.com clickhype.com
ciceros.co clickprotects.com
clickaider.com clmforexeu.com
clickcash.com clubfashionista.com
clknsee.com cmd.kz
cntravelre.com
coccoc.com coccoc.com
coin-hive.com codq.info
compliance-alex.xyz collegerentals.com
compliance-alexa.xyz cometorussia.net
compliance-irvin.xyz compliance-brian.top
compliance-josh.top compliance-donald.xyz
compliance-mary.top compliance-jess.top
compliance-nelson.top compliance-margo.top
compucelunlock.net compliance-mark.top
cookingmeat.ru conocer-sanabria.com
coolingoods.com coolgramgoods.com
counter.yadro.ru cornomase.win
creatives.adbetclickin.pink covetnica.com
cpabegins.ru
credit.co.ua credit.co.ua
creditnation.ru cubook.supernew.org
dailyfinancefix.com customsua.com.ua
daydream-studio.ru db.speedup.gdn
daymusam.com dbmkfhqk.bloger.index.hr
deluxedumps.com ddospanel.com
dengi-pod-zalog-nedvizhimosti.ru degerlund.net
detki-opt.ru delfin-aqua.com.ua
dfiles.me delta-line.men
digitalassetmanagement.site den-noch24.ru
discountbarbour.online deutschehobbyhuren.net
discountliv.com dichvuvesinhhanoi.com
djekxa.ru digitalfaq.com
dkvorota.ru dignitasdata.se
dmmspy.com dikx.gdn
doska-vsem.ru discovertreasurenow.com
dosugrostov.site divatraffic.com
dptaughtme.com divisioncore.com
drillsaw.ru djstools.com
druzhbany.ru dnsrsearch.com
dvrlists.com doctissimo.fr
eandsgallery.com doctormakarova.ru
eas-seo.com dogsrun.net
easyshoppermac.com doktoronline.no
ecommerce-seo.org dollartree.info
ednorxmed.com dolohen.com
eduserver.net domainroam.win
ekaterinburg.xrus.org downtuptv.gq
eldiariodeguadalajara.com doxysexy.com
elektronischezigarettekaufen1.beeplog.com dvr.biz.ua
epngo.bz e90post.com
erotikstories.ru ecookna.com.ua
editmedios.com
edudocs.net
ekosmetyki.net.pl
ekspertmed.com
electronicadirect.com
elektronischezigarette2.mex.tl
elektrozigarette1.dreamwidth.org
elektrozigaretten1.tumblr.com
elmifarhangi.com
elvenmachine.com
emediate.eu
enbersoft.com
energydiet24.ru
ero-advertising.com
erotik-kostenlos.net
essay-edu.biz essay-edu.biz
essaypro.com etur.ru
eugenevaultstorage.com eu-cookie-law.blogspot.com
evehemming.blogspot.com.au evaashop.ru
executehosting.com excitacion.info
eyes-on-you.ga expertblog.info
f07.de expertnaya-ocenka.ru
f0815.de extractorandburner.com
face.hostingx.eu ez8motelseaworldsandiego.com
fanoboi.com ezigarettekaufen2.blox.pl
farmingworm.com ezigarettenkaufen1.pagina.gr
fdzone.org ezigarettenshop1.yolasite.com
fearcrow.com facialporntube.com
fetroshok.ru fanrto.com
filesmonster.porn fashionindeed.ml
financehint.eu fil.ru
financetip.eu fingerprintjs.com
findacheaplawyers.com firstsiteguide.com
flipper.top flavors.me
forexunion.net footbalive.org
forum.poker4life.ru forex.osobye.ru
forodvd.com
forum.doctissimo.fr
forumprofi.de forumprofi.de
fotopop.club foxydeal.com
fotosfotos.eu fr-bearings.ru
free-traffic.xyz free-video-tool.com
freecamdollars.com freemags.cc
freenom.link friendflnder.com
freshwallpapers.info fyxabomiw.ru
frustrated-favorable.gq g7m.pl
frvo.alptandem.ru game300.ru
ftns.ru gamezblox.com
fuck-paid-share-buttons.xyz gardene.ru
gael-s.ru gerhardhealer.com
gambarkatabaru.com getpopunder.com
gamedayhouse.com girls-ufa.ru
gameprimary.com golfresa.lucania.se
gamevalue7.weebly.com goodhousekeeping.com
geileweiber.tk gotcher.us
go2album.com groupmoney.ru
gogps.me growmyfunds.ca
gorabagrata.ru guardlink.org
graphics8.info hahashka.ru
greatgrace.ru hamptonoaks.ca
guidefs.ru hanwei.us
hao123.com hccoder.info
harmonyglen.us helpmymacfaster.trade
haveinc.xyz hentai-manga.porn
healthcarestore.info hermesbirkinhandbagoutlets.com
hermesbelts.xyz hetmanship.xyz
hidefiles.org
hitcpm.com hitcpm.com
hledejvshopech.cz hostcritique.com
hoholikik.club hotxnights.info
hornymatches.com hoverboard360.es
housediz.com howlongdoestizanidinestayinyoursystem.blogspot.com
howmuchdoestizanidinecost.blogspot.com hspline.com
i-midias.net.br humbmosquina.tk
idealtits.net hunthillfarmtrust.org
ilovevitaly.ru hystersister.com
imadedinner.net images.gyffu.com
immigrational.info imediadesk.com
in-tandem.co imicrovision.com
inbabes.sexushost.com imperia31.ru
impisr.edunsk.ru
inboundlinks.win
infogame.name
infospot.pt infospot.pt
infowarcraft.ru intellektmedia.at
insider.pro intervsem.ru
insurple.com invitefashion.com
inzn.ru iqoption.com
iopeninghours.co.uk isitpaleo.info
ip-guide.com isoveti.ru
iron-age.info itronics.ca
irunfar.com iwanttodeliver.com
istanbulit.com jabimgo.pw
istizanidinelikexanax.blogspot.com jamiembrown.com
itis4you.com javstock.com
izhevsk.xrus.org jbl-charge.info
jerseywholesalebizchina.com jerseyssportsshop.com
justucalling32211123456789.tk jerseywholesaleelitestore.com
joy-penguin.com
jwss.cc
kalandranis.gr
kaliningrad.zrus.org
kamorel.com
karachev-city.ru
karpun-iris.ru
karting196.ru
kazinogames.lv
kellyonline.xyz kellyonline.xyz
kenaba.su
kerei.ru kerei.ru
keywords-monitoring-success.com keywordglobal.co.za
kihi.gdn kino-ecran.ru
kinocccp.net kinopolet.net
kinosed.net kinostorm.net
kiskinhouse.com kirov.zrus.org
klejonka.info kiwe-analytics.com
komputers-best.ru kol-energo.ru
konica.kz
koptims.tiu.ru
koronirealestate.gr
kosynka-games.ru kosynka-games.ru
kunstaktien.info kraljeva-sutjeska.com
kurbappeal.info kupiproday.com.ua
labafydjxa.su labvis.host.sk
landinez.co lapitec.eu
landmania.ru larger.io
laxdrills.com laxob.com
leboard.ru lechenie-gemorroya.com
ledis.top ledis.top
legalrc.biz libertybilisim.com
lego4x4.ru lifebyleese.com
lennatin.info likesuccess.com
letmacworkfaster.site linksharingt.com
letsart.ru livefixer.com
letslowbefast.site liveu.infoteka.hu
lexaprogeneric.link
lidoradio.com
likrot.com
linkbux.ru
liran-locks.com
liveinternet.ro
livingroomdecor.info livingroomdecor.info
louboutinbooties.xyz lombia.com
lubetube.com longgreen.info
lumb.co lotto6888.com
lunamedia.co low-format.ru
m0r0zk0-krava.ru lsex.xyz
madot.onlinewebshop.net macrotek.ru
maju.bluesalt.co magicdiet.gq
mall.uk
manifestation.betteroffers.review
manifestyourmillion.com manifestyourmillion.com
masterseek.com manimpotence.com
masthopehomes.com manve.info
meble-bogart.info mapquestz.us
medicinacom.ru markjaybeefractal.com
medkritika.ru matb3aa.com
memberty.com mature.free-websites.com
mensandals.xyz mbiologi.ru
metallosajding.ru metarip.ru
microsearch.ru minharevisao.com
mir-business-24.ru missclub.info
mirobuvi.com.ua mmog-play.ru
misandesign.se moje-recenze.cz
missis.top moncleroutletonline.xyz
mlvc4zzw.space moviezbonkerssk.cf
mnogabukaff.net mp3films.ru
mobplayer.ru mttwtrack.com
mojpreskumanie.com musicktab.com
moncleronline.xyz musicstock.me
moyakuhnia.ru myhydros.org
msk-diplomat.com nardulan.com
multgo.ru narkologiya-orel.ru
mwtpludn.review naturalpharm.com.ua
myftpupload.com nhl17coins.exblog.jp
mygameplus.com nudejapan.net
myra.top
mysexpics.ru
nakrutka.cc
namecrumilchlet.tk
nastydollars.com
nero-us.com
netfacet.net
nfljerseyscheapbiz.us
niki-mlt.ru
njpalletremoval.com
northfacestore.online
nottyu.xyz
novatech.vn
nvformula.ru nvformula.ru
oakridgemo.com offer.camp
obuv-kupit.ru
oda.as
offside2.5v.pl
ok-ua.info
ok.ru ok.ru
oklogistic.ru okel.co
okmedia.sk
olgacvetmet.com
onefilms.net
oneminutesite.it oneminutesite.it
ooomeru.ru online7777.com
openmultipleurl.com ontargetseo.us
otbelivanie-zubov.com orsonet.ru
owohho.com osoznanie-narkotikam.net
p-business.ru outclicks.net
panicatack.com ownshop.win
pavlodar.xkaz.org paceform.com
pc-services.ru paparazzistudios.com.au
pearlisland.ru papasdelivery.ru
pierrehardysale.online partner-high.men
piulatte.cz pawli.eu
plastweb.ru paydayloanslocal.com
playfortuna-play.ru pb-dv.ru
perosan.com
pestomou.info
pharmacyincity.com
picsfair.com
picturesboss.com
picturesfrom.com
picturesify.com
pijoto.net
pixelrz.com
pixshark.com
pk-services.ru
placid-rounded-coast.glitch.me
plastgran.com
pliks.pl
plohaya-kreditnaya-istoriya.ru
poems.com.ua
pokemongooo.ml
polimga.pw polimga.pw
pons-presse.com
popugaychiki.com popugaychiki.com
pornolook.net popunder.net
pornosemki.info poquoson.org
pornoslive.net porno-chaman.info
powc.r.ca.d.sendibm2.com porno.simple-image.com.ua
pravoholding.ru pornohd1080.online
printdirectforless.com pornosmola.info
privatefx.all4invest.info pornotubs.com
prodvigator.ua powitania.pl
pronorm.fr pozdrawleniya.ru
proxyradar.com pr-ten.de
pxhdwsm.com privhosting.com
quangcaons.com profitkode.com
queerspace.com profolan.pl
quelle.ru prointer.net.ua
ptr.ruvds.com
purchasepillsnorx.com
push-ad.com
q-moto.ru
qexyfu.bugs3.com
quality-traffic.com
quickcashlimited.com quickcashlimited.com
ranksonic.com quit-smoking.ga
rapevideosmovies.com quizzitch.net
real-time-analytics.com radiogambling.com
recipedays.com rankia.com
reeyanaturopathy.com rasteniya-vs-zombi.ru
registry-clean-up.net ratemodels.net
remont-komputerov-notebook.ru registratciya-v-moskve.ru
remote-dba.de reversing.cc
renecaovilla.online reyel1985.webnode.fr
rennlist.com rniaeba.ga
rentaremotecomputer.com rocis.site
rentehno.ru rocketchange.ru
resellerclub.com rosbalt.com.ua
ros-ctm.ru royalcar-ufa.ru
rumamba.com rucrypt.com
ruscoininvest.company rus-teh.narod.ru
russia-today-video.ru rustic-quiver.win
s.lollypopgaming.com sa-live.com
sadaholding.com saltspray.ru
saddiechoua.com sammlungfotos.online
satellite.maps.ilovevitaly.com samo-soznanie.ru
scanhub.ru sanyuprojects.com
scanmyphones.com satoristudio.net
scanner-andrew.top sbetodiodnye-lampy.ru
scanner-barak.top scenarii-1-sentyabrya.uroki.org.ua
scanner-donald.top search-goo.com
scanner-george.top
scanner-julianna.top
scanner-willy.top
scenicmissouri.us
scmor.ilxc.cc
sdi-pme.com
search.alot.com search.alot.com
searchimage.co search.xtconnect.com
searchmywindow-a.akamaihd.net segol.tv
secret.xn--oogle-wmc.com sel-hoz.com
security60-e.com seminarykansas904.ml
seeingmeerkat.com seocheckupx.com
seinterface.com seotoolsagency.com
seminarygeorgia59.ga servethis.com
seomarketings.online servicecenter.co.ua
sex-watch.com sex-pr.net
shell-pmr.ru sex-tracker.com
shodanhq.com sex.hotblog.top
shop-electron.ru sexobzor.info
sibtest.ru sexvporno.ru
site-speed-checker.site shemalegalls.blogporn.in
sk.golden-praga.ru shoesonlinebuy.xyz
slayerlife.com sigmund-freud.co.uk
slonechka.ru silktide.com
slowmac.tech sindragosa.comxa.com
sluganarodu.ru sinestesia.host.sk
smartadserver.com siteonomy.com
snip.to sloopyjoes.com
snjatie-geroinovoy-lomki.ru smailik.org
soc-econom-problems.ru smallseotools.com
social-search.me smart-scripts.com
smartpet.ru
socas.pluto.ro
solitaire-game.ru
sonnikforme.ru
space4update.pw
spammen.de
splendorsearch-a.akamaihd.net splendorsearch-a.akamaihd.net
sports-supplements.us spravka-medosmotr.ru
spy-sts.com stardevine.com
squidoo.com stariy-baku.com
steamoff.net staticfs.host
steebook.com sticken.co
sterva.cc strangeduckfilms.com
strag-invest.ru stromerrealty.com
streamin.to superfish.com
streetfire.net supermama.top
support.nopeas.sk surgut.zrus.org
tacbibirfa.tk svbur.ru
swagbucks.com
swinger-mobil.net
tamada69.com tamada69.com
tandvardshuset.net tattomedia.com
thenetinfo.com td-33.ru
thesmartsearch.net tds-advert005.info
titelhelden.eu technika-remont.ru
tizanidineinteractions.blogspot.com teenforporn.com
tizanidinesideeffects.blogspot.com testingads.pro
tkanorganizma.ru thefds.net
top-study.work theguardlan.com
topflownews.com thfox.com
topsiteminecraft.com tizanidinehcl2mgtablet.blogspot.com
torrent-to-magnet.com tizanidineorflexeril.blogspot.com
tracklead.net topmira.com
trafficgenius.xyz torrentdownloadhub.com
trahic.ru torrents.cd
trion.od.ua tourismvictoria.com
truck-land.life trafficmp.com
trudogolik.net trafficstars.com
tversvet.ru trahvid.com
twodollarshows.com truck-rece.life
um-razum.ru turbabitload.weebly.com
univerfiles.com turizmus.us
unrealcommander.biz turn-up-life.life
uptime-gamma.net tvory.predmety.in.ua
us-america.ru tvteleport.ru
twitlinks.com
ucoz.ru
uhdtv.website
ukr-lit.com
uogonline.com
uvozdeckych.info uvozdeckych.info
validdomain.xyz
valkiria-tk.ru valkiria-tk.ru
vandrie-ict.nl vapomnoncri.tk
varbergsvind.se
varikozdok.ru
vashsvet.com vashsvet.com
vbabule.net vchulkah.net
vclicks.net
vedomstvo.net
velpanex.ru
vertaform.com vertaform.com
video--production.com videooko.weebly.com
video-production.com vipps.com.my
videojam.tv vksaver-all.ru
videospornogratisx.net voluumtrk.com
virtualbb.com vvpg.ru
visionwell.com.cn watsonrealtycorp.com
voditeltrezviy.ru
voloslove.ru
vzlomfb.com
w3javascript.com
wallpapersist.com
webextract.profound.net
webix.me webix.me
webjam.com wedding0venues.tk
weekes.biz.tc wejdz-tu.pl
welck.octopis.com wheelchairliftsarea.com
westen-z.life where-toget.com
wholesalejerseysgaa.com wiosenny-bon-1500.pl
wholinkstome.com wnhjavlhezp.gq
wildnatureimages.com womens-journal.net
win-spy.com workle.website
wladimirpayen.com worldinternetauthority.com
womensplay.net worldtraveler.world
woodyguthrie.se www.bookmaker-bets.com
woweb.com.ua www.ehscloud.cn
writingservices17.blogspot.ru xdoza.com
wsgames.ru xingzi-vision.com
www.timer4web.com xn----7sbahjd3btneuw1joc.xn--p1ai
x-lime.net xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
xbaboon.com xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xblognetwork.com xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--80ak6aa92e.com xsfetish.org
xn--80aodinpgi.xn--p1ai yatrk.xyz
xn--b1agm2d.net yoursearch.me
xolodremont.ru yousense.info
xxart.ru z-master.ru
xxxsiterips.xyz zarabiaj-dzis.pl
y8games-free.com zeleznobeton.ru
yogamatsexpert.com zetmaster.ru
yuweng.info zixizop.net.ru
zambini.ru zoogdinsney.com
zaobao.com.sg zreloeporno.tv
zapiszto.pl zynax.ua
zed21.net zzbroya.com.ua
zirondelli.it
znakomstvaonlain.ru
zoodrawings.com
zophim.me
zrizvtrnpale.tk
zverokruh-shop.cz

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
108shot.com 1001desert.com
123cha.com 123locker.com
2020iscoming.info 300richmond.co.nz
76brighton.co.uk abbanreddy.com
999webdesign.com absolute-s.ru
a.pr-cy.ru actulite.com
aa08daf7e13b6345e09e92f771507fa5f4.com adminshop.com
abcguru.xyz adobereader-free.ru
above.com adviceforum.info
academiacsmendoza.org alphavisions.net
advokat-grodno.by anal-acrobats.hol.es
ahhjf.com androids-store.com
aksonural.ru
alibestsale.com
alphahoverboards.com
angkortours.vn
apelsinnik.website
apiallgeniusinfo-a.akamaihd.net
app5.letmacworkfaster.world app5.letmacworkfaster.world
apple.com-cleaner.systems apple.com-cleaner.systems
arquivo.pt arabgirls.adultgalls.com
astana.xxxkaz.org arclk.net
availit.weebly.com arcteryxstore.online
arewater.com
arkartex.ru
artefakct.com
artparquet.ru
asociatia-tipografilor-transilvania.ro
asrv-a.akamaihd.net
asscenihotosidea.blogspot.com
avirasecureserver.com avirasecureserver.com
avtoarenda.by babespcs.com
balkanfarma.ru bestmarriages.com
battleforupdating.site bezcmexa.ru
bausparen.bz.it bg6s0.com
beachtoday.ru bgviagrachrx.com
beetpics.pw bingo8888.com
bestfortraders.com bjetjt.com
besthoro.ru blacktwhite.com
bestserials.com blobar.org
betshuckclean.com blogig.org
bget.ru bloglag.com
bitcoins-live.ru bluesman.nu
blog.koreadaily.com bonkers.name
blogos.kz bonvillan.com
blpmovies.com brateg.xyz
boanonihaca.tk brothers-smaller.ru
bpro1.top browserprotecter.com
brainzpod.win buchananshardware.com
brakehawk.com cannibalcheerleader.com
brk-rti.ru
buy-meds24.com
buzzzg.men
canacopegdl.com
cardsharp1.ru cardsharp1.ru
carrauterie.be cartierreplica.win
cartierlove.xyz cartujano-pre.de
cbb1smartlist12.click cfjump.com
celebsopics.com cheapjerseysa.com
chelnytruck.ru cherrypointplace.ca
chelyabinsk.xrus.org cicaki.net
chinajerseyswholesalecoupons.com cienum.fr
cialisndbrx.com cineacademy.ru
cityadspix.com cinemaenergy-hd.ru
classicakuhni.ru
clickso.com
clknsee.com
compliance-alex.xyz compliance-alex.xyz
compliance-alexa.top computernetworksonline.com
compliance-barak.xyz connectionstrenth.com
compliance-julianna.top cornomase.win
compliance-mark.top crackguru.tk
constantaservice.net data-mining.tk
dame-ns.kz dengi-pod-zalog-nedvizhimosti.ru
darmebel.com.ua destinationrealestate.com
data.vtc.pw dikx.gdn
dieswaene.com dispo.de
documentsite.net dollartree.info
downloadkakaotalk.com dpihatinh.gov.vn
drev.biz drugstoreforyou.com
drpornogratisx.xxx ekspresihati.info
dumuelave.xyz epngo.bz
eachdayisagift.review espaceinventoristes.com
earl-brown.info eugenevaultstorage.com
edudocs.net excaliburfilms.com
electronicadirect.com extstat.com
elite-sex-finders.com filmania-x.ru
emergencyneeds.org
enginebay.ru
enpolis.ru
enthuse.computernetworksonline.com
eu-cookie-law.info
europeanwatches.ru
evehemming.blogspot.com.au
ezb.elvenmachine.com
ezigarettekaufen.myblog.de
ezigarettekaufen2.yolasite.com
fanoboi.com
financehint.eu financehint.eu
financepoints.eu firesub.pl
finuse.com forexunion.net
flprog.com foxydeal.com
fm-upgrade.ru free-fbook-traffic.com
forum20.smailik.org free-floating-buttons.com
freelotto.com freewebs.com
freewhatsappload.com fungirlsgames.net
friendflnder.com fyl.com.ru
galleryawesome.com getaclueamerica.com
golaya.pw getprismatic.com
gfaq.ru
ghazel.ru
googlefeud.com
googleseo.com.tr googleseo.com.tr
greenidesign.co gopixdatabase.com
grtyi.com gratis-sexkontakte.com
gta-club.ru gsmtlf.ru
gta-top.ru gz2.bbsoldes.fr
guiadeserraazul.com hausfrauensex18.com
habermetre.com havepussy.com
hermesbirkinhandbagoutlets.com healgastro.com
hotel-mkad.ru histats.com
howmuchdoestizanidinecost.blogspot.com hit-kino.ru
iamsport.org hotdl.in
id-forex.com hoverboard360.nl
ilbe.club howtowhitenteethfast.xyz
imagine-ex.co incest-ru.com
img.wallpaperstock.net indo-export.ru
infobabki.ru infodocsportal.com
instasexyblog.com iqs.biz.ua
intellektmedia.at je7.us
isitwp.com jeremyeaton.co
itag.pw jewelryandfiligree.com
japfm.com kazan.xrus.org
javlibrary.cc kazan.zrus.org
jmat.cn keywordbasket.com
jyvopys.com keywordhut.com
kamen-e.ru khafre.us
kartiniresto.com klin3952.ru
kchaxton.com kol-energo.ru
ketrzyn.pl kotaku.com
kino-key.info kountrylife.com
kinogonew.ru kvartiry-remont.ucoz.ru
kinostorm.net lafourmiliaire.com
kostenlos-sexvideos.com lashstudia.ru
krasula.pp.ua lernur.net
kreativperlen.ch letsrepair.in
letslowbefast.site lifebyleese.com
lexaprogeneric.link linkarena.com
lirunet.ru lion.snapmap.us
liveinternet.ro livefixer.com
lob.com.ru
loginduepunti.it
lol-smurfs.com
lordzfilmz.me
lostfilm-online.ru
louboutinbooties.xyz louboutinbooties.xyz
lutherstable.org lovi-moment.com.ua
m0r0zk0-krava.ru ltvperf.com
mac-shield.com machicon-akihabara.info
magento-crew.net mantramusic.ru
make-money-online.com manve.info
mansparskats.com marcoislandvacations.net
maranbrinfo.com.br marinetraffic.com
massage-info.nl meddesk.ru
max-p.men medictube.ru
merryhouse.co.uk metarip.ru
mirzonru.net mikozstop.com
mitrasound.ru
mmoguider.ru mmoguider.ru
moviemail-online.co.uk moncleronline.xyz
msfsaar.de moyakuhnia.ru
musflashtv.com mrlmedia.net
musicpro.monster myyour.eu
myhealthcare.com nacap.ru
myliveblog.ru nastydollars.com
naughtyconnect.com nosecret.com.ua
nomuos.it ofanda.com
norththeface.store onlainbesplatno.ru
obiavo.in openstat.com
offtime.ru orgasmatrix.com
ons-add.men orgasmus-virtual.com
partner-host.men pdamods.ru
pb-dv.ru perm.xrus.org
picsearch.com photostudiolightings.com
platinumdeals.gr picurams.pw
play-movie.pl pijoto.net
playlott.com poisk-zakona.ru
plohaya-kreditnaya-istoriya.ru poptool.net
polybuild.ru pornhubforum.tk
porn-w.org pornobrazzers.biz
pornohd1080.online pornoreino.com
pornoklad.ru pornoted.com
privetsochi.ru prostitutki-astana.org
proekt-gaz.ru pshare.biz
rankscanner.com pskcijdc.bloger.index.hr
rating-casino2021.ru py100.ru
realizmobi.com pyrodesigns.com.au
relatodelpresente.com.ar questionmarque.ch
resant.ru radiogambling.com
resultshub-a.akamaihd.net rentalcarnavi.info
rixpix.ru rosbalt.com.ua
russintv.fr rusvideos.su
screpy.com rvtv.ru
searchencrypt.com saecsa.co
servethis.com sashagreyblog.ga
sex-foto.pw sawin.beth.webd.pl
sexs-foto.top search.1and1.com
shitmovs.com seoboxes.com
silverage.ru sergiorossistore.online
siteonomy.com sex-znakomstva.online
skanninge.se shmetall.com.ua
sklad-24.ru shopfishing.com.ua
soheavyblog.com sientalyric.co
spinazdrav.ru smartshoppymac.com
starwars.wikia.com snegozaderzhatel.ru
strongsignal-a.akamaihd.net social-fun.ru
success-seo.com socialbutton.xyz
svetodiodoff.ru squidoo.com
t3chtonic.com steame.ru
thedownloadfreeonlinegames.blogspot.com steebook.com
tizanidine4mgstreetprice.blogspot.com stretchingabuckblog.com
tizanidineandcipro.blogspot.com strongholdsb.ru
tizanidineformigraines.blogspot.com susanholtphotography.com
tizanidinestreetvalue.blogspot.com symphonyintegratedhealthcare.com
top-l2.com taaaak.com
topflownews.com tdsing.ru
topquality.cf titan-ads.life
torrentred.games tizanidineandgabapentin.blogspot.com
toyota.7zap.com tizanidinecapsules.blogspot.com
tramadolandtizanidine.blogspot.com tizanidinehcl4mgtabinfo.blogspot.com
trustedmaccleaner.com tizanidinesideeffectsweightloss.blogspot.com
tt-ipd.info tizanidinewithdrawal.blogspot.com
tupper-posuda.ru top250movies.ru
tversvet.ru torrents.cd
twodollarshows.com toursmaps.com
uk-zheu20.ru traffmonster.info
ukr-lit.com tvnewsclips.info
upproar.com twittruth.com
uptime.com ua-company.ru
uptimebot.net ukkala.xyz
uroffer.link url-extractor.xyz
vellings.info valmetrundan.se
verymes.xyz vesnatehno.ru
visa-china.ru vezuviy.su
visitcambridge.org viagraneggrx.com
vizag.kharkov.ua video-hollywood.ru
volgograd.xrus.org vzlom-na-zakaz.com
voloslove.ru w3data.co
vseigry.fun wait3sec.org
watsonrealtycorp.com watchmygf.net
webs.com website-datenbank.de
wladimirpayen.com wedding0venues.tk
woman-h.ru weddingdresses.xyz
wrz0iuebwhp5fg.freeddns.com weprik.ru
www.pinnacle-bets.com wesharepics.info
x-stars.ru whatsupinfoley.com
xaijo.com winx-play.ru
xclicks.net ww2awards.info
xn----dtbndd4ae7eub.top www.bookmaker-bets.com
xxxnatelefon.ru wyniki-lista.pl
zapatosenventa.info x-lime.com
zocaparj.kz xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
zoodrawings.com xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
xn--80aeahghtf8ac5i.xn--p1ai
xxx-treker.ru
xzlive.com
yadro.ru
yebocasino.com
yoopsie.com
youradulthosting.com
ytmnd.com
yuweng.info
z-master.ru
zeroredirect2.com
znakomstva-moskva77.ru
zrizvtrnpale.tk
zrus.org

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

17
dev-tools/test_units/ratelimittest-master.txt
View file

@ -1,20 +1,3 @@
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html> <html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.126 0; 10.1.1.96 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.07.3235 ### Version: V4.2022.07.3236
### Updated: Mon Jul 18 22:01:13 UTC 2022 ### Updated: Tue Jul 19 10:03:40 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.07.3235 ### Version: V4.2022.07.3236
### Updated: Mon Jul 18 22:01:13 UTC 2022 ### Updated: Tue Jul 19 10:03:40 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##