Isolate browser extension experimental implementation.

This commit is contained in:
Zied Aouini 2022-10-30 19:34:18 +01:00
parent b322b45911
commit 8737ee0a09
22 changed files with 594 additions and 0 deletions

68
browser/README.md Normal file
View file

@ -0,0 +1,68 @@
# NFStream Browser Extensions (experimental)
This repository contains NFStream browser extensions (Chrome and Firefox) that was implemented
as part of a exploratory work for end-host ground truth generation.
The main idea was to explore what could be extracted from a browser context data (tab_id, tab_url,
etc.) to enrich end-host visibility when the process generating the traffic is a browser.
This implementation was first ([**this commit**][commit]) introduced in NFStream as experimental/draft code
(not officially documented), we decided for clarity purposes to isolate it and provide it in this separate repository.
Feel free to use it or extend it as part of your research experiments.
## How it works?
NFStream browser extension is simply a listener on `onResponseStarted` of [**webRequest**][webrequest] browser API that
perform the following actions:
* extract features such as timestamp, tab identifier, request identifier, tab is active, server IP address and URL.
* export it as HTTP POST it a JSON format to **localhost:export_port**.
`export_port` is configurable by the user in the nfstream extension options menu and is set by default to `28314`.
## How to use it?
### Setup your environment
``` bash
python3 -m pip install --upgrade pip
virtualenv venv-py -p /usr/bin/python3
source venv-py/bin/activate
```
### Load the extensions
#### On Chrome
* Open: chrome://extensions
* Enable `Developer Mode`.
* Click on `Load unpacked`.
* Select `nfstream/browser/chrome` folder.
* Now NFStream extension is loaded, you can navigate to details menu and then select extension options to configure
a specific export port.
#### On Firefox
* Open `about:debugging`
* Click `This Firefox`
* Click `Load Temporary Add-on`
* Open the `nfstream/browser/firefox` and select any file inside the extension,
* Now NFStream extension is loaded, you can navigate to `about:addons` and then select extension preferences
to configure specific export port.
### extension_server
```
usage: python3 extension_server.py listening_port_number
positional arguments:
listening_port_number port number to listen to (default 28314).
```
The script will display exported data from the extensions. That's it!
[webrequest]: https://developer.chrome.com/docs/extensions/reference/webRequest/
[commit]: https://github.com/nfstream/nfstream/commit/729e69d97bac69c12609b28271d32542e9f5dc45

View file

@ -0,0 +1,36 @@
/*
------------------------------------------------------------------------------------------------------------------------
background.js
Copyright (C) 2019-22 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
*/
function send_event(event) {
chrome.tabs.query({active:true, currentWindow:true},
function(tabs) {
if ((! String(event.ip).match("127.0.0.1")) && (! (event.ip == undefined))
&& (! (tabs[0] == undefined)) && (! (tabs[0].url == undefined))
&& (! (tabs[0].id == undefined)) && (! (event.timeStamp == undefined))) { // Filter out
chrome.tabs.sendMessage(tabs[0].id, {browser: "chrome",
version: "1.0",
timestamp: event.timeStamp,
tab_id: tabs[0].id,
req_id: event.requestId,
ip_address: event.ip,
tab_is_active: tabs[0].active,
tab_url: tabs[0].url});
}
});
}
chrome.webRequest.onResponseStarted.addListener(send_event, {urls: ["<all_urls>"]}, ["extraHeaders",
"responseHeaders"]);

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 688 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.4 KiB

View file

@ -0,0 +1,28 @@
{
"manifest_version": 2,
"version": "1.0",
"name": "NFStream for Chrome",
"description": "NFStream Chrome extension for system visibility.",
"author": "NFStream Developers",
"homepage_url" : "https://www.nfstream.org/",
"icons": {
"16": "images/nfstream_icon_16.png",
"32": "images/nfstream_icon_32.png",
"48": "images/nfstream_icon_48.png",
"128": "images/nfstream_icon_128.png"
},
"background" : {
"scripts": ["background.js"]
},
"options_ui": {
"page": "options.html",
"open_in_tab": false
},
"content_scripts": [
{
"matches": ["<all_urls>"],
"js": ["nfstream.js"]
}
],
"permissions": ["webRequest", "storage", "tabs", "<all_urls>"]
}

View file

@ -0,0 +1,33 @@
/*
------------------------------------------------------------------------------------------------------------------------
nfstream.js
Copyright (C) 2019-22 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
*/
function upload_to_nfstream(msg) {
chrome.storage.local.get({export_port: '28314'},
function (items) {
var xhr = new XMLHttpRequest();
var uid = "nfstream-" + msg.browser + "-" + msg.tab_id + "-" + msg.req_id + ".json";
try {
xhr.open("POST",
"http://localhost:" + items.export_port + "/" + uid,
true);
xhr.setRequestHeader("Content-type", "application/json");
xhr.send(JSON.stringify(msg));
} catch (e) {
// ignore
}});
}
chrome.runtime.onMessage.addListener(upload_to_nfstream);

View file

@ -0,0 +1,33 @@
<!---
------------------------------------------------------------------------------------------------------------------------
options.html
Copyright (C) 2019-22 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
--->
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="style.css">
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>NFStream for Chrome</title>
</head>
<hr />
<label>
&nbsp;&nbsp;&nbsp;NFStream listening port number:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</label>
<input type="number" id="export_port" min="0" max="65525">
<br><br>
<a class="custom_button" style="background-color:#f14e4e" id="update">Update</a>
<script src="options.js"></script>

44
browser/chrome/options.js Normal file
View file

@ -0,0 +1,44 @@
/*
------------------------------------------------------------------------------------------------------------------------
options.js
Copyright (C) 2019-22 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
*/
function restore_options() {
chrome.storage.local.get({export_port: '28314'},
function(items) {
document.getElementById('export_port').value = items.export_port;
});
}
function update_input(element, color) {
element.style.border = '3px solid';
element.style.borderColor = color;
setTimeout(function() {
element.style.border = '';
element.style.borderColor = '';
restore_options();
}, 750);
}
function update_options() {
var export_port = document.getElementById('export_port');
if ((export_port.value < 0) || (export_port.value > 65525) || (export_port.value == undefined)) {
update_input(export_port, "red");
} else {
chrome.storage.local.set({export_port: export_port.value},
update_input(export_port, "green"));
}
}
document.addEventListener('DOMContentLoaded', restore_options);
document.getElementById('update').addEventListener('click', update_options);

43
browser/chrome/style.css Normal file
View file

@ -0,0 +1,43 @@
/*
------------------------------------------------------------------------------------------------------------------------
style.css
Copyright (C) 2019-22 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
*/
div {text-align: center;}
a.custom_button{
display:inline-block;
padding:0.3em 1.2em;
margin:0 0.1em 0.1em 0;
border:0.16em solid rgba(255,255,255,0);
border-radius:2em;
box-sizing: border-box;
text-decoration:none;
font-family:'Roboto',sans-serif;
font-weight:300;
color:#FFFFFF;
text-shadow: 0 0.04em 0.04em rgba(0,0,0,0.35);
text-align:center;
transition: all 0.2s;
}
a.custom_button:hover{
border-color: rgba(255,255,255,1);
}
@media all and (max-width:30em){
a.custom_button{
display:block;
margin:0.2em auto;
}
}

View file

@ -0,0 +1,93 @@
"""
------------------------------------------------------------------------------------------------------------------------
extension_server.py
Copyright (C) 2019-22 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
"""
from http.server import HTTPServer, BaseHTTPRequestHandler
from collections import namedtuple
import json
import sys
NFRequest = namedtuple('NFRequest', ['browser',
'timestamp',
'remote_ip',
'tab_id',
'request_id',
'tab_is_active',
'tab_url'])
class NFRequestHandler(BaseHTTPRequestHandler):
""" Handler for HTTP request from browser extension """
def _set_headers(self):
""" headers setter """
self.send_header('Access-Control-Allow-Origin', '*')
self.send_header('Access-Control-Allow-Methods', 'GET, POST, OPTIONS')
self.send_header("Access-Control-Allow-Headers", "X-Requested-With")
self.send_header("Access-Control-Allow-Headers", "content-type")
self.end_headers()
def do_OPTIONS(self):
""" OPTIONS handler """
self.send_response(200)
self._set_headers()
def do_POST(self):
""" POST handler """
if self.path.endswith('.json') and (self.path.startswith('/nfstream-chrome') or
self.path.startswith('/nfstream-firefox')):
length = self.headers['content-length']
data = json.loads(self.rfile.read(int(length)))
self.send_response(200)
self._set_headers()
request = NFRequest(data["browser"],
float(data["timestamp"]),
data["ip_address"],
data["tab_id"],
data["req_id"],
data["tab_is_active"],
data["tab_url"])
# For sake of brevity, we print it only
print(request)
# However you can use this server in a child process and poll from it using a Queue (channel)
# self.server.channel.put(request)
else:
self.send_response(400)
self._set_headers()
def log_message(self, fmt, *args):
return
class NFRequestServer(HTTPServer):
""" NFRequest HTTP server"""
def __init__(self, *args):
HTTPServer.__init__(self, *args)
self.stopped = False
# self.channel = channel
if __name__ == '__main__': # Mandatory if you are running on Windows Platform
try:
port = int(sys.argv[1])
except IndexError: # not specified
port = 28314
server_address = ('', port) # localhost with configurable port
server = NFRequestServer(server_address, NFRequestHandler)
try:
while not server.stopped:
server.handle_request()
except KeyboardInterrupt:
exit(0)

View file

@ -0,0 +1,35 @@
/*
------------------------------------------------------------------------------------------------------------------------
background.js
Copyright (C) 2019-22 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
*/
function send_event(event) {
browser.tabs.query({active:true, currentWindow:true},
function(tabs) {
if ((! String(event.ip).match("127.0.0.1")) && (! (event.ip == undefined))
&& (! (tabs[0] == undefined)) && (! (tabs[0].url == undefined))
&& (! (tabs[0].id == undefined)) && (! (event.timeStamp == undefined))) { // Filter out
browser.tabs.sendMessage(tabs[0].id, {browser: "firefox",
version: "1.0",
timestamp: event.timeStamp,
tab_id: tabs[0].id,
req_id: event.requestId,
ip_address: event.ip,
tab_is_active: tabs[0].active,
tab_url: tabs[0].url});
}
});
}
browser.webRequest.onResponseStarted.addListener(send_event, {urls: ["<all_urls>"]}, ["responseHeaders"]);

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 688 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.4 KiB

View file

@ -0,0 +1,28 @@
{
"manifest_version": 2,
"version": "1.0",
"name": "NFStream for Firefox",
"description": "NFStream Firefox extension for system visibility.",
"author": "NFStream Developers",
"homepage_url" : "https://www.nfstream.org/",
"icons": {
"16": "images/nfstream_icon_16.png",
"32": "images/nfstream_icon_32.png",
"48": "images/nfstream_icon_48.png",
"128": "images/nfstream_icon_128.png"
},
"background" : {
"scripts": ["background.js"]
},
"options_ui": {
"page": "options.html",
"open_in_tab": false
},
"content_scripts": [
{
"matches": ["<all_urls>"],
"js": ["nfstream.js"]
}
],
"permissions": ["webRequest", "storage", "tabs", "<all_urls>"]
}

View file

@ -0,0 +1,33 @@
/*
------------------------------------------------------------------------------------------------------------------------
nfstream.js
Copyright (C) 2019-22 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
*/
function upload_to_nfstream(msg) {
browser.storage.local.get({export_port: '28314'},
function (items) {
var xhr = new XMLHttpRequest();
var uid = "nfstream-" + msg.browser + "-" + msg.tab_id + "-" + msg.req_id + ".json";
try {
xhr.open("POST",
"http://localhost:" + items.export_port + "/" + uid,
true);
xhr.setRequestHeader("Content-type", "application/json");
xhr.send(JSON.stringify(msg));
} catch (e) {
// ignore
}});
}
browser.runtime.onMessage.addListener(upload_to_nfstream);

View file

@ -0,0 +1,33 @@
<!---
------------------------------------------------------------------------------------------------------------------------
options.html
Copyright (C) 2019-22 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
--->
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="style.css">
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>NFStream for Chrome</title>
</head>
<hr />
<label>
&nbsp;&nbsp;&nbsp;NFStream listening port number:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</label>
<input type="number" id="export_port" min="0" max="65525">
<br><br>
<a class="custom_button" style="background-color:#f14e4e" id="update">Update</a>
<script src="options.js"></script>

View file

@ -0,0 +1,44 @@
/*
------------------------------------------------------------------------------------------------------------------------
options.js
Copyright (C) 2019-22 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
*/
function restore_options() {
browser.storage.local.get({export_port: '28314'},
function(items) {
document.getElementById('export_port').value = items.export_port;
});
}
function update_input(element, color) {
element.style.border = '3px solid';
element.style.borderColor = color;
setTimeout(function() {
element.style.border = '';
element.style.borderColor = '';
restore_options();
}, 750);
}
function update_options() {
var export_port = document.getElementById('export_port');
if ((export_port.value < 0) || (export_port.value > 65525) || (export_port.value == undefined)) {
update_input(export_port, "red");
} else {
browser.storage.local.set({export_port: export_port.value},
update_input(export_port, "green"));
}
}
document.addEventListener('DOMContentLoaded', restore_options);
document.getElementById('update').addEventListener('click', update_options);

43
browser/firefox/style.css Normal file
View file

@ -0,0 +1,43 @@
/*
------------------------------------------------------------------------------------------------------------------------
style.css
Copyright (C) 2019-22 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
*/
div {text-align: center;}
a.custom_button{
display:inline-block;
padding:0.3em 1.2em;
margin:0 0.1em 0.1em 0;
border:0.16em solid rgba(255,255,255,0);
border-radius:2em;
box-sizing: border-box;
text-decoration:none;
font-family:'Roboto',sans-serif;
font-weight:300;
color:#FFFFFF;
text-shadow: 0 0.04em 0.04em rgba(0,0,0,0.35);
text-align:center;
transition: all 0.2s;
}
a.custom_button:hover{
border-color: rgba(255,255,255,1);
}
@media all and (max-width:30em){
a.custom_button{
display:block;
margin:0.2em auto;
}
}