initial commit

g3proxy/doc/protocol/client/egress_path_selection.rst

@@ -0,0 +1,58 @@
+.. _protocol_client_path_selection:
+
+#####################
+Egress Path Selection
+#####################
+
+Usually there are many outgoing ip addresses on proxy machine, and we may provide one to one server port mapping to
+each of them. In many cases, we may have one server port mapping to many outgoing ip addresses, and we default to
+use a random selection policy, which will match the most common use cases. But for some users, they may want to
+connect to such one to many server port, but bind each connection to a specific ip address, so we need a path selection
+policy which the user can tell us within each connection negotiation stage.
+
+For path selection to work, the escapers used must support and enable it.
+Not all escapers support it, see the config documentation for each for confirmation.
+
+server support
+==============
+
+custom http header
+------------------
+
+Only http proxy server can support this.
+
+See :ref:`path_selection_header <config_server_http_proxy_egress_path_selection_header>` for more info.
+
+socks extension
+---------------
+
+Only socks proxy server can support this.
+
+No implementation for now.
+
+username extension
+------------------
+
+All servers which support user auth with a username can support this.
+
+No implementation for now.
+
+selection methods
+=================
+
+default
+-------
+
+**value**: constant("default")
+
+The default one, just like no path selection enabled.
+
+by index
+--------
+
+**value**: usize
+
+For escapers with multiple nodes (may be next escapers or ip addresses), the node with the specified index will be used.
+
+The value will be wrapped into range *1 - len(nodes)*.
+**NOTE*** the start value is *1*, *0* is the same as *len(nodes) - 1*.

g3proxy/doc/protocol/client/http_custom_codes.rst

@@ -0,0 +1,32 @@
+.. _protocol_client_http_custom_codes:
+
+#################
+Http Custom Codes
+#################
+
+The following custom HTTP reply codes are used:
+
+* 521 WEB_SERVER_IS_DOWN
+
+  The upstream or the next peer has refused or reset our connection request.
+
+* 522 CONNECTION_TIMED_OUT
+
+  Timeout to connect to upstream or next peer.
+
+* 523 ORIGIN_IS_UNREACHABLE
+
+  For network error, such network unreachable and host unreachable, occurred while connecting to upstream or next peer.
+
+* 525 SSL_HANDSHAKE_FAILED
+
+  Tls handshake with upstream failed.
+
+  .. note::
+
+    Tls handshake with next proxy peer (it's not upstream) will generate internal server error instead,
+    as we usually use different tls client config for proxy peers.
+
+* 530 ORIGIN_DNS_ERROR
+
+  Failed to resolve the ip address of upstream or next peer.

g3proxy/doc/protocol/client/http_custom_headers.rst

@@ -0,0 +1,90 @@
+.. _protocol_client_http_custom_headers:
+
+###################
+Http Custom Headers
+###################
+
+Chained Final Info Headers
+==========================
+
+We won't reset or append new values. So we can make sure the value is set by the proxy that is nearest to upstream.
+
+X-BD-Upstream-Id
+----------------
+
+If this header is set, it means the response is from remote. The value for this value is *server_id*.
+
+If we see this header in error response, we can know that the response is coming from the server after the one with the
+same *server_id*, which may be upstream server, or another chained proxy server.
+
+This header is controlled by http_proxy server option
+:ref:`http_forward_mark_upstream <config_server_http_proxy_http_forward_mark_upstream>`.
+
+X-BD-Upstream-Addr
+------------------
+
+If set, it will contains the remote address we are trying connect to from the far-most proxy server.
+
+This header is controlled by http_proxy server option
+:ref:`echo_chained_info <config_server_http_proxy_echo_chained_info>`.
+
+X-BD-Outgoing-Ip
+----------------
+
+If set, it will contains the local bind ip address we are using to connect to remote from the far-most proxy server.
+
+This header is controlled by http_proxy server option
+:ref:`echo_chained_info <config_server_http_proxy_echo_chained_info>`.
+
+Local Info Headers
+==================
+
+Every proxy configured will append new values. The value comes first if the proxy is nearer to upstream.
+
+X-BD-Remote-Connection-Info
+---------------------------
+
+The value format:
+
+::
+
+    <server_id>[; bind=<bind_ip>][; remote=<remote_addr>][; local=<local_addr>][; expire=<expire_rfc3339>]
+
+* bind_ip
+
+  the ip address we decide to bind to before connection.
+
+* remote_addr
+
+  the socket address we decide to connect to before connection.
+
+* local_addr
+
+  the local socket address we bound to after the connection established.
+
+* expire_rfc3339
+
+  expire time of the remote peer. This field won't be set if the remote side is the target upstream.
+
+This header is controlled by http_proxy server option :ref:`server_id <config_server_http_proxy_server_id>`.
+
+X-BD-Dynamic-Egress-Info
+------------------------
+
+The value format:
+
+::
+
+    <server_id>[; isp=<isp>][; ip=<ip>][; area=<area>]
+
+* isp
+
+  ISP for the egress ip address.
+
+* ip
+
+  The egress ip address from external view.
+
+* area
+
+  Area of the egress ip address. The format is strings joined with '/', like 中国/山东/济南.

g3proxy/doc/protocol/client/index.rst

@@ -0,0 +1,37 @@
+.. _protocol_client:
+
+###############
+Client Protocol
+###############
+
+We support the following protocols for clients:
+
+* http proxy
+
+  - Both HttpForward and HttpConnect are supported.
+  - HttpsForward is also supported, but not enabled by default.
+  - Only http version 1.0 and 1.1. Currently no support for http version 2 and 3.
+  - Only Basic auth is supported.
+  - TLS 1.2 and above can be enabled.
+  - see :doc:`http_custom_headers` for custom headers.
+  - see :doc:`http_custom_codes` for custom reply codes.
+
+* socks proxy
+
+  - socks4 and socks4a are supported (no ident verification) with most escapers.
+  - socks5 TcpConnect is supported with most escapers.
+  - socks5 UdpAssociate is supported with some escapers but disabled by default at server side. The default enabled one
+    is UdpConnect which is much simplified, but require the target address for each packet to be the same.
+    The address family for the tcp and udp connection at client side should be the same if no explicit bind ip set.
+  - socks5 User auth is the only one that we support yet.
+  - no TLS and DTLS support yet.
+  - no socks6 support yet.
+  - see :doc:`socks5_custom_reply` for socks5 custom reply field.
+
+.. toctree::
+   :hidden:
+
+   http_custom_headers
+   http_custom_codes
+   socks5_custom_reply
+   egress_path_selection

g3proxy/doc/protocol/client/socks5_custom_reply.rst

@@ -0,0 +1,11 @@
+.. _protocol_client_socks5_custom_reply:
+
+###################
+Socks5 Custom Reply
+###################
+
+In addition to the standard reply field defined in `rfc1928`_, the following reply field are also used:
+
+* X'09' Connection timeout
+
+.. _rfc1928: https://tools.ietf.org/html/rfc1928

g3proxy/doc/protocol/helper/index.rst

@@ -0,0 +1,15 @@
+.. _protocol_helper:
+
+###############
+Helper Protocol
+###############
+
+route_query
+===========
+
+This protocol is used to make queries in route_query escaper. See :doc:`route_query`.
+
+.. toctree::
+   :hidden:
+
+   route_query

g3proxy/doc/protocol/helper/route_query.rst

@@ -0,0 +1,86 @@
+.. _protocol_helper_route_query:
+
+===========
+Route Query
+===========
+
+We need a peer service in :ref:`route_query escaper <configuration_escaper_route_query>`. This page describes the
+protocol we used to communicate with the peer service.
+
+The peer service should listen on a UDP port, which may be IPv4 or IPv6 based, we will sending requests to this port.
+
+Each UDP packet from our side to the peer service will contains exactly one request. And each UDP packet from the peer
+service should contains exactly one response.
+
+Both the request and the response are structured data and should be encoded in `msgpack`_ format.
+
+.. _msgpack: https://msgpack.org/
+
+The root of the request and the response should be a map, we will describe the keys of them in the following.
+
+request
+=======
+
+id
+--
+
+**required**, **type**: uuid binary
+
+Set the id of the request.
+
+user
+----
+
+**required**, **type**: string
+
+Set the username of the proxy request. The value may be an empty string if auth is disabled on the proxy side.
+
+host
+----
+
+**required**, **type**: string
+
+Set the target host of the proxy request. May be a domain or an IP address.
+
+client_ip
+---------
+
+Set the client ip address. This will be set only of :ref:`query_pass_client_ip <configuration_escaper_route_query_pass_client_ip>` is enabled.
+
+response
+========
+
+id
+--
+
+**required**, **type**: uuid binary | uuid string
+
+Set the id of the corresponding request.
+
+nodes
+-----
+
+**optional**, **type**: string | seq
+
+Set the next escaper(s) those can be selected.
+
+For *seq* value, each of its element must be :ref:`weighted name str <conf_value_weighted_name_str>`.
+
+If empty, the :ref:`fallback node <configuration_escaper_route_query_fallback_node>` escaper config will take effect.
+
+**default**: empty
+
+ttl
+---
+
+**optional**, **type**: u32
+
+Set the expire ttl of this response.
+
+If 0, the :ref:`protective cache ttl <configuration_escaper_route_query_protective_cache_ttl>` escaper config will
+take effect
+
+.. note:: expired records will be cached some more time before cleared, see
+ :ref:`vanish_after_expired <configuration_escaper_route_query_vanish_after_expired>` for more info.
+
+**default**: 0

g3proxy/doc/protocol/index.rst

@@ -0,0 +1,21 @@
+.. _protocol:
+
+########
+Protocol
+########
+
+The protocols we used can be described as two categories:
+
+* client protocol
+
+  The protocols we used to communicate to clients. See :doc:`client/index`.
+
+* helper protocol
+
+  The protocols we used to communicate to helper programs. See :doc:`helper/index`.
+
+.. toctree::
+   :hidden:
+
+   client/index
+   helper/index