initial commit

g3proxy/doc/configuration/auditors/index.rst

@@ -0,0 +1,127 @@
+.. _configuration_auditor:
+
+*******
+Auditor
+*******
+
+The type for each auditor config is *map*, the keys are as follows:
+
+name
+----
+
+**required**, **type**: str
+
+Set the auditor name, which will can be referenced in :ref:`server config <conf_server_common_auditor>`.
+
+protocol_inspection
+-------------------
+
+**optional**, **type**: :ref:`protocol inspection <conf_value_dpi_protocol_inspection>`
+
+Set basic config for protocol inspection.
+
+**default**: set with default value
+
+server_tcp_portmap
+------------------
+
+**optional**, **type**: :ref:`server tcp portmap <conf_value_dpi_server_tcp_portmap>`
+
+Set the portmap for protocol inspection based on server side tcp port.
+
+**default**: set with default value
+
+client_tcp_portmap
+------------------
+
+**optional**, **type**: :ref:`client tcp portmap <conf_value_dpi_client_tcp_portmap>`
+
+Set the portmap for protocol inspection based on client side tcp port.
+
+**default**: set with default value
+
+tls_cert_generator
+------------------
+
+**optional**, **type**: :ref:`tls cert generator <conf_value_dpi_tls_cert_generator>`
+
+Set certificate generator for TLS interception.
+
+If not set, TLS interception will be disabled.
+
+**default**: not set
+
+tls_interception_client
+-----------------------
+
+**optional**, **type**: :ref:`tls interception client <conf_value_dpi_tls_interception_client>`
+
+Set the tls client config for server handshake in TLS interception.
+
+**default**: set with default value
+
+log_uri_max_chars
+-----------------
+
+**optional**, **type**: usize
+
+Set the max chars for the log of URI.
+
+**default**: 1024
+
+h1_interception
+---------------
+
+**optional**, **type**: :ref:`h1 interception <conf_value_dpi_h1_interception>`
+
+Set http 1.x interception config.
+
+**default**: set with default value
+
+h2_interception
+---------------
+
+**optional**, **type**: :ref:`h2 interception <conf_value_dpi_h2_interception>`
+
+Set http 2.0 interception config.
+
+**default**: set with default value
+
+icap_reqmod_service
+-------------------
+
+**optional**, **type**: :ref:`icap service config <conf_value_audit_icap_service_config>`
+
+Set the ICAP REQMOD service config.
+
+**default**: not set
+
+.. versionadded:: 1.7.3
+
+icap_respmod_service
+--------------------
+
+**optional**, **type**: :ref:`icap service config <conf_value_audit_icap_service_config>`
+
+Set the ICAP RESPMOD service config.
+
+**default**: not set
+
+.. versionadded:: 1.7.3
+
+.. _conf_auditor_application_audit_ratio:
+
+application_audit_ratio
+-----------------------
+
+**optional**, **type**: :ref:`random ratio <conf_value_random_ratio>`
+
+Set the application audit (like ICAP REQMOD/RESPMOD) ratio for incoming requests.
+
+This also controls whether protocol inspection is really enabled for a specific request.
+
+User side settings may override this.
+
+**default**: 1.0
+
+.. versionadded:: 1.7.4

g3proxy/doc/configuration/escapers/direct_fixed.rst

@@ -0,0 +1,97 @@
+.. _configuration_escaper_direct_fixed:
+
+direct_fixed
+============
+
+This escaper will access the target upstream from local machine directly.
+
+The following interfaces are supported:
+
+* tcp connect
+* udp relay
+* udp connect
+* http(s) forward
+* ftp over http
+
+The following common keys are supported:
+
+* :ref:`shared_logger <conf_escaper_common_shared_logger>`
+* :ref:`resolver <conf_escaper_common_resolver>`, **required**
+* :ref:`resolve_strategy <conf_escaper_common_resolve_strategy>`
+
+  The user custom resolve strategy will be taken into account.
+
+* :ref:`tcp_sock_speed_limit <conf_escaper_common_tcp_sock_speed_limit>`
+* :ref:`udp_sock_speed_limit <conf_escaper_common_udp_sock_speed_limit>`
+* :ref:`no_ipv4 <conf_escaper_common_no_ipv4>`
+* :ref:`no_ipv6 <conf_escaper_common_no_ipv6>`
+* :ref:`tcp_connect <conf_escaper_common_tcp_connect>`
+
+  The user tcp connect params will be taken into account.
+
+* :ref:`tcp_misc_opts <conf_escaper_common_tcp_misc_opts>`
+* :ref:`udp_misc_opts <conf_escaper_common_udp_misc_opts>`
+* :ref:`extra_metrics_tags <conf_escaper_common_extra_metrics_tags>`
+
+bind_ip
+-------
+
+**optional**, **type**: :ref:`ip addr str <conf_value_ip_addr_str>` | seq
+
+Set the bind ip address(es) for sockets.
+
+For *seq* value, each of its element must be :ref:`ip addr str <conf_value_ip_addr_str>`.
+Only random select is supported. Use *route* type escapers if is doesn't meet your needs.
+
+**default**: not set
+
+egress_network_filter
+---------------------
+
+**optional**, **type**: :ref:`egress network acl rule <conf_value_egress_network_acl_rule>`
+
+Set the network filter for the (resolved) remote ip address.
+
+**default**: all permitted except for loop-back and link-local addresses
+
+happy_eyeballs
+--------------
+
+**optional**, **type**: :ref:`happy eyeballs <conf_value_happy_eyeballs>`
+
+Set the HappyEyeballs config.
+
+**default**: default HappyEyeballs config
+
+.. versionadded:: 1.5.3
+
+tcp_keepalive
+-------------
+
+**optional**, **type**: :ref:`tcp keepalive <conf_value_tcp_keepalive>`
+
+Set tcp keepalive.
+
+The tcp keepalive set in user config will be taken into account.
+
+**default**: no keepalive set
+
+resolve_redirection
+-------------------
+
+**optional**, **type**: :ref:`resolve redirection <conf_value_resolve_redirection>`
+
+Set the dns redirection rules at escaper level.
+
+**default**: not set
+
+enable_path_selection
+---------------------
+
+**optional**, **type**: bool
+
+Weather we should enable path selection.
+
+.. note:: Path selection on server side should be open, or this option will have no effects.
+
+**default**: false

g3proxy/doc/configuration/escapers/direct_float.rst

@@ -0,0 +1,156 @@
+.. _configuration_escaper_direct_float:
+
+************
+direct_float
+************
+
+This escaper will access the target upstream from local machine directly. The local bind ip, which is required,
+can be set via the `publish` rpc method.
+
+The following interfaces are supported:
+
+* tcp connect
+* http(s) forward
+
+The Cap'n Proto RPC publish command is supported on this escaper, the published data should be a map, with the keys:
+
+* ipv4
+
+  Set the IPv4 bind ip address(es).
+  The value could be an array of or just one :ref:`bind ip <config_escaper_dynamic_bind_ip>`.
+
+* ipv6
+
+  Set the IPv6 bind ip address(es).
+  The value could be an array of or just one :ref:`bind ip <config_escaper_dynamic_bind_ip>`.
+
+There is no path selection support for this escaper.
+
+Config Keys
+===========
+
+The following common keys are supported:
+
+* :ref:`shared_logger <conf_escaper_common_shared_logger>`
+* :ref:`resolver <conf_escaper_common_resolver>`, **required**
+* :ref:`resolve_strategy <conf_escaper_common_resolve_strategy>`
+
+  The user custom resolve strategy will be taken into account.
+
+* :ref:`tcp_sock_speed_limit <conf_escaper_common_tcp_sock_speed_limit>`
+* :ref:`udp_sock_speed_limit <conf_escaper_common_udp_sock_speed_limit>`
+* :ref:`no_ipv4 <conf_escaper_common_no_ipv4>`
+* :ref:`no_ipv6 <conf_escaper_common_no_ipv6>`
+* :ref:`tcp_connect <conf_escaper_common_tcp_connect>`
+
+  The user tcp connect params will be taken into account.
+
+* :ref:`tcp_misc_opts <conf_escaper_common_tcp_misc_opts>`
+* :ref:`extra_metrics_tags <conf_escaper_common_extra_metrics_tags>`
+
+cache_ipv4
+----------
+
+**recommend**, **type**: :ref:`file path <conf_value_file_path>`
+
+Set the cache file for published IPv4 IP Address(es).
+
+It is recommended to set this as the fetch of peers at startup may be finished after the first batch of requests.
+
+The file will be created if not existed.
+
+**default**: not set
+
+cache_ipv6
+----------
+
+**recommend**, **type**: :ref:`file path <conf_value_file_path>`
+
+Set the cache file for published IPv6 IP Address(es).
+
+It is recommended to set this as the fetch of peers at startup may be finished after the first batch of requests.
+
+The file will be created if not existed.
+
+**default**: not set
+
+egress_network_filter
+---------------------
+
+**optional**, **type**: :ref:`egress network acl rule <conf_value_egress_network_acl_rule>`
+
+Set the network filter for the (resolved) remote ip address.
+
+**default**: all permitted except for loopback and link-local addresses
+
+happy_eyeballs
+--------------
+
+**optional**, **type**: :ref:`happy eyeballs <conf_value_happy_eyeballs>`
+
+Set the HappyEyeballs config.
+
+**default**: default HappyEyeballs config
+
+.. versionadded:: 1.5.3
+
+tcp_keepalive
+-------------
+
+**optional**, **type**: :ref:`tcp keepalive <conf_value_tcp_keepalive>`
+
+Set tcp keepalive.
+
+The tcp keepalive set in user config will be taken into account.
+
+**default**: 60s
+
+resolve_redirection
+-------------------
+
+**optional**, **type**: :ref:`resolve redirection <conf_value_resolve_redirection>`
+
+Set the dns redirection rules at escaper level.
+
+**default**: not set
+
+.. _config_escaper_dynamic_bind_ip:
+
+Bind IP
+=======
+
+We use json string to represent a dynamic bind ip, with a map type as root element.
+
+* ip
+
+  **required**, **type**: :ref:`ip addr str <conf_value_ip_addr_str>`
+
+  Set the IP address. The address family should match the type of the publish key described above.
+
+* isp
+
+  **optional**, **type**: str
+
+  ISP for the egress ip address.
+
+* eip
+
+  **optional**, **type**: :ref:`ip addr str <conf_value_ip_addr_str>`
+
+  The egress ip address from external view.
+
+* area
+
+  **optional**, **type**: :ref:`egress area <conf_value_egress_area>`
+
+  Area of the egress ip address.
+
+* expire
+
+  **optional**, **type**: :ref:`rfc3339 datetime str <conf_value_rfc3339_datetime_str>`
+
+  Set the expire time of this dynamic ip.
+
+  **default**: not set
+
+If all optional fields can be set with the default value, the root element can be just a *ip*.

g3proxy/doc/configuration/escapers/dummy_deny.rst

@@ -0,0 +1,16 @@
+.. _configuration_escaper_dummy_deny:
+
+**********
+dummy_deny
+**********
+
+This is the dummy escaper designed to deny all requests.
+
+There is no path selection support for this escaper.
+
+Config Keys
+===========
+
+The following common keys are supported:
+
+* :ref:`extra_metrics_tags <conf_escaper_common_extra_metrics_tags>`

g3proxy/doc/configuration/escapers/index.rst

@@ -0,0 +1,208 @@
+.. _configuration_escaper:
+
+*******
+Escaper
+*******
+
+The type for each escaper config is *map*, with two always required keys:
+
+* *name*, which specify the name of the escaper.
+* *type*, which specify the real type of the escaper, decides how to parse other keys.
+
+There are many types of escaper, each with a section below.
+
+Escapers
+========
+
+.. toctree::
+   :maxdepth: 2
+
+   dummy_deny
+   direct_fixed
+   direct_float
+   proxy_float
+   proxy_http
+   proxy_https
+   proxy_socks5
+   route_mapping
+   route_query
+   route_resolved
+   route_select
+   route_upstream
+   route_client
+   trick_float
+
+Common Keys
+===========
+
+This section describes the common keys, they may be used by many escapers.
+
+.. _conf_escaper_common_shared_logger:
+
+shared_logger
+-------------
+
+**optional**, **type**: ascii
+
+Set the escaper to use a logger running on a shared thread.
+
+**default**: not set
+
+.. _conf_escaper_common_resolver:
+
+resolver
+--------
+
+**type**: str
+
+Set the resolver to use for this escaper.
+
+If the specified resolver doesn't exist in configure, a default DenyAll resolver will be used.
+
+.. _conf_escaper_common_resolve_strategy:
+
+resolve_strategy
+-----------------
+
+**optional**, **type**: :ref:`resolve strategy <conf_value_resolve_strategy>`
+
+Set the resolve strategy.
+
+.. _conf_escaper_common_tcp_sock_speed_limit:
+
+tcp_sock_speed_limit
+--------------------
+
+**optional**, **type**: :ref:`tcp socket speed limit <conf_value_tcp_sock_speed_limit>`
+
+Set speed limit for each tcp socket.
+
+**default**: no limit, **alias**: tcp_conn_speed_limit | tcp_conn_limit
+
+.. versionchanged:: 1.4.0 changed name to tcp_sock_speed_limit
+
+.. _conf_escaper_common_udp_sock_speed_limit:
+
+udp_sock_speed_limit
+--------------------
+
+**optional**, **type**: :ref:`udp socket speed limit <conf_value_udp_sock_speed_limit>`
+
+Set speed limit for each udp socket.
+
+**default**: no limit, **alias**: udp_relay_speed_limit | udp_relay_limit
+
+.. versionchanged:: 1.4.0 changed name to udp_sock_speed_limit
+
+.. _conf_escaper_common_no_ipv4:
+
+no_ipv4
+-------
+
+**optional**, **type**: bool
+
+Disable IPv4. This setting should be compatible with :ref:`resolve_strategy <conf_escaper_common_resolve_strategy>`.
+
+**default**: false
+
+.. _conf_escaper_common_no_ipv6:
+
+no_ipv6
+-------
+
+**optional**, **type**: bool
+
+Disable IPv6. This setting should be compatible with :ref:`resolve_strategy <conf_escaper_common_resolve_strategy>`.
+
+**default**: false
+
+.. _conf_escaper_common_tcp_connect:
+
+tcp_connect
+-----------
+
+**optional**, **type**: :ref:`tcp connect <conf_value_tcp_connect>`
+
+Set tcp connect params.
+
+.. note:: For *direct* type escapers, the user level tcp connect params will be taken to limit the final value.
+
+.. _conf_escaper_common_tcp_misc_opts:
+
+tcp_misc_opts
+-------------
+
+**optional**, **type**: :ref:`tcp misc sock opts <conf_value_tcp_misc_sock_opts>`
+
+Set misc tcp socket options.
+
+**default**: not set, nodelay is default enabled
+
+.. _conf_escaper_common_udp_misc_opts:
+
+udp_misc_opts
+-------------
+
+**optional**, **type**: :ref:`udp misc sock opts <conf_value_udp_misc_sock_opts>`
+
+Set misc udp socket options.
+
+**default**: not set
+
+.. _conf_escaper_common_default_next:
+
+default_next
+------------
+
+**required**, **type**: str
+
+Set the default next escaper for *route* type escapers.
+
+.. _conf_escaper_common_pass_proxy_userid:
+
+pass_proxy_userid
+-----------------
+
+**optional**, **type**: bool
+
+Set if we should pass userid (username) to next proxy.
+
+If set, the native basic auth method will be used when negotiation with next proxy, and the username field will be set
+to the real username, the password field set to our package name (g3proxy if not forked).
+
+**default**: false
+
+.. note:: This will conflict with the real auth of next proxy.
+
+.. _conf_escaper_common_use_proxy_protocol:
+
+use_proxy_protocol
+------------------
+
+**optional**, **type**: :ref:`proxy protocol version <conf_value_proxy_protocol_version>`
+
+Set the version of PROXY protocol we use for outgoing tcp connections.
+
+**default**: not set, which means PROXY protocol won't be used
+
+.. _conf_escaper_common_peer_negotiation_timeout:
+
+peer_negotiation_timeout
+------------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the negotiation timeout for next proxy peers.
+
+**default**: 10s
+
+.. _conf_escaper_common_extra_metrics_tags:
+
+extra_metrics_tags
+------------------
+
+**optional**, **type**: :ref:`static metrics tags <conf_value_static_metrics_tags>`
+
+Set extra metrics tags that should be added to escaper stats and user stats already with escaper tags added.
+
+**default**: not set

g3proxy/doc/configuration/escapers/proxy_float.rst

@@ -0,0 +1,391 @@
+.. _configuration_escaper_proxy_float:
+
+***********
+proxy_float
+***********
+
+This escaper provide the capability to access the target upstream through dynamic remote proxies.
+
+The following interfaces are supported:
+
+* tcp connect
+* http(s) forward
+
+The following remote proxy protocols are supported:
+
+* Http Proxy
+* Socks5 Proxy
+
+The Cap'n Proto RPC publish command is supported on this escaper, the published data should be an array of
+or just one :ref:`peer <config_escaper_dynamic_peer>`.
+
+There is no path selection support for this escaper.
+
+Config Keys
+===========
+
+The following common keys are supported:
+
+* :ref:`shared_logger <conf_escaper_common_shared_logger>`
+* :ref:`tcp_sock_speed_limit <conf_escaper_common_tcp_sock_speed_limit>`
+* :ref:`tcp_misc_opts <conf_escaper_common_tcp_misc_opts>`
+* :ref:`peer negotiation timeout <conf_escaper_common_peer_negotiation_timeout>`
+* :ref:`extra_metrics_tags <conf_escaper_common_extra_metrics_tags>`
+
+source
+------
+
+**required**, **type**: :ref:`url str <conf_value_url_str>` | map | null
+
+Set the fetch source for peers.
+
+We support many type of sources. The type is detected by reading the *scheme* field of url,
+or the *type* key of the map. See :ref:`sources <config_escaper_dynamic_source>` for all supported type of sources.
+
+cache
+-----
+
+**recommend**, **type**: :ref:`file path <conf_value_file_path>`
+
+Set the cache file.
+
+It is recommended to set this as the fetch of peers at startup may be finished after the first batch of requests.
+
+The file will be created if not existed.
+
+**default**: not set
+
+refresh_interval
+----------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the refresh interval to update peers from the configured source.
+
+**default**: 1s
+
+bind_ipv4
+---------
+
+**optional**, **type**: :ref:`ipv4 addr str <conf_value_ipv4_addr_str>`
+
+Set the bind ip address for inet sockets.
+
+**default**: not set
+
+bind_ipv6
+---------
+
+**optional**, **type**: :ref:`ipv6 addr str <conf_value_ipv6_addr_str>`
+
+Set the bind ip address for inet6 sockets.
+
+**default**: not set
+
+tls_client
+----------
+
+**optional**, **type**: bool | :ref:`openssl tls client config <conf_value_openssl_tls_client_config>`
+
+Enable https peer, and set TLS parameters for this local TLS client.
+If set to true or empty map, a default config is used.
+
+**default**: not set
+
+tcp_connect_timeout
+-------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the tcp connect application level timeout value.
+
+**default**: 30s
+
+tcp_keepalive
+-------------
+
+**optional**, **type**: :ref:`tcp keepalive <conf_value_tcp_keepalive>`
+
+Set tcp keepalive.
+
+The tcp keepalive set in user config won't be taken into account.
+
+**default**: 60s
+
+expire_guard_duration
+---------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+If the peer has an expire value, we won't connect to it if we can reach the expire time after adding this value.
+
+**default**: 5s
+
+.. _config_escaper_dynamic_source:
+
+Sources
+=======
+
+For *map* format, the **type** key should always be set.
+
+passive
+-------
+
+Do not fetch peers. Only publish is needed.
+
+The root value of source may be set to *null* to use passive source.
+
+redis
+-----
+
+Fetch peers from a redis db.
+
+The keys used in the *map* format are:
+
+* addr
+
+  **required**, **type**: :ref:`upstream str <conf_value_upstream_str>`
+
+  Set the address of the redis instance. The default port is 6379 which can be omitted.
+
+* db
+
+  **optional**, **type**: int
+
+  Set the database.
+
+  **default**: 0
+
+* username
+
+  **optional**, **type**: str
+
+  Set the username for redis 6 database if needed. It is required if connect to an ACL enabled redis 6 database.
+
+  **default**: not set
+
+* password
+
+  **optional**, **type**: str
+
+  Set the password.
+
+  **default**: not set
+
+* connect_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the connect timeout.
+
+  **default**: 5s
+
+* read_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout for redis read operation.
+
+  **default**: 2s
+
+* sets_key
+
+  **required**, **type**: str
+
+  Set the key for the sets that store the peers. Each string record in the set is a single peer.
+  See :ref:`peers <config_escaper_dynamic_peer>` for its formats.
+
+For *url* str values, the format is:
+
+    redis://[username][:<password>@]<addr>/<db>?sets_key=<sets_key>
+
+redis_cluster
+-------------
+
+Fetch peers from a redis cluster.
+
+The value should be a *map*, with these keys:
+
+* initial_nodes
+
+  **required**, **type**: :ref:`upstream str <conf_value_upstream_str>`
+
+  Set the address of the startup nodes.
+
+* username
+
+  **optional**, **type**: str
+
+  Set the username.
+
+  .. versionadded:: 1.7.0
+
+* password
+
+  **optional**, **type**: str
+
+  Set the password.
+
+  **default**: not set
+
+* read_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout for redis read operation.
+
+  **default**: 2s
+
+* sets_key
+
+  **required**, **type**: str
+
+  Set the key for the sets that store the peers. Each string record in the set is a single peer.
+  See :ref:`peers <config_escaper_dynamic_peer>` for its formats.
+
+.. _config_escaper_dynamic_peer:
+
+Peers
+=====
+
+We use json string to represent a peer, with a map type as root element.
+
+Common keys
+-----------
+
+* type
+
+  **required**, **type**: str
+
+  It tells us the peer type.
+
+* addr
+
+  **required**, **type**: :ref:`sockaddr str <conf_value_sockaddr_str>`
+
+  Set the socket address we can connect to the peer.
+  No domain name is allowed here.
+
+* isp
+
+  **optional**, **type**: str
+
+  ISP for the egress ip address.
+
+* eip
+
+  **optional**, **type**: :ref:`ip addr str <conf_value_ip_addr_str>`
+
+  The egress ip address from external view.
+
+* area
+
+  **optional**, **type**: :ref:`egress area <conf_value_egress_area>`
+
+  Area of the egress ip address.
+
+* expire
+
+  **optional**, **type**: :ref:`rfc3339 datetime str <conf_value_rfc3339_datetime_str>`
+
+  Set the expire time for this peer.
+
+* tcp_sock_speed_limit
+
+  **optional**, **type**: :ref:`tcp socket speed limit <conf_value_tcp_sock_speed_limit>`
+
+  Set the speed limit for each tcp connections to this peer.
+
+  .. versionchanged:: 1.4.0 changed name to tcp_sock_speed_limit
+
+The following types are supported:
+
+http
+----
+
+* username
+
+  **optional**, **type**: :ref:`username <conf_value_username>`
+
+  Set the username for HTTP basic auth.
+
+* password
+
+  **optional**, **type**: :ref:`password <conf_value_password>`
+
+  Set the password for HTTP basic auth.
+
+* http_connect_rsp_header_max_size
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set the max header size for received CONNECT response.
+
+  **default**: 4KiB
+
+* extra_append_headers
+
+  **optional**, **type**: map
+
+  Set extra headers append to the requests sent to upstream.
+  The key should be the header name, both the key and the value should be in ascii string type.
+
+  .. note:: No duplication check is done here, use it with caution.
+
+
+https
+-----
+
+* username
+
+  **optional**, **type**: :ref:`username <conf_value_username>`
+
+  Set the username for HTTP basic auth.
+
+* password
+
+  **optional**, **type**: :ref:`password <conf_value_password>`
+
+  Set the password for HTTP basic auth.
+
+* tls_name
+
+  **optional**, **type**: :ref:`tls name <conf_value_tls_name>`
+
+  Set the tls server name for server certificate verification.
+
+  .. note:: IP address is not supported by now. So if not set, the connection will fail.
+
+  **default**: not set
+
+* http_connect_rsp_header_max_size
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set the max header size for received CONNECT response.
+
+  **default**: 4KiB
+
+* extra_append_headers
+
+  **optional**, **type**: map
+
+  Set extra headers append to the requests sent to upstream.
+  The key should be the header name, both the key and the value should be in ascii string type.
+
+  .. note:: No duplication check is done here, use it with caution.
+
+socks5
+------
+
+* username
+
+  **optional**, **type**: :ref:`username <conf_value_username>`
+
+  Set the username for Socks5 User auth.
+
+* password
+
+  **optional**, **type**: :ref:`password <conf_value_password>`
+
+  Set the password for Socks5 User auth.

g3proxy/doc/configuration/escapers/proxy_http.rst

@@ -0,0 +1,113 @@
+.. _configuration_escaper_proxy_http:
+
+proxy_http
+==========
+
+This escaper will access the target upstream through another http proxy.
+
+The following interfaces are supported:
+
+* tcp connect
+* http(s) forward
+
+There is no path selection support for this escaper.
+
+The following common keys are supported:
+
+* :ref:`shared_logger <conf_escaper_common_shared_logger>`
+* :ref:`resolver <conf_escaper_common_resolver>`, **required** only if *proxy_addr* is domain
+* :ref:`resolve_strategy <conf_escaper_common_resolve_strategy>`
+* :ref:`tcp_sock_speed_limit <conf_escaper_common_tcp_sock_speed_limit>`
+* :ref:`no_ipv4 <conf_escaper_common_no_ipv4>`
+* :ref:`no_ipv6 <conf_escaper_common_no_ipv6>`
+* :ref:`tcp_connect <conf_escaper_common_tcp_connect>`
+* :ref:`tcp_misc_opts <conf_escaper_common_tcp_misc_opts>`
+* :ref:`pass_proxy_userid <conf_escaper_common_pass_proxy_userid>`
+* :ref:`use_proxy_protocol <conf_escaper_common_use_proxy_protocol>`
+* :ref:`peer negotiation timeout <conf_escaper_common_peer_negotiation_timeout>`
+* :ref:`extra_metrics_tags <conf_escaper_common_extra_metrics_tags>`
+
+proxy_addr
+----------
+
+**required**, **type**: :ref:`upstream str <conf_value_upstream_str>` | seq
+
+Set the target proxy address. The default port is 3128 which can be omitted.
+
+For *seq* value, each of its element must be :ref:`weighted upstream addr <conf_value_weighted_upstream_addr>`.
+
+proxy_addr_pick_policy
+----------------------
+
+**optional**, **type**: :ref:`selective pick policy <conf_value_selective_pick_policy>`
+
+Set the policy to select next proxy address.
+
+The key for rendezvous/jump hash is *<client-ip>[-<username>]-<upstream-host>*.
+
+**default**: random
+
+proxy_username
+--------------
+
+**optional**, **type**: :ref:`username <conf_value_username>`
+
+Set the proxy username. The Basic auth scheme is used by default.
+
+.. note::
+
+  Conflict with :ref:`pass_proxy_userid <conf_escaper_common_pass_proxy_userid>`
+
+proxy_password
+--------------
+
+**optional**, **type**: :ref:`password <conf_value_password>`
+
+Set the proxy password. Required if username is present.
+
+bind_ipv4
+---------
+
+**optional**, **type**: :ref:`ipv4 addr str <conf_value_ipv4_addr_str>`
+
+Set the bind ip address for inet sockets.
+
+**default**: not set
+
+bind_ipv6
+---------
+
+**optional**, **type**: :ref:`ipv6 addr str <conf_value_ipv6_addr_str>`
+
+Set the bind ip address for inet6 sockets.
+
+**default**: not set
+
+http_forward_capability
+-----------------------
+
+**optional**, **type**: :ref:`http forward capability <conf_value_http_forward_capability>`
+
+Set the http forward capability if the next proxy.
+
+**default**: all capability disabled
+
+http_connect_rsp_header_max_size
+--------------------------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the max header size for received CONNECT response.
+
+**default**: 4KiB
+
+tcp_keepalive
+-------------
+
+**optional**, **type**: :ref:`tcp keepalive <conf_value_tcp_keepalive>`
+
+Set tcp keepalive.
+
+The tcp keepalive set in user config won't be taken into account.
+
+**default**: no keepalive set

g3proxy/doc/configuration/escapers/proxy_https.rst

@@ -0,0 +1,134 @@
+.. _configuration_escaper_proxy_https:
+
+proxy_https
+===========
+
+This escaper will access the target upstream through another https proxy.
+
+The following interfaces are supported:
+
+* tcp connect
+* http(s) forward
+
+There is no path selection support for this escaper.
+
+The following common keys are supported:
+
+* :ref:`shared_logger <conf_escaper_common_shared_logger>`
+* :ref:`resolver <conf_escaper_common_resolver>`, **required** only if *proxy_addr* is domain
+* :ref:`resolve_strategy <conf_escaper_common_resolve_strategy>`
+* :ref:`tcp_sock_speed_limit <conf_escaper_common_tcp_sock_speed_limit>`
+* :ref:`no_ipv4 <conf_escaper_common_no_ipv4>`
+* :ref:`no_ipv6 <conf_escaper_common_no_ipv6>`
+* :ref:`tcp_connect <conf_escaper_common_tcp_connect>`
+* :ref:`tcp_misc_opts <conf_escaper_common_tcp_misc_opts>`
+* :ref:`pass_proxy_userid <conf_escaper_common_pass_proxy_userid>`
+* :ref:`use_proxy_protocol <conf_escaper_common_use_proxy_protocol>`
+* :ref:`peer negotiation timeout <conf_escaper_common_peer_negotiation_timeout>`
+* :ref:`extra_metrics_tags <conf_escaper_common_extra_metrics_tags>`
+
+proxy_addr
+----------
+
+**required**, **type**: :ref:`upstream str <conf_value_upstream_str>` | seq
+
+Set the target proxy address. The default port is 3128 which can be omitted.
+
+For *seq* value, each of its element must be :ref:`weighted upstream addr <conf_value_weighted_upstream_addr>`.
+
+proxy_addr_pick_policy
+----------------------
+
+**optional**, **type**: :ref:`selective pick policy <conf_value_selective_pick_policy>`
+
+Set the policy to select next proxy address.
+
+The key for rendezvous/jump hash is *<client-ip>[-<username>]-<upstream-host>*.
+
+**default**: random
+
+tls_client
+----------
+
+**required**, **type**: :ref:`openssl tls client config <conf_value_openssl_tls_client_config>`
+
+Set TLS parameters for this local TLS client.
+If set to empty map, a default config is used.
+
+tls_name
+--------
+
+**optional**, **type**: :ref:`tls name <conf_value_tls_name>`
+
+Set the tls server name to verify tls certificate for all peers.
+
+If not set, the host part of each peer will be used.
+
+.. note:: IP address is not supported by now
+
+**default**: not set
+
+proxy_username
+--------------
+
+**optional**, **type**: :ref:`username <conf_value_username>`
+
+Set the proxy username. The Basic auth scheme is used by default.
+
+.. note::
+
+  Conflict with :ref:`pass_proxy_userid <conf_escaper_common_pass_proxy_userid>`
+
+proxy_password
+--------------
+
+**optional**, **type**: :ref:`password <conf_value_password>`
+
+Set the proxy password. Required if username is present.
+
+bind_ipv4
+---------
+
+**optional**, **type**: :ref:`ipv4 addr str <conf_value_ipv4_addr_str>`
+
+Set the bind ip address for inet sockets.
+
+**default**: not set
+
+bind_ipv6
+---------
+
+**optional**, **type**: :ref:`ipv6 addr str <conf_value_ipv6_addr_str>`
+
+Set the bind ip address for inet6 sockets.
+
+**default**: not set
+
+http_forward_capability
+-----------------------
+
+**optional**, **type**: :ref:`http forward capability <conf_value_http_forward_capability>`
+
+Set the http forward capability if the next proxy.
+
+**default**: all capability disabled
+
+http_connect_rsp_header_max_size
+--------------------------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the max header size for received CONNECT response.
+
+**default**: 4KiB
+
+tcp_keepalive
+-------------
+
+**optional**, **type**: :ref:`tcp keepalive <conf_value_tcp_keepalive>`
+
+Set tcp keepalive.
+
+The tcp keepalive set in user config won't be taken into account.
+
+**default**: no keepalive set

g3proxy/doc/configuration/escapers/proxy_socks5.rst

@@ -0,0 +1,93 @@
+.. _configuration_escaper_proxy_socks5:
+
+proxy_socks5
+============
+
+This escaper will access the target upstream through another http proxy.
+
+The following interfaces are supported:
+
+* tcp connect
+* udp_relay
+* udp_connect
+* http(s) forward
+
+There is no path selection support for this escaper.
+
+The following common keys are supported:
+
+* :ref:`shared_logger <conf_escaper_common_shared_logger>`
+* :ref:`resolver <conf_escaper_common_resolver>`, **required** only if *proxy_addr* is domain
+* :ref:`resolve_strategy <conf_escaper_common_resolve_strategy>`
+* :ref:`tcp_sock_speed_limit <conf_escaper_common_tcp_sock_speed_limit>`
+* :ref:`udp_sock_speed_limit <conf_escaper_common_udp_sock_speed_limit>`
+* :ref:`no_ipv4 <conf_escaper_common_no_ipv4>`
+* :ref:`no_ipv6 <conf_escaper_common_no_ipv6>`
+* :ref:`tcp_connect <conf_escaper_common_tcp_connect>`
+* :ref:`tcp_misc_opts <conf_escaper_common_tcp_misc_opts>`
+* :ref:`udp_misc_opts <conf_escaper_common_udp_misc_opts>`
+* :ref:`peer negotiation timeout <conf_escaper_common_peer_negotiation_timeout>`
+* :ref:`extra_metrics_tags <conf_escaper_common_extra_metrics_tags>`
+
+proxy_addr
+----------
+
+**required**, **type**: :ref:`upstream str <conf_value_upstream_str>` | seq
+
+Set the target proxy address. The default port is 1080 which can be omitted.
+
+For *seq* value, each of its element must be :ref:`weighted upstream addr <conf_value_weighted_upstream_addr>`.
+
+proxy_addr_pick_policy
+----------------------
+
+**optional**, **type**: :ref:`selective pick policy <conf_value_selective_pick_policy>`
+
+Set the policy to select next proxy address.
+
+The key for rendezvous/jump hash is *<client-ip>[-<username>]-<upstream-host>*.
+
+**default**: random
+
+proxy_username
+--------------
+
+**optional**, **type**: :ref:`username <conf_value_username>`
+
+Set the proxy username. The User auth scheme is used by default.
+
+proxy_password
+--------------
+
+**optional**, **type**: :ref:`password <conf_value_password>`
+
+Set the proxy password. Required if username is present.
+
+bind_ipv4
+---------
+
+**optional**, **type**: :ref:`ipv4 addr str <conf_value_ipv4_addr_str>`
+
+Set the bind ip address for inet sockets.
+
+**default**: not set
+
+bind_ipv6
+---------
+
+**optional**, **type**: :ref:`ipv6 addr str <conf_value_ipv6_addr_str>`
+
+Set the bind ip address for inet6 sockets.
+
+**default**: not set
+
+tcp_keepalive
+-------------
+
+**optional**, **type**: :ref:`tcp keepalive <conf_value_tcp_keepalive>`
+
+Set tcp keepalive.
+
+The tcp keepalive set in user config won't be taken into account.
+
+**default**: 60s

g3proxy/doc/configuration/escapers/route_client.rst

@@ -0,0 +1,60 @@
+.. _configuration_escaper_route_client:
+
+route_client
+============
+
+.. versionadded:: 1.1.3
+
+This escaper allows to select a next escaper based on rules on client address.
+
+There is no path selection support for this escaper.
+
+The following common keys are supported:
+
+* :ref:`default_next <conf_escaper_common_default_next>`
+
+exact_match
+-----------
+
+**optional**, **type**: seq
+
+If the client ip exactly match the one in the rules, that escaper will be selected.
+
+Each rule is in *map* format, with two keys:
+
+* next
+
+  **required**, **type**: str
+
+  Set the next escaper.
+
+* ips
+
+  **optional**, **type**: seq
+
+  Each element should be :ref:`ip addr str <conf_value_ip_addr_str>`.
+
+  An ip should not be set duplicated in rules for different next escapers.
+
+subnet_match
+------------
+
+**optional**, **type**: seq
+
+If the client ip match the longest subnet in the rule, that escaper will be selected.
+
+Each rule is in *map* format, with two keys:
+
+* next
+
+  **required**, **type**: str
+
+  Set the next escaper.
+
+* subnets
+
+  **optional**, **type**: seq
+
+  Each element should be :ref:`ip network str <conf_value_ip_network_str>`.
+
+  A subnet should not be set duplicated in rules for different next escapers.

g3proxy/doc/configuration/escapers/route_mapping.rst

@@ -0,0 +1,19 @@
+.. _configuration_escaper_route_mapping:
+
+route_mapping
+=============
+
+This escaper allows to select a next escaper based on the user specified path selection index.
+
+If no index can be get from the path selection method, the default random one will be used.
+
+No common keys are supported.
+
+next
+----
+
+**required**, **type**: seq
+
+This set all the next escapers. Each element should be the name of the target float escaper.
+
+.. note:: No duplication of next escapers is allowed.

g3proxy/doc/configuration/escapers/route_query.rst

@@ -0,0 +1,132 @@
+.. _configuration_escaper_route_query:
+
+route_query
+===========
+
+This escaper allows to select a next escaper based on query to another service through a UDP socket.
+
+There is no path selection support for this escaper.
+
+No common keys are supported.
+
+.. _configuration_escaper_route_query_fallback_node:
+
+fallback_node
+-------------
+
+**required**, **type**: string
+
+Set the fallback escaper name.
+
+query_allowed_next
+------------------
+
+**required**, **type**: seq
+
+Set all the next escapers those are allowed to use in the query result. Each element should be the next escaper name.
+If the selected escaper name is not found in this list, the fallback escaper will be used.
+
+.. _configuration_escaper_route_query_pass_client_ip:
+
+query_pass_client_ip
+--------------------
+
+**optional**, **type**: bool
+
+Set whether we should also send client_ip in the query message.
+
+**default**: false
+
+cache_request_batch_count
+-------------------------
+
+**optional**, **type**: usize
+
+Set how many consequent query requests we should handle in the cache runtime before yield out to the next loop.
+
+**default**: 10
+
+cache_request_timeout
+---------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set how many time we should spend on waiting responses from cache runtime after sending query request.
+
+The fallback node will be used if timeout occur.
+
+**default**: 100ms
+
+cache_pick_policy
+-----------------
+
+**optional**, **type**: :ref:`selective pick policy <conf_value_selective_pick_policy>`
+
+Set the policy to select next proxy address from the query result.
+
+The key for rendezvous/jump hash is *<client-ip>*.
+
+**default**: rendezvous
+
+query_peer_addr
+---------------
+
+**optional**, **type**: :ref:`sockaddr str <conf_value_sockaddr_str>`
+
+Set the socket address of the service that we should send queries to.
+
+**default**: 127.0.0.1:1053
+
+query_socket_buffer
+-------------------
+
+**optional**, **type**: :ref:`socket buffer config <conf_value_socket_buffer_config>`
+
+Set the socket buffer config for the UDP socket we will use.
+
+**default**: not set
+
+query_wait_timeout
+------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set how many time we should wait for response from the peer service.
+
+Empty reply will be send back to cache runtime if timeout occur.
+
+**default**: 10s
+
+.. _configuration_escaper_route_query_protective_cache_ttl:
+
+protective_cache_ttl
+--------------------
+
+**optional**, **type**: usize
+
+Set the cache ttl for failed or zero-ttl query results.
+
+**default**: 10
+
+maximum_cache_ttl
+-----------------
+
+**optional**, **type**: usize
+
+Set the maximum cache ttl for query results.
+
+**default**: 1800
+
+.. _configuration_escaper_route_query_vanish_after_expired:
+
+cache_vanish_wait
+-----------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Clean the record from the cache if it has been expired such many time.
+
+We still cache expired records some time before clean them as a new query will spend more time and the new query result
+will have a big chance to be the same with the expired one.
+
+**default**: 30s, **alias**: vanish_after_expire

g3proxy/doc/configuration/escapers/route_resolved.rst

@@ -0,0 +1,52 @@
+.. _configuration_escaper_route_resolved:
+
+route_resolved
+==============
+
+This escaper allows to select a next escaper based on rules on the resolved upstream ip address.
+
+There is no path selection support for this escaper.
+
+The resolve method in Happy Eyeballs algorithm is used.
+
+The following common keys are supported:
+
+* :ref:`resolver <conf_escaper_common_resolver>`, **required**
+* :ref:`resolve_strategy <conf_escaper_common_resolve_strategy>`
+* :ref:`default_next <conf_escaper_common_default_next>`
+
+lpm_match
+---------
+
+**optional**, **type**: seq
+
+If the resolved upstream ip address lpm match the network in the rules, that escaper will be selected.
+
+Each rule is in *map* format, with two keys:
+
+* next
+
+  **required**, **type**: str
+
+  Set the next escaper.
+
+* networks
+
+  **optional**, **type**: seq
+
+  Each element should be valid network string. Both IPv4 and IPv6 are supported.
+
+  Each network should not be set for different next escapers.
+
+resolution_delay
+----------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+The resolution delay time for the wait of the preferred address family after another one is returned.
+
+The meaning is the same as *resolution_delay* field in :ref:`happy eyeballs <conf_value_happy_eyeballs>`.
+
+**default**: 50ms
+
+.. versionadded:: 1.5.5

g3proxy/doc/configuration/escapers/route_select.rst

@@ -0,0 +1,30 @@
+.. _configuration_escaper_route_select:
+
+route_select
+============
+
+This escaper allows to select a next escaper based on the specified pick policy.
+
+There is no path selection support for this escaper.
+
+No common keys are supported.
+
+next_nodes
+----------
+
+**required**, **type**: string | seq
+
+Set the next escaper(s) those can be selected.
+
+For *seq* value, each of its element must be :ref:`weighted name str <conf_value_weighted_name_str>`.
+
+next_pick_policy
+----------------
+
+**optional**, **type**: :ref:`selective pick policy <conf_value_selective_pick_policy>`
+
+Set the policy to select next proxy address.
+
+The key for rendezvous/jump hash is *<client-ip>[-<username>]-<upstream-host>*.
+
+**default**: rendezvous

g3proxy/doc/configuration/escapers/route_upstream.rst

@@ -0,0 +1,105 @@
+.. _configuration_escaper_route_upstream:
+
+route_upstream
+==============
+
+This escaper allows to select a next escaper based on rules on upstream address.
+
+There is no path selection support for this escaper.
+
+The following common keys are supported:
+
+* :ref:`default_next <conf_escaper_common_default_next>`
+
+exact_match
+-----------
+
+**optional**, **type**: seq
+
+If the host part of upstream address exactly match the one in the rules, that escaper will be selected.
+
+Each rule is in *map* format, with two keys:
+
+* next
+
+  **required**, **type**: str
+
+  Set the next escaper.
+
+* hosts
+
+  **optional**, **type**: seq
+
+  Each element should be :ref:`host <conf_value_host>`.
+
+  A host should not be set duplicated in rules for different next escapers.
+
+subnet_match
+------------
+
+**optional**, **type**: seq
+
+If the host is an IP address and match the longest subnet in the rule, that escaper will be selected.
+
+Each rule is in *map* format, with two keys:
+
+* next
+
+  **required**, **type**: str
+
+  Set the next escaper.
+
+* subnets
+
+  **optional**, **type**: seq
+
+  Each element should be :ref:`ip network str <conf_value_ip_network_str>`.
+
+  A subnet should not be set duplicated in rules for different next escapers.
+
+child_match
+-----------
+
+**optional**, **type**: seq
+
+If the domain of the upstream address is children of domains in the rules, that escaper will be selected.
+
+Each rule is in *map* format, with two keys:
+
+* next
+
+  **required**, **type**: str
+
+  Set the next escaper.
+
+* domains
+
+  **optional**, **type**: seq
+
+  Each element should be :ref:`domain <conf_value_domain>`.
+
+  Each domain should not be set for different next escapers.
+
+radix_match
+-----------
+
+**optional**, **type**: seq
+
+If the domain of the upstream address exactly match the one of the domain suffixes in the rules,
+that escaper will be selected.
+
+Each rule is in *map* format, with two keys:
+
+* next
+
+  **required**, **type**: str
+
+  Set the next escaper.
+
+* suffixes
+
+  **optional**, **type**: seq
+
+  Each element should be :ref:`domain <conf_value_domain>`.
+
+  Each domain suffix should not be set for different next escapers.

g3proxy/doc/configuration/escapers/trick_float.rst

@@ -0,0 +1,17 @@
+.. _configuration_escaper_trick_float:
+
+trick_float
+===========
+
+This escaper allows to select a next float escaper weighted randomly.
+
+No common keys are supported.
+
+next
+----
+
+**required**, **type**: seq
+
+This set all the next escapers. Each element should be the name of the target float escaper.
+
+.. note:: Duplication of next escapers will be ignored.

g3proxy/doc/configuration/example.rst

@@ -0,0 +1,11 @@
+.. _configuration_example:
+
+*******
+Example
+*******
+
+Structure
+=========
+
+.. graphviz:: rd-relay_conf.dot
+

g3proxy/doc/configuration/index.rst

@@ -0,0 +1,58 @@
+.. _configuration:
+
+#############
+Configuration
+#############
+
+YAML is used as the configuration file format. The main conf file,
+which should be specified with the command line option *-c*,
+is make up of the following entries:
+
++-----------+----------+-------+------------------------------------------------+
+|Key        |Type      |Reload |Description                                     |
++===========+==========+=======+================================================+
+|group_name |Str       |no     |Process group name, default to be empty, can be |
+|           |          |       |overridden by the *-G* command line option.     |
++-----------+----------+-------+------------------------------------------------+
+|runtime    |Map       |no     |Runtime config, see :doc:`runtime`              |
++-----------+----------+-------+------------------------------------------------+
+|worker     |Map [#w]_ |no     |An unaided runtime will be started if present.  |
++-----------+----------+-------+------------------------------------------------+
+|log        |Map       |no     |Log config, see :doc:`log/index`                |
++-----------+----------+-------+------------------------------------------------+
+|stat       |Map       |no     |Stat config, see :doc:`stat`                    |
++-----------+----------+-------+------------------------------------------------+
+|controller |Seq       |no     |Controller config                               |
++-----------+----------+-------+------------------------------------------------+
+|resolver   |Mix [#m]_ |yes    |Resolver config, see :doc:`resolvers/index`     |
++-----------+----------+-------+------------------------------------------------+
+|escaper    |Mix [#m]_ |yes    |Escaper config, see :doc:`escapers/index`       |
++-----------+----------+-------+------------------------------------------------+
+|user_group |Mix [#m]_ |yes    |User group config, see :doc:`user_group/index`  |
++-----------+----------+-------+------------------------------------------------+
+|auditor    |Mix [#m]_ |yes    |Auditor config, see :doc:`auditors/index`       |
++-----------+----------+-------+------------------------------------------------+
+|server     |Mix [#m]_ |yes    |Server config, see :doc:`servers/index`         |
++-----------+----------+-------+------------------------------------------------+
+
+Example config: :doc:`example config for rd-relay service <example>`
+
+.. rubric:: Footnotes
+
+.. [#m] *Mix* is not a yaml type, see :ref:`hybrid map <conf_value_hybrid_map>` for the real format.
+.. [#w] See :ref:`unaided runtime config <conf_value_unaided_runtime_config>`.
+
+.. toctree::
+   :hidden:
+
+   values/index
+   runtime
+   log/index
+   stat
+   resolvers/index
+   escapers/index
+   auditors/index
+   user_group/index
+   servers/index
+   example
+

g3proxy/doc/configuration/log/driver/fluentd.rst

@@ -0,0 +1,140 @@
+.. _configuration_log_driver_fluentd:
+
+fluentd
+=======
+
+.. versionadded:: 1.5.0
+
+The fluentd driver config is is map format.
+
+We can set it to send logs to fluentd / fluent-bit by using it's `Forward Protocol`_.
+
+.. _Forward Protocol: https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1
+
+The tags in the fluentd event message will be g3proxy.Task / g3proxy.Escape / g3proxy.Resolve for the corresponding logs.
+
+The keys are described below.
+
+address
+-------
+
+**optional**, **type**: :ref:`sockaddr str <conf_value_sockaddr_str>`
+
+Set the tcp address of the fluentd server.
+
+**default**: 127.0.0.1:24224
+
+bind_ip
+-------
+
+**optional**, **type**: :ref:`ip addr str <conf_value_ip_addr_str>`
+
+Set the ip address to bind to for the local socket.
+
+**default**: not set
+
+shared_key
+----------
+
+**optional**, **type**: str
+
+Set the shared key if authentication is required.
+
+The handshake stage will be skipped if shared key is not set.
+
+**default**: not set
+
+username
+--------
+
+**optional**, **type**: str
+
+Set the username if authorization is required.
+
+This will only be used if authorization is required by the server.
+
+**default**: not set
+
+password
+--------
+
+**optional**, **type**: str
+
+Set the password if authorization is required.
+
+This will only be used if authorization is required by the server.
+
+**default**: not set
+
+hostname
+--------
+
+**optional**, **type**: str
+
+Set a custom hostname.
+
+**default**: local hostname
+
+tcp_keepalive
+-------------
+
+**optional**, **type**: :ref:`tcp keepalive <conf_value_tcp_keepalive>`
+
+Set the tcp keepalive config for the connection to fluentd server.
+
+**default**: enabled with system default values
+
+tls_client
+----------
+
+**optional**, **type**: :ref:`openssl tls client config <conf_value_openssl_tls_client_config>`
+
+Enable tls and set the config.
+
+**default**: not set
+
+connect_timeout
+---------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the timeout value for the connection to fluentd server, including tcp connect, tls handshake, fluentd handshake.
+
+**default**: 10s
+
+connect_delay
+-------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the delay time if the connect to fluentd server failed. All messages received will be dropped during this stage.
+
+**default**: 10s
+
+write_timeout
+-------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the write timeout for each message. The message will be dropped if timeout.
+
+default: 1s
+
+flush_interval
+--------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the flush interval for the connection to fluentd server.
+
+**default**: 100ms
+
+retry_queue_len
+---------------
+
+**optional**, **type**: usize
+
+Set how many events will be queued up to retry when connect or write failed.
+Note the write timeout events will be dropped directly.
+
+**default**: 10

g3proxy/doc/configuration/log/driver/index.rst

@@ -0,0 +1,20 @@
+.. _configuration_log_driver:
+
+**********
+Log Driver
+**********
+
+We support the following log drivers:
+
+* journald
+
+* syslog
+
+* fluentd
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Details:
+
+   syslog
+   fluentd

g3proxy/doc/configuration/log/driver/syslog.rst

@@ -0,0 +1,159 @@
+.. _configuration_log_driver_syslog:
+
+syslog
+======
+
+The syslog driver config is is map format.
+
+We can set it to send logs to syslogd, which can be listening on
+
+ * unix socket, which is default
+ * udp socket
+
+The message format can be
+
+ * rfc3164, which is default
+ * rfc5424
+
+The keys are described below.
+
+target_unix
+-----------
+
+**optional**, **type**: mix
+
+You can set this if you want to send syslog to a custom unix socket path.
+
+The value can be a map, with the following keys:
+
+* path
+
+  **required**, **type**: :ref:`absolute path <conf_value_absolute_path>`
+
+  The syslogd daemon listen socket path.
+
+If the value type is str, the value should be the same as the value as *path* above.
+
+**default**: not set
+
+target_udp
+----------
+
+**optional**, **type**: mix
+
+You can set this if you want to send syslog to a remote syslogd which listening on a udp socket.
+
+The value can be a map, with the following keys:
+
+* address
+
+  **required**, **type**: :ref:`sockaddr str <conf_value_sockaddr_str>`
+
+  Set the remote socket address.
+
+* bind_ip
+
+  **optional**, **type**: :ref:`ip addr str <conf_value_ip_addr_str>`
+
+  Set the ip address to bind to for the local socket.
+
+  **default**: not set
+
+If the value type is str, the value should be the same as the value as *address* above.
+
+**default**: not set
+
+target
+------
+
+**optional**, **type**: map
+
+This is just another form to set syslog target address.
+
+The key *udp* is just handled as *target_udp* as above.
+
+The key *unix* is just handled as *target_unix* as above.
+
+.. versionadded:: 1.3.5
+
+format_rfc5424
+--------------
+
+**optional**, **type**: mix
+
+Set this to use rfc5424 message format.
+
+The value can be a map, with the following keys:
+
+* enterprise_id
+
+  **optional**, **type**: int
+
+  Set the enterprise id value as described in `rfc5424`_.
+
+  See `PRIVATE ENTERPRISE NUMBERS`_ for IANA allocated numbers.
+
+  **default**: 0, which is reserved
+
+  .. _rfc5424: https://tools.ietf.org/html/rfc5424
+  .. _PRIVATE ENTERPRISE NUMBERS: https://www.iana.org/assignments/enterprise-numbers/enterprise-numbers
+
+* message_id
+
+  **optional**, **type**: str
+
+  Set the message id.
+
+  **default**: not set
+
+If the value type is int, the value should be the same as the value as *enterprise_id* above.
+If the value type is str, the value should be the same as the value as *message_id* above.
+
+**default**: not set
+
+use_cee_log_syntax
+------------------
+
+**optional**, **type**: bool
+
+Set if we should use `CEE Log Syntax`_.
+
+Enable this option if you need to use rsyslog `mmjsonparse`_ module.
+
+**default**: not set
+
+.. _mmjsonparse: https://www.rsyslog.com/files/temp/doc-indent/configuration/modules/mmjsonparse.html
+.. _CEE Log Syntax: https://cee.mitre.org/language/1.0-beta1/cls.html
+
+cee_event_flag
+--------------
+
+**optional**, **type**: ascii string
+
+Set a custom CEE event flag value. Only meaningful if *use_cee_log_syntax* is set.
+
+The one defined by `CLT`_ is *@cee:*, you can override it by using this option.
+
+**default**: @cee:
+
+.. _CLT: https://cee.mitre.org/language/1.0-beta1/clt.html
+
+emit_hostname
+-------------
+
+**optional**, **type**: bool
+
+Set if we should set hostname in the syslog message header.
+
+**default**: false
+
+.. versionadded:: 1.5.4
+
+append_report_ts
+----------------
+
+**optional**, **type**: bool
+
+Set if we should add :ref:`report_ts <log_shared_keys_report_ts>` to logs.
+
+**default**: false

g3proxy/doc/configuration/log/index.rst

@@ -0,0 +1,117 @@
+.. _configuration_log:
+
+***
+Log
+***
+
+This is the initial *log* config, which is optional and can not be reloaded.
+If set, it must reside in the main conf file.
+
+Root Value
+==========
+
+The value could be a simple string, which is the driver name, such as
+
+- discard
+
+  drop the logs. This is the **default**.
+
+- journal
+
+  send logs to journald directly.
+
+- syslog
+
+  send logs to syslogd directly.
+
+In such case, a default driver is used as default log config for all loggers.
+
+The value could be a map, with the following keys:
+
+- default
+
+  **optional**, **type**: :ref:`log config <configuration_log_config>`
+
+  Set default log config for loggers with no explicit config.
+
+  **default**: journal log config
+
+- task
+
+  **optional**, **type**: :ref:`log config <configuration_log_config>`
+
+  Set log config for *task* loggers.
+
+  **default**: not set
+
+- escape
+
+  **optional**, **type**: :ref:`log config <configuration_log_config>`
+
+  Set log config for *escape* loggers.
+
+  **default**: not set
+
+- resolve
+
+  **optional**, **type**: :ref:`log config <configuration_log_config>`
+
+  Set log config for *resolve* loggers.
+
+  **default**: not set
+
+.. _configuration_log_config:
+
+log config
+==========
+
+The detailed log config may be a simple driver name, or a map, with the following keys:
+
+- journal
+
+  **optional**, **type**: map
+
+  Use *journal* log driver. An empty map should be used, as no keys are defined by now.
+
+- syslog
+
+  **optional**, **type**: :ref:`syslog <configuration_log_driver_syslog>`
+
+  Use *syslog* log driver.
+
+- async_channel_size
+
+  **optional**, **type**: usize
+
+  Set the internal async channel size.
+
+  **default**: 4096
+
+- async_thread_number
+
+  **optional**, **type**: usize
+
+  Set the number of async threads.
+
+  This has no effect on *discard* and *journal* log driver.
+
+  **default**: 1
+
+- io_error_sampling_offset
+
+  **optional**, **type**: usize, **max**: 16
+
+  The logger may encounter io error, we should report it anyhow. We will log this error every *2^n* times,
+  where *n* can be set here.
+
+  This has no effect on *discard* and *journal* log driver.
+
+  **default**: 10
+
+.. note:: The *discard* driver has no config options, so it doesn't has a corresponding map field.
+
+.. toctree::
+   :maxdepth: 2
+   :caption: More:
+
+   driver/index

g3proxy/doc/configuration/rd-relay_conf.dot

@@ -0,0 +1,52 @@
+
+strict digraph G {
+  user [style=filled];
+  subgraph cluster_resolver {
+    graph [style=dashed][label=resolver];
+    r_intranet [label=intranet];
+    r_unicom [label=unicom];
+  };
+  subgraph cluster_escaper {
+    graph [label=escaper];
+    node [style=dotted];
+    e_rt_domain [label="rt-domain"];
+    e_rt_dstip [label="rt-dstip"];
+    e_intranet [label=intranet];
+    e_unicom [label=unicom];
+    e_alisg [label=alisg];
+    e_deny [label=deny];
+    e_rt_domain -> e_rt_dstip [label=default];
+    e_rt_domain -> e_deny;
+    e_rt_domain -> e_intranet;
+    e_rt_domain -> e_unicom;
+    e_rt_domain -> e_alisg;
+    e_rt_dstip -> e_unicom [label=default];
+    e_rt_dstip -> e_deny;
+    e_rt_dstip -> e_intranet;
+  };
+  subgraph cluster_server {
+    graph [label=server];
+    s_http [label=http];
+    s_socks [label=socks];
+  }
+  subgraph cluster_path {
+    graph [label=path];
+    node [style=filled];
+    intranet;
+    unicom;
+    alisg;
+  }
+  e_rt_dstip -> r_intranet [style=dashed][dir=both];
+  e_intranet -> r_intranet [style=dashed][dir=both];
+  e_unicom -> r_unicom [style=dashed][dir=both];
+  s_http -> e_rt_domain;
+  s_socks -> e_rt_domain;
+  user -> s_http;
+  user -> s_socks;
+  e_intranet -> intranet;
+  e_unicom -> unicom;
+  e_alisg -> alisg;
+  r_intranet -> intranet [dir=both];
+  r_unicom -> unicom [dir=both];
+}
+

g3proxy/doc/configuration/resolvers/c_ares.rst

@@ -0,0 +1,100 @@
+.. _configuration_resolver_c_ares:
+
+c_ares
+======
+
+This is the resolver based on c-ares library.
+
+server
+------
+
+**required**, **type**: str | seq
+
+Set the nameservers.
+
+For *str* value, it may be one or more :ref:`sockaddr str <conf_value_sockaddr_str>` joined with whitespace characters.
+
+For *seq* value, each of its value should be :ref:`sockaddr str <conf_value_sockaddr_str>`.
+
+The default port *53* will be used, if not port is specified in the value string.
+
+Servers in different address families can be set in together.
+
+each_timeout
+------------
+
+**optional**, **type**: int, **unit**: ms
+
+The number of milliseconds each name server is given to respond to a query on the first try.
+After the first try, the timeout algorithm becomes more complicated, but scales linearly with the value of timeout.
+
+**default**: 5000
+
+each_tries
+----------
+
+**optional**, **type**: int
+
+The number of tries the resolver will try contacting each name server before giving up.
+
+**default**: 2
+
+round_robin
+-----------
+
+**optional**, **type**: bool
+
+If true, perform round-robin selection of the nameservers configured for the channel for each resolution.
+
+**default**: false
+
+socket_send_buffer_size
+-----------------------
+
+**optional**, **type**: u32
+
+Set the send buffer size for the socket.
+
+**default**: not set, which should be the value of /proc/sys/net/core/wmem_default
+
+socket_recv_buffer_size
+-----------------------
+
+**optional**, **type**: u32
+
+Set the recv buffer size for the socket.
+
+**default**: not set, which should be the value of /proc/sys/net/core/rmem_default
+
+bind_ipv4
+---------
+
+**optional**, **type**: :ref:`ipv4 addr str <conf_value_ipv4_addr_str>`
+
+Set the IPv4 bind ip for the resolver while setting up sockets.
+
+bind_ipv6
+---------
+
+**optional**, **type**: :ref:`ipv6 addr str <conf_value_ipv6_addr_str>`
+
+Set the IPv6 bind ip for the resolver while setting up sockets.
+
+negative_ttl
+------------
+
+**optional**, **type**: u32
+
+Time-to-Live (TTL) for negative caching of failed DNS lookups.
+This also sets the lower cache limit on positive lookups.
+
+**default**: 30
+
+positive_ttl
+------------
+
+**optional**, **type**: u32
+
+Upper limit on how long we will cache positive DNS responses. It should long than *negative_ttl*.
+
+**default**: 3600

g3proxy/doc/configuration/resolvers/deny_all.rst

@@ -0,0 +1,8 @@
+.. _configuration_resolver_deny_all:
+
+deny_all
+========
+
+This is the dummy resolver designed to drop all requests.
+
+There are no extra config keys for this kind of resolver.

g3proxy/doc/configuration/resolvers/fail_over.rst

@@ -0,0 +1,44 @@
+.. _configuration_resolver_fail_over:
+
+fail_over
+=========
+
+This is a virtual resolver designed to fail over between (real) resolvers.
+
+Rules for result selection:
+
+1. The **success** result of the primary resolver will always be used before the timeout.
+2. The first **success** result either from the primary or the standby resolver will be used after the timeout.
+3. If no success result, the last error one will be used.
+
+primary
+-------
+
+**required**, **type**: string
+
+Set the primary resolver to use.
+
+standby
+-------
+
+**required**, **type**: string
+
+Set the standby resolver to use.
+
+timeout
+-------
+
+**optional**, **type**:
+
+Set the timeout for primary lookup.
+
+**default**: 100ms
+
+negative_ttl
+------------
+
+**optional**, **type**: u32
+
+Time-to-Live (TTL) for negative caching of failed DNS lookups.
+
+**default**: 30

g3proxy/doc/configuration/resolvers/index.rst

@@ -0,0 +1,53 @@
+.. _configuration_resolver:
+
+********
+Resolver
+********
+
+The type for each resolver config is *map*, with two always required keys:
+
+* *name*, which specify the name of the resolver.
+* *type*, which specify the real type of the resolver, decides how to parse other keys.
+
+There are many types of resolver, each with a section below.
+
+Resolvers
+=========
+
+.. toctree::
+   :maxdepth: 2
+
+   deny_all
+   fail_over
+   c_ares
+   trust_dns
+
+Common Keys
+===========
+
+This section describes the common keys, they may be used by many resolvers.
+
+Most of them are the runtime (of the standalone resolver thread) config.
+
+graceful_stop_wait
+------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the wait duration before really shutdown the resolver thread.
+
+There may be queries running inside the resolver,
+we don't wait all of them to finish but instead wait for a fixed time interval.
+
+**default**: 30s
+
+protective_query_timeout
+------------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the query timeout value for queries sent to driver.
+
+The value should be larger than the value set in the driver specific timeout config.
+
+**default**: 60s

g3proxy/doc/configuration/resolvers/trust_dns.rst

@@ -0,0 +1,98 @@
+.. _configuration_resolver_trust_dns:
+
+trust_dns
+=========
+
+.. versionadded:: 1.1.4
+
+This is the resolver based on trust-dns library.
+
+server
+------
+
+**required**, **type**: str | seq
+
+Set the nameservers.
+
+For *str* value, it may be one or more :ref:`ip addr str <conf_value_ip_addr_str>` joined with whitespace characters.
+
+For *seq* value, each of its value should be :ref:`ip addr str <conf_value_ip_addr_str>`.
+
+server_port
+-----------
+
+**optional**, **type**: u16
+
+Set the port if the default port is not usable.
+
+**default**: 53 for udp and tcp, 853 for dns-over-tls, 443 for dns-over-https
+
+encryption
+----------
+
+**optional**, **type**: :ref:`dns encryption config <conf_value_dns_encryption_config>`
+
+Set the encryption config.
+
+**default**: not set
+
+each_timeout
+------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Specify the timeout for a request.
+
+**default**: 5s
+
+retry_attempts
+--------------
+
+**optional**, **type**: usize
+
+Number of retries after lookup failure before giving up.
+
+**default**: 2
+
+bind_ip
+-------
+
+**optional**, **type**: :ref:`ip addr str <conf_value_ip_addr_str>`
+
+Set the bind ip for the resolver while setting up sockets.
+
+positive_min_ttl
+----------------
+
+**optional**, **type**: u32
+
+Minimum TTL for positive responses.
+
+**default**: 30
+
+positive_max_ttl
+----------------
+
+**optional**, **type**: u32
+
+Maximum TTL for positive responses.
+
+**default**: 3600
+
+negative_min_ttl
+----------------
+
+**optional**, **type**: u32
+
+Minimum TTL for negative responses.
+
+**default**: 30
+
+negative_max_ttl
+----------------
+
+**optional**, **type**: u32
+
+Maximum TTL for negative responses.
+
+**default**: 3600

g3proxy/doc/configuration/runtime.rst

@@ -0,0 +1,105 @@
+.. _configuration_runtime:
+
+*******
+Runtime
+*******
+
+This is the *runtime* config, which is optional. If set, it must reside in the main conf file.
+
+All the options in this config are optional with a reasonable default value.
+Set them only if you really known their meaning.
+
+The options can be grouped into the following sections:
+
+tokio main runtime
+==================
+
+This section describes the options for the main tokio runtime, which is used for all servers.
+
+thread_number
+-------------
+
+**optional**, **type**: int | str
+
+Set the scheduler and core number of worker threads.
+
+if *0*, a basic scheduler is used.
+if not *0*, a threaded scheduler with the specified number of worker thread is used.
+
+**default**: threaded scheduler with worker threads on each all available CPU core.
+
+thread_name
+-----------
+
+**optional**, **type**: str
+
+Set name of worker threads spawned. Only ASCII characters is allowed.
+Note that the length of thread name will be restricted at the OS level.
+
+**default**: "tokio"
+
+thread_stack_size
+-----------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the stack size for worker threads. For *<int>* value, the unit is bytes.
+
+**default**: `tokio thread_stack_size`_
+
+.. _tokio thread_stack_size: https://docs.rs/tokio/0.2.21/tokio/runtime/struct.Builder.html#method.thread_stack_size
+
+max_io_events_per_tick
+----------------------
+
+**optional**, **type**: usize
+
+Configures the max number of events to be processed per tick.
+
+**default**: 1024, tokio default value
+
+.. versionadded: 1.7.6
+
+daemon quit control
+===================
+
+This section describes the options used during graceful quit of the daemon.
+
+server_offline_delay
+--------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the time duration before offline all servers after received daemon quit signals.
+All listen server sockets will be closed after this duration, so it should be more than the time used to
+start the new daemon process if you depends on it for graceful restart.
+
+**default**: 4s
+
+task_wait_delay
+---------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the time duration before checking alive tasks after all servers going into offline mode.
+Tasks are marked as alive only if auth success, so we should leave some time for those tasks in negotiation
+state to run into their next state, which may be alive or really dead.
+
+**default**: 2s
+
+task_wait_timeout
+-----------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the time duration before force quit alive tasks after we decide to wait for them to end gracefully.
+
+**default**: 10h
+
+task_quit_timeout
+-----------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the time duration before we shutdown the process after entering force quit status for all tasks.
+The tasks dropped after this timeout won't have any logs.

g3proxy/doc/configuration/servers/dummy_close.rst

@@ -0,0 +1,8 @@
+.. _configuration_server_dummy_close:
+
+dummy_close
+===========
+
+This is the dummy server designed to close all connections.
+
+There are no extra config keys for this kind of server.

g3proxy/doc/configuration/servers/http_proxy.rst

@@ -0,0 +1,250 @@
+.. _configuration_server_http_proxy:
+
+http_proxy
+==========
+
+This server provides http proxy, including http forward and http connect.
+
+The following common keys are supported:
+
+* :ref:`escaper <conf_server_common_escaper>`
+* :ref:`auditor <conf_server_common_auditor>`
+* :ref:`user_group <conf_server_common_user_group>`
+* :ref:`shared_logger <conf_server_common_shared_logger>`
+* :ref:`listen <conf_server_common_listen>`
+* :ref:`listen_in_worker <conf_server_common_listen_in_worker>`
+* :ref:`tls_server <conf_server_common_tls_server>`
+* :ref:`tcp_sock_speed_limit <conf_server_common_tcp_sock_speed_limit>`
+* :ref:`ingress_network_filter <conf_server_common_ingress_network_filter>`
+* :ref:`dst_host_filter_set <conf_server_common_dst_host_filter_set>`
+* :ref:`dst_port_filter <conf_server_common_dst_port_filter>`
+* :ref:`tcp_copy_buffer_size <conf_server_common_tcp_copy_buffer_size>`
+* :ref:`tcp_copy_yield_size <conf_server_common_tcp_copy_yield_size>`
+* :ref:`tcp_misc_opts <conf_server_common_tcp_misc_opts>`
+* :ref:`task_idle_check_duration <conf_server_common_task_idle_check_duration>`
+* :ref:`task_idle_max_count <conf_server_common_task_idle_max_count>`
+* :ref:`extra_metrics_tags <conf_server_common_extra_metrics_tags>`
+
+The auth scheme supported by the server is determined by the type of the specified user group.
+
++-------------+---------------------------+-------------------+
+|auth scheme  |user group type            |is supported       |
++=============+===========================+===================+
+|Basic        |hashed_user                |yes                |
++-------------+---------------------------+-------------------+
+|Negotiate    |gss_api                    |not yet            |
++-------------+---------------------------+-------------------+
+
+.. _config_server_http_proxy_server_id:
+
+server_id
+---------
+
+**optional**, **type**: :ref:`http server id <conf_value_http_server_id>`
+
+Set the server id. If set, the header *X-BD-Remote-Connection-Info* will be added to response.
+
+**default**: not set
+
+auth_realm
+----------
+
+**optional**, **type**: :ref:`ascii str <conf_value_ascii_str>`
+
+Set the auth realm.
+
+**default**: proxy
+
+tls_client
+----------
+
+**optional**, **type**: :ref:`openssl tls client config <conf_value_openssl_tls_client_config>`
+
+Set TLS client parameters for https forward requests.
+
+**default**: set with default value
+
+ftp_client
+----------
+
+**optional**, **type**: :ref:`ftp client config <conf_value_ftp_client_config>`
+
+Set the ftp client config for FTP over Http requests.
+
+**default**: set with default value
+
+req_header_recv_timeout
+-----------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the max time to wait a full request header after the client connection become readable.
+
+**default**: 30s
+
+rsp_header_recv_timeout
+-----------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the max time duration after the full request sent and before receive of the whole response header.
+
+**default**: 60s
+
+req_header_max_size
+-------------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the max request header size.
+
+**default**: 64KiB
+
+rsp_header_max_size
+-------------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the max response header size.
+
+**default**: 64KiB
+
+.. _config_server_http_proxy_log_uri_max_chars:
+
+log_uri_max_chars
+-----------------
+
+**optional**, **type**: usize
+
+Set the max number of characters of uri should be logged in logs.
+
+The user level config value will take effect if set, see this :ref:`user config option <config_user_log_uri_max_chars>`.
+
+**default**: 1024
+
+pipeline_size
+-------------
+
+**optional**, **type**: int
+
+Set the pipeline size for HTTP 1.0/1.1.
+
+**default**: 10
+
+.. note::
+
+  We only pipeline requests with no body.
+
+pipeline_read_idle_timeout
+--------------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the idle timeout of the client side IDLE http connections.
+
+**default**: 5min
+
+no_early_error_reply
+--------------------
+
+**optional**, **type**: bool
+
+Set to true if no error reply should be sent out before user auth succeeded, the connection will be just closed
+in such case.
+
+**default**: false
+
+allow_custom_host
+-----------------
+
+**optional**, **type**: bool
+
+Set if custom *Host* header is allowed. If set to false, the *Host* header in http headers should have the same domain
+or ip address with the one in the request method line.
+
+**default**: true
+
+.. note:: we don't require the *Host* header to be present in http headers no matter what have been set for this
+
+body_line_max_length
+--------------------
+
+**optional**, **type**: int
+
+Set the max line length for lines (trailer and chunk size) in http body.
+
+**default**: 8192
+
+http_forward_upstream_keepalive
+-------------------------------
+
+**optional**, **type**: :ref:`http keepalive <conf_value_http_keepalive>`
+
+Set http keepalive config at server level.
+
+**default**: set with default value
+
+.. _config_server_http_proxy_http_forward_mark_upstream:
+
+http_forward_mark_upstream
+--------------------------
+
+**optional**, **type**: bool
+
+If set, the header *X-BD-Upstream-Id* header will be added to the response from upstream, with the value to be
+:ref:`server_id <config_server_http_proxy_server_id>`.
+Local generated response will not contains this header.
+
+**default**: false
+
+.. _config_server_http_proxy_echo_chained_info:
+
+echo_chained_info
+-----------------
+
+**optional**, **type**: bool
+
+Set whether to add custom header in response that provides chained information
+about the direct connection to upstream.
+
+The custom headers are:
+
+- X-BD-Upstream-Addr
+- X-BD-Outgoing-IP
+
+**default**: false
+
+untrusted_read_speed_limit
+--------------------------
+
+**optional**, **type**: :ref:`tcp socket speed limit <conf_value_tcp_sock_speed_limit>`
+
+Enable untrusted read of the body of requests with no auth info, and set the read rate limit.
+
+Set this if you need to be compatible with buggy java http clients which won't handle the 407 error response in time.
+
+**default**: not set, which means untrusted read is disabled, **alias**: untrusted_read_limit
+
+.. versionchanged:: 1.4.0 changed name to untrusted_read_speed_limit
+
+.. _config_server_http_proxy_egress_path_selection_header:
+
+egress_path_selection_header
+----------------------------
+
+**optional**, **type**: str, **alias**: path_selection_header
+
+Set the http custom header name to be used for path selection.
+
+**default**: not set
+
+.. _config_server_http_proxy_steal_forwarded_for:
+
+steal_forwarded_for
+-------------------
+
+**optional**, **type**: bool
+
+Set if we should delete the *Forwarded* and *X-Forwarded-For* headers from the client's request.
+
+**default**: false

g3proxy/doc/configuration/servers/http_rproxy.rst

@@ -0,0 +1,302 @@
+.. _configuration_server_http_rproxy:
+
+http_rproxy
+===========
+
+This server provides http reverse proxy.
+
+The following common keys are supported:
+
+* :ref:`escaper <conf_server_common_escaper>`
+* :ref:`auditor <conf_server_common_auditor>`
+* :ref:`user_group <conf_server_common_user_group>`
+* :ref:`shared_logger <conf_server_common_shared_logger>`
+* :ref:`listen <conf_server_common_listen>`
+* :ref:`listen_in_worker <conf_server_common_listen_in_worker>`
+* :ref:`tcp_sock_speed_limit <conf_server_common_tcp_sock_speed_limit>`
+* :ref:`ingress_network_filter <conf_server_common_ingress_network_filter>`
+* :ref:`tcp_copy_buffer_size <conf_server_common_tcp_copy_buffer_size>`
+* :ref:`tcp_copy_yield_size <conf_server_common_tcp_copy_yield_size>`
+* :ref:`tcp_misc_opts <conf_server_common_tcp_misc_opts>`
+* :ref:`task_idle_check_duration <conf_server_common_task_idle_check_duration>`
+* :ref:`task_idle_max_count <conf_server_common_task_idle_max_count>`
+* :ref:`extra_metrics_tags <conf_server_common_extra_metrics_tags>`
+
+The auth scheme supported by the server is determined by the type of the specified user group.
+
++-------------+---------------------------+-------------------+
+|auth scheme  |user group type            |is supported       |
++=============+===========================+===================+
+|Basic        |hashed_user                |yes                |
++-------------+---------------------------+-------------------+
+|Negotiate    |gss_api                    |not yet            |
++-------------+---------------------------+-------------------+
+
+.. _config_server_http_rproxy_server_id:
+
+server_id
+---------
+
+**optional**, **type**: :ref:`http server id <conf_value_http_server_id>`
+
+Set the server id. If set, the header *X-BD-Remote-Connection-Info* will be added to response.
+
+**default**: not set
+
+auth_realm
+----------
+
+**optional**, **type**: :ref:`ascii str <conf_value_ascii_str>`
+
+Set the auth realm.
+
+**default**: proxy
+
+req_header_recv_timeout
+-----------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the max time to wait a full request header after the client connection become readable.
+
+**default**: 30s
+
+rsp_header_recv_timeout
+-----------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the max time duration after the full request sent and before receive of the whole response header.
+
+**default**: 60s
+
+req_header_max_size
+-------------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the max request header size.
+
+**default**: 64KiB
+
+rsp_header_max_size
+-------------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the max response header size.
+
+**default**: 64KiB
+
+.. _config_server_http_rproxy_log_uri_max_chars:
+
+log_uri_max_chars
+-----------------
+
+**optional**, **type**: usize
+
+Set the max number of characters of uri should be logged in logs.
+
+The user level config value will take effect if set, see this :ref:`user config option <config_user_log_uri_max_chars>`.
+
+**default**: 1024
+
+pipeline_size
+-------------
+
+**optional**, **type**: int
+
+Set the pipeline size for HTTP 1.0/1.1.
+
+**default**: 10
+
+.. note::
+
+  We only pipeline requests with no body.
+
+pipeline_read_idle_timeout
+--------------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the idle timeout of the client side IDLE http connections.
+
+**default**: 5min
+
+no_early_error_reply
+--------------------
+
+**optional**, **type**: bool
+
+Set to true if no error reply should be sent out before user auth succeeded, the connection will be just closed
+in such case.
+
+**default**: false
+
+body_line_max_length
+--------------------
+
+**optional**, **type**: int
+
+Set the max line length for lines (trailer and chunk size) in http body.
+
+**default**: 8192
+
+http_forward_upstream_keepalive
+-------------------------------
+
+**optional**, **type**: :ref:`http keepalive <conf_value_http_keepalive>`
+
+Set http keepalive config at server level.
+
+**default**: set with default value
+
+untrusted_read_speed_limit
+--------------------------
+
+**optional**, **type**: :ref:`tcp socket speed limit <conf_value_tcp_sock_speed_limit>`
+
+Enable untrusted read of the body of requests with no auth info, and set the read rate limit.
+
+Set this if you need to be compatible with buggy java http clients which won't handle the 407 error response in time.
+
+**default**: not set, which means untrusted read is disabled, **alias**: untrusted_read_limit
+
+.. versionchanged:: 1.4.0 changed name to untrusted_read_speed_limit
+
+append_forwarded_for
+--------------------
+
+**optional**, **type**: :ref:`http forwarded header type <conf_value_http_forwarded_header_type>`
+
+Set if we should append a corresponding forwarded header to the request send out to the next proxy.
+
+See :ref:`steal_forwarded_for <config_server_http_proxy_steal_forwarded_for>` config option in http_proxy for more info
+if you want to delete existing forwarded headers.
+
+See the doc of supported escapers for detailed protocol info.
+
+**default**: classic, which means *X-Forwarded-\** headers will be appended
+
+hosts
+-----
+
+**required**, **type**: :ref:`host matched object <conf_value_host_matched_object>` <:ref:`host <configuration_server_http_rproxy_host>`>
+
+Set the hosts we should handle based on host match rules.
+
+Example 1:
+
+.. code-block:: yaml
+
+  hosts:
+    services:
+      upstream: www.example.net
+
+Example 2:
+
+.. code-block:: yaml
+
+  hosts:
+    - exact_match:
+        - www.example.net
+        - example.net
+      services:
+        upstream: www.example.net
+    - child_match: example.org
+      set_default: true
+      services:
+        upstream: www.example.org
+
+**default**: not set
+
+.. _configuration_server_http_rproxy_host:
+
+Host
+^^^^
+
+This is the config for each local host on this server.
+
+services
+""""""""
+
+**required**, **type**: :ref:`uri path matched object <conf_value_uri_path_matched_object>` <:ref:`service <configuration_server_http_rproxy_service>`>
+
+Set the sites we should handle based on url path match rules.
+
+tls_server
+""""""""""
+
+**optional**, **type**: :ref:`rustls server config <conf_value_rustls_server_config>`
+
+Set TLS server config for this local site.
+
+If not set, the :ref:`global tls server <configuration_server_http_rproxy_global_tls_server>` config will be used.
+
+**default**: not set
+
+.. _configuration_server_http_rproxy_service:
+
+Service
+^^^^^^^
+
+This set the config for a upstream http service.
+
+upstream
+""""""""
+
+**required**, **type**: :ref:`upstream str <conf_value_upstream_str>`
+
+Set the target upstream address. The default port is 80 which can be omitted.
+
+tls_client
+""""""""""
+
+**optional**, **type**: :ref:`openssl tls client config <conf_value_openssl_tls_client_config>`
+
+Set TLS parameters for this local TLS client if https is needed.
+If set to empty map, a default config is used.
+
+**default**: not set
+
+tls_name
+""""""""
+
+**optional**, **type**: :ref:`tls name <conf_value_tls_name>`
+
+Set the tls server name to verify tls certificate of the upstream site.
+
+If not set, the host part of the upstream address will be used.
+
+**default**: not set
+
+enable_tls_server
+-----------------
+
+**optional**, **type**: bool
+
+Set whether tls is enabled for all local sites.
+
+Requests to local sites without valid tls server config will be dropped.
+
+**default**: false
+
+.. _configuration_server_http_rproxy_global_tls_server:
+
+global_tls_server
+-----------------
+
+**optional**, **type**: :ref:`rustls server config <conf_value_rustls_server_config>`
+
+Set global TLS server config on the server. This will be used if no tls server config set on the matched local site.
+
+**default**: not set
+
+client_hello_recv_timeout
+-------------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the timeout value for the receive of the complete TLS ClientHello message.
+
+**default**: 1s

g3proxy/doc/configuration/servers/index.rst

@@ -0,0 +1,281 @@
+.. _configuration_server:
+
+******
+Server
+******
+
+The type for each server config is *map*, with two always required keys:
+
+* *name*, which specify the name of the escaper.
+* *type*, which specify the real type of the escaper, decides how to parse other keys.
+
+There are many types of server, each with a section below.
+
+Servers
+=======
+
+.. toctree::
+   :maxdepth: 2
+
+   dummy_close
+   tcp_stream
+   tls_stream
+   http_proxy
+   socks_proxy
+   http_rproxy
+   sni_proxy
+   plain_tcp_port
+   plain_tls_port
+   intelli_proxy
+
+Common Keys
+===========
+
+This section describes the common keys, they may be used by many escapers.
+
+.. _conf_server_common_escaper:
+
+escaper
+-------
+
+**required**, **type**: str
+
+Set the escaper to use with this server.
+
+If the specified escaper doesn't exist in configure, a default DummyDeny escaper will be used.
+
+.. _conf_server_common_auditor:
+
+auditor
+-------
+
+**optional**, **type**: str
+
+Set the auditor to use with this server.
+
+If the specified auditor doesn't exist in configure, a default auditor will be used.
+
+.. versionadded:: 1.7.0
+
+.. _conf_server_common_user_group:
+
+user_group
+----------
+
+**optional**, **type**: str
+
+Set the user group for auth.
+
+If the specified user group doesn't exist in configure, a default user group with no users will be used.
+
+**default**: no auth enabled
+
+.. _conf_server_common_shared_logger:
+
+shared_logger
+-------------
+
+**optional**, **type**: ascii
+
+Set the server to use a logger running on a shared thread.
+
+**default**: not set
+
+.. _conf_server_common_listen:
+
+listen
+------
+
+**required**, **type**: :ref:`tcp listen <conf_value_tcp_listen>`
+
+Set the listen config for this server.
+
+The instance count setting will be ignored if *listen_in_worker* is correctly enabled.
+
+.. _conf_server_common_listen_in_worker:
+
+listen_in_worker
+----------------
+
+**optional**, **type**: bool
+
+Set if we should listen in each worker runtime if you have worker enabled.
+
+The listen instance count will be the same with the worker number count.
+
+**default**: false
+
+.. versionadded:: 1.7.8
+
+.. _conf_server_common_tls_server:
+
+tls_server
+----------
+
+**optional**, **type**: :ref:`rustls server config <conf_value_rustls_server_config>`
+
+Enable TLS on the listening socket and set TLS parameters.
+
+**default**: disabled
+
+.. _conf_server_common_ingress_network_filter:
+
+ingress_network_filter
+----------------------
+
+**optional**, **type**: :ref:`ingress network acl rule <conf_value_ingress_network_acl_rule>`
+
+Set the network filter for clients.
+
+**default**: not set
+
+.. _conf_server_common_dst_host_filter_set:
+
+dst_host_filter_set
+-------------------
+
+**optional**, **type**: :ref:`dst host acl rule set <conf_value_dst_host_acl_rule_set>`
+
+Set the filter for dst host of each request.
+
+.. note:: This won't limit the Host header in http protocol.
+
+**default**: not set
+
+.. _conf_server_common_dst_port_filter:
+
+dst_port_filter
+---------------
+
+**optional**, **type**: :ref:`exact port acl rule <conf_value_exact_port_acl_rule>`
+
+Set the filter for dst port of each request.
+
+**default**: not set
+
+.. _conf_server_common_tcp_sock_speed_limit:
+
+tcp_sock_speed_limit
+--------------------
+
+**optional**, **type**: :ref:`tcp socket speed limit <conf_value_tcp_sock_speed_limit>`
+
+Set speed limit for each tcp socket.
+
+**default**: no limit, **alias**: tcp_conn_speed_limit | tcp_conn_limit
+
+.. versionchanged:: 1.4.0 changed name to tcp_sock_speed_limit
+
+.. _conf_server_common_udp_sock_speed_limit:
+
+udp_sock_speed_limit
+--------------------
+
+**optional**, **type**: :ref:`udp socket speed limit <conf_value_udp_sock_speed_limit>`
+
+Set speed limit for each udp socket.
+
+**default**: no limit, **alias**: udp_relay_speed_limit | udp_relay_limit
+
+.. versionchanged:: 1.4.0 changed name to udp_sock_speed_limit
+
+.. _conf_server_common_tcp_copy_buffer_size:
+
+tcp_copy_buffer_size
+--------------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the buffer size for internal tcp copy.
+
+**default**: 16K, **minimal**: 4K
+
+.. _conf_server_common_tcp_copy_yield_size:
+
+tcp_copy_yield_size
+-------------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the yield out size for the internal copy task.
+
+**default**: 1M, **minimal**: 256K
+
+.. _conf_server_common_udp_relay_packet_size:
+
+udp_relay_packet_size
+---------------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the udp packet size for udp relay.
+
+**default**: 4K, **maximum**: 16K
+
+.. _conf_server_common_udp_relay_yield_size:
+
+udp_relay_yield_size
+--------------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the yield out size for the internal relay task.
+
+**default**: 1M, **maximum**: 256K
+
+.. _conf_server_common_tcp_misc_opts:
+
+tcp_misc_opts
+-------------
+
+**optional**, **type**: :ref:`tcp misc sock opts <conf_value_tcp_misc_sock_opts>`
+
+Set misc tcp socket options on accepted tcp sockets.
+
+**default**: not set, nodelay is default enabled
+
+.. _conf_server_common_udp_misc_opts:
+
+udp_misc_opts
+-------------
+
+**optional**, **type**: :ref:`udp misc sock opts <conf_value_udp_misc_sock_opts>`
+
+Set misc udp socket options on created udp sockets.
+
+**default**: not set
+
+.. _conf_server_common_task_idle_check_duration:
+
+task_idle_check_duration
+------------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the idle check duration for task.
+
+**default**: 5min, **max**: 30min
+
+.. _conf_server_common_task_idle_max_count:
+
+task_idle_max_count
+-------------------
+
+**optional**, **type**: i32
+
+The task will be closed if the idle check return IDLE the times as this value.
+
+.. note:: The value set at user side will overwrite this.
+
+**default**: 1
+
+.. _conf_server_common_extra_metrics_tags:
+
+extra_metrics_tags
+------------------
+
+**optional**, **type**: :ref:`static metrics tags <conf_value_static_metrics_tags>`
+
+Set extra metrics tags that should be added to server stats and user stats already with server tags added.
+
+**default**: not set

g3proxy/doc/configuration/servers/intelli_proxy.rst

@@ -0,0 +1,59 @@
+.. _configuration_server_intelli_proxy:
+
+intelli_proxy
+=============
+
+Intelligent Proxy port, it will do protocol detection and then send to other servers if detected.
+
+The following common keys are supported:
+
+* :ref:`listen <conf_server_common_listen>`
+* :ref:`listen_in_worker <conf_server_common_listen_in_worker>`
+* :ref:`ingress_network_filter <conf_server_common_ingress_network_filter>`
+
+http_server
+-----------
+
+**required**, **type**: str
+
+Set name of the next http_proxy server to send the accepted connections to.
+
+socks_server
+------------
+
+**required**, **type**: str
+
+Set name of the next socks_proxy server to send the accepted connections to.
+
+protocol_detection_channel_size
+-------------------------------
+
+**optional**, **type**: usize
+
+The connection will be send to a task channel after it's protocol is detected. This config option set the channel size.
+
+If the channel is full, the connection will be closed silently.
+
+**default**: 4096
+
+protocol_detection_timeout
+--------------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the timeout duration that we wait to check the protocol of each connection.
+
+If timeout, the connection will be closed silently.
+
+**default**: 4s
+
+protocol_detection_max_jobs
+---------------------------
+
+**optional**, **type**: usize
+
+Set the limit of protocol detection jobs.
+
+If the limit is reached, the connection will be closed silently.
+
+**default**: 4096

g3proxy/doc/configuration/servers/plain_tcp_port.rst

@@ -0,0 +1,21 @@
+.. _configuration_server_plain_tcp_port:
+
+plain_tcp_port
+==============
+
+This server provides plain tcp port, which can be placed in front of other servers.
+
+The following common keys are supported:
+
+* :ref:`listen <conf_server_common_listen>`
+* :ref:`listen_in_worker <conf_server_common_listen_in_worker>`
+* :ref:`ingress_network_filter <conf_server_common_ingress_network_filter>`
+
+server
+------
+
+**required**, **type**: str
+
+Set name of the next server to send the accepted connections to.
+
+The next server should be able to accept tcp connections.

g3proxy/doc/configuration/servers/plain_tls_port.rst

@@ -0,0 +1,24 @@
+.. _configuration_server_plain_tls_port:
+
+plain_tls_port
+==============
+
+This server provides plain tls port, which can be placed in front of other servers.
+
+The following common keys are supported:
+
+* :ref:`listen <conf_server_common_listen>`
+* :ref:`listen_in_worker <conf_server_common_listen_in_worker>`
+* :ref:`ingress_network_filter <conf_server_common_ingress_network_filter>`
+* :ref:`tls_server <conf_server_common_tls_server>`
+
+  This is required for this server.
+
+server
+------
+
+**required**, **type**: str
+
+Set name of the next server to send the accepted connections to.
+
+The next server should be able to accept tls connections.

g3proxy/doc/configuration/servers/sni_proxy.rst

@@ -0,0 +1,125 @@
+.. _configuration_server_sni_proxy:
+
+sni_proxy
+=========
+
+A tcp forward proxy server based on TLS SNI / HTTP Host.
+
+The following common keys are supported:
+
+* :ref:`escaper <conf_server_common_escaper>`
+* :ref:`auditor <conf_server_common_auditor>`
+* :ref:`shared_logger <conf_server_common_shared_logger>`
+* :ref:`listen <conf_server_common_listen>`
+* :ref:`listen_in_worker <conf_server_common_listen_in_worker>`
+* :ref:`tcp_sock_speed_limit <conf_server_common_tcp_sock_speed_limit>`
+* :ref:`ingress_network_filter <conf_server_common_ingress_network_filter>`
+* :ref:`tcp_copy_buffer_size <conf_server_common_tcp_copy_buffer_size>`
+* :ref:`tcp_copy_yield_size <conf_server_common_tcp_copy_yield_size>`
+* :ref:`tcp_misc_opts <conf_server_common_tcp_misc_opts>`
+* :ref:`task_idle_check_duration <conf_server_common_task_idle_check_duration>`
+* :ref:`task_idle_max_count <conf_server_common_task_idle_max_count>`
+* :ref:`extra_metrics_tags <conf_server_common_extra_metrics_tags>`
+
+request_wait_timeout
+--------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the timeout value for the wait of initial client data.
+
+**default**: 60s
+
+request_recv_timeout
+--------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the timeout value for the receive of the complete initial request after the arriving of initial data,
+which may be a TLS ClientHello message or a HTTP Request.
+
+**default**: 4s
+
+protocol_inspection
+-------------------
+
+**optional**, **type**: :ref:`protocol inspection <conf_value_dpi_protocol_inspection>`
+
+Set basic config for protocol inspection.
+
+**default**: set with default value
+
+.. versionadded:: 1.7.0
+
+server_tcp_portmap
+------------------
+
+**optional**, **type**: :ref:`server tcp portmap <conf_value_dpi_server_tcp_portmap>`
+
+Set the portmap for protocol inspection based on server side tcp port.
+
+**default**: set with default value
+
+.. versionadded:: 1.7.0
+
+client_tcp_portmap
+------------------
+
+**optional**, **type**: :ref:`client tcp portmap <conf_value_dpi_client_tcp_portmap>`
+
+Set the portmap for protocol inspection based on client side tcp port.
+
+**default**: set with default value
+
+.. versionadded:: 1.7.0
+
+allowed_hosts
+-------------
+
+**optional**, **type**: :ref:`host matched object <conf_value_host_matched_object>` <:ref:`host <configuration_server_sni_proxy_host>`>
+
+Set the list of hosts we should handle based on host match rules.
+
+If not set, all requests will be handled.
+
+Example:
+
+.. code-block:: yaml
+
+  hosts:
+    - exact_match:
+        - www.example.net
+        - example.net
+      redirect_host: www.example.net:443 # all redirect to www.example.net:*
+    - child_match: example.org # pass all *.example.org:*
+
+**default**: not set
+
+.. versionadded:: 1.1.1
+
+.. _configuration_server_sni_proxy_host:
+
+Host
+^^^^
+
+.. versionadded:: 1.1.1
+
+This set the config for a SNI host.
+
+redirect_host
+"""""""""""""
+
+**optional**, **type**: :ref:`host <conf_value_host>`
+
+Change the host field of the upstream address.
+
+**default**: not set
+
+redirect_port
+"""""""""""""
+
+**optional**, **type**: u16
+
+Change the port field of the upstream address.
+
+**default**: not set

g3proxy/doc/configuration/servers/socks_proxy.rst

@@ -0,0 +1,120 @@
+.. _configuration_server_socks_proxy:
+
+socks_proxy
+===========
+
+This server provides socks proxy, which support tcp connect and udp associate.
+
+The following common keys are supported:
+
+* :ref:`escaper <conf_server_common_escaper>`
+* :ref:`auditor <conf_server_common_auditor>`
+* :ref:`user_group <conf_server_common_user_group>`
+* :ref:`shared_logger <conf_server_common_shared_logger>`
+* :ref:`listen <conf_server_common_listen>`
+* :ref:`listen_in_worker <conf_server_common_listen_in_worker>`
+* :ref:`tcp_sock_speed_limit <conf_server_common_tcp_sock_speed_limit>`
+* :ref:`udp_sock_speed_limit <conf_server_common_udp_sock_speed_limit>`
+* :ref:`ingress_network_filter <conf_server_common_ingress_network_filter>`
+* :ref:`dst_host_filter_set <conf_server_common_dst_host_filter_set>`
+* :ref:`dst_port_filter <conf_server_common_dst_port_filter>`
+* :ref:`tcp_copy_buffer_size <conf_server_common_tcp_copy_buffer_size>`
+* :ref:`tcp_copy_yield_size <conf_server_common_tcp_copy_yield_size>`
+* :ref:`udp_relay_packet_size <conf_server_common_udp_relay_packet_size>`
+* :ref:`udp_relay_yield_size <conf_server_common_udp_relay_yield_size>`
+* :ref:`tcp_misc_opts <conf_server_common_tcp_misc_opts>`
+* :ref:`udp_misc_opts <conf_server_common_udp_misc_opts>`
+* :ref:`task_idle_check_duration <conf_server_common_task_idle_check_duration>`
+* :ref:`task_idle_max_count <conf_server_common_task_idle_max_count>`
+* :ref:`extra_metrics_tags <conf_server_common_extra_metrics_tags>`
+
+The auth type supported by the server is determined by the type of the specified user group.
+
++-------------+---------------------------+-------------------+
+|auth scheme  |user group type            |is supported       |
++=============+===========================+===================+
+|user         |hashed_user                |yes                |
++-------------+---------------------------+-------------------+
+|gssapi       |gss_api                    |not yet            |
++-------------+---------------------------+-------------------+
+
+use_udp_associate
+-----------------
+
+**optional**, **type**: bool, **alias**: enable_udp_associate
+
+Set whether we should use udp associate instead of udp connect.
+
+**default**: false
+
+negotiation_timeout
+-------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the max time duration for negotiation, before we start to handle the real socks commands.
+
+**default**: 4s
+
+udp_client_initial_timeout
+--------------------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the max time duration to wait before the first udp packet after we send back the udp port info.
+
+**default**: 30s
+
+udp_bind_ipv4
+-------------
+
+**optional**, **type**: :ref:`list <conf_value_list>` of :ref:`ipv4 addr str <conf_value_ipv4_addr_str>`
+
+Set the ipv4 addresses for udp associate local binding to socks client.
+If not set, the server ip for the tcp connection will be used when setup the udp listen socket.
+
+If set, the tcp connect can be in ipv6 address family.
+
+**default**: not set
+
+udp_bind_ipv6
+-------------
+
+**optional**, **type**: :ref:`list <conf_value_list>` of :ref:`ipv6 addr str <conf_value_ipv6_addr_str>`
+
+Set the ipv6 addresses for udp associate local binding to socks client.
+If not set, the server ip for the tcp connection will be used when setup the udp listen socket.
+
+If set, the tcp connect can be in ipv4 address family.
+
+**default**: not set
+
+udp_bind_port_range
+-------------------
+
+**optional**, **type**: :ref:`port range <conf_value_port_range>`
+
+Set the UDP port-range for udp associate local binding to socks client.
+If not set, the port will be selected by the OS.
+
+udp_socket_buffer
+-----------------
+
+**optional**, **type**: :ref:`socket buffer config <conf_value_socket_buffer_config>`
+
+Set the buffer config for the udp socket.
+
+.. note:: The buffer size of the socket at escaper side will also be set.
+
+**default**: not set
+
+auto_reply_local_ip_map
+-----------------------
+
+**optional**, **type**: map
+
+Set this if you want to reply another ip other then the real bind ip for the udp listen socket to the client.
+
+The key of the map should be the local ip, and the value should be the ip you want the client to use.
+
+**default**: not set

g3proxy/doc/configuration/servers/tcp_stream.rst

@@ -0,0 +1,70 @@
+.. _configuration_server_tcp_stream:
+
+tcp_stream
+==========
+
+A simple tcp stream server. Map local tcp port to remote tcp port.
+
+The following common keys are supported:
+
+* :ref:`escaper <conf_server_common_escaper>`
+* :ref:`auditor <conf_server_common_auditor>`
+* :ref:`shared_logger <conf_server_common_shared_logger>`
+* :ref:`listen <conf_server_common_listen>`
+* :ref:`listen_in_worker <conf_server_common_listen_in_worker>`
+* :ref:`tcp_sock_speed_limit <conf_server_common_tcp_sock_speed_limit>`
+* :ref:`ingress_network_filter <conf_server_common_ingress_network_filter>`
+* :ref:`tcp_copy_buffer_size <conf_server_common_tcp_copy_buffer_size>`
+* :ref:`tcp_copy_yield_size <conf_server_common_tcp_copy_yield_size>`
+* :ref:`tcp_misc_opts <conf_server_common_tcp_misc_opts>`
+* :ref:`task_idle_check_duration <conf_server_common_task_idle_check_duration>`
+* :ref:`task_idle_max_count <conf_server_common_task_idle_max_count>`
+* :ref:`extra_metrics_tags <conf_server_common_extra_metrics_tags>`
+
+upstream
+--------
+
+**required**, **type**: :ref:`upstream str <conf_value_upstream_str>` | seq
+
+Set the remote address(es) and port. The *port* field is always required.
+
+For *seq* value, each of its element must be :ref:`weighted upstream addr <conf_value_weighted_upstream_addr>`.
+
+**alias**: proxy_pass
+
+.. versionchanged:: 1.5.3 Allow set multiple upstream addresses.
+
+upstream_pick_policy
+----------------------
+
+**optional**, **type**: :ref:`selective pick policy <conf_value_selective_pick_policy>`
+
+Set the policy to select upstream address.
+
+The key for rendezvous/jump hash is *<client-ip>*.
+
+**default**: random
+
+.. versionadded:: 1.5.3
+
+tls_client
+----------
+
+**optional**, **type**: bool | :ref:`openssl tls client config <conf_value_openssl_tls_client_config>`
+
+Set if we should do tls handshake with upstream.
+
+**default**: disabled
+
+upstream_tls_name
+-----------------
+
+**optional**, **type**: :ref:`tls name <conf_value_tls_name>`
+
+Set an explicit tls server name to do upstream tls certificate verification.
+
+If not set, the host of upstream address will be used.
+
+.. note:: IP address is not supported by now
+
+**default**: not set

g3proxy/doc/configuration/servers/tls_stream.rst

@@ -0,0 +1,74 @@
+.. _configuration_server_tls_stream:
+
+tls_stream
+==========
+
+A simple tls stream server. Add tls layer to remote tcp port.
+
+The following common keys are supported:
+
+* :ref:`escaper <conf_server_common_escaper>`
+* :ref:`auditor <conf_server_common_auditor>`
+* :ref:`shared_logger <conf_server_common_shared_logger>`
+* :ref:`listen <conf_server_common_listen>`
+* :ref:`listen_in_worker <conf_server_common_listen_in_worker>`
+* :ref:`tls_server <conf_server_common_tls_server>`
+
+  This is **required**.
+
+* :ref:`tcp_sock_speed_limit <conf_server_common_tcp_sock_speed_limit>`
+* :ref:`ingress_network_filter <conf_server_common_ingress_network_filter>`
+* :ref:`tcp_copy_buffer_size <conf_server_common_tcp_copy_buffer_size>`
+* :ref:`tcp_copy_yield_size <conf_server_common_tcp_copy_yield_size>`
+* :ref:`tcp_misc_opts <conf_server_common_tcp_misc_opts>`
+* :ref:`task_idle_check_duration <conf_server_common_task_idle_check_duration>`
+* :ref:`task_idle_max_count <conf_server_common_task_idle_max_count>`
+* :ref:`extra_metrics_tags <conf_server_common_extra_metrics_tags>`
+
+upstream
+--------
+
+**required**, **type**: :ref:`upstream str <conf_value_upstream_str>` | seq
+
+Set the remote address(es) and port. The *port* field is always required.
+
+For *seq* value, each of its element must be :ref:`weighted upstream addr <conf_value_weighted_upstream_addr>`.
+
+**alias**: proxy_pass
+
+.. versionchanged:: 1.5.3 Allow set multiple upstream addresses.
+
+upstream_pick_policy
+----------------------
+
+**optional**, **type**: :ref:`selective pick policy <conf_value_selective_pick_policy>`
+
+Set the policy to select upstream address.
+
+The key for rendezvous/jump hash is *<client-ip>*.
+
+**default**: random
+
+.. versionadded:: 1.5.3
+
+tls_client
+----------
+
+**optional**, **type**: bool | :ref:`openssl tls client config <conf_value_openssl_tls_client_config>`
+
+Set if we should do tls handshake with upstream.
+
+**default**: disabled
+
+upstream_tls_name
+-----------------
+
+**optional**, **type**: :ref:`tls name <conf_value_tls_name>`
+
+Set an explicit tls server name to do upstream tls certificate verification.
+
+If not set, the host of upstream address will be used.
+
+.. note:: IP address is not supported by now
+
+**default**: not set

g3proxy/doc/configuration/stat.rst

@@ -0,0 +1,11 @@
+.. _configuration_stat:
+
+****
+Stat
+****
+
+This file described the stat config, which is optional and can not be reloaded.
+If set, it must reside in the main conf file.
+
+The value should be of type :ref:`statsd client config <conf_value_statsd_client_config>`,
+with the default *prefix* set to "g3proxy".

g3proxy/doc/configuration/user_group/audit.rst

@@ -0,0 +1,44 @@
+.. _configuration_user_group_user_audit:
+
+**********
+User Audit
+**********
+
+.. versionadded:: 1.7.0
+
+The user audit config is in map format. We will use this to specify user level audit actions.
+
+enable_protocol_inspection
+--------------------------
+
+**optional**, **type**: bool
+
+Whether protocol inspection functionality should be enabled.
+
+Protocol inspection will be enabled if true, and if audit is also enabled at both server and user side, for a specific user request.
+
+**default**: false
+
+prohibit_unknown_protocol
+-------------------------
+
+**optional**, **type**: bool
+
+Whether unknown protocol will be prohibited when protocol inspection is enabled.
+
+**default**: false
+
+application_audit_ratio
+-----------------------
+
+**optional**, **type**: :ref:`random ratio <conf_value_random_ratio>`
+
+Set the application audit (like ICAP REQMOD/RESPMOD) ratio for incoming user requests.
+
+This also controls whether protocol inspection is really enabled for a specific user request.
+
+If set, this will override the :ref:`application audit ratio <conf_auditor_application_audit_ratio>` config at auditor side.
+
+**default**: not set
+
+.. versionadded:: 1.7.4

g3proxy/doc/configuration/user_group/index.rst

@@ -0,0 +1,67 @@
+.. _configuration_user_group:
+
+**********
+User Group
+**********
+
+.. toctree::
+   :maxdepth: 2
+
+   user
+   source
+   audit
+   site
+
+The type for each user group config is *map*, with two always required keys:
+
+* *name*, which specify the name of the user group.
+* *type*, which specify the real type of the user group, decides how to parse other keys.
+
+For now, we only support *hashed_user* type of user group. We may add a *gss_api* type after sometime.
+
+The real auth type used in each protocol is determined by the type of user group.
+See documentation for each server type for the mapping.
+
+Group types
+===========
+
+hashed_user
+-----------
+
+This type of user group is consist of users that store hashed passwords. The clear text password must be transported
+to server, so we can calc its hash and compare with the ones in config.
+
+The users are split into two groups: static and dynamic. The static users is configured with key *static_users*
+in config file, each in yaml format. The dynamic users are fetched from *dynamic_source* periodically, each in json
+format. Both can be optional and share the same data structure.
+
+* static_users
+
+  **optional**, **type**: seq
+
+  Static user can be added in this array.
+
+  See :ref:`user <configuration_user_group_user>` for detailed structure of user.
+
+* source
+
+  **optional**, **type**: :ref:`url str <conf_value_url_str>` | map
+
+  Set the fetch source for dynamic users.
+
+  We support many type of sources. The type is detected by reading the *scheme* field of url,
+  or the *type* key of the map. See :ref:`source <configuration_user_group_source>` for all supported type of sources.
+
+  Dynamic users will be updated if they exist already. If the fetch fail, we will continue keep the old users.
+
+  **default**: not set
+
+.. _conf_user_group_refresh_interval:
+
+* refresh_interval
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the check interval for user expiration and the fetch interval for dynamic users.
+
+  **default**: 60s

g3proxy/doc/configuration/user_group/site.rst

@@ -0,0 +1,70 @@
+.. _configuration_user_group_user_site:
+
+*********
+User Site
+*********
+
+.. versionadded:: 1.3.4
+
+The user site config is in map format. We can set how to match this site, enable site level metrics, or do any other
+site level config.
+
+.. _conf_user_group_user_site_id:
+
+id
+--
+
+**required**, **type**: :ref:`metrics name <conf_value_metrics_name>`
+
+Each site should have an ID, and it will be used in metrics name if enabled.
+
+exact_match
+-----------
+
+**optional**, **type**: :ref:`host <conf_value_host>`
+
+Set the exact domain or the exact target IP in user request which we should match.
+
+.. note:: the value should be different within all sites config of the current user.
+
+child_match
+-----------
+
+**optional**, **type**: :ref:`domain <conf_value_domain>`
+
+Set the parent domain and all it's child domains will be matched.
+
+.. note:: the value should be different within all sites config of the current user.
+
+subnet_match
+------------
+
+**optional**, **type**: :ref:`ip network str <conf_value_ip_network_str>`
+
+Set the network to match if the target is IP address in user request.
+
+.. note:: the value should be different within all sites config of the current user.
+
+emit_stats
+----------
+
+**optional**, **type**: bool
+
+Set whether we should emit site level stats for this site.
+
+See :ref:`user site metrics <metrics_user_site>` for the definition of metrics.
+
+**default**: false
+
+resolve_strategy
+----------------
+
+**optional**, **type**: :ref:`resolve strategy <conf_value_resolve_strategy>`
+
+Set a custom resolve strategy at user-site level, which will override the one at user level,
+but still within the range of the one set on the escaper.
+Not all escapers support this, see the documentation for each escaper for more info.
+
+**default**: not custom resolve strategy is set
+
+.. versionadded:: 1.7.10

g3proxy/doc/configuration/user_group/source.rst

@@ -0,0 +1,250 @@
+.. _configuration_user_group_source:
+
+******
+Source
+******
+
+For the *map* format, the **type** key should always be set.
+
+file
+====
+
+Fetch dynamic users from a local file.
+
+The content of the file should be the json encoded string of all dynamic users.
+
+The keys used in *map* format are:
+
+* path
+
+  **required**, **type**: :ref:`file path <conf_value_file_path>`
+
+  Set path for the file. The file should be existed before start the daemon.
+
+* format
+
+  **optional**, **type**: :ref:`config file format <conf_value_config_file_format>`
+
+  Set the file format for the file specified in *path*.
+
+  **default**: If the file has extension, the extension will be used to detect the format.
+  If not format can be detected through extension, *yaml* will be used.
+
+For *url* str values, the *path* should be an absolute path with the following format:
+
+    file://<path>[?[format=<format>]]
+
+http
+====
+
+Fetch dynamic users through an http GET request.
+
+The success response code should be 200, and the body should be the json encoded string of all dynamic users.
+
+The keys used in *map* format are:
+
+* url
+
+  **required**, **type**: :ref:`url str <conf_value_url_str>`
+
+  Set the url to GET.
+
+* cache_file
+
+  **required**, **type**: :ref:`file path <conf_value_file_path>`
+
+  The local file to cache remote results, it will be used during initial load of the user group.
+
+  The file will be created if not existed.
+
+* timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout value for the whole request.
+
+  It's not recommended to set the timeout value greater the :ref:`refresh_interval <conf_user_group_refresh_interval>`
+  in group config.
+
+  **default**: 10s
+
+* connect_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the connect timeout value.
+
+  It's not recommended to set the timeout value greater the :ref:`refresh_interval <conf_user_group_refresh_interval>`
+  in group config.
+
+  **default**: 5s
+
+  .. versionadded:: 1.2.0
+
+* interface
+
+  **optional**, **type**: str
+
+  Set the bind IP or bind interface.
+
+  **default**: not set
+
+  .. versionadded:: 1.2.0
+
+* max_body_size
+
+  **optional**, **type**: :ref:`humanize size <conf_value_humanize_usize>`
+
+  Set the max body size.
+
+  **default**: 64MiB
+
+lua
+===
+
+.. versionadded:: 1.1.3
+
+Fetch users through local lua script.
+
+.. note::
+
+  Environment variable `LUA_PATH`_ and `LUA_CPATH`_ can be set to include more lua module files.
+  Any ";;" in the value of the *LUA_PATH* environment variable will be replaced by the default path.
+
+  .. _LUA_PATH: https://www.lua.org/manual/5.1/manual.html#pdf-package.path
+  .. _LUA_CPATH: https://www.lua.org/manual/5.1/manual.html#pdf-package.cpath
+
+
+The return value of the script should be the json encoded string of all dynamic users.
+
+The keys used in *map* format are:
+
+* cache_file
+
+  **required**, **type**: :ref:`file path <conf_value_file_path>`
+
+  The local file to cache results, it will be used during initial load of the user group.
+
+  The file will be created if not existed.
+
+* fetch_script
+
+  **required**, **type**: :ref:`file path <conf_value_file_path>`
+
+  The path of the lua script to fetch dynamic users.
+
+  The content of this script file should be like:
+
+  .. code-block:: lua
+
+    -- TODO fetch users
+    local result = "[]"
+    -- return the json encoded string
+    return result
+
+  **alias**: script
+
+* fetch_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout value for the execution of the fetch script.
+
+  It's not recommended to set the timeout value greater the :ref:`refresh_interval <conf_user_group_refresh_interval>`
+  in group config.
+
+  **default**: 30s, **alias**: timeout
+
+* report_script
+
+  **optional**, **type**: :ref:`file path <conf_value_file_path>`
+
+  The path of the lua script to report the parse result of the fetched dynamic users.
+
+  Two global functions should be defined in this script, like this:
+
+  ..  code-block:: lua
+
+    function reportOk ()
+      -- takes no argument
+    end
+
+    function reportErr (errMsg)
+      -- takes one argument, which the error message string
+    end
+
+  .. versionadded:: 1.4.1
+
+* report_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout value for the execution of the report script.
+
+  It's not recommended to set the timeout value greater the :ref:`refresh_interval <conf_user_group_refresh_interval>`
+  in group config.
+
+  **default**: 15s, **alias**: timeout
+
+  .. versionadded:: 1.4.1
+
+python
+======
+
+.. versionadded:: 1.5.0
+
+Fetch users through local python script.
+
+The keys used in *map* format are:
+
+* cache_file
+
+  **required**, **type**: :ref:`file path <conf_value_file_path>`
+
+  The local file to cache results, it will be used during initial load of the user group.
+
+  The file will be created if not existed.
+
+* script
+
+  **required**, **type**: :ref:`file path <conf_value_file_path>`
+
+  The path of the python script.
+
+  Three global functions should be defined in this script, like this:
+
+  ..  code-block:: python
+
+    def fetch_users():
+        # required, takes no argument, returns the json string
+        return "[]"
+
+    def report_ok():
+        # optional, takes no argument
+        pass
+
+    def report_err(errmsg):
+        # optional, takes one positional argument, which is the error message string
+        pass
+
+* fetch_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout value for the execution of the fetch function.
+
+  It's not recommended to set the timeout value greater the :ref:`refresh_interval <conf_user_group_refresh_interval>`
+  in group config.
+
+  **default**: 30s, **alias**: timeout
+
+* report_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout value for the execution of the report functions.
+
+  It's not recommended to set the timeout value greater the :ref:`refresh_interval <conf_user_group_refresh_interval>`
+  in group config.
+
+  **default**: 15s, **alias**: timeout

g3proxy/doc/configuration/user_group/user.rst

@@ -0,0 +1,324 @@
+.. _configuration_user_group_user:
+
+****
+User
+****
+
+The user config is in map format. We can specify how to authenticate the user, set limitations and we may also specify
+som custom actions for each user.
+
+name
+----
+
+**required**, **type**: :ref:`username <conf_value_username>`
+
+Set the username.
+
+token
+-----
+
+**required**, **type**: mix
+
+Set the token used to authenticate the user. The token can be in the following types:
+
+* str
+
+  The value should be a string in unix format, see crypt(5).
+
+* map
+
+  The key *type* specify the real type.
+
+  * fast_hash
+
+    A custom type. We use salt, and one or more value of md5, sha1, blake3. The hash is weak, but fast.
+    The values for *salt*, *md5*, *sha1*, *blake3* should be in hex encoded ascii string.
+
+  * xcrypt_hash
+
+    The required key is *value*, which value should be a valid crypt(5) string.
+
+The currently supported crypt(5) methods are: md5, sha256, sha512.
+
+expire
+------
+
+**optional**, **type**: :ref:`rfc3339 datetime str <conf_value_rfc3339_datetime_str>`
+
+Set when the user should be considered expired. The check interval is set by
+:ref:`refresh interval <conf_user_group_refresh_interval>` in group config.
+
+**default**: not set
+
+block_and_delay
+---------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Block the user, and delay sending of the error response by the specified duration.
+
+The response code for blocked user will be forbidden instead of auth failed.
+
+**default**: not set
+
+proxy_request_filter
+--------------------
+
+**optional**, **type**: :ref:`proxy request acl rule <conf_value_proxy_request_acl_rule>`
+
+Set the proxy request types that we should handle.
+
+**default**: not set
+
+dst_host_filter_set
+-------------------
+
+**optional**, **type**: :ref:`dst host acl rule set <conf_value_dst_host_acl_rule_set>`
+
+Set the filter for dst host of each request, which means it won't apply to udp associate tasks.
+
+**default**: not set
+
+dst_port_filter
+---------------
+
+**optional**, **type**: :ref:`exact port acl rule <conf_value_exact_port_acl_rule>`
+
+Set the filter for dst port of each request, which means it won't apply to udp associate tasks.
+
+**default**: not set
+
+http_user_agent_filter
+----------------------
+
+**optional**, **type**: :ref:`user agent acl rule <conf_value_user_agent_acl_rule>`
+
+Set the filter for HTTP User-Agent header.
+
+.. note:: This only applies to layer-7 http traffic, including http forward and https forward.
+
+**default**: not set
+
+tcp_connect
+-----------
+
+**optional**, **type**: :ref:`tcp connect <conf_value_tcp_connect>`
+
+Set user level tcp connect params, which will take effect for *direct* type escapers.
+And this will be limited by the escaper level settings.
+
+**default**: not set
+
+tcp_sock_speed_limit
+--------------------
+
+**optional**, **type**: :ref:`tcp socket speed limit <conf_value_tcp_sock_speed_limit>`
+
+Set speed limit for each tcp socket.
+
+**default**: no limit, **alias**: tcp_conn_speed_limit | tcp_conn_limit
+
+.. versionchanged:: 1.4.0 changed name to tcp_sock_speed_limit
+
+udp_sock_speed_limit
+---------------------
+
+**optional**, **type**: :ref:`udp socket speed limit <conf_value_udp_sock_speed_limit>`
+
+Set speed limit for each udp socket.
+
+**default**: no limit, **alias**: udp_relay_speed_limit | udp_relay_limit
+
+.. versionchanged:: 1.4.0 changed name to udp_sock_speed_limit
+
+tcp_remote_keepalive
+--------------------
+
+**optional**, **type**: :ref:`tcp keepalive <conf_value_tcp_keepalive>`
+
+Set tcp keepalive for the remote tcp socket.
+
+The tcp keepalive set in user config will only be taken into account in Direct type escapers.
+
+**default**: no keepalive set
+
+tcp_remote_misc_opts
+--------------------
+
+**optional**, **type**: :ref:`tcp misc sock opts <conf_value_tcp_misc_sock_opts>`
+
+Set misc tcp socket options for the remote tcp socket.
+
+The user level TOS and Mark config will overwrite the one set at escaper level.
+Other fields will be limited to the smaller ones.
+
+**default**: not set
+
+udp_remote_misc_opts
+--------------------
+
+**optional**, **type**: :ref:`udp misc sock opts <conf_value_udp_misc_sock_opts>`
+
+Set misc udp socket options for the remote udp socket.
+
+The user level TOS and Mark config will overwrite the one set at escaper level.
+Other fields will be limited to the smaller ones.
+
+**default**: not set
+
+tcp_client_misc_opts
+--------------------
+
+**optional**, **type**: :ref:`tcp misc sock opts <conf_value_tcp_misc_sock_opts>`
+
+Set misc tcp socket options for the client tcp socket before task connecting stage.
+
+The user level TOS and Mark config will overwrite the one set at escaper level.
+Other fields will be limited to the smaller ones.
+
+**default**: not set
+
+udp_client_misc_opts
+--------------------
+
+**optional**, **type**: :ref:`udp misc sock opts <conf_value_udp_misc_sock_opts>`
+
+Set misc udp socket options for the client udp socket.
+
+The user level TOS and Mark config will overwrite the one set at server level.
+Other fields will be limited to the smaller ones.
+
+**default**: not set
+
+http_upstream_keepalive
+-----------------------
+
+**optional**, **type**: :ref:`http keepalive <conf_value_http_keepalive>`
+
+Set http keepalive config at user level.
+
+**default**: set with default value
+
+tcp_conn_rate_limit
+-------------------
+
+**optional**, **type**: :ref:`rate limit quota <conf_value_rate_limit_quota>`
+
+Set rate limit on client side new connections.
+
+The same connection used for different users will be counted for each of them.
+
+**default**: no limit, **alias**: tcp_conn_limit_quota
+
+.. versionadded:: 1.4.0
+
+request_rate_limit
+------------------
+
+**optional**, **type**: :ref:`rate limit quota <conf_value_rate_limit_quota>`
+
+Set rate limit on request.
+
+**default**: no limit, **alias**: request_limit_quota
+
+request_max_alive
+-----------------
+
+**optional**, **type**: usize, **alias**: request_alive_max
+
+Set max alive requests at user level.
+
+Even if not set, the max alive requests should not be more than usize::MAX.
+
+**default**: no limit
+
+resolve_strategy
+----------------
+
+**optional**, **type**: :ref:`resolve strategy <conf_value_resolve_strategy>`
+
+Set an user custom resolve strategy, within the range of the one set on the escaper.
+Not all escapers support this, see the documentation for each escaper for more info.
+
+**default**: not custom resolve strategy is set
+
+resolve_redirection
+-------------------
+
+**optional**, **type**: :ref:`resolve redirection <conf_value_resolve_redirection>`
+
+Set the dns redirection rules at user level.
+
+**default**: not set
+
+log_rate_limit
+--------------
+
+**optional**, **type**: :ref:`rate limit quota <conf_value_rate_limit_quota>`
+
+Set rate limit on log request.
+
+**default**: no limit, **alias**: log_limit_quota
+
+.. _config_user_log_uri_max_chars:
+
+log_uri_max_chars
+-----------------
+
+**optional**, **type**: usize
+
+Set the max number of characters of uri should be logged in logs.
+
+If set, this will override the one set in server level.
+
+If not set, the one in server level will take effect.
+
+The password in uri will be replaced by *xyz* before logging.
+
+**default**: not set
+
+task_idle_max_count
+-------------------
+
+**optional**, **type**: i32
+
+The task will be closed if the idle check return IDLE the times as this value.
+
+This will overwrite the one set at server side,
+see :ref:`server task_idle_max_count <conf_server_common_task_idle_max_count>`.
+
+The idle check interval can only set at server side,
+see :ref:`server task_idle_check_duration <conf_server_common_task_idle_check_duration>`.
+
+**default**: 1
+
+socks_use_udp_associate
+-----------------------
+
+**optional**, **type**: bool
+
+Set if we should use socks udp associate instead of the simplified udp connect method.
+
+**default**: false
+
+.. versionadded:: 1.3.0
+
+audit
+-----
+
+**optional**, **type**: :ref:`user audit <configuration_user_group_user_audit>`
+
+Set audit config for this user.
+
+**default**: set with default values
+
+.. versionadded:: 1.7.0
+
+explicit_sites
+--------------
+
+**optional**, **type**: seq of :ref:`user site <configuration_user_group_user_site>`
+
+Set explicit sites for this user.
+
+.. versionadded:: 1.3.4

g3proxy/doc/configuration/values/acl.rst

@@ -0,0 +1,207 @@
+
+.. _configure_acl_value_types:
+
+***
+ACL
+***
+
+All acl value types are described here.
+
+Basic Type
+==========
+
+.. _conf_value_acl_action:
+
+acl action
+----------
+
+**yaml value**: str
+
+There are 4 types of acl actions:
+
+* permit
+
+  Permit if match the rule. Alternatives: allow, accept.
+
+* permit_log
+
+  Permit if match the rule and log. Alternatives: allow_log, accept_log.
+
+* forbid
+
+  Forbid if match the rule. Alternatives: deny, reject.
+
+* forbid_log
+
+  Forbid if match the rule and log. Alternatives: deny_log, reject_log.
+
+.. _conf_value_acl_rule:
+
+acl rule
+--------
+
+**yaml value**: mix
+
+All the rules share the same config format described in this section.
+
+An acl rule is consisted of many records, each of them has an associated :ref:`acl action <conf_value_acl_action>`.
+A default missed action can be set in the rule, it set the default action if no record matches.
+
+The value in map format is consisted of the following fields:
+
+* default
+
+  Set the default acl action if no rule match.
+
+  Default action if rule is set but with *default* omitted: forbid if not specified in the rule's doc.
+
+* any of the acl actions as the key str
+
+  The value should be a valid record or a list of them, with the key string as the acl action.
+  See detail types for the format of each record type.
+
+The value could also be a single record or a list of them, which means only them are permitted with no log.
+
+The default missed action is **forbid** and the default found action is **permit**,
+if they are not specified in the detail types.
+
+.. _conf_value_acl_rule_set:
+
+acl rule set
+------------
+
+**yaml value**: seq
+
+Acl rule set is a group of at least 2 acl rules. The rules are matched in order, see detail types for the real order.
+
+If any record in any rules is matched, that acl action will be used. If missed in all rules, all default missed actions
+will be compared and the most strict one will be used, so there is no default missed action at rule set level.
+
+Detail Type
+===========
+
+.. _conf_value_network_acl_rule:
+
+network acl rule
+----------------
+
+**yaml value**: :ref:`acl rule <conf_value_acl_rule>`
+
+The record type should be :ref:`ip network str <conf_value_ip_network_str>`.
+
+.. _conf_value_egress_network_acl_rule:
+
+egress network acl rule
+-----------------------
+
+**yaml value**: :ref:`network acl rule <conf_value_network_acl_rule>`
+
+The same type as network acl rule. Default added: forbid unspecified, loopback, link-local and discard-only addresses.
+
+.. _conf_value_ingress_network_acl_rule:
+
+ingress network acl rule
+------------------------
+
+**yaml value**: :ref:`network acl rule <conf_value_network_acl_rule>`
+
+The same type as network acl rule. Default added: permit 127.0.0.1 and ::1.
+
+.. _conf_value_dst_subnet_acl_rule:
+
+dst subnet acl rule
+-------------------
+
+**yaml value**: :ref:`network acl rule <conf_value_network_acl_rule>`
+
+The same type as network acl rule. Default added: forbid unspecified, loopback and link-local addresses.
+
+.. _conf_value_exact_host_acl_rule:
+
+exact host acl rule
+-------------------
+
+**yaml value**: :ref:`acl rule <conf_value_acl_rule>`
+
+The record type should be :ref:`host <conf_value_host>`.
+
+.. _conf_value_exact_port_acl_rule:
+
+exact port acl rule
+-------------------
+
+**yaml value**: :ref:`acl rule <conf_value_acl_rule>`
+
+The record type should be :ref:`ports <conf_value_ports>`.
+
+.. _conf_value_child_domain_acl_rule:
+
+child domain acl rule
+---------------------
+
+**yaml value**: :ref:`acl rule <conf_value_acl_rule>`
+
+Specify the parent domain to match, all children domain in this domain will be matched.
+
+The record type should be :ref:`domain <conf_value_domain>`.
+
+.. _conf_value_regex_set_acl_rule:
+
+regex set acl rule
+------------------
+
+**yaml value**: :ref:`acl rule <conf_value_acl_rule>`
+
+The record type should be a valid regex string.
+
+.. _conf_value_dst_host_acl_rule_set:
+
+dst host acl rule set
+---------------------
+
+**yaml value**: :ref:`acl rule set <conf_value_acl_rule_set>`
+
+This rule set is used to match dst host for each request.
+
+Consisted of the following rules:
+
+* exact_match
+
+  :ref:`exact host acl rule <conf_value_exact_host_acl_rule>`
+
+* child_match
+
+  :ref:`child domain acl rule <conf_value_child_domain_acl_rule>`
+
+* regex_match
+
+  :ref:`regex set acl rule <conf_value_regex_set_acl_rule>`
+
+* subnet_match
+
+  Match only if the host is an IP Address. The type is
+  :ref:`dst subnet acl rule <conf_value_dst_subnet_acl_rule>`
+
+The match order is the same as the list order above.
+
+.. _conf_value_user_agent_acl_rule:
+
+user agent acl rule
+-------------------
+
+**yaml value**: :ref:`acl rule <conf_value_acl_rule>`
+
+The record type should be a valid **product** string as specified in `rfc7231 User-Agent`_.
+
+The default missed action is **permit** and the default found action is **forbid**.
+
+.. _rfc7231 User-Agent: https://tools.ietf.org/html/rfc7231#section-5.5.3
+
+.. _conf_value_proxy_request_acl_rule:
+
+proxy request acl rule
+----------------------
+
+**yaml value**: :ref:`acl rule <conf_value_acl_rule>`
+
+The record type should be a valid :ref:`proxy request type <conf_value_proxy_request_type>`.

g3proxy/doc/configuration/values/audit.rst

@@ -0,0 +1,117 @@
+
+.. _configure_audit_value_types:
+
+*****
+Audit
+*****
+
+All audit value types are described here.
+
+ICAP
+====
+
+.. _conf_value_audit_icap_service_config:
+
+icap service config
+-------------------
+
+**type**: map | str
+
+Config ICAP service.
+
+For *str* value, the value will be treated as *url* as described following.
+
+For *map* value, the keys are:
+
+* url
+
+  **required**, **type**: :ref:`url str <conf_value_url_str>`
+
+  Set the ICAP service url.
+
+* tcp_keepalive
+
+  **optional**, **type**: :ref:`tcp keepalive <conf_value_tcp_keepalive>`
+
+  Set the keep-alive config for the tcp connection to ICAP server.
+
+  **default**: enabled with default value
+
+* icap_connection_pool
+
+  **optional**, **type**: :ref:`icap connection pool <conf_value_audit_icap_connection_pool>`
+
+  Set the connection pool config.
+
+  **default**: set with default value
+
+* icap_max_header_size
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set the max header size when parsing response from the ICAP server.
+
+  **default**: 8KiB
+
+* preview_data_read_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout value for the read of preview data.
+  If timeout, preview will not be used in the request send to the ICAP server.
+
+  **default**: 4s
+
+* respond_shared_names
+
+  **optional**, **type**: :ref:`http header name <conf_value_http_header_name>` or seq of this
+
+  Set the headers returned by ICAP server in REQMOD response that we should send in the following RESPMOD request.
+
+  This config option now only apply to REQMOD service.
+
+  **default**: not set
+
+  .. versionadded:: 1.7.4
+
+* bypass
+
+  **optional**, **type**: bool
+
+  Set if we should bypass if we can't connect to the ICAP server.
+
+  **default**: false
+
+.. _conf_value_audit_icap_connection_pool:
+
+icap connection pool
+--------------------
+
+**type**: map
+
+The keys are:
+
+* check_interval
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the min idle check interval.
+  New connections will be established if the idle connections are less than *min_idle_count*.
+
+  **default**: 10s
+
+* max_idle_count
+
+  **optional*, **type**: usize
+
+  Set the maximum idle connections count.
+
+  **default**: 128
+
+* min_idle_count
+
+  **optional**, **type**: usize
+
+  Set the minimum idle connections count.
+
+  **default**: 16

g3proxy/doc/configuration/values/base.rst

@@ -0,0 +1,213 @@
+
+.. _configure_base_value_types:
+
+****
+Base
+****
+
+.. _conf_value_nonzero_u32:
+
+nonzero u32
+===========
+
+**yaml value**: int
+
+A non-zero u32 in value type.
+
+.. _conf_value_humanize_usize:
+
+humanize usize
+==============
+
+**yaml value**: int | str
+
+For *str* value, it support units of 2^10 like "KiB", "MiB", or units of 1000 like "KB", "MB".
+
+For *int* value or *str* value without unit, the unit will be bytes.
+
+.. seealso::
+
+   `humanize_rs bytes <https://docs.rs/humanize-rs/0.1.5/humanize_rs/bytes/index.html>`_
+
+.. _conf_value_humanize_u32:
+
+humanize u32
+============
+
+**yaml value**: int | str
+
+For *str* value, it support units of 2^10 like "KiB", "MiB", or units of 1000 like "KB", "MB".
+
+For *int* value or *str* value without unit, the unit will be bytes.
+
+.. _conf_value_humanize_duration:
+
+humanize duration
+=================
+
+**yaml value**: int | str
+
+For *str* value, at least one unit is required. Multiple units string like "1h 30m 71s" is also supported.
+See `duration units`_ for all supported units.
+
+For *int* and *real* value, the unit will be seconds.
+
+.. seealso::
+
+   `humanize_rs duration <https://docs.rs/humanize-rs/0.1.5/humanize_rs/duration/index.html>`_
+
+.. _duration units: https://docs.rs/humanize-rs/0.1.5/src/humanize_rs/duration/mod.rs.html#115
+
+.. _conf_value_username:
+
+username
+========
+
+**yaml value**: str
+
+The UTF-8 username to be used in different contexts.
+Should be less than or equal to 255 bytes.
+
+.. _conf_value_password:
+
+password
+========
+
+**yaml value**: str
+
+The UTF-8 password to be used in different contexts.
+Should be less than or equal to 255 bytes.
+
+.. _conf_value_upstream_str:
+
+upstream str
+============
+
+**yaml value**: str
+
+The string should be in *<ip>[:<port>]* or *<domain>[:<port>]* format.
+
+If omitted, the *port* will be set to *0*.
+
+.. _conf_value_url_str:
+
+url str
+=======
+
+**yaml value**: str
+
+The string should be a valid url.
+
+.. _conf_value_ascii_str:
+
+ascii str
+=========
+
+**yaml value**: str
+
+The string should only consists of ascii characters.
+
+.. _conf_value_rfc3339_datetime_str:
+
+rfc3339 datetime str
+====================
+
+**yaml value**: str
+
+The string should be a value rfc3339 datetime string.
+
+.. _conf_value_selective_pick_policy:
+
+selective pick policy
+=====================
+
+**yaml value**: str
+
+The policy to select item from selective vectors.
+
+The following values are supported:
+
+* random
+
+  The default one.
+
+* serial | sequence
+
+  For nodes with the same weights, the order is kept as in the config.
+
+* round_robin | rr
+
+  For nodes with the same weights, the order is kept as in the config.
+
+* rendezvous
+
+  Rendezvous Hash. The key format is defined in the context of each selective vector.
+
+* jump_hash
+
+  Jump Consistent Hash. The key format is defined in the context of each selective vector.
+
+.. _conf_value_weighted_upstream_addr:
+
+weighted upstream addr
+======================
+
+**yaml value**: map | string
+
+A upstream str with weight set, which make can be grouped into selective vector.
+
+The map consists 2 fields:
+
+* addr
+
+  **required**, **type**: :ref:`upstream str <conf_value_upstream_str>`
+
+  The real value.
+
+* weight
+
+  **optional**, **type**: f64
+
+  The weight of the real value.
+
+  **default**: 1.0
+
+If the value type is string, then it's value will be the *addr* field, with *weight* set to default value.
+
+.. _conf_value_weighted_name_str:
+
+weighted name str
+=================
+
+**yaml value**: map | string
+
+A name string with weight set, which make can be grouped into selective vector.
+
+The map consists 2 fields:
+
+* name
+
+  **required**, **type**: string
+
+  The name. The meaning of the name is depending on the config context.
+
+* weight
+
+  **optional**, **type**: f64
+
+  The weight of the name.
+
+  **default**: 1.0
+
+If the value type is string, then it's value will be the *name* field, with *weight* set to default value.
+
+.. _conf_value_list:
+
+list
+====
+
+**yaml value**: mix
+
+A list container type for type T.
+
+The value could be a single value of type T, or a sequence of values of type T.

g3proxy/doc/configuration/values/dpi.rst

@@ -0,0 +1,482 @@
+
+.. _configure_dpi_value_types:
+
+***
+DPI
+***
+
+All dpi value types are described here.
+
+Protocol Inspection
+===================
+
+.. _conf_value_dpi_inspection_size_limit:
+
+inspection size limit
+---------------------
+
+**type**: map
+
+This will set size limit for each protocol with no explicit size limit in their specification.
+
+The keys ars:
+
+* ftp_greeting_msg
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set for FTP server greeting message.
+
+  **default**: 512
+
+* http_request_uri
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set for HTTP client request URI.
+
+  **default**: 4096
+
+* imap_greeting_msg
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set for IMAP server greeting message.
+
+  **default**: 512
+
+* nats_info_line
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set for NATS server info line.
+
+  **default**: 1024
+
+* smtp_greeting_msg
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set for SMTP server greeting message.
+
+  **default**: 512
+
+.. _conf_value_dpi_protocol_inspection:
+
+protocol inspection
+-------------------
+
+**type**: map
+
+This set the basic protocol inspection config.
+
+The keys are:
+
+* inspect_max_depth
+
+  **optional**, **type**: usize
+
+  Set the max inspection depth. The stream will be treated as unknown protocol if it's nested too much.
+
+  **default**: 4
+
+* data0_buffer_size
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set the stream buffer size for protocol inspection.
+
+  **default**: 4096
+
+* data0_wait_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the wait timeout for the initial data, from either the client side or the server side.
+
+  **default**: 60s
+
+* data0_read_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the read timeout value when reading initial data for protocol inspection after it's arrival.
+
+  If timeout, the protocol will be considered as unknown.
+
+  **default**: 4s
+
+* data0_size_limit
+
+  **optional**, **type**: :ref:`inspection size limit <conf_value_dpi_inspection_size_limit>`
+
+  Set inspection size limit for each protocol.
+
+  **default**: set with default value
+
+.. _conf_value_dpi_maybe_protocol:
+
+maybe protocol
+--------------
+
+**type**: str
+
+The following values are supported:
+
+* http
+* https
+* smtp
+* ssh
+* ftp
+* pop3
+* pop3s
+* nntp
+* nntps
+* imap
+* imaps
+* nats
+* bittorrent
+
+.. _conf_value_dpi_portmap:
+
+portmap
+-------
+
+**type**: seq | map
+
+Set the protocol indication for each port.
+
+For *seq* value, each element should be a map, with two keys:
+
+* port
+
+  **required**, **type**: u16
+
+  Set the port number.
+
+* protocol
+
+  **required**, **type**: :ref:`maybe protocol <conf_value_dpi_maybe_protocol>` | seq
+
+  Set the protocol(s).
+
+For *map* value, the key should be the port, and the value should be the same as the *protocol* above.
+
+.. _conf_value_dpi_server_tcp_portmap:
+
+server tcp portmap
+------------------
+
+**type**: :ref:`portmap <conf_value_dpi_portmap>`
+
+Set the protocol indication for each server side tcp port.
+
+See the code `lib/g3-dpi/src/protocol/portmap.rs` for default set ports.
+
+.. _conf_value_dpi_client_tcp_portmap:
+
+client tcp portmap
+------------------
+
+**type**: :ref:`portmap <conf_value_dpi_portmap>`
+
+Set the protocol indication for each client side tcp port.
+
+See the code `lib/g3-dpi/src/protocol/portmap.rs` for default set ports.
+
+TLS Interception
+================
+
+.. _conf_value_dpi_tls_cert_generator:
+
+tls cert generator
+------------------
+
+**type**: map
+
+Set the config for tls certificate generator.
+
+The keys are:
+
+* query_peer_addr
+
+  **optional**, **type**: :ref:`sockaddr str <conf_value_sockaddr_str>`
+
+  Set the peer udp socket address.
+
+  **default**: 127.0.0.1:2999
+
+* query_socket_buffer
+
+  **optional**, **type**: :ref:`socket buffer config <conf_value_socket_buffer_config>`
+
+  Set the socket buffer config for the socket to peer.
+
+  **default**: not set
+
+* query_wait_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout for the cache runtime to wait response from the query runtime.
+
+  **default**: 400ms
+
+* protective_cache_ttl
+
+  **optional**, **type**: u32
+
+  Set the protective cache ttl for certificates returned by peer.
+
+  **default**: 10
+
+* maximum_cache_ttl
+
+  **optional**, **type**: u32
+
+  Set the maximum cache ttl for certificates returned by peer.
+
+  **default**: 300
+
+* cache_request_batch_count
+
+  **optional**, **type**: usize
+
+  Set the batch request count in cache runtime.
+
+  **default**: 10
+
+* cache_request_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the request timeout for the caller.
+
+  **default**: 800ms
+
+* cache_vanish_wait
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the vanish time after the record is considered expired (not the certificate expire time).
+
+  **default**: 300s
+
+.. _conf_value_dpi_tls_interception_client:
+
+tls interception client
+-----------------------
+
+**type**: map
+
+Set the tls client config for tls interception.
+
+The keys are:
+
+* ca_certificate
+
+  **optional**, **type**: :ref:`tls certificates <conf_value_tls_certificates>`
+
+  Add CA certificate for certificate verification of the upstream server.
+
+  **default**: not set
+
+* no_default_ca_certificate
+
+  **optional**, **type**: false
+
+  Set if we should not load the system default CA certificates.
+
+  **default**: false
+
+* handshake_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout for upstream tls handshake.
+
+  **default**: 10s
+
+* no_session_cache
+
+  **optional**, **type**: bool
+
+  Set if we should disable tls session cache.
+
+  **default**: false
+
+* session_cache_lru_max_sites
+
+  **optional**, **type**: usize
+
+  Set how many LRU sites should have cached sessions.
+
+  **default**: 128
+
+* session_cache_each_capacity
+
+  **optional**, **type**: usize
+
+  Set how many sessions should be kept for each site.
+
+  **default**: 16
+
+HTTP Interception
+=================
+
+.. _conf_value_dpi_h1_interception:
+
+h1 interception
+---------------
+
+**type**: map
+
+Set the config for HTTP 1.x interception.
+
+The keys are:
+
+* pipeline_size
+
+  **optional**, **type**: usize
+
+  Set the pipeline size.
+
+  **default**: 10
+
+* pipeline_read_idle_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the idle timeout of the client side IDLE http connections.
+
+  **default**: 5min
+
+* req_header_recv_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the max time to wait a full request header after the client connection become readable.
+
+  **default**: 30s
+
+* rsp_header_recv_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the max time duration after the full request sent and before receive of the whole response header.
+
+  **default**: 60s
+
+* req_header_max_size
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set the max request header size.
+
+  **default**: 64KiB
+
+* rsp_header_max_size
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set the max response header size.
+
+  **default**: 64KiB
+
+* body_line_max_length
+
+  **optional**, **type**: int
+
+  Set the max line length for lines (trailer and chunk size) in http body.
+
+  **default**: 8192
+
+.. _conf_value_dpi_h2_interception:
+
+h2 interception
+---------------
+
+**type**: map
+
+Set the config for HTTP 2.0 interception.
+
+The keys are:
+
+* max_header_list_size
+
+  **optional**, **type**: :ref:`humanize u32 <conf_value_humanize_u32>`
+
+  Set the max header size.
+
+  **default**: 64KiB
+
+* max_concurrent_streams
+
+  **optional**, **type**: u32
+
+  Set the max concurrent stream for each http2 connection.
+
+  **default**: 16
+
+* max_frame_size
+
+  **optional**, **type**: :ref:`humanize u32 <conf_value_humanize_u32>`
+
+  Set the max frame size.
+
+  **default**: 1MiB
+
+* max_send_buffer_size
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set the max send buffer size.
+
+  **default**: 16MiB
+
+* disable_upstream_push
+
+  **optional**, **type**: bool
+
+  Set if we should disable server push.
+
+  **default**: false
+
+* upstream_handshake_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the http2 handshake timeout to upstream.
+
+  **default**: 10s
+
+* upstream_stream_open_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the upstream stream open timeout.
+
+  **default**: 10s
+
+* client_handshake_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the http2 handshake timeout to client.
+
+  **default**: 4s
+
+* rsp_header_recv_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the max time duration after the full request sent and before receive of the whole response header.
+
+  **default**: 60s
+
+* silent_drop_expect_header
+
+  **optional**, **type**: bool
+
+  Set if we should drop the *Expect* http header silently.
+  If not set, a *417 Expectation Failed* response will be sent to client.

g3proxy/doc/configuration/values/fs.rst

@@ -0,0 +1,71 @@
+
+.. _configure_fs_value_types:
+
+**********
+Filesystem
+**********
+
+.. _conf_value_hybrid_map:
+
+hybrid map
+==========
+
+**yaml value**: seq | str
+
+This is a hybrid container for a sequence of maps which may reside in other files.
+
+For *seq* value, all of it's values should be the final map, or a valid *str* value as described below.
+
+For *str* value, it should be a valid path, which may be absolute or relative to the directory of the main conf file.
+
+The path may be a file or directory:
+
+* If the path is a directory, the non-symbolic files in it with extension *.conf* will be parsed as described below.
+* If the path is a file, it should contains one or many yaml docs, each doc will be the final map.
+
+.. _conf_value_file_path:
+
+file path
+=========
+
+**yaml value**: str
+
+This set the path for a regular file to be used.
+
+The file should be an absolute path, or relative to a predefined path.
+
+The path should be existed, or can be auto created, according to the specific config.
+
+.. _conf_value_file:
+
+file
+====
+
+**yaml value**: str
+
+This set a file to be read. The file should be an absolute path, or relative to a predefined path.
+
+.. _conf_value_absolute_path:
+
+absolute path
+=============
+
+**yaml value**: str
+
+The set a file path to be used. The path should be absolute.
+
+.. _conf_value_config_file_format:
+
+config file format
+==================
+
+**yaml value**: str
+
+Set the format for the related config file.
+
+The following values are supported:
+
+* yaml
+* json
+
+The default vary in different contexts.

g3proxy/doc/configuration/values/index.rst

@@ -0,0 +1,22 @@
+
+******************
+Common Value Types
+******************
+
+There are many common values used across this document.
+The meanings and formats of them are described as follows:
+
+.. toctree::
+
+   base
+   fs
+   network
+   acl
+   audit
+   dpi
+   tls
+   rate_limit
+   resolve
+   metrics
+   route
+   runtime

g3proxy/doc/configuration/values/metrics.rst

@@ -0,0 +1,128 @@
+
+.. _configure_metrics_value_types:
+
+*******
+Metrics
+*******
+
+.. _conf_value_metrics_value:
+
+metrics value
+=============
+
+**yaml value**: limited str
+
+Only the following characters are allowed:
+
+a to z, A to Z, 0 to 9, -, _, ., / or Unicode letters (as per the specification)
+
+The character range is the same as `OpenTSDB metrics-and-tags`_.
+
+.. _OpenTSDB metrics-and-tags: http://opentsdb.net/docs/build/html/user_guide/writing/index.html#metrics-and-tags
+
+.. _conf_value_static_metrics_tags:
+
+static metrics tags
+===================
+
+**yaml value**: map
+
+This should be a map, each key will be the tag name, and it's value will be the tag value.
+
+The tag name and the tag value should be of type :ref:`metrics value <conf_value_metrics_value>`.
+
+.. _conf_value_metrics_name:
+
+metrics name
+============
+
+**yaml value**: :ref:`metrics value <conf_value_metrics_value>`
+
+The metrics name
+
+
+.. _conf_value_statsd_client_config:
+
+Statsd Client Config
+====================
+
+The full format of the root value should be a map, with the following keys:
+
+target_unix
+-----------
+
+**optional**, **type**: mix
+
+You can set this if you want to send statsd metrics to a custom unix socket path.
+
+The value can be a map, with the following keys:
+
+* path
+
+  **required**, **type**: :ref:`absolute path <conf_value_absolute_path>`
+
+  The syslogd daemon listen socket path.
+
+If the value type is str, the value should be the same as the value as *path* above.
+
+**default**: not set
+
+.. versionadded:: 1.3.5
+
+target_udp
+----------
+
+**optional**, **type**: mix
+
+You can set this if you want to send statsd metrics to a remote statsd which listening on a udp socket.
+
+The value can be a map, with the following keys:
+
+* address
+
+  **optional**, **type**: :ref:`sockaddr str <conf_value_sockaddr_str>`
+
+  Set the remote socket address.
+
+  **default**: 127.0.0.1:8125
+
+* bind_ip
+
+  **optional**, **type**: :ref:`ip addr str <conf_value_ip_addr_str>`
+
+  Set the ip address to bind to for the local socket.
+
+  **default**: not set
+
+If the value type is str, the value should be the same as the value as *address* above.
+
+.. versionadded:: 1.3.5
+
+target
+------
+
+**optional**, **type**: map
+
+This is just another form to set statsd target address.
+
+The key *udp* is just handled as *target_udp* as above.
+
+The key *unix* is just handled as *target_unix* as above.
+
+prefix
+------
+
+**optional**, **type**: :ref:`metrics name <conf_value_metrics_name>`
+
+Set the global prefix for all metrics.
+
+**default**: "g3proxy"
+
+emit_duration
+-------------
+
+**optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+Set the emit duration for local stats. All stats will be send out in sequence.
+
+**default**: 200ms

g3proxy/doc/configuration/values/network.rst

@@ -0,0 +1,817 @@
+
+.. _configure_network_value_types:
+
+*******
+Network
+*******
+
+.. _conf_value_sockaddr_str:
+
+sockaddr str
+============
+
+**yaml value**: str
+
+The string should be in *<ip>[:<port>]* format, in which the port may be omitted if a default value is available.
+
+.. _conf_value_ip_addr_str:
+
+ip addr str
+===========
+
+**yaml value**: str
+
+The string should be in *<ip>* format.
+
+.. _conf_value_ipv4_addr_str:
+
+ipv4 addr str
+=============
+
+**yaml value**: str
+
+The string should be in *<ipv4 address>* format.
+
+.. _conf_value_ipv6_addr_str:
+
+ipv6 addr str
+=============
+
+**yaml value**: str
+
+The string should be in *<ipv6 address>* format.
+
+Ipv4 mapped address should not be set when this type is required.
+
+.. _conf_value_ip_network_str:
+
+ip network str
+==============
+
+**yaml value**: str
+
+The string should be a network address in CIDR format, or just an ip address.
+
+.. _conf_value_egress_area:
+
+egress area
+===========
+
+**yaml value**: str
+
+Area of the egress ip address. The format is strings joined with '/', like "中国/山东/济南".
+
+.. _conf_value_host:
+
+host
+====
+
+**yaml value**: str
+
+A host value. Which should be either a valid domain, or a valid IP address.
+
+.. _conf_value_domain:
+
+domain
+======
+
+**yaml value**: str
+
+A domain value. The string value should be able to convert to a IDNA domain.
+
+Leading '.' is not allowed.
+
+.. _conf_value_ports:
+
+ports
+=====
+
+**yaml value**: mix
+
+A collection of ip ports. The base type may be:
+
+* u16 port
+
+  A single port.
+
+* <start>-<end> port range
+
+  A port range, which includes both *start* and *end*. *end* should be greater than *start*.
+
+* comma separated discrete port(s)
+
+  A list of port. Each could be a port or a range of ports.
+
+The yaml value could be:
+
+* int
+
+  int base type.
+
+* str
+
+  str base types.
+
+* array
+
+  array of base types.
+
+.. _conf_value_port_range:
+
+port range
+==========
+
+**yaml value**: mix
+
+A consequent range of ip ports. It consists of 2 fields:
+
+* start
+
+  **required**, **type**: u16, **inclusive**
+
+  The start of the port range. Should be greater than zero.
+
+* end
+
+  **required**, **type**: u16, **inclusive**
+
+  The end of the port range. Should be greater than *start*.
+
+The yaml value for *port range* can be in the following formats:
+
+* str
+
+  In format <start>-<end>. Extra whitespaces is allowed.
+
+* map
+
+  The keys of this map are the fields as described above.
+
+.. _conf_value_socket_buffer_config:
+
+socket buffer config
+====================
+
+**yaml value**: mix
+
+It consists of 2 fields:
+
+* recv
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set the recv buf size.
+
+  **default**: not set
+
+* send
+
+  **optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  Set the send buf size.
+
+  **default**: not set
+
+The yaml value for *socket buffer config* can be in the following formats:
+
+* int | string
+
+  The value will be set for both **recv** and **send** fields above.
+
+* map
+
+  The keys of this map are the fields as described above.
+
+.. _conf_value_tcp_listen:
+
+tcp listen
+==========
+
+**yaml value**: mix
+
+It consists of 4 fields:
+
+* address
+
+  **required**, **type**: :ref:`sockaddr str <conf_value_sockaddr_str>`
+
+  Set the listen socket address.
+
+  **default**: [::]:0, which has empty port
+
+* backlog
+
+  **optional**, **type**: unsigned int
+
+  Set the listen backlog number for tcp sockets. The default value will be used if the specified value is less than 8.
+
+  **default**: 4096
+
+  .. note::
+
+    If the backlog argument is greater than the value in /proc/sys/net/core/somaxconn, then it is silently truncated
+    to that value. Since Linux 5.4, the default in this file is 4096; in earlier kernels, the default value is 128.
+
+* ipv6_only
+
+  **optional**, **type**: bool
+
+  Listen only to ipv6 address only if address is set to [::].
+
+  **default**: false
+
+* instance
+
+  **optional**, **type**: int
+
+  Set how many listen instances. If *scale* is set, this will be the least value.
+
+  **default**: 1
+
+* scale
+
+  **optional**, **type**: float | string
+
+  Set the listen instance count scaled according to available parallelism.
+
+  For string value, it could be in percentage (n%) or fractional (n/d) format.
+
+  Example:
+
+  .. code-block:: yaml
+
+    scale: 1/2
+    # or
+    scale: 0.5
+    # or
+    scale: 50%
+
+  **default**: 0
+
+  .. versionadded:: 1.7.8
+
+The yaml value for *listen* can be in the following formats:
+
+* int
+
+  Set the port only.
+
+* :ref:`sockaddr str <conf_value_sockaddr_str>`
+
+  Set ip and port. The port field is required.
+
+* map
+
+  The keys of this map are the fields as described above.
+
+.. _conf_value_tcp_connect:
+
+tcp connect
+===========
+
+**yaml value**: map
+
+This set TCP connect params.
+
+It consists of 2 fields:
+
+* max_retry
+
+  **optional**, **type**: int
+
+  Set the max tcp connect retry for a single upstream connection of the same address family.
+  The total tcp connect tries will be *1 + max_retry*.
+
+  Each resolved IP addr will be tried at most once.
+
+  **default**: 2, which means the total tries is 3
+
+* each_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the max timeout for each connection to the resolved addr of the upstream.
+
+  **default**: 30s
+
+.. _conf_value_happy_eyeballs:
+
+happy eyeballs
+==============
+
+**yaml value**: map
+
+This set Happy Eyeballs params for multiple tcp connections.
+
+It consists of the following fields:
+
+* resolution_delay
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  The resolution delay time for the wait of the preferred address family after another one is returned.
+
+  **default**: 50ms
+
+* second_resolution_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  The timeout time for the wait of the second resolution after no running connection attempts.
+
+  **default**: 2s
+
+* first_address_family_count
+
+  **optional**, **type**: usize
+
+  The address to try before use the addresses from another address family.
+
+  **default**: 1
+
+* connection_attempt_delay
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  The delay time before start a new connection after the previous one.
+
+  **default**: 250ms, **min**: 100ms, **max**: 2s
+
+.. versionadded:: 1.5.3
+
+.. _conf_value_tcp_keepalive:
+
+tcp keepalive
+=============
+
+**yaml value**: mix
+
+This set TCP level keepalive settings.
+
+It consists of 2 fields:
+
+* enable
+
+  **optional**, **type**: bool
+
+  Set whether tcp keepalive should be enabled.
+
+  **default**: false, which means you can set limit on other values in case keepalive is needed somewhere
+
+* idle_time
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the keepalive idle time.
+
+  **default**: 60s
+
+* probe_interval
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the probe interval after idle.
+
+  **default**: not set, which means the OS default value will be used
+
+* probe_count
+
+  **optional**, **type**: u32
+
+  Set the probe count.
+
+  **default**: not set, which means the OS default value will be used
+
+If the root value type is bool, the value will be parsed the same as the *enable* key.
+
+If the root value type is not map and not bool, the value will be parsed the same as the *idle_time* key, but with
+*enable* set to true.
+
+.. _conf_value_tcp_misc_sock_opts:
+
+tcp misc sock opts
+==================
+
+**yaml value**: map
+
+This set misc tcp socket options.
+
+Keys:
+
+* no_delay
+
+  **optional**, **type**: bool
+
+  Set value for tcp level socket option TCP_NODELAY. If set to true, disable the Nagle algorithm.
+
+  **default**: the default value varies, check the doc of the outer option
+
+* mss
+
+  **optional**, **type**: u32, **alias**: max_segment_size
+
+  Set value for tcp level socket option TCP_MAXSEG, the maximum segment size for outgoing TCP packets.
+
+  **default**: not set
+
+* ttl
+
+  **optional**, **type**: u32, **alias**: time_to_live
+
+  Set value for ip level socket option IP_TTL, the time-to-live field in each sent packet.
+
+  **default**: not set
+
+* tos
+
+  **optional**, **type**: u8, **alias**: type_of_service
+
+  Set value for ip level socket option IP_TOS, the type-of-service field in each sent packet.
+
+  **default**: not set
+
+* mark
+
+  **optional**, **type**: u32, **alias**: netfilter_mark
+
+  Set value for socket level socket option SO_MARK, the netfilter mark value for our tcp sockets.
+
+  **default**: not set
+
+.. _conf_value_udp_misc_sock_opts:
+
+udp misc sock opts
+==================
+
+**yaml value**: map
+
+This set misc udp socket options.
+
+Keys:
+
+* ttl
+
+  **optional**, **type**: u32, **alias**: time_to_live
+
+  Set value for ip level socket option IP_TTL, the time-to-live field in each sent packet.
+
+  **default**: not set
+
+* tos
+
+  **optional**, **type**: u8, **alias**: type_of_service
+
+  Set value for ip level socket option IP_TOS, the type-of-service field in each sent packet.
+
+  **default**: not set
+
+* mark
+
+  **optional**, **type**: u32, **alias**: netfilter_mark
+
+  Set value for socket level socket option SO_MARK, the netfilter mark value for our tcp sockets.
+
+  **default**: not set
+
+.. _conf_value_http_header_name:
+
+http header name
+================
+
+**yaml value**: str
+
+This string should be a valid HTTP header name.
+
+.. _conf_value_http_keepalive:
+
+http keepalive
+==============
+
+**yaml value**: mix
+
+This set HTTP level keepalive settings.
+
+It consists of 2 fields:
+
+* enable
+
+  **optional**, **type**: bool
+
+  Set whether tcp keepalive should be enabled.
+
+  **default**: true
+
+* idle_expire
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the idle expire time for the saved connection.
+  If the last active time for the connection has elapsed, the connection will be dropped.
+
+  **default**: 60s
+
+If the root value type is bool, the value will be parsed the same as the *enable* key.
+
+If the root value type is not map and not bool, the value will be parsed the same as the *idle_expire* key, but with
+*enable* set to true.
+
+.. _conf_value_http_forwarded_header_type:
+
+http forwarded header type
+==========================
+
+**yaml value**: str | bool
+
+This set the header type we set in requests for identifying the originating IP address of a client connected to us.
+
+The string values are:
+
+* none
+
+  Do not set any header.
+
+* classic
+
+  Use the de-facto standard header *X-Forwarded-For*, this is widely used.
+
+* standard
+
+  Use the standard header *Forwarded* defined in rfc7239. We set both the *for* and the *by* parameter in this case.
+
+If the yaml value type is bool, *true* will be *classic*, and *false* will be none.
+
+.. _conf_value_http_forward_capability:
+
+http forward capability
+=======================
+
+**yaml value**: map
+
+The following fields can be set:
+
+* forward_https
+
+  **optional**, **type**: bool
+
+  Whether we should forward request of https url to next proxy.
+
+  If not, we will do tls handshake with upstream locally.
+
+  **default**: false
+
+* forward_ftp
+
+  **optional**, **type**: bool
+
+  Whether we should forward all requests of ftp url to next proxy.
+
+  If not, we will act as a ftp client.
+
+  It can be overwritten by the specific forward_ftp_* options as described below for the corresponding http methods.
+
+  **default**: false
+
+* forward_ftp_get
+
+  **optional**, **type**: bool
+
+  Whether we should forward the GET request of ftp url to next proxy.
+
+  If not, we will act as a ftp client.
+
+  **default**: false
+
+* forward_ftp_put
+
+  **optional**, **type**: bool
+
+  Whether we should forward the PUT request of ftp url to next proxy.
+
+  If not, we will act as a ftp client.
+
+  **default**: false
+
+* forward_ftp_del
+
+  **optional**, **type**: bool
+
+  Whether we should forward the DELETE request of ftp url to next proxy.
+
+  If not, we will act as a ftp client.
+
+  **default**: false
+
+.. _conf_value_http_server_id:
+
+http server id
+==============
+
+**yaml value**: str
+
+Set http server id (server name) for http forwarding services.
+
+All characters should be ASCII in range '0x20' - '0x7E', except for ';' and ','.
+
+.. _conf_value_proxy_protocol_version:
+
+proxy protocol version
+======================
+
+**yaml value**: u8
+
+Set the PROXY protocol version.
+
+We support version 1 and version 2 for outgoing tcp connections.
+
+.. _conf_value_ftp_control_config:
+
+ftp control config
+==================
+
+**yaml value**: map
+
+The following fields can be set:
+
+* max_line_len
+
+  **optional**, **type**: usize
+
+  Set the max line length.
+
+  **default**: 2048
+
+* max_multi_lines
+
+  **optional**, **type**: usize
+
+  Set the max lines for multi-line reply.
+
+  **default**: 128
+
+* command_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the general command timeout value for commands with no explicit timeout config.
+
+  **default**: 10s
+
+.. _conf_value_ftp_transfer_config:
+
+ftp transfer config
+===================
+
+**yaml value**: map
+
+The following fields can be set:
+
+* list_max_line_len
+
+  **optional**, **type**: usize
+
+  Set the max line length for list reply.
+
+  **default**: 2048
+
+* list_max_entries
+
+  **optional**, **type**: usize
+
+  Set the max lines will be handled in list reply.
+
+  **default**: 1024
+
+* list_all_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout value for listing.
+
+  **default**: 120s, **max**: 300s
+
+* end_wait_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout value when waiting for the end of the transfer action at both the control and the transfer channel.
+
+  **default**: 10s
+
+.. _conf_value_ftp_client_config:
+
+ftp client config
+=================
+
+**yaml value**: map
+
+The following fields can be set:
+
+* control
+
+  **optional**, **type**: :ref:`ftp control config <conf_value_ftp_control_config>`
+
+  Set config for the ftp control channel.
+
+* transfer
+
+  **optional**, **type**: :ref:`ftp transfer config <conf_value_ftp_transfer_config>`
+
+  Set config for the ftp transfer channels.
+
+* connect_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the connection timeout for both control and transfer channels.
+
+  **default**: 30s
+
+* greeting_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the timeout for waiting of the greeting message from the server.
+
+  **default**: 10s
+
+* always_try_epsv
+
+  **optional**, **type**: bool
+
+  Set if we should always try EPSV command even server doesn't set it in feature.
+
+  **default**: true
+
+.. _conf_value_dns_encryption_protocol:
+
+dns encryption protocol
+=======================
+
+**yaml value**: enum
+
+The followings values are supported:
+
+* dns-over-tls | dot | tls
+
+  If `dns over tls`_ should be used.
+
+.. _dns over tls: https://datatracker.ietf.org/doc/html/rfc7858
+
+* dns-over-https | doh | https
+
+  If `dns over https`_ should be used.
+
+.. _dns over https: https://datatracker.ietf.org/doc/html/rfc8484
+
+.. _conf_value_dns_encryption_config:
+
+dns encryption config
+=====================
+
+**yaml value**: map | str
+
+The following fields can be set:
+
+* tls_name
+
+  **required**, **type**: :ref:`tls name <conf_value_tls_name>`
+
+  Set the tls server name.
+
+* protocol
+
+  **optional**, **type**: :ref:`dns encryption protocol <conf_value_dns_encryption_protocol>`
+
+  Set the encryption protocol.
+
+  **default**: dns-over-tls
+
+* tls_client
+
+  **optional**, **type**: :ref:`rustls client config <conf_value_rustls_client_config>`
+
+  Set the tls client config.
+
+  .. note:: not all fields will be used, check the doc of each key has the value *dns encryption config*.
+
+  **default**: not set
+
+If in str format, the value will be treated as field *tls_name*.
+
+.. versionadded:: 1.1.4
+
+.. _conf_value_proxy_request_type:
+
+proxy request type
+==================
+
+**yaml type**: enum string
+
+The values are:
+
+* HttpForward
+* HttpsForward
+* FtpOverHttp
+* HttpConnect
+* SocksTcpConnect
+* SocksUdpAssociate

g3proxy/doc/configuration/values/rate_limit.rst

@@ -0,0 +1,176 @@
+
+.. _configure_rate_limit_value_types:
+
+**********
+Rate Limit
+**********
+
+.. _conf_value_tcp_sock_speed_limit:
+
+tcp socket speed limit
+======================
+
+**yaml value**: mix
+
+It consists of 3 fields:
+
+* shift_millis | shift
+
+  **type**: int
+
+  The time slice we use to count is *2 ^ N* milliseconds, where N is set by this key and should be in range [0 - 12].
+  If N is 10, and the time slice is 1024ms. If omitted, this means the limit is not set.
+
+* upload | north | upload_bytes | north_bytes
+
+  **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  This sets the upload bytes in the time slice. *0* means delay forever.
+
+* download | south | download_bytes | south_bytes
+
+  **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  This set the max download bytes in the time slice. *0* means delay forever.
+
+The yaml value for *tcp_sock_speed_limit* can be in varies formats:
+
+* :ref:`humanize usize <conf_value_humanize_usize>`
+
+  This will set upload and download to the same value, with shift_millis set to 10.
+
+* map
+
+  The keys of this map are the fields as described above.
+
+.. _conf_value_udp_sock_speed_limit:
+
+udp socket speed limit
+======================
+
+**yaml value**: mix
+
+It consists of 4 fields:
+
+* shift_millis | shift
+
+  **type**: int
+
+  The time slice we use to count is *2 ^ N* milliseconds, where N is set by this key and should be in range [0 - 12].
+  If N is 10, and the time slice is 1024ms. If omitted, this means the limit is not set.
+
+* upload_bytes | north_bytes
+
+  **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  This sets the upload bytes in the time slice. *0* means no limit.
+
+* download_bytes | south_bytes
+
+  **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+  This set the max download bytes in the time slice. *0* means no limit.
+
+* upload_packets | north_packets
+
+  **type**: int [usize]
+
+  This set the max upload packets in the time slice. *0* means no limit.
+
+* download_packets | south_packets
+
+  **type**: int [usize]
+
+  This set the max download packets in the time slice. *0* means no limit.
+
+The yaml value for *udp_sock_speed_limit* can be in varies formats:
+
+* :ref:`humanize usize <conf_value_humanize_usize>`
+
+  This will set upload and download bytes to the same value, set shift_millis to 10 and disable check on packets.
+
+* map
+
+  The keys of this map are the fields as described above.
+
+.. _conf_value_request_limit:
+
+request limit
+=============
+
+**yaml value**: mix
+
+It consists of 2 fields:
+
+* shift_millis | shift
+
+  **type**: int
+
+  The time slice we use to count is *2 ^ N* milliseconds, where N is set by this key and should be in range [0 - 12].
+  If N is 10, and the time slice is 1024ms. If omitted, this means the limit is not set.
+
+* requests
+
+  **type**: usize
+
+  This sets the max requests in the time slice. 0 is not allowed.
+
+.. _conf_value_rate_limit_quota:
+
+rate limit quota
+================
+
+**yaml value**: mix
+
+It consists of 3 fields:
+
+* rate
+
+  **type**: :ref:`nonzero u32 <conf_value_nonzero_u32>`
+
+  If int or str without any unit, the default unit will be per second.
+
+  Supported units for str:
+
+    - /s, per second
+    - /m, per minute
+    - /h, per hour
+
+* replenish_interval
+
+  **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Construct a quota that replenishes one cell in a given interval. The default max_burst value is 1 is its not specified
+  along with this option.
+
+* max_burst
+
+  Adjusts the maximum burst size for a quota to construct a rate limiter with a capacity
+  for at most the given number of cells
+
+.. note:: *rate* and *replenish_interval* is conflict with each other, the latter one in conf will take effect.
+
+The yaml value for *u32 limit quota* can be in varies formats:
+
+* simple rate
+
+  Just the rate value. The max_burst value is the same as the one set in the rate.
+
+* map
+
+  The keys of this map are the fields as described above.
+
+.. _conf_value_random_ratio:
+
+random ratio
+============
+
+**yaml value**: f64 | str | bool | integer
+
+Set a random ratio between 0.0 and 1.0 (inclusive).
+
+For *str* value, it can be in fraction form (n/d), in percentage form (n%), or just a float string.
+
+For *bool* value, *false* means 0.0, *true* means 1.0.
+
+For *integer* value, only 0 and 1 is allowed.

g3proxy/doc/configuration/values/resolve.rst

@@ -0,0 +1,89 @@
+
+.. _configure_resolve_value_types:
+
+*******
+Resolve
+*******
+
+.. _conf_value_resolve_strategy:
+
+Resolve Strategy
+================
+
+**yaml value**: mix
+
+The *Resolve Strategy* config is not for resolvers, but for the users of resolvers.
+
+The type for this is *map*, which consists of keys as follows:
+
+query
+-----
+
+**optional**, **type**: enum str
+
+The query strategy, which will be used by the resolver while resolving.
+
+The value should be:
+
+* Ipv4Only
+* Ipv6Only
+* Ipv4First (default)
+* Ipv6First
+
+pick
+----
+
+**optional**, **type**: enum str
+
+The pick strategy, which will be used when selecting the best ip address from all the results.
+
+The value should be:
+
+* Random (default)
+* First
+
+.. _conf_value_resolve_redirection:
+
+Resolve Redirection
+===================
+
+**yaml value**: mix
+
+The *Resolve Redirection* config is not for resolvers, but for the users of resolvers.
+
+The type for this could be *seq*, which consists of many rules of type
+:ref:`resolve redirection rule <conf_value_resolve_redirection_rule>`.
+
+The type for this could also be *map*, in such case, each kv pair will be one rule,
+with the key as it's *exact* value, and the value as it's *to* value.
+
+.. _conf_value_resolve_redirection_rule:
+
+Resolve Redirection Rule
+------------------------
+
+Each rule should be a map with the following keys:
+
+* exact
+
+  **required**: false, **type**: :ref:`domain <conf_value_domain>`
+
+  Set the exact domain to replace.
+
+* parent
+
+  **required**: false, **type**: :ref:`domain <conf_value_domain>`
+
+  Set the parent domain to replace.
+
+* to
+
+  **required**: true, **type**: mix
+
+  Set the replacement value for the match.
+
+  For *exact* match, the value should be :ref:`host <conf_value_host>` or an array of ip addresses.
+
+  For *parent* match, the value should be :ref:`domain <conf_value_domain>`.
+
+Either *exact* or *parent* should be set for the rule.

g3proxy/doc/configuration/values/route.rst

@@ -0,0 +1,75 @@
+.. _configure_route_value_types:
+
+*****
+Route
+*****
+
+.. _conf_value_host_matched_object:
+
+Host Matched Object
+===================
+
+**yaml value**: map | seq of map
+
+This set a host based match object for generic type T, which will be specified in the real config options.
+
+The yaml value for T will be a map, but with the following keys as reserved as they are used by the match rules:
+
+* exact_match
+
+  **optional**, **type**: :ref:`host <conf_value_host>`
+
+  Match if this is the exact host.
+
+* child_match
+
+  **optional**, **type**: :ref:`domain <conf_value_domain>`
+
+  Match if the target host is a child domain of this parent domain.
+
+* set_default
+
+  **optional**, **type**: bool
+
+  If true, also set this T as default value
+
+  **default**: false
+
+If none of the above keys found, the parsed T value will also be used as the default value.
+
+A match object can contains one or more T(s), which means the yaml type for this object could be a single T,
+or a sequence of T.
+
+Only a single T is allowed for each match rules, including the default one.
+
+.. _conf_value_uri_path_matched_object:
+
+Uri Path Matched Object
+=======================
+
+**yaml value**: map | seq of map
+
+This set a url path based match object for generic type T, which will be specified in the real config options.
+
+The yaml value for T will be a map, but with the following keys as reserved as they are used by the match rules:
+
+* prefix_match
+
+  **optional**, **type**: str
+
+  Match if the target url path has this prefix.
+
+* set_default
+
+  **optional**, **type**: bool
+
+  If true, also set this T as default value
+
+  **default**: false
+
+If none of the above keys found, the parsed T value will also be used as the default value.
+
+A match object can contains one or more T(s), which means the yaml type for this object could be a single T,
+or a sequence of T.
+
+Only a single T is allowed for each match rules, including the default one.

g3proxy/doc/configuration/values/runtime.rst

@@ -0,0 +1,81 @@
+
+.. _configure_runtime_value_types:
+
+*******
+Runtime
+*******
+
+.. _conf_value_cpu_set:
+
+cpu set
+=======
+
+**yaml value** seq | usize
+
+A `CPU_SET(3)`_ for use with `sched_setaffinity(2)`_.
+
+The value should be a CPU ID, starting from 0, or a sequence of CPU IDs.
+
+.. _CPU_SET(3): https://man7.org/linux/man-pages/man3/CPU_SET.3.html
+.. _sched_setaffinity(2): https://man7.org/linux/man-pages/man2/sched_setaffinity.2.html
+
+.. versionadded:: 1.3.1
+
+.. _conf_value_unaided_runtime_config:
+
+unaided runtime config
+======================
+
+**yaml value**: map
+
+This is the config for unaided runtime.
+
+The keys are:
+
+thread_number
+-------------
+
+**optional**, **type**: usize
+
+Set the thread number.
+
+**default**: the number of logic CPU cores
+
+thread_stack_size
+-----------------
+
+**optional**, **type**: :ref:`humanize usize <conf_value_humanize_usize>`
+
+Set the stack size for worker threads. For *<int>* value, the unit is bytes.
+
+**default**: system default
+
+sched_affinity
+--------------
+
+**optional**, **type**: map | bool
+
+Set the sched affinity for each threads.
+
+For map value, the key should be the thread id starting from 0, and the value should be :ref:`cpu set <conf_value_cpu_set>`.
+
+For bool value:
+
+* if true, a default CPU SET will be set for each thread, the CPU ID in the set will match the thread ID.
+
+* if false, no sched affinity will be set, just as if this config option is not present.
+
+**default**: no sched affinity set
+
+.. versionadded:: 1.3.1
+
+max_io_events_per_tick
+----------------------
+
+**optional**, **type**: usize
+
+Configures the max number of events to be processed per tick.
+
+**default**: 1024, tokio default value
+
+.. versionadded:: 1.7.6

g3proxy/doc/configuration/values/tls.rst

@@ -0,0 +1,405 @@
+
+.. _configure_tls_value_types:
+
+***
+TLS
+***
+
+.. _conf_value_tls_name:
+
+tls name
+========
+
+**yaml type**: string
+
+Set the dns name / ip address for server certificate verification.
+If not set, the corresponding peer address will be used.
+
+.. note:: IP address is not supported by now
+
+.. _conf_value_tls_certificates:
+
+tls certificates
+================
+
+**yaml type**: :ref:`file <conf_value_file>` | seq
+
+Set the certificate file(s), which should be in PEM format(`openssl-req(1)`_).
+
+If relative, it will be searched in the directory that contains current config file.
+
+.. _openssl-req(1): https://www.openssl.org/docs/manmaster/man1/openssl-req.html
+
+.. _conf_value_tls_private_key:
+
+tls private_key
+===============
+
+**yaml type**: :ref:`file <conf_value_file>`
+
+Set the private key file, which should be in PKCS#8(`openssl-genpkey(1)`_) or traditional PEM format.
+
+If relative, it will be searched in the directory that contains current config file.
+The last one in the file will be used if many keys are found.
+
+.. versionchanged:: support traditional PEM format since version 1.3.2
+
+.. _openssl-genpkey(1): https://www.openssl.org/docs/manmaster/man1/openssl-genpkey.html
+
+.. _conf_value_tls_cert_pair:
+
+tls cert pair
+=============
+
+**yaml value**: map
+
+A pair value contains tls certificate and private key.
+
+The keys are:
+
+* certificate
+
+  **required**, **type**: :ref:`tls certificates <conf_value_tls_certificates>`
+
+  Set client certificates if client auth is needed by remote server.
+  Private key must also be set if client auth is needed.
+
+  **default**: not set
+
+* private_key
+
+  **required**, **type**: :ref:`tls private_key <conf_value_tls_private_key>`
+
+  Set the private key for client if client auth is needed by remote server.
+  Client certificates are also needed if client auth is needed.
+
+  **default**: not set
+
+.. versionadded:: 1.7.7
+
+.. _conf_value_openssl_protocol:
+
+openssl protocol
+================
+
+**yaml value**: string
+
+Set openssl protocol version.
+
+Current supported values are:
+
+- tls1.2
+- tls1.3
+
+.. versionadded:: 1.7.7
+
+.. _conf_value_openssl_ciphers:
+
+openssl ciphers
+===============
+
+**yaml value**: string or seq
+
+Set openssl cipher list or ciphersuites for the specified protocol.
+
+Values can be obtained from `openssl ciphers -v` command.
+
+For string value, it can be ciphers joined by ':'.
+
+For seq value, each one should be a cipher string.
+
+.. versionadded:: 1.7.7
+
+.. _conf_value_openssl_tls_client_config:
+
+openssl tls client config
+=========================
+
+**yaml value**: map
+
+The tls config to be used as a tls client.
+
+The map is consists of the following fields:
+
+* protocol
+
+  **optional**, **type**: :ref:`openssl protocol <conf_value_openssl_protocol>`
+
+  Set to use a specific protocol version.
+
+  **default**: not set
+
+  .. versionadded:: 1.7.7
+
+* ciphers
+
+  **optional**, **type**: :ref:`openssl ciphers <conf_value_openssl_ciphers>`
+  **require**: protocol
+
+  Set to use a specific set of ciphers for the specified protocol version.
+
+  **default**: not set
+
+  .. versionadded:: 1.7.7
+
+* disable_sni
+
+  **optional**, **type**: bool
+
+  Whether to send the Server Name Indication (SNI) extension during the client handshake.
+
+  **default**: false
+
+* cert_pair
+
+  **optional**, **type**: :ref:`tls cert pair <conf_value_tls_cert_pair>`
+  **conflict**: certificate, private_key
+
+  Set the client certificate and private key pair.
+
+  **default**: not set
+
+  .. versionadded:: 1.7.7
+
+* certificate
+
+  **optional**, **type**: :ref:`tls certificates <conf_value_tls_certificates>`
+  **conflict**: cert_pair
+
+  Set client certificates if client auth is needed by remote server.
+  Private key must also be set if client auth is needed.
+
+  **default**: not set
+
+* private_key
+
+  **optional**, **type**: :ref:`tls private_key <conf_value_tls_private_key>`
+  **conflict**: cert_pair
+
+  Set the private key for client if client auth is needed by remote server.
+  Client certificates are also needed if client auth is needed.
+
+  **default**: not set
+
+* ca_certificate | server_auth_certificate
+
+  **optional**, **type**: :ref:`tls certificates <conf_value_tls_certificates>`
+
+  A list of certificates for server auth. If not set, the system default ca certificates will be used.
+
+  **default**: not set
+
+* no_default_ca_certificate
+
+  **optional**, **type**: bool
+
+  Set if you don't want to load default ca certificates.
+
+  **default**: false
+
+* handshake_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the tls handshake timeout value.
+
+  **default**: 10s
+
+* no_session_cache
+
+  **optional**, **type**: bool
+
+  Set if you want to disable cache of TLS sessions.
+
+  **default**: false
+
+* use_builtin_session_cache
+
+  **optional**, **type**: bool
+
+  Set if we should use OpenSSL builtin session cache.
+
+  **default**: false
+
+  .. versionadded:: 1.7.7
+
+* session_cache_lru_max_sites
+
+  **optional**, **type**: usize
+
+  Set how many LRU sites should have cached sessions.
+
+  Only in use if this tls client is used by many sites.
+
+  **default**: 128
+
+* session_cache_each_capacity
+
+  **optional**, **type**: usize
+
+  Set how many sessions should be kept for each site.
+
+  **default**: 16
+
+.. versionadded:: 1.1.4
+
+.. _conf_value_rustls_client_config:
+
+rustls client config
+====================
+
+**yaml value**: map
+
+The tls config to be used as a tls client.
+
+The map is consists of the following fields:
+
+* no_session_cache
+
+  **optional**, **type**: bool
+
+  Set if you want to disable cache of TLS sessions.
+
+  **default**: false
+
+  .. versionadded:: 1.1.4
+
+* disable_sni
+
+  **optional**, **type**: bool
+
+  Whether to send the Server Name Indication (SNI) extension during the client handshake.
+
+  **default**: false
+
+  .. versionadded:: 1.1.4
+
+* max_fragment_size
+
+  **optional**, **type**: usize
+
+  Set the maximum size of TLS message we'll emit.
+
+  **default**: default value in tls driver
+
+* cert_pair
+
+  **optional**, **type**: :ref:`tls cert pair <conf_value_tls_cert_pair>`
+  **conflict**: certificate, private_key
+
+  Set the client certificate and private key pair.
+
+  **default**: not set
+
+  .. versionadded:: 1.7.8
+
+* certificate
+
+  **optional**, **type**: :ref:`tls certificates <conf_value_tls_certificates>`
+
+  Set client certificates if client auth is needed by remote server.
+  Private key must also be set if client auth is needed.
+
+  **default**: not set
+
+* private_key
+
+  **optional**, **type**: :ref:`tls private_key <conf_value_tls_private_key>`
+
+  Set the private key for client if client auth is needed by remote server.
+  Client certificates are also needed if client auth is needed.
+
+  **default**: not set
+
+* ca_certificate | server_auth_certificate
+
+  **optional**, **type**: :ref:`tls certificates <conf_value_tls_certificates>`
+
+  A list of certificates for server auth. If not set, the system default ca certificates will be used.
+
+  **default**: not set
+
+* no_default_ca_certificate
+
+  **optional**, **type**: bool
+
+  Set if you don't want to load default ca certificates.
+
+  **default**: false
+
+  .. versionadded:: 1.1.4
+
+* use_builtin_ca_certificate
+
+  **optional**, **type**: bool
+
+  Set to true if you want to use built in webpki-roots ca certificates as default ca certificates.
+
+  **default**: false
+
+* handshake_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the tls handshake timeout value.
+
+  **default**: 10s
+
+.. _conf_value_rustls_server_config:
+
+rustls server config
+====================
+
+**yaml value**: map
+
+The tls config to be used as a tls server.
+
+The map is consists of the following fields:
+
+* cert_pairs
+
+  **optional**, **type**: :ref:`tls cert pair <conf_value_tls_cert_pair>` or seq
+
+  Set certificate and private key pairs for this TLS server.
+
+  .. note:: At least set this or certificate & private_key.
+
+  .. versionadded:: 1.7.8
+
+* certificate
+
+  **optional**, **type**: :ref:`tls certificates <conf_value_tls_certificates>`
+
+  Set the certificates for this TLS server.
+
+  .. note:: At least set this or cert_pairs
+
+* private_key
+
+  **optional**, **type**: :ref:`tls private_key <conf_value_tls_private_key>`
+
+  Set the private key for this TLS server.
+
+  .. note:: At least set this or cert_pairs
+
+* enable_client_auth
+
+  **optional**, **type**: bool
+
+  Set if you want to enable client auth.
+
+* ca_certificate | client_auth_certificate
+
+  **optional**, **type**: :ref:`tls certificates <conf_value_tls_certificates>`
+
+  A list of certificates for client auth. If not set, the system default ca certificates will be used.
+
+  **default**: not set
+
+* handshake_timeout
+
+  **optional**, **type**: :ref:`humanize duration <conf_value_humanize_duration>`
+
+  Set the tls handshake timeout value.
+
+  **default**: 10s