initial commit

g3proxy/doc/log/task/ftp_over_http.rst

@@ -0,0 +1,187 @@
+.. _log_task_ftp_over_http:
+
+*************
+FTP Over HTTP
+*************
+
+The following keys are available for TcpConnect task log:
+
+server_addr
+-----------
+
+**required**, **type**: socket address string
+
+The listening address of the server.
+
+client_addr
+-----------
+
+**required**, **type**: socket address string
+
+The client address.
+
+upstream
+--------
+
+**required**, **type**: domain:port | socket address string
+
+The target upstream that the client want to access.
+
+next_bind_ip
+------------
+
+**optional**, **type**: ip address string
+
+The selected bind IP before we really connect to the remote peer.
+
+Present only if bind ip config is enabled on the corresponding escaper.
+
+next_expire
+-----------
+
+**optional**, **type**: rfc3339 timestamp string with microseconds
+
+The expected expire time of the next peer.
+
+Present only if the next escaper is dynamic and we have selected the remote peer.
+
+ftp_c_bound_addr
+----------------
+
+**optional**, **type**: socket address string
+
+The local address for the remote ftp control connection.
+
+Present only if we have connected to the remote peer.
+
+ftp_c_peer_addr
+---------------
+
+**optional**, **type**: socket address string
+
+The peer address for the remote ftp control connection.
+
+The peer may be the upstream, or will be a next proxy address, which depends on the type of escaper.
+
+Present only if we have selected the ip address of the next peer.
+
+ftp_c_connect_tries
+-------------------
+
+**optional**, **type**: int
+
+How many times we have tried to connect to the remote peer to establish the ftp control connection.
+
+ftp_c_connect_spend
+-------------------
+
+**optional**, **type**: time duration string
+
+How many time we have spent during the ftp control connection of the remote peer (all tries count in).
+
+ftp_d_bound_addr
+----------------
+
+**optional**, **type**: socket address string
+
+The local address for the remote ftp data connection.
+
+Present only if we have connected to the remote peer.
+
+ftp_d_peer_addr
+---------------
+
+**optional**, **type**: socket address string
+
+The peer address for the remote ftp data connection.
+
+The peer may be the upstream, or will be a next proxy address, which depends on the type of escaper.
+
+Present only if we have selected the ip address of the next peer.
+
+ftp_d_connect_tries
+-------------------
+
+**optional**, **type**: int
+
+How many times we have tried to connect to the remote peer to establish the ftp data connection.
+
+ftp_d_connect_spend
+-------------------
+
+**optional**, **type**: time duration string
+
+How many time we have spent during the ftp data connection of the remote peer (all tries count in).
+
+method
+------
+
+**required**, **type**: http method string
+
+Show the http method string of the client request.
+
+uri
+---
+
+**required**, **type**: http uri string
+
+Show the uri of the client request. All non-printable characters will be escaped.
+
+The max allowed number of characters of the uri is configurable at
+:ref:`server <config_server_http_proxy_log_uri_max_chars>` or :ref:`user <config_user_log_uri_max_chars>` level.
+
+user_agent
+----------
+
+**optional**, **type**: string
+
+Show the first User-Agent header value in the client request.
+
+rsp_status
+----------
+
+**optional**, **type**: int
+
+Show the status code in the response that we send to the client.
+
+c_rd_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have received from client.
+
+c_wr_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have sent to client.
+
+ftp_c_rd_bytes
+--------------
+
+**optional**, **type**: int
+
+How many bytes we have received from the remote peer through the ftp control connection.
+
+ftp_c_wr_bytes
+--------------
+
+**optional**, **type**: int
+
+How many bytes we have sent to the remote peer through the ftp control connection.
+
+ftp_d_rd_bytes
+--------------
+
+**optional**, **type**: int
+
+How many bytes we have received from the remote peer through the ftp data connection.
+
+ftp_d_wr_bytes
+--------------
+
+**optional**, **type**: int
+
+How many bytes we have sent to the remote peer through the ftp data connection.

g3proxy/doc/log/task/http_forward.rst

@@ -0,0 +1,90 @@
+.. _log_task_http_forward:
+
+************
+Http Forward
+************
+
+All config keys for TcpConnect task log also applies to HttpForward task log,
+see :ref:`TcpConnect <log_task_tcp_connect>` for details.
+
+The following keys are available only for HttpForward task log:
+
+pipeline_wait
+-------------
+
+**required**, **type**: time duration string
+
+Show the time spent from the receive of the http request header to the creation of the task.
+
+reuse_connection
+----------------
+
+**required**, **type**: bool
+
+Show if this task reuse old remote connection.
+
+method
+------
+
+**required**, **type**: http method string
+
+Show the http method string of the client request.
+
+uri
+---
+
+**required**, **type**: http uri string
+
+Show the uri of the client request. All non-printable characters will be escaped.
+
+The max allowed number of characters of the uri is configurable at
+:ref:`server <config_server_http_proxy_log_uri_max_chars>` or :ref:`user <config_user_log_uri_max_chars>` level.
+
+user_agent
+----------
+
+**optional**, **type**: string
+
+Show the first User-Agent header value in the client request.
+
+rsp_status
+----------
+
+**optional**, **type**: int
+
+Show the status code in the response that we send to the client.
+
+origin_status
+-------------
+
+**optional**, **type**: int
+
+Show the status code in the response we receive from the remote peer.
+
+dur_req_send_hdr
+----------------
+
+**optional**, **type**: time duration string
+
+Show the time spent from the creation of the task to when we sent out the request header to the remote peer.
+
+dur_req_send_all
+----------------
+
+**optional**, **type**: time duration string
+
+Show the time spent from the creation of the task to when we sent out the total request to the remote peer.
+
+dur_rsp_recv_hdr
+----------------
+
+**optional**, **type**: time duration string
+
+Show the time spent from the creation of the task to when we received the response header from the remote peer.
+
+dur_rsp_recv_all
+----------------
+
+**optional**, **type**: time duration string
+
+Show the time spent from the creation of the task to when we received the total response from the remote peer.

g3proxy/doc/log/task/index.rst

@@ -0,0 +1,152 @@
+.. _log_task:
+
+********
+Task Log
+********
+
+Each valid request will be a task. Each task will generate one log when finished.
+
+Shared Keys
+===========
+
+The following shared keys are set in all type of task logs:
+
+server_type
+-----------
+
+**required**, **type**: enum string
+
+The type of the server that accepted the request.
+
+server_name
+-----------
+
+**required**, **type**: string
+
+The name of the server that accepted the request.
+
+task_type
+---------
+
+**required**, **type**: enum string
+
+The subtype of this task log. The meaning of non-shared keys are depend on this value.
+
+task_id
+-------
+
+**required**, **type**: uuid in simple string format
+
+UUID of the task.
+
+The *task_id* will appear in other logs such as escape log if they have any association with this task.
+
+stage
+-----
+
+**required**, **type**: enum string
+
+The stage of the task.
+
+The values available for each task depend on the server protocol. Here is all values:
+
+* Created
+
+  The task has just been created.
+
+* Preparing
+
+  We are preparing internal resources.
+
+* Connecting
+
+  We are trying to connect to remote peer.
+
+* Connected
+
+  We have just connected to remote peer.
+
+* Replying
+
+  We are trying to reply to clients that we have connected to remote peer.
+
+* LoggedIn
+
+  The upstream needs login and we have logged in.
+
+* Relaying
+
+  Both client and remote channel established, we are relaying data now.
+
+* Finished
+
+  The task has finished with no error. Only available for layer 7 protocols.
+
+start_at
+--------
+
+**required**, **type**: rfc3339 timestamp string with microseconds
+
+The time that the task is created (after validation).
+
+.. note:: Not every request will be a task, only the valid ones.
+
+user
+----
+
+**optional**, **type**: string
+
+The username. Set only if user auth is enabled on server.
+
+escaper
+-------
+
+**optional**, **type**: string
+
+The selected escaper name.
+
+reason
+------
+
+**required**, **type**: enum string
+
+The brief reason why the task ends.
+
+See the definition of **ServerTaskError** in code file *src/serve/error.rs*.
+
+wait_time
+---------
+
+**optional**, **type**: time duration string
+
+Show how many time spent from the acceptation of the request to the creation of the task.
+
+For requests that reuse old connection, the start time will be the time we start to polling the next request,
+so you may see very large wait_time in logs. This behaviour may change in future.
+
+ready_time
+----------
+
+**optional**, **type**: time duration string
+
+Show how many time spent from the creation of the task to the relaying stage, which means both the client channel
+and the remote channel have been established. The value may be empty if the task failed early.
+
+total_time
+----------
+
+**required**, **type**: time duration string
+
+Show the time from the creation of the task to the end of the task.
+
+Sub Types
+=========
+
+.. toctree::
+   :maxdepth: 2
+
+   tcp_connect
+   http_forward
+   ftp_over_http
+   udp_associate
+   udp_connect

g3proxy/doc/log/task/tcp_connect.rst

@@ -0,0 +1,108 @@
+.. _log_task_tcp_connect:
+
+***********
+Tcp Connect
+***********
+
+The following keys are available for TcpConnect task log:
+
+server_addr
+-----------
+
+**required**, **type**: socket address string
+
+The listening address of the server.
+
+client_addr
+-----------
+
+**required**, **type**: socket address string
+
+The client address.
+
+upstream
+--------
+
+**required**, **type**: domain:port | socket address string
+
+The target upstream that the client want to access.
+
+next_bind_ip
+------------
+
+**optional**, **type**: ip address string
+
+The selected bind IP before we really connect to the remote peer.
+
+Present only if bind ip config is enabled on the corresponding escaper.
+
+next_bound_addr
+---------------
+
+**optional**, **type**: socket address string
+
+The local address for the remote connection.
+
+Present only if we have connected to the remote peer.
+
+next_peer_addr
+--------------
+
+**optional**, **type**: socket address string
+
+The peer address for the remote connection.
+
+The peer may be the upstream, or will be a next proxy address, which depends on the type of escaper.
+
+Present only if we have selected the ip address of the next peer.
+
+next_expire
+-----------
+
+**optional**, **type**: rfc3339 timestamp string with microseconds
+
+The expected expire time of the next peer.
+
+Present only if the next escaper is dynamic and we have selected the remote peer.
+
+tcp_connect_tries
+-----------------
+
+**optional**, **type**: int
+
+How many times we have tried to connect to the remote peer.
+
+tcp_connect_spend
+-----------------
+
+**optional**, **type**: time duration string
+
+How many time we have spent during connection of the remote peer (all tries count in).
+
+c_rd_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have received from client.
+
+c_wr_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have sent to client.
+
+r_rd_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have received from the remote peer.
+
+r_wr_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have sent to the remote peer.

g3proxy/doc/log/task/udp_associate.rst

@@ -0,0 +1,98 @@
+.. _log_task_udp_associate:
+
+*************
+Udp Associate
+*************
+
+The following keys are available for UdpAssociate task log:
+
+tcp_server_addr
+---------------
+
+**required**, **type**: socket address string
+
+The server address for the tcp control connection.
+
+tcp_client_addr
+---------------
+
+**required**, **type**: socket address string
+
+The client address for the tcp control connection.
+
+udp_server_addr
+---------------
+
+**optional**, **type**: socket address string
+
+The server address for the udp data connection.
+
+udp_client_addr
+---------------
+
+**optional**, **type**: socket address string
+
+The client address for the udp data connection.
+
+initial_peer
+------------
+
+**optional**, **type**: socket address string
+
+The target peer address in the first udp packet.
+
+c_rd_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have received from client.
+
+c_rd_packets
+------------
+
+**optional**, **type**: int
+
+How many packets we have received from client.
+
+c_wr_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have sent to client.
+
+c_wr_packets
+------------
+
+**optional**, **type**: int
+
+How many packets we have sent to client.
+
+r_rd_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have received from the remote peer.
+
+r_rd_packets
+------------
+
+**optional**, **type**: int
+
+How many packets we have received from the remote peer.
+
+r_wr_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have sent to the remote peer.
+
+r_wr_packets
+------------
+
+**optional**, **type**: int
+
+How many packets we have sent to the remote peer.

g3proxy/doc/log/task/udp_connect.rst

@@ -0,0 +1,132 @@
+.. _log_task_udp_connect:
+
+*************
+Udp Associate
+*************
+
+The following keys are available for UdpAssociate task log:
+
+tcp_server_addr
+---------------
+
+**required**, **type**: socket address string
+
+The server address for the tcp control connection.
+
+tcp_client_addr
+---------------
+
+**required**, **type**: socket address string
+
+The client address for the tcp control connection.
+
+udp_server_addr
+---------------
+
+**optional**, **type**: socket address string
+
+The server address for the udp data connection.
+
+udp_client_addr
+---------------
+
+**optional**, **type**: socket address string
+
+The client address for the udp data connection.
+
+upstream
+--------
+
+**required**, **type**: domain:port | socket address string
+
+The target upstream that the client want to access.
+
+next_bind_ip
+------------
+
+**optional**, **type**: ip address string
+
+The selected bind IP before we really setup the remote side udp socket.
+
+Present only if bind ip config is enabled on the corresponding escaper.
+
+next_bound_addr
+---------------
+
+**optional**, **type**: socket address string
+
+The local address for the remote udp socket.
+
+next_peer_addr
+--------------
+
+**optional**, **type**: socket address string
+
+The peer address for the remote udp socket.
+
+The peer may be the upstream, or will be a next proxy address, which depends on the type of escaper.
+
+next_expire
+-----------
+
+**optional**, **type**: rfc3339 timestamp string with microseconds
+
+The expected expire time of the next peer.
+
+Present only if the next escaper is dynamic and we have selected the remote peer.
+
+c_rd_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have received from client.
+
+c_rd_packets
+------------
+
+**optional**, **type**: int
+
+How many packets we have received from client.
+
+c_wr_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have sent to client.
+
+c_wr_packets
+------------
+
+**optional**, **type**: int
+
+How many packets we have sent to client.
+
+r_rd_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have received from the remote peer.
+
+r_rd_packets
+------------
+
+**optional**, **type**: int
+
+How many packets we have received from the remote peer.
+
+r_wr_bytes
+----------
+
+**optional**, **type**: int
+
+How many bytes we have sent to the remote peer.
+
+r_wr_packets
+------------
+
+**optional**, **type**: int
+
+How many packets we have sent to the remote peer.