mirror of
https://github.com/bytedance/g3.git
synced 2026-04-28 03:30:31 +00:00
update doc
This commit is contained in:
Zhang Jingqiang
Zhang Jingqiang
parent
301f6f2d82
commit
055195ab32
9 changed files with 217 additions and 207 deletions
|
|
@ -46,26 +46,26 @@ G3 プロジェクトは多数のアプリケーションで構成されてお
|
|||
- 豊富な監視メトリクス、入口/出口/ユーザー/ユーザーサイトレベルで
|
||||
- さまざまな観測ツールをサポート
|
||||
|
||||
詳しい機能紹介とユーザーガイドは[g3proxy](g3proxy/README.md)をご覧ください。
|
||||
[詳細な紹介](g3proxy/README.md) | [ユーザーガイド](g3proxy/UserGuide.en_US.md) |
|
||||
[リファレンスドキュメント](https://g3-project.readthedocs.io/projects/g3proxy/en/latest/)
|
||||
|
||||
Sphinx を使用して生成された g3proxy リファレンス
|
||||
ドキュメントは、[Read the Docs](https://g3-project.readthedocs.io/projects/g3proxy/en/latest/)
|
||||
でオンラインで参照できます。 詳細な設定形式、ログ形式、メトリクスの定義、メトリクスの定義などを含みます。
|
||||
#### g3statsd
|
||||
|
||||
StatsD互換の統計アグリゲータ。
|
||||
|
||||
[詳細な紹介](g3statsd/README.md) | [リファレンスドキュメント](https://g3-project.readthedocs.io/projects/g3statsd/en/latest/)
|
||||
|
||||
### g3tiles
|
||||
|
||||
作業中のリバースプロキシソリューション。
|
||||
|
||||
Sphinx を使用して生成された g3tiles リファレンス
|
||||
ドキュメントは、[Read the Docs](https://g3-project.readthedocs.io/projects/g3tiles/en/latest/)
|
||||
でオンラインで参照できます。
|
||||
詳細な設定形式、ログ形式、メトリクスの定義などを含みます。
|
||||
[リファレンスドキュメント](https://g3-project.readthedocs.io/projects/g3tiles/en/latest/)
|
||||
|
||||
### g3bench
|
||||
|
||||
HTTP 1.x、HTTP 2、HTTP 3、TLSハンドシェイク、DNS、Cloudflare Keylessをサポートするベンチマークツール。
|
||||
|
||||
詳細な紹介については、[g3bench](g3bench/README.md) を参照してください。
|
||||
[詳細な紹介](g3bench/README.md)
|
||||
|
||||
### g3mkcert
|
||||
|
||||
|
|
@ -83,6 +83,9 @@ g3proxyのGeoIPサポートのためのIPロケーションルックアップサ
|
|||
|
||||
Cloudflare keylessサーバーの簡単な実装。
|
||||
|
||||
[詳細な紹介](g3keymess/README.md) |
|
||||
[リファレンスドキュメント](https://g3-project.readthedocs.io/projects/g3keymess/en/latest/)
|
||||
|
||||
## 対応プラットフォーム
|
||||
|
||||
現在、完全にサポートされているのはLinuxのみです。コードはFreeBSD、NetBSD、macOS、Windowsでコンパイルされますが、そこでのテストは行っていません。
|
||||
|
|
@ -99,73 +102,7 @@ Cloudflare keylessサーバーの簡単な実装。
|
|||
|
||||
## リリースとパッケージング
|
||||
|
||||
各アプリの各リリースには *\<name\>-v\<version\>* の形式でタグが設定されます。
|
||||
これらのタグを使用してソースtarballを生成できます。
|
||||
また、配布準備が整った各アプリにはdebおよびrpmパッケージファイルが追加されています。
|
||||
|
||||
リリースビルドを行う場合:
|
||||
|
||||
1. リリースtarballを生成する
|
||||
|
||||
```shell
|
||||
# <name>-v<version> のタグがある場合
|
||||
./scripts/release/build_tarball.sh <name>-v<version>
|
||||
# 使用可能なタグがない場合、gitリビジョン（例: HEAD）を指定する必要があります
|
||||
./scripts/release/build_tarball.sh <name> <rev>
|
||||
```
|
||||
|
||||
すべてのベンダーソースはソースtarballに追加されるため、ソースtarballを保存し、コンパイラと依存関係がインストールされている任意の場所でオフラインでビルドできます。
|
||||
|
||||
2. パッケージをビルドする
|
||||
|
||||
debパッケージの場合:
|
||||
```shell
|
||||
tar xf <name>-<version>.tar.xz
|
||||
cd <name>-<version>
|
||||
./build_deb_from_tar.sh
|
||||
```
|
||||
|
||||
rpmパッケージの場合:
|
||||
```shell
|
||||
rpmbuild -ta ./<name>-<version>.tar.xz
|
||||
# 失敗した場合、次のコマンドを手動で実行できます:
|
||||
tar xvf <name>-<version>.tar.xz ./<name>-<version>/<name>.spec
|
||||
cp <name>-<version>.tar.xz ~/rpmbuild/SOURCES/
|
||||
rpmbuild -ba ./<name>-<version>/<name>.spec
|
||||
```
|
||||
|
||||
gitリポジトリから直接パッケージをビルドする場合:
|
||||
|
||||
- debパッケージの場合:
|
||||
|
||||
```shell
|
||||
./build_deb_from_git.sh <name>
|
||||
```
|
||||
|
||||
- rpmパッケージの場合:
|
||||
|
||||
```shell
|
||||
./build_rpm_from_git.sh <name>
|
||||
```
|
||||
|
||||
### 事前ビルドパッケージ
|
||||
|
||||
本番環境にインストールする場合は、自分でパッケージをビルドすることをお勧めします。
|
||||
|
||||
テスト目的の場合、いくつかのパッケージをビルドして
|
||||
[cloudsmith](https://cloudsmith.io/~g3-oqh/repos/) にアップロードしました。インストール手順はそこにあります。
|
||||
|
||||
### Dockerイメージのビルド
|
||||
|
||||
各アプリの *docker* フォルダーの下にDockerfile(s)があります。ビルドコマンドは次のようになります
|
||||
|
||||
```shell
|
||||
# ソースルートディレクトリで実行します
|
||||
docker build -f <app>/docker/debian.Dockerfile . -t <app>:<tag>
|
||||
# ソースコードなしでビルドします
|
||||
docker build -f <app>/docker/debian.Dockerfile github.com/bytedance/g3 -t <app>:<tag>
|
||||
# ソースtarballがある場合、そのtarballのURLも使用できます
|
||||
```
|
||||
[Packaging](doc/packaging.md) を参照してください。
|
||||
|
||||
### 静的リンク
|
||||
|
||||
|
|
|
|||
75
README.md
75
README.md
|
|
@ -50,6 +50,12 @@ as we have basic support built in.
|
|||
[README](g3proxy/README.md) | [User Guide](g3proxy/UserGuide.en_US.md) |
|
||||
[Reference Doc](https://g3-project.readthedocs.io/projects/g3proxy/en/latest/)
|
||||
|
||||
#### g3statsd
|
||||
|
||||
A StatsD compatible stats aggregator.
|
||||
|
||||
[README](g3statsd/README.md) | [Reference Doc](https://g3-project.readthedocs.io/projects/g3statsd/en/latest/)
|
||||
|
||||
### g3tiles
|
||||
|
||||
A work in progress reverse proxy solution.
|
||||
|
|
@ -98,74 +104,7 @@ Follow [Standards](doc/standards.md).
|
|||
|
||||
## Release and Packaging
|
||||
|
||||
We will set tags for each release of each application in the form *\<name\>-v\<version\>*.
|
||||
You can use these tags to generate source tarballs.
|
||||
And we have added deb and rpm package files for each application that is ready for distribution.
|
||||
|
||||
If you want to do a release build:
|
||||
|
||||
1. generate a release tarball
|
||||
|
||||
```shell
|
||||
# if we have a tag <name>-v<version>
|
||||
./scripts/release/build_tarball.sh <name>-v<version>
|
||||
# if no tags usable, you need to specify the git revision (e.g. HEAD)
|
||||
./scripts/release/build_tarball.sh <name> <rev>
|
||||
```
|
||||
|
||||
All vendor sources will be added to the source tarball, so you can save the source tarball and build it offline at
|
||||
anywhere that has the compiler and dependencies installed.
|
||||
|
||||
2. build the package
|
||||
|
||||
For deb package:
|
||||
```shell
|
||||
tar xf <name>-<version>.tar.xz
|
||||
cd <name>-<version>
|
||||
./build_deb_from_tar.sh
|
||||
```
|
||||
|
||||
For rpm package:
|
||||
```shell
|
||||
rpmbuild -ta ./<name>-<version>.tar.xz
|
||||
# if failed, you can run the following commands manually:
|
||||
tar xvf <name>-<version>.tar.xz ./<name>-<version>/<name>.spec
|
||||
cp <name>-<version>.tar.xz ~/rpmbuild/SOURCES/
|
||||
rpmbuild -ba ./<name>-<version>/<name>.spec
|
||||
```
|
||||
|
||||
If you want to build a package directly from the git repo:
|
||||
|
||||
- For deb package:
|
||||
|
||||
```shell
|
||||
./build_deb_from_git.sh <name>
|
||||
```
|
||||
|
||||
- For rpm package:
|
||||
|
||||
```shell
|
||||
./build_rpm_from_git.sh <name>
|
||||
```
|
||||
|
||||
### Pre-Built Packages
|
||||
|
||||
It is recommended to build packages yourself if you want to install them in a production environment.
|
||||
|
||||
For testing purpose, we have built and uploaded some packages to
|
||||
[cloudsmith](https://cloudsmith.io/~g3-oqh/repos/), you can find installation instructions there.
|
||||
|
||||
### Build Docker Image
|
||||
|
||||
You can find Dockerfile(s) under *docker* folder of each application. The build command will be like
|
||||
|
||||
```shell
|
||||
# run this in the source root dir
|
||||
docker build -f <app>/docker/debian.Dockerfile . -t <app>:<tag>
|
||||
# build without the source code
|
||||
docker build -f <app>/docker/debian.Dockerfile github.com/bytedance/g3 -t <app>:<tag>
|
||||
# if you have a source tarball, you can also use the URL of that tarball
|
||||
```
|
||||
See [Packaging](doc/packaging.md).
|
||||
|
||||
### Static Linking
|
||||
|
||||
|
|
|
|||
|
|
@ -47,6 +47,12 @@ G3项目包含许多应用，每一个应用程序单独一个子目录，包含
|
|||
[详细介绍](g3proxy/README.md) | [用户指南](g3proxy/UserGuide.zh_CN.md) |
|
||||
[参考文档](https://g3-project.readthedocs.io/projects/g3proxy/en/latest/)
|
||||
|
||||
#### g3statsd
|
||||
|
||||
StatsD兼容的监控打点指标聚合服务。
|
||||
|
||||
[详细介绍](g3statsd/README.md) | [参考文档](https://g3-project.readthedocs.io/projects/g3statsd/en/latest/)
|
||||
|
||||
### g3tiles
|
||||
|
||||
通用反向代理解决方案，开发中。
|
||||
|
|
@ -94,69 +100,7 @@ Cloudflare Keyless Server的简单实现。
|
|||
|
||||
## 发布及打包
|
||||
|
||||
每个应用程序的每个发布版本都会有对应的tag，格式为 *\<name\>-v\<version\>* 。
|
||||
使用对应的tag生成源码tar包，该tar包可以用于生成deb、rpm等发行版原生包文件。
|
||||
|
||||
如果需要对正式发布的版本打包:
|
||||
|
||||
1. 生成版本发布包
|
||||
|
||||
```shell
|
||||
./scripts/release/build_tarball.sh <name>-v<version>
|
||||
```
|
||||
|
||||
所有引用第三方源码都会放在tar包的vendor目录下，打包时只需要在目标机器上安装好编译器及系统依赖库即可，无需额外的网络连接。
|
||||
|
||||
2. 打包指令
|
||||
|
||||
deb包:
|
||||
```shell
|
||||
tar xf <name>-<version>.tar.xz
|
||||
cd <name>-<version>
|
||||
./build_deb_from_tar.sh
|
||||
```
|
||||
|
||||
rpm包:
|
||||
```shell
|
||||
rpmbuild -ta ./<name>-<version>.tar.xz
|
||||
# 如果失败，可以手动执行以下指令：
|
||||
tar xvf <name>-<version>.tar.xz ./<name>-<version>/<name>.spec
|
||||
cp <name>-<version>.tar.xz ~/rpmbuild/SOURCES/
|
||||
rpmbuild -ba ./<name>-<version>/<name>.spec
|
||||
```
|
||||
|
||||
如果需要直接从git打包:
|
||||
|
||||
- deb包:
|
||||
|
||||
```shell
|
||||
./build_deb_from_git.sh <name>
|
||||
```
|
||||
|
||||
- rpm包:
|
||||
|
||||
```shell
|
||||
./build_rpm_from_git.sh <name>
|
||||
```
|
||||
|
||||
### 预构建安装包
|
||||
|
||||
如需在生产环境使用，建议自行打包。
|
||||
|
||||
测试环境的话，部分包已经编译上传到
|
||||
[cloudsmith](https://cloudsmith.io/~g3-oqh/repos/), 可参考该链接页面的说明进行安装。
|
||||
|
||||
### 制作Docker镜像
|
||||
|
||||
每个应用的*docker*文件夹下有可参考的Dockerfile(s)，命令如下：
|
||||
|
||||
```shell
|
||||
# 在源码根目录可执行
|
||||
docker build -f <app>/docker/debian.Dockerfile . -t <app>:<tag>
|
||||
# 本地没有源码时，可用远程URL执行
|
||||
docker build -f <app>/docker/debian.Dockerfile github.com/bytedance/g3 -t <app>:<tag>
|
||||
# 如果已经制作了源码tar包，也可以把URL路径换成源码tar包路径
|
||||
```
|
||||
参考 [Packaging](doc/packaging.md)。
|
||||
|
||||
### 静态链接
|
||||
|
||||
|
|
|
|||
|
|
@ -3,6 +3,10 @@ Dev-Setup
|
|||
|
||||
# Toolchain
|
||||
|
||||
The minimum required rust version now is 1.86.
|
||||
|
||||
It is recommended to always using the lastest stable rust version.
|
||||
|
||||
## Install rustup
|
||||
|
||||
See [rustup.rs](https://rustup.rs/) to install **rustup**.
|
||||
|
|
|
|||
74
doc/packaging.md
Normal file
74
doc/packaging.md
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
# Release and Packaging
|
||||
|
||||
`deb` and `rpm` package files are provided by default for each application, you can use them to build binary packages to
|
||||
install on other machines.
|
||||
|
||||
## Package a tagged version
|
||||
|
||||
We set tags for each release of each application in the form *\<name\>-v\<version\>*.
|
||||
You can use these tags to generate source tarballs and then build packages from these tarballs.
|
||||
|
||||
1. generate a release tarball
|
||||
|
||||
```shell
|
||||
# if we have a tag <name>-v<version>
|
||||
./scripts/release/build_tarball.sh <name>-v<version>
|
||||
# if no tags usable, you need to specify the git revision (e.g. HEAD)
|
||||
./scripts/release/build_tarball.sh <name> <rev>
|
||||
```
|
||||
|
||||
All vendor sources will be added to the source tarball, so you can save the source tarball and build it offline at
|
||||
anywhere that has the compiler and dependencies installed.
|
||||
|
||||
2. build the package
|
||||
|
||||
For deb package:
|
||||
```shell
|
||||
tar xf <name>-<version>.tar.xz
|
||||
cd <name>-<version>
|
||||
./build_deb_from_tar.sh
|
||||
```
|
||||
|
||||
For rpm package:
|
||||
```shell
|
||||
rpmbuild -ta ./<name>-<version>.tar.xz
|
||||
# if failed, you can run the following commands manually:
|
||||
tar xvf <name>-<version>.tar.xz ./<name>-<version>/<name>.spec
|
||||
cp <name>-<version>.tar.xz ~/rpmbuild/SOURCES/
|
||||
rpmbuild -ba ./<name>-<version>/<name>.spec
|
||||
```
|
||||
|
||||
## Package a specific git commit
|
||||
|
||||
- Check out that git commit first
|
||||
|
||||
- For deb package:
|
||||
|
||||
```shell
|
||||
./build_deb_from_git.sh <name>
|
||||
```
|
||||
|
||||
- For rpm package:
|
||||
|
||||
```shell
|
||||
./build_rpm_from_git.sh <name>
|
||||
```
|
||||
|
||||
## Pre-Built Packages
|
||||
|
||||
It is recommended to build packages yourself if you want to install them in a production environment.
|
||||
|
||||
For testing purpose, we have built and uploaded some packages to
|
||||
[cloudsmith](https://cloudsmith.io/~g3-oqh/repos/), you can find installation instructions there.
|
||||
|
||||
## Build Docker Image
|
||||
|
||||
You can find Dockerfile(s) under *docker* folder of each application. The build command will be like
|
||||
|
||||
```shell
|
||||
# run this in the source root dir
|
||||
docker build -f <app>/docker/debian.Dockerfile . -t <app>:<tag>
|
||||
# build without the source code
|
||||
docker build -f <app>/docker/debian.Dockerfile github.com/bytedance/g3 -t <app>:<tag>
|
||||
# if you have a source tarball, you can also use the URL of that tarball
|
||||
```
|
||||
|
|
@ -4,6 +4,24 @@
|
|||
|
||||
g3keymess is a server implementation of Cloudflare Keyless protocol.
|
||||
|
||||
## How to build
|
||||
|
||||
You need to follow the [dev-setup](../doc/dev-setup.md) guide to set up your build environment first.
|
||||
|
||||
To build debug binaries:
|
||||
|
||||
```shell
|
||||
cargo build -p g3keymess -p g3keymess-ctl
|
||||
```
|
||||
|
||||
To build release binaries:
|
||||
|
||||
```shell
|
||||
cargo build --profile release-lto -p g3keymess -p g3keymess-ctl
|
||||
```
|
||||
|
||||
See [Packaging](../doc/packaging.md) if you want to build binary packages or docker images.
|
||||
|
||||
## Features
|
||||
|
||||
g3keymess dynamically links to libcrypto on the system as the crypto engine by default.
|
||||
|
|
|
|||
|
|
@ -11,6 +11,8 @@ tcp streaming / tls streaming / transparent proxy / reverse proxy.
|
|||
|
||||
## How to build
|
||||
|
||||
You need to follow the [dev-setup](../doc/dev-setup.md) guide to set up your build environment first.
|
||||
|
||||
To build debug binaries:
|
||||
|
||||
```shell
|
||||
|
|
@ -23,6 +25,8 @@ To build release binaries:
|
|||
cargo build --profile release-lto -p g3proxy -p g3proxy-ctl
|
||||
```
|
||||
|
||||
See [Packaging](../doc/packaging.md) if you want to build binary packages or docker images.
|
||||
|
||||
## Documents
|
||||
|
||||
You can view the reference documentation generated by sphinx online at
|
||||
|
|
|
|||
88
g3statsd/README.md
Normal file
88
g3statsd/README.md
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
[](https://g3-project.readthedocs.io/projects/g3statsd/)
|
||||
|
||||
# g3statsd
|
||||
|
||||
g3statsd is [statsd](https://github.com/statsd/statsd)-compatible stats aggregator.
|
||||
|
||||
It is developed to meet the needs in G3 project as all applications use StatsD as metrics sending protocol.
|
||||
|
||||
The features make it different from other statsd server implementations are:
|
||||
|
||||
- written in async rust, which make it efficient and safe
|
||||
- compatible with [DogStatsD](https://docs.datadoghq.com/developers/dogstatsd/datagram_shell/) protocol, tags supported
|
||||
- each exporter has its own emit interval
|
||||
- can aggregate gauge metric values when dropping tags
|
||||
|
||||
There are still many features missing as the current focus is our internal usage, feel free to submit feature request
|
||||
issues. PRs are also welcomed.
|
||||
|
||||
## Building
|
||||
|
||||
You need to follow the [dev-setup](../doc/dev-setup.md) guide to set up your build environment first.
|
||||
|
||||
To build debug binaries:
|
||||
|
||||
```shell
|
||||
cargo build -p g3statsd -p g3statsd-ctl
|
||||
```
|
||||
|
||||
To build release binaries:
|
||||
|
||||
```shell
|
||||
cargo build --profile release-lto -p g3statsd -p g3statsd-ctl
|
||||
```
|
||||
|
||||
See [Packaging](../doc/packaging.md) if you want to build binary packages or docker images.
|
||||
|
||||
## Supported Metric Types
|
||||
|
||||
- c - COUNT
|
||||
- g - GAUGE
|
||||
- h - HISTOGRAM (unsupported yet)
|
||||
- ms - TIMER (unsupported yet)
|
||||
|
||||
## Supported Importers
|
||||
|
||||
- statsd
|
||||
|
||||
Accept StatsD metrics, and send them to collectors.
|
||||
|
||||
Only UDP is supported at this time.
|
||||
|
||||
## Supported Collectors
|
||||
|
||||
- aggregate
|
||||
|
||||
Aggregate received metrics and send them to exporters.
|
||||
|
||||
You can set `join_tags` in this collector to join metrics when drop tags.
|
||||
|
||||
- regulate
|
||||
|
||||
Make some changes to the received metrics and send directly to exporters.
|
||||
|
||||
The supported actions are:
|
||||
|
||||
* prefix - add a common name prefix to all metrics
|
||||
* drop_tags - drop tags for all metrics
|
||||
|
||||
## Supported Exporters
|
||||
|
||||
| Exporter | Introduction | Aggregate | Global prefix and tags |
|
||||
|-------------|-------------------------------------------------------|-----------|------------------------|
|
||||
| console | Log all metrics to stdout | no | no |
|
||||
| discard | Discard all metrics | no | no |
|
||||
| memory | Store all metrics values in memory | no | no |
|
||||
| graphite | Emit to graphite by using the plaintext protocol | yes | yes |
|
||||
| opentsdb | Emit to OpenTSDB by using the /api/put API | yes | yes |
|
||||
| influxdb_v2 | Emit to InfluxDB v2 by using the /api/v2/write API | yes | yes |
|
||||
| influxdb_v3 | Emit to InfluxDB v3 by using the /api/v3/write_lp API | yes | yes |
|
||||
|
||||
## Documents
|
||||
|
||||
You can view the reference documentation generated by sphinx online at
|
||||
[Read the Docs](https://g3-project.readthedocs.io/projects/g3statsd/en/latest/).
|
||||
|
||||
## Examples
|
||||
|
||||
You can find example config in the [examples](examples) directory.
|
||||
|
|
@ -8,11 +8,13 @@ You can visit the `code repo`_ to get more detailed introduction.
|
|||
Follow the links below to see documentations for each application:
|
||||
|
||||
- `g3proxy`_
|
||||
- `g3statsd`_
|
||||
- `g3tiles`_
|
||||
- `g3keymess`_
|
||||
|
||||
.. _code repo: https://github.com/bytedance/g3
|
||||
|
||||
.. _g3proxy: /projects/g3proxy/en/latest/
|
||||
.. _g3statsd: /projects/g3statsd/en/latest/
|
||||
.. _g3tiles: /projects/g3tiles/en/latest/
|
||||
.. _g3keymess: /projects/g3keymess/en/latest/
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue