feat(telemetry): define HTTP OTLP endpoint behavior and signal routing (#3779)

* feat(telemetry): define HTTP OTLP endpoint behavior and signal routing - Add resolveHttpOtlpUrl() that appends /v1/traces, /v1/logs, /v1/metrics to base HTTP OTLP endpoints per the OpenTelemetry specification - Add per-signal endpoint overrides (otlpTracesEndpoint, otlpLogsEndpoint, otlpMetricsEndpoint) for backends with non-standard paths (e.g. Alibaba Cloud) - Add LogToSpanProcessor that bridges OTel log records to spans for traces-only backends, with session-based traceId correlation and error status propagation - Auto-wire LogToSpanProcessor when traces URL exists but logs URL doesn't - Validate per-signal URLs gracefully (log error + skip, don't crash) - Preserve query strings when appending signal paths to URLs - Guard gRPC branch against missing base endpoint with per-signal config - Update telemetry documentation with signal routing semantics and Alibaba Cloud HTTP per-signal endpoint examples Closes #3734 Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(telemetry): fix TS noPropertyAccessFromIndexSignature errors in tests Use typed ExportedSpan interface and bracket notation for index signature properties to satisfy strict TypeScript checks in CI. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(telemetry): replace MD5 with SHA-256 for traceId derivation CodeQL flagged MD5 as a weak cryptographic algorithm when used with session.id (considered sensitive data). Switch to SHA-256 truncated to 32 hex chars to satisfy CodeQL while maintaining the same traceId format required by the OTel specification. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(telemetry): address review feedback for LogToSpanProcessor robustness - Wrap JSON.stringify in try/catch to handle circular refs and BigInt - Add export timeout (30s) and try/catch to prevent hung shutdown - Track in-flight exports to avoid interval-vs-shutdown race condition - Fix deriveSpanStatus: use truthy checks (!!), drop success===false heuristic since declined tool calls are normal, not errors - Enforce http(s) scheme in validateUrl to reject file:/javascript: URLs - Change DiagLogLevel from ERROR to WARN to preserve operational diagnostics - Preserve logRecord.instrumentationScope instead of hardcoding - Forward severityNumber/severityText as span attributes - Add tests for circular refs, error status edge cases, severity Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(telemetry): flush sdk shutdown through cleanup Remove async process exit handlers from telemetry initialization and route SDK shutdown through Config cleanup so normal CLI exit paths await pending telemetry exports. Keep shutdown idempotent while an SDK shutdown is in flight. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(telemetry): harden bridged log shutdown Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(telemetry): address review follow-ups Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> --------- Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-05-04 06:30:53 +00:00 · 2026-05-01 22:47:01 +08:00 · 2026-05-01 22:47:01 +08:00 · 5d1052a358
commit 5d1052a358
parent 35fe97e0f6
11 changed files with 1387 additions and 102 deletions
--- a/docs/developers/development/telemetry.md
+++ b/docs/developers/development/telemetry.md
@ -58,20 +58,36 @@ observability framework — Qwen Code's observability system provides:
 All telemetry behavior is controlled through your `.qwen/settings.json` file.
 These settings can be overridden by environment variables or CLI flags.

-| Setting        | Environment Variable           | CLI Flag                                                 | Description                                       | Values            | Default                 |
-| -------------- | ------------------------------ | -------------------------------------------------------- | ------------------------------------------------- | ----------------- | ----------------------- |
-| `enabled`      | `QWEN_TELEMETRY_ENABLED`       | `--telemetry` / `--no-telemetry`                         | Enable or disable telemetry                       | `true`/`false`    | `false`                 |
-| `target`       | `QWEN_TELEMETRY_TARGET`        | `--telemetry-target <local\|gcp>`                        | Where to send telemetry data                      | `"gcp"`/`"local"` | `"local"`               |
-| `otlpEndpoint` | `QWEN_TELEMETRY_OTLP_ENDPOINT` | `--telemetry-otlp-endpoint <URL>`                        | OTLP collector endpoint                           | URL string        | `http://localhost:4317` |
-| `otlpProtocol` | `QWEN_TELEMETRY_OTLP_PROTOCOL` | `--telemetry-otlp-protocol <grpc\|http>`                 | OTLP transport protocol                           | `"grpc"`/`"http"` | `"grpc"`                |
-| `outfile`      | `QWEN_TELEMETRY_OUTFILE`       | `--telemetry-outfile <path>`                             | Save telemetry to file (overrides `otlpEndpoint`) | file path         | -                       |
-| `logPrompts`   | `QWEN_TELEMETRY_LOG_PROMPTS`   | `--telemetry-log-prompts` / `--no-telemetry-log-prompts` | Include prompts in telemetry logs                 | `true`/`false`    | `true`                  |
-| `useCollector` | `QWEN_TELEMETRY_USE_COLLECTOR` | -                                                        | Use external OTLP collector (advanced)            | `true`/`false`    | `false`                 |
+| Setting               | Environment Variable                   | CLI Flag                                                 | Description                                          | Values            | Default                 |
+| --------------------- | -------------------------------------- | -------------------------------------------------------- | ---------------------------------------------------- | ----------------- | ----------------------- |
+| `enabled`             | `QWEN_TELEMETRY_ENABLED`               | `--telemetry` / `--no-telemetry`                         | Enable or disable telemetry                          | `true`/`false`    | `false`                 |
+| `target`              | `QWEN_TELEMETRY_TARGET`                | `--telemetry-target <local\|gcp>`                        | Where to send telemetry data                         | `"gcp"`/`"local"` | `"local"`               |
+| `otlpEndpoint`        | `QWEN_TELEMETRY_OTLP_ENDPOINT`         | `--telemetry-otlp-endpoint <URL>`                        | OTLP collector endpoint                              | URL string        | `http://localhost:4317` |
+| `otlpProtocol`        | `QWEN_TELEMETRY_OTLP_PROTOCOL`         | `--telemetry-otlp-protocol <grpc\|http>`                 | OTLP transport protocol                              | `"grpc"`/`"http"` | `"grpc"`                |
+| `otlpTracesEndpoint`  | `QWEN_TELEMETRY_OTLP_TRACES_ENDPOINT`  | -                                                        | Per-signal endpoint override for traces (HTTP only)  | URL string        | -                       |
+| `otlpLogsEndpoint`    | `QWEN_TELEMETRY_OTLP_LOGS_ENDPOINT`    | -                                                        | Per-signal endpoint override for logs (HTTP only)    | URL string        | -                       |
+| `otlpMetricsEndpoint` | `QWEN_TELEMETRY_OTLP_METRICS_ENDPOINT` | -                                                        | Per-signal endpoint override for metrics (HTTP only) | URL string        | -                       |
+| `outfile`             | `QWEN_TELEMETRY_OUTFILE`               | `--telemetry-outfile <path>`                             | Save telemetry to file (overrides `otlpEndpoint`)    | file path         | -                       |
+| `logPrompts`          | `QWEN_TELEMETRY_LOG_PROMPTS`           | `--telemetry-log-prompts` / `--no-telemetry-log-prompts` | Include prompts in telemetry logs                    | `true`/`false`    | `true`                  |
+| `useCollector`        | `QWEN_TELEMETRY_USE_COLLECTOR`         | -                                                        | Use external OTLP collector (advanced)               | `true`/`false`    | `false`                 |

 **Note on boolean environment variables:** For the boolean settings (`enabled`,
 `logPrompts`, `useCollector`), setting the corresponding environment variable to
 `true` or `1` will enable the feature. Any other value will disable it.

+**HTTP OTLP signal routing:** When using HTTP protocol (`otlpProtocol: "http"`),
+Qwen Code automatically appends signal-specific paths (`/v1/traces`, `/v1/logs`,
+`/v1/metrics`) to the base `otlpEndpoint`. For example, `http://collector:4318`
+becomes `http://collector:4318/v1/traces` for traces. If the URL already ends
+with a signal path, it is used as-is. Per-signal endpoint overrides
+(`otlpTracesEndpoint`, etc.) take precedence over the base endpoint and are used
+verbatim. gRPC protocol uses service-based routing and does not append paths.
+
+The per-signal endpoint environment variables also accept the standard
+OpenTelemetry names: `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`,
+`OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`, `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT`.
+The `QWEN_TELEMETRY_OTLP_*` variants take precedence over the `OTEL_*` variants.
+
 For detailed information about all configuration options, see the
 [Configuration Guide](./cli/configuration.md).

@ -91,6 +107,9 @@ sent to Alibaba Cloud.

 1. Enable telemetry in your `.qwen/settings.json` and set the OTLP
   endpoint:
+
+   **Option A: gRPC protocol** (standard OTLP endpoint):
+
   ```json
   {
     "telemetry": {
@ -101,6 +120,29 @@ sent to Alibaba Cloud.
     }
   }
   ```
+
+   **Option B: HTTP protocol with per-signal endpoints** (for backends
+   that use non-standard paths, e.g., `/api/otlp/traces` instead of
+   `/v1/traces`):
+
+   ```json
+   {
+     "telemetry": {
+       "enabled": true,
+       "otlpProtocol": "http",
+       "otlpTracesEndpoint": "http://<host>/<token>/api/otlp/traces",
+       "otlpLogsEndpoint": "http://<host>/<token>/api/otlp/logs",
+       "otlpMetricsEndpoint": "http://<host>/<token>/api/otlp/metrics"
+     }
+   }
+   ```
+
+   > **Note:** When using HTTP protocol with only `otlpEndpoint` (no
+   > per-signal overrides), Qwen Code appends standard OTLP paths
+   > (`/v1/traces`, `/v1/logs`, `/v1/metrics`) to the base URL. If your
+   > backend uses different paths, use per-signal endpoint overrides as
+   > shown in Option B.
+
 2. If your Alibaba Cloud endpoint requires authentication, provide OTLP
   headers through standard OpenTelemetry environment variables such as
   `OTEL_EXPORTER_OTLP_HEADERS` (or the signal-specific variants). Qwen