qwen-code/docs/developers/development/telemetry.md

Observability with OpenTelemetry

Learn how to enable and setup OpenTelemetry for Qwen Code.

• Observability with OpenTelemetry
  • Key Benefits
  • OpenTelemetry Integration
  • Configuration
  • Aliyun Telemetry
    • Manual OTLP Export
  • Local Telemetry
    • File-based Output (Recommended)
    • Collector-Based Export (Advanced)
  • Logs and Metrics
    • Logs
    • Metrics

Key Benefits

• 🔍 Usage Analytics: Understand interaction patterns and feature adoption across your team
• ⚡ Performance Monitoring: Track response times, token consumption, and resource utilization
• 🐛 Real-time Debugging: Identify bottlenecks, failures, and error patterns as they occur
• 📊 Workflow Optimization: Make informed decisions to improve configurations and processes
• 🏢 Enterprise Governance: Monitor usage across teams, track costs, ensure compliance, and integrate with existing monitoring infrastructure

OpenTelemetry Integration

Built on OpenTelemetry — the vendor-neutral, industry-standard observability framework — Qwen Code's observability system provides:

• Universal Compatibility: Export to any OpenTelemetry backend (Aliyun, Jaeger, Prometheus, Datadog, etc.)
• Standardized Data: Use consistent formats and collection methods across your toolchain
• Future-Proof Integration: Connect with existing and future observability infrastructure
• No Vendor Lock-in: Switch between backends without changing your instrumentation

Configuration

All telemetry behavior is controlled through your .qwen/settings.json file. These settings can be overridden by environment variables or CLI flags.

Setting	Environment Variable	CLI Flag	Description	Values	Default
enabled	QWEN_TELEMETRY_ENABLED	--telemetry / --no-telemetry	Enable or disable telemetry	true/false	false
target	QWEN_TELEMETRY_TARGET	--telemetry-target <local|gcp> (deprecated)	Informational destination label; does not control exporter routing — set otlpEndpoint or outfile to configure where data is sent	"gcp"/"local"	"local"
otlpEndpoint	QWEN_TELEMETRY_OTLP_ENDPOINT	--telemetry-otlp-endpoint <URL>	OTLP collector endpoint	URL string	http://localhost:4317
otlpProtocol	QWEN_TELEMETRY_OTLP_PROTOCOL	--telemetry-otlp-protocol <grpc|http>	OTLP transport protocol	"grpc"/"http"	"grpc"
otlpTracesEndpoint	QWEN_TELEMETRY_OTLP_TRACES_ENDPOINT	-	Per-signal endpoint override for traces (HTTP only)	URL string	-
otlpLogsEndpoint	QWEN_TELEMETRY_OTLP_LOGS_ENDPOINT	-	Per-signal endpoint override for logs (HTTP only)	URL string	-
otlpMetricsEndpoint	QWEN_TELEMETRY_OTLP_METRICS_ENDPOINT	-	Per-signal endpoint override for metrics (HTTP only)	URL string	-
outfile	QWEN_TELEMETRY_OUTFILE	--telemetry-outfile <path>	Save telemetry to file (overrides OTLP export)	file path	-
logPrompts	QWEN_TELEMETRY_LOG_PROMPTS	--telemetry-log-prompts / --no-telemetry-log-prompts	Include prompts in telemetry logs	true/false	true
includeSensitiveSpanAttributes	QWEN_TELEMETRY_INCLUDE_SENSITIVE_SPAN_ATTRIBUTES	-	Include user prompts, system prompts, tool I/O, and model output as native span attributes (in addition to log-to-span bridge spans)	true/false	false
resourceAttributes	OTEL_RESOURCE_ATTRIBUTES (+ OTEL_SERVICE_NAME)	-	Static resource attributes attached to every exported span / log / metric. See Resource attributes below.	key=value,…	{}
metrics.includeSessionId	QWEN_TELEMETRY_METRICS_INCLUDE_SESSION_ID	-	Include session.id on metric data points. Disabled by default to protect metric backends from time-series fan-out.	true/false	false

Note on boolean environment variables: For the boolean settings (enabled, logPrompts, includeSensitiveSpanAttributes), setting the corresponding environment variable to true or 1 will enable the feature. Any other value will disable it.

Sensitive span attributes: When includeSensitiveSpanAttributes is enabled, two things happen:

1. Native span attributes (qwen-code.interaction, api.generateContent*, tool.<name>) carry verbatim conversation content:

   • User prompts (new_context)
   • System prompts (system_prompt — full text once per session, deduped by SHA-256 hash; subsequent spans only carry system_prompt_hash + system_prompt_preview + system_prompt_length)
   • Tool schemas (emitted as tool_schema events, also hash-deduped)
   • Tool inputs (tool_input) and tool results (tool_result)
   • Model output (response.model_output)

   Each value is truncated at 60 KB; *_truncated and *_original_length flags surface when truncation occurs.

2. Log-to-span bridge spans (used when HTTP traces are exported without a logs endpoint) keep their existing prompt, function_args, and response_text fields, instead of being dropped.

⚠️ Security warning: enabling this flag streams full conversation history, file contents read by read_file, shell commands and their output (including secrets in env vars or arguments), and model responses to the configured OTLP backend. Treat the backend as a privileged data sink. The flag defaults to false.

Cost / payload size: A heavy turn (60 KB system prompt + 10 tool calls, each up to 60 KB input + 60 KB result, plus 60 KB model output) can produce up to ~1.5 MB of attribute payload before OTLP compression. When pointing tools that read large files (read_file, etc.) at long-running sessions, monitor exporter throughput.

This setting does not disable sensitive data in OTel logs or other telemetry sinks; non-internal API response telemetry can populate response_text, so OTel logs, UI telemetry, and chat recording may receive response text independently of this setting. QwenLogger does not include response_text.

HTTP OTLP signal routing: When using HTTP protocol (otlpProtocol: "http"), Qwen Code automatically appends signal-specific paths (/v1/traces, /v1/logs, /v1/metrics) to the base otlpEndpoint. For example, http://collector:4318 becomes http://collector:4318/v1/traces for traces. If the URL already ends with a signal path, it is used as-is. Per-signal endpoint overrides (otlpTracesEndpoint, etc.) take precedence over the base endpoint and are used verbatim. gRPC protocol uses service-based routing and does not append paths.

The per-signal endpoint environment variables also accept the standard OpenTelemetry names: OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, OTEL_EXPORTER_OTLP_LOGS_ENDPOINT, OTEL_EXPORTER_OTLP_METRICS_ENDPOINT. The QWEN_TELEMETRY_OTLP_* variants take precedence over the OTEL_* variants.

For detailed information about all configuration options, see the Configuration Guide.

Resource attributes

Resource attributes are static key-value pairs attached to every span, log, and metric exported via OTLP. Use them to slice telemetry by team, environment, deployment region, or any other dimension your backend cares about.

Two sources, merged in priority order (lowest → highest):

1. The standard OTEL_RESOURCE_ATTRIBUTES env var
2. telemetry.resourceAttributes in .qwen/settings.json (overrides env on key conflict)

OTEL_SERVICE_NAME is a separate escape hatch — when set, it overrides service.name from any other source (per the OpenTelemetry spec).

Examples

Slice all telemetry by team / environment:

export OTEL_RESOURCE_ATTRIBUTES="team=platform,env=prod,cost_center=eng-123"

Route to a per-tenant collector via service.name:

export OTEL_SERVICE_NAME=qwen-code-ci

Fleet baseline (~/.qwen/settings.json) + per-host override:

{
  "telemetry": {
    "resourceAttributes": {
      "deployment.environment": "production",
      "service.namespace": "engineering-tooling"
    }
  }
}

# Add a one-off tag without touching settings:
export OTEL_RESOURCE_ATTRIBUTES="debug_run=true"

Reserved keys

Some keys are runtime-controlled and cannot be overridden:

• service.version — always set to the running CLI version. Setting it from any source is silently dropped with a warning.
• session.id — runtime-injected per session. User-provided values from either env or settings are dropped with a warning. The reason is that Resource attributes auto-attach to every metric data point; allowing user override would bypass Cardinality controls below. Spans and logs always carry session.id.

service.name is not reserved; it follows the precedence chain above.

Format

OTEL_RESOURCE_ATTRIBUTES follows the OpenTelemetry spec: key1=value1,key2=value2 with values percent-encoded. Spaces in values must be encoded as %20, commas as %2C (unencoded commas split the value at the wrong boundary and the second half is dropped as malformed). Malformed pairs are skipped with a warning rather than failing telemetry startup.

Troubleshooting: when a user-provided attribute appears not to take effect

Reserved keys (service.version, session.id), malformed pairs, non-string settings values, and invalid percent-encoding are all silently dropped with a warning logged via the OpenTelemetry diagnostics channel. That channel routes to the debug log file (~/.qwen/log/otel-*.log), not the console, so the behavior can look like silent failure.

If a custom resource attribute isn't appearing on exported telemetry:

1. Check ~/.qwen/log/otel-*.log for lines matching cannot override (reserved key dropped), Skipping malformed (bad env var pair), or must be a string (non-string settings value).
2. Verify the env var is set in the qwen-code process's environment (not just your shell) and that values are percent-encoded.
3. Confirm telemetry.enabled is true — telemetry init only runs if enabled.

Cardinality controls

Metrics are aggregated by attribute set at the backend — every distinct combination of attribute values produces a new time series. Attaching a high-cardinality field like session.id to a metric causes time-series fan-out proportional to the number of sessions, which quickly exhausts metric backend storage.

To prevent this, Qwen Code keeps high-cardinality attributes off metric data points by default. Spans and logs are per-event and unaffected, so they continue to carry session.id for trace and log correlation.

telemetry.metrics.includeSessionId (default: false)

Setting this to true (via settings or QWEN_TELEMETRY_METRICS_INCLUDE_SESSION_ID=true) re-attaches session.id to every metric data point.

⚠️ Warning: each CLI session creates a new value. Leaving this on for a fleet will blow up metric storage. Recommended only for short-term debugging. For long-term session correlation, query trace or log backends instead.

Migration from earlier versions

Prior to this release, session.id was attached to metrics by default. If your Prometheus queries / Grafana dashboards / alert rules reference session_id on a metric, you have two options:

Option A — restore the previous behavior for short-term debugging:

export QWEN_TELEMETRY_METRICS_INCLUDE_SESSION_ID=true

or:

{
  "telemetry": {
    "metrics": { "includeSessionId": true }
  }
}

Option B (recommended) — move session-level analysis off metrics. Spans and logs still carry session.id, and trace / log backends (Jaeger, Tempo, Loki, Aliyun SLS / ARMS Tracing) handle per-session slicing natively without cardinality pressure.

Client-side HTTP span on outbound fetch

When telemetry is enabled, Qwen Code registers UndiciInstrumentation which creates a client-side HTTP span for every outbound fetch() request originated by the process — including the LLM SDKs (openai, @google/genai, @anthropic-ai/sdk), the MCP StreamableHTTP client, the WebFetch tool, and any IDE-extension out-of-process calls. The span lets you see network latency (TTFB / response body transfer) separately from upstream model processing time, which the existing api.generateContent span alone can't distinguish.

These spans go to your own OTLP collector (or file outfile) just like the rest of the telemetry — they do not affect what is written onto the outbound HTTP request itself. Whether the W3C traceparent header is also written into the outgoing request stream is controlled by a separate, security-relevant setting documented in outbound correlation below.

Feedback-loop avoidance. OTel SDK uses fetch internally to upload OTLP data. Without protection, instrumenting fetch would trace those uploads, which would themselves be uploaded, causing an infinite loop. Qwen Code's undici instrumentation is configured with an ignoreRequestHook that skips URLs matching the configured telemetry.otlpEndpoint / telemetry.otlpTracesEndpoint / telemetry.otlpLogsEndpoint / telemetry.otlpMetricsEndpoint prefixes. In file-outfile mode there are no outbound HTTP uploads, so the hook is a no-op.

Outbound correlation (SECURITY-RELEVANT)

These settings live in a separate top-level namespace from telemetry.* on purpose: telemetry controls data flow into the operator's own observability backend, while outboundCorrelation.* controls what client-side correlation data qwen-code writes into outbound LLM API request streams that reach third-party LLM provider endpoints (DashScope, OpenAI, Anthropic, etc.). Different recipients, different consent decision. All values default to off. See PR #4390 review discussion for the framing rationale.

outboundCorrelation.propagateTraceContext

"outboundCorrelation": {
  "propagateTraceContext": false // default
}

When false (default), Qwen Code installs a no-op TextMapPropagator on the OTel SDK. UndiciInstrumentation still creates client HTTP spans for your OTLP collector, but propagation.inject() is a no-op so no traceparent is written onto outbound requests. Trace IDs stay internal to the operator's collector.

When true, the SDK's default W3C composite propagator (tracecontext + baggage) is installed and the standard traceparent header is written on every outbound fetch:

traceparent: 00-<32-hex traceId>-<16-hex parentSpanId>-<01-sampled | 00-not-sampled>

Opt in only when the LLM provider also reports into your OTel collector for cross-process trace stitching — e.g. ARMS Tracing serving DashScope. For most operators the value is false; cross-vendor trace continuation is niche.

Depends on telemetry.enabled: true. The OTel SDK only initializes when telemetry is enabled, so propagateTraceContext only takes effect in that state. Setting it to true while telemetry is disabled is a silent no-op — no SDK, no propagator, no traceparent on the wire. Verify both flags when wiring an ARMS+DashScope correlation setup:

{
  "telemetry": {
    "enabled": true,
    "otlpTracesEndpoint": "http://tracing-analysis-...",
  },
  "outboundCorrelation": {
    "propagateTraceContext": true,
  },
}

Other outbound correlation headers

X-Qwen-Code-Session-Id and X-Qwen-Code-Request-Id are not part of this PR. They will be designed and proposed in their own follow-up PR(s) under the same outboundCorrelation.* namespace, each with its own threat model and operator-consent flow. PR #4390 review (LaZzyMan) established the principle: "telemetry's scope of work doesn't include sending identifiers to LLM providers"; correlation-header work moves to its own design discussion rather than landing under telemetry.

Aliyun Telemetry

Manual OTLP Export

To view Qwen Code telemetry in Alibaba Cloud Managed Service for OpenTelemetry, configure Qwen Code to export to the OTLP endpoint provided by ARMS.

Setting "target": "gcp" alone does not configure the export destination. If otlpEndpoint is not set, Qwen Code still defaults to http://localhost:4317. If outfile is set, it overrides otlpEndpoint and telemetry is written to the file instead of being sent to Alibaba Cloud.

1. Enable telemetry in your .qwen/settings.json and set the OTLP endpoint:

   Option A: gRPC protocol (standard OTLP endpoint):

   {
     "telemetry": {
       "enabled": true,
       "target": "gcp",
       "otlpEndpoint": "https://<your-otlp-endpoint>",
       "otlpProtocol": "grpc"
     }
   }
   

   Option B: HTTP protocol with per-signal endpoints (for backends that use non-standard paths, e.g., /api/otlp/traces instead of /v1/traces):

   {
     "telemetry": {
       "enabled": true,
       "otlpProtocol": "http",
       "otlpTracesEndpoint": "http://<host>/<token>/api/otlp/traces",
       "otlpLogsEndpoint": "http://<host>/<token>/api/otlp/logs",
       "otlpMetricsEndpoint": "http://<host>/<token>/api/otlp/metrics"
     }
   }
   

   Note: When using HTTP protocol with only otlpEndpoint (no per-signal overrides), Qwen Code appends standard OTLP paths (/v1/traces, /v1/logs, /v1/metrics) to the base URL. If your backend uses different paths, use per-signal endpoint overrides as shown in Option B.

2. If your Alibaba Cloud endpoint requires authentication, provide OTLP headers through standard OpenTelemetry environment variables such as OTEL_EXPORTER_OTLP_HEADERS (or the signal-specific variants). Qwen Code does not currently expose OTLP auth headers directly in .qwen/settings.json.

3. Run Qwen Code and send prompts.

4. View telemetry in Managed Service for OpenTelemetry:

   • Product overview: What is Managed Service for OpenTelemetry?
   • Getting started: Get started with Managed Service for OpenTelemetry
   • Console entry points:
     • China mainland: trace.console.aliyun.com (legacy console: tracing.console.aliyun.com)
     • International: arms.console.alibabacloud.com
   • In the console, use Applications to inspect traces and service topology.
   • To locate the OTLP endpoint and access information:
     • New console (trace.console.aliyun.com or international): navigate to Integration Center.
     • Legacy console (tracing.console.aliyun.com): navigate to Cluster Configurations → Access point information.

Local Telemetry

For local development and debugging, you can capture telemetry data locally:

File-based Output (Recommended)

1. Enable telemetry in your .qwen/settings.json:

   {
     "telemetry": {
       "enabled": true,
       "outfile": ".qwen/telemetry.log"
     }
   }
   

   Note: When outfile is set, OTLP export is automatically disabled. The target and otlpEndpoint settings are not needed for file-only output and can be safely omitted from your config.

2. Run Qwen Code and send prompts.

3. View logs and metrics in the specified file (e.g., .qwen/telemetry.log).

Collector-Based Export (Advanced)

1. Run the automation script:

   npm run telemetry -- --target=local
   

   This will:
   • Download and start Jaeger and OTEL collector
   • Configure your workspace for local telemetry
   • Provide a Jaeger UI at http://localhost:16686
   • Save logs/metrics to ~/.qwen/tmp/<projectHash>/otel/collector.log
   • Stop collector on exit (e.g. Ctrl+C)
2. Run Qwen Code and send prompts.
3. View traces at http://localhost:16686 and logs/metrics in the collector log file.

Logs and Metrics

The following section describes the structure of logs and metrics generated for Qwen Code.

• A sessionId is included as a common attribute on all logs and metrics.

Logs

Logs are timestamped records of specific events. The following events are logged for Qwen Code:

• qwen-code.config: This event occurs once at startup with the CLI's configuration.

  • Attributes:
    • model (string)
    • sandbox_enabled (boolean)
    • core_tools_enabled (string)
    • approval_mode (string)
    • file_filtering_respect_git_ignore (boolean)
    • debug_mode (boolean)
    • truncate_tool_output_threshold (number)
    • truncate_tool_output_lines (number)
    • hooks (string, comma-separated hook event types, omitted if hooks disabled)
    • ide_enabled (boolean)
    • interactive_shell_enabled (boolean)
    • mcp_servers (string)
    • output_format (string: "text" or "json")

• qwen-code.user_prompt: This event occurs when a user submits a prompt.

  • Attributes:
    • prompt_length (int)
    • prompt_id (string)
    • prompt (string, this attribute is excluded if log_prompts_enabled is configured to be false)
    • auth_type (string)

• qwen-code.tool_call: This event occurs for each function call.

  • Attributes:
    • function_name
    • function_args
    • duration_ms
    • success (boolean)
    • decision (string: "accept", "reject", "auto_accept", or "modify", if applicable)
    • error (if applicable)
    • error_type (if applicable)
    • content_length (int, if applicable)
    • metadata (if applicable, dictionary of string -> any)

• qwen-code.file_operation: This event occurs for each file operation.

  • Attributes:
    • tool_name (string)
    • operation (string: "create", "read", "update")
    • lines (int, if applicable)
    • mimetype (string, if applicable)
    • extension (string, if applicable)
    • programming_language (string, if applicable)
    • diff_stat (json string, if applicable): A JSON string with the following members:
      • ai_added_lines (int)
      • ai_removed_lines (int)
      • user_added_lines (int)
      • user_removed_lines (int)

• qwen-code.api_request: This event occurs when making a request to Qwen API.

  • Attributes:
    • model
    • request_text (if applicable)

• qwen-code.api_error: This event occurs if the API request fails.

  • Attributes:
    • model
    • error
    • error_type
    • status_code
    • duration_ms
    • auth_type

• qwen-code.api_response: This event occurs upon receiving a response from Qwen API.

  • Attributes:
    • model
    • status_code
    • duration_ms
    • error (optional)
    • input_token_count
    • output_token_count
    • cached_content_token_count
    • thoughts_token_count
    • response_text (if applicable)
    • auth_type

• qwen-code.tool_output_truncated: This event occurs when the output of a tool call is too large and gets truncated.

  • Attributes:
    • tool_name (string)
    • original_content_length (int)
    • truncated_content_length (int)
    • threshold (int)
    • lines (int)
    • prompt_id (string)

• qwen-code.malformed_json_response: This event occurs when a generateJson response from Qwen API cannot be parsed as a json.

  • Attributes:
    • model

• qwen-code.flash_fallback: This event occurs when Qwen Code switches to flash as fallback.

  • Attributes:
    • auth_type

• qwen-code.slash_command: This event occurs when a user executes a slash command.

  • Attributes:
    • command (string)
    • subcommand (string, if applicable)

• qwen-code.extension_enable: This event occurs when an extension is enabled

• qwen-code.extension_install: This event occurs when an extension is installed

  • Attributes:
    • extension_name (string)
    • extension_version (string)
    • extension_source (string)
    • status (string)

• qwen-code.extension_uninstall: This event occurs when an extension is uninstalled

Metrics

Metrics are numerical measurements of behavior over time. The following metrics are collected for Qwen Code (metric names remain qwen-code.* for compatibility):

• qwen-code.session.count (Counter, Int): Incremented once per CLI startup.

• qwen-code.tool.call.count (Counter, Int): Counts tool calls.

  • Attributes:
    • function_name
    • success (boolean)
    • decision (string: "accept", "reject", or "modify", if applicable)
    • tool_type (string: "mcp", or "native", if applicable)

• qwen-code.tool.call.latency (Histogram, ms): Measures tool call latency.

  • Attributes:
    • function_name
    • decision (string: "accept", "reject", or "modify", if applicable)

• qwen-code.api.request.count (Counter, Int): Counts all API requests.

  • Attributes:
    • model
    • status_code
    • error_type (if applicable)

• qwen-code.api.request.latency (Histogram, ms): Measures API request latency.

  • Attributes:
    • model

• qwen-code.token.usage (Counter, Int): Counts the number of tokens used.

  • Attributes:
    • model
    • type (string: "input", "output", "thought", or "cache")

• qwen-code.file.operation.count (Counter, Int): Counts file operations.

  • Attributes:
    • operation (string: "create", "read", "update"): The type of file operation.
    • lines (Int, if applicable): Number of lines in the file.
    • mimetype (string, if applicable): Mimetype of the file.
    • extension (string, if applicable): File extension of the file.
    • model_added_lines (Int, if applicable): Number of lines added/changed by the model.
    • model_removed_lines (Int, if applicable): Number of lines removed/changed by the model.
    • user_added_lines (Int, if applicable): Number of lines added/changed by user in AI proposed changes.
    • user_removed_lines (Int, if applicable): Number of lines removed/changed by user in AI proposed changes.
    • programming_language (string, if applicable): The programming language of the file.

• qwen-code.chat_compression (Counter, Int): Counts chat compression operations

  • Attributes:
    • tokens_before: (Int): Number of tokens in context prior to compression
    • tokens_after: (Int): Number of tokens in context after compression