mirror of
https://github.com/alibaba/open-code-review.git
synced 2026-07-09 17:28:58 +00:00
Feat/telemetry http exporter (#314)
* feat(telemetry): add OTLP HTTP exporter and print TraceID - Add HTTP/protobuf exporter support alongside existing gRPC exporter - Route based on OTEL_EXPORTER_OTLP_PROTOCOL config (http/protobuf vs grpc) - Print TraceID to stderr when telemetry is enabled for easier correlation - Add corresponding unit tests * feat(telemetry): add span coverage for LLM calls, tool execution, plan/filter phases - Add StartLLMSpan / RecordLLMResult helpers (span.go), symmetric with existing StartToolSpan / RecordToolResult - Wrap LLM completion calls in llmloop.RunPerFile with llm.request spans - Wrap all three tool execution paths in executeToolCall with tool.execute.* spans (dynamic tools, code_comment sync/async, other tools) - Add plan.execute span around executePlanPhase - Add main.loop span around RunPerFile call in executeSubtask - Add review_filter.execute span around executeReviewFilter, with comments.before / comments.filtered attributes - Record llm.error attribute on LLM failures for diagnosability - Record review.repo / review.from / review.to / review.model on the top-level review.run span - Metrics (RecordLLMRequest / RecordToolCall) are preserved alongside the new spans — they serve different purposes (aggregate dashboards vs per-run diagnosis) Verified end-to-end against Sunfire (OTLP HTTP gateway): full span tree observed for review.run -> subtask.execute -> plan.execute/main.loop/ review_filter.execute -> llm.request/tool.execute.* * fix(telemetry): address CR findings — span error handling, async span lifecycle, protocol robustness - Add span.RecordError(err) to RecordLLMResult and RecordToolResult for consistency with EndSpan - Use OTel standard pattern (span.SetStatus + span.RecordError) in error paths of review.run, plan.execute, main.loop, review_filter.execute - Move async code_comment span end into pool.Submit callback so span duration reflects actual execution time - Unify time.Since(startTime) in code_comment error path to a single dur - Remove http/json from supported OTLP protocols (not actually implemented) - Add stderr warning when unknown OTLP protocol falls back to gRPC * feat(telemetry): include trace_id in JSON output, restrict stderr to text format - Add trace_id as top-level field in jsonOutput struct (omitempty) - JSON format: trace_id in structured response for programmatic extraction - Text format: TraceID printed to stderr for human debugging - Telemetry disabled: trace_id field omitted entirely * fix: address PR review findings - loop.go: wrap async span lifecycle in defer to prevent leak on panic - exporter.go: update parseOTLPEndpoint comment to reflect gRPC+HTTP usage - scan_cmd.go: align traceID extraction and OTel error handling with review_cmd - output.go/shared.go: propagate traceID to outputJSONNoFiles for consistency - agent.go: move comments.filtered attribute before early return so 0 is distinguishable from not-executed * feat(telemetry): address PR review — http/json routing, LLM span coverage, trace_id tests - Route http/json to HTTP exporter (Go OTel SDK HTTP transport only supports protobuf serialization; users need HTTP transport, not JSON encoding) - Add llm.request spans to executePlanPhase, executeReviewFilter, and ReLocateComment with Usage nil-safety consistent with loop.go - Add trace_id assertions to output helper tests and emitRunResult end-to-end tests using real TracerProvider * docs: add OTLP protocol selection and endpoint format to telemetry section Sync across all 5 README language versions (en, zh-CN, ja-JP, ko-KR, ru-RU). * fix: unify time.Since in async code_comment defer to single dur variable
This commit is contained in:
parent
14f1c22a60
commit
46dde274d8
21 changed files with 471 additions and 24 deletions
|
|
@ -762,6 +762,15 @@ ocr config set telemetry.otlp_endpoint localhost:4317
|
||||||
|
|
||||||
エクスポートデータにLLMのプロンプトとレスポンスを含めるには、`telemetry.content_logging`を設定してください。
|
エクスポートデータにLLMのプロンプトとレスポンスを含めるには、`telemetry.content_logging`を設定してください。
|
||||||
|
|
||||||
|
**プロトコル選択:** 環境変数 `OTEL_EXPORTER_OTLP_PROTOCOL` でエクスポートプロトコルを選択できます:
|
||||||
|
|
||||||
|
| 値 | トランスポート | 説明 |
|
||||||
|
|---|---|---|
|
||||||
|
| `grpc`(デフォルト) | gRPC | デフォルトポート 4317 |
|
||||||
|
| `http/protobuf` | HTTP | デフォルトポート 4318 |
|
||||||
|
|
||||||
|
**Endpoint 形式:** `telemetry.otlp_endpoint` は `host:port` または `http://host:port` 形式のベースURLを指定します。パスを含める必要はありません。SDKが [OTLP仕様](https://opentelemetry.io/docs/specs/otlp/#otlphttp-request)に従いシグナルパス(例:`/v1/traces`)を自動的に付加します。
|
||||||
|
|
||||||
## コントリビューション
|
## コントリビューション
|
||||||
|
|
||||||
開発環境のセットアップ、コーディングガイドライン、プルリクエストの提出方法については[CONTRIBUTING.md](CONTRIBUTING.md)を参照してください。
|
開発環境のセットアップ、コーディングガイドライン、プルリクエストの提出方法については[CONTRIBUTING.md](CONTRIBUTING.md)を参照してください。
|
||||||
|
|
|
||||||
|
|
@ -719,6 +719,15 @@ ocr config set telemetry.otlp_endpoint localhost:4317
|
||||||
|
|
||||||
exported data에 LLM prompt와 response를 포함하려면 `telemetry.content_logging`을 설정합니다.
|
exported data에 LLM prompt와 response를 포함하려면 `telemetry.content_logging`을 설정합니다.
|
||||||
|
|
||||||
|
**프로토콜 선택:** 환경 변수 `OTEL_EXPORTER_OTLP_PROTOCOL`로 export 프로토콜을 선택할 수 있습니다:
|
||||||
|
|
||||||
|
| 값 | 전송 방식 | 설명 |
|
||||||
|
|---|---|---|
|
||||||
|
| `grpc` (기본값) | gRPC | 기본 포트 4317 |
|
||||||
|
| `http/protobuf` | HTTP | 기본 포트 4318 |
|
||||||
|
|
||||||
|
**Endpoint 형식:** `telemetry.otlp_endpoint`는 `host:port` 또는 `http://host:port` 형식의 base URL을 지정합니다. 경로를 포함할 필요가 없습니다. SDK가 [OTLP 사양](https://opentelemetry.io/docs/specs/otlp/#otlphttp-request)에 따라 signal 경로(예: `/v1/traces`)를 자동으로 추가합니다.
|
||||||
|
|
||||||
## Contributing
|
## Contributing
|
||||||
|
|
||||||
개발 환경 설정, coding guideline, pull request 제출 방법은 [CONTRIBUTING.ko-KR.md](CONTRIBUTING.ko-KR.md)를 참고하세요.
|
개발 환경 설정, coding guideline, pull request 제출 방법은 [CONTRIBUTING.ko-KR.md](CONTRIBUTING.ko-KR.md)를 참고하세요.
|
||||||
|
|
|
||||||
10
README.md
10
README.md
|
|
@ -767,6 +767,16 @@ ocr config set telemetry.otlp_endpoint localhost:4317
|
||||||
|
|
||||||
Set `telemetry.content_logging` to include LLM prompts and responses in exported data.
|
Set `telemetry.content_logging` to include LLM prompts and responses in exported data.
|
||||||
|
|
||||||
|
**Protocol selection:** Set the environment variable `OTEL_EXPORTER_OTLP_PROTOCOL` to choose the export protocol:
|
||||||
|
|
||||||
|
| Value | Transport | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `grpc` (default) | gRPC | Default port 4317 |
|
||||||
|
| `http/protobuf` | HTTP | Default port 4318 |
|
||||||
|
|
||||||
|
**Endpoint format:** `telemetry.otlp_endpoint` expects a base URL in `host:port` or `http://host:port` format, without a path component. The SDK appends the signal path (e.g. `/v1/traces`) automatically per the [OTLP specification](https://opentelemetry.io/docs/specs/otlp/#otlphttp-request).
|
||||||
|
|
||||||
|
|
||||||
## Contributing
|
## Contributing
|
||||||
|
|
||||||
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, coding guidelines, and how to submit pull requests.
|
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, coding guidelines, and how to submit pull requests.
|
||||||
|
|
|
||||||
|
|
@ -764,6 +764,15 @@ ocr config set telemetry.otlp_endpoint localhost:4317
|
||||||
|
|
||||||
Установите `telemetry.content_logging`, чтобы включать промпты и ответы LLM в экспортируемые данные.
|
Установите `telemetry.content_logging`, чтобы включать промпты и ответы LLM в экспортируемые данные.
|
||||||
|
|
||||||
|
**Выбор протокола:** Переменная окружения `OTEL_EXPORTER_OTLP_PROTOCOL` определяет протокол экспорта:
|
||||||
|
|
||||||
|
| Значение | Транспорт | Описание |
|
||||||
|
|---|---|---|
|
||||||
|
| `grpc` (по умолчанию) | gRPC | Порт по умолчанию 4317 |
|
||||||
|
| `http/protobuf` | HTTP | Порт по умолчанию 4318 |
|
||||||
|
|
||||||
|
**Формат endpoint:** `telemetry.otlp_endpoint` принимает базовый URL в формате `host:port` или `http://host:port` без компонента пути. SDK автоматически добавляет путь сигнала (например, `/v1/traces`) в соответствии со [спецификацией OTLP](https://opentelemetry.io/docs/specs/otlp/#otlphttp-request).
|
||||||
|
|
||||||
## Участие в разработке
|
## Участие в разработке
|
||||||
|
|
||||||
В [CONTRIBUTING.ru-RU.md](CONTRIBUTING.ru-RU.md) описаны настройка окружения разработки, рекомендации по коду и порядок отправки pull request'ов.
|
В [CONTRIBUTING.ru-RU.md](CONTRIBUTING.ru-RU.md) описаны настройка окружения разработки, рекомендации по коду и порядок отправки pull request'ов.
|
||||||
|
|
|
||||||
|
|
@ -752,6 +752,15 @@ ocr config set telemetry.otlp_endpoint localhost:4317
|
||||||
|
|
||||||
设置 `telemetry.content_logging` 可在导出数据中包含 LLM 提示词和响应。
|
设置 `telemetry.content_logging` 可在导出数据中包含 LLM 提示词和响应。
|
||||||
|
|
||||||
|
**协议选择:** 通过环境变量 `OTEL_EXPORTER_OTLP_PROTOCOL` 选择导出协议:
|
||||||
|
|
||||||
|
| 值 | 传输方式 | 说明 |
|
||||||
|
|---|---|---|
|
||||||
|
| `grpc`(默认) | gRPC | 默认端口 4317 |
|
||||||
|
| `http/protobuf` | HTTP | 默认端口 4318 |
|
||||||
|
|
||||||
|
**Endpoint 格式:** `telemetry.otlp_endpoint` 的值为 `host:port` 或 `http://host:port`,无需包含路径。SDK 会根据 [OTLP 规范](https://opentelemetry.io/docs/specs/otlp/#otlphttp-request)自动追加信号路径(如 `/v1/traces`)。
|
||||||
|
|
||||||
## 贡献
|
## 贡献
|
||||||
|
|
||||||
参见 [CONTRIBUTING.zh-CN.md](CONTRIBUTING.zh-CN.md) 了解开发环境搭建、编码规范以及如何提交 Pull Request。
|
参见 [CONTRIBUTING.zh-CN.md](CONTRIBUTING.zh-CN.md) 了解开发环境搭建、编码规范以及如何提交 Pull Request。
|
||||||
|
|
|
||||||
|
|
@ -7,6 +7,9 @@ import (
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"go.opentelemetry.io/otel"
|
||||||
|
sdktrace "go.opentelemetry.io/otel/sdk/trace"
|
||||||
|
|
||||||
"github.com/open-code-review/open-code-review/internal/agent"
|
"github.com/open-code-review/open-code-review/internal/agent"
|
||||||
"github.com/open-code-review/open-code-review/internal/model"
|
"github.com/open-code-review/open-code-review/internal/model"
|
||||||
)
|
)
|
||||||
|
|
@ -175,3 +178,61 @@ func TestEmitRunResult_NilQuietHandle(t *testing.T) {
|
||||||
})
|
})
|
||||||
_ = got
|
_ = got
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestEmitRunResult_JSONTraceIDFromContext(t *testing.T) {
|
||||||
|
tp := sdktrace.NewTracerProvider()
|
||||||
|
defer tp.Shutdown(context.Background())
|
||||||
|
otel.SetTracerProvider(tp)
|
||||||
|
|
||||||
|
ctx, span := tp.Tracer("test").Start(context.Background(), "test-root")
|
||||||
|
wantTraceID := span.SpanContext().TraceID().String()
|
||||||
|
defer span.End()
|
||||||
|
|
||||||
|
ag := &mockResultProvider{
|
||||||
|
filesReviewed: 2,
|
||||||
|
inputTokens: 10,
|
||||||
|
outputTokens: 5,
|
||||||
|
totalTokens: 15,
|
||||||
|
}
|
||||||
|
got := captureStdout(t, func() {
|
||||||
|
err := emitRunResult(ctx, ag, nil, time.Now(), "json", "developer", nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
var out jsonOutput
|
||||||
|
if err := json.Unmarshal([]byte(got), &out); err != nil {
|
||||||
|
t.Fatalf("unmarshal: %v", err)
|
||||||
|
}
|
||||||
|
if out.TraceID != wantTraceID {
|
||||||
|
t.Errorf("trace_id = %q, want %q", out.TraceID, wantTraceID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEmitRunResult_JSONNoFilesTraceID(t *testing.T) {
|
||||||
|
tp := sdktrace.NewTracerProvider()
|
||||||
|
defer tp.Shutdown(context.Background())
|
||||||
|
otel.SetTracerProvider(tp)
|
||||||
|
|
||||||
|
ctx, span := tp.Tracer("test").Start(context.Background(), "test-root")
|
||||||
|
wantTraceID := span.SpanContext().TraceID().String()
|
||||||
|
defer span.End()
|
||||||
|
|
||||||
|
ag := &mockResultProvider{filesReviewed: 0}
|
||||||
|
got := captureStdout(t, func() {
|
||||||
|
err := emitRunResult(ctx, ag, nil, time.Now(), "json", "developer", nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
var out jsonOutput
|
||||||
|
if err := json.Unmarshal([]byte(got), &out); err != nil {
|
||||||
|
t.Fatalf("unmarshal: %v", err)
|
||||||
|
}
|
||||||
|
if out.Status != "skipped" {
|
||||||
|
t.Errorf("status = %q, want skipped", out.Status)
|
||||||
|
}
|
||||||
|
if out.TraceID != wantTraceID {
|
||||||
|
t.Errorf("trace_id = %q, want %q", out.TraceID, wantTraceID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
|
||||||
|
|
@ -240,6 +240,7 @@ type jsonToolCalls struct {
|
||||||
|
|
||||||
type jsonOutput struct {
|
type jsonOutput struct {
|
||||||
Status string `json:"status"`
|
Status string `json:"status"`
|
||||||
|
TraceID string `json:"trace_id,omitempty"`
|
||||||
Message string `json:"message,omitempty"`
|
Message string `json:"message,omitempty"`
|
||||||
Summary *jsonSummary `json:"summary,omitempty"`
|
Summary *jsonSummary `json:"summary,omitempty"`
|
||||||
ToolCalls *jsonToolCalls `json:"tool_calls"`
|
ToolCalls *jsonToolCalls `json:"tool_calls"`
|
||||||
|
|
@ -263,9 +264,10 @@ func outputJSON(comments []model.LlmComment) error {
|
||||||
|
|
||||||
func outputJSONWithWarnings(comments []model.LlmComment, warnings []agent.AgentWarning,
|
func outputJSONWithWarnings(comments []model.LlmComment, warnings []agent.AgentWarning,
|
||||||
filesReviewed, inputTokens, outputTokens, totalTokens, cacheReadTokens, cacheWriteTokens int64,
|
filesReviewed, inputTokens, outputTokens, totalTokens, cacheReadTokens, cacheWriteTokens int64,
|
||||||
duration time.Duration, projectSummary string, toolCalls map[string]int64) error {
|
duration time.Duration, projectSummary string, toolCalls map[string]int64, traceID string) error {
|
||||||
out := jsonOutput{
|
out := jsonOutput{
|
||||||
Status: "success",
|
Status: "success",
|
||||||
|
TraceID: traceID,
|
||||||
Comments: comments,
|
Comments: comments,
|
||||||
Summary: &jsonSummary{
|
Summary: &jsonSummary{
|
||||||
FilesReviewed: filesReviewed,
|
FilesReviewed: filesReviewed,
|
||||||
|
|
@ -311,9 +313,10 @@ func outputJSONWithWarnings(comments []model.LlmComment, warnings []agent.AgentW
|
||||||
return enc.Encode(out)
|
return enc.Encode(out)
|
||||||
}
|
}
|
||||||
|
|
||||||
func outputJSONNoFiles() error {
|
func outputJSONNoFiles(traceID string) error {
|
||||||
out := jsonOutput{
|
out := jsonOutput{
|
||||||
Status: "skipped",
|
Status: "skipped",
|
||||||
|
TraceID: traceID,
|
||||||
Message: "No supported files changed.",
|
Message: "No supported files changed.",
|
||||||
Comments: []model.LlmComment{},
|
Comments: []model.LlmComment{},
|
||||||
ToolCalls: &jsonToolCalls{
|
ToolCalls: &jsonToolCalls{
|
||||||
|
|
|
||||||
|
|
@ -168,8 +168,7 @@ func TestOutputJSONWithWarnings_NoCommentsSubtaskError(t *testing.T) {
|
||||||
os.Stdout = w
|
os.Stdout = w
|
||||||
|
|
||||||
warnings := []agent.AgentWarning{{Type: "subtask_error", File: "x.go", Message: "fail"}}
|
warnings := []agent.AgentWarning{{Type: "subtask_error", File: "x.go", Message: "fail"}}
|
||||||
err := outputJSONWithWarnings(nil, warnings, 1, 10, 5, 15, 0, 0, time.Second, "", nil)
|
err := outputJSONWithWarnings(nil, warnings, 1, 10, 5, 15, 0, 0, time.Second, "", nil, "abc123trace")
|
||||||
|
|
||||||
w.Close()
|
w.Close()
|
||||||
os.Stdout = old
|
os.Stdout = old
|
||||||
|
|
||||||
|
|
@ -188,6 +187,9 @@ func TestOutputJSONWithWarnings_NoCommentsSubtaskError(t *testing.T) {
|
||||||
if !strings.Contains(out.Message, "errors") {
|
if !strings.Contains(out.Message, "errors") {
|
||||||
t.Errorf("message = %q, expected to mention errors", out.Message)
|
t.Errorf("message = %q, expected to mention errors", out.Message)
|
||||||
}
|
}
|
||||||
|
if out.TraceID != "abc123trace" {
|
||||||
|
t.Errorf("trace_id = %q, want abc123trace", out.TraceID)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestStatusBadge(t *testing.T) {
|
func TestStatusBadge(t *testing.T) {
|
||||||
|
|
@ -275,8 +277,7 @@ func TestOutputJSONWithWarnings(t *testing.T) {
|
||||||
|
|
||||||
comments := []model.LlmComment{{Path: "b.go", Content: "test"}}
|
comments := []model.LlmComment{{Path: "b.go", Content: "test"}}
|
||||||
warnings := []agent.AgentWarning{{Type: "subtask_error", File: "c.go", Message: "failed"}}
|
warnings := []agent.AgentWarning{{Type: "subtask_error", File: "c.go", Message: "failed"}}
|
||||||
err := outputJSONWithWarnings(comments, warnings, 5, 100, 50, 150, 10, 5, 3*time.Second, "summary", map[string]int64{"file_read": 3})
|
err := outputJSONWithWarnings(comments, warnings, 5, 100, 50, 150, 10, 5, 3*time.Second, "summary", map[string]int64{"file_read": 3}, "trace-xyz-789")
|
||||||
|
|
||||||
w.Close()
|
w.Close()
|
||||||
os.Stdout = old
|
os.Stdout = old
|
||||||
|
|
||||||
|
|
@ -301,6 +302,9 @@ func TestOutputJSONWithWarnings(t *testing.T) {
|
||||||
if out.ToolCalls == nil || out.ToolCalls.Total != 3 {
|
if out.ToolCalls == nil || out.ToolCalls.Total != 3 {
|
||||||
t.Errorf("ToolCalls.Total = %v", out.ToolCalls)
|
t.Errorf("ToolCalls.Total = %v", out.ToolCalls)
|
||||||
}
|
}
|
||||||
|
if out.TraceID != "trace-xyz-789" {
|
||||||
|
t.Errorf("trace_id = %q, want trace-xyz-789", out.TraceID)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestOutputJSONWithWarnings_NoCommentsNoErrors(t *testing.T) {
|
func TestOutputJSONWithWarnings_NoCommentsNoErrors(t *testing.T) {
|
||||||
|
|
@ -309,8 +313,7 @@ func TestOutputJSONWithWarnings_NoCommentsNoErrors(t *testing.T) {
|
||||||
os.Stdout = w
|
os.Stdout = w
|
||||||
|
|
||||||
warnings := []agent.AgentWarning{{Type: "warning", Message: "something"}}
|
warnings := []agent.AgentWarning{{Type: "warning", Message: "something"}}
|
||||||
err := outputJSONWithWarnings(nil, warnings, 2, 50, 20, 70, 0, 0, time.Second, "", nil)
|
err := outputJSONWithWarnings(nil, warnings, 2, 50, 20, 70, 0, 0, time.Second, "", nil, "")
|
||||||
|
|
||||||
w.Close()
|
w.Close()
|
||||||
os.Stdout = old
|
os.Stdout = old
|
||||||
|
|
||||||
|
|
@ -336,7 +339,7 @@ func TestOutputJSONNoFiles(t *testing.T) {
|
||||||
r, w, _ := os.Pipe()
|
r, w, _ := os.Pipe()
|
||||||
os.Stdout = w
|
os.Stdout = w
|
||||||
|
|
||||||
err := outputJSONNoFiles()
|
err := outputJSONNoFiles("test-trace-id-456")
|
||||||
|
|
||||||
w.Close()
|
w.Close()
|
||||||
os.Stdout = old
|
os.Stdout = old
|
||||||
|
|
@ -353,6 +356,9 @@ func TestOutputJSONNoFiles(t *testing.T) {
|
||||||
if out.Status != "skipped" {
|
if out.Status != "skipped" {
|
||||||
t.Errorf("status = %q, want skipped", out.Status)
|
t.Errorf("status = %q, want skipped", out.Status)
|
||||||
}
|
}
|
||||||
|
if out.TraceID != "test-trace-id-456" {
|
||||||
|
t.Errorf("trace_id = %q, want test-trace-id-456", out.TraceID)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func captureStdout(t *testing.T, fn func()) string {
|
func captureStdout(t *testing.T, fn func()) string {
|
||||||
|
|
|
||||||
|
|
@ -13,6 +13,8 @@ import (
|
||||||
"github.com/open-code-review/open-code-review/internal/mcp"
|
"github.com/open-code-review/open-code-review/internal/mcp"
|
||||||
"github.com/open-code-review/open-code-review/internal/telemetry"
|
"github.com/open-code-review/open-code-review/internal/telemetry"
|
||||||
"github.com/open-code-review/open-code-review/internal/tool"
|
"github.com/open-code-review/open-code-review/internal/tool"
|
||||||
|
|
||||||
|
"go.opentelemetry.io/otel/codes"
|
||||||
)
|
)
|
||||||
|
|
||||||
func runReview(args []string) error {
|
func runReview(args []string) error {
|
||||||
|
|
@ -104,11 +106,23 @@ func runReview(args []string) error {
|
||||||
|
|
||||||
ctx, span := telemetry.StartSpan(context.Background(), "review.run")
|
ctx, span := telemetry.StartSpan(context.Background(), "review.run")
|
||||||
defer span.End()
|
defer span.End()
|
||||||
|
telemetry.SetAttr(span, "review.repo", cc.RepoDir)
|
||||||
|
telemetry.SetAttr(span, "review.from", opts.from)
|
||||||
|
telemetry.SetAttr(span, "review.to", opts.to)
|
||||||
|
telemetry.SetAttr(span, "review.model", rt.Model)
|
||||||
|
var traceID string
|
||||||
|
if telemetry.IsEnabled() {
|
||||||
|
traceID = telemetry.TraceIDFromContext(ctx)
|
||||||
|
if opts.outputFormat != "json" {
|
||||||
|
fmt.Fprintf(os.Stderr, "[ocr] TraceID: %s\n", traceID)
|
||||||
|
}
|
||||||
|
}
|
||||||
startTime := time.Now()
|
startTime := time.Now()
|
||||||
|
|
||||||
comments, err := ag.Run(ctx)
|
comments, err := ag.Run(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
telemetry.SetAttr(span, "error", err.Error())
|
span.SetStatus(codes.Error, err.Error())
|
||||||
|
span.RecordError(err)
|
||||||
return fmt.Errorf("review failed: %w", err)
|
return fmt.Errorf("review failed: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -3,6 +3,7 @@ package main
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"os"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
|
@ -11,6 +12,8 @@ import (
|
||||||
"github.com/open-code-review/open-code-review/internal/scan"
|
"github.com/open-code-review/open-code-review/internal/scan"
|
||||||
"github.com/open-code-review/open-code-review/internal/telemetry"
|
"github.com/open-code-review/open-code-review/internal/telemetry"
|
||||||
"github.com/open-code-review/open-code-review/internal/tool"
|
"github.com/open-code-review/open-code-review/internal/tool"
|
||||||
|
|
||||||
|
"go.opentelemetry.io/otel/codes"
|
||||||
)
|
)
|
||||||
|
|
||||||
// scanOptions mirrors reviewOptions for the full-scan subcommand. The two
|
// scanOptions mirrors reviewOptions for the full-scan subcommand. The two
|
||||||
|
|
@ -207,11 +210,19 @@ func runScan(args []string) error {
|
||||||
|
|
||||||
ctx, span := telemetry.StartSpan(context.Background(), "scan.run")
|
ctx, span := telemetry.StartSpan(context.Background(), "scan.run")
|
||||||
defer span.End()
|
defer span.End()
|
||||||
|
var traceID string
|
||||||
|
if telemetry.IsEnabled() {
|
||||||
|
traceID = telemetry.TraceIDFromContext(ctx)
|
||||||
|
if opts.outputFormat != "json" {
|
||||||
|
fmt.Fprintf(os.Stderr, "[ocr] TraceID: %s\n", traceID)
|
||||||
|
}
|
||||||
|
}
|
||||||
startTime := time.Now()
|
startTime := time.Now()
|
||||||
|
|
||||||
comments, err := ag.Run(ctx)
|
comments, err := ag.Run(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
telemetry.SetAttr(span, "error", err.Error())
|
span.SetStatus(codes.Error, err.Error())
|
||||||
|
span.RecordError(err)
|
||||||
return fmt.Errorf("scan failed: %w", err)
|
return fmt.Errorf("scan failed: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -276,8 +276,10 @@ func emitRunResult(
|
||||||
telemetry.RecordCommentsGenerated(ctx, int64(len(comments)))
|
telemetry.RecordCommentsGenerated(ctx, int64(len(comments)))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
traceID := telemetry.TraceIDFromContext(ctx)
|
||||||
|
|
||||||
if outputFormat == "json" && len(comments) == 0 && ag.FilesReviewed() == 0 {
|
if outputFormat == "json" && len(comments) == 0 && ag.FilesReviewed() == 0 {
|
||||||
return outputJSONNoFiles()
|
return outputJSONNoFiles(traceID)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Agent-text audiences need stdout back before PrintTraceSummary so the
|
// Agent-text audiences need stdout back before PrintTraceSummary so the
|
||||||
|
|
@ -296,7 +298,7 @@ func emitRunResult(
|
||||||
return outputJSONWithWarnings(comments, ag.Warnings(), ag.FilesReviewed(),
|
return outputJSONWithWarnings(comments, ag.Warnings(), ag.FilesReviewed(),
|
||||||
ag.TotalInputTokens(), ag.TotalOutputTokens(), ag.TotalTokensUsed(),
|
ag.TotalInputTokens(), ag.TotalOutputTokens(), ag.TotalTokensUsed(),
|
||||||
ag.TotalCacheReadTokens(), ag.TotalCacheWriteTokens(), duration,
|
ag.TotalCacheReadTokens(), ag.TotalCacheWriteTokens(), duration,
|
||||||
ag.ProjectSummary(), ag.ToolCalls())
|
ag.ProjectSummary(), ag.ToolCalls(), traceID)
|
||||||
}
|
}
|
||||||
outputTextWithWarnings(comments, ag.Warnings())
|
outputTextWithWarnings(comments, ag.Warnings())
|
||||||
if summary := ag.ProjectSummary(); summary != "" {
|
if summary := ag.ProjectSummary(); summary != "" {
|
||||||
|
|
|
||||||
2
go.mod
2
go.mod
|
|
@ -13,7 +13,9 @@ require (
|
||||||
github.com/pkoukk/tiktoken-go v0.1.8
|
github.com/pkoukk/tiktoken-go v0.1.8
|
||||||
go.opentelemetry.io/otel v1.44.0
|
go.opentelemetry.io/otel v1.44.0
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.44.0
|
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.44.0
|
||||||
|
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.44.0
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.44.0
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.44.0
|
||||||
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.44.0
|
||||||
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.44.0
|
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.44.0
|
||||||
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.44.0
|
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.44.0
|
||||||
go.opentelemetry.io/otel/metric v1.44.0
|
go.opentelemetry.io/otel/metric v1.44.0
|
||||||
|
|
|
||||||
4
go.sum
4
go.sum
|
|
@ -109,10 +109,14 @@ go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
|
||||||
go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
|
go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.44.0 h1:SUplec5dp06reu1zaXmOXdvqH398taqrDXqUl99jxSc=
|
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.44.0 h1:SUplec5dp06reu1zaXmOXdvqH398taqrDXqUl99jxSc=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.44.0/go.mod h1:ho2g4N+ane+swq5I/VBkKWnRDY4kUINH3FuqyZqX/Ug=
|
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.44.0/go.mod h1:ho2g4N+ane+swq5I/VBkKWnRDY4kUINH3FuqyZqX/Ug=
|
||||||
|
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.44.0 h1:RuynHbfU8JUEw7DyONgkVYg2SVtsoF28y0LGIr69jgA=
|
||||||
|
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.44.0/go.mod h1:qZF+/lBs71APw8mlnEZcqZHMzqrYrsFiJOv83lX1OGo=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.44.0 h1:4YsVu3B8+3qtWYYrsUYgn0OG78pN0rnNPRGX4SbokQI=
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.44.0 h1:4YsVu3B8+3qtWYYrsUYgn0OG78pN0rnNPRGX4SbokQI=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.44.0/go.mod h1:+wnlSn0mD1ADVMe3v9Z/WIaiz6q6gL2J/ejaAmdmv80=
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.44.0/go.mod h1:+wnlSn0mD1ADVMe3v9Z/WIaiz6q6gL2J/ejaAmdmv80=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.44.0 h1:qazEJlUOQzhCpzQpFETGby7EdqjI1wsd0W+6Gg1SCTU=
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.44.0 h1:qazEJlUOQzhCpzQpFETGby7EdqjI1wsd0W+6Gg1SCTU=
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.44.0/go.mod h1:fOD2Yefuxixkx3ahVNf0O/PERb6r4OlbxfATVnYvzCo=
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.44.0/go.mod h1:fOD2Yefuxixkx3ahVNf0O/PERb6r4OlbxfATVnYvzCo=
|
||||||
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.44.0 h1:lgh3PiVrRUWMLOVSkQicxzZll5NjF1r+AtsX1XRIHw0=
|
||||||
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.44.0/go.mod h1:5Cnhth3m/AgOeTgE3ex12pPmiu/gGtZit03kSzx9X7s=
|
||||||
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.44.0 h1:hqxVTu/GtBF+vJ8d1fzW7fRxZFvgoDjWcxwwCaFDYpU=
|
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.44.0 h1:hqxVTu/GtBF+vJ8d1fzW7fRxZFvgoDjWcxwwCaFDYpU=
|
||||||
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.44.0/go.mod h1:z5fVEF4X5v0ESvlJqBrrFlBVoj5EQuefZpzsu7R+x5Q=
|
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.44.0/go.mod h1:z5fVEF4X5v0ESvlJqBrrFlBVoj5EQuefZpzsu7R+x5Q=
|
||||||
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.44.0 h1:bl2S7Ubua0Nms+D/gAmznQTd4dxxMA93aKbcpKqiTCs=
|
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.44.0 h1:bl2S7Ubua0Nms+D/gAmznQTd4dxxMA93aKbcpKqiTCs=
|
||||||
|
|
|
||||||
|
|
@ -22,6 +22,8 @@ import (
|
||||||
"github.com/open-code-review/open-code-review/internal/stdout"
|
"github.com/open-code-review/open-code-review/internal/stdout"
|
||||||
"github.com/open-code-review/open-code-review/internal/telemetry"
|
"github.com/open-code-review/open-code-review/internal/telemetry"
|
||||||
"github.com/open-code-review/open-code-review/internal/tool"
|
"github.com/open-code-review/open-code-review/internal/tool"
|
||||||
|
|
||||||
|
"go.opentelemetry.io/otel/codes"
|
||||||
)
|
)
|
||||||
|
|
||||||
// AgentWarning is re-exported from llmloop for backwards compatibility with
|
// AgentWarning is re-exported from llmloop for backwards compatibility with
|
||||||
|
|
@ -487,7 +489,17 @@ func (a *Agent) executeSubtask(ctx context.Context, d model.Diff) error {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
err := a.runner.RunPerFile(ctx, messages, newPath)
|
err := func() error {
|
||||||
|
ctx, mainSpan := telemetry.StartSpan(ctx, "main.loop")
|
||||||
|
defer mainSpan.End()
|
||||||
|
telemetry.SetAttr(mainSpan, "file.path", newPath)
|
||||||
|
if err := a.runner.RunPerFile(ctx, messages, newPath); err != nil {
|
||||||
|
mainSpan.SetStatus(codes.Error, err.Error())
|
||||||
|
mainSpan.RecordError(err)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}()
|
||||||
if err == nil {
|
if err == nil {
|
||||||
// REVIEW_FILTER_TASK runs after the main loop and decides which of the
|
// REVIEW_FILTER_TASK runs after the main loop and decides which of the
|
||||||
// just-collected comments to drop. It needs to see comments produced by
|
// just-collected comments to drop. It needs to see comments produced by
|
||||||
|
|
@ -503,6 +515,10 @@ func (a *Agent) executeSubtask(ctx context.Context, d model.Diff) error {
|
||||||
// executeReviewFilter runs the REVIEW_FILTER_TASK to remove comments that are
|
// executeReviewFilter runs the REVIEW_FILTER_TASK to remove comments that are
|
||||||
// provably incorrect based solely on the diff. Errors are logged and silently ignored.
|
// provably incorrect based solely on the diff. Errors are logged and silently ignored.
|
||||||
func (a *Agent) executeReviewFilter(ctx context.Context, d model.Diff, newPath string) {
|
func (a *Agent) executeReviewFilter(ctx context.Context, d model.Diff, newPath string) {
|
||||||
|
ctx, span := telemetry.StartSpan(ctx, "review_filter.execute")
|
||||||
|
defer span.End()
|
||||||
|
telemetry.SetAttr(span, "file.path", newPath)
|
||||||
|
|
||||||
ft := a.args.Template.ReviewFilterTask
|
ft := a.args.Template.ReviewFilterTask
|
||||||
if ft == nil || len(ft.Messages) == 0 {
|
if ft == nil || len(ft.Messages) == 0 {
|
||||||
return
|
return
|
||||||
|
|
@ -512,6 +528,7 @@ func (a *Agent) executeReviewFilter(ctx context.Context, d model.Diff, newPath s
|
||||||
if len(comments) == 0 {
|
if len(comments) == 0 {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
telemetry.SetAttr(span, "comments.before", len(comments))
|
||||||
|
|
||||||
commentsJSON := buildFilterCommentsJSON(comments)
|
commentsJSON := buildFilterCommentsJSON(comments)
|
||||||
|
|
||||||
|
|
@ -534,20 +551,33 @@ func (a *Agent) executeReviewFilter(ctx context.Context, d model.Diff, newPath s
|
||||||
rec := fs.AppendTaskRecord(session.ReviewFilterTask, messages)
|
rec := fs.AppendTaskRecord(session.ReviewFilterTask, messages)
|
||||||
startTime := time.Now()
|
startTime := time.Now()
|
||||||
|
|
||||||
|
_, llmSpan := telemetry.StartLLMSpan(ctx, a.args.Model)
|
||||||
resp, err := a.args.LLMClient.CompletionsWithCtx(ctx, llm.ChatRequest{
|
resp, err := a.args.LLMClient.CompletionsWithCtx(ctx, llm.ChatRequest{
|
||||||
Model: a.args.Model,
|
Model: a.args.Model,
|
||||||
Messages: messages,
|
Messages: messages,
|
||||||
MaxTokens: a.args.Template.MaxTokens,
|
MaxTokens: a.args.Template.MaxTokens,
|
||||||
})
|
})
|
||||||
|
duration := time.Since(startTime)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
rec.SetError(err, time.Since(startTime))
|
telemetry.RecordLLMResult(llmSpan, duration, 0, err)
|
||||||
|
llmSpan.End()
|
||||||
|
rec.SetError(err, duration)
|
||||||
fmt.Fprintf(stdout.Writer(), "[ocr] Review filter failed for %s: %v\n", newPath, err)
|
fmt.Fprintf(stdout.Writer(), "[ocr] Review filter failed for %s: %v\n", newPath, err)
|
||||||
|
span.SetStatus(codes.Error, err.Error())
|
||||||
|
span.RecordError(err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
rec.SetResponse(resp, time.Since(startTime))
|
var totalTokens int64
|
||||||
|
if resp.Usage != nil {
|
||||||
|
totalTokens = resp.Usage.TotalTokens
|
||||||
|
}
|
||||||
|
telemetry.RecordLLMResult(llmSpan, duration, totalTokens, nil)
|
||||||
|
llmSpan.End()
|
||||||
|
rec.SetResponse(resp, duration)
|
||||||
a.runner.RecordUsage(resp.Usage)
|
a.runner.RecordUsage(resp.Usage)
|
||||||
|
|
||||||
indices := parseFilterResponse(resp.Content(), len(comments))
|
indices := parseFilterResponse(resp.Content(), len(comments))
|
||||||
|
telemetry.SetAttr(span, "comments.filtered", len(indices))
|
||||||
if len(indices) == 0 {
|
if len(indices) == 0 {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
@ -722,6 +752,10 @@ func (a *Agent) extFromPath(path string) string {
|
||||||
// executePlanPhase runs the plan task for a single file, sending template messages
|
// executePlanPhase runs the plan task for a single file, sending template messages
|
||||||
// with resolved placeholders and collecting the LLM response as plan guidance.
|
// with resolved placeholders and collecting the LLM response as plan guidance.
|
||||||
func (a *Agent) executePlanPhase(ctx context.Context, newPath, rawDiff, changeFiles, rule string) (string, error) {
|
func (a *Agent) executePlanPhase(ctx context.Context, newPath, rawDiff, changeFiles, rule string) (string, error) {
|
||||||
|
ctx, span := telemetry.StartSpan(ctx, "plan.execute")
|
||||||
|
defer span.End()
|
||||||
|
telemetry.SetAttr(span, "file.path", newPath)
|
||||||
|
|
||||||
pt := a.args.Template.PlanTask
|
pt := a.args.Template.PlanTask
|
||||||
messages := make([]llm.Message, 0, len(pt.Messages))
|
messages := make([]llm.Message, 0, len(pt.Messages))
|
||||||
for _, m := range pt.Messages {
|
for _, m := range pt.Messages {
|
||||||
|
|
@ -740,16 +774,28 @@ func (a *Agent) executePlanPhase(ctx context.Context, newPath, rawDiff, changeFi
|
||||||
rec := fs.AppendTaskRecord(session.PlanTask, messages)
|
rec := fs.AppendTaskRecord(session.PlanTask, messages)
|
||||||
startTime := time.Now()
|
startTime := time.Now()
|
||||||
|
|
||||||
|
_, llmSpan := telemetry.StartLLMSpan(ctx, a.args.Model)
|
||||||
resp, err := a.args.LLMClient.CompletionsWithCtx(ctx, llm.ChatRequest{
|
resp, err := a.args.LLMClient.CompletionsWithCtx(ctx, llm.ChatRequest{
|
||||||
Model: a.args.Model,
|
Model: a.args.Model,
|
||||||
Messages: messages,
|
Messages: messages,
|
||||||
MaxTokens: a.args.Template.MaxTokens,
|
MaxTokens: a.args.Template.MaxTokens,
|
||||||
})
|
})
|
||||||
|
duration := time.Since(startTime)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
rec.SetError(err, time.Since(startTime))
|
telemetry.RecordLLMResult(llmSpan, duration, 0, err)
|
||||||
|
llmSpan.End()
|
||||||
|
rec.SetError(err, duration)
|
||||||
|
span.SetStatus(codes.Error, err.Error())
|
||||||
|
span.RecordError(err)
|
||||||
return "", fmt.Errorf("plan request: %w", err)
|
return "", fmt.Errorf("plan request: %w", err)
|
||||||
}
|
}
|
||||||
rec.SetResponse(resp, time.Since(startTime))
|
var totalTokens int64
|
||||||
|
if resp.Usage != nil {
|
||||||
|
totalTokens = resp.Usage.TotalTokens
|
||||||
|
}
|
||||||
|
telemetry.RecordLLMResult(llmSpan, duration, totalTokens, nil)
|
||||||
|
llmSpan.End()
|
||||||
|
rec.SetResponse(resp, duration)
|
||||||
a.runner.RecordUsage(resp.Usage)
|
a.runner.RecordUsage(resp.Usage)
|
||||||
fmt.Fprintf(stdout.Writer(), "[ocr] Plan completed for %s\n", newPath)
|
fmt.Fprintf(stdout.Writer(), "[ocr] Plan completed for %s\n", newPath)
|
||||||
return resp.Content(), nil
|
return resp.Content(), nil
|
||||||
|
|
|
||||||
|
|
@ -10,6 +10,7 @@ import (
|
||||||
"github.com/open-code-review/open-code-review/internal/llm"
|
"github.com/open-code-review/open-code-review/internal/llm"
|
||||||
"github.com/open-code-review/open-code-review/internal/model"
|
"github.com/open-code-review/open-code-review/internal/model"
|
||||||
"github.com/open-code-review/open-code-review/internal/stdout"
|
"github.com/open-code-review/open-code-review/internal/stdout"
|
||||||
|
"github.com/open-code-review/open-code-review/internal/telemetry"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ReLocateComment calls the LLM to regenerate a precise existing_code snippet
|
// ReLocateComment calls the LLM to regenerate a precise existing_code snippet
|
||||||
|
|
@ -44,15 +45,26 @@ func ReLocateComment(
|
||||||
messages = append(messages, llm.NewTextMessage(m.Role, content))
|
messages = append(messages, llm.NewTextMessage(m.Role, content))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
startTime := time.Now()
|
||||||
|
_, llmSpan := telemetry.StartLLMSpan(ctx, modelName)
|
||||||
resp, err := client.CompletionsWithCtx(ctx, llm.ChatRequest{
|
resp, err := client.CompletionsWithCtx(ctx, llm.ChatRequest{
|
||||||
Model: modelName,
|
Model: modelName,
|
||||||
Messages: messages,
|
Messages: messages,
|
||||||
MaxTokens: maxTokens,
|
MaxTokens: maxTokens,
|
||||||
})
|
})
|
||||||
|
duration := time.Since(startTime)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
telemetry.RecordLLMResult(llmSpan, duration, 0, err)
|
||||||
|
llmSpan.End()
|
||||||
fmt.Fprintf(stdout.Writer(), "[ocr] Re-location LLM call failed for %s: %v\n", cm.Path, err)
|
fmt.Fprintf(stdout.Writer(), "[ocr] Re-location LLM call failed for %s: %v\n", cm.Path, err)
|
||||||
return false, nil, messages
|
return false, nil, messages
|
||||||
}
|
}
|
||||||
|
var totalTokens int64
|
||||||
|
if resp.Usage != nil {
|
||||||
|
totalTokens = resp.Usage.TotalTokens
|
||||||
|
}
|
||||||
|
telemetry.RecordLLMResult(llmSpan, duration, totalTokens, nil)
|
||||||
|
llmSpan.End()
|
||||||
|
|
||||||
code := extractCodeBlock(resp.Content())
|
code := extractCodeBlock(resp.Content())
|
||||||
if code == "" {
|
if code == "" {
|
||||||
|
|
|
||||||
|
|
@ -163,6 +163,7 @@ func (r *Runner) RunPerFile(ctx context.Context, messages []llm.Message, newPath
|
||||||
rec := fs.AppendTaskRecord(session.MainTask, append([]llm.Message(nil), messages...))
|
rec := fs.AppendTaskRecord(session.MainTask, append([]llm.Message(nil), messages...))
|
||||||
startTime := time.Now()
|
startTime := time.Now()
|
||||||
|
|
||||||
|
_, llmSpan := telemetry.StartLLMSpan(ctx, r.deps.Model)
|
||||||
resp, err := r.deps.LLMClient.CompletionsWithCtx(ctx, llm.ChatRequest{
|
resp, err := r.deps.LLMClient.CompletionsWithCtx(ctx, llm.ChatRequest{
|
||||||
Model: r.deps.Model,
|
Model: r.deps.Model,
|
||||||
Messages: messages,
|
Messages: messages,
|
||||||
|
|
@ -172,6 +173,8 @@ func (r *Runner) RunPerFile(ctx context.Context, messages []llm.Message, newPath
|
||||||
duration := time.Since(startTime)
|
duration := time.Since(startTime)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
rec.SetError(err, duration)
|
rec.SetError(err, duration)
|
||||||
|
telemetry.RecordLLMResult(llmSpan, duration, 0, err)
|
||||||
|
llmSpan.End()
|
||||||
telemetry.RecordLLMRequest(ctx, r.deps.Model, duration, 0, "error")
|
telemetry.RecordLLMRequest(ctx, r.deps.Model, duration, 0, "error")
|
||||||
return fmt.Errorf("LLM completion error: %w", err)
|
return fmt.Errorf("LLM completion error: %w", err)
|
||||||
}
|
}
|
||||||
|
|
@ -184,6 +187,8 @@ func (r *Runner) RunPerFile(ctx context.Context, messages []llm.Message, newPath
|
||||||
atomic.AddInt64(&r.totalCacheReadTokens, resp.Usage.CacheReadTokens)
|
atomic.AddInt64(&r.totalCacheReadTokens, resp.Usage.CacheReadTokens)
|
||||||
atomic.AddInt64(&r.totalCacheWriteTokens, resp.Usage.CacheWriteTokens)
|
atomic.AddInt64(&r.totalCacheWriteTokens, resp.Usage.CacheWriteTokens)
|
||||||
}
|
}
|
||||||
|
telemetry.RecordLLMResult(llmSpan, duration, totalTokens, nil)
|
||||||
|
llmSpan.End()
|
||||||
telemetry.RecordLLMRequest(ctx, r.deps.Model, duration, totalTokens, "ok")
|
telemetry.RecordLLMRequest(ctx, r.deps.Model, duration, totalTokens, "ok")
|
||||||
|
|
||||||
content := resp.Content()
|
content := resp.Content()
|
||||||
|
|
@ -272,14 +277,19 @@ func (r *Runner) executeToolCall(ctx context.Context, newPath string, call llm.T
|
||||||
return tool.Of(fmt.Sprintf("Error parsing tool arguments for %s: %v", call.Function.Name, err))
|
return tool.Of(fmt.Sprintf("Error parsing tool arguments for %s: %v", call.Function.Name, err))
|
||||||
}
|
}
|
||||||
telemetry.PrintToolCallStarted(call.Function.Name, dynArgs)
|
telemetry.PrintToolCallStarted(call.Function.Name, dynArgs)
|
||||||
|
_, toolSpan := telemetry.StartToolSpan(ctx, call.Function.Name)
|
||||||
startTime := time.Now()
|
startTime := time.Now()
|
||||||
result, err := p.Execute(ctx, dynArgs)
|
result, err := p.Execute(ctx, dynArgs)
|
||||||
dur := time.Since(startTime)
|
dur := time.Since(startTime)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
telemetry.RecordToolResult(toolSpan, call.Function.Name, dur.Milliseconds(), err)
|
||||||
|
toolSpan.End()
|
||||||
telemetry.RecordToolCall(ctx, call.Function.Name, dur, false)
|
telemetry.RecordToolCall(ctx, call.Function.Name, dur, false)
|
||||||
telemetry.PrintToolCallError(call.Function.Name, err)
|
telemetry.PrintToolCallError(call.Function.Name, err)
|
||||||
return tool.Of(fmt.Sprintf("Error executing tool %s: %v", call.Function.Name, err))
|
return tool.Of(fmt.Sprintf("Error executing tool %s: %v", call.Function.Name, err))
|
||||||
}
|
}
|
||||||
|
telemetry.RecordToolResult(toolSpan, call.Function.Name, dur.Milliseconds(), nil)
|
||||||
|
toolSpan.End()
|
||||||
telemetry.RecordToolCall(ctx, call.Function.Name, dur, true)
|
telemetry.RecordToolCall(ctx, call.Function.Name, dur, true)
|
||||||
telemetry.PrintToolCallFinished(call.Function.Name, dur)
|
telemetry.PrintToolCallFinished(call.Function.Name, dur)
|
||||||
if rec != nil {
|
if rec != nil {
|
||||||
|
|
@ -314,10 +324,14 @@ func (r *Runner) executeToolCall(ctx context.Context, newPath string, call llm.T
|
||||||
|
|
||||||
if t == tool.CodeComment {
|
if t == tool.CodeComment {
|
||||||
telemetry.PrintToolCallStarted(t.Name(), args)
|
telemetry.PrintToolCallStarted(t.Name(), args)
|
||||||
|
_, toolSpan := telemetry.StartToolSpan(ctx, t.Name())
|
||||||
|
|
||||||
comments, errMsg := tool.ParseComments(args)
|
comments, errMsg := tool.ParseComments(args)
|
||||||
if errMsg != "" {
|
if errMsg != "" {
|
||||||
telemetry.RecordToolCall(ctx, t.Name(), time.Since(startTime), false)
|
dur := time.Since(startTime)
|
||||||
|
telemetry.RecordToolResult(toolSpan, t.Name(), dur.Milliseconds(), fmt.Errorf("%s", errMsg))
|
||||||
|
toolSpan.End()
|
||||||
|
telemetry.RecordToolCall(ctx, t.Name(), dur, false)
|
||||||
return tool.Of(errMsg)
|
return tool.Of(errMsg)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -361,8 +375,13 @@ func (r *Runner) executeToolCall(ctx context.Context, newPath string, call llm.T
|
||||||
asyncCtx := context.WithoutCancel(ctx)
|
asyncCtx := context.WithoutCancel(ctx)
|
||||||
toolName := t.Name()
|
toolName := t.Name()
|
||||||
pool.Submit(func() ([]model.LlmComment, error) {
|
pool.Submit(func() ([]model.LlmComment, error) {
|
||||||
|
defer func() {
|
||||||
|
dur := time.Since(startTime)
|
||||||
|
telemetry.RecordToolResult(toolSpan, toolName, dur.Milliseconds(), nil)
|
||||||
|
toolSpan.End()
|
||||||
|
telemetry.PrintToolCallFinished(toolName, dur)
|
||||||
|
}()
|
||||||
resolveAndCollect(asyncCtx)
|
resolveAndCollect(asyncCtx)
|
||||||
telemetry.PrintToolCallFinished(toolName, time.Since(startTime))
|
|
||||||
return []model.LlmComment{}, nil
|
return []model.LlmComment{}, nil
|
||||||
})
|
})
|
||||||
telemetry.RecordToolCall(asyncCtx, toolName, time.Since(startTime), true)
|
telemetry.RecordToolCall(asyncCtx, toolName, time.Since(startTime), true)
|
||||||
|
|
@ -371,6 +390,8 @@ func (r *Runner) executeToolCall(ctx context.Context, newPath string, call llm.T
|
||||||
|
|
||||||
resolveAndCollect(ctx)
|
resolveAndCollect(ctx)
|
||||||
dur := time.Since(startTime)
|
dur := time.Since(startTime)
|
||||||
|
telemetry.RecordToolResult(toolSpan, t.Name(), dur.Milliseconds(), nil)
|
||||||
|
toolSpan.End()
|
||||||
telemetry.RecordToolCall(ctx, t.Name(), dur, true)
|
telemetry.RecordToolCall(ctx, t.Name(), dur, true)
|
||||||
telemetry.PrintToolCallFinished(t.Name(), dur)
|
telemetry.PrintToolCallFinished(t.Name(), dur)
|
||||||
if rec != nil {
|
if rec != nil {
|
||||||
|
|
@ -381,9 +402,12 @@ func (r *Runner) executeToolCall(ctx context.Context, newPath string, call llm.T
|
||||||
|
|
||||||
// Synchronous path for all other tools
|
// Synchronous path for all other tools
|
||||||
telemetry.PrintToolCallStarted(t.Name(), args)
|
telemetry.PrintToolCallStarted(t.Name(), args)
|
||||||
|
_, toolSpan := telemetry.StartToolSpan(ctx, t.Name())
|
||||||
result, err := p.Execute(ctx, args)
|
result, err := p.Execute(ctx, args)
|
||||||
dur := time.Since(startTime)
|
dur := time.Since(startTime)
|
||||||
ok := err == nil
|
ok := err == nil
|
||||||
|
telemetry.RecordToolResult(toolSpan, t.Name(), dur.Milliseconds(), err)
|
||||||
|
toolSpan.End()
|
||||||
telemetry.RecordToolCall(ctx, t.Name(), dur, ok)
|
telemetry.RecordToolCall(ctx, t.Name(), dur, ok)
|
||||||
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|
|
||||||
|
|
@ -65,7 +65,7 @@ func TestResolveEnv(t *testing.T) {
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "otlp protocol override",
|
name: "otlp protocol passthrough",
|
||||||
envs: map[string]string{"OTEL_EXPORTER_OTLP_PROTOCOL": "http/json"},
|
envs: map[string]string{"OTEL_EXPORTER_OTLP_PROTOCOL": "http/json"},
|
||||||
check: func(t *testing.T, cfg Config) {
|
check: func(t *testing.T, cfg Config) {
|
||||||
if cfg.OTLPProtocol != "http/json" {
|
if cfg.OTLPProtocol != "http/json" {
|
||||||
|
|
|
||||||
|
|
@ -11,7 +11,9 @@ import (
|
||||||
sdktrace "go.opentelemetry.io/otel/sdk/trace"
|
sdktrace "go.opentelemetry.io/otel/sdk/trace"
|
||||||
|
|
||||||
otlpmetricgrpc "go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc"
|
otlpmetricgrpc "go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc"
|
||||||
|
otlpmetrichttp "go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp"
|
||||||
otlptracegrpc "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc"
|
otlptracegrpc "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc"
|
||||||
|
otlptracehttp "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
|
||||||
stdoutmetric "go.opentelemetry.io/otel/exporters/stdout/stdoutmetric"
|
stdoutmetric "go.opentelemetry.io/otel/exporters/stdout/stdoutmetric"
|
||||||
stdouttrace "go.opentelemetry.io/otel/exporters/stdout/stdouttrace"
|
stdouttrace "go.opentelemetry.io/otel/exporters/stdout/stdouttrace"
|
||||||
)
|
)
|
||||||
|
|
@ -27,11 +29,11 @@ func newStdoutMetricExporter() (sdkmetric.Exporter, error) {
|
||||||
}
|
}
|
||||||
|
|
||||||
// parseOTLPEndpoint strips a http:// or https:// scheme from the endpoint and
|
// parseOTLPEndpoint strips a http:// or https:// scheme from the endpoint and
|
||||||
// reports whether the connection should be insecure (plaintext gRPC).
|
// reports whether the connection should be plaintext (insecure).
|
||||||
// Scheme matching is case-insensitive per RFC 3986. A bare host:port (no
|
// Scheme matching is case-insensitive per RFC 3986. A bare host:port (no
|
||||||
// scheme) is left unchanged and defaults to TLS. Any trailing slash left
|
// scheme) is left unchanged and defaults to TLS. Any trailing slash left
|
||||||
// over from a URL-style endpoint (e.g. "http://localhost:4317/") is
|
// over from a URL-style endpoint (e.g. "http://localhost:4317/") is
|
||||||
// trimmed, since otlptracegrpc/otlpmetricgrpc's WithEndpoint expects a bare
|
// trimmed, since both gRPC and HTTP exporters' WithEndpoint expects a bare
|
||||||
// host:port with no path.
|
// host:port with no path.
|
||||||
func parseOTLPEndpoint(endpoint string) (addr string, insecure bool) {
|
func parseOTLPEndpoint(endpoint string) (addr string, insecure bool) {
|
||||||
switch {
|
switch {
|
||||||
|
|
@ -44,8 +46,21 @@ func parseOTLPEndpoint(endpoint string) (addr string, insecure bool) {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// initOTLPProviders sets up OTLP gRPC exporters for traces and metrics.
|
// initOTLPProviders dispatches to the gRPC or HTTP exporter based on cfg.OTLPProtocol.
|
||||||
func initOTLPProviders(ctx context.Context, res *resource.Resource, cfg Config) {
|
func initOTLPProviders(ctx context.Context, res *resource.Resource, cfg Config) {
|
||||||
|
switch cfg.OTLPProtocol {
|
||||||
|
case "http/protobuf", "http/json":
|
||||||
|
initOTLPHTTPProviders(ctx, res, cfg)
|
||||||
|
case "", "grpc":
|
||||||
|
initOTLPGRPCProviders(ctx, res, cfg)
|
||||||
|
default:
|
||||||
|
fmt.Fprintf(os.Stderr, "[ocr] WARNING: unsupported OTLP protocol %q, falling back to gRPC\n", cfg.OTLPProtocol)
|
||||||
|
initOTLPGRPCProviders(ctx, res, cfg)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// initOTLPGRPCProviders sets up OTLP gRPC exporters for traces and metrics.
|
||||||
|
func initOTLPGRPCProviders(ctx context.Context, res *resource.Resource, cfg Config) {
|
||||||
addr, insecure := parseOTLPEndpoint(cfg.OTLPEndpoint)
|
addr, insecure := parseOTLPEndpoint(cfg.OTLPEndpoint)
|
||||||
|
|
||||||
traceOpts := []otlptracegrpc.Option{otlptracegrpc.WithEndpoint(addr)}
|
traceOpts := []otlptracegrpc.Option{otlptracegrpc.WithEndpoint(addr)}
|
||||||
|
|
@ -83,6 +98,45 @@ func initOTLPProviders(ctx context.Context, res *resource.Resource, cfg Config)
|
||||||
shutdownFuncs = append(shutdownFuncs, func(ctx context.Context) error { return mp.Shutdown(ctx) })
|
shutdownFuncs = append(shutdownFuncs, func(ctx context.Context) error { return mp.Shutdown(ctx) })
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// initOTLPHTTPProviders sets up OTLP HTTP exporters for traces and metrics.
|
||||||
|
func initOTLPHTTPProviders(ctx context.Context, res *resource.Resource, cfg Config) {
|
||||||
|
addr, insecure := parseOTLPEndpoint(cfg.OTLPEndpoint)
|
||||||
|
|
||||||
|
traceOpts := []otlptracehttp.Option{otlptracehttp.WithEndpoint(addr)}
|
||||||
|
if insecure {
|
||||||
|
traceOpts = append(traceOpts, otlptracehttp.WithInsecure())
|
||||||
|
}
|
||||||
|
traceExp, err := otlptracehttp.New(ctx, traceOpts...)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "[ocr] WARNING: failed to create OTLP HTTP trace exporter: %v\n", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
tp := sdktrace.NewTracerProvider(
|
||||||
|
sdktrace.WithBatcher(traceExp),
|
||||||
|
sdktrace.WithResource(res),
|
||||||
|
)
|
||||||
|
tracerProvider = tp
|
||||||
|
shutdownFuncs = append(shutdownFuncs, func(ctx context.Context) error { return tp.Shutdown(ctx) })
|
||||||
|
|
||||||
|
metricOpts := []otlpmetrichttp.Option{otlpmetrichttp.WithEndpoint(addr)}
|
||||||
|
if insecure {
|
||||||
|
metricOpts = append(metricOpts, otlpmetrichttp.WithInsecure())
|
||||||
|
}
|
||||||
|
metricExp, err := otlpmetrichttp.New(ctx, metricOpts...)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "[ocr] WARNING: failed to create OTLP HTTP metric exporter: %v\n", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
mp := sdkmetric.NewMeterProvider(
|
||||||
|
sdkmetric.WithReader(sdkmetric.NewPeriodicReader(metricExp)),
|
||||||
|
sdkmetric.WithResource(res),
|
||||||
|
)
|
||||||
|
meterProvider = mp
|
||||||
|
shutdownFuncs = append(shutdownFuncs, func(ctx context.Context) error { return mp.Shutdown(ctx) })
|
||||||
|
}
|
||||||
|
|
||||||
// initConsoleProviders sets up stdout exporters for traces and metrics.
|
// initConsoleProviders sets up stdout exporters for traces and metrics.
|
||||||
func initConsoleProviders(res *resource.Resource) {
|
func initConsoleProviders(res *resource.Resource) {
|
||||||
traceExp, err := newStdoutTraceExporter()
|
traceExp, err := newStdoutTraceExporter()
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,11 @@
|
||||||
package telemetry
|
package telemetry
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"go.opentelemetry.io/otel/sdk/resource"
|
"go.opentelemetry.io/otel/sdk/resource"
|
||||||
|
|
@ -103,3 +107,94 @@ func TestInitOTLPProviders_InvalidEndpoint(t *testing.T) {
|
||||||
t.Error("expected tracerProvider to be set (OTLP exporter creation is lazy)")
|
t.Error("expected tracerProvider to be set (OTLP exporter creation is lazy)")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestInitOTLPHTTPProviders_InvalidEndpoint(t *testing.T) {
|
||||||
|
tracerProvider = nil
|
||||||
|
meterProvider = nil
|
||||||
|
shutdownFuncs = nil
|
||||||
|
defer func() {
|
||||||
|
for _, fn := range shutdownFuncs {
|
||||||
|
_ = fn(context.Background())
|
||||||
|
}
|
||||||
|
tracerProvider = nil
|
||||||
|
meterProvider = nil
|
||||||
|
shutdownFuncs = nil
|
||||||
|
}()
|
||||||
|
|
||||||
|
cfg := Config{
|
||||||
|
Exporter: "otlp",
|
||||||
|
OTLPEndpoint: "localhost:0",
|
||||||
|
OTLPProtocol: "http/protobuf",
|
||||||
|
}
|
||||||
|
initOTLPHTTPProviders(context.Background(), resource.Default(), cfg)
|
||||||
|
if tracerProvider == nil {
|
||||||
|
t.Error("expected tracerProvider to be set (OTLP HTTP exporter creation is lazy)")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInitOTLPProviders_ProtocolRouting(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
protocol string
|
||||||
|
wantWarning bool // default branch emits a gRPC fallback warning
|
||||||
|
}{
|
||||||
|
{"grpc default", "grpc", false},
|
||||||
|
{"empty defaults to grpc", "", false},
|
||||||
|
{"http/protobuf routes to http", "http/protobuf", false},
|
||||||
|
{"http/json routes to http", "http/json", false},
|
||||||
|
{"unknown falls back to grpc", "foo", true},
|
||||||
|
}
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
tracerProvider = nil
|
||||||
|
meterProvider = nil
|
||||||
|
shutdownFuncs = nil
|
||||||
|
|
||||||
|
// Capture stderr to assert on the fallback warning.
|
||||||
|
oldStderr := os.Stderr
|
||||||
|
r, w, _ := os.Pipe()
|
||||||
|
os.Stderr = w
|
||||||
|
|
||||||
|
cfg := Config{
|
||||||
|
Exporter: "otlp",
|
||||||
|
OTLPEndpoint: "localhost:0",
|
||||||
|
OTLPProtocol: tc.protocol,
|
||||||
|
}
|
||||||
|
initOTLPProviders(context.Background(), resource.Default(), cfg)
|
||||||
|
w.Close()
|
||||||
|
os.Stderr = oldStderr
|
||||||
|
|
||||||
|
defer func() {
|
||||||
|
for _, fn := range shutdownFuncs {
|
||||||
|
_ = fn(context.Background())
|
||||||
|
}
|
||||||
|
tracerProvider = nil
|
||||||
|
meterProvider = nil
|
||||||
|
shutdownFuncs = nil
|
||||||
|
}()
|
||||||
|
|
||||||
|
if tracerProvider == nil {
|
||||||
|
t.Error("expected tracerProvider to be set")
|
||||||
|
}
|
||||||
|
if meterProvider == nil {
|
||||||
|
t.Error("expected meterProvider to be set")
|
||||||
|
}
|
||||||
|
if len(shutdownFuncs) != 2 {
|
||||||
|
t.Errorf("expected 2 shutdown funcs, got %d", len(shutdownFuncs))
|
||||||
|
}
|
||||||
|
|
||||||
|
var buf bytes.Buffer
|
||||||
|
io.Copy(&buf, r)
|
||||||
|
stderrOut := buf.String()
|
||||||
|
if tc.wantWarning {
|
||||||
|
if !strings.Contains(stderrOut, "falling back to gRPC") {
|
||||||
|
t.Errorf("expected gRPC fallback warning in stderr, got %q", stderrOut)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if strings.Contains(stderrOut, "falling back to gRPC") {
|
||||||
|
t.Errorf("expected no fallback warning, got %q", stderrOut)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
|
||||||
|
|
@ -3,6 +3,7 @@ package telemetry
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"time"
|
||||||
|
|
||||||
"go.opentelemetry.io/otel"
|
"go.opentelemetry.io/otel"
|
||||||
"go.opentelemetry.io/otel/attribute"
|
"go.opentelemetry.io/otel/attribute"
|
||||||
|
|
@ -23,6 +24,16 @@ func StartSpan(ctx context.Context, name string, opts ...trace.SpanStartOption)
|
||||||
return getTracer().Start(ctx, name, opts...)
|
return getTracer().Start(ctx, name, opts...)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TraceIDFromContext returns the hex-encoded trace ID of the span carried by
|
||||||
|
// ctx, or "" if ctx carries no valid span (e.g. telemetry is disabled).
|
||||||
|
func TraceIDFromContext(ctx context.Context) string {
|
||||||
|
sc := trace.SpanContextFromContext(ctx)
|
||||||
|
if !sc.TraceID().IsValid() {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return sc.TraceID().String()
|
||||||
|
}
|
||||||
|
|
||||||
// EndSpan ends the span and records error status if present.
|
// EndSpan ends the span and records error status if present.
|
||||||
func EndSpan(span trace.Span, err error) {
|
func EndSpan(span trace.Span, err error) {
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|
@ -69,11 +80,35 @@ func RecordToolResult(span trace.Span, toolName string, durationMs int64, err er
|
||||||
SetAttr(span, "tool.status", "error")
|
SetAttr(span, "tool.status", "error")
|
||||||
SetAttr(span, "tool.error", err.Error())
|
SetAttr(span, "tool.error", err.Error())
|
||||||
span.SetStatus(codes.Error, err.Error())
|
span.SetStatus(codes.Error, err.Error())
|
||||||
|
span.RecordError(err)
|
||||||
} else {
|
} else {
|
||||||
SetAttr(span, "tool.status", "ok")
|
SetAttr(span, "tool.status", "ok")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// StartLLMSpan creates a span for an LLM request with standard attributes.
|
||||||
|
func StartLLMSpan(ctx context.Context, model string) (context.Context, trace.Span) {
|
||||||
|
return StartSpan(ctx, "llm.request",
|
||||||
|
trace.WithAttributes(attribute.String("llm.model", model)))
|
||||||
|
}
|
||||||
|
|
||||||
|
// RecordLLMResult sets the outcome of an LLM request on the span.
|
||||||
|
func RecordLLMResult(span trace.Span, duration time.Duration, totalTokens int64, err error) {
|
||||||
|
if span == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
SetAttr(span, "llm.duration_ms", duration.Milliseconds())
|
||||||
|
SetAttr(span, "llm.total_tokens", totalTokens)
|
||||||
|
if err != nil {
|
||||||
|
SetAttr(span, "llm.status", "error")
|
||||||
|
SetAttr(span, "llm.error", err.Error())
|
||||||
|
span.SetStatus(codes.Error, err.Error())
|
||||||
|
span.RecordError(err)
|
||||||
|
} else {
|
||||||
|
SetAttr(span, "llm.status", "ok")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// AnyToAttr converts an arbitrary value to an OTel attribute.KeyValue.
|
// AnyToAttr converts an arbitrary value to an OTel attribute.KeyValue.
|
||||||
func AnyToAttr(k string, v interface{}) attribute.KeyValue {
|
func AnyToAttr(k string, v interface{}) attribute.KeyValue {
|
||||||
switch val := v.(type) {
|
switch val := v.(type) {
|
||||||
|
|
|
||||||
|
|
@ -5,6 +5,7 @@ import (
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
"go.opentelemetry.io/otel/attribute"
|
"go.opentelemetry.io/otel/attribute"
|
||||||
"go.opentelemetry.io/otel/trace"
|
"go.opentelemetry.io/otel/trace"
|
||||||
|
|
@ -138,3 +139,34 @@ func TestRecordToolResult_NilSpan(t *testing.T) {
|
||||||
RecordToolResult(nil, "tool", 100, nil)
|
RecordToolResult(nil, "tool", 100, nil)
|
||||||
RecordToolResult(nil, "tool", 100, fmt.Errorf("err"))
|
RecordToolResult(nil, "tool", 100, fmt.Errorf("err"))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestStartLLMSpan(t *testing.T) {
|
||||||
|
setupEnabledTelemetry(t)
|
||||||
|
ctx, span := StartLLMSpan(context.Background(), "qwen-max")
|
||||||
|
if !span.SpanContext().IsValid() {
|
||||||
|
t.Error("expected valid span context")
|
||||||
|
}
|
||||||
|
if ctx == nil {
|
||||||
|
t.Error("expected non-nil context")
|
||||||
|
}
|
||||||
|
span.End()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRecordLLMResult_Success(t *testing.T) {
|
||||||
|
setupEnabledTelemetry(t)
|
||||||
|
_, span := StartSpan(context.Background(), "test.llm")
|
||||||
|
RecordLLMResult(span, 500*time.Millisecond, 1200, nil)
|
||||||
|
span.End()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRecordLLMResult_Error(t *testing.T) {
|
||||||
|
setupEnabledTelemetry(t)
|
||||||
|
_, span := StartSpan(context.Background(), "test.llm")
|
||||||
|
RecordLLMResult(span, 100*time.Millisecond, 0, errors.New("timeout"))
|
||||||
|
span.End()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRecordLLMResult_NilSpan(t *testing.T) {
|
||||||
|
RecordLLMResult(nil, 100*time.Millisecond, 0, nil)
|
||||||
|
RecordLLMResult(nil, 100*time.Millisecond, 0, fmt.Errorf("err"))
|
||||||
|
}
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue