mirror of
https://github.com/alibaba/open-code-review.git
synced 2026-08-04 14:01:45 +00:00
* feat(session): add run manifest coverage data model and builder First slice of issue #367 (run manifest coverage contract): the data model and state machine only. Not yet wired into the agent or CLI, so existing review/scan output is unchanged. Introduce the versioned, immutable RunManifest (schema ocr.run-manifest/v1) and a concurrency-safe ManifestBuilder that tracks per-file coverage (selected/completed/reused/failed/waived) and freezes into a terminal state. - terminal state derived solely from coverage sets, never comments/warnings (complete/partial/failed/skipped) - Finalize sweeps any undecided selected item to failed/unknown so no item is silently dropped - single-mutex builder: first terminal state wins, frozen after Finalize, nil-receiver safe - fixed failure classification enum with an unknown catch-all - redaction floor on failure/waive reasons (strip secrets, cap length) as a single write entry so callers cannot bypass it - 22 unit tests, race-clean Refs: issue #367 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(session): harden run manifest per adversarial review Address findings from the concurrency / JSON-contract / PR#306-coupling adversarial review of the manifest data model (still slice 1; not wired to agent or CLI). - SetSweepClass: Finalize can classify undispatched items as cancelled/budget instead of a blanket unknown (the one real model gap the review found) - ItemID(fingerprint)=SHA-256 canonical mint helper; an item_id is never a raw fingerprint, keeping the resume cross-reference explicit and mix-ups caught - sanitizeReason: strip control/ANSI chars, coerce valid UTF-8, redact quoted secret values, guarantee single line - Finalize returns deep-copied coverage slices so the frozen snapshot is never aliased across the two outlets - RegisterSelected: nil-safe (lazy-init map) + documents that only the post-deletion/post-filter dispatchable set may be registered +7 unit tests (29 total), race-clean. Refs: issue #367 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(manifest): wire input identity, config hashes and run-level failure (shard ②d) - Freeze per-mode input identity (mode + resolved_base/head + exact_range + source_artifact_sha256) via diff.ResolveInput/commitParents, and repository identity via RemoteIdentity/canonicalRemote (credential-free). - Add rule_config_sha256 and runtime_config_sha256 over an allowlist of non-secret fields using a length-prefixed SHA-256 framework (no tokens/URLs). - Replace SetRunLevelFailure(bool) with structured SetRunFailure(class, reason) and set ManifestInput.mode; fill execution.* (ocr version, provider, model, concurrency, config hashes). - Thread error returns through Finalize/WriteSessionEnd (main review path surfaces them; skip/all-failed/scan paths hardened in follow-up). - Tests: manifest_hash, canonical_config, git_resolve. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(manifest): propagate persistence errors and harden remote/error classification Merged review themes A/B/E from the 07-22 consolidated assessment. Theme A — Finalize / session_end delivery errors no longer swallowed: - agent.go no-files path returns the Finalize error instead of nil (A1) - agent.go loadDiffs failure joins the Finalize error via errors.Join (A2) - session.Finalize uses sync.Once + cached finalizeErr: written exactly once, concurrency-safe, and every caller replays the same result so a retry cannot falsely report success (A3) - scan/agent.go wires both Finalize call sites to surface the error (A4) Theme B — canonicalRemote rewritten (internal/diff/git.go): - keep the port (u.Host, not u.Hostname) so endpoints differing only by port stay distinct (B1) - split scp syntax on the first ':' so an '@' inside the path survives (B2) - recognize local/file/Windows/UNC remotes and omit identity rather than misparsing a path as a host (B3; local-remote policy still open) Theme E — main_task-empty is now a sentinel (errMainTaskEmpty) classified via errors.Is instead of matching error text. Theme D (TOCTOU) deferred to shard 4 per issue #367 open-issues OI-12. Tests: go build ./... + go vet + go test ./... all green (23 pkgs). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(manifest): report both dispatch and persistence errors on the normal path The success-path Finalize wiring used `ferr != nil && err == nil`, so when the review (or scan) failed AND session_end also failed to persist, the persistence error was dropped and only the dispatch error surfaced — the caller never learned the session/manifest was not saved. Join both with errors.Join when both occur (matching the loadDiffs path), so a persistence failure is always reported even alongside a dispatch failure. This closes the last gap in the OI-10 contract. - internal/agent/agent.go: review normal path - internal/scan/agent.go: scan normal path (+ errors import) Tests: go build ./... + go vet + go test ./... all green (23 pkgs). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(manifest): 接入 CLI 与 viewer 并补齐验收用例 - 使用冻结 manifest 统一 review JSON、文本与退出状态\n- session CLI 和 viewer 展示五集合覆盖并兼容 legacy/aborted\n- 补充本地 mock、跨出口一致性及安全验收用例 * test(manifest): 补齐验收矩阵缺口并修复审核发现的缺陷 验收用例:configuration 分类(run 级 sweep + item 级映射)、budget/timeout/panic 混合 partial 隔离、跨出口一致性改为规范化原始字节比对、flag 校验失败无产物断言。 代码修复:sanitizeReason 先剥控制字符再脱敏(堵控制字节绕过)、失败项异分类二次标记报冲突错误、source_artifact_sha256 按 item_id 去重并稳定排序、sortItems 改 SliceStable 对齐设计用词。 全仓 go test 23 包通过。 * test(manifest): 补充 provider transition resume 测试用例 覆盖 issue #367 验收标准 provider transition:resume 时 provider/model 改变后,子 manifest 记录当前值而非继承父运行,并经 parent_run_id 链接父会话以支持审计。用 mock client,不依赖真实 provider key。 * fix(manifest): 对齐预算终态与持久化语义 统一聚合预算停止时的 coverage、status 与退出码。传播 session writer 初始化错误,并补齐 merge first-parent 输入身份及回归测试。移除代码注释中的外部设计文档引用。 --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: kite <254839944+lizhengfeng101@users.noreply.github.com>
364 lines
13 KiB
Go
364 lines
13 KiB
Go
package main
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"net/url"
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/alibaba/open-code-review/internal/agent"
|
|
"github.com/alibaba/open-code-review/internal/config/rules"
|
|
"github.com/alibaba/open-code-review/internal/config/template"
|
|
"github.com/alibaba/open-code-review/internal/config/toolsconfig"
|
|
"github.com/alibaba/open-code-review/internal/diff"
|
|
"github.com/alibaba/open-code-review/internal/gitcmd"
|
|
"github.com/alibaba/open-code-review/internal/llm"
|
|
"github.com/alibaba/open-code-review/internal/model"
|
|
"github.com/alibaba/open-code-review/internal/session"
|
|
"github.com/alibaba/open-code-review/internal/stdout"
|
|
"github.com/alibaba/open-code-review/internal/telemetry"
|
|
"github.com/alibaba/open-code-review/internal/tool"
|
|
)
|
|
|
|
// commonContext bundles the state that both `ocr review` and `ocr scan`
|
|
// need to load *before* deciding whether to dispatch a preview or a real
|
|
// LLM session: a validated template, the resolved repo path, review rules,
|
|
// and a shared git subprocess limiter.
|
|
type commonContext struct {
|
|
Template *template.Template
|
|
RepoDir string
|
|
Resolver rules.Resolver
|
|
FileFilter *rules.FileFilter
|
|
GitRunner *gitcmd.Runner
|
|
// IsGitRepo reports whether RepoDir is inside a git repository. Always
|
|
// true when requireGit was set; may be false when scan accepts non-git
|
|
// directories.
|
|
IsGitRepo bool
|
|
}
|
|
|
|
// loadCommonContext validates the working directory, loads the embedded
|
|
// template, raises MaxToolRequestTimes when maxTools exceeds the default,
|
|
// resolves the absolute repo path, loads system review rules, and creates
|
|
// the global git subprocess limiter. Both review and scan callers go
|
|
// through this so the startup sequence stays consistent.
|
|
//
|
|
// requireGit=true fails fast when the directory is not a git repo (review
|
|
// path: diff concept requires git). requireGit=false allows non-git
|
|
// directories (scan path: provider falls back to filepath.Walk).
|
|
func loadCommonContext(repoDirInput, rulePath string, maxTools, maxGitProcs int, requireGit bool) (*commonContext, error) {
|
|
tpl, err := template.LoadDefault()
|
|
if err != nil {
|
|
return nil, fmt.Errorf("load default template: %w", err)
|
|
}
|
|
if maxTools > tpl.MaxToolRequestTimes {
|
|
tpl.MaxToolRequestTimes = maxTools
|
|
}
|
|
if err := tpl.Validate(); err != nil {
|
|
return nil, fmt.Errorf("invalid config: %w", err)
|
|
}
|
|
|
|
repoDir, isGit, err := resolveWorkingDir(repoDirInput, requireGit)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
resolver, fileFilter, err := rules.NewResolver(repoDir, rulePath)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("load rules: %w", err)
|
|
}
|
|
|
|
return &commonContext{
|
|
Template: tpl,
|
|
RepoDir: repoDir,
|
|
Resolver: resolver,
|
|
FileFilter: fileFilter,
|
|
GitRunner: gitcmd.New(maxGitProcs),
|
|
IsGitRepo: isGit,
|
|
}, nil
|
|
}
|
|
|
|
// resolveWorkingDir returns (absPath, isGitRepo, err). When requireGit is
|
|
// true, returns an error if the directory is not a git repo. When false,
|
|
// returns IsGitRepo=false instead of erroring (scan path uses this).
|
|
func resolveWorkingDir(input string, requireGit bool) (string, bool, error) {
|
|
if input == "" {
|
|
wd, err := os.Getwd()
|
|
if err != nil {
|
|
return "", false, fmt.Errorf("get working directory: %w", err)
|
|
}
|
|
input = wd
|
|
}
|
|
absPath, err := filepath.Abs(input)
|
|
if err != nil {
|
|
return "", false, fmt.Errorf("resolve absolute path: %w", err)
|
|
}
|
|
if _, statErr := os.Stat(absPath); statErr != nil {
|
|
return "", false, fmt.Errorf("stat %s: %w", absPath, statErr)
|
|
}
|
|
out, err := runGitCmd(absPath, "rev-parse", "--git-dir")
|
|
isGit := err == nil && len(out) > 0
|
|
if !isGit && requireGit {
|
|
return "", false, fmt.Errorf("%s is not a git repository", absPath)
|
|
}
|
|
// #287: git reports diff and `git show HEAD:<path>` paths relative to the
|
|
// repository root, not the current directory. When `ocr review` runs from a
|
|
// subdirectory of a monorepo, anchor RepoDir at the git top-level so those
|
|
// root-relative paths resolve for both disk reads and git-show reads.
|
|
// requireGit is true only for the review path; scan (requireGit=false) keeps
|
|
// the CWD so its `git ls-files` walk stays scoped to the subdirectory.
|
|
if isGit && requireGit {
|
|
// runGitCmdStdout captures stdout only so git stderr notices can't
|
|
// pollute the resolved path. --show-toplevel fails (or is empty) when
|
|
// there is no work tree — e.g. a bare repo, where --git-dir succeeds so
|
|
// isGit is true. Fail loudly there instead of silently reusing the
|
|
// subdir, which would reproduce the #287 root-relative-path bug.
|
|
top, topErr := runGitCmdStdout(absPath, "rev-parse", "--show-toplevel")
|
|
t := strings.TrimSpace(string(top))
|
|
if topErr != nil || t == "" {
|
|
return "", false, fmt.Errorf("%s is a git repository without a work tree (bare repo?); cannot resolve its top level for review", absPath)
|
|
}
|
|
absPath = t
|
|
}
|
|
return absPath, isGit, nil
|
|
}
|
|
|
|
// llmRuntime bundles the LLM-side state both subcommands need once they've
|
|
// decided to actually run a session: tool definitions, an app-language
|
|
// adjusted template (mutated in place via ApplyLanguage), the LLM client,
|
|
// the resolved model name, and a fresh comment collector.
|
|
type llmRuntime struct {
|
|
Client llm.LLMClient
|
|
Model string
|
|
Provider string // configured provider name (non-secret label; empty for env-resolved endpoints)
|
|
PlanToolDefs []llm.ToolDef
|
|
MainToolDefs []llm.ToolDef
|
|
Collector *tool.CommentCollector
|
|
AppCfg *Config
|
|
// RuntimeConfig holds the allowlisted, non-secret runtime settings (protocol,
|
|
// sanitized endpoint host, language, timeout) derived from the resolved
|
|
// endpoint and app config, for the run manifest's runtime_config_sha256. It
|
|
// never carries the token or full URL.
|
|
RuntimeConfig agent.RuntimeConfig
|
|
}
|
|
|
|
// loadLLMRuntime loads tool defs from toolConfigPath, reads the app config
|
|
// from the user's default config path (applying the configured language to
|
|
// tpl — defaulting when the config file is absent), resolves the LLM
|
|
// endpoint (honoring modelOverride from --model when non-empty), and
|
|
// returns the runtime bundle. tpl is mutated in place.
|
|
func loadLLMRuntime(tpl *template.Template, toolConfigPath, modelOverride string) (*llmRuntime, error) {
|
|
toolEntries, err := toolsconfig.Load(toolConfigPath)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("load tools: %w", err)
|
|
}
|
|
planToolDefs := agent.BuildToolDefs(toolEntries, true)
|
|
mainToolDefs := agent.BuildToolDefs(toolEntries, false)
|
|
|
|
cfgPath, err := defaultConfigPath()
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
appCfg, err := LoadAppConfig(cfgPath)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("load app config: %w", err)
|
|
}
|
|
// Apply the language directive even when the config file is missing
|
|
// (upstream #fix: ApplyLanguage with empty lang falls back to default).
|
|
var lang, provider string
|
|
if appCfg != nil {
|
|
lang = appCfg.Language
|
|
provider = appCfg.Provider
|
|
}
|
|
tpl.ApplyLanguage(lang)
|
|
|
|
ep, err := llm.ResolveEndpointWithModelOverride(cfgPath, modelOverride)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("resolve LLM endpoint: %w", err)
|
|
}
|
|
|
|
return &llmRuntime{
|
|
Client: llm.NewLLMClient(ep),
|
|
Model: ep.Model,
|
|
Provider: provider,
|
|
PlanToolDefs: planToolDefs,
|
|
MainToolDefs: mainToolDefs,
|
|
Collector: tool.NewCommentCollector(),
|
|
AppCfg: appCfg,
|
|
RuntimeConfig: agent.RuntimeConfig{
|
|
Protocol: ep.Protocol,
|
|
EndpointHost: sanitizeEndpointHost(ep.URL),
|
|
Language: lang,
|
|
Timeout: ep.Timeout,
|
|
},
|
|
}, nil
|
|
}
|
|
|
|
// sanitizeEndpointHost extracts the credential-free host[:port] from a full LLM
|
|
// endpoint URL, dropping scheme, any embedded userinfo, path, query and fragment
|
|
// so no secret material survives into the manifest's runtime_config hash. The
|
|
// host is lowercased for a stable identity (DNS is case-insensitive). An empty
|
|
// or unparseable URL, or one without a host, yields "".
|
|
func sanitizeEndpointHost(rawURL string) string {
|
|
if strings.TrimSpace(rawURL) == "" {
|
|
return ""
|
|
}
|
|
u, err := url.Parse(rawURL)
|
|
if err != nil || u.Host == "" {
|
|
return ""
|
|
}
|
|
return strings.ToLower(u.Host) // u.Host is host[:port]; userinfo lives in u.User
|
|
}
|
|
|
|
// applyCLIExcludes appends user-supplied --exclude patterns (already split
|
|
// into a []string) onto cc.FileFilter.Exclude. Creates the FileFilter if
|
|
// none was returned by rule.json layers. Idempotent on empty input.
|
|
func applyCLIExcludes(cc *commonContext, patterns []string) {
|
|
if len(patterns) == 0 {
|
|
return
|
|
}
|
|
if cc.FileFilter == nil {
|
|
cc.FileFilter = &rules.FileFilter{}
|
|
}
|
|
cc.FileFilter.Exclude = append(cc.FileFilter.Exclude, patterns...)
|
|
}
|
|
|
|
// excludeToolDef returns a copy of defs with any entries whose function name
|
|
// matches name removed. Used by `ocr scan` to hide tools that don't make
|
|
// sense in full-scan mode (e.g. file_read_diff).
|
|
func excludeToolDef(defs []llm.ToolDef, name string) []llm.ToolDef {
|
|
out := make([]llm.ToolDef, 0, len(defs))
|
|
for _, d := range defs {
|
|
if d.Function.Name == name {
|
|
continue
|
|
}
|
|
out = append(out, d)
|
|
}
|
|
return out
|
|
}
|
|
|
|
// quietHandle wraps a stdout.Quiet() restorer so callers can `defer
|
|
// q.Restore()` for safety while emitRunResult restores it early when the
|
|
// agent-text audience needs the trace summary on the user's terminal.
|
|
// Restore is idempotent.
|
|
type quietHandle struct {
|
|
fn func()
|
|
}
|
|
|
|
// newQuietHandle silences stdout when outputFormat=="json" or
|
|
// audience=="agent"; otherwise the returned handle is a no-op restorer.
|
|
func newQuietHandle(outputFormat, audience string) *quietHandle {
|
|
h := &quietHandle{}
|
|
if outputFormat == "json" || audience == "agent" {
|
|
h.fn = stdout.Quiet()
|
|
}
|
|
return h
|
|
}
|
|
|
|
// Restore re-enables stdout. Safe to call multiple times.
|
|
func (h *quietHandle) Restore() {
|
|
if h == nil || h.fn == nil {
|
|
return
|
|
}
|
|
h.fn()
|
|
h.fn = nil
|
|
}
|
|
|
|
// ResultProvider abstracts the metadata both internal/agent.Agent and
|
|
// internal/scan.Agent expose post-run, so emitRunResult can finalize
|
|
// either without knowing which kind it has.
|
|
type ResultProvider interface {
|
|
Diffs() []model.Diff
|
|
FilesReviewed() int64
|
|
TotalInputTokens() int64
|
|
TotalOutputTokens() int64
|
|
TotalTokensUsed() int64
|
|
TotalCacheReadTokens() int64
|
|
TotalCacheWriteTokens() int64
|
|
Warnings() []agent.AgentWarning
|
|
// ProjectSummary is the markdown project-level summary produced by
|
|
// scan's PROJECT_SUMMARY_TASK. Empty for review mode and for scans
|
|
// that skipped / failed the summary phase.
|
|
ProjectSummary() string
|
|
ToolCalls() map[string]int64
|
|
// SessionID returns the persisted session identifier so callers can show it
|
|
// in JSON output or failure diagnostics. Returns "" when no session was
|
|
// created.
|
|
SessionID() string
|
|
// BudgetExceeded reports whether the aggregate token budget gate stopped the
|
|
// run before all files were reviewed. It is a diagnostic signal only — it
|
|
// feeds summary.budget_exceeded and the failure usage record, and never
|
|
// decides the run's terminal state. The terminal state comes solely from the
|
|
// manifest's coverage: the stop marks the undispatched items
|
|
// failed(budget) without recording a run_failure, so it reads as partial
|
|
// whenever anything was covered.
|
|
BudgetExceeded() bool
|
|
// RunManifest returns the frozen v1 coverage result for review runs. Scan
|
|
// remains legacy and returns nil.
|
|
RunManifest() *session.RunManifest
|
|
}
|
|
|
|
type resumeInfoProvider interface {
|
|
ResumeInfo() *agent.ResumeInfo
|
|
}
|
|
|
|
// emitRunResult is the post-LLM-run finalization shared by `ocr review` and
|
|
// `ocr scan`: resolves comment line numbers, records telemetry, restores
|
|
// stdout early for agent-text audiences so the summary is visible, prints
|
|
// the trace summary, and writes the result in the requested format.
|
|
//
|
|
// q is the silencing handle returned by newQuietHandle; pass nil if no
|
|
// silencing was set up (in which case the early restore is a no-op).
|
|
func emitRunResult(
|
|
ctx context.Context,
|
|
ag ResultProvider,
|
|
comments []model.LlmComment,
|
|
startTime time.Time,
|
|
outputFormat, audience string,
|
|
q *quietHandle,
|
|
) error {
|
|
comments = diff.ResolveLineNumbers(comments, ag.Diffs())
|
|
|
|
duration := time.Since(startTime)
|
|
telemetry.RecordReviewDuration(ctx, duration)
|
|
if len(comments) > 0 {
|
|
telemetry.RecordCommentsGenerated(ctx, int64(len(comments)))
|
|
}
|
|
|
|
traceID := telemetry.TraceIDFromContext(ctx)
|
|
manifest := ag.RunManifest()
|
|
|
|
if outputFormat == "json" && manifest == nil && len(comments) == 0 && ag.FilesReviewed() == 0 {
|
|
return outputJSONNoFiles(traceID)
|
|
}
|
|
|
|
// Agent-text audiences need stdout back before PrintTraceSummary so the
|
|
// summary line lands on their terminal.
|
|
if audience == "agent" && outputFormat != "json" {
|
|
q.Restore()
|
|
}
|
|
|
|
if outputFormat != "json" {
|
|
telemetry.PrintTraceSummary(ag.FilesReviewed(), int64(len(comments)),
|
|
ag.TotalInputTokens(), ag.TotalOutputTokens(), ag.TotalTokensUsed(),
|
|
ag.TotalCacheReadTokens(), ag.TotalCacheWriteTokens(), duration)
|
|
}
|
|
|
|
if outputFormat == "json" {
|
|
var resumeInfo *agent.ResumeInfo
|
|
if p, ok := ag.(resumeInfoProvider); ok {
|
|
resumeInfo = p.ResumeInfo()
|
|
}
|
|
return outputJSONWithWarnings(comments, ag.Warnings(), ag.FilesReviewed(),
|
|
ag.TotalInputTokens(), ag.TotalOutputTokens(), ag.TotalTokensUsed(),
|
|
ag.TotalCacheReadTokens(), ag.TotalCacheWriteTokens(), duration,
|
|
ag.ProjectSummary(), ag.ToolCalls(), traceID, resumeInfo, ag.SessionID(), manifest, ag.BudgetExceeded())
|
|
}
|
|
outputTextWithWarnings(comments, ag.Warnings(), manifest)
|
|
if summary := ag.ProjectSummary(); summary != "" {
|
|
fmt.Printf("\n\n──────── Project Summary ────────\n\n%s\n", summary)
|
|
}
|
|
return nil
|
|
}
|