Add conditional Verify fix button on Patrol findings

Fourth of the seven contextual Assistant entries. Verify fix is
the post-remediation confirmation step: after a fix has run, the
operator asks the Assistant whether the underlying condition
actually cleared, rather than trusting the fix command's exit
code or the LLM's prior self-verification.

- Widens PatrolAssistantFindingIntent to include 'verify_fix'.
- buildPatrolAssistantFindingPrompt gains a verify_fix branch
  that directs the LLM to check the current evidence against the
  original signal that fired the finding (metrics, resource
  state, recent alerts, service health), then synthesize: is the
  condition cleared, what evidence supports that judgment, how
  confident, and is there residual risk to monitor for. Tool
  calls are allowed; state-changing commands are explicitly
  forbidden — verification is read-only.
- FindingsPanel adds a Verify fix button after Why, gated by
  hasAppliedFix() which returns true for investigation outcomes
  fix_executed, fix_verified, fix_verification_failed, and
  fix_verification_unknown. For fix_queued (no fix has run yet)
  and fix_failed (fix didn't complete) the button is hidden
  because there is nothing applied to verify.
- autoSendInitialPrompt extends to verify_fix; Discuss with
  Assistant unchanged.

Test: new verify_fix-intent prompt-builder case asserts the
verification dimensions (condition cleared, evidence, confidence,
residual / monitor) and the read-only safety boundary, and
isn't either Discuss or Investigate phrasing.
This commit is contained in:
rcourtman 2026-05-11 09:21:36 +01:00
parent dee757c927
commit ac5f140802
3 changed files with 102 additions and 4 deletions

View file

@ -518,7 +518,7 @@ export const FindingsPanel: Component<FindingsPanelProps> = (props) => {
const openFindingInAssistant = async (
finding: UnifiedFinding,
intent: 'discuss' | 'explain' | 'investigate' | 'why',
intent: 'discuss' | 'explain' | 'investigate' | 'why' | 'verify_fix',
) => {
await aiIntelligenceStore.loadPendingApprovals();
const subject = getFindingSubjectPresentation(finding).label;
@ -569,7 +569,10 @@ export const FindingsPanel: Component<FindingsPanelProps> = (props) => {
aiChatStore.openWithPrompt(handoff.prompt, {
...handoff.context,
autoSendInitialPrompt:
intent === 'explain' || intent === 'investigate' || intent === 'why',
intent === 'explain' ||
intent === 'investigate' ||
intent === 'why' ||
intent === 'verify_fix',
});
};
@ -604,6 +607,34 @@ export const FindingsPanel: Component<FindingsPanelProps> = (props) => {
await openFindingInAssistant(finding, 'why');
};
// Verify fix is the post-remediation check. After a fix has been
// executed against this finding, the operator asks the Assistant
// whether it actually worked — confirming the underlying condition
// cleared rather than trusting the fix command's exit code. Only
// meaningful when a fix has been applied (see hasAppliedFix).
const handleVerifyFixFinding = async (finding: UnifiedFinding, e: Event) => {
e.stopPropagation();
await openFindingInAssistant(finding, 'verify_fix');
};
// True when the finding has an investigation outcome indicating that
// some remediation step has run against it — anything past "fix
// queued." For these states, Verify fix is a meaningful action; for
// fix_queued (still awaiting approval) and earlier states there is
// nothing applied yet to verify, and for fix_failed the fix didn't
// complete so verification doesn't apply.
const hasAppliedFix = (finding: UnifiedFinding): boolean => {
switch (finding.investigationOutcome) {
case 'fix_executed':
case 'fix_verified':
case 'fix_verification_failed':
case 'fix_verification_unknown':
return true;
default:
return false;
}
};
// Copy a Markdown summary of the finding to the clipboard so the operator
// can paste it into a chat, ticket, or incident channel. The shape mirrors
// the seven-question schema (title + impact + recommendation + trust
@ -1240,6 +1271,24 @@ export const FindingsPanel: Component<FindingsPanelProps> = (props) => {
</svg>
Why
</button>
<Show when={hasAppliedFix(finding)}>
<button
type="button"
onClick={(e) => handleVerifyFixFinding(finding, e)}
class="px-2 py-1 rounded border border-border hover:bg-surface-hover flex items-center gap-1"
title="Ask Pulse Assistant to verify whether the applied fix actually resolved the underlying condition"
>
<svg class="w-3 h-3" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
d="M9 12l2 2 4-4m5.618-4.016A11.955 11.955 0 0112 2.944a11.955 11.955 0 01-8.618 3.04A12.02 12.02 0 003 9c0 5.591 3.824 10.29 9 11.622 5.176-1.332 9-6.03 9-11.622 0-1.042-.133-2.052-.382-3.016z"
/>
</svg>
Verify fix
</button>
</Show>
<button
type="button"
onClick={(e) => handleCopyFindingSummary(finding, e)}

View file

@ -951,6 +951,35 @@ describe('patrolInvestigationContextModel', () => {
expect(explainPrompt).toContain('vm-101');
});
it('seeds a verify_fix-intent prompt that asks the LLM to confirm the fix actually worked', () => {
// Verify fix is the post-remediation check. After a fix has run,
// the operator asks "did that actually clear the underlying
// condition" — and the LLM should check via Pulse tools rather
// than trust the fix command's exit code. Verification is
// read-only; no state-changing tool calls.
const prompt = buildPatrolAssistantFindingPrompt({
title: 'Backup job failing',
subject: 'vm-101',
description: 'Datastore quota exhausted',
intent: 'verify_fix',
});
expect(prompt).toContain('Verify the fix applied to this Patrol finding');
expect(prompt).toContain('Backup job failing');
expect(prompt).toContain('vm-101');
// The verification dimensions: condition cleared, evidence,
// confidence, residual risk.
expect(prompt.toLowerCase()).toContain('condition');
expect(prompt.toLowerCase()).toContain('cleared');
expect(prompt.toLowerCase()).toContain('evidence');
expect(prompt.toLowerCase()).toContain('confident');
expect(prompt.toLowerCase()).toMatch(/residual|monitor/);
// Read-only safety: no state-changing commands during verification.
expect(prompt.toLowerCase()).toContain('read-only');
expect(prompt).not.toContain("I'd like to discuss");
expect(prompt).not.toContain('Investigate this Patrol finding now');
});
it('seeds a why-intent prompt that focuses on cause, not current state', () => {
// Why-did-this-happen is the diagnostic counterpart to Explain. Where
// Explain says "tell me what we know" and Investigate says "go find

View file

@ -97,7 +97,16 @@ export interface PatrolInvestigationRecordPresentation {
// rather than current state: recent changes around detection time,
// learned correlations, prior incident memory, regression history.
// Action-style and auto-sent like Explain and Investigate.
export type PatrolAssistantFindingIntent = 'discuss' | 'explain' | 'investigate' | 'why';
// 'verify_fix' = post-remediation verification — ask the LLM to confirm
// whether the recently-applied fix actually resolved the underlying
// condition. Only meaningful when a fix has been applied to the
// finding (investigation outcome is one of the fix-* states).
export type PatrolAssistantFindingIntent =
| 'discuss'
| 'explain'
| 'investigate'
| 'why'
| 'verify_fix';
export interface PatrolAssistantFindingPromptInput {
title: string;
@ -557,6 +566,16 @@ export function buildPatrolAssistantFindingPrompt(
'and what would have to be true for the cause to recur. ' +
'If the cause requires verification through a Pulse tool call, do that; do not run anything ' +
'that changes state without operator approval.';
} else if (input.intent === 'verify_fix') {
prompt =
`Verify the fix applied to this Patrol finding: "${title}" on ${subject}. ` +
'A remediation step has been executed against this finding — confirm whether the underlying ' +
'condition has actually cleared. Use the Pulse tools (metrics, resource state, recent alerts, ' +
'service health) to check the current evidence against the original signal that fired this ' +
'finding, not against an unrelated state. ' +
'Then answer: is the condition cleared, what evidence supports that judgment, ' +
'how confident are you, and is there any residual risk the operator should monitor for. ' +
'Do not execute any new state-changing commands; verification is read-only.';
} else {
prompt = `I'd like to discuss this Patrol finding: "${title}" on ${subject}.`;
}
@ -564,7 +583,8 @@ export function buildPatrolAssistantFindingPrompt(
hasRecord &&
input.intent !== 'explain' &&
input.intent !== 'investigate' &&
input.intent !== 'why'
input.intent !== 'why' &&
input.intent !== 'verify_fix'
) {
prompt +=
'\n\nPulse Patrol has a structured investigation record for this finding. Use that record as the main context before suggesting next actions.';