deer-flow/backend/tests/blocking_io/test_skills_install.py
AochenShen99 658c39ccf7
Some checks failed
Backend Blocking IO / backend-blocking-io (push) Waiting to run
Unit Tests / backend-unit-tests (push) Waiting to run
Frontend Unit Tests / frontend-unit-tests (push) Waiting to run
Lint Check / lint-backend (push) Waiting to run
Lint Check / lint-frontend (push) Waiting to run
Replay E2E (front-back contract) / Layer 1 — backend golden (no API key) (push) Waiting to run
Replay E2E (front-back contract) / Layer 2 — full-stack render (no API key) (push) Waiting to run
E2E Tests / e2e-tests (push) Has been cancelled
feat(skills): Add native SkillScan phase 1 for skills (#3033)
* Add phase 1 skill static scanning

* Rework SkillScan phase 1 as native scanner

* refactor(skillscan): align phase 1 with trimmed RFC contract

- SecurityFinding: 7 fields (rule_id, severity, file, line, message,
  remediation, evidence); category/analyzer derive from the rule_id
  prefix, confidence/column/fingerprint/metadata removed
- scan_archive_preflight()/scan_skill_dir() are pure functions: no
  ScanContext, no policy schema; CRITICAL-blocks is a code constant and
  skill_scan.enabled is applied by enforce_static_scan()/callers
- secret-* evidence is redacted before findings leave the scanner
- de-dup keys on (rule_id, file, line) so repeated occurrences keep
  distinct locations for agent self-correction
- cloud-metadata detection consolidated into network-cloud-metadata
- nested zip members get a one-level stdlib magic-byte peek; an
  executable member escalates package-nested-archive to CRITICAL
- install metadata sidecar removed (Phase 7 decides if it is needed)
- rule specs moved next to their analyzers; skillscan/rules/ removed
- tests updated + new anchors: redaction, dedup lines, nested-zip
  escalation, single cloud-metadata rule, bundled-skill zero-CRITICAL

* fix(skillscan): tighten reverse-shell/secret/archive scan rules from review

Address PR #3033 review feedback on the native SkillScan analyzers:

- Reverse-shell false positives: split shell detection by signal strength
  (/dev/tcp/, nc -e stay CRITICAL; bash -i, mkfifo -> new HIGH
  shell-reverse-shell-heuristic, warn->LLM). The Python check is now
  AST-anchored on real socket.socket/os.dup2/subprocess call sites instead
  of raw-text substring matching, so prose/docstrings no longer hard-block.
- Secret evidence: _redact_secret_evidence returns [redacted] with no secret
  bytes (was value[:6], which leaked 2 real token bytes past the prefix).
- Archive DoS: cap outer archive member count (_MAX_ARCHIVE_MEMBERS=4096);
  scan_archive_preflight early-aborts with a package-too-many-members CRITICAL
  finding (routes through the existing blocked->400 fail-closed path).
- shell-destructive-command: broaden the rm -rf matcher to sensitive system
  roots (/home, /usr, /*, --no-preserve-root /) while leaving safe subpaths
  unflagged.
- Dead code: collapse _decode_text_for_analysis to a single decode path and
  drop the unused _TEXT_SUFFIXES set and _has_text_shebang helper.
- local_skill_storage: document why the host_path branch keeps app_config
  possibly-None (lazy kill-switch resolution; avoids eager get_app_config in
  config-free environments such as CI).

Tests: new negative/positive coverage in test_skillscan_native.py. Full
backend suite 6616 passed, 26 skipped.
2026-07-07 21:44:28 +08:00

73 lines
3 KiB
Python

"""Regression anchor: skill archive installation must not block the event loop.
``LocalSkillStorage.ainstall_skill_from_archive`` is the async entry point the
gateway ``POST /skills/install`` route awaits. It extracts the archive,
validates frontmatter, security-scans every installable file, and stages the
skill into the custom directory — all filesystem work that previously ran
inline on the event loop (zip extract, ``rglob`` enumeration, ``read_text``,
``shutil.copytree``). The fix offloads those phases via ``asyncio.to_thread``
while keeping the per-file LLM security scan as the only awaited work; if any
phase regresses back onto the loop, the strict Blockbuster gate raises
``BlockingError`` and this test fails.
Only the external LLM boundary (``scan_skill_content``) is stubbed — the
archive, extraction, validation, and staging all run against the real local
filesystem. Test-side setup IO is itself offloaded with ``asyncio.to_thread``
(matching ``test_agents_router``) so only the production path is exercised on
the loop.
"""
from __future__ import annotations
import asyncio
import zipfile
from pathlib import Path
from types import SimpleNamespace
import pytest
from deerflow.skills.storage.local_skill_storage import LocalSkillStorage
pytestmark = pytest.mark.asyncio
_SKILL_MD = """---
name: loop-skill
description: Anchor fixture skill for the blocking-IO gate.
---
# Loop Skill
Drives the full install pipeline under the Blockbuster gate.
"""
_SUPPORT_MD = "Reference notes scanned by the per-file security pass.\n"
def _build_archive(archive: Path) -> None:
with zipfile.ZipFile(archive, "w") as zf:
zf.writestr("loop-skill/SKILL.md", _SKILL_MD)
zf.writestr("loop-skill/references/usage.md", _SUPPORT_MD)
async def test_install_skill_archive_does_not_block_event_loop(tmp_path: Path, monkeypatch) -> None:
archive = tmp_path / "loop-skill.skill"
await asyncio.to_thread(_build_archive, archive)
async def _allow_scan(content: str, *, executable: bool = False, location: str = "SKILL.md", app_config=None, static_findings=None):
return SimpleNamespace(decision="allow", reason="anchor stub")
# External dependency boundary only: the security scanner is an LLM call.
monkeypatch.setattr("deerflow.skills.installer.scan_skill_content", _allow_scan)
# Constructor resolves paths (one-time, cached in production via
# get_or_new_skill_storage); offloaded here so the anchor exercises only
# the install pipeline itself on the loop.
storage = await asyncio.to_thread(LocalSkillStorage, host_path=str(tmp_path / "skills"))
result = await storage.ainstall_skill_from_archive(archive)
assert result["success"] is True
assert result["skill_name"] == "loop-skill"
installed_md = tmp_path / "skills" / "custom" / "loop-skill" / "SKILL.md"
assert await asyncio.to_thread(installed_md.exists)
assert await asyncio.to_thread((tmp_path / "skills" / "custom" / "loop-skill" / "references" / "usage.md").exists)