open-notebook/scripts/check_md_links.py
Luis Novo b38c74cf29
docs: restructure documentation around AGENTS.md, VISION.md and decision records (#1032)
* docs: restructure documentation around AGENTS.md, VISION.md and decision records

- Consolidate 17 CLAUDE.md files into 3 AGENTS.md (root, backend, frontend);
  CLAUDE.md files become @AGENTS.md pointers
- Add VISION.md: product identity + current posture with horizon clusters
- Add docs/7-DEVELOPMENT/decisions/ with 4 retroactive ADRs and 2 PDRs
- Add 5 new engineering docs pages (credentials, content-processing,
  podcasts, prompts, frontend) absorbing knowledge from removed CLAUDE.md
- Dismember TRIAGE.md: label taxonomy into maintainer-guide.md, product
  jurisprudence into VISION.md, operator heuristics stay local (gitignored)
- Add AI-assisted/agent-generated PR guidelines to contributing.md
- Convert README.dev.md into a pointer after migrating its unique content
  (make workflow matrix, Docker publishing, add-a-language playbook)
- Fix stale docs: migration path/format, provider count, locale list;
  fix broken links (docs/index.md, PR template, CONFIGURATION.md)

* docs: fix README doc links and add markdown link check to CI

- Repoint 9 README links to pages that actually exist in docs/
- Replace literal (link) placeholder in maintainer-guide templates
- Add scripts/check_md_links.py validating relative links in tracked
  markdown (skips URLs, anchors and code spans)
- Add docs-links workflow running the check on PRs that touch markdown

* docs: add documentation restructure to changelog

* feat: add cubic.yaml with project-aware AI review agents

Three custom review agents (vision & principles alignment backed by
VISION.md, known mechanical caveats, security & testability), PR-contract
review instructions, and automatic ultrareviews for auth, credential,
encryption and migration changes.

* docs: graduate issue-first policy by change size

Small obvious fixes (typos, docs, tiny bugs, i18n completions) no longer
require an issue; features and architecture changes still do. Sizeable
PRs opened without an issue convert to draft while the issue goes
through triage (1-2 days). Applied consistently across contributing
guide, root CONTRIBUTING pointer, PR template, maintainer guide red
flags and cubic review instructions.

* docs: align PR template Related Issue section with graduated issue-first policy

* docs: address review — generalize ADR-002/004, unwrap hard-wrapped lines

- ADR-002 now records the general delegation rule (platform/media support
  that needs heavy coding lives in focused external libraries) covering
  Esperanto, Content Core and podcast-creator
- ADR-004 now records the durable decision (long-running work runs on
  background workers — heavy content, varied machine sizes, never lock
  usage) with the queue technology as a swappable implementation detail
  pending #381
- Remove mid-paragraph hard line wrapping from authored docs to match
  repo convention (one line per paragraph)

* fix: address cubic review — stale doc facts, make dev/full targets, link checker query strings

- credentials.md: only PROVIDER_CONFIG exists as a map; Vertex/Azure/
  OpenAI-compatible provisioning is inline in _provision_*() functions
- content-processing.md: correct ContextConfig priority weights
  (source 100 > insight 75 > note 50)
- development-setup.md + Makefile: make dev/full pointed at root compose
  files that don't exist; targets now use examples/docker-compose-dev.yml
  and examples/docker-compose-full-local.yml with --project-directory .
- check_md_links.py: strip query strings before file-existence checks
2026-07-10 15:33:19 -03:00

75 lines
2.2 KiB
Python

#!/usr/bin/env python3
"""Check that relative markdown links point to files that exist.
Scans every tracked *.md file in the repo and validates relative link targets
(external URLs, anchors and mailto links are skipped; code blocks and inline
code spans are ignored). Exits 1 if any broken link is found.
Run locally: python3 scripts/check_md_links.py
"""
import os
import re
import subprocess
import sys
LINK_RE = re.compile(r"\[[^\]]*\]\(([^)\s]+)\)")
FENCED_CODE_RE = re.compile(r"```.*?```", re.DOTALL)
INLINE_CODE_RE = re.compile(r"`[^`\n]*`")
SKIP_PREFIXES = ("http://", "https://", "mailto:", "#", "<")
def repo_root() -> str:
return subprocess.check_output(
["git", "rev-parse", "--show-toplevel"], text=True
).strip()
def tracked_markdown_files(root: str) -> list[str]:
out = subprocess.check_output(["git", "ls-files", "*.md"], text=True, cwd=root)
return [line for line in out.splitlines() if line]
def main() -> int:
root = repo_root()
broken: list[str] = []
for rel in tracked_markdown_files(root):
path = os.path.join(root, rel)
try:
with open(path, encoding="utf-8") as fh:
text = fh.read()
except OSError:
continue
text = FENCED_CODE_RE.sub("", text)
text = INLINE_CODE_RE.sub("", text)
for match in LINK_RE.finditer(text):
target = match.group(1)
if target.startswith(SKIP_PREFIXES):
continue
file_part = target.split("#")[0].split("?")[0]
if not file_part:
continue
if file_part.startswith("/"):
resolved = os.path.join(root, file_part.lstrip("/"))
else:
resolved = os.path.normpath(
os.path.join(os.path.dirname(path), file_part)
)
if not os.path.exists(resolved):
broken.append(f"{rel}: {target}")
if broken:
print(f"{len(broken)} broken relative link(s):")
for entry in broken:
print(f" {entry}")
return 1
print("All relative markdown links resolve.")
return 0
if __name__ == "__main__":
sys.exit(main())