mirror of
https://github.com/razzant/ouroboros.git
synced 2026-10-02 19:58:46 +00:00
Derive the usage-ledger compaction floor from the ledger, not from process memory
The growth guard was a per-process memo keyed by (inode, device, size), and a committed pass REPLACES the file. So at the instant one process folds, every other process's memo stops matching and re-enters a full pass on its next reservation, and a process that has just started has no memo at all while the residue that cannot fold — group rows, retained idempotent and review-attributed rows, terminal rows younger than the 48-hour fold horizon — holds the file above the trigger for good. Each of those passes rewrites the whole monetary authority under the held money lock and copies the entire live file into a new archive segment. Measured on the owner's live install: 100 passes archiving 2.37 GB in three days for 4.88 MB of live-file gain, with 93 of 99 consecutive passes entered after less than 1 MB of growth. A guard that must hold across processes cannot live inside one of them. The pass already stamps what it needs into line 1: the baseline header records source_size_bytes, the size that pass read. maybe_compact_usage_ledger_locked now declines while the file is below that plus the existing USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES, read lock-free through the existing _live_baseline_header (one readline; appends never touch line 1 and the swap is atomic). No new file, no header field, no constant, no lock ordering. The per-process memo stays, because an abort changes no bytes and so leaves the ledger's own floor naming the window that let that pass in: only the memo can throttle the retry. It is also the whole guard on a ledger that states no floor — nothing has folded here yet, the recorded size is not a positive count, or the leading row cannot be read at all. That last case is contained exactly where it was: the floor neither raises into the caller's reservation nor declines in place of the pass, so the corruption still surfaces from the normal ledger read. Disclosed cost: the stamp names the PRE-pass size, so after a high-gain fold the next pass waits until the file outgrows what the last one read. The 20 MB USAGE_LEDGER_WARN_BYTES tripwire and the startup note on state/usage_attempts.jsonl now name that as a third cause of growth, because a decline happens before the pass and leaves no typed event. test_a_committed_fold_arms_the_same_growth_guard_as_an_abort kept its claim but had simulated growth by narrowing the retry window to one byte; it now appends real chains, since the ledger's floor is the size the last pass read.
This commit is contained in:
Ouroboros
parent
ef33a0a956
commit
825563217b
6 changed files with 322 additions and 33 deletions
|
|
@ -479,21 +479,48 @@ exactly the per-row branch taken `weight` times with the sums pre-added.
|
|||
because they report a lost race the next pass simply repeats rather than a
|
||||
cause an operator has to diagnose. A structurally corrupt ledger still fails
|
||||
in the normal read path with the normal error.
|
||||
- Thrash guard: a per-process memo of the last attempted (inode, size); after
|
||||
ANY pass — unprofitable (nothing foldable / no shrink / verify-abort) or
|
||||
committed — the next pass runs only once the file grows by
|
||||
`…_RETRY_GROWTH_BYTES` beyond the size that pass left, or the file is
|
||||
replaced by someone else. A success arms the memo with the COMPACTED size
|
||||
and the new inode, because the threshold alone is no brake: the unfoldable
|
||||
residue (group rows, retained idempotent and review-attributed rows) only
|
||||
grows, so once it reaches the trigger every reservation would run a full
|
||||
rewrite of the authority under the held lock and copy the whole live file
|
||||
into a new archive segment for a gain of a few kilobytes. Profitability is
|
||||
not the question the guard asks; a pass is worth its cost only after real
|
||||
- Thrash guard, in two parts, and a pass runs only past both. The threshold
|
||||
alone is no brake: the unfoldable residue (group rows, retained idempotent
|
||||
and review-attributed rows, terminal rows younger than the fold horizon)
|
||||
only grows, so once it reaches the trigger every reservation would run a
|
||||
full rewrite of the authority under the held lock and copy the whole live
|
||||
file into a new archive segment for a gain of a few kilobytes. Profitability
|
||||
is not the question the guard asks; a pass is worth its cost only after real
|
||||
growth.
|
||||
- **The floor is the ledger's own, so it holds across processes.** The
|
||||
header the last committed pass stamped into line 1 records
|
||||
`source_size_bytes`, the size that pass READ, and no pass runs until the
|
||||
live file reaches it plus `…_RETRY_GROWTH_BYTES`. Read lock-free by
|
||||
`_live_baseline_header` (one `readline`; appends never touch line 1 and
|
||||
the swap is atomic), so every process answers from the same number. This
|
||||
is what a per-process memo cannot do: a committed pass REPLACES the file,
|
||||
so at that instant every other process's memo stops matching and re-enters
|
||||
a pass, and a freshly started process has no memo at all while the residue
|
||||
holds the file above the trigger for good. Measured on the owner's live
|
||||
install before this floor: 100 passes archiving 2.37 GB in three days for
|
||||
4.88 MB of live-file gain, 93 of 99 consecutive passes entered after less
|
||||
than 1 MB of growth. **Disclosed cost:** the stamp names the PRE-pass
|
||||
size, so after a high-gain fold the next pass waits until the file
|
||||
outgrows what the last one read — deliberately conservative, and the 20 MB
|
||||
tripwire below is what watches it. No floor is claimed when the ledger
|
||||
states none: no stamp (nothing has folded here yet), a leading row that
|
||||
cannot be read at all (the caller's own read reports that corruption — the
|
||||
guard neither raises it early nor stands in for the pass), or a recorded
|
||||
size that is not a positive count.
|
||||
- **The per-process memo remains, for the pass that changed nothing.** It
|
||||
holds the last attempted (inode, size) and arms after ANY pass —
|
||||
unprofitable (nothing foldable / no shrink / verify-abort) or committed,
|
||||
where it takes the COMPACTED size and the new inode. An abort leaves the
|
||||
bytes untouched, so the ledger's own floor still names the window that let
|
||||
that pass in; only the memo can throttle the retry. It is also the whole
|
||||
guard on a ledger that states no floor.
|
||||
- `USAGE_LEDGER_WARN_BYTES` (20 MB) stays as the regression tripwire above the
|
||||
mechanism, exactly like the rotation-bounded log warns: it now fires only if
|
||||
compaction is broken or the unfoldable residue itself reaches 20 MB.
|
||||
mechanism, exactly like the rotation-bounded log warns: it fires if
|
||||
compaction is broken, if the unfoldable residue itself reaches 20 MB, or if
|
||||
the floor above is still holding a ledger that has not outgrown what its
|
||||
last pass read. The startup note on `state/usage_attempts.jsonl`
|
||||
(`agent_startup_checks._hot_store_thresholds`) names all three, because the
|
||||
last one is declined before the pass and so leaves no typed event.
|
||||
|
||||
## 10. History readers: model-send reconciliation and audits
|
||||
|
||||
|
|
@ -730,7 +757,11 @@ tests/fixtures_usage_compaction.py)
|
|||
6. **Idempotency survives**: subscription/external replays after compaction
|
||||
dedup (no double charge) and still conflict-check; legacy import stays
|
||||
correct with and without its watermark.
|
||||
7. **Trigger policy**: no compaction below threshold; thrash guard holds;
|
||||
7. **Trigger policy**: no compaction below threshold; the growth guard holds
|
||||
ACROSS processes — the floor a pass stamps into the ledger throttles every
|
||||
other process and every freshly started one, and the per-process memo still
|
||||
throttles a pass that aborted, a ledger that states no floor, and one whose
|
||||
leading row cannot be read (which stays the normal read's error to report);
|
||||
verify-abort leaves the ledger untouched; the pass is entered with the
|
||||
monetary lock demonstrably HELD *and* with that lock's heartbeat wired
|
||||
through — the parameter is required, so a caller that drops it raises
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@ This chapter is the short list of properties the rest of the book must not contr
|
|||
23. **Every call has a bound, and a recorder speaks only for what it collected.** A reviewer's tool call runs under the loop's per-tool timeout narrowed by the inherited dispatch deadline; a call that outlives it is abandoned — its late value sources no receipt and no coverage. A deadline recorder reconciles the turn's own panel at $0 before it writes a terminal reason. Owners: `review_native_episode.py`, `loop_tool_execution.py`, `acceptance_settlement.py`.
|
||||
24. **The thread that answers workers runs only queue-bounded work.** Work that scales with history, the daemon or the network runs off-thread, reads its candidates before it reads liveness (one in-memory live source under `_queue_lock`), stops mutating when its loop generation ends and reaches the daemon attach-only once a stop is in flight. Named residuals on the loop thread: the 300-s zombie reconcile and the usage-ledger lock in the heartbeat handler. Owner: `ouroboros/server_maintenance.py`.
|
||||
25. **An answer that has not arrived is a gap — never a refusal, a failure, a verdict or an owner message.** A direct turn applies its acceptance fence in-process (admission lock, then `_queue_lock` — never the reverse); a pooled request is idempotent by token, acknowledged per request (`<token>.<req>.json`), and a transition is re-sent once while a read never is; only `sealed` is a seal, an absent row is never read as one, and a fence that did not answer buys no model round: the panel runs as advice on `admission_fence_available=false`, delivery seals again, and a blocking install accepts a reviewer-approved answer with the typed note `admission_close_unconfirmed`. Owners: `ouroboros/agent.py`, `supervisor/queue_transitions.py`, `ouroboros/loop_delivery.py`.
|
||||
26. **A guard that must hold across processes derives from the durable artifact it guards, never from process memory.** The usage-ledger compaction floor is the `source_size_bytes` the last committed pass stamped into the live header, so one process's pass throttles every other and every fresh one; a per-process memo throttles only a pass that changed no bytes. Owner: `ouroboros/usage_compaction.py`.
|
||||
|
||||
### 10.1 Continuity data-flow map
|
||||
|
||||
|
|
|
|||
|
|
@ -751,8 +751,10 @@ def _hot_store_thresholds() -> Tuple[Tuple[str, int, str], ...]:
|
|||
"(~0.5s hold at 20MB — see usage_ledger.py); size-triggered "
|
||||
"compaction (usage_compaction.py, CPL4-C6) should hold the file "
|
||||
"far below this — growth can mean broken compaction, a large "
|
||||
"unfoldable residue, a policy abort, or refusal on the name tier "
|
||||
"(no kernel locks). Check usage_ledger_compaction_refused or "
|
||||
"unfoldable residue, a policy abort, refusal on the name tier "
|
||||
"(no kernel locks), or a file that has not yet outgrown the floor "
|
||||
"its last committed pass stamped into the ledger header (declined "
|
||||
"before the pass, so no event). Check usage_ledger_compaction_refused or "
|
||||
"usage_ledger_compaction_skipped in events.jsonl; the two snapshot-race "
|
||||
"exits before archive/swap only log warnings, without a typed event.",
|
||||
),
|
||||
|
|
|
|||
|
|
@ -247,8 +247,10 @@ SCRATCHPAD_MAX_CONTENT_CHARS = 60_000
|
|||
# starving concurrent workers (the 2026-07-23 lock-timeout incident). Warn at
|
||||
# exactly that measured degradation point. Since CPL4-C6, size-triggered
|
||||
# compaction (config.USAGE_LEDGER_COMPACT_BYTES, usage_compaction.py) should
|
||||
# hold the file far below this. Growth can reflect a large unfoldable residue
|
||||
# or compaction that is broken, refused, or skipped. The name tier (no kernel
|
||||
# hold the file far below this. Growth can reflect a large unfoldable residue,
|
||||
# compaction that is broken or refused, or a file that has not yet outgrown the
|
||||
# growth floor its last committed pass stamped into the ledger header (declined
|
||||
# before the pass, so no typed event). The name tier (no kernel
|
||||
# locks) emits usage_ledger_compaction_refused once per process per data root;
|
||||
# a policy abort (_Abort) emits usage_ledger_compaction_skipped once per process
|
||||
# per (data root, reason). The two snapshot-race exits before archive/swap only
|
||||
|
|
|
|||
|
|
@ -939,6 +939,39 @@ def compact_usage_ledger_locked(
|
|||
return receipt
|
||||
|
||||
|
||||
def _durable_growth_floor(root: pathlib.Path) -> Optional[int]:
|
||||
"""The size the last committed pass READ, taken from the ledger itself.
|
||||
|
||||
A pass swaps the file, so the moment one process folds, every other
|
||||
process's inode-keyed memo stops matching and re-enters a full pass on its
|
||||
next reservation — and a process that has just started has no memo at all
|
||||
while the residue keeps the file above the trigger for good. The guard
|
||||
therefore cannot live in process memory: it is a property of the ledger,
|
||||
and the pass already stamps it into line 1 as ``source_size_bytes``, so
|
||||
every process reads the same number. Lock-free, one ``readline``: appends
|
||||
never touch line 1 and the swap is atomic, so the row is complete
|
||||
whichever generation answers.
|
||||
|
||||
``None`` means this ledger states no floor — no stamp, a leading row that
|
||||
cannot be read (the caller's own read reports that, and masking it here
|
||||
would hide it), or a recorded size that is not a positive count. The
|
||||
per-process memo then stays the only guard, exactly as before.
|
||||
|
||||
Disclosed cost: the stamp names the PRE-pass size, so after a high-gain
|
||||
fold the next pass waits until the file outgrows what the last one read.
|
||||
"""
|
||||
try:
|
||||
header = _live_baseline_header(root)
|
||||
except (UsageLedgerCorrupt, OSError):
|
||||
return None
|
||||
if not header:
|
||||
return None
|
||||
size = header.get("source_size_bytes")
|
||||
if not isinstance(size, int) or isinstance(size, bool) or size <= 0:
|
||||
return None
|
||||
return size
|
||||
|
||||
|
||||
def maybe_compact_usage_ledger_locked(
|
||||
root: pathlib.Path | str,
|
||||
*,
|
||||
|
|
@ -947,18 +980,21 @@ def maybe_compact_usage_ledger_locked(
|
|||
"""Opportunistic trigger on the monetary write path (under the held lock).
|
||||
|
||||
``os.stat`` fast-path below ``config.USAGE_LEDGER_COMPACT_BYTES``; above
|
||||
it, a per-process growth guard throttles re-attempts after ANY pass, not
|
||||
only an unprofitable one. A success used to clear the memo, which left the
|
||||
threshold as the only brake: the unfoldable residue (group rows, retained
|
||||
idempotent and review-attributed rows) never shrinks, so once it reaches
|
||||
the trigger every reservation ran a full rewrite of the authority under
|
||||
the held lock and copied the whole live file into a new archive segment
|
||||
for a gain of a few kilobytes. Remembering the COMPACTED size instead
|
||||
makes the memo mean "the ledger size when this process last ran a pass",
|
||||
so the next one waits for ``USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES`` of
|
||||
real growth whatever the last outcome was. Every failure is contained:
|
||||
this never raises into the caller's reservation (a corrupt ledger still
|
||||
fails in the normal read)."""
|
||||
it, TWO growth guards, and a pass runs only past both. The unfoldable
|
||||
residue (group rows, retained idempotent and review-attributed rows, and
|
||||
terminal rows younger than the fold horizon) never shrinks below the
|
||||
trigger, so without a brake every reservation would rewrite the whole
|
||||
authority under the held lock and copy the live file into a new archive
|
||||
segment for a gain of a few kilobytes.
|
||||
|
||||
The durable one is the floor this ledger carries (``_durable_growth_floor``):
|
||||
it is what makes the brake hold ACROSS processes, which a memo cannot.
|
||||
The per-process memo records the size this process's last pass left behind,
|
||||
whatever its outcome, and throttles a pass that aborted — an abort changes
|
||||
no bytes, so the stamp still names the window that let it in.
|
||||
|
||||
Every failure is contained: this never raises into the caller's
|
||||
reservation (a corrupt ledger still fails in the normal read)."""
|
||||
try:
|
||||
root = pathlib.Path(_drive_root(root))
|
||||
except Exception:
|
||||
|
|
@ -972,13 +1008,17 @@ def maybe_compact_usage_ledger_locked(
|
|||
|
||||
if stat.st_size < int(config.USAGE_LEDGER_COMPACT_BYTES):
|
||||
return False
|
||||
retry_growth = int(config.USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES)
|
||||
key = str(root.resolve(strict=False))
|
||||
with _COMPACT_ATTEMPTS_LOCK:
|
||||
prior = _COMPACT_ATTEMPTS.get(key)
|
||||
if prior is not None and prior[:2] == (stat.st_ino, stat.st_dev) and (
|
||||
stat.st_size < prior[2] + int(config.USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES)
|
||||
stat.st_size < prior[2] + retry_growth
|
||||
):
|
||||
return False
|
||||
return False # this process already ran a pass on these bytes
|
||||
floor = _durable_growth_floor(root)
|
||||
if floor is not None and stat.st_size < floor + retry_growth:
|
||||
return False # somebody's pass read this ledger; it has not regrown since
|
||||
receipt: Optional[Dict[str, Any]] = None
|
||||
try:
|
||||
receipt = compact_usage_ledger_locked(root, heartbeat=heartbeat)
|
||||
|
|
|
|||
|
|
@ -239,6 +239,36 @@ def _folds(data_root):
|
|||
return sum(1 for row in _ledger_rows(data_root) if row.get("kind") == "usage_baseline")
|
||||
|
||||
|
||||
def _count_passes(monkeypatch):
|
||||
"""Every entry into the pass itself, whatever its outcome."""
|
||||
entered = []
|
||||
original = uc.compact_usage_ledger_locked
|
||||
|
||||
def counting(root, **kwargs):
|
||||
entered.append(1)
|
||||
return original(root, **kwargs)
|
||||
|
||||
monkeypatch.setattr(uc, "compact_usage_ledger_locked", counting)
|
||||
return entered
|
||||
|
||||
|
||||
def _grow_past(data_root, monkeypatch, target, tag):
|
||||
"""Append real chains until the live file is larger than ``target`` bytes.
|
||||
|
||||
The trigger is lifted while growing, so only the explicit ``_trigger``
|
||||
calls in each test exercise the guard (``reserve_attempt`` runs it too).
|
||||
"""
|
||||
path = data_root / ua.LEDGER_REL
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_BYTES", 10 ** 12)
|
||||
for index in range(400):
|
||||
if path.stat().st_size > target:
|
||||
break
|
||||
_settle(data_root, cost=0.25, cost_final=True, task_id="%s-%d" % (tag, index))
|
||||
else:
|
||||
raise AssertionError("the fixture never grew past %d bytes" % target)
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_BYTES", 1)
|
||||
|
||||
|
||||
def test_a_committed_fold_arms_the_same_growth_guard_as_an_abort(data_root, monkeypatch):
|
||||
"""A pass that COMMITTED must throttle the next one exactly like a pass that
|
||||
aborted: the memo means "the size this process last ran a pass on", not
|
||||
|
|
@ -267,8 +297,11 @@ def test_a_committed_fold_arms_the_same_growth_guard_as_an_abort(data_root, monk
|
|||
assert _skip_events(data_root) == []
|
||||
assert (data_root / ua.LEDGER_REL).read_bytes().startswith(compacted_bytes)
|
||||
|
||||
# Real growth past the threshold releases it, and the new pass folds again.
|
||||
# Real growth releases it, and the new pass folds again. Growth means bytes
|
||||
# that arrived: the ledger's own floor is the size the last pass READ, so a
|
||||
# narrowed retry window no longer stands in for an append.
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES", 1)
|
||||
_grow_past(data_root, monkeypatch, _ledger_rows(data_root)[0]["source_size_bytes"], "regrowth")
|
||||
assert _trigger(data_root) is True
|
||||
assert len(entered) == 2
|
||||
assert _folds(data_root) == 1 # the header is replaced, never accumulated
|
||||
|
|
@ -307,3 +340,183 @@ def test_repeated_chains_cost_at_most_one_pass_per_growth_window(data_root, monk
|
|||
"%d passes for %d appended bytes at a %d-byte guard" % (len(passes), added, growth)
|
||||
)
|
||||
assert all(passes), "a pass that aborts would prove nothing about the guard"
|
||||
|
||||
|
||||
# --- The growth floor ACROSS processes ---------------------------------------
|
||||
#
|
||||
# The memo above is per-process and keyed by the file's identity, and a
|
||||
# committed pass REPLACES that file: at the moment one process folds, every
|
||||
# other process's memo stops matching and re-enters a full pass on the next
|
||||
# reservation, and a process that has just started has no memo at all while the
|
||||
# residue younger than the fold horizon holds the ledger above the trigger for
|
||||
# good. Measured on the owner's live install: 100 passes archiving 2.37 GB in
|
||||
# three days for 4.88 MB of live-file gain, 93 of 99 consecutive passes entered
|
||||
# after less than 1 MB of growth. A guard that must hold across processes
|
||||
# cannot live in one process's memory — and the pass already stamps what it
|
||||
# needs into line 1: `source_size_bytes` is the size it read.
|
||||
|
||||
|
||||
|
||||
def test_a_peer_process_does_not_refold_what_another_just_folded(data_root, monkeypatch):
|
||||
"""Two process memos over one ledger, with the swap between them. The peer's
|
||||
memo names the inode the pass replaced, so the per-process guard is blind to
|
||||
a pass it did not run; only a floor read off the ledger can throttle it."""
|
||||
_fixtures._seed_mixed_ledger(data_root)
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_BYTES", 1)
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES", 1_000_000)
|
||||
uc._COMPACT_ATTEMPTS.clear()
|
||||
key = str(data_root.resolve(strict=False))
|
||||
before = (data_root / ua.LEDGER_REL).stat()
|
||||
peer_memo = (before.st_ino, before.st_dev, before.st_size) # the peer looked first
|
||||
entered = _count_passes(monkeypatch)
|
||||
|
||||
assert _trigger(data_root) is True # this process folds
|
||||
assert (data_root / ua.LEDGER_REL).stat().st_ino != before.st_ino # the swap is real
|
||||
assert _ledger_rows(data_root)[0]["source_size_bytes"] == before.st_size
|
||||
|
||||
uc._COMPACT_ATTEMPTS.clear()
|
||||
uc._COMPACT_ATTEMPTS[key] = peer_memo # the peer, carrying the memo it built
|
||||
for index in range(3):
|
||||
_settle(data_root, cost=0.25, cost_final=True, task_id="peer-%d" % index)
|
||||
assert _trigger(data_root) is False
|
||||
assert len(entered) == 1, "the peer refolded a ledger another process had just folded"
|
||||
assert _folds(data_root) == 1
|
||||
assert _skip_events(data_root) == [] # declined before the pass, not aborted inside it
|
||||
|
||||
# Quiet on real growth: once the file passes what that pass READ plus the
|
||||
# retry window, the peer folds on its own.
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES", 1)
|
||||
_grow_past(data_root, monkeypatch, before.st_size + 1, "peer-growth")
|
||||
assert _trigger(data_root) is True
|
||||
assert len(entered) == 2
|
||||
assert int(_ledger_rows(data_root)[0]["compaction_epoch"]) == 2
|
||||
|
||||
|
||||
def test_a_fresh_process_declines_a_ledger_that_was_just_folded(data_root, monkeypatch):
|
||||
"""`prior is None` is the other half of the cascade: the residue that cannot
|
||||
fold keeps a compacted ledger above the trigger for good, so every newly
|
||||
started process used to buy one full rewrite on its first reservation."""
|
||||
_fixtures._seed_mixed_ledger(data_root)
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_BYTES", 1)
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES", 1_000_000)
|
||||
uc._COMPACT_ATTEMPTS.clear()
|
||||
source_size = (data_root / ua.LEDGER_REL).stat().st_size
|
||||
entered = _count_passes(monkeypatch)
|
||||
assert _trigger(data_root) is True
|
||||
assert (data_root / ua.LEDGER_REL).stat().st_size < source_size # still above the trigger
|
||||
|
||||
uc._COMPACT_ATTEMPTS.clear() # a process that has just started, nothing remembered
|
||||
assert _trigger(data_root) is False
|
||||
assert _trigger(data_root) is False
|
||||
assert len(entered) == 1
|
||||
assert uc._COMPACT_ATTEMPTS == {}, "a declined pass is not a pass this process ran"
|
||||
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES", 1)
|
||||
_grow_past(data_root, monkeypatch, source_size + 1, "fresh-growth")
|
||||
uc._COMPACT_ATTEMPTS.clear()
|
||||
assert _trigger(data_root) is True # real growth, and a fresh process still folds
|
||||
assert len(entered) == 2
|
||||
|
||||
|
||||
def test_an_unstamped_ledger_keeps_exactly_the_per_process_guard(data_root, monkeypatch):
|
||||
"""No pass has ever run here, so line 1 carries no floor. The trigger must
|
||||
behave exactly as it did: the pass runs, and an unprofitable one is still
|
||||
throttled by the memo of the process that ran it."""
|
||||
for task in ("a", "b"): # in-flight only: nothing foldable
|
||||
reservation = ua.reserve_attempt(_fixtures._request(data_root, task_id=task))
|
||||
ua.mark_dispatched(reservation)
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_BYTES", 1)
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES", 10_000_000)
|
||||
uc._COMPACT_ATTEMPTS.clear()
|
||||
entered = _count_passes(monkeypatch)
|
||||
|
||||
assert uc._live_baseline_header(data_root) is None # no stamp, no floor to read
|
||||
assert _trigger(data_root) is False
|
||||
assert len(entered) == 1 # the pass ran and aborted
|
||||
assert [row["reason"] for row in _skip_events(data_root)] == ["nothing foldable"]
|
||||
assert _trigger(data_root) is False
|
||||
assert len(entered) == 1, "the abort is still throttled inside this process"
|
||||
|
||||
|
||||
def test_a_stamp_without_a_usable_size_falls_back_to_the_memo(data_root, monkeypatch):
|
||||
"""A header whose recorded source size is not a positive int answers no
|
||||
floor at all. Falling back to the per-process guard keeps the ledger
|
||||
compactable instead of freezing it behind a number nobody can read."""
|
||||
_fixtures._seed_mixed_ledger(data_root)
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_BYTES", 1)
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES", 1_000_000)
|
||||
uc._COMPACT_ATTEMPTS.clear()
|
||||
assert _trigger(data_root) is True
|
||||
path = data_root / ua.LEDGER_REL
|
||||
lines = path.read_text(encoding="utf-8").splitlines()
|
||||
header = json.loads(lines[0])
|
||||
header.pop("source_size_bytes")
|
||||
lines[0] = uc._dumps_row(header)
|
||||
path.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
|
||||
entered = _count_passes(monkeypatch)
|
||||
uc._COMPACT_ATTEMPTS.clear() # a fresh process meets the unusable stamp
|
||||
assert _trigger(data_root) is False # ran, then aborted on the broken header
|
||||
assert len(entered) == 1
|
||||
assert _trigger(data_root) is False
|
||||
assert len(entered) == 1, "the memo is what throttles a ledger with no readable floor"
|
||||
|
||||
|
||||
def test_an_unreadable_leading_row_surfaces_where_it_always_did(data_root, monkeypatch):
|
||||
"""The floor reads line 1, so it meets a corrupt ledger before anything else
|
||||
does. It may neither raise into the caller's reservation nor quietly decline
|
||||
in place of the pass: the corruption is still reported by the normal read."""
|
||||
_fixtures._seed_mixed_ledger(data_root)
|
||||
path = data_root / ua.LEDGER_REL
|
||||
lines = path.read_text(encoding="utf-8").splitlines()
|
||||
lines[0] = "{not json at all"
|
||||
path.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_BYTES", 1)
|
||||
uc._COMPACT_ATTEMPTS.clear()
|
||||
entered = _count_passes(monkeypatch)
|
||||
|
||||
try:
|
||||
uc._live_baseline_header(data_root)
|
||||
raise AssertionError("an unreadable leading row must be typed corruption")
|
||||
except uc.UsageLedgerCorrupt:
|
||||
pass
|
||||
assert _trigger(data_root) is False # contained, never raised into the reservation
|
||||
assert len(entered) == 1, "the guard swallowed the corrupt ledger instead of the pass"
|
||||
try:
|
||||
ua.reserve_attempt(_fixtures._request(data_root, task_id="after-corrupt"))
|
||||
raise AssertionError("the corrupt ledger must still fail the normal read")
|
||||
except uc.UsageLedgerCorrupt:
|
||||
pass
|
||||
|
||||
|
||||
def test_a_restored_older_ledger_uses_the_floor_it_carries(data_root, monkeypatch):
|
||||
"""A ledger restored from a backup brings its own, older stamp. The floor is
|
||||
whatever THAT header records: the guard reads the file in front of it, never
|
||||
a size belonging to a generation that is no longer there."""
|
||||
_fixtures._seed_mixed_ledger(data_root)
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_BYTES", 1)
|
||||
monkeypatch.setattr("ouroboros.config.USAGE_LEDGER_COMPACT_RETRY_GROWTH_BYTES", 1)
|
||||
uc._COMPACT_ATTEMPTS.clear()
|
||||
path = data_root / ua.LEDGER_REL
|
||||
first_source = path.stat().st_size
|
||||
assert _trigger(data_root) is True
|
||||
restored_bytes = path.read_bytes() # the backup: epoch 1, its own stamp
|
||||
|
||||
_grow_past(data_root, monkeypatch, first_source * 3, "epoch2")
|
||||
uc._COMPACT_ATTEMPTS.clear()
|
||||
second_source = path.stat().st_size
|
||||
assert _trigger(data_root) is True
|
||||
assert int(_ledger_rows(data_root)[0]["compaction_epoch"]) == 2
|
||||
|
||||
path.write_bytes(restored_bytes) # the operator restores the older generation
|
||||
uc._COMPACT_ATTEMPTS.clear()
|
||||
entered = _count_passes(monkeypatch)
|
||||
assert _ledger_rows(data_root)[0]["source_size_bytes"] == first_source
|
||||
assert _trigger(data_root) is False # below the floor the restored file carries
|
||||
assert len(entered) == 0
|
||||
|
||||
_grow_past(data_root, monkeypatch, first_source + 1, "restored-growth")
|
||||
assert path.stat().st_size < second_source, "the fixture overshot the discriminator"
|
||||
assert _trigger(data_root) is True # its own header released it, not the one it lost
|
||||
assert len(entered) == 1
|
||||
assert int(_ledger_rows(data_root)[0]["compaction_epoch"]) == 2
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue