path: root/tests/CLAUDE-PROGRESS-ARCHIVE.md

# Progress summary (for a new Claude session)

Written by Claude Code at the user's request, 2026-10-14.  Read this first,
then `CLAUDE.md` (project rules) and `forest-examples.md` (reference cases).
Longer notes from earlier in the work: `tests/CLAUDE SUGGESTIONS.org`.

## START HERE (state at the end of session 2, 2026-10-14)

**Reading order for a new session:** `CLAUDE.md` -> this section -> 
`tests/DESIGN-leading-chains.md` (design + status, section 7).  The rest of this
file is chronological notes of session 1 and 2; parts are SUPERSEDED (see
"Superseded claims" below).  `forest-examples.md` holds the reference cases;
the memory notes (auto-loaded) hold the working style.

### Where the work stands (one paragraph)
Recognition works.  Forest construction is fixed for: root-level left recursion
(`S -> S a | a` up to 4 tokens, several variants), non-root left recursion
(`T -> x S`, `T -> S x`), an ambiguous left-recursive root
(`S -> x S | S a | a` on `x a a`), a simple indirect case (`S -> A b; A -> S c | d`
on `d b`).  All of this is a PROTOTYPE in a scratch copy; NOTHING is applied to
the user's tree (the user writes all source by hand).  The task now is: finish
the failing cases below, then hand the user a consolidated, ordered change list
(file, function, what, why; NO diff unless asked) plus regression tests.

### Scratch copy (the ONLY place the prototype exists; no diff was saved)
- MOVED INTO THE REPO at the user's request (end of session 2):
  `<repo>/claude scratch/ws` (the prototype, a full copy of the workspace with
  its own `[workspace]`, so it is a separate cargo workspace root) and
  `<repo>/claude scratch/target` (its own, independent build directory, ~420 MB).
  `<repo>` = `/Users/durand/Desktop/Centre/A propos de programmes/Rust/rep`.
  The paths contain spaces: always quote them.
- Run (verified from the new location; identical probe results):
  `cd "<repo>/claude scratch/ws" && CARGO_TARGET_DIR="<repo>/claude scratch/target" cargo test --test oracle -- --test-threads=1`
  This is the ONE exception to "write only inside `tests/`" and was explicitly
  requested by the user; the user's real sources are still untouched.  Do NOT
  copy it anywhere (move/edit in place); do not touch the repo's own `target/`.
- Notes: `claude scratch/` is untracked and NOT covered by `.gitignore`
  (`/target` only matches the root one); rust-analyzer may index the nested
  workspace.  Whether to ignore it is the user's decision (not done).
- Safe to delete (ask the user first), still in the old per-session scratchpads
  under `/private/tmp/claude-501/-Users-durand-Desktop-Centre-A-propos-de-programmes-Rust-rep/`:
  `2f4b067a-.../scratchpad/ws` (old, ~29 MB) and `8010540b-.../scratchpad/`
  `{base,target_base,*.txt}` (`base` = pristine copy of the user's tree, used to
  show that `test_from_grammar` fails without my changes; `target_base` 121 MB).
- Risk reduced: the prototype no longer lives under `/private/tmp`.  Still no
  diff of it exists in `tests/`; ask before writing one.
- macOS: `sed -i ''`, BSD sed has no `\b` (use python for edits).  zsh: avoid
  `for g in $'...'` loops with newlines in the grammar (spurious `.Trash` errors).

### The prototype = these edits to the user's tree (in dependency order)
(S) = scratch only; status: NEEDED / UNVERIFIED / DEBUG.
1. NEEDED `chain/src/item/reduction.rs` leaf branch (childless Non node closed by
   `close_pavi(Virtual)`): after `correct_ends.insert(top, correctness.into())`
   add `if correctness && accept_root { order_of_correct_ends.push(top); }`
   (FIX-A; the `accept_root` gate is required).
2. NEEDED `graph/src/lib.rs` new module `det` (aliases `HashMap`/`HashSet` over
   `BuildHasherDefault<DefaultHasher>`), used in graph (`adset`,
   `labelled/double`, `labelled/binary`), grammar (`lib`, `first_set`, `abnf`),
   chain (`atom/default`, `reducer`, `default`, `item/reduction`); `new()` ->
   `default()`, `with_capacity(n)` -> `with_capacity_and_hasher(n, Default::default())`.
   Reason: results of failing cases FLIPPED between runs with random hash order
   (measured), so nothing could be diagnosed.  Same SipHash speed (inferred).
3. NEEDED `chain/src/default.rs` `derive` (~674): remove the dummy wrapper node 1
   as soon as it exists: `nodes_len() >= 2` instead of `> 2`.
4. NEEDED `chain/src/item/default/mod.rs` `plant_at_start`: `clone_threshold`
   always 0 (no dummy child of the root).  (S) also `remove_prefix_cut_cycles`
   (DFS back-edge cut) replaced `remove_prefix(1)`; with the leading chains
   (edit 7) there are no cycles, so it is probably a no-op: UNVERIFIED whether
   it can be dropped; try `remove_prefix(1)` again.
5. NEEDED `chain/src/item/default/splone.rs`:
   a. `open_ancestors`: add `new_top` as a child of the new open ancestor AFTER
      the preserved children, in BOTH branches (clone branch never added it;
      plain branch added it first, wrong order when `parent_edge > 0`).
   b. `create_new_label`: a parentless existing node is legitimate (not stale)
      when `node` is top-level (no parents, or its only parent is a packed node
      that is the root or parentless); apply to both the packed and plain
      lookups.  Fixes two packed clones of the root closing to the same label
      (children of both merged into one node).  Reproduced without left
      recursion: `S -> a A | A b; A -> a | b` on `a b`.
   c. New `splone_left(node, pos, edge, wrapper) -> (closed, open)`: if `node`
      is open, force the split path (closed copy `C`, `split_parents`
      duplicates parents) and keep `node` as the open continuation `O`; if
      `node` is already closed at `pos`, `C = node` and `O` is an open copy via
      `create_new_label(node, None, ..)` + `split_node`.  Then remove all
      children of `O` and give it the single child `Rule(wrapper)[p,pos] -> C`
      (packed node above `C` if `C` is a clone).
6. NEEDED `chain/src/item/genins.rs` `insert_item`:
   a. climb hook before the `splone` of a reduction level: `r` = fragment root
      rule, `h = atom.get_rule_num(r)`, `w = atom.left_wrapper(h, reduction_nt, r)`;
      if `Some(w)`, the node has no wrapper-parent labelled `Non(h)` with the
      SAME START, is plain and (open or closed at `pos`): call `splone_left`,
      push `Parent(O, 0)` on `second_stack`, `continue`.
   b. virtual branch: loop over ALL fragments from `generate_virtual_frags`
      (was `frag.first().unwrap()`), deduplicating pushed parents.
   c. UNVERIFIED (no measured effect): `is_last_level` restriction of the hook,
      and a final filter keeping only stack nodes labelled `get_rule_num(r)`.
      Theory says the filter is correct; drop the level restriction.
7. NEEDED `chain/src/atom/default.rs` (design in `DESIGN-leading-chains.md`):
   `first_literals`, `lead`, `chains`, `left_wrappers` computed from the NFA
   before `nfa.closure(..)`; `virtual_frags: Map<VirtualNode, Map<slot, Vec<frag>>>`
   built from `chains`; `generate_virtual_frags` returns all alternatives;
   accessors `left_wrapper(h, y, r)` and `chains_of(nt, t)` (tests).
   `expansion_map` is no longer read by the parser.
8. DEBUG (S): `eprintln`/`dbg_dump` under env `DBG`, `DBGLEAD`, `DBGEXP`;
   `generate_virtual_frags` made `pub` (the user's tree has it `pub(crate)`);
   scratch-only tests in `tests/oracle.rs`: `diag_left` (env `DIAG_GRAMMAR`,
   `DIAG_INPUT`, `DIAG_END`), `probe_cases`, `probe_one`.  `probe_cases` and
   `probe_one` were copied into the user's `tests/oracle.rs` as `#[ignore]`.

### Results now (deterministic; heights 1..5 against the oracle)
PASS: layer 1 (incl. right recursion n=1), `layer4_left_recursion` (n=1..4),
`S -> S a | b`, `S -> S a b | c`, `S -> S a | a | b`, `T -> x S; S -> S a | a`
(`x a`, `x a a`, `x a a a`), `T -> S x; S -> S a | a` (`a a x`),
`S -> x S | S a | a` (`x a a`), `S -> S S | a` (`a a`),
`S -> A b; A -> S c | d` (`d b`), `S -> a A | A b; A -> a | b` (`a b`).
FAIL (not yet analysed unless noted):
- `S -> x S | S a | a` on `x x a a a`: panic "attempt to subtract with overflow".
- `S -> A b | d; A -> S c` on `d c b`, `d c b c b`: CannotPlant at token 1.
- `S -> S S | a` on `a a a` (layer 2, blocks Case 1 and 2): EXTRA
  `(S (S a) (S a) (S a))` (an `S` with three `S` children) and MISSING
  `(S (S (S a) (S a)) (S a))`.  Smallest layer-2 failure: START HERE.
- layer 3 (epsilon): epsilon nodes missing.  layer 5 (cycles): cyclic
  derivations missing (chains drop back edges, nothing creates `S -> R -> S`);
  case 5d CannotPlant at token 1.
- chain crate unit test `atom::default::test_default_atom::test_from_grammar`
  fails 8/8 on a pristine copy of the user's tree too (hard-coded virtual node
  numbers 166..=173 and an absolute path): NOT caused by the prototype.

### Next steps (suggested order)
1. Trace `S -> S S | a` on `a a a` with `DBG=1 DIAG_END=1 DIAG_GRAMMAR=... DIAG_INPUT=...
   cargo test --test oracle diag_left -- --nocapture --test-threads=1`.
2. The overflow panic on `x x a a a`; then `d c b` (indirect: the chain for
   `[A]^d` exists, check the climb with `h != nt`).
3. Design layer 5 (who creates the cycle `S -> R -> S` for unit/cyclic
   productions) and layer 3 (epsilon nodes; nullable leading symbols in chains).
4. `reduction_map` is built by the same iterative closure with the same
   "longer wins" guard as `expansion_map` was: possible analogous defect, not
   measured.
5. When the probe set and `layer2_*` pass: write the consolidated change list
   for the user; every fix gets a regression test that fails before it.

### Superseded claims (older sections of this file)
- "OPEN DESIGN QUESTION" / "Suggested next steps" / "Scratch state at the time of
  writing": superseded by this section.
- "Nondeterminism ... HARMLESS": wrong for the failing cases; they flipped
  between runs.  The user allowed restoring order (deterministic hasher, edit 2).
- "junk expansion is the cause" (finding 1): partly right.  Measured later:
  `(0,13)` is first written correctly, then overwritten by a longer entry from
  another composition; now bypassed by the leading chains.
- The "last level only" restriction of the continuation fork was a guess with
  no measured effect (see edit 6c).
- User preferences confirmed this session: deterministic hashing is fine; a
  redesign of how information is recorded is fine (the user confirms first);
  the paper is NOT to be consulted (their forest construction differs from
  Herman's).

## How the user wants to work (keep following this)

- The user writes all source by hand.  Claude plans, explains, and writes files
  ONLY inside `tests/`.  Do NOT write diffs unless the user asks for one.
- Run experiments ONLY on a scratch copy of the repo, never in the user's tree
  (the user said so; cargo in the real tree would also write to its `target/`).
  Keep scratch space small and delete it when debugging is done.
  Copy: `chain grammar graph graph_macro forest nfa receme semiring tokens viz
  src tests examples Cargo.toml Cargo.lock`, and build with
  `CARGO_TARGET_DIR=<scratch>/target cargo test ...`.
- Say what is MEASURED, what is INFERRED, what is a guess.  Do not claim
  more.  Ask before opening any paper PDF (see `CLAUDE.md`).
- macOS: use `sed -i ''`.  The parser prints a lot; the tests capture it.
- The parser's debug dumps write to a relative `output/` directory; the
  test harness `chdir`s into `$TMPDIR/rep-tests` (with `output/`) first.

## Project in one paragraph

Rust implementation of Herman's relational parsing (chain-rule machine over an
NFA of "atomic languages").  Recognition works; the FOREST (shared packed
parse forest) is wrong.  Goal: the forest represents exactly the derivation
trees of the input, shared properly.  Construction is in `chain/`
(`chain/src/default.rs` `derive` / `end_of_input`; `chain/src/item/genins.rs`
`insert_item`; `item/default/{mod,splone,extract}.rs`; `item/reduction.rs`).

## The test tooling (DONE): `tests/oracle.rs`

- Brute-force oracle `oracle_trees(grammar, tokens, h)`, validated against all
  of `forest-examples.md` (10 checks pass).  Case 1 in that file says "5
  packed alternatives"; the real count is 7 (user confirmed it is a typo).
- `parser_side` module: builds the parser from the same grammar text, runs it
  (per-case timeout, default 20 s, `REP_TIMEOUT`; panics and errors are caught),
  `unpack(forest, names, h, limit)`, `classify` (missing / extra / duplicate),
  `validate_forest` (span coverage, unique nodes, terminals, productions,
  cycles within one span, size bound), and the test layers of `CLAUDE.md`
  (layers 1-5, plus layer 6 random grammars, `#[ignore]`, `REP_SEED`,
  `REP_CASES`).  Run: `cargo test --test oracle -- --test-threads=1`.
- Grammar syntax in tests: `"S -> S S | a"`, several lines for several
  nonterminals, `eps` for an empty alternative.  Names are mapped to letters
  because `scan_tnt` only accepts letters in `T<name>` / `N<name>`.
- `tests/spike.rs`: early probes (partly obsolete).  `tests/exp.diff`,
  `tests/fix-experiments.diff`: old probe/experiment diffs (obsolete; the
  experiments below supersede them).
- Empty input is NOT supported by the API: `end_of_input(pos, ter)` needs the
  last token, and `ter` selects virtual fragments (atom/default.rs:489-549,
  genins.rs:730), so an arbitrary terminal is not safe.  Deferred to the
  epsilon layer.  The "END_OF_INPUT token" is NOT a convention of this project.

## Forest convention (verified on forests of passing grammars)

- A nonterminal node's children are "position wrappers" `Rule(pos)`, one per
  right-hand-side symbol, in order; each wrapper has exactly one child, the
  symbol.  Wrapper span = its child's span.  Example for `S -> a a` on `aa`:
  `S[0,2] -> [Rule(2)[0,1] -> a, Rule(3)[1,2] -> a]`.
- Ambiguity: a `Packed` node (labelled like the nonterminal) whose children are
  `Cloned(i)` nodes, one per alternative.
- Nodes are unique by label = (symbol, start, end) (`forest/src/design.org`,
  axiom 1).  `plant` identifies fragment nodes with forest nodes BY LABEL.
- `extract(pos)` takes the node labelled `Non(0)[0,pos]` (closed copy of the
  root) as the answer root; any node without an end infects its relatives.
  So a closed `S[0,pos]` MUST exist at the end.
- Unverified assumption in `unpack`: an epsilon production is a childless
  nonterminal node.  Observed instead: epsilon nodes are MISSING (layer 3).
- The dummy root: `unit()` makes root `S[0,None]`; the first step plants
  `[Rule(6), Non(0)]` under it, which merges back onto the root: a cycle
  `S -> Rule(6) -> S`.  Per the user it only exists so the code can work on an
  almost empty forest and SHOULD be removed early (`derive`, default.rs:673-683,
  and `end_of_input`).  Do not keep it alive (experiment E5 did; it was wrong).

## Results on the ORIGINAL code (baseline, as in the user's tree)

| layer / case | result |
|---|---|
| 1 fixed length, nested nonterminals | pass |
| 1 right recursion `S -> a S \| a` | n >= 2 pass; **n = 1: empty forest** |
| 2 Case 1 `S -> S S \| a`, Case 2 | n = 1 empty forest; n >= 2 `CannotPlant` at token 1 |
| 3 epsilon (Case 3, 4, nullable middle) | epsilon nodes missing / empty forest |
| 4 left recursion, hidden left recursion | `CannotPlant` at token 1 (n=1 empty) |
| 5 cyclic: 5a, 5c, highly ambiguous with eps | TIMEOUT: the parser never terminates |
| 5 Case 5d | `CannotPlant` at token 1 |

## Findings

### Verified fix FIX-A (not applied to the user's tree)
`item/reduction.rs` ~line 357, in the leaf branch (childless Non node closed via
`close_pavi(Virtual)`): after `correct_ends.insert(top, correctness.into());`
add `if correctness && accept_root { order_of_correct_ends.push(top); }`.
Without it no closed `S[0,pos]` is made and `extract` returns nothing.  The
`&& accept_root` gate is REQUIRED: without it two layer-1 tests regress
(`reduction` is also called mid-parse from `insert_item`).  Regression test:
`layer1_right_recursion` (now starts at n = 1) fails before, passes after.
It does not fix left-recursive one-token inputs.

### Root cause of `CannotPlant` on left recursion (MEASURED)
1. `grammar/src/lib.rs:722-781` (`transform_label_null_epsilon`): the
   left-linear `expansion_map` is keyed by (source, target), so the direct edge
   and the left-recursion composite edge share one entry, and every closure
   pass re-extends it.  The recorded expansion is junk for left recursion
   (length 4-10 depending on order and iteration count).  Right recursion gives
   a correct length-1 expansion.
2. `generate_fragment` (genins.rs:79-88) merges nodes by label, so the repeated
   pairs collapse into a CYCLE: `S -> R2 -> S` plus `S -> R5 -> a`.
3. `plant_at_start` (item/default/mod.rs:963) calls `fragment.remove_prefix(1)`
   (deletes node 0 and all its edges; assumes a chain): the cycle and the
   second root child are dropped, leaving `R4` (childless, planted) and a
   floating `R2 -> a`.  `insert_item` then finds that floating rule node by
   `query_label` (no parents), the lift in step 4 gives `[]`, empty stack,
   `CannotPlant`.
4. The `trace`/`virtual_frags` tables DO agree (trace value == fragment key);
   my earlier "key mismatch" claim was wrong.

### Nondeterminism
Hash iteration order made atom numbering vary between runs (graph/labelled/
double.rs, grammar/lib.rs, first_set.rs, atom/default.rs).  Measured
HARMLESS for the results above (identical over repeated runs with the original
hashing); the user deliberately keeps hash order for speed.  If a result is
flaky, repeat the run before suspecting anything else.

### Left-recursion fragment experiments (scratch only)
| shape | root `S -> S a \| a` | non-root `T -> x S` |
|---|---|---|
| E3 base only `S -> R5 -> a` (cut expansion at first pair with nt = the nonterminal) | 1 token ok; continuation lost: 2nd `a` attaches to T's rule | same |
| E4 one wrapper level + explicit closed flags (`generate_fragment_with_closing`) | correct up to 2 tokens | spurious unit layer `(T x (S (S a)))` |
| E6 keep the cycle, graft root children, `set_pos` does not close past root | `splone` assertions ("a rule node should not have more than one child"); closed copy keeps the cycle wrapper | same |
Beyond 2 tokens with the dummy removed early: closing `S[0,2]` leaves no
parent to climb to (the wrapper `Rule(2)[0,2] -> S[0,2]` is never created).

## DESIGN DECISION (session 2, 2026-10-14): the climb creates the continuation

Decided with the user: when a left-recursive `S` closes (e.g. `S[0,1]`), the
CLIMB in `insert_item` (through `splone`-like closing) creates the wrapper
`R4[0,1] -> S[0,1]` and the new open `S[0,None]` above it.  The fragment does
NOT carry the continuation.  The user thinks their forest construction differs
from Herman's, so the paper is not to be consulted.

Evidence (MEASURED on `S -> S a | a`, input `a a`, scratch copy with a
diagnostic test `diag_left`, not in the repo):
- NFA (state numbers): `0 -a(moved 2)-> 5`, `0 -S(moved 4, left_p)-> 9`,
  `5 -a(moved 5)-> 11` (reduction [0]), `11 -a(moved 5)-> 11` (reduction [0],
  expansion only [(0,5)]), virtual node `[S]^a` = 14 with
  `14 -a(moved 5)-> 11` (reduction [0], accepting).
- `expansion(0,5) = [(0,2),(0,4),(0,4),(0,4),(0,4)]` (junk repeat of (0,4)).
  Harmless for the stored virtual fragment: merging by label collapses it.
- Stored virtual fragment of `[S]^a` (rule 2) is
  `S[0,None] -> {R4[0,None] -> S (back edge: CYCLE), R2[0,1] -> a[0,1]}`.
  `trace(0,0,11) = [2]`; `trace(0,0,5)` and `trace(0,0,9)` are empty.
- After token 0 the forest holds only the dummy root (`S -> Rule(6) -> S`):
  the virtual fragment is materialized LAZILY at token 1 (`PaVi::Virtual`
  branch of `insert_item`, `plant_at_start` + `remove_prefix(1)`), which drops
  the cycle: result `R4[0,1]` childless + floating `R2[0,1] -> a`, then
  `CannotPlant` (parents = []).

Why the climb (INFERRED from the above):
1. Axiom 1: outer and inner `S` both start at 0 and are open, so they get the
   same label; a fragment can only express the pair as a cycle.
2. The loop `11 -a-> 11` repeats the continuation at EVERY further token, but
   its expansion is just [(0,5)]: nothing in a fragment supplies `R4` for the
   2nd, 3rd... iteration.  Only the climb runs every time.
3. Commit `a80db17` says reductions are meant to close nodes at every step.
4. The wrapper rule number is available: it is the `moved` of the left-linear
   `Non` edge (here 4), and the second component of the `(nt, rule)` pairs of
   the expansion.

Sketch for the user's implementation (not written, to be checked):
(a) plant only the acyclic base `S[p,None] -> R2[p,q] -> a[p,q]` (cut the back
edge instead of `remove_prefix`); (b) on reduction [S] splone/close `S[p,q]`;
(c) if `S` is left-recursively continued, add `R4[p,q] -> S[p,q]` as the first
child of a fresh open `S[p,None]`; (d) the next terminal's wrapper `R5` goes
after `R4`.  OPEN: indirect left recursion (`S -> A b`, `A -> S c`), and
non-root nonterminals (`T -> x S`, E4 gave a spurious unit layer there).

### PROTOTYPE RESULT (session 2, scratch copy only; MEASURED)

Scratch copy: `<scratchpad>/ws` (original code + the edits below + debug
`eprintln` under env `DBG` + a `diag_left`/`probe_cases` test in its
`tests/oracle.rs`).  Edits, described in words (no diff written):
1. FIX-A (gated on `accept_root`), as before.
2. `derive` (default.rs ~line 674): remove the dummy wrapper node 1 as soon as
   it exists (`nodes_len() >= 2` instead of `> 2`).
3. `plant_at_start` (item/default/mod.rs ~963): new `remove_prefix_cut_cycles`
   instead of `remove_prefix(1)`: also drops the wrapper nodes that are parents
   of the fragment root (the cycle back edges) and the nodes they leave
   isolated; and `clone_threshold` is always 0 (no dummy child of the root).
4. New `left_continue(closed, wrapper_rule)` (item/default/mod.rs, next to
   `remove_node`): adds `wrapper[p,q] -> closed` and an open `cont[p,None] ->
   wrapper` (reusing existing nodes by label), and makes `cont` the forest root
   if `closed` was the root.
5. Climb hook (genins.rs, just before `let parents_iter = self.parents_of(
   node.node())` after the `splone` in step 1).  FIRST VERSION (superseded):
   "closed node has no parents" + wrapper = fragment slot - 1 (arithmetic
   hack, wrong in general: slot numbers are regex NODE ids, not a sequence).
   CURRENT VERSION = condition (1), using only existing accessors:
   `r` = fragment root's `Rule(r)`; `h = atom.get_rule_num(r)` (head of the
   slot; `accumulators` are in regex-node units, same unit as `moved`);
   `w = left_wrapper(atom, h, nt, r)`: look in `atom.labels_of(2 *
   atom.nth_accumulator(h))` for an edge with `is_left_p()` and value
   `Non(nt)` (nt = the reduced nonterminal), whose target state has an
   outgoing edge with `get_moved() == r`; `w` is that left-linear edge's
   `get_moved()` (e.g. 4 for `S -> S a`).  Then, unless the closed node
   already has a rule-wrapper parent whose parent is labelled `Non(h)`, call
   `left_continue(closed, w, h)` (continuation labelled `Non(h)`, so it also
   covers indirect left recursion, modulo the longer cycle, see below).
   `regexp` in `DefaultAtom` is private and has no accessor; none is needed.

Results (heights 1..5 vs the oracle, `validate_forest` included):
- PASS: `S -> S a | a` for n = 1..4 (stable over 8 repeated runs, so hash
  order is not an issue here); `S -> S a | b`; `S -> S a b | c`;
  `S -> S a | a | b`; `S -> S S | a` on `a a`; `T -> x S; S -> S a | a` on `x a`.
  Layer 1 still passes (fixed length, nested, right recursion).
  With condition (1) the root-level results are identical (same passes).
- FAIL, non-root left recursion: `T -> x S; S -> S a | a`: `x a a` gives EXTRA
  `(T x (S a) a)` and MISSING `(T x (S (S a) a))`; `x a a a` MISSING one tree;
  `T -> S x` with `a a x`: CannotPlant at token 1.  MEASURED: condition (1)
  DOES fire here (`r=7 h=S nt=S w=6`) and builds `S'[1,None] -> R6 -> S[1,2]`,
  so the condition is not the problem.  The problem (measured on the forest
  dump): the closed `S[1,2]` has two wrapper parents, T's `R3[1,2]` and the new
  `R6`; the existing lift goes through BOTH, so the fragment is planted under
  T as well as under `S'` (spurious), and `S'` has no open wrapper under an
  open copy of T.  Needed (design, not yet tried): in the continuation case
  lift ONLY through the new wrapper, and give `S'` an open wrapper
  `R3'[1,None]` under an open alternative of T (split: closed T alternative
  with `R3[1,2]`, open one with `R3'`); `split_node`/`split_parents` already
  do the closed/open duplication for ordinary closings.  Alternative worth
  trying: force the SPLIT path of `splone` (closed copy C, parents duplicated
  by `split_parents`) and rewrite the surviving open node O to have the single
  child `R_w[p,q] -> C`, so root and non-root cases are handled uniformly.
- FAIL, ambiguity: `S -> S S | a` on `a a a` (2 trees missing).
- FAIL, indirect left recursion (`S -> A b; A -> S c | d`, and
  `S -> A b | d; A -> S c`): "assumption fails" panic / CannotPlant.  Inferred:
  the virtual fragment has a cycle of length 2 (`S -> R -> A -> R' -> S`);
  `remove_prefix_cut_cycles` only drops the wrapper `R'` that points to the
  root, leaving `A` and its wrapper planted childless.  A general cut must drop
  the whole cycle path through the root.
- Layers 2-5 other cases still fail as in the baseline table (not re-examined).

### OPTION B RESULT (session 2, last scratch run; MEASURED)

Implemented in the scratch copy (now unavailable, see below): in the climb, BEFORE
the `splone` of a reduction, if condition (1) holds for (fragment root slot
`r`, head `h = get_rule_num(r)`, reduced `nt`) and the open node has no wrapper
parent labelled `Non(h)` with the SAME START (the start check is new) and is a
plain open node, then call a new method `splone_left(node, pos, edge, w)`
(splone.rs): `create_new_label(node, Some(pos), edge, None)` + `split_node` to
get the closed copy `C` (parents duplicated by `split_parents`), then remove
all children of the surviving open node `O` and give it the single child
`Rule(w)[p,pos] -> C` (the packed node above `C` if `C` is a clone).  The lift
then pushes only `Parent(O, 0)` (skips the generic lift through `C`'s
parents).  The forest root is NOT reassigned any more (`O` is the root).

- Now PASS: `T -> x S; S -> S a | a` on `x a`, `x a a`, `x a a a` (fixed by
  option B; before: extra/missing trees); `S -> S a | a` n = 1..4 (no regression);
  all other earlier passes unchanged.
- Still FAIL:
  - `T -> S x; S -> S a | a` on `a a x`: CannotPlant at token 1 (not analysed).
  - `S -> x S | S a | a` on `x a a`: MISSING `(S (S x (S a)) a)`, EXTRA
    `(S x (S (S (S a)) a))` at h = 4; on `x x a a a`: panic "attempt to subtract
    with overflow" at genins.rs:646 (not analysed; line numbers refer to the
    scratch file).
  - `S -> S S | a` on `a a a`: now 1 missing + 1 extra tree (was 2 missing).
  - Indirect left recursion: unchanged (assumption-fails panic / CannotPlant).
- Layers 2-5 of the oracle test file: not re-run after option B.

### SESSION 2, LATER (MEASURED unless marked)

**Scratch location.**  The scratchpad path of the session changed
(`2f4b067a-...` -> `8010540b-...`); the old copy was NOT deleted.  It was copied to
`.../8010540b-.../scratchpad/ws` (build dir `.../scratchpad/target`).  The old
directory `.../2f4b067a-.../scratchpad/{ws,target}` (~250 MB) is untouched and
can be deleted by the user.  Still no diff saved: the edits are described here.

**Determinism (new scratch edit, E0).**  Results of the failing probe cases
FLIPPED between runs (CannotPlant / "assumption fails" / missing / extra), so
nothing could be diagnosed.  Measured: 6 runs, passing cases identical, 4 of the
failing cases varied.  Fix tried in scratch: a `graph::det` module with
`HashMap`/`HashSet` aliases over `BuildHasherDefault<DefaultHasher>` (fixed
SipHash keys), used in graph (adset, labelled/double, labelled/binary),
grammar (lib, first_set, abnf), chain (atom/default, reducer, default,
item/reduction); constructors `new()` -> `default()`,
`with_capacity(n)` -> `with_capacity_and_hasher(n, Default::default())`.
After that, 6 runs of the probe set gave byte-identical results.  Speed: same
SipHash as the default hasher, only a constant key, so lookups stay O(1)
(INFERRED, not benchmarked).  A BTree container would be O(log n) and needs
`Ord` keys.

**Reduction levels (E7, option B refinement).**  The continuation fork is only
done at the LAST level of `reduction_info` (the level at which the fragment is
planted); earlier levels use the normal `splone`.  (Hypothesis from reading;
it changed no probe result by itself, but it is the logical rule.)

**Two calls per token (measured on `S -> x S | S a | a`, `x a a`).**  At the
token after `x a`, `derive` makes TWO `insert_item` calls (edges 17 and 20)
sharing `true_source = Virtual(0,1,5)`.  Call A closes the virtual node 5
(`S[1,..]`) IN PLACE through `close_pavi`, so in call B the node is already
`S[1,2]`.  Hence `splone_left` now has a second case: node already closed at
`pos` -> make the open continuation `O` by the standard re-open path
(`create_new_label(node, None, ..)` + `split_node`, which calls
`open_ancestors`), then rewrite `O` to `R_w[p,pos] -> C`.  The hook's condition
accepts `end == None` or `end == Some(pos)`.  `splone_left` returns `(C, O)` and
the climb pushes `Parent(O, 0)`.

**Bug found in `open_ancestors` (splone.rs ~ line 590; measured on the forest
dump).**  (1) In the clone branch the new open ancestor never receives the
reopened child `new_top` (orphan wrapper, e.g. `R8[1,None]` without parent);
(2) in the plain branch `new_top` is added BEFORE the preserved children, so
the child order is wrong whenever `parent_edge > 0`.  Scratch fix: add
`new_top` after the preserved children in both branches (remove the edge from
the plain `else`).  With it the forest of `x a a` after token 2 is exactly
right: root packed with clone0 `[R4[0,2]->S[0,2], R5->a]` and clone1
`[R7->x, R8'[1,None]->S[1,None]->[R4[1,2]->S[1,2], R5->a]]`.

**Results now (deterministic):**
- PASS (unchanged): all earlier passes, incl. `T -> x S; S -> S a | a` on
  `x a`, `x a a`, `x a a a`; `S -> S a | a` n = 1..4.  Layer 1 and
  `layer4_left_recursion` pass; no regression in the full run.
- FAIL, now a DIFFERENT bug class (ambiguity at closing, NOT left recursion):
  `S -> x S | S a | a` on `x a a` fails only when `end_of_input` closes the two
  packed clones of the root: both close to the label `S[0,3]` and the node
  ends up with the children of BOTH (`S -> [S, a, x, S]`, not a production).
  Reproduced WITHOUT left recursion: `S -> a A | A b; A -> a | b` on `a b`
  gives EXTRA `(S (A a) b a (A b))` and MISSING `(S (A a) b)`, `(S a (A b))`;
  also `S -> a b | A b; A -> a | a b`.  Suspect: `reduction()` +
  `splone` on cloned nodes / `create_new_label` returning `Cloned` and then
  `split_node` (INFERRED, not yet traced).  This is the baseline "layer 2"
  failure.
- FAIL, unchanged class: `T -> S x` (`a a x`) CannotPlant at token 1;
  `S -> x S | S a | a` on `x x a a a` overflow panic; `S -> S S | a` on
  `a a a` (EXTRA); indirect left recursion (panic "this should be a rule
  parent" / CannotPlant).

### SESSION 2, LEADING CHAINS PROTOTYPE (design: `tests/DESIGN-leading-chains.md`)

User confirmed: multiple virtual fragments per (virtual node, final slot) are
fine and must each be inspected by the consumers; the leading edges are read
from the NFA (in `DefaultAtom::from_grammar`, where it exists).

Implemented in the scratch copy (prose, no diff):
- `first_literals(nfa, starts, state)`: literal edges reachable from `state` by
  epsilon edges WITHOUT entering another nonterminal's start state.  Note: in
  the pre-closure NFA the first epsilon edge out of a start state has NO label
  entry (`children_of(0) = [2]` but `labels_of(0)` is empty); unlabelled edges
  must be treated as epsilon.  The edge `4 -eps-> 8` into `S`'s start state is
  the left-linear link and must not be followed.
- Before the closure: `lead[N]` = `first_literals(start(N))`;
  `chains[(N,t)]` = simple paths in the leading graph (no repeated nonterminal;
  repeats only stop the search); `left_wrappers[(h, Y, r)] = w`.
- `virtual_frags` is now `Map<VirtualNode, Map<slot, Vec<frag>>>`, filled from
  `chains` (a chain is used for the closure edge whose `moved` equals its last
  slot; duplicates skipped).  `generate_virtual_frags` returns ALL alternatives.
- `insert_item` virtual branch loops over all fragments (parents deduplicated);
  `left_wrapper` is now `atom.left_wrapper(h, nt, r)`, a table lookup.
- `expansion_map` is no longer read by the parser (still written).

MEASURED chains (slots vary with the encoding): `S -> A b; A -> a`:
`chains(S,a) = [[(S,2),(A,6)]]`; `S -> S a | a`: `[[(S,2)]]`, left wrapper
`(S,S,5) -> 4`; `T -> S x; S -> S a | a`: `chains(T,a) = [[(T,2),(S,6)]]`
(the previously missing fragment); indirect `S -> A b | d; A -> S c`:
`chains(A,d) = [[(A,8),(S,2)]]`.

Probe results now (deterministic, heights 1..5): PASS: all earlier passes plus
`T -> S x; S -> S a | a` on `a a x`, `S -> x S | S a | a` on `x a a` (after the
`create_new_label` top-level fix), `S -> A b; A -> S c | d` on `d b`.
FAIL: `S -> x S | S a | a` on `x x a a a` (panic "attempt to subtract with
overflow", not analysed); `S -> S S | a` on `a a a`; `S -> A b | d; A -> S c`
on `d c b` and `d c b c b` (CannotPlant at token 1).  Layer 1 and
`layer4_left_recursion` still pass.  The full oracle run now takes 0.16 s (the
cyclic layer-5 cases used to time out; they now fail fast).

Layer 2-5 symptoms now:
- layer 2 (`S -> S S | a`, `E -> E + E | n`), 3 inputs: EXTRA
  `(S (S a) (S a) (S a))` = an `S` node with THREE `S` children (not a
  production) and MISSING `(S (S (S a) (S a)) (S a))`.  Looks like the
  continuation/wrapper order for a left-recursive rule whose leading symbol
  is followed by the SAME nonterminal (INFERRED, not traced).
- layer 3 (epsilon): epsilon nodes missing (as in the baseline).
- layer 5 (`S -> S | a`, `S -> A | a; A -> S`): MISSING the cyclic derivations.
  The chains drop back edges, so nothing creates the cycle `S -> R -> S`.
  Expected: the completer must create it when `S[0,1]` closes (to be designed).
  Case 5d: CannotPlant at token 1.

Not a regression: the chain crate unit test `atom::default::test_default_atom::
test_from_grammar` fails 8/8 on a pristine copy of the user's tree (it asserts
fixed virtual-node numbers 166..=173 for an absolute-path grammar file).

Extra scratch dirs now: `<scratchpad>/base` (pristine copy of the user's tree)
and `<scratchpad>/target_base`; delete when no longer needed.

## Understanding of `insert_item` (genins.rs:178)
Steps: close `true_source` if virtual; choose initial `parents` per `PaVi`
(`Parent` / `Virtual` -> materialize via `trace` + `generate_virtual_frags`
then `plant_at_start` / `Empty` -> root); `reduction()` on the child for
`Parent` sources; lift parents from rule nodes to Non nodes; per `atom_child`
climb through the nonterminals in `query_reduction` (`splone` each); empty stack
= `CannotPlant`; `splant` the fragment; return a new `PaVi`.  A redesign was
discussed (split into named phases; move the "virtual marker" call at
default.rs:807 out of `insert_item`) but nothing depends on it yet.

## Suggested next steps (SUPERSEDED, see START HERE)
1. (Design settled, see above.)  Trace what `splone` and `reduction` do when
   `S[0,1]` closes, and where step (c) would go.
2. Apply FIX-A (user's hands) and re-run `--test oracle`.
3. Fix left recursion per the answer; the expansion junk (finding 1) must be
   handled either way.
4. Then Case 1 / Case 2 cycles (E4 gave "depth bound exceeded"; unanalysed),
   epsilon nodes (layer 3), and the TIMEOUT on cyclic grammars (layer 5).
5. Memoization (step 5 of `CLAUDE.md`) is skipped; none exists.

## Scratch state at the time of writing (SUPERSEDED, see START HERE)
Scratch copy `.../scratchpad/wsb` (original code + gated FIX-A + oracle tests)
and `.../scratchpad/targetb` (~240 MB) under
`/private/tmp/claude-501/-Users-durand-Desktop-Centre-A-propos-de-programmes-Rust-rep/`.
They are session-specific; delete them when no longer needed.

## Session rep-3 (2026-10-14/15): Case 1 (S -> S S | a) fixed up to 7 tokens in scratch

All edits are in `claude scratch/ws` only (nothing in the user's tree).  Result: `layer2_case_1`
passes; every previously-passing test still passes; layer3/4-hidden/5 and the probes
`x x a a a`, `d c b`, `d c b c b` still fail (unchanged).

Root causes found this session (MEASURED via DBG traces, fixes verified by the oracle):
1. `genins.rs` hook condition required `clone_index().is_none()`; clones are the normal case in
   ambiguous grammars.  Removed.
2. `graph/src/labelled/binary.rs` `PLChildren::remove` never removed the key from `indices`
   (HEAD bug): `has_edge` stayed true after `remove_edge`, so re-adding an edge was skipped.
3. `splone_left` was not idempotent (several `insert_item` calls per position): early return when
   the open node's first child is already the wrapper.
4. `open_ancestors`, `Eon::Ex` branch: do not add an edge blindly; push a (parent, new_top) pair.
5. Hook "already hooked" check (closed node under wrapper `w` of an open non-packed `h` node).
6. NEW cascade: after hooking `closed`, the open `h` ancestors whose LAST child is a rule wrapper
   (closed at pos, rule == fragment root rule) over `closed` complete at `pos` too; push them on
   `stack`.  (Without it S[0,4] / S[0,5] lose the split-at-2 alternative.)
7. NEW `splone_left` dedupe: two alternatives of one span produce the same continuation; reuse
   the existing open sibling (closed-node branch: before reopening; open-node branch: `bury` the
   duplicate clone).  `bury` (splone.rs) drops edges and relabels with a unique dummy label so
   label lookups never find an orphan (orphans broke `has_same_children`/`clone_node` asserts).
8. NEW `close_pavi(Virtual)`: do not replant the virtual alternative into a node that was already
   turned into a left-recursive continuation (first child is a closed left-wrapper rule node;
   `atom.is_left_wrapper_rule`).  It created a bogus open clone `S[2,None] -> [a]`.
9. NEW `reduction.rs` `repair_stale_last_children` (called in `end_of_input`): top-down `splone`
   in `reduction` leaves closed copies whose last child is still the open rule node although a
   closed twin exists (the `Eon::Ex` path never rewires).  Point the edge at the twin, or `bury`
   the copy if a sibling alternative already equals the repaired one.
Debug-only: `dbg_dump` calls in `end_of_input`, `SL ...`, `CASCADE`, `HOOKED_ALREADY`, `REPAIR`.
Not yet: regression tests in tests/ for 2 (graph unit test), 6-9 (Case 1 at 4,5,6 tokens already
covered by `layer2_case_1`), mid-parse application of the repair (only EOI so far).

### Regression tests added (tests/oracle.rs, module parser_side; also in the scratch copy)
Verified fail-before with scratch-only `NOFIX_<name>` env switches (each fix off, others on):
- `regression_graph_remove_edge_clears_index`  <- GRAPH (`PLChildren::remove` indices)
- `regression_case1_three_tokens_idempotent_hook`  <- IDEMP (`splone_left` early return)
- `regression_case1_four_tokens_clone_hook`  <- CLONE (hook on clones), HOOKED (already-hooked reuse), IDEMP
- `regression_case1_five_tokens_cascade`  <- CASCADE, DEDUPE, REPAIR (and the three above)
- `regression_case1_seven_tokens_virtual_and_ex`  <- VIRT (`close_pavi` replant), EX (`open_ancestors` Ex), and all above
Caveat: CLONE/HOOKED, CASCADE/DEDUPE/REPAIR and VIRT/EX are not isolated from each other by any
input I found; the test names the group.  Still-open failures found by the matrix (not fixed):
`S -> S S | S a | a` on `a a a a` (panic), `E -> E + E | n` on 5 operands, `S -> S S S | a` on 5 tokens.

## Archived from CLAUDE-PROGRESS.md on 2026-10-16 (prototype edits, regression tests, Updates 1-14)

The prototype edits 13-24 below are OBSOLETE (see `tests/DESIGN-forest-from-spans.md`).
Headings are demoted one level.

### Current results (scratch, deterministic, heights 1..5 vs the oracle)
PASS: layer 1, `layer2_case_1` (`S -> S S | a`, up to 7 tokens), `layer2_case_2`,
`layer4_left_recursion`, `validate_oracle::*`, the 10 regression tests, and probes:
`S -> S a | b`, `S -> S a b | c`, `S -> S a | a | b`, `T -> x S; S -> S a | a`, `T -> S x; ...`,
`S -> x S | S a | a` (`x a a`, `x x a a a`, `x x x a a a`), `S -> A b; A -> S c | d` (`d b`),
`S -> S S | S a | a` on 1..6 tokens, `E -> E + E | n` on 3 and 4 operands, `S -> S S S | a` on 3 tokens.
FAIL (open):
- `S -> S S | S a | a` on 7 and 8 tokens: MISSING trees only (no extras, no duplicates, invariants OK).
  MEASURED (`DIAG_ALTS`): `S[0,6]` has only the splits k=1 and k=5; the splits k=2,3,4 are lost.
  Cause (MEASURED in the dumps): `reduction()` at the entry of an `insert_item` closes the frame named by the
  item's forest_source; `split_parents` then makes OPEN clones of the ancestors whose last child is the new
  CLOSED wrapper (e.g. node 177 `S[0,None] -> [R7[0,2], R8[2,6]]`).  Nothing completes them: the cascade
  (edit 6) only follows wrappers whose rule equals the item's fragment-root rule (here 8 vs 5), and the item's
  `stack` only holds the parents of ONE reduced node.  INFERRED: the frames of S[2,*] need an item of their own
  at that position, and the obsolete (stale) items that used to serve them were dropped by edit 13.
  Tried and rejected: letting the cascade follow every wrapper that is the last position of its production
  (`atom.is_accepting(rule*2+1)`): 6 tokens then lose `S[0,5]` splits.  Not kept.
- `E -> E + E | n` on 5 operands: 1 MISSING tree (`(E (E (E (E n) + (E n)) + (E (E n) + (E n))) + (E n))`).
- `S -> S S S | a` on 4 tokens (NOT derivable: the forest must be empty) is ACCEPTED with trees like
  `(S (S (S a) (S a) (S a)) (S a))`, i.e. a node with 2 children for a 3-symbol rule: an incomplete
  continuation closed (same family as edit 12).  On 5 tokens: 3 missing, 5 extra, 5 duplicates.
- `S -> A b | d; A -> S c` on `d c b`, `d c b c b`: CannotPlant at token 1 (indirect left recursion).
- layer 3 (epsilon), `layer4_hidden_left_recursion`, layer 5 (cyclic): not started.
- `test_from_grammar` fails on the PRISTINE tree too (not caused by the prototype).

### The prototype = edits to the user's tree (dependency order; all scratch only)
Old items 1-8 (details in the archive, "The prototype = these edits"): FIX-A in
`item/reduction.rs` leaf branch; deterministic hasher `graph::det` (random hash order flipped
results); dummy root removal in `derive`; `plant_at_start` threshold 0; `open_ancestors` child
order; `create_new_label` top-level fix; `splone_left`; climb hook in `genins.rs insert_item`
(+ all virtual fragments); leading-chains tables in `atom/default.rs` (design in
`DESIGN-leading-chains.md`).  Added in session rep-3:
1. `graph/src/labelled/binary.rs` `PLChildren::remove`: also `self.indices.remove(&key)`.
   BUG IN THE USER'S TREE (HEAD too): `has_edge` stayed true after `remove_edge`, later
   `add_edge` to the same child was skipped.  `remove_edge` has one caller in the user's tree,
   `chain/src/item/default/mod.rs:1213`.
2. `genins.rs` hook condition: drop `clone_index().is_none()` (clones are normal when ambiguous).
3. `splone.rs splone_left`: early return if the open node's first child is already the wrapper
   `Rule(w)[start,pos]` (several `insert_item` calls visit one reduction per position).
4. `splone.rs open_ancestors`, `Eon::Ex` branch: if the existing node already has a child at
   `parent_edge`, push `(Parent(ex, parent_edge), new_top)` instead of `add_edge`.
5. `genins.rs` hook: "already hooked" reuse: closed node under wrapper `w` of an open non-packed
   `h` node with the same start -> push that open node on `second_stack`, `continue`.
6. `genins.rs` cascade: after `splone_left`, open `h` ancestors whose LAST child is the rule
   wrapper (rule == fragment root rule, closed at `pos`) over `closed` complete at `pos` too:
   push them on `stack` (not `second_stack`).
7. `splone.rs splone_left` dedupe: two alternatives of one span give the same continuation;
   closed-node branch reuses the existing open sibling before reopening, open-node branch
   buries the duplicate clone.  New helper `bury` (drop edges, unique dummy label), because a
   detached clone with a registered label broke `has_same_children`/`clone_node` asserts
   (also two `assert_eq!(parentsa.len(), 1)` made tolerant of zero parents).
8. `genins.rs close_pavi(Virtual)`: do not replant the virtual alternative into a node already
   turned into a continuation (first child is a closed left-wrapper rule node;
   new `atom.is_left_wrapper_rule`).
9. `item/reduction.rs`: new `repair_stale_last_children(only)`; in `reduction` step 3 the nodes
   returned by `splone` are collected and repaired right after the loop: a closed node whose
   last child is still the open rule node although the closed twin exists gets its last edge
   repointed (or is buried if a sibling alternative already equals the repaired one).  The
   `splone` `Eon::Ex` path never rewires; this is a patch around that.  MEASURED: no stale
   edge exists mid-parse, so the end-of-input call was dropped.
10. `genins.rs` hook, `has_h_parent`: only count a wrapper whose rule == `w` (a node hooked for
    the OTHER left-recursive rule of the same head must not block this rule's hook).
    Switch `NOFIX_HASHP`.  Smallest failing input: `S -> S S | S a | a` on `a a`.
11. `DefaultForest` field `buried: HashMap<usize, usize>` (buried node -> replacement), filled by
    `bury(node, replacement)`; `resolve_buried` follows the map until a node that is not buried;
    `insert_item` resolves `forest_source`/`true_source` through it at entry (`resolve_pavi`).
    Reason: pending chain edges keep the old node number in their `PaVi`; without it the
    debug assertion at `genins.rs` ~293 panicked (`a a a`).  Alternative considered: rewrite
    the chain edges when burying (needs patching `derive`'s pending items too; not done).
12. `splone.rs splone_left(.., atom)`: if the open node's first child is a closed wrapper of
    ANOTHER left rule (`atom.is_left_wrapper_rule`), the node is already a continuation with
    incomplete children and must not be closed again (it gave `S[1,2] -> [R7]`, i.e. `(S (S a))`
    extras).  Continue from its existing closed counterpart instead (`NOFIX_OTHERW`).
13. `genins.rs insert_item`, stack loop: a stack frame whose label is closed at a position `q < pos` is
    DROPPED (`stale_frame = true`), but only when the item's own forest_source child is closed before `pos`
    too (`DROP_ALL=1` drops every such frame; same results so far).  An item left with an empty stack plants
    nothing (no `CannotPlant`), and when its fragment root was not planted by another item the function
    returns the item's own `pavi` instead of panicking on `query_label(root_label)`.  Switch `NOFIX_STALE`.
    Why (MEASURED, 5 tokens): the lineage whose forest_source is `Parent(25,0,26)` (frame `S[2,*]` under the
    2nd symbol of `S S`) keeps that pavi after the hook closed node 26 in place at pos 3 and moved the live
    frame into the continuations (nodes 44/46/53).  At pos 4 the item still climbs from the closed wrapper 25,
    finds `S[0,3]`, `S[1,3]` (closed at 3 < 4) in its stack and `splone` re-ends them at 4 while keeping the
    old last child `R8[2,3]`: nodes whose children end before the node does (89, 90, 92).  The live
    continuations are served by the lineage with forest_source = true_source = the freshly planted
    `Parent(48,0,43)` (its stack was `[44,53,46]`, correct).  The same test exists in
    `chain/src/default.rs:758` for the `Non` branch (`Some(end) if end != pos => continue`), which is why
    I believe the item is obsolete, not needed.  Side effect: also fixes `S -> x S | S a | a` on `x x a a a`.
    OPEN QUESTION for the user: the right cure may be to give the hooked frame a new forest_source
    (remap `Parent(25,0,26)` to the continuation) instead of dropping; dropping loses the k=2..4 splits
    from 7 tokens on (see FAIL above).
Debug-only (do not copy): `dbg_dump`, `SL`/`CASCADE`/`REPAIR`/`HOOKED_ALREADY`/`STALE_FRAME`/`OA create`/
`CLONE_NODE` traces, probes, NOFIX switches, `DROP_ALL`.

### Regression tests (tests/oracle.rs, module parser_side; all fail without their fix)
`regression_graph_remove_edge_clears_index` (GRAPH); `..._three_tokens_idempotent_hook` (IDEMP);
`..._four_tokens_clone_hook` (CLONE, HOOKED); `..._five_tokens_cascade` (CASCADE, DEDUPE,
REPAIR); `..._seven_tokens_virtual_and_ex` (VIRT, EX).  New: `regression_two_left_rules_two_tokens_hook_per_rule`
(HASHP); `..._three_tokens` (HASHP, OTHERW; no longer isolates BURIED); `..._four_tokens_other_wrapper`
(HASHP, OTHERW); `..._five_tokens_stale_frame` (STALE, BURIED, HASHP, OTHERW; 5 and 6 tokens);
`regression_nested_left_recursion_stale_frame` (STALE; `x x a a a`).  MEASURED with the switches.
No input separates fixes inside the older groups.

### Still to do
1. `S -> S S | S a | a` on 7+ tokens (lost splits, see FAIL); decide drop vs remap for edit 13.
2. `S -> S S S | a` (incomplete continuation accepted on 4 tokens), `E -> E + E | n` with 5 operands,
   indirect `d c b` (CannotPlant), then layer 3 (epsilon) and layer 5 (cyclic): the reference Cases 3-5
   of `forest-examples.md` are still untouched and matter more than the exotic grammars above.
3. Final deliverable: a consolidated, ordered change list for the user (file, function,
   what, why; no diff unless asked) plus the regression tests.  Mention that
   `repair_stale_last_children` and edit 13 are patches around the `splone` `Eon::Ex` path and the stale
   forest_source, and that the scratch-only debug code and `NOFIX_*` switches must not be copied.

### Update (rep-3, after edit 13): stale-frame criterion and 7-token loss

Scratch-only experiments (`claude scratch/ws`), all MEASURED unless marked.

- New scratch field `hooked: HashMap<usize, Vec<usize>>` (closed copy C -> open continuations O), filled at the end of `splone_left`.
- All 39 frames dropped by edit 13 on `S -> S S | S a | a`, 7 tokens, are recorded hook-closed copies (`hooked` has an entry). So `q < pos` coincides with "closed by the hook" here; not shown for other grammars.
- Only 18/39 have a continuation O in some item's stack at the same pos; 21 do not (some O were closed and re-hooked, so the live node is further along the chain; not separated).
- `REMAP=1` (replace a dropped frame by its live continuations, following the chain, pushed into the same item): WORSE than the drop. 5 tokens ok; 6 tokens 24 missing; 7: 128 missing (drop: 104); 8: 272 (drop: 200). Kept behind `REMAP`, off by default.
- 7-token loss, cause located: node 177 (`S[0,None] -> [R7[0,2], R8[2,6]]`, clone of 55 made by `reduction()` at `insert_item` entry) has no closed twin `S[0,6]`, so the splits k=2..4 of `S[0,6]` are missing. At pos 5 the analogous twin (node 122) is made by the cascade (`CASCADE closed=116 -> [118,119]`); at pos 6 the cascade skips it because the wrapper rule (8) differs from the item's fragment-root rule (5).
- `CASC_ACC=1` (cascade also for wrappers with `atom.is_accepting(rule*2+1)`): 7 tokens 104 -> 48 missing, but 6 tokens now 48 missing (`S[0,5]` loses splits k=2,3). At pos 4 it fires 5 nonempty cascades instead of 3, changing the `SL dedupe` / `bury` outcomes. Cause of the 6-token regression NOT identified (guess: `SL dedupe` buries an open clone whose pending content differs).
- Next: understand why extra cascades at pos 4 change pos-5 structure (compare `SL dedupe` decisions), before deciding on a cascade rule.

### Update 2: 7 tokens of `S -> S S | S a | a` now pass (edits 14, 15; scratch)

MEASURED. Both are on by default in scratch; off switches `NOFIX_REMAP`, `NOFIX_CASCACC`. Both are needed.

- Root cause of the 7-token loss: an open node `S[0,None] -> [R7[0,j], R8[j,pos]]` with a complete last child never got its closed twin `S[0,pos]`.
  - Cause (a): the cascade after `splone_left` only followed wrappers of the item's own fragment-root rule.
  - Cause (b): when `CASC_ACC` was tried alone, the cascade-pushed node was processed with the item's rule (wrong hook wrapper `w`), and its open continuation was pushed into the item's `second_stack`, so the item planted its own fragment under a foreign-rule continuation (node `[R7[0,2], R5[2,3]]`, an extra/duplicate).
- Edit 14 (`genins.rs`, `insert_item`, cascade): also cascade over wrappers `Rule(r)` with `atom.is_accepting(r*2+1)`, remember `(node, r)` in the local `casc_rule`, use that rule instead of the fragment-root rule in the COND1 hook logic, and do not push the continuation of a foreign-rule cascade node into `second_stack`. Skip a node whose identical closed twin already exists (`NOFIX_CASCTWIN`; did not change results by itself).
- Edit 15 (`genins.rs`, stale-frame drop): instead of only dropping a stale frame, replace it by its live continuations: follow `DefaultForest::hooked` (closed copy -> open continuations, filled at the end of `splone_left`, `pub(crate)` field in `item/default/mod.rs`) until nodes that are open or end >= pos, and push `Parent(o, edge)`.
- Alone, neither works: remap without the cascade fix gives 24 missing at 6 tokens; the cascade fix without remap gives 104 missing at 7.
- New regression test `regression_two_left_rules_seven_tokens_cascade` (repo and scratch `tests/oracle.rs`). Fails with `NOFIX_REMAP` and with `NOFIX_CASCACC`.
- Suite in scratch: 30/39 pass (was 29/38 plus the new test), same failing set as before.
- Still failing: `S -> S S | S a | a` at 8 tokens (nodes `S -> ["S"]`, an incomplete continuation closed); `E -> E + E | n` at 5 operands is WORSE than before (4 missing + extras, was 1 missing); `S -> S S S | a` 4 tokens still accepted (extras), 5 tokens slightly better (1 missing, 3 extra, 1 dup, was 3/5/5); indirect `S -> A b | d; A -> S c`; layers 3, hidden left recursion, 5.

### Update 3: 8+ tokens of `S -> S S | S a | a` pass (edit 16; scratch)

MEASURED. `S -> S S | S a | a` now passes 3..10 tokens (heights 1..5). Suite in scratch: 30/39 (same failing set: layer 3 x3, hidden left recursion, layer 5 x4, layer6_random_grammars).

- Symptom at 8 tokens: invariant violations `S[1..3,7] -> ["S"]` (nodes `S[k,7] -> [R7[k,7]]`, one child).
- Root cause: `reduction()` step 2 (`chain/src/item/reduction.rs`, the `if let Some(end) = top_label.label().end()` branch) marks every closed node ending at `pos` as Correct. A closed hook wrapper `Rule(w)` (first symbol of a left-recursive rule) is not a complete rule instance, so the open continuation `S[k,None] -> [R7[k,7]]` (one child) looked complete and step 3 closed it with that single child. RED3 trace (`DBG`): nodes 295, 316, 280 with `deg=1`.
- Edit 16: in that branch, a closed node whose rule satisfies `atom.is_left_wrapper_rule(r)` is Incorrect (switch `NOFIX_WRAPLAST`).
- Regression test `regression_two_left_rules_eight_tokens_wrapper_last_child` (repo + scratch `tests/oracle.rs`); fails with `NOFIX_WRAPLAST` (11/12 regression tests pass).
- Other probes (unchanged by edit 16): `E -> E + E | n` at 5 operands still worse than before edit 14 (4 missing + extras); `S -> S S S | a` 4/5 tokens still wrong.

### Update 4: `E -> E + E | n` fixed (edits 17-20; scratch)

MEASURED. `E -> E + E | n` passes 3..7 operands; `E -> E + E | E * E | n` (4 operands) and `E -> E + T | T; T -> T * n | n` (5 operands) pass; `S -> S S | S a | a` still passes 3..10 tokens. Suite in scratch: 32/44 pass; failing: layer 3 x3, hidden left recursion, layer 5 x4, and the open tests (indirect left recursion, `S -> S S S | a` 4 and 5 tokens). `layer6_random_grammars` is `#[ignore]`d.

New known-open tests added to `tests/oracle.rs` (repo and scratch), named `open_*`: `open_three_symbol_rule_four_tokens`, `open_three_symbol_rule_five_tokens`, `open_indirect_left_recursion`. They fail on purpose until fixed.

Causes (E at 5 operands; 3 missing + extras + invariants `E -> [E + E + E]`, children ending later than the node):
- Edit 17 (`genins.rs`, cascade): `left_wrapper` is keyed by the slot AFTER the wrapper (`R5` for `E -> E + E`), which coincides with the last slot only for two-symbol rules. A cascade node of another rule was hooked with the rule of its last slot (`R6`): `left_wrapper(0, E, 6) = None`, so it was never hooked. Now the hook slot is `first_child_rule + 1` when the first child is a left wrapper (`NOFIX_CASCWRAP`).
- Edit 18 (`genins.rs`, `insert_item` entry): `repair_stale_last_children(None)` sweep at every entry (`NOFIX_REPAIRALL`). The remapped continuation closed at pos 7 had an open last child; the correct closed twin appeared later, so the clone must be repointed (and dropped if it duplicates a sibling).
- Edit 19 (`reduction.rs`, `repair_stale_last_children`): also repair a last child that is closed but ends elsewhere than the node (was: open only) (`NOFIX_REPAIRANY`); and (`default.rs`) run the sweep at the end of input by default (was `EOI_REPAIR`, now `NOFIX_EOIREPAIR`).
- Edit 20 (`splone.rs`, `splone_left`, open-node branch): when the node is in progress (more than one child, last child open) the hook continuation is a separate open clone (made with edge 0, since edge `edge_index` resolves to the node itself) instead of repurposing the node, which lost `E[0,None] -> [R4[0,5], R5[5,6], R6[6,None]]` and with it the split `E[0,5] + E[6,9]` (`NOFIX_INPROG`).
- Interactions (MEASURED with the switches): `NOFIX_CASCWRAP` and `NOFIX_INPROG` fail the new E test; `NOFIX_REPAIRALL`, `NOFIX_REPAIRANY`, `NOFIX_EOIREPAIR` fail the 7- and 8-token S tests (edit 20 leaves more stale last children for the sweeps to fix). An earlier experiment, skipping remapped in-progress continuations (`REMAP_SKIP_INPROG`), fixed E but lost 48 trees at 8 tokens of S; kept off.
- Regression test: `regression_three_symbol_left_rule_five_operands` (repo and scratch `tests/oracle.rs`). 13/13 regression tests pass.
- Still wrong: `S -> S S S | a` at 4 tokens (extras, accepted though not derivable), 5 and 7 tokens; the indirect grammar; layers 3, hidden left recursion, 5.

### Update 5: edit 17 without arithmetic

The `first_child_rule + 1` of edit 17 was an unproven assumption. `atom/default.rs` fills `left_wrappers[(h, y, r)] = slot` with `r` ranging over the FIRST LITERALS reachable after the leading symbol (`first_literals`), not "the next slot", so `w+1` is not guaranteed. `r` is only used to look up `w`; a cascade node's own first child already is its wrapper. Now `casc_rule` entries carry `Some(w)` (the first child's rule, when `atom.is_left_wrapper_rule`), COND1 uses `known_w.or_else(|| left_wrapper(...))`, and the head comes from `get_rule_num(w)`. Suite unchanged (32/44, same failing set). Added `coverage_longer_left_recursive_rules` (4-symbol rule, shared leading symbol with different lengths, `S x S y S`); passes. Not covered: leading nonterminal that is not the head (indirect), regexp rules with alternatives after the leading symbol.

### Update 6: `S -> S S S | a` (edits 21-23; scratch)

MEASURED. Passes 3..6 and 8 tokens (4, 6, 8 are not derivable: the forest must be empty). 7 and 9 tokens: no invalid node left, 1 tree missing at h=3 (`(S (S a a a) (S a a a) a)`) and more at larger heights. Suite in scratch: 35/46 (failing: layer 3 x3, hidden left recursion, layer 5 x4, `open_indirect_left_recursion`, and the new `open_three_symbol_rule_seven_tokens`).

- Harness fix (tests only): `check_case` returned the invariant "the forest is empty (no root)" even for inputs the oracle says are not derivable. Now no root is correct iff the oracle finds no tree at any height.
- Edit 21 (`genins.rs`, cascade after `splone_left`): every cascade node needs an ACCEPTING wrapper (`atom.is_accepting(rule*2+1)`), also in the same-rule case (edit 6 assumed the item's slot is the last slot of its rule, true only for two-symbol rules). Closed `S[0,2] -> [R4, R5]` of `S -> S S S`. Switch `NOFIX_CASCACCEPT`. Test `regression_three_symbol_rule_incomplete_cascade` (4, 5, 6 tokens).
- Edit 22 (`genins.rs` + `atom/default.rs`): new `DefaultAtom::follows` / `can_follow(prev, next)`: slot `next` may directly follow slot `prev` (through the target of each literal edge, `first_literals`). In the `HFILTER` loop a frame `(node, edge)` is kept only if `can_follow(rule of child at edge, fragment root rule)`. Fixes planting `R6` after `R4` (frame `edge=0` on `[R4, R5]`). Switch `NOFIX_FOLLOW`. No passing test needs it; at 7 and 9 tokens it removes 10 / 17 invalid nodes.
- Edit 23 (`genins.rs`, stack loop): a frame whose node is the reduced nonterminal completes it only if the child at `edge` has an accepting slot; hook wrapper slots (`is_left_wrapper_rule`) are exempt (they are legitimately non-accepting and are pushed with edge 0). Closed `S -> [R4, R5]` made by the hook with `edge=1`. Switch `NOFIX_ACCFRAME`. Test `regression_three_symbol_rule_eight_tokens_frame_filters`. Without the exemption the 7- and 8-token tests of `S -> S S | S a | a` fail.
- Open: 7 tokens. The open node `S[0,None] -> [R4[0,3], R5[3,6]]` (node 95, made during the pos-6 items) is in no item's stack, so the last symbol is never planted under it. Test `open_three_symbol_rule_seven_tokens`.

### Update 7: what a stack frame is (investigation; no behaviour change kept)

- `accepting_vec` (`atom/default.rs` ~499-555) is a fixpoint: a state is accepting if it ends a rule OR a nullable edge leads to an accepting state. So a wrapper followed only by nullable symbols IS accepting (read from code, not measured; the `2*slot+1` indexing is the original author's convention). Edit 23's wrapper exemption is only about non-nullable remainders.
- Convention (MEASURED, `DBGF`): a frame `(node, edge)` means "the child at `edge` is the one before the planting point" (`splant` keeps `edge+1` children). The planting filter (`HFILTER`) runs AFTER the reduction loop, so an invalid frame is hooked/closed before it is ever checked (cause of closed `S -> [R4, R5]`).
- Scratch diagnostics (off by default): `origin` side table in `insert_item` (init / remap / hooked_already / hook_open / casc / climb), `DBG_ORIGIN=1` prints `FRM origin lvl r childrule nt_match follow_would_drop`; `FOLLOW_ENTRY=1` applies `can_follow(child rule at edge, fragment root slot)` at the entry of reduction level 0.
- Result: `FOLLOW_ENTRY` breaks `regression_two_left_rules_{seven,eight}_tokens_*` (S S | S a | a): `S[0,6]` loses the splits k=2,3 (`[R7[0,j], R8[j,6]]`). Init frames whose child at `edge` is the OTHER production's wrapper (e.g. child slot 7, item slot 5) are needed: the item completes an inner `S` (via `S a`) which is then planted as the second child of the `S S` continuation. So there are at least two roles: (P) plant the item's fragment root under the node (needs `can_follow(child slot, r)`), (C) receive the CLOSED reduced nonterminal as the next child (needs a following slot whose symbol is `Non(reduction_nt)`; independent of `r`). Edit 23 (accepting child at `edge`) and edit 22 (`can_follow`) mix the two, hence the wrapper exemption.
- `can_follow` is too permissive at rule ends: `(8, 5)` is reported as allowed (`first_literals` crosses epsilon edges at the end of a rule into other rules' starts); fine for the filters that use it only in role P but should be restricted.
- Next: tag frames with role (P or C) at creation (from the relation above), apply the P check only to P frames and a C check ("next slot after child at `edge` has symbol `Non(nt)`") to C frames, and see whether edit 23 and the wrapper exemption can go. Open question for the user: read the paper's definition of the stack (ask first).

### Update 8 (roles of the stack, MEASURED)
- At reduction level 0 every surviving input frame has nt_match=true (8-token S S S test: 34 frames, 7 of them would be dropped by `can_follow`, and are needed).
  So the two roles are two phases: input frames (C, they close their node) and the output stack after the levels (P, where the fragment root is planted).
  `can_follow` (edit 22) belongs only to P, where it already runs (HFILTER).
- Full suite with and without `NOFIX_FOLLOW`: identical (35 passed, 10 failed). `can_follow`, strict or coarse, does not decide any remaining failure.
- Next: the open 7-token `S -> S S S | a` case (open node 95 in no item's stack).

### Update 9 (7-token `S -> S S S | a`, traced)
- MEASURED: the hook (`splone_left` of node 81, closed copy 91 = S[3,6]) clones three containers: 94 = S[1,None]->[R4,R5,R6[3,6]] (gets a cascade frame, last slot accepting), 95 = S[0,None]->[R4[0,3],R5[3,6]] and 96 = S[2,None]->[R4[2,3],R5[3,6]] (need one more sibling, get NO frame).
- INFERRED: the hook branch `continue`s before the climb into `second_stack` that the non-hook branch does, so open containers with a non-last slot never become planting frames.
- Experiment (opt-in `HOOKCLIMB=1`, off): push `(gp, idx)` for open NT parents of each parent of `closed` ending at pos. Result: breaks 4+ passing tests (extra and missing trees), 7-token tree still missing, also when restricted to the last level. Not a fix; frames of the wanted kind (`edge=1, crules=[4,5]`) do appear, so more is wrong (maybe which clones exist, or the cascade twin check).

### Update 10 (7-token `S -> S S S | a` FIXED)
- MEASURED: the `r=6` item's start stack contained the closed cascade node 91 (`S[3,6]`, edge 2 = its last child). The planting filter (HFILTER) dropped it (a closed node takes no child) and nothing replaced it, so containers 94/95/96 (open, waiting for a third child) got no frame.
- Fix (`genins.rs`, just before the HFILTER loop, switch `NOFIX_CLOSEDCLIMB`): a frame whose node is closed at `pos`, not packed, with `edge` = last child, is replaced by `(gp, edge)` for every open non-packed NT grandparent through its wrapper parents. The normal head and `can_follow` filters then apply (e.g. `(81,0)` after R4 is dropped for `r=6`, `(95,1)` after R5 is kept).
- Needed: `NOFIX_CLOSEDCLIMB=1` fails the test `regression_three_symbol_rule_seven_tokens_closed_frame_climb` (formerly `open_three_symbol_rule_seven_tokens`). Suite: 36 passed, 9 failed, 1 ignored.
- Earlier `HOOKCLIMB` experiment (frames added at the hook, off by default) is superseded: it put the frames at the `r=5` item where they are invalid.
- `S S S|a`: 7 passes; 9 and 11 tokens still lose trees (3 at h=4 for 9); 10 is non-derivable (correct, empty).

### PLAN for next session (agreed with the user; Update 10's fix is NOT yet considered sound)
The closed-frame climb (`NOFIX_CLOSEDCLIMB`, genins.rs before the HFILTER loop) works on the tests but has unjustified choices:
1. Packed nodes: treat a packed node as the base (the "closed node"), its parent as the wrapper, and that parent's parent as `gp`, instead of rejecting them (a packed parent currently makes `gp` a rule node and the case is dropped).
2. Last-ness (`edge + 1 == degree`): the user thinks it is wrong, middle nodes can grow new follow-ups. Measure on other grammars whether last-ness always holds, or derive it from the definitions; otherwise remove it (mind that `splant` keeps only `edge+1` children).
3. One level only: the user thinks it is wrong too. Measure or infer the stopping condition of the climb (what makes a container final: open + non-packed? closed copies served by other items? cascade completion?).
Also check what else the fix drops: closed `gp`, containers whose wrapper is not last. Add a regression test for each case found, and keep NOFIX switches until justified.

### Update 11 (plan steps 1-3 done; MEASURED)
- Closed-frame climb now uses packed nodes as bases too (`NOFIX_CCPACKED` disables): 8 occurrences in the suite; S S S at 11 tokens: missing 21->19 (h=4), 74->62 (h=5). Suite unchanged: 36 passed, 9 failed.
- Last-ness of the closed frame (`edge+1==degree`): never violated (0 events in suite, 9 and 11 token S S S). `CC_NOLAST=1` disables it, no change. Containers whose wrapper is not their last child: 10 events, already handled (the code never required last-ness for `gp`).
- Stopping condition: closed `gp` skipped 5 times, all with end == pos. Recursive climb from them (`CC_REC=1`, off) changes nothing.
- 9 tokens `S S S`, the missing tree: open `170 = S[0,None] -> [R4[0,3], R5[3,8]]` (wrapper 168 over packed 195, alternatives incl. closed 199) never gets its third child. The closed frame `(199,2)` is in the stack of the `r=5` item only (pavi child 116 = S[7,8]) and is consumed there by the hook (SPLONE_LEFT 199, open=158), so the climb never sees it. The `r=6` items have frames from another source (R5 node 148). OPEN: how should the `r=6` item receive containers found when the `r=5` item consumed a closed frame? (design question for the user)
- Diagnostics added (no behaviour change): `DBG_ANC=1`, `ANC_TARGETS=a,b,c` (path from the pavi child to targets), `DBG_CC=1`.

### Update 12 (9/11/13-token `S -> S S S | a` FIXED)
- MEASURED cause: two items at pos 8 share the pavi child 116 (S[7,8]) but start from different rule nodes (r=5 item: R6[7,8] 115; r=6 item: R5[7,8] 148). In the 7-token case the r=6 item's pavi child was still OPEN, so the init `reduction` produced a closed copy and the item took `parents_of(reduced)`, i.e. the parents of ALL wrappers of the shared closed node (R5 and R6), reaching the closed S[3,6] frame. At 9 tokens the pavi child was already closed at `pos` (closed by an earlier item), `reduced == nth_child`, so only the parents of the planted R5 node were used and the wrapper R6[7,8] -> closed S[3,8] (199) -> packed 195 -> R5[3,8] 168 -> container 170 was never reached.
- Fix (`genins.rs`, the `reduced != nth_child` test after `reduction` in `insert_item`, switch `NOFIX_CLOSEDPARENTS`): treat a pavi child already closed at `pos` like a freshly reduced one (take all its parents).
- Needed: `NOFIX_CLOSEDPARENTS=1` fails the new test `regression_three_symbol_rule_nine_and_eleven_tokens_closed_pavi_child`. Suite: 37 passed, 9 failed, 1 ignored. S S S passes for 3,5,7,9,12,13 tokens (even counts that are non-derivable give an empty forest).
- With this fix on: `NOFIX_CLOSEDCLIMB=1` fails the 7-token regression test, and `NOFIX_CCPACKED=1` fails the 9/11 test (the packed-base part is needed, 9 tokens: 3 missing at h=4 again). `CC_REC=1` (recursive climb) and `CC_NOLAST=1` still change nothing: still no evidence for them.
- Separate, pre-existing gap (same with and without this fix): `S -> S S | a` at 8, 9, 10, 12 tokens misses trees at h=5 (2, 2, 4, 12); 7 tokens is fine.

### Update 13 (2026-10-15): change of approach — READ `tests/DESIGN-forest-from-spans.md` FIRST
- MEASURED: the current forest construction depends on the item processing order. Adding a constant field to `Edge`'s hash (scratch `HASH_START=1`) changes `labels_of` order and breaks 3 regression tests (37 -> 32 passed).
- MEASURED: with frame starts carried on edges, the recognizer's completion events give spans covering every nonterminal span of the oracle trees in 20/21 cases (scratch test `probe_spans`, `PROBE_START=1`), independent of hash order. The 21st is a recognition bug.
- New failing test (both copies): `open_recognition_nullable_rest_of_virtual_node` (`S -> A A; A -> a | eps` rejects `a`; virtual node `[S]^a` not accepting). This is why `layer3_case_3` fails.
- Decision proposed: record spans in the recognizer, build the forest after the input from spans + grammar (productive fixpoint, keyed nodes). Plan in the design doc, section 5. Prototype edits 13-24 become obsolete.
- Scratch-only probe code (default behaviour unchanged): `Edge.start` + `with_start` (no-op unless `PROBE_START`), manual `Hash for Edge` (start hashed only with `HASH_START`), `DefaultChain.{spans,marks,tail_links}`, `node_pos`, `probe_spans`, `probe_pos`; env `PROBE_OFF`, `PROBE_TR`, `PROBE_CASE`, `NO_TAIL`. Scratch suite: 38 passed (incl. the print-only `probe_spans`), 10 failed (the 10th is the new recognition test), 1 ignored.

### Update 14 (2026-10-15): facts with nonterminals verified; recognizer bugs — see `tests/DESIGN-forest-from-spans.md` (rewritten)
- Edges carry `start` and `frame_nt`; facts `(X, i, j)` recorded (E1-E4 + closures). MEASURED: no missing fact on 27 hand cases and 597 random grammars; "derivable from facts == exactly derivable" on 398 random grammars (needs the virtual-node fix).
- Recognizer bugs (design doc section 6): 6.1 virtual-node acceptance ignores left-linear edges (needed; scratch `FIX_VACC`); 6.2 `extend` acceptance order-dependent (scratch `FIX_EXTEND`); 6.3 reduction lists lose nonterminals (scratch `FIX_REDSET`, union sets); 6.4 false acceptances (acceptance not relative to the frame nonterminal; harmless for the facts approach).
- New failing test (both copies): `open_recognition_unreachable_rule_accepts_too_much`.
- CORRECTION: an earlier round used `env $v` with several switches in zsh, which sets only the first; the claims "FIX_SECOND needed?", "FIX_EXTEND has no effect" and the first hash-order check were invalid. Redone with `env ${=v}`; the design doc has the corrected results.
- Scratch suite (probes skipped): 35 passed, 11 failed (9 old + 2 recognition tests), 1 ignored.