Add safe snapshot and segment assembly

Add envelope-aware snapshot and segment assembly for structured Monopoly and Arc history records.
This commit is contained in:
Golumpa 2026-07-15 19:48:11 +01:00
parent a5312520e7
commit 73bb100cd6
8 changed files with 601 additions and 23 deletions

View file

@ -13,6 +13,9 @@ Known limitations:
- Structured Monopoly and Arc blocks are parsed in enrichment/fallback mode.
The existing decoder remains authoritative when structured rows do not agree
on record count, reward ID, and timestamp.
- Structured snapshot/segment assembly is limited to runs where every decoded
row came from structured fallback. Existing primary-decoder runs are never
reordered. Ambiguous generations retain the last proven snapshot.
- Live capture prefers Npcap on Windows and automatically falls back to the built-in raw-socket backend if Npcap is unavailable. Linux and macOS require the system libpcap runtime.
- The file adapter reads mitmproxy `.flows` captures for research and testing.
- Npcap is Windows-only and is not redistributed with this project; Linux and macOS use their system libpcap.

View file

@ -21,8 +21,29 @@ The structured parser is deliberately compatibility-gated:
- Malformed, incomplete, mismatched, or ambiguous structured data is ignored;
it cannot overwrite a successfully decoded primary row.
Structured protocol envelopes identify a history stream, page, query side, and
segment index. For an all-structured fallback run, the snapshot assembler:
- orders segments by their protocol index while retaining row order inside
every segment;
- ignores exact retransmissions;
- starts a new generation when an existing segment index changes;
- replaces an older snapshot only when the new generation covers at least the
same segment range;
- merges a partial generation only when its suffix has one unique overlap with
the proven snapshot; and
- retains the proven snapshot and records an assembly warning when a merge is
ambiguous.
Assembly never runs on a history run containing a successfully decoded primary
row. Such runs retain their existing packet/page order exactly. Timestamp-group
ordinals and UIDs are calculated only after any fallback assembly, using the
same inputs and algorithms as before.
`decoder_mode`, `structured_protocol_view`, `structured_pool_id`,
`secondary_reward_id`, and `secondary_quantity` are research/debug CSV fields.
`structured_generation_index`, `structured_assembly`,
`structured_assembly_warning_count`, `secondary_reward_id`, and
`secondary_quantity` are research/debug CSV fields.
They are intentionally omitted from the public JSON export, whose format stays
at version 1.