goober/nte-exporter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

0.1.1 - Add libpcap/Npcap backend and robust live capture

Browse source 
Introduce a cross-platform libpcap/Npcap capture backend with a Windows raw-socket fallback and plumbing to select backends via --capture-backend. Add ctypes-based libpcap wrapper (open_libpcap_capture, CaptureStats, LibpcapUnavailable) and a backend factory; refactor live capture runner to use the new backend, StopKeyMonitor, and improved clipboard handling. Make session and protocol decoders resilient to pipelined/multi-page responses and non-byte-aligned streams (alignment iterator, alignment-aware decoding, embedded-record trimming, slice support). Update mitmproxy flows adapter to reuse LiveHistorySession. Expand console messages and docs/README to explain cross-platform requirements, usage, and capture diagnostics. Minor: update package description in pyproject and adjust ARC/response handling and row bookkeeping to record row indices.
This commit is contained in:
Golumpa 2026-06-12 12:58:15 +01:00
parent 1144e33a57
commit 0e78906e3a
20 changed files with 1234 additions and 147 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/export-format.md
View file

@ -10,7 +10,7 @@ The sanitized JSON export uses:
"source": "packet_capture",
"exporter": {
"name": "nte-history-exporter",
"version": "0.1.0"
"version": "0.1.1"
},
"banner": {
"id": "Lottery_Permanent",

7
docs/limitations.md
View file

@ -8,10 +8,11 @@ Known limitations:
- The game appears not to provide a unique server-side roll ID in the decoded record body.
- UIDs are generated deterministically from decoded fields and timestamp-group order.
- A partially captured oldest timestamp group is still exported; its captured prefix has stable UIDs, and a later deeper scan adds the rest with the same UIDs.
- Pages are anchored to the continuous run starting at page 1; pages after the first gap are ignored and reported as warnings.
- Live capture currently uses Windows raw sockets and requires administrator permission.
- Pages are anchored to the continuous run starting at page 1. Live capture reports gaps immediately and accepts replacement pages from another pass while the exporter remains open. Scrolling backward does not request cached pages again, so close and reopen the history board before rescanning. Any gap left when capture ends causes later pages to be ignored and reported as warnings.
- Pipelined page requests and multi-page responses are supported, including observed 10-record responses containing two consecutive Monopoly pages. Non-byte-aligned response payloads are realigned before decoding.
- Live capture prefers Npcap on Windows and automatically falls back to the built-in raw-socket backend if Npcap is unavailable. Linux and macOS require the system libpcap runtime.
- The file adapter reads mitmproxy `.flows` captures for research and testing.
- The more stable and reliable Npcap/libpcap capture is not implemented yet.
- Npcap is Windows-only and is not redistributed with this project; Linux and macOS use their system libpcap.
- Reward keys decode to their reward id string, so unknown rewards still export a usable `reward_id`; display names/ranks come from the mapping JSON files and should be expanded as new rewards appear.
Privacy guardrail: raw packet data must not be included in sanitized exports.

13
docs/packet-format.md
View file

@ -5,12 +5,20 @@ This prototype supports separate Monopoly and Arc/Gashapon history decoders.
## Monopoly
- History is fetched over the UDP game connection.
- Client history-page requests are 45 bytes.
- A client history-page request has a 45-byte request prefix. UDP payloads may
contain additional coalesced transport data after that prefix.
- History request constant: `4220` / `0x107c`.
- Request selector `4`: `Lottery_Permanent`.
- Request selector `8`: `Lottery_LimitedCharacter`.
- Request page cursor: `page_number * 4`.
- Normal server responses contain 5 history records.
- The client can pipeline several page requests before responses arrive.
- Under load, one server response can contain multiple consecutive pages. The
observed format contained 10 records representing two five-record pages,
with an internal response header before the second page's first record.
- Some batched responses begin at a non-byte-aligned position in the UDP
payload. The decoder tests all LSB bit offsets; captures have been observed
where the record stream begins five bits into the byte stream.
- The final page may contain fewer than 5 records.
Decoded fields:
@ -40,7 +48,8 @@ Pages are anchored to the continuous run starting at page 1 (history always load
## Arc / Gashapon
- Arc history uses a separate 34-byte request.
- Arc history uses a separate 34-byte request prefix and may likewise have
coalesced transport data after it.
- Request constant: `2060` / `0x080c`.
- Cursor step: `2`.
- Pool: `Arc_MiracleBox`.