Add achievement export support

.github/workflows/update-mappings.yml

@@ -1,4 +1,4 @@
-name: Update reward mappings
+name: Update mappings
 
 on:
   workflow_dispatch:
@@ -14,7 +14,7 @@ permissions:
   pull-requests: write
 
 concurrency:
-  group: update-reward-mappings
+  group: update-mappings
   cancel-in-progress: false
 
 jobs:
@@ -42,7 +42,7 @@ jobs:
       - name: Install project
         run: python -m pip install -e .
 
-      - name: Rebuild reward mappings
+      - name: Rebuild mappings
         run: python tools/update_mappings.py --source-ref "$NTE_ASSETS_REF" --apply
 
       - name: Run tests
@@ -108,7 +108,7 @@ jobs:
               f"- {changes['additions']} additions\n"
               f"- {changes['updates']} updates\n"
               f"- {changes['deletions']} deletions\n\n"
-              "The workflow rebuilt all reward mappings, resolved English names through "
+              "The workflow rebuilt all reward and achievement mappings, resolved English text through "
               "`Localization/en/game.json`, and passed the complete test suite. Pool mapping "
               "files and UID inputs were not changed.\n\n"
               f"[Workflow run]({os.environ['RUN_URL']})\n"
@@ -125,12 +125,12 @@ jobs:
 
           if [ -n "$pr_number" ]; then
             gh pr edit "$pr_number" \
-              --title "chore: update reward mappings from NTE_Assets" \
+              --title "chore: update mappings from NTE_Assets" \
               --body-file build/mapping-update/pull-request-body.md
           else
             gh pr create \
               --head "$AUTOMATION_BRANCH" \
               --base "$BASE_BRANCH" \
-              --title "chore: update reward mappings from NTE_Assets" \
+              --title "chore: update mappings from NTE_Assets" \
               --body-file build/mapping-update/pull-request-body.md
           fi

README.md

@@ -4,7 +4,7 @@
 
 # NTE History Exporter
 
-Prototype CLI exporter for **Neverness to Everness** pull history — decodes your own game traffic into sanitized JSON ready for tracker import.
+Prototype CLI exporter for **Neverness to Everness** pull history and achievements — decodes your own game traffic into sanitized JSON.
 
 ![Python](https://img.shields.io/badge/python-3.10+-blue)
 ![Platform](https://img.shields.io/badge/platform-Windows%20%7C%20Linux%20%7C%20macOS-0078D6)
@@ -23,10 +23,17 @@ Prototype CLI exporter for **Neverness to Everness** pull history — decodes yo
 
 ## What It Does
 
-The exporter decodes Permanent Board, Limited Character Board, Arc Miracle Box, and Mystery Box history pages from captured UDP data, applies conservative timestamp-boundary handling, and writes sanitized JSON suitable for tracker import.
+The exporter decodes Permanent Board, Limited Character Board, Arc Miracle Box,
+and Mystery Box history pages from captured UDP data. It also captures tracked
+achievements during login, including completed and in-progress records. Pull
+history is written as tracker-ready JSON and achievements use a separate
+versioned JSON format.
 
 > [!NOTE]
-> The import JSON contains decoded history rows and the shareable NTE user UID when it can be detected. It does **not** export tokens, account IDs, role IDs, device IDs, server IPs, raw packets, cookies, session data, or other capture metadata.
+> Export JSON contains decoded history or achievement records and the shareable
+> NTE user UID when it can be detected. It does **not** export tokens, account
+> IDs, role IDs, device IDs, server IPs, raw packets, cookies, session data, or
+> other capture metadata.
 
 ## Requirements
 
@@ -90,7 +97,11 @@ From source:
 Or simply double-click **`run-exporter.cmd`**.
 
 > [!IMPORTANT]
-> For automatic user UID and account-server detection, launch the tool **before pressing Start on the game's main menu**. If you are already in game, history capture can still work; the tool will ask for missing account details before saving.
+> Start the exporter **before pressing Start on the game's main menu**. This is
+> required for achievement capture and enables automatic user UID and
+> account-server detection. If already in game, pull-history capture still
+> works, but achievements require returning to the main menu and logging in
+> again; missing account details are requested before saving.
 
 Once running, open any supported history board in game. The tool keeps listening until you press any key. After it prints and saves the results, press any key again to close the exporter. Exports are written under `exports\` as:
 
@@ -98,6 +109,7 @@ Once running, open any supported history board in game. The tool keeps listening
 - `<user_uid>_Limited_<date_time>.json`
 - `<user_uid>_Arc_<date_time>.json`
 - `<user_uid>_MysteryBox_<date_time>.json`
+- `<user_uid>_Achievements_<date_time>.json`
 
 If the user UID is not detected automatically, the console asks for it before saving. Leaving it blank saves as `unknown_<banner>_<date_time>.json`, but may prevent import on some trackers.
 
@@ -112,19 +124,22 @@ If a page response is missed, the exporter reports the missing page number while
 
 ### Achievement export
 
-Start the live exporter before launching the game. The completed achievement
+Start the live exporter before pressing Start on the game's main menu. The
-list is loaded during login, captured automatically, and written to
+tracked achievement state is loaded during login, captured automatically, and written to
 `exports/<user_uid>_Achievements_*.json` as a versioned achievement export with
 account metadata, completion totals, and records grouped by achievement category.
-Both completed and incomplete achievements are included with their current
+Both completed and in-progress achievements are included with their current
 progress, status, and completion time when available. Generated metadata adds
 the localized title and description, target, quality, and rewards from the
 current `NTE_Assets` achievement table. Newly captured IDs that are not yet in
 the bundled mapping still export normally; only their optional display metadata
 is omitted until the mappings are refreshed.
-Completed PlayStation trophy entries are retained and identified separately
+PlayStation trophy entries are retained and identified separately
 even though they are not shown in the in-game achievement list. The console
-reports the visible and PlayStation counts as soon as the list has been decoded.
+reports completed and in-progress totals for in-game and PlayStation records as
+soon as the list has been decoded. Every observed record without a completion
+timestamp is classified as `in_progress`, even when its numeric progress is
+zero; compound achievements can track hidden checklist progress this way.
 
 #### Linux/macOS
 
@@ -191,22 +206,22 @@ The exporter automatically includes the shareable NTE user UID when it appears i
 
 ## Mapping maintenance
 
-Reward metadata can be rebuilt directly from the latest NTE_Assets tables and
+Reward and achievement metadata can be rebuilt directly from the latest
-English translation files. The reward snapshot may change IDs while UID inputs
+NTE_Assets tables and English translation files. Mapping snapshots may change
-remain untouched:
+IDs while UID inputs remain untouched:
 
-Run **Update reward mappings** from the GitHub Actions tab to generate and test
+Run **Update mappings** from the GitHub Actions tab to generate and test
 the snapshot in a reviewable pull request, or run it locally:
 
 ```powershell
 python tools/update_mappings.py
 ```
 
-This rebuilds reviewable reward-map candidates directly from NTE_Assets under
+This rebuilds reviewable mapping candidates directly from NTE_Assets under
 `build/mapping-update/`; committed mappings are untouched unless `--apply` is
 explicitly supplied. The snapshot may include additions, updates, and removals,
 while pool mappings and UID inputs remain untouched. See
-[Reward mapping updates](docs/mapping-updates.md) for the source rules and
+[Mapping updates](docs/mapping-updates.md) for the source rules and
 review workflow.
 
 ## Privacy

docs/export-format.md

@@ -15,7 +15,7 @@ decoder-only offsets.
   "capture_source": "npcap",
   "exporter": {
     "name": "nte-history-exporter",
-    "version": "0.2.1"
+    "version": "0.3.0"
   },
   "banner": {
     "id": "Lottery_Permanent",
@@ -176,6 +176,79 @@ Mystery Box:
 }
 ```
 
+## Achievement export
+
+Achievements are written separately as
+`<user_uid>_Achievements_<date_time>.json`. The document contains only records
+observed in the account's login response; definitions found only in the asset
+mapping are not synthesized into the export.
+
+```json
+{
+  "format": "nte-achievement-export",
+  "format_version": 1,
+  "game": "Neverness to Everness",
+  "source": "live_capture",
+  "capture_source": "npcap",
+  "exporter": {
+    "name": "nte-history-exporter",
+    "version": "0.3.0"
+  },
+  "scan": {
+    "in_game": {
+      "total_achievements": 367,
+      "completed_achievements": 310,
+      "in_progress_achievements": 57
+    },
+    "playstation": {
+      "total_achievements": 35,
+      "completed_achievements": 31,
+      "in_progress_achievements": 4
+    }
+  },
+  "user_uid": "218216016349",
+  "server_id": "23003",
+  "account_region": "EU",
+  "categories": {
+    "battle": [
+      {
+        "id": "Battle_25",
+        "platform": "in_game",
+        "status": "in_progress",
+        "progress": 20,
+        "completed": false,
+        "completed_at": null,
+        "name": "Devil Within II",
+        "description": "Trigger Hexed ×50.",
+        "quality": "high",
+        "target": 50,
+        "rewards": [
+          {"item_id": "Annulith", "amount": 10}
+        ]
+      }
+    ]
+  }
+}
+```
+
+Achievement fields:
+
+- `scan.in_game` and `scan.playstation` separate records visible in game from
+  PlayStation trophy records and report completed/in-progress totals.
+- `categories` groups records by the prefix of their captured ID. The category
+  is therefore not repeated on every record.
+- `status` is `completed` when completion ticks are present and `in_progress`
+  otherwise. An observed compound achievement can be in progress while its
+  numeric `progress` is zero because individual checklist state is server-side.
+- `completed_at` is a decoded UTC time for completed records and `null` for
+  in-progress records.
+- `name`, `description`, `quality`, `target`, and `rewards` are optional display
+  metadata from `mappings/achievements.json`.
+- A newly captured ID missing from the bundled mapping still exports its core
+  captured fields; only optional display metadata is omitted.
+- Asset definitions absent from the network response are not exported because
+  absence cannot distinguish unstarted, locked, or unavailable content.
+
 ## CSV diagnostics
 
 CSV exports are mainly for debugging the decoder. Tools should prefer JSON

docs/limitations.md

@@ -21,5 +21,13 @@ Known limitations:
 - The file adapter reads mitmproxy `.flows` captures for research and testing.
 - Npcap is Windows-only and is not redistributed with this project; Linux and macOS use their system libpcap.
 - Reward keys decode to their reward id string, so unknown rewards still export a usable `reward_id`; display names/ranks come from the mapping JSON files and should be expanded as new rewards appear.
+- Achievement exports contain only records observed in the login response.
+  Asset-only definitions are not synthesized because absence cannot distinguish
+  unstarted, locked, or unavailable content. Unknown captured achievement IDs
+  still export without optional display metadata.
+- Observed achievements have only `completed` and `in_progress` states. Some
+  compound achievements expose numeric progress `0` while the server tracks
+  hidden checklist progress, so every observed record without completion ticks
+  is classified as `in_progress`.
 
 Privacy guardrail: raw packet data must not be included in sanitized exports.

docs/mapping-updates.md

@@ -1,4 +1,4 @@
-# Reward mapping updates
+# Mapping updates
 
 `tools/update_mappings.py` rebuilds `arcs.json`, `characters.json`,
 `items.json`, and `achievements.json` directly from the current
@@ -19,7 +19,7 @@ game whose asset tables evolve over time.
 
 ### GitHub Actions
 
-Run **Update reward mappings** from the repository's Actions tab. The optional
+Run **Update mappings** from the repository's Actions tab. The optional
 `source_ref` input accepts an NTE_Assets branch, tag, or commit and defaults to
 `main`.
 
@@ -68,7 +68,7 @@ python -m pytest -q
 ```
 
 For CI, `--check` exits with status 1 whenever the generated snapshot differs
-from the committed reward mappings. It still writes the staged artifacts.
+from the committed mappings. It still writes the staged artifacts.
 
 ## Source rules
 
@@ -100,7 +100,7 @@ reads or writes the permanent, limited, beginner, or Arc pool mapping files.
 Banner IDs, timestamps, record ordering, format version, and all UID inputs are
 therefore unchanged by mapping synchronization.
 
-Reward IDs and display metadata may change or disappear when the authoritative
+Reward and achievement IDs or display metadata may change or disappear when
-NTE_Assets snapshot changes. Existing exported history files remain unchanged;
+the authoritative NTE_Assets snapshot changes. Existing exports remain
-new exports describe rewards using the current asset snapshot. UID stability is
+unchanged; new exports use the current asset snapshot. UID stability is
-independent of reward display mappings.
+independent of display mappings.

docs/packet-format.md

@@ -1,6 +1,21 @@
 # Packet Format Notes
 
-This prototype supports separate Monopoly, Arc/Gashapon, and Mystery Box history decoders.
+This prototype supports separate Monopoly, Arc/Gashapon, Mystery Box history,
+and achievement decoders.
+
+## Achievements
+
+The account's achievement component arrives in an inbound TCP game-data block
+during login. The block is LZ4-decoded and contains `AchievementRecord` entries.
+Each record provides an achievement ID, an unsigned numeric progress value, and
+a .NET completion timestamp. A non-zero timestamp marks a completed record.
+Every observed record without completion ticks is treated as in progress;
+compound achievements can have server-side checklist progress while exposing a
+numeric value of zero.
+
+Only records present in the network response are exported. Display metadata is
+looked up case-insensitively in `mappings/achievements.json`, because captured
+and asset-table ID casing can differ. Missing metadata never prevents export.
 
 ## Decoder strategy
 

pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "nte-history-exporter"
-version = "0.2.1"
+version = "0.3.0"
-description = "Cross-platform Neverness to Everness pull-history exporter."
+description = "Cross-platform Neverness to Everness pull-history and achievement exporter."
 requires-python = ">=3.10"
 dependencies = []
 

src/nte_history_exporter/__init__.py

@@ -1 +1 @@
-__version__ = "0.2.1"
+__version__ = "0.3.0"

src/nte_history_exporter/console.py

@@ -94,7 +94,7 @@ def print_live_instructions(local_ip: str, backend: str = "windows_raw", detail:
     print(style("    You can capture more than one board in the same session.", DIM))
     print()
     print(style("  Achievements", BOLD))
-    print("    Your completed achievements are captured automatically when")
+    print("    Your tracked achievements are captured automatically when")
     print("    you log in after following the step above.")
     print()
     print(style("  Ready - use the game, then press any key here when finished.", BOLD, GREEN))
@@ -106,12 +106,26 @@ def print_page_captured(label: str, page: int | None, *, recaptured: bool = Fals
     print(style("  + ", GREEN, BOLD) + label + style(f"  {action} {page}", DIM))
 
 
-def print_achievements_captured(total: int, playstation: int) -> None:
+def print_achievements_captured(
-    visible = total - playstation
+    in_game_completed: int,
+    in_game_in_progress: int,
+    playstation_completed: int,
+    playstation_in_progress: int,
+) -> None:
+    print(style("  + ", GREEN, BOLD) + "Achievements captured")
     print(
-        style("  + ", GREEN, BOLD)
+        style(
-        + "Achievements"
+            f"      In-game      {in_game_completed} completed, "
-        + style(f"  captured {visible} visible + {playstation} PlayStation", DIM)
+            f"{in_game_in_progress} in progress",
+            DIM,
+        )
+    )
+    print(
+        style(
+            f"      PlayStation  {playstation_completed} completed, "
+            f"{playstation_in_progress} in progress",
+            DIM,
+        )
     )
 
 
@@ -184,7 +198,7 @@ def print_problem(text: str) -> None:
 def print_update_available(current_version: str, latest_version: str, release_url: str) -> None:
     print()
     print(style(f"  Update available: v{current_version} -> {latest_version}", YELLOW, BOLD))
-    print(style("  Some banner/reward mappings may be incomplete in this version.", YELLOW))
+    print(style("  Some reward or achievement mappings may be incomplete in this version.", YELLOW))
     print(style("  Updating is recommended for the most accurate export labels.", YELLOW))
     print(style(f"  Download when ready: {release_url}", YELLOW))
 

src/nte_history_exporter/constants.py

@@ -13,7 +13,7 @@ ARC_BANNER_NAME = "Arc Miracle Box"
 MYSTERY_BOX_BANNER_ID = "Gashapon_MysteryBox"
 MYSTERY_BOX_BANNER_NAME = "Mystery Box"
 EXPORTER_NAME = "nte-history-exporter"
-EXPORTER_VERSION = "0.2.1"
+EXPORTER_VERSION = "0.3.0"
 
 HISTORY_REQUEST_BANNER = 4220
 HISTORY_REQUEST_LENGTH = 45

src/nte_history_exporter/decoder/achievement.py

@@ -27,7 +27,10 @@ class AchievementRecord:
     def status(self) -> str:
         if self.completed:
             return "completed"
-        return "in_progress" if self.progress else "not_started"
+        # Presence in the server response means the achievement has tracked
+        # state. Compound achievements can have hidden checklist progress while
+        # exposing a numeric progress value of zero to the client.
+        return "in_progress"
 
     @property
     def completed_at(self) -> str | None:

src/nte_history_exporter/export/json_export.py

@@ -94,13 +94,11 @@ def build_achievement_export_json(
             "total_achievements": 0,
             "completed_achievements": 0,
             "in_progress_achievements": 0,
-            "not_started_achievements": 0,
         },
         "playstation": {
             "total_achievements": 0,
             "completed_achievements": 0,
             "in_progress_achievements": 0,
-            "not_started_achievements": 0,
         },
     }
     for achievement in achievement_records:
@@ -110,10 +108,8 @@ def build_achievement_export_json(
         summary["total_achievements"] += 1
         if achievement.completed:
             summary["completed_achievements"] += 1
-        elif achievement.progress:
-            summary["in_progress_achievements"] += 1
         else:
-            summary["not_started_achievements"] += 1
+            summary["in_progress_achievements"] += 1
         category, separator, _number = achievement_id.partition("_")
         category_key = category.casefold() if separator else "uncategorized"
         record = {

src/nte_history_exporter/live_capture/runner.py

@@ -111,12 +111,27 @@ def run_live_capture(
                             reassemble_tcp_segments(tcp_segments[flow])
                         )
                         if achievement_records:
-                            completed = [record for record in achievement_records if record.completed]
+                            in_game = [
+                                record
+                                for record in achievement_records
+                                if not record.achievement_id.casefold().startswith(
+                                    "playstation_"
+                                )
+                            ]
+                            playstation = [
+                                record
+                                for record in achievement_records
+                                if record.achievement_id.casefold().startswith(
+                                    "playstation_"
+                                )
+                            ]
                             console.print_achievements_captured(
-                                len(completed),
+                                sum(record.completed for record in in_game),
+                                sum(record.status == "in_progress" for record in in_game),
+                                sum(record.completed for record in playstation),
                                 sum(
-                                    record.achievement_id.lower().startswith("playstation_")
+                                    record.status == "in_progress"
-                                    for record in completed
+                                    for record in playstation
                                 ),
                             )
                 pair_count_before = len(session.pairs)

tests/test_achievement.py

@@ -28,7 +28,7 @@ class AchievementPacketTests(unittest.TestCase):
         self.assertEqual(records[0].achievement_id, "Battle_30")
         self.assertTrue(records[0].completed)
         self.assertEqual(records[1].achievement_id, "Playstation_035")
-        self.assertEqual(records[1].status, "not_started")
+        self.assertEqual(records[1].status, "in_progress")
 
     def test_reassembles_tcp_data(self):
         self.assertEqual(

tests/test_export_contract.py

@@ -1,11 +1,16 @@
 from tests.support import *  # noqa: F401,F403
 from nte_history_exporter import __version__
+from nte_history_exporter.constants import EXPORTER_VERSION
 from nte_history_exporter.decoder.achievement import AchievementRecord
 from nte_history_exporter.export.json_export import build_achievement_export_json
 from nte_history_exporter.live_capture.runner import _achievement_path
 
 
 class ExportContractTests(unittest.TestCase):
+    def test_public_version_references_match(self):
+        self.assertEqual(__version__, "0.3.0")
+        self.assertEqual(EXPORTER_VERSION, __version__)
+
     def test_sanitized_export_omits_raw_packet_fields(self):
         annotated = annotate_groups(fixture_session().build_rows("permanent"))
         export = build_export_json(annotated, [])
@@ -153,13 +158,11 @@ class ExportContractTests(unittest.TestCase):
                     "total_achievements": 2,
                     "completed_achievements": 1,
                     "in_progress_achievements": 1,
-                    "not_started_achievements": 0,
                 },
                 "playstation": {
                     "total_achievements": 1,
                     "completed_achievements": 0,
-                    "in_progress_achievements": 0,
+                    "in_progress_achievements": 1,
-                    "not_started_achievements": 1,
                 },
             },
         )