Pristine post-embark state
Each cohort binds one admitted source with its digest. The configured 2 model × 2 harness, one-repeat integration matrix remains local and private; no LLM cell outcome or ranking is published here.
first-fortress-v1 · contract 1
Model × harness comparison
A broad after-embark LLM × harness benchmark: establish a functioning early fortress while the model chooses strategy through bounded, player-equivalent observations and actions.
Rows appear only from sanitized matrix evidence. Observation, labor, replay, and cleanup have native smoke evidence; comparison runtime availability is reported only by a sanitized artifact and is unverified while none is published. This page cannot start a run or control a game.
Runtime · unverifiedPaired cohorts onlyNo cross-cohort ranking
ObjectiveEarly fortressBroad strategy after embark
Matrix rows0 publishedNever synthesized from prior stages
ValidityReplay firstObjective fail ≠ invalid evidence
PublicationSanitized JSONStatic and read-only
Controlled protocol
Comparable only inside a cohort.
A cohort fingerprint binds the scenario and source identity, observation and action contracts, and every budget. Model and harness identity remain visible on each repeated cell.
Fixed identities + budgets
Provider, model, harness and revisions; requests, actions, total and per-request output tokens, wall time, timeout, and request bytes are explicit. Per-turn hashes bind the runtime request, model-visible request, and prompt.
Deterministic pairing
Configured repetitions are all counted. Missing, blocked, invalid, and attempted infrastructure failures are retained rather than cherry-picked.
Dimensions before comparison
Survival, food and drink, shelter, excavation, stockpiles, production, costs, and stability are shown with measured usage. One repetition is integration evidence, not a stable ranking.
Harness boundary. minimal-observe-act retains no prior request/response pairs. history-aware retains exactly the four most recent complete model-visible request/assistant-response pairs, discarding older pairs without summaries or repair. Both current requests contain only the current observation and latest tool-feedback event. The harness configuration digest binds that history count. This is a bounded functional comparator, not a guarantee that an arbitrary provider context window can fit the request.
Public rubric source. The objective draws operational inspiration from the Dwarf Fortress Wiki v0.47.05 quickstart guide. It is public tutorial context, not a hidden holdout or evidence that an episode passed.
Reference-policy capability proof
Native reference run
An accepted hand-authored reference policy completed 55 applied actions across 56 replay-verified native frames. These pictures establish capability on the admitted source; they are not an LLM matrix row, model result, or ranking.
Not an LLM matrix row
display palette — original colors not recorded
Pictures are derived only from validated native frames in accepted trajectory 315e365d6ed02a1423bcc02fa75a1a4c1b20deeea94aca007b3514903b6387b3.
Trajectory 315e365d6ed02a1423bcc02fa75a1a4c1b20deeea94aca007b3514903b6387b3
Trajectory 315e365d6ed02a1423bcc02fa75a1a4c1b20deeea94aca007b3514903b6387b3
Trajectory 315e365d6ed02a1423bcc02fa75a1a4c1b20deeea94aca007b3514903b6387b3Full sanitized progression
Inspect the accepted reference replay
Loading the replay-verified reference projection…
Sanitized evidence
Published matrix
Checking for a published model-harness-matrix-v1 artifact…
Valid pass
Strict replay verified and the objective passed.
Valid fail
Strict replay verified and the objective failed. It counts in its cohort.
Invalid
Integrity, identity, budget, or accounting invalidity; not an objective failure.
Blocked / failed
A prerequisite blocked the attempt, or runtime/inference failed. Neither is a replay-valid comparison row.
Replay viewer boundary. Replay-valid rows include bounded 80 × 25 native-text frames, deterministic display-paletted ASCII, and up to three unique initial/middle/final run pictures derived only from those validated frames. A one-frame run shows one picture. Original colors were not recorded, so the palette is presentation—not recovered game evidence. The exact plain text remains beside the selected picture with previous and next inspection. Full offline replay uses the private trajectory; this viewer has no network access or live game controls.
No comparative evidence
No sanitized matrix artifact is published.
The benchmark is accepted, but there are no episodes to rank or summarize. Stage 1 and Stage 2 evidence remains separate and is not promoted into comparison rows.
Inspect a local sanitized export
Choose a model-harness-matrix-v1 JSON file. It stays in this browser; nothing is uploaded or executed.
No local file selected.
Interpretation boundary
Terrain proof is not fortress performance.
The earlier terrain survey establishes terrain conformance. Stage 1 founding stability and Stage 2 farming/workshop evidence retain their original prepared identities and replay claims. None is a fresh-embark first-fortress-v1 episode or evidence of generalization.