Skip to content

Test Template — the tests every surface must have

Companion to TESTING-CONSTITUTION.md. The constitution says what the loop is (spec → approve → test → code); this is the checklist so you never have to remember which tests to write. Copy the “Per-surface matrix” into a spec’s Test matrix section, then create the files. The check_specs_have_tests.py guardrail keeps the spec and its tests bound together.

When you start a surface (a view, a feature, an endpoint — e.g. the Library icon view, the KG tables), you owe tests across the legs the surface touches. Not every surface touches every leg; the matrix says which, and each leg below is a fill-in-the-blank skeleton grounded in the real harness so writing it is mechanical.

Per-surface matrix (copy into the spec’s Test matrix section)

Leg Required when… Pins Lives in
Pure rule (Swift) there is any non-trivial logic (a filter, a mapping, a state machine) the rule, off-main, no UI fichero/Tests/Unit/**/<Area>Tests.swift
Availability (Swift) a capability must be REACHABLE in a surface “this surface wires this capability” same, source-string via AppSource.root()
Visual / A11y (Apple tools) a surface has UI to verify performAccessibilityAudit() passes + .xctestplan captures a screenshot (no pixel-diff) in the Click-around leg below (fichero/Tests/UI/**)
Backend (pytest) the surface calls an endpoint / writes data the endpoint’s contract + that data was DELIVERED fichero-server/tests/**
MCP the capability should be agent-reachable the MCP tool maps + routes fichero-mcp/tests/test_mcp_full.py
CLI the capability should be scriptable the CLI command wires the endpoint fichero-cli/tests/test_*.py
Click-around (XCUITest, Mac) a user drives it in the UI the end-to-end path: click → effect fichero/Tests/UI/** (subclass FicheroUISessionTests)
iPhone (iOS) the surface ships on iPhone the touch path exists on iPhone fichero/Tests/UI/ios + the fichero-ui-ios plan
iPad the surface ships on iPad the touch path exists on iPad (the split/regular-width IA) fichero/Tests/UI/ipad + the fichero-ui-ipad plan
Load (#4634) the surface acts on many rows bounded, no peg, timed fichero-server/tests/perf/**

Hard-gate (Testing Constitution Art. 4): the cross-surface invariant (same result from backend, MCP, CLI, UX) and capability availability are hard-gates. The rest is tracked debt — but the matrix is how you SEE the debt instead of forgetting it.

Skeletons

1. Pure rule (Swift, off-main)

@testable import Fichero
import XCTest
/// spec: <area> — `<behavior.id>`.
final class <Area>RuleTests: XCTestCase {
    func testRule() {
        XCTAssertEqual(<Type>.<pureFunc>(<input>), <expected>)
    }
}

2. Availability (Swift — “the capability is present in this surface”)

private static func appSource(_ p: String) throws -> String {
    try String(contentsOf: AppSource.root().appendingPathComponent(p), encoding: .utf8)
}
func testSurfaceWiresCapability() throws {
    let src = try Self.appSource("Views/<Path>/<View>.swift")
    XCTAssertTrue(src.contains("<the wiring token>"), "<surface> must wire <capability>")
}

2b. Visual + accessibility (Apple tools — no pixel-diff)

There is no pixel-diff snapshot layer — Apple ships no snapshot assertion and we don’t hand-roll one (the old SnapshotSupport.swift was deleted; see specs/testing/ui-testing-strategy.md). Visual/a11y verification uses Apple’s own tools, inside the XCUITest leg (§ Click-around): - Accessibility correctness: each XCUITest flow ends with try app.performAccessibilityAudit() — Apple-first-party, catches missing labels/identifiers + contrast on every destination. - Doc screenshots: the .xctestplan captures screenshots per run (Apple’s “keep all”), and Xcode RenderPreview captures a surface on demand. Capture (for review-by-eye + the guides), NOT auto-diff. Copy the capture into docs/assets/<milestone>/ to satisfy the Documentation matrix’s user-manual screenshot — no separate screenshot step. See DOC-TEMPLATE.md → user-manual leg.

3. Backend (pytest — prove DELIVERY, not just a 200; see check_import_tests_prove_delivery)

def test_<verb>_persists(client, db):
    r = client.post("/api/<route>", json={...})
    assert r.status_code == 200
    assert r.json()["id"]                 # evidence something was delivered
    assert db.get(<Model>, r.json()["id"]) is not None

4. MCP (mirror test_mcp_full.py: a fake client records the call)

def test_<tool>_routes(monkeypatch):
    fake = FakeClient(); ...
    mcp_full.<tool>(<Input>(...))
    assert fake.calls[-1][0] == "<client_method>"

5. CLI (typer runner; assert it dials the right endpoint)

def test_<command>_wires_endpoint(monkeypatch):
    result = runner.invoke(cli_main.app, ["<group>", "<command>", ...])
    assert result.exit_code == 0
    assert recorded_path == "/api/<route>"

6. Click-around (XCUITest — the weak leg; this is the whole point of the template)

Subclass FicheroUISessionTests — it gives you a launched app over a seeded library (auto-skips when no venv/seeder). Drive by accessibility identifiers, assert the effect. 

@MainActor
final class <Area>FlowUITests: FicheroUISessionTests {
    func testUserDoesThingAndSeesResult() {
        waitForLibraryReady()
        // 1. reach the surface (a mode button / sidebar row by its a11y id)
        app.buttons["<surface.entry.id>"].firstMatch.tap()
        // 2. act (select a seeded row, invoke the verb)
        let row = app.descendants(matching: .any)
            .matching(identifier: "<row.id.for(seeded.ids[...])>").firstMatch
        XCTAssertTrue(row.waitForExistence(timeout: readyTimeout))
        row.rightClick(); app.menuItems["<verb.id>"].tap()
        // 3. assert the effect (the row is gone / a value changed)
        XCTAssertFalse(row.waitForExistence(timeout: 5))
    }
}
Prerequisite for this leg (per surface): the views need stable accessibility identifiers on their key controls and rows (e.g. .accessibilityIdentifier("kg.entity.row.\(id)"), "kg.menu.delete"). No a11y ids ⇒ the click-around test can’t find anything. Adding the ids is part of building the surface, not an afterthought — list them in the spec.

7. iPad/iOS

Same as the click-around skeleton, but the test targets the fichero-ui-ipad / fichero-ui-ios plans; keep it to the touch path (tap, not right-click) and the reduced first-run IA.

8. Load (#4634)

def test_<verb>_1k_is_bounded(client, benchmark):
    ids = seed_many(1000)
    # act on 1000; assert it completes within budget and concurrency stays bounded

The process — spec-lead development, every surface, every time

Everything is done systematically, the same way each time — spec-lead development. Run the design half in plan mode with a ponytail lens (research → plan → approve before code; the laziest design that meets the intent). Each step has a guardrail gate, so the process is enforced, not remembered. The loop produces four bound artifacts for one <name>: the spec, the GitHub milestone, the tests (tagged), and the docs (contributor AND user).

  1. Scaffold + survey prior art. cp docs/contributor_manual/specs/_TEMPLATE.md docs/contributor_manual/specs/<surface>.md. Fill Intent, then the Prior art / best practices section — survey how the field already solves this (digital humanities standards, NLP/NLG pipelines, Hugging Face, relevant libraries) and cite what we adopt/reuse rather than invent. · Gate: _-prefixed scaffolds are skipped; real specs are tracked.
  2. Behaviors. Write one line per behavior with a stable id (<surface>.<behavior>) and a tag ([OK]/[MISSING]/[PARTIAL]). These ids are what tests cite.
  3. Approve. The creative director approves the intent; flip Status: DRAFT → APPROVED. · Gate: an APPROVED spec MUST be cited by ≥1 test, carry a Test-matrix section (check_specs_have_tests.py, Rules B + C), and declare Milestone: <name> matching a GitHub milestone of the same name (check_spec_milestones.py). On approval, create or rename the milestone to the spec name and point its description back at the spec — the link is bidirectional (spec name == milestone name == test tag). Tag the tests to the same area name: Swift @Tag (TestTags.swift), pytest markers (fichero-server/pyproject.toml).
  4. Fill the matrix. Tick the legs this surface touches; list the accessibility identifiers the click-around leg needs (add them to the views as you build).
  5. Test-first, per leg. One file per ticked leg, from the skeletons above; the docstring cites the behavior id. Hard-gate legs (cross-surface invariant + availability) first.
  6. Reuse, don’t duplicate — THEN implement. BEFORE writing, search the codebase for an existing path (jCodemunch search_symbols / find_references / find_similar_symbols) — we too often grow a SECOND implementation of something that already exists. If a path exists, EXTEND it; never add a parallel one (one audited action layer, one endpoint per capability). After implementing, confirm no duplicate/parallel path was introduced. Then make the tests pass; add the a11y ids alongside the views.
  7. Verify. python scripts/check_specs_have_tests.py + the full gate (verify_all.sh runs every check_*.py, including this one).
  8. Document each audience — by its owner. Fill the spec’s Documentation matrix. The agent authors what it owns: the contributor docs (docs/contributor_manual/… + this spec), the reference (docs/reference_manual/…), and the MCP tool description / CLI --help where the feature is agent- or script-reachable. The user manual (docs/user_manual/…) is the maintainer’s own — authored in Tinderbox, exported to the GitHub folder; the agent does NOT write it, but DOES produce the accurate raw material the maintainer needs (the .xctestplan/RenderPreview screenshot capture doubles as the doc screenshot, and the behavior list/a11y ids are current). A feature isn’t done until the reader who never opens the code can use it — and the facts that reader will rely on are pinned by tests, not by prose that can drift.

Weakest leg today is click-around — treat its skeleton as non-optional for any surface a user touches. A surface without a click-around test is not “done”, it’s “unproven in the one way the user actually experiences it”.

Worked reference

kg-interactions.md and kg-tables.md carry filled-in Test matrices. Use them as examples when you spec the Library icon view (or any new surface).