Final Non-User System Verification Bank¶
Owner issue: #870 — establish final non-user verification bank and execution matrix Milestone: Milestone 4 — Submission Purpose: technical/system verification of the final release candidate Initial state: every case is
NOT RUNuntil evidence is produced
1. Purpose and boundary¶
This page is the authoritative final non-user technical verification bank for the Sport Analytics Tool.
It is deliberately separate from formal user testing. The existing
User Testing Task Bank is reused only as a coverage map because its
AUTH, PUB, SUB, BAT, COR, REV, ADM, DATA and API families already describe the
project's major journeys. Technical results recorded here do not create participant evidence,
usability findings or formal user-feedback outcomes.
Each verification case defines the capability to test, prerequisites, method, expected result and evidence requirement before execution. The final execution issues update the status/evidence fields as the release candidate is tested.
2. Execution ownership¶
| Execution issue | Technical-verification scope | Primary case families |
|---|---|---|
| #871 | Frontend, authentication and roles | AUTH-*, PUB-* |
| #872 | Submission, review, batch ingestion and corrections | SUB-*, REV-*, BAT-*, COR-*, selected ADM-* |
| #873 | Statistics, aggregates, provenance and dataset releases | STAT-*, DATA-*, selected COR-*, ADM-* |
| #874 | External API, contracts, consumer controls and integrations | API-*, INT-*, selected SEC-* |
| #875 | Database, worker and asynchronous reliability | DB-*, WRK-* |
| #876 | Performance, load, accessibility and responsiveness | PERF-*, A11Y-*, RESP-* |
| #877 | Automated suites, coverage, CI/CD and quality gates | AUTO-*, COV-*, CI-*, DEP-* |
The bank must exist before #871–#877 execute. #810 owns final production deployment and release close-out after the execution lanes are complete.
3. Result vocabulary and evidence rule¶
Use only these result states:
| State | Meaning |
|---|---|
NOT RUN |
Final release-candidate verification has not yet been executed. |
PASS |
The expected result was observed and evidence is linked. |
FAIL |
Incorrect behaviour was observed. Link a bug/issue. |
BLOCKED |
The check could not execute. Record the concrete blocker. |
N/A |
The capability is demonstrably outside the final implemented scope. Record the reason. |
A case may not be marked PASS from memory, an old Sprint result or the existence of an automated test.
Evidence must identify the final candidate or the exact run used for the conclusion.
For every executed case retain, where relevant:
- date and tester;
- commit/tag/release candidate;
- environment and deployed URL;
- role/account type, without credentials;
- fixture/package/dataset used;
- command/test suite or deterministic steps;
- observed result;
- evidence link;
- linked defect;
- retest result.
Use evidence/validation/final-system-verification/execution-record-template.md for retained execution
records.
4. Mapping from the formal user-testing bank¶
This mapping provides consistent product coverage only.
| Existing user-testing family | Technical use in this bank |
|---|---|
AUTH-* |
Authentication, account state, sessions and access-control boundaries |
PUB-* |
Public navigation, fixture/statistics discovery, filters, export and comparison |
SUB-* |
Submitter access, direct/guided submission and validation |
BAT-* |
Season/back-catalogue ingestion, progress, recovery and reporting |
COR-* |
Correction, stable identity, provenance and recomputation |
REV-* |
Review queue, reference resolution, approval/rejection and new-fixture onboarding |
ADM-* |
Access administration and privileged provenance/audit operations |
DATA-* |
Versioned dataset releases and reproducibility |
API-* |
Consumer API, aggregate queries, versioning/deprecation and usage controls |
The formal user-testing task bank remains authoritative for participant-facing goals and outcomes.
5. Verification bank¶
A. Authentication, accounts, roles and frontend flow — #871¶
AUTH-TECH-01 — Public access without authentication¶
Maps from: AUTH-04, PUB-*
Preconditions
- Final frontend and API deployment are reachable.
- Browser starts without an authenticated application session.
Method
- Open the final public application in a clean/private browser context.
- Navigate through the public home/data/statistics/API-discovery surfaces.
- Inspect representative network calls and direct/deep-linked public routes.
Expected
- Public data remains available without login.
- Private controls are not exposed as usable authenticated actions.
- Public API reads used by these pages do not unexpectedly require an authenticated session.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
AUTH-TECH-02 — Sign-in completion¶
Maps from: AUTH-01
Preconditions
- Approved project test identity exists.
- No credentials are committed or retained in evidence.
Method
- Start from the public application.
- Use the configured authentication flow.
- Return to the application after provider completion.
- Verify the application and backend recognise the authenticated account.
Expected
- Authentication completes successfully.
- Account/session state is consistent between frontend and protected backend requests.
- Authentication cancellation/failure does not create a false signed-in state.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
AUTH-TECH-03 — Sign-out clears private session state¶
Maps from: AUTH-04
Method
- Sign in with a test account.
- Visit at least one authenticated route.
- Sign out.
- Refresh the previous authenticated route and retry a protected API action.
Expected
- Private session state is cleared.
- Protected actions are rejected after sign-out.
- Public browsing continues normally.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
AUTH-TECH-04 — Session refresh/reopen behaviour¶
Maps from: AUTH-01
Method
- Sign in.
- Refresh an authenticated route.
- Reopen the application in the supported session window.
- Observe any token/session refresh requests.
Expected
- Valid supported sessions survive refresh/reopen without corrupting account state.
- Expired/invalid state is handled safely rather than leaving stale privileged controls.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
AUTH-TECH-05 — Role and permission boundary¶
Maps from: AUTH-02, SUB-01, REV-*, ADM-*
Preconditions
- Test identities for relevant final roles are available.
Method
For public/viewer, submitter, reviewer and administrator roles that exist in the final product:
- Attempt the routes/actions the role should be allowed to use.
- Attempt at least one representative privileged action the role must not use.
- Verify the backend response as well as the UI.
Expected
- Each role can perform only its authorised actions.
- Server-side authorisation rejects forbidden operations even if a route/request is attempted directly.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
AUTH-TECH-06 — Account deletion boundary¶
Maps from: AUTH-03
Preconditions
- Disposable account if destructive execution is required.
Method
- Locate the account-deletion path.
- Verify the confirmation boundary.
- Where a disposable account is approved, complete deletion and retry authentication/protected access.
Expected
- Destructive action requires explicit confirmation.
- Completed deletion removes/invalidates the account according to documented provider/application ownership.
- Evidence contains no credentials or tokens.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
AUTH-TECH-07 — Password recovery ownership/path¶
Maps from: account-lifecycle requirement; no direct user-task-bank equivalent
Method
- Use the documented password/authentication recovery path for the configured provider.
- Verify the application directs recovery to the correct provider-owned flow.
- Confirm no custom secret-reset mechanism bypasses the configured authentication provider.
Expected
- Recovery behaviour matches final security/authentication documentation.
- No application secret is exposed.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
PUB-TECH-01 — Public fixture discovery and detail¶
Maps from: PUB-01
Method
- Start from the public application.
- Find a representative fixture.
- Open its available details and data.
Expected
- Fixture discovery and detail routes return valid data.
- No broken route, uncaught frontend error or incorrect empty state occurs.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
PUB-TECH-02 — Public statistics discovery¶
Maps from: PUB-02
Method
- Open a representative fixture.
- Navigate to its available statistics.
- Compare displayed values with the corresponding API response/reference used by #873.
Expected
- Statistics routes render successfully.
- Displayed values correspond to the selected fixture/scope.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
PUB-TECH-03 — Filter/narrowing behaviour¶
Maps from: PUB-03
Method
- Apply a representative public filter/narrowing control.
- Record request parameters and returned/displayed scope.
- Clear/change the filter.
Expected
- Selected scope is reflected consistently in UI and API result.
- Clearing/changing the filter does not retain stale data.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
PUB-TECH-04 — Export behaviour¶
Maps from: PUB-04
Method
- Select a representative supported export scope.
- Request available CSV/JSON export(s).
- Open/parse the resulting file and compare scope/identifiers with the request.
Expected
- Export downloads successfully.
- File format/content matches the documented selected scope.
- Export does not leak privileged fields.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
PUB-TECH-05 — API Explorer discovery and deep link¶
Maps from: PUB-05
Method
- Reach the API Explorer from public navigation.
- Open
/apidirectly. - Refresh the route.
Expected
- Explorer is publicly discoverable and deep-linkable.
- Route refresh does not return an unrelated SPA/server error.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
PUB-TECH-06 — Meaningful comparison flow¶
Maps from: PUB-06
Method
- Select two supported comparison subjects/scopes.
- Record compared values and underlying API responses.
- Navigate between comparison subjects.
Expected
- Comparison uses compatible scope/units.
- Values do not silently mix fixture/season/career scopes.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
PUB-TECH-07 — Loading, empty and failure states¶
Maps from: PUB-*
Method
Exercise at least one deterministic case for:
- loading;
- a valid empty result;
- an API/network failure.
Expected
- The application shows an appropriate state/message.
- No unhandled crash or misleading stale data is shown.
Status: PASS
Evidence / defect / retest: #871 deterministic execution record; no product defect observed.
B. Submission, review, batch ingestion and correction — #872¶
SUB-TECH-01 — Submitter access and competition scope¶
Maps from: AUTH-02, SUB-01
Method
- Sign in with an approved submitter.
- Verify submission capability for an authorised competition.
- Attempt submission outside permitted competition scope.
Expected
- Authorised scope is accepted.
- Out-of-scope operation is rejected server-side.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
SUB-TECH-02 — Valid guided single-fixture package¶
Maps from: SUB-02
Preconditions
- Known-valid disposable/approved single-fixture package.
Method
- Upload through the guided single-fixture path.
- Record durable reference/receipt.
- Follow status until the intended next state.
Expected
- Package is safely received.
- Receipt/status identifies the submission.
- Processing may continue independently after upload.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
SUB-TECH-03 — Invalid guided package validation¶
Maps from: SUB-03, SUB-04
Preconditions
- Package containing known deliberate validation errors.
Method
- Upload invalid package.
- Record file/row/field error detail.
- Correct the known error and resubmit using approved disposable data.
Expected
- Invalid input is rejected without partial authoritative publication.
- Error identifies actionable source context.
- Corrected replacement can proceed.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
SUB-TECH-04 — Advanced canonical JSON valid path¶
Maps from: SUB-05
Method
- Select the advanced technical JSON mode.
- Submit a known-valid canonical event array to the intended fixture.
- Record accepted/conflict/rejected response.
Expected
- Valid schema is accepted according to documented direct-submission semantics.
- No unrelated competition/fixture data is changed.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
SUB-TECH-05 — Advanced JSON validation errors¶
Maps from: SUB-06
Method
Submit a canonical event array containing known cricket/business-rule errors.
Expected
- Rejection identifies the correct event/field/problem.
- Multiple independent errors remain distinguishable where supplied.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
SUB-TECH-06 — Genuinely new fixture proposal¶
Maps from: SUB-07
Preconditions
- Valid package for a fixture not already canonical and within allowed competition scope.
Method
- Submit/propose readable fixture metadata without internal IDs.
- Record staged/new-fixture state.
- Verify it awaits the correct validation/review path.
Expected
- New fixture is not rejected merely because it lacks a pre-existing fixture ID.
- Duplicate protection and review boundary remain active.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
REV-TECH-01 — Review queue exposes eligible staged work¶
Maps from: REV-01
Method
Create/use a staged batch expected to require review and inspect the reviewer queue.
Expected
- Eligible work appears with correct lifecycle/status information.
- Unauthorised roles cannot make the review decision.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
REV-TECH-02 — Reviewer can inspect validation/reference detail¶
Maps from: REV-02
Method
Open representative staged work containing accepted/rejected/unresolved content.
Expected
- Validation, source/provenance and reference information required for a decision is available.
- Blocking state prevents unsafe publication.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
REV-TECH-03 — Resolve an ambiguous reference¶
Maps from: REV-03
Method
Use a staged item with a controlled ambiguous/unresolved reference and save an approved mapping.
Expected
- Mapping is persisted.
- Eligibility/revalidation state updates consistently.
- Mapping does not alter unrelated references.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
REV-TECH-04 — Approve and publish¶
Maps from: REV-04
Method
Approve known-valid staged work and follow the durable publication path.
Expected
- Decision is persisted once.
- Publication proceeds through the documented backend/worker path.
- Public/canonical state eventually reflects the approved data.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
REV-TECH-05 — Return/reject unsafe work¶
Maps from: REV-05
Method
Use a scenario that must not publish as-is and exercise the appropriate return/reject action.
Expected
- Reason/decision is retained.
- Rejected/returned data does not publish.
- Resulting state is distinguishable from approval.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
REV-TECH-06 — Review/onboard a new fixture¶
Maps from: REV-06
Method
Review a staged genuinely-new-fixture proposal and perform the correct create/link/return action.
Expected
- Competition/date/participants/source data are validated.
- Existing duplicate fixture is not silently duplicated.
- Approved new fixture becomes safely canonical/linked before publication proceeds.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
BAT-TECH-01 — Whole-season package¶
Maps from: BAT-01
Method
Upload a representative season package using approved test/acceptance data and follow processing.
Expected
- Durable batch reference is returned.
- Package progresses through validation/staging/review states without silent loss.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
BAT-TECH-02 — Multi-season/back-catalogue package¶
Maps from: BAT-02
Method
Upload a representative package spanning more than one season using readable external references.
Expected
- Season/fixture ownership and references resolve correctly.
- Package does not require internal database IDs from the submitter.
- Distinct seasons remain distinguishable.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
BAT-TECH-03 — Batch status and item results¶
Maps from: BAT-03
Method
Open a batch in at least one in-progress and one terminal state.
Expected
- State, counts and accepted/rejected item information agree with backend/report data.
- Terminal/in-progress state is unambiguous.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
BAT-TECH-04 — Failed/correction-required recovery¶
Maps from: BAT-04
Method
Use a controlled batch with rejected data or correction requirement and follow the documented recovery path.
Expected
- Affected source item/error is identifiable.
- Corrected replacement/retry behaviour does not duplicate prior accepted work.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
BAT-TECH-05 — Complete batch report download¶
Maps from: BAT-05
Method
Download the complete report for a processed batch and compare it with the on-screen/backend result.
Expected
- Report is retrievable.
- Counts/items/status match authoritative batch state.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
BAT-TECH-06 — Idempotent retry/resubmission¶
Maps from: BAT-*
Method
Retry/redeliver the same durable work according to the supported retry path.
Expected
- Already accepted/published authoritative data is not double-counted.
- Duplicate/conflict result is explicit where applicable.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
COR-TECH-01 — Correct published event and preserve history¶
Maps from: COR-01
Method
Using approved disposable/correction data:
- identify a published event;
- submit an authorised correction;
- inspect canonical event and correction/audit history.
Expected
- Corrected value is authoritative.
- Prior state/history remains traceable.
- Stable logical identity/provenance remains coherent.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
COR-TECH-02 — Correction recomputes dependent statistics¶
Maps from: COR-01, selected PUB-*
Method
Record a known dependent statistic before correction, apply correction, then query it again.
Expected
- Affected statistic changes correctly.
- Unrelated representative statistics remain stable.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
ADM-TECH-01 — Submitter-access administration¶
Maps from: ADM-01
Method
Where the final feature is available, approve/reject a controlled submitter-access request and verify competition scope.
Expected
- Decision is persisted correctly.
- Resulting submitter permissions match the selected competition scope.
Status: PASS
Evidence / defect / retest: #872 execution record; accepted final verification outcome; no new product defect recorded.
C. Statistics, aggregates, provenance and dataset releases — #873¶
STAT-TECH-01 — Fixture/innings totals against known result¶
Maps from: PUB-01, PUB-02
Method
Choose at least one representative fixture with trusted reference totals and compare final API/UI output.
Expected
- Fixture/innings totals agree with the trusted reference after applying documented domain rules.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
STAT-TECH-02 — Batting calculations¶
Maps from: PUB-02, PUB-06
Method
For representative batters, independently derive/check exposed batting figures from the accepted event data/reference fixture.
Expected
- Exposed batting values match documented calculations and selected scope.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
STAT-TECH-03 — Bowling calculations¶
Maps from: PUB-02, PUB-06
Method
For representative bowlers, independently derive/check exposed bowling figures, including legal-delivery/extras handling.
Expected
- Values match documented cricket rules and selected scope.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
STAT-TECH-04 — Fielding/appearance calculations where exposed¶
Maps from: PUB-02
Method
Compare representative final exposed fielding/appearance values with source events/reference data.
Expected
- Implemented values are correct and scoped to the intended fixture/aggregate.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
STAT-TECH-05 — Cricket extras and edge cases¶
Maps from: PUB-*
Method
Run focused automated/reference checks for implemented edge cases such as wides/no-balls with zero-valued payload fields, repeated printed ball numbers and other documented Cricsheet/domain constraints.
Expected
- Derived statistics follow the project's documented event/domain interpretation.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
STAT-TECH-06 — Season/career/competition aggregates¶
Maps from: PUB-06, API-02
Method
Compare representative aggregate values with aggregation over the accepted underlying fixture data.
Expected
- Aggregate totals/derived values equal the underlying authoritative data and documented formulae.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
STAT-TECH-07 — Comparison/leaderboard ordering¶
Maps from: PUB-06
Method
Verify at least one implemented comparison/leaderboard with known values, including tie/scope behaviour where applicable.
Expected
- Ordering and units are consistent and reproducible.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
ADM-TECH-02 — Provenance from published data to submission¶
Maps from: ADM-02
Method
Select a published fixture/event/statistic and follow available provenance/audit information back to its source submission/batch.
Expected
- Source relationship is traceable without exposing secret credentials.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
DATA-TECH-01 — Find/retrieve a versioned dataset release¶
Maps from: DATA-01
Method
Identify a generated release and retrieve the downloadable artefact plus metadata.
Expected
- Release has stable version/identifier, documented scope/schema and retrievable artefact.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
DATA-TECH-02 — Release checksum and metadata integrity¶
Maps from: DATA-01
Method
Compute/verify the documented checksum of a retrieved release and compare metadata/schema/version.
Expected
- Checksum matches the released artefact.
- Metadata refers to the same release/scope.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
DATA-TECH-03 — Reproduce representative statistic from release¶
Maps from: DATA-02
Method
Using only the release plus linked schema/calculation documentation, reproduce one representative published statistic.
Expected
- Required source fields/rules are available and reproduced result matches the expected value.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
DATA-TECH-04 — Asynchronous release generation¶
Maps from: DATA-*
Method
Request a release-generation workflow and follow API/job/worker state.
Expected
- Request does not require one long blocking API process.
- Durable background work reaches a correct terminal state.
- Repeated delivery does not generate inconsistent duplicate authoritative releases.
Status: PASS
Evidence / defect / retest: #873 execution record; no new correctness failure observed.
D. External API, contracts, consumer controls and integrations — #874¶
API-TECH-01 — External API and OpenAPI availability¶
Maps from: PUB-05, API-*
Method
From outside the local development environment:
- request representative public API health/read endpoint;
- request
/openapi.yaml; - record status/content type.
Expected
- Deployed endpoints are externally reachable.
- OpenAPI document is served successfully and anonymously where documented.
Status: PASS
Evidence / defect / retest: #874 execution record; health 200 with API-Version: v1 and /openapi.yaml 200 as application/yaml, both anonymous; first request paid a 32.5 s scale-to-zero cold start.
API-TECH-02 — Public API Explorer against deployed contract¶
Maps from: PUB-05
Method
Open the final /api explorer and execute at least one implemented public operation.
Expected
- Explorer loads the deployed OpenAPI contract.
- Operation executes against the intended deployed backend.
- Planned/non-implemented operations are not misleadingly executable.
Status: PASS
Evidence / defect / retest: #874 execution record; the explorer loaded at sport-analytics-tool-web.pages.dev/api and issued GET /api/v1/competitions?limit=2 to the deployed host, returning 200 with two competitions and a nextCursor; performed in a browser by Ben Swartz on 2026-10-10 and recorded with its status, body, headers and duration (no screenshot retained).
API-TECH-03 — Contract matches implementation¶
Maps from: API-*
Method
Run the API contract suite and manually spot-check representative success/validation/auth/rate-limit responses against OpenAPI.
Expected
- Implemented method/path/status/shape agrees with the published contract.
Status: FAIL
Evidence / defect / retest: #874 execution record; six of seven representative deployed responses validated against the deployed specification; GET /api/v1/participants returns an undocumented totalRecords on every item (finding F1, not submission-blocking, code left unchanged).
API-TECH-04 — HTTP design and non-redundant endpoint behaviour¶
Maps from: API-*
Method
Review/exercise representative create/read/update-style operations and nested resources.
Expected
- HTTP methods/status codes match operation semantics.
- No final consumer workflow requires redundant duplicate endpoints for the same operation without documented reason.
Status: PASS
Evidence / defect / retest: #874 execution record; 404 NOT_FOUND, and 400 VALIDATION_FAILED with a field-naming details[] for a bad limit and for a missing required identifier; envelope matches the documented shape.
API-TECH-05 — Filtering, pagination and stable identifiers¶
Maps from: PUB-03, API-*
Method
Exercise representative list/filter/pagination operations across at least two pages/scopes.
Expected
- Filters are applied deterministically.
- Pagination does not silently duplicate/skip stable records under the tested stable dataset.
- Stable identifiers remain consistent between related responses.
Status: PASS
Evidence / defect / retest: #874 execution record; cursor pagination followed to a second page with no overlap, totalPages present where documented, and opaque immutable identifiers throughout.
API-TECH-06 — Consumer key authentication¶
Maps from: API-01
Method
Using a disposable/test key supplied outside Git:
- make a valid authenticated consumer request;
- repeat with missing/invalid key.
Expected
- Valid key is accepted where required.
- Missing/invalid credential is rejected distinctly.
- Secret key is absent from retained screenshots/logs.
Status: PASS
Evidence / defect / retest: #874 execution record; key lifecycle and the 401 on a missing, invalid or revoked key covered by test:api and test:api-contract; WWW-Authenticate: ApiKey observed live, and the anonymous allowance observed being metered live — see API-TECH-07.
API-TECH-07 — Quota/rate-limit enforcement¶
Maps from: API-01
Method
Exercise the documented test-safe quota/rate-limit boundary.
Expected
- Limit state is enforced.
- Response status/headers/body expose only documented safe guidance.
- Retry/reset state is coherent.
Status: PASS
Evidence / defect / retest: #874 execution record; per-minute limit, UTC daily quota, the header split and the fail-closed 503 asserted by the contract suite; the documented 30-per-source-per-minute anonymous allowance was additionally observed live during the manual step (ratelimit-limit: 30, ratelimit-remaining: 29, ratelimit-reset: 22), so the limiter is confirmed active in the deployed environment. Enforcement was not flooded against the live API by design.
API-TECH-08 — Aggregate API capability¶
Maps from: API-02
Method
Run a representative aggregate query and compare the result with #873's underlying accepted-data calculation.
Expected
- Aggregate response scope/units/values are correct.
Status: PASS
Evidence / defect / retest: #874 execution record; competition-scoped leaderboard returned ranked entries with tie-breakers and validated against the deployed schema; a missing scope identifier returned 400.
API-TECH-09 — Deprecation/versioning behaviour¶
Maps from: API-03
Method
For any final deprecated operation/version, inspect documentation/OpenAPI/response signalling and replacement guidance.
Expected
- Deprecated operation is clearly identified.
- Replacement/lifecycle guidance is accurate.
- If no operation is deprecated in the final release, mark
N/Awith reason rather than fabricating a deprecation.
Status: PASS
Evidence / defect / retest: #874 execution record; deprecation: ?1 and link: rel="successor-version" observed on the deprecated consumer alias, with no Sunset.
API-TECH-10 — Per-consumer usage visibility¶
Maps from: API-04
Method
Where implemented, use an authorised admin/reviewer path to inspect a test consumer's usage.
Expected
- Usage counts/time window/operations agree with controlled requests.
- Secret key material is not displayed.
Status: PASS
Evidence / defect / retest: #874 execution record; usage views covered by test:api; the live route refused an anonymous request with 401, and reading usage needs a key.
API-TECH-11 — Natural-language analytics query¶
Maps from: no formal task-bank equivalent
Method
- Run the repository natural-language evaluation command.
- Exercise representative deployed queries if included in the release candidate.
- Include casual/follow-up phrasing owned by #868.
Expected
- Implemented query classes meet their documented/evaluated behaviour.
- Unsupported/ambiguous input fails safely instead of fabricating a result.
- If the feature is not part of the final candidate, record
N/Awith reason.
Repository command: npm run evaluate:natural-language-queries
Status: PASS
Evidence / defect / retest: #874 execution record; no call made; cited the #940 70/70 evaluation run, the #940 live verification including a follow-up question, and the three #868 runs.
INT-TECH-01 — External weather integration success¶
Method
Exercise the documented weather/external integration through the final deployed system.
Expected
- External result is integrated into the intended application/API behaviour.
- Application does not expose provider credentials.
Status: PASS
Evidence / defect / retest: #874 execution record; deployed weather read returned live Open-Meteo values for a real coordinate and date.
INT-TECH-02 — External integration failure handling¶
Method
Use a deterministic test/mocked/failure-safe path to exercise provider failure/timeout handling.
Expected
- Failure is handled predictably.
- Core application does not crash.
- Secret/provider internals are not returned to public consumers.
Status: PASS
Evidence / defect / retest: #874 execution record; out-of-range coordinates returned 400 with a safe message and no provider detail; timeout and upstream-error paths covered by test:api with a stubbed provider.
SEC-TECH-01 — Public-response secret/privilege leakage check¶
Method
Inspect representative public API/application responses and built frontend assets for accidental secret/private-field exposure.
Expected
- Server credentials, service credentials and secret consumer keys are absent.
- Privileged-only data is not exposed through public endpoints.
Status: PASS
Evidence / defect / retest: #874 execution record; eight public responses scanned for private-looking keys and address-shaped strings with zero hits; five protected routes each returned 401 with a non-disclosing message.
E. Database, worker and asynchronous reliability — #875¶
DB-TECH-01 — Deployed database connectivity and health¶
Method
Run the documented deployed backend/database health/smoke path.
Expected
- Intended services can connect.
- Public response does not reveal credentials.
Status: PASS
Evidence / defect / retest: #875 execution record; live worker.probe completed after its database dependency check.
DB-TECH-02 — Clean migration/integration setup¶
Method
Run the repository database integration setup against the approved test database.
Repository command: npm run test:database
Expected
- Committed migrations/schema initialise successfully in the test environment.
- Database integration suite passes.
Status: PASS
Evidence / defect / retest: #875 execution record; clean disposable PostgreSQL migration, seed and integration run passed: 38 files passed, 1 skipped; 278 tests passed, 2 skipped.
DB-TECH-03 — Relational integrity constraints¶
Method
Use database integration tests to exercise representative invalid foreign-key/uniqueness/business-integrity cases.
Expected
- Invalid relational state is rejected.
- Valid authoritative relationships remain queryable.
Status: PASS
Evidence / defect / retest: #875 execution record; covered by the completed isolated database integration suite.
DB-TECH-04 — Transactional consistency¶
Method
Exercise at least one multi-write workflow with an induced/controlled failure at the supported test layer.
Expected
- Workflow does not leave a falsely-complete partial authoritative state.
Status: PASS
Evidence / defect / retest: #875 execution record; covered by the completed isolated database integration suite.
DB-TECH-05 — Representative query/index behaviour¶
Method
For important final read/aggregate paths, inspect representative-scale timing/query-plan evidence where existing performance work exposes it.
Expected
- No obvious final high-volume query depends on an unintended full-table/per-item N+1 path without documented acceptance.
Status: PASS
Evidence / defect / retest: #875 execution record; completed database suite includes the representative query-plan/index integration coverage.
DB-TECH-06 — Production/test data distinction¶
Method
Record final production dataset source/scale and the policy used to distinguish test/demo/writable acceptance data.
Expected
- Final documentation can answer what production data is present and what data was introduced only for testing.
Status: PASS
Evidence / defect / retest: #875 execution record; aggregate-only deployed inventory found 3,207,623 deliveries, 14,020 fixtures and four immutable releases, classified as one local and three dev releases.
WRK-TECH-01 — Worker liveness/readiness¶
Method
Use deployed worker health/readiness endpoints/runbook.
Expected
- Liveness reports running process.
- Readiness reflects required dependencies accurately.
Status: PASS
Evidence / defect / retest: #875 execution record; current internal status is ready with database, object storage, and Service Bus all up. Earlier failure logs were transient/historical.
WRK-TECH-02 — Representative queued job completes¶
Method
Enqueue an approved representative ingestion/background job and follow the durable state.
Expected
- Message is received and work reaches the correct terminal state.
- API/request process is not required to remain open for the full job.
Status: PASS
Evidence / defect / retest: #875 execution record; live worker.probe was received, verified dependencies, and completed. The 19 historical dead-letter messages require separate review.
WRK-TECH-03 — Failure diagnostics and retry¶
Method
Exercise a controlled failing job or approved failure fixture.
Expected
- Failure is durable/observable with useful diagnostics.
- Supported retry changes state predictably.
Status: PASS
Evidence / defect / retest: #875 execution record; Peek-only inspection classified all 19 historical dead-letter messages without changing them. The retained reasons establish the terminal routing mechanism, not an unproven infrastructure/RBAC/network root cause.
WRK-TECH-04 — Duplicate delivery/idempotency¶
Method
Exercise retry/redelivery of the same durable command in the supported integration test path.
Expected
- Canonical publication/release side effects are not duplicated.
Status: PASS
Evidence / defect / retest: #875 execution record; final-candidate worker suite passed its transient-redelivery, completed-delivery idempotency and duplicate-release prevention paths.
WRK-TECH-05 — Interruption/restart recovery¶
Method
Use the documented safe worker recovery exercise or existing automated recovery test.
Expected
- In-progress durable work can resume/retry according to lease/checkpoint semantics.
- Work does not silently disappear or duplicate authoritative results.
Status: PASS
Evidence / defect / retest: #875 execution record; final-candidate worker suite passed interrupted-object cleanup, lease release and resumed-publication recovery paths.
F. Performance, accessibility and responsiveness — #876¶
PERF-TECH-01 — Representative API response times¶
Method
Run the repository response-time measurement against the approved representative environment/data.
Repository command: npm run measure:api-response-times
Expected
- Measurements record endpoint, environment/data size and timings.
- Regressions beyond documented targets/baseline are dispositioned rather than ignored.
Status: BLOCKED
Evidence / defect / retest: #876 execution record retains two local ten-sample reports. Their configured #289 targets were missed, and the environment was not established as an approved isolated representative database. The owner scope disposition accepts that residual risk for #876; it is not a PASS result.
PERF-TECH-02 — Production-scale acceptance¶
Method
Run the production-scale deployment acceptance procedure against the intended environment.
Repository command: npm run verify:production-scale-deployment
Expected
- Representative-scale workflow meets the acceptance criteria recorded by the script/runbook.
- Failure is retained as evidence/bug rather than rerun until green without explanation.
Status: BLOCKED
Evidence / defect / retest: #876 execution record: the supported runner needs a runtime administrator token and creates or reuses a dataset release. Final release mutation and hosted verification remain owned by #810.
PERF-TECH-03 — Frontend Lighthouse measurement¶
Method
Run the final supported Lighthouse path on representative public routes and authenticated routes where configured.
Repository command: npm run test:lighthouse
Expected
- Final measurements are retained with environment/context.
- Severe performance/accessibility regression is dispositioned.
Status: BLOCKED
Evidence / defect / retest: Retained #797 production-preview evidence has a passing representative public-route baseline, but it does not establish Production Performance >=90 for every frontend page: four parameterised mobile routes were below target and protected routes were not audited. No new final-candidate hosted audit or role storage state was available.
PERF-TECH-04 — API/background-work coexistence¶
Method
During representative background ingestion/release work, make representative health/public-read requests.
Expected
- Background processing does not make normal API health/read paths unusable beyond documented capacity limits.
Status: BLOCKED
Evidence / defect / retest: #876 execution record: this requires a live representative background release/ingestion run and concurrent API reads. It was not simulated or run against an unapproved environment.
Final #876 disposition: Closed by owner scope disposition dated 2026-10-09. The BLOCKED performance statuses remain accurate technical results and are not converted to PASS. Their residual risks are accepted for this issue only; final production deployment and release acceptance remain with #810.
RESP-TECH-01 — Public responsive layout¶
Maps from: PUB-*
Method
Exercise primary public routes at representative desktop, tablet and mobile viewports.
Expected
- Supported layouts remain usable without unintended page-level horizontal overflow.
- If a form factor is intentionally unsupported, the product gives the documented clear warning.
Status: PASS
Evidence / defect / retest: #876 execution record: tests/e2e/responsive-layout.spec.ts passed 3/3 against a local production preview. It checks signed-out and administrator headers at 320, 360, 390, 600, 768, 900, 960, 1024, 1100, 1180, 1280 and 1440 CSS pixels, plus no page-level overflow at 320px across the reviewed routes.
RESP-TECH-02 — Authenticated workflow responsiveness¶
Maps from: SUB-*, REV-*
Method
Exercise representative submitter/reviewer screens at supported narrow/wide viewports.
Expected
- Required controls, validation and status information remain usable.
Status: PASS
Evidence / defect / retest: #876 execution record: the 320px reviewed-route reflow check includes the mocked administrator workspace routes for submission and review. It found no page-level horizontal overflow; it is local mocked evidence, not deployed role-session verification.
A11Y-TECH-01 — Automated accessibility regression¶
Method
Run the project's Playwright/axe accessibility coverage on the final candidate.
Expected
- No unresolved serious/critical project-owned accessibility violation remains without disposition.
Status: PASS
Evidence / defect / retest: #876 execution record: tests/e2e/accessibility.spec.ts passed 69/69 local Chromium audits. It found no serious or critical Axe violation across reviewed public and administrator workspace routes, both themes, and the mobile public/authentication/dialog cases.
A11Y-TECH-02 — Keyboard navigation and focus¶
Method
Keyboard-only navigate primary public/authentication/form/reviewer paths.
Expected
- Interactive controls are reachable/operable in a logical order.
- Focus is visible.
- No keyboard trap is present in project-owned UI.
Status: PASS
Evidence / defect / retest: #876 execution record: targeted public-navigation, mobile-menu, combobox and route-focus tests passed. An initial 38/39 focused run had a separate /fixtures title assertion failure; the exact three-file rerun passed 39/39 without a code change. No keyboard/focus assertion failed.
A11Y-TECH-03 — Labels, semantics and accessible names¶
Method
Inspect primary forms/navigation/status regions with browser accessibility tree/automated checks.
Expected
- Inputs/controls have accessible labels.
- Semantic/ARIA usage supports the intended control/region meaning.
Status: PASS
Evidence / defect / retest: #876 execution record: the 69 Axe checks completed without a serious/critical semantic or accessible-name finding. Its reviewed routes assert visible headings and loading-state completion before the audit; targeted component tests also cover labelled navigation and combobox interaction.
A11Y-TECH-04 — Contrast/readability¶
Method
Run automated contrast checks and review final project-owned text/control states, including both supported themes.
Expected
- No unresolved severe contrast/readability problem remains without disposition.
Status: PASS
Evidence / defect / retest: Axe's colour-contrast rule ran within the 69 audits across day/night themes and mobile coverage; no serious/critical violation was reported. The production build reported a non-blocking large-chunk advisory, not a contrast/readability finding.
G. Automated suites, coverage, CI/CD and deployment — #877¶
AUTO-TECH-01 — Frontend test suite¶
Repository command: npm run test:frontend
Expected: final frontend suite passes from the candidate.
Status: PASS
Evidence / defect / retest: #877: the CI-equivalent two-worker run passed 45 files/460 tests; the default configuration now uses that established worker limit without retries.
AUTO-TECH-02 — Backend unit/API suites¶
Repository commands
npm run test:unitnpm run test:apinpm run test:api-contract
Expected: final backend/API/contract suites pass.
Status: PASS
Evidence / defect / retest: #877: full npm run check passed, including backend unit/API/API-contract suites.
AUTO-TECH-03 — Worker suite¶
Repository command: npm run test:worker
Expected: final worker suite passes.
Status: PASS
Evidence / defect / retest: #877: full npm run check passed, including the worker suite.
AUTO-TECH-04 — Contract package suite¶
Repository command: npm run test:contracts
Expected: shared contract suite passes.
Status: PASS
Evidence / defect / retest: #877: full npm run check passed, including contracts.
AUTO-TECH-05 — Database integration suite¶
Repository command: npm run test:database
Expected: database integration suite passes against the approved test database.
Status: PASS
Evidence / defect / retest: #877: npm run test:database passed in Dean's terminal.
AUTO-TECH-06 — Browser end-to-end suite¶
Repository command: npm run test:e2e
Expected: intended final Playwright matrix passes.
Status: PASS
Evidence / defect / retest: #877: npm run test:e2e passed in Dean's terminal.
AUTO-TECH-07 — Deployment/infrastructure suite¶
Repository command: npm run test:deployment
Expected: deployment/infrastructure regression suite passes.
Status: PASS
Evidence / defect / retest: #877: full npm run check passed, including deployment-helper tests.
AUTO-TECH-08 — Intermediate ingestion integration gate¶
Repository command: npm run verify:intermediate-ingestion
Expected: retained ingestion invariants/integration acceptance passes.
Status: PASS
Evidence / defect / retest: #877: npm run ci:local completed with LOCAL CI: PASS.
AUTO-TECH-09 — Repository quality check¶
Repository command: npm run check
Expected: required structure, formatting, lint, type checking, automated tests, OpenAPI lint and builds pass.
Status: PASS
Evidence / defect / retest: #877: npm run check passed.
AUTO-TECH-10 — Repository hygiene/architecture check¶
Repository command: npm run hygiene
Expected: Knip, dependency-version and architecture-boundary checks pass or documented intentional exceptions are current.
Status: PASS
Evidence / defect / retest: #877: npm run hygiene passed.
COV-TECH-01 — Repository-wide coverage generation¶
Repository command: npm run test:coverage
Expected
- Required first-party workspaces emit coverage.
- Aggregated final report is retained.
- Missing workspace artefacts cause failure rather than silently lowering the denominator.
Status: PASS
Evidence / defect / retest: #877: all five required workspaces generated coverage.
COV-TECH-02 — Coverage quality review¶
Method
Review final coverage by first-party workspace/file and identify high-risk weakly tested code.
Expected
- Final evidence records headline coverage and material weak/uncovered areas.
- Files are not excluded only to inflate the result.
Status: PASS
Evidence / defect / retest: #877: 80.77% lines; 79.49% statements; 84.92% functions; 72.27% branches; thresholds remain informational.
COV-TECH-03 — Explicit untested-area register¶
Method
After #871–#877 execution, list final capabilities that remain untested or partially tested and why.
Expected
- Register is non-empty only where gaps really exist.
- Each gap has risk/mitigation/owner or reason for acceptance.
Status: PASS
Evidence / defect / retest: #877 execution record explicitly retains the failures, unrun gates and dependency findings.
CI-TECH-01 — Local final CI plan¶
Repository command: npm run ci:local
Expected: final local change-aware CI completes for the candidate/changes being verified.
Status: PASS
Evidence / defect / retest: #877: npm run ci:local completed with LOCAL CI: PASS.
CI-TECH-02 — Hosted Gitea CI for exact candidate¶
Method
Record the hosted Gitea Actions run for the exact commit/tag proposed for submission.
Expected
- Intended required checks are green.
- No final failing lane is hidden/disabled merely to obtain a green result.
Status: BLOCKED
Evidence / defect / retest: #877: hosted validation remains pending for the rebased exact candidate; no passing run is claimed before it completes.
DEP-TECH-01 — Strict documentation build¶
Repository command: python -m mkdocs build --strict
Expected: public documentation builds without strict warnings/errors.
Status: PASS
Evidence / defect / retest: #877: strict MkDocs build passed.
DEP-TECH-02 — OpenAPI lint¶
Repository command: npm run openapi:lint
Expected: final OpenAPI contract passes configured Redocly lint.
Status: PASS
Evidence / defect / retest: #877: OpenAPI lint passed.
DEP-TECH-03 — Final deployed component smoke¶
Method
Record reachability/health for final:
- frontend;
- backend/API;
- worker;
- database through documented service health;
- documentation site.
Expected
- Submitted deployment endpoints are reachable and correspond to the intended final candidate.
- Final deployment/release sign-off remains owned by #810.
Status: BLOCKED
Evidence / defect / retest: #877: no exact final deployed candidate; #810 owns release sign-off.
DEP-TECH-04 — Dependency/security audit¶
Method
Run the repository's final dependency/security audit commands/policy and review findings rather than applying unsafe forced upgrades.
Expected
- Unresolved findings are documented and dispositioned.
- No known severe production-impacting dependency issue is silently ignored.
Status: PASS
Evidence / defect / retest: #877 applied reviewed non-breaking npm fixes; the post-fix production audit has no critical/high findings. Four moderate Swagger dependency-chain advisories require a breaking forced change and are explicitly retained in the execution record.
6. Final execution summary¶
Do not complete this table while #870 is only establishing the bank. #871–#877 update it from retained evidence.
| Lane | Issue | Status | Candidate | Evidence | Open defects / blockers |
|---|---|---|---|---|---|
| Frontend/auth/roles | #871 | PASS |
955f30105ed02858e42ccf9f3605d48d136c0717 |
#871 execution record | None observed |
| Submission/review/batch/corrections | #872 | PASS |
9b1dbf5fbaa933f682f24f08bed1edf32507a01a |
#872 execution record | None observed |
| Statistics/data/provenance/releases | #873 | PASS |
d963e138d |
#873 execution record | None observed |
| API/contracts/consumer/integration | #874 | PASS WITH ONE FAIL |
74cf6827 |
#874 execution record | API-TECH-03 FAIL: undocumented totalRecords in participants response; explicit follow-up and deployed contract retest required (see #874 F1). |
| Database/worker/reliability | #875 | PASS |
ed28ee025 |
#875 execution record | None blocking; retained logs did not expose the second probe receipt. |
| Performance/accessibility/responsive | #876 | BLOCKED |
83d02d5803cba7ecefd074cbb0a89dd5eb02c130 |
#876 execution record | Performance/hosted checks require an approved environment. |
| Automated suites/coverage/CI/deployment | #877 | LOCAL PASS; RELEASE CHECKS BLOCKED |
26fc2857b |
#877 execution record | Local audited candidate; pre-remediation baseline ab216b987; later documentation/evidence preservation 4a676b554 is not a new test run. CI-TECH-02 and DEP-TECH-03 remain BLOCKED; #810 owns final deployment and production smoke testing. |
Known untested / partially tested areas¶
877 owns the final consolidation of this register.¶
| Area | Reason | Risk / mitigation | Owner / linked issue |
|---|---|---|---|
| Final automated release-candidate matrix | Full local #877 suites and coverage were observed PASS for tested candidate 26fc2857b; exact-candidate hosted CI and final deployed smoke remain BLOCKED. |
Retain candidate-scoped local evidence; #810 must establish the exact release-candidate hosted CI and deployed smoke. | #877 |
| Production dependency advisories | Post-fix audit retains 4 moderate Swagger dependency-chain advisories; no high/critical finding remains. | Plan the breaking Swagger remediation separately; do not force it during the final audit. | #877 |
| Hosted CI and deployed smoke | No exact #877 candidate has been pushed or deployed. | Push only after local repair and approval; retain hosted SHA/run and release smoke under #810. | #877; #810 |
| #876 performance and hosted verification | No approved representative API/database, final hosted preview or authenticated Lighthouse storage state was available. | Run the blocked measurement, Lighthouse and background-work checks only in the approved environment; retain the results under #876/#810. | #876; #810 |
| Populate only from actual execution evidence. | — | — | #877 |
#870 closure checklist¶
870 is complete when the bank itself is ready for execution:¶
- [ ] This page exists and is linked from the public documentation.
- [ ] Technical verification is explicitly separated from formal user testing.
- [ ] All seven execution issues #871–#877 have a defined section/family.
- [ ] Every test case has a unique ID, prerequisites/method, expected result, status and evidence field.
- [ ] Every case begins
NOT RUN; no result has been fabricated. - [ ] Existing user-testing families are mapped only for coverage consistency.
- [ ] Additional technical-only families cover database, worker, performance, accessibility, automated suites, CI and deployment.
- [ ] Execution evidence template exists.
- [ ] Documentation regression test passes.
- [ ] Strict MkDocs build passes before merge.
- [ ] #871–#877 each depend on #870 in Gitea.
After #870 merges, #871–#877 may execute in parallel.
AI Declaration¶
The preceding document was planned, generated, reviewed and edited with the assistance of ChatGPT-Web[GPT-5.6 Sol] and Codex[GPT-5].