Evaluation | NERVE by AIC

The sequence

Ten steps that prove the chain end to end.

Work through the sequence in order. Each step produces something you can inspect, so the result is an acceptance record rather than an impression.

Choose your sources

Select two or three materially different approved or synthetic sources, so the chain you test is representative rather than convenient.

Set the policy boundary

Define purpose, handling, authority scope and expected volumes before anything is acquired.

Preserve and verify

Preserve the originals, then verify acquisition context and cryptographic hashes against exactly what was accepted.

Exercise transformation

Run at least two transformation paths, so canonical mapping is tested across genuinely different content types.

Search across families

Search a common concept across source families and confirm results arrive from more than one origin.

Resolve an exact reference

Resolve a single evidence reference and inspect its full lineage back to the original bytes.

Reprocess and re-check

Reprocess with a new version and confirm the earlier reference still resolves to the same thing.

Test denied discovery

Confirm unauthorised material stays undiscoverable, with no hidden-result leakage through counts or facets.

Interrupt the pipeline

Interrupt workers mid-run and confirm durable, idempotent recovery with no duplicates and no gaps.

Exercise offline paths

Where relevant, run offline operation and a fresh restore that resolves a pre-existing evidence reference.

Evaluation scope depends on your agreed evidence boundary. We will set that boundary with you before any step begins.

What success looks like

Agree the measures before you start.

These are the eleven things worth measuring. Pick the ones that matter to your service and write the target down, so the outcome is a decision rather than a debate.

Onboarding effort

How much work it takes to bring a new source family into the fabric.

Accepted bytes and records

Volume accepted into preservation, counted against what was offered.

Source-to-search time

Elapsed time from acquisition to an authorised result.

Canonical mapping coverage

Share of fields mapped into the common envelope without loss.

Warnings and failures

Rate and type of warnings, rejections and hard failures across the run.

Citation resolution after reprocessing

Whether an earlier evidence reference still resolves once a new version has run.

Authorised search usefulness

Whether the results an authorised user sees actually answer the question asked.

Storage overhead

Cost of holding originals, canonical records and derived artefacts together.

Worker recovery

Behaviour after interruption: duplicates, gaps and time to catch up.

Restore performance

Time to a working system from a fresh restore, and whether references survive it.

Effort to explain provenance

How long it takes to explain where a result came from, to someone who was not there.

After you send the brief

Four things happen, in this order.

Step 1

We confirm the boundary

We agree the evidence boundary, the sources in scope and who owns each part of the environment.

Step 2

We stand up the profile

Local, private or approved cloud, configured to the profile you chose, with the policy set you supplied.

Step 3

We run the sequence together

Your people drive it. We record the measures against each step as the run progresses.

Step 4

You keep the acceptance record

A written record of what was accepted, what was transformed, what resolved and what it cost to hold.

Plan an evaluation around your information.