What Should an AI Evaluation Evidence Package Contain?

QUICK ANSWER

An AI evaluation evidence package should identify exactly what decision was evaluated, which baseline and candidate were compared, what data and time window were used, which evidence was available at the relevant decision time, how the comparison was executed, what metrics and guardrails were declared, what uncertainty and limitations remain, and who retains authority over the next action.

The goal is not more paperwork. It is to prevent a result from losing meaning when it moves across Data Science, Fraud, Model Risk, Architecture, Security, Audit or an executive sponsor.

WHY AN EVIDENCE PACKAGE MATTERS

Many AI evaluations exist as a notebook, spreadsheet, slide or analyst explanation. That may be sufficient for the person who ran the test, but weak for a second reviewer trying to answer basic questions later:

Which model version was tested?

Was the baseline reconstructed correctly?

Were all signals available before the decision point?

Did the candidate use the same review budget?

Which metric was primary?

What changed after the test?

What was the result actually allowed to support?

A review-ready evidence package makes those answers explicit.

1. DECISION QUESTION

Start with the decision the evaluation is meant to inform.

Weak: “Test the new fraud model.”

Stronger: “Under the same Top-K review capacity and approved historical evidence, does candidate C provide enough review value versus baseline B to justify a non-customer-impacting shadow stage?”

The decision question defines the scope of every other artifact.

2. WORKFLOW AND DECISION OWNER

Identify the workflow in which the candidate may be used and the owner responsible for deciding what happens next.

Relevant context can include:

• workflow purpose;
• current human and system roles;
• where AI output enters the process;
• operational capacity;
• consequential customer or production boundaries;
• named business or risk owner.

If no realistic decision could change because of the result, the evaluation may not be worth running.

3. BASELINE IDENTITY

The evidence package should record the current reference process sufficiently to reconstruct the comparison.

That can include:

• model or rule version;
• thresholds;
• feature set;
• workflow configuration;
• relevant dependencies;
• policy state;
• date or release identity.

A challenger cannot be evaluated fairly against an ambiguous baseline.

4. CANDIDATE IDENTITY

Record exactly what is changing.

The candidate may be a model, signal, threshold, rule, prompt, retrieval configuration, ranking treatment or another bounded change. If the candidate changes materially during evaluation, the package should record that a new basis was created.

5. DATASET, POPULATION AND HISTORICAL WINDOW

Document which cases were eligible and why.

At minimum record:

• source data;
• population filters;
• historical start and end dates;
• outcome or label definition;
• label-maturity rules;
• exclusions;
• missingness handling;
• relevant sampling decisions.

This helps reviewers distinguish a valid comparison from a convenient sample.

6. EVIDENCE TIMING AND LEAKAGE CONTROLS

For each material signal, ask whether it existed before the decision being reconstructed.

Temporal leakage can make a retrospective result look strong while being impossible to reproduce in the intended workflow. The package should therefore document event-time semantics, observation cutoffs, delayed labels and any late-arriving evidence.

7. PROTOCOL

A protocol states how baseline and candidate were compared.

Useful elements include:

• preprocessing steps;
• model execution or scoring procedure;
• ranking or threshold logic;
• fixed review-capacity definition;
• evaluation metric definitions;
• uncertainty method;
• subgroup or slice checks;
• stop conditions;
• permitted reruns;
• random seeds where relevant.

The objective is enough precision for another qualified reviewer to understand or reproduce the declared comparison.

8. PRIMARY METRIC

The primary metric should be chosen because it represents the decision, not because it produces the most attractive result.

For fraud review, examples may include precision at K, recall at K, positive cases surfaced at a fixed review capacity or another directly measured operational metric.

For other workflows, the primary metric may be calibration, latency, error severity, task success or a different measure.

9. GUARDRAILS

Guardrails prevent a candidate from winning by creating unacceptable trade-offs.

Possible guardrails include:

• false escalations;
• investigator workload;
• customer friction;
• latency;
• stability;
• subgroup behaviour;
• security or privacy constraints;
• policy constraints.

A primary metric improvement should not automatically override a material guardrail breach.

10. RESULTS AND UNCERTAINTY

Record the result in a form that can be checked independently.

That may include point estimates, confidence intervals, bootstrap intervals, repeated-run stability, overlap analysis, error analysis or other appropriate uncertainty measures.

Do not convert statistical evidence into unmeasured business claims. More positive cases surfaced in a benchmark, for example, does not automatically establish prevented fraud losses.

11. LIMITATIONS AND UNRESOLVED ISSUES

A useful evidence package says what it does not establish.

Examples:

• historical data may not represent current conditions;
• investigator-hours were not measured;
• security integration was not tested;
• the evaluation did not include customer impact;
• a third-party dependency could not be reproduced;
• production monitoring remains undefined.

Limitations preserve the authority of the result by preventing it from being reused for a broader claim later.

12. VERSION, PROVENANCE AND INTEGRITY REFERENCES

Where appropriate, record artifact identities, software versions, hashes, manifests, execution environment and data lineage. The level of detail should be proportionate to the decision and risk.

Integrity evidence is not the same as methodological correctness. A hash can show that an artifact has not changed; it does not prove that the evaluation design was valid. Both questions matter and should remain separate.

13. DECISION OUTPUT

End with an explicit next-step status.

Continue — evidence supports progression to the next declared stage.

Refine — candidate or evidence needs correction before progression.

Stop — current evidence does not justify progression.

Not evaluable — the evidence basis cannot answer the declared question.

The package should also state who retains authority to make the final decision.

WHY THIS MATTERS FOR AI ASSURANCE

AI Verify Foundation’s real-world testing work emphasises that assurance involves more than running tests. It spans risk assessment, test selection, execution, configuration and result interpretation, and it requires engagement from technical and non-technical stakeholders.

An evidence package provides the common object around which those stakeholders can review the same result.

HOW AEGI FRAMES EVIDENCE PACKAGING

AEGI Shield’s Controlled Evaluation produces review-ready evidence for one bounded change. AEGI’s proof architecture deliberately separates mechanism evidence, empirical evaluation evidence, operating evidence and institution-specific evidence so one proof family is not silently used to support another claim.

AEGI Core separately verifies declared evidence properties and scope within its protocol boundaries. Core does not determine whether the business methodology is correct or whether an institution should approve production.

FREQUENTLY ASKED QUESTIONS

Does every evidence package need cryptographic hashes?

No. Integrity controls should be proportionate. Hashes and manifests are useful where artifact identity matters, but they do not replace sound evaluation design.

Should raw customer data be included in the package?

Not necessarily. Evidence packaging should respect the institution’s approved data-handling and locality requirements. The package can reference controlled data sources and lineage without exporting sensitive records.

Who should review the package?

That depends on the use case. Relevant reviewers may include model owners, Fraud or Risk, Model Risk, Security, Architecture, Compliance, Audit or an executive sponsor.

Can the package support production approval directly?

It can contribute evidence, but production approval remains an institution-specific authority decision and may require additional validation, security and operational gates.

CONCLUSION

A good AI evaluation result should survive handoff. If the meaning of the result depends on the original analyst being present to explain it, the evidence is fragile.

A review-ready package ties the decision to the baseline, candidate, data, protocol, metrics, limitations and authority that produced it.

NEXT STEP

If your team has one proposed fraud or risk workflow change, AEGI can first assess whether the evidence path can be bounded and packaged for a Controlled Evaluation.

CLAIM BOUNDARY

This article is educational. Evidence-package requirements vary by institution, use case and applicable governance framework. Integrity mechanisms do not by themselves establish methodological correctness, regulatory compliance or production readiness.

SOURCES

[1] AI Verify Foundation — test lifecycle and multi-stakeholder engagement:

https://assurance.aiverifyfoundation.sg/report/whats-next/

Related AEGI resources

• Evidence
• AEGI Core