An AEO observation is useful when another reviewer can inspect the answer and understand how it was collected. A screenshot of a favorable sentence often omits the prompt, conversation history, settings, and surrounding qualifications that change its meaning.
Create a small capture manifest for every observation. It should connect the exact question, available environment details, answer evidence, and coding decision. Keep the storage method appropriate to the interface's current terms and your organization's data rules.
This is an original evidence-management template. It does not authorize automated collection from a particular service or claim that every interface exposes the same metadata.
Record the observation and its conditions
Use a stable observation ID that does not depend on a product name appearing in the answer. A missing brand is still a valid observation if the capture itself is complete.
| Field | Record |
|---|---|
| Panel and prompt | Panel version, prompt ID, exact wording, permitted follow-up context |
| Environment | Product, available model label, account state, relevant settings |
| Time | Actual observation time and time zone when known |
| Evidence | Complete answer or permitted capture, citation destinations, limitations |
| Review | Coder, rubric version, result, unresolved questions |
Use “not exposed” when an interface does not reveal a model version. Do not infer a precise backend model from branding or a remembered announcement. The manifest should preserve available evidence, not manufacture completeness.
Keep raw evidence separate from interpretation
Store the answer capture as the observed record. Store mention, recommendation, citation, and accuracy codes in a separate review layer. If a reviewer changes a code, the original answer should remain unchanged.
In a hypothetical observation, the answer names a product but says it is unsuitable for offline work. The capture contains the full paragraph. The coding layer records a mention, no recommendation for the requested offline task, and any relevant citation. Cropping the warning away would change the evidence.
Keep links to the exact cited destinations as observed. A later redirect or page update can change what a reviewer sees when reopening a source. Record the check date and relevant source passage when permitted so that change is visible.
Redact deliberately and document the effect
Remove unnecessary personal data, account details, or private conversation content from shareable evidence. Preserve a note describing what was removed and whether the redaction affects interpretation.
If the prompt depended on private customer information that cannot be shared, do not present a redacted screenshot as a fully reproducible public benchmark. It can still support an internal review with appropriate access controls and stated limits.
Use an access-controlled location for sensitive raw records and a separate redacted review artifact when needed. Do not expose cookies, tokens, or unrelated account information in screenshots or exports.
Handle incomplete captures explicitly
A truncated answer, inaccessible citation, or failed generation needs a status of its own. Keep it in the collection record with the failure reason. Do not silently replace it with a favorable retry and pretend the original attempt never happened.
If a retry is part of the method, assign it a related attempt ID and preserve the rule. A panel can distinguish collection failures from valid answers that contain no mention. Those states have different implications for the denominator.
Use the evidence log and the frozen panel together. The finished packet should allow another reviewer to locate the question, inspect the answer, understand its limitations, and follow the reasoning behind the code.