Quality cockpit

Purpose

Whether the chat is good is not decided in testing but in the real conversations. dAI Pro logs every question with its answer, sources, rating and outcome and turns them into metrics, topics and concrete work lists. The quality cockpit is not a single screen but the interplay of the log, the metrics under "Usage" and the agents that evaluate the log.

The components

Component	Where	What it shows
Log	Per collection, visible to editors	Every question with answer, sources, language, channel (chat, search, endpoint for agents), rating and outcome.
Conversation outcome	Usage (Chapter 19)	Resolved, not resolved, resolution rate and cost per resolved request, from signals without an AI verdict.
Answer rating	Chat, setting in Chapter 4	Visitors' thumbs below every answer.
Quality audit (agent)	Agents (Chapter 15)	Weekly report: resolution rate, topics with trend, AI sample of the answers against the knowledge excerpts the chat actually had.
Content gap radar (agent)	Agents	Gap topics with example questions and a recommendation.
Golden set (agent)	Agents	Regression test with fixed test questions; reports when an answer regresses.
Trace	Log	For each answer, the path to it: which sections were found, which strategy ran, which model answered. For root-cause analysis.

The conversation outcome

Every answer gets an outcome from signals:

Resolved: answered; confirmed by the visitor with a thumbs up; a follow-up question came; an action was triggered.
Not resolved: knowledge gap (the chat had to decline); rejected (off topic); rephrased (the visitor asked the same thing differently); handover to a human; error.

The resolution rate and the cost per resolved request are shown under "Usage". The weekly report of the quality audit compares them with the previous period.

The working routine

Weekly: read the quality audit report. Topics with a trend show what is currently on visitors' minds; the AI sample names answers that are weak or need checking.
Go through the unresolved requests: knowledge gap means content is missing, rephrasing means content is unclear, handover means the chat got no further. The suggestions of the FAQ generator and the content gap radar take this work off your hands.
Check thumbs down: open the source, correct the content, re-index, clear the cache (Chapter 7).
Extend the golden set as soon as a question is important enough that its answer must never regress again.

Data protection

The log stores questions and answers without linking them to persons and deletes them after the configured period (Chapter 20). Details from handover forms are not in the log; they go to the team by email.

Frequently asked questions

The resolution rate is low, but the answers are good. Check the distribution: many "rejected" point to a topic filter threshold that is too strict, many "rephrased" to unclear content, many "knowledge gap" to missing pages.

Where do I see which sections the chat had for an answer? In the trace of the answer in the log. It shows the sections found, with source and proximity to the question.

Can I export the log? Usage can be exported as Excel and CSV; in TYPO3 the log is available as its own module with export, in the standalone version via the REST API (Chapter 29).