Requirements
Design a Siri response-generation flow for an assistant that calls tools and then speaks a concise answer.
You should cover:
- How the system grounds the response in tool output instead of unsupported model memory.
- How to evaluate factuality, usefulness, and voice-chat length.
- Why hallucination happens and what product / model guardrails reduce it.
- How to handle long context from conversation history, user state, and tool payloads.
- How to keep the answer short enough for spoken interaction without dropping required facts.
Notes
A strong answer separates retrieval / tool execution, evidence selection, response generation, verification, and final voice formatting. Treat the tool result as the source of truth; the model should summarize, transform, or ask for clarification rather than invent missing data.
Evaluation should mix offline labeled sets, adversarial queries, tool-result consistency checks, human preference review, and online metrics such as correction rate, repeat-query rate, and user abandonment. For long context, discuss summarization with citations to retained evidence, recency windows, structured memory, and hard limits on what the generator may rely on.
Preparation
- Practice a 10-minute whiteboard flow: user query -> intent -> tool call -> evidence extraction -> grounded generation -> verifier -> voice response.
- Prepare a compact hallucination taxonomy: missing evidence, stale memory, ambiguous tool result, prompt injection, and over-compression.
- Drill eval metrics for conversational assistants: factuality, helpfulness, latency, answer length, and user-correction signals.
- Be ready to explain why voice UX changes the design: shorter responses, explicit uncertainty, and follow-up questions beat long text-style answers.