Owner/openspec/changes/separate-wake-and-realtime-transcript/specs/voice-pet-pipeline/spec.md

## ADDED Requirements

### Requirement: Live assistant pipeline events
The live runtime SHALL execute as a stage-based voice assistant pipeline with stable events for every externally observable stage.

#### Scenario: Successful live turn emits ordered events
- **WHEN** a live turn completes successfully
- **THEN** the event stream SHALL include wake listening, wake detected, acknowledgement, capture started, speech started, speech ended, STT started, final transcript, LLM started, TTS started, playback finished, and standby resumed in that order

#### Scenario: Terminal reporting consumes events
- **WHEN** the terminal reporter prints user-facing live status
- **THEN** it SHALL derive the status from pipeline events rather than duplicating stage logic

### Requirement: Primary speaker endpointing
The live runtime SHALL support a per-turn primary speaker endpoint mode that ends the user utterance when the speaker who started the utterance is no longer present.

#### Scenario: Primary speaker disappears
- **WHEN** the user starts speaking after wake and their temporary per-turn voice profile is established
- **THEN** capture SHALL end after the configured primary speaker absence duration even if background noise or non-primary speech remains

#### Scenario: Temporary profile is insufficient
- **WHEN** the live runtime cannot establish a reliable per-turn voice profile because speech is too short or too quiet
- **THEN** capture SHALL fall back to configured VAD silence endpointing and SHALL NOT hang indefinitely

#### Scenario: No persistent voiceprint
- **WHEN** the process exits
- **THEN** no primary speaker profile SHALL be stored or reused across runs

### Requirement: Low latency capture and first utterance preservation
The live runtime SHALL preserve the start of the user's formal utterance after wake acknowledgement and SHALL end capture promptly after the primary speaker stops.

#### Scenario: Acknowledgement finishes before the user speaks
- **WHEN** acknowledgement playback completes and the runtime enters capture
- **THEN** the runtime SHALL NOT perform an additional timed microphone read-and-discard window before listening for the user's formal question

#### Scenario: Queued microphone frames are available
- **WHEN** the live microphone transport has multiple frames queued
- **THEN** the transport SHALL return the available batch to the pipeline so wake and capture stages do not process stale audio one frame at a time

#### Scenario: Primary speaker profile becomes ready quickly
- **WHEN** the user starts the formal utterance after wake
- **THEN** the primary speaker endpoint SHALL use an independent profile readiness threshold no greater than the configured `OWNER_SPEAKER_PROFILE_MIN_MS` rather than waiting for generic VAD minimum duration

#### Scenario: User stops after a short first question
- **WHEN** the primary speaker profile is ready and the primary speaker disappears for the configured absence duration
- **THEN** capture SHALL close the utterance and proceed to STT without waiting for a second repeated question or generic VAD minimum duration

### Requirement: Realtime transcript terminal output
The live runtime SHALL display recognized user utterance text in the terminal both during recording when partial transcript is available and after final STT succeeds before the LLM request is sent.

#### Scenario: User utterance partial is available during recording
- **WHEN** the user is speaking after wake and the local realtime STT session produces a changed partial transcript
- **THEN** the terminal output SHALL include a realtime transcript message containing the partial text before final STT completes

#### Scenario: Partial transcript is displayed
- **WHEN** a partial transcript is emitted
- **THEN** it SHALL be treated as user-visible feedback only and SHALL NOT be appended to the conversation context or sent to the LLM

#### Scenario: Realtime transcript stops advancing
- **WHEN** the user utterance has started, at least one partial transcript has been displayed, and no new partial transcript text is emitted for `OWNER_REALTIME_TRANSCRIPT_IDLE_TIMEOUT_MS`
- **THEN** capture SHALL end the current utterance with end reason `partial_transcript_idle`
- **AND** the runtime SHALL proceed to final STT for the captured audio instead of waiting for VAD maximum recording duration

#### Scenario: User utterance is transcribed
- **WHEN** a live turn captures a user utterance and STT returns non-empty text
- **THEN** the terminal output SHALL include a transcript message containing the recognized text before the thinking/LLM status is emitted

#### Scenario: Transcript is empty
- **WHEN** STT returns empty text, punctuation-only text, or an invalid transcript
- **THEN** the runtime SHALL NOT emit a misleading transcript as valid user input and SHALL recover to standby without invoking the LLM

### Requirement: Local voice chain and noise filtering
The live runtime SHALL run wake, VAD, STT, realtime transcript, TTS, and capture-stage noise filtering locally by default, while continuing to send only the final user text and session history to the configured cloud LLM.

#### Scenario: Default speech provider is local
- **WHEN** the user starts `run-live` without overriding `OWNER_SPEECH_PROVIDER`
- **THEN** the runtime SHALL use local sherpa-onnx STT and local macOS TTS
- **AND** it SHALL NOT call cloud ASR or cloud TTS providers during the turn

#### Scenario: Capture audio is denoised before downstream stages
- **WHEN** `OWNER_NOISE_FILTER_ENABLED=1` and the user speaks after wake acknowledgement
- **THEN** the capture stage SHALL pass microphone frames through the configured local denoiser before VAD analysis, realtime partial STT, and final STT segment assembly
- **AND** all three downstream consumers SHALL observe the same denoised frame content for a given captured frame

#### Scenario: Wake remains raw by default
- **WHEN** `OWNER_WAKE_DENOISE_ENABLED` is unset or `0`
- **THEN** wake word detection SHALL receive raw microphone frames rather than denoised frames

#### Scenario: Denoiser model is missing
- **WHEN** `models/denoise/gtcrn_simple.onnx` is absent
- **THEN** `model-check` SHALL fail with a structured missing model error
- **AND** `run-live` SHALL NOT enter live microphone listening as if noise filtering were available

#### Scenario: Denoiser runtime fails
- **WHEN** the local denoiser raises an error while processing a formal user utterance frame
- **THEN** the current turn SHALL emit a stage error and recover to standby
- **AND** the runtime SHALL NOT invoke final STT, LLM, or TTS for that failed turn

#### Scenario: Local ASR uses the 2025 CTC model by default
- **WHEN** the default manifest is written by `download_speech_models.py`
- **THEN** the STT provider SHALL be declared as `sherpa-onnx-streaming-zipformer2-ctc`
- **AND** required STT files SHALL be `tokens.txt` and `model.int8.onnx`, not transducer encoder/decoder/joiner files

#### Scenario: Realtime partial filters noise
- **WHEN** local streaming STT emits a single meaningful character or a short transient regression
- **THEN** the terminal reporter SHALL NOT display that text as `实时转写`
- **AND** final STT SHALL remain responsible for the text sent to the LLM

#### Scenario: Audio privacy is preserved
- **WHEN** local noise filtering and local STT run during capture
- **THEN** raw PCM, denoised PCM, VAD features, partial transcript metadata, and temporary speaker profile data SHALL remain process-local and SHALL NOT be sent to the cloud LLM

### Requirement: Simulated microphone live acceptance
The system SHALL provide a deterministic simulated microphone acceptance command that exercises the current live `VoiceAssistantPipeline` without requiring a human to speak into a physical microphone.

#### Scenario: Simulated live command completes repeated turns
- **WHEN** the developer runs `.venv/bin/python -m owner_voice_pet simulate-live --turns 2`
- **THEN** the command SHALL feed simulated microphone audio frames into the live pipeline and complete two wake-to-playback turns
- **AND** the JSON output SHALL report two completed turns, zero failed turns, two final transcripts, two LLM calls, at least two playback completions, and standby resumed after each turn

#### Scenario: Simulated input includes noisy realtime partials
- **WHEN** simulated formal question frames contain short noisy partial text such as `家` or `家确`
- **THEN** realtime transcript output SHALL filter those short noise partials and SHALL only report stable question text

#### Scenario: Simulated input includes background speech after the user stops
- **WHEN** simulated background speech remains after the primary speaker frames end
- **THEN** primary speaker endpointing SHALL close the formal utterance and SHALL NOT merge later simulated turns into the current question

#### Scenario: Simulated command can write and replay fixtures
- **WHEN** the developer passes `--write-fixture path.jsonl`
- **THEN** the command SHALL write the generated simulated microphone frames as JSONL
- **AND** a later run using `--fixture path.jsonl` SHALL replay the same frames and produce equivalent successful live acceptance results

#### Scenario: Simulated microphone frames are exhausted unexpectedly
- **WHEN** the simulated microphone transport runs out of frames before the expected turn completes
- **THEN** the command SHALL fail with a structured error instead of waiting indefinitely

### Requirement: Automatic continuous dialog decision
The live runtime SHALL automatically decide after each assistant reply whether to continue listening for a direct follow-up answer or to end the continuous session and return to wake standby.

#### Scenario: Assistant asks the user a direct follow-up question
- **WHEN** an assistant reply clearly asks the user a question, requests missing information, asks the user to choose among options, or otherwise waits for the user's next answer
- **THEN** the continuation decision SHALL be `continue` with confidence at or above the configured threshold
- **AND** the runtime SHALL enter follow-up listening after playback without requiring the wake word again

#### Scenario: Assistant completes the answer
- **WHEN** an assistant reply only provides an answer, reports a result, refuses, reports an error, ends a greeting, or is ambiguous about needing more user input
- **THEN** the continuation decision SHALL be `standby`
- **AND** the runtime SHALL return to wake standby after playback

#### Scenario: Hybrid decision falls back to an LLM classifier
- **WHEN** deterministic rules cannot decide continue or standby
- **THEN** the hybrid provider SHALL call the configured cloud LLM with a classifier prompt that can only resolve to `continue` or `standby`
- **AND** a low-confidence, malformed, timed-out, or failed classifier result SHALL be treated as `standby`

#### Scenario: Follow-up answer arrives in the listening window
- **WHEN** the decision is `continue` and the user starts speaking within `OWNER_FOLLOWUP_LISTEN_TIMEOUT_MS`
- **THEN** the runtime SHALL capture, realtime-transcribe, final-transcribe, send to LLM, synthesize, and play the answer without local wake detection and without the wake acknowledgement phrase

#### Scenario: Follow-up listening times out
- **WHEN** the decision is `continue` but no user speech is detected before `OWNER_FOLLOWUP_LISTEN_TIMEOUT_MS`
- **THEN** the runtime SHALL emit `followup_timeout`
- **AND** it SHALL emit `continuous_session_ended`
- **AND** it SHALL return to wake standby

### Requirement: Barge-in playback interruption
The live runtime SHALL support a first-version barge-in path that can stop assistant playback when the user starts a valid new utterance during TTS playback.

#### Scenario: Echo guard suppresses early playback echo
- **WHEN** TTS playback starts and microphone input appears before `OWNER_BARGE_IN_ECHO_GUARD_MS`
- **THEN** the runtime SHALL NOT treat that input as a user barge-in

#### Scenario: Valid user speech interrupts playback
- **WHEN** TTS playback has passed the echo guard, VAD observes at least `OWNER_BARGE_IN_MIN_SPEECH_MS` of user speech, and realtime STT produces a valid partial transcript
- **THEN** the runtime SHALL emit `barge_in_detected`
- **AND** it SHALL stop remaining playback chunks and emit `playback_interrupted`
- **AND** it SHALL process the user's new utterance as the next input without requiring wake

#### Scenario: Interrupted assistant context is partial
- **WHEN** playback is interrupted before the full assistant reply is spoken
- **THEN** the process-local conversation context SHALL append only the assistant text that had been fully spoken before interruption
- **AND** unspoken assistant text SHALL NOT be appended to the session history

### Requirement: Real provider live fixture acceptance
The system SHALL support scripted full-chain acceptance that drives the live pipeline with generated microphone PCM while using real local speech providers, the configured cloud LLM, and local TTS/playback capability.

#### Scenario: Real provider fixture completes one turn
- **WHEN** the developer runs `.venv/bin/python -m owner_voice_pet real-live-check --turns 1`
- **THEN** the pipeline SHALL use the real local KWS, local VAD, local denoiser, local STT, configured cloud LLM, local TTS, and playback provider path to complete a wake-to-standby turn

#### Scenario: Real provider fixture completes repeated turns with temporary context
- **WHEN** the developer runs `.venv/bin/python -m owner_voice_pet real-live-check --turns 2` and the first generated turn gives a user fact
- **THEN** the second LLM request SHALL include the first turn's user and assistant history from process-local memory
- **AND** no history SHALL be read from disk or persisted after the process exits
- **AND** JSON output SHALL include checks for completed turns, event counts, final transcripts, playback, flush count, and temporary context

#### Scenario: Zero playback drain still flushes queued echo
- **WHEN** ACK or TTS playback completes and `OWNER_POST_PLAYBACK_DRAIN_MS=0`
- **THEN** the runtime SHALL call `flush_input()` once to clear already queued playback echo
- **AND** it SHALL NOT perform an additional timed microphone read-and-discard loop

#### Scenario: No acknowledgement playback is configured
- **WHEN** `OWNER_WAKE_ACK_TEXT` is empty and no wake acknowledgement audio is played
- **THEN** the runtime SHALL NOT run post-playback drain before capture starts

#### Scenario: Fixture transport preserves future frames
- **WHEN** a memory or fixture transport represents pre-generated future microphone frames
- **THEN** tests SHALL be able to configure flush to record the cleanup call without deleting those future frames

#### Scenario: Real provider check runs quietly
- **WHEN** the developer runs `.venv/bin/python -m owner_voice_pet real-live-check --turns 2 --no-playback`
- **THEN** the command SHALL still synthesize local TTS output and emit pipeline playback events
- **AND** it SHALL skip actual speaker output for automated or quiet environments

#### Scenario: Real provider check reports timing diagnostics
- **WHEN** the developer runs `.venv/bin/python -m owner_voice_pet real-live-check --turns 2`
- **THEN** the JSON output SHALL include `timing.started_at`, `timing.finished_at`, `timing.duration_ms`, `timing.phases`, `timing.events`, and `timing.stage_timings`
- **AND** `timing.stage_timings` SHALL include turn-scoped durations for wake wait, acknowledgement, wait-for-speech, capture, final STT, LLM-to-first-TTS, TTS playback, and full turn when the corresponding events are emitted
- **AND** the JSON output SHALL include `llm_request_timings` with one item per LLM call containing `sent_at`, `sent_offset_ms`, `first_delta_ms`, `finished_at`, `duration_ms`, `message_count`, and a short last-user preview
- **AND** the timing output SHALL NOT contain API keys, raw audio payloads, denoised audio payloads, or full secret-bearing provider request headers

### Requirement: Wake acknowledgement before recording
The live runtime SHALL provide an audible local acknowledgement after local wake detection and before it starts recording the user's formal question.

#### Scenario: Wake is detected
- **WHEN** the local wakeword provider detects “小杰小杰”
- **THEN** the runtime SHALL play a short acknowledgement such as “我在” before entering the user utterance recording state

#### Scenario: Acknowledgement playback finishes
- **WHEN** the acknowledgement playback completes
- **THEN** the runtime SHALL clear buffered microphone input captured during acknowledgement playback before starting VAD recording for the user's question and SHALL NOT discard additional post-acknowledgement user speech by default

### Requirement: Fast user utterance endpointing
The live runtime SHALL use project-local VAD capability with an energy-threshold fallback by default for user utterance endpoint detection and SHALL expose configurable silence timing so the recording stops promptly after the user stops speaking.

#### Scenario: User stops speaking after wake
- **WHEN** VAD observes the configured continuous silence duration after a started utterance
- **THEN** the runtime SHALL close the utterance segment and proceed to STT without waiting for the maximum recording duration

#### Scenario: Local VAD model is available
- **WHEN** `OWNER_VAD_PROVIDER=local`
- **THEN** the runtime SHALL use the project-local `sherpa-onnx` VAD model rather than a raw energy threshold for live user utterance endpointing

#### Scenario: Hybrid VAD is configured
- **WHEN** `OWNER_VAD_PROVIDER=hybrid`
- **THEN** the runtime SHALL load both the project-local VAD model and an energy-threshold fallback and SHALL treat a frame as speech when either detector reports speech

#### Scenario: Wake acknowledgement prompt ordering
- **WHEN** local wake detection succeeds and `OWNER_WAKE_ACK_TEXT` is non-empty
- **THEN** the terminal SHALL emit wake hit, acknowledgement, question prompt, and recording states in that order so the user is cued to speak only after acknowledgement playback

## MODIFIED Requirements

### Requirement: Wake word detection
The system SHALL listen locally for the Chinese wake word “小杰小杰” with a dedicated local wake word provider before accepting user speech for a conversation turn, and the default live runtime implementation SHALL use a project-local KWS model rather than cloud ASR or the formal user STT provider for wake detection.

#### Scenario: Wake word is detected
- **WHEN** the user says “小杰小杰” and the local wakeword provider returns confidence above the configured threshold
- **THEN** the pipeline SHALL transition from wake listening to speech detection without invoking the configured STT provider for the wake audio

#### Scenario: Wake word is not detected
- **WHEN** background speech or noise does not match “小杰小杰”
- **THEN** the pipeline SHALL remain in wake listening and SHALL NOT invoke STT, LLM, or TTS

#### Scenario: Wake model fails
- **WHEN** the wakeword provider cannot load or process audio
- **THEN** the system SHALL report a wakeword error and SHALL NOT crash the desktop pet process

#### Scenario: Wake audio is isolated from user utterance
- **WHEN** the local wakeword provider detects “小杰小杰”
- **THEN** the runtime SHALL reset the user utterance recorder and SHALL NOT add wake audio or wake transcript text to the LLM conversation context

### Requirement: Local speech model management
The system SHALL provide project-local speech model preparation and diagnostics for live wake/VAD/STT operation, storing downloaded model artifacts under `models/` without committing them to Git.

#### Scenario: Models are downloaded
- **WHEN** the user runs `python3.11 scripts/download_speech_models.py --dir models`
- **THEN** the script SHALL create or update a project-local model directory with the files required by the configured wake, VAD, STT, and denoiser providers

#### Scenario: Model check succeeds
- **WHEN** required wake, VAD, STT, denoiser dependencies and model files are available
- **THEN** `PYTHONPATH=src python3.11 -m owner_voice_pet model-check` SHALL exit successfully and report the model directory and checked providers

#### Scenario: Model check fails
- **WHEN** `sherpa-onnx` is unavailable, a wake model file is missing, a VAD/STT/denoiser model file is missing, or a model cannot be loaded
- **THEN** `model-check` SHALL fail with a structured model error and SHALL NOT start live microphone listening

### Requirement: Live repeat voice runtime
The system SHALL provide a `run-live` command that performs real repeated voice conversation with local microphone input, local model wake detection, configured speech recognition and speech synthesis providers, cloud LLM reply generation, local speaker playback, and automatic return to standby.

#### Scenario: Live runtime starts in standby
- **WHEN** the user runs `PYTHONPATH=src python3.11 -m owner_voice_pet run-live`
- **THEN** the system SHALL load `.env`, validate required live dependencies, initialize local wake/audio/model providers, and enter a standby listening loop

#### Scenario: Live runtime completes two turns
- **WHEN** the user wakes the system with “小杰小杰”, asks a question, hears the reply, then wakes it again and asks another question
- **THEN** the system SHALL complete local wake, audible acknowledgement, recording, STT, transcript display, LLM, TTS, playback for both turns and SHALL return to standby after each turn

#### Scenario: Once mode completes one turn
- **WHEN** the user runs `PYTHONPATH=src python3.11 -m owner_voice_pet run-live --once`
- **THEN** the system SHALL run at most one local-wake-to-playback turn and exit after the turn completes or fails with a documented live error

### Requirement: Live terminal state reporting
The live runtime SHALL emit concise Chinese terminal status messages for observable runtime states, including explicit user transcript output.

#### Scenario: Normal turn status
- **WHEN** a live turn succeeds
- **THEN** terminal output SHALL include states equivalent to standby, wake hit, recording, transcribing, transcript result, thinking, speaking, and returning to standby

#### Scenario: Recoverable error status
- **WHEN** a live turn encounters empty STT, LLM failure, TTS failure, or playback failure
- **THEN** terminal output SHALL include the failing stage and a stable error code or recoverable explanation

### Requirement: Testability
The system SHALL be designed so each stage can be tested with mock providers, file-based audio fixtures, and fake live runtime components without requiring real devices in automated tests.

#### Scenario: Wake and STT separation is unit tested
- **WHEN** fake live providers run a turn with a wake frame and a user utterance
- **THEN** tests SHALL verify wake detection happens through the wake provider and the formal STT provider is called only for the user utterance

#### Scenario: Repeated runtime is unit tested
- **WHEN** fake live providers produce two deterministic turns
- **THEN** tests SHALL verify two wake detections, two STT calls, two LLM calls, two TTS calls, two playback calls, transcript output, and final return to standby

#### Scenario: Wake keyword does not pollute LLM input
- **WHEN** a wake frame contains “小杰小杰” and the later user utterance transcript is “第一问”
- **THEN** the current user message sent to the LLM SHALL be exactly “第一问” rather than a concatenation with the wake keyword