Owner/openspec/changes/separate-wake-and-realtime-transcript/specs/voice-pet-pipeline/spec.md

ADDED Requirements

Requirement: Live assistant pipeline events

The live runtime SHALL execute as a stage-based voice assistant pipeline with stable events for every externally observable stage.

Scenario: Successful live turn emits ordered events

• WHEN a live turn completes successfully
• THEN the event stream SHALL include wake listening, wake detected, acknowledgement, capture started, speech started, speech ended, STT started, final transcript, LLM started, TTS started, playback finished, and standby resumed in that order

Scenario: Terminal reporting consumes events

• WHEN the terminal reporter prints user-facing live status
• THEN it SHALL derive the status from pipeline events rather than duplicating stage logic

Requirement: Primary speaker endpointing

The live runtime SHALL support a per-turn primary speaker endpoint mode that ends the user utterance when the speaker who started the utterance is no longer present.

Scenario: Primary speaker disappears

• WHEN the user starts speaking after wake and their temporary per-turn voice profile is established
• THEN capture SHALL end after the configured primary speaker absence duration even if background noise or non-primary speech remains

Scenario: Temporary profile is insufficient

• WHEN the live runtime cannot establish a reliable per-turn voice profile because speech is too short or too quiet
• THEN capture SHALL fall back to configured VAD silence endpointing and SHALL NOT hang indefinitely

Scenario: No persistent voiceprint

• WHEN the process exits
• THEN no primary speaker profile SHALL be stored or reused across runs

Requirement: Realtime transcript terminal output

The live runtime SHALL display the recognized user utterance text in the terminal after STT succeeds and before the LLM request is sent.

Scenario: User utterance is transcribed

• WHEN a live turn captures a user utterance and STT returns non-empty text
• THEN the terminal output SHALL include a transcript message containing the recognized text before the thinking/LLM status is emitted

Scenario: Transcript is empty

• WHEN STT returns empty text, punctuation-only text, or an invalid transcript
• THEN the runtime SHALL NOT emit a misleading transcript as valid user input and SHALL recover to standby without invoking the LLM

Requirement: Wake acknowledgement before recording

The live runtime SHALL provide an audible local acknowledgement after local wake detection and before it starts recording the user's formal question.

Scenario: Wake is detected

• WHEN the local wakeword provider detects “小杰小杰”
• THEN the runtime SHALL play a short acknowledgement such as “我在” before entering the user utterance recording state

Scenario: Acknowledgement playback finishes

• WHEN the acknowledgement playback completes
• THEN the runtime SHALL clear buffered microphone input captured during acknowledgement playback before starting VAD recording for the user's question

Requirement: Fast user utterance endpointing

The live runtime SHALL use project-local VAD capability with an energy-threshold fallback by default for user utterance endpoint detection and SHALL expose configurable silence timing so the recording stops promptly after the user stops speaking.

Scenario: User stops speaking after wake

• WHEN VAD observes the configured continuous silence duration after a started utterance
• THEN the runtime SHALL close the utterance segment and proceed to STT without waiting for the maximum recording duration

Scenario: Local VAD model is available

• WHEN OWNER_VAD_PROVIDER=local
• THEN the runtime SHALL use the project-local sherpa-onnx VAD model rather than a raw energy threshold for live user utterance endpointing

Scenario: Hybrid VAD is configured

• WHEN OWNER_VAD_PROVIDER=hybrid
• THEN the runtime SHALL load both the project-local VAD model and an energy-threshold fallback and SHALL treat a frame as speech when either detector reports speech

Scenario: Wake acknowledgement prompt ordering

• WHEN local wake detection succeeds and OWNER_WAKE_ACK_TEXT is non-empty
• THEN the terminal SHALL emit wake hit, acknowledgement, question prompt, and recording states in that order so the user is cued to speak only after acknowledgement playback

MODIFIED Requirements

Requirement: Wake word detection

The system SHALL listen locally for the Chinese wake word “小杰小杰” with a dedicated local wake word provider before accepting user speech for a conversation turn, and the default live runtime implementation SHALL use a project-local KWS model rather than cloud ASR or the formal user STT provider for wake detection.

Scenario: Wake word is detected

• WHEN the user says “小杰小杰” and the local wakeword provider returns confidence above the configured threshold
• THEN the pipeline SHALL transition from wake listening to speech detection without invoking the configured STT provider for the wake audio

Scenario: Wake word is not detected

• WHEN background speech or noise does not match “小杰小杰”
• THEN the pipeline SHALL remain in wake listening and SHALL NOT invoke STT, LLM, or TTS

Scenario: Wake model fails

• WHEN the wakeword provider cannot load or process audio
• THEN the system SHALL report a wakeword error and SHALL NOT crash the desktop pet process

Scenario: Wake audio is isolated from user utterance

• WHEN the local wakeword provider detects “小杰小杰”
• THEN the runtime SHALL reset the user utterance recorder and SHALL NOT add wake audio or wake transcript text to the LLM conversation context

Requirement: Local speech model management

The system SHALL provide project-local speech model preparation and diagnostics for live wake/VAD/STT operation, storing downloaded model artifacts under models/ without committing them to Git.

Scenario: Models are downloaded

• WHEN the user runs python3.11 scripts/download_speech_models.py --dir models
• THEN the script SHALL create or update a project-local model directory with the files required by the configured wake, VAD, and STT providers

Scenario: Model check succeeds

• WHEN required wake, VAD, STT dependencies and model files are available
• THEN PYTHONPATH=src python3.11 -m owner_voice_pet model-check SHALL exit successfully and report the model directory and checked providers

Scenario: Model check fails

• WHEN sherpa-onnx is unavailable, a wake model file is missing, a VAD/STT model file is missing, or a model cannot be loaded
• THEN model-check SHALL fail with a structured model error and SHALL NOT start live microphone listening

Requirement: Live repeat voice runtime

The system SHALL provide a run-live command that performs real repeated voice conversation with local microphone input, local model wake detection, configured speech recognition and speech synthesis providers, cloud LLM reply generation, local speaker playback, and automatic return to standby.

Scenario: Live runtime starts in standby

• WHEN the user runs PYTHONPATH=src python3.11 -m owner_voice_pet run-live
• THEN the system SHALL load .env, validate required live dependencies, initialize local wake/audio/model providers, and enter a standby listening loop

Scenario: Live runtime completes two turns

• WHEN the user wakes the system with “小杰小杰”, asks a question, hears the reply, then wakes it again and asks another question
• THEN the system SHALL complete local wake, audible acknowledgement, recording, STT, transcript display, LLM, TTS, playback for both turns and SHALL return to standby after each turn

Scenario: Once mode completes one turn

• WHEN the user runs PYTHONPATH=src python3.11 -m owner_voice_pet run-live --once
• THEN the system SHALL run at most one local-wake-to-playback turn and exit after the turn completes or fails with a documented live error

Requirement: Live terminal state reporting

The live runtime SHALL emit concise Chinese terminal status messages for observable runtime states, including explicit user transcript output.

Scenario: Normal turn status

• WHEN a live turn succeeds
• THEN terminal output SHALL include states equivalent to standby, wake hit, recording, transcribing, transcript result, thinking, speaking, and returning to standby

Scenario: Recoverable error status

• WHEN a live turn encounters empty STT, LLM failure, TTS failure, or playback failure
• THEN terminal output SHALL include the failing stage and a stable error code or recoverable explanation

Requirement: Testability

The system SHALL be designed so each stage can be tested with mock providers, file-based audio fixtures, and fake live runtime components without requiring real devices in automated tests.

Scenario: Wake and STT separation is unit tested

• WHEN fake live providers run a turn with a wake frame and a user utterance
• THEN tests SHALL verify wake detection happens through the wake provider and the formal STT provider is called only for the user utterance

Scenario: Repeated runtime is unit tested

• WHEN fake live providers produce two deterministic turns
• THEN tests SHALL verify two wake detections, two STT calls, two LLM calls, two TTS calls, two playback calls, transcript output, and final return to standby

Scenario: Wake keyword does not pollute LLM input

• WHEN a wake frame contains “小杰小杰” and the later user utterance transcript is “第一问”
• THEN the current user message sent to the LLM SHALL be exactly “第一问” rather than a concatenation with the wake keyword