## ADDED Requirements ### Requirement: Live assistant pipeline events The live runtime SHALL execute as a stage-based voice assistant pipeline with stable events for every externally observable stage. #### Scenario: Successful live turn emits ordered events - **WHEN** a live turn completes successfully - **THEN** the event stream SHALL include wake listening, wake detected, acknowledgement, capture started, speech started, speech ended, STT started, final transcript, LLM started, TTS started, playback finished, and standby resumed in that order #### Scenario: Terminal reporting consumes events - **WHEN** the terminal reporter prints user-facing live status - **THEN** it SHALL derive the status from pipeline events rather than duplicating stage logic ### Requirement: Primary speaker endpointing The live runtime SHALL support a per-turn primary speaker endpoint mode that ends the user utterance when the speaker who started the utterance is no longer present. #### Scenario: Primary speaker disappears - **WHEN** the user starts speaking after wake and their temporary per-turn voice profile is established - **THEN** capture SHALL end after the configured primary speaker absence duration even if background noise or non-primary speech remains #### Scenario: Temporary profile is insufficient - **WHEN** the live runtime cannot establish a reliable per-turn voice profile because speech is too short or too quiet - **THEN** capture SHALL fall back to configured VAD silence endpointing and SHALL NOT hang indefinitely #### Scenario: No persistent voiceprint - **WHEN** the process exits - **THEN** no primary speaker profile SHALL be stored or reused across runs ### Requirement: Low latency capture and first utterance preservation The live runtime SHALL preserve the start of the user's formal utterance after wake acknowledgement and SHALL end capture promptly after the primary speaker stops. #### Scenario: Acknowledgement finishes before the user speaks - **WHEN** acknowledgement playback completes and the runtime enters capture - **THEN** the runtime SHALL NOT perform an additional timed microphone read-and-discard window before listening for the user's formal question #### Scenario: Queued microphone frames are available - **WHEN** the live microphone transport has multiple frames queued - **THEN** the transport SHALL return the available batch to the pipeline so wake and capture stages do not process stale audio one frame at a time #### Scenario: Primary speaker profile becomes ready quickly - **WHEN** the user starts the formal utterance after wake - **THEN** the primary speaker endpoint SHALL use an independent profile readiness threshold no greater than the configured `OWNER_SPEAKER_PROFILE_MIN_MS` rather than waiting for generic VAD minimum duration #### Scenario: User stops after a short first question - **WHEN** the primary speaker profile is ready and the primary speaker disappears for the configured absence duration - **THEN** capture SHALL close the utterance and proceed to STT without waiting for a second repeated question or generic VAD minimum duration ### Requirement: Realtime transcript terminal output The live runtime SHALL display recognized user utterance text in the terminal both during recording when partial transcript is available and after final STT succeeds before the LLM request is sent. #### Scenario: User utterance partial is available during recording - **WHEN** the user is speaking after wake and the local realtime STT session produces a changed partial transcript - **THEN** the terminal output SHALL include a realtime transcript message containing the partial text before final STT completes #### Scenario: Partial transcript is displayed - **WHEN** a partial transcript is emitted - **THEN** it SHALL be treated as user-visible feedback only and SHALL NOT be appended to the conversation context or sent to the LLM #### Scenario: User utterance is transcribed - **WHEN** a live turn captures a user utterance and STT returns non-empty text - **THEN** the terminal output SHALL include a transcript message containing the recognized text before the thinking/LLM status is emitted #### Scenario: Transcript is empty - **WHEN** STT returns empty text, punctuation-only text, or an invalid transcript - **THEN** the runtime SHALL NOT emit a misleading transcript as valid user input and SHALL recover to standby without invoking the LLM ### Requirement: Wake acknowledgement before recording The live runtime SHALL provide an audible local acknowledgement after local wake detection and before it starts recording the user's formal question. #### Scenario: Wake is detected - **WHEN** the local wakeword provider detects “小杰小杰” - **THEN** the runtime SHALL play a short acknowledgement such as “我在” before entering the user utterance recording state #### Scenario: Acknowledgement playback finishes - **WHEN** the acknowledgement playback completes - **THEN** the runtime SHALL clear buffered microphone input captured during acknowledgement playback before starting VAD recording for the user's question and SHALL NOT discard additional post-acknowledgement user speech by default ### Requirement: Fast user utterance endpointing The live runtime SHALL use project-local VAD capability with an energy-threshold fallback by default for user utterance endpoint detection and SHALL expose configurable silence timing so the recording stops promptly after the user stops speaking. #### Scenario: User stops speaking after wake - **WHEN** VAD observes the configured continuous silence duration after a started utterance - **THEN** the runtime SHALL close the utterance segment and proceed to STT without waiting for the maximum recording duration #### Scenario: Local VAD model is available - **WHEN** `OWNER_VAD_PROVIDER=local` - **THEN** the runtime SHALL use the project-local `sherpa-onnx` VAD model rather than a raw energy threshold for live user utterance endpointing #### Scenario: Hybrid VAD is configured - **WHEN** `OWNER_VAD_PROVIDER=hybrid` - **THEN** the runtime SHALL load both the project-local VAD model and an energy-threshold fallback and SHALL treat a frame as speech when either detector reports speech #### Scenario: Wake acknowledgement prompt ordering - **WHEN** local wake detection succeeds and `OWNER_WAKE_ACK_TEXT` is non-empty - **THEN** the terminal SHALL emit wake hit, acknowledgement, question prompt, and recording states in that order so the user is cued to speak only after acknowledgement playback ## MODIFIED Requirements ### Requirement: Wake word detection The system SHALL listen locally for the Chinese wake word “小杰小杰” with a dedicated local wake word provider before accepting user speech for a conversation turn, and the default live runtime implementation SHALL use a project-local KWS model rather than cloud ASR or the formal user STT provider for wake detection. #### Scenario: Wake word is detected - **WHEN** the user says “小杰小杰” and the local wakeword provider returns confidence above the configured threshold - **THEN** the pipeline SHALL transition from wake listening to speech detection without invoking the configured STT provider for the wake audio #### Scenario: Wake word is not detected - **WHEN** background speech or noise does not match “小杰小杰” - **THEN** the pipeline SHALL remain in wake listening and SHALL NOT invoke STT, LLM, or TTS #### Scenario: Wake model fails - **WHEN** the wakeword provider cannot load or process audio - **THEN** the system SHALL report a wakeword error and SHALL NOT crash the desktop pet process #### Scenario: Wake audio is isolated from user utterance - **WHEN** the local wakeword provider detects “小杰小杰” - **THEN** the runtime SHALL reset the user utterance recorder and SHALL NOT add wake audio or wake transcript text to the LLM conversation context ### Requirement: Local speech model management The system SHALL provide project-local speech model preparation and diagnostics for live wake/VAD/STT operation, storing downloaded model artifacts under `models/` without committing them to Git. #### Scenario: Models are downloaded - **WHEN** the user runs `python3.11 scripts/download_speech_models.py --dir models` - **THEN** the script SHALL create or update a project-local model directory with the files required by the configured wake, VAD, and STT providers #### Scenario: Model check succeeds - **WHEN** required wake, VAD, STT dependencies and model files are available - **THEN** `PYTHONPATH=src python3.11 -m owner_voice_pet model-check` SHALL exit successfully and report the model directory and checked providers #### Scenario: Model check fails - **WHEN** `sherpa-onnx` is unavailable, a wake model file is missing, a VAD/STT model file is missing, or a model cannot be loaded - **THEN** `model-check` SHALL fail with a structured model error and SHALL NOT start live microphone listening ### Requirement: Live repeat voice runtime The system SHALL provide a `run-live` command that performs real repeated voice conversation with local microphone input, local model wake detection, configured speech recognition and speech synthesis providers, cloud LLM reply generation, local speaker playback, and automatic return to standby. #### Scenario: Live runtime starts in standby - **WHEN** the user runs `PYTHONPATH=src python3.11 -m owner_voice_pet run-live` - **THEN** the system SHALL load `.env`, validate required live dependencies, initialize local wake/audio/model providers, and enter a standby listening loop #### Scenario: Live runtime completes two turns - **WHEN** the user wakes the system with “小杰小杰”, asks a question, hears the reply, then wakes it again and asks another question - **THEN** the system SHALL complete local wake, audible acknowledgement, recording, STT, transcript display, LLM, TTS, playback for both turns and SHALL return to standby after each turn #### Scenario: Once mode completes one turn - **WHEN** the user runs `PYTHONPATH=src python3.11 -m owner_voice_pet run-live --once` - **THEN** the system SHALL run at most one local-wake-to-playback turn and exit after the turn completes or fails with a documented live error ### Requirement: Live terminal state reporting The live runtime SHALL emit concise Chinese terminal status messages for observable runtime states, including explicit user transcript output. #### Scenario: Normal turn status - **WHEN** a live turn succeeds - **THEN** terminal output SHALL include states equivalent to standby, wake hit, recording, transcribing, transcript result, thinking, speaking, and returning to standby #### Scenario: Recoverable error status - **WHEN** a live turn encounters empty STT, LLM failure, TTS failure, or playback failure - **THEN** terminal output SHALL include the failing stage and a stable error code or recoverable explanation ### Requirement: Testability The system SHALL be designed so each stage can be tested with mock providers, file-based audio fixtures, and fake live runtime components without requiring real devices in automated tests. #### Scenario: Wake and STT separation is unit tested - **WHEN** fake live providers run a turn with a wake frame and a user utterance - **THEN** tests SHALL verify wake detection happens through the wake provider and the formal STT provider is called only for the user utterance #### Scenario: Repeated runtime is unit tested - **WHEN** fake live providers produce two deterministic turns - **THEN** tests SHALL verify two wake detections, two STT calls, two LLM calls, two TTS calls, two playback calls, transcript output, and final return to standby #### Scenario: Wake keyword does not pollute LLM input - **WHEN** a wake frame contains “小杰小杰” and the later user utterance transcript is “第一问” - **THEN** the current user message sent to the LLM SHALL be exactly “第一问” rather than a concatenation with the wake keyword