[Wake/VAD/STT 与 Live runtime]:完成真实重复语音运行,包含云端ASR/TTS开关、run-live和临时上下文测试
This commit is contained in:
+22
-7
@@ -1,7 +1,7 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Live repeat voice runtime
|
||||
The system SHALL provide a `run-live` command that performs real repeated voice conversation with local microphone input, local speech processing, cloud LLM reply generation, local TTS, local speaker playback, and automatic return to standby.
|
||||
The system SHALL provide a `run-live` command that performs real repeated voice conversation with local microphone input, configured speech recognition and speech synthesis providers, cloud LLM reply generation, local speaker playback, and automatic return to standby.
|
||||
|
||||
#### Scenario: Live runtime starts in standby
|
||||
- **WHEN** the user runs `PYTHONPATH=src python3.11 -m owner_voice_pet run-live`
|
||||
@@ -46,6 +46,21 @@ The system SHALL provide project-local speech model preparation and diagnostics
|
||||
- **WHEN** `sherpa-onnx` is unavailable, a model file is missing, or a model cannot be loaded
|
||||
- **THEN** `model-check` SHALL fail with a structured model error and SHALL NOT start live microphone listening
|
||||
|
||||
### Requirement: Configurable speech provider mode
|
||||
The system SHALL read `OWNER_SPEECH_PROVIDER` from `.env` to choose between cloud speech providers and local speech providers for first-version live runtime.
|
||||
|
||||
#### Scenario: Cloud speech provider is selected
|
||||
- **WHEN** `OWNER_SPEECH_PROVIDER=cloud`
|
||||
- **THEN** live ASR SHALL use the configured cloud model from `OWNER_ASR_MODEL`, live TTS SHALL use the configured cloud model from `OWNER_TTS_MODEL`, and the implementation SHALL default those models to `mimo-v2.5-asr` and `mimo-v2.5-tts`
|
||||
|
||||
#### Scenario: Local speech provider is selected
|
||||
- **WHEN** `OWNER_SPEECH_PROVIDER=local`
|
||||
- **THEN** live ASR/VAD SHALL use project-local speech model assets and local TTS SHALL use a local playback-capable provider
|
||||
|
||||
#### Scenario: Speech provider is invalid
|
||||
- **WHEN** `OWNER_SPEECH_PROVIDER` is neither `cloud` nor `local`
|
||||
- **THEN** startup validation SHALL fail with a structured configuration error
|
||||
|
||||
### Requirement: Live audio device readiness
|
||||
The system SHALL provide live audio device diagnostics and SHALL use `sounddevice` for first-version real microphone and speaker access.
|
||||
|
||||
@@ -123,11 +138,11 @@ The system SHALL use the local microphone as the first-version input Transport a
|
||||
- **WHEN** the microphone or speaker is missing, denied, unsupported, or inaccessible through `sounddevice`
|
||||
- **THEN** the system SHALL expose a recoverable Transport error with a stable error code and SHALL NOT silently fall back to fixture audio in live mode
|
||||
|
||||
### Requirement: Local STT transcription
|
||||
The system SHALL transcribe captured user utterances through a local STT provider, with `sherpa-onnx` as the first live implementation candidate and project-local models under `models/`.
|
||||
### Requirement: Configured STT transcription
|
||||
The system SHALL transcribe captured user utterances through the configured STT provider; cloud mode SHALL use NewAPI-compatible ASR and local mode SHALL use project-local `sherpa-onnx` model assets under `models/`.
|
||||
|
||||
#### Scenario: STT succeeds
|
||||
- **WHEN** local STT returns non-empty text for a captured live audio segment
|
||||
- **WHEN** configured STT returns non-empty text for a captured live audio segment
|
||||
- **THEN** the pipeline SHALL add the trimmed text as a user message to the current process conversation context
|
||||
|
||||
#### Scenario: STT returns empty text
|
||||
@@ -135,11 +150,11 @@ The system SHALL transcribe captured user utterances through a local STT provide
|
||||
- **THEN** the live runtime SHALL skip LLM invocation and return to standby with a recoverable status
|
||||
|
||||
#### Scenario: STT provider fails
|
||||
- **WHEN** the STT provider raises an error or cannot load its local model
|
||||
- **WHEN** the STT provider raises an error, cloud ASR fails, or local model loading fails
|
||||
- **THEN** the system SHALL emit an STT or model error code and SHALL recover to a state where future wake attempts are possible if startup can continue safely
|
||||
|
||||
### Requirement: Local TTS synthesis and playback
|
||||
The system SHALL synthesize assistant replies through a local TTS provider and SHALL play synthesized speech through the local system output path, with macOS `say/afplay` accepted as the first live implementation.
|
||||
### Requirement: Configured TTS synthesis and playback
|
||||
The system SHALL synthesize assistant replies through the configured TTS provider and SHALL play synthesized speech through the local system output path; cloud mode SHALL use NewAPI-compatible TTS and local mode SHALL use a local playback-capable TTS provider.
|
||||
|
||||
#### Scenario: Reply text is ready for speech
|
||||
- **WHEN** the LLM produces a non-empty reply for a live turn
|
||||
|
||||
Reference in New Issue
Block a user