[本地语音降噪]：完成本地语音链路和噪音过滤，包含降噪模型、本地ASR和实时字幕稳定策略

2026-06-17 22:55:33 +08:00
parent a77a172412
commit 8c75fc5baf
22 changed files with 803 additions and 64 deletions
@@ -64,6 +64,47 @@ The live runtime SHALL display recognized user utterance text in the terminal bo
 - **WHEN** STT returns empty text, punctuation-only text, or an invalid transcript
 - **THEN** the runtime SHALL NOT emit a misleading transcript as valid user input and SHALL recover to standby without invoking the LLM

+### Requirement: Local voice chain and noise filtering
+The live runtime SHALL run wake, VAD, STT, realtime transcript, TTS, and capture-stage noise filtering locally by default, while continuing to send only the final user text and session history to the configured cloud LLM.
+
+#### Scenario: Default speech provider is local
+- **WHEN** the user starts `run-live` without overriding `OWNER_SPEECH_PROVIDER`
+- **THEN** the runtime SHALL use local sherpa-onnx STT and local macOS TTS
+- **AND** it SHALL NOT call cloud ASR or cloud TTS providers during the turn
+
+#### Scenario: Capture audio is denoised before downstream stages
+- **WHEN** `OWNER_NOISE_FILTER_ENABLED=1` and the user speaks after wake acknowledgement
+- **THEN** the capture stage SHALL pass microphone frames through the configured local denoiser before VAD analysis, realtime partial STT, and final STT segment assembly
+- **AND** all three downstream consumers SHALL observe the same denoised frame content for a given captured frame
+
+#### Scenario: Wake remains raw by default
+- **WHEN** `OWNER_WAKE_DENOISE_ENABLED` is unset or `0`
+- **THEN** wake word detection SHALL receive raw microphone frames rather than denoised frames
+
+#### Scenario: Denoiser model is missing
+- **WHEN** `models/denoise/gtcrn_simple.onnx` is absent
+- **THEN** `model-check` SHALL fail with a structured missing model error
+- **AND** `run-live` SHALL NOT enter live microphone listening as if noise filtering were available
+
+#### Scenario: Denoiser runtime fails
+- **WHEN** the local denoiser raises an error while processing a formal user utterance frame
+- **THEN** the current turn SHALL emit a stage error and recover to standby
+- **AND** the runtime SHALL NOT invoke final STT, LLM, or TTS for that failed turn
+
+#### Scenario: Local ASR uses the 2025 CTC model by default
+- **WHEN** the default manifest is written by `download_speech_models.py`
+- **THEN** the STT provider SHALL be declared as `sherpa-onnx-streaming-zipformer2-ctc`
+- **AND** required STT files SHALL be `tokens.txt` and `model.int8.onnx`, not transducer encoder/decoder/joiner files
+
+#### Scenario: Realtime partial filters noise
+- **WHEN** local streaming STT emits a single meaningful character or a short transient regression
+- **THEN** the terminal reporter SHALL NOT display that text as `实时转写`
+- **AND** final STT SHALL remain responsible for the text sent to the LLM
+
+#### Scenario: Audio privacy is preserved
+- **WHEN** local noise filtering and local STT run during capture
+- **THEN** raw PCM, denoised PCM, VAD features, partial transcript metadata, and temporary speaker profile data SHALL remain process-local and SHALL NOT be sent to the cloud LLM
+
 ### Requirement: Wake acknowledgement before recording
 The live runtime SHALL provide an audible local acknowledgement after local wake detection and before it starts recording the user's formal question.

@@ -120,14 +161,14 @@ The system SHALL provide project-local speech model preparation and diagnostics

 #### Scenario: Models are downloaded
 - **WHEN** the user runs `python3.11 scripts/download_speech_models.py --dir models`
- **THEN** the script SHALL create or update a project-local model directory with the files required by the configured wake, VAD, and STT providers
+- **THEN** the script SHALL create or update a project-local model directory with the files required by the configured wake, VAD, STT, and denoiser providers

 #### Scenario: Model check succeeds
- **WHEN** required wake, VAD, STT dependencies and model files are available
+- **WHEN** required wake, VAD, STT, denoiser dependencies and model files are available
 - **THEN** `PYTHONPATH=src python3.11 -m owner_voice_pet model-check` SHALL exit successfully and report the model directory and checked providers

 #### Scenario: Model check fails
- **WHEN** `sherpa-onnx` is unavailable, a wake model file is missing, a VAD/STT model file is missing, or a model cannot be loaded
+- **WHEN** `sherpa-onnx` is unavailable, a wake model file is missing, a VAD/STT/denoiser model file is missing, or a model cannot be loaded
 - **THEN** `model-check` SHALL fail with a structured model error and SHALL NOT start live microphone listening

 ### Requirement: Live repeat voice runtime