[Pipeline OpenSpec]:完成开源语音助手式重构规划,包含事件总线、TurnController和主说话人端点

This commit is contained in:
mkbk
2026-06-17 21:23:57 +08:00
parent 95e4996434
commit 1ee9f8ba95
4 changed files with 130 additions and 9 deletions
@@ -25,11 +25,15 @@
```text
run-live
-> AppConfig(.env)
-> VoiceAssistantPipeline
-> PipelineEventBus
-> TurnController
-> SoundDeviceAudioTransport
-> SherpaOnnxKeywordWakeWordProvider(local models/wake)
-> VadRecorder(user utterance, hybrid local+energy VAD by default)
-> AcknowledgeStage(local "我在")
-> CaptureStage(primary speaker endpoint by default)
-> CloudAsrSttProvider or SherpaOnnxSttProvider
-> TerminalRuntimeReporter.transcript()
-> DialogStage(session memory)
-> ConversationContext
-> OpenAICompatibleLlmProvider
-> CloudTtsProvider or MacSayTtsProvider
@@ -46,11 +50,12 @@ run-live
6. Wait for KWS wake event by feeding frames directly into wake provider.
7. On wake hit, reset wake stream and VAD recorder.
8. Record user utterance with VAD. The default provider is `hybrid`: project-local `sherpa-onnx` VAD remains the primary detector, and an energy threshold fallback prevents low microphone gain from being treated as no speech.
9. Transcribe user utterance.
10. Emit transcript to terminal.
11. Append user text and call LLM.
12. Synthesize/play reply.
13. Append assistant reply and return to standby.
9. When `OWNER_ENDPOINT_MODE=primary_speaker`, build a temporary per-turn speaker profile from the first valid user speech frames and end the capture when the primary speaker is absent for `OWNER_SPEAKER_ABSENT_MS`.
10. Transcribe user utterance.
11. Emit transcript event to terminal.
12. Append user text and call LLM.
13. Synthesize/play reply.
14. Append assistant reply and return to standby.
## Interfaces
@@ -73,6 +78,29 @@ class RuntimeReporter(Protocol):
def error(self, stage: str, code: str, message: str, *, turn_id: int | None = None) -> None: ...
```
### Pipeline events
```python
class PipelineEventBus:
def subscribe(self, listener): ...
def emit(self, event_type, *, turn_id=None, state=None, message="", payload=None): ...
```
Required event types are `pipeline_started`, `wake_listening`, `wake_detected`, `ack_started`, `capture_started`, `speech_started`, `speech_ended`, `stt_started`, `transcript_final`, `llm_started`, `tts_started`, `playback_finished`, `standby_resumed`, and `stage_error`.
### `TurnController`
```python
class TurnController:
def run_turn(self, turn_id: int) -> TurnResult: ...
```
The controller owns one turn state machine and delegates work to provider-backed stages. It does not write terminal text directly; it emits events only.
### Primary speaker endpoint
The first implementation is a per-turn heuristic endpoint, not persistent voiceprint recognition. It extracts local PCM features from speech frames and compares future frames against the profile. If profile creation fails because the user speech is too short or too quiet, capture falls back to existing VAD silence endpoint.
## Model Files
```text
@@ -99,6 +127,8 @@ models/
4. Empty user STT: existing `STT_EMPTY_TRANSCRIPT`.
5. Invalid wake provider config: `CONFIG_MISSING_VALUE`.
6. Real microphone VAD miss: default `OWNER_VAD_PROVIDER=hybrid` SHALL accept speech when either the local model or the energy fallback detects speech.
7. Primary speaker endpoint profile failure: fallback to VAD silence endpoint.
8. Pipeline stage failure: emit `stage_error`, recover to standby, and keep the process alive unless startup dependencies are missing.
## Testing Strategy
@@ -108,6 +138,8 @@ models/
4. Unit test LLM user content excludes wake keyword.
5. Unit test KWS provider missing model raises structured error.
6. Model-check test validates wake required files.
7. Pipeline event order test validates successful two-turn event sequence.
8. Primary speaker endpoint test validates that background noise or a later repeated utterance does not extend the current turn after the main speaker disappears.
## Migration