Owner/openspec/changes/separate-wake-and-realtime-transcript/design.md

# 独立本地唤醒与终端转写显示设计

## Context

真实运行日志显示，当前 `run-live` 会先截取语音段并使用 STT 判断是否包含“小杰小杰”。该实现把 wake detection 与 user utterance transcription 绑定在一起，导致唤醒慢、唤醒词污染正式问题、终端输出难以区分 STT 和 LLM 阶段。本设计把 wake word detection 升级为本地 KWS 模型路径，并把正式问题转写作为独立可见事件输出。

## Goals

1. 唤醒词“小杰小杰”由本地模型检测。
2. wake 阶段不调用云 ASR，也不调用正式 STT provider。
3. wake 命中后才开始正式问题录音和 STT。
4. 终端在 LLM 前显示正式问题转写文本。
5. 模型下载和检查覆盖 wake KWS、VAD、STT。
6. 自动化测试证明 wake/STT 分离、重复对话、上下文和错误恢复仍正常。

## Non-Goals

1. 不实现 GUI 桌宠窗口。
2. 不实现跨进程长期记忆。
3. 不在本阶段实现逐字 partial ASR 字幕；本阶段先保证正式问题 STT 完成后立即可见，且出现在 LLM 前。
4. 不把连续麦克风流上传到云端。

## Architecture

```text
run-live
  -> AppConfig(.env)
  -> VoiceAssistantPipeline
  -> PipelineEventBus
  -> TurnController
  -> SoundDeviceAudioTransport
  -> SherpaOnnxKeywordWakeWordProvider(local models/wake)
  -> AcknowledgeStage(local "我在")
  -> CaptureStage(primary speaker endpoint by default)
  -> CloudAsrSttProvider or SherpaOnnxSttProvider
  -> DialogStage(session memory)
  -> ConversationContext
  -> OpenAICompatibleLlmProvider
  -> CloudTtsProvider or MacSayTtsProvider
  -> speaker playback
```

## Runtime Lifecycle

1. Load config.
2. Validate `OWNER_WAKE_PROVIDER=local_kws`.
3. Load KWS model from `OWNER_SPEECH_MODELS_DIR`.
4. Load VAD, STT, LLM, TTS.
5. Open microphone stream.
6. Wait for KWS wake event by feeding frames directly into wake provider.
7. On wake hit, reset wake stream and VAD recorder.
8. Record user utterance with VAD. The default provider is `hybrid`: project-local `sherpa-onnx` VAD remains the primary detector, and an energy threshold fallback prevents low microphone gain from being treated as no speech.
9. When `OWNER_ENDPOINT_MODE=primary_speaker`, build a temporary per-turn speaker profile from the first valid user speech frames and end the capture when the primary speaker is absent for `OWNER_SPEAKER_ABSENT_MS`.
10. Transcribe user utterance.
11. Emit transcript event to terminal.
12. Append user text and call LLM.
13. Synthesize/play reply.
14. Append assistant reply and return to standby.

## Interfaces

### `SherpaOnnxKeywordWakeWordProvider`

```python
class SherpaOnnxKeywordWakeWordProvider:
    def __init__(self, models_dir, keyword, keywords_file=None, threshold=0.25, score=1.0, sherpa_module=None): ...
    def load(self) -> None: ...
    def detect(self, frame: AudioFrame) -> WakeEvent | None: ...
    def reset(self) -> None: ...
```

### `RuntimeReporter`

```python
class RuntimeReporter(Protocol):
    def status(self, state: str, message: str, *, turn_id: int | None = None) -> None: ...
    def transcript(self, text: str, *, final: bool, turn_id: int | None = None) -> None: ...
    def error(self, stage: str, code: str, message: str, *, turn_id: int | None = None) -> None: ...
```

### Pipeline events

```python
class PipelineEventBus:
    def subscribe(self, listener): ...
    def emit(self, event_type, *, turn_id=None, state=None, message="", payload=None): ...
```

Required event types are `pipeline_started`, `wake_listening`, `wake_detected`, `ack_started`, `capture_started`, `speech_started`, `speech_ended`, `stt_started`, `transcript_final`, `llm_started`, `tts_started`, `playback_finished`, `standby_resumed`, and `stage_error`.

### `TurnController`

```python
class TurnController:
    def run_turn(self, turn_id: int) -> TurnResult: ...
```

The controller owns one turn state machine and delegates work to provider-backed stages. It does not write terminal text directly; it emits events only.

### Primary speaker endpoint

The first implementation is a per-turn heuristic endpoint, not persistent voiceprint recognition. It extracts local PCM features from speech frames and compares future frames against the profile. If profile creation fails because the user speech is too short or too quiet, capture falls back to existing VAD silence endpoint.

## Model Files

```text
models/
  manifest.json
  wake/
    sherpa-onnx-kws-zipformer-wenetspeech-3.3M-2024-01-01-mobile/
      tokens.txt
      encoder-epoch-12-avg-2-chunk-16-left-64.int8.onnx
      decoder-epoch-12-avg-2-chunk-16-left-64.onnx
      joiner-epoch-12-avg-2-chunk-16-left-64.int8.onnx
    keywords.txt
  vad/
    silero_vad.onnx
  stt/
    sherpa-onnx-streaming-zipformer-zh-14M-2023-02-23/
```

## Error Handling

1. Missing KWS model: `WAKE_MODEL_MISSING`.
2. KWS load failure: `WAKE_MODEL_LOAD_FAILED`.
3. KWS runtime failure: `WAKE_MODEL_LOAD_FAILED` with retryable true.
4. Empty user STT: existing `STT_EMPTY_TRANSCRIPT`.
5. Invalid wake provider config: `CONFIG_MISSING_VALUE`.
6. Real microphone VAD miss: default `OWNER_VAD_PROVIDER=hybrid` SHALL accept speech when either the local model or the energy fallback detects speech.
7. Primary speaker endpoint profile failure: fallback to VAD silence endpoint.
8. Pipeline stage failure: emit `stage_error`, recover to standby, and keep the process alive unless startup dependencies are missing.

## Testing Strategy

1. Unit test fake wake provider detects wake without STT calls.
2. Unit test repeated runtime runs two turns with exactly two STT calls.
3. Unit test terminal reporter records transcript before LLM stage.
4. Unit test LLM user content excludes wake keyword.
5. Unit test KWS provider missing model raises structured error.
6. Model-check test validates wake required files.
7. Pipeline event order test validates successful two-turn event sequence.
8. Primary speaker endpoint test validates that background noise or a later repeated utterance does not extend the current turn after the main speaker disappears.

## Migration

No database migration. Users should run:

```bash
python3.11 scripts/download_speech_models.py --dir models
.venv/bin/python -m owner_voice_pet model-check --models-dir models
```

Existing `.env` remains valid because new wake keys have defaults.