Owner/openspec/changes/complete-live-repeat-voice-runtime/design.md

# 补齐真实可重复对话的实时语音运行版技术设计

## Context

`add-voice-pet-pipeline` 已归档并生成 `voice-pet-pipeline` 主规范。当前仓库已有 Python 包、CLI acceptance、`.env` 配置读取、Provider 协议、mock 测试、桌宠资产和 OpenSpec 主规范，但真实运行能力仍不完整：没有 `run-live` 常驻命令，没有真实麦克风输入和扬声器输出，没有本地模型下载/检查，没有证明同一运行进程内可以连续多轮携带历史。

本设计聚焦第一版“无 GUI 的真实实时语音版”。GUI 桌宠窗口、长期记忆、播放中打断、跨设备 Transport 和云端 STT/TTS 均不在本变更范围内。

## Goals

1. 提供 `owner_voice_pet run-live`，默认常驻重复语音对话。
2. 提供 `--once`，便于单轮真实验收和自动化测试。
3. 使用 `.env` 作为默认配置来源，避免要求用户导出 shell 环境变量。
4. 使用 `sounddevice` 真实读取本机麦克风。
5. 使用 `sherpa-onnx` 本地 VAD/STT 模型，模型存放在项目 `models/`。
6. 使用 macOS `say/afplay` 先保证本地 TTS 真实播出。
7. 在同一个 `run-live` 进程内保留最近 user/assistant 历史，进程退出即丢弃。
8. 新增自动化测试证明两轮 runtime、临时上下文、进程隔离和错误恢复。

## Non-Goals

1. 不实现 GUI 桌宠窗口实时联动。
2. 不实现长期记忆、数据库或跨进程会话恢复。
3. 不实现播放中用户打断。
4. 不把模型文件提交到 Git。
5. 不把 acceptance fixture 当作 `run-live` 默认输入。
6. 不要求自动化测试依赖真实麦克风、扬声器或真实模型。

## Architecture

```text
CLI
  -> AppConfig.from_dotenv(".env")
  -> LiveVoiceRuntimeFactory
    -> SoundDeviceAudioTransport
    -> WakeDetector("小杰小杰")
    -> SherpaOnnxVadProvider
    -> SherpaOnnxSttProvider
    -> OpenAI/NewAPI LlmProvider
    -> MacOsSayTtsProvider
    -> ConversationContext(empty per process)
    -> TerminalRuntimeReporter

LiveVoiceRuntime.run()
  -> standby loop
  -> wait_wake()
  -> record_utterance_with_vad()
  -> transcribe()
  -> build messages with temporary history
  -> generate reply
  -> synthesize/play
  -> append assistant
  -> recover to standby
```

## Runtime Lifecycle

1. Parse CLI args.
2. Load `.env` and merge only explicitly supported defaults.
3. Validate LLM config.
4. Validate models when live mode requires local VAD/STT.
5. Validate audio devices.
6. Create one `ConversationContext` for the process.
7. Enter loop:
   1. report `待机`;
   2. wait for wake word;
   3. report `唤醒命中`;
   4. record utterance with VAD;
   5. report `转写中`;
   6. transcribe;
   7. append user to context;
   8. report `思考中`;
   9. call LLM with system + retained history + current user;
   10. report `播放中`;
   11. synthesize and play;
   12. append assistant to context;
   13. report `恢复待机`;
   14. break only if `--once`.
8. On Ctrl-C, close audio streams and exit without persisting context.

## Interfaces

### `LiveVoiceRuntime`

```python
class LiveVoiceRuntime:
    def run(self, *, once: bool = False) -> RuntimeSummary: ...
    def run_turn(self, turn_id: int) -> TurnResult: ...
    def shutdown(self) -> None: ...
```

`run()` owns the loop. `run_turn()` owns one wake -> record -> STT -> LLM -> TTS/play -> standby cycle. `shutdown()` closes transport streams and removes temporary TTS files.

### `RuntimeReporter`

```python
class RuntimeReporter(Protocol):
    def status(self, state: str, message: str, *, turn_id: int | None = None) -> None: ...
    def error(self, stage: str, code: str, message: str, *, turn_id: int | None = None) -> None: ...
```

The default implementation prints short Chinese status lines to stdout/stderr and redacts credentials.

### `SpeechModelManifest`

```python
@dataclass(frozen=True)
class SpeechModelManifest:
    models_dir: Path
    vad_model_path: Path
    stt_model_dir: Path
    sample_rate: int = 16000
```

The manifest is used by `download_speech_models.py`, `model-check`, and live provider construction.

### `SoundDeviceAudioTransport`

```python
class SoundDeviceAudioTransport:
    def open_input(self, sample_rate: int, channels: int, device_id: int | None = None) -> None: ...
    def read_chunk(self, timeout_s: float) -> AudioFrame: ...
    def record_until_endpoint(self, vad: VadProvider, limits: RecordingLimits) -> AudioSegment: ...
    def play_pcm(self, segment: AudioSegment) -> PlaybackResult: ...
    def close(self) -> None: ...
```

The implementation keeps `sounddevice` imports optional so unit tests and non-audio commands still import without audio dependencies.

### `MacOsSayTtsProvider`

```python
class MacOsSayTtsProvider:
    def synthesize_to_file(self, text: str) -> Path: ...
    def synthesize(self, text: str) -> AudioSegment: ...
```

First version may return a file-backed audio segment or call `say -o <tmpfile> --data-format=LEF32@16000`. Playback may use `afplay`.

## State Machine

```text
initializing
  -> standby
  -> wake_listening
  -> wake_hit
  -> recording
  -> transcribing
  -> thinking
  -> speaking
  -> recovering_to_standby
  -> standby

recoverable turn error
  -> error
  -> recovering_to_standby
  -> standby

fatal startup error
  -> startup_failed
  -> exit
```

Fatal startup errors include invalid `.env`, missing required model directory, no audio input device, and no usable live audio dependency. Recoverable turn errors include no speech after wake, empty STT text, LLM timeout, TTS failure, and playback failure.

## Temporary Context Design

Each `LiveVoiceRuntime` receives a newly constructed `ConversationContext`. This object:

1. starts with only system prompt behavior;
2. appends user text after valid STT;
3. appends assistant text after non-empty LLM reply;
4. builds messages for every LLM request using retained history;
5. truncates by `context_max_messages` and `context_max_chars`;
6. is never serialized;
7. is never stored in a module-level global;
8. is lost when the process exits.

Test requirements:

1. A fake two-turn runtime must prove the second LLM call includes first turn user/assistant.
2. A new runtime instance must start with no prior messages.
3. Context truncation must still preserve current user input.

## Wake/VAD/STT Design

First implementation can use one of two local strategies:

1. Preferred: sherpa-onnx VAD for endpointing plus sherpa-onnx STT for short wake windows and user utterances.
2. Future replacement: dedicated local KWS provider for “小杰小杰”.

The implementation must not send raw audio to the cloud for wake, VAD, or STT. If a dedicated wake model is unavailable, wake detection may transcribe short local segments and search for the normalized wake word.

## Model Download Design

`scripts/download_speech_models.py --dir models` creates:

```text
models/
  manifest.json
  vad/
    ...
  stt/
    ...
```

`manifest.json` records model names, source URLs, expected key files, and sample rate. The script must be idempotent: if files exist and pass checks, it should skip or report already present.

## Error Handling

All live errors must convert to structured codes:

1. `LIVE_CONFIG_INVALID`
2. `LIVE_MODEL_UNAVAILABLE`
3. `LIVE_AUDIO_DEVICE_UNAVAILABLE`
4. `LIVE_WAKE_TIMEOUT`
5. `LIVE_NO_SPEECH`
6. `LIVE_STT_EMPTY`
7. `LIVE_LLM_FAILED`
8. `LIVE_TTS_FAILED`
9. `LIVE_PLAYBACK_FAILED`
10. `LIVE_INTERRUPTED`

Startup errors return non-zero CLI exit codes. Turn-level errors in default loop log and return to standby. In `--once`, turn-level errors return non-zero except no-speech or empty transcript may return a documented recoverable code.

## Testing Strategy

1. Unit tests use fake audio/provider objects and do not require live audio devices.
2. `test_live_runtime_repeats_two_turns` verifies two STT, two LLM, two TTS, two playback events and final standby.
3. `test_live_runtime_temporary_context` verifies second LLM messages include first user/assistant.
4. `test_live_runtime_process_local_context` verifies a new runtime has empty history.
5. `test_model_check` verifies missing/valid manifest behavior.
6. `test_device_check` mocks sounddevice device query.
7. Existing tests for acceptance, assets, security, config, transport, wake/VAD/STT remain passing.

## Migration

No persistent data migration is needed. Existing `.env` remains valid. New optional `.env` keys receive defaults. Existing acceptance command remains available.

## Open Questions

1. Which exact sherpa-onnx Chinese STT model gives the best latency/accuracy tradeoff on this Mac.
2. Whether future wake detection should use a dedicated KWS model instead of short-window STT.
3. Whether TTS should later move from macOS `say` to sherpa-onnx TTS for consistent voice and offline packaging.