[实时转写]:完成录音期间转写显示,包含partial事件、本地streaming STT和终端实时反馈

This commit is contained in:
mkbk
2026-06-17 22:16:56 +08:00
parent a64bb86da4
commit a77a172412
16 changed files with 331 additions and 17 deletions
@@ -111,6 +111,16 @@ The first implementation is a per-turn heuristic endpoint, not persistent voicep
4. 主说话人画像就绪后,`OWNER_SPEAKER_ABSENT_MS` 是结束正式问题采集的主条件;主说话人连续缺席达到该值后直接进入 STT,不再额外等待普通 VAD 最小时长。
5. 普通 VAD 静音仍作为画像不足或音色特征不可用时的兜底,最大录音时长仍作为最终保护。
## Realtime Transcript Revision
录音期间新增 partial transcript 通道,用于解决“说话时看不到文字结果”的体验问题:
1. Pipeline 事件新增 `transcript_partial`。终端 reporter 将其显示为 `实时转写:<文本>`,最终结果仍显示为 `转写结果:<文本>`
2. partial transcript 只用于用户反馈,不写入 `ConversationContext`,不触发 LLMLLM 仍只消费 final STT 结果。
3. 当 final ASR/TTS 走 cloud 时,partial transcript 使用本地 `sherpa-onnx` streaming STT,避免对云端 ASR 高频请求。
4. `VoiceAssistantPipeline``speech_started` 后把 capture 阶段已开始录音的帧 feed 给 realtime STT session;当 partial 文本变化时才 emit,避免刷屏。
5. `OWNER_REALTIME_TRANSCRIPT_ENABLED=1` 默认启用;设置为 `0` 可临时回退到只显示 final transcript。
## Model Files
```text
@@ -141,6 +151,7 @@ models/
8. Pipeline stage failure: emit `stage_error`, recover to standby, and keep the process alive unless startup dependencies are missing.
9. Playback drain misconfiguration: negative `OWNER_POST_PLAYBACK_DRAIN_MS` remains invalid; non-zero values are treated as explicit user tuning rather than default behavior.
10. Speaker profile threshold misconfiguration: non-positive `OWNER_SPEAKER_PROFILE_MIN_MS` fails config validation.
11. Realtime STT failure: startup model缺失按 `model-check` 暴露;capture 中 partial session 失败不得把 partial 文本写入上下文。
## Testing Strategy
@@ -155,6 +166,8 @@ models/
9. First utterance preservation test validates that frames immediately after ACK are not discarded by post-playback drain.
10. Low-latency endpoint test validates that a short first question ends by primary speaker absence without waiting for a repeated second question.
11. Transport batching test validates that queued SoundDevice frames are returned together.
12. Partial transcript event test validates that realtime text appears after speech start and before final transcript.
13. Context isolation test validates partial transcript does not enter LLM messages.
## Migration