[实时转写]:完成录音期间转写显示,包含partial事件、本地streaming STT和终端实时反馈
This commit is contained in:
@@ -111,6 +111,16 @@ The first implementation is a per-turn heuristic endpoint, not persistent voicep
|
||||
4. 主说话人画像就绪后,`OWNER_SPEAKER_ABSENT_MS` 是结束正式问题采集的主条件;主说话人连续缺席达到该值后直接进入 STT,不再额外等待普通 VAD 最小时长。
|
||||
5. 普通 VAD 静音仍作为画像不足或音色特征不可用时的兜底,最大录音时长仍作为最终保护。
|
||||
|
||||
## Realtime Transcript Revision
|
||||
|
||||
录音期间新增 partial transcript 通道,用于解决“说话时看不到文字结果”的体验问题:
|
||||
|
||||
1. Pipeline 事件新增 `transcript_partial`。终端 reporter 将其显示为 `实时转写:<文本>`,最终结果仍显示为 `转写结果:<文本>`。
|
||||
2. partial transcript 只用于用户反馈,不写入 `ConversationContext`,不触发 LLM;LLM 仍只消费 final STT 结果。
|
||||
3. 当 final ASR/TTS 走 cloud 时,partial transcript 使用本地 `sherpa-onnx` streaming STT,避免对云端 ASR 高频请求。
|
||||
4. `VoiceAssistantPipeline` 在 `speech_started` 后把 capture 阶段已开始录音的帧 feed 给 realtime STT session;当 partial 文本变化时才 emit,避免刷屏。
|
||||
5. `OWNER_REALTIME_TRANSCRIPT_ENABLED=1` 默认启用;设置为 `0` 可临时回退到只显示 final transcript。
|
||||
|
||||
## Model Files
|
||||
|
||||
```text
|
||||
@@ -141,6 +151,7 @@ models/
|
||||
8. Pipeline stage failure: emit `stage_error`, recover to standby, and keep the process alive unless startup dependencies are missing.
|
||||
9. Playback drain misconfiguration: negative `OWNER_POST_PLAYBACK_DRAIN_MS` remains invalid; non-zero values are treated as explicit user tuning rather than default behavior.
|
||||
10. Speaker profile threshold misconfiguration: non-positive `OWNER_SPEAKER_PROFILE_MIN_MS` fails config validation.
|
||||
11. Realtime STT failure: startup model缺失按 `model-check` 暴露;capture 中 partial session 失败不得把 partial 文本写入上下文。
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
@@ -155,6 +166,8 @@ models/
|
||||
9. First utterance preservation test validates that frames immediately after ACK are not discarded by post-playback drain.
|
||||
10. Low-latency endpoint test validates that a short first question ends by primary speaker absence without waiting for a repeated second question.
|
||||
11. Transport batching test validates that queued SoundDevice frames are returned together.
|
||||
12. Partial transcript event test validates that realtime text appears after speech start and before final transcript.
|
||||
13. Context isolation test validates partial transcript does not enter LLM messages.
|
||||
|
||||
## Migration
|
||||
|
||||
|
||||
Reference in New Issue
Block a user