[低延迟端点]:完成首句保留和快速结束修正,包含ACK缓冲策略、批量读帧和主说话人端点回归测试

This commit is contained in:
mkbk
2026-06-17 22:04:25 +08:00
parent e74ec28e7d
commit a64bb86da4
15 changed files with 136 additions and 15 deletions
@@ -101,6 +101,16 @@ The controller owns one turn state machine and delegates work to provider-backed
The first implementation is a per-turn heuristic endpoint, not persistent voiceprint recognition. It extracts local PCM features from speech frames and compares future frames against the profile. If profile creation fails because the user speech is too short or too quiet, capture falls back to existing VAD silence endpoint.
## Low Latency Capture Revision
真人运行反馈显示,当前 capture 仍可能把用户第一句话开头吞掉或需要用户重复提问才能结束。本修正把低延迟 capture 作为 pipeline 内部约束:
1. `OWNER_POST_PLAYBACK_DRAIN_MS` 默认改为 `0`。ACK 播放结束后仅清掉播放期间积压在输入队列中的帧,不再主动等待并丢弃后续音频。
2. `SoundDeviceAudioTransport.read_frames()` 在拿到首帧后立即返回队列中所有可用帧,避免真实麦克风回调积压时 pipeline 逐帧追赶。
3. `OWNER_SPEAKER_PROFILE_MIN_MS` 控制临时主说话人画像最低就绪语音长度,默认 `120` ms,不再复用 `OWNER_VAD_MIN_DURATION_MS`
4. 主说话人画像就绪后,`OWNER_SPEAKER_ABSENT_MS` 是结束正式问题采集的主条件;主说话人连续缺席达到该值后直接进入 STT,不再额外等待普通 VAD 最小时长。
5. 普通 VAD 静音仍作为画像不足或音色特征不可用时的兜底,最大录音时长仍作为最终保护。
## Model Files
```text
@@ -129,6 +139,8 @@ models/
6. Real microphone VAD miss: default `OWNER_VAD_PROVIDER=hybrid` SHALL accept speech when either the local model or the energy fallback detects speech.
7. Primary speaker endpoint profile failure: fallback to VAD silence endpoint.
8. Pipeline stage failure: emit `stage_error`, recover to standby, and keep the process alive unless startup dependencies are missing.
9. Playback drain misconfiguration: negative `OWNER_POST_PLAYBACK_DRAIN_MS` remains invalid; non-zero values are treated as explicit user tuning rather than default behavior.
10. Speaker profile threshold misconfiguration: non-positive `OWNER_SPEAKER_PROFILE_MIN_MS` fails config validation.
## Testing Strategy
@@ -140,6 +152,9 @@ models/
6. Model-check test validates wake required files.
7. Pipeline event order test validates successful two-turn event sequence.
8. Primary speaker endpoint test validates that background noise or a later repeated utterance does not extend the current turn after the main speaker disappears.
9. First utterance preservation test validates that frames immediately after ACK are not discarded by post-playback drain.
10. Low-latency endpoint test validates that a short first question ends by primary speaker absence without waiting for a repeated second question.
11. Transport batching test validates that queued SoundDevice frames are returned together.
## Migration