[唤醒灵敏度与端点恢复]：完成唤醒提示顺序和Hybrid VAD优化，包含阈值默认值、缓冲时序和测试覆盖
This commit is contained in:
mkbk
+3
-3
@@ -8,11 +8,11 @@ OWNER_ASSET_DIR=assets/pet
|
||||
OWNER_LOG_DIR=logs
|
||||
OWNER_WAKE_PROVIDER=local_kws
|
||||
OWNER_WAKE_KEYWORDS_FILE=
|
||||
OWNER_WAKE_KWS_THRESHOLD=0.25
|
||||
OWNER_WAKE_KWS_THRESHOLD=0.15
|
||||
OWNER_WAKE_KWS_SCORE=1.0
|
||||
OWNER_WAKE_ACK_TEXT=我在
|
||||
OWNER_POST_PLAYBACK_DRAIN_MS=250
|
||||
OWNER_VAD_PROVIDER=local
|
||||
OWNER_POST_PLAYBACK_DRAIN_MS=50
|
||||
OWNER_VAD_PROVIDER=hybrid
|
||||
OWNER_VAD_THRESHOLD=0.5
|
||||
OWNER_VAD_MIN_DURATION_MS=250
|
||||
OWNER_VAD_END_SILENCE_MS=350
|
||||
|
||||
@@ -34,11 +34,11 @@ OWNER_LLM_MODEL=mimo-v2.5
|
||||
OWNER_LLM_API_STYLE=chat_completions
|
||||
OWNER_WAKE_PROVIDER=local_kws
|
||||
OWNER_WAKE_KEYWORDS_FILE=
|
||||
OWNER_WAKE_KWS_THRESHOLD=0.25
|
||||
OWNER_WAKE_KWS_THRESHOLD=0.15
|
||||
OWNER_WAKE_KWS_SCORE=1.0
|
||||
OWNER_WAKE_ACK_TEXT=我在
|
||||
OWNER_POST_PLAYBACK_DRAIN_MS=250
|
||||
OWNER_VAD_PROVIDER=local
|
||||
OWNER_POST_PLAYBACK_DRAIN_MS=50
|
||||
OWNER_VAD_PROVIDER=hybrid
|
||||
OWNER_VAD_THRESHOLD=0.5
|
||||
OWNER_VAD_MIN_DURATION_MS=250
|
||||
OWNER_VAD_END_SILENCE_MS=350
|
||||
@@ -63,9 +63,9 @@ python3.11 scripts/download_speech_models.py --dir models
|
||||
|
||||
`models/` 已在 `.gitignore` 中，不会提交大模型文件。
|
||||
|
||||
本地唤醒关键词文件位于 `models/wake/keywords.txt`。如果真人唤醒不灵敏，可以先把 `.env` 中 `OWNER_WAKE_KWS_THRESHOLD` 调低，例如 `0.15`，再重新运行。
|
||||
本地唤醒关键词文件位于 `models/wake/keywords.txt`。默认 `OWNER_WAKE_KWS_THRESHOLD=0.15` 已偏向灵敏；如果真人唤醒仍不灵敏，可以继续降到 `0.10`，若误唤醒变多再回调到 `0.20`。
|
||||
|
||||
如果唤醒后你已经停说但还长时间显示“录音中”，优先调小 `OWNER_VAD_END_SILENCE_MS`，例如 `250`；如果房间噪声较大，再略调高 `OWNER_VAD_THRESHOLD`，例如 `0.6`。
|
||||
默认 `OWNER_VAD_PROVIDER=hybrid` 会同时使用本地 `sherpa-onnx` VAD 和能量阈值兜底，避免本地 VAD 对麦克风音量过保守导致 `VAD_TIMEOUT_NO_SPEECH`。如果唤醒后你已经停说但还长时间显示“录音中”，优先调小 `OWNER_VAD_END_SILENCE_MS`，例如 `250`；如果房间噪声较大，再略调高 `OWNER_VAD_THRESHOLD`，例如 `0.6`。
|
||||
|
||||
## 设备检查
|
||||
|
||||
@@ -89,7 +89,7 @@ python3.11 scripts/download_speech_models.py --dir models
|
||||
.venv/bin/python -m owner_voice_pet run-live
|
||||
```
|
||||
|
||||
运行后终端会显示待机、唤醒命中、录音中、转写中、转写结果、思考中、播放中、恢复待机等状态。说“小杰小杰”等待唤醒命中后再提问；听到回复后可以再次说“小杰小杰”继续下一轮。本次进程内会携带临时历史，程序退出后不保存。
|
||||
运行后终端会显示待机、唤醒命中、应答中、请说出问题、录音中、转写中、转写结果、思考中、播放中、恢复待机等状态。说“小杰小杰”，听到“我在”且看到“请说出问题”后再提问；听到回复后可以再次说“小杰小杰”继续下一轮。本次进程内会携带临时历史，程序退出后不保存。
|
||||
|
||||
## 验证
|
||||
|
||||
|
||||
@@ -27,7 +27,7 @@ run-live
|
||||
-> AppConfig(.env)
|
||||
-> SoundDeviceAudioTransport
|
||||
-> SherpaOnnxKeywordWakeWordProvider(local models/wake)
|
||||
-> VadRecorder(user utterance)
|
||||
-> VadRecorder(user utterance, hybrid local+energy VAD by default)
|
||||
-> CloudAsrSttProvider or SherpaOnnxSttProvider
|
||||
-> TerminalRuntimeReporter.transcript()
|
||||
-> ConversationContext
|
||||
@@ -45,7 +45,7 @@ run-live
|
||||
5. Open microphone stream.
|
||||
6. Wait for KWS wake event by feeding frames directly into wake provider.
|
||||
7. On wake hit, reset wake stream and VAD recorder.
|
||||
8. Record user utterance with VAD.
|
||||
8. Record user utterance with VAD. The default provider is `hybrid`: project-local `sherpa-onnx` VAD remains the primary detector, and an energy threshold fallback prevents low microphone gain from being treated as no speech.
|
||||
9. Transcribe user utterance.
|
||||
10. Emit transcript to terminal.
|
||||
11. Append user text and call LLM.
|
||||
@@ -98,6 +98,7 @@ models/
|
||||
3. KWS runtime failure: `WAKE_MODEL_LOAD_FAILED` with retryable true.
|
||||
4. Empty user STT: existing `STT_EMPTY_TRANSCRIPT`.
|
||||
5. Invalid wake provider config: `CONFIG_MISSING_VALUE`.
|
||||
6. Real microphone VAD miss: default `OWNER_VAD_PROVIDER=hybrid` SHALL accept speech when either the local model or the energy fallback detects speech.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
|
||||
@@ -196,6 +196,8 @@ standby
|
||||
| --- | --- | --- | --- |
|
||||
| KWS 关键词拼音格式不匹配模型 | 中 | 高 | 默认写入 sherpa 示例格式；保留 keywords 文件可编辑；测试下载后用真实模型加载；README 标明关键词文件位置 |
|
||||
| KWS 模型误唤醒或漏唤醒 | 中 | 中 | 暴露 threshold/score 配置；保留状态输出；后续可替换 KWS Provider |
|
||||
| 唤醒应答期间用户抢说被缓冲清理吞掉 | 中 | 高 | 终端提示顺序改为“唤醒命中 -> 应答中 -> 请说出问题 -> 录音中”，用户只在应答完成后收到提问提示；默认播放后排水从 250 ms 降为 50 ms |
|
||||
| 本地 VAD 对真实麦克风音量过保守 | 中 | 高 | 默认 VAD provider 改为 `hybrid`，本地模型判断和能量阈值兜底任一命中即认为有语音；保留 `local` 和 `energy` 可配置回退 |
|
||||
| 本地 KWS 增加启动加载时间 | 低 | 中 | 模型约 15 MB，启动加载一次；不在每轮重复加载 |
|
||||
| 正式问题和唤醒词连在同一句导致问题前半段丢失 | 中 | 中 | 第一版交互明确为先唤醒再提问；后续可做 pre-roll buffer，但不得把 wake 音频直接进 LLM |
|
||||
| 终端转写不是逐字流式 | 中 | 低 | 第一版至少在 LLM 前即时显示最终 STT 文本；后续可接入本地 streaming STT partial |
|
||||
@@ -232,6 +234,20 @@ standby
|
||||
- [ ] 4.3 执行 run-live 单轮/两轮验收；前置条件：模型、设备、.env 齐全；验收标准：唤醒命中不等待云 ASR，终端显示转写结果；优先级：P0；预计：60 分钟。
|
||||
- [ ] 4.4 最终门禁、归档、提交；前置条件：全部任务完成；验收标准：compileall、unittest、security-check、model-check、device-check、OpenSpec strict、工作树干净；优先级：P0；预计：45 分钟。
|
||||
|
||||
### 5. 唤醒应答与快速端点修正
|
||||
|
||||
- [ ] 5.1 增加唤醒后本地语音应答；前置条件：本地 KWS 已可唤醒；验收标准：wake 命中后播放“我在”再进入录音；测试要点：fake runtime 播放顺序；优先级：P0；预计：45 分钟。
|
||||
- [ ] 5.2 清理应答播放期间的麦克风缓冲；前置条件：5.1 完成；验收标准：应答音频不进入正式问题 VAD/STT；测试要点：transport flush 测试；优先级：P0；预计：45 分钟。
|
||||
- [ ] 5.3 live 默认改用本地 `sherpa-onnx` VAD 并缩短静音端点；前置条件：模型已下载；验收标准：`.env` 可配置 VAD provider、静音结束时间和最大录音时长；测试要点：配置和 runtime 构造测试；优先级：P0；预计：45 分钟。
|
||||
- [ ] 5.4 验证并提交“唤醒应答与快速端点”模块；前置条件：5.1 至 5.3 完成；验收标准：compileall、unittest、security-check、model-check、OpenSpec strict 通过后 commit；优先级：P0；预计：20 分钟。
|
||||
|
||||
### 6. 灵敏度与录音端点恢复
|
||||
|
||||
- [ ] 6.1 调整唤醒提示顺序；前置条件：真人验收暴露用户会在“我在”播放前抢说；验收标准：终端顺序为“唤醒命中 -> 应答中：我在 -> 请说出问题 -> 录音中：正在听取问题”；测试要点：runtime reporter 顺序断言；优先级：P0；预计：30 分钟。
|
||||
- [ ] 6.2 降低默认 KWS 阈值；前置条件：真人反馈唤醒难触发；验收标准：默认 `OWNER_WAKE_KWS_THRESHOLD=0.15`，README 说明 0.10 至 0.20 调参范围；测试要点：配置默认值测试；优先级：P0；预计：20 分钟。
|
||||
- [ ] 6.3 增加 `hybrid` VAD；前置条件：真人验收出现 `VAD_TIMEOUT_NO_SPEECH`；验收标准：本地 VAD 和能量阈值任一判断为语音即可开始录音，默认 `OWNER_VAD_PROVIDER=hybrid`；测试要点：能量兜底单测；优先级：P0；预计：45 分钟。
|
||||
- [ ] 6.4 验证并提交“唤醒灵敏度与端点恢复”模块；前置条件：6.1 至 6.3 完成；验收标准：compileall、unittest、security-check、model-check、device-check、OpenSpec strict 通过后 commit；优先级：P0；预计：30 分钟。
|
||||
|
||||
## Spec Deltas
|
||||
|
||||
### 新增能力
|
||||
@@ -244,6 +260,8 @@ standby
|
||||
2. `Live terminal state reporting`：新增终端转写结果输出要求。
|
||||
3. `Local speech model management`：新增 wake KWS 模型和关键词文件管理。
|
||||
4. `Testability`：新增 wake/STT 分离测试和唤醒污染回归测试。
|
||||
5. `Wake acknowledgement before recording`：明确“请说出问题”必须在应答播放完成后输出，避免用户抢说被清缓冲。
|
||||
6. `Fast user utterance endpointing`：默认 VAD provider 改为 `hybrid`，用能量阈值兜底真实麦克风音量差异。
|
||||
|
||||
### 删除项
|
||||
|
||||
@@ -259,6 +277,7 @@ standby
|
||||
2. M2：KWS 模型下载、manifest、config、model-check 完成并提交。
|
||||
3. M3：Runtime 独立 wake provider 和转写输出完成并提交。
|
||||
4. M4：README、真实验收、archive 和最终提交完成。
|
||||
5. M5：真人验收反馈修正唤醒提示顺序、KWS 阈值和 hybrid VAD，并在门禁通过后提交。
|
||||
|
||||
估时：
|
||||
|
||||
|
||||
+9
-1
@@ -23,7 +23,7 @@ The live runtime SHALL provide an audible local acknowledgement after local wake
|
||||
- **THEN** the runtime SHALL clear buffered microphone input captured during acknowledgement playback before starting VAD recording for the user's question
|
||||
|
||||
### Requirement: Fast user utterance endpointing
|
||||
The live runtime SHALL use the project-local VAD model by default for user utterance endpoint detection and SHALL expose configurable silence timing so the recording stops promptly after the user stops speaking.
|
||||
The live runtime SHALL use project-local VAD capability with an energy-threshold fallback by default for user utterance endpoint detection and SHALL expose configurable silence timing so the recording stops promptly after the user stops speaking.
|
||||
|
||||
#### Scenario: User stops speaking after wake
|
||||
- **WHEN** VAD observes the configured continuous silence duration after a started utterance
|
||||
@@ -33,6 +33,14 @@ The live runtime SHALL use the project-local VAD model by default for user utter
|
||||
- **WHEN** `OWNER_VAD_PROVIDER=local`
|
||||
- **THEN** the runtime SHALL use the project-local `sherpa-onnx` VAD model rather than a raw energy threshold for live user utterance endpointing
|
||||
|
||||
#### Scenario: Hybrid VAD is configured
|
||||
- **WHEN** `OWNER_VAD_PROVIDER=hybrid`
|
||||
- **THEN** the runtime SHALL load both the project-local VAD model and an energy-threshold fallback and SHALL treat a frame as speech when either detector reports speech
|
||||
|
||||
#### Scenario: Wake acknowledgement prompt ordering
|
||||
- **WHEN** local wake detection succeeds and `OWNER_WAKE_ACK_TEXT` is non-empty
|
||||
- **THEN** the terminal SHALL emit wake hit, acknowledgement, question prompt, and recording states in that order so the user is cued to speak only after acknowledgement playback
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Wake word detection
|
||||
|
||||
@@ -38,3 +38,10 @@
|
||||
- [x] 5.2 清理应答播放期间的麦克风缓冲；前置条件：5.1 完成；验收标准：应答音频不进入正式问题 VAD/STT；测试要点：transport flush 测试；优先级：P0；预计：45 分钟。
|
||||
- [x] 5.3 live 默认改用本地 `sherpa-onnx` VAD 并缩短静音端点；前置条件：模型已下载；验收标准：`.env` 可配置 VAD provider、静音结束时间和最大录音时长；测试要点：配置和 runtime 构造测试；优先级：P0；预计：45 分钟。
|
||||
- [x] 5.4 验证并提交“唤醒应答与快速端点”模块；前置条件：5.1 至 5.3 完成；验收标准：compileall、unittest、security-check、model-check、OpenSpec strict 通过后 commit；优先级：P0；预计：20 分钟。
|
||||
|
||||
## 6. 灵敏度与录音端点恢复
|
||||
|
||||
- [x] 6.1 调整唤醒提示顺序；前置条件：真人验收显示用户可能在“我在”播放前抢说；验收标准：终端顺序为“唤醒命中 -> 应答中：我在 -> 请说出问题 -> 录音中：正在听取问题”；测试要点：runtime reporter 顺序断言；优先级：P0；预计：30 分钟。
|
||||
- [x] 6.2 降低默认 KWS 阈值；前置条件：真人反馈“小杰小杰”难触发；验收标准：默认 `OWNER_WAKE_KWS_THRESHOLD=0.15`，README 给出 `0.10` 至 `0.20` 调参建议；测试要点：配置默认值测试；优先级：P0；预计：20 分钟。
|
||||
- [x] 6.3 增加 `hybrid` VAD 并设为默认；前置条件：真人验收出现 `VAD_TIMEOUT_NO_SPEECH`；验收标准：本地 VAD 和能量阈值任一判断为语音即可开始/继续录音，默认 `OWNER_VAD_PROVIDER=hybrid`；测试要点：能量兜底单测、配置校验测试；优先级：P0；预计：45 分钟。
|
||||
- [x] 6.4 验证并提交“唤醒灵敏度与端点恢复”模块；前置条件：6.1 至 6.3 完成；验收标准：compileall、unittest、security-check、model-check、device-check、OpenSpec strict 通过后 commit；优先级：P0；预计：30 分钟。
|
||||
|
||||
@@ -17,7 +17,7 @@ from .models import (
|
||||
)
|
||||
from .transport import AudioRingBuffer, FileReplayTransport, MemoryAudioTransport
|
||||
from .wakeword import KeywordWakeWordProvider, SherpaOnnxKeywordWakeWordProvider
|
||||
from .vad import EnergyVadProvider, VadRecorder
|
||||
from .vad import EnergyVadProvider, HybridVadProvider, VadRecorder
|
||||
from .stt import CloudAsrSttProvider, MetadataSttProvider, SherpaOnnxSttProvider, is_valid_transcript_text
|
||||
from .conversation import ConversationContext
|
||||
from .llm import MockLlmProvider, OpenAICompatibleLlmProvider
|
||||
@@ -38,6 +38,7 @@ __all__ = [
|
||||
"KeywordWakeWordProvider",
|
||||
"SherpaOnnxKeywordWakeWordProvider",
|
||||
"EnergyVadProvider",
|
||||
"HybridVadProvider",
|
||||
"VadRecorder",
|
||||
"CloudAsrSttProvider",
|
||||
"MetadataSttProvider",
|
||||
|
||||
@@ -22,11 +22,11 @@ class AppConfig:
|
||||
log_dir: Path = Path("logs")
|
||||
wake_provider: str = "local_kws"
|
||||
wake_keywords_file: Path | None = None
|
||||
wake_kws_threshold: float = 0.25
|
||||
wake_kws_threshold: float = 0.15
|
||||
wake_kws_score: float = 1.0
|
||||
wake_ack_text: str = "我在"
|
||||
post_playback_drain_ms: int = 250
|
||||
vad_provider: str = "local"
|
||||
post_playback_drain_ms: int = 50
|
||||
vad_provider: str = "hybrid"
|
||||
vad_threshold: float = 0.5
|
||||
vad_min_duration_ms: int = 250
|
||||
vad_end_silence_ms: int = 350
|
||||
@@ -63,11 +63,11 @@ class AppConfig:
|
||||
log_dir=Path(get("LOG_DIR", "logs") or "logs"),
|
||||
wake_provider=(get("WAKE_PROVIDER", "local_kws") or "local_kws").lower(),
|
||||
wake_keywords_file=Path(value) if (value := get("WAKE_KEYWORDS_FILE")) else None,
|
||||
wake_kws_threshold=float(get("WAKE_KWS_THRESHOLD", "0.25") or "0.25"),
|
||||
wake_kws_threshold=float(get("WAKE_KWS_THRESHOLD", "0.15") or "0.15"),
|
||||
wake_kws_score=float(get("WAKE_KWS_SCORE", "1.0") or "1.0"),
|
||||
wake_ack_text=get("WAKE_ACK_TEXT", "我在") or "我在",
|
||||
post_playback_drain_ms=int(get("POST_PLAYBACK_DRAIN_MS", "250") or "250"),
|
||||
vad_provider=(get("VAD_PROVIDER", "local") or "local").lower(),
|
||||
post_playback_drain_ms=int(get("POST_PLAYBACK_DRAIN_MS", "50") or "50"),
|
||||
vad_provider=(get("VAD_PROVIDER", "hybrid") or "hybrid").lower(),
|
||||
vad_threshold=float(get("VAD_THRESHOLD", "0.5") or "0.5"),
|
||||
vad_min_duration_ms=int(get("VAD_MIN_DURATION_MS", "250") or "250"),
|
||||
vad_end_silence_ms=int(get("VAD_END_SILENCE_MS", "350") or "350"),
|
||||
@@ -178,11 +178,11 @@ class AppConfig:
|
||||
"startup",
|
||||
)
|
||||
)
|
||||
if self.vad_provider not in {"local", "energy"}:
|
||||
if self.vad_provider not in {"hybrid", "local", "energy"}:
|
||||
errors.append(
|
||||
ProviderError(
|
||||
ErrorCode.CONFIG_MISSING_VALUE,
|
||||
"OWNER_VAD_PROVIDER must be local or energy",
|
||||
"OWNER_VAD_PROVIDER must be hybrid, local, or energy",
|
||||
False,
|
||||
"config",
|
||||
"startup",
|
||||
|
||||
@@ -12,7 +12,7 @@ from .protocols import AudioTransport, LlmProvider, SttProvider, TtsProvider, Wa
|
||||
from .stt import CloudAsrSttProvider, SherpaOnnxSttProvider, is_valid_transcript_text
|
||||
from .transport import SoundDeviceAudioTransport
|
||||
from .tts import CloudTtsProvider, MacSayTtsProvider, SentenceBuffer
|
||||
from .vad import EnergyVadProvider, SherpaOnnxVadProvider, VadRecorder
|
||||
from .vad import EnergyVadProvider, HybridVadProvider, SherpaOnnxVadProvider, VadRecorder
|
||||
from .wakeword import SherpaOnnxKeywordWakeWordProvider
|
||||
|
||||
|
||||
@@ -145,10 +145,11 @@ class LiveVoiceRuntime:
|
||||
wake_error = self._wait_for_local_wake(turn_id)
|
||||
if wake_error is not None:
|
||||
return wake_error
|
||||
self._state(PipelineState.SPEECH_DETECTING, "唤醒命中：请说出问题", turn_id=turn_id)
|
||||
self._state(PipelineState.SPEECH_DETECTING, "唤醒命中", turn_id=turn_id)
|
||||
ack_error = self._acknowledge_wake(turn_id)
|
||||
if ack_error is not None:
|
||||
return ack_error
|
||||
self._state(PipelineState.SPEECH_DETECTING, "请说出问题", turn_id=turn_id)
|
||||
user_segment = self._capture_segment(turn_id, state_message="录音中：正在听取问题")
|
||||
if isinstance(user_segment, ProviderError):
|
||||
return user_segment
|
||||
@@ -277,7 +278,12 @@ def build_live_runtime(config: AppConfig, reporter: RuntimeReporter | None = Non
|
||||
else:
|
||||
stt = SherpaOnnxSttProvider(str(config.speech_models_dir))
|
||||
tts = MacSayTtsProvider()
|
||||
if config.vad_provider == "local":
|
||||
if config.vad_provider == "hybrid":
|
||||
vad_provider = HybridVadProvider(
|
||||
SherpaOnnxVadProvider(config.speech_models_dir, threshold=config.vad_threshold),
|
||||
EnergyVadProvider(),
|
||||
)
|
||||
elif config.vad_provider == "local":
|
||||
vad_provider = SherpaOnnxVadProvider(config.speech_models_dir, threshold=config.vad_threshold)
|
||||
else:
|
||||
vad_provider = EnergyVadProvider()
|
||||
|
||||
@@ -157,6 +157,53 @@ class SherpaOnnxVadProvider:
|
||||
self._model.reset()
|
||||
|
||||
|
||||
class HybridVadProvider:
|
||||
"""Combine local model VAD with energy fallback for live microphone variance."""
|
||||
|
||||
def __init__(self, primary: Any, fallback: Any) -> None:
|
||||
self.primary = primary
|
||||
self.fallback = fallback
|
||||
self.loaded = False
|
||||
self._speech_ms = 0
|
||||
self._silence_ms = 0
|
||||
|
||||
def load(self) -> None:
|
||||
self.primary.load()
|
||||
self.fallback.load()
|
||||
self.loaded = True
|
||||
|
||||
def analyze(self, frame: AudioFrame) -> VadResult:
|
||||
if not self.loaded:
|
||||
raise ProviderError(
|
||||
ErrorCode.VAD_MODEL_LOAD_FAILED,
|
||||
"hybrid VAD provider is not loaded",
|
||||
False,
|
||||
"hybrid-vad",
|
||||
"vad",
|
||||
)
|
||||
primary = self.primary.analyze(frame)
|
||||
fallback = self.fallback.analyze(frame)
|
||||
is_speech = primary.is_speech or fallback.is_speech
|
||||
frame_ms = int(frame.metadata.get("duration_ms", 20))
|
||||
if is_speech:
|
||||
self._speech_ms += frame_ms
|
||||
self._silence_ms = 0
|
||||
else:
|
||||
self._silence_ms += frame_ms
|
||||
return VadResult(
|
||||
is_speech=is_speech,
|
||||
confidence=max(primary.confidence, fallback.confidence),
|
||||
speech_ms=self._speech_ms,
|
||||
silence_ms=self._silence_ms,
|
||||
)
|
||||
|
||||
def reset(self) -> None:
|
||||
self._speech_ms = 0
|
||||
self._silence_ms = 0
|
||||
self.primary.reset()
|
||||
self.fallback.reset()
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class VadRecorder:
|
||||
provider: Any
|
||||
|
||||
@@ -104,6 +104,9 @@ class LiveRuntimeTests(unittest.TestCase):
|
||||
self.assertEqual(len(transport.played_segments), 4)
|
||||
self.assertEqual(reporter.transcripts, ["第一问", "第二问"])
|
||||
self.assertIn("应答中：我在", reporter.statuses)
|
||||
self.assertLess(reporter.statuses.index("唤醒命中"), reporter.statuses.index("应答中：我在"))
|
||||
self.assertLess(reporter.statuses.index("应答中：我在"), reporter.statuses.index("请说出问题"))
|
||||
self.assertLess(reporter.statuses.index("请说出问题"), reporter.statuses.index("录音中：正在听取问题"))
|
||||
self.assertIn("恢复待机：可继续唤醒", reporter.statuses[-1])
|
||||
|
||||
def test_temporary_context_is_sent_to_second_llm_call(self) -> None:
|
||||
|
||||
@@ -70,11 +70,11 @@ class ModelsConfigTests(unittest.TestCase):
|
||||
self.assertEqual(config.llm_api_key, "secret-value")
|
||||
self.assertEqual(config.llm_model, "test-model")
|
||||
self.assertEqual(config.wake_provider, "local_kws")
|
||||
self.assertEqual(config.wake_kws_threshold, 0.25)
|
||||
self.assertEqual(config.wake_kws_threshold, 0.15)
|
||||
self.assertEqual(config.wake_kws_score, 1.0)
|
||||
self.assertEqual(config.wake_ack_text, "我在")
|
||||
self.assertEqual(config.post_playback_drain_ms, 250)
|
||||
self.assertEqual(config.vad_provider, "local")
|
||||
self.assertEqual(config.post_playback_drain_ms, 50)
|
||||
self.assertEqual(config.vad_provider, "hybrid")
|
||||
self.assertEqual(config.vad_threshold, 0.5)
|
||||
self.assertEqual(config.vad_min_duration_ms, 250)
|
||||
self.assertEqual(config.vad_end_silence_ms, 350)
|
||||
@@ -98,7 +98,7 @@ class ModelsConfigTests(unittest.TestCase):
|
||||
errors = config.validate_basic()
|
||||
self.assertTrue(any("OWNER_WAKE_PROVIDER" in error.message for error in errors))
|
||||
|
||||
def test_vad_provider_must_be_local_or_energy(self) -> None:
|
||||
def test_vad_provider_must_be_hybrid_local_or_energy(self) -> None:
|
||||
config = AppConfig(vad_provider="invalid")
|
||||
errors = config.validate_basic()
|
||||
self.assertTrue(any("OWNER_VAD_PROVIDER" in error.message for error in errors))
|
||||
|
||||
@@ -7,7 +7,7 @@ import unittest
|
||||
from owner_voice_pet.models import AudioFrame, AudioSegment, ErrorCode, ProviderError
|
||||
from owner_voice_pet.config import AppConfig
|
||||
from owner_voice_pet.stt import CloudAsrSttProvider, MetadataSttProvider, SherpaOnnxSttProvider, is_valid_transcript_text
|
||||
from owner_voice_pet.vad import EnergyVadProvider, VadRecorder
|
||||
from owner_voice_pet.vad import EnergyVadProvider, HybridVadProvider, VadRecorder
|
||||
from owner_voice_pet.wakeword import (
|
||||
KeywordWakeWordProvider,
|
||||
MissingWakeWordModelProvider,
|
||||
@@ -87,6 +87,30 @@ class WakeVadSttTests(unittest.TestCase):
|
||||
self.assertIsInstance(result, ProviderError)
|
||||
self.assertEqual(result.code, ErrorCode.VAD_TIMEOUT_NO_SPEECH)
|
||||
|
||||
def test_hybrid_vad_accepts_energy_fallback_speech(self) -> None:
|
||||
class SilentVadProvider:
|
||||
def __init__(self) -> None:
|
||||
self.loaded = False
|
||||
|
||||
def load(self) -> None:
|
||||
self.loaded = True
|
||||
|
||||
def analyze(self, frame: AudioFrame):
|
||||
return type(
|
||||
"Result",
|
||||
(),
|
||||
{"is_speech": False, "confidence": 0.1, "speech_ms": 0, "silence_ms": 20},
|
||||
)()
|
||||
|
||||
def reset(self) -> None:
|
||||
return None
|
||||
|
||||
provider = HybridVadProvider(SilentVadProvider(), EnergyVadProvider())
|
||||
provider.load()
|
||||
result = provider.analyze(make_frame(1, 0, speech=True))
|
||||
self.assertTrue(result.is_speech)
|
||||
self.assertEqual(result.speech_ms, 20)
|
||||
|
||||
def test_metadata_stt_transcribes_fixture_text(self) -> None:
|
||||
provider = MetadataSttProvider()
|
||||
provider.load()
|
||||
|
||||
Reference in New Issue
Block a user