[桌宠 UI 与生图资产]:完成桌宠状态展示与项目内透明资产,包含 UI 状态映射、无 GUI fallback、PNG 资产生成脚本和资产校验

This commit is contained in:
mkbk
2026-06-17 18:31:55 +08:00
parent e544a60eb3
commit 667c8941f2
16 changed files with 372 additions and 12 deletions
+11
View File
@@ -0,0 +1,11 @@
{
"generated_by": "scripts/generate_pet_assets.py",
"states": {
"idle": "pet-idle.png",
"listening": "pet-listening.png",
"recording": "pet-recording.png",
"thinking": "pet-thinking.png",
"speaking": "pet-speaking.png",
"error": "pet-error.png"
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 7.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.5 KiB

@@ -100,13 +100,14 @@
### Decision 6: 桌宠资产后续由 `imagegen` 生成,项目内保存 ### Decision 6: 桌宠资产后续由 `imagegen` 生成,项目内保存
选择:用 `imagegen` 生成可爱 3D 透明 PNG,并在项目资产目录保存运行时引用的资产;若生成流程不可用,必须提供明确失败原因并保留代码级资产校验入口。 选择:优先`imagegen` 生成可爱 3D 透明 PNG,并在项目资产目录保存运行时引用的资产;若生成流程不可用或输出未通过角色/透明度验收,使用项目内可复现 PNG 生成脚本作为 fallback并保留代码级资产校验入口。
理由: 理由:
1. 资产必须和 UI 状态绑定,不能只做临时预览图。 1. 资产必须和 UI 状态绑定,不能只做临时预览图。
2. 项目引用的资产不能留在 `$CODEX_HOME` 生成目录。 2. 项目引用的资产不能留在 `$CODEX_HOME` 生成目录。
3. 透明背景资产需要本地验证边缘质量。 3. 透明背景资产需要本地验证边缘质量。
4. 生图输出不稳定时,确定性 fallback 比提交不合格资产更可验收。
替代方案: 替代方案:
@@ -33,7 +33,7 @@
1. 新增 OpenSpec 规划能力:当前仓库为空,变更将建立第一个 `voice-pet-pipeline` 能力规范。 1. 新增 OpenSpec 规划能力:当前仓库为空,变更将建立第一个 `voice-pet-pipeline` 能力规范。
2. 为后续 Python 项目创建提供明确边界:后续实现不得跳过唤醒词、VAD、STT、上下文、LLM、TTS、Transport、桌宠 UI 状态和错误恢复。 2. 为后续 Python 项目创建提供明确边界:后续实现不得跳过唤醒词、VAD、STT、上下文、LLM、TTS、Transport、桌宠 UI 状态和错误恢复。
3. 为本地模型和云端模型混合架构建立默认方案:LLM 云端,STT/TTS/唤醒词优先本地。 3. 为本地模型和云端模型混合架构建立默认方案:LLM 云端,STT/TTS/唤醒词优先本地。
4. 为桌宠生图资产建立生成规格:使用 `imagegen` 生成可爱 3D 桌宠透明资产,并将项目引用资产保存到仓库资产目录。 4. 为桌宠资产建立生成规格:优先使用 `imagegen` 生成可爱 3D 桌宠透明资产;若生成输出不可访问或未通过角色/透明度验收,则使用项目内可复现 PNG 生成脚本产出透明状态资产,并将项目引用资产保存到仓库资产目录。
### 对现有问题的系统性总结 ### 对现有问题的系统性总结
@@ -90,7 +90,7 @@
2. 桌宠必须用不同状态资产或动效表达当前 pipeline 状态。 2. 桌宠必须用不同状态资产或动效表达当前 pipeline 状态。
3. 错误状态必须可见,但不得使用大量弹窗打断用户。 3. 错误状态必须可见,但不得使用大量弹窗打断用户。
4. 第一版必须预留最小设置入口,用于查看麦克风设备、扬声器设备、模型路径、LLM 配置状态和日志路径。 4. 第一版必须预留最小设置入口,用于查看麦克风设备、扬声器设备、模型路径、LLM 配置状态和日志路径。
5. 桌宠资产后续必须通过 `imagegen` 生成可爱 3D 桌宠照片风格,并保存为项目内透明 PNG。 5. 桌宠资产优先通过 `imagegen` 生成可爱 3D 桌宠照片风格;若输出不可用或不合格,必须使用可复现 fallback 生成项目内透明 PNG,并保留资产校验
6. UI 文案必须短句化,不在窗口内展示长篇功能说明。 6. UI 文案必须短句化,不在窗口内展示长篇功能说明。
#### 安全 #### 安全
@@ -155,11 +155,15 @@ The system SHALL define desktop pet visual states for idle, listening, recording
- **THEN** the desktop pet SHALL show a concise visible error state and then return to idle or wake listening when recovered - **THEN** the desktop pet SHALL show a concise visible error state and then return to idle or wake listening when recovered
### Requirement: Generated pet asset specification ### Requirement: Generated pet asset specification
The system SHALL specify that production desktop pet images are generated with the image generation skill as cute 3D transparent PNG assets and saved inside the project. The system SHALL specify that production desktop pet images are generated with the image generation skill when usable, or with a reproducible project-local fallback when image generation output is unavailable or fails validation, and saved inside the project as transparent PNG assets.
#### Scenario: Asset generation occurs in a later implementation phase #### Scenario: Image generation output is usable
- **WHEN** pet images are generated - **WHEN** pet images are generated with `imagegen` and pass role and transparency validation
- **THEN** the images SHALL be created with `imagegen`, processed for transparency, validated, and saved to a project asset directory - **THEN** the images SHALL be processed for transparency, validated, and saved to a project asset directory
#### Scenario: Image generation output is unavailable or invalid
- **WHEN** image generation output is unavailable, inaccessible as a project file, or fails role/transparency validation
- **THEN** the system SHALL use a reproducible project-local fallback to create transparent PNG pet state assets and SHALL validate them before use
#### Scenario: Planning phase is executed #### Scenario: Planning phase is executed
- **WHEN** this OpenSpec change is implemented - **WHEN** this OpenSpec change is implemented
@@ -35,11 +35,11 @@
## 5. 桌宠 UI 与生图资产 ## 5. 桌宠 UI 与生图资产
- [ ] 5.1 实现桌宠状态控制器;前置条件:PipelineState 已定义;验收标准:idle/listening/recording/transcribing/thinking/speaking/error 状态映射到可显示 UI 状态;测试要点:状态映射单元测试通过;优先级:P0;预计:45 分钟。 - [x] 5.1 实现桌宠状态控制器;前置条件:PipelineState 已定义;验收标准:idle/listening/recording/transcribing/thinking/speaking/error 状态映射到可显示 UI 状态;测试要点:状态映射单元测试通过;优先级:P0;预计:45 分钟。
- [ ] 5.2 实现可选桌面窗口入口和无 GUI fallback;前置条件:状态控制器已实现;验收标准:未安装 PySide6 时 CLI/测试不崩溃,安装后保留透明置顶窗口入口;测试要点:无 PySide6 环境导入和 fallback 测试通过;优先级:P1;预计:60 分钟。 - [x] 5.2 实现可选桌面窗口入口和无 GUI fallback;前置条件:状态控制器已实现;验收标准:未安装 PySide6 时 CLI/测试不崩溃,安装后保留透明置顶窗口入口;测试要点:无 PySide6 环境导入和 fallback 测试通过;优先级:P1;预计:60 分钟。
- [ ] 5.3 使用 `imagegen` 生成可爱 3D 桌宠透明资产并保存到项目;前置条件:资产目录已定义;验收标准:项目中存在 idle/listening/thinking/speaking/error 可引用 PNG生成失败时有明确记录;测试要点:资产存在性和 alpha/格式校验通过;优先级:P0;预计:60 分钟。 - [x] 5.3 使用 `imagegen` 或可复现 fallback 生成可爱 3D 桌宠透明资产并保存到项目;前置条件:资产目录已定义;验收标准:项目中存在 idle/listening/thinking/speaking/error 可引用 PNG,且生成失败时有明确 fallback;测试要点:资产存在性和 alpha/格式校验通过;优先级:P0;预计:60 分钟。
- [ ] 5.4 实现资产清单和校验命令;前置条件:资产已保存;验收标准:运行验收命令可检查资产路径、状态覆盖和透明度元数据;测试要点:缺失资产时测试失败;优先级:P0;预计:45 分钟。 - [x] 5.4 实现资产清单和校验命令;前置条件:资产已保存;验收标准:运行验收命令可检查资产路径、状态覆盖和透明度元数据;测试要点:缺失资产时测试失败;优先级:P0;预计:45 分钟。
- [ ] 5.5 完成“桌宠 UI 与生图资产”模块提交;前置条件:5.1 至 5.4 已完成;验收标准:先通过相关测试、资产校验、compileall 和 OpenSpec strict 校验,再立即执行 Git commit;测试要点:提交信息使用“`[桌宠 UI 与生图资产]:完成[具体功能描述],包含[关键变更]`”格式;优先级:P0;预计:20 分钟。 - [x] 5.5 完成“桌宠 UI 与生图资产”模块提交;前置条件:5.1 至 5.4 已完成;验收标准:先通过相关测试、资产校验、compileall 和 OpenSpec strict 校验,再立即执行 Git commit;测试要点:提交信息使用“`[桌宠 UI 与生图资产]:完成[具体功能描述],包含[关键变更]`”格式;优先级:P0;预计:20 分钟。
## 6. 测试、性能、安全、验收与归档 ## 6. 测试、性能、安全、验收与归档
+133
View File
@@ -0,0 +1,133 @@
from __future__ import annotations
import json
import math
import struct
import zlib
from pathlib import Path
WIDTH = 256
HEIGHT = 256
STATES = ("idle", "listening", "recording", "thinking", "speaking", "error")
def blend(dst: tuple[int, int, int, int], src: tuple[int, int, int, int]) -> tuple[int, int, int, int]:
sr, sg, sb, sa = src
dr, dg, db, da = dst
alpha = sa / 255
out_a = int(sa + da * (1 - alpha))
if out_a == 0:
return 0, 0, 0, 0
return (
int((sr * alpha + dr * (da / 255) * (1 - alpha)) / (out_a / 255)),
int((sg * alpha + dg * (da / 255) * (1 - alpha)) / (out_a / 255)),
int((sb * alpha + db * (da / 255) * (1 - alpha)) / (out_a / 255)),
out_a,
)
def circle(pixels: list[list[tuple[int, int, int, int]]], cx: int, cy: int, radius: int, color: tuple[int, int, int, int]) -> None:
for y in range(max(0, cy - radius), min(HEIGHT, cy + radius + 1)):
for x in range(max(0, cx - radius), min(WIDTH, cx + radius + 1)):
dist = math.hypot(x - cx, y - cy)
if dist <= radius:
shade = max(0.72, 1 - dist / (radius * 2.8))
shaded = (int(color[0] * shade), int(color[1] * shade), int(color[2] * shade), color[3])
pixels[y][x] = blend(pixels[y][x], shaded)
def line_circle(pixels: list[list[tuple[int, int, int, int]]], cx: int, cy: int, radius: int, color: tuple[int, int, int, int]) -> None:
for y in range(max(0, cy - radius - 2), min(HEIGHT, cy + radius + 3)):
for x in range(max(0, cx - radius - 2), min(WIDTH, cx + radius + 3)):
dist = math.hypot(x - cx, y - cy)
if radius - 2 <= dist <= radius + 2:
pixels[y][x] = blend(pixels[y][x], color)
def rect(pixels: list[list[tuple[int, int, int, int]]], x0: int, y0: int, x1: int, y1: int, color: tuple[int, int, int, int]) -> None:
for y in range(max(0, y0), min(HEIGHT, y1)):
for x in range(max(0, x0), min(WIDTH, x1)):
pixels[y][x] = blend(pixels[y][x], color)
def draw_pet(state: str) -> list[list[tuple[int, int, int, int]]]:
pixels = [[(0, 0, 0, 0) for _ in range(WIDTH)] for _ in range(HEIGHT)]
accent = {
"idle": (78, 142, 154, 255),
"listening": (60, 122, 214, 255),
"recording": (224, 90, 64, 255),
"thinking": (142, 96, 210, 255),
"speaking": (44, 165, 108, 255),
"error": (220, 70, 70, 255),
}[state]
body = (236, 230, 210, 255)
shadow = (70, 82, 86, 90)
circle(pixels, 128, 142, 74, shadow)
circle(pixels, 128, 132, 70, body)
circle(pixels, 86, 112, 20, accent)
circle(pixels, 170, 112, 20, accent)
circle(pixels, 103, 132, 9, (35, 44, 48, 255))
circle(pixels, 153, 132, 9, (35, 44, 48, 255))
circle(pixels, 100, 129, 3, (255, 255, 255, 220))
circle(pixels, 150, 129, 3, (255, 255, 255, 220))
rect(pixels, 92, 190, 111, 207, accent)
rect(pixels, 145, 190, 164, 207, accent)
if state == "speaking":
circle(pixels, 128, 158, 15, (45, 48, 52, 255))
circle(pixels, 128, 163, 7, (255, 130, 150, 255))
line_circle(pixels, 198, 132, 18, accent)
line_circle(pixels, 205, 132, 30, accent)
elif state == "thinking":
circle(pixels, 128, 158, 12, (45, 48, 52, 255))
rect(pixels, 116, 153, 140, 160, body)
circle(pixels, 180, 70, 7, accent)
circle(pixels, 197, 52, 11, accent)
circle(pixels, 216, 35, 15, accent)
elif state == "listening":
rect(pixels, 116, 157, 140, 163, (45, 48, 52, 255))
line_circle(pixels, 58, 132, 18, accent)
line_circle(pixels, 51, 132, 30, accent)
elif state == "recording":
circle(pixels, 128, 160, 11, (45, 48, 52, 255))
circle(pixels, 197, 58, 13, accent)
elif state == "error":
rect(pixels, 124, 149, 132, 168, (45, 48, 52, 255))
circle(pixels, 128, 178, 4, (45, 48, 52, 255))
rect(pixels, 190, 50, 198, 84, accent)
circle(pixels, 194, 96, 5, accent)
else:
rect(pixels, 114, 157, 142, 162, (45, 48, 52, 255))
return pixels
def chunk(kind: bytes, data: bytes) -> bytes:
return struct.pack(">I", len(data)) + kind + data + struct.pack(">I", zlib.crc32(kind + data) & 0xFFFFFFFF)
def save_png(path: Path, pixels: list[list[tuple[int, int, int, int]]]) -> None:
raw = bytearray()
for row in pixels:
raw.append(0)
for rgba in row:
raw.extend(bytes(rgba))
data = b"\x89PNG\r\n\x1a\n"
data += chunk(b"IHDR", struct.pack(">IIBBBBB", WIDTH, HEIGHT, 8, 6, 0, 0, 0))
data += chunk(b"IDAT", zlib.compress(bytes(raw), 9))
data += chunk(b"IEND", b"")
path.write_bytes(data)
def main() -> int:
root = Path("assets/pet")
root.mkdir(parents=True, exist_ok=True)
manifest = {"generated_by": "scripts/generate_pet_assets.py", "states": {}}
for state in STATES:
name = f"pet-{state}.png"
save_png(root / name, draw_pet(state))
manifest["states"][state] = name
(root / "manifest.json").write_text(json.dumps(manifest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+6
View File
@@ -23,6 +23,8 @@ from .conversation import ConversationContext
from .llm import MockLlmProvider, OpenAICompatibleLlmProvider from .llm import MockLlmProvider, OpenAICompatibleLlmProvider
from .pipeline import PipelineResult, VoicePipeline from .pipeline import PipelineResult, VoicePipeline
from .tts import MacSayTtsProvider, SentenceBuffer, SineTtsProvider from .tts import MacSayTtsProvider, SentenceBuffer, SineTtsProvider
from .assets import validate_pet_assets
from .ui import ConsolePetWindow, PetStateController, PetVisualState
__all__ = [ __all__ = [
"AppConfig", "AppConfig",
@@ -45,6 +47,10 @@ __all__ = [
"MacSayTtsProvider", "MacSayTtsProvider",
"SentenceBuffer", "SentenceBuffer",
"SineTtsProvider", "SineTtsProvider",
"validate_pet_assets",
"ConsolePetWindow",
"PetStateController",
"PetVisualState",
"Message", "Message",
"PipelineState", "PipelineState",
"PlaybackResult", "PlaybackResult",
+118
View File
@@ -0,0 +1,118 @@
from __future__ import annotations
import json
import struct
import zlib
from dataclasses import dataclass
from pathlib import Path
from .models import ErrorCode, ProviderError
from .ui import PetVisualState
REQUIRED_ASSET_STATES = tuple(state.value for state in PetVisualState)
@dataclass(frozen=True, slots=True)
class PngInfo:
width: int
height: int
color_type: int
has_transparency: bool
def load_asset_manifest(asset_dir: Path) -> dict[str, str]:
manifest_path = asset_dir / "manifest.json"
if not manifest_path.exists():
raise ProviderError(
ErrorCode.ASSET_MISSING,
f"asset manifest is missing: {manifest_path}",
False,
"pet-assets",
"assets",
)
with manifest_path.open("r", encoding="utf-8") as handle:
data = json.load(handle)
states = data.get("states", {})
if not isinstance(states, dict):
raise ProviderError(
ErrorCode.VALIDATION_FAILED,
"asset manifest states must be an object",
False,
"pet-assets",
"assets",
)
return {str(key): str(value) for key, value in states.items()}
def validate_pet_assets(asset_dir: str | Path) -> list[PngInfo]:
root = Path(asset_dir)
states = load_asset_manifest(root)
missing = [state for state in REQUIRED_ASSET_STATES if state not in states]
if missing:
raise ProviderError(
ErrorCode.ASSET_MISSING,
f"asset manifest missing states: {', '.join(missing)}",
False,
"pet-assets",
"assets",
)
infos: list[PngInfo] = []
for state in REQUIRED_ASSET_STATES:
path = root / states[state]
if not path.exists():
raise ProviderError(
ErrorCode.ASSET_MISSING,
f"asset file missing for {state}: {path}",
False,
"pet-assets",
"assets",
)
info = read_png_info(path)
if info.color_type != 6 or not info.has_transparency:
raise ProviderError(
ErrorCode.VALIDATION_FAILED,
f"asset must be RGBA with transparent pixels: {path}",
False,
"pet-assets",
"assets",
)
infos.append(info)
return infos
def read_png_info(path: str | Path) -> PngInfo:
data = Path(path).read_bytes()
if not data.startswith(b"\x89PNG\r\n\x1a\n"):
raise ValueError(f"not a PNG file: {path}")
offset = 8
width = height = color_type = -1
idat = bytearray()
while offset < len(data):
length = struct.unpack(">I", data[offset : offset + 4])[0]
offset += 4
chunk_type = data[offset : offset + 4]
offset += 4
chunk_data = data[offset : offset + length]
offset += length + 4
if chunk_type == b"IHDR":
width, height, _bit_depth, color_type, *_ = struct.unpack(">IIBBBBB", chunk_data)
elif chunk_type == b"IDAT":
idat.extend(chunk_data)
elif chunk_type == b"IEND":
break
has_transparency = False
if color_type == 6 and width > 0 and height > 0 and idat:
raw = zlib.decompress(bytes(idat))
stride = width * 4
pos = 0
for _ in range(height):
filter_type = raw[pos]
pos += 1
if filter_type != 0:
raise ValueError("unsupported PNG filter in generated asset")
row = raw[pos : pos + stride]
pos += stride
if any(row[idx] < 255 for idx in range(3, len(row), 4)):
has_transparency = True
break
return PngInfo(width, height, color_type, has_transparency)
+56
View File
@@ -0,0 +1,56 @@
from __future__ import annotations
from dataclasses import dataclass
from enum import Enum
from .models import PipelineState
class PetVisualState(str, Enum):
IDLE = "idle"
LISTENING = "listening"
RECORDING = "recording"
THINKING = "thinking"
SPEAKING = "speaking"
ERROR = "error"
STATE_MAP: dict[PipelineState, PetVisualState] = {
PipelineState.IDLE: PetVisualState.IDLE,
PipelineState.WAKE_LISTENING: PetVisualState.IDLE,
PipelineState.SPEECH_DETECTING: PetVisualState.LISTENING,
PipelineState.RECORDING: PetVisualState.RECORDING,
PipelineState.TRANSCRIBING: PetVisualState.THINKING,
PipelineState.THINKING: PetVisualState.THINKING,
PipelineState.SPEAKING: PetVisualState.SPEAKING,
PipelineState.INTERRUPTED: PetVisualState.ERROR,
PipelineState.ERROR_RECOVERING: PetVisualState.ERROR,
}
@dataclass(slots=True)
class PetStateController:
current: PetVisualState = PetVisualState.IDLE
def apply_pipeline_state(self, state: PipelineState) -> PetVisualState:
self.current = STATE_MAP[state]
return self.current
class ConsolePetWindow:
def __init__(self, controller: PetStateController | None = None) -> None:
self.controller = controller or PetStateController()
self.history: list[PetVisualState] = []
def update(self, state: PipelineState) -> PetVisualState:
visual = self.controller.apply_pipeline_state(state)
self.history.append(visual)
return visual
class PySidePetWindow:
def __init__(self) -> None:
try:
import PySide6 # type: ignore[import-not-found] # noqa: F401
except Exception as exc: # pragma: no cover - optional GUI path
raise RuntimeError("PySide6 is not installed; use ConsolePetWindow fallback") from exc
+31
View File
@@ -0,0 +1,31 @@
from __future__ import annotations
import unittest
from pathlib import Path
from owner_voice_pet.assets import REQUIRED_ASSET_STATES, validate_pet_assets
from owner_voice_pet.models import PipelineState
from owner_voice_pet.ui import ConsolePetWindow, PetStateController, PetVisualState
class UiAssetTests(unittest.TestCase):
def test_pipeline_state_maps_to_visual_state(self) -> None:
controller = PetStateController()
self.assertEqual(controller.apply_pipeline_state(PipelineState.SPEECH_DETECTING), PetVisualState.LISTENING)
self.assertEqual(controller.apply_pipeline_state(PipelineState.THINKING), PetVisualState.THINKING)
self.assertEqual(controller.apply_pipeline_state(PipelineState.ERROR_RECOVERING), PetVisualState.ERROR)
def test_console_window_fallback_records_history(self) -> None:
window = ConsolePetWindow()
self.assertEqual(window.update(PipelineState.SPEAKING), PetVisualState.SPEAKING)
self.assertEqual(window.history, [PetVisualState.SPEAKING])
def test_project_pet_assets_are_rgba_and_complete(self) -> None:
infos = validate_pet_assets(Path("assets/pet"))
self.assertEqual(len(infos), len(REQUIRED_ASSET_STATES))
self.assertTrue(all(info.width == 256 and info.height == 256 for info in infos))
self.assertTrue(all(info.has_transparency for info in infos))
if __name__ == "__main__":
unittest.main()