Initial TrueGrowth source import
This commit is contained in:
219
openspec/changes/add-suno-lyrics-task-and-canvas-flow/design.md
Normal file
219
openspec/changes/add-suno-lyrics-task-and-canvas-flow/design.md
Normal file
@@ -0,0 +1,219 @@
|
||||
## Context
|
||||
|
||||
仓库当前已经为 Suno 音乐生成铺好了主路径:
|
||||
|
||||
- `runtime-model-discovery` 会把 `lyrics`、`music`、`suno` 等标识统归到 `audio`
|
||||
- `provider-routing` 会为 Suno 推断 `tuzi.suno.music` 绑定
|
||||
- `audio-api-service` 负责提交 `/suno/submit/music`、轮询 `/suno/fetch/{taskId}` 并提取音频 URL
|
||||
- 任务队列、自动插入和任务面板会把 `TaskType.AUDIO` 默认当成“可播放音频资产”
|
||||
|
||||
这条链路在音乐生成场景下是成立的,但在歌词场景下不成立:
|
||||
|
||||
- 歌词任务仍然属于 Suno 的异步任务
|
||||
- 但 fetch 结果里的核心字段是:
|
||||
- `data.data.text`
|
||||
- `data.data.title`
|
||||
- `data.data.tags`
|
||||
- 没有 `audio_url`
|
||||
|
||||
因此,当前代码里“路由类型 = 结果类型 = 画布落地方式”的隐含绑定需要被拆开。
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
- Goals:
|
||||
- 在不新增独立模态的前提下支持 Suno `lyrics` 动作
|
||||
- 保持 `audio` 路由、默认预设和主入口不变
|
||||
- 显式区分 Suno 的提交动作和最终结果类型
|
||||
- 让歌词任务进入现有任务队列、恢复、重试和画布插入链路
|
||||
- 最大化复用现有文本插入能力,而不是新造歌词节点系统
|
||||
- Non-Goals:
|
||||
- 不把歌词能力改造成独立 `lyrics` 模态
|
||||
- 不在本次中实现歌词编辑器、版本对比或歌词协作能力
|
||||
- 不覆盖所有 Suno 高级动作
|
||||
- 不重构整个任务系统为严格的 discriminated union 体系
|
||||
|
||||
## Decisions
|
||||
|
||||
- Decision: 歌词能力继续归属 `audio` 路由族
|
||||
|
||||
- 预设配置仍然只维护 `audio` 路由
|
||||
- AI 输入栏也仍然通过 `音频` 模式进入
|
||||
- 通过 `sunoAction = 'music' | 'lyrics'` 区分当前执行动作
|
||||
|
||||
- Decision: 显式拆分“提交动作”和“结果类型”
|
||||
|
||||
- `music` 与 `lyrics` 决定 submit path、请求字段和结果提取器
|
||||
- `resultKind` 决定任务队列展示和画布落地方式
|
||||
- `TaskType` 首批保持为 `AUDIO`,避免把歌词任务拆成新的队列大类
|
||||
|
||||
- Decision: `lyrics` 走独立 submit,但继续共用 fetch
|
||||
|
||||
- `music -> POST /suno/submit/music`
|
||||
- `lyrics -> POST /suno/submit/lyrics`
|
||||
- 两者都通过 `GET /suno/fetch/{task_id}` 获取状态与结果
|
||||
|
||||
- Decision: 任务结果模型以增量字段扩展为主
|
||||
|
||||
- 保持现有图片、视频、音频路径影响最小
|
||||
- 为任务结果新增:
|
||||
- `resultKind`
|
||||
- `lyricsText`
|
||||
- `lyricsTitle`
|
||||
- `lyricsTags`
|
||||
- `url` 对歌词结果不再视为必填
|
||||
|
||||
- Decision: 歌词落画布复用现有文本插入链
|
||||
|
||||
- 任务面板点击“插入”时,把歌词结果格式化成 markdown / text 内容
|
||||
- 自动插入同样走文本插入能力
|
||||
- 不创建音频节点,也不伪造音频卡片
|
||||
|
||||
- Decision: UI 参数按动作分域
|
||||
|
||||
- `music` 保留现有参数:
|
||||
- `mv`
|
||||
- `title`
|
||||
- `tags`
|
||||
- `continueSource`
|
||||
- `continueClipId`
|
||||
- `continueAt`
|
||||
- `lyrics` 只需要:
|
||||
- `prompt`
|
||||
- 可选的内部 `notifyHook`
|
||||
- 音乐专属参数在歌词动作下不显示、不提交
|
||||
|
||||
## Proposed Data Shape
|
||||
|
||||
```ts
|
||||
type SunoAction = 'music' | 'lyrics';
|
||||
|
||||
type SunoResultKind = 'audio' | 'lyrics';
|
||||
|
||||
interface SunoGenerationRequest {
|
||||
prompt: string;
|
||||
model?: string;
|
||||
modelRef?: ModelRef | null;
|
||||
sunoAction?: SunoAction;
|
||||
notifyHook?: string;
|
||||
title?: string;
|
||||
tags?: string;
|
||||
mv?: string;
|
||||
continueClipId?: string;
|
||||
continueAt?: number;
|
||||
params?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
interface TaskResult {
|
||||
resultKind?: 'image' | 'video' | 'audio' | 'lyrics' | 'chat';
|
||||
url?: string;
|
||||
urls?: string[];
|
||||
format: string;
|
||||
size: number;
|
||||
title?: string;
|
||||
previewImageUrl?: string;
|
||||
providerTaskId?: string;
|
||||
primaryClipId?: string;
|
||||
clipIds?: string[];
|
||||
clips?: AudioClipResult[];
|
||||
lyricsText?: string;
|
||||
lyricsTitle?: string;
|
||||
lyricsTags?: string[];
|
||||
}
|
||||
```
|
||||
|
||||
## Provider Binding Direction
|
||||
|
||||
首批不建议为 `lyrics` 新增独立模态 binding,而是在现有 audio binding metadata 上增加动作表:
|
||||
|
||||
```ts
|
||||
interface ProviderAudioBindingMetadata {
|
||||
defaultAction?: 'music';
|
||||
supportedActions?: Array<'music' | 'lyrics' | string>;
|
||||
actions?: Record<
|
||||
string,
|
||||
{
|
||||
submitPath: string;
|
||||
resultKind: 'audio' | 'lyrics';
|
||||
requestFields: string[];
|
||||
}
|
||||
>;
|
||||
versionField?: string;
|
||||
versionOptions?: string[];
|
||||
defaultVersion?: string;
|
||||
supportsContinuation?: boolean;
|
||||
supportsUploadContinuation?: boolean;
|
||||
supportsTags?: boolean;
|
||||
supportsTitle?: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
对 Suno 的首批映射可收敛为:
|
||||
|
||||
```ts
|
||||
{
|
||||
operation: 'audio',
|
||||
protocol: 'tuzi.suno.music',
|
||||
pollPathTemplate: '/suno/fetch/{taskId}',
|
||||
metadata: {
|
||||
audio: {
|
||||
defaultAction: 'music',
|
||||
supportedActions: ['music', 'lyrics'],
|
||||
actions: {
|
||||
music: {
|
||||
submitPath: '/suno/submit/music',
|
||||
resultKind: 'audio',
|
||||
requestFields: ['prompt', 'mv', 'title', 'tags', 'continue_clip_id', 'continue_at']
|
||||
},
|
||||
lyrics: {
|
||||
submitPath: '/suno/submit/lyrics',
|
||||
resultKind: 'lyrics',
|
||||
requestFields: ['prompt', 'notify_hook']
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Canvas Formatting Strategy
|
||||
|
||||
歌词插入画布时优先生成语义化文本,而不是直接丢一大段纯文本:
|
||||
|
||||
```md
|
||||
# 战斗进行时
|
||||
|
||||
标签: EDM, 激烈的
|
||||
|
||||
[Chorus]
|
||||
...
|
||||
```
|
||||
|
||||
这样可以直接复用现有:
|
||||
|
||||
- markdown 解析为卡片
|
||||
- 纯文本回退为文本元素
|
||||
|
||||
两条路径。
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- 风险: `TaskType.AUDIO` 继续承载歌词任务会让部分旧代码仍然误判为音频资产
|
||||
- Mitigation: 统一改为优先判断 `result.resultKind`
|
||||
|
||||
- 风险: 把 `url` 改为非必填会触发较多类型修正
|
||||
- Mitigation: 先用增量字段方式扩展,聚焦实际消费点改造
|
||||
|
||||
- 风险: 任务队列过滤仍归入“音频”类,用户可能会觉得歌词不是音频
|
||||
- Mitigation: 在 item 级别增加明确的动作标签,如“歌词”
|
||||
|
||||
- 风险: 画布文本插入格式不稳定,影响可读性
|
||||
- Mitigation: 在服务层统一生成 markdown 模板,不把格式拼装分散到多个组件里
|
||||
|
||||
## Implementation Outline
|
||||
|
||||
1. 扩展 Suno 请求和 binding metadata,加入 `lyrics` 动作
|
||||
2. 抽出共享 fetch 轮询与 action-specific 结果提取器
|
||||
3. 扩展任务结果模型与 IndexedDB 读写结构
|
||||
4. 让 AI 输入栏按动作切换参数面板
|
||||
5. 让任务队列、恢复与重试按 `resultKind` 分支
|
||||
6. 让手动插入与自动插入把歌词结果落到文本插入链路
|
||||
@@ -0,0 +1,81 @@
|
||||
# Change: 增加 Suno 歌词任务与画布落地适配
|
||||
|
||||
## Why
|
||||
|
||||
当前仓库已经具备 Suno 音乐生成的基础闭环:
|
||||
|
||||
- `audio` 已经是一等生成模态
|
||||
- Suno 音乐提交与轮询已经有专门的 adapter / service
|
||||
- 任务队列、自动插入和画布音频节点都已经围绕“音频资产”建立了展示和落地路径
|
||||
|
||||
但 `suno lyrics` 会打破当前实现里的一个默认前提:
|
||||
|
||||
- 它沿用 Suno 的提交与查询链路
|
||||
- 但最终产物不是音频 URL,而是 `title / tags / text` 形式的文本结果
|
||||
|
||||
如果继续沿用当前“`audio` 任务必然产出可播放 URL”的假设,就会出现三类问题:
|
||||
|
||||
- 提交层无法选择 `/suno/submit/lyrics`
|
||||
- 结果提取层会因缺少 `audio_url` 而把任务判定为无结果
|
||||
- 任务队列、自动插入和画布会把歌词结果错误地当成音频卡片处理
|
||||
|
||||
因此需要把 Suno 的“提交动作”和“最终结果类型”显式拆开,并把歌词任务融入现有队列与画布能力中。
|
||||
|
||||
## What Changes
|
||||
|
||||
- 在现有 `audio / suno` 路由体系内新增 `lyrics` 动作
|
||||
- 让 Suno 请求显式区分:
|
||||
- 提交动作:`music` / `lyrics`
|
||||
- 结果类型:`audio` / `lyrics`
|
||||
- 新增 `POST /suno/submit/lyrics` 提交适配,并继续复用 `/suno/fetch/{task_id}` 轮询
|
||||
- 为歌词查询结果增加标准化提取,保留 `text / title / tags / providerTaskId`
|
||||
- 调整任务结果模型与持久化结构,使文本歌词结果不再依赖媒体 URL
|
||||
- 在任务队列中为歌词任务提供独立展示与操作,而不是复用音频卡片假设
|
||||
- 在画布中将歌词结果插入为文本或 markdown 卡片,而不是音频节点
|
||||
- 在音频输入模式中增加动作切换,并让音乐专属参数只在 `music` 动作下展示
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
- `lyrics` 动作的 submit / fetch / normalize 闭环
|
||||
- 继续复用现有 `audio` 默认路由,不新增独立的 `lyrics` 默认路由
|
||||
- 新增 `sunoAction` 和歌词结果字段
|
||||
- 任务队列中的歌词任务展示、重试、恢复和插入能力
|
||||
- 画布文本插入路径对歌词结果的适配
|
||||
- 自动插入路径根据结果类型在音频卡片与文本内容之间分流
|
||||
|
||||
### Out Of Scope
|
||||
|
||||
- 独立的歌词编辑器或画布内富文本歌词组件
|
||||
- 一键把歌词任务自动串联成音乐生成任务
|
||||
- `suno-tags`、`suno_act_tags`、`suno-midi` 等其他 Suno 辅助动作
|
||||
- 为歌词结果新增独立的默认模型类型或独立的产品入口页
|
||||
|
||||
## Impact
|
||||
|
||||
- Affected specs:
|
||||
- `audio-generation`
|
||||
- `generation-task-queue`
|
||||
- `canvas-generated-text`
|
||||
- Affected code:
|
||||
- `packages/drawnix/src/services/audio-api-service.ts`
|
||||
- `packages/drawnix/src/services/model-adapters/*`
|
||||
- `packages/drawnix/src/services/provider-routing/*`
|
||||
- `packages/drawnix/src/types/shared/core.types.ts`
|
||||
- `packages/drawnix/src/services/task-queue-service.ts`
|
||||
- `packages/drawnix/src/services/task-storage-reader.ts`
|
||||
- `packages/drawnix/src/services/media-executor/task-storage-writer.ts`
|
||||
- `packages/drawnix/src/components/ai-input-bar/*`
|
||||
- `packages/drawnix/src/components/task-queue/*`
|
||||
- `packages/drawnix/src/hooks/useAutoInsertToCanvas.ts`
|
||||
- `packages/drawnix/src/services/media-result-handler.ts`
|
||||
- `packages/drawnix/src/services/sw-capabilities/handler.ts`
|
||||
|
||||
## Relationship To Existing Changes
|
||||
|
||||
- 建立在 `add-audio-generation-suno-routing` 已定义的 `audio` 模态与 Suno 路由方案之上
|
||||
- 与 `add-canvas-audio-playback` 互补:
|
||||
- 音乐结果继续进入音频节点
|
||||
- 歌词结果进入文本插入链路
|
||||
- 本变更不会把歌词能力建成新的模态,而是在现有 `audio` 能力族中补齐新的动作和结果类型
|
||||
@@ -0,0 +1,64 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Support Suno Lyrics As A Provider-Backed Audio Action
|
||||
|
||||
The system SHALL support Suno lyrics generation as an action within the existing audio routing model.
|
||||
|
||||
#### Scenario: Submit a lyrics generation request through the Suno lyrics endpoint
|
||||
|
||||
- **GIVEN** the active audio binding supports Suno lyrics generation
|
||||
- **WHEN** the user selects the lyrics action and submits a prompt
|
||||
- **THEN** the system SHALL send the request to `/suno/submit/lyrics`
|
||||
- **AND** SHALL include `prompt`
|
||||
- **AND** MAY include `notify_hook` when configured by the caller
|
||||
|
||||
#### Scenario: Poll a lyrics task through the shared Suno fetch endpoint
|
||||
|
||||
- **GIVEN** a Suno lyrics submit request returns a task identifier
|
||||
- **WHEN** the system tracks that task asynchronously
|
||||
- **THEN** it SHALL query `/suno/fetch/{task_id}`
|
||||
- **AND** SHALL normalize provider status into the internal task state
|
||||
|
||||
#### Scenario: Extract title tags and text from a completed lyrics task
|
||||
|
||||
- **GIVEN** a Suno fetch response for a lyrics task reports success
|
||||
- **WHEN** the nested result payload contains `text`, `title`, and `tags`
|
||||
- **THEN** the system SHALL store the generated lyrics text
|
||||
- **AND** SHALL preserve the returned title and tags when available
|
||||
- **AND** SHALL keep the provider task identifier for follow-up actions
|
||||
|
||||
### Requirement: Distinguish Suno Submit Action From Final Result Kind
|
||||
|
||||
The system SHALL distinguish the selected Suno action from the normalized result kind used by task storage and UI rendering.
|
||||
|
||||
#### Scenario: Lyrics action completes without an audio URL
|
||||
|
||||
- **GIVEN** the selected Suno action is `lyrics`
|
||||
- **WHEN** the provider returns a completed task with text output but no playable audio URL
|
||||
- **THEN** the task SHALL still be considered successfully completed
|
||||
- **AND** the normalized result SHALL be marked as a lyrics result rather than an audio asset
|
||||
- **AND** the system SHALL not fail only because `audio_url` is absent
|
||||
|
||||
#### Scenario: Music action continues to produce audio assets
|
||||
|
||||
- **GIVEN** the selected Suno action is `music`
|
||||
- **WHEN** the provider returns generated clips with playable audio URLs
|
||||
- **THEN** the normalized result SHALL remain an audio result
|
||||
- **AND** the existing audio insertion and playback paths SHALL remain available
|
||||
|
||||
### Requirement: Scope User-Facing Parameters By Suno Action
|
||||
|
||||
The system SHALL expose only the parameters relevant to the currently selected Suno action.
|
||||
|
||||
#### Scenario: Music-only parameters are hidden in lyrics mode
|
||||
|
||||
- **GIVEN** the user has selected the lyrics action in the audio flow
|
||||
- **WHEN** the request form is rendered
|
||||
- **THEN** the UI SHALL not present music-only submit controls such as `mv`, `title`, `tags`, `continue_clip_id`, or `continue_at`
|
||||
- **AND** the primary lyrics flow SHALL not require the user to fill any of those music-specific fields
|
||||
|
||||
#### Scenario: Music mode preserves existing Suno parameter controls
|
||||
|
||||
- **GIVEN** the user has selected the music action in the audio flow
|
||||
- **WHEN** the request form is rendered
|
||||
- **THEN** the UI SHALL continue to expose the existing Suno music parameters supported by the binding
|
||||
@@ -0,0 +1,27 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Insert Generated Lyrics Onto The Canvas As Text Content
|
||||
|
||||
The system SHALL insert generated lyrics results onto the canvas as text-oriented content instead of audio components.
|
||||
|
||||
#### Scenario: Manual canvas insertion formats lyrics as user-readable text
|
||||
|
||||
- **GIVEN** a completed lyrics task includes generated `text` and optional `title` or `tags`
|
||||
- **WHEN** the user manually inserts that result onto the canvas
|
||||
- **THEN** the system SHALL generate a text or markdown representation that includes the lyrics body
|
||||
- **AND** SHALL include the title and tags when available
|
||||
- **AND** SHALL use the existing text insertion capability rather than the audio node insertion capability
|
||||
|
||||
#### Scenario: Auto insert uses the same lyrics text rendering path
|
||||
|
||||
- **GIVEN** a lyrics task is configured for automatic insertion
|
||||
- **WHEN** the task completes successfully
|
||||
- **THEN** the automatic insertion flow SHALL route the result through the same text-oriented insertion path
|
||||
- **AND** SHALL not create an audio node or audio playback component for the lyrics result
|
||||
|
||||
#### Scenario: Plain text fallback remains available when markdown card parsing is not applicable
|
||||
|
||||
- **GIVEN** a generated lyrics payload cannot be meaningfully parsed into markdown cards
|
||||
- **WHEN** the system inserts the result onto the canvas
|
||||
- **THEN** the canvas SHALL still insert the lyrics as readable plain text
|
||||
- **AND** SHALL not fail the insertion only because card parsing is unavailable
|
||||
@@ -0,0 +1,37 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Represent Lyrics Tasks In The Queue Without Audio Asset Assumptions
|
||||
|
||||
The task queue SHALL support completed Suno lyrics tasks even when the result does not contain a media URL.
|
||||
|
||||
#### Scenario: Queue item shows lyrics-specific summary after completion
|
||||
|
||||
- **GIVEN** a Suno lyrics task has completed successfully
|
||||
- **WHEN** the task is rendered in the queue
|
||||
- **THEN** the queue SHALL show that the task action is lyrics generation
|
||||
- **AND** SHALL display available semantic fields such as title, tags, or a lyrics excerpt
|
||||
- **AND** SHALL not render the task as if it were a playable audio card
|
||||
|
||||
#### Scenario: Lyrics task survives persistence and refresh recovery
|
||||
|
||||
- **GIVEN** a completed lyrics task has been written to browser storage
|
||||
- **WHEN** the application restores tasks after a refresh or session recovery
|
||||
- **THEN** the restored task SHALL keep its normalized lyrics result fields
|
||||
- **AND** the queue SHALL still be able to show the same lyrics summary and status
|
||||
|
||||
### Requirement: Provide Lyrics-Specific Queue Actions
|
||||
|
||||
The task queue SHALL expose follow-up actions appropriate for lyrics results.
|
||||
|
||||
#### Scenario: Copy lyrics text from the queue
|
||||
|
||||
- **GIVEN** a completed lyrics task exists in the queue
|
||||
- **WHEN** the user chooses a copy action
|
||||
- **THEN** the system SHALL copy the generated lyrics text rather than trying to open or download an audio URL
|
||||
|
||||
#### Scenario: Insert lyrics into the canvas from the queue
|
||||
|
||||
- **GIVEN** a completed lyrics task exists in the queue
|
||||
- **WHEN** the user chooses to insert the result
|
||||
- **THEN** the queue SHALL route the insertion through the text insertion path
|
||||
- **AND** SHALL not attempt to insert an audio node for that task
|
||||
@@ -0,0 +1,46 @@
|
||||
## 1. Architecture
|
||||
|
||||
- [ ] 1.1 为 Suno 请求增加 `sunoAction`,并定义 `music / lyrics` 两类执行动作
|
||||
- [ ] 1.2 为任务结果增加 `resultKind` 与歌词结果字段,解除“所有 audio 任务都有媒体 URL”的假设
|
||||
- [ ] 1.3 明确歌词结果的统一 markdown / text 画布格式
|
||||
|
||||
## 2. Provider Routing And Service Layer
|
||||
|
||||
- [ ] 2.1 为 Suno audio binding metadata 增加 `lyrics` 动作描述
|
||||
- [ ] 2.2 实现 `POST /suno/submit/lyrics`
|
||||
- [ ] 2.3 复用 `GET /suno/fetch/{task_id}`,并新增歌词结果标准化提取:
|
||||
- `text`
|
||||
- `title`
|
||||
- `tags`
|
||||
- `providerTaskId`
|
||||
- [ ] 2.4 让 `music` 与 `lyrics` 共用轮询框架,但分别走各自的 submit body 与 result extractor
|
||||
|
||||
## 3. Task Queue And Persistence
|
||||
|
||||
- [ ] 3.1 调整 `TaskResult`、任务存储读写和恢复逻辑,支持无 URL 的歌词结果
|
||||
- [ ] 3.2 调整任务队列 item 展示,使歌词任务显示动作、标题、标签和歌词摘要
|
||||
- [ ] 3.3 为歌词任务补齐重试、刷新恢复和搜索行为
|
||||
- [ ] 3.4 为歌词任务增加队列操作:
|
||||
- 复制歌词
|
||||
- 插入画布
|
||||
|
||||
## 4. UI And Canvas
|
||||
|
||||
- [ ] 4.1 在音频模式中增加 `music / lyrics` 动作切换
|
||||
- [ ] 4.2 在 `lyrics` 动作下隐藏音乐专属参数:
|
||||
- `mv`
|
||||
- `title`
|
||||
- `tags`
|
||||
- `continueSource`
|
||||
- `continueClipId`
|
||||
- `continueAt`
|
||||
- [ ] 4.3 调整手动“插入到画布”路径,让歌词结果走文本插入而不是音频节点
|
||||
- [ ] 4.4 调整自动插入路径,按 `resultKind` 在音频节点和文本插入之间分流
|
||||
|
||||
## 5. Verification
|
||||
|
||||
- [ ] 5.1 验证歌词任务能正确提交到 `/suno/submit/lyrics`
|
||||
- [ ] 5.2 验证 `/suno/fetch/{task_id}` 返回歌词结果时能正确完成任务
|
||||
- [ ] 5.3 验证刷新页面后歌词任务仍可恢复与展示
|
||||
- [ ] 5.4 验证歌词任务在队列中可复制、可插入,且不会被误渲染为音频卡片
|
||||
- [ ] 5.5 验证音乐生成既有链路不回归
|
||||
Reference in New Issue
Block a user