Initial TrueGrowth source import
This commit is contained in:
@@ -0,0 +1,219 @@
|
||||
## Context
|
||||
|
||||
当前标准 Kling 视频接口具有两个明显特征:
|
||||
|
||||
- 提交动作分为:
|
||||
- `POST /kling/v1/videos/text2video`
|
||||
- `POST /kling/v1/videos/image2video`
|
||||
- 真正执行版本通过请求体 `model_name` 指定,而不是通过发现到的能力 ID 直接调用
|
||||
|
||||
同时,运行时发现已经能够识别出能力型视频标识,例如:
|
||||
|
||||
- `kling_video`
|
||||
- `kling_image`
|
||||
- `kling_effects`
|
||||
- `kling_extend`
|
||||
- `kling_lip_sync`
|
||||
- `kling_virtual_try_on`
|
||||
|
||||
这说明 Kling 与 Suno 有相似的结构特征:
|
||||
|
||||
- 发现层暴露的是能力入口
|
||||
- 执行层需要额外版本字段
|
||||
|
||||
但当前标准 Kling adapter 仍然主要围绕版本型模型 ID 运转,例如 `kling-v1-6`。这让以下几层出现语义错位:
|
||||
|
||||
1. discovery 发现的是能力
|
||||
2. selector 选中的可能是能力
|
||||
3. adapter 执行时却把 `model` 理解为可直接提交的执行版本
|
||||
|
||||
此外,标准 Kling 视频与 `kling-video-o1`、`kling-video-o1-edit` 不是同一条协议语义:
|
||||
|
||||
- 标准 Kling 视频走 `/kling/v1/videos/{action}`
|
||||
- `o1` 系列是另一类特例模型,不应被标准 Kling 能力路由误吞
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
- Goals:
|
||||
- 让标准 Kling 视频沿用 Suno 式“能力入口 + 执行版本”模式
|
||||
- 让 `kling_video` 成为标准 Kling 视频的一等能力模型
|
||||
- 让 `model_name` 成为标准 Kling 视频的执行版本字段
|
||||
- 让 `text2video / image2video` 的版本限制显式建模
|
||||
- 保持旧版本型模型 ID 的兼容,避免历史任务和用户设置立刻失效
|
||||
- Non-Goals:
|
||||
- 本次不抽象所有 Kling 能力为统一多动作产品面板
|
||||
- 本次不处理 `o1` 系列的专用协议
|
||||
- 本次不一次性把所有 Kling 高级字段做成复杂 UI 控件
|
||||
|
||||
## Decisions
|
||||
|
||||
- Decision: 区分标准 Kling 能力模型和执行版本
|
||||
|
||||
- `kling_video` 表示标准 Kling 视频能力入口
|
||||
- `model_name` 表示具体执行版本,例如 `kling-v3`
|
||||
- 发现层与选择层优先使用 `kling_video`
|
||||
- 执行层始终向标准接口提交 `model_name`
|
||||
|
||||
- Decision: action 继续由请求语义决定,而不是再拆成两个能力模型
|
||||
|
||||
- 显式传入 `klingAction2` 时优先使用
|
||||
- 否则:
|
||||
- 有参考图 => `image2video`
|
||||
- 无参考图 => `text2video`
|
||||
|
||||
- Decision: 版本合法性绑定到 action
|
||||
|
||||
- `text2video` 的可选版本:
|
||||
- `kling-v3`
|
||||
- `kling-v2-6`
|
||||
- `kling-v2-1`
|
||||
- `kling-v1-6`
|
||||
- `kling-v1-5`
|
||||
- `image2video` 的可选版本:
|
||||
- `kling-v3`
|
||||
- `kling-v2-6`
|
||||
- `kling-v2-1`
|
||||
- `kling-v1-6`
|
||||
- `kling-v1-5`
|
||||
|
||||
- Decision: 历史版本型模型 ID 作为兼容输入,而不是主路径
|
||||
|
||||
- 若请求模型为 `kling-v1-5` 或 `kling-v1-6`
|
||||
- 运行时仍允许提交
|
||||
- 但内部按“标准 Kling 能力 + 指定 model_name”解释
|
||||
|
||||
- Decision: `o1` 系列与标准 Kling 视频能力显式隔离
|
||||
|
||||
- `kling-video-o1`
|
||||
- `kling-video-o1-edit`
|
||||
- 不参与标准 `kling.video` binding 推断
|
||||
- 不复用 `/kling/v1/videos/{action}` 的标准版本校验
|
||||
|
||||
- Decision: UI 先暴露最小必要版本选择
|
||||
|
||||
- 第一阶段保证用户可以为标准 Kling 能力选择:
|
||||
- `model_name`
|
||||
- `klingAction2`
|
||||
- `mode`
|
||||
- `cfg_scale`
|
||||
- `negative_prompt`
|
||||
- `camera_control` 通过平铺字段纳入参数体系:
|
||||
- `camera_control_type`
|
||||
- `camera_horizontal / vertical / pan / tilt / roll / zoom`
|
||||
- `image_tail` 与 `callback_url` 暂不纳入本阶段可视化参数体系
|
||||
|
||||
- Decision: Kling 数值参数同时在 UI 元数据和 adapter 层校验
|
||||
|
||||
- `cfg_scale` 取值范围固定为 `[0, 1]`
|
||||
- `camera_control` 的六个数值字段固定为 `[-10, 10]` 且必须是整数
|
||||
- 参数面板与 MCP schema 暴露相同的边界信息
|
||||
- adapter 仍负责最终校验,避免 workflow / MCP 直接绕过 UI
|
||||
|
||||
## Proposed Data Model
|
||||
|
||||
```ts
|
||||
interface ProviderVideoBindingMetadata {
|
||||
allowedDurations?: string[];
|
||||
defaultDuration?: string;
|
||||
durationMode?: 'request-param' | 'model-alias';
|
||||
durationField?: string;
|
||||
durationToModelMap?: Record<string, string>;
|
||||
strictDurationValidation?: boolean;
|
||||
resultMode?: 'inline-url' | 'download-content';
|
||||
downloadPathTemplate?: string;
|
||||
|
||||
versionField?: string;
|
||||
versionOptions?: string[];
|
||||
defaultVersion?: string;
|
||||
versionOptionsByAction?: Record<string, string[]>;
|
||||
}
|
||||
|
||||
interface KlingResolvedRequest {
|
||||
action: 'text2video' | 'image2video';
|
||||
modelName: string;
|
||||
}
|
||||
```
|
||||
|
||||
标准 Kling 视频 binding 期望类似:
|
||||
|
||||
```ts
|
||||
{
|
||||
operation: 'video',
|
||||
protocol: 'kling.video',
|
||||
requestSchema: 'kling.video.auto-action-json',
|
||||
submitPath: '/kling/v1/videos/{action}',
|
||||
pollPathTemplate: '/kling/v1/videos/{action}/{taskId}',
|
||||
metadata: {
|
||||
video: {
|
||||
versionField: 'model_name',
|
||||
defaultVersion: 'kling-v1-6',
|
||||
versionOptions: [
|
||||
'kling-v3',
|
||||
'kling-v2-6',
|
||||
'kling-v2-1',
|
||||
'kling-v1-6',
|
||||
'kling-v1-5'
|
||||
],
|
||||
versionOptionsByAction: {
|
||||
text2video: ['kling-v3', 'kling-v2-6', 'kling-v2-1', 'kling-v1-6', 'kling-v1-5'],
|
||||
image2video: ['kling-v3', 'kling-v2-6', 'kling-v2-1', 'kling-v1-6', 'kling-v1-5']
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Execution Flow
|
||||
|
||||
1. 发现层识别到 `kling_video`
|
||||
2. 选择器保留 `profileId::kling_video`
|
||||
3. 运行时 planner 解析到标准 Kling 视频 binding
|
||||
4. adapter 根据请求判断 action:
|
||||
- `text2video`
|
||||
- `image2video`
|
||||
5. adapter 从以下来源解析执行版本:
|
||||
- `params.model_name`
|
||||
- binding `defaultVersion`
|
||||
- 历史兼容的 `request.model === kling-v*`
|
||||
6. adapter 按 action 校验版本是否允许
|
||||
7. adapter 构造标准 Kling 请求体并提交
|
||||
|
||||
## Compatibility Strategy
|
||||
|
||||
### Discovery / Selection
|
||||
|
||||
- 新发现到的 `kling_video` 应作为标准主路径能力模型
|
||||
- 老的静态版本型模型可以继续存在,但不再作为未来扩展的主心智
|
||||
|
||||
### Runtime
|
||||
|
||||
- 当请求模型是 `kling_video` 时:
|
||||
- 必须解析 `model_name`
|
||||
- 当请求模型是 `kling-v1-5` 或 `kling-v1-6` 时:
|
||||
- 直接将该值视为显式执行版本
|
||||
- 同时仍走标准 Kling adapter
|
||||
|
||||
### UI
|
||||
|
||||
- 若当前选择的是 `kling_video`
|
||||
- 参数区应能选择 `model_name`
|
||||
- 若当前选择的是旧版本型模型
|
||||
- UI 可以继续工作
|
||||
- 但内部行为应与显式 `model_name` 一致
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- 风险: 运行时同时支持 `kling_video` 和 `kling-v*`,可能产生双轨心智
|
||||
- Mitigation: 将 `kling_video` 作为主路径,仅保留 `kling-v*` 兼容输入
|
||||
|
||||
- 风险: Kling 版本集合仍按 action 维护,后续若再次分叉,UI 或 workflow 可能产生非法组合
|
||||
- Mitigation: 在 adapter 层做最终校验,并在参数层尽量按 action 过滤
|
||||
|
||||
- 风险: `o1` 系列误被标准 Kling 识别规则吞掉
|
||||
- Mitigation: 明确在 binding inference 和 adapter 匹配中排除 `kling-video-o1*`
|
||||
|
||||
## Open Questions
|
||||
|
||||
- 默认版本是否应从 `kling-v1-6` 逐步切换到 `kling-v3`
|
||||
- `image_tail` 是否在第一阶段就进入 UI 参数面板
|
||||
- `mode / cfg_scale / camera_control` 是否应在后续改为动态 binding 参数,而不是继续依赖静态透传
|
||||
@@ -0,0 +1,71 @@
|
||||
# Change: 调整 Kling 视频能力路由为“能力模型 + 执行版本”
|
||||
|
||||
## Why
|
||||
|
||||
当前 Kling 标准视频链路和 Suno 已经采用的“能力入口 / 执行版本”模式不一致:
|
||||
|
||||
- 运行时发现更接近能力标识,例如 `kling_video`
|
||||
- 实际提交接口要求通过请求体 `model_name` 传入执行版本
|
||||
- Kling 标准视频仍需要按 action 建模版本合法性,即使当前 `text2video` 与 `image2video` 已共享同一组主版本
|
||||
- 当前实现仍主要把 `kling-v1-6` 这类版本字符串当作直接调用模型,导致发现层、路由层和执行层的语义混在一起
|
||||
|
||||
这会带来几个具体问题:
|
||||
|
||||
- 发现到 `kling_video` 时,运行时无法稳定映射到正确的执行版本
|
||||
- `text2video` 与 `image2video` 的版本约束只能散落在 adapter 条件分支中
|
||||
- 老的版本型模型 ID 与新的能力型模型 ID 并存时,默认路由和参数表单容易失真
|
||||
- `kling-video-o1`、`kling-video-o1-edit` 这类特例模型与标准 Kling 视频能力共用识别规则,后续扩展风险高
|
||||
|
||||
## What Changes
|
||||
|
||||
- 将标准 Kling 视频能力明确建模为能力模型 `kling_video`
|
||||
- 将真正执行版本明确建模为请求字段 `model_name`
|
||||
- 为标准 Kling 视频 binding 增加版本字段元数据,包括:
|
||||
- `versionField`
|
||||
- `defaultVersion`
|
||||
- `versionOptions`
|
||||
- `versionOptionsByAction`
|
||||
- 保留 `text2video / image2video` 的 action 分流,但把版本合法性和默认值绑定到 action
|
||||
- 保持对历史版本型模型 ID 的兼容,例如:
|
||||
- `kling-v1-5`
|
||||
- `kling-v1-6`
|
||||
- 显式将 `kling-video-o1` 与 `kling-video-o1-edit` 排除在标准 Kling 视频能力路由之外
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
- 标准 Kling 视频能力 `kling_video` 的路由与执行建模
|
||||
- `model_name` 版本参数的运行时解析与默认值策略
|
||||
- `text2video` 与 `image2video` 的版本约束建模
|
||||
- 旧版本型模型 ID 到新能力型路由的兼容策略
|
||||
- 最小必要的参数暴露,用于选择标准 Kling 执行版本
|
||||
|
||||
### Out Of Scope
|
||||
|
||||
- 本次不接入 `kling_effects`
|
||||
- 本次不接入 `kling_extend`
|
||||
- 本次不接入 `kling_lip_sync`
|
||||
- 本次不接入 `kling_virtual_try_on`
|
||||
- 本次不将 `kling-video-o1` 与 `kling-video-o1-edit` 迁入这套标准 Kling 视频能力模型
|
||||
- 本次不一次性为 Kling 全量高级字段设计完整可视化表单,例如复杂 `camera_control` 编辑器
|
||||
|
||||
## Impact
|
||||
|
||||
- Affected specs:
|
||||
- `provider-protocol-routing`
|
||||
- Affected code:
|
||||
- `packages/drawnix/src/services/provider-routing/*`
|
||||
- `packages/drawnix/src/services/model-adapters/kling-adapter.ts`
|
||||
- `packages/drawnix/src/services/model-adapters/registry.ts`
|
||||
- `packages/drawnix/src/services/video-binding-utils.ts`
|
||||
- `packages/drawnix/src/constants/model-config.ts`
|
||||
- `packages/drawnix/src/constants/video-model-config.ts`
|
||||
- `packages/drawnix/src/components/ttd-dialog/ai-video-generation.tsx`
|
||||
- `packages/drawnix/src/components/ai-input-bar/*`
|
||||
|
||||
## Relationship To Existing Changes
|
||||
|
||||
- 本变更建立在 `add-provider-protocol-routing` 的协议绑定架构之上
|
||||
- 本变更复用 `add-audio-generation-suno-routing` 已采用的“能力标识不等于执行版本”设计原则
|
||||
- 本变更不改变标准 Kling 接口的 submit / poll URL,而是调整运行时如何解释能力模型与版本字段
|
||||
@@ -0,0 +1,75 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Resolve Standard Kling Video Capability Separately From Executable Versions
|
||||
|
||||
The system SHALL treat standard Kling video capability identifiers separately from the executable version strings required by submit requests.
|
||||
|
||||
#### Scenario: Discovered Kling capability is not submitted as executable version directly
|
||||
|
||||
- **GIVEN** a provider profile exposes the discovered standard Kling video capability `kling_video`
|
||||
- **WHEN** the user submits a standard Kling video generation request
|
||||
- **THEN** the system SHALL route the request through the standard Kling video binding
|
||||
- **AND** SHALL send the executable version through the request field `model_name`
|
||||
- **AND** SHALL not assume that `kling_video` itself is a valid executable version string
|
||||
|
||||
#### Scenario: Legacy Kling version-style model IDs remain executable
|
||||
|
||||
- **GIVEN** an existing task or setting still references a legacy standard Kling model ID such as `kling-v1-6`
|
||||
- **WHEN** the request is executed through the current runtime
|
||||
- **THEN** the system SHALL continue to accept that input
|
||||
- **AND** SHALL interpret it as an explicit `model_name` for the standard Kling binding
|
||||
- **AND** SHALL keep the request on the standard Kling submit and poll endpoints
|
||||
|
||||
### Requirement: Validate Standard Kling Version Choices Against The Selected Action
|
||||
|
||||
The system SHALL validate standard Kling executable versions against the selected submit action.
|
||||
|
||||
#### Scenario: Text-to-video allows the current standard Kling version set
|
||||
|
||||
- **GIVEN** the standard Kling request resolves to `text2video`
|
||||
- **WHEN** the runtime selects or receives an executable version
|
||||
- **THEN** the system SHALL allow only the versions supported by the text-to-video endpoint
|
||||
- **AND** SHALL include `kling-v2-6` in that allowed set when provided by binding metadata
|
||||
|
||||
#### Scenario: Image-to-video allows the extended Kling version set
|
||||
|
||||
- **GIVEN** the standard Kling request resolves to `image2video`
|
||||
- **WHEN** the runtime selects or receives an executable version
|
||||
- **THEN** the system SHALL allow the versions supported by the image-to-video endpoint
|
||||
- **AND** SHALL include `kling-v2-6` in that allowed set when provided by binding metadata
|
||||
|
||||
#### Scenario: Image-to-video still requires a reference image
|
||||
|
||||
- **GIVEN** the standard Kling request resolves to `image2video`
|
||||
- **WHEN** the user submits the request without a reference image
|
||||
- **THEN** the system SHALL fail validation before submit
|
||||
- **AND** SHALL not send an invalid image-to-video request to the provider
|
||||
|
||||
### Requirement: Exclude Non-Standard Kling O1 Models From Standard Kling Capability Routing
|
||||
|
||||
The system SHALL keep `kling-video-o1` style models outside the standard Kling capability routing path.
|
||||
|
||||
#### Scenario: O1 model does not reuse standard Kling submit action routing
|
||||
|
||||
- **GIVEN** the selected video model is `kling-video-o1` or `kling-video-o1-edit`
|
||||
- **WHEN** the runtime resolves provider protocol bindings
|
||||
- **THEN** the system SHALL not treat that model as the standard `kling_video` capability
|
||||
- **AND** SHALL not force it through the standard `/kling/v1/videos/{action}` routing and version validation rules
|
||||
|
||||
### Requirement: Enforce Standard Kling Numeric Parameter Constraints Before Submit
|
||||
|
||||
The system SHALL enforce the documented numeric constraints for standard Kling video requests before submit.
|
||||
|
||||
#### Scenario: cfg_scale stays within the documented range
|
||||
|
||||
- **GIVEN** a standard Kling request includes `cfg_scale`
|
||||
- **WHEN** the request is prepared for submit
|
||||
- **THEN** the system SHALL accept only values in the inclusive range `[0, 1]`
|
||||
- **AND** SHALL fail validation before submit when the value falls outside that range
|
||||
|
||||
#### Scenario: camera_control uses bounded integer values
|
||||
|
||||
- **GIVEN** a standard Kling text-to-video request includes `camera_control` values through flat params or a nested `camera_control` object
|
||||
- **WHEN** the request is prepared for submit
|
||||
- **THEN** the system SHALL accept only integer values in the inclusive range `[-10, 10]` for `horizontal`, `vertical`, `pan`, `tilt`, `roll`, and `zoom`
|
||||
- **AND** SHALL fail validation before submit when any of those values is non-integer or out of range
|
||||
@@ -0,0 +1,33 @@
|
||||
## 1. Architecture
|
||||
|
||||
- [ ] 1.1 为标准 Kling 视频定义“能力模型 + 执行版本”路由约束
|
||||
- [ ] 1.2 设计标准 Kling 的 binding metadata,补齐 `versionField / defaultVersion / versionOptions / versionOptionsByAction`
|
||||
- [ ] 1.3 明确标准 Kling 与 `kling-video-o1*` 的边界和排除规则
|
||||
|
||||
## 2. Provider Routing And Adapter Layer
|
||||
|
||||
- [ ] 2.1 调整 Kling binding 推断,使 `kling_video` 作为标准 Kling 视频能力入口
|
||||
- [ ] 2.2 在标准 Kling binding 中声明 `model_name` 版本元数据
|
||||
- [ ] 2.3 更新 Kling adapter,使其按 action 解析并校验 `model_name`
|
||||
- [ ] 2.4 保持对 `kling-v1-5`、`kling-v1-6` 等历史版本型模型 ID 的兼容
|
||||
- [ ] 2.5 显式排除 `kling-video-o1` 与 `kling-video-o1-edit`,避免误走标准 Kling 路由
|
||||
- [ ] 2.6 在标准 Kling image2video 请求中预留 `image_tail` 透传能力
|
||||
- [ ] 2.7 在标准 Kling adapter 中校验 `cfg_scale` 与 `camera_control` 的官方取值范围
|
||||
|
||||
## 3. UI And Parameter Exposure
|
||||
|
||||
- [ ] 3.1 为标准 Kling 能力模型暴露最小必要的 `model_name` 版本选择
|
||||
- [ ] 3.2 保证 `duration / size / aspect_ratio` 与标准 Kling binding 元数据保持一致
|
||||
- [ ] 3.3 暴露 `klingAction2 / mode / cfg_scale / negative_prompt` 参数,不阻塞主链路
|
||||
- [ ] 3.4 通过平铺字段组装 `camera_control`,暂不处理 `image_tail / callback_url`
|
||||
- [ ] 3.5 为 Kling 数值参数补齐 `min / max / step` 约束元数据,并同步 MCP 描述
|
||||
|
||||
## 4. Verification
|
||||
|
||||
- [ ] 4.1 验证 `kling_video` 在无参考图时走 `text2video`
|
||||
- [ ] 4.2 验证 `kling_video` 在有参考图时走 `image2video`
|
||||
- [ ] 4.3 验证 `text2video` 可使用 `kling-v2-6`
|
||||
- [ ] 4.4 验证 `image2video` 可使用 `kling-v2-6`
|
||||
- [ ] 4.5 验证历史 `kling-v1-6` 请求仍可成功映射到标准 Kling adapter
|
||||
- [ ] 4.6 验证 `kling-video-o1*` 不会误走标准 Kling `/kling/v1/videos/{action}` 路由
|
||||
- [ ] 4.7 验证超范围 `cfg_scale` 与非法 `camera_control` 会在提交前失败
|
||||
Reference in New Issue
Block a user