ONNX 模型库
返回模型

说明文档

<div align ="center"> <h1> Proteus-ID </h1> <h3> Proteus-ID：身份一致且运动连贯的视频定制 </h3> <div align="center"> </div>

项目主页  arXiv  </div>

作者：Guiyu Zhang<sup>1</sup>、Chen Shi<sup>1</sup>、Zijian Jiang<sup>1</sup>、Xunzhi Xiang<sup>2</sup>、Jingjing Qian<sup>1</sup>、Shaoshuai Shi<sup>3</sup>、Li Jiang†<sup>1</sup>

<sup>1</sup> 香港中文大学（深圳） <sup>2</sup> 南京大学  <sup>3</sup> 滴滴出行 Voyager Research

待办事项

  • [x] 发布 arXiv 技术报告
  • [x] 发布完整代码
  • [ ] 发布数据集（即将推出）

🛠️ 环境要求与安装

环境配置

# 0. 克隆仓库
git clone --depth=1 https://github.com/grenoble-zhang/Proteus-ID.git

cd /nfs/dataset-ofs-voyager-research/guiyuzhang/Opensource/code/Proteus-ID-main

# 1. 创建 conda 环境
conda create -n proteusid python=3.11.0
conda activate proteusid

# 3. 安装 PyTorch 和其他依赖
# CUDA 12.6
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu126
# 4. 安装 pip 依赖
pip install -r requirements.txt

下载模型

cd util
python download_weights.py
python down_raft.py

准备就绪后，权重文件将按以下格式组织：

🔦 ckpts/
├── 📂 face_encoder/
├── 📂 scheduler/
├── 📂 text_encoder/
├── 📂 tokenizer/
├── 📂 transformer/
├── 📂 vae/
├── 📄 configuration.json
├── 📄 model_index.json

🏋️ 训练

# 单卡训练
bash train_single_rank.sh
# 多卡训练
bash train_multi_rank.sh

🏄️ 推理

python inference.py --img_file_path assets/example_images/1.png --json_file_path assets/example_images/1.json

BibTeX

如果您觉得我们的工作对您的研究有帮助，请考虑引用我们的论文：

@article{zhang2025proteus,
  title={Proteus-ID: ID-Consistent and Motion-Coherent Video Customization},
  author={Zhang, Guiyu and Shi, Chen and Jiang, Zijian and Xiang, Xunzhi and Qian, Jingjing and Shi, Shaoshuai and Jiang, Li},
  journal={arXiv preprint arXiv:2506.23729},
  year={2025}
}

致谢

感谢以下优秀的开源工作和模型：CogVideoX；ConsisID；diffusers。

fateforward/Proteus-ID

作者 fateforward

diffusers
↓ 1 ♥ 1

创建时间: 2026-01-14 17:31:37+00:00

更新时间: 2026-02-02 17:06:14+00:00

在 Hugging Face 上查看

文件 (39)

.gitattributes
.hfd/aria2c_urls.txt
.hfd/last_download_command
.hfd/repo_metadata.json
README.md
configuration.json
data_process/step1_yolov8_face.pt
data_process/step1_yolov8_head.pt
face_encoder/EVA02_CLIP_L_336_psz14_s6B.pt
face_encoder/detection_Resnet50_Final.pth
face_encoder/glint360k_curricular_face_r101_backbone.bin
face_encoder/models/antelopev2/1k3d68.onnx ONNX
face_encoder/models/antelopev2/2d106det.onnx ONNX
face_encoder/models/antelopev2/genderage.onnx ONNX
face_encoder/models/antelopev2/glintr100.onnx ONNX
face_encoder/models/antelopev2/scrfd_10g_bnkps.onnx ONNX
face_encoder/models/buffalo_l/1k3d68.onnx ONNX
face_encoder/models/buffalo_l/2d106det.onnx ONNX
face_encoder/models/buffalo_l/det_10g.onnx ONNX
face_encoder/models/buffalo_l/genderage.onnx ONNX
face_encoder/models/buffalo_l/w600k_r50.onnx ONNX
face_encoder/parsing_bisenet.pth
face_encoder/parsing_parsenet.pth
model_index.json
scheduler/scheduler_config.json
text_encoder/config.json
text_encoder/model-00001-of-00002.safetensors
text_encoder/model-00002-of-00002.safetensors
text_encoder/model.safetensors.index.json
tokenizer/added_tokens.json
tokenizer/special_tokens_map.json
tokenizer/spiece.model
tokenizer/tokenizer_config.json
transformer/config.json
transformer/diffusion_pytorch_model-00001-of-00002.safetensors
transformer/diffusion_pytorch_model-00002-of-00002.safetensors
transformer/diffusion_pytorch_model.safetensors.index.json
vae/config.json
vae/diffusion_pytorch_model.safetensors