camenduru/unianimate

UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation

This repo include the checkpoints for UniAnimate:

• "models/dw-ll_ucoco_384.onnx": the checkpoint for dwpose extraction.

• "models/open_clip_pytorch_model.bin": the checkpoint for clip embedding.

• "models/unianimate_16f_32f_non_ema_223000.pth": the checkpoint for human image animation in UniAnimate (16/32 frames).

• "models/yolox_l.onnx": the checkpoint for dwpose extraction.

• "models/v2-1_512-ema-pruned.ckpt": the checkpoint for Stable Diffusion.

BibTeX

If this repo is useful to you, please cite our corresponding technical paper.

@article{wang2024unianimate,
      title={UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation},
      author={Wang, Xiang and Zhang, Shiwei and Gao, Changxin and Wang, Jiayu and Zhou, Xiaoqiang and Zhang, Yingya and Yan, Luxin and Sang, Nong},
      journal={arXiv preprint arXiv:2406.01188},
      year={2024}
}
@inproceedings{TFT2V,
      title={A Recipe for Scaling up Text-to-Video Generation with Text-free Videos},
      author={Wang, Xiang and Zhang, Shiwei and Yuan, Hangjie and Qing, Zhiwu and Gong, Biao and Zhang, Yingya and Shen, Yujun and Gao, Changxin and Sang, Nong},
      booktitle={CVPR},
      year={2024}
}
@article{VideoComposer,
      title={VideoComposer: Compositional Video Synthesis with Motion Controllability},
      author={Wang, Xiang and Yuan, Hangjie and Zhang, Shiwei and Chen, Dayou and Wang, Jiuniu and Zhang, Yingya and Shen, Yujun and Zhao, Deli and Zhou, Jingren},
      journal={NeurIPS},
      year={2023}
}