AXERA-TECH/LivePortrait

By AXERA-TECH · ↓ 4 · ♥ 4 · Updated 2025-06-21

What's in this repo

This page lists the files in AXERA-TECH/LivePortrait, including warp.onnx, 1k3d68.onnx, 2d106det.onnx and more.

7 ONNX models (.onnx files)
147 files, including 63 .py, 27 .pyc, 17 .jpg, 8 .mp4, 7 .onnx, 5 .pkl
Last updated 2025-06-21

ONNX files

File	Size	Download
python/axmodels/warp.onnx	—	Download
python/pretrained_weights/insightface/models/buffalo_l/1k3d68.onnx	—	Download
python/pretrained_weights/insightface/models/buffalo_l/2d106det.onnx	—	Download
python/pretrained_weights/insightface/models/buffalo_l/det_10g.onnx	—	Download
python/pretrained_weights/insightface/models/buffalo_l/genderage.onnx	—	Download
python/pretrained_weights/insightface/models/buffalo_l/w600k_r50.onnx	—	Download
python/pretrained_weights/liveportrait/landmark.onnx	—	Download

How to use

These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.

README

showcase

LivePortrait

This version of LivePortrait has been converted to run on the Axera NPU using w8a16 quantization.

This model has been optimized with the following:

Compatible with Pulsar2 version: 3.4

Convert tools links:

For those who are interested in model conversion:

Support Platform

AX650/AX8850
- M4N-Dock(爱芯派Pro)
- M.2 Accelerator card

How to use

Download all files from this repository to the device.

(py310) axera@dell:~/samples/LivePortrait$ tree -L 2
.
├── assets
│   └── examples
├── config.json
├── python
│   ├── axmodels
│   ├── cropper.py
│   ├── infer_onnx.py
│   ├── infer.py
│   ├── pretrained_weights
│   ├── requirements.txt
│   └── utils
└── README.md

7 directories, 6 files

python env requirement

pyaxengine

https://github.com/AXERA-TECH/pyaxengine

wget https://github.com/AXERA-TECH/pyaxengine/releases/download/0.1.3.rc1/axengine-0.1.3-py3-none-any.whl
pip install axengine-0.1.3-py3-none-any.whl

others

pip install -r python/requirements.txt

Inference with AX650 or AX8850 Host, such as AX650 DEMO BOARD, M4N-DOCK(爱芯派Pro)

root@ax650 ~/yongqiang/LivePortrait.axera # python3 ./python/infer.py --source ./assets/examples/source/s0.jpg --driving ./assets/examples/driving/d8.jpg --models ./python/axmodels/ --output-dir ./axmodel_infer
[INFO] Available providers:  ['AxEngineExecutionProvider']
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Chip type: ChipType.MC50
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Engine version: 2.12.0s
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 0f7260e8
[INFO] Using provider: AxEngineExecutionProvider
[INFO] Model type: 2 (triple core)
[INFO] Compiler version: 3.3 144960ad
FaceAnalysisDIY warmup time: 0.598s
LandmarkRunner warmup time: 0.769s
2025-05-30 09:56:12.247 | INFO     | __main__:main:727 - Start making driving motion template...
2025-05-30 09:56:14.770 | INFO     | __main__:main:747 - Prepared pasteback mask done.
2025-05-30 09:56:17.219 | INFO     | __main__:main:787 - The output of image-driven portrait animation is an image.
2025-05-30 09:56:30.701 | DEBUG    | __main__:warp_decode:647 - warp time: 13.475s
2025-05-30 09:56:31.118 | INFO     | __main__:main:881 - Animated image: ./axmodel_infer/s0--d8.jpg
2025-05-30 09:56:31.118 | INFO     | __main__:main:882 - Animated image with concat: ./axmodel_infer/s0--d8_concat.jpg
2025-05-30 09:56:31.167 | DEBUG    | __main__:<module>:894 - LivePortrait axmodel infer time: 32.455s

Inference with M.2 Accelerator card

What is M.2 Accelerator card?, Show this DEMO based on x86.

Image

(py310) axera@dell:~/samples/LivePortrait$ python ./python/infer.py --source ./assets/examples/source/s0.jpg --driving ./assets/examples/driving/d8.jpg --models ./python/axmodels/ --output-dir ./axmodel_infer
[INFO] Available providers:  ['AXCLRTExecutionProvider']
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 144960ad
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 0f7260e8
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 144960ad
FaceAnalysisDIY warmup time: 0.024s
[20:02:20] LandmarkRunner warmup time: 0.031s    human_landmark_runner.py:95
2025-05-29 20:02:20.727 | INFO     | __main__:main:727 - Start making driving motion template...
2025-05-29 20:02:20.972 | INFO     | __main__:main:747 - Prepared pasteback mask done.
2025-05-29 20:02:21.449 | INFO     | __main__:main:787 - The output of image-driven portrait animation is an image.
2025-05-29 20:02:25.475 | DEBUG    | __main__:warp_decode:647 - warp time: 4.017s
2025-05-29 20:02:25.892 | INFO     | __main__:main:881 - Animated image: ./axmodel_infer/s0--d8.jpg
2025-05-29 20:02:25.892 | INFO     | __main__:main:882 - Animated image with concat: ./axmodel_infer/s0--d8_concat.jpg
2025-05-29 20:02:25.904 | DEBUG    | __main__:<module>:894 - LivePortrait axmodel infer time: 8.165s
(py310) axera@dell:~/samples/LivePortrait$

Here, --models specifies the storage path for the *.axmodel model.

The output of axmodel-infer is as follows:

output_concat output

Video

python3 ./python/infer.py --source ./assets/examples/source/s0.jpg --driving ./assets/examples/driving/d0.mp4 --models ./python/axmodels/ --output-dir ./axmodel_infer

The output of axmodel-infer is as follows:

output_concat output

Model info

image-to-video

onnx image-to-video en base_model:KlingTeam/LivePortrait base_model:quantized:KlingTeam/LivePortrait license:mit region:us

↓ 4 ♥ 4

Created: 2025-05-29

Updated: 2025-06-21

View on Hugging Face

Files (147)

.gitattributes

README.md

assets/examples/driving/d0.mp4

assets/examples/driving/d14_trim.mp4

assets/examples/driving/d18.mp4

assets/examples/driving/d19.mp4

assets/examples/driving/d6_trim.mp4

assets/examples/driving/d8.jpg

assets/examples/driving/d8.pkl

assets/examples/driving/doug-i-can't-understand-it.mp4

assets/examples/result/01.gif

assets/examples/result/02.gif

assets/examples/result/s0--d0_axmodel.mp4

assets/examples/result/s0--d0_concat_axmodel.mp4

assets/examples/result/s0--d8.jpg

assets/examples/result/s0--d8_axmodel.jpg

assets/examples/result/s0--d8_concat.jpg

assets/examples/result/s0--d8_concat_axmodel.jpg

assets/examples/source/s0.jpg

assets/examples/source/s10.jpg

assets/examples/source/s12.jpg

assets/examples/source/s5.jpg

assets/examples/source/s6.jpg

assets/examples/source/s7.jpg

assets/examples/source/s9.jpg

assets/showcase2.gif

config.json

python/axmodels/feature_extractor.axmodel

python/axmodels/motion_extractor.axmodel

python/axmodels/spade_generator.axmodel

python/axmodels/stitching_retargeting.axmodel

python/axmodels/warp.onnx ONNX

python/cropper.py

python/infer.py

python/infer_onnx.py

python/pretrained_weights/insightface/models/buffalo_l/1k3d68.onnx ONNX

python/pretrained_weights/insightface/models/buffalo_l/2d106det.onnx ONNX

python/pretrained_weights/insightface/models/buffalo_l/det_10g.onnx ONNX

python/pretrained_weights/insightface/models/buffalo_l/genderage.onnx ONNX

python/pretrained_weights/insightface/models/buffalo_l/w600k_r50.onnx ONNX

python/pretrained_weights/liveportrait/landmark.onnx ONNX

python/requirements.txt

python/utils/__init__.py

python/utils/__pycache__/__init__.cpython-310.pyc

python/utils/__pycache__/crop.cpython-310.pyc

python/utils/__pycache__/human_landmark_runner.cpython-310.pyc

python/utils/__pycache__/rprint.cpython-310.pyc

python/utils/__pycache__/timer.cpython-310.pyc

python/utils/crop.py

python/utils/dependencies/XPose/config_model/UniPose_SwinT.py

python/utils/dependencies/XPose/config_model/coco_transformer.py

python/utils/dependencies/XPose/models/UniPose/__init__.py

python/utils/dependencies/XPose/models/UniPose/attention.py

python/utils/dependencies/XPose/models/UniPose/backbone.py

python/utils/dependencies/XPose/models/UniPose/deformable_transformer.py

python/utils/dependencies/XPose/models/UniPose/fuse_modules.py

python/utils/dependencies/XPose/models/UniPose/mask_generate.py

python/utils/dependencies/XPose/models/UniPose/ops/functions/__init__.py

python/utils/dependencies/XPose/models/UniPose/ops/functions/ms_deform_attn_func.py

python/utils/dependencies/XPose/models/UniPose/ops/modules/__init__.py

python/utils/dependencies/XPose/models/UniPose/ops/modules/ms_deform_attn.py

python/utils/dependencies/XPose/models/UniPose/ops/modules/ms_deform_attn_key_aware.py

python/utils/dependencies/XPose/models/UniPose/ops/setup.py

python/utils/dependencies/XPose/models/UniPose/ops/src/cpu/ms_deform_attn_cpu.cpp

python/utils/dependencies/XPose/models/UniPose/ops/src/cpu/ms_deform_attn_cpu.h

python/utils/dependencies/XPose/models/UniPose/ops/src/cuda/ms_deform_attn_cuda.cu

python/utils/dependencies/XPose/models/UniPose/ops/src/cuda/ms_deform_attn_cuda.h

python/utils/dependencies/XPose/models/UniPose/ops/src/cuda/ms_deform_im2col_cuda.cuh

python/utils/dependencies/XPose/models/UniPose/ops/src/ms_deform_attn.h

python/utils/dependencies/XPose/models/UniPose/ops/src/vision.cpp

python/utils/dependencies/XPose/models/UniPose/ops/test.py

python/utils/dependencies/XPose/models/UniPose/position_encoding.py

python/utils/dependencies/XPose/models/UniPose/swin_transformer.py

python/utils/dependencies/XPose/models/UniPose/transformer_deformable.py

python/utils/dependencies/XPose/models/UniPose/transformer_vanilla.py

python/utils/dependencies/XPose/models/UniPose/unipose.py

python/utils/dependencies/XPose/models/UniPose/utils.py

python/utils/dependencies/XPose/models/__init__.py

python/utils/dependencies/XPose/models/registry.py

python/utils/dependencies/XPose/predefined_keypoints.py

python/utils/dependencies/XPose/transforms.py

python/utils/dependencies/XPose/util/addict.py

python/utils/dependencies/XPose/util/box_ops.py

python/utils/dependencies/XPose/util/config.py

python/utils/dependencies/XPose/util/keypoint_ops.py

python/utils/dependencies/XPose/util/misc.py

python/utils/dependencies/insightface/__init__.py

python/utils/dependencies/insightface/__pycache__/__init__.cpython-310.pyc

python/utils/dependencies/insightface/app/__init__.py

python/utils/dependencies/insightface/app/__pycache__/__init__.cpython-310.pyc

python/utils/dependencies/insightface/app/__pycache__/common.cpython-310.pyc

python/utils/dependencies/insightface/app/__pycache__/face_analysis.cpython-310.pyc

python/utils/dependencies/insightface/app/common.py

python/utils/dependencies/insightface/app/face_analysis.py

python/utils/dependencies/insightface/data/__init__.py

python/utils/dependencies/insightface/data/__pycache__/__init__.cpython-310.pyc

python/utils/dependencies/insightface/data/__pycache__/image.cpython-310.pyc

python/utils/dependencies/insightface/data/__pycache__/pickle_object.cpython-310.pyc

python/utils/dependencies/insightface/data/image.py

python/utils/dependencies/insightface/data/images/Tom_Hanks_54745.png

python/utils/dependencies/insightface/data/images/mask_black.jpg

python/utils/dependencies/insightface/data/images/mask_blue.jpg

python/utils/dependencies/insightface/data/images/mask_green.jpg

python/utils/dependencies/insightface/data/images/mask_white.jpg

python/utils/dependencies/insightface/data/images/t1.jpg

python/utils/dependencies/insightface/data/objects/meanshape_68.pkl

python/utils/dependencies/insightface/data/pickle_object.py

python/utils/dependencies/insightface/data/rec_builder.py

python/utils/dependencies/insightface/model_zoo/__init__.py

python/utils/dependencies/insightface/model_zoo/__pycache__/__init__.cpython-310.pyc

python/utils/dependencies/insightface/model_zoo/__pycache__/arcface_onnx.cpython-310.pyc

python/utils/dependencies/insightface/model_zoo/__pycache__/attribute.cpython-310.pyc

python/utils/dependencies/insightface/model_zoo/__pycache__/inswapper.cpython-310.pyc

python/utils/dependencies/insightface/model_zoo/__pycache__/landmark.cpython-310.pyc

python/utils/dependencies/insightface/model_zoo/__pycache__/model_zoo.cpython-310.pyc

python/utils/dependencies/insightface/model_zoo/__pycache__/retinaface.cpython-310.pyc

python/utils/dependencies/insightface/model_zoo/__pycache__/scrfd.cpython-310.pyc

python/utils/dependencies/insightface/model_zoo/arcface_onnx.py

python/utils/dependencies/insightface/model_zoo/attribute.py

python/utils/dependencies/insightface/model_zoo/inswapper.py

python/utils/dependencies/insightface/model_zoo/landmark.py

python/utils/dependencies/insightface/model_zoo/model_store.py

python/utils/dependencies/insightface/model_zoo/model_zoo.py

python/utils/dependencies/insightface/model_zoo/retinaface.py

python/utils/dependencies/insightface/model_zoo/scrfd.py

python/utils/dependencies/insightface/utils/__init__.py

python/utils/dependencies/insightface/utils/__pycache__/__init__.cpython-310.pyc

python/utils/dependencies/insightface/utils/__pycache__/constant.cpython-310.pyc

python/utils/dependencies/insightface/utils/__pycache__/download.cpython-310.pyc

python/utils/dependencies/insightface/utils/__pycache__/face_align.cpython-310.pyc

python/utils/dependencies/insightface/utils/__pycache__/filesystem.cpython-310.pyc

python/utils/dependencies/insightface/utils/__pycache__/storage.cpython-310.pyc

python/utils/dependencies/insightface/utils/__pycache__/transform.cpython-310.pyc

python/utils/dependencies/insightface/utils/constant.py

python/utils/dependencies/insightface/utils/download.py

python/utils/dependencies/insightface/utils/face_align.py

python/utils/dependencies/insightface/utils/filesystem.py

python/utils/dependencies/insightface/utils/storage.py

python/utils/dependencies/insightface/utils/transform.py

python/utils/face_analysis_diy.py

python/utils/human_landmark_runner.py

python/utils/resources/clip_embedding_68.pkl

python/utils/resources/clip_embedding_9.pkl

python/utils/resources/lip_array.pkl

python/utils/resources/mask_template.png

python/utils/rprint.py

python/utils/timer.py

Related models

More image-to-video models

Browse all image-to-video models

AXERA-TECH/LivePortrait

What's in this repo

ONNX files

How to use

README

LivePortrait

Convert tools links:

Support Platform

How to use

python env requirement

pyaxengine

others

Inference with AX650 or AX8850 Host, such as AX650 DEMO BOARD, M4N-DOCK(爱芯派Pro)

Inference with M.2 Accelerator card

Image

Video

Model info

Files (147)

Related models

More from AXERA-TECH

More image-to-video models

Popular models