aamocualg-hse/sdxl-onnx

## SDXL Turbo (ONNX Runtime, CUDA)

This repository provides **ONNX-converted SDXL Turbo models** optimized for inference with **ONNX Runtime CUDA Execution Provider**.  
It **requires an NVIDIA GPU** and does **not** support CPU or DirectML.

Models are generated using **Olive**:

```bash
python stable_diffusion_xl.py \
  --provider cuda \
  --model_id stabilityai/sdxl-turbo \
  --optimize \
  --use_fp16_fixed_vae

Model Overview

• Developer: Stability AI
• Type: Diffusion-based text-to-image model
• License: STABILITY AI Non-Commercial Research Community License
• Base model: stabilityai/sdxl-turbo
• VAE: madebyollin/sdxl-vae-fp16-fix

The VAE output slightly differs from the original SDXL VAE, but visual quality is generally comparable and suitable for most use cases.

Requirements

• NVIDIA GPU
• CUDA 12.x
• Docker + NVIDIA Container Toolkit
• ≥64 GB RAM recommended (otherwise reduce build parallelism)

Usage

0. Install NVIDIA Container Toolkit

Follow the official NVIDIA installation guide.

1. Clone ONNX Runtime

git clone https://github.com/microsoft/onnxruntime
cd onnxruntime

2. Download SDXL Turbo ONNX models

git lfs install
git clone https://huggingface.co/tlwu/sdxl-turbo-onnxruntime

3. Start Docker container

docker run --rm -it --gpus all \
  -v $PWD:/workspace \
  nvcr.io/nvidia/pytorch:23.10-py3 \
  /bin/bash

4. Build ONNX Runtime with CUDA support

export CUDACXX=/usr/local/cuda-12.2/bin/nvcc
git config --global --add safe.directory '*'

sh build.sh \
  --config Release \
  --build_shared_lib \
  --parallel \
  --use_cuda \
  --cuda_version 12.2 \
  --cuda_home /usr/local/cuda-12.2 \
  --cudnn_home /usr/lib/x86_64-linux-gnu/ \
  --build_wheel \
  --skip_tests \
  --use_tensorrt \
  --tensorrt_home /usr/src/tensorrt \
  --cmake_extra_defines onnxruntime_BUILD_UNIT_TESTS=OFF \
  --cmake_extra_defines CMAKE_CUDA_ARCHITECTURES=80 \
  --allow_running_as_root

Install the generated wheel:

pip install build/Linux/Release/dist/onnxruntime_gpu-*.whl --force-reinstall

Notes:

• CMAKE_CUDA_ARCHITECTURES:

  • A100 → 80
  • RTX 3090 → 86
  • RTX 4090 → 89

• If system RAM < 64 GB, use:

  --parallel 4 --nvcc_threads 1
  

5. Install dependencies

pip install --upgrade pip
cd onnxruntime/python/tools/transformers/models/stable_diffusion
pip install -r requirements-cuda12.txt
pip install --upgrade polygraphy onnx-graphsurgeon \
  --extra-index-url https://pypi.ngc.nvidia.com

6. Run inference

python demo_txt2img_xl.py \
  "starry night over Golden Gate Bridge by van gogh" \
  --version xl-turbo \
  --engine-dir /workspace/sdxl-turbo-onnxruntime