SebRincon/anycalib-onnx

AnyCalib ONNX — Ray Prediction Head

ONNX export of the AnyCalib ray prediction neural network, ready for deployment with ONNX Runtime (Python, C++, Web/WASM, Mobile).

Variants

Architecture

• Backbone: DINOv2 ViT-L/14 (304M params)
• Decoder: LightDPT (15.2M params)
• Head: ConvexTangentDecoder (0.6M params)
• Source model: anycalib_gen

Usage — Python

import onnxruntime as ort
import numpy as np

sess = ort.InferenceSession("model_fp16.onnx")

# RGB [0,1], size must be divisible by 14
image = np.random.rand(1, 3, 518, 518).astype(np.float32)

rays, tangent_coords = sess.run(None, {"image": image})
# rays: (1, 3, 518, 518) — unit rays per pixel
# tangent_coords: (1, 2, 518, 518) — tangent space coords

Usage — ONNX Runtime Web (WASM)

import * as ort from 'onnxruntime-web';

// Use WASM backend
ort.env.wasm.wasmPaths = 'https://cdn.jsdelivr.net/npm/onnxruntime-web/dist/';

const session = await ort.InferenceSession.create('./model_int8.onnx', {
  executionProviders: ['wasm'],
});

// Prepare input: (1, 3, 518, 518) RGB float32
const imageData = new Float32Array(1 * 3 * 518 * 518);
// ... fill with normalized RGB data ...

const inputTensor = new ort.Tensor('float32', imageData, [1, 3, 518, 518]);
const results = await session.run({ image: inputTensor });

const rays = results.rays;           // (1, 3, 518, 518)
const tangentCoords = results.tangent_coords;  // (1, 2, 518, 518)

Usage — Transformers.js

import { env } from '@huggingface/transformers';

// Point to this repo
env.allowLocalModels = false;

// Load ONNX model directly
const session = await ort.InferenceSession.create(
  'https://huggingface.co/SebRincon/anycalib-onnx/resolve/main/model_int8.onnx'
);

Input/Output Spec

• Input: image — (B, 3, H, W) RGB float32 in [0, 1], H and W divisible by 14
• Output: rays — (B, 3, H, W) unit rays on S^2 manifold
• Output: tangent_coords — (B, 2, H, W) tangent space coordinates

Note

The Calibrator (RANSAC + Gauss-Newton camera fitting) is NOT included in the ONNX model. It must run as a lightweight CPU post-processing step. See the calibrator implementation for details.

Related

• AnyCalib Raw — Raw PyTorch weights (safetensors)
• AnyCalib Source — Original repository