bollscoasts/pix2act-onnx
By bollscoasts · ↓ 14 · ♥ 0 · Updated 2026-01-24
What's in this repo
This page lists the files in bollscoasts/pix2act-onnx, including decoder_model.onnx, decoder_model_merged.onnx, decoder_with_past_model.onnx and more.
- 4 ONNX models (.onnx files)
- 12 files, including 6 .json, 4 .onnx, 1 .md
- Last updated 2026-01-24
ONNX files
| File | Size | Download |
|---|---|---|
| decoder_model.onnx | — | Download |
| decoder_model_merged.onnx | — | Download |
| decoder_with_past_model.onnx | — | Download |
| encoder_model.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
Pix2Act ONNX
ONNX export of google/pix2struct-base for GUI automation via Transformers.js/WebGPU.
Model Details
| Property | Value |
|---|---|
| Original Model | google/pix2struct-base |
| Architecture | Pix2Struct (Vision2Seq) |
| Parameters | ~282M |
| Export Format | ONNX (opset 17) |
| Quantization | INT8 dynamic |
| Browser Eligible | Yes |
Description
Pix2Struct is a pretrained image-to-text model designed for understanding visually situated language. It uses a pixel-only approach making it ideal for:
- Screenshot understanding
- GUI automation
- Document parsing
- Web interaction
Usage
With ONNX Runtime (Python)
import onnxruntime as ort
from transformers import AutoProcessor
from PIL import Image
# Load processor
processor = AutoProcessor.from_pretrained("google/pix2struct-base")
# Load ONNX model
session = ort.InferenceSession("encoder_model.onnx")
# Process image
image = Image.open("screenshot.png")
inputs = processor(images=image, return_tensors="np")
# Run inference
outputs = session.run(None, dict(inputs))
With Transformers.js (Browser)
import { pipeline } from "@xenova/transformers";
// Load the ONNX model
const pipe = await pipeline("image-to-text", "bollscoasts/pix2act-onnx");
// Run inference on screenshot
const result = await pipe(image);
console.log(result);
Export Details
Exported using HotelBench ONNX export utilities.
hotelbench export-onnx pix2act -o models/pix2act-onnx --quantize
License
Apache 2.0
Model info
Created: 2026-01-24
Updated: 2026-01-24
View on Hugging Face