samwell/laborview-ultrasound

LaborView Ultrasound (Edge)

Lightweight multi-task AI for real-time intrapartum ultrasound on mobile devices

Model Description

LaborView Ultrasound is an edge-optimized multi-task model for comprehensive labor monitoring on mobile devices. It performs three tasks simultaneously in a single forward pass:

Key Features

• Lightweight: ~21 MB (vs 1.6 GB full model)
• Fast: <50ms on mobile devices
• Multi-task: Complete clinical assessment in one pass
• Cross-platform: ONNX, CoreML (iOS), TFLite (Android)
• Uncertainty Weighting: Learned task balancing

Architecture

Input Image (256×256 RGB)
         │
         ▼
┌─────────────────────────┐
│      MobileViT-S        │  Vision Encoder
│   apple/mobilevit-small │  640-dim features
│       (~5.6M params)    │
└───────────┬─────────────┘
            │
     ┌──────┴──────┐
     ▼             ▼
┌─────────┐  ┌─────────────┐
│ Pooled  │  │  Sequence   │
│Features │  │  Features   │
│  (640)  │  │  (N×640)    │
└────┬────┘  └──────┬──────┘
     │              │
     ▼              ▼
┌─────────┐  ┌─────────────┐
│Projector│  │ Seg Decoder │
│  (256)  │  │(128→64→32)  │
└────┬────┘  └──────┬──────┘
     │              │
  ┌──┴──┐           │
  ▼     ▼           ▼
┌────┐┌────┐  ┌──────────┐
│Cls ││Reg │  │ Seg Mask │
│Head││Head│  │(3×H×W)   │
└────┘└────┘  └──────────┘
  │     │          │
  ▼     ▼          ▼
Plane  AoP,    Symphysis,
Logits HSD     Head Masks
(6)    (2)     (3×256×256)

Model Outputs

# ONNX outputs (in order)
outputs = session.run(None, {"image": img})
plane_logits = outputs[0]   # (1, 6) - Plane classification
seg_masks = outputs[1]      # (1, 3, 256, 256) - Segmentation
labor_params = outputs[2]   # (1, 2) - [AoP, HSD]

Training

• Dataset: HAI-DEF Challenge
• Base Model: apple/mobilevit-small (640-dim, ~5.6M params)
• Input Size: 256×256 (edge-optimized)
• Multi-Task Loss: Uncertainty-weighted (Kendall et al.)
  • Segmentation: Dice + Cross-Entropy
  • Classification: Cross-Entropy
  • Regression: Smooth L1
• Full Fine-tuning: MobileViT trained end-to-end
• Optimization: AdamW, OneCycleLR, mixed precision

Output Interpretation

Segmentation Classes

Plane Classification

Labor Parameters

AoP Clinical Interpretation:

How to Use

ONNX Runtime (Python)

import onnxruntime as ort
import numpy as np
from PIL import Image

# Load model
session = ort.InferenceSession("laborview_mobilevit.onnx")

# Preprocess (256x256, ImageNet normalization)
image = Image.open("ultrasound.png").convert("RGB").resize((256, 256))
img = np.array(image).astype(np.float32) / 255.0
img = (img - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]
img = img.transpose(2, 0, 1)[np.newaxis, ...].astype(np.float32)

# Multi-task inference
plane_logits, seg_masks, labor_params = session.run(None, {"image": img})

# Parse outputs
plane = ["transperineal", "transabdominal", "oblique",
         "sagittal", "axial", "other"][np.argmax(plane_logits)]
mask = np.argmax(seg_masks, axis=1)[0]
aop, hsd = labor_params[0]

print(f"Plane: {plane}")
print(f"AoP: {aop:.1f}°, HSD: {hsd:.1f}px")

Flutter (Dart)

import 'package:flutter_onnxruntime/flutter_onnxruntime.dart';

class LaborViewService extends ChangeNotifier {
  OrtSession? _session;

  Future<void> loadModel() async {
    final options = OrtSessionOptions();
    _session = await OrtSession.fromAsset(
      'assets/models/laborview_mobilevit.onnx',
      options,
    );
  }

  Future<LaborViewResult> analyze(Uint8List imageBytes) async {
    // Preprocess to 256x256, ImageNet normalize
    final input = _preprocessImage(imageBytes);

    // Run multi-task inference
    final outputs = await _session!.run([input]);

    return LaborViewResult(
      planeClass: _argmax(outputs[0]),
      segMask: _argmax2D(outputs[1]),
      aop: outputs[2][0],
      hsd: outputs[2][1],
    );
  }
}

iOS (Swift + CoreML)

import CoreML
import Vision

class LaborViewAnalyzer {
    private let model: VNCoreMLModel

    init() throws {
        let config = MLModelConfiguration()
        config.computeUnits = .cpuAndNeuralEngine
        let laborview = try LaborView(configuration: config)
        model = try VNCoreMLModel(for: laborview.model)
    }

    func analyze(image: CGImage) async throws -> LaborViewResult {
        let request = VNCoreMLRequest(model: model)
        let handler = VNImageRequestHandler(cgImage: image)
        try handler.perform([request])

        guard let results = request.results as? [VNCoreMLFeatureValueObservation] else {
            throw AnalysisError.noResults
        }

        return LaborViewResult(
            planeLogits: results[0].featureValue.multiArrayValue!,
            segMask: results[1].featureValue.multiArrayValue!,
            laborParams: results[2].featureValue.multiArrayValue!
        )
    }
}

Android (Kotlin + TFLite)

class LaborViewInterpreter(context: Context) {
    private val interpreter: Interpreter

    init {
        val options = Interpreter.Options().apply {
            setNumThreads(4)
            addDelegate(GpuDelegate())
        }
        val model = FileUtil.loadMappedFile(context, "laborview.tflite")
        interpreter = Interpreter(model, options)
    }

    fun analyze(bitmap: Bitmap): LaborViewResult {
        val input = preprocessBitmap(bitmap)  // 1x3x256x256

        val planeLogits = Array(1) { FloatArray(6) }
        val segMask = Array(1) { Array(3) { Array(256) { FloatArray(256) } } }
        val laborParams = Array(1) { FloatArray(2) }

        val outputs = mapOf(
            0 to planeLogits,
            1 to segMask,
            2 to laborParams
        )

        interpreter.runForMultipleInputsOutputs(arrayOf(input), outputs)

        return LaborViewResult(
            plane = planeLogits[0].argmax(),
            mask = segMask[0].argmax2D(),
            aop = laborParams[0][0],
            hsd = laborParams[0][1]
        )
    }
}

Clinical Metrics (Post-processing)

from clinical_metrics import compute_all_metrics

# Get comprehensive clinical assessment from segmentation
metrics = compute_all_metrics(
    segmentation_mask=mask,
    symphysis_class=1,
    head_class=2
)

# Full output
print(f"=== Clinical Assessment ===")
print(f"AoP: {metrics.aop:.1f}° - {metrics.aop_interpretation}")
print(f"HSD: {metrics.hsd:.1f}px - {metrics.hsd_interpretation}")
print(f"Head Circumference: {metrics.head_circumference:.0f}px")
print(f"Head Area: {metrics.head_area:.0f}px²")
print(f"Quality: {metrics.segmentation_quality} ({metrics.confidence:.0%})")
print(f"Progress: {metrics.labor_progress.upper()}")
print(f"Recommendation: {metrics.recommendation}")

Model Files

Performance

Multi-Task Metrics

Inference Latency

Size Comparison

Limitations

1. Resolution Trade-off: 256×256 may miss fine details
2. Training Data: Single dataset/protocol
3. Image Quality: Sensitive to artifacts/shadows
4. Anatomical Variations: May struggle with unusual cases
5. Calibration: Needs device-specific pixel-to-mm conversion
6. Regression Accuracy: Direct predictions less accurate than geometry-computed

Safety & Regulatory

Intended Use

Research and development only. Not approved for clinical diagnosis.

Before Clinical Deployment

1. Obtain FDA 510(k) / CE marking
2. Validate on local patient population
3. Verify hardware integration
4. Train operators on limitations
5. Implement adverse event monitoring

Known Risks

• False negatives may delay intervention
• False positives may cause unnecessary intervention
• Model drift with different equipment

Ethical Considerations

• Health Equity: Designed for resource-limited settings
• Privacy: Local inference, no data transmission
• Bias: Validate across diverse populations
• Transparency: Full documentation provided

Citation

@software{laborview_edge_2024,
  title = {LaborView Ultrasound: Edge Multi-Task Model for Labor Monitoring},
  author = {Samuel},
  year = {2024},
  url = {https://huggingface.co/samwell/laborview-ultrasound},
  note = {Multi-task MobileViT: segmentation + classification + regression}
}

Related Resources

• laborview-medsiglip - Full model (higher accuracy)
• Demo Space - Try online
• HAI-DEF Challenge - Dataset
• MobileViT Paper - Base architecture

License

Apache 2.0