opencv/handpose_estimation_mediapipe

Hand pose estimation from MediaPipe Handpose

This model estimates 21 hand keypoints per detected hand from palm detector. (The image below is referenced from MediaPipe Hands Keypoints)

Hand gesture classification demo (0-9)

This model is converted from TFlite to ONNX using following tools:

• TFLite model to ONNX: https://github.com/onnx/tensorflow-onnx
• simplified by onnx-simplifier

Note:

• The int8-quantized model may produce invalid results due to a significant drop of accuracy.
• Visit https://github.com/google/mediapipe/blob/master/docs/solutions/models.md#hands for models of larger scale.
• handpose_estimation_mediapipe_2023feb_int8bq.onnx represents the block-quantized version in int8 precision and is generated using block_quantize.py with block_size=64.

Demo

Run the following commands to try the demo:

# detect on camera input
python demo.py
# detect on an image
python demo.py -i /path/to/image -v

Example outputs

License

All files in this directory are licensed under Apache 2.0 License.

Reference

• MediaPipe Handpose: https://developers.google.com/mediapipe/solutions/vision/hand_landmarker
• MediaPipe hands model and model card: https://github.com/google/mediapipe/blob/master/docs/solutions/models.md#hands
• Handpose TFJS:https://github.com/tensorflow/tfjs-models/tree/master/handpose
• Int8 model quantized with rgb evaluation set of FreiHAND: https://lmb.informatik.uni-freiburg.de/resources/datasets/FreihandDataset.en.html