Esperanto/llava-dermatology-7b-kvc-AWQ-int4-onnx
By Esperanto · ↓ 0 · ♥ 1 · Updated 2024-12-05
What's in this repo
This page lists the files in Esperanto/llava-dermatology-7b-kvc-AWQ-int4-onnx, including model.onnx and more.
- 4 ONNX models (.onnx files)
- 745 files, including 224 .qweight, 224 .qzeros, 224 .scales, 66 .weight, 4 .onnx, 1 .md
- Last updated 2024-12-05
ONNX files
| File | Size | Download |
|---|---|---|
| clip/model.onnx | — | Download |
| embeds/model.onnx | — | Download |
| llama/model.onnx | — | Download |
| mmp/model.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
Model Card for Model ID
Model Details
Fine tuned version of LLaVA 7B through transfer learning on Dermatology Images. This version is the ONNX model in int4 precision with KVC (obtained with AWQ) for the LLM and the rest is in fp16 precision. Stay tuned for instructions on how to run this ONNX pipeline with OnnxRuntime!
Model Description
Given the rise in demand for reliable and immediate medical advice, particularly in dermatology, this tool empowers users to consult an "online dermatologist" by providing essential information, an image of the affected area, and asking follow-up questions, closely mimicking an actual consultation.
To achieve this, we fine-tuned LLaVA 7B with LoRA on the Skin Condition Image Network (SCIN) dataset, equipping it with the ability to process and analyze user-provided images, such as rashes, inflammations, or other skin conditions, and give patients diagnoses with high accuracy.
- Developed by: Esperanto Technologies
- Finetuned from model: llava-hf/llava-1.5-7b-hf
- Blog: Updating soon
Uses
Put the information in the demo questionnaire together and form context such as:
'This is a CLOSE UP picture of a patient's skin problem. The patient is within the age group of AGE 18 TO 29. The patient is a MALE. The patient is Hispanic Latino or Spanish Origin. The texture of the skin problem is flat, rough or flaky. The skin problem comes with increasing size, itching and burning symptoms. The skin problem comes with pain. The skin problem doesn't come with other symptoms. The patient's skin problem has been ONE TO FOUR WEEKS.'
Along with system prompt and user question like "Please diagnose what skin condition is most likely in the picture."
Model info
Created: 2024-11-06
Updated: 2024-12-05
View on Hugging FaceFiles (745)
Related models
More from Esperanto
More models
No task type declared.