RedHatAI/mpt-7b-chat-pruned50-quant-ds

By RedHatAI · ↓ 8 · ♥ 4 · Updated 2023-10-19

What's in this repo

This page lists the files in RedHatAI/mpt-7b-chat-pruned50-quant-ds, including model-orig.onnx, model.onnx.

2 ONNX models (.onnx files)
10 files, including 4 .json, 2 .onnx, 1 .md, 1 .py, 1 .data
Last updated 2023-10-19

ONNX files

File	Size	Download
model-orig.onnx	—	Download
model.onnx	—	Download

How to use

These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.

README

Sparse MPT-7B-Chat - DeepSparse

Chat-aligned MPT 7b model pruned to 50% and quantized using SparseGPT for inference with DeepSparse

from deepsparse import TextGeneration
model = TextGeneration(model="hf:neuralmagic/mpt-7b-chat-pruned50-quant")
model("Tell me a joke.", max_new_tokens=50)

Model info

text-generation transformers

transformers onnx mpt text-generation deepsparse custom_code region:us

↓ 8 ♥ 4

Created: 2023-09-20

Updated: 2023-10-19

View on Hugging Face

Files (10)

.gitattributes

README.md

config.json

configuration_mpt.py

model-orig.onnx ONNX

model.data

model.onnx ONNX

special_tokens_map.json

tokenizer.json

tokenizer_config.json

Related models

More text-generation models

Browse all text-generation models