vuiseng9/bert-base-squadv1-qat-bt

By vuiseng9 · ↓ 1 · ♥ 0 · Updated 2022-12-24

What's in this repo

This page lists the files in vuiseng9/bert-base-squadv1-qat-bt, including bert-base-squadv1-qat-bt.onnx, nncf_network.onnx, optimum-model.onnx and more.

6 ONNX models (.onnx files)
83 files, including 22 .json, 17 .bin, 11 .xml, 9 .mapping, 6 .onnx, 4 .md
Last updated 2022-12-24

ONNX files

File	Size	Download
bert-base-squadv1-qat-bt.onnx	—	Download
ir/jpqd-lt-r0.01/mo-2022.3.0-9078/nncf_network.onnx	—	Download
ir/jpqd-lt-r0.02-8eph-optimum-cfg-QEmb/nncf_network.onnx	—	Download
ir/jpqd-lt-r0.02-8eph-optimum-cfg/nncf_network.onnx	—	Download
ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/nncf_network.onnx	—	Download
ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/optimum-model.onnx	—	Download

How to use

These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.

README

This model is a quantized-aware transfer learning of bert-base-uncased on Squadv1 using OpenVINO/NNCF. Applied optimization includes:

NNCF Quantize-Aware Training - Symmetric 8-bit for both weight and activation on all learnable layers.
Custom distillation with fine-tuned model csarron/bert-base-uncased-squad-v1

  eval_exact_match = 80.8136
  eval_f1          = 88.2594
  eval_samples     =   10784

Setup

# OpenVINO/NNCF
git clone https://github.com/vuiseng9/nncf && cd nncf
git checkout tld-poc
git reset --hard 1dec7afe7a4b567c059fcf287ea2c234980fded2
python setup.py develop
pip install -r examples/torch/requirements.txt

# Huggingface nn_pruning
git clone https://github.com/vuiseng9/nn_pruning && cd nn_pruning
git checkout reproduce-evaluation
git reset --hard 2d4e196d694c465e43e5fbce6c3836d0a60e1446
pip install -e ".[dev]"

# Huggingface Transformers
git clone https://github.com/vuiseng9/transformers && cd transformers
git checkout tld-poc
git reset --hard 10a1e29d84484e48fd106f58957d9ffc89dc43c5
pip install -e .
head -n 1 examples/pytorch/question-answering/requirements.txt | xargs -i pip install {}

# Additional dependencies
pip install onnx

Train

wget https://huggingface.co/vuiseng9/bert-base-squadv1-qat-bt/raw/main/nncf_bert_squad_qat.json
NNCF_CFG=/path/to/downloaded_nncf_cfg_above #to-revise

OUTROOT=/path/to/train_output_root #to-revise
WORKDIR=transformers/examples/pytorch/question-answering #to-revise
RUNID=bert-base-squadv1-qat-bt

cd $WORKDIR

OUTDIR=$OUTROOT/$RUNID
mkdir -p $OUTDIR

export CUDA_VISIBLE_DEVICES=0
NEPOCH=2

python run_qa.py \
    --model_name_or_path bert-base-uncased \
    --dataset_name squad \
    --do_eval \
    --do_train \
    --evaluation_strategy steps \
    --eval_steps 250 \
    --learning_rate 3e-5 \
    --lr_scheduler_type cosine_with_restarts \
    --warmup_ratio 0.25 \
    --cosine_cycles 1 \
    --teacher csarron/bert-base-uncased-squad-v1 \
    --teacher_ratio 0.9 \
    --num_train_epochs $NEPOCH \
    --per_device_eval_batch_size 128 \
    --per_device_train_batch_size 16 \
    --max_seq_length 384 \
    --doc_stride 128 \
    --save_steps 250 \
    --nncf_config $NNCF_CFG \
    --logging_steps 1 \
    --overwrite_output_dir \
    --run_name $RUNID \
    --output_dir $OUTDIR

Eval

This repo must be cloned locally.

git clone https://huggingface.co/vuiseng9/bert-base-squadv1-qat-bt
MODELROOT=/path/to/cloned_repo_above #to-revise

export CUDA_VISIBLE_DEVICES=0

OUTDIR=eval-bert-base-squadv1-qat-bt
WORKDIR=transformers/examples/pytorch/question-answering #to-revise
cd $WORKDIR
mkdir $OUTDIR

nohup python run_qa.py  \
    --model_name_or_path vuiseng9/bert-base-uncased-squad  \
    --dataset_name squad  \
    --qat_checkpoint $MODELROOT/checkpoint-10750  \
    --nncf_config $MODELROOT/nncf_bert_squad_qat.json  \
    --to_onnx $OUTDIR/bert-base-squadv1-qat-bt.onnx  \
    --do_eval  \
    --per_device_eval_batch_size 128  \
    --max_seq_length 384  \
    --doc_stride 128  \
    --overwrite_output_dir \
    --output_dir $OUTDIR 2>&1 | tee $OUTDIR/run.log &

Model info

transformers

transformers pytorch onnx bert endpoints_compatible region:us

↓ 1 ♥ 0

Created: 2022-03-02

Updated: 2022-12-24

View on Hugging Face

Files (83)

.gitattributes

README.md

all_results.json

bert-base-squadv1-qat-bt.onnx ONNX

checkpoint-10750/config.json

checkpoint-10750/optimizer.pt

checkpoint-10750/pytorch_model.bin

checkpoint-10750/rng_state.pth

checkpoint-10750/scheduler.pt

checkpoint-10750/special_tokens_map.json

checkpoint-10750/tokenizer.json

checkpoint-10750/tokenizer_config.json

checkpoint-10750/trainer_state.json

checkpoint-10750/training_args.bin

checkpoint-10750/vocab.txt

compressed_graph.dot

config.json

eval_nbest_predictions.json

eval_predictions.json

eval_results.json

ir/bert-base-squadv1-qat-bt.bin

ir/bert-base-squadv1-qat-bt.mapping

ir/bert-base-squadv1-qat-bt.xml

ir/jpqd-lt-r0.01/mo-2022.3.0-9078/mo-pruned-ir/nncf_network.bin

ir/jpqd-lt-r0.01/mo-2022.3.0-9078/mo-pruned-ir/nncf_network.mapping

ir/jpqd-lt-r0.01/mo-2022.3.0-9078/mo-pruned-ir/nncf_network.xml

ir/jpqd-lt-r0.01/mo-2022.3.0-9078/nncf_network.bin

ir/jpqd-lt-r0.01/mo-2022.3.0-9078/nncf_network.mapping

ir/jpqd-lt-r0.01/mo-2022.3.0-9078/nncf_network.onnx ONNX

ir/jpqd-lt-r0.01/mo-2022.3.0-9078/nncf_network.xml

ir/jpqd-lt-r0.02-8eph-optimum-cfg-QEmb/mo-pruned-ir/nncf_network.bin

ir/jpqd-lt-r0.02-8eph-optimum-cfg-QEmb/mo-pruned-ir/nncf_network.mapping

ir/jpqd-lt-r0.02-8eph-optimum-cfg-QEmb/mo-pruned-ir/nncf_network.xml

ir/jpqd-lt-r0.02-8eph-optimum-cfg-QEmb/nncf_network.bin

ir/jpqd-lt-r0.02-8eph-optimum-cfg-QEmb/nncf_network.mapping

ir/jpqd-lt-r0.02-8eph-optimum-cfg-QEmb/nncf_network.onnx ONNX

ir/jpqd-lt-r0.02-8eph-optimum-cfg-QEmb/nncf_network.xml

ir/jpqd-lt-r0.02-8eph-optimum-cfg-QEmb/openvino_config.json

ir/jpqd-lt-r0.02-8eph-optimum-cfg-QEmb/openvino_model.bin

ir/jpqd-lt-r0.02-8eph-optimum-cfg-QEmb/openvino_model.xml

ir/jpqd-lt-r0.02-8eph-optimum-cfg/mo-pruned-ir/nncf_network.bin

ir/jpqd-lt-r0.02-8eph-optimum-cfg/mo-pruned-ir/nncf_network.mapping

ir/jpqd-lt-r0.02-8eph-optimum-cfg/mo-pruned-ir/nncf_network.xml

ir/jpqd-lt-r0.02-8eph-optimum-cfg/nncf_network.bin

ir/jpqd-lt-r0.02-8eph-optimum-cfg/nncf_network.mapping

ir/jpqd-lt-r0.02-8eph-optimum-cfg/nncf_network.onnx ONNX

ir/jpqd-lt-r0.02-8eph-optimum-cfg/nncf_network.xml

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/mo-pruned-ir/nncf_network.bin

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/mo-pruned-ir/nncf_network.mapping

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/mo-pruned-ir/nncf_network.xml

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/nncf_network.bin

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/nncf_network.mapping

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/nncf_network.onnx ONNX

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/nncf_network.xml

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/config.json

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/openvino_config.json

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/openvino_model.bin

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/openvino_model.xml

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/optimum-model.onnx ONNX

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/pytorch_model.bin

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/special_tokens_map.json

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/tokenizer.json

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/tokenizer_config.json

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/training_args.bin

ir/jpqd-lt-r0.02-8eph/pt1.13.0+cu117_ov2022.3.0-8831-4f0b846d1a5/optimum-save/vocab.txt

ir/mo.log

layer_wise_sparsity_global_rate_0.00.csv

layer_wise_sparsity_global_rate_0.00.md

linear_layer_sparsity_85M_params_0.00_sparsity.csv

linear_layer_sparsity_85M_params_0.00_sparsity.md

nncf_bert_squad_qat.json

onnx_sparsity.csv

onnx_sparsity.md

original_graph.dot

pytorch_model.bin

run.log

special_tokens_map.json

tokenizer.json

tokenizer_config.json

train_results.json

trainer_state.json

training_args.bin

vocab.txt

Related models

More models

No task type declared.