models deepset minilm uncased squad2 - Azure/azureml-assets GitHub Wiki

deepset-minilm-uncased-squad2

Overview

Training Details

Hyperparameters

seed=42
batch_size = 12
n_epochs = 4
base_LM_model = "microsoft/MiniLM-L12-H384-uncased"
max_seq_len = 384
learning_rate = 4e-5
lr_schedule = LinearWarmup
warmup_proportion = 0.2
doc_stride=128
max_query_length=64
grad_acc_steps=4

Evaluation Results

Evaluated on the SQuAD 2.0 dev set with the official eval script.

"exact": 76.13071675229513,
"f1": 79.49786500219953,
"total": 11873,
"HasAns_exact": 78.35695006747639,
"HasAns_f1": 85.10090269418276,
"HasAns_total": 5928,
"NoAns_exact": 73.91084945332211,
"NoAns_f1": 73.91084945332211,
"NoAns_total": 5945

Model Evaluation samples

Task Use case Dataset Python sample (Notebook) CLI with YAML
Question Answering Extractive Q&A Squad v2 evaluate-model-question-answering.ipynb evaluate-model-question-answering.yml

Inference samples

Inference type Python sample (Notebook)
Real time sdk-example.ipynb
Real time question-answering-online-endpoint.ipynb

Sample inputs and outputs

Sample input

{
    "input_data": {
        "question": "What's my name?",
        "context": "My name is John and I live in Seattle"
    }
}

Sample output

[
  "John"
]

Version: 13

View in Studio: https://ml.azure.com/registries/azureml/models/deepset-minilm-uncased-squad2/version/13

Properties

SHA: 8a0111575f579698cae155af069d4b4db6fbe6a7

⚠️ **GitHub.com Fallback** ⚠️