Shenkezs/nbme-deberta-v3-finetuned

NBME DeBERTa v3 Fine-tuned Model

---

language: en license: apache-2.0 tags: - deberta - fine-tuning - nbme - medical - token-classification datasets: - nbme-dataset model_name: nbme-deberta-v3-finetuned widget: - text: "Patient reports chest pain and shortness of breath after medication."

This model is a fine-tuned version of microsoft/deberta-v3-base for the NBME clinical note scoring task.
It was trained on a subset of the NBME dataset to identify and classify relevant spans in patient notes.

---

🧠 Model Details

• Base Model: microsoft/deberta-v3-base
• Framework: PyTorch + Transformers
• Task: Token classification (NER-style scoring)
• Loss Function: BCEWithLogitsLoss
• Optimizer: AdamW
• Epochs: 5
• Batch size: hyperparameters['batch_size']
• Learning rate: hyperparameters['lr']

---

📊 Training Information

• Training Dataset: NBME Clinical Notes Dataset
• Train/Validation Split: 80/20
• Validation Loss: ~0.0965
• F1 Score: ~0.82

---

📁 Files

File	Description
nbme_full_model.pth	Full fine-tuned model weights
nbme_full_model.pkl	Pickle version for Python serialization
nbme_bert_v2.pth	Early checkpoint version
README.md	Model card file

---

🚀 How to Use

from transformers import AutoTokenizer
import torch
from your_model_file import CustomModel

tokenizer = AutoTokenizer.from_pretrained("microsoft/deberta-v3-base")
model = CustomModel(hyperparameters)
model.load_state_dict(torch.load("nbme_full_model.pth", map_location="cpu"))
model.eval()

text = "Patient reports severe headache after taking medication."
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
outputs = model(**inputs)
preds = torch.sigmoid(outputs)
print(preds)