BJH-perioperative-notes-bioGPT
This clinical foundational model is designed to predict potential risk factors / post-operative surgical outcomes from clinical notes taken during perioperative care. It was finetuned from the microsoft/biogpt
model through a multi-task learning approach, spanning the following 6 outcomes:
- Death in 30 days
- Deep vein thrombosis (DVT)
- pulmonary embolism (PE)
- Pneumonia
- Acute Knee Injury
- delirium
Also check out cja5553/BJH-perioperative-notes-bioClinicalBERT
, which is the bioClinicalBERT variant of our model!
Dataset
We used 84,875 perioperative clinical notes spanning 3 years worth of anonymized patient data from the Barnes Jewish Healthcare (BJH) system in St Louis, MO. BJH is the largest hospital in the state of Missouri and the largest employer in the greater St. Louis region! The following are the characteristics for the data:
- vocabulary size: 3203
- averaging words per clinical note: 8.9 words
- all single sentenced clinical notes
How to use model
from transformers import BioGptTokenizer, AutoModelForCausalLM
model=AutoModelForCausalLM.from_pretrained("cja5553/BJH-perioperative-notes-bioGPT")
tokenizer = BioGptTokenizer.from_pretrained("microsoft/biogpt")
Note: Because of our distinct model architecture, you are required to train a distinct predictor or use a respective fully-connected network above the hidden state when deploying the model via transformers
Codes
Codes used to train the model are publicly available at: https://github.com/cja5553/LLMs_in_perioperative_care
Citation
If you find this model useful, please cite the following paper:
@article{
author={Bing Xue, Charles Alba, Joanna Abraham, Thomas Kannampallil, Christopher King, Michael Avidan, Chenyang Lu}
title={"Prescribing Large Language Models for Perioperative Care: What’s The Right Dose for Pretrained Models?"},
year={2024}
}
Questions?
contact me at [email protected]
- Downloads last month
- 0
Model tree for cja5553/BJH-perioperative-notes-bioGPT
Base model
microsoft/biogpt