Resources
16 results for "llm evaluation"
Database
Credentialed Access
EHRNoteQA: An LLM Benchmark for Real-World Clinical Practice Using Discharge Summaries
Discharge summaries in Electronic Health Records (EHRs) are crucial for clinical decision-making, but their length and complexity make information extraction challenging, especially when dealing with accumulated summaries across multiple patient adm…
Model
Credentialed Access
Me-LLaMA: Foundation Large Language Models for Medical Applications
Recent advancements in large language models (LLMs) such as ChatGPT and LLaMA have hinted at their potential to revolutionize medical applications, yet their application in clinical settings often reveals limitations due to a lack of specialized tra…
Database
Credentialed Access
MIMIC-III-Ext-VeriFact-BHC: Labeled Propositions From Brief Hospital Course Summaries for Long-form Clinical Text Evaluation
The VeriFact-BHC dataset is designed to verify the factuality of long-form text written about a patient against their own electronic health record. There is increasing interest in using large language models (LLMs) to generate clinical text in patie…
natural language processing
clinical informatics
formal logic
chart review
text embedding
brief hospital course
long-form text
logical atomism
retrieval-augmented generation
text reranking
clinical medicine
atomic claim
fact verification
hybrid retrieval
llm-as-a-judge
llm evaluation
clinical notes
large language models
electronic health records
artificial intelligence
Database
Credentialed Access
MIMIC-IV-Ext-SDBench: Sequential Diagnosis Benchmark Cases Derived from MIMIC-IV
MIMIC-IV-Ext-SDBench contains 488 case records derived from MIMIC-IV emergency department encounters in which the patient was subsequently admitted as an inpatient. Each case is formatted primarily for use with the Sequential Diagnosis Benchmark, an…
Database
Credentialed Access
PatientSim: A Persona-Driven Simulator for Realistic Doctor-Patient Interactions
Doctor-patient consultations require multi-turn, context-aware communication tailored to diverse patient personas. Training or evaluating doctor LLMs in such settings requires realistic patient interaction systems. However, existing simulators often…
Model
Credentialed Access
Asclepius-R : Clinical Large Language Model Built On MIMIC-III Discharge Summaries
The development of large language models tailored for handling patients’ clinical notes is often hindered by the limited accessibility and usability of these notes due to strict privacy regulations. To address these challenges, we first create synth…