Resources
16 results for "llm evaluation"
Database
Contributor Review
ER-REASON: A Benchmark Dataset for LLM-Based Clinical Reasoning in the Emergency Room
The ER-Reason dataset is a benchmark designed to evaluate LLM-based clinical reasoning and decision-making in the emergency room (ER), a high-stakes setting where clinicians make rapid, consequential decisions across diverse patient presentations an…
Database
Credentialed Access
MIMIC-III-Ext-SBDH: An annotated social and behavioral determinants of health dataset
This study presents MIMIC-III-Ext-SBDH, a manually reviewed and consolidated annotation dataset of social and behavioral determinants of health (SBDH) derived from clinical notes in the MIMIC-III database. We previously developed SBDH-Reader, a larg…
Database
Credentialed Access
Medical-CXR-VQA dataset: A Large-Scale LLM-Enhanced Medical Dataset for Visual Question Answering on Chest X-Ray Images
Medical Visual Question Answering (VQA) is an important task in medical multi-modal Large Language Models (LLMs), aiming to answer clinically relevant questions regarding input medical images. This technique has the potential to improve the efficien…
Database
Credentialed Access
MIMIC-Ext-SynNotes: Large-Scale Synthetic Clinical Notes via Rephrasing MIMIC-III/IV with LLMs
MIMIC-III and MIMIC-IV are foundational resources for clinical NLP, but their scale is modest relative to the data volumes needed to train language models. Building on recent advances in synthetic data generation through LLM-based rephrasing, we con…
Database
Credentialed Access
MeDiSumQA: Patient-Oriented Question-Answer Generation from Discharge Letters
While increasing patients' access to medical documents improves medical care, this benefit is limited by varying health literacy levels and complex medical terminology. Large language models (LLMs) offer solutions by simplifying medical information.…
Database
Credentialed Access
RaDialog Instruct Dataset
Conversational AI tools that can generate and discuss clinically correct radiology reports for a given medical image have the potential to transform radiology. Such a human-in-the-loop radiology assistant could facilitate a collaborative diagnostic …
Database
Credentialed Access
MIMIC-III-Ext-MIMIC-Patient: Structured Per-Patient JSON Records for Clinical Question Answering
This project provides MIMIC-Patient, a structured, per-patient representation of the MIMIC-III database designed for large language model (LLM)–based clinical question answering. For each of 500 admissions, we reconstruct the patient’s episode into …
Database
Credentialed Access
MIMIC-III-Ext-Notes
Unstructured clinical documentation, such as progress notes, contains rich contextual information critical for clinical decision-making but remains underutilized in computational research due to the limited availability of annotated datasets. The MI…
Database
Credentialed Access
MIMIC-IV-Ext-MedicalBench: Evaluating Large Language Models Towards Improved Medical Concept Extraction
Medical concept extraction from electronic health records underpins many downstream applications, yet remains challenging because medically meaningful concepts, such as diagnosis, are frequently implied rather than explicitly stated in medical narra…
Database
Credentialed Access
AIPatient KG: MIMIC-III and CORAL Electronic Health Records based Patient Knowledge Graph
This study integrates the MIMIC-III and CORAL electronic health records into knowledge graphs to enhance their utility for advanced medical analysis and decision-making. MIMIC-III contains comprehensive data from over 40,000 patients, while CORAL fo…