Resources

16 results for "llm evaluation"

Database Contributor Review

ER-REASON: A Benchmark Dataset for LLM-Based Clinical Reasoning in the Emergency Room

Mel Molina, Nikita Mehandru, Niloufar Golchini, et al.

The ER-Reason dataset is a benchmark designed to evaluate LLM-based clinical reasoning and decision-making in the emergency room (ER), a high-stakes setting where clinicians make rapid, consequential decisions across diverse patient presentations an…
Published: Oct. 23, 2025. Version: 1.0.0
Database Credentialed Access

MIMIC-III-Ext-SBDH: An annotated social and behavioral determinants of health dataset

Zifan Gu, Lesi He, Donghan Yang

This study presents MIMIC-III-Ext-SBDH, a manually reviewed and consolidated annotation dataset of social and behavioral determinants of health (SBDH) derived from clinical notes in the MIMIC-III database. We previously developed SBDH-Reader, a larg…
Published: Sept. 1, 2026. Version: 1.0.0
Database Credentialed Access

Medical-CXR-VQA dataset: A Large-Scale LLM-Enhanced Medical Dataset for Visual Question Answering on Chest X-Ray Images

Xinyue Hu, Lin Gu, Kazuma Kobayashi, et al.

Medical Visual Question Answering (VQA) is an important task in medical multi-modal Large Language Models (LLMs), aiming to answer clinically relevant questions regarding input medical images. This technique has the potential to improve the efficien…
Published: Jan. 21, 2025. Version: 1.0.0
Database Credentialed Access

MIMIC-Ext-SynNotes: Large-Scale Synthetic Clinical Notes via Rephrasing MIMIC-III/IV with LLMs

Jinghui Liu, Sarvesh Soni, Anthony Nguyen

MIMIC-III and MIMIC-IV are foundational resources for clinical NLP, but their scale is modest relative to the data volumes needed to train language models. Building on recent advances in synthetic data generation through LLM-based rephrasing, we con…
Published: Oct. 5, 2026. Version: 1.0
Database Credentialed Access

MeDiSumQA: Patient-Oriented Question-Answer Generation from Discharge Letters

Amin Dada, Osman Alperen Koras, Marie Bauer, et al.

While increasing patients' access to medical documents improves medical care, this benefit is limited by varying health literacy levels and complex medical terminology. Large language models (LLMs) offer solutions by simplifying medical information.…
Published: May 5, 2025. Version: 1.0.0
Database Credentialed Access

RaDialog Instruct Dataset

Chantal Pellegrini, Ege Özsoy, Benjamin Busam, et al.

Conversational AI tools that can generate and discuss clinically correct radiology reports for a given medical image have the potential to transform radiology. Such a human-in-the-loop radiology assistant could facilitate a collaborative diagnostic …
Published: July 12, 2024. Version: 1.1.0
Database Credentialed Access

MIMIC-III-Ext-MIMIC-Patient: Structured Per-Patient JSON Records for Clinical Question Answering

Tianqi Shang, Weiqing He, Charles Zheng, et al.

This project provides MIMIC-Patient, a structured, per-patient representation of the MIMIC-III database designed for large language model (LLM)–based clinical question answering. For each of 500 admissions, we reconstruct the patient’s episode into …
Published: Sept. 11, 2026. Version: 1.0.0
Database Credentialed Access

MIMIC-III-Ext-Notes

Darren Liu, Monique Bouvier, Delgersuren Bold, et al.

Unstructured clinical documentation, such as progress notes, contains rich contextual information critical for clinical decision-making but remains underutilized in computational research due to the limited availability of annotated datasets. The MI…
Published: Feb. 27, 2026. Version: 1.0.0
Database Credentialed Access

MIMIC-IV-Ext-MedicalBench: Evaluating Large Language Models Towards Improved Medical Concept Extraction

Zhichao Yang, Gregory Lyng, Sanjit Batra, et al.

Medical concept extraction from electronic health records underpins many downstream applications, yet remains challenging because medically meaningful concepts, such as diagnosis, are frequently implied rather than explicitly stated in medical narra…
Published: March 23, 2026. Version: 1.0.0
Database Credentialed Access

AIPatient KG: MIMIC-III and CORAL Electronic Health Records based Patient Knowledge Graph

Lizhou Fan, Huizi Yu

This study integrates the MIMIC-III and CORAL electronic health records into knowledge graphs to enhance their utility for advanced medical analysis and decision-making. MIMIC-III contains comprehensive data from over 40,000 patients, while CORAL fo…
Published: April 15, 2025. Version: 1.0.0