Resources

This project focuses on Task 2 of the 2014 ShARe/CLEF eHealth evaluation lab, which extended Task 1 of the 2013 ShARe/CLEF eHealth evaluation lab by focusing on identification and normalization of disorder attributes. The task comprises two subtasks…

Automatically generated radiology reports often receive high scores from existing evaluation metrics but fail to earn clinicians' trust. This gap reveals fundamental flaws in how current metrics assess the quality of generated reports. We rethink th…

Large Vision Language Models (LVLMs) have recently achieved superior performance in various tasks on natural image and text data, which inspires a large amount of studies for LVLMs fine-tuning and training. Despite their advancements, there has been…

Recent progress in Large Vision-Language Models (LVLMs) has enabled promising applications in medical tasks such as report generation and visual question answering. However, existing benchmarks focus mainly on the final diagnostic answer, offering l…

The Radiology Report Generation Models Evaluation Dataset For Chest X-rays (RadEvalX) is publicly available and developed similarly to the ReXVal dataset. Just like ReXVal, RadEvalX focuses on radiologist evaluations of errors found in automatically…

The Radiology Report Expert Evaluation (ReXVal) Dataset is a publicly available dataset of radiologist evaluations of errors in automatically generated radiology reports. The dataset contains annotations from 6 board certified radiologists on clinic…

MIMIC-IV-Ext-DeVisE contains counterfactual ICU admission notes designed to enable controlled behavioral evaluation of medical large language models (LLMs) to clinically meaningful perturbations in patient data. It was created as part of the DeVisE …

Medical concept extraction from electronic health records underpins many downstream applications, yet remains challenging because medically meaningful concepts, such as diagnosis, are frequently implied rather than explicitly stated in medical narra…

Laparoscopic surgery represents a significant advancement in healthcare, aiming to enhance patient outcomes. However, evaluating performance in laparoscopic surgery tasks can be challenging due to subjective assessments. To address this, this study …

Surgical training and patient safety depend on objective and accurate performance evaluation. Robot-assisted surgery (RAS) is newer and its training less well defined, but resources are not easily accessed to support data collection from participant…