Resources
123 results for "evaluation"
Challenge
Credentialed Access
ShAReCLEF eHealth Evaluation Lab 2014 (Task 2): Disorder Attributes in Clinical Reports
This project focuses on Task 2 of the 2014 ShARe/CLEF eHealth evaluation lab, which extended Task 1 of the 2013 ShARe/CLEF eHealth evaluation lab by focusing on identification and normalization of disorder attributes. The task comprises two subtasks…
Database
Credentialed Access
ReEvalMed: Rethinking Medical Report Evaluation by Aligning Metrics with Real-World Clinical Judgment
Automatically generated radiology reports often receive high scores from existing evaluation metrics but fail to earn clinicians' trust. This gap reveals fundamental flaws in how current metrics assess the quality of generated reports. We rethink th…
Database
Credentialed Access
MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context
Large Vision Language Models (LVLMs) have recently achieved superior performance in various tasks on natural image and text data, which inspires a large amount of studies for LVLMs fine-tuning and training. Despite their advancements, there has been…
Database
Credentialed Access
CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest X-rays
Recent progress in Large Vision-Language Models (LVLMs) has enabled promising applications in medical tasks such as report generation and visual question answering. However, existing benchmarks focus mainly on the final diagnostic answer, offering l…
Database
Open Access
Radiology Report Generation Models Evaluation Dataset For Chest X-rays (RadEvalX)
The Radiology Report Generation Models Evaluation Dataset For Chest X-rays (RadEvalX) is publicly available and developed similarly to the ReXVal dataset. Just like ReXVal, RadEvalX focuses on radiologist evaluations of errors found in automatically…
Database
Credentialed Access
Radiology Report Expert Evaluation (ReXVal) Dataset
The Radiology Report Expert Evaluation (ReXVal) Dataset is a publicly available dataset of radiologist evaluations of errors in automatically generated radiology reports. The dataset contains annotations from 6 board certified radiologists on clinic…
Database
Credentialed Access
MIMIC-IV-Ext-MedicalBench: Evaluating Large Language Models Towards Improved Medical Concept Extraction
Medical concept extraction from electronic health records underpins many downstream applications, yet remains challenging because medically meaningful concepts, such as diagnosis, are frequently implied rather than explicitly stated in medical narra…
Database
Open Access
Integration of Electroencephalogram and Eye-Gaze Datasets for Performance Evaluation in Fundamentals of Laparoscopic Surgery (FLS) Tasks
Laparoscopic surgery represents a significant advancement in healthcare, aiming to enhance patient outcomes. However, evaluating performance in laparoscopic surgery tasks can be challenging due to subjective assessments. To address this, this study …
Database
Open Access
Electroencephalogram and eye-gaze datasets for robot-assisted surgery performance evaluation
Surgical training and patient safety depend on objective and accurate performance evaluation. Robot-assisted surgery (RAS) is newer and its training less well defined, but resources are not easily accessed to support data collection from participant…
Database
Credentialed Access
MIMIC-III-Ext-VeriFact-BHC: Labeled Propositions From Brief Hospital Course Summaries for Long-form Clinical Text Evaluation
The VeriFact-BHC dataset is designed to verify the factuality of long-form text written about a patient against their own electronic health record. There is increasing interest in using large language models (LLMs) to generate clinical text in patie…
natural language processing
clinical informatics
formal logic
chart review
text embedding
brief hospital course
long-form text
logical atomism
retrieval-augmented generation
text reranking
clinical medicine
atomic claim
fact verification
hybrid retrieval
llm-as-a-judge
llm evaluation
clinical notes
large language models
electronic health records
artificial intelligence