Structured Viewing Classification Annotations From the MIMIC-IV-ECHO Dataset (ECHOVIEW)

@article{PhysioNet-echoview-0.1,
  author = {Rapuri, Sampath and Dias, Sofia Sapeta and Carvalho, Maria Salomé and Lizzappi, Malcolm and Harris, Carl and Stevens, Robert},
  title = {{Structured Viewing Classification Annotations From the MIMIC-IV-ECHO Dataset (ECHOVIEW)}},
  journal = {{PhysioNet}},
  year = {2026},
  month = mar,
  note = {Version 0.1},
  doi = {10.13026/ywz0-5b62},
  url = {https://doi.org/10.13026/ywz0-5b62}
}

Abstract

The development of robust echocardiography analyses has been limited by the scarcity of well-annotated, labeled echocardiogram datasets. We introduce the Structured Viewing Classification Annotations From the MIMIC-IV-ECHO Dataset (ECHOVIEW), a derived dataset providing transthoracic (TTE) echocardiogram viewing classifications for a derived subset of the MIMIC-IV-ECHO dataset. This derived dataset includes viewing classifications for 29,196 echocardiograms across 717 studies, from 463 patients. Each study contains multiple echocardiogram views and sometimes uses multiple ultrasound techniques, with changes in the position and angle of the ultrasound probe corresponding to different views. Likewise, each patient can have multiple studies. The ECHOVIEW dataset features 23 separate viewing classifications predicted for each echocardiogram, which are produced by a machine learning (ML) classifier. Our model provides classification probabilities for echocardiograms with at least 10 frames of data. We validated our predictions by having a licensed cardiologist and an intensivist independently review a sample of 50 echocardiograms. Their evaluations were then compared against the ML view classifications. With the release of this dataset, we provide machine-generated view classifications for the MIMIC-IV-ECHO dataset and hope to encourage the development of further echocardiography analyses and models.

Background

Echocardiography is a rapid, noninvasive imaging study that can be used to evaluate the heart’s structure and function. A sonographer or other provider typically positions and maneuvers a transducer across the chest in a transthoracic echocardiogram while it emits high-frequency ultrasound waves in the range of 2 to 12 MHz [1]. These waves reflect and scatter within according to the acoustic properties of different tissues, allowing for the dynamic study of the distance, size, and shape of internal anatomical structures such as the heart. Along with other clinical data, echocardiograms represent a powerful class of imaging studies for both the diagnosis of heart abnormalities and monitoring of cardiac function.

Echocardiograms are increasingly relied upon as diagnostic modalities with an estimated 3% annualized growth rate [2]. However, while the number of echocardiographic studies is exponentially increasing, few open-source echocardiography datasets are available to foster further research into advanced predictive analytics and diagnostics. Additionally, there is a wide scarcity of available clinical annotations for existing echocardiographic datasets, further inhibiting the development of further research.

In this annotated dataset, we build upon the MIMIC-IV clinical dataset [3–5], which contains electronic health record (EHR) and related clinical data, along with the MIMIC-IV-ECHO dataset [6] that provides the corresponding echocardiogram videos, from which we derive echocardiogram view annotations. Because of the variability in transducer position and angle during an echocardiogram study, one of the first steps in many echocardiographic analyses is view classification. We describe the process of classifying unlabeled echocardiogram data by view type, facilitating large-scale analyses of these data.

Methods

We classify views of echocardiograms within the MIMIC-IV-ECHO dataset (v0.1) [6]. The MIMIC-IV-ECHO dataset contains 500,000 echocardiograms across 7,243 studies from 4,579 distinct patients admitted to Beth Israel Deaconess Medical Center in Boston, MA between 2017 and 2019. To create view classifications, we rely on a pretrained convolutional neural network (CNN) trained on an internal dataset of 7,168 echocardiograms from within the University of California San Francisco by Zhang et al. [7]. The CNN model is a 13-layer VGG network which outputs a vector of 23 class probabilities for each processed image from 10 randomly selected frames of the echocardiogram video (using a fixed random seed of 42 for reproducibility). The final view classification probability is an average across all the 23-dimensional output vectors for each frame and the resulting assigned view type corresponds to the class with the maximum average probability. The pretrained model and inference code used to generate the annotations is available through a Bitbucket repository [8].

We perform inference on all videos with at least 10 frames using this pretrained CNN model; thus, we select videos from the MIMIC-IV-ECHO dataset with at least 10 frames of data. As we do not have any ground truth view classifications in the dataset, to deal with the possible domain shift from different scanning equipment and acquisition conditions, we investigate the accuracy of our ML-classified view findings through a manual review with a licensed cardiologist and an intensivist.

In our analysis we were primarily interested in apical 4-chamber (A4C) videos, selecting A4C videos with an identified probability of 0.70 for manual review. A random sample of 50 echocardiograms predicted to correspond to an A4C view was rated by the expert reviewers. The reviewers were blinded to the ML predictions during evaluation. We assessed inter-rater reliability between reviewers using Cohen's kappa statistic [9], obtaining a value of 0.69, indicating substantial agreement. Confidence intervals for the Cohen's kappa statistic were not computed. Our machine learning classifier achieved an average accuracy of 79% when using a decision threshold of 0.7. For more conservative studies requiring higher precision at the expense of recall, we recommend increasing this probability threshold.

Data Description

The ECHOVIEW dataset consists of a single file, MIMIC_ECHO_View_Classifications.csv, containing 29,196 echocardiograms and 25 columns.

The structure of the ECHOVIEW CSV is as follows:

Image: The unique echocardiogram video being analyzed. The image name follows the format ZZZZZZZZ_VVVV, where ZZZZZZZZ is the study_id and VVVV is the view number. The corresponding echocardiogram videos are stored in the parent MIMIC-IV-ECHO dataset following the directory path pattern /pNN/pXXXXXXXX/sZZZZZZZZ/ZZZZZZZZ_VVVV, where NN are the first two characters of the subject_id that can be linked with the original MIMIC-IV dataset, XXXXXXXX is the subject_id, ZZZZZZZZ is the study_id, and VVVV is the view number.

• prob_plax_far: Probability of Parasternal Long Axis – remote
• prob_plax_plax: Probability of Parasternal Long Axis
• prob_plax_laz: Probability of Parasternal Long Axis – zoom of left atrium
• prob_psax_az: Probability of Parasternal Short Axis – aortic valve
• prob_psax_mv: Probability of Parasternal Short Axis – mitral valve
• prob_psax_pap: Probability of Parasternal Short Axis – papillary muscle
• prob_a2c_lvocc_s: Probability of Apical 2-chamber – occluded left ventricle
• prob_a2c_laocc: Probability of Apical 2-chamber – occluded left atrium
• prob_a2c: Probability of Apical 2-chamber – no occlusions
• prob_a3c_lvocc_s: Probability of Apical 3-chamber – occluded left ventricle
• prob_a3c_laocc: Probability of Apical 3-chamber – occluded left atrium
• prob_a3c: Probability of Apical 3-chamber – no occlusions
• prob_a4c_lvocc_s: Probability of Apical 4-chamber – occluded left ventricle
• prob_a4c_laocc: Probability of Apical 4-chamber – occluded left atrium
• prob_a4c: Probability of Apical 4-chamber – no occlusions
• prob_a5c: Probability of Apical 5-chamber
• prob_other: Probability of Other Views (with Doppler, IV contrast, or unable to classify)
• prob_rvinf: Probability of RV inflow
• prob_psax_avz: Probability of Parasternal Short Axis – aortic valve – zoom
• prob_suprasternal: Probability of Suprasternal Views
• prob_subcostal: Probability of Subcostal Views
• prob_plax_lac: Probability of Parasternal Long Axis – centered over left atrium
• prob_psax_apex: Probability of Parasternal Short Axis – apex

Usage Notes

The ECHOVIEW dataset provides granular view classifications for 29,196 echocardiogram videos within the MIMIC-IV-ECHO dataset, which links to the broader MIMIC-IV database, enabling future use of echocardiogram data in the MIMIC-IV database. By providing these view classifications, we expand research possibilities using the MIMIC-IV-ECHO dataset, addressing the inherent inhomogeneity of echocardiogram data and the wide variability of views that can be acquired. Additional linking information can be found within the MIMIC-IV database by utilizing the subject_id recorded in our dataset. Researchers requesting access must do so through PhysioNet.

We acknowledge limitations in our annotations. As we employ a pretrained ML classifier for view annotation, there is potential for classification errors. However, the model demonstrated high performance in the internal test dataset of Zhang et al. To address concerns about domain shift and generalizability, we verified that model predictions are generally accurate as determined by a licensed cardiologist and intensivist when using a probability threshold of 0.70.

We have included two example snippets of Python code to extract the highest probability viewing classification and filter echocardiograms by the view probability threshold.

df = pd.read_csv("MIMIC_ECHO_View_Classifications.csv")
view_cols = list(df.columns)[1:]
df["max_view"] = df[view_cols].idxmax(axis=1).str.replace("prob_", "")
#compute the most likely view for each video
df["max_prob"] = df[view_cols].max(axis=1)
#filter to videos with a certain probability of a viewing classification
a4c_high_conf = df.loc[df["prob_a4c"] >= 0.9, ["image", "prob_a4c"]]

Release Notes

Version 1.0.0: Initial public release of the dataset.

Ethics

This project is derived entirely from the deidentified MIMIC-IV and MIMIC-IV-ECHO datasets, which were approved by the Beth Israel Deaconess Medical Center IRB with a waiver of informed consent. No new data were collected for this study.

Conflicts of Interest

The authors have no conflicts of interest to declare. 

References

1. Lucas VS, Burk RS, Creehan S, Grap MJ. Utility of High-Frequency Ultrasound: Moving Beyond the Surface to Detect Changes in Skin Integrity. Plast Surg Nurs. 2014 Jan;34(1):34–8.
2. Papolos A, Narula J, Bavishi C, Chaudhry FA, Sengupta PP. U.S. Hospital Use of Echocardiography. J Am Coll Cardiol. 2016 Feb;67(5):502–11.
3. Johnson AEW, Bulgarelli L, Shen L, Gayles A, Shammout A, Horng S, et al. MIMIC-IV, a freely accessible electronic health record dataset. Sci Data. 2023 Jan 3;10(1):1.
4. Goldberger AL, Amaral LAN, Glass L, Hausdorff JM, Ivanov PC, Mark RG, et al. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation. 2000;101(23):e215–e220.
5. Johnson A, Bulgarelli L, Pollard T, Horng S, Celi L A, Mark R. MIMIC-IV (version 2.2). PhysioNet. 2023. RRID:SCR_007345. Available from: https://doi.org/10.13026/6mm1-ek67
6. Gow B, Pollard T, Greenbaum N, Moody B, Johnson A, Herbst E, Waks J W, Eslami P, Chaudhari A, Carbonati T, Berkowitz S, Mark R, Horng S. MIMIC-IV-ECHO: Echocardiogram Matched Subset (version 0.1). PhysioNet. 2023. RRID:SCR_007345. Available from: https://doi.org/10.13026/ef48-v217
7. Zhang J, Gajjala S, Agrawal P, Tison GH, Hallock LA, Beussink-Nelson L, et al. Fully Automated Echocardiogram Interpretation in Clinical Practice: Feasibility and Diagnostic Accuracy. Circulation. 2018 Oct 16;138(16):1623–35.
8. Zhang J, Gajjala S, Agrawal P, Tison GH, Hallock LA, Beussink-Nelson L, et al. EchoCV: Echocardiogram view classification [Internet]. Bitbucket; [cited 2025 Aug 18]. Available from: https://bitbucket.org/rahuldeo/echocv/src/master/ (commit 04f61c73a53a73677dfdea7defc14a37fc8ad7b5)
9. McHugh ML. Interrater reliability: the kappa statistic. Biochem Medica. 2012;22(3):276–82.