TEEGLLTEEG: This EEG Looks Like That EEG

Abstract

This dataset comprises 50,697 electroencephalography (EEG) segments from 2,711 patients who underwent continuous EEG monitoring at Massachusetts General Hospital between July 2006 and March 2020. The EEG segments are labeled for six classes of patterns: seizures, lateralized periodic discharges (LPDs), generalized periodic discharges (GPDs), lateralized rhythmic delta activity (LRDA), generalized rhythmic delta activity (GRDA), and other patterns. These annotations were done by 124 domain experts and trained annotators. The dataset was used to develop and validate interpretable deep learning model that accurately classifies these patterns while providing faithful case-based explanations for its classifications. This comprehensive EEG dataset with expert annotations provides a valuable resource for developing and validating automated methods for EEG interpretation.

Background

Seizures and other seizure-like patterns of brain activity known as "ictal-interictal-injury continuum" (IIIC) patterns are commonly seen during brain monitoring with electroencephalography (EEG) in patients with epilepsy or critical illness. These IIIC events, which include seizures, lateralized periodic discharges (LPDs), generalized periodic discharges (GPDs), lateralized rhythmic delta activity (LRDA), and generalized rhythmic delta activity (GRDA), can damage the brain, especially when prolonged.

In intensive care units (ICUs), critically ill patients are monitored with EEG to prevent serious brain injury. Recent studies have found that seizure or status epilepticus occurs in approximately 20% of these patients, and every hour of seizures detected on EEG further increases the risk of permanent disability or death. Even more concerning, intermediate seizure-like patterns occur in nearly 40% of patients undergoing EEG monitoring, and evidence suggests these patterns also increase the risk of disability and death if they persist.

While the need for accurate EEG interpretation is clear, several critical challenges remain. First, EEG monitoring is constrained by the scarcity of qualified neurophysiologists, limiting access to brain monitoring services in many parts of the world. Second, EEG interpretation is inherently subjective and prone to interobserver variability, especially for ambiguous patterns along the IIIC spectrum. This variability can lead to inconsistent diagnoses and treatment decisions.

Recent advances in deep learning have made possible the development of automated algorithms to detect and classify seizures and other EEG patterns, with some models achieving accuracy comparable to that of physician experts. However, a fundamental limitation of existing approaches is their lack of interpretability. These "black-box" models cannot explain their decision-making processes, making them unsuitable for clinical decision support at the point of care. Uninterpretable models are prone to silent failures during clinical operations due to poor generalization or reliance on clinically irrelevant features, potentially leading to misdiagnoses and increased patient risks.

The need for interpretability in medical AI systems has been recognized by regulatory bodies, with both the U.S. Food and Drug Administration and the European Union publishing requirements for interpretability and explainability in AI used for medical applications. While post hoc explainability techniques like Gradient-Weighted Class Activation Mapping and Shapley Additive Explanations attempt to elucidate model decisions after the fact, these methods only approximate model reasoning and often give conflicting explanations for the same input.

In contrast, our approach focuses on building interpretability directly into the model architecture, creating a system where the explanation exactly matches the predictor network's underlying calculations. This "This EEG Looks Like That EEG" (TEEGLLTEEG) approach provides case-based explanations using prototypes, allowing clinicians to understand and validate the model's reasoning. By developing an inherently interpretable deep learning model for IIIC EEG pattern classification, we aimed to reduce human subjectivity, improve diagnostic accuracy, and gain insights into the relationships among EEG patterns along the IIIC spectrum.

Methods

Our model, called ProtoPMed-EEG or "TEEGLLTEEG" (This EEG Looks Like That EEG), was trained and tested on a large-scale EEG study consisting of 50,697 events from 2,711 patients hospitalized between July 2006 and March 2020 who underwent continuous EEG as part of clinical care at Massachusetts General Hospital. A total of 124 EEG raters from 18 centers labeled the middle 10 seconds of 50-second EEG segments. Raters produced one of six labels: seizure, lateralized periodic discharges (LPDs), generalized periodic discharges (GPDs), lateralized rhythmic delta activity (LRDA), generalized rhythmic delta activity (GRDA), and other patterns. "Other" included all patterns (including normal) except seizures and the four rhythmic and periodic patterns.

Patterns obscured by artifacts were scored by experts in the same way that it is done in clinical practice so that when artifacts were present but experts were still able to discern that one of the five target patterns was present, they were instructed to assign the target pattern as the label.

Mean rater-to-rater interrater reliability (IRR) was moderate (agreement, 52%; kappa, 42%), and mean rater-to-majority IRR was substantial (agreement, 65%; kappa, 61%). In total, 180,000 ten-second EEG segments received labels from at least 3 experts. A subset of 71,982 segments were designated as having "high-quality labels" that had received at least 10 independent annotations from a group of 20 experts who had both completed clinical neurophysiology fellowship training and had each annotated ≥1,000 EEG segments.

We split the 71,982 EEG segments with high-quality labels into approximately equal-sized training and test sets. Rather than allowing any training set sample to become a prototype, we limited our prototype candidates to 10,641 samples that were thoroughly examined in the data-labeling process (≥20 expert votes).
 

Data Description

The dataset consists of 50,697 EEG segments collected from 6,095 EEGs from 2,711 patients. Each segment contains 50 seconds of continuous EEG, with annotations provided for the middle 10 seconds. The data are organized into six pattern classes:

1. Seizures: Ictal patterns showing clear evolution in frequency, amplitude, or spatial distribution
2. Lateralized Periodic Discharges (LPDs): Periodic discharges occurring in one hemisphere
3. Generalized Periodic Discharges (GPDs): Periodic discharges occurring in both hemispheres
4. Lateralized Rhythmic Delta Activity (LRDA): Rhythmic delta activity occurring in one hemisphere
5. Generalized Rhythmic Delta Activity (GRDA): Rhythmic delta activity occurring in both hemispheres
6. Other patterns: All patterns (including normal) except seizures and the four rhythmic and periodic patterns

Each segment has annotations from multiple experts, with a subset of 71,982 segments receiving at least 10 annotations from fellowship-trained neurophysiologists. The dataset includes patient demographics (age, gender), as well as technical information about the EEG recordings.

The dataset was split into training and test sets, ensuring that all data from any patient appeared only in the training or test set, and both sets were approximately equal in the number of patients, proportion of each pattern type, and agreement among experts.

Usage Notes

Data and code to generate all results and figures from the publication are provided here.

Ethics

This study was conducted under protocols approved by the Mass General Brigham Institutional Review Board (Protocol Number: 2013P001024) and Beth Israel Deaconess Medical Center Institutional Review Boards (BIDMC: #2022P000417, 2022P000481). All data is deidentified

Acknowledgements

We acknowledge all authors who contributed to the original papers: Alina Jade Barnett, Zhicheng Guo, Jin Jing, Wendong Ge, Peter W. Kaplan, Wan Yee Kong, Ioannis Karakis, Aline Herlopian, Lakshman Arcot Jayagopal, Olga Taraschenko, Olga Selioutski, Gamaleldin Osman, Daniel Goldenholz, Cynthia Rudin, M. Brandon Westover, Shenda Hong, Marta Bento Fernandes, Zhen Lin, Chaoqi Yang, Sungtae An, Aaron F. Struck, Jonathan J. Halford, Marcus C. Ng, Emily L. Johnson, Brian L. Appavu, Rani A. Sarkis, Monica B. Dhakar, Zubeda Sheikh, Sarah Schmitt, Hiba A. Haider, Jennifer A. Kim, Christa B. Swisher, Nicolas Gaspard, Mackenzie C. Cervenka, Andres Rodriguez, Jong Woo Lee, Mohammad Tabaeizadeh, Emily J. Gilmore, Kristy Nordstrom, Ji Yeoun Yoo, Manisha Holmes, Susan T. Herman, Jennifer A. Williams, Jay Pathmanathan, Fabio A. Nascimento, Ziwei Fan, Samaneh Nasiri, Mouhsin M. Shafi, Sydney S. Cash, Daniel B. Hoch, Andrew J. Cole, Eric S. Rosenthal, Sahar F. Zafar, and Jimeng Sun. We thank them for their contributions to "Improving Clinician Performance in Classifying EEG Patterns on the Ictal–Interictal Injury Continuum Using Interpretable Machine Learning" and "Development of Expert-Level Classification of Seizures and Rhythmic and Periodic Patterns During EEG Interpretation."
 

Conflicts of Interest

M.B.W. is a co-founder of and holds equity in Beacon Biosignals. Beacon Biosignals did not contribute funding nor played any role in the study.

References

1. Barnett AJ, Guo Z, Jing J, Ge W, Kaplan PW, Kong WY, Karakis I, Herlopian A, Jayagopal LA, Taraschenko O, Selioutski O, Osman G, Goldenholz D, Rudin C, Westover MB. Improving Clinician Performance in Classifying EEG Patterns on the Ictal-Interictal Injury Continuum Using Interpretable Machine Learning. NEJM AI. 2024 Jun;1(6):10.1056/aioa2300331. doi: 10.1056/aioa2300331. Epub 2024 May 23. PMID: 38872809; PMCID: PMC11175595.