SPaRCNet data: Seizures, Rhythmic and Periodic Patterns in ICU Electroencephalography

Abstract

Seizures and seizure-like rhythmic and periodic brain activity known as “ictal-interictal-injury continuum” (IIIC) patterns are frequently detected during brain monitoring with electroencephalography (EEG) in patients with epilepsy or critical illness. Prior efforts to automate detection of IIIC patterns have been limited by lack of large well-annotated datasets to train/evaluate algorithms, and there have been only a few attempts to detect IIIC events other than seizures. The IIIC dataset includes  50,697 labeled EEG samples from 2,711 patients’ and 6,095 EEGs that were annotated by physician experts from 18 institutions. These samples were used to train SPaRCNet (Seizures, Periodic and Rhythmic Continuum patterns Deep Neural Network), a computer program that classifies IIIC events with an accuracy matching clinical experts. 

Associated GitHub repositories: 

• https://github.com/bdsp-core/IIIC-IRR
• https://github.com/bdsp-core/IIIC-SPaRCNet

Background

Seizures and other seizure-like types of brain activity known as “ictal-interictal-injury continuum” (IIIC) patterns (aka “ictal-interictal continuum” (IIC) patterns), are seen commonly during brain monitoring with electroencephalography (EEG) in patients with epilepsy or critical illness. IIIC events can damage the brain, especially when prolonged, and are indicators of impending delayed cerebral ischemia and risk for post-traumatic epilepsy. Fellowship-trained clinical neurophysiologists are the gold standard for identifying IIIC events. However, subspecialists are scarce, and in most of the world brain monitoring services are unavailable. There is a critical need for methods that detect IIIC events automatically without compromising accuracy.

Prior efforts to automate seizure detection have been limited by lack of large well-annotated datasets to train and evaluate algorithms, and there have been only a few attempts to detect IIIC events other than seizures.

To address this gap, we created a set of 50,697 IIIC and non-IIIC events from 2,711 patients’ (6,095 EEGs) and obtained independent annotations from 124 raters, 20 of whom annotated sufficient data to compare against algorithm performance and qualified as experts (physicians with subspecialty EEG training). We used these data to train a computer program (deep neural network) to classify IIIC events and distinguish them from non-IIIC events with accuracy matching clinical experts. 
 

Methods

Methods used in preparing this data are described in detail in reference [1] below. 

Standard Protocol Approvals, Registrations, and Patient Consents: The study was approved by the Massachusetts General Hospital (MGH) IRB, which waived the requirement for informed consent.

Subjects: We selected 2,711 recordings from patients hospitalized between July 2006 and March 2020 who underwent EEG as part of clinical care at MGH in medical, neurological and surgical intensive care and general care units. EEG electrodes were placed according to the International 10-20 system. Patients were selected in two stages, based on clinical notes mentioning IIIC events (see eTable 2 in [1]). We placed no restrictions on age (eTable 1 in [1]). The large group was intended to ensure broad coverage of all variations of IIIC events encountered in practice. 

EEG Labeling: Labeling of 10-second EEG segments was done using custom local- and web-based interfaces, in two stages (supplemental S2). The first stage involved targeted annotations by small groups of independent experts. The second stage involved multiple labeling rounds by larger groups of independent experts guided by automated selection of new segments to be labeled. Experts could pan 20 seconds before or after the target segment, change montages, and adjust the signal gain. A 10-minute spectrogram was provided for additional context. Raters were given a forced choice of six options: seizure (SZ), lateralized periodic discharges (LPD), generalized periodic discharges (GPD), lateralized rhythmic delta activity (LRDA), generalized rhythmic delta activity (GRDA), and “Other” if none of those patterns was present. The final ground truth label assigned to each EEG segment was the category chosen most often for the segment by expert reviewers. 

EEG raters: 124 EEG raters from 18 centers labeled varying numbers of EEG segments, including 30 fellowship-trained physicians (“experts”) and 94 technicians and trainees. Experts who participated in Stage 2 of labeling and scored ≥1,000 segments were included in IRR analysis. 
 

REFERENCES:
[1]  Interrater Reliability of Expert Electroencephalographers Identifying Seizures and Rhythmic and Periodic Patterns in Electroencephalograms. Neurology. 2022 Dec 2:10.1212/WNL.0000000000201670. doi: 10.1212/WNL.0000000000201670. Epub ahead of print. PMID: 36460472.  https://pubmed.ncbi.nlm.nih.gov/36460472/ 

[2] Development of Expert-Level Classification of Seizures and Rhythmic and Periodic Patterns During EEG Interpretation. Neurology. 2023 Mar 6:10.1212/WNL.0000000000207127. doi: 10.1212/WNL.0000000000207127. Epub ahead of print. PMID: 36878708. https://pubmed.ncbi.nlm.nih.gov/36878708/

(authors are listed in Acknowledgements)

Data Description

Overall, 124 raters scored 50,697 EEG segments from 2,711 patients, including 49% females, with a median (IQR) age of 55 (41) years old, and EEG duration of 18 (22) hours (eTable 1 of reference [1]). Limiting analysis to experts with training in clinical neurophysiology yielded 30 experts; all scored ≥1,000 EEG segments. Among these 30, there were 20 experts with enough data for calibration analysis; and 30 for pairwise IRR, majority IRR, and noise-bias analyses. The median [range] of EEG segments scored per expert was 6,287.5 [1,002, 45,267], and the median number of labels per EEG segment was 15 [10, 29]. The consensus (majority) labels were “Other” for 37% (N=18,582). Among the remaining 32,115 IIIC patterns, 28% (9,141) were SZ, 11% (3,547) were LPD, 7% (2,141) were “GPD”, 10% (3,348) were LRDA, and 43% (13,938) were GRDA. A flow diagram of the sample selection is shown in Figure 1 of reference [1].

These data were used to train a deep neural network, SPaRCNet, described in reference [2]. 

Training data is shared here. We do not make the test data available publicly. If you wish to have your algorithm evaluated on the test data, please contact the authors. 

Usage Notes

The data for training SPaRCNet is provided here in this repository. Code for training SPaRCNet is available in the associated GitHub repository (https://github.com/bdsp-core/IIIC-SPaRCNet). 

Training data includes both labeled and unlabeled samples. 

Labels for testing data are not being made available.

Investigators who wish to have their model evaluated against the test data may email the first and last author of the paper (Drs. Jing and Westover),  who will do their best to accommodate all reasonable requests. Investigators making such requests will be required to provide working code (training and testing), and must commit to making their code publicly available under an MIT license. 

Ethics

 In this dataset, all data were anonymized with all identifiable patient information removed.

Acknowledgements

Preparation of this dataset was done by the following clinical experts and scientific collaborators, through the Critical Care EEG Monitoring Research Consortium (CCEMRC): 

Reference [1]: 

Interrater Reliability of Expert Electroencephalographers Identifying Seizures and Rhythmic and Periodic Patterns in Electroencephalograms.

Jin Jing1,2, *Wendong Ge1,2­,  Aaron F. Struck3,4 , *Marta Bento Fernandes1,2, Shenda Hong5, Sungtae An7, Safoora Fatima3,  Aline Herlopian8, Ioannis Karakis9, Jonathan J. Halford10, Marcus Ng11, Emily L. Johnson12, Brian Appavu13, Rani A. Sarkis14, Gamaleldin Osman15, Peter W. Kaplan12, Monica B. Dhakar16, Lakshman Arcot Jayagopal17, Zubeda Sheikh18, Olha Taraschenko17, Sarah Schmitt10, Hiba A. Haider19, Jennifer A. Kim8, Christa B. Swisher20, Nicolas Gaspard21, Mackenzie C. Cervenka12, Andres Rodriguez9, Jong Woo Lee14, Mohammad Tabaeizadeh1,2, Emily J. Gilmore8, Kristy Nordstrom1, Ji Yeoun Yoo22, Manisha Holmes23, Susan T. Herman24, Jennifer A. Williams25, Jay Pathmanathan26, Fábio A. Nascimento1,2, Ziwei Fan1,2, Samaneh Nasiri1,2, Mouhsin M. Shafi27, Sydney S. Cash1,2, Daniel B. Hoch1,2, Andrew J. Cole1,2, Eric S. Rosenthal1,2, Sahar F. Zafar1,2, Jimeng Sun5, M. Brandon Westover1,2

1-Massachusetts General Hospital/Harvard Medical School Department of Neurology, MA
2-Massachusetts General Hospital Clinical Data Animation Center (CDAC), MA
3-University of Wisconsin-Madison Department of Neurology
4-William S Middleton Memorial Veterans Hospital Madison, WI
5-National Institute of Health Data Science, Peking University, Beijing, China
6-University of Illinois at Urbana-Champaign, College of Computing, Champaign, IL
7-Georgia Institute of Technology, College of Computing, Atlanta, GA
8-Yale University-Yale New Haven Hospital, CT
9-Emory University School of Medicine, GA
10-Medical University of South Carolina, SC
11-University of Manitoba, Canada
12-Johns Hopkins School of Medicine, MD
13-University of Arizona College of Medicine, AZ
14-Brigham and Women's Hospital, MA
15-Mayo Clinic-Rochester, MN
16-Warren Alpert School of Medicine of Brown University, Providence, RI
17-University of Nebraska Medical Center, NE
18-West Virginia University Hospitals, WV
19-University of Chicago, Chicago, IL
20-Atrium Health, NC
21-Université Libre de Bruxelles - Hôpital Erasme, Belgium
22-Icahn School of Medicine, Mount Sinai, NY
23-New York University (NYU) Grossman School of Medicine, NY
24-Barrow Neurological Institute, Phoenix, AZ
25-Mater Misericordiae University Hospital, Dublin, Ireland.
26-University of Pennsylvania, PA
27-Beth Israel Deaconess Medical Center/Harvard Medical School, MA

Reference [2]

Development of Expert-level Classification of Seizures and Rhythmic and Periodic Patterns During EEG Interpretation
Jin Jing1,2, Wendong Ge1,2, Shenda Hong3, Marta Bento Fernandes1,2, Zhen Lin4, Chaoqi Yang4,  Sungtae An5, Aaron F Struck6,7 , Aline Herlopian8, Ioannis Karakis9, Jonathan J. Halford10, Marcus Ng11, Emily L. Johnson12, Brian Appavu13, Rani A. Sarkis14, Gamaleldin Osman15, Peter W. Kaplan12, Monica B. Dhakar16, Lakshman Arcot Jayagopal17, Zubeda Sheikh18, Olha Taraschenko17, Sarah Schmitt10, Hiba A. Haider19, Jennifer A. Kim8, Christa B. Swisher20, Nicolas Gaspard21, Mackenzie C. Cervenka12, Andres Rodriguez9, Jong Woo Lee14, Mohammad Tabaeizadeh1,2, Emily J. Gilmore8, Kristy Nordstrom1, Ji Yeoun Yoo22, Manisha Holmes23, Susan T. Herman24, Jennifer A. William25, Jay Pathmanathan26, Fábio A. Nascimento1,2, Ziwei Fan1,2, Nasiri  Samane1,2, Mouhsin M. Shafi27, Sydney S. Cash1,2, Daniel B. Hoch1,2, Andrew J Cole1, Eric S. Rosenthal1, Sahar Zafar1,2, Jimeng Sun4, M Brandon Westover1,2

1-Massachusetts General Hospital/Harvard Medical School Department of Neurology, MA
2-Massachusetts General Hospital Clinical Data Animation Center (CDAC), MA
3-National Institute of Health Data Science, Peking University, Beijing, China
4-University of Illinois at Urbana-Champaign, College of Computing, Champaign, IL
5-Georgia Institute of Technology, College of Computing, Atlanta, GA
6-University of Wisconsin-Madison Department of Neurology
7-William S Middleton Memorial Veterans Hospital Madison, WI
8-Yale University-Yale New Haven Hospital, CT
9-Emory University School of Medicine, GA
10-Medical University of South Carolina, SC
11-University of Manitoba, Canada
12-Johns Hopkins School of Medicine, MD
13-University of Arizona College of Medicine Phoenix, AZ
14-Brigham and Women's Hospital, MA
15-Mayo Clinic-Rochester, MN
16-Warren Alpert School of Medicine of Brown University, Providence, RI
17-University of Nebraska Medical Center, Omaha, NE
18-West Virginia University Hospitals, WV
19-University of Chicago, Chicago, IL
20-Atrium Health, NC
21-Université Libre de Bruxelles - Hôpital Erasme, Belgium
22-Icahn School of Medicine, Mount Sinai, NY
23-New York University (NYU) Grossman School of Medicine, NY
24-Barrow Neurological Institute, Phoenix, AZ
25-Mater Misericordiae University Hospital, Dublin, Ireland.
26-University of Pennsylvania, PA
27-Beth Israel Deaconess Medical Center/Harvard Medical School, MA
 

Conflicts of Interest

This work was supported by grants from the NIH (R01NS102190, R01NS102574, R01NS107291, RF1AG064312, RF1NS120947, R01AG073410, R01HL161253, R01NS126282, R01AG073598).