Name: Noise in the diagnosis of epilepsy by experts
Published: July 7, 2026
License: https://github.com/bdsp-core/bdsp-license-and-dua

Software Credentialed Access

Fábio A. Nascimento , John McLaren , Wei Zhao , Roohi Katyal , Irfan S. Sheikh , Wan-Yee Kong , Danah Aljaafari , Nirav Barot , Selim Benbadis , Daniel Friedman , Jay R Gavvala , Jonathan Halford , R. Edward Hogan , Peter Kaplan , Ioannis Karakis , Atul Maheshwari , Rebecca Matthews , Cormac O'Donovan , Stefan Rampp , Stephan Schuele , Joseph Sirven , William O. Tatum , Jonathan Williams , Elza Marcia Yacubian , Doyle Yuan , Sándor Beniczky , Olivier Sibony , M. Brandon Westover

Published: July 7, 2026. Version: 1.0.0

When using this resource, please cite: (show more options)
Nascimento, F. A., McLaren, J., Zhao, W., Katyal, R., Sheikh, I. S., Kong, W., Aljaafari, D., Barot, N., Benbadis, S., Friedman, D., Gavvala, J. R., Halford, J., Hogan, R. E., Kaplan, P., Karakis, I., Maheshwari, A., Matthews, R., O'Donovan, C., Rampp, S., ... Westover, M. B. (2026). Noise in the diagnosis of epilepsy by experts (version 1.0.0). Brain Data Science Platform. https://doi.org/10.60508/zhep-m488.

MLA	Nascimento, Fábio A., et al. "Noise in the diagnosis of epilepsy by experts" (version 1.0.0). Brain Data Science Platform (2026), https://doi.org/10.60508/zhep-m488.
APA	Nascimento, F. A., McLaren, J., Zhao, W., Katyal, R., Sheikh, I. S., Kong, W., Aljaafari, D., Barot, N., Benbadis, S., Friedman, D., Gavvala, J. R., Halford, J., Hogan, R. E., Kaplan, P., Karakis, I., Maheshwari, A., Matthews, R., O'Donovan, C., Rampp, S., ... Westover, M. B. (2026). Noise in the diagnosis of epilepsy by experts (version 1.0.0). Brain Data Science Platform. https://doi.org/10.60508/zhep-m488.
Chicago	Nascimento, Fábio A., McLaren, John, Zhao, Wei, Katyal, Roohi, Sheikh, Irfan S., Kong, Wan-Yee, Aljaafari, Danah, Barot, Nirav, Benbadis, Selim, Friedman, Daniel, Gavvala, Jay R, Halford, Jonathan, Hogan, R. Edward, Kaplan, Peter, Karakis, Ioannis, Maheshwari, Atul, Matthews, Rebecca, O'Donovan, Cormac, Rampp, Stefan, Schuele, Stephan, Sirven, Joseph, Tatum, William O., Williams, Jonathan, Yacubian, Elza Marcia, Yuan, Doyle, Beniczky, Sándor, Sibony, Olivier, and M. Brandon Westover. "Noise in the diagnosis of epilepsy by experts" (version 1.0.0). Brain Data Science Platform (2026). https://doi.org/10.60508/zhep-m488.
Harvard	Nascimento, F. A., McLaren, J., Zhao, W., Katyal, R., Sheikh, I. S., Kong, W., Aljaafari, D., Barot, N., Benbadis, S., Friedman, D., Gavvala, J. R., Halford, J., Hogan, R. E., Kaplan, P., Karakis, I., Maheshwari, A., Matthews, R., O'Donovan, C., Rampp, S., Schuele, S., Sirven, J., Tatum, W. O., Williams, J., Yacubian, E. M., Yuan, D., Beniczky, S., Sibony, O., and Westover, M. B. (2026) 'Noise in the diagnosis of epilepsy by experts' (version 1.0.0), Brain Data Science Platform. Available at: https://doi.org/10.60508/zhep-m488.
Vancouver	Nascimento F A, McLaren J, Zhao W, Katyal R, Sheikh I S, Kong W, Aljaafari D, Barot N, Benbadis S, Friedman D, Gavvala J R, Halford J, Hogan R E, Kaplan P, Karakis I, Maheshwari A, Matthews R, O'Donovan C, Rampp S, Schuele S, Sirven J, Tatum W O, Williams J, Yacubian E M, Yuan D, Beniczky S, Sibony O, Westover M B. Noise in the diagnosis of epilepsy by experts (version 1.0.0). Brain Data Science Platform. 2026. Available from: https://doi.org/10.60508/zhep-m488.

Abstract

OBJECTIVE: To measure the relative levels of signal and noise in expert diagnosis of epilepsy.

METHODS: Twenty multinational epileptologists independently reviewed 50 vignettes of adult and pediatric patients presenting with suspected seizure(s) on two separate occasions with a ≥30-day washout period. Experts provided a diagnosis of epilepsy or non-epilepsy based on clinical information and, if requested, routine EEG and neuroimaging data. Cases had an established clinical diagnosis of epilepsy or non-epilepsy based on capture of habitual paroxysmal events on video-EEG or long-term clinical follow-up. Experts' judgments were analyzed to decompose variability into different sources: signal (objective differences between cases), level noise (experts' bias toward over/under-diagnosis), pattern noise (experts' idiosyncratic reactions to specific case features), and occasion noise (inconsistency across occasions).

RESULTS: The probability of an expert making a different diagnosis for a given case on two different occasions was 16%. The probability of two different experts making a different diagnosis for the same case was 26%. Signal (case "difficulty") accounted for 66-69% of total variation, with 31-34% attributable to noise. Level noise was the largest contributor in the absence of EEG/neuroimaging results (23%), while pattern noise dominated when test results were available (24%). Occasion noise contributed relatively little (1%) but was still sufficient to cause diagnostic reversals in 16-22% between occasions.

SIGNIFICANCE: The degree of noise in expert diagnosis of epilepsy is substantial, stemming primarily from physicians' idiosyncratic interpretations of case features and variable dispositions toward over- or under-diagnosis. Strategies to improve reliability are needed, including standardized data collection protocols and structured decision algorithms. For "difficult cases," where expert reliability and accuracy are lowest, our findings support current clinical practice which favors early referral for video-EEG monitoring over reliance on diagnostic anchoring. This diagnostic pathway may become more accessible with advances in EEG technology (e.g., wearable devices) and artificial intelligence.

Background

Diagnostic decisions vary between clinicians (bias) and within the same clinician across occasions (inconsistency, or 'noise'). Quantifying how much of expert epilepsy diagnosis is signal versus noise, and which kinds of noise dominate, is essential for understanding and improving diagnostic reliability.

Software Description

This is a code + de-identified data release (not a large credentialed dataset). Everything needed to reproduce the paper is in the GitHub repository:

Monkey2_Cases_Mastersheet_deID.xlsx - the 50 anonymized patient vignettes.
Monkey2_Responses_deID.xlsx + dataQ1/Q2/QEEG.csv - the 20 experts' responses across two occasions.
*_latent_effects.csv - committed Bayesian model outputs (variance decomposition).
Analysis: a0_Monkey2_Bayesian_v2.ipynb (PyMC model + figures), a1_MakeSummaryStats.m (MATLAB).

Experts and cases are anonymized (index only); no names, MRNs, or dates.

Technical Implementation

Twenty multinational epileptologists independently reviewed 50 de-identified vignettes of adult and pediatric patients with suspected seizures on two occasions (>=30-day washout). For each case they answered whether it was epilepsy based on history alone (Q1), whether they would order an EEG (QEEG), and a final diagnosis with EEG/imaging revealed (Q2).

Model

For each question the log-odds of a 'yes' response for expert i, case j, occasion t is modeled as mu + c_j + l_i + p_ij + o_ijt, with non-centered normal priors: c_j = case signal, l_i = level noise (expert baseline), p_ij = pattern noise (expert x case interaction), o_ijt = occasion noise (within-expert inconsistency). Fit with PyMC (NUTS). Supporting descriptive agreement statistics are computed alongside.

Installation and Requirements

Python 3.10+: pip install -r requirements.txt (pymc, arviz, numpy, pandas, scipy, matplotlib, openpyxl). MATLAB for the summary-statistics script.

Usage Notes

Code & reproduction: https://github.com/bdsp-core/Noise_in_Diagnosing_Epilepsy (BSD/CC). Verified 2026-07-07: Figures 2-3 and the variance-component matrices regenerate from the committed model outputs via a0_Monkey2_Bayesian_v2.ipynb; the full Bayesian MCMC and MATLAB summary statistics are included. See REPRODUCE.md and DATA_SOURCE.md.

Release Notes

Version 1.0.0 - initial release accompanying Nascimento et al., Epileptic Disorders 2026 (doi:10.1002/epd2.70181).

Ethics

The study used de-identified patient vignettes reviewed by expert readers. [AUTHOR CHECK: confirm IRB determination/approval language.]

Acknowledgements

[AUTHOR CHECK: funding/acknowledgements from the manuscript.]

Conflicts of Interest

M. Brandon Westover is a co-founder of, scientific advisor to, consultant for, and has a personal equity interest in Beacon Biosignals. [AUTHOR CHECK: confirm co-author disclosures.]