Explainable paroxysmal atrial fibrillation diagnosis using an artificial intelligence-enabled electrocardiogram

Data sources and study population

We extracted standard 12-lead ECGs for patients with at least one instance of normal sinus rhythm (NSR) recorded between May 16, 2013 and December 31, 2020. Each ECG was captured at a 500 Hz sampling rate, with raw data stored in the MUSE cardiology information system (GE Healthcare, Chicago, IL, USA).

The extracted ECG data included 10-s recordings. The quantitative measurements from ECG clinical reports were analyzed to identify diagnostic class and 18 patient features. These features were age, sex, ventricular/atrial rate, QRS duration/count, QTc (Bazett/Fridericia correction), QT interval, PR interval, the axes of P, R, and T waves, T-offset and on/offset of P and Q. Each variable had a missing value rate of 0–24.7%, with the highest percentage being sex. For cases with missing values, due to reasons such as privacy constraints, we imputed values to denote the absence of information.

In the prepared data, we excluded 95,398 ECGs where patient ID was not tracked and 287,247 ECGs where diagnostic classification was unavailable for AF or NSR. Rhythm diagnosis and labeling were carefully performed by clinical experts to ensure reliability before further analysis. These annotations were further validated using the electronic medical records of the patients, cross-referencing for AF diagnosis codes or documented history. Only AF diagnosis codes or previous history without a documented 12-lead ECG were not classified as PAF.

In the ECG clinical report of a patient, NSR recorded within 31 days following the first AF episode was labeled as “PAF.” This broader definition, extending beyond the standard definition of PAF (AF episodes lasting under 7 days without intervention), enabled a comprehensive assessment of early AF episodes. The ECGs recorded more than 31 days after the first AF episode and those taken before the first onset were excluded. ECGs with consistent NSR across clinical records were annotated as ‘Normal.’ After annotation, we categorized patients into PAF and control groups and then deidentified the ECG data to remove personal ID. This approach ensured that subtle ECG patterns related to PAF were robustly identified, independent of individual characteristics.

We initially extracted 1,014,617 raw ECGs from 422,664 patients. From these, 95,398 ECGs with incomplete PID tracking or missing data were excluded, as well as 287,247 ECGs without AF records or NSR. Additionally, sinus rhythm ECGs immediately preceding the first AF event were excluded to avoid recording transitional ECG patterns that may not accurately reflect the early characteristics of PAF [7], thereby reducing noise and enhancing model prediction accuracy. ECGs were annotated as normal if all serial recordings for a patient showed NSR, while ECG records within the target window were labeled as PAF. After applying certain exclusions (Fig. 1), the dataset included 552,372 ECGs from 318,321 patients. Applying predefined labeling protocol, 26,541 ECGs (4.8%) were annotated as PAF, reflecting similar prevalence rates in the general population [24]. The processed ECGs were divided into training (60%), validation (20%), and test datasets (20%) for model training.

Figure 1

Summary of data used in the study. Diagram summarizing patient selection for training, validation, and testing cohorts. ECG, electrocardiogram; PID, patient identification.

Algorithm development and evaluation for PAF early detection

The convolutional neural network (CNN)-based statistical model for early PAF detection was developed using patient demographics and raw ECG data as input. The CNN model was designed to identify patterns and extract local spatial features from global maps using filters that apply each input subregion through dot product operations [25]. Given the seasonality and fixed length of ECG signals, 1d-CNN is the most appropriate to apply [26]. The network architecture for analyzing standard 12-lead ECGs (sampled at 500 Hz) included 50 convolutional layers, using the skip-connection method of the residual network [27] to ensure effective training. The model we used consisted of 16 residual blocks, each containing three convolution layers, followed by a Batch Normalization and rectified linear activation function (ReLU) applied after each layer [28,29]. Where the output feature map dimensions decrease, a convolutional layer with a stride of 2 was applied to ensure alignment between input and output dimensions. Pooling was conducted at the first and last layers after the nonlinear activation function. A feature vector was then extracted from the ECG by applying two fully connected (FC) layers with ReLU activations. Finally, this feature vector was concatenated with a 12-lead ECG feature vector, which produced class probabilities via a final FC layer and softmax function. The network weights were initialized following the He method, and the model was optimized using the Adam algorithm with default parameters β1 = 0.9 and β2 = 0.999 [30,31]. The batch size was 32, with a learning rate of 1 × 10⁻⁴, halved every 10 epochs. Figure 2 depicts the overall architect of our model.

Figure 2

Network architecture. Network architecture used in the study. Our network processes raw ECG data (sampled at 500 Hz) and ECG features to predict AF onset within a month. ECG, electrocardiogram; ReLU, rectified linear activation function; PAF, paroxysmal atrial fibrillation; AF, atrial fibrillation.

Additionally, focal loss was used to address data imbalance. This loss function—a variation of cross entropy—counteracts extreme interclass imbalances by assigning lower weights to easily classified negatives to reduce their contribution to learning while increasing weights for challenging positives that are harder to classify [32]. Hyperparameters for the network structure and loss function used were set through grid search and manual tuning.

We trained 30 models using different dataset configurations generated through pseudo-random number sampling. The model performance was evaluated using the area under the curve (AUC) metric, a key performance indicator in binary classification where a higher AUC indicates better performance. To further evaluate the generalizability and stability of the model, we recorded the AUC of the developed models.

Identify contributors to PAF early detection through LRP

The interpretability of complex deep-learning models remains a major concern in the medical field. Enhancing model transparency can improve acceptance in clinical decision-making and justify specific diagnoses and treatment recommendations [33]. Layer-wise relevance propagation (LRP) is a prominent method for interpreting complex deep-learning models by measuring the contribution of each input to the output of the model [34]. It functions by estimating and decomposing the layer-level contributions presenting relevance scores for input features as a heatmap. We calculated the relevance score of each input value by backpropagating from the class score of the output node to the input layer, scaling values between 0 and 1 across all leads. This produced a scaled relevance score for each input dimension. To elucidate factors influencing early PAF diagnosis, we interpreted LRP from the following perspectives: leads and intervals. Lead contributions were identified by averaging relevance scores for each lead across classifications. To maintain consistency and facilitate model interpretation, the ECG summary process was referenced and applied to relevance scores. Next, the ECG intervals were confirmed by summarizing key ECG measurements obtaining the average value of the relevance score for each interval.

Ethics statement

The Institutional Review Board of Chonnam National University Hospital approved this study with a waiver of consent (CNUH-2021-158), adhering to institutional patient privacy policies.

Evaluation of model performance

The limited number of positive cases required for effective training was a critical barrier when applying deep learning to diagnose PAF. To address this class imbalance, we employed focal loss as a loss function of CNN. This technique reduces the influence of easy negative cases while assigning greater weight to hard positive cases, thereby enhancing model accuracy. Here, we implemented a CNN model that inputs ECG data with features such as age, sex, and ECG parameters. Model performance was evaluated using the area under the curve of the receiver operating characteristic curve (AUROC). To understand the generalizability and stability of the model, 30 datasets were generated through pseudo-random sampling and trained individually against the corresponding test dataset. The performance results showed a sensitivity, specificity, and F1 score of 0.722 ± 0.015, 0.906 ± 0.006, and 0.542 ± 0.011, respectively. The AUC for early PAF diagnosis was 0.905 ± 0.007. Class-specific predictive accuracy was 0.826 ± 0.011 for negative cases (Fig. 3, blue) and 0.702 ± 0.019 for the positive cases (Fig. 3, red).

Figure 3

Model performance. ROC curves of the trained model for all data (black), negative prediction ECG subset (blue), and positive prediction ECG subset (red). ECG, electrocardiogram; AUC, area under the curve; ROC, receiver operating characteristic.

Interpreting the inference process of the deep learning model

Although our model showed strong performance in diagnosing PAF using large-scale data, interpreting the inference process of the deep learning model remains challenging. We used the LRP method to explore the ECG leads and intervals that affected PAF prediction. The P wave is clinically known to affect the diagnosis of PAF [18,21,35,36]. We hypothesized that critical factors could influence prediction more subtly, which could be revealed by analyzing ECG contribution.

To explain the deep learning explanation technique on ECG records, we used the LRP method to identify key ECG leads and intervals. First, the average relevance scores of each ECG lead in the confusion matrix were calculated (Table 2). Regardless of prediction results, the aVR lead had the highest relevance score, indicating it as the most referenced by the model in PAF diagnosis. In contrast, other ECG features showed low relevance scores, suggesting limited predictive effect. To identify the most significant interval for PAF diagnosis and minimize mutual cancellation of ECG signals, we constructed an averaged ECG by aligning, sorting, and interpolating each QRS complex based on the R-peaks. Figure 4 displays this summarized ECG along with LRP intensities for the true positive cases. We summarized ECGs for true positives (Fig. 4, black) and negatives (Fig. 4, blue) for each lead. These summaries reveal a significant difference in the ST segment and T wave regions. In particular, the true positives showed T wave depression, suggesting that an inverted (or flattened) T wave may influence the PAF diagnosis. The T waves, indicating ventricular repolarization [37], are commonly classified as nonspecific ST-T abnormalities (NSTTA) when flattened. To identify the most significant ECG interval in PAF diagnosis, the average LRP weights across intervals were compared (Table 3). The ST segment and QRS complex emerged as the strongest predictors for PAF. We concluded that NSTTA could be a characteristic associated with PAF. Studies show that ST segment depression and T wave inversion are significant markers for cardiovascular disease, with T wave inversion linked to increased mortality risk [38,39]. Additionally, transient ST segment depression has been observed during PAF episodes and linked to underlying coronary artery disease [40]. These findings indicate the association between NSTTA and PAF, which is consistent with our results, underscoring the significance of NSTTA and inverted T waves in predicting early PAF diagnosis, further emphasizing their clinical relevance in arrhythmia detection. However, the relationship between NSTTA and PAF has received limited attention, and our analysis confirms that a flattened T wave influences PAF diagnosis. This suggests that T waves could serve as novel predictors for early PAF detection and that deep learning models can effectively reveal complex mechanisms in PAF diagnosis.

Table 2

Average of relevance scores for each ECG lead in the confusion matrix

Figure 4

Identification of major contributors to PAF diagnosis through model explainability with the LRP method. Overlay of summarized ECG signals for all true positive (black) and negative (blue) cases. The ECG signals for each lead are aligned and interpolated into one signal. The LRP projection map for true positives is highlighted in red, with the darker red regions indicating a stronger contribution to PAF diagnosis. LRP, layer-wise relevance propagation; ECG, electrocardiogram; PAF, paroxysmal atrial fibrillation.

Table 3

Relevance scores for each ECG interval