Patients hospitalized in the Armed Forces Capital Hospital from 15 November 2016 through 25 December 2019 were screened. The Armed Forces Capital Hospital is the highest-level referral center among 17 military hospitals in South Korea, and is the only center capable of performing bronchoscopy. Therefore, all patients suspected to have AEP are transferred to our center [10,12,15]. For this study, patients diagnosed with either AEP or pneumonia were selected. Because the aim of our study was to aid the diagnosis of AEP among military smokers, those who did not smoke at the time of admission were excluded. We also excluded patients without CT results, because chest CT findings are one of the key characteristics of AEP [6,16].

The diagnosis of AEP was confirmed according to the modified Philit criteria [7]: (1) acute respiratory illness (≤ 1 month); (2) pulmonary infiltrates on chest imaging; (3) pulmonary eosinophilia (> 25% eosinophils in the bronchoalveolar lavage fluid); and (4) absence of other specific pulmonary eosinophilic diseases. Some patients could not undergo bronchoscopy due to their urgent medical condition, unavailability of emergent bronchoscopy, or a temporary shortage of medical staff. Therefore, they were clinically suspected to have AEP and were treated accordingly. These patients were excluded because the fulfilment of the modified Philit criteria could not be assessed, nor could they be classified as having other pneumonias. Thus, we excluded patients whose diagnosis was inconclusive, who were transferred to other hospitals, and who died before the final diagnosis. The patients who were hospitalized from 15 November 2016 through 31 December 2018 were assigned to the derivation group, and those hospitalized after this period were assigned to the validation group.

Data were collected retrospectively from the patients’ medical records; these included data on patient demographics, smoking history, comorbidities, and symptoms. Smoking history included conventional cigarettes only. We evaluated the duration (weeks), quantity (packs per day), and any change in habits of smoking. Patients who restarted smoking in the preceding 4 weeks after quitting for longer than 4 weeks were considered new-onset smokers, and patients who doubled their quantity of daily smoking within the preceding 4 weeks were considered to have increased their quantity of smoking. The comorbidities inspected were hypertension, diabetes mellitus, chronic lung disease (asthma, chronic obstructive pulmonary disease, bronchiectasis, or tuberculosis-destroyed lung), chronic liver disease, history of tuberculosis, history of AEP, and any type of allergy. Symptoms consisted of dyspnea, fever, chills, night sweats, cough, sputum, chest pain, rhinorrhea, myalgia, fatigue, palpitation, and sore throat, along with the onset of the chief complaint. Laboratory findings that could be commonly evaluated were extracted: white blood cell count, neutrophil count, lymphocyte count, eosinophil count, platelet count, and C-reactive protein levels.

The included chest CT findings were ground glass opacity, interlobular septal thickening, bronchovascular bundle thickening, pleural effusion, bilateral involvement, consolidation, and centrilobular nodules [6,17,18]. The CT findings were independently inspected by two blinded researchers (S.P. and D.H.) and any discrepancy was resolved by another researcher (H.J.K.).

Our study was conducted in accordance with the amended Declaration of Helsinki and was approved by the Institutional Review Board of the Armed Forces Capital Hospital (protocol number: AFCH-20-IRB-025). The requirement for informed consent was waived because of the retrospective design of the study, but the records were anonymized prior to the analyses. There were no relevant missing data found during the data collection process.

The scoring system was constructed according to the recent recommendation for the development and reporting of prediction models in respiratory medicine [19]. The candidate variables included the majority of the inspected characteristics. Since this scoring system was designed for bedside use, continuous variables were transformed into generally acceptable forms. The white blood cell count was transformed to leukocytosis (≥ 10,000/μL) or not, platelet count to thrombocytopenia (≤ 150,000/μL) or not, eosinophil count to eosinophilia (≥ 500/μL) or not, and neutrophil and lymphocyte counts to the neutrophil-to-lymphocyte ratio. The smoking history was simplified to ‘new-onset or a recent increase in the quantity of smoking (≤ 4 weeks)’ [20,21]. The onset of the chief complaint was simplified as acute onset (≤ 3 days) or otherwise. Consequently, a total of 36 candidate variables were included to construct the scoring system (Supplementary Table 1).

For variable selection and score derivation, the least absolute shrinkage and selection operator (LASSO) regression was used [22]. With LASSO regression, a simplified list of variables was derived with 10-fold cross-validation, and the penalized coefficients were calculated. To simplify the coefficients for use in the scoring system, they were multiplied by 5 and rounded to the nearest integer.

Each patient’s total score was calculated using the derived scoring system. The total score was used to evaluate its performance by calculating the area under the receiver-operating characteristics curve (AUC) to predict the diagnosis of AEP for the derivation and validation groups. To evaluate the goodness-of-fit of our model, a calibration plot was obtained and the Hosmer-Lemeshow test was performed for both groups.

Our study was performed in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement [23]. The TRIPOD checklist is available in Appendix 1.

During the study period, 1,178 patients were screened. After excluding the non-eligible patients, 414 patients were included in our study and were divided into the derivation (n = 279) and validation groups (n = 135); 54 of 279 patients (19.4%) and 18 of 135 patients (13.3%) were diagnosed with AEP in the two groups, respectively (Fig. 1).

All the patients were male, with a median age of 20 years (interquartile range, 20 to 21). The median duration of smoking was 37.5 months (interquartile range, 12.0 to 62.5), which was shorter for those diagnosed with AEP than for those diagnosed with other types of pneumonia (median 0.8 months vs. 48.0 months, p < 0.001). Fifty-seven of 72 patients (79.2%) diagnosed with AEP were classified as new-onset smokers, as compared to 13 of 342 patients (3.8%) diagnosed with other types of pneumonia (p < 0.001). The distribution of comorbidities did not differ between patients diagnosed with AEP and those without. The patients diagnosed with AEP had a higher white blood cell count (median 14,015/μL vs. 7,260/μL, p < 0.001), neutrophil count (median 10,940/μL vs. 5,175/μL, p < 0.001), lymphocyte count (median 1,375/μL vs. 1,230/μL, p = 0.042), platelet count (median 241 × 103/μL vs. 189 × 103/μL, p < 0.001), and higher levels of C-reactive protein (median 7.9 mg/dL vs. 5.4 mg/dL, p = 0.006). Radiographic findings differed significantly according to the diagnosis of AEP. Among the patients with AEP, ground glass opacity (100.0% vs. 77.5%, p < 0.001), interlobular septal thickening (94.4% vs. 6.1%, p < 0.001), pleural effusion (36.1% vs. 7.6%, p < 0.001), bronchovascular bundle thickening (33.3% vs. 1.5%, p < 0.001), and bilateral involvement (98.6% vs. 37.1%, p < 0.001) were more commonly found than in patients diagnosed with other types of pneumonia. The prevalence of consolidation (40.3% vs. 87.1%, p < 0.001) was lower in patients with AEP. Results from microbiological evaluation were less likely to be positive in patients with AEP compared to those without AEP in both bacterial (56.9% vs. 75.0%, p = 0.002) and viral origin (19.3% vs. 73.4%, p < 0.001) from respiratory specimens (Table 1). Details of the microbiological evaluation are presented in Supplementary Table 2.

Patients in the derivation and validation groups revealed similar characteristics, although there were several differences. Patients in the derivation group had a higher incidence of interlobular septal thickening (24.4% vs. 15.6%, p = 0.041) and bilateral involvement (51.3% vs. 40.7%, p = 0.045). Patient symptoms and laboratory findings did not differ between the two groups (Supplementary Table 3).

After LASSO regression, 14 variables were selected from the 36 candidate variables. The selected variables were new-onset or a recent increase in the quantity of smoking (≤ 4 weeks), interlobular septal thickening, ground glass opacity, the presence of sputum, bronchovascular bundle thickening, dyspnea, chest pain, leukocytosis (≥ 10,000/μL), bilateral involvement, acute onset (≤ 3 days), fever, fatigue, pleural effusion, and a higher neutrophil-to-lymphocyte ratio. The penalized coefficients were calculated, multiplied by 5, and subsequently rounded off to the nearest integer. After this process, the coefficients of four variables (fever, fatigue, pleural effusion, and the neutrophil-to-lymphocyte ratio) became zero and were excluded from the final score (Table 2).

The scores were organized as follows: 8 points for new-onset or a recent increase in the quantity of smoking (≤ 4 weeks); 5 points for interlobular septal thickening; 3 points for both the absence of sputum and ground glass opacity; and 2 points each for acute onset (≤ 3 days), dyspnea, chest pain, leukocytosis (≥ 10,000/μL), bronchovascular bundle thickening, and bilateral involvement (Table 3). The score distribution ranged from 0 to 31 points. We named this scoring system ‘AEP Diagnostic Index for Military Smokers,’ which was abbreviated as ADIMS. The ADIMS score was calculated for each patient and showed a similar distribution among the derivation and validation groups (Supplementary Fig. 1).

The AUC of the ADIMS to distinguish AEP from other types of pneumonia was 0.997 (95% confidence interval [CI], 0.992 to 1.000) in the derivation group, and 0.985 (95% CI, 0.965 to 1.000) in the validation group, which refers to an excellent performance (Fig. 2). We evaluated the calibration plots and Hosmer-Lemeshow tests, which revealed that the ADIMS fit well (Supplementary Fig. 2).

Various cut-off values were used to calculate the sensitivities and specificities to correctly diagnose AEP. With cut-off values of ≥ 18, 96.30% of patients could be correctly classified. The cut-off of ≥ 18 points showed a sensitivity of 94.44% and a specificity of 96.58% (Supplementary Table 4).