Validation and modification of HEART score components for patients with chest pain in the emergency department
Article information
Abstract
Objective
This study aimed to clarify the relative prognostic value of each History, Electrocardiography, Age, Risk Factors, and Troponin (HEART) score component for major adverse cardiac events (MACE) within 3 months and validate the modified HEART (mHEART) score.
Methods
This study evaluated the HEART score components for patients with chest symptoms visiting the emergency department from November 19, 2018 to November 19, 2019. All components were evaluated using logistic regression analysis and the scores for HEART, mHEART, and Thrombolysis in Myocardial Infarction (TIMI) were determined using the receiver operating characteristics curve.
Results
The patients were divided into a derivation (809 patients) and a validation group (298 patients). In multivariate analysis, age did not show statistical significance in the detection of MACE within 3 months and the mHEART score was calculated after omitting the age component. The areas under the receiver operating characteristics curves for HEART, mHEART and TIMI scores in the prediction of MACE within 3 months were 0.88, 0.91, and 0.83, respectively, in the derivation group; and 0.88, 0.91, and 0.81, respectively, in the validation group. When the cutoff value for each scoring system was determined for the maintenance of a negative predictive value for a MACE rate >99%, the mHEART score showed the highest sensitivity, specificity, positive predictive value, and negative predictive value (97.4%, 54.2%, 23.7%, and 99.3%, respectively).
Conclusion
Our study showed that the mHEART score better detects short-term MACE in high-risk patients and ensures the safe disposition of low-risk patients than the HEART and TIMI scores.
INTRODUCTION
Chest pain is a frequently cited and potentially serious complaint among patients visiting the emergency department (ED). Therefore, it is vital for emergency physicians to identify whether this chest pain is consistent with acute coronary syndrome and to predict the outcomes associated with chest pain in the ED. Several studies have evaluated the efficacy of various clinical risk stratification scores in the prediction of outcomes in patients with chest pain, which are as follows: Thrombolysis in Myocardial Infarction (TIMI) risk score; History, Electrocardiography (ECG), Age, Risk Factors, and Troponin (HEART) score; and Global Registry of Acute Coronary Events (GRACE) score [1-5]. However, only the HEART score was designed for use in the ED. Owing to its ease of use and superior accuracy in the prediction of short-term major adverse cardiac events (MACE) [6-11], the American College of Emergency Physicians recently categorized the use of the HEART score as a level B recommendation for the initial evaluation of patients presenting to the ED with chest pain [12].
The HEART score is a prognostic tool employed to assign undifferentiated chest pain patients to low-, intermediate-, and high-risk short-term MACE groups. However, most previous studies on the topic primarily examined the use of the HEART score in the identification of those with a safe disposition in terms of MACE incidence. Moreover, previous studies did not clarify the actual extent of the individual impact of each HEART score component on MACE incidence [7-11,13-20].
Accordingly, we performed a retrospective external validation study to clarify the relative prognostic value of each component of the score in terms of MACE presenting within 3 months and to validate the modified HEART (mHEART) score.
METHODS
Study design and patient selection
We conducted our study at the ED of an urban academic tertiarycare hospital in South Korea. This 900-bed facility has a regional emergency medical center, as designated by the government, and receives approximately 75,000 ED annual visits. Participants were selected through a systematic review of electronic medical records (EMRs). We included all adult patients (aged 18 years or older) with a chief complaint of chest pain, chest tightness, chest pressure, or chest discomfort, as found in a keyword search for “chief complaint” of our EMRs from November 19, 2018 to November 19, 2019. The exclusion criteria were as follows: (1) age under 18 years, (2) presence of fever or cough, (3) presence of chest pain due to trauma, (4) negative response for the presence of chest-related symptoms on questioning by a primary ED physician, (5) presence of abdominal pain (e.g., epigastric pain) when examined by a primary ED physician, which was later established to be a symptom of the gastrointestinal tract or an accessory organ-related condition using computed tomography (e.g., acute cholangitis), (6) presence of an altered level of consciousness upon arrival at the ED, (7) absence of initial troponin-I measurements or ECG data or patient refusal to undergo evaluation in the ED, (8) insufficient data for the calculation of the HEART score, and (9) presence of ST-segment elevation myocardial infarction on the initial ECG. We also excluded participants who were lost to follow-up at 3 months after ED presentation although we performed a telephone interview at the end of the data collection period (e.g., participants who did not answer the phone or had provided the wrong phone number). Patients included during the first 8 months (November 2018 to July 2019) were assigned to the derivation group and those included during the latter 4 months (August 2019 to November 2019) to the internal validation group.
Data collection
The EMRs of the enrolled patients were reviewed. Since our ED has been using the HEART score as a routine protocol for the safe disposition and risk stratification of patients presenting to the ED with chest symptoms since March 1, 2018, we were able to determine the factors for calculating the HEART score from the EMRs of most of the eligible patients during the data collection period. Data on patients’ baseline characteristics (sex, age, smoking, familial history of coronary artery disease [CAD], aspirin use in the past 7 days), underlying disease (hypertension, diabetes mellitus, obesity, dyslipidemia, atherosclerotic disease), initial vital signs (systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate), history, initial troponin-I level, ECG results, HEART score, and MACE incidence within the previous 3 months were obtained and evaluated retrospectively. MACE were defined as a composite of acute myocardial infarction, percutaneous intervention, coronary artery bypass graft, and all-cause mortality. The HEART score incorporates the elements of history, ECG, age, risk factors, and cardiac troponin levels, and ranges from 0 to 2 in each of these five categories, with 0 being the lowest possible score and 10 the highest. A 3-month follow-up was performed through the evaluation of patients’ hospital records or if these data were unavailable, by a telephone interview. The component of history was evaluated by a senior resident specializing in emergency medicine, and in cases involving ambiguity, it was further evaluated by faculty members specializing in emergency medicine.
Statistical analyses
The normality of the data distributions was evaluated using the Kolmogorov-Smirnov test for the selection of appropriate parametric and nonparametric statistical methods. Categorical variables were analyzed using chi-squared or Fisher exact tests and expressed as total numbers (percentages). Continuous variables were expressed as medians (25th–75th percentiles) and analyzed using the Mann-Whitney U-test. Independent risk factors associated with MACE occurrence within 3 months in terms of the HEART score components were evaluated using multivariate backward stepwise logistic regression. Based on the multivariate analyses, we calculated the mHEART score and then evaluated the original HEART score, mHEART score, and the TIMI score using receiver operating characteristic (ROC) curves in the derivation and validation groups. Subsequently, we evaluated the cutoff values to maintain a negative predictive value (NPV) with a MACE rate >99% for the HEART, mHEART, and TIMI scores in the validation group. With these cutoff values, we compared the performance for the sensitivity, specificity, positive predictive value (PPV), and NPV in the prediction of MACE within 3 months. For all comparisons, the tests were two-tailed, and between-group differences were regarded as statistically significant at a P-value lower than 0.05. PASW Statistics ver. 18.0 (SPSS Inc., Chicago, IL, USA) was used for all analyses.
Ethics statement
This study was approved by the institutional review board of Hallym University Sacred Heart Hospital (2020-07-004). Owing to the purely observational, retrospective, and noninterventional nature of the study, the need for informed consent was waived.
RESULTS
Baseline and clinical characteristics of the patients
The study population was enrolled from among 1,634 consecutive patients with chest-related symptoms who presented to the ED for evaluation. Overall, 524 patients were excluded, and 1,110 patients met the inclusion criteria. Three patients were lost to follow-up, as they could not be reached for a telephone interview or due to a lack of EMR data. Finally, 1,107 patients were included in the analysis and categorized into the derivation group (809 patients) and the validation group (298 patients) (Fig. 1). The differences in baseline characteristics between the derivation and validation groups are summarized in Table 1. Although there were no significant differences in the patients’ age, comorbidities, smoking status, aspirin use in the past 7 days, family history of CAD, initial vital signs, or initial troponin-I levels between the two groups, there was a significant difference in terms of sex (proportion of men: 63.5% in the derivation group and 53.4% in the validation group, P=0.002). The overall MACE incidence rate within 3 months was not significantly different between the 2 groups (12.9% vs. 12.8%, respectively) (Table 2).
HEART and mHEART scores
Table 2 describes the HEART score components and risk classification in both groups. Among the components, history and ECG results showed significant differences between the groups (P<0.001 and P=0.003, respectively); however, age, initial troponin-I levels, and risk factors did not show significant differences. In both groups, the proportion of patients with a score of 1 was found to be the highest in terms of the components of history, age, and risk factors, whereas the proportion of those with a score of 0 was the highest for troponin-I. In terms of ECG results, we observed the highest numbers of patients with a score of 1 and 0 in the derivation and validation groups, respectively. In terms of risk classification, the proportion of those with a high risk (7–10 points) was low in both groups (6.7% and 6.7%, respectively). A tendency of increasing MACE incidence was observed with increasing scores, and comparisons were performed between the HEART, mHEART, and TIMI scoring systems (Fig. 2). Univariate and multivariate analyses of the HEART score components were performed in the derivation group (Table 3). In the univariate logistic regression analysis, any score greater than 0 for the components of history, age, risk factors, or initial troponin-I level and a score of 2 for the ECG component indicated statistical significance compared to a score of 0 for each component, except in cases with a score of 1 for the ECG component. In the subsequent multivariate analysis, the age component did not show statistical significance in the detection of MACE occurrence within 3 months. Based on the results of the multivariate analysis, we calculated the mHEART score by omitting the age component, with a score ranging from a minimum of 0 to a maximum of 8 (Table 4).
HEART, mHEART, and TIMI scores for the detection of MACE incidence within 3 months
The areas under the ROC curves (AUROCs) for the HEART, mHEART, and TIMI scores in the prediction of MACE incidence within 3 months were 0.88 (95% confidence interval [CI], 0.85–0.91; P<0.001), 0.91 (95% CI, 0.86–0.94; P<0.001), and 0.83 (95% CI, 0.79–0.87; P<0.001), respectively, in the derivation group and 0.88 (95% CI, 0.82–0.30; P<0.001), 0.91 (95% CI, 0.86–0.95; P<0.001), and 0.81 (95% CI, 0.75–0.87; P<0.001), respectively, in the validation group (Fig. 3). The performances of the cutoff values for the HEART, mHEART, and TIMI scores in the validation group are shown in Table 5. The cutoff value for each scoring system was determined to maintain a negative predictive value (NPV) for a MACE rate >99% (HEART score 3, mHEART score 2, TIMI score 0). The mHEART score showed the highest sensitivity, specificity, positive predictive value (PPV), and NPV (97.4%, 54.2%, 23.7%, and 99.3%, respectively). Comparison of overall MACE between low- and intermediate/high-risk groups in the validation group and details are presented in Table 6.
DISCUSSION
In the present retrospective external validation study that sought to clarify the relative prognostic value of each HEART score component for MACE occurrence within 3 months, the age component did not show statistical significance in the multivariate logistic regression analysis. The mHEART score was calculated without the age component, with a range of a minimum of 0 points to a maximum of 8 points. The mHEART score performed better, with the highest AUROC, when the detection of MACE incidence within 3 months was considered compared to the HEART and TIMI scores.
MACE within 3 months
According to an earlier systematic review, which screened 514 studies regarding clinical scores for risk stratification of chest pain patients in the ED that were published between 1 January 2012 and 25 September 2017, a total of 29 studies on the validation and modification of HEART, TIMI, GRACE, and heart rate variability based scores were reviewed. The duration of the short-term MACE ranged from 72 hours to 6 months after the time when patients’ disposition was made and 12 months was classified as the long-term duration for MACE [21]. In addition, in a study conducted by Six et al. [10], who first proposed the HEART score, the long-term follow-up data of 120 out of a total of 122 eligible patients were available, with a duration of 423±106 days. During this period, MACE were reported in 29 patients, all of which occurred within a time frame of 3 months. Accordingly, our study assessed a more efficient period of short-term MACE which was of 3 months to maximize the ability to detect these events.
Baseline and clinical characteristics of the patients
The demographic parameters, including those of the original HEART score that were observed in this study, were similar to those noted in previous HEART score-based studies, except for obesity and a family history of CAD [10,13,14,16-18,22-24]. However, compared to these previous studies, we reported significantly lower rates of obesity in both the derivation and validation groups (3.0% and 2.7%, respectively). In 2017, the Organization for Economic Cooperation and Development (OECD) stated that 19.5% of the adult population in OECD countries were obese. The prevalence rate of obesity in the adult population of South Korea is 5.3%, which is the third lowest among the OECD countries, after those found in Japan and India, with the adult population in the United States showing the highest obesity rate at 38.2% [25]. Since the present study was performed in South Korea, this may explain the lower rates of obesity. Indeed, the HEART score validation studies conducted in China and Hong Kong, which showed obesity rates in the adult population similar to South Korea, have been found to demonstrate similar results. Additionally, similar results were observed in those with a family history of CAD. These differences indicate a potentially unique risk factor profile in South Korean patients, or Asian populations in general, with regards to MACE incidence. Overall, MACE occurred within 3 months in 12.9% and 12.8% of patients in the derivation and validation groups, respectively, which was consistent with previous findings [7,11].
HEART and mHEART scores
The HEART score is used to ensure the safe disposition of low-risk chest pain patients presenting to the ED and has gained popularity among ED physicians worldwide. With the accumulation of external validation data, there is now ample evidence to support the score’s value in guiding the patient’s safe disposition. However, although some studies have investigated the relationship between each element of the HEART score and short-term MACE incidence, none have demonstrated the exact extent of their impact or identified whether all the elements have the same influence on MACE incidence [10,11,18,24]. Initially, Six et al. [10] while proposing the HEART score, suggested that all components had the same score ranges (0–2 points), and assumed that each component would have the same impact on the MACE prevalence. It is notable that this was based on experts’ clinical experiences, but not on traditional multivariate regression analysis, which is commonly used in clinical score derivation [21,26]. In this context, we evaluated whether all components of the HEART score had the same effect on MACE incidence.
In this study, the general characteristics of the components of the HEART score were in line with those observed in previous related studies. Particularly, we reported the highest P-value for the age component than for the other components showing correlations with MACE incidence in the univariate analysis. A retrospective validation study in China reported similar results [24]. All other HEART score components showed P-values lower than 0.001, whereas the age component had a P-value of 0.025, which was the highest. Additionally, a similar finding was shown in the original HEART score study [10]. The age component had the highest P-value among all components, with a P-value of 0.2847. This finding suggests that the age component may have weaker statistical significance in the prediction of short-term MACE incidence than the other HEART score components. Finally, in the subsequent multivariate analysis, age did not show statistical significance. Thus, we calculated the mHEART score without the age component and evaluated whether this score was more appropriate for use in the prediction of short-term MACE risk than the original HEART score.
HEART, mHEART, and TIMI scores for the detection of MACE within 3 months
Several studies have reported the AUROC for the HEART score in the prediction of short-term MACE incidence. Streitz et al. [18] reported an AUROC of 0.885 (95% CI, 0.838–0.931), whereas in an external validation study conducted in the United States, the value was 0.898 (95% CI, 0.847–0.950) [16]. In our study, the AUROC for the HEART score was 0.880 (95% CI, 0.850–0.910; P<0.001) in the derivation group and 0.880 (95% CI, 0.820–0.300; P<0.001) in the validation group, similar to the values observed in other studies. However, the AUROCs for the mHEART score were 0.910 (95% CI, 0.860–0.940; P<0.001) and 0.910 (95% CI, 0.860–0.950; P<0.001) in the derivation and validation groups, respectively, which are higher than those for the HEART score, as observed in our study as well as other HEART-related studies. In the comparison of observed sensitivity, specificity, PPV, and NPV for MACE occurrence with those observed in a HEART score validation study in an Asian-Pacific population performed in the Netherlands [11], when the cutoff levels for the HEART and mHEART scores were set at ≤3 and ≤2, respectively, all four values were higher for the mHEART score than for the HEART score.
Multiple ED-based external validation studies have concluded that the accuracy of the HEART score is superior to that of the TIMI score in the prediction of short-term MACE incidence [7,10,13,18,27]. In the aforementioned study conducted in the Netherlands [11], when the cutoff level was set at ≤3 for the HEART score and 0 for the TIMI score, the PPV rates for short-term MACE incidence for the HEART score and TIMI score were 17.3% and 15.4%, respectively, and the corresponding NPV rates were 98.3% and 98.7%, respectively; the corresponding specificity values for MACE incidence were 31.8% and 20.3%, respectively. These results suggest that the HEART score allows for the identification of high-risk patients with greater accuracy than the TIMI score, while also allowing for the safe disposition of low-risk patients. Additionally, our study showed that the mHEART score had the highest PPV and NPV for short-term MACE incidence for the three scoring tools mentioned, with a PPV rate of 23.7% and NPV rate of 99.3%. The fact that the mHEART score showed the highest PPV when we calculated the PPV with the NPV fixed at a rate of 99% or higher indicates that the mHEART score performed better in terms of detecting high-risk patients for short-term MACE incidence and in ensuring the safe disposition of low-risk patients than both the HEART and TIMI scores.
With the increase in the number of external validation studies related to the HEART score, sufficient data have already been collected to support its diagnostic value in the identification of lowrisk patients. Recent external validation studies on the HEART score are now predominantly focused on the validation of its prognostic value in distinguishing high-risk patients for short-term MACE incidence rather than the safe disposition of low-risk patients, which has already been heavily researched. Given these current trends, our results may be meaningful as they support the use of the mHEART score as a prognostic tool for groups with a high risk of MACE and also as the basis for the disposition of low-risk patients. Moreover, our findings suggest that each component of the HEART score may have a different degree of influence on short-term MACE incidence. If this degree is reflected in the HEART score, the prognostic accuracy of the score would be maximized. Future studies with larger sample sizes might yield more knowledge on this subject.
Our study has several limitations. First, its single-center, retrospective study design may be associated with a selection bias. Second, the sample size was not calculated, and only patients who visited the ED over a period of 1 year were included. Thus, the generalizability of the results may be limited. Third, patients were included in the study only if they had chest pain, chest tightness, chest pressure, or chest discomfort. Patients with atypical symptoms without such chest-related symptoms who may have presented with only nausea, dyspnea, or an impaired level of consciousness were not included. This may have influenced the outcomes, as acute coronary syndrome patients with initial atypical symptoms are known to have worse prognoses than those with chest symptoms [28,29]. Our findings should be validated in a general population including various ethnicities. A more comprehensive and well-designed multicenter prospective validation study is needed to address the aforementioned issues. Lastly, as a biomarker of myocardial cell necrosis, troponin-I is very sensitive, but lacks the specificity for cardiac ischemia associated with acute coronary syndrome. Troponin-I may also increase due to acute decompensated heart failure, myocarditis, and renal failure. Further studies on the troponin component of the HEART score may be another important area for future investigation.
In summary, our study showed that the mHEART score performed better in the detection of patients at high risk for short-term MACE incidence as well as in ensuring the safe disposition of low-risk patients than the HEART or TIMI score. Our findings suggest that each component of the HEART score may have a different degree of influence on short-term MACE incidence. If this degree is reflected in the HEART score, the prognostic accuracy of the score would be maximized. Further, more comprehensive and well-designed multicenter prospective validation studies are needed in the future.
Notes
No potential conflict of interest relevant to this article was reported.
References
Article information Continued
Notes
Capsule Summary
What is already known
The History, Electrocardiography, Age, Risk Factors, and Troponin (HEART) score is a prognostic and clinical risk stratification tool to assign undifferentiated chest pain patients to low-, intermediate-, and high-risk groups for short-term major adverse cardiac events (MACE). Although some studies have investigated the relationship between each element of the HEART score and short-term MACE incidence, none have demonstrated the extent of the individual impact or identified whether all the elements have the same influence on the incidence of MACE.
What is new in the current study
In this retrospective study that sought to clarify the relative prognostic value of each HEART score component for MACE occurrence within 3 months, we developed and validated the modified HEART score to maximize its ability to predict short-term major adverse cardiac events.