| Home | E-Submission | Sitemap | Contact Us |  
Clin Exp Emerg Med > Volume 7(2); 2020 > Article
Patel, Klim, Ritchie, Johns, and Kelly: External validation of the modified HOPPE score to predict low risk pulmonary embolism suitable for early discharge

Abstract

Objective

Recently, a novel score for risk stratification of patients with pulmonary embolism (PE)—the HOPPE score—was derived. We aimed to externally validate the HOPPE score in emergency department-diagnosed PE, using SpO2 as a surrogate for PaO2—the modified HOPPE score.

Methods

Retrospective observational study of adult patients with an emergency department diagnosis of PE was performed. Data collected included demographics, co-morbidities, clinical features, electrocardiogram and test results, in-hospital mortality and non-fatal major adverse clinical events (MACE; survived cardiac arrest, cardiogenic shock or thrombolysis administration). The primary outcome of interest was clinical performance of the modified HOPPE score for inhospital mortality and the composite outcome of in-hospital death and MACE. A secondary outcome was comparison of predictive performance between the modified HOPPE score and the simplified Pulmonary Embolism Severity Index score.

Results

Two hundred and six patients were studied (median age 61, 55% female). There were no deaths or MACE in patients with a low risk modified HOPPE score of 0 to 6 (0%; 95% confidence interval, 0% to 1.8%). Negative predictive value of a low risk score was 100% (95% confidence interval, 92.2% to 100%) for in-hospital mortality and for the composite of in-hospital mortality or MACE. The modified HOPPE score had similar predictive performance to the simplified Pulmonary Embolism Severity Index score with an area under the curve of 0.88 vs. 0.80 for the composite outcome of in-hospital mortality or MACE (P=0.052). Twenty-eight percent of the patients were classified as low risk and potentially suitable for management as outpatients.

Conclusion

The modified HOPPE score showed good clinical performance. Prospective validation is warranted.

INTRODUCTION

Pulmonary embolism (PE) is an uncommon condition in emergency departments (ED) accounting for approximately 1.2% of ED presentations for shortness of breath [1]. It carries a mortality of approximately 4% and a risk of serious non-fatal major adverse clinical events (MACE) of approximately a further 2.5% [2].
There has been a move towards outpatient management of low risk PE [3]. The keys to safe and effective outpatient treatment are appropriate risk stratification and close follow-up [4]. The Pulmonary Embolism Severity Index (PESI) is the most widely validated risk stratification score—in either its original or simplified version. If the PESI score is ≤65 (classified as very low risk) or 66–85 (low risk) there is a low risk of mortality or morbidity and outpatient management can be considered [5]. A simplified PESI (sPESI) score of 0 has a 1.1% risk of death and a 1.5% risk of recurrent thromboembolism or non-fatal bleeding [6]. The PESI score in both its forms has been criticized for its reliance on co-morbidity data and the difficulty of obtaining this accurately in the ED. The components of the sPESI score are shown in Table 1.
Recently a new score has been derived, the HOPPE score—an acronym derived from its elements heart rate, oxygenation, systolic and diastolic blood pressure and electrocardiogram (ECG) features [2]. Its components are shown in Table 1. In internal and a small external validation, low risk patients (score ≤6) had no 30-day mortality or non-fatal MACE [2].0 External validation to date has been small (total of 82 patients, only 27 low risk). To be clinically acceptable, further external validation is required. In addition, the HOPPE score relies on the measured arterial partial pressure of oxygen (PaO2) obtained by arterial blood gas analysis. This test is painful for patients and may only be performed in patients with other indicators of hypoxia such as low peripheral oxygen saturation on pulse oximetry (SpO2) or respiratory failure. SpO2 has been shown to correlate well with PaO2 [7], so it may be that SpO2 can be a less invasive measure of oxygenation for the purposes of this score.
The primary objective of this study was to validate the HOPPE PE risk score in ED-diagnosed PE, using SpO2 as a surrogate for PaO2 in the score (the modified HOPPE score) for the outcomes of in-hospital mortality or the composite outcome of in-hospital mortality or MACE. The secondary objective was to compare the clinical performance of the modified HOPPE score with the sPESI score.

METHODS

Study design and setting

We performed a retrospective observational study by medical record review. It was undertaken in the EDs of two university-affiliated urban community teaching hospitals with a combined adult ED census of approximately 100,000. Participants were all adult patients with an ED diagnosis of PE (as coded on ED data management system) between 1 January 2014 and 30 June 2017 who were admitted to the hospital for treatment. Exclusion criteria were age <18 years, unconfirmed diagnosis of PE (confirmation by computed tomography pulmonary angiogram or ventilation-perfusion scan was required), missing ECG or medical record, previous inclusion and patients who were interhospital transfers.

Data collection

Collected data included demographics, PE confirmation methods, past medical history of cancer, heart failure or chronic lung disease (defined as a documented history of chronic obstructive pulmonary disease, asthma requiring daily medication, bronchiectasis, pulmonary fibrosis or cystic fibrosis), vital signs (heart rate, blood pressure, respiratory rate, SpO2 on room air), ECG analysis, troponin and blood gas analysis results (if taken), in-hospital mortality, non-fatal MACE (defined as survived cardiac arrest, cardiogenic shock or administration of thrombolysis). With respect to the clinical features, we used the first recorded measurements in the ED. For SpO2 if a room air reading was not available in the ED records we used first one recorded by paramedics.
Data was collected onto a piloted, project-specific data form by trained clinical researchers (ED doctors and specialist ED nurses) with access to a project-specific data dictionary. Data collectors were not blinded to study hypotheses.

Outcomes of interest

Outcomes of interest were 1) clinical performances (sensitivity, specificity, negative predictive value [NPV]) of the modified HOPPE score for in-hospital mortality and the composite outcome of in-hospital mortality or non-fatal MACE and 2) comparison of the predictive performance of the modified HOPPE score to the sPESI score.

Analysis

Categorical variables are reported as number and percentages, with 95% confidence intervals (CIs) as required. Continuous variables are reported as median and interquartile range. Analysis is descriptive for clinical performance. Receiver operating characteristic curve analysis was used to compare the modified HOPPE score and the sPESI score. All analyses were performed using Analyse-It for Excel (https://analyse-it.com; Analyse-it Software, Leeds, UK). The previous study [2] reported NPV for 30-day mortality of 95% to 96%. Based on this, 160 patients were required to give a confidence interval +/- 5% for NPV of the outcome in-hospital mortality or composite mortality or MACE. Inter-rater reliability was assessed for 20 of cases. This was analyzed using kappa analysis and percent agreement.

Ethical approval

This study was approved by the Western Health Low Risk Ethics Panel (QA 2019.085). Patient consent for data collection was not required.

RESULTS

Two hundred and six patients were studied. Sample derivation is shown in Fig. 1. Median age was 61 years and 55% were female. Characteristics of the patients and their modified HOPPE risk score distribution are shown in Table 2. Fifty eight (28.2%) were classified as low risk by the modified HOPPE score. Inpatient mortality was 4/206 (1.9%; 95% CI, 0.8% to 4.9%). A further 15 patients suffered non-fatal MACE. The distribution of non-fatal MACE is shown in Supplementary Table 1.
Outcome by modified HOPPE score risk category classification is shown in Table 3. There were no deaths or MACE in patients with a modified HOPPE score of ≤6 (low risk group, 0%; 95% CI, 0% to 6.2%). For in-hospital mortality, the modified HOPPE score had sensitivity of 100% (95% CI, 39.6% to 100%), specificity of 28.7% (95% CI, 22.7% to 35.6%) and NPV of 100% (95% CI, 92.2% to 100%). For the composite outcome of in-hospital mortality or MACE, the modified HOPPE score had sensitivity of 100% (95% CI, 79.1% to 100%), specificity of 31% (95% CI, 24.6% to 38.2%) and NPV of 100% (95% CI, 92.2% to 100%) (Table 4).
The area under the receiver operating characteristic curve of sPESI score for in-hospital mortality was 0.79 (95% CI, 0.61 to 0.96) and for the composite outcome of in-hospital mortality or MACE was 0.88 (95% CI, 0.80 to 0.95). In comparison to the sPESI score, the modified HOPPE score had similar predictive performance with area under the receiver operating characteristic curve of 0.88 vs. 0.80 for the composite outcome of in-hospital mortality or MACE (P=0.052) (Fig. 2).
Inter-rater reliability for data collection is shown in Supplementary Table 2. Agreement was 100% for the items PE confirmed by computed tomography pulmonary angiogram, presence of incomplete right bundle branch block, presence of S1Q3T3 pattern on ECG, in-hospital mortality and occurrence of non-fatal adverse events. Agreement was moderate-to-good for the variables heart rate group, oxygen saturation group and systolic blood pressure group.

DISCUSSION

Outpatient treatment of venous thromboembolism (including, PE and deep venous thrombosis) is becoming more common [4]. Outpatient treatment releases capacity in hospitals, reduces patient exposure to hospital-acquired infections and other complications associated with hospitalization and is associated with high levels of patient satisfaction [8]. The keys to safe and effective outpatient treatment are appropriate risk stratifications and close follow-up arrangements [4].
In this external validation, the modified HOPPE score performed well, with very high NPV for in-hospital mortality and the composite outcomes of in-hospital mortality and non-fatal MACE. It showed similar predictive performance to the sPESI score for the composite outcome of in-hospital mortality and non-fatal MACE. Taken together with previous derivation, internal and external validation, the data suggests that this score may be useful in assessing which patients with PE may be suitable for treatment in the community rather than in hospital.
The modified HOPPE score has the advantage of being calculated using variables readily available at the bedside and it is not subject to missing data or patient recall error regarding past medical history. It is also not reliant on laboratory tests or advanced imaging (such as echocardiography) so may be particularly useful in those EDs where these modalities are not available. That the modified HOPPE score performed well despite a quite different risk score classification distribution in our cohort compared to the derivation and validation studies is promising with respect to generalizability between populations. While there was a similar proportion of low risk patients, the proportion of high-risk patients in our study was substantially lower (17% vs. 34%). Reasons for this are unclear.
The modified HOPPE score classified about half as many patients as low risk compared to that reported for the sPESI score. However it did identify a very low risk patient group. The sPESI classification would see more patients classified as low risk and suitable for outpatient management but at the cost of about 3% risk of death or MACE. What level of risk clinicians are prepared to accept and which score has better utility and acceptability in the ED clinical setting is unknown.
The modified HOPPE and sPESI scores share three key variables—pulse rate, systolic blood pressure and oxygen saturation. All have high face validity and have been shown to be associated with outcome [2,9]. A potential criticism of the HOPPE score is that it does not take into account co-morbidities which may be an important factor in physician decision-making. Incorporation of co-morbidities in the sPESI score has been seen as a weakness of that score. The justification given for omitting co-morbidities from the HOPPE score was difficulty getting accurate data for the co-morbidity components of the score, with missing data potentially resulting in an under-estimation of true risk. Risk stratification tools are useful to inform and support physician decision-making but do not supplant the responsibility of clinicians to make decisions considering all of the available data and the individual patient context.
The proportion of patients with low risk PE treated as outpatients is small and appears to be changing slowly over time despite evidence of safety and feasibility [4,10]. Reasons may include lack of confidence in the evidence, medicolegal concerns and lack of systems to support outpatient care of these patients. Use of further risk stratification with investigation for right ventricular dysfunction and troponin and lactate assays may help overcome the first two of these concerns, as abnormal results on these tests have been shown to be associated with poorer outcomes [11,12].
This study has some limitations that should be considered when interpreting the results. It was conducted at a single health service, so results may not be generalizable. Data were collected retrospectively, so it may be subject to missing data [13]. There may also have been an element of subjectivity in ECG interpretation, especially as data collectors were not blinded to the study objectives. Confidence intervals are wide due to unexpectedly low mortality and MACE. Our study did not assess either utility or usefulness of the score due to study design. The score was not used in practice. Cases of PE may have been missed in the ED. However, this is the ‘real world’ of ED practice.
The clinical implications of our findings are limited as the modified HOPPE score has not been subjected to prospective external validation. Given its good performance in this study, we think further research is justified.
In this study, the modified HOPPE score showed good clinical performance. Given its ease of use in the ED setting and good predictive performance, further research to validate it prospectively with a view to incorporation into treatment pathways for PE is justified.

NOTES

No potential conflict of interest relevant to this article was reported.

SUPPLEMENTARY MATERIAL

Supplementary Tables are available from: https://doi.org/10.15441/ceem.19.051.

REFERENCES

1. Kelly AM, Keijzers G, Klim S, et al. An observational study of dyspnea in emergency departments: the Asia, Australia, and New Zealand Dyspnea in Emergency Departments Study (AANZDEM). Acad Emerg Med 2017; 24:328-36.
crossref pmid
2. Subramanian M, Gopalan S, Ramadurai S, et al. Derivation and validation of a novel prediction model to identify lowrisk patients with acute pulmonary embolism. Am J Cardiol 2017; 120:676-81.
crossref pmid
3. Roy PM, Moumneh T, Penaloza A, Sanchez O. Outpatient management of pulmonary embolism. Thromb Res 2017; 155:92-100.
crossref pmid
4. Kabrhel C, Rosovsky R, Baugh C, et al. Multicenter implementation of a novel management protocol increases the outpatient treatment of pulmonary embolism and deep vein thrombosis. Acad Emerg Med 2019; 26:657-69.
crossref pmid
5. Erkens PM, Gandara E, Wells PS, et al. Does the Pulmonary Embolism Severity Index accurately identify low risk patients eligible for outpatient treatment? Thromb Res 2012; 129:710-4.
crossref pmid
6. Jimenez D, Aujesky D, Moores L, et al. Simplification of the pulmonary embolism severity index for prognostication in patients with acute symptomatic pulmonary embolism. Arch Intern Med 2010; 170:1383-9.
crossref pmid
7. Collins JA, Rudenski A, Gibson J, Howard L, O’Driscoll R. Relating oxygen partial pressure, saturation and content: the haemoglobin-oxygen dissociation curve. Breathe (Sheff) 2015; 11:194-201.
crossref pmid pmc
8. Bledsoe JR, Woller SC, Stevens SM, et al. Management of lowrisk pulmonary embolism patients without hospitalization: the low-risk pulmonary embolism prospective management study. Chest 2018; 154:249-56.
crossref pmid
9. Aujesky D, Obrosky DS, Stone RA, et al. Derivation and validation of a prognostic model for pulmonary embolism. Am J Respir Crit Care Med 2005; 172:1041-6.
crossref pmid pmc
10. Vinson DR, Ballard DW, Mark DG, et al. Risk stratifying emergency department patients with acute pulmonary embolism: does the simplified Pulmonary Embolism Severity Index perform as well as the original? Thromb Res 2016; 148:1-8.
crossref pmid
11. Singer AJ, Thode HC Jr, Peacock WF 4th. Admission rates for emergency department patients with venous thromboembolism and estimation of the proportion of low risk pulmonary embolism patients: a US perspective. Clin Exp Emerg Med 2016; 3:126-31.
crossref pmid pmc pdf
12. Barco S, Mahmoudpour SH, Planquette B, Sanchez O, Konstantinides SV, Meyer G. Prognostic value of right ventricular dysfunction or elevated cardiac biomarkers in patients with low-risk pulmonary embolism: a systematic review and metaanalysis. Eur Heart J 2019; 40:902-10.
crossref pmid pdf
13. Gilbert EH, Lowenstein SR, Koziol-McLain J, Barta DC, Steiner J. Chart reviews in emergency medicine research: Where are the methods? Ann Emerg Med 1996; 27:305-8.
crossref pmid

Fig. 1.
Sample derivation. PE, pulmonary embolism; ECG, electrocardiogram; RVD, right ventricular dysfunction.
ceem-19-051f1.jpg
Fig. 2.
Comparison of receiver operating characteristic curves for simplified Pulmonary Embolism Severity Index (sPESI) and modified HOPPE scores with respect to the composite outcome of in-hospital mortality or major adverse clinical event.
ceem-19-051f2.jpg
Table 1.
Calculation of the modified HOPPE and sPESI scores
HOPPE score Variable cut-offs Points allocated
Heart rate (beats per minute) ≤ 80 1
81–100 2
> 100 3
Oxygenation (%, SpO2 on room air) ≥ 95 1
90–94 2
< 90 3
Systolic blood pressure (mmHg) ≥ 120 1
100–119 2
< 100 3
Diastolic blood pressure (mmHg) ≥ 80 1
65–79 2
< 65 3
ECG score groupa) <2 1
2–4 2
>4 3
Minimum score is 5; maximum score is 15
 Low risk ≤6
 Intermediate risk 7–10
 High risk > 10
sPESI score
 Age > 80 years 1
 History of cancer 1
 History of chronic cardiopulmonary disease 1
 Heart rate > 110 beats per minute 1
 Systolic blood pressure < 100 mmHg 1
 Oxygen saturation < 90% 1
Minimum score is 0; maximum score is 6
 Low risk 0
 Non-low risk ≥1

sPESI, simplified Pulmonary Embolism Severity Index; ECG, electrocardiogram.

a) ECG score calculation: 1 point for incomplete right bundle branch block, 2 points for tachycardia, 3 points for complete right bundle branch block, 4 points each for T wave inversion in V1-3 or S1Q3T3 pattern.

Table 2.
Patient demographics and clinical features
Parameter Value
Age (yr) 61 (44–72)
Sex, female 114 (55)
Patient characteristics
 Confirmation method CT pulmonary angiogram 150 (66.4%–78.4%)
Ventilation-perfusion scan 54 (20.7%–32.6%)
Both 2 (0.27%–3.5%)
 Heart rate distribution (beats per minute) ≤ 80 66 (26.1%–38.7%)
81–100 65 (25.6%–38.2%)
> 100 75 (30.1%–43.2%)
 Oxygenation (SpO2 on air) ≥ 95% 120 (51.4%–64.8%)
90–94% 55 (21.1%–33.1%)
< 90% 31 (10.8%–20.6%)
 Systolic blood pressure (mmHg) ≥ 120 156 (69.4%–81.1%)
100–119 41 (15%–25.9%)
< 100 9 (2.3%–8.1%)
 Diastolic blood pressure (mmHg) ≥ 80 98 (40.9%–54.4%)
65–79 70 (27.9%–40.7%)
< 65 38 (13.8%–24.3%)
 Electrocardiogram score <2 124 (53.4%–66.6%)
2–4 44 (16.3%–27.5%)
>4 38 (13.8%–24.3%)
 Risk score distribution—modified HOPPE ≤ 6 (low) 58 (22.5%–34.7%)
7–10 (intermediate) 113 (48%–61.5%)
> 10 (high) 35 (12.5%–22.7%)
 Risk score distribution—sPESI 0 (low) 102 (42.8%–56.3%)
≥ 1 (high) 104 (43.7%–57.3%)
Outcome
 In-hospital mortality 4 (0.8%–4.8%)
 Nonfatal major adverse clinical event 19 (6%–14%)
 Right ventricular dysfunction on investigation 58 (22.5%–34.7%)

Values are presented as median (interquartile range); number (%); or number (%), 95% confidence interval.

CT, computed tomography; sPESI, simplified Pulmonary Embolism Severity Index.

Table 3.
Outcome by modified HOPPE and sPESI score category
In-hospital mortality In-hospital MACE In-hospital mortality or MACE Right ventricular dysfunction
Modified HOPPE score
 ≤ 6 (low) 0/58 (0), 0%–6.2% 0/58 (0), 0%–6.2% 0/58 (0), 0%–6.2% 1/58 (1.7), 0.3%–9.1%
 7–10 (intermediate) 1/113 (0.8), 0.2%–0.5% 4/113 (3.5), 1.4%–8.8% 5/113 (4.4), 1.9%–9.9% 34/113 (30), 22.4%–39.1%
 > 10 (high) 3/35 (8.6), 3%–22.4% 11/35 (31.4), 18.6%–48% 14/35 (40), 25.6%–56.4% 23/35 (35.7), 49.2%–79.2%
sPESI
 0 (low) 1/102 (1), 0.2%–5.4% 2/102 (2), 0.5%–6.9% 3/102 (2.9), 1%–8.3% 12/102 (11.8), 6.9%–19.4%
 ≥ 1 (high) 3/104 (2.9), 1%–8.1% 13/104 (12.5), 7.5%–20.2% 16/104 (15.4), 9.7%–23.5% 46/104 (44.2), 35%–53.8%

Values are presented as number (%), 95% confidence interval.

MACE, major adverse clinical events; sPESI, simplified Pulmonary Embolism Severity Index.

Table 4.
Clinical performance of the low risk modified HOPPE score
Sensitivity (95% CI) Specificity (95% CI) Negative predictive value (95% CI)
In-hospital mortality 100% (39.6%–100%) 28.2% (22.2%–34.9%) 100% (92.2%–100%)
In-hospital mortality or MACE 100% (82.8%–100%) 28.2% (22.2%–34.9%) 100% (92.2%–100%)

CI, confidence interval; MACE, major adverse clinical event.

Editorial Office
The Korean Society of Emergency Medicine
101-3104, Brownstone Seoul, 464 Cheongpa-ro, Jung-gu, Seoul 04510, Korea
TEL: +82-31-709-0918   E-mail: office@ceemjournal.org
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Copyright © by The Korean Society of Emergency Medicine.                 Developed in M2PI