External validation of the modified HOPPE score to predict low risk pulmonary embolism suitable for early discharge
Article information
Abstract
Objective
Recently, a novel score for risk stratification of patients with pulmonary embolism (PE)—the HOPPE score—was derived. We aimed to externally validate the HOPPE score in emergency department-diagnosed PE, using SpO2 as a surrogate for PaO2—the modified HOPPE score.
Methods
Retrospective observational study of adult patients with an emergency department diagnosis of PE was performed. Data collected included demographics, co-morbidities, clinical features, electrocardiogram and test results, in-hospital mortality and non-fatal major adverse clinical events (MACE; survived cardiac arrest, cardiogenic shock or thrombolysis administration). The primary outcome of interest was clinical performance of the modified HOPPE score for inhospital mortality and the composite outcome of in-hospital death and MACE. A secondary outcome was comparison of predictive performance between the modified HOPPE score and the simplified Pulmonary Embolism Severity Index score.
Results
Two hundred and six patients were studied (median age 61, 55% female). There were no deaths or MACE in patients with a low risk modified HOPPE score of 0 to 6 (0%; 95% confidence interval, 0% to 1.8%). Negative predictive value of a low risk score was 100% (95% confidence interval, 92.2% to 100%) for in-hospital mortality and for the composite of in-hospital mortality or MACE. The modified HOPPE score had similar predictive performance to the simplified Pulmonary Embolism Severity Index score with an area under the curve of 0.88 vs. 0.80 for the composite outcome of in-hospital mortality or MACE (P=0.052). Twenty-eight percent of the patients were classified as low risk and potentially suitable for management as outpatients.
Conclusion
The modified HOPPE score showed good clinical performance. Prospective validation is warranted.
INTRODUCTION
Pulmonary embolism (PE) is an uncommon condition in emergency departments (ED) accounting for approximately 1.2% of ED presentations for shortness of breath [1]. It carries a mortality of approximately 4% and a risk of serious non-fatal major adverse clinical events (MACE) of approximately a further 2.5% [2].
There has been a move towards outpatient management of low risk PE [3]. The keys to safe and effective outpatient treatment are appropriate risk stratification and close follow-up [4]. The Pulmonary Embolism Severity Index (PESI) is the most widely validated risk stratification score—in either its original or simplified version. If the PESI score is ≤65 (classified as very low risk) or 66–85 (low risk) there is a low risk of mortality or morbidity and outpatient management can be considered [5]. A simplified PESI (sPESI) score of 0 has a 1.1% risk of death and a 1.5% risk of recurrent thromboembolism or non-fatal bleeding [6]. The PESI score in both its forms has been criticized for its reliance on co-morbidity data and the difficulty of obtaining this accurately in the ED. The components of the sPESI score are shown in Table 1.
Recently a new score has been derived, the HOPPE score—an acronym derived from its elements heart rate, oxygenation, systolic and diastolic blood pressure and electrocardiogram (ECG) features [2]. Its components are shown in Table 1. In internal and a small external validation, low risk patients (score ≤6) had no 30-day mortality or non-fatal MACE [2].0 External validation to date has been small (total of 82 patients, only 27 low risk). To be clinically acceptable, further external validation is required. In addition, the HOPPE score relies on the measured arterial partial pressure of oxygen (PaO2) obtained by arterial blood gas analysis. This test is painful for patients and may only be performed in patients with other indicators of hypoxia such as low peripheral oxygen saturation on pulse oximetry (SpO2) or respiratory failure. SpO2 has been shown to correlate well with PaO2 [7], so it may be that SpO2 can be a less invasive measure of oxygenation for the purposes of this score.
The primary objective of this study was to validate the HOPPE PE risk score in ED-diagnosed PE, using SpO2 as a surrogate for PaO2 in the score (the modified HOPPE score) for the outcomes of in-hospital mortality or the composite outcome of in-hospital mortality or MACE. The secondary objective was to compare the clinical performance of the modified HOPPE score with the sPESI score.
METHODS
Study design and setting
We performed a retrospective observational study by medical record review. It was undertaken in the EDs of two university-affiliated urban community teaching hospitals with a combined adult ED census of approximately 100,000. Participants were all adult patients with an ED diagnosis of PE (as coded on ED data management system) between 1 January 2014 and 30 June 2017 who were admitted to the hospital for treatment. Exclusion criteria were age <18 years, unconfirmed diagnosis of PE (confirmation by computed tomography pulmonary angiogram or ventilation-perfusion scan was required), missing ECG or medical record, previous inclusion and patients who were interhospital transfers.
Data collection
Collected data included demographics, PE confirmation methods, past medical history of cancer, heart failure or chronic lung disease (defined as a documented history of chronic obstructive pulmonary disease, asthma requiring daily medication, bronchiectasis, pulmonary fibrosis or cystic fibrosis), vital signs (heart rate, blood pressure, respiratory rate, SpO2 on room air), ECG analysis, troponin and blood gas analysis results (if taken), in-hospital mortality, non-fatal MACE (defined as survived cardiac arrest, cardiogenic shock or administration of thrombolysis). With respect to the clinical features, we used the first recorded measurements in the ED. For SpO2 if a room air reading was not available in the ED records we used first one recorded by paramedics.
Data was collected onto a piloted, project-specific data form by trained clinical researchers (ED doctors and specialist ED nurses) with access to a project-specific data dictionary. Data collectors were not blinded to study hypotheses.
Outcomes of interest
Outcomes of interest were 1) clinical performances (sensitivity, specificity, negative predictive value [NPV]) of the modified HOPPE score for in-hospital mortality and the composite outcome of in-hospital mortality or non-fatal MACE and 2) comparison of the predictive performance of the modified HOPPE score to the sPESI score.
Analysis
Categorical variables are reported as number and percentages, with 95% confidence intervals (CIs) as required. Continuous variables are reported as median and interquartile range. Analysis is descriptive for clinical performance. Receiver operating characteristic curve analysis was used to compare the modified HOPPE score and the sPESI score. All analyses were performed using Analyse-It for Excel (https://analyse-it.com; Analyse-it Software, Leeds, UK). The previous study [2] reported NPV for 30-day mortality of 95% to 96%. Based on this, 160 patients were required to give a confidence interval +/- 5% for NPV of the outcome in-hospital mortality or composite mortality or MACE. Inter-rater reliability was assessed for 20 of cases. This was analyzed using kappa analysis and percent agreement.
Ethical approval
This study was approved by the Western Health Low Risk Ethics Panel (QA 2019.085). Patient consent for data collection was not required.
RESULTS
Two hundred and six patients were studied. Sample derivation is shown in Fig. 1. Median age was 61 years and 55% were female. Characteristics of the patients and their modified HOPPE risk score distribution are shown in Table 2. Fifty eight (28.2%) were classified as low risk by the modified HOPPE score. Inpatient mortality was 4/206 (1.9%; 95% CI, 0.8% to 4.9%). A further 15 patients suffered non-fatal MACE. The distribution of non-fatal MACE is shown in Supplementary Table 1.

Sample derivation. PE, pulmonary embolism; ECG, electrocardiogram; RVD, right ventricular dysfunction.
Outcome by modified HOPPE score risk category classification is shown in Table 3. There were no deaths or MACE in patients with a modified HOPPE score of ≤6 (low risk group, 0%; 95% CI, 0% to 6.2%). For in-hospital mortality, the modified HOPPE score had sensitivity of 100% (95% CI, 39.6% to 100%), specificity of 28.7% (95% CI, 22.7% to 35.6%) and NPV of 100% (95% CI, 92.2% to 100%). For the composite outcome of in-hospital mortality or MACE, the modified HOPPE score had sensitivity of 100% (95% CI, 79.1% to 100%), specificity of 31% (95% CI, 24.6% to 38.2%) and NPV of 100% (95% CI, 92.2% to 100%) (Table 4).
The area under the receiver operating characteristic curve of sPESI score for in-hospital mortality was 0.79 (95% CI, 0.61 to 0.96) and for the composite outcome of in-hospital mortality or MACE was 0.88 (95% CI, 0.80 to 0.95). In comparison to the sPESI score, the modified HOPPE score had similar predictive performance with area under the receiver operating characteristic curve of 0.88 vs. 0.80 for the composite outcome of in-hospital mortality or MACE (P=0.052) (Fig. 2).

Comparison of receiver operating characteristic curves for simplified Pulmonary Embolism Severity Index (sPESI) and modified HOPPE scores with respect to the composite outcome of in-hospital mortality or major adverse clinical event.
Inter-rater reliability for data collection is shown in Supplementary Table 2. Agreement was 100% for the items PE confirmed by computed tomography pulmonary angiogram, presence of incomplete right bundle branch block, presence of S1Q3T3 pattern on ECG, in-hospital mortality and occurrence of non-fatal adverse events. Agreement was moderate-to-good for the variables heart rate group, oxygen saturation group and systolic blood pressure group.
DISCUSSION
Outpatient treatment of venous thromboembolism (including, PE and deep venous thrombosis) is becoming more common [4]. Outpatient treatment releases capacity in hospitals, reduces patient exposure to hospital-acquired infections and other complications associated with hospitalization and is associated with high levels of patient satisfaction [8]. The keys to safe and effective outpatient treatment are appropriate risk stratifications and close follow-up arrangements [4].
In this external validation, the modified HOPPE score performed well, with very high NPV for in-hospital mortality and the composite outcomes of in-hospital mortality and non-fatal MACE. It showed similar predictive performance to the sPESI score for the composite outcome of in-hospital mortality and non-fatal MACE. Taken together with previous derivation, internal and external validation, the data suggests that this score may be useful in assessing which patients with PE may be suitable for treatment in the community rather than in hospital.
The modified HOPPE score has the advantage of being calculated using variables readily available at the bedside and it is not subject to missing data or patient recall error regarding past medical history. It is also not reliant on laboratory tests or advanced imaging (such as echocardiography) so may be particularly useful in those EDs where these modalities are not available. That the modified HOPPE score performed well despite a quite different risk score classification distribution in our cohort compared to the derivation and validation studies is promising with respect to generalizability between populations. While there was a similar proportion of low risk patients, the proportion of high-risk patients in our study was substantially lower (17% vs. 34%). Reasons for this are unclear.
The modified HOPPE score classified about half as many patients as low risk compared to that reported for the sPESI score. However it did identify a very low risk patient group. The sPESI classification would see more patients classified as low risk and suitable for outpatient management but at the cost of about 3% risk of death or MACE. What level of risk clinicians are prepared to accept and which score has better utility and acceptability in the ED clinical setting is unknown.
The modified HOPPE and sPESI scores share three key variables—pulse rate, systolic blood pressure and oxygen saturation. All have high face validity and have been shown to be associated with outcome [2,9]. A potential criticism of the HOPPE score is that it does not take into account co-morbidities which may be an important factor in physician decision-making. Incorporation of co-morbidities in the sPESI score has been seen as a weakness of that score. The justification given for omitting co-morbidities from the HOPPE score was difficulty getting accurate data for the co-morbidity components of the score, with missing data potentially resulting in an under-estimation of true risk. Risk stratification tools are useful to inform and support physician decision-making but do not supplant the responsibility of clinicians to make decisions considering all of the available data and the individual patient context.
The proportion of patients with low risk PE treated as outpatients is small and appears to be changing slowly over time despite evidence of safety and feasibility [4,10]. Reasons may include lack of confidence in the evidence, medicolegal concerns and lack of systems to support outpatient care of these patients. Use of further risk stratification with investigation for right ventricular dysfunction and troponin and lactate assays may help overcome the first two of these concerns, as abnormal results on these tests have been shown to be associated with poorer outcomes [11,12].
This study has some limitations that should be considered when interpreting the results. It was conducted at a single health service, so results may not be generalizable. Data were collected retrospectively, so it may be subject to missing data [13]. There may also have been an element of subjectivity in ECG interpretation, especially as data collectors were not blinded to the study objectives. Confidence intervals are wide due to unexpectedly low mortality and MACE. Our study did not assess either utility or usefulness of the score due to study design. The score was not used in practice. Cases of PE may have been missed in the ED. However, this is the ‘real world’ of ED practice.
The clinical implications of our findings are limited as the modified HOPPE score has not been subjected to prospective external validation. Given its good performance in this study, we think further research is justified.
In this study, the modified HOPPE score showed good clinical performance. Given its ease of use in the ED setting and good predictive performance, further research to validate it prospectively with a view to incorporation into treatment pathways for PE is justified.
Notes
No potential conflict of interest relevant to this article was reported.
SUPPLEMENTARY MATERIAL
Supplementary Tables are available from: https://doi.org/10.15441/ceem.19.051.
References
Article information Continued
Notes
Capsule Summary
What is already known
Low risk pulmonary embolism is increasingly treated as outpatients. The Pulmonary Embolism Severity Index is the only validated risk stratification tool for pulmonary embolism. A new score (HOPPE) has been proposed.
What is new in the current study
HOPPE score had high predictive value. It performed similarly to the simplified Pulmonary Embolism Severity Index for the composite outcome (in-hospital mortality, major adverse events). This score is easily applied in the emergency department and less reliant on co-morbidity, which is open to missing data and interpretation.