The number of deaths due to acute poisoning (AP) is on the increase. It is crucial to predict AP patient mortality to identify those requiring intensive care for providing appropriate patient care as well as preserving medical resources. The aim of this study is to predict the risk of in-hospital mortality associated with AP using an artificial neural network (ANN) model.
In this multicenter retrospective study, ANN and logistic regression models were constructed using the clinical and laboratory data of 1,304 patients seeking emergency treatment for AP. The ANN model was first trained on 912/1,304 (70%) randomly selected patients and then tested on the remaining 392/1,304 (30%). Receiver operating characteristic curve analysis was used to evaluate the mortality prediction of the two models.
Age, endotracheal intubation status, and intensive care unit admission were significant predictors of mortality in patients with AP in the multivariate logistic regression model. The ANN model indicated age, Glasgow Coma Scale, intensive care unit admission, and endotracheal intubation status were critical factors among the 12 independent variables related to in-hospital mortality. The area under the receiver operating characteristic curve for mortality prediction was significantly higher in the ANN model compared to the logistic regression model.
This study establishes that the ANN model could be a valuable tool for predicting the risk of death following AP. Thus, it may facilitate effective patient triage and improve the outcomes.
The number of deaths from acute poisoning accounts for a significant proportion of deaths from external causes. The poisoning severity score is a good predictor of mortality. However, the poisoning severity score needs to consider the overall clinical progression, and the clinician needs to collect a large amount of data from the 12 organ systems. Therefore, a new scoring tool that is a simple and efficient prognosis prediction system for high throughput clinical data is required.
A novel artificial neural network model was developed to measure the risk of death in patients presenting with acute poisoning on initial emergency department assessment. Trained artificial neural networks approach the functionality of a cluster in a fundamental manner, which can inform triage in emergency departments more effectively than logistic regression to provide an additional tool in efficient resource and human resource management.
The number of deaths due to acute poisoning (AP) because of drug or substance ingestion tripled from 6,100 in 1999 to 36,500 in 2008 in the United States of America, exceeding the deaths due to traffic accidents [
In general, patients with AP have a mortality of 1% to 3%, which is lower compared to those of other diseases needing intensive care [
A logistic regression (LR) model is the most typical means of predicting the risk of binary outcomes occurring due to risk factor exposure; however, this model has poor predictability for data based on small sample sizes [
In comparison, artificial neural networks (ANNs), which attempt to emulate neuronal networks in the human brain, have been created to address classification and prediction problems, and have particular relevance and application in medical science [
Data to support the replacement of standard statistical approaches by ANNs as the method of choice for the classification of medical data are insufficient [
This study was designed as a multicenter, retrospective, and medical record review of patients with AP. Patients older than 15 years, who visited the emergency center of two tertiary care university hospitals in South Korea (Incheon St. Mary’s Hospital and Yeouido St. Mary’s Hospital) within 24 hours of drug or toxic substance ingestion, between January 2010 and December 2016, were enrolled in the study. Patients with poisoning associated with a suicide attempt or accidental exposure were also included. The study protocol was approved by the institutional review board of the Catholic Medical Center of Korea (DC20ZIS10034) and written informed consent was waived as all the data were retrospectively collected through retrospective chart review. The initial AP severity was classified based on the worst poisoning severity score (PSS) during the first 24-hour period in the ED [
Patient monitoring and laboratory measurements were performed as part of routine patient management in the ED, and immediate appropriate medical care was offered for the clinical condition by the emergency medical personnel.
To estimate the toxic effects in patients who had ingested multiple toxins simultaneously, we estimated the sum of risks of the poisonous substances. These substances were classified according to the ICD-10 (International Statistical Classification of Diseases and Related Health Problems) (T36–T50) [
To identify the potential predictors of in-hospital death following AP, univariate LR analyses were performed incorporating the demographic informations, clinical characteristics, and laboratory variables. Multivariate LR analysis was performed for a combination of selected risk factors (screened in univariate analyses) exhibiting P-values less than 0.20. Results of the regression analyses were presented as odds ratios (ORs) with the accompanying 95% confidence intervals (CIs).
All the data were randomly divided into training and test datasets at a ratio of 7:3. The backpropagation (BP) ANN model comprised three layers, input, hidden, and output, containing 14, 30, and 1 neuron, respectively. A total of 12 input variables, including age of the patient, diastolic blood pressure, pulse rate, Glasgow Coma Scale (GCS), PSS, PSS group, admission to the ICU, length of ICU stay, endotracheal intubation status, diabetes mellitus, and gastric lavage were employed in the model. All the hidden layers were fully connected, and the rectified linear unit was used as the activation function for each neuron in this layer (
Continuous data were presented as the median and interquartile range (IQR), whereas dichotomous data were presented as the number and percentage. The significance of differences between the two groups were evaluated through the unpaired Wilcoxon rank-sum test for continuous variables, whereas the chi-squared test or Fisher exact test, as appropriate, was applied for dichotomous variables. LR analysis and the ANN model were developed for predicting the occurrence of mortality in patients with AP. The ANN structure included 30 hidden layers, and each hidden layer had 12 nodes. The receiver operating characteristic (ROC) curves were plotted, and the empirical method of Patel and Goyal [
A total of 1,304 poisoned patients (>15 years) had visited the ED during the previously mentioned study period. Among these, 434 men and 870 women were included in the study.
Univariate LR analysis identified 12 factors relevant for multivariate analysis (
In the BP ANN model, age, GCS score, ICU admission, and endotracheal intubation status were the crucial factors among the 12 independent variables indicated for mortality. Thus, according to the LR as well as ANN models, age, ICU admission, and endotracheal intubation status were the common risk factors.
The area under the ROC curve (AUROC) was obtained applying the two models constructed using the test data set for mortality identification (
The ability to predict the exact risk of mortality at an early stage during the treatment of critical patients in the ED is crucial for patient triage and improving outcomes. It has been demonstrated that the ANN model has good prognosis prediction for numerous disease conditions and superiority over conventional predictive models, even when the same input variables are used for model generation [
A preceding study on critical patients with sepsis treated in the ED indicated that the ANN could predict the possibility of death more precisely than LR [
The ANN algorithm possesses an automatic handling function for missing values and performs feature selection enabling the model to perform comparable functions as the LR model, requiring considerably less effort. Another advantage of the ANN model is that preprocessing is not necessary. The LR model has a relatively simple approach for modeling NNs employing a fully connected layer of the feed-forward network without hidden layers, using a sigmoid activation function. Thus, the ANN can potentially model more complicated nonlinear relationships than the LR [
In comparison, the LR model can clarify the variables that provide the strongest predictability of an outcome based on the magnitude of the coefficients and the associated ORs. Moreover, LR analysis can eliminate independent variables that are not related to a particular outcome of interest through a stepwise variable selection process, whereas the ANN model may contain several unimportant predictor variables that may remain unrecognized, compromising or complicating model application. The existence of a statistical relationship between a predictor variable and an outcome alone in an ANN model does not imply causality. LR models are superior to NNs in identifying possible causal relationships [
Previous studies have shown that among patients with AP, older individuals were more prone to death or prolonged ICU stay, with each 10-year increase in age associated with a 0.36 increase in the OR for death [
Five PSS grades (0, none; 4, fatal poisoning) were created for symptoms or signs of organ failure following AP [
The number of toxic substances did not constitute a significant factor in mortality prediction, even though toxic substances received four points from the PSS as the associated complications are highly related to death. Patients with AP due to multiple agents or mixed drug overdoses are common in the ED. A study from Taiwan showed that 208/1,507 (13.8%) patients were exposed to more than one agent [
This study has several limitations, including small sample size and relatively sporadic outcome events. However, more than 10 outcome events for each independent variable are generally acceptable [
Liisanantti et al. [
A predictive tool is urgently required to estimate the mortality risk for patients with AP in the ED, which can serve as a guideline for medical decisions and patient disposition. In this study, a novel ANN model was developed and validated to determine the mortality risk for patients with AP during initial ED assessment. Trained ANNs approach the functionality of a cluster in fundamental manner, which we expect would perform triage in EDs as effectively as LR and provide an additional tool for efficient resource and human resource management. Nevertheless, further optimization of the model is necessary for more accurate predictions.
No potential conflict of interest relevant to this article was reported.
This work was supported by The Catholic University of Korea Daejeon St. Mary’s Hospital. The Clinical Research Institute Grant was funded by The Catholic University of Korea Daejeon St. Mary’s Hospital (CMCDJ-P-2021-013).
Schematic diagram of the backpropagation artificial neural network architecture used to predict the probability of in-hospital mortality. The input layer contained 12 neurons and the hidden layers were 30 with 12 neurons. The output layer has one node (in-hospital mortality). GCS, Glasgow Coma Scale.
Receiver operating characteristics curves for in-hospital mortality after acute poisoning constructed using (A) logistic regression model and (B) artificial neural network model. The area under receiver operating characteristics curves were 0.74, 0.88 for logistic regression and artificial neural network models. TP, true positive; FP, false positive.
Descriptive statistics by mortality after acute poisoning
Variable | Survival (n = 1,292) | Death (n=12) | P-value |
---|---|---|---|
Age | 44 (29–57) | 72 (57–75) | < 0.001 |
Sex | 0.352 | ||
Female | 864 (66.9) | 6 (50.0) | |
Male | 428 (33.1) | 6 (50.0) | |
No. of poisoning substances | 2.2 (1–3) | 1.8 (1–2) | 0.410 |
Vital signs | |||
Systolic blood pressure | 122 (108–139) | 127 (113–138) | 0.810 |
Diastolic blood pressure | 75 (71–140) | 70 (64–77) | 0.131 |
Pulse rate | 87 (78–104) | 99 (88–115) | 0.099 |
Respiration rate | 19 (18–20) | 20 (18–20) | 0.700 |
Body temperature | 36.5 (36.0–36.5) | 36.4 (36.2–36.8) | 0.691 |
Glasgow Coma Scale | 15 (13–15) | 12 (7–13) | 0.049 |
Interval from the time of ingestion to ED arrival (hr) | 1.7 (1.0–4.0) | 2.2 (1.5–7.5) | 0.462 |
Hospital transportation method | 0.368 | ||
Paramedic ambulance | 927 (71.7) | 11 (91.7) | |
Hospital ambulance | 97 (7.5) | 1 (8.3) | |
Automobile | 267 (20.7) | 0 (0) | |
Ambulatory | 1 (0.1) | 0 (0) | |
Poisoning severity score | < 0.001 | ||
1 | 97 (7.5) | 0 (0) | |
2 | 962 (74.5) | 0 (0) | |
3 | 171 (13.2) | 2 (16.7) | |
4 | 62 (4.8) | 10 (83.3) | |
Poisoning severity score group | < 0.001 | ||
0 | 0 (0) | 0 (0) | |
Low (0–2) | 1,059 (82.0) | 0 (0) | |
High (3,4) | 233 (18.0) | 12 (100.0) | |
ICU admission | |||
Yes | 366 (28.3) | 8 (66.7) | 0.003 |
Length of ICU stay | 0 (0–2) | 1.5 (1–13) | 0.162 |
Sum of risk points | |||
Diabetes | 0.85 (0.78–2.52) | 4.34 (1.19–6.08) | 0.385 |
Yes | 137 (10.6) | 4 (33.3) | 0.042 |
Hypertension | |||
Yes | 232 (18.0) | 4 (33.3) | 0.315 |
Tuberculosis | |||
Yes | 15 (1.2) | 1 (8.3) | 0.352 |
CLD | |||
Yes | 18 (1.4) | 1 (8.3) | 0.439 |
Hepatobiliary disease | |||
Yes | 13 (1.0) | 0 (0.0) | 0.990 |
Cancer | |||
Yes | 19 (1.5) | 1 (8.3) | 0.452 |
Major depression disorder | |||
Yes | 224 (17.3) | 1 (8.3) | 0.671 |
Coronary artery disease | |||
Yes | 14 (1.1) | 0 (0) | 1.000 |
Cerebrovascular disease | |||
Yes | 7 (0.5) | 0 (0) | 1.000 |
Endotracheal intubation status | |||
Yes | 84 (6.5) | 7 (58.3) | < 0.001 |
Gastric lavage | |||
Yes | 433 (33.5) | 7 (58.3) | 0.130 |
Charcoal | |||
Yes | 647 (50.1) | 6 (50.0) | 1.000 |
Values are presented as median (interquartile range) or as number (%).
ED, emergency department; ICU, intensive care unit; CLD, chronic lung disease including asthma and chronic obstructive pulmonary disease.
Results of univariate logistic regression analysis for mortality after acute poisoning
Variable | OR | 95% CI | P-value |
---|---|---|---|
Age | 1.05 | 1.02–1.09 | < 0.001 |
Sex | 2.02 | 0.63–6.49 | 0.221 |
No. of poisoning substances | 1.21 | 0.06–3.21 | 0.406 |
Vital signs | |||
Systolic blood pressure | 1.00 | 0.98–1.02 | 0.812 |
Diastolic blood pressure | 0.98 | 0.95–1.01 | 0.123 |
Pulse rate | 1.01 | 0.99–1.01 | 0.154 |
Respiration rate | 0.95 | 0.80–1.07 | 0.653 |
Body temperature | 0.98 | 0.83–2.15 | 0.902 |
Glasgow Coma Scale | 0.83 | 0.74–0.95 | < 0.001 |
Interval from the time of ingestion to ED arrival | 0.98 | 0.85–1.05 | 0.699 |
Hospital transport method | 0.31 | 0.02–0.96 | 0.154 |
ICU admission | 7.74 | 0.41–43.82 | 0.022 |
Length of ICU stay | 3.24 | 1.11–5.42 | 0.210 |
Sum of risk points | 1.00 | 1.00–1.00 | 0.119 |
Diabetes | 4.22 | 1.11–13.57 | 0.047 |
Hypertension | 2.28 | 0.61–7.32 | 0.187 |
Tuberculosis | 1.25 | 0.32–4.32 | 0.256 |
CLD | 8.94 | 1.07–74.42 | 1.000 |
Cancer | 6.09 | 0.33–33.79 | 0.092 |
Major depressive disorder | 0.43 | 0.02–2.24 | 0.422 |
Coronary artery disease | 0.00 | 0.00–0.00 | 1.000 |
Cerebrovascular disease | 0.00 | 0.00–0.00 | 1.000 |
Endotracheal intubation | 20.13 | 6.3–69.3 | < 0.001 |
Gastric lavage | 2.77 | 0.88–9.42 | 0.082 |
Charcoal | 1.00 | 0.31–3.20 | 0.901 |
OR, odds ratio; CI, confidence interval; ED, emergency department; ICU, intensive care unit; CLD, chronic lung disease including asthma and chronic obstructive pulmonary disease.
Results of multivariate logistic regression for mortality after acute poisoning
Variable | Estimate | Adjusted OR | 95% CI | P-value |
---|---|---|---|---|
Age | 0.45 | 5.81 | 3.21–9.01 | 0.032 |
Endotracheal intubation | 2.35 | 20.13 | 6.25–64.79 | < 0.001 |
ICU admission | 0.35 | 5.06 | 1.51–16.91 | 0.031 |
OR, odds ratio; CI, confidence interval; ICU, intensive care unit.
Performance comparison of LR and ANN models for predicting in-hospital mortality after acute poisoning
AUC | 95% CI | CA | Precision | |
---|---|---|---|---|
LR | 0.74 | 0.59–0.90 | 0.99 | 0.98 |
ANN | 0.88 | 0.77–1.00 | 0.99 | 0.98 |
LR, logistic regression; ANN, artificial neural network; AUC, area under curve; CI, confidence interval; CA, classification accuracy.