Combined anatomic and physiologic scoring systems for predicting in-hospital mortality in ICU patients with severe trauma: A multicenter observational cohort study

Xiao-Yuan Ma; Huai-Jian Jin; Li-Xing Tian; Qian Wang; Jun-Yu Zhu; Zhi-Gao He; Wen Tao; Tao Chen; Bin Wang; Hua-Ping Liang

doi:10.4103/1995-7645.271976

ORIGINAL ARTICLE

Year : 2019 | Volume : 12 | Issue : 14 | Page : 17-24

Combined anatomic and physiologic scoring systems for predicting in-hospital mortality in ICU patients with severe trauma: A multicenter observational cohort study

Xiao-Yuan Ma¹, Huai-Jian Jin², Li-Xing Tian³, Qian Wang⁴, Jun-Yu Zhu³, Zhi-Gao He⁴, Wen Tao⁴, Tao Chen⁵, Bin Wang⁴, Hua-Ping Liang³
¹ Department of Critical Care Medicine, ChongGang General Hospital; State Key Laboratory of Trauma, Burns and Combined Injury, Department of Wound Infection and Drug, Research Institute of Surgery, Daping Hospital, Army Medical University, Chongqing, PR China
² Department of Spine Surgery, Center of Orthopedics, Daping Hospital, Army Medical University, Chongqing, PR China
³ State Key Laboratory of Trauma, Burns and Combined Injury, Department of Wound Infection and Drug, Research Institute of Surgery, Daping Hospital, Army Medical University, Chongqing, PR China
⁴ Department of Critical Care Medicine, ChongGang General Hospital, Chongqing, PR China
⁵ Department of Critical Care Medicine, the Affiliated Hospital of Zunyi Medical College, Zunyi, Guizhou, PR China

Date of Submission	06-Sep-2019
Date of Decision	13-Nov-2019
Date of Acceptance	16-Nov-2019
Date of Web Publication	03-Dec-2019

Correspondence Address:
Dr. Bin Wang
ChongGang General Hospital, Chongqing
PR China
Dr. Hua-Ping Liang
State Key Laboratory of Trauma, Burns and Combined Injury, Daping Hospital, Army Medical University, Chongqing
PR China

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/1995-7645.271976

Abstract

Objective: To evaluate the ability of new injury severity score (NISS), acute physiology and chronic health evaluation II (APACHE II), Glasgow coma scale (GCS), a combination of NISS and GCS, a combination of APACHE II and GCS, a combination of NISS and APACHE II to predict all-cause mortality of patients with severe trauma in mainland China.
Methods: This was a multicenter observational cohort study conducted in the ICU of the Chonggang General Hospital, Daping Hospital of the Army Medical University and Affiliated Hospital of Zunyi Medical College from January 2012 to August 2016. The score of NISS, APACHE II, GCS, a combination of NISS and GCS, a combination of APACHE II and GCS, a combination of NISS and APACHE II were calculated based on data from the first 24 hours of ICU admission. Data were processed with Student’s t-test, chi-square test, and receiver operating characteristic (ROC) curve of six scoring systems. Calibration was assessed with the Hosmer-Lemeshow test. The primary endpoint was death from any cause during ICU stay.
Results: A total of 852 and 238 patients with severe trauma were assigned to the derivation group and validation group, respectively. Area under the ROC curve (AUC) was 0.826 [95% confidence interval (CI)=0.794-0.855)] for NISS, 0.802 (95% CI=0.768-0.832) for APACHE II, 0.808 (95% CI=0.774-0.838) for NGCS, 0.859 (95% CI=0.829 -0.886) for NISS+NGCS, 0.864 (95% CI=0.835-0.890) for APACHE II +NGCS, 0.896 (95% CI=0.869-0.929) for NISS+APACHE II in the derivation cohort. Similarly, the score of NISS+APACHE II was also better than the other five scores in the validation cohort (AUC=0.782; 95% CI=0.725-0.833) and had a good calibration (P=0.41).
Conclusions: Taking into account anatomical and physiological parameters completely, the combination of NISS and APACHE II performs better than NISS, APACHE II , NGCS, NISS+NGCS, APACHE II +NGCS for predicting mortality in ICU severe trauma patients. It is needful to develop models that contain various types of accessible predictors (demographic variables, injury cause/mechanism, physiological and anatomical variables, etc.) as comprehensive as possible.

Keywords: New injury severity score, Acute physiology and chronic health evaluation II, Glasgow coma scale, In-hospital mortality, Predictive value, Severe trauma, ICU

How to cite this article:
Ma XY, Jin HJ, Tian LX, Wang Q, Zhu JY, He ZG, Tao W, Chen T, Wang B, Liang HP. Combined anatomic and physiologic scoring systems for predicting in-hospital mortality in ICU patients with severe trauma: A multicenter observational cohort study. Asian Pac J Trop Med 2019;12, Suppl S2:17-24

How to cite this URL:
Ma XY, Jin HJ, Tian LX, Wang Q, Zhu JY, He ZG, Tao W, Chen T, Wang B, Liang HP. Combined anatomic and physiologic scoring systems for predicting in-hospital mortality in ICU patients with severe trauma: A multicenter observational cohort study. Asian Pac J Trop Med [serial online] 2019 [cited 2023 Apr 2];12, Suppl S2:17-24. Available from: https://www.apjtm.org/text.asp?2019/12/14/17/271976

1. Introduction

According to the WHO report, the global mortality caused by severe trauma (especially traffic injury) remains high^[1]. If the wound infection following injury is not well controlled, it can be further developed into local infection, endogenous infection (such as intestinal infection) or nosocomial infection (such as surgical infection, ventilator-associated pneumonia, etc.), and then evolved to sepsis, multiple organ dysfunction syndrome even death, which seriously threaten the health of patients^[2]. Consequently, in the event that we make an early prediction and diagnosis for trauma patients with complications, their prognosis will be better improved.

The predictive indicators for post-traumatic mortality are mainly composed of epidemiological information of patients (such as gender, age, injury mechanism and severity, etc. )^{[3],[4],[5],[6],[7]}, physiological and biological indicators (such as blood lactate, activated prothrombin time, inflammatory cytokines, vital signs, plasma arginine bioavailability, etc.)^{[8],[9],[10],[11],[12]}. In addition, Injury Severity Score (ISS), New Injury Severity Score (NISS), APACHE system, Revised Trauma Score (RTS), Glasgow Coma Scale (GCS) and Trauma & Injury Severity Score (TRISS) are well-known for predicting mortality in trauma patients. However, some researches on death prediction in trauma patients used physiological and biological indicators alone, or trauma score plus biochemical index^[13],[14], which did not fully consider the bias of univariate prediction and weak prediction ability.

Previously published finding has indicated that the ISS can be replaced by the NISS which takes the three most severe injuries regardless of body region into account^[15], and NISS shows better predictive value than ISS both in adult and pediatric trauma population^[16]. Owing to the extremely poor physiological status of critically III patients in a trauma center, the APACHE scoring system was developed. APACHE II and H are widely used for predicting outcomes of trauma patients^[17]. The GCS following modification is most commonly used to evaluate the severity of traumatic brain injury, in adults as well as in children, which are also used as a predictor of mortality^[18]. A systematic review showed that the basic TRISS model was perceived as outdated and adding more predictors to it did not always prove higher performance in the general trauma population^[19]. Although there are many different trauma scales for predicting patients’ outcomes following trauma, the combined application of the scoring system is extremely limited in mainland China. Therefore, the aim of this multicenter observational cohort study was to evaluate the ability of NISS, APACHE II, GCS, a combination of NISS and GCS, a combination of APACHE II and GCS, a combination of NISS and APACHE II to predict mortality in ICU severe trauma patients.

2. Materials and methods

2.1. Study design and setting

All data in the derivation group and validation group were collected at the ICU of Chonggang General Hospital, Daping Hospital of the Army Medical University and Affiliated Hospital of Zunyi Medical College from January 2012 to August 2016.

2.2. Ethics with study approval

This multicenter observational cohort study was performed after receiving institutional review board approval from the Chonggang General Hospital, Daping Hospital and Affiliated Hospital of Zunyi Medical College. The Human Ethics Committee of the Third Affiliated Hospital of Army Medical University approved the study procedures and consent form (approval number 2014-51).

2.3. Patients

A total of 1 090 patients with severe trauma hospitalized in ICU were included, and all met the following criteria: ≥16 years old, incoming ICU within 24 hours after injury, the length of ICU stay ≥48 hours; ISS≥16 and without coexisting illness. Patients who abandoned treatment or transferred to another hospital were excluded.

2.4. Data collection

The primary endpoint was death from any cause during hospital stay. The clinical data of demographical characteristics, physiological and biological indicators were collected and NISS, APACHE II, NGCS, NISS+NGCS, APACHE II +NGCS, NISS+APACHE II score of patients on the first day of ICU admission were calculated and compared between two groups. NISS is the improvement on the basis of the ISS, which adds to the squares of the three highest scoring Abbreviated Injury Scale injuries no matter the affected body area^[20]. APACHE II is a revised version of APACHE-1, which consists of age scores, acute physiology scores, and chronic health scores cited the weights of 45 acute diseases^[21]. A score of 15 minus the original GCS to get the modified GCS named NGCS.

2.5. Data analysis

Data within two groups were compared by Student’s t test for continuous variables and chi-square test for categorical variables. The area under the receiver operator characteristic (ROC) curves (AUC) of the six scoring systems was compared and calibration was evaluated using the Hosmer-Lemeshow statistic. DeLong-DeLong non-parametric test was used to analyze the predictive ability of these six scoring systems and P<0.05 was considered significant. All analyses were performed with SAS 9.3.

3. Results

A total of 1090 severe trauma patients were enrolled from 1 January 2012 to 15 August 2016. There were 852 patients with severe trauma in the 2012-2014 derivation database and 238 valid cases in the 2015-2016 database, respectively [Figure 1].

Figure 1: Chart flow.

Click here to view

3.1. Results of derivation cohort

A total of 852 participants with severe trauma were studied, including 684 males (80.28%) and 168 (19.72%) females. Mean ages in death group and survival group were (48.32±15.75) years, (45.51±13.68) years, respectively. The demographic information is summarized in [Table 1]. The most common causes of severe trauma were road traffic injuries (50.35%), followed by falling from a high place (31.22%), blunt instrument injuries (10.92%), sharp instrument injuries (4.58%), assault (2.11%) and others (0.83%). In the end, the overall mortality of patients in ICU was 33.75%. The scores of NISS, APACHE II , NGCS, NISS+NGCS, APACHE II +NGCS, and NISS+APACHE II in the death group were significantly higher than survival group (P<0.001).

Table 1: Comparison of demographic information between the death group and the survival group in the derivation cohort (n=852).

Click here to view

[Figure 2]A shows the ROC curves of these six scoring systems for mortality prediction. The area under ROC curves was 0.826 for NISS, 0.802 for APACHE II , 0.808 for NGCS, 0.859 for NISS+NGCS, 0.864 for APACHE II +NGCS, 0.896 for NISS+APACHE II , and the NISS + APACHE II showed the best calibration (χ²=3.06, P=0.47).

Figure 2: ROC curves of six scoring systems in derivation cohort (A) (n=852) and in validation cohort (B) (n=238).

Click here to view

DeLong-DeLong non parametric test results of AUC were as follows: NISS versus APACHE II, P=0.391; NISS versus NGCS, P=0.449; NISS versus APACHE II +NGCS, P=0.117; NISS versus NISS+NGCS, P<0.001; NISS versus NISS+APACHE II, P<0.001; APACHE II versus NGCS, P=0.818; APACHE II versus APACHE II +NGCS, P<0.001; APACHE II versus NISS+NGCS, P=0.030; APACHE II versus NISS+APACHE II, P<0.001; NGCS versus APACHE II +NGCS, P<0.001; NGCS versus NISS+NGCS, P=0.003; NGCS versus NISS+APACHE II, P<0.001; APACHE II +NGCS versus NISS+NGCS, P=0.807; APACHE II +NGCS versus NISS+APACHE II , P=0.045; NISS+NGCS versus NISS+APACHE II, P=0.002.

The best cut-off points for mortality prediction were 40 (sensitivity=62.67%; specificity=86.85%) for NISS, 18 (sensitivity=81.03%; specificity=66.38%) for APACHE II , 6 (sensitivity=75.33%; specificity=75.86%) for NGCS, 43 (sensitivity=73.36%; specificity=85.34%) for NISS+NGCS, 24 (sensitivity=86.00%; specificity=74.47%) for APACHE II +NGCS, 56 (sensitivity=79.45%; specificity=87.93%) for NISS+APACHE II [Table 2].

Table 2: Efficacy of the six scoring systems in the derivation cohort.

Click here to view

3.2. Results of validation cohort

A total of 238 participants with severe trauma were studied, including 186 males (78.15%) and 52 (21.85%) females. The mean ages in death group and survival group were (48.38±14.26) years, (46.36±14.35), years, respectively. The demographic information is summarized in [Table 3]. In the end, the overall mortality of patients in ICU was 19.75%. The scores of NISS, APACHE II , NGCS, NISS+NGCS, APACHE II +NGCS, and NISS+APACHE II in the death group were significantly higher than survival group (P<0.001). [Figure 2]B shows the ROC curves of these six scoring systems for mortality prediction. The area under ROC curves were 0.690 for NISS, 0.719 for APACHE II, 0.739 for NGCS, 0.740 for NISS+NGCS, 0.763 for APACHE II +NGCS, 0.786 for NISS+APACHE II , and the NISS + APACHE II showed the best calibration (χ²=4.34, P=0.41) [Table 4]. DeLong-DeLong non parametric test results of AUC were as follows in [Table 4].

Table 3: Comparison of demographic information between the death group and the survival group in the validation cohort (n=238).

Click here to view

Table 4: Efficacy of the six scoring systems in the validation cohort.

Click here to view

4. Discussion

To a large extent, the prognoses of severe trauma patients in ICU are depended on patients’ characteristics, such as injury severity, and the scoring system is the main method to measure the severity of trauma. Considering the heterogeneity of patients, it is inappropriate to apply only one trauma scoring model for all trauma population. Various scoring systems have been developed in recent decades to predict mortality or survival of trauma patients, and ISS, Abbreviated Injury Scale, NISS, APACHE II and GCS are frequently used in mainland China. According to the results of our study, the combination of NISS and APACHE II performed better than the other five scores (NISS, APACHE II , NGCS, NISS+NGCS, APACHE II +NGCS) during mortality prediction in severe trauma patients.

Among all patients in this study, cases due to road traffic injuries accounted for the highest proportion, followed by high-falling injuries, blunt injuries, sharp injuries and assaults, which were consistent with the distribution of trauma types in other studies^[22],[23]. NISS was proposed by Osler^[20] based on ISS and demonstrated correlation with the length of hospital stay and mortality of severe trauma patients inn ICU^[24]. Besides, several studies found that NISS was superior to ISS for predicting functional recovery and mortality in road traffic injuries and skeletal trauma^[25], so NISS was selected to use in this study. However, Dewar et al.^[26] found that neither NISS nor ISS could predict the occurrence of post-traumatic multiple organ failure. This discordance between the results can be mainly attributed to the use of anatomical score alone.

APACHE II consists of age score, acute physiology score, and chronic health score, which can better reflect the patients’ physiological status. Multiple studies showed that APACHE II had some ability to predict the death of emergency trauma patients and ICU trauma patients^{[27],[28],[29]}. In contrast, our study found that the predictive efficacy of APACHE II was lower with the minimum AUC (0.802) among six scoring systems, implying that the physiological score used alone was inadequate to predict the mortality of trauma patients. GCS is better stool to evaluate the level of consciousness in trauma patients, especially those with brain traumatic injury (TBI). Yousefzadeh-Chabok et al.^[18] declared that GCS might be a better predictor of mortality in children trauma cases compared to ISS (AUC: 0.997 versus 0.929, P<0.05). McNett et al.^[30] found that GCS performed better than FOUR scores when predicted 24 h and 72 h mortality after TBI (24 h FOUR versus GCS: 0.913, 0.935; 72 h FOUR versus GCS: 0.837, 0.884). In this study, we found that NGCS after modification had analogical AUC to APACHE II (0.808 versus 0.802, P>0.05). This is in contrast to Zali et al.^[31]. They compared the ability to predict mortality and functional outcome of GCS and APACHE II in ICU patients with multiple trauma, and found that APACHE II was superior to GCS since it involved the principle physiologic parameters of patients (AUC: 0.892±0.028 versus 0.621±0.029, P<0.05).

On account of the lower accuracy of a single scoring system, some researchers investigated the predictive efficacy among combined scoring systems. Kahloul et al.^[32] compared the predictive performance of two anatomic scales (ISS, NISS) with two physiologic scales (Revised Trauma Scale, Simplified Acute Physiology Scale II ) in 1 136 trauma patients. They found that the combination of NISS with SAPS II, or combination of ISS with SAPS did not improve the prediction performance. However, our study findings showed a better predictive value of the combination of anatomic scale with physiologic one (NISS+APACHE II , AUC=0.896). In addition, our previous study found that the combination of NISS with APACHE II was superior to NISS and APACHE II used alone for multiple organ dysfunction syndrome diagnosis in ICU severe trauma patients^[33]. In fact, scoring systems that incorporate anatomic and physiologic variables are beneficial in predicting the mortality of trauma patients. Combination of APACHE II with NGCS and combination of NISS with NGCS did not show high predictive value, which might be related to the calculation method of GCS itself. The GCS scale does not include pupil and sensory examinations, language assessment of patients with artificial airways, and that was why Majdan et al.^[34] used GCS and pupillary reaction to predict six-month mortality in patients with TBI. It also lacks indicators for assessing the severity of coma, such as brain-stem function and breathing patterns and it fails to fully reflect the patients’ physiology because the subtle neurological system changes could not be found. Notably, the combined application of NISS and APACHE II has comprehensive manifestation in anatomy and physiology aspects so that it could show the severity of the injury.

Our study has limitations that should not be neglected. First, this evaluation included 852 cases who met the inclusion criteria, hence, prospective verification is needed for multi-center of severe trauma patients. Second, although we trained medical technicians before the study, there were still computational errors during calculation. Finally, this study focused on severe trauma patients (ISS≥16) who were in critical condition. In order to obtain comprehensive results, trauma patients who had ISS<16 and admission within 24 hours should be considered in subsequent studies.

Taking into account anatomical and physiological parameters completely, the combination of NISS and APACHE II performed better than NISS, APACHE II , NGCS, NISS+NGCS, APACHE II +NGCS as predicting mortality of severe trauma patients in ICU. These findings provide basis for developing diagnostic models that contain various types of accessible predictors (demographic variables, injury cause/mechanism, physiological and anatomical variables etc.).

Conflicts of interest statement

The authors declare that there are no conflicts of interest.

Sources of funding

This study was supported by the National Nature Science Foundation of China (NSFC, No. 81671906), Military Twelve-Five Key Project of China (BWS11J038), Military Medical Innovation Engineering Fundation of China (18CXZ002).

Acknowledgements

The author thanks ICU of Chonggang General Hospital, Chongqing Daping Hospital and the Affiliated Hospital of Zunyi Medical College for providing valuable information.

Authors’ contributions

H.P.L. had an initial idea. X.Y.M., H.J.J., B.W. designed the study. X.Y.M., H.J.J., L.X.T., Q.W., J.Y.Z. collected the derivation cohort clinical data. Z.G.H., W.T., T.C. collected the validation cohort clinical data. X.Y.M., H.J.J., L.X.T., J.Y.Z. proofread all data. X.Y.M., H.J.J. summarized, extracted and processed all results. X.Y.M., H.J.J., B.W. assessed and statistically analyzed the data. X.Y.M., H.J.J., H.P.L. drafted the manuscript. All author read and revised the manuscript, and approved the final submission. H.P.L. and B.W. take responsibility for completeness and accuracy of the data and analyses.

References

1.	Kilpatrick C, Saito H, Allegranzi B, Pittet D. Preventing sepsis in health care - It’s in your hands: A World Health Organization call to action. J Infect Prev 2018; 19(3): 104-106.
2.	Wafaisade A, Lefering R, Bouillon B, Sakka SG, Thamm OC, Paffrath T, et al. Epidemiology and risk factors of sepsis after multiple trauma: an analysis of 29,829 patients from the Trauma Registry of the German Society for Trauma Surgery. Crit Care Med 2011; 39(4): 621-628.
3.	Ley EJ, Short SS, Liou DZ, Singer MB, Mirocha J, Melo N, et al. Gender impacts mortality after traumatic brain injury in teenagers. J Trauma Acute Care Surg 2013; 75(4): 682-686.
4.	Kesmarky K, Delhumeau C, Zenobi M, Walder B. Comparison of two predictive models for short-term mortality in patients after severe traumatic brain injury. J Neurotrauma 2017; 34(14): 2235-2242.
5.	Yuen MS, Mann SK, Chow DH. A simplified emergency trauma score for predicting mortality in emergency setting. Nurs Crit Care 2016; 21(4): 9-15.
6.	Sardinha DS, de Sousa RM, Nogueira Lde S, Damiani LP. Risk factors for the mortality of trauma victims in the intensive care unit. Intensive Crit Care Nurs 2015; 31(2): 76-82.
7.	Baghi I, Shokrgozar L, Herfatkar MR, Nezhad Ehsan K, Mohtasham Amiri Z. Mechanism of injury, glasgow coma scale, age, and systolic blood pressure: a new trauma scoring system to predict mortality in trauma patients. Trauma Mon 2015; 20(3): e24473.
8.	Umebachi R, Taira T, Wakai S, Aoki H, Otsuka H, Nakagawa Y, et al. Measurement of blood lactate, D-dimer, and activated prothrombin time improves prediction of in-hospital mortality in adults blunt trauma. Am J Emerg Med 2018; 36(3): 370-375.
9.	Dekker SE, de Vries HM, Lubbers WD, van de Ven PM, Toor EJ, Bloemers FW, et al. Lactate clearance metrics are not superior to initial lactate in predicting mortality in trauma. Eur J Trauma Emerg Surg 2017; 43(6): 841-851.
10.	Qiao Z, Wang W, Yin L, Luo P, Greven J, Horst K, et al. Using IL-6 concentrations in the first 24 h following trauma to predict immunological complications and mortality in trauma patients: a meta-analysis. Eur J Trauma Emerg Surg 2018; 44(5): 679-687.
11.	Liu NT, Holcomb JB, Wade CE, Salinas J. Inefficacy of standard vital signs for predicting mortality and the need for prehospital life-saving interventions in blunt trauma patients transported via helicopter: A repeated call for new measures. J Trauma Acute Care Surg 2017; 83(1 Suppl 1): S98-S103.
12.	Costa BP, Martins P, Verissimo C, Simoes M, Tome M, Grazina M, et al. Argininemia and plasma arginine bioavailability - predictive factors of mortality in the severe trauma patients? Nutr Metab (Lond) 2016; 13(1): 60.
13.	Timmermans K, Vaneker M, Scheffer GJ, Maassen P, Janssen S, Kox M, et al. Soluble urokinase-type plasminogen activator levels are related to plasma cytokine levels but have low predictive value for mortality in trauma patients. J Crit Care 2015; 30(3): 476-480.
14.	Kim SC, Kim DH, Kim TY, Kang C, Lee SH, Jeong JH, et al. The Revised Trauma Score plus serum albumin level improves the prediction of mortality in trauma patients. Am J Emerg Med 2017; 35(12): 1882-1886.
15.	Jamulitrat S, Sangkerd P, Thongpiyapoom S, Na Narong M. A comparison of mortality predictive abilities between NISS and ISS in trauma patients. J Med Assoc Thai 2001; 84(10): 1416-1421.
16.	Sullivan T, Haider A, DiRusso SM, Nealon P, Shaukat A, Slim M. Prediction of mortality in pediatric trauma patients: new injury severity score outperforms injury severity score in the severely injured. J Trauma 2003; 55(6): 1083-1087.
17.	Darbandsar Mazandarani P, Heydari K, Hatamabadi H, Kashani P, Jamali Danesh Y. Acute Physiology and Chronic Health Evaluation (APACHE) II Score compared to Trauma-Injury Severity Score (TRISS) in predicting mortality of trauma patients. Emerg (Tehran) 2016; 4(2): 88-91.
18.	Yousefzadeh Chabok S, Kazemnejad Leili E, Kouchakinejad Eramsadati L, Hosseinpour M, Ranjbar F, Malekpouri R, et al. Comparing Pediatric Trauma, Glasgow Coma Scale and Injury Severity scores for mortality prediction in traumatic children. Ulus Travma Acil Cerrahi Derg 2016; 22(4): 328-332.
19.	de Munter L, Polinder S, Lansink KW, Cnossen MC, Steyerberg EW, de Jongh MA. Mortality prediction models in the general trauma population: A systematic review. Injury 2017; 48(2): 221-229.
20.	Osler T, Baker SP, Long W. A modification of the injury severity score that both improves accuracy and simplifies scoring. J Trauma 1997; 43(6): 922-925.
21.	Knaus WA, Draper EA, Wagner DP, Zimmerman JE. Apache II : A severity of disease classification system. Crit Care Med 1985; 13(10): 818-829.
22.	Lalwani S, Rajkumari N, Bindra A, Mathur P. Profile of fatal patients admitted to a neuro trauma critical care unit. Eur J Trauma Emerg Surg 2015; 41(1): 65-67.
23.	Salentijn EG, Collin JD, Boffano P, Forouzanfar T. A ten year analysis of the traumatic maxillofacial and brain injury patient in Amsterdam: complications and treatment. J Craniomaxillofac Surg 2014; 42(8): 1717-1722.
24.	Staff T, Eken T, Wik L, Roislien J, Sovik S. Physiologic, demographic and mechanistic factors predicting New Injury Severity Score (NISS) in motor vehicle accident victims. Injury 2014; 45(1): 9-15.
25.	Deng Q, Tang B, Xue C, Liu Y, Liu X, Lv Y, et al. Comparison of the ability to predict mortality between the injury severity score and the new injury severity score: A meta-analysis. Int J Environ Res Public Health 2016; 13(8): 825.
26.	Dewar DC, Tarrant SM, King KL, Balogh ZJ. Changes in the epidemiology and prediction of multiple-organ failure after injury. J Trauma Acute Care Surg 2013; 74(3): 774-779.
27.	Polita JR, Gomez J, Friedman G, Ribeiro SP. Comparison of APACHE II and three abbreviated APACHE II scores for predicting outcome among emergency trauma patients. Rev Assoc Med Bras 2014; 60(4): 381-386.
28.	Thanapaisal C, Saksaen P. A comparison of the Acute Physiology and Chronic Health Evaluation (APACHE) II score and the Trauma-Injury Severity Score (TRISS) for outcome assessment in Srinagarind Intensive Care Unit trauma patients. J Med Assoc Thai 2012; 95(Suppl 11): S25-S33.
29.	Olsson T, Lind L. Comparison of the rapid emergency medicine score and APACHE II in nonsurgical emergency department patients. Acad Emerg Med 2003; 10(10): 1040-1048.
30.	McNett M, Amato S, Gianakis A, Grimm D, Philippbar SA, Belle J, et al. The FOUR score and GCS as predictors of outcome after traumatic brain injury. Neurocrit Care 2014; 21(1): 52-57.
31.	Zali AR, Seddighi AS, Seddighi A, Ashrafi F. Comparison of the acute physiology and chronic health evaluation score (APACHE) II with GCS in predicting hospital mortality of neurosurgical intensive care unit patients. Glob J Health Sci 2012; 4(3): 179-184.
32.	Kahloul M, Bouida W, Boubaker H, Toumi S, Grissa MH, Jaafar A, et al. Value of anatomic and physiologic scoring systems in outcome prediction of trauma patients. Eur J Emerg Med 2014; 21(2): 125-129.
33.	Ma XY, Xiao Y, Chen T, Jiang DP, Zhou J, Yan J, et al. Predictive value of combination of anatomical scores with physiologic scores for multiple organ dysfunction syndrome diagnosis in severe trauma patients. Chin J Burns 2016; 32(2): 105-108.
34.	Majdan M, Steyerberg EW, Nieboer D, Mauritz W, Rusnak M, Lingsma HF. Glasgow coma scale motor score and pupillary reaction to predict six-month mortality in patients with traumatic brain injury: comparison of field and admission assessment. J Neurotrauma 2015; 32(2): 101-108.