Evaluation of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction model for the patients in the intensive care unit

Introduction. Delirium not only compromises patient care, but is also associated with poorer outcomes: increased duration of mechanical ventilation, higher mortality, and greater long-term cognitive dysfunction. The PRE-DELIRIC model is a tool used to calculate the risk of the development of delirium. The classification of the patients into groups by risk allows efficient initiation of preventive measures. The goal of this study was to validate the PRE-DELIRIC model using the CAM-ICU (The Confusion Assessment Method for the Intensive Care Unit) method for the diagnosis of delirium. Materials and methods. Patients admitted to the University Hospital of Vilnius during February 2015 were enrolled. Every day, data were collected for APACHE-II and PRE-DELIRIC scores. Out of 167 patients, 38 (23%) were included and screened using the CAM-ICU method within 24 hours of admission to the ICU. We defined patients as having delirium when they had at least one positive CAM-ICU screening or haloperidol administration due to sedation. To validate the PRE-DELIRIC model, we calculated the area under receiver operating characteristic curve. Results. The mean age of the patients was 69.2 ± 17.2 years, 19 (50%) were male, APACHE-II mean score 18.0 ± 7.4 points. Delirium was diagnosed in 22 (58%) of 38 patients. Data used for validation of the PRE-DELIRIC model resulted in an area under the curve of 0.713 (p < 0.05, 95% CI 0.539–0.887); sensitivity and specificity for the patients with 20% risk were, accordingly, 77.3% and 50%; 40% risk – 45.5% and 81.3%, 60% – 36.4%, and 87.5%. Conclusions. The PRE-DELIRIC model predicted delirium in the patients within 24 hours of admission to the ICU. Preventive therapy could be efficiently targeted at high-risk patients if both of the methods are to be implemented.


INTRODUCTION
Delirium is defined as an acute, usually reversible, cerebral dysfunction that manifests in a wide range of neuropsychiatric symptoms. The main characteristics are a sudden onset, behavioural changes (from aggression to apathy), and inattention. Even to this day, the aetiology and pathophysiology of this condition remain unclear. However, it is known that delirium is associated with such negative outcomes as prolonged stay in the hospital, decreased physical functionality, and increased health care costs (1,2).
The incidence of delirium in different departments ranges from 19% to 82%, the highest being in the intensive care unit (ICU) (3). The patients who develop delirium in the ICU have a two-tofour-times increased risk of death both in and out of hospital compared to the patients who do not (4,5). As a result, delirium has attracted attention of many researchers worldwide for the past decade. This has led to the development of diagnostic screening tools, prognostic risk models, and effective prevention methods, which are now becoming more and more part of routine in the care of critically ill patients.
Despite the fact that the development of delirium has definite organic causes, there are no laboratory investigations or imaging tests that could confirm this condition. Thus, the diagnosis of delirium is based on clinical criteria, which are often unrecognized and overlooked. In the view of the fluctuating nature of delirium, it has been acknowledged that cognitive functions of the patients have to be monitored in the same manner as other vital functions (heart rate, blood pressure, etc.). For this reason, various screening tools for early detection of delirium in critically ill patients were developed.
Seven tests have been developed to evaluate delirium in the ICU. Only four tests have been validated against DSM-IV criteria (Diagnostic and Statistical Manual of Mental Disorders IV) and only two are useful for intubated patients (6)(7)(8)(9)(10).
The most widely used delirium screening method is CAM-ICU (The Confusion Assessment Method for the Intensive Care Unit), which has been validated in high-quality studies, with sensitivity of 79-100%, specificity of 87-100% (6)(7)(8)(9)(10)(11). Even though the effectiveness of these instruments is clearly proven, the implementation is difficult due to the lack of staff members, a misunderstanding regarding delir-ium risk factors, outcomes, and disbelief that this screening approach may benefit (7,12).
Prevention is an important delirium intervention. According to the Pain, Agitation and Delirium Guidelines (13), non-pharmacological measures such as early mobilization, patient orientation, sleep, and friendly environment are highly recommended for the prevention while the pharmacological approach is still lacking clear evidence. However, the implementation of such interventions requires considerable time and increases the workload significantly. Therefore, various prediction models that could help identify high-risk patients have been developed.
One of the models to predict delirium according to risk factors -the PRE-DELIRIC model -was internationally validated, though its true predictive value can vary from one ICU to another (14). The PRE-DELIRIC model was created in 2013 in the Netherlands based on a recent systemic review of delirium risk factors (15). This model includes ten objectively and clearly defined risk factors known within 24 hours of admission to the ICU: age, Acute Physiology and Chronic Health Evaluation II (APACHE II) score, coma, urgent admission (unplanned ICU admission), admission category (surgical, medical, trauma, neurology/neurosurgical), infection, coma, use of sedatives, morphine use (three dosage groups), serum urea, and metabolic acidosis ( Table 1). Neither alcohol and drug abuse The intercept of the scoring system is expressed as -4.0369; the other numbers represent the shrunken regression coefficients (weight) of each risk factor.
The values were recalibrated during an international study for the PRE-DELIRIC model (14).
Our aim was to test the PRE-DELIRIC model and compare the results with other studies. The main goal of the study was to validate the PRE-DE-LIRIC model for our ICU patients by using novel diagnostic tools. Indirectly, we aimed to spread the up-to-date knowledge about delirium to our staff members.

METHODS
This prospective observational study was performed at the 32-bed mixed medical-surgical ICU at the Centre for Anaesthesiology, Reanimatology, All consecutive 167 patients older than 18 years of age admitted to the ICU from 1 to 28 February 2015, were enrolled to the study. In order to reliably detect delirium, we used a validated delirium assessment tool, the CAM-ICU (Fig. 1). Out of no change in the level of consciousness during the shift, we asked the nurses to describe patient's consciousness during the night and checked the medication list for haloperidol administration due to sedation.
Delirium was defined as at least one positive CAM-ICU screening during a patient's complete stay and/or a patient was treated with haloperidol. Additionally, we distinguished three different types of delirium according to psychomotor symptoms indicated by the Richmond Agitation Sedation Scale (RASS) score: hyperactive (RASS > 0, hyperalert, agitated), hypoactive (RASS ≤ 0, hypoalert, lethargic), and mixed (RASS score varied) (17,18). The ten predictors of the PRE-DE-LIRIC model were collected within the first 24 h of ICU admission. Additionally, we collected data about patient's demographics, outcome, and duration of mechanical ventilation, the length of stay in the ICU and in the hospital. The bioethical agreement was not obtained because the design of the study was observational, without intervention, and had no influence on the treatment decision.

Statistical analysis
Quantitative normally distributed variables were presented as means ± standard deviation (SD) and non-normally distributed variables as median and the interquartile range (IQR). Categorical variables were expressed as actual numbers and percentages. Variables were assessed in the groups of the patients stratified by the delirium status: delirious (n = 22) and non-delirious (n = 16).
Statistical analysis was performed using SPSS 19 (SPSS, Inc., Chicago, IL, USA). Tests were chosen according to the sample size and data distribution. Quantitative and not normally distributed variables were compared using the Mann-Whitney test. For normally distributed variables, we applied the t-test. Categorical variables were compared by using χ 2 or a Fisher exact test. To assess the accuracy of the PRE-DELIRIC prediction model ROC curve was designed. All statistical tests were two-sided. P < 0.05 was considered significant for all tests.
We confirmed delirium in 22 of 38 (58%) of included patients by screening them with the CAM-ICU, eight of them had hyperactive delirium, 12 -hypoactive, two were mixed cases ( Table 2). Haloperidol was administered due to sedation to six out of ten patients with hyperactive or mixed delirium. A median day to the onset of delirium Data are medians with the 25th-75th quartiles (IQR) range and mean with standard deviation or number of patients (%).
The comparison of groups of delirious patients (n = 22) and non-delirious patients (n = 16) showed that delirium was associated with significantly more frequent mechanical ventilation (p = 0.002), longer duration of mechanical ventilation (p = 0.001), a higher RASS score (p = 0.001), and a PRE-DELIRIC score (p = 0.048) ( Table 3). APACHE II scores, days with sedation, duration of the stay in the ICU, and mortality rate were not significantly different between the groups.
The ability of the PRE-DELIRIC model to predict the prognosis correctly was tested by classification matrices ( Table 4). The highest overall correct classification of 0.74 was obtained using a decision criterion of 28.4%, with the sensitivity of 72.7% and the specificity of 75.0%.
The ROC curve for the PRE-DELIRIC model equation applied to our data is shown in Fig. 3. The area under the ROC curve was 0.713 (standard error 0.088, 95% CI 0.539-0.887, p < 0.05) and confirmed the good discrimination of the PRE-DE-LIRIC model.

DISCUSSION
We tested an internationally validated delirium prediction model (PRE-DELIRIC) for our ICU patients. The main goal of the study was to evaluate the PRE-DELIRIC model for delirium prediction after admission to the ICU. The model resulted in an area under the curve of 0.713, indicating a moderate prediction value. In the original study, the results were slightly better -the AUROC was equal to 0.76 (14). The biggest difference lies in the broader range of the confidence interval that can be explained by a smaller sample size in our study.
The original delirium prediction model was developed in 2012. The data for the model were derived from 1613 patients from a single hospital, 25.5% of whom developed delirium (19). Six original research papers, including five prospective cohort studies and one retrospective record analysis, were selected in the systematic review resulting in the identification of 25 risk factors (15). Of these potential risk factors, alcohol misuse, dementia, use of an epidural catheter, hyperam-ylasaemia, hyponatraemia, use of dopamine, and use of lorazepam were excluded because of a prevalence rate below 10%. After multivariate logistic regression analysis with the remaining risk factors, the PRE-DELIRIC model, which consisted of ten risk factors, was constructed. In the derivation cohort, the model had 87% accuracy rate for distinguishing between patients who did and did not develop delirium, whereas the predictions of physicians and nurses attending a subgroup of 124 patients were just 59% accurate. This team validated the model in a further 549 patients from the same hospital, finding that it had an accuracy of 89%.
In 2013, the model was tested in six other countries (Australia, Belgium, Germany, Spain, Sweden, and United Kingdom) (14). This resulted in the recalibration of the model. Despite differences of the predictors between the countries, the discriminative power of the PRE-DELIRIC model was not affected, indicating that the most important predictors for the development of delirium in the ICU are included in the model. In our study, the incidence of delirium was higher in comparison to other similar design studies (58% vs. 12-39%). One of the main reasons might be the fact that there were a lot of neurological and neurosurgical patients (45% vs. 2-20%). Brain injury is especially associated with an increased risk of delirium and this is reflected in the PRE-DE-LIRIC model by high coefficients given for the patients in coma states and neurological and neurosurgical pathology (respectively, 0.25-1.34 and 0.65). Otherwise, the data showed high variability in regard to patient demographics depending on specific case-mix: 42% vs. 12-46% for comatose patients, 32% vs. 6-58% for morphine use, 40% vs. 16-82% for use of sedation, 87% vs. 35-97% for urgent admission and 34% vs. 15-55% for presence of infection (14).
In proportion to risk factors, these differences reflect the need to recalibrate PRE-DELIRIC model as each ICU can have its own specific patient demographics or tendencies in treatment that might influence the predictive power of the model. Even though it was proved to be superior to prediction of the attending caregiver (19), it remains essential to test the model in order to refine the coefficients for each risk factor while also leaving a possibility to add new risk factors.
Additionally, in our study we compared delirious and non-delirious patients. The results were similar to the general trends in scientific literature: delirious patients had higher median RASS scores, spent a significantly longer time on mechanical ventilation and in the ICU (1,2,5,20,21).
High-risk patients can be easily and reliably identified and can benefit from various preventive measures. It is proven that non-pharmacological interventions such as early mobilization, keeping the normal circadian rhythm, cognitive stimulation, reduction of sedatives and narcotic analgesics reduce the incidence of delirium as well as its duration (3,14,(19)(20)(21). It is even suggested that primary prevention of delirium is probably the most effective treatment strategy. Ideally, such measures should be provided for all ICU patients. However, it is not possible as they require a lot of resources and time. Moreover, there might be some benefits of haloperidol prophylaxis for the patients undergoing surgery (23). In order to avoid potential side effects of the drugs, only high-risk patients should be given drug prophylaxis. However, pharmacological preventive strategies still lack solid evidence and they have not been recommended yet (13). Lastly, the ability to stratify risk can help physicians explain risks to patients and their families, and can help the families to better understand the possible outcomes.
The only challenge left is the implementation of the model. Even though the risk factors of the model are easy to note down, it requires a significant effort to collect them. Thus, the workload of nurses increases and this adds a temporary barrier for the use of the model. However, an important part of this workload will be eliminated if the model is implemented in the electronic medical system calculating prediction values automatically for each patient.
Otherwise, we experienced a negative attitude from our staff towards application of the CAM-ICU and the need to use a significant amount of resources for delirium prevention. From the experience in other countries, the negative attitude is a common problem (7,11). However, the situation tends to change significantly after the implementation. With a better understanding and experience, nurses and physicians started to value the tool.
We strongly believe that a better understanding of delirium consequences and new ways to manage it would lead to a positive change in the culture of the ICU. Thus, in order to ease the implementation of new diagnostic and prognostic methods for delirium in the ICU, the first step would be the education of staff members and understanding of their current knowledge and attitude towards this condition.

CONCLUSIONS
The PRE-DELIRIC model can reliably predict a manifestation of delirium within 24 hours of admission to the ICU. However, the true value of the model can only be evaluated after its implementation into clinical practice. Only then we could see if identifying high-risk delirium patients has an effect on the attitude of physicians and nurses towards prevention.