Bayesian analysis of a systematic review of early versus late tracheostomy in ICU patients

Background A recent systematic review and meta-analysis of RCTs of early vs late tracheostomy in mechanically ventilated patients suggest that early tracheostomy reduces the duration of ICU stay and mechanical ventilation, but does not reduce short-term mortality or ventilator-associated pneumonia (VAP). Meta-analysis of randomised trials is typically performed using a frequentist approach, and although reporting confidence intervals, interpretation is usually based on statistical significance. To provide a robust basis for clinical decision-making, we completed the search used from the previous review and analysed the data using Bayesian methods to estimate posterior probabilities of the effect of early tracheostomy on clinical outcomes. Methods The search was completed for RCTS comparing early vs late tracheostomy in the databases PubMed, EMBASE, and Cochrane library in June 2022. Effect estimates and 95% confidence intervals were calculated for the outcomes short-term mortality, VAP, duration of ICU stay, and mechanical ventilation. A Bayesian meta-analysis was performed with uninformative priors. Risk ratios (RRs) and standardised mean differences (SMDs) with 95% credible intervals were reported alongside posterior probabilities for any benefit (RR<1; SMD<0), a small benefit (number needed to treat, 200; SMD<–0.5), or modest benefit (number needed to treat, 100; SMD<–1). Results Nineteen RCTs with 3508 patients were included. Comparing patients with early vs late tracheostomy, the posterior probabilities for any benefit, small benefit, and modest benefit, respectively, were: 99%, 99%, and 99% for short-term mortality; 94%, 78%, and 51% for VAP; 97%, 43%, and 1% for duration of mechanical ventilation; and 97%, 75%, and 27% and for length of ICU stay. Conclusions Bayesian meta-analysis suggests a high probability that early tracheostomy compared with delayed tracheostomy has at least some benefit across all clinical outcomes considered.


Editor's key points
A previous systematic review using frequentist methods concluded that early tracheostomy was more effective in reducing length of ICU stay and duration of mechanical ventilation but not shortterm mortality or ventilator-associated pneumonia. This Bayesian analysis provides evidence that early tracheostomy improves all of the above outcomes. Investigators should provide probabilities of effect sizes rather than simply dichotomise outcomes based on a hypothesis test.
Tracheostomy is a common surgical procedure performed in critically ill patients who require prolonged mechanical ventilation. 1 Tracheostomy is hypothesised to reduce ventilator-associated pneumonia (VAP), duration of mechanical ventilation, length of intensive care unit (ICU) stay, and risk of death, 2 in part through permitting a reduction in sedation, improving patient comfort, and facilitating communication. However, tracheostomy is associated with bleeding, wound infection, tracheal stenosis, accidental displacement, and occasionally death. 3 The timing of tracheostomy in a mechanically ventilated patient is a clinically important question in ICU practice; several studies have attempted to answer this, including several metaanalyses. 2e6 Deng and colleagues 4 completed a recent metaanalysis based on RCTs to investigate whether early tracheostomy compared with late tracheostomy can improve clinical outcomes in critically ill patients undergoing mechanical ventilation. This study concluded that early tracheostomy reduces the length of ICU stay and mechanical ventilation but does not reduce the risk of short-term mortality or VAP. Although the effects sizes for short-term mortality (risk ratio [RR]¼0.87; 95% confidence interval [CI], 0.74e1.03) and VAP (RR¼0.90; 95% CI, 0.78e1.04) suggested a reduction in risk, the results were not statistically significant at the 5% level.
The above analysis used frequentist methods by dichotomising results into just 'null' or positive. 7 However, we think it would be more of use to present clinical decision-makers with the probability of the intervention on the magnitude of a particular outcome. Such an approach requires a Bayesian calculation of 'posterior probabilities'. To obtain a source of data for calculation of these probabilities, we completed a search using the strategy described by Deng and colleagues in their previous systematic review. 7 We then analysed the data using standard frequentist methods followed by a Bayesian meta-analysis. We estimated the probability of any benefit, a small benefit (defined as a number needed to treat [NNT] of 200 for binary outcomes or standardised mean difference [SMD] <e0.5 for continuous outcomes) and a modest benefit (defined as an NNT of 100 or SMD <e1.0).

Search strategy and selection criteria
Our analysis is based on a previous review and frequentist meta-analysis described by Deng and colleagues 4 comparing the risk of clinical outcomes in ICU patients who had 'early' tracheostomy vs 'late' tracheostomy. This review was carried out following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis recommendations. The databases PubMed, EMBASE, and the Cochrane library were searched systematically in June 2022 using the keywords 'tracheotomy' OR 'tracheostomy' AND 'mechanical ventilation' OR 'intracheal intubation' AND 'randomised controlled trials'. The search was independently performed by two reviewers, and discrepancies were resolved by a third observer. The inclusion criterion was RCTs with a population of ICU patients requiring mechanical ventilation, an intervention group with 'early' tracheostomy, a comparison group with 'late' tracheostomy, and one of the following clinical outcomes: short-term mortality, VAP, length of ICU stay, duration of mechanical ventilation, and length of hospital stay. Another inclusion criterion was use of an intention-to-treat analysis. Some studies clearly stated this approach, whereas others did not include a specific statement to this effect. We included these studies in our review unless there was some indication in the numbers that such an approach had not been followed. For example we excluded papers where there was wide disparity in control and intervention participants despite a 1:1 randomisation ratio.

Data extraction for this review
Information on the study setting, population, timing of early and late tracheostomy, and clinical outcomes was extracted. We excluded one of the outcomes in the review of Deng and colleagues, length of hospital stay, on the ground that only six studies in the review included this outcome. For the binary outcomes (short-term mortality and VAP), we extracted the number of times a clinical outcome occurred and the number of patients in the early and late tracheostomy groups. For the continuous outcomes (length of ICU stay and duration of mechanical ventilation), we extracted means and standard deviations in both groups or medians and inter-quartile ranges when the mean and standard deviation were not reported.

Statistical methods
A Bayesian random effects meta-analysis was performed to account for between trial variation in treatment effects, and variability within a trial. Heterogeneity in the effect of early tracheostomy on clinical outcomes across the studies was measured using tau and a value of more than one was considered 'fairly extreme'. 8 Posterior probabilities were calculated for the heterogeneity parameters and plots of the distribution of the probabilities were created.

Binary outcomes
For binary outcomes, the number of times a clinical outcome occurred and the number of patients in the early and late tracheostomy groups were used to calculate the RRs and standard errors. For the analysis of the binary outcomes, uninformative priors were used to minimise assumptions about the model, including N (0, 1000) prior for the effect parameter and a uniform prior for the heterogeneity parameter, and the data were pooled on the log-scale to estimate for the distribution of the log RRs. The pooled log RRs were transformed through exponentiation to calculate RRs. RRs with their 95% credible intervals (95% CrL) were reported, and plots of the posterior distribution of the log RRs were created. In addition, we report posterior probabilities for any benefit (RR<1); a small beneficial effect defined as an absolute risk difference of at least 0.5% (equivalent to a number needed to treat [NNT] 200); and a modest beneficial effect defined as a risk difference of at least 1% (NNT 100). The risk differences of 0.5% and 1% were chosen as they was used in a previous Bayesian re-analysis, 9 and it seemed plausible that patients and their families would be willing to offset the disadvantages of tracheostomy against potential reductions in mortality. To estimate the posterior probability of these risk differences, we extracted the risk of the outcome under the control arm from the largest studies with available data in the previous systematic review; risk of short-term mortality was 31.5% in the study of Young and colleagues, 10 and risk of VAP was 9% in the study of Diaz-Prieto and colleagues. 11 We then calculated the equivalent log RR for each of the two putative absolute risk differences postulated above (0.5% and 1%). From the pooled distribution of the log relative risk, we then calculated the posterior probability of these two log RRs.

Continuous outcomes
For continuous outcomes, means, and standard deviations were used to calculate SMDs. When standard deviations were not reported, standard errors or inter-quartile ranges were extracted and converted to standard deviations. The SMDs were then calculated using Cohen's d. For the analysis of the continuous outcomes (duration of mechanical ventilation and ICU stay), uninformative priors were again used, including N (0, 1000) prior for the effect parameter and a uniform prior for the heterogeneity parameter which provided a pooled estimate for the distribution of the SMDs. The SMDs and 95% CrLs were reported. Plots of the posterior distribution of the SMDs were created. In addition, we report posterior probabilities for any benefit (defined as a SMD <0), a small beneficial effect (which we have defined as a SMD <e0.5), and a modest beneficial effect (which we have defined as a SMD <e1.0). The small and modest benefits defined here are different to the standard definition of small, medium, and large SMDs used by Cohen. 12 To estimate the posterior probabilities on a clinically meaningful scale, we extracted the standard deviation of the outcome for the control group from the largest study with available data in the previous review by Young and colleagues. 10 We then multiplied the SMDs by the standard deviation of the outcome in the control group standard deviation to get an estimate of the mean difference (MD) in days.

Frequentist method for comparison
A frequentist meta-analysis was performed for all of the clinical outcomes. RRs for the binary outcomes and SMDs for the continuous outcomes were pooled using a random-effects model. Estimates with 95% confidence intervals were reported.

Results
In total, 19 RCTs encompassing 3508 patients were included in the systematic review ( Fig. 1). Baseline characteristics of the included studies are reported in Table 1. A summary of the estimated effect sizes, heterogeneity parameters, and posterior probabilities for beneficial effects and extreme heterogeneity are reported for each outcome in Table 2. For each clinical outcome, the plots of the posterior distribution for effect parameters ( Fig. 2) and for heterogeneity parameters were reported (see Supplementary material Figs. 1e4).

Short-term mortality
The pooled RR for short-term mortality was 0.82 (95% CrL, 0.70 to 0.96) for patients with early tracheostomy compared with late tracheostomy (Fig. 3). The posterior probability of any beneficial effect (RR<1) was 99%, a small beneficial effect (NNT200) was 99%, and a modest beneficial effect (NNT100) was 99%. The posterior probability for extreme heterogeneity (tau>1) in the effect of early tracheostomy on short-term mortality across the studies was 0%.

Duration of mechanical ventilation
The pooled SMD for the duration of mechanical ventilation was e0.46 (95% CrL, e0.93 to e0.01; MD, e7 days) for patients with early tracheostomy compared with late tracheostomy (Fig. 5). The posterior probability of any beneficial effect (SMD<0 or a mean difference <0 day) was 97%, a small beneficial effect (SMD<e0.5 or a MD<e7 days) was 43%, and a

Causal pathway
In Figure 2, we represent a plausible causal pathway whereby early tracheostomy might improve patient outcomes relative to delayed tracheostomy. According to this model, the relationship between early tracheostomy and short-term mortality is mediated by a reduction in VAP. Likewise, a reduction in sedation used to facilitate orotracheal tube tolerance might improve mucociliary function, communication and early Casual pathway of clinical outcomes comparing early tracheostomy to late tracheostomy, including posterior probabilities plots of effect sizes. Posterior probability of log relative risks for ventilator-associated pneumonia and short-term mortality and standardised mean differences for duration of mechanical ventilation and length of ICU stay. Vertical line represents no difference with an effect size of zero. ⁺Other mechanisms include reduction in sedation dose, earlier mobilisation and possibly improved mucociliary action which generally have not been reported in the studies included in our review. eve represents reduced probability of outcome. eve* represents reduced probability of outcome owing to competing risks. mobilisation resulting in a reduction in short-term mortality, duration of mechanical ventilation, and length of ICU stay. 30e32 In Figure 2, we have explored this possible casual pathway, which we have populated with credible limits calculated from our Bayesian re-analysis of the data in this review. The pattern in the data provides support for the theory that early tracheostomy leads to beneficial outcomes and that reduction in pneumonia is an important mediating variable.  This does not mean that this is the only pathway, but we cannot test for the additional putative pathway relating to communication, early mobilisation, and mucociliary function because the salient data were not collected in the studies on which we rely, perhaps because the authors think that pneumonia is the most important variable linking intervention to outcome.

Discussion
This Bayesian meta-analysis assessed whether early tracheostomy compared with late tracheostomy can improve clinical outcomes in critically ill patients undergoing mechanical ventilation. The results suggest that early tracheostomy has a beneficial effect on clinical outcomes. The pattern in the data is consistent with a plausible theoretical model or causal pathway.

Comparison with previous results: binary outcomes
A meta-analysis by Deng and colleagues 4 examined whether early tracheostomy compared with late tracheostomy can improve clinical outcomes using a frequentist approach. The review concluded that 'early tracheotomy can reduce the length of ICU stay and mechanical ventilation duration, but the timing of the tracheotomy was not associated with the -2.5 -2 -1. short-term clinical endpoints'. Based on a frequentist approach, the results from this study with the updated search suggested a reduction of risk (18% for short-term mortality and 10% for VAP). The results for short-term mortality were statistically significant and those for VAP were not statistically significant. The effect estimates were very similar from the Bayesian approach but the interpretation is different. For short-term mortality, the Bayesian approach produced posterior probabilities of more than 99% for any benefit, a small benefit (NNT<200), and a modest benefit (NNT<100). This means there is a 99% probability that for every 100 patients given an early tracheostomy, there was one additional survivor compared with late tracheostomy. For VAP, which had a non-statistically significant results from the frequentist approach, the posterior probability for any benefit was high at 94% but dropped to 78% for a small benefit and 51% for a modest benefit.

Comparison with previous results: continuous outcomes
For the duration of mechanical ventilation and the length of ICU stay, the results from this study include new studies found in the search and some other changes the previous review. First, for a few studies the standard errors were not converted to standard deviations before consolidating all the individual study findings. Second, one of the studies 11 in the previous review did not include a measure of variability for the outcome length of ICU stay and therefore we had to exclude it.
From a frequentist approach, following these amendments the results still showed a statistically significant reduction in the duration of mechanical ventilation and length of ICU stay. However, the reduction was much less than compared with the previous review. Again, the effect estimates were very similar to the Bayesian approach but again the interpretation is different.
For duration of mechanical ventilation and length of ICU stay, the Bayesian results are rather less impressive than were those of the frequentist analysis with lower probabilities for small and modest benefits. The posterior probabilities for extreme heterogeneity in the effect of early tracheostomy across the studies were very high for length of ICU stay which indicates the pooled estimate should be interpreted with caution.

Implications
Scientifically, Bayesian analysis leaves considerable room for doubt; the question is not resolved and there is a case for further trials. The clinical issue is rather different as decisions have to be made on the best current evidence. As the point estimate is positive and 'no evidence of effect is not evidence of no effect', the hypothesis test is silent on what action a patient/clinician should take. This is where the Bayesian analysis proves its worth. In discussing care with the intensive care team, the clinician can now say something like; 'there is no proof which is better, but the evidence available suggests that there is a probability of 99% that the early tracheostomy policy will reduce the risk of death in the hospital by at least one percentage point'. When speaking with the patient or family, and translating to natural frequencies, this could be described as follows: 'There is a high chance that among 100 people in your situation, performing an early tracheostomy would mean at least one extra person would survive'.
The Bayesian analysis may also help in the design or decision to fund a further trial. The probability that the casual effect is 'large' with a percentage point effect of, at least one percentage point, is 99%. A frequentist trial to exclude such a difference, from a baseline of 31.5%, would require 89,898 patients across an early and late group to have 90% power at the 5% one-tailed false positive level.
Bayesian probabilities invite a consideration of preferences as they provide evidence of effects of various sizes. We think a one percentage point improvement in survival is worth known downsides of modern tracheostomy. However, it would be interesting to measure the trade-off functions for clinicians and, more importantly, their patients/family members. If most people think that a one percentage point improvement is insufficient to justify tracheostomy, then a future trial can select a larger effect size. For example, the probabilities that effect sizes exceed two and five percentage points are 96% and 62%, respectively.

Casual pathway
The casual pathway of early tracheostomy and clinical outcomes presented in Figure 2 shows the possible mediator of VAP or other mechanisms such as reduction in sedation dose, earlier mobilisation and possibly improved mucociliary action for short-term mortality and the competing risks between short-term mortality, length of ICU stay, and duration of mechanical ventilation. Future RCTs could use this casual pathway as a basis for mediation analysis by means of structural equation modelling. Such an analysis would provide evidence for or against alternative routes from tracheostomy to outcome.

Strengths and limitations
This study has many strengths and limitations. A strength of this study is the methods used. Bayesian meta-analysis allows for a study conclusion to be based on posterior probabilities and effect estimates which are interpreted more intuitively 33 compared with effect estimates, CIs, and significance tests in a frequentist approach, which are often misunderstood. 34,35 For Bayesian analysis, a choice of prior distributions for effect and heterogeneity parameters are required. Future research could use informative priors to calculate posterior probabilities as in our previous work. 36 Our study, by repeating Deng's search identified and corrected two issues in the previous review.
The limitations of this study include differences in tracheostomy methods and definitions of early and late tracheostomy across studies. Only summary data could be extracted, so the definitions of early and late tracheostomy could not be altered and no subgroup analysis could be carried out, for example comparing before or after 9 days on a ventilator. The level of heterogeneity was very high for length of ICU stay and the results for this clinical outcome should therefore be interpreted with caution.

Conclusions
In conclusion, the previous frequentist meta-analysis concluded that the risk of short-term mortality and VAP were not reduced for early tracheostomy compared with late tracheostomy. The frequentist results using the updated research showed a statistically significant result for short-term mortality and not for VAP. Using a Bayesian approach in this meta-analysis, we can see there is evidence that the risk of all adverse clinical outcomes is reduced for early vs late tracheostomy with posterior probabilities of more than 90% for any benefit and more than 75% for a small benefit apart from duration of mechanical ventilation, whereas probabilities of a modest benefit varied across outcomes from 1% to 99%.