Increased adherence to perioperative safety guidelines associated with improved patient safety outcomes: a stepped-wedge, cluster-randomised multicentre trial

Background: National Dutch guidelines have been introduced to improve suboptimal perioperative care. A multifaceted implementation programme (IMPlementatie Richtlijnen Operatieve VEiligheid [IMPROVE]) has been developed to support hospitals in applying these guidelines. This study evaluated the effectiveness of IMPROVE on guideline adherence and the association between guideline adherence and patient safety. Methods: Nine hospitals participated in this unblinded, superiority, stepped-wedge, cluster RCT in patients with major noncardiac surgery (mortality risk 1%). IMPROVE consisted of educational activities, audit and feedback, reminders, organisational, team-directed, and patient-mediated activities. The primary outcome of the study was guideline adherence measured by nine patient safety indicators on the process (stop moments from the composite STOP bundle, and timely administration of antibiotics) and on the structure of perioperative care. Secondary safety outcomes included in-hospital complications, postoperative wound infections, mortality, length of hospital stay, and unplanned care. Results: Data were analysed for 1934 patients. The IMPROVE programme improved one stop moment: ‘discharge from recovery room’ (þ16%; 95% confidence interval [CI], 9e23%). This stop moment was related to decreased mortality (e3%; 95% CI, e4% to e1%), fewer complications (e8%; 95% CI, e13% to e3%), and fewer unscheduled transfers to the ICU (e6%; 95% CI, e9% to e3%). IMPROVE negatively affected one other stop moment e ‘discharge from the hospital’ e possibly because of the limited resources of hospitals to improve all stop moments together. Conclusions: Mixed implementation effects of IMPROVE were found. We found some positive associations between guideline adherence and patient safety (i.e. mortality, complications, and unscheduled transfers to the ICU) except for the timely administration of antibiotics. Clinical trial registration: NTR3568 (Dutch Trial Registry).


Editor's key points
This study evaluated a multifaceted implementation bundle of perioperative safety 'stop moments' in a randomised design. Most structure indicators had very modest or no improvement over the study period. There were generally positive associations between guideline adherence and patient outcomes.
Surgical care accounts for a large proportion of hospital care and in-hospital adverse events. 1e3 Studies from industrialised countries indicate that permanent disability and mortality rates range between 0.4 and 0.8% of all surgical procedures. 4 Complications are common and occur in 3e16% of such procedures. 5,6 This suggests that a minimum of 1 million patients die after surgery and 7 million patients are injured worldwide by surgical complications annually. 7 Several studies report that up to 50% of surgical adverse events are preventable. 5,8,9 Analyses by the Dutch Health Care Inspectorate in 2007e9 showed that perioperative care in the Netherlands was lacking standards in information transfer, clinical documentation, teamwork, and coordination. 10e12 The WHO designed the Surgical Safety Checklist to decrease adverse events by increasing the use of evidencebased patient safety practices and encouraging communication among operating room team members. 13 Since the introduction of this checklist, more than 4000 hospitals worldwide implemented modified versions of the checklist. 14 In 2009, Haynes and colleagues 4 documented a significant reduction in mortality and other postoperative complications with the use of this surgical safety checklist. In addition, de Vries and colleagues 9 showed a significant reduction in inhospital mortality and overall complications after implementation of a comprehensive surgical checklist, the SURPASS (SURgical PAtient Safety System).
In response to the recommendations of the Dutch Health Care Inspectorate and the two studies mentioned above, national evidence-based perioperative safety guidelines, including patient safety indicators, were developed in the Netherlands (2010e3). 15e17 These guidelines are partly based on the SURPASS checklist and cover the entire surgical pathway from admission to discharge, in line with international patient safety goals of the Joint Commission International. 18 In general, guidelines facilitate the practice of evidence-based medicine, intending to help reduce unnecessary variations in practice, discourage outdated and inefficient practice, and improve the efficiency of healthcare delivery. 19 The patient safety indicators reflect compliance to the guidelines. Hospitals were expected to implement the national perioperative safety guidelines. However, it is not easy to implement new guidelines and sustain change. 20e23 Key to getting guidelines work is a successful implementation strategy. Many hospitals lack the resources or expertise to organise and lead an implementation effort or to manage the changes needed, collect data, and initiate improvement teams. We therefore planned to develop an implementation programme and evaluate its effect and feasibility. It was hypothesised that a multifaceted implementation programme leads to higher guideline adherence than unsupported implementation and that higher adherence leads to a higher patient safety in terms of less mortality, less complications and less unplanned care.

Study design
Between 2012 and 2015, we undertook a cross-section design multicentre, stepped-wedge cluster-randomised trial in a diverse sample of nine Dutch hospitals. A stepped-wedge trial has a crossover design with repeated measurements and random allocation of time points for crossing over to the intervention (Fig. 1). In each hospital, !50 patients were included at each measurement point (four measurements). The multifaceted programme, IMPROVE (Implementation of Perioperative Safety Guidelines; in Dutch: IMPlementatie Richtlijnen Operatieve VEiligheid), was implemented at different time points in each group of three hospitals. In the following text, we will use the term IMPROVE to refer to the IMPROVE implementation programme.
The protocol of the IMPROVE trial has been previously published. 24 There were no changes to the methods after trial commencement. The study was conducted and reported with fidelity to the study protocol 24 and in accordance with the relevant guidelines, regulations and ethical principles for medical research involving human subjects and data (e.g. Declaration of Helsinki). The study was approved by the Medical Ethical Committee of the Radboud University Medical Center following Dutch and European legislation on August 26, 2011 (registration number: 2011/318). Written consent was obtained from all participating hospitals before randomisation. Because outcome data were routinely collected by the Stepped-wedge design with three groups of three hospitals each, and four steps in which 'C' represents the Control situation and 'I' the Intervention phase. T0, baseline measurement (no interventions are implemented yet); T1, measurement at T1 (one group of three hospitals received the intervention); T2, measurement at T2 (six hospitals received interventions); T3, measurement at T3 (all hospitals received the interventions).
hospitals and no personal identifiers are transmitted, individual consent of patients was not required. This study was registered on August 2, 2012 at trialregister.nl, NTR3568. The reporting of this trial followed the Consolidated Standards of Reporting Trials (CONSORT) statement extended to steppedwedge cluster randomised trials. 25,26 Participants IMPROVE was conducted across nine hospitals in the Netherlands. This purposeful sample comprised two academic, four tertiary teaching, and three regional hospitals (

Randomisation and masking
Hospitals were the unit of randomisation and intervention. Randomisation was restricted to hospital type as we wanted groups to represent a balanced mix of hospital types. Randomisation of the order of the switch was performed by an independent statistician (ST). Because of the nature of the intervention and study design, participants and research staff could not be blinded. For logistical reasons, there was no concealment of the randomisation schedule.

Intervention
The intervention was the IMPROVE programme. IMPROVE was conducted over a 2 yr period. IMPROVE was applied at different time points in each group. At baseline, no groups were implementing IMPROVE (control situation). Groups crossed over from the control situation to the intervention situation once every 7 months: the first group started after the baseline measurement, the second group after 7 months, and the third group after 14 months. When a hospital switched to the intervention situation, a period of 4 months was used to implement IMPROVE and the remaining 3 months were allocated for data collection. After 14 months, all groups were in the intervention phase for the remaining 4 months. Groups were therefore in the intervention phase between 4 months (group 3) and 18 months (group 1). The intervention situation persisted for the duration of the trial. The study ended with the completion of the final measurement. IMPROVE consisted of a standard package of five activities and a set of six additional activities that were offered optionally. The intervention components are summarised in Table 1 Context and intervention variables. *Local hospital interventions initiated to implement the perioperative safety guidelines additional to IMPROVE (thus unrelated to IMPROVE), for example: educational meetings, local opinion leaders and reminders. These ranged from 0 to 7. y Average involvement during the intervention phases; the involvement rate per intervention phase could vary between 0 and 11, based on the number of IMPROVE activities with the degree of involvement per activity as described in our IMPROVE manual (see web supplement). IMPROVE, IMPlementatie Richtlijnen Operatieve VEiligheid.

Outcomes
The primary outcome measure was guideline adherence defined by nine patient safety indicators that were part of the national guideline development 17 (Supplementary Digital Content 1): two indicators on the perioperative process (a composite measure of a core set of safety checks at stop moments and timely administration of antibiotic prophylaxis) and seven indicators on the structure of care on hospital level. Pre-specified secondary patient safety outcomes included postoperative wound infections, complications, and mortality during the initial hospital stay, length of hospital stay, unscheduled transfer to the ICU, unplanned reoperation, and non-elective hospital re-admission within 30 days after the initial surgery. The definitions used in this study were adopted

Standard package Small-scale educational meetings
Workshop or skills training for perioperative key disciplines including assignments, role playing, own presentations, patient stories, or discussion and problem solving of hypothetical patient situations/case studies. Provided by an opinion leader within the field of patient safety or a highly respected colleague. Based on active participation in small groups: multi-or monodisciplinary groups (i.e. per discipline, e.g. surgeons, recovery nurses, etc. separately). The content is based on the key constraints and most important obstacles in applying the guidelines for a hospital (based on the results of the audit) and a brainstorming session during the training or pre-handed topics that participants find important to discuss.

Audit and feedback
Feedback is based on the indicator measurement(s), structural observation, barrier analysis and the Team Climate Inventory and Hospital Survey on Patient Safety questionnaires. The feedback consists of a local paper report with the hospital's own results, benchmarked and presented in relation to all nine participating hospitals. The hospitals in the intervention phase receive this report shortly after a measurement period. The feedback report is presented and discussed with the key professionals during a meeting.

Structural observation
Observation by a trained expert of the pre-, per-, and postoperative trajectory of one surgical patient (on the ward, operating room, and recovery ward) based on a structured observation list. The list is divided into six subjects: identification, execution of perioperative stop moments, hygiene and infection prevention, postoperative wound infection-bundle compliance, medication safety, and medical equipment. Feedback is based on the completed observation list. The observation list shows the degree of patient safety as a percentage with an explanation and additional comments and findings. The hospitals receive the feedback immediately afterwards.

Local embedding of the guidelines
Concrete and visible integration into and/or completion of a local protocol, checklist, or both. For example, the adaptation of the guidelines in a local protocol, conducting audits (indicator measurements), structural observations, and visitation to monitor the implementation of the guidelines, the use of reminder systems (completing existing checklists based on the guidelines, if possible, new digital checklist may be installed in electronic patient records), decision support, and feedback on the implementation of the protocol (using information and communications technology), incorporation of the guidelines in the clinical pathway, e.g. resignation letter to the general practitioner.

Patient safety cards
Two patient safety cards (with cartoons and explanations) based on the perioperative guidelines, entitled 'Help us with your safe surgery' and 'Discharge from the hospital', are offered to the patients in order to explicitly invite patients to ask questions.

Additional package Personal information letter in mailbox
Personal information letter to all key disciplines about the (use of the) guidelines. Exchange platform A digital platform for the hospitals to exchange best practices, ideas, and experiences. Scan of the total perioperative process A practice scan consisting of five parts: e Hospital staff complete an online survey about the perioperative process e Interviews with hospital staff (more background information regarding remarkable survey answers) e Structured observation on-site e Paper report by post (contains findings, a top 5 list of strengths and weaknesses and recommendations) e Feedback meeting to discuss the report.

Electronic reminder message
Catchy quotes on behalf of an opinion leader, based on bottlenecks within the perioperative process of a hospital.

Posters
Visual representation of, for example, the perioperative trajectory of the patient with the stop moments, shown as a subway line.

Multi-professional team training
The IMPROVE team facilitates contacts between the participating hospitals and organisations that provide trainings aimed at improving team culture, like crew resource management.
from previous studies 29e33 to enable comparison of the results. There were no changes to the trial outcomes after the trial commenced.
We expected that better guideline adherence would improve patient safety outcomes, as increased surgical checklist use was associated with significant reductions in complications and deaths. 4,9 This is consistent with the model proposed by Donabedian 34 that the presence of good structures and the presence good processes increase the likelihood of achieving favourable outcomes of care.

Data collection
Guideline adherence and patient safety outcomes were examined in 1934 patients and retrieved from administrative data sources: chart review, including (electronic) hospital records, discharge reports, and correspondence between the hospital physician and the general practitioner. All complications that hospitals listed in their official complication registration and in the discharge letter to the general practitioner were recorded. If no data on process indicators were found in the administrative sources, the entry was registered as 'not performed'. Quality control was performed using descriptive and frequency analyses in order to find out-of-range answers and missing data.
The inter-rater agreement 35 among the two researchers and research assistants and the participating hospitals was high, kappa !0.71 in a 20% sample.
The structure indicators were measured by the contact person in each hospital with a questionnaire on a dichotomous scale (yes/no).
Finally, each hospital provided information about their local implementation of IMPROVE, for example information on the actual activity content and the number of participants. This information was quantified into an involvement rating measure combining the number of activities with the degree of involvement per IMPROVE activity as described in our IMPROVE manual for implementation activities. For the five standard activities, hospitals could receive 0e2 points per activity or 3 points in case of the audit and feedback activity. The (overall) involvement rating score ranged from 0 to 11, with 0e3 points indicating unsatisfactory involvement, 4e7 points moderate, and 8e11 points satisfactory involvement.

Power analysis
We made the following assumptions: (1) nine hospitals in the Netherlands cooperate in this study; (2) an average effect of 10e15% increase in the process indicators (primary outcome measure) might be expected from the literature 36 ; (3) baseline adherence to the guidelines is unknown, but a 15% increase from a baseline level of 50% requires the largest sample size and therefore the sample size calculated will also suffice if the baseline adherence is different from 50%; and (4) the intraclass correlation coefficient is unknown but can be estimated between 0.1 and 0.3 as intraclass correlation coefficients have been shown to vary between 0 and 0.4 with a median intraclass correlation coefficient of 0.06 in other cluster randomised trials in secondary care. 37 Also, using process variables results in a comparable range and median of intraclass correlation coefficients. 37 Based on the availability of nine hospitals, a baseline adherence of 50%, an increase of 15% by the intervention, a significance level alpha of 0.05, an intraclass correlation coefficient between 0.1 and 0.3, and four time points (three steps in the stepped design), 50 patients per cluster (equal sized) and time point are needed to reach a power of at least 0.85, using the formula of Hussey and Hughes. 38 As a result, 1800 (50 patients Â 9 hospitals Â 4 measurements) patients will be included over all hospitals in the course of the trial.

Statistical analysis
Basic descriptive statistics (e.g. intraclass correlation coefficient) and the coefficient of variation were calculated. The coefficient of variation was calculated as the ratio of the standard deviation (SD) to the mean. Patient and procedure characteristics, processes, and outcomes of care were described at the patient level. Categorical variables are presented as percentages and continuous variables as mean and SD when normally distributed or median, inter-quartile range (IQR), and full range when not. We used SPSS version 20.0 for data analysis. The primary outcomes (nine patient safety indicators) were corrected according to Bonferroni, that is they were only considered statistically significant if P<0.05/9. For all analyses, two-tailed nominal P values and nominal 95% confidence interval (CI) were reported.

Primary stepped-wedge analysis
To assess the influence of IMPROVE, and adjust for possible influence of covariates on outcomes and for time trends over the four measurements, intention-to-treat, linear mixed model analyses were performed with a random effect for hospitals to account for the clustering of patients within hospitals, and fixed effects for time to account for time trend (categorical, i.e. for each measurement separately, not assuming a linear trend), treatment and possibly covariate(s) (see the section on Confounders). The dependent variable included the primary patient safety indicators and secondary outcome measures. Independent variables were intervention (i.e. being in the intervention phase, yes/no) and categorical measurement (from the baseline to the final measurement) to assess whether there were general changes over time. Following the intention-to-treat principle, intervention was defined using the randomisation schedule (independent of the degree of involvement in IMPROVE). Missing data in the dependent variables were dealt with by the multilevel analysis under the missing-at-random assumption. There were no independent variables missing.
To account for deviations from the randomisation schedule, a sensitivity analysis in the form of an 'as-implemented' analysis was conducted, replicating the above analyses but using the involvement rating instead of randomised intervention to define involvement in IMPROVE.
Similar mixed-effect models as used for the implementation analyses were used for the association analyses to answer the second study objective about the associations between the patient safety indicators (independent variable) and patient safety outcomes (dependent variable), but without the terms for intervention, involvement, or both.

Additional beforeeafter comparison
Post-hoc explorative beforeeafter comparisons for guideline adherence and patient safety outcomes were additionally performed. Estimation of the difference between the baseline and the final measurement was conducted using the same mixed model, but with a term for the beforeeafter effect replacing the terms for intervention, involvement, or both.

Confounders
Primary and beforeeafter analyses were also accounted for potential confounders. Patient characteristics (i.e. sex, age, socioeconomic status, ASA score, type of surgery, and surgical duration in minutes) and hospital characteristics (type of hospital, number of local interventions initiated to implement the perioperative safety guidelines, and control visits of the Health Care Inspectorate) were added as covariates to adjust the intervention (IMPROVE) and the time effect for possible differences in patient population and other confounding factors.
We derived the socioeconomic status from the zip code area in which a person lives. The average socioeconomic status in the Netherlands is þ0.17 (range, e4.0 to þ4.0). 39 Type of surgery was divided into two risk groups: a mortality risk between 1.00% and 4.02% or between 5.93% and 8.96%. 28 Per measurement period, we also listed whether a hospital received audits by the Health Care Inspectorate.

Study population
Nine hospitals participated in the trial, and a total of 1934 patients were enrolled. Primary and additional analyses included all patients who were enrolled. There were no exclusions after randomisation (trial profile in Supplementary Digital Content 2 for the hospital and participant enrolment).
Characteristics of the patients and surgical procedures in the intervention phase and control situation are shown in Table 3. Patient characteristics, type of surgery, and surgical duration were comparable between the intervention phase and control situation at all four measurement points (not shown).
All hospitals initiated local interventions (range, 0e7) to implement the perioperative safety guidelines in their hospital (Table 1). In seven hospitals, one or more audits were conducted by the Health Care Inspectorate during the study period (range, 1e5 visits). The planned IMPROVE interventions were performed with varying degrees of success; the involvement in IMPROVE of the participating hospitals was unsatisfactory to moderate (range 0e6). There was considerable variation in the hospitals' involvement in IMPROVE and adherence to the randomisation schedule.
Both unadjusted and adjusted analyses with all potential confounders (results not shown) were performed, and all P values remained similar.
The (degree of) implementation was not related to other primary outcome measures of the IMPROVE study (i.e. other stop moments from the composite STOP bundle and timely administration of antibiotic prophylaxis).
After correcting for confounders, the estimated difference decreased for the association between stop moment VI 'discharge from the recovery room' and complications; half of the association can be explained by the confounders. Other estimates remained the same after correcting for confounders.
Guideline adherence on the two process indicators (the stop moments from the composite STOP bundle and timely administration of antibiotic prophylaxis) was not associated with decreased wound infections, length of hospital stay, unscheduled reoperations, and non-elective re-admissions. Most associations were found between adherence to the two process indicators and unscheduled transfer to the ICU. Carrying out stop moment VI 'discharge from the recovery room' was most often associated with improved patient safety outcomes. There was no association between carrying Table 4 Primary analysis of the implementation effect of IMPROVE on guideline adherence measured by the patient safety indicators. *'As-implemented' analysis using the involvement rating instead of randomised intervention to define involvement in IMPROVE. IMPROVE, IMPlementatie Richtlijnen Operatieve VEiligheid; CI, confidence interval. Guideline adherence on the two process indicators  (Table 5).

Additional beforeeafter comparison
The original analysis plan was intended to study the implementation effect of IMPROVE on guideline adherence and the association between guideline adherence and patient safety outcomes (see previous sections). However, as there was no change in guideline adherence as a result of IMPROVE, we focused on exploring what has improved over time and learning as much as possible from the data by beforeeafter analyses. In this section, we therefore report the changes in outcome measures (i.e. guideline adherence and patient safety outcomes) from the start (baseline measurement) to the end (final measurement) of the study. Characteristics of the patients and surgical procedures at the baseline and final measurement are shown in Supplementary Digital Content 3. Overall, the study group consisted of more men (51.4%) than women and had a mean age of 62 (15) yr. Abdominal procedures (86.1%) were the most common. The mean surgical duration was 3.00 (1.43) h. The low intraclass correlation coefficients (i.e. 0 or close to 0) in Supplementary Digital Content 5 show that patients from the same hospital were not very similar.
In the figures of Supplementary Digital Content 4, the time course is shown of the adherence to the separate stop moments, the STOP bundle and timely administration of antibiotic prophylaxis. The last chart on timely antibiotic administration is based on eight hospitals owing to unavailability of data in one hospital (in that hospital, only the time of documenting that antibiotic prophylaxis had been given was reported, not the time of the actual antibiotic prophylaxis gift).
Supplementary Digital Content 5 shows the results for guideline adherence on the two process indicators (the stop moments from the composite STOP bundle and timely administration of antibiotic prophylaxis) and patient safety outcomes. The improvement in guideline adherence from the baseline to the final measurement had a nominal P<0.001, for all process patient safety indicators. These nominal P values remained significant (P<0.05) after controlling for potential confounding factors. In addition, the smaller SDs over time in the figures of Supplementary Digital Content 4 indicate decreasing variation in the execution of the stop moments I 'preoperative risk management', III 'check of the current situation', IV 'time-out', V 'sign-out', VI 'discharge from the recovery room', and administration of antibiotic prophylaxis. A reverse result was found for stop moment VII 'discharge from the hospital', with increasing variation in the final measurements. The low indicator scores on the STOP bundle reflect the low adherence to the stop moment VII 'discharge from the hospital'.
At the final measurement, two hospitals scored 100% on the registration of stop moment 'preoperative risk management' (I), 'check of the current situation' (III), and 'discharge from the recovery room' (VI). Four hospitals scored 100% on the recording of stop moment IV the 'time-out' and V the 'signout'. None of the participating hospitals achieved 100% compliance to stop moment VII 'discharge from the hospital', the STOP bundle, and timely administration of antibiotic prophylaxis.
Data on the structure indicators are presented in Supplementary Digital Content 6. Most structure indicators remained stable or showed a slight improvement over the study period.
At the end of the study, we perceived more accessibility of protocols and more regular updates (for details, see Supplementary Digital Content 6). The mortality rate (1.9% at the baseline measurement and 1.8% at the final measurement) and complication rate (27.9% at the baseline measurement and 26% at the final measurement) remained stable as did unscheduled transfers to the intensive care, unscheduled returns to the operating room, and nonelective hospital re-admissions. The percentage of postoperative wound infections declined by more than 80%, from 13.6% at baseline to 2.6% in the final measurement (nominal P<0.001). The median length of hospital stay decreased from 8 days at the baseline measurement (IQR, 5e14 days) to 6 days (IQR 3e9 days) at the final measurement (nominal P<0.001) (Supplementary Digital Content 5).

Discussion
Our findings were mixed, but there was an overall marked improvement in perioperative safety during the 3-yr study period.
The primary stepped-wedge analyses showed that IMPROVE was significantly related to increased performance of stop moment VI 'discharge recovery room' and to decreased performance of stop moment VII 'hospital discharge' after adjustment for multiple comparisons. There was no further evidence of a positive implementation effect, probably because of the inadequate programme implementation and insufficient involvement into IMPROVE. The negative effect of IMPROVE could also indicate competition between the stop moments. Hospitals could not deal with improving multiple stop moments at the same time. This is in line with our results showing that hospitals seemed to focus on improving selected stop moments, which may have led to a temporally reducing effect on others. Hospitals were making choices about which stop moments of the pathway to focus on first, in recognition that improving the entire pathway may be beyond the limited time and resources they had. Compliance to stop moment VII 'hospital discharge' showed a mixed picture with fluctuations in the adherence for the individual hospitals. Overall, the stepped-wedge analyses showed a picture of positive and negative effects of IMPROVE, but generally positive associations between guideline adherence and patient outcomes.
Over time there was improvement, as shown in the beforeeafter comparisons. These additional analyses seem to suggest that outcomes improved over time almost regardless of the formal programme implementation. Moreover, two patient outcomes improved: postoperative wound infections decreased from 13.6% to 2.6%, and length of hospital stay decreased from 8 to 6 days. We were, however, not able to demonstrate an association between improved antibiotic administration and a reduction of wound infections. Whereas, timely administration of antibiotic prophylaxis is part of a bundle with three other perioperative interventions for the prevention of wound infections of the national patient safety campaign for hospitals 'Prevent harm, work safely', the VMS programme. 40 However, we did not measure these other interventions as timely administration of antibiotic prophylaxis is the only indicator of the bundle included in the perioperative safety guidelines. Therefore, we have no information whether these other interventions improved, stayed the same, or worsened. When complying meticulously to all four interventions in daily practice, evidence from quality improvement projects in the Netherlands, USA, and Canada show that it is possible to significantly reduce wound infection rates. 41 However, adherence to the total bundle per patient was at the onset of this study still very low: only in 10% of Dutch patients there was adherence to all four interventions. 42 This may explain the lack of association between timely administration of antibiotic prophylaxis and postoperative wound infections in the current study. In addition, various patient and surgical characteristics affect the incidence of postoperative wound infections. In this study, we also analysed whether some of these (i.e. age, ASA score, type of surgery, and surgical duration in minutes) perhaps affected infection rate. We only found a positive relationship between surgical duration and postoperative wound infections.
Other patient outcomes remained fairly stable over the measurements, with figures comparable with those of other studies. 4,9,28,43,44 In addition, all process indicators showed an increase, indicating increased guideline adherence. However, special attention should be given to the stop moment 'hospital discharge' (final compliance rate, 28.9%). Poor adherence of the STOP bundle can be explained by the later release of the definitive version of the postoperative guideline (thus with a later introduction of the stop moment 'hospital discharge') compared with the publication of the pre-and perioperative guidelines with the other stop moments. This assumption is supported by the differences between separate indicator scores and the aggregated STOP bundle score.
During the study period, there were improvements in perioperative safety in the nine participating hospitals. It can be concluded that a favourable development has taken place, although we do not know the exact reason(s). Rapid guideline implementation in a low motivated target group showed a picture of mixed implementation effects. There was hardly any direct positive implementation effect of IMPROVE. In none of the participating hospitals was IMPROVE carried out as planned. Involvement of the participating hospitals was unsatisfactory to moderate. 45 We encountered several barriers relating to society (co-interventions by the Dutch Health Care Inspectorate) and implementation (the stepped-wedge design, which turned out to be less suitable for implementing change in such a complex setting; the implementation context with a lack of time and priority, lack of mandate of our contact persons, and lack of support within the hospitals; and the size and complexity [tailoring] of IMPROVE). Other barriers were institutional (local intervention activities, besides fundamental changes occurred in some hospitals), social (stand-alone surgeons who resisted participating in IMPROVE), and provider (complexity of the multi-professional perioperative team) related. 45 Some of these implementation challenges are confirmed by more recent studies. 46,47 The planned association analyses showed quite a few positive associations between perioperative guideline adherence and patient outcomes, which makes it likely that patient safety varies with the adherence to the patient safety indicators. Results showed the impact of the whole perioperative trajectory on patient outcomes, except for the stop moments 'time-out' and 'hospital discharge'. The effect of the time-out procedure was the subject of several studies conducted, in particular on the basis of the WHO Surgical Safety Checklist. 48e50 Because these studies are mainly observational in nature, this has resulted in a low GRADE (Grading of Recommendations Assessment, Development and Evaluation) rating of the conclusions found. However, the literature review performed for the update of the perioperative guidelines has shown that a time-out, if performed correctly and completely, results in lower mortality and fewer complications. 51 No changes in outcomes associated with 'hospital discharge' may be explained by a lack of reliable outcome data after hospital discharge. Data on wound infections, complications and deaths occurring after discharge were not collected. The current study was also restricted to re-admissions occurring to the same hospital. In addition, stop moment 'discharge recovery room' seemed to offer the greatest opportunities to improve patient safety. This is in line with diverse other publications, 3,52e54 which stated that the majority of surgical adverse events occur in the postoperative care process. Results of the study of Greenberg and colleagues 55 suggest that standardisation of handoff and transfer protocols are among the prevention strategies with the greatest potential to improve patient safety. The structured transfer between the recovery nurse and the ward nurse on the basis of a checklist may explain what happens during stop moment 'discharge recovery room' that gives this stop moment the potential to make a difference in patient outcomes. The WHO Surgical Safety Checklist is universally adopted and has been shown to reduce complications and mortality in many, but not all, studies. 48e50 Specific items, such as certain stop moments from the STOP bundle, may offer more benefits than others.
We also found one negative association: between timely administration of antibiotic prophylaxis and the number of complications. This process indicator may be more sensitive to information bias. We saw this in one hospital, which we excluded from the analyses of timely administration of antibiotic prophylaxis. We cannot rule out that this was also the case in more hospitals.

Limitations
This study has several limitations. The stepped-wedge design was viewed as the most ethical option and a pragmatic design to assess IMPROVE on a large scale, within a reasonable time frame and with restricted resources and staffing. However, the implementation of IMPROVE was challenging with the restrictions on logistics, time, and funding, especially when dealing with an intervention requiring behavioural changes and implementation in complex healthcare systems. In retrospect, it may therefore have been necessary to make the implementation periods longer and probably to steer them more intensively, taking into account time needed for prioritising before actually implementing.
There may have been selection bias as participation was on a voluntary basis, without financial support. This may have affected baseline adherence to the process indicators or capacity for change as only better (resourced) hospitals may have participated. This could have resulted in higher baseline performance and smaller performance improvements, potentially impacting the power to detect significant effects on the primary outcome. Improved guideline adherence over the measurements likely reflects a combination of actual changes and improvements in data registration and storage. We were, however, unable to distinguish these effects.
Moreover, the quality of the collected data was dependent on the quality of documentation in the participating hospitals. Overall, the completeness of registrations varied between hospitals in this study. The poor level of documentation in most hospitals turned out to be a real challenge. Eight percent of the administration time point of antibiotic prophylaxis, 11% of the postoperative wound infection rates, and 5% of the complication rates turned out to be missing during the IMPROVE study. Overall, only 1.9% of the (52 218) required key data remained missing, owing to a vigorous approach to follow up on missing data.
A further limitation of the study is related to external factors that may or may not cause residual confounding. Patient safety and quality improvement received increasing attention in journals, professional meetings, and conferences during the period when the guidelines were implemented. It is possible that additional factors are related to, or responsible for, the reductions in wound infections, such as the national patient safety campaign for hospitals. 40 Finally, part of the improvements in this study might also result from the influence of observation (the Hawthorne effect).

Conclusions
In this stepped-wedge cluster randomised trial, use of IMPROVE did not significantly improve guideline adherence. Mixed results could be ascribed to IMPROVE, including a negative implementation effect of IMPROVE on one stop moment. When performing additional beforeeafter analyses, we observed improvements in the safety of perioperative care between the baseline and final measurement, including decreasing wound infection rates and length of hospital stay. In addition, an increasing number of patients received perioperative care as recommended in the national perioperative safety guidelines. The climate of process and safety focus associated with IMPROVE seemed to have a very positive influence on processes and outcomes over the 3 yr of the study. Therefore, it is potentially beneficial to raise the consciousness of quality and safety with such programmes, even if a statistical significant correlation with the programme is more difficult to prove.
Probably because of modest implementation success, we were not able to demonstrate improvements in perioperative patient safety solely as a result of IMPROVE. There was only one positive effect of the intervention, probably because IMPROVE could only be implemented to a limited extent. Therefore, most of the favourable outcomes could not be directly attributed to the implementation efforts of IMPROVE. External influences and a Hawthorne effect may also explain the improved results. Moreover, the local implementation of IMPROVE turned out to be more difficult than anticipated. Delivering a complex intervention into a complex system, such as the perioperative care pathway in a hospital, is challenging with many barriers to achieving intended outcomes. There was no simple reality. Adherence to the stepped-wedge randomisation schedule was variable as hospitals were often late to start the IMPROVE activities whereas some could not wait to start. The stepped-wedge design did not allow us to anticipate in a flexible manner to all types of circumstances that hindered the implementation. In retrospect, it is fair to say that we expected too much change in a too short time frame. Many participants were insufficiently motivated to change established behaviour patterns and procedures. In addition, adequate infrastructure e such as information and communications technology e was often lacking. In conclusion, implementation is not a quick fix, but requires a considerable amount of time and effort. The extent of the tasks required, combined with many organizational challenges, may have meant that many hospitals simply ran out of time to implement the guidelines with all recommendations and stop moments within the intervention phase. Optimal use of the perioperative safety guidelines will also require a culture shift in perioperative teams, and the benefits are only realised if most support the change. Further research is warranted to learn from the best-practice hospitals and reveal key factors supportive of guideline implementation.