SICOT

Editorial

Intervention, change and quality improvement

   

Abstract

Clinical governance is a systematic approach to maintaining and improving the quality of patient care. In quality improvement projects such as a complete loop audit, interventions are often evaluated retrospectively. This presents several challenges including background changes known as secular trends. Techniques to address this issue include cluster randomised controlled trials, interrupted time-series analysis and segmented linear regression methods. These have their strengths and weaknesses and it is important that orthopaedic surgeons understand their underlying principles.

In recent years there has been a surge in large-scale quality improvement projects in Orthopaedics. The concurrent rapid growth in the field of implementation science has meant implementation strategies have become more sophisticated and targeted at multiple levels of an organisation. These projects allow valid conclusions to be drawn about healthcare interventions, but the adoption of appropriate statistical techniques to deal with the limitations of retrospective studies has been variable. In the context of rapidly changing patient demographics, standards of care and healthcare austerity, the need for routine adoption of appropriate statistical techniques in all quality improvement projects is more important than ever before.

This editorial outlines the key challenges to retrospective time-series analysis in the context of quality improvement, and highlights the main methods used to address these. We also identify some areas for future work and highlight the need to evaluate change using mathematically rigorous methodology.

Keywords: Change; Quality improvement; Orthopaedics; Intervention; Secular trends

Introduction

As part of the medical community, orthopaedic surgeons are constantly seeking ways to improve the quality of their practice. Clinical governance is “a systematic approach to maintaining and improving the quality of patient care” within healthcare services. A prerequisite to improvement is change. In healthcare systems, change is often marked by interventions; these are formal attempts at implementing and maintaining change. Crucially, when seeking to evaluate the effectiveness of an intervention, we must re-assess performance and compare it with the pre-intervention status quo. This is the final step in an audit cycle: closing the loop. Indeed, reflecting on the effect of an intervention is a requirement for its maintenance and widespread adoption. But did the intervention cause the observed improvement? This brings us to the biggest challenge that healthcare professionals are faced with in quality improvement: time.

Time

Wouldn’t it be ideal to travel back in time and evaluate performance with and without the intervention? Clearly this would allow for a fair comparison to draw conclusions from. Unfortunately for us, time is change, and quality improvement projects must compare performance before and after an intervention in different environments. This makes it difficult to distinguish causation from association. In an actively evolving environment with ageing patients, healthcare austerity and constantly changing guidelines, how do we know what to credit for any observed change? Although randomised trials eloquently control for known and unknown such factors, quality improvement projects are usually retrospectively evaluated. They are sensitive to secular trends.

Secular trends

Secular trends are changes in the measured performance outcome over time that are due to background factors, unrelated to the intervention under investigation. Commonly, audit cycles evaluate an intervention by statistically comparing two groups: one before and one after the intervention. Figure 1 shows what can go wrong when secular trends are unaccounted for - where observed changes are misattributed to the (seemingly unrelated) intervention. So how can we control for these time-dependent, inevitable secular trends?

Cluster randomised trials (c-RCT)

The defining feature of a c-RCT is that the unit of allocation is a group rather than an individual [1]. For example, some hospitals would be randomised to implement a new intervention whereas others would be randomised to continue offering the current standard of care. This allows evaluation of performance in the same time-period, hopefully controlling for externally originating secular trends. Challenges with this approach include ethical approval, blinding, and population matching. Group consent is not a substitute for individual approval, and a balance must be achieved between individual autonomy and the common good. c-RCTs are particularly susceptible to selection bias as blinding is logistically more challenging. Just like for standard RCTs, baseline characteristics of control and intervention groups must be demonstrated to be similar. However, geographically distinct patient populations and regional differences pose a challenge.

Interrupted time-series analysis

Cochrane recommends the use of interrupted time-series (ITS) analysis for evaluating the effectiveness of an intervention [2][3]. This is a useful tool for evaluating the effectiveness of interventions, where data is collected at several time-points before and after the intervention, to determine whether any change could be explained by secular trends. Although it addresses non-stationarity of data, it does have some limitations. Importantly, it displays bias for detecting change at the time of the studied intervention. It also does not allow for time-series to be modelled with anything more complex than a single straight line before and after the intervention. Finally, ITS does not allow more information to be drawn about other sudden changes that may have occurred during the study period.

Segmented linear regression techniques

The limitations of ITS can be somewhat addressed by using segmented linear regression techniques. By incorporating more than one linear segment before and after the intervention, other important changes in the time-series can be detected - not just the specific intervention studied. A limitation of this technique is deciding the maximum number of linear segments to use, and further work is required in this field [4].

Are these techniques being used in Orthopaedic literature?

A multi-centre cluster randomised controlled trial in 11 centres in Denmark investigated the effectiveness of group-based reassuring information in altering the behaviour of low back pain [5]. This study found that a simple non-threatening explanation for low back pain improved days at work and work ability.

Matharu and colleagues used ITS to investigate the changing trends in revision of metal-on-metal total hip arthroplasty [6]. They concluded that the increased revision rates observed over time were likely due to increased surveillance and changes in revision thresholds rather than the introduction of the Pinnacle acetabular system.

Recently, segmented linear regression techniques were used to evaluate the effectiveness of a dedicated hip fracture unit in improving outcomes for patients with proximal femoral fragility fractures [4]. This study found that improvements in mortality rate were likely due to concurrent improvements over the last decade including introduction of the Best Practice Tariff, neck of femur clerking proformas and changes to hip implants. Employing this technique revealed the effect of the Best Practice Tariff in reducing the time to surgery early in the study period, and unrelated to the studied intervention.

What next?

Several multi-centre pragmatic randomised trials in trauma have been completed over the last decade, and more are currently underway [7][8]. In the United Kingdom, the orthopaedic community is facing major changes with centralisation and standardisation of care through the ‘Getting It Right First Time’ programme [9]. It is important to evaluate surgeons’ change in practice following interventions as well as publication of trials with a mathematically rigorous manner. This approach must also be adopted in smaller scale local departmental audits. Changing practice is only worth it if we can accurately determine whether the change made a difference.

1. Hemming K, Eldridge S, Forbes G, Weijer C, Taljaard M. How to design efficient cluster randomised trials. BMJ 2017;[PMID: 28710062 DOI: 10.1136/bmj.j3064]
2. Lopez Bernal J, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol 2016;1–8 [PMID: 27283160 DOI: 10.1093/ije/dyw098]
3. Higgins JPT, Green S, (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. 2011.
4. Valsamis E, Ricketts D, Husband H, Rogers B. Segmented Linear Regression Models for Assessing Change in Retrospective Studies in Healthcare. Comput Math Methods Med 2019;  [DOI: https://doi.org/10.1155/2019/9810675]Available from: https://www.hindawi.com/journals/cmmm/2019/9810675/#B18
5. Frederiksen P, Indahl A, Andersen LL, Burton K, Hertzum-Larsen R, Bendix T. Can group-based reassuring information alter low back pain behavior? A cluster-randomized controlled trial. PLoS One 2017;[PMID: 28346472 DOI: 10.1371/journal.pone.0172003]
6. Matharu GS, Hunt LP, Murray DW, Howard P, Pandit HG, Blom AW, Bolland B, Judge A. Is the rate of revision of 36 mm metal-onmetal total hip arthroplasties with Pinnacle acetabular components related to the year of the initial operation? An interrupted time-series analysis using data from the national joint registry for England and Wales. Bone Jt J 2018;[PMID: 29305448 DOI: 10.1302/0301-620X.100B1.BJJ-2017-0625.R2]
7. 7Costa ML, Achten J, Caroline P, Parsons NR, Rangan A, Tubeuf S, Yu G, Lamb SE. UK DRAFFT: A randomised controlled trial of percutaneous fixation with kirschner wires versus volar locking-plate fixation in the treatment of adult patients with a dorsally displaced fracture of the distal radius. Health Technol Assess (Rockv) 2015;[PMID: 21914196 DOI: 10.3310/hta19170]
8. Bhandari M, Devereaux PJ, Einhorn TA, Thabane L, Schemitsch EH, Koval KJ, Frihagen F, Poolman RW, Tetsworth K, Guerra-Farfán E, Madden K, Sprague S, Guyatt G. Hip fracture evaluation with alternatives of total hip arthroplasty versus hemiarthroplasty (HEALTH): Protocol for a multicentre randomised trial. BMJ Open 2015;[PMID: 25681312 DOI: 10.1136/bmjopen-2014-006263]
9. Barratt H, Turner S, Hutchings A, Pizzo E, Hudson E, Briggs T, Hurd R, Day J, Yates R, Gikas P, Morris S, Fulop NJ, Raine R. Mixed methods evaluation of the Getting it Right First Time programme - improvements to NHS orthopaedic care in England: study protocol. BMC Health Serv Res 2017;[PMID: 28115018 DOI: 10.1186/s12913-017-2012-y]