In recent years, real-world evidence generated from UK primary care data has risen significantly in response to increasing accessibility of large electronic primary care health records and rising costs of clinical trials. Here, we discuss the strengths and limitations of real-world observational studies and publicise a new initiative in cardiometabolic medicine – ERICAS (Evidence Review on Cardiometabolic Observational Studies) – to address some of the quality limitations of published real-world observational studies.
Strengths of real-world evidence
It has long been accepted that patients who are enrolled in clinical trials are very different from those seen by clinicians in the real-world setting, be it primary or secondary care. The often stringent inclusion and exclusion criteria of clinical trials limit generalisability to the wider population. More complex individuals, such as those who are multimorbid, taking multiple medications or lack capacity, are traditionally excluded from clinical trials which tend to focus on safety and efficacy outcomes, rather than replicating real-world clinical practice.
Real-world evidence plays an important complementary role in supporting clinical trials because it includes a broader, more diverse range of patients. It also allows exploration of a wider range of outcomes over longer follow-up periods and allows the identification of rare side effects that would not be possible in the smaller clinical trial setting. Real-world evidence can provide unique insights into the costs and clinical effectiveness of treatments, healthcare systems, unmet needs and best clinical practice. It can also be used to establish “proof of concept” before conducting large efficacy trials for treatments with unlicensed indications.
Limitations of real-world evidence
While evidence suggests that well-conducted observational studies tend to show similar treatment effects to randomised clinical trials in the same topic area (Benson and Hart, 2000), real-world data can also produce a wealth of messy, large, unreliable and unreproducible estimates. The main concern with real-world evidence is the distinct lack of guidance in relation to how it should be assessed and evaluated (de Lusignan et al, 2015). Where there is no transparency in the process of data collection, analysis and reporting, it is understandable that clinicians question the value of real-world evidence and its importance in clinical practice.
Evidence Review on Cardiometabolic Observation Studies (ERICAS)
The authors have set up a new initiative, the Evidence Review on Cardiometabolic Observational Studies (ERICAS), to improve the availability of good quality, real-world evidence in cardiometabolic medicine. We have developed a bespoke tool called the ArROWS (Assessment of Real World Observational Studies; soon to be published) to help break down the recognised barriers in identifying whether real-world observational studies produce adequate real-world evidence. The ArROWS contains questions to assess research articles, including the assessment of research questions, data sources, clinical codes and statistical analyses. Our quality assessment team of statisticians and epidemiologists review publications identified using a pre-defined search strategy. Only articles identified as being of good quality make it onto the ERICAS website (www.ericas.org). They then receive a clinical rating and review from one of our clinical faculty members. We hope to see more initiatives like ERICAS that help to standardise reporting and assessment of real-world evidence in this new era of healthcare innovation.
The authors would like to acknowledge the support received from the NIHR CLAHRC East Midlands.
