All healthcare professionals are under a certain amount of pressure to deliver evidence based care. However, finding the evidence can be a problem. It may be difficult to locate rigorous research on certain subjects. Although, for many topics the problem is that there is too much research, and clinicians are burdened with trying to identify it and make some sense of all the different papers that have been published.
Why do we need systematic reviews?
It is estimated that over two million research papers on biomedical topics are published annually (Mulrow, 1995); a situation which Silverman (1998) refers to as a ‘glut of information’. In addition to the overwhelming volume of information, there is also the question of the quality of the available material. Although we might take comfort in the view that an article must be fairly sound if it has been published, Silverman (1998) is critical of the standards of published articles and points out that as many as 80% of reported studies might be methodologically flawed. The publishing industry relies on the process of peer reviewing, in which individuals with appropriate experience critique the work submitted for publication. In many journals two independent reviewers read each paper and in some journals, for example Diabetes Care, there are three reviewers per paper. But Silverman (1998) is not reassured
because he claims that reviewing is essentially based on opinion. The recently reported ‘MMR research scandal’ (Deer, 2004) reminds us that while research is rarely discredited in such a dramatic way, even journals such as The Lancet may be criticised about the quality of published work. However, clinicians will never have the time to personally review all the research that may be of value to them. Therefore, processes that can identify good quality evidence are valuable in today’s healthcare climate. The situation in which healthcare providers, researchers and policy makers are overwhelmed with unmanageable amounts of information of variable integrity and rigor gave rise to the need for systematic reviewing (Mulrow, 1995).
What is a systematic review?
A systematic review is a process of reviewing all papers that can be located on a particular topic, which uses a specified approach that has been designed to minimise bias and errors that may have influenced the results published in individual papers. This process should result in a single paper or report to which a reader can refer, rather than reading many individual articles. Each paper is evaluated and those that are considered flawed, according to the criteria applied, are identified. The process of a systematic review ‘separates the insignificant, unsound, or redundant deadwood from the salient and critical studies that are worthy of reflection’ (Morgan, 1986).
By drawing together the results from a range of studies it is possible to check whether they are consistent across investigations or whether they are contradictory. If different teams in different localities have investigated a topic, possibly using different methods, yet the research yields consistent results, then they can be used as a robust form of evidence. Conflicting results or inconsistent data will be identified, as will gaps or limitations in the research on a topic and this is also an important finding.
Some systematic reviews will also include a meta-analysis. This is a technique in which data from several studies are combined using statistical techniques, with the aim of producing a single estimate of the effect of the intervention or treatment. It is a bringing together of the analyses of separate investigations of the same general phenomenon (a synthesis). The original investigators have done the analysing; the meta-analysis synthesises the results of those analyses (Powers and Knapp, 1990).
Systematic reviews as a research method
In drawing together the individual results of a range of research studies, a systematic review creates new knowledge and understanding on the subject. As they do not involve the gathering of fresh data systematic reviews may be termed secondary research. The process of the review is thorough and explicit and the method of the review is published as part of the study and could, if warranted, be replicated by others. According to the hierarchy of evidence proposed by Muir Grey (1997), systematic reviews are at the pinnacle of research evidence. This is due to their ability to locate all the randomised controlled trials (RCTs) conducted on a topic. Although a single RCT is a good source of evidence the results of a systematic review which can draw together the results of a range of robust studies is thought to offer a more dependable level of evidence.
The Cochrane Library: a database of systematic reviews
The Cochrane Library is one example of a database that may be consulted to find evidence to inform healthcare. It is a source of reliable and up-to-date information on the effects of interventions in healthcare, which is based on systematic reviews of specific topics. It was formed on the premise that healthcare in the 21st century not only needs the skills of clinicians but also depends upon ‘the best information on the effectiveness of each intervention being accessible to practitioners, patients and policy makers’ (The Cochrane Library, 2004). The Cochrane Library currently contains 30 reviews on diabetes mellitus covering a range of subjects, including for example, a review of specialist nurses in diabetes care (Loveman et al, 2004).
Monitoring glucose: an example of systematic reviewing
The value of self-monitoring of blood glucose is a controversial subject that has generated a vast amount of literature. This activity involves a multi-million pound industry yet we do not know the extent to which patients, clinicians and society reap the benefits of such an immense financial outlay. Recognising the need to identify the evidence available on this subject the Health Technology Assessment NHS R&D Programme (HTA) commissioned a systematic review to identify the evidence relating to glucose monitoring in diabetes. A systematic review was subsequently conducted and the report (Coster et al, 2000) is freely available to all those working in the NHS and can be purchased or reviewed on the web (www.ncchta.org) by all others interested in this subject. The process of this review will be used to explain the steps taken in a systematic review.
Defining the remit of the review
The aim of the review was to systematically search for research data on the clinical effectiveness and cost-effectiveness of different methods of monitoring blood glucose control in diabetes (Coster et al, 2000). Thus the remit was specific and clear. The authors did not set out to review all the information ever written on the topic, they were specifically seeking research data that could be regarded as evidence of clinical effectiveness. Other published work, such as discussions, teaching interventions to enable patients to monitor, patients’ views of monitoring were all beyond the remit. However, in a literature review it might well be possible to mention a wider range of topics which may shed light on a subject but would not necessarily systematically locate evidence about the activity. There is a place for both types of review but the outcomes are different.
The search strategy
As the review is also a form of research the search strategy is presented as an important part of the report; this enables readers to appreciate the range of material consulted. If nurses were to do their own literature review to inform care they may interrogate a database such as CINAHL (Cumulative Index of Nursing and Allied Health, www.cinahl.com). However, as Bird (2003), who has produced a very helpful guide to unearthing literature and to navigating routes through electronic databases relevant to nurses, has pointed out:
‘Unfortunately, there is no single super discovery tool that will, by itself, search all the relevant literature of health care.’
Coster et al (2000) present a search strategy in their report. They searched several electronic databases: MEDLINE (the US National Library of Medicines, an international database of over 4000 biomedical journals, www.bmn.com) from 1976–99; EMBASE (which consists of Excerpta Medica, drugs and pharmacology and psychiatry databases, www.embase.com) from 1980–98; The Index of the Bibliography of Social Science (IBSS, www.bids.ac.uk) from 1975–98; and the database of the Diabetes Health Economic Study Group (www.pitt.edu/~tjs/diabecon.html). In addition, the personal articles of the research team were consulted. A manual search, known as a hand search, was conducted of the 1990–99 volumes of Diabetic Medicine and Diabetes Care. The British Diabetes Association and Bayer and Roche were contacted in an attempt to locate further material. The references from located papers were also checked for relevant research references. All relevant material from the obvious to the obscure should have been exposed through this strategy.
The key words used to conduct the searches and the order and combinations in which they were used are all presented. This allows readers to know exactly what was searched and how, and enables them to replicate the search if necessary. Other systematic reviews may use different databases and key words, but all must explicitly state what strategy was used. It is worth noting that only dedicated databases are used. General interest search engines such as Google are not recommended because general internet searching is not appropriate for locating specific literature in a thorough and systematic way.
The work of Coster et al (2000) covered several aspects of glucose monitoring, however, for the remainder of this article only the work to determine whether blood or urine glucose self-monitoring was effective at contributing to improved blood glucose control in type 2 diabetes will be used, to illustrate the process of systematic reviewing.
Judging the quality of published studies
Relevant papers were reviewed for quality by two members of the research team using a checklist developed by Downs and Black (1998). Four categories of criteria are shown in Figure 1.
Each question was scored 1 for ‘Yes’ or 0 for ‘No’ with the exception of one question which used scores of 0,1 or 2. Thus the total scores for the questions could range from 0–28. Each reviewer rated the studies independently and then compared the scores and the results of these comparisons are included in the report.
Statistical power was assessed for each study to determine, for example, if sample sizes were large enough to detect differences between two groups of patients. The smaller the difference between the two groups the greater the number of patients that will be needed to make it evident.
It would be expected that such a thorough search would yield a prolific volume of papers yet only eight RCTs were identified as shown in Figure 2.
In the quest to locate evidence to determine clinical efficacy all other study designs were excluded. It is important to be aware that even in the presence of a ‘glut of information’ only eight studies were deemed to provide robust evidence on the efficacy of self-monitoring in type 2 diabetes.
The eight studies in the review were compared and contrasted according to a range of variables, such as participants’ age, drug treatment, how they were randomised, sample size, drop out rates, main measures, and adherence to the regimen and quality of the studies. Other types of research design, such as observational studies were also appraised to see if further evidence could be provided. Ten non-randomised studies were identified but due to the potential for bias and the influence of confounding variables these studies were excluded from the evaluation of clinical effectiveness, although the main points from these studies are reported separately.
The results of the systematic review
The main conclusion from the systematic review was that there was no evidence to show that self-monitoring of blood or urine improves blood glucose control measured using HbA1c or fasting plasma glucose (Coster et al, 2000). In addition, there was no evidence that blood glucose monitoring is more effective than urine monitoring to improve control. Conclusions were also drawn about limitations in the studies and it was noted that patients’ views of monitoring ‘were neither completely nor rigorously studied’ and further research into these topics was recommended.
Limitations of systematic reviews
Although systematic reviews are reported to be the ‘gold standard’ of evidence they are not without their limitations (Egger et al, 2001). For example, out of all the treatment, management and service delivery issues that are relevant to the way nurses practice there are only a very limited number of topics that have been subjected to systematic review. Secondly, as the work of Coster et al illustrates, the majority of the research on the topic is not included because it is considered methodologically flawed. However, this may mean that potentially valuable studies are overlooked because they were not generated through a clinical trial. It seems hard to believe that out of all the work conducted on blood glucose monitoring in type 2 diabetes only eight studies are to be taken into account. In nursing, many subjects do not lend themselves to clinical trials since individual perspectives are explicitly sought, therefore topics important to nursing may never be investigated in a way that will allow them ‘best evidence’ acclaim.
Conclusion
Whatever personal views we hold about self-monitoring of blood glucose in people with type 2 diabetes, after reading the detail in the Coster et al report it is clear that every effort was made to ensure this work was done thoroughly and fairly. Clinicians can use these results to justify clinical behaviour, but must also be mindful that further work is being published all the time and that reviews need to be updated.
This article has discussed the need for systematic reviews and indicated that they are a useful technique in the current climate of information overload. Clearly, not all areas of care will have been subjected to such intense scrutiny, but where controversial areas of practice are identified it is worth checking to see if a review has been published. Although systematic reviews are considered ‘the gold standard’ of evidence they are not above criticism and should not be regarded as the perfect research method. This stated, they are widely accepted as a means of identifying best available evidence.
Su Down looks back on a year of change and achievement.
17 Dec 2024