Evidence-based medicine has been criticised for drawing too heavily on quantitative research and for requiring numerical skills confined to a minority. A randomised controlled trial (RCT) clearly demonstrating the benefit of an intervention may fail to influence health outcomes due to unforeseen barriers. An individual’s acceptance over the need to intervene and the stigma of a “disease label” particularly in a pre-symptomatic phase, are pertinent issues in diabetes care. Similarly, questionnaire surveys counting responses to closed questions often generate lots of useful data, but have limited ability to explore the complex beliefs of respondents only found in the free text. The beliefs, attitudes and behaviours of all involved in health care need to be understood if novel interventions are to have impact, and research methods need to be equal to the task.
Qualitative methods respond to this need by generating and analysing data based largely on words rather than numbers. They address questions for which numerical data are irrelevant, but also support quantitative methods. The contrast between the two approaches is sometimes exaggerated to the point of stereotypy: quantitative research lacks real-life validity and is out of touch with what matters to patients, while qualitative research is nebulous, lacking reproducibility and rigour, according to this caricature. In fact, both are essential components of a comprehensive research effort.
Qualitative methods have a long tradition in the social sciences but their importance to health care took time to become established. Borrowing a contemporary advertising slogan for beer, a memorable article from 1995 claimed that they “reach the parts other methods cannot reach” (Pope and Mays, 1995). This article emphasised the complementary rather than antithetical roles of the two methods, and since the mid-1990s research programmes have tended to include both approaches. Over the same time period, user involvement in the research process has become increasingly important.
Addressing answerable questions
All research attempts to address answerable questions and create new knowledge that is generalisable to wider populations. This is what distinguishes research from audit, service evaluation, or other types of investigation (National Patient Safety Agency, 2009). A randomised controlled trial (RCT) involves a test of a clearly defined pre-specified hypothesis, set in stone in the protocol. In a qualitative study, the investigators are hoping to find out things they cannot predict at the outset as well as perhaps confirming (or refuting) expected patterns, so hypotheses must be allowed to adapt flexibly according to what starts to be found out when people respond to questions. These questions need to be “open”, in contrast to the necessarily closed questions of a computer processed questionnaire survey.
The words that provide the data for qualitative research take a variety of forms. Very often they are gathered during interviews with individuals, but may also involve focus groups where a number of participants engage in a discussion guided by a researcher. In either setting, the researcher starts out with a “topic guide” that outlines the areas to be explored. This topic guide is designed during the planning phase of the study and is approved by an ethics committee. A basic rule is that the researcher avoids leading questions or putting words into the participants’ mouths. However, a project cannot start from a blank slate and the ideas for the study and why it is important arise on a background of established knowledge and perhaps recent developments requiring further exploration. So inevitably, the topic guide will be coloured by the interests and preconceptions of the researchers.
The receptiveness of the interviewer to unanticipated ideas is essential for the creation of new knowledge. The challenge of qualitative interviewing is to allow the interviewee to digress into productive, new areas without losing relevance to the research questions. Interviews may be “semi-structured” (which ensures that a basic set of issues is covered, and may make responses more comparable between interviewees), or “in-depth”, which involves the deeper probing of the person’s beliefs and attitudes to yield richer data.
Recording interview or focus group data
Interviews and focus group sessions are usually audio-recorded and then transcribed (i.e. written down in some form), preferably by an experienced transcriber. Identifying the unseen owner of each audiotaped utterance in a focus group session is extremely difficult and requires training in itself. Video-taping makes this easier but requires more resource. The transcripts are then analysed by the research team. The idea is to recognise identifiable themes arising from the discussion.
The simplest method is called “thematic content analysis”. Recurrent themes, arising more than once and mentioned by several participants, are likely to be particularly relevant and may have been foreseen in the study protocol. The topic guide is designed to draw out and unpack these themes. New, unanticipated emerging themes are also identified and may throw important new light on the subject.
It may be necessary to revise underlying hypotheses, review previously analysed transcripts, or reformulate the topic guide in response to emerging data. There are numerous more complicated analytical techniques beyond the scope of this article. An excellent introduction to qualitative methods is given by Green and Thorogood (2004). Software packages such as NVivo are available to assist in data analysis.
In addition to interview transcripts, the field notes of qualitative researchers provide another source of data. Field notes may be taken during the conduct of a study and include a record of unspoken phenomena, such as observed behaviours. Qualitative researchers also use existing documentation (such as written policy statements, legislation, standard operating procedures) to understand the basis for the behaviour of those under study.
Observing from inside the system: The ethnographic approach
Awareness of participation in a research project may influence behaviour, a phenomenon known as the Hawthorne effect. This was originally identified in a study of worker productivity in 1950, and has recently been measured (McCarney et al, 2007). While the term is usually used in the context of a controlled trial or observational study, interviewees’ responses to audio-taped or video-taped questions may similarly be affected by what they believe the researcher is expecting or wanting to hear. Skilled interviewers can minimise this effect, but there is always some risk of it.
One qualitative method designed to overcome it is the ethnographic approach, in which the researcher becomes embedded in the study environment as a participant observer, blending into the background as far as possible, and witnessing events from an internal rather than external perspective.
Ethnographic methods have long been used by anthropologists and naturalists. Many may remember David Attenborough studying a group of wild mountain gorillas for the BBC in 1979 by sitting quietly among them with a camera crew. As one of his colleagues later reminisced, “Once they’ve accepted you, you become irrelevant”. This is the essence of ethnographic research. Attenborough himself was “astonished to discover just how gentle these giants were when undisturbed”. Such insights can rarely be gained in traditional research environments where the investigators are clearly visible as external, detached, but potentially disruptive observers. In a healthcare setting, ethnographic research might involve spending time in a hospital bed or in an accident and emergency department, making field notes to record the behaviours of others in the natural, undisturbed process of routine care.
The traditional recording of raw data in qualitative research, i.e. as audiotapes, video-tapes or in notebooks, is evolving to include new media with the expansion of web-based resources. The “chat room” environment that has exploded as a forum for health-related conversations is a potentially fertile ground for qualitative research.
Validity and reproducibility of qualitative research compared with RCTs
An RCT of an intervention addresses its research question in an undeniably rigorous way. In fact, measurement of the intervention’s effect is made with a precision that is itself estimable. In other words, we may or may not be able to exclude or confirm a treatment effect, but we can estimate the probabilities that the effect has been missed or erroneously confirmed by the trial. This involves calculating statistical power, confidence intervals, and P values, as discussed in earlier articles in this series (Holt, 2011a; 2011b) and elsewhere (Bland, 2000).
There is little in qualitative research that can match this precision. But criticisms of quantitative methods include the charges that the research question may be irrelevant to the population under study; the outcome measures are designed by academics and not by patients, who have different priorities; and that statistically significant results may have such small clinical significance that hundreds of people may need to receive an apparently beneficial intervention for just one of them to actually benefit.
To add to this list, the sample size of an RCT is generally based on the detection of benefits, not harms, which are (hopefully) much rarer. So a rare but important harm may escape detection in an RCT and its recognition must wait for the post-marketing phase when tens of thousands of people have been exposed. This issue will be covered in the next article of this series.
The reputation of RCT evidence for rigour is based on a trial’s internal validity i.e. lack of bias, including unbiased randomisation and effective follow-up according to the protocol. Most trials are less than perfect in terms of adherence to protocol when examined closely, often for unavoidable practical reasons – an RCT is a very difficult undertaking. External validity (i.e. generalisability to wider populations) is also problematic. Many RCTs have (rightly or wrongly) confined themselves to narrow study populations, excluding those with important comorbidities, general infirmity, limited age groups, one gender or another, etc. The generalisability of such research becomes questionable.
Similarly, the need for unbiased sampling in postal questionnaire surveys is well known, and there is usually a doubt over external validity unless response rates are high, simply because those sufficiently motivated to respond may be a non-representative subgroup of the overall population.
Qualitative research is designed to understand and investigate the beliefs, experiences, behaviours and priorities of a study population. An example might be the experiences of homeless people with diabetes accessing retinopathy screening services in central London. Interviewing even a small sample of such people may throw important light on the issues and obstacles encountered by this community in general, issues that are beyond the reach of any quantitative method. A recruitment strategy aimed randomly at the population of all Londoners would of course be inappropriate. The validity of qualitative research depends not on random sampling but on appropriate “purposive sampling”. Purposive sampling is an attempt to recruit individuals likely to yield the most relevant data and to include a diverse but representative range of opinions.
How can we estimate the necessary sample size for a qualitative project? How many gorillas would David Attenborough need to meet in the wild to conclude that these animals are generally gentle when undisturbed? This is always a difficult question, and there is no well-defined answer. It depends to some extent on the confidence of the research team that the sample is representative of the population under study. For interview projects, a clue that the necessary number has been reached is the phenomenon termed “data saturation”. This means that the same recurrent themes are arising in the transcribed data with no new themes emerging. Similarly, it may be difficult for qualitative researchers to say how reproducible their findings are, i.e. how likely it is that an independent researcher investigating the same research question would identify the same themes, and reach the same conclusions.
An individual qualitative project may answer a research question on its own, but more often it contributes to an accumulating body of evidence that over time influences health care, as further independent researchers investigate similar issues and support (or refute) earlier findings. The same is true of RCT evidence, which is continually reviewed through quantitative meta-analysis. There is increasing interest in making qualitative data synthesis more rigorous. The meta-ethnographic technique (Atkins et al, 2008) has been applied to diabetes studies (Campbell et al, 2003). Most important research questions are only answerable through the synthesis of data from a number of independent studies.
Qualitative research in diabetes
An important issue in recent years concerns whether or not people with type 2 diabetes who are not using insulin, benefit from self-monitoring of blood glucose (SMBG). This research question is relevant to quality of life, to symptom control, potentially to development of complications, and to NHS prescribing budgets. It has predictably attracted significant interest and investment. Clinicians and patients tended to assume in the past that SMBG would probably improve HbA1c levels (as it does in type 1 diabetes), and at least improve an individual’s feeling of autonomy, with raised awareness of responses to carbohydrate challenges. Improved autonomy should, in turn, improve quality of life. This assumption started to be challenged during the 1980s, and became the basis for a number of later studies, including two fairly recent randomised trials of SMBG (Farmer et al, 2007; O’Kane et al, 2008). Other quantitative studies used patient quality-of-life surveys and epidemiological methods. However, all of this research would be incomplete without the inclusion of purely qualitative methods.
In a series of interview studies based in Scotland, Elizabeth Peel and colleagues studied a group of 40 people newly diagnosed with type 2 diabetes (Peel et al, 2004). The participants were recruited purposively from both primary and secondary care sites to ensure a diversity of representative opinions. In-depth interviews lasting approximately 1 hour were conducted, and in all but two cases a repeat interview took place 6-months later. This two-stage process provided longitudinal data on changes in attitudes over time, and also allowed hypotheses generated during the first round to be explored more effectively in the second.
The result section of their report is divided into evidence for the pros and the cons of SMBG, and each of these is divided into a number of thematic headings. There are many quotations in each sub-section, which indicate a wide variety of individual experience of SMBG, both positive and negative. Some people believed it was helping them, providing an ongoing reminder of the presence of diabetes, and reassurance when low results were obtained. Other comments suggested frustration that doctors were not interested in looking at the results. In some cases, unexplained results could be distressing and amplify a feeling of failure.
In a further similar study published 3 years later, the same research group reported as a conclusion: “Clinical uncertainty about the efficacy and role of SMBG in people with type 2 diabetes is mirrored in patients’ own accounts” (Peel et al, 2007). No amount of quantitative data collection could “reach these parts” and describe what people actually believe and experience in this particular setting.
The combined results of all of these projects support the conclusion that SMBG in people with type 2 diabetes who are not using insulin does not generally improve glycaemic control and that, perhaps contrary to expectation, there is some risk of adverse psychological outcomes for some individuals. However, a proportion of people believe that they benefit and the overall conclusion might, in principle, change in the future if people were trained more effectively in understanding and interpreting self-monitored data. The studies published to date provide a good foundation with which future research can be compared.
Conclusion
Qualitative research has become increasingly important in health care since the mid-1990s, as the limitations of quantitative methods to investigate health beliefs, behaviours, attitudes and priorities have become evident, and the importance of these issues to effective healthcare delivery even more so. In parallel with the development of clinical trial methodology and meta-analysis, qualitative research is also evolving more rigorous means of synthesising data from multiple studies. An individual project in either arena can usually only begin to address, rather than single-handedly resolve, a research question. While quantitative research still provides the basis for much current evidence, there are many questions for which the only effective approach is to sit down and ask the people who are likely to know the answers.
Scotland-wide advice to inform the process of making injectable weight management drugs available and to prevent variation between Health Boards.
14 Nov 2024