Surveying the scene: using surveys in nursing research

The research design is the overall plan for conducting an investigation and, while there does not appear to be any hard and fast classification of designs, Parahoo (1997) suggests that the three main designs are: i) experimental; ii) survey; and iii) case study. In the hierarchy of research evidence developed by Muir Grey (1997), systematic reviews and experimental studies, especially randomised controlled trials, are listed as generating the most robust evidence upon which to base practice.

Less rigorous forms of experimental clinical trials are followed by well-designed non-experimental studies, preferably from more than one research centre or group. Surveys belong in this category. According to the hierarchy, ranging from best evidence derived from systematic reviews (ranked first) through to expert opinions (ranked fifth), non-experimental research is ranked fourth, which implies that it is considered by some to produce a weaker type of evidence. Surveys are viewed by some researchers as rather shallow and only capable of making a limited contribution to scientific knowledge. As Burns and Grove (2001, p256) point out:

‘This belief has led to a bias in the scientific community against survey research. In this context, the term survey is used derisively.’

However, nursing practice often does not lend itself to experimental work and therefore non-experimental research designs, such as surveys, are both widely used and valued. In non-experimental research no interventions are introduced or tested; there is no intention to intervene with the natural course of events. There is no manipulation of an independent variable, no control group and no randomisation of groups within the study. In non-experimental research the plan is to observe or ‘capture’ what is naturally occurring.

Surveys
Surveys are one type of non-experimental research and are designed ‘to obtain information about the prevalence, distribution and interrelations of variables within a population’ (Polit and Beck, 2004, p234). Surveys tend to gather information regarding activities, beliefs, preferences and attitudes, usually by asking a sample of people direct questions (Polit and Beck, 2004). In addition to creating a picture of what is happening, they can also be designed to investigate links or associations between variables, known as correlational research.

The advantages of surveys include that they can be applied to different populations, to different topics, that they can be broad in scope and that they are an economical way of gathering data. However, it should also be noted that surveys tend to gather data at a broad but superficial level, rather than to probe a situation in detail. They may not be helpful in clarifying complex situations or behaviour. Polit and Beck (2004) suggest that they are better suited to extensive rather than intensive analysis.

Key principles underlying survey research
It would be a mistake to think that conducting a survey is a simple task. If an accurate picture is to be created there are key principles that must be taken into account. Surveys are a form of quantitative research and therefore measurement, accuracy, reproducibility and generalisability are important. Such qualities are fostered by having a structured approach, in which all aspects of the design are specified in advance. Once data gathering starts no changes are made.

Samples
Sampling is a vital part of a survey and if done well the results from the sample can be used to describe the whole study population. Samples must be selected to ensure that neither the researcher nor other influential factors affect the sampling procedure (Atkinson, 2000). For example, if the intention is to describe the exercise habits of patients registered at a local diabetes centre, it is probably not feasible to include everyone in the survey, and therefore a sample must be selected. How can an investigator be sure that when selecting a sample that it is not in some way flawed or biased? If only those who have lost weight in the last year were selected then this would introduce a bias into the results. Whilst the potential for bias in this example is obvious, in practice the sources of bias can often be much more subtle. If the sample is to be representative of the wider clinic population then steps must be taken to minimise any bias. Random sampling is the method of choice:

‘The term random does not mean haphazard or careless but refers to a precise method of selection where all individuals in a defined population stand an equal chance of being selected for inclusion in the study sample’ (Atkinson, 2000, p265).

In contrast, a convenience sample is based on those available when data gathering is scheduled to occur. For example, those who have appointments at a clinic during a specified timescale when the researcher happens to be collecting data. As Parahoo (1997, p231) points out, this method of attaining a sample is open to bias as there may be some subjectivity in the selection ‘as the researcher does not always choose everyone who happens to be available’.

Other forms of sampling include volunteer sampling, in which individuals volunteer to take part and therefore are self-selected; snowball sampling, in which one member of the sample is asked to suggest other people who could be involved in the study; or quota sampling in which different groups need to be represented in proportion to their size within the overall population (Parahoo, 1997). There are advantages and disadvantages to all forms of sampling but they are not discussed here as they are not the focus of this paper, further detail can be gained, if required, from Blacktop (1996) and Newell (1996).

Sample size is an important issue. A sample that is too small is unlikely to enable an accurate description to be portrayed, yet a sample that is too big is wasting resources. The importance of an adequate sample in clinical trials was discussed in the previous paper (Coates, 2004) but even in non-experimental, descriptive studies, sample size must be carefully determined.

Sample size is affected by the type of data analysis required – for example, if the sample is to be divided into sub-categories such as age, gender or type of diabetes, then a larger sample will be required to ensure that there is adequate representation in each sub-group. Data gathering instruments such as questionnaires must be valid and reliable and sample sizes must be big enough to allow appropriate testing to occur. The need for a large sample must then be balanced against the constraints of time and resources as the bigger the sample, the longer it will take to complete and the more resources it will require. As a general rule, after taking account of the type of analysis required, aim for ‘as large a sample as time, availability of subjects, and funds permit’ (Thomas, 1990, p95).

Data gathering instruments
Accuracy is vitally important in surveys and is affected by the method of data gathering. In surveys the most common approach to gathering data is by asking questions – this is usually by a questionnaire although interviews can also be used. However, in a quantitative study, an interview will be structured and will be based on an interview schedule or questionnaire.

Questionnaires tend to be used ‘to collect information on facts, attitudes, knowledge, beliefs, opinions, perceptions, expectations, experiences and the behavior of clients and staff ’ (Parahoo, 1997, p249). Behaviour may be more suited to observation but if large numbers of people are involved then a questionnaire may be the most feasible approach. Questionnaires can be used with large samples either by direct distribution or through the mail.

When using physiological measures it is often possible to use very precise methods but when measuring psychosocial variables it can be difficult to ensure the instruments are precise due to the abstract nature of the construct to be measured. Questionnaires must be carefully planned and constructed to ensure that they are capable of gathering the right data to meet the aim and objectives of the study and to ensure that the data are accurate and precise. In quantitative research they are standardised in that the same questions are asked in the same order of all the participants. Questionnaires can be used on their own or in combination with other methods of gathering data, such as by observation, clinical records, or databases.

It is worth noting that although questionnaires may appear to be deceptively easy to construct, a well- designed instrument requires considerable

effort, and, not surprisingly, much has been written about their construction. Questionnaire design is not covered in this paper but helpful advice can be gained elsewhere (e.g. Burns and Grove, 2001; Murphy-Black, 2000; Oppenheim, 1992).

Case study: compliance of adolescents with chronic illness

A study by Kyngäs (2000) to describe the factors that affect compliance in adolescents with a chronic illness will be used to illustrate some of the important features of survey design. The different steps used in this investigation will be discussed to illustrate how surveys can be rigorously conducted and the results applied to nursing practice.

Defining abstract or complex terms
Kyngäs’s study was about compliance – a widely debated construct, often intangible, and which is particularly notorious to measure (Cramer and Spilker, 1991). For the purposes of research, however, the meaning must be made clear and it must be expressed in a way that enables it to be measured. Helvi Kyngäs is a seasoned researcher in this field and has spent much time teasing out the components of this construct. For the purpose of her investigation we are informed that:

‘Compliance is an active, intentional and responsible process whereby adolescents with a chronic disease aim to maintain their health in collaboration with healthcare staff’ (p550).

While others may interpret ‘compliance’ differently, it is important that it has been defined for the purpose of this study as this will influence how it is measured and investigated.

The sample
Kyngäs was able to define the entire study population, as all individuals in Finland with a chronic illness are registered with the Social Insurance Institution of Finland, which covers the cost of their treatment and care. Adolescence was defined as those people aged between 13–17 years. In this investigation four chronic conditions were selected; asthma, epilepsy, junior rheumatoid arthritis (JRA) and type 1 diabetes, as they share the features of an unpredictable course, necessary behavioural adjustments, long-term dependency on treatment and the need for a serious commitment to self-management strategies. Only those individuals diagnosed with the disease for least a year were to be included to ensure that they had had time to develop self-management strategies and views about them.

There were found to be 8671 people in the study population who met the above criteria. As this would be too large a population to use in its entirety it was important to have a sample selection strategy.

To get a fair representation of individuals from each disease category, Kyngäs (2000) used a method of proportional sampling in which every fifteenth person on the list with asthma, every fifth person with type 1 diabetes and with epilepsy, and every second person with JRA were selected. This generated a total sample of 1200 people in which each disease category was represented in keeping with the overall prevalence of the condition on the register.

Data collection
The data were collected by means of a questionnaire, whose content was initially developed for those with diabetes but then modified to include the other conditions. Development of the questionnaire was guided by a literature review and earlier work on a previously constructed theoretical model of compliance in adolescents with diabetes (Kyngäs, 1999).

The importance of accurate measurement was stated above and it is crucial to estimate the extent to which an instrument (in this case the questionnaire) is accurate. To do this, the properties of validity and reliability are investigated, and we learn that the procedures of factor analysis, correlation and covariance coefficients, face validity and Cronbach’s alpha values were used to establish that the questionnaire could validly and reliably measure compliance with diabetes. Details of these tests and how to use and interpret them can be gained from research books such as Polit and Beck (2004), Burns and Grove (2001), and LoBiondo-Wood and Haber (1998).

Results
One-thousand-and-sixty-one questionnaires were returned, giving a response rate of 88%, which is considered high. Polit and Beck (2004) suggest that a response rate of 65 % or more is usually sufficient for most research purposes. In quantitative research the response rate is important as it affects the extent to which it can be said that the results are gathered from a representative sample. The worry associated with a low response rate is that it may introduce a bias into the study, as it may mean that a particular section of the sample has not responded and thus the results cannot be generalised to the total study population.

The analytical techniques used in the study were firstly frequencies and descriptive statistics, so that a broad picture of background information such as gender, age, duration of disease, exercise patterns and alcohol intake could be obtained. For example, the study indicated that, overall, 23% of adolescents with a chronic illness showed good compliance, 60% showed satisfactory compliance and 17 % had poor compliance.

When those with diabetes are considered, however, it was found that 54 % had good compliance, 75 % satisfactory and only 6% showed poor compliance. Thus, the extent to which the teenagers with diabetes comply with their health care is more favourable than those of the other groups.

Kyngäs then wanted to test whether there were any relationships between the variables that might enable compliance behaviour to be more fully understood. Cross tabulations were used to analyse relationships between variables. Cross tabulations (also known as contingency tables) enable data to be displayed, summarised and visually compared. The data can simply be presented as a descriptive summary or, as in this case, it can be further analysed using a chi-squared test (pronounced ‘kie’) (χ²), which tests for significant differences between the data. According to the theoretical model previously developed by Kyngäs (1999), a range of factors such as motivation, the results of care, a sense of normality and adequate energy and will-power for care, are purported to improve compliance. In this study, it was possible to use χ² to test whether the differences in reported compliance with health care were significantly linked to these factors.

Each patient group was found to have a statistically significant link between compliance and motivation; sense of normality; attitude; energy and will-power; experience of results; support from parents, nurses, and physicians; and threat to social well-being. The significance level is reported to be p<0.001, which means that the probability that this relationship would occur by chance was less than one in a thousand.

In the case of the people with diabetes, there were also statistically significant relationships (p<0.001) between support from friends and compliance; threat to physical well-being and compliance; and emotional well-being and compliance. It was found that the more the adolescents exercised, the less they smoked and drank alcohol and these factors were statistically connected with compliance (p<0.001). The individuals with diabetes for 1–3 years indicated better compliance than those diagnosed for over three years. Gender and family size were not found to have statistically significant links to compliance.

Kyngäs (2000) drew attention to implications for nursing arising from her results such as:

‘It is essential that adolescents are involved in transition planning, and that their interests, needs and aspirations are considered’ (p555).

She advocated that ‘customised treatment plans’ were required, as the research highlighted that the chronic illness and its treatment are only one aspect of an individual’s life and in reality will be treated as such.

Evidence informed practice
In this final section of this article, the importance of appraising the suitability of research which might be used as evidence to inform practice will be discussed. The questions developed by Muir Grey (1997) will be considered as in the previous paper in this series (Coates, 2004):

Is this the best type of research method for this question?
The purpose of the research by Kyngäs (2000) was to describe the factors that affect compliance in adolescents with a chronic illness and so enable a greater understanding of the construct and the factors that might affect it. As there are many factors (variables) which influence behaviour, a large sample will be required to allow for variation across the individuals. A large scale survey approach was an appropriate research design for this study. Had she wanted to test some innovation in practice to see if it could alter compliance behaviour then an experimental design would have been more appropriate.

Is the research of adequate quality?
Yes, this study is of good quality. It generated evidence that can be used to inform practice and is an example of research that produces ‘evidence from well-designed non-experimental studies from more than one centre or research group’ as per Muir Grey’s (1997) hierarchy. This survey was thoroughly and carefully designed and the steps taken are presented in detail. This study had a defined population that was randomly sampled. Individuals were drawn from across Finland, which means that local influences should not bias results. Ethical aspects of the study were addressed and serve to show that the adolescents were treated appropriately, had not been taken advantage of, and that each person had given informed consent to prior to participating in the research.

The selection of variables in the study was based on a theoretical framework – thus, there is a rationale guiding inclusion of variables rather than this being dependent upon anecdotal reasons. All instruments are well developed and tested for validity and reliability and the results of the tests are included in the publication. These points all add to the quality of this research. However, it should be mentioned that this research design is based on patient self-report, and there is no means of verifying the extent to which actual behaviour concords with the self-report.

What is the size of the beneficial effect and of the adverse effect?
Effect size indicates the presence of a phenomenon, and is usually used in experimental research. Effect sizes play an important part in determining sample size in experimental studies but were not applicable in this study.

Is the research generalisable to the whole population from which the research sample was drawn?
As there was a defined study population and a process of random sampling was used, the results of this study could be generalised to the wider population of adolescents in Finland from which the sample was drawn.

Are the results applicable to the ‘local’ population?
While this study is a good example of a survey and its results may be applicable to the wider adolescent population in Finland, it cannot be assumed to be applicable to other adolescent populations. There may be differences in the patient profiles from one country to another, for example in ethnicity, that would affect the results.

Furthermore, compliance is a construct, it is abstract rather than tangible, and therefore its meaning is likely to be affected by social and cultural issues. Your local population of teenagers is likely to interpret compliance differently in some way (Burns and Grove, 2001). It can be difficult to generalise results from a single study but as evidence from research accumulates, then the opportunities to generalise increase.

Are the results applicable to this patient?
The results of the survey identify relationships between variables, chronic conditions and compliance to health care. It is not possible to state, with confidence,

that a general trend or relationship is directly attributable to a specific patient with whom you are working. These results are valuable as they illustrate trends and demonstrate links and as such help us to understand adolescent healthcare behaviour but they cannot be said to be directly transferable to a specific patient.

For example, it was found that in Finland compliance was promoted amongst individuals with diabetes who had support from their friends. This result could be said to apply to other adolescents in Finland with diabetes, but it could not be assumed that a specific patient in your clinic will be more compliant if they have greater support from their friends. This result draws attention to the importance of friends’ support, which might be applicable to your patient, but it is not definitely applicable.

Conclusion
In this article the design of surveys to generate results that might be used as evidence to inform nursing practice have been discussed. The value of surveys relative to other research designs has been outlined and important aspects of the research design that will enhance the robustness of the study have been discussed. Then, using the work of Kyngäs (2000) as an example, issues relating to defining the topic under investigation, sample definition, selection and size, validity of data gathering instruments such as questionnaires and interpretation of results have been further explored. Finally, the application of the results to inform nursing practice were discussed.

Surveys are an important and widely used research design and they have the potential to provide evidence to inform practice. However, even when research has been thoroughly and rigorously conducted, the generalisability of the results must be considered with caution. Results from one research setting may not be taken literally and directly as evidence to inform practice in other localities. However, as nursing as a discipline extends its foundations upon robust research and as evidence accumulates, the opportunities to apply results more broadly will also increase.