Indonesia is the world’s fourth most populous country, with a population of 237.6 million in 2010 (Statistics Indonesia National Population and Family Planning Board Ministry of Health, 2012). According to a study of diabetes prevalence, Indonesia ranks seventh highest globally, with 7.6 million patients recorded in 2012, despite a relatively low prevalence of the disease (Soewondo et al, 2013). The two regions with the highest prevalence of diabetes in the country are Ternate, a small, remote island in eastern Indonesia, where 19.6% of the suburban population has diabetes, and the province of West Kalimantan, with a prevalence of 11.1% (Soewondo et al, 2013).
Diabetes is associated with multifaceted complications. One of the most common of these is foot ulcers, which often result in lower extremity amputations (Singh et al, 2005), and which have a prevalence of 4–10% among people with diabetes (Wu et al, 2007). Diabetic foot ulcers affect patients’ quality of life and have social and economic consequences (Deribe et al, 2014). Diabetes-related complications increase the length of hospital stays and, therefore, the cost of care (Cichero et al, 2013). The average cost of an ulcer in a person with diabetes has been estimated at $13,179 per episode (Stockl et al, 2004).
The high costs of treating diabetic ulcers emphasise the value of intensive inpatient or outpatient interventions designed to prevent ulcer progression. The management of wounds is an essential part of intensive care. In people with diabetes, inadequate treatment of diabetic foot ulcers will result in poor treatment outcomes (Itani et al, 2015). For example, the foot ulcer may become chronic and show delayed healing.
In recent years, interest in chronic wound management has increased, focusing not only on the correct treatment, but also on optimal preventive care (Restrepo-Medrano and Soriano, 2012).
Drawing on the evidence base, various wound management strategies have been developed to accelerate wound healing (Woodbury et al, 2004). The determinants of wound healing include the bioburden and severity of the wound, and these factors should be included in all wound assessments (Houghton et al, 2000).
To assess the effectiveness of wound treatment, a measurement tool is required that can describe the current condition of the wound and detect improvement or deterioration over time. The wound assessment process is essential for the development of an adequate treatment plan. Appropriate topical care and assessments of a wound are only possible when the observations and results of interventions are recorded.
Monitoring tools used in Indonesia include the leg ulcer measurement tool (LUMT) and the photographic wound assessment tool (PAWT) (Houghton et al, 2000; Woodbury et al, 2004). It is very important to observe diabetic ulcers at every change of dressing to determine ulcer progression and to identify potential barriers to wound healing and serious complications.
Developing a new tool
The authors have created a new assessment tool for evaluating wound healing progression. The MUNGS tool (maceration, undermining/tunnelling, necrotic, granulation and signs or symptoms) was developed based on the authors’ clinical observations of diabetic ulcer patients in Indonesia (Figure 1).
The total MUNGS score for each wound is calculated by summing the scores assigned to each of the five domains. Thus, the range of possible total MUNGS scores is between 0 and 15, with 0 representing a completely healed ulcer, and higher scores indicating poor wound healing progress.
PAWT is the tool used in the authors’ clinic currently. This is a modified version of the pressure sore status tool. The PAWT consists of six parameters: wound edges, necrotic tissue type and amount, skin colour surrounding wound, granulation tissue type, and epithelialisation.
There are a number of classification systems available to grade ulcers according to the presence and extent of various physical characteristics of the wound (International Best Practice Guidelines, 2013). However, the authors felt there was a need to develop a specific assessment tool for the purposes of ongoing monitoring of progression.
MUNGS is the first instrument developed in Indonesia specifically to evaluate the appearance of diabetic foot wounds. Unlike PAWT, MUNGS does not include an assessment of the wound size and depth. The rationale for this lies in the difficulties in gaining accurate measurement of the depth and size of diabetic foot ulcers, which can often be on the toes, where it is difficult to accurately measure the size of the wound. The MUNGS tool includes signs or symptoms and uses this as a parameter of healing progress rather than size.
Assessing the wound for clinical signs and symptoms of inflammation and infection is of particular importance in people with diabetes, and clinicians can use assessment tools such as NERDS (non-healing wounds; exudating wounds; red and bleeding granulation tissue; debris on wound surface (yellow/black); and smell) and STONES (size – bigger; temperature – increased; Os – probe to or exposed bone; new or satellite areas of breakdown; exudate, oedema, erythema; and smell). NERDS is for detection of superficial infection, while STONES is to detect deep infection.
The authors believe the parameters identified in the MUNGS tool include components that will recognise infection and ischaemia, but this will require further research to gain confidence in this aspect of assessment.
Prior to using an assessment tool, the reliability of the instrument must be evaluated. The goal of the present study was to compare the inter-rater reliability of MUNGS and PAWT in assessing wound healing progression in diabetic patients. Inter-rater reliability is the degree of concordance or consistence of the performance of two or more observers in recording the same responses at the same time (Karanicolas et al, 2009).
Method
A cross-sectional pilot study of the inter-rater reliability of the MUNGS and PAWT tools was conducted at the Kitamura Wound Clinic in Pontianak city, Indonesia.
Fourteen adult outpatients with diabetic ulcers were asked to participate in the evaluation of the assessment tools. The inclusion criteria were patients who were physically able to participate and whose diabetic ulcers included a variety of wound sizes. A pre-study sample size calculation indicated, that with 14 subjects, a two-tailed test, kappa (P≥0.00) and 80% power were detected (Sim and Wright, 2005).
Raters
Eight raters conducted the assessment of the diabetic wounds: four experienced wound care nurses (WCNs) with a minimum of 2 years’ experience and four inexperienced raters. The inexperienced raters were final year undergraduate nursing students. The WCNs had previous training in diabetic wound care and did not receive any further instructions on how to use the assessment tools. The student nurses were educated about diabetic ulcers and how to use the assessment tools (MUNGS and PAWT) over 2 days. On the first day, students were taught the characteristics of the wound bed in diabetic foot ulcer using photographs, and on the second day they used the tools for assessment on three diabetic foot ulcer patients.
Study protocol
All patients in this study were present at the outpatient clinic during the data collection period and were recruited by the wound care nurse in the routine care process. The patients were informed about the study goals and procedures, and when they agreed to participate in the study they signed consent forms.
Two WCNs and two students rated the diabetic wounds using MUNGS, and two nurses and two students rated the wound using PAWT. The patients were assessed by the raters, and other wound care nurses who did not participate in this study changed the dressings. Each assessment was performed simultaneously and independently. Participants were blind to the ratings of the other evaluators. The total MUNGS and PAWT scores were calculated.
In all the assessments, precautions were taken to avoid cross-contamination of the wounds. Fresh gloves were used during each evaluation, a waterless hand cleanser was used between the evaluations, and no measurement instruments were transferred between patients.
Statistical analysis
The inter-rater reliability was expressed in terms of Cohen’s kappa coefficient. This is a measure of association that indicates the agreement of scores measured by two raters. For example, a value of 0.60 denotes acceptable agreement between assessors, and a value of 0.80 denotes satisfactory or good agreement.
The scale is: score of <0=poor, 0–0.20=slight 0.21–0.40=fair,0.41–0.60=moderate, 0.61–0.80=substantial and 0.81–1.00=almost perfect agreement (Landis and Koch, 1977). Cohen’s kappa was calculated using MedCalc® version 15.8.
Results
In this study, wounds were assessed in 14 people with diabetes. The mean age of the patients was 54.6 years (SD 7.98; range 38–65 years), and 64.3% were women. The mean length of the wounds on the time of the assessment was 27.6 (SD 28.5; range 1–90 days). Each patient had one ulcer.
The inter-rater reliability of MUNGS among the WCNs was almost perfect agreement (0.81), whereas it was substantial (0.69) among the students (Table 1). For the individual MUNGS items used by the WCNs, agreement was satisfactory, except for signs or symptoms, where it was moderate (0.51); and for the students, each item was satisfactory, except for maceration, where it was poor (Table 1).
Using the PAWT, overall agreement was moderate for the WCNs (0.60) and fair for the students (0.43) (Table 2). There was moderate agreement for three of the PAWT items assessed by the WCNs, and for wound edges and epithelialisation the level of agreement was less than 0.40 (Table 2). Agreement for the PAWT items assessed by the students was moderate for granulation tissue (0.52) and epithelialisation (0.49). The level of agreement was below 0.40 for the necrotic tissue type (0.00), amount of necrotic tissue (0.25), skin colour surrounding the wound (0.05) and wound edges (−0.14). Wound edge had a negative rating — a negative kappa represents agreement worse than expected, or disagreement among raters. The kappa value of 0 for necrotic tissue type indicates no more rater agreement than that expected by chance.
Discussion
The results of this study demonstrated that the inter-rater reliability of the MUNGS tool was higher than that of the PAWT among both groups of raters (WCNs and students). Interestingly, the overall agreement of the PAWT was moderate for the two groups of raters, whereas the coefficients of agreement for most of the PAWT items were less than 0.60. This is probably due to the variation in the number of years of experience of the raters, especially that of the nurses.
Another study reported that the reliability of the PAWT was greater in individuals who had at least 5 years of clinical experience in chronic wound management (Houghton et al, 2000).
Based on the findings of this study, the authors can conclude that clinical experience provides the necessary training to distinguish various descriptors of a wound, such as the condition of the wound bed, the skin surrounding the wound and other signs or symptoms specific to diabetic wounds.
The MUNGS results showed that signs and symptoms was moderate for WCNs. This may be caused by several factors. In this study, the ability of two raters to obtain the same results when performing a clinical test can be affected by a number of signs and symptoms on wound condition and/or examiner error. For the item, examiner error could occur if signs and symptoms were identified and/or interpreted inconsistently, and the exam results may be different. The item maceration was poor for students. A possible explanation for this finding is years of clinical experience. The training period that was completed prior to data collection may have improved test performance. However, this training period needs to be considered when applying the results of this study in practice.
The need for prior clinical experience has also been reported with other wound assessment tools, such as the Pressure Sore Status tool (Bates-Jensen, 1992; 1997).
The coefficient of agreement for most of the MUNGS items was greater than 0.80 for the experts. The only mean coefficient of agreement below 0.80 for the WCNs was for additional signs or symptoms. Cohen’s kappa value for inter-rater reliability was almost perfect (Landis and Koch, 1977).
Cohen’s kappa value for the agreement among the students on the undermining/tunnelling MUNGS items was perfect. This may be explained by the amount of undermining/tunnelling being easily identified and recorded in the assessment of wounds.
A previous study by Houghton et al of the use of PAWT for assessing leg ulcers reported that the inter-rater reliability of experienced individuals was high (Houghton et al, 2000). However, that study not only used photographs of diabetic ulcers, but also ischaemic and venous ulcers. Houghton et al postulated that the coefficients of inter-rater reliability were higher when the photographic assessment tool was applied to pressure ulcers compared to leg ulcers (Houghton et al, 2000).
The current study was based on diabetic ulcers of patients in a clinical setting and differs from Houghton et al in the clinical setting, type of wounds assessed and statistical analysis (Houghton et al, 2000). In practical situations many more elements can be taken into account, such as whether or not undermining/tunnelling and signs or symptoms in the wound are present. Wounds in people with diabetes may differ greatly between individuals.
Prior to their application in practice, all assessment tools must be shown to provide a reliable measure of wound healing (Pillen et al, 2009). As with other tools that have been developed to assess wound appearance, the MUNGS tool must be used by individuals with clinical experience. The reliability of this tool will be compromised in the hands of inexperienced trainees. Using the MUNGS to assess wound appearance has advantages, but has limitations as well. Responsiveness of the MUNGS has not been demonstrated by its ability to adequately detect change in diabetic ulcer appearance over time and to demonstrate a difference between healers and non-healers. In addition, aspects related to ischaemia need further development.
Conclusion
This cross-sectional pilot study showed that the inter-rater reliability of the MUNGS tool for assessing diabetic wounds was almost perfect. These findings provide evidence supporting the application of the MUNGS tool in the assessment of the progression and healing of wounds, especially diabetic wounds, of inpatients and outpatients. Further studies with larger groups of subjects are required to determine the validity of the study and the utility of the MUNGS tool over time.
This article first appeared in Wounds International (2016) 7(2): 5–9
