Developing the Korean Educational Needs Assessment Tool (Korean ENAT) in rheumatoid arthritis: cross-cultural validation using Rasch analysis
Article information
Abstract
Background/Aims
This study was performed to undertake cross-cultural adaptation and validation of the Educational Needs Assessment Tool (ENAT) in rheumatoid arthritis (RA) for use in Korea.
Methods
The study involved two main phases: cross-cultural adaptation of the ENAT from English into Korean, and validation of the Korean ENAT. The first phase followed the established process of cross-cultural adaptation of self-report measures, and in the second phase, the Korean ENAT data were analyzed using the Rasch measurement model. Fit to the model was determined using the observed data infit and outfit statistics. Additional tests of validity included unidimensionality and internal consistency.
Results
Adequate conceptual equivalence was achieved following the adaptation process. A total of 123 patients completed the Korean ENAT. The mean age was 46.7 ± 12.3 years and the majority of patients (81.3%) were female. Thirty-five of the 39 items gave good fit to the model. The four items deviating from the model had infit and outfit > 1.50. The item separation index (5.26) and item reliability index (0.97) provided evidence for good reliability of items. All seven domains of the Korean ENAT fit the Rasch model. The internal consistency of the Korean ENAT was high, and unidimensionality was confirmed (person separation index, 3.41; reliability index, 0.92; item separation index, 16.82; reliability index, 1.00).
Conclusions
Using the standard procedure for cross-cultural adaptation, the ENAT has been adapted into Korean, and Rasch analysis has confirmed the construct validity, reliability, and unidimensionality of the Korean ENAT.
INTRODUCTION
Rheumatoid arthritis (RA) is a systemic inflammatory disease characterized by the presence of destructive polyarthritis. This chronic disabling disease can affect the individual’s physical and mental functioning [1,2]. It is managed by early intensive treatment with combination drugs, and tight monitoring to achieve remission or low disease activity [3]. Patient education is recommended as an integral part of disease management because (1) the pathophysiology of RA is complex, (2) symptoms and disease impact are different for each patient, (3) the effects and adverse events of disease-modifying antirheumatic drugs (DMARDs) vary, and (4) patients are expected to self-manage and cope with the disease impact at home [4,5]. There is a growing body of evidence showing that patient needs and individual learning capabilities play important roles in patient education [6-8]. Patient education targeting the specific needs of patients with RA is likely to be effective in increasing their self-efficacy and some aspects of their health status [9,10]. Consequently, European League Against Rheumatism (EULAR) recommendations have specified that patient education for people with RA should be individually tailored and need-based [11].
The patient’s perspective regarding educational needs is important since they have experiential knowledge of their disease and carry out daily self-care activities. In addition, patient expectations determine whether education is likely to lead to behavioral change [12]. Therefore, assessing patient educational needs and priorities before providing education is very important for health professionals [13].
The Educational Needs Assessment Tool (ENAT) was developed in the UK to help patients identify their educational needs so that practitioners could address the priority needs as perceived by patients. The ENAT has 39 items, organized into seven domains: managing pain, movement, feelings, disease process, treatments, self-help measures, and support systems [14]. The tool has been adapted into various languages and validated in rheumatic diseases [13-17]; however, there is no Korean version of the questionnaire. The present study aimed to adapt and validate the ENAT into Korean for use in patients with RA.
METHODS
Study design and population
This was a cross-sectional study conducted in two phases: (1) cross-cultural adaptation of the ENAT into Korean and (2) validation of the Korean ENAT in patients with RA. The first phase followed the standardized guidelines for cross-cultural adaptation of patient-reported outcome measures suggested by Beaton et al. [18]. The second phase was conducted using a cross-sectional survey requiring patient completion of the adapted version of the Korean ENAT. To check test-retest reliability, the ENAT was administered twice over a 2-week period.
Cross-cultural adaptation
The cross-cultural adaptation process described by Beaton et al. [18] consisted of five stages. (1) Forward translation from English into Korean performed by two independent translators. Each translator produced a written report containing comments on difficult phrases or uncertainties along with the rationale for their word choices. (2) A third unbiased person held a meeting to discuss translation differences, and a combined version was produced, together with a report documenting the process and how issues were resolved. (3) Back-translation was performed by two bilingual back-translators whose mother tongue was English and who were blinded to the original version. This was a process of validity checking to ensure the translated version accurately reflected the item content of the original version. (4) An expert committee reviewed all the versions and components of the questionnaire and all translated versions, discussed the discrepancies raised in previous stages, and reached consensus on all items. A pre-final version of the Korean ENAT was produced for field testing. (5) The field test of the adapted Korean ENAT involved 30 patients recruited from the rheumatology outpatient clinic of a university hospital in Korea. These 30 patients completed the Korean ENAT; they were then asked what they thought was meant by each questionnaire item, and provided their responses.
Cross-cultural validation
The final translated version of the Korean ENAT was then completed by a consecutive sample of patients with RA. The ENAT was anonymous but contained patient demographic data such as age, gender, educational background, and self-reported disease duration. If they were willing, patients replied by mail after completing the same questionnaire two weeks after the first survey.
Study population
Patients were recruited from the rheumatology outpatient clinic of a university hospital in Korea. The inclusion criteria were (1) RA patients diagnosed by the American College of Rheumatology (ACR) and ACR/EULAR classification criteria, (2) adults (19 years of age or older), and (3) willingness to complete and return the questionnaire. Patients having more than one form of rheumatic disease were excluded. This study was approved by the Institutional Review Board of Hanyang University Hospital (IRB No: HYUN 2015-07-026-001). All patients provided informed consent.
The Educational Needs Assessment Tool
The ENAT is a simple patient-completed questionnaire comprising 39 items grouped into the following seven domains: managing pain (six items), movement (five items), feelings (four items), disease process (seven items), treatments (seven items), self-help measures (six items), and support systems (four items). Items were scored on a Likert scale from 0 “not important at all” to 4 “extremely important” [14].
Statistical analysis
Rasch analysis was conducted using the Rasch computer program Winsteps version 3.91.1 (http://www.winsteps.com/winsteps.htm). The Rasch model provides a formal representation of fundamental measurement, and in Rasch analysis, the data derived from questionnaires are measured against the Rasch model to assess how well they fit the model [15]. Fit mean-square statistics (infit and outfit) and fit standardized as a z-score (ZSTD) statistics were computed to determine whether items fit the expected model. Infit takes into consideration the difference between the observed and expected responses for items that have a difficulty level near the person’s ability. Outfit includes the differences for all items, irrespective of how far the item difficulties are from the individual’s ability level [19], and was determined from the observed data fit statistics (≥ 0.50 and ≤ 1.50) [20]. ZSTD are t-tests of the hypothesis “Do the data fit the model?” and expected values for a perfect model fit are 0.0. More than three of ZSTD means data unexpected if they fit the model perfectly, but, with a large sample size, the substantive misfit may be small [21]. Point-measure correlation was used to identify the validity of the responses with a cut-off value > 0.3. Unidimensionality of the scale was determined based on item and person separation index ≥ 2.00 and reliability index ≥ 0.80. Internal consistency reliability was determined by calculating Cronbach’s alpha. Internal consistency was considered good if Cronbach’s alpha was > 0.70 [22].
In addition, the quartile, median, and extents of ceiling and floor effects were assessed. Floor and ceiling effects between 1% and 15% were defined as optimal [23]. After test-retest, reliability was tested using the intraclass correlation coefficient (ICC).
To compare patient needs for education between domains, domain scores were normalized by dividing by the maximum possible score (%) for each domain. SAS version 9.2 (SAS Institute, Cary, NC, USA) was used for these analyses.
RESULTS
Cultural adaptation into Korean
Ambiguity, multiple meanings, inexactness of certain concepts, and idiomatic expressions were all issues for the translation. Other problems arose due to differences in the style of formulating questionnaire items in English and Korean. For example, since Koreans are not well-acquainted with the term ‘relaxation’ in Korean health care services, patients would have difficulty envisioning how to use it. Therefore, it was defined by describing how to relax the muscles and reduce tension. Another example concerned the information in the movement section regarding ways to save energy. As the word ‘energy’ has multiple meanings in Korean and is most often used to describe environmental resources it was described as the power needed to move muscles or parts of the body (Supplementary Table 1). The expert committee discussed and solved the above-mentioned problems by finding Korean equivalents that were understandable but also accurate from a medical point of view.
Cross-cultural validation in patients with rheumatoid arthritis
The validation study included patients (n = 123) from a tertiary referral hospital outpatient clinic who had completed the Korean ENAT. The clinical and demographic characteristics of the participants are presented in Table 1. The mean (± standard deviation [SD]) age was 46.5 ± 12.2 years, and disease duration was 5.0 ± 6.0 years. The patient means disease activity score for 28 joints with erythrocyte sedimentation rate (DAS28-ESR) was 3.5 ± 1.5, and 12.2% of the patients were on biologic DMARDs. The retest response rate was 87.0% (n = 107).
Participants’ characteristics (n = 123)
To estimate the construct validity, reliability, and statistical sufficiency of the Korean ENAT, fit statistics for individual items and each of the seven domains were performed and are presented in Table 2. Thirty-five of the 39 items displayed good fit to the mode, while four pain and self-help domain items exhibited misfit, deviating from the model with infit and outfit > 1.50. However, when analyzed as testlets, all seven domains of the Korean ENAT were found to fit the Rasch model, and all values of a point-measure correlation analysis were within the acceptable specified range of the Rasch model. Internal consistency was high (Cronbach’s alpha, 0.92), and unidimensionality was confirmed (person separation index, 3.4; reliability index, 0.92/item separation index, 16.82; reliability index, 1.00).
Fit statistics for the Korean ENAT subscales
With regard to ceiling and floor effects, 3.25% of the patients scored in the floor area of the feelings domain, and 1.63% in the support domain, and for the ceiling effect, 21.95%, 24.39%, and 21.14% of the patients scored in the feelings, disease process, and treatment domains, respectively (Table 3).
Internal consistency and test-retest results
A further test of reliability (test-retest reliability) showed an excellent degree of agreement in the feelings domain (ICC, 0.75), good degrees of agreement in the self-help (ICC, 0.72), support (ICC, 0.72), disease process (ICC, 0.65), and treatment (ICC, 0.60) domains, and fair degrees of agreement in the managing pain (ICC, 0.59) and movement (ICC, 0.59) domains.
Estimates of educational needs using the Korean ENAT
The mean (± SD) Korean ENAT total score was 109.1 ± 28.9. Table 4 shows the educational needs of the Korean patients with RA. In terms of percentage of the maximum possible domain score, the highest educational needs were for disease process (81% of maximum score), treatment (77% of maximum score), and feelings (70% of maximum score). The percentages for other domains (self-help, movement, support and managing pain) were somewhat lower (Table 4).
ENAT in Korean RA patients
Levels of educational needs were shown to be slightly different depending on sex. The total ENAT scores of males and females did not differ (102.9 vs. 110.6, p = 0.26), and the highest educational needs in both groups were disease process and treatment domain. However, the score for the feelings domain was significantly higher in female (mean, 11.7 ± 4.0) than in male (mean, 9.3 ± 4.3), p = 0.01.
DISCUSSION
This study was successful in generating a valid ENAT that could assess the educational needs of Korean RA patients; the mean total score of 109.1 showed that Korean patients with RA have considerable educational needs. The highest educational needs were for disease process and treatment domains.
In this study, we employed Beaton’s method [18], which uses forward-backward translation with an expert committee meeting, as well as a field test with patients, thus ensuring thoroughness in translation. The outcomes also demonstrated conceptual equivalence between the English and Korean versions of the ENAT. However, several items did not achieve linguistic or idiomatic equivalence; consequently, additional cultural-specific terms were used to ensure they were understood by the target population. To examine the validity of the Korean ENAT, we used the Rasch model [19]. Rasch analysis is the formal testing of an outcome scale against a math ematical measurement model developed by the Danish mathematician Georg Rasch [24]: data from questionnaires are measured against the Rasch model to assess how well they fit the model [15].
In the fit statistics results, four items in the pain and self-help domains exhibited misfit deviating from the model with infit and outfit > 1.50. However, when analyzed as testlets, all seven domains displayed good fit to the model. Breaching of the local independence assumption has been shown to drive misfit to the Rasch model, and this can be accommodated through the testlet design [14]. Including all the items in the scale is advantageous in that they may inform practitioners about educational needs at a finer level, while grouping them into testlets accounts effectively for local dependence, so satisfying the psychometric requirements [25]. Several domains such as feelings, disease process, and treatment showed a tendency toward ceiling effects. This indicates that the scales of 5 for these domains could be reduced to less than 5 in Korean patients.
The total ENAT score in Korean patients with RA was higher than in Polish patients (75.0 in those aged ≤ 40 years, 67.7 in those aged between 41 and 60 years, and 58.7 in those aged > 60 years) [26]. The report on Polish patients with RA suggested that health education should be targeted at patients with early RA [26]. When we calculated total ENAT scores in Korean patients with RA after stratifying age groups, we also found that total ENAT score decreased in patients aged > 60 years (data not shown). Disease process, treatment, and feelings domains showed high scores in Korea, while in the Netherlands, disease process, treatment, and self-help domains showed high scores [27]. The lower educational needs for managing pain and movement in Korea may be due to the Korean health care environment, in which the use of assistive devices and discussion of exercises with health professionals are not common. Previous reports of racial or ethnic group differences with regard to pain and depression have shown that Asian patients are more sensitive to experimental pain, and have higher levels of depressive symptoms [28,29]. These findings may be connected with the high educational needs of female Korean patients in the feelings domain. Therefore, our findings add to the evidence that this group of patients need psychological support as part of patient education and self-management interventions. Future work on educational needs taking into account patients’ demographic and clinical characteristics can provide evidence for constructing educational program for Korean patients with RA.
This study had several limitations. First, the patients were enrolled from a single university hospital, so the results may not be representative of the entire Korean patient population. However, this does not affect the conclusions of this study, since the Korean language has no major variations across the country and the sample size requirements for Rasch analysis were met. Second, the ENAT is a self-completed questionnaire, and consequently, does not reach patients who cannot read or write. However, using the ENAT in Korean patients is appropriate because the literacy rate in Korea is very high (98% in 2013).
This study had several strengths. First, the thorough validation process was based on the standardized guidelines for cross-cultural adaptation of patient-reported outcome measures. Furthermore, several tests of validity and reliability were conducted to ensure that the conclusions were robust.
In this study, the Korean ENAT was established as a valid and reliable tool providing an accurate measure of educational needs for Korean patients with RA. The study provides further evidence for the validity of the ENAT as a generic questionnaire in rheumatic diseases. This Korean version of the ENAT can help assess patients’ educational needs and plan targeted, patient-centered education in clinical practice.
KEY MESSAGE
1. This study resulted in a valid Educational Needs Assessment Tool that can assess the educational needs of Korean rheumatoid arthritis patients; the mean total score of 109.1 showed that these patients have considerable educational needs.
2. The highest educational needs were found for the disease process and treatment domains.
Notes
No potential conflict of interest relevant to this article was reported.
Acknowledgements
This work was supported by the research fund of Rheumatology Research Foundation (RRF-2015-02).
The ENAT is free to use for educational and non-profit research purposes but permission is required from the University of Leeds who owns the copyright of the instrument. To obtain copies of the original and translated versions of the ENAT and permission to use the ENAT please contact the Psychometric Laboratory at the University of Leeds, Leeds Institute of Rheumatic and Musculoskeletal Medicine, E-mail: RehabMed@leeds.ac.uk.