J Neurogastroenterol Motil 2013; 19(4): 509-515  https://doi.org/10.5056/jnm.2013.19.4.509
Development and Validation of the Korean Rome III Questionnaire for Diagnosis of Functional Gastrointestinal Disorders
Kyung Ho Song1, Hye-Kyung Jung2*, Byung-Hoon Min3, Young Hoon Youn4, Kee Don Choi5, Bo Ra Keum6 and Kyu Chan Huh1

1Konyang University, College of Medicine, Daejeon, Korea.

2Ewha Womans University, School of Medicine, Seoul, Korea.

3Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.

4Gangnam Severance Hospital, Yonsei University College of Medicine, Seoul, Korea.

5University of Ulsan, College of Medicine, Seoul, Korea.

6Korea University, College of Medicine, Seoul, Korea.

Correspondence to: Correspondence: Hye-Kyung Jung, MD, PhD. Department of Internal Medicine, Ewha Womans University, School of Medicine, Mokdong Hospital, 1071, Anyangcheon-ro, Yangcheon-gu, Seoul 158-710, Korea. Tel: +82-2-2650-2874, Fax: +82-2-2650-2874, junghk@ewha.ac.kr
Received: June 5, 2013; Revised: June 25, 2013; Accepted: July 2, 2013; Published online: October 7, 2013.
© The Korean Society of Neurogastroenterology and Motility. All rights reserved.

cc This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.


A self-report questionnaire is frequently used to measure symptoms reliably and to distinguish patients with functional gastrointestinal disorders (FGIDs) from those with other conditions. We produced and validated a cross-cultural adaptation of the Rome III questionnaire for diagnosis of FGIDs in Korea.


The Korean version of the Rome III (Rome III-K) questionnaire was developed through structural translational processes. Subsequently, reliability was measured by a test-retest procedure. Convergent validity was evaluated by comparing self-reported questionnaire data with the subsequent completion of the questionnaire by the physician based on an interview and with the clinical diagnosis. Concurrent validation using the validated Korean version of the Short Form-36 Health Survey (SF-36) was adopted to demonstrate discriminant validity.


A total of 306 subjects were studied. Test-retest reliability was good, with a median Cronbach's α value of 0.83 (range, 0.71-0.97). The degree of agreement between patient-administered and physician-administered questionnaires to diagnose FGIDs was excellent; the κ index was 0.949 for irritable bowel syndrome, 0.883 for functional dyspepsia and 0.927 for functional heartburn. The physician's clinical diagnosis of functional dyspepsia showed the most marked discrepancy with that based on the self-administered questionnaire. Almost all SF-36 domains were impaired in participants diagnosed with one of these FGIDs according to the Rome III-K.


We developed the Rome III-K questionnaire though structural translational processes, and it revealed good test-retest reliability and satisfactory construct validity. These results suggest that this instrument will be useful for clinical and research assessments in the Korean population.

Keywords: Dyspepsia, Functional gastrointestinal disorders, Irritable bowel syndrome, Questionnaires, Validation studies

Since the historical meeting of experts in Rome in 1988, several revisions have been made to standardize the diagnostic criteria of functional gastrointestinal disorders (FGIDs).1-3 This standardization has enabled international comparisons of epidemiology, clinical features and clinical outcomes of FGIDs. The medical terms of the Rome criteria are in English, and the majority of self-administered symptom questionnaires are intended for use with English-speakers.4 A simple translation of a published questionnaire usually fails to contain the original meaning because of language and cultural differences.5 Therefore, a systematic translation and the cross-cultural adaptation process are essential for research using questionnaires.6 None of self-administered questionnaires related to FGIDs have undergone a structural validation study in Korea. Thus, the aim of this study was to develop, validate and confirm the reliability of the Korean version of the self-administered symptom questionnaire based on the Rome III criteria (Rome III-K).

Materials and Methods

Development of the Questionnaire

Stepwise processing was used to develop the Rome III-K. First, translation of the Rome III criteria and wording were performed according to guidelines.6,7 Two physicians were involved: a native Korean physician with long-term residence in the USA (Hong YS) and another Korean gastroenterology-subspecialist physician (Jung HK). Based on the comments of Richard Brislin,8 back-translation of the translated draft demonstrated that the target version was functionally equivalent to the Rome III. A translation committee included the above 2 physicians, a psychologist experienced in questionnaire-based studies, a teacher of Korean and 2 native non-medical people. This committee reviewed the translation and back-translation and confirmed that the questionnaire drafted was developed at the level of understanding of sixth grader in elementary school. Several pre-tests with native Korean outpatients provided feedback regarding the readability and comprehensibility of the questionnaire to the translation committee. After these pilot studies, we compiled the final edition of the Rome III-K to be validated. It was composed of 35 questions mainly regarding the presence or frequency of symptoms, including the diagnosis of irritable bowel syndrome (IBS), functional dyspepsia (FD), functional heartburn (FH), functional constipation, functional diarrhea and functional bloating.

Questionnaire Pilot Study

The draft of the Rome III-K was tested on patients presenting at the gastroenterology clinic of Ewha Womans University, Mokdong Hospital. The patients may have complained of gastrointestinal symptoms or had no such symptoms. All patients were native Korean speakers (age, 33-71 years). Patients were requested to fill out the questionnaire, and one gastroenterology-subspecialist judged the FGID clinical diagnosis in a subsequent interview with the patient. The agreement between the clinical diagnosis and diagnosis based on the self-administered questionnaire was the key outcome. Tests of feasibility included completeness of the questionnaire, completion time, length and relevance. Feedback from patients on readability was reviewed by the translation committee. The translation committee revised the questionnaire several times to enhance its comprehensibility.

Validation Study

Participants for the validation study were enrolled from outpatients visiting gastroenterology clinics. Overall, 5 institutes (Ewha Womans University, Mokdong Hospital, Konyang University Hospital, Seoul Samsung Medical Center, Gangnam Severance Hospital and Seoul Asan Medical Center), 4 located in Seoul and one located in Daejeon, were engaged in the survey for content validity and reliability. Each institute had 1 gastroenterologist who interviewed and examined the patients. Recruitment took 7 months from August 2011 to February 2012. During the first visit, the participating gastroenterologists inferred whether the patient had minimal comprehensibility to complete the questionnaire, and asked the relevant people (age, 19-70 years) whether they were willing to take the questionnaire. Those who provided verbal consent were provided with details of the aim and methodology of the study, and gave written consent. After the participants completed the questionnaire, the attending gastroenterologists, who were blinded to the participants' responses, interviewed the participants and subsequently filled out the questionnaire. The content validity of the self-administered Rome III-K was investigated using the physician-administered Rome III-K as the gold standard. The patients may have undergone endoscopy or ultrasound before their next visit. However, those who had started medical therapy or who already were in the course of medications that would affect gastrointestinal symptoms were excluded to avoid confounders. Mental health patients, alcohol or drug abusers, patients suffering from any medical disease that could affect comprehensibility or readability, such as dementia and stroke, were also excluded. The test-retest method was used to assess reliability of the Rome III-K. The retest was the same questionnaire that was completed by the enrolled persons at the second visit, usually 1 week after the first visit. Another survey for additional convergent validity and discriminant validity was performed in an independent study population. The clinical diagnosis judged by a physician (Jung HK), who was blinded to the patients' responses to the questionnaire, was used as the reference to assess convergent validity. We obtained responses for the Korean version of Short Form-36 Heath Survey questionnaire (SF-36-K) concurrently, which is known to be valid and reliable for measuring general health-related quality of life (HR-QOL).9 Participants diagnosed with an FGID according to the Rome III-K are supposed to have lower HR-QOL.10 Therefore, we conducted this concurrent validation to assess the validity of the Rome III-K indirectly through the validated SF-36-K.

Statistical Methods

The majority of Rome III-K question items were categorical or dichotomous items, and the test-retest reliability of the Rome III-K was assessed by Cronbach's α coefficient. Coefficient > 0.7 indicates acceptable consistency. The coefficient for each item is presented as a median with a 95% confidence interval. Convergent validity was judged with the physician-administered questionnaire as the reference. The self-administered questionnaire and physician-administered questionnaire have the same key outcome of the presence or absence of a FGID, which can be calculated according to the Rome III criteria. For example, we diagnosed IBS if the response of abdominal pain was present for the last 3 months (item 1), with a frequency of symptoms > 3 days per month (items 1 and 2) associated with 2 or more subsets of the following items (items 2-6) ranked > 1, and with symptom onset at least 6 months prior to the day of examination (item 1-1). The overall ratios of agreements for the presence of FGIDs between the 2 questionnaires were expressed as a percentage and verified using Cohen's κ coefficient. Convergent validity was expressed as percentage agreement and assessed statistically using McNemar's test. Another survey for additional convergent validity was assessed using the clinical diagnosis of a physician as the reference value. The Wald- Wolfowitz two-sample run test was used to determine whether the subjects without or with FGIDs had an identical SF-36 score distribution. If the overall rate of administered items was < 80% (< 28 of 35 questions), such data were defined as a low responder and excluded from any analyses.

Ethical Approval

All institutes engaged in the validation process approved this study via their own institutional review boards. All participants submitted a written informed consent form at enrollment.


Demographic Characteristics

Overall, 297 subjects agreed to participate inthe reliability and content validity survey. Among them, 65 were excluded with the response rate of < 80%, a duration too long for the test-retest, or dropping out on the second visit.

Finally, 232 subjects were enrolled (median age, 47.5 years; range, 19-73 years; male:female, 1:1.7).


The questionnaire was developed at the sixth grade elementary school reading level. We measured the time required to complete the Rome III-K in a subset of participants, and the mean measured run-time was about 12 minutes (12.0 ± 4.0 minutes).


The median visit interval between the test and retest was 7.0 days (range, 1-19 days). Almost all items had acceptable to excellent consistency with a Cronbach's alpha > 0.7 (range, 0.71-0.97) (Table 1). The median Cronbach's alpha of all IBS items was 0.73 (range, 0.51-0.87), which indicated excellent consistency; FD was 0.84 (range, 0.60-0.86); functional constipation was 0.83 (range, 0.73-0.86); functional diarrhea was 0.83; nausea was 0.83; vomiting was 0.81; and bloating was 0.89. Four questions of suboptimal value were related to IBS or FD. One question for IBS (features of accompanying decreased bowel movement) and one question regarding FD (epigastric pain not relieved by defecation) showed poor consistency.


The degree of agreement between patient-administered and physician-administered questionnaires to diagnose FGIDs was > 95% for all kinds of disorders (Table 2). The κ index of each disorder also confirmed that all agreement degrees were significant. Subtyping of IBS or FD was accurate. Thirty-six of 194 participants were diagnosed with IBS in both patient-administered and physician-administered questionnaires, and the concordance rate of IBS subtyping was as high as 94.4% among IBS patients (34/36) (2 diarrhea-predominant patients were classified as mixed subtype according to the patient-administered questionnaire). Subtyping of patients with FD by self-reported responses was also highly concordant with the physician-completed results, as all corresponding patient subtypes were equivalent (17/17).

We compared the physician's clinical diagnosis with the self-administered responses using a number of subjects (n = 74). McNemar's test verified that there was no significant difference for determining the presence or absence of all kinds of FGIDs in the population (Table 3). The percentage agreement between physicians' clinical diagnosis and diagnosis by patients' self-reported questionnaires was excellent: IBS (80.9%), FH (87.2%), functional constipation (90.9%), functional diarrhea (95.1%) and functional bloating (87.7%). However, a marked discrepancy was observed in the diagnosis of FD (34%), although the difference was not significant.

Rome III-K was sufficiently valid to distinguish patients with FGIDs from normal subjects with higher HR-QOL as determined by validation using the SF-36-K. HR-QOL tended to be lower in patients suffering from FGIDs compared to that in subjects without FGIDs in 7 domains of the SF-36, with the exception of physical functioning (Table 4). Scores of patients with FD did not decline in some additional domains (role physical, bodily pain and social functioning). Age, gender ratio and presence of on-going medical problems were not significantly different between the normal group and each FGID; the FH group was significantly older than the normal group (52.5 ± 14.0 vs. 44.0 ± 14.7 years, P = 0.037 by Mann-Whitney test). Similar patterns of difference in HR-QOL were also observed when subjects with overlapping FGIDs were compared with non-overlapping patients. Seven of 8 SF-36 domains were reduced in patients with overlapping syndromes (Table 5). Age, gender ratio and the presence of on-going medical problems were not different between the 2 groups.


We demonstrated that the Rome III-K is a reliable and valid self-reported tool for diagnosis of FGIDs. A validated questionnaire related to the diagnosis of FGIDs is crucial to assess the prevalence of FGIDs in epidemiological studies or to enroll patients in clinical trials. Systematic translation and independent back-translation are essential for accurate translation of the original source. Without such processes, cross-language studies are vulnerable to the risk that between-group differences, a different prevalence for example, are an artifact of differences in the adequacy of translation.5 We produced several reconciled versions and a reliable version of the Rome III questionnaire in Korean. We have demonstrated that the Rome III-K is valid for diagnosis of FGIDs.

Many symptoms related to FGIDs, including pain with bowel habit change, early satiation and heartburn, are difficult to express clearly in Korean and can be easily misunderstood. Anatomical misunderstanding between the epigastrium and substernal area is one example. The Rome III-K contains a small illustration to minimize such anatomical misunderstandings, and the satisfactory reliability shown by the test-retest method suggests that this questionnaire was translated accurately. However, the question "onset of pain associated with decreased frequency of stool" had poor reliability. This suboptimal result might be related to a lower positive response rate among patients. Cronbach's α was also < 0.6 for the question "epigastric pain not relieved by defecation." It was not clear whether this poor reliability was related to significant overlap with IBS. A Portuguese validation of the Rome III questionnaire for FD showed a similar level of reliability,11 whereas a Malay translation of the Rome III questionnaire for IBS had relatively higher test-retest reliability compared with that in the present study.12

Convergent validity was excellent when we used the physician-completed questionnaire as the reference. The percent agreement between the physician-completed and self-administered Rome III-K was > 97% for all FGID items. FD showed excellent agreement at 97.9%; however, that between clinical diagnosis and self-reported ROME III-K decreased to 66.0%. This reduction can be explained partly by the fact that the Rome III definition of FGIDs is stricter than physician's clinical diagnosis. The Rome III criteria were developed for selecting pathogenetically homogenous patients for research.13 Therefore, strict definitions of symptom onset and duration were included. The major changes in the Rome III criteria compared to the Rome II criteria were symptom onset and duration; these were intended to decrease recall bias. Interestingly, the reliability of symptom onset and duration of each IBS and FD subscale had a Cronbach's α coefficient > 0.8 in the present study. Therefore, these questions may be important for discrimination of FGIDs from organic gastrointestinal symptoms. The clinical diagnosis of FD showed a marked discrepancy with the self-reported diagnosis, as 34.0% of the study population was diagnosed differently. However, this discrepancy was minimized to 2.1% when we interviewed the participants and diagnosed FD based on the formal questionnaire. A questionnaire-based FD diagnosis may be far more accurate than a physician's diagnosis for several reasons. The Rome III criteria for epigastric pain syndrome (EPS) are relatively strict, and a physician is more likely to fail to remember the criteria without a formal questioning form. Additionally, a marked overlap in non-erosive reflux disease with EPS was reported recently in Korea, even though the authors used an insufficiently validated questionnaire.14 Heartburn or acid regurgitation is the most bothersome symptom in only 24.6% of patients with reflux esophagitis, according to a nationwide endoscopy-based study in Korea.15 More than 13% of patients with reflux esophagitis complained of epigastric pain as the most bothersome symptom in that report. Therefore, it is difficult to distinguish FH from EPS according to the Rome III criteria in Koreans complaining of epigastric pain, and our data suggest a relatively low accuracy for the clinical diagnosis of FD. The Rome III-K contained only questions related to typical symptoms, heartburn and acid regurgitation, for the diagnosis of FH. In contrast, the absence of extraesophageal symptoms is the main disadvantage of the Rome III-K. Therefore, we developed questions regarding extraesophageal symptoms, including cough, asthma, globus pharyngitis, hoarseness and chest pain, to enhance the sensitivity of FH; however, these items have yet to be validated. These data confirm the convergent validity of the Rome III-K. The substantial degree of agreement with clinical diagnosis is comparable to questionnaires in other Asian languages.12

Discriminant validity is another pillar of construct validity, as introduced by Campbell.16 However, no standardized value is available for discriminant validity. In addition, it is theoretically difficult to validate when there is no previously validated tool that measures the same concept or disorder. Because the Rome III-K is the first systematically translated Korean questionnaire for diagnosis of FGIDs, we adopted a concurrent validation process. The previously validated SF-36-K can discriminate low HR-QOL in participants suffering from FGIDs. One report of a Korean population demonstrated that the HR-QOL is significantly impaired in subjects with IBS, uninvestigated dyspepsia and FH; this finding is consistent with data from other countries.10 Another study from Japan demonstrated worse HR-QOL in patients with an overlap of these 3 diseases.17 In the present study, almost all SF-36 domains were impaired in participants who were diagnosed with one of these FGIDs according to the Rome III-K. Furthermore, participants with overlapping disorders tended to reveal a worse QOL. These results suggest that the Rome III-K can discriminate the 2 groups of participants. The weakness of our study is the lack of demographic data regarding QOL (marital status, presence of religion, level of education, or economic status), which may be a confounder regarding discriminant validity.

We conclude that the Rome III-K questionnaire, developed using a structural translational process, had acceptable test-retest reliability and good construct validity. Nationwide epidemiological studies and clinical trials should now be conducted using this first Korean instrument for diagnosis of FGIDs.

Table. 1. Cronbach's α Coefficient for Reliability of the Korean Version of Rome III Questionnaire for Functional Gastrointestinal Disorders

The data are presented as medians (95% confidence interval); FGIDs, functional gastrointestinal disorders; BM, bowel movement.

Table. 2. Convergent Validity Judged Using a Physician-completed Questionnaire as the Reference

The data are presented as medians (95% confidence interval); FGIDs, functional gastrointestinal disorders; BM, bowel movement.

Table. 3. Convergent Validity Judged Using Clinical Diagnosis as the Reference

The data are presented as medians (95% confidence interval); FGIDs, functional gastrointestinal disorders; BM, bowel movement.

Table. 4. Short Form-36 Health Survey Scores in Subjects Without and With the 3 Functional Gastrointestinal Disorders According to the Korean Version of Rome III Questionnaire

The data are presented as medians (95% confidence interval); FGIDs, functional gastrointestinal disorders; BM, bowel movement.

Table. 5. Short Form-36 Health Survey Scores in Subjects Without and With Overlapping Functional Gastrointestinal Disorders

The data are presented as medians (95% confidence interval); FGIDs, functional gastrointestinal disorders; BM, bowel movement.

  1. Thompson WG, Creed FH, Drossman DA, Heaton KW, Mazzacca G. Functional bowel disorders and functional abdominal pain. Gastroenterol Int. 1992;5;75-91.
  2. Thompson WG, Longstreth GF, Drossman DA, Heaton KW, Irvine EJ, M?ller-Lissner SA. Functional bowel disease and functional abdominal pain. Gut. 1999;45;II43-II47.
  3. Longstreth GF, Thompson WG, Chey WD, Houghton LA, Mearin F, Spiller RC. Functional bowel disorders. Gastroenterology. 2006;130;1480-1491.
  4. Drossman DA, Talley NJ, Whitehead WE, Corazziari E. In: Drossman DA, Corazziari E, Talley NJ, Thompson WG. Rome II: the functional gastrointestinal disorders. McLean, VA: Degnon Associates; 2000. p. 669-714.
  5. Berkanovic, E. The effect of inadequate language translation on Hispanics' responses to health surveys. Am J Public Health. 1980;70;1273-1276.
  6. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of helth-related quality of life measures: literature reviewand proposed guidelines. J Clin Epidemiol. 1993;46;1417-1432.
  7. Brislin RW, Lonner WJ, Thorndike RM. In: Brislin RW, Lonner WJ, Thorndike RM. Cross-cultural research methods. New York: John Wiley; 1973. p. 32-58.
  8. Brislin, RW. Back-translation for cross-cultural research. J Cross Cult Psychol. 1970;1;185-216.
  9. Nam BH, Lee SW. Testing the validity of the Korean SF-36 health survey. J Korean Soc Health Stat. 2003;28;3-24.
  10. Jeong JJ, Choi MG, Cho YS, et al. Chronic gastrointestinal symptoms and quality of life in the Korean population. World J Gastroenterol. 2008;14;6388-6394.
  11. Reisswitz RS, Mazzoleni LE, Sander GB, Francisconi CF. Portuguese validation of the Rome III diagnostic questionnaire for functional dyspepsia. Arq Gastroenterol. 2010;47;354-360.
  12. Lee YY, Waid A, Tan HJ, Chua SB, Whitehead WE. Validity and reliability of the Malay-language translation of the Rome III diagnostic questionnaire for irritable bowel syndrome. J Gastroenterol Hepatol. 2012;27;746-750.
  13. Drossman, DA. The functional gastrointestinal disorders and the Rome III process. Gastroenterology. 2006;130;1377-1390.
  14. Noh YW, Jung HK, Kim SE, Jung SA. Overlap of erosive and non-erosive reflux diseases with functional gastrointestinal disorders according to Rome III criteria. J Neurogastroenterol Motil. 2010;16;148-156.
  15. Shim KN, Hong SJ, Sung JK, et al. Clinical spectrum of reflux esophagitis among 25,536 Koreans who underwent a health check-up: a nationwide multicenter prospective, endoscopy-based study. J Clin Gastroenterol. 2009;43;632-638.
  16. Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull. 1959;56;81-105.
  17. Kaji M, Fujiwara Y, Shiba M, et al. Prevalence of overlaps between GERD, FD and IBS andimpact on health-related quality of life. J Gastroenterol Hepatol. 2010;25;1151-1156.

This Article



Aims and Scope