J Neurogastroenterol Motil 2022; 28(3): 376-389  https://doi.org/10.5056/jnm22004
A Systematic Review of the Development and Psychometric Properties of Constipation-Related Patient-Reported Outcome Measures: Opportunities for Digital Health
V Vien Lee,1* Ni Yin Lau,1 David J Y Xi,1 Anh T L Truong,1,2 Agata Blasiak,1,2,3* Kewin T H Siah,4,5* and Dean Ho1,2,3*
1The N.1 Institute for Health, National University of Singapore, Singapore; 2Department of Biomedical Engineering, National University of Singapore, Singapore; 3The Institute for Digital Medicine (WisDM); Department of Pharmacology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore; 4Division of Gastroenterology and Hepatology, Department of Medicine, National University Hospital, Singapore; and 5Department of Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
Correspondence to: *V Vien Lee, PhD
The N.1 Institute for Health (N.1), National University of Singapore, Center for Life Sciences, 28 Medical Drive, #05-COR, 117456, Singapore
Tel: +65-6516-6666, Fax: +65-6873-3905, E-mail: vvien.lee@nus.edu.sg
Agata Blasiak, PhD
The N.1 Institute for Health (N.1), National University of Singapore, Center for Life Sciences, 28 Medical Drive, #05-COR, 117456, Singapore
Tel: +65-6516-6666, Fax: +65-6873-3905, E-mail: agata.blasiak@nus.edu.sg
Kewin T H Siah, MBBS (London), MRCP
Division of Gastroenterology and Hepatology, National University Health System, 1E Kent Ridge Road, Level 10, 119228, Singapore
Tel: +65-6779-5555, Fax: +65-6775-1598, E-mail: kewin_siah@nuhs.edu.sg
Dean Ho, PhD
The N.1 Institute for Health (N.1), National University of Singapore, Center for Life Sciences, 28 Medical Drive, #05-COR, 117456, Singapore
Tel: +65-6601-7515, Fax: +65-6873-3905, E-mail: biedh@nus.edu.sg

V Vien Lee, Agata Blasiak, Kewin T H Siah, and Dean Ho are equally responsible for this study.
Received: January 5, 2022; Revised: February 25, 2022; Accepted: May 24, 2022; Published online: June 20, 2022
© The Korean Society of Neurogastroenterology and Motility. All rights reserved.

cc This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
Background/Aims
Constipation can be a chronic condition that impacts daily functioning and quality of life (QoL). To aid healthcare providers in accurately assessing patient symptoms and treatment outcomes, patient-related outcome measures (PROMs) have been increasingly adopted in clinical settings. This review aims to (1) evaluate the methodological quality and measurement properties of constipation-related PROMs, using the COnsensus-based Standards for the selection of health Measurement INtruments (COSMIN) criteria; and (2) assess the modes of digital dissemination of constipation-related PROMs.
Methods
PubMed, Embase, and PsycINFO databases were searched and 11 011 records ranging from 1989 to 2020 were screened by 2 independent reviewers. A total of 26 studies (23 PROMs; 18 measuring symptom-related items and 5 measuring constipation-related QoL items) were identified for the review and assessed.
Results
There were multiple variations between PROMs, including subtypes of constipation, methods of administration, length of PROM and recall period. While no PROM met all the COSMIN quality standards for development and measurement properties, 5 constipation-related PROMs received at least 4 (out of 7) sufficient ratings. Only 2 PROMs were developed in Asia. Five PROMs were administered through digital methods during the validation process but methods of adapting the PROMs into digital formats were not reported.
Conclusions
The constipation-related PROMs identified in this review present varying quality of development and validation, with an overall need for improvement. Further considerations should be given towards more consistent methodology and reporting of PROM development, increase in culturally-specific PROMs, and better reporting of protocol for the digitisation of PROMs.
Keywords: Constipation; Digital health; Patient-reported outcome measures; Quality of life
Introduction

Chronic constipation is a prevalent worldwide problem that affects up to 10-15% of the adult population.1 Symptoms of primary and secondary constipation include hard stools, excessive straining, infrequent bowel movements, bloating, and abdominal pain.2 While constipation can often be managed by medication and lifestyle modification, prolonged constipation can significantly decreases quality of life (QoL).2,3 To better assess patients’ health status and QoL, it is important to have an accessible tool that can accurately assess patients’ symptoms and treatment outcomes, which may enable personalized intervention strategies. The usage of reliable and validated patient-related outcome measure (PROM) can help provide a consistent method of measuring clinical symptoms and QoL outcomes in patients.4 PROMs are standardized, validated questionnaires that measure patients’ perception of their own health status and well-being.5 While PROMs were initially developed for research use, they have been increasingly adopted in clinical practice to aid clinicians provide better and more patient-centered care.6

To date, 2 reviews have examined existing assessment scales measuring constipation symptoms.7,8 A combination of 9 self-reported measures, developed between 1989 to 2010, were assessed by both reviews. While the reviews provided an insight on the reliability and validity of existing constipation PROMs, the reviews were not conducted systematically and constipation-related QoL PROMs were not included. Given the impact of constipation on QoL, including mental, social, and physical functioning,9,10 it is important to consider QoL in treatment outcomes.

As the capabilities and adoption of digital technology expand in healthcare, it is also important for us to explore the potential of digitizing PROMs. This could sustain longitudinal patient assessment, which can further support the individualization of patient care. In patients with inflammatory bowel disease, collecting consistent electronic patient reported outcomes (ePRO) on a cloud-based digital therapeutics and monitoring application has been shown to significantly reduce yearly hospitalizations and emergency room visit rates most likely due to immediate interventions prompted by concerning questionnaire scores. Patients also reported having a better understanding of the nature and causes of their health condition after a year.11 Given the importance of incorporating QoL into treatment outcomes and the potential of incorporating digital health technologies that are patient-centric into constipation management, the current review aims to (1) systematically review constipation-related PROMs, including QoL reporting, using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN)12 guideline to evaluate the methodological quality of included studies and the quality of the measurement properties themselves, and (2) assess the current modes of digitization of constipation-related PROMs.

Methods

A systematic review protocol was developed in accordance with the Preferred Reporting for Items for Systematic Reviews and Meta-analyses (PRISMA) and the COSMIN guidelines. The study protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO) database (No. CRD42021236257).

Search Strategy

The PRISMA guidelines were used to identify studies for this review. A comprehensive literature search was performed using PubMed, Embase, and PsycINFO to identify all articles on the development or validation of constipation-related PROMs. The search was conducted up to February 2021. Searches in all 3 databases were performed using the following keywords: (constipation OR gastrointestinal) AND (question* OR [patient AND outcome AND measure]) AND (validation OR development).

Study Selection

The initial search yielded 11 011 articles after duplicates were removed. Four articles were identified via hand-checking of reference lists of published reviews and were included retrospectively. Two authors (V.V.L. and N.Y.L.) independently reviewed titles and abstracts of the identified records for preliminary inclusion. Articles were included for further screening based on inclusion criteria listed in Table 1. Five hundred and seventy-nine articles satisfied the preliminary inclusion criteria and were accepted for a full review. Interrater agreement was assessed with Cohen’s κ indicator, where κ of 0.60-0.79 was classified as “moderate,” 0.80-0.90 as “strong,” and above 0.90 as “almost perfect” interrater agreement.13 There was a moderate interrater agreement for study selection (Cohen’s κ = 0.72; 95% CI, 0.68-0.75) and discrepancies were resolved by discussion.

Table 1 . Inclusion/Exclusion Criteria for Preliminary Screening and Full-text Screening

Screening criteria
Inclusion criteria for preliminary (abstract and title) screening
1) Gastrointestinal-related
2) Developed or validated PROM or questionnaire or survey or scale
Exclusion criteria for full-text screening
1) PROM did not measure constipation symptoms or constipation-related QoL
2) Revalidated an existing questionnaire for a different language or patient population
3) Review papers or conference abstracts
4) PROM examined constipation as subset or question
5) Paediatric-related PROM
6) Not in English

PROM, patient-reported outcome measures; QoL, quality of life.



Full texts of the eligible articles were retrieved and reviewed. Articles were excluded based the exclusion criteria listed in Table 1. Five hundred and fifty-three articles did not meet the eligibility criteria and were excluded. There was strong interrater agreement for the second screening (Cohen’s κ = 0.80; 95% CI, 0.67-0.93) and discrepancies were resolved by discussion. An independent third reviewer (A.T.) was brought in when discrepancies were not resolved. Figure depicts the flow diagram of the study selection.

Figure 1. Flow diagram of study selection.

Quality Assessment

The methodological quality of the studies and the quality of the PROM itself was assessed using the COSMIN guidelines. Firstly, the COSMIN Risk of Bias checklist14 consisting of 117 questions was used to assess the methodological quality of the studies. The following measurement properties were assessed: PROM development, content validity, structural validity, internal consistency, reliability, measurement error, criterion validity, hypothesis testing for constructive validity, and responsiveness. A 4-point rating system of “very good,” “adequate,” “doubtful,” and “inadequate” was used to rate each property. The final rating was determined by taking the lowest score of an assessment area (ie, “worst score counts” principle). No rating was given if measurement property was not assessed or described.

Following that, the quality of the PROM itself was assessed using the COSMIN updated criteria for good measurement properties.12 The following psychometric properties were assessed: internal consistency, reliability, measurement error, content validity, structural validity, hypotheses testing, criterion validity, and responsiveness. Using the criteria provided, a rating of “+” for sufficient, “–” for insufficient, or “?” for indeterminate was given to each measurement property.

Two authors (V.V.L. and D.J.Y.X.) independently reviewed the included studies using the COSMIN risk of bias checklist and updated criteria for good measurement properties. There was moderate interrater agreement for risk of bias (Cohen’s κ = 0.71; 95% CI, 0.69-0.74) and almost perfect interrater agreement for criteria for good measurement properties (Cohen’s κ = 0.93; 95% CI, 0.88-0.98). Any discrepancies were resolved by discussion.

Results

Summary of Included Studies

A total of 23 PROMs measuring constipation symptoms15-35 or constipation-related QoL36-40 were identified. The PROMs were reported in 26 different studies with publication years ranging from 1989 to 2020. The Bowel Function Index and Patient Assessment of Constipation–Symptom (PAC-SYM) had more than 1 validation study with additional information on measurement properties. A summary of included studies is presented in Table 2.

Table 2 . Characteristics of Included Studies

PROMAuthorsType of constipationParticipants
(validation study)
Method of administrationRecall periodItems and subsetsLocation of studyOriginal language
Constipation symptoms
BF-DiaryCamilleri et al,15 2011OIC238 patients with chronic painHandheld electronic PDA devicePast 24 hours and immediately after each defecation event for some items10 items (3 modules per bowel movement, daily assessment, and treatments used)USAEnglish
BFIRentz et al,16 2009OIC985 patients with chronic pain (202 Phase II; 460 Phase III; 323 Phase III)Clinician administeredLast 7 days3 itemsGermanyEnglish
Ducrotté and Caussé,17 2012OIC987 patients with chronic pain (202 Phase II, 463 Phase III, 322 Phase III)Clinician administeredLast 7 days3 itemsFranceEnglish
Abramowitz et al,18 2013OIC520 patients with pain induced by cancerClinician administeredLast 7 days3 itemsFranceEnglish
Chinese Constipation QuestionnaireChan et al,19 2005Functional constipation221 (111 patients with chronic constipation, 110 healthy controls)Pen and paperPast 2 weeks6 itemsHong KongChinese
CC Symptom Severity MeasuresNelson et al,20 2014Chronic constipation1579 with chronic constipation (307 Phase IIb; 1272 Phase III)Telephone-based IVRSAdministered daily during trial7 items (2 subscales: bowel and abdominal symptoms)USAEnglish
CASMcMillan and Williams,21 1989Iatrogenic constipation64 (32 at risk for constipation, 32 healthy controls)Pen and paperPast week8 itemsUSAEnglish
Constipation during pregnancy questionnairePonce et al,22 2008Constipation during pregnancy207 pregnant womenPen and paper, telephone interview (follow ups)None reported42 items (5 subscales: demographic, obstetric history, bowel habits and laxative consumption, lifestyle and eating habits, and urinary or fecal incontinence)SpainNot reported
CSIVarma et al,23 2008Multiple subtypes294 (191 patients with constipation, 103 healthy controls)Pen and paperLast month (2 items), no recall period (14 items)16 items (3 subscales: obstructive defecation, colonic inertia, and pain)USAEnglish
CSSAgachan et al,24 1996Idiopathic constipation232 patients with constipationPen and paperNone reported8 itemsUSAEnglish
DIBSS-CCoon et al,25 2020IBS-C532 IBS-C patients from multicentre phase IIb studyHandheld electronic diaryEvery BM event (3 items), past 24 hours (3 items)7 items (2 subcales: bowel movement-related symptoms and abdominal symptoms severity)USAEnglish
FICABharucha et al,26 2004Functional constipation & faecal incontinence83 womenPen and paperLast 12 months (symptoms-related items)98 items (8 subscales: general bowel habits, abdominal pain, treatment of constipation, faecal incontinence, urinary symptoms, anorectal disease, surgical history and other, health care utilization, and quality of life)USAEnglish
Fecal Incontinence and Constipation QuestionnaireÖsterberg et al,27 1996Constipation & faecal incontinence90 (36 with faecal incontinence, 38 with constipation, 16 healthy controls)Pen and paperNone reported38 itemsSwedenNot reported
IBS-C Symptom Severity MeasuresWilliams et al,28 2014IBS-C1608 patients with IBS-CTelephone-based IVRSAdministered daily during trial9 items (2 subscales: bowel and abdominal symptoms subsets)USAEnglish
KESSKnowles et al,29 2000Chronic constipation91 (71 patients with chronic constipation, 20 healthy controls)Researcher administeredNone reported11 itemsUKEnglish
ODS-SRenzi et al,30 2013ODS200 (100 patients with ODS, 100 healthy controls)Pen and paperNone reported5 itemsItalyNot reported
ODS ScoreAltomare et al,31 2008ODS106 (76 patients with ODS, 30 healthy controls)Researcher administeredNone reported8 itemsItalyNot reported
PAC-SYMFrank et al,32 1999Chronic constipation216 patients with chronic constipationPen and paperPast 2 weeks12 items (3 subscales: stool, rectal, and abdominal symptoms)USAEnglish
Modified PAC-SYMNeri et al,33 2015Chronic constipation2203 patients with chronic constipationPen and paperPast 2 weeks11 items (2 subscales: stool and abdominal symptoms)ItalyEnglish
Rome III Criteria QuestionnaireDigesu et al,34 2010Functional constipation (Rome III)201 womenPen and paperLast 3 months (9 items), last 6 months (2 items), no recall period (6 items)17 itemsUKEnglish
VSAQPamuk et al,35 2003Constipation (unspecified)369 hospital personnelPen and paperLast 12 months (symptoms-related items)5 itemsTurkeyNot reported
Constipation-related quality of life
CTSAT-QSzeinbach et al,36 2009Chronic constipation and IBS-C311 patients with chronic constipation and IBS-COnline questionnaireNone reported12 items (5 subscales: expectations, value, treatment satisfaction, activities, and effectiveness)USAEnglish
Constipation-related Disability ScaleHart et al,37 2012Chronic constipation343 (240 patients with constipation, 103 healthy controls)Pen and paperPast week13 items (2 subscales: leisure/work activities and activities of daily living)USAEnglish
CRQOLWang et al,38 2009Chronic constipation343 (240 patients with constipation, 103 healthy controls)Pen and paperPast 12 months18 items (4 subscales: social impairment, distress, eating habits, and bathroom attitudes)USAEnglish
E-CISAbdul Wahab et al,39 2020Chronic constipation470 elderly people with chronic constipationResearcher administeredNone reported22 items (7 subscales: daily activities, treatment satisfaction, lack of control of bodily function, diet restriction, symptom intensity, anxiety, and preventive actions)MalaysiaMalay
PAC-QOLMarquis et al,40 2005Chronic constipation223 patients with chronic constipationPen and paperPast 2 weeks28 items (4 subscales: worries/concerns, physical discomfort, psychosocial discomfort, and satisfaction)USA, UK, France, SwedenEnglish

PROM, patient-reported outcome measure; BF-Diary, Bowel Function Diary; BFI, Bowel Function Index; CC, chronic constipation; CAS, Constipation Assessment Scale; CSI, Constipation Severity Instrument; CSS, Constipation Scoring System; DIBSS-C, Diary for Irritable Bowel Syndrome Symptoms–Constipation; FICA, Fecal Incontinence and Constipation Assessment; IBS-C, constipation-predominant irritable bowel syndrome; KESS, Knowles-Eccersley-Scott Symptom Questionnaire; ODS-S, Obstructive Defecation Syndrome Score; PAC-SYM, Patient Assessment of Constipation–Symptom; VSAQ, Visual Scale Analog Questionnaire; CTSAT-Q, Chronic Constipation Treatment Satisfaction Questionnaire; CRQOL, Constipation-related Quality of Life; E-CIS, Elderly-constipation Impact Scale; PAC-QOL, Patient Assessment of Constipation–Quality of Life; OIC, opioid-induced constipation; PDA, personal digital assistant; IVRS, interactive voice response system; USA, United States of America; UK, United Kingdom.

Method of administration was assumed to be pen and paper unless reported otherwise.



Patient-related Outcome Measures Measuring Constipation Symptoms

Eighteen out of the 23 identified PROMs evaluated constipation symptoms. The majority of the PROMs (n = 10) were developed with intentions to assess severity of constipation in patients.16,19,21,23,24,26,27,30-32 Seven PROMs were developed as a potential diagnosis tool to detect clinically significant constipation, with some having dual functionality for diagnosis and measurement of severity.19,21,27,29,30,34,35 A subset of PROMs (n = 4) were created and/or validated for research purposes, specifically to assess treatment benefits in patients during varying stages of clinical trials.15,20,25,28

The items included in the PROMs can be categorised into 5 categories: abdominal symptoms, bowel movement-related symptoms, stool-related symptoms, anal or rectal symptoms, and others. The most common questionnaire items were incomplete evacuation during bowel movement15,16,19,22-24,26,27,29-32,34,35 and stool consistency,15,19-22,25-29,31,32,34,35 with both items included in 77.8% of PROMs assessed. To measure incomplete bowel movement, PROMs have included a combination of frequency and/or severity related questions. To measure stool consistency, questions include rating of consistency based on the 7-point Bristol stool form scale15,20,28 or a self-constructed scale,27,29,31,35 frequency or severity of hard or lumpy stools,19,32,34 and presence of hard and loose/water stools.21,22,26

More than half of the studies included a measure of abdominal pain,15,20,24-28,30,32,34 abdominal bloating,15,19-21,25-29,32 frequency of bowel movements,19-29,34,35 and straining during bowel movement.15,20,22,23,25-28,30-32,34,35 Abdominal pain, abdominal bloating, and straining during bowel movement have been measured through both severity and/or frequency while questions on frequency of bowel movements have generally been measured using a self-constructed time scale. Two PROMs differentiated complete spontaneous bowel movement from spontaneous bowel movement.20,28 A summary of questionnaire items in PROMs measuring constipation symptoms is presented in Table 3.

Table 3 . Summary of Questionnaire Items in Patient-reported Outcome Measures Measuring Constipation Symptoms

Questionnaire itemsn (out of 18)%
Overall rating for constipation422.2
Abdominal symptoms
Pain1055.6
Bloating1055.6
Discomfort633.3
Gas316.7
Cramping211.1
Distention15.6
Fullness15.6
Pressure during defecation15.6
Bowel movement-related symptoms
Incomplete evacuation1477.8
Frequency1372.2
Straining1372.2
Inability to pass738.9
Ease/pain during bowel movement527.8
Urgency211.1
Attempts a day15.6
Lack of urge15.6
Stool-related symptoms
Consistency1477.8
Amount316.7
Anal/rectal symptoms
Pain422.1
Bleeding211.1
Anus blockage211.1
Burning15.6
Fullness/pressure15.6
Pruritus ani15.6
Others
Use of laxatives/enemas950.0
Use of digital manoeuvres844.4
Time spent in toilet527.8
History (duration of constipation)422.2
Lack of appetite15.6
Changes to diet15.6


Patient-related Outcome Measures Measuring Constipation-related Quality of Life

Five out of the 23 PROMs assessed constipation-related QoL. Three PROMs were developed to measure the impact of constipation on multiple aspects of QoL, including social relationships, treatment satisfaction, physical symptoms, diet, daily activities, and psychological state.38-40 While all 3 PROMs are suitable for patients with chronic constipation, the Elderly-constipation Impact Scale (E-CIS) was developed for elderly Malay speaking individuals aged 60 years and above.39

Two PROMs evaluated specific aspects of QoL in patients with constipation. The Chronic Constipation Treatment Satisfaction Questionnaire (CTSAT-Q) specifically focused on treatment satisfaction in patients with chronic constipation and constipation-predominant irritable bowel syndrome (IBS-C).36 Items include patient’s expectations on and attitude towards medication, value of medication, interference due to treatment, and effectiveness of treatment. On the other hand, the Constipation-related Disability Scale37 focused on the impact of constipation symptoms on day-to-day activities. The PROM includes a rating of difficulty in performing various leisure, work, and daily activities.

COnsensus-based Standards for the Selection of Health Measurement INstruments Risk of Bias

The COSMIN risk of bias assessment demonstrated very few studies with consistent “very good” and/or “adequate” ratings across all domains. A summary of risk of bias scores for each study are presented in Table 4. Cross-cultural validity was not assessed as the current review only included studies that assessed the original version of the PROM.

Table 4 . Individual PAC-SYM Category Scores for Each Study as Assessed by the COnsensus-based Standards for the Selection of Health Measurement INstruments Risk of Bias Checklist14

PROMAuthorsPROM developmentContent validityStructural validityInternal consistencyReliabilityMeasurement errorCriterion validityConstruct validityResponsive-ness
Constipation symptoms
BF-DiaryCamilleri et al,15 2011DoubtfulDoubtfulAdequateAdequateDoubtfulDoubtful
BFIRentz et al,16 2009InadequateVery goodDoubtfulInadequateDoubtfulDoubtful
Ducrotté and Caussé,17 2012Very goodDoubtfulDoubtfulDoubtful
Abramowitz et al,18 2013Very goodDoubtful
Chinese Constipation QuestionnaireChan et al,19 2005DoubtfulDoubtfulDoubtfulVery goodDoubtfulVery goodInadequate
CC Symptom Severity MeasuresNelson et al,20 2014InadequateVery goodDoubtfulDoubtfulDoubtful
CASMcMillan & Williams,21 1989InadequateVery goodInadequateAdequate
Constipation during pregnancy questionnairePonce et al,22 2008InadequateVery good
CSIVarma et al,23 2008DoubtfulDoubtfulVery goodVery goodAdequateVery good
CSSAgachan et al,24 1996InadequateDoubtful
DIBSS-CCoon et al,25 2020DoubtfulDoubtfulAdequateVery goodAdequateDoubtfulDoubtfulDoubtful
FICABharucha et al,26 2004InadequateDoubtfulDoubtful
Fecal Incontinence and Constipation QuestionnaireÖsterberg et al,27 1996InadequateInadequateInadequate
IBS-C Symptoms Severity MeasuresWilliams et al,28 2014InadequateVery goodAdequateDoubtfulDoubtful
KESSKnowles et al,29 2000InadequateDoubtful
ODS-SRenzi et al,30 2013DoubtfulDoubtfulDoubtfulDoubtfulVery goodDoubtful
ODS ScoreAltomare et al,31 2008InadequateDoubtfulDoubtful
PAC-SYMFrank et al,32 1999DoubtfulDoubtfulAdequateVery goodAdequateAdequateInadequate
Modified PAC-SYMNeri et al,33 2015Very goodVery goodDoubtfulDoubtful
Rome III Criteria QuestionnaireDigesu et al,34 2010InadequateDoubtfulVery goodAdequateInadequate
VSAQPamuk et al,35 2003InadequateInadequate
Constipation-related quality of life
CTSAT-QSzeinbach et al,36 2009DoubtfulDoubtfulVery goodVery good
Constipation-related Disability ScaleHart et al,37 2012InadequateVery goodVery goodDoubtfulVery good
CRQOLWang et al,38 2009DoubtfulDoubtfulVery goodVery goodDoubtfulVery good
E-CISAbdul Wahab et al,39 2020DoubtfulDoubtfulVery goodVery good
PAC-QOLMarquis et al,40 2005InadequateAdequateVery goodDoubtfulInadequateDoubtful

PROM, patient-reported outcome measure; BF-Diary, Bowel Function Diary; BFI, Bowel Function Index; CC, chronic constipation; CAS, Constipation Assessment Scale; CRQOL, Constipation-Related Quality of Life; CSI, Constipation Severity Instrument; CSS, Constipation Scoring System; CTSAT-Q, Chronic Constipation Treatment Satisfaction Questionnaire; DIBSS-C, Diary for Irritable Bowel Syndrome Symptoms–Constipation; E-CIS, Elderly-Constipation Impact Scale; FICA, Fecal Incontinence and Constipation Assessment; IBS-C, constipation-predominant irritable bowel syndrome; KESS, Knowles-Eccersley-Scott Symptom Questionnaire; PAC-SYM, Patient Assessment of Constipation–Symptom; ODS-S, Obstructive Defecation Syndrome Score; PAC-QOL, Patient Assessment of Constipation–Quality of Life; VSAQ, Visual Scale Analog Questionnaire.



Twenty-three studies were rated on PROM development and the majority of the studies (n = 14) scored “inadequate” due to the lack of a PROM development study involving the target population or a cognitive interview study to assess the comprehensibility or comprehensiveness of the PROM. The remaining 9 studies scored “doubtful” due to poor reporting of study methods including the use of skilled group moderators or interviewers, interview guides, recording and transcription process of interviews and independent coding of data. Poor reporting of methods similarly resulted in “doubtful” ratings for content validity. Only 2 studies19,38 comprehensively examined content validity (ie, asking patients and professionals about relevance, comprehensibility, and comprehensiveness).

Construct validity was the most common measurement properties analysed (n = 23), nevertheless, not all studies examined both convergent and discriminative validity. Four studies15,21,24,31 only examined discriminative validity while 2 studies26,35 only examined convergent validity. Half of the studies that examined construct validity scored “doubtful” due to the lack of detailed description of comparator instruments and/or important characteristics of subgroups.

Following construct validity, reliability (n = 17), and internal consistency (n = 16) were the second and third most analyzed measurement properties. The majority of studies that scored “doubtful” and “inadequate” for reliability did not fulfill appropriate design requirements (eg, patients’ stability in the interim period, similarity of test conditions, and appropriate time interval). Most of the studies that analyzed internal consistency fulfilled the COSMIN criteria for “very good.” “Doubtful” ratings for internal consistency were given due to lack of clarity if scale or subscale was unidimensional.

Less than half of the studies analyzed structural validity, measurement error, criterion validity, and responsiveness. Studies that examined structural validity mostly scored “very good” and “adequate.” Only 1 study scored “doubtful” for structural validity due to the lack of description of rotation method. Two studies scored “doubtful” and “inadequate” for measurement error due to unclear description on stability of patients in the interim period and inadequate calculations of standard error of measurement. All studies that examined responsiveness only examined comparison between subgroups accordingly, ratings were based on that aspect. Scores of “doubtful” and “inadequate” for responsiveness were due to poor description of important characteristics of subgroups and inadequate statistical methods applied.

COnsensus-based Standards for the Selection of Health Measurement INstruments Rating of Measurement Properties

Due to the limited amount of validation studies per PROM, the studies were assessed individually and the total ratings were not provided. The individual ratings for each measurement property of all the studies are presented in Table 5.

Table 5 . Individual Rating for Each Measurement Properties Based on the Updated COnsensus-based Standards for the Selection of Health Measurement INstruments Criteria12

PROMAuthorsStructural validityInternal consistencyReliabilityMeasurement errorCriterion validityConstruct validityResponsive-ness
Constipation symptoms
BF-DiaryCamilleri et al,15 2011?++
BFIRentz et al,16 2009+?+?
Ducrotté and Caussé,17 2012+???
Abramowitz et al,18 2013+
Chinese Constipation QuestionnaireChan et al,19 2005?++++
CC Symptom Severity MeasuresNelson et al,20 2014++?
CASMcMillan and Williams,21 1989+?+
Constipation during pregnancy questionnairePonce et al,22 2008?
CSIVarma et al,23 2008++++
CSSAgachan et al,24 1996?
DIBSS-CCoon et al,25 2020?+?+?
FICABharucha et al,26 2004?
Fecal Incontinence and Constipation QuestionnaireÖsterberg et al,27 1996?+
IBS-C Symptom Severity MeasuresWilliams et al,28 2014?++?
KESSKnowles et al,29 2000+
ODS-SRenzi et al,30 2013+?++
ODS ScoreAltomare et al,31 2008+
PAC-SYMFrank et al,32 1999?++++
Modified PAC-SYMNeri et al,33 2015+++
Rome III Criteria QuestionnaireDigesu et al,34 2010++
VSAQPamuk et al,35 2003?
Constipation-related quality of life
CTSAT-QSzeinbach et al,36 2009++
Constipation-related Disability ScaleHart et al,37 2012++++
PAC-QOLWang et al,38 2009++++
E-CISAbdul Wahab et al,39 2020+
CRQOLMarquis et al,40 2005?+??

PROM, patient-reported outcome measure; BF-Diary, Bowel Function Diary; BFI, Bowel Function Index; CC, chronic constipation; CAS, Constipation Assessment Scale; CRQOL, Constipation-Related Quality of Life; CSI, Constipation Severity Instrument; CSS, Constipation Scoring System; CTSAT-Q, Chronic Constipation Treatment Satisfaction Questionnaire; DIBSS-C, Diary for Irritable Bowel Syndrome Symptoms–Constipation; E-CIS, Elderly-Constipation Impact Scale; FICA, Fecal Incontinence and Constipation Assessment; IBS-C, constipation-predominant irritable bowel syndrome; KESS, Knowles-Eccersley-Scott Symptom Questionnaire; PAC-SYM, Patient Assessment of Constipation–Symptom; ODS-S, Obstructive Defecation Syndrome Score; PAC-QOL, Patient Assessment of Constipation–Quality of Life; VSAQ, Visual Scale Analog Questionnaire.

Ratings for measurement properties: +, sufficient; ?, indeterminate; −, insufficient.



Twenty-one studies had at least 1 insufficient (–) or indeterminate (?) rating, and no PROM was fully assessed in all measurement properties, with measurement error and criterion validity most commonly missing. The PROMs with the most sufficient (+) ratings include the Chinese Constipation Questionnaire, Constipation Severity Instrument (CSI), PAC-SYM, Constipation-related Disability Questionnaire, and Patient Assessment of Constipation–Quality of Life (PAC-QOL). To improve ratings of measurements properties, focus should be given to obtaining sufficient rating for measurement error (smallest detectable change/limits of agreement < minimal important change), criterion validity (correlation with gold standard or area under the curve ≥ 0.70), and responsiveness (results in accordance with hypothesis or area under the curve ≥ 0.70).

Digitization of Patient-related Outcome Measures

Five studies reported using digital formats to administer PROMs during the validation process.15,20,25,28,36 Both the Bowel Function Diary and Diary for Irritable Bowel Syndrome Symptoms-Constipation (DIBSS-C) were completed as part of an electronic diary and were administered using a handheld device. For the Bowel Function Diary, participants were given a handheld electronic personal digital assistant (PDA) device while the type of device was not specified for the DIBSS-C. The CTSAT-Q was described to be disseminated online however, no further information was provided.

The Chronic Constipation Symptom Severity Measures and IBS-C Symptom Severity Measures were administered using interactive voice response system technology, a computer-automated telephone system that collects data through spoken answers or keypad responses.41 For all 5 studies, methods of digitizing and validating the digital formats of the PROMs were not reported.

Discussion

Digitizing constipation-related PROMs represents a promising step towards individualizing patient intervention in a longitudinal and scalable manner. Therefore, the current systematic review provides an overview of constipation-related PROMs that have been developed and validated over the past 32 years. The review identified 23 different constipation-related PROMs, with 18 measuring symptom-related measures and 5 measuring constipation-related QoL measures.

The review revealed a large amount of variation between PROMs used to measure symptom-related constipation outcomes. Variations include outcome measures targeting different subtypes of constipation (eg, opioid-induced constipation, obstructive defecation syndrome, and IBS-C), functions of PROM (eg, clinical use and research purposes), methods of administration (eg, pen and paper, clinician administered, and electronic diary), length of PROM (range = 3-98 items), and recall period (eg, last 2 weeks and past 24 hours). Given the multiple possible etiologies of constipation, subtype-specific PROMs can be a useful tool to further facilitate customization of outcome measures monitoring, and to provide more accurate feedback to clinicians about treatment progress. Nevertheless, the heterogeneity of variables between PROMs can indicate the lack of standardization during the PROM development process. For instance, multiple studies scored “inadequate” for PROM development based on the COSMIN checklist due to the lack of patients’ involvement and input when developing PROM items. While physical symptoms and functioning are vital aspects of disease monitoring, patients may be more focused on regaining or preserving QoL, including emotional wellbeing and social functioning.42

Considering the known impact of constipation on QoL,9 the review also examined constipation-specific QoL-related PROMs. Similar to studies that developed symptom-related outcomes measures, none of the QoL-related PROMs reviewed met all the COSMIN quality standards for development and measurement properties. While all 5 of QoL PROMs involved patients in the development process through individual interviews or focus groups, issues include incomplete reporting of interview methods and lack of a follow-up cognitive testing session. Based on the ratings of measurement properties, out of the reviewed PROMs, the current review recommends the Chinese Constipation Questionnaire, CSI, PAC-SYM, Constipation-related Disability Questionnaire, and PAC-QOL to measure constipation symptoms or constipation-related quality of life. Nevertheless, none of the reviewed PROMs report all measurement properties indicated in the COSMIN checklist thus, there is a need for better standardization of PROM creation, from the development stages to the final reporting of validation studies. In their efforts to better regulate PROM creation and usage, the Food and Drug Administration (FDA) released a guidance for the industry on recommendations for PROM development and validation,43 and more recently, a draft on selecting, developing or adapting PROMs for medical device evaluation.44 To ensure more consistent methodological quality and reporting, it would be ideal for future PROM development studies to familiarize themselves with the FDA recommendations and COSMIN guidelines.

Despite the variety of PROMs identified in this review, only 2 PROMs were developed and validated within the Asian context. Chan et al19 developed and validated the Chinese Constipation Questionnaire in the Chinese language with an ethnic Chinese population. Similarly, Abdul Wahab et al39 developed and validated the E-CIS using the Malay language spoken in the local dialects of Terengganu and Kelantan in Malaysia. While there are translated versions of questionnaires including the PAC-SYM32 and PAC-QOL,40 culture and language are intertwined, and language should be examined in conjunction with culturally specific health beliefs and understanding.45 Regardless of English fluency, patients of different ethnic groups may differ in terms of pronunciation, speech delivery, grammar/vocabulary and culturally specific presentation styles when describing their issues to medical practitioners.45 Hence, a relatable and culturally-specific PROM can be beneficial in increasing the efficacy of patient-clinician communication, and further facilitate a more personalised, patient-centric symptoms monitoring and treatment.

The current review also assessed the modes of digital dissemination of currently available PROMs and identified 5 PROMs that used digital formats to administer the questionnaire during the validation process. Methods of dissemination were varied, ranging from electronic diary formats on PDA devices to computer-automated telephone systems. Given the widespread adoption of smart devices, such as smartphones and tablets, the use of ePROs presents as a viable option for remote monitoring led by patients themselves. Consistent symptom reporting through digital means can improve patient-clinician communication, detection of unrecognised problems, and patients’ health behaviors, including patient self-management and patient empowerment.46 Nevertheless, to ensure reliable reporting of ePROs, evidence is needed to support measurement equivalence between the electronic and paper-based PROMs.47 The 5 studies in the current review that utilized digital PROMs did not report methods undertaken to digitize the PROMs. Accordingly, there is a need for better standardization for digitization of PROMs to maximize the potential of ePRO tools. Recommendations from the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) ePRO Task Force include cognitive debriefing, usability testing or full psychometric testing of the electronic versions, depending on the level of modification. Future ePRO development studies can benefit by reporting level of modification and relevant methods undertaken to ensure measurement equivalence.

With growing interests to integrate technology into healthcare, PROM development and implementation should keep pace with the fast and evolving field of digital health. Digital health has the potential to offer new modalities of probing patient state in real time through minimally invasive methods (eg, experience sampling method, day reconstruction method). Furthermore, increasing accessibility to ePROs can open doors to personalization of PROMs. A recent real-world longitudinal study of patients with rheumatic and musculoskeletal diseases allowed patients to customize their tracking on the Arthritis Power mobile application between 3 and 10 PRO symptom measures over a period of 3 months.48 While some PROM items were prioritized more, there were variations between patients in the items chosen for tracking and in ranking of importance. It should also be noted that minimal changes to items tracked were observed, suggesting that patients continue to only track symptoms that are important to them. The ePROs can further benefit from concepts commonly employed in digital health, such as gamification and behavioral nudges, to sustain users’ engagement.49 As the concept of personalized medicine continues to grow, digital technologies can aid in the continual evolution and optimization of PROMs.

A limitation of the current systematic review is the subjective nature of the COSMIN evaluation methods. Multiple items of the COSMIN checklist require subjective judgement of the reviewer based on experience and knowledge, hence, there is possibility of subjectivity within the review.50 Furthermore, we acknowledge that there may be other methods to assess psychometric measures beyond the COSMIN guidelines. Nevertheless, the current review endeavored to reduce subjectivity by utilizing 2 independent reviewers with good interrater reliability and a third for any discrepancies. Secondly, the current review did not include PROMs assessing the pediatrics population. Constipation-related pediatric PROMs rely on patient-reported measures, parent/caregiver-reported measures or a combination of both, and assessing the differences between the method of reporting is beyond the scope of the current review. Poor concordance between parent- and child-reporting have been observed when assessing gastrointestinal symptoms. For instance, children tended to rate their pain/discomfort intensity more severely than parents did, and up to 60% of parents of 10- to 19-year-olds could not answer items relating to defecation habits.51 Therefore, it would be beneficial for future studies to focus on the differences in reporting methods and the involvement of both parent and child in the PROM development process.

In conclusion, this review assessed constipation symptoms and constipation-related QoL PROMs using the COSMIN guidelines and identified a lack of consistent methodology and reporting of development and validation studies. Furthermore, more culturally-specific PROMs, especially in the Asian context, will be beneficial. There are varying modes of digital dissemination of constipation-related PROMs however, greater standardization of the process is required to ensure transparency and consistency. As PROM is a useful tool that can provide clinicians and researchers insights into patients’ health status and health-related QoL, further developments of constipation-related PROMs can be made by more consistent methodology and reporting of PROM development, increase in culturally-specific PROMs, and better reporting of protocol for digitization of PROMs.

Financial support

This study was supported funding from the Institute for Digital Medicine (WisDM) Translational Research Programme (Grant No. R-719-000-037-733) at the Yong Loo Lin School of Medicine, National University of Singapore (Dean Ho).

Conflicts of interest

Agata Blasiak and Dean Ho are co-inventors or previously filed pending patents on artificial intelligence-based therapy development. Dean Ho is a shareholder of KYAN Therapeutics, which has licensed intellectual property pertaining to artificial intelligence -based drug development. Others have nothing to disclose.

Author contributions

V Vien Lee: development of the parameters of the systematic review, review of articles for eligibility and quality, data analysis, and preparation of the manuscript; Ni Yin Lau: review of articles for eligibility and editing of the manuscript; David J Y Xi: review of articles for quality and editing of the manuscript; Anh T L Truong: review of articles for eligibility and editing of the manuscript; Agata Blasiak: development of the parameters of the systematic review and editing of the manuscript; Kewin T H Siah: evaluation of quality and editing of the manuscript; and Dean Ho: evaluation of quality and editing of the manuscript.

References
  1. Barberio B, Judge C, Savarino EV, Ford AC. Global prevalence of functional constipation according to the Rome criteria: a systematic review and meta-analysis.
    Pubmed CrossRef
  2. Camilleri M, Ford AC, Mawe GM, et al. Chronic constipation. Nat Rev Dis Primers 2017;3:17095.
    Pubmed CrossRef
  3. Belsey J, Greenfield S, Candy D, Geraint M. Systematic review: impact of constipation on quality of life in adults and children. Aliment Pharmacol Ther 2010;31:938-949.
    Pubmed CrossRef
  4. Almario CV, Spiegel BMR. Employing Irritable Bowel Syndrome Patient-Reported Outcomes in the Clinical Trenches. Clin Gastroenterol Hepatol 2018;16:462-466.e462.
    Pubmed CrossRef
  5. Dawson J, Doll H, Fitzpatrick R, Jenkinson C, Carr AJ. The routine use of patient reported outcome measures in healthcare settings. Bmj 2010;340:c186.
    Pubmed CrossRef
  6. Black N. Patient reported outcome measures could help transform healthcare. Bmj 2013;346:f167.
    Pubmed CrossRef
  7. Izumi K. The measures to evaluate constipation: a review article. Gastroenterol Nurs 2014;37:137-146.
    Pubmed CrossRef
  8. McCrea GL, Miaskowski C, Stotts NA, Macera L, Hart SA, Varma MG. Review article: self-report measures to evaluate constipation. Aliment Pharmacol Ther 2008;27:638-648.
    Pubmed CrossRef
  9. Dennison C, Prasad M, Lloyd A, Bhattacharyya SK, Dhawan R, Coyne K. The health-related quality of life and economic burden of constipation. Pharmacoeconomics 2005;23:461-476.
    Pubmed CrossRef
  10. Wald A, Scarpignato C, Kamm MA, et al. The burden of constipation on quality of life: results of a multinational survey. Aliment Pharmacol Ther 2007;26:227-236.
    Pubmed CrossRef
  11. Zhen J, Marshall JK, Nguyen GC, Atreja A, Narula N. Impact of Digital Health Monitoring in the Management of Inflammatory Bowel Disease. J Med Syst 2021;45:23.
    Pubmed KoreaMed CrossRef
  12. Prinsen CAC, Mokkink LB, Bouter LM, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 2018;27:1147-1157.
    Pubmed KoreaMed CrossRef
  13. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012;22:276-282.
    Pubmed KoreaMed CrossRef
  14. Mokkink LB, de Vet HCW, Prinsen CAC, et al. COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures. Qual Life Res 2018;27:1171-1179.
    Pubmed KoreaMed CrossRef
  15. Camilleri M, Rothman M, Ho KF, Etropolski M. Validation of a bowel function diary for assessing opioid-induced constipation. Am J Gastroenterol 2011;106:497-506.
    Pubmed CrossRef
  16. Rentz AM, Yu R, Müller-Lissner S, Leyendecker P. Validation of the Bowel Function Index to detect clinically meaningful changes in opioid-induced constipation. J Med Econ 2009;12:371-383.
    Pubmed CrossRef
  17. Ducrotté P, Caussé C. The Bowel Function Index: a new validated scale for assessing opioid-induced constipation. Curr Med Res Opin 2012;28:457-466.
    Pubmed CrossRef
  18. Abramowitz L, Béziaud N, Caussé C, Chuberre B, Allaert FA, Perrot S. Further validation of the psychometric properties of the Bowel Function Index for evaluating opioid-induced constipation (OIC). J Med Econ 2013;16:1434-1441.
    Pubmed CrossRef
  19. Chan AO, Lam KF, Hui WM, et al. Validated questionnaire on diagnosis and symptom severity for functional constipation in the Chinese population. Aliment Pharmacol Ther 2005;22:483-488.
    Pubmed CrossRef
  20. Nelson LM, Williams VS, Fehnel SE, et al. Psychometric validation of patient-reported outcome measures assessing chronic constipation. Clin Exp Gastroenterol 2014;7:385-394.
    Pubmed KoreaMed CrossRef
  21. McMillan SC, Williams FA. Validity and reliability of the Constipation Assessment Scale. Cancer Nurs 1989;12:183-188.
    Pubmed CrossRef
  22. Ponce J, Martínez B, Fernández A, et al. Constipation during pregnancy: a longitudinal survey based on self-reported symptoms and the Rome II criteria. Eur J Gastroenterol Hepatol 2008;20:56-61.
    Pubmed CrossRef
  23. Varma MG, Wang JY, Berian JR, Patterson TR, McCrea GL, Hart SL. The constipation severity instrument: a validated measure. Dis Colon Rectum 2008;51:162-172.
    Pubmed CrossRef
  24. Agachan F, Chen T, Pfeifer J, Reissman P, Wexner SD. A constipation scoring system to simplify evaluation and management of constipated patients. Dis Colon Rectum 1996;39:681-685.
    Pubmed CrossRef
  25. Coon CD, Hanlon J, Abel JL, Lundy JJ, Carson RT, Reasner DS. Psychometric Analysis of the Abdominal Score From the Diary for Irritable Bowel Syndrome Symptoms-Constipation Using Phase IIb Clinical Trial Data. Value Health 2020;23:362-369.
    Pubmed CrossRef
  26. Bharucha AE, Locke GR, Seide BM, Zinsmeister AR; 3rd. A new questionnaire for constipation and faecal incontinence. Aliment Pharmacol Ther 2004;20:355-364.
    Pubmed CrossRef
  27. Österberg A, Graf W, Karlbom U, Påhlman L. Evaluation of a questionnaire in the assessment of patients with faecal incontinence and constipation. Scand J Gastroenterol 1996;31:575-580.
    Pubmed CrossRef
  28. Williams VS, Nelson LM, Fehnel SE, et al. Psychometric validation of symptom severity measures in irritable bowel syndrome with constipation. Aliment Pharmacol Ther 2014;40:298-308.
    Pubmed CrossRef
  29. Knowles CH, Eccersley AJ, Scott SM, Walker SM, Reeves B, Lunniss PJ. Linear discriminant analysis of symptoms in patients with chronic constipation: validation of a new scoring system (KESS). Dis Colon Rectum 2000;43:1419-1426.
    Pubmed CrossRef
  30. Renzi A, Brillantino A, Di Sarno G; d'Aniello F. Five-item score for obstructed defecation syndrome: study of validation. Surg Innov 2013;20:119-125.
    Pubmed CrossRef
  31. Altomare DF, Spazzafumo L, Rinaldi M, Dodi G, Ghiselli R, Piloni V. Set-up and statistical validation of a new scoring system for obstructed defaecation syndrome. Colorectal Dis 2008;10:84-88.
    Pubmed CrossRef
  32. Frank L, Kleinman L, Farup C, Taylor L, Miner P; Jr. Psychometric validation of a constipation symptom assessment questionnaire. Scand J Gastroenterol 1999;34:870-877.
    Pubmed CrossRef
  33. Neri L, Conway PM, Basilisco G. Confirmatory factor analysis of the Patient Assessment of Constipation-Symptoms (PAC-SYM) among patients with chronic constipation. Qual Life Res 2015;24:1597-1605.
    Pubmed CrossRef
  34. Digesu GA, Panayi D, Kundi N, Tekkis P, Fernando R, Khullar V. Validity of the Rome III Criteria in assessing constipation in women. Int Urogynecol J 2010;21:1185-1193.
    Pubmed CrossRef
  35. Pamuk ON, Pamuk GE, Celik AF. Revalidation of description of constipation in terms of recall bias and visual scale analog questionnaire. J Gastroenterol Hepatol 2003;18:1417-1422.
    Pubmed CrossRef
  36. Szeinbach SL, Baran RW, Bratten J, Jones MP. Psychometric development and validation of the chronic constipation treatment satisfaction questionnaire (CTSAT-Q). Value Health 2009;12:1004-1010.
    Pubmed CrossRef
  37. Hart SL, Albiani JJ, Crangle CJ, Torbit LA, Varma MG. Development and assessment of the constipation-related disability scale. Aliment Pharmacol Ther 2012;35:183-192.
    Pubmed CrossRef
  38. Wang JY, Hart SL, Lee J, Berian JR, McCrea GL, Varma MG. A valid and reliable measure of constipation-related quality of life. Dis Colon Rectum 2009;52:1434-1442.
    Pubmed CrossRef
  39. Abdul Wahab P, Mohd Yusoff D, Abdul Kadir A, Ali SH, Lee YY, Kueh YC. Psychometric evaluation of a newly developed Elderly-Constipation Impact Scale. PeerJ 2020;8:e8581.
    Pubmed KoreaMed CrossRef
  40. Marquis P, De La Loge C, Dubois D, McDermott A, Chassany O. Development and validation of the Patient Assessment of Constipation Quality of Life questionnaire. Scand J Gastroenterol 2005;40:540-551.
    Pubmed CrossRef
  41. Lee H, Friedman ME, Cukor P, Ahern D. Interactive voice response system (IVRS) in health care services. Nurs Outlook 2003;51:277-283.
    Pubmed CrossRef
  42. Kluetz PG, O'Connor DJ, Soltys K. Incorporating the patient experience into regulatory decision making in the USA, Europe, and Canada. Lancet Oncol 2018;19:e267-e274.
    Pubmed CrossRef
  43. The United States Food and Drug Administration. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes 2006;4:79.
    Pubmed KoreaMed CrossRef
  44. The United States Food and Drug Administration. Principles for Selecting, Developing, Modifying, and Adapting Patient-Reported Outcome Instruments for Use in Medical Device Evaluation, 2020. Available from URL: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/principles-selecting-developing-modifying-and-adapting-patient-reported-outcome-instruments-use (accessed 4 Jul 2022).
  45. Roberts C, Moss B, Wass V, Sarangi S, Jones R. Misunderstandings: a qualitative study of primary care consultations in multilingual settings, and educational implications. Med Educ 2005;39:465-475.
    Pubmed CrossRef
  46. Gandrup J, Ali SM, McBeth J, van der Veer SN, Dixon WG. Remote symptom monitoring integrated into electronic health records: A systematic review. J Am Med Inform Assoc 2020;27:1752-1763.
    Pubmed KoreaMed CrossRef
  47. Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value Health 2009;12:419-429.
    Pubmed CrossRef
  48. Nowell WB, Gavigan K, Kannowski CL, et al. Which patient-reported outcomes do rheumatology patients find important to track digitally? A real-world longitudinal study in ArthritisPower. Arthritis Res Ther 2021;23:53.
    Pubmed KoreaMed CrossRef
  49. Sardi L, Idri A, Fernández-Alemán JL. A systematic review of gamification in e-Health. J Biomed Inform 2017;71:31-48.
    Pubmed CrossRef
  50. Mokkink LB, Terwee CB, Gibbons E, et al. Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) checklist. BMC Med Res Methodol 2010;10:82.
    Pubmed KoreaMed CrossRef
  51. Caplan A, Walker L, Rasquin A. Development and preliminary validation of the questionnaire on pediatric gastrointestinal symptoms to assess functional gastrointestinal disorders in children and adolescents. J Pediatr Gastroenterol Nutr 2005;41:296-304.
    Pubmed CrossRef


This Article

e-submission

Archives

Aims and Scope