Despite numerous randomized clinical trials and meta-analyses, there is no increased evidence for the efficacy of probiotics in the treatment of irritable bowel syndrome (IBS). We review this evidence, identify and analyse the reasons for this lack of evidence and propose methodological improvements for future studies.
Based on a literature search, we identified 56 papers that matched the purpose of our analyses. Twenty-seven studies used multi-species bacterial preparations and 29 used single-strain probiotics. They were analysed regarding patients included, treatment duration, probiotic dosage, and outcome measures.
Trials in both groups suffered from heterogeneity with respect to probiotic concentration, duration of treatment, and other methodological issues (crossover design and underpowered studies). This heterogeneity did not allow the application of a meta-analytic approach and a systematic review was therefore performed instead. Multi-strain preparations combined 2 to 8 different bacterial subspecies, mostly lactobacilli or bifidobacteria, and used variable lengths of treatments. Overall, more than 50% of trials presented negative outcomes. The majority of the single-strain probiotic trials employing lactobacilli or
The heterogeneity of the studies of probiotics in IBS questions the value of meta-analyses. The use of different bacterial strains and different mixtures of these strains, as well as different dosages, are the main contributors to this heterogeneity. Current data provides limited evidence for the efficacy of a small number of single-strain probiotics in IBS (mostly bifidobacteria) and sound studies following strict trial guidelines (Food and Drug Administration and European Medicines Agency guidelines for clinical trials) are needed. We summarised and proposed some methodological issues for future studies in the field.
“Probiotic” therapy-in contrast to antibiotic therapy-defines the use of single bacterial strains or combinations of bacterial strains to influence the commensal gut microbiota, despite the fact that the term “probiotic” was not used before the 1950s.1,2 Parker3 first established a definition in 1974. Fuller4 defined probiotic as a “living microbial food supplement” that improves the host by improving its intestinal bacterial balance. His definition is still valid today. This also resembles the position of the WHO/FAS (2001) which state that probiotics are living organisms that, when ingested in sufficient amounts, may be beneficial for the host.5 This definition avoids any speculation about the presumed mechanism of probiotics’ action. A 2014 update of the definition by the International Scientific Association for Probiotics and Prebiotics (ISAPP) confirmed6 the latter definition. In addition, bacterial yoghurt cultures usually consisting of
The use of probiotics in the treatment of gastrointestinal (GI) tract disorders increased during the 1990s. Among the disorders that have been shown to benefit from probiotic treatment in placebo-controlled double-blinded studies are antibiotic-associated diarrhea,7 traveller’s diarrhea,8 rotavirus induced diarrhea,9 and
However, clinical trials prior to the formation of the international Rome consensus for functional bowel disorders in 1988 mostly lack credibility because a homogenous disease definition was lacking.15 The consequence was that in some trials, patients with or without abdominal pain in association with altered bowel habits were classified as IBS sufferers (or irritable colon, mucous colitis, or otherwise).16 In studies based on the Rome consensus IBS definition,17 efficacy was often based on improvement of individual symptoms including abdominal pain, diarrhea, and/or constipation.
Over the years general assessments have remained rather similar, presuming an overall positive effect whilst claiming a lack of high-quality data: “Probiotics may improve symptoms of irritable bowel syndrome and can be used as supplement to standard therapy.”19 “While our analyses suggest that probiotic use may be associated with improvement in IBS symptoms compared to placebo, these results should be interpreted with caution given the methodological limitations of contributing studies. Probiotics warrant further study as a potential therapy for IBS”20: “Probiotics may have a role in alleviating some of the symptoms of IBS; however further research should focus on the type, optimal dose of probiotics and the subgroups of patients who are likely to benefit the most”23: “Six of the eight diseases (...irritable bowel syndrome...) showed positive significant effects.... Across all diseases and probiotic species, positive significant effects of probiotics were observed for all age groups, single vs multiple species, and treatment lengths.”25
Among these meta-analyses, the largest (including 35 trials) found a significant effect of probiotics on global IBS symptom rating for all 24 mono-strain preparations pooled26; however neither lactobacilli nor bifidobacteria was effective when tested separately. There was also an overall benefit of the 15 combination trial; however, when specific combinations were tested, neither yielded a significant benefit over placebo. A significant effect of probiotics on individual IBS symptoms was only found for bloating and flatulence. A later meta-analysis covering data published between 2007 and 2013, which included only 14 RCTs, found probiotics to have a positive effect on abdominal pain only.28
The initial idea of our project was a meta-analysis on the efficacy of probiotics in the management of IBS. However, after the initial search and data extraction (see Methods below), we concluded that the current data did not allow a methodologically sound meta-analysis to be performed. We therefore performed a new literature search with a changed scope: to critically review the body of evidence and to identify and assess the reasons for poor evidence in the current literature, as reflected in recent meta-analyses. We also proposed ways to improve methods for studies in the future. Our new analysis was conducted in 4-steps: (1) literature search, (2) identification of systematic reviews and meta-analyses, (3) comparison of lists from Step 1 and 2, and lastly (4) sorting the final list according to specific criteria.
In our initial search early in 2014, we performed a PubMed literature search using the search terms “(probiotic OR prebiotic) AND (irritable-bowel-syndrome OR IBS OR functional-bowel-disorder) AND trial” to retrieve randomized trials comparing a probiotic or prebiotic to a control treatment in functional bowel disorders such as IBS.
The abstracts of retrieved papers were screened to match the following inclusion criteria:
(1) The study including IBS patients according to pre-defined criteria (Manning, Kruis, and Rome);
(2) The study only including adult patients;
(3) The study design including a placebo arm;
(4) The study being described as randomized;
(5) The study with attempted blinding of patient assignment;
(6) The study describing the prebiotic or the probiotic bacterial strain in sufficient detail;
(7) The study assessing either global IBS measures or single symptoms or QOL;
(8) The full paper being available and written in English.
If only an abstract was available, we excluded the study. After exclusion of non-relevant studies, the remaining articles were screened for the following criteria:
(1) Whether the study population was IBS, IBS-C or IBS-D, and IBS-M;
(2) The number of patients included in the study;
(3) Whether the probiotic was single-strain or multiple-strain;
(4) In the case of multi-strain products, the strains included;
(5) Whether it was a pharmacological or nutritional probiotic;
(6) Whether the design was a parallel-group design or a cross-over design;
(7) The duration of treatment.
All screenings were performed by 2 authors (E.B. and M.S.). In case of disagreement, a third author (N.M.) was consulted for a final decision. We also assessed the Jadad score for each selected study in order to assess the methodological quality of the clinical trials.
After the first review of the retrieved data and the decision to change the purpose and methodology of the project, we implemented a 4-step approach that helped us achieve our goals.
In Step 1 we performed a new literature search using the same key words- “(probiotic OR prebiotic) AND (irritable-bowel-syndrome OR IBS OR functional-bowel-disorder) AND trial” -which resulted in the generation of new literature set. The last update was performed on April 20, 2015. Retrieved papers were screened again using the inclusion and exclusion criteria described above.
In Step 2 we identified systematic reviews and meta-analyses in our literature search published between 2006 and 2015, which summarise the respective actual state of knowledge. They were reviewed for the included studies as well their major findings across all studies. A complete list of trials included in the reviewed meta-analyses and systematic reviews was generated independently from the literature list from Step 1.
In Step 3 we compared both retrieved literature lists from Step 1 and from Step 2 and identified papers included in one or both lists as well as those that were excluded by us, but which were included in one or more of the performed meta-analyses. For each discrepancy found, we discussed the respective paper among all authors to reach a final decision about it’s inclusion or exclusion using the same criteria described in Step 1.
Step 4: the final list of included studies was sorted according to the following criteria:
(1) Single-strain or multi-strain probiotic;
(2) Manufacturer of the preparation;
(3) If multi-strain, which bacterial strains were present;
(4) The dosage of each strain;
(5) The form of preparation either as a nutritional product or a drug;
(6) If single-strain, the bacterial strain used, and at which dosage;
(7) Trial length and patient groups included;
(8) Study outcome as defined by the authors-positive or negative.
Based on our critical review of the current literature, we finally discussed the potential cornerstones of studies to investigate the efficacy of probiotics in IBS in the future.
Step 1: Figure 1 shows the results of the literature search, last performed on April 20, 2015. After exclusion of systematic reviews and meta-analyses (n = 9), uncontrolled trials (n = 20), narrative reviews, editorial, letters and comments (n = 78), animal research (n = 2), studies of children and respective reviews (n = 10), non-English papers and studies (n = 8), and others (n = 8), 60 studies remained. Data published in abstract form only was not included.
Following a full-paper review, we excluded 8 further studies due to various reasons: only 1 study reported data from prebiotic treatment,29 1 study did not use a probiotic in a strict sense (no living bacteria),30 2 were microbial investigations of earlier studies,31,32 2 were single-blinded,33,34 and 2 were uncontrolled interventions.35,36 Based on hand-searches of the published meta-analyses, we added 3 more studies,37-39 resulting in a total of 55 articles.
Step 2: we identified 10 meta-analyses and systematic reviews (published between 2008 and 2015) related to probiotic trials in IBS,18-26,28 reporting a total of 44 RCTs with variable number of trials in each of them that increased over the years. One “systematic review” of complementary and alternative therapies in IBS was not comprehensive at all for IBS probiotic trials18; it included only 4 studies that were also analysed and discussed in all other reviews. Another review21 summarized previous systematic reviews and meta-analyses,19,22-24,40,41 but did not provide the full list of citations of papers and was therefore of no further help for the purpose of this review.
Step 3: we then compared the literature lists from Step 1 and Step 2 to identify the degree of overlap between each analysis. Of the 45 papers listed in Step 2, we excluded 14 for different reasons.
Six citations were available as abstracts only42-47 and were thus of no value for our study. One of those43 has been published as a full paper later48 and was included in our sample; another44 was also published as full paper30 but was excluded since it was not a probiotic in a strict sense; 245,46 had only an English abstract however the full papers were written in Korean and Chinese each.
Two frequently included papers reported studies in children.49,50 Two were placebo controlled but single-blinded studies,33,51 one combined acupuncture with probiotic treatment,46 and one32 reported microbiota changes from a probiotic trial published before52 which was already included in our list.
One study was not able to be verified53 but was, by title, identical to another article by the same authors published in French54 It was excluded because it was published too long before the first standardized diagnostic criteria for IBS to become available.55
One more study56 from Step 2 was added to our list of Step 1 studies, resulting in 56 studies available for our analysis.
Taking into account only the studies on our list and in at least one of the other analyses, overall agreement was low, as can be seen in Figure 2: only 5 RCTs from all 56 were included in all 7 previous analyses (unless the analyses restricted their timeframe, as in28) as well as in ours. Seven RCTs were exclusively analysed by the largest study26 and our analysis added another 20 studies to the list.
Step 4: of the 56 RCTs, 27 reported data from multi-strain probiotics, and 29 from single-strain probiotic products. Such a distinction was not made in the majority of the meta-analyses. Except in 2 studies,24,26 all others pooled data and provided overall statistics and relative risk calculations.19,20,23,25,28 One22 did perform a systematic review instead of a meta-analysis, but nevertheless mixed both groups.
The number of patients included ranged between 24 and 186 (median = 90). One study included patients with functional gastrointestinal disorders, ie, with diarrhea but not constipation and not necessarily suffering from abdominal pain,72 one was restricted to constipation predominant IBS (IBS-C) patients65 and 4 included only diarrhea predominant IBS (IBS-D) patients.62,68,69,80 All studies claimed the use of Rome (II or III) criteria, nevertheless, a more detailed description of the patient population (the number of IBS-D, IBS-C, and mixed IBS [IBS-M] patients) was lacking.
Treatment duration varied from 7 days to 6 months, thus not fulfilling Rome criteria for treatment duration in many cases,17 with the majority of studies assessing treatment results after 8 weeks. A common outcome-reporting period (4 weeks for example) was not available for comparison across all studies and neither were the primary nor the secondary outcome variables according to the FDA/EMA or Rome criteria in most studies.Bacterial strain combinations
The largest heterogeneity was noted with respect to the bacterial strain combinations used (Table 2). Most of the multi-strain preparations were nutritional supplements from different suppliers worldwide. The same preparation (VSL#3) was used in only 3 studies.69,77,80 Up to 8 different bacterial strains are used in some preparations and the combinations are numerous, since in many cases subspecies of specific strains were used that are only available to (and eventually patented by) individual companies. Two studies were performed by the same research group with the same preparation.74,76
The use of
While some strains (such as
The concentration of bacteria in these studies was not always reported57,78 and varied substantially, ranging from 108 (Shavakhi et al57) to 1011 (Michail et al69), providing that the concentrations listed in the papers are correct and that (specifically for dairy products) the cooling chain had not been interrupted. Only one study compared a high dose (3 × 1010 colony forming unit [cfu]) with a low dose (6 × 109 cfu) and found both equally ineffective compared to placebo.59
Due to the heterogeneity of the different probiotic compositions, it appeared inappropriate to perform a meta-analysis; except for the 3 VLS#3 studies which have been reported to provide no benefit over placebo in IBS.26Global outcome
Most studies included here reported their outcome in global terms. We used these explicit summaries of the efficacy of the respective probiotic treatment to retrieve a globally rated “positive” or “negative” IBS treatment outcome. However, we are aware of the fact that this may be biased by the respective authors that tend to underestimate or undervalue negative results, for instance in comparison to registered trial reports.81,82
We found that 14 of the 27 studies reported negative outcome on global symptoms, however individual symptoms (bloating67,77 and satiety69) were occasionally reported as being responsive to treatment. Quality of life (QoL) improved in one study but symptomatic improvement was not different from placebo.59 Thirteen studies were positive on global symptom reports. While the size of the study populations has increased, this did not affect the (positive or negative) outcome of the studies, as more patients were included in the studies reporting negative outcomes than in the studies with positive outcomes (Table 1).
In summary, the balance of positive and negative studies indicates an arbitrary and random result rather than an effective outcome of treatment attempts in IBS using multi-strain probiotics. This contrasts with the summary of Ford et al26 that found an overall positive effect of multi-strain probiotics in IBS when all included studies were pooled, but not of individual combinations.
The 29 single-strain probiotic intervention studies exhibited a large variability with respect to various design features, as shown illustrated in Table 3.37,52,56,83-108 Four studies employed a cross-over design,52,56,90,105 which is frequently used for motivational purposes (patients are easier to recruit when they are offered an effective treatment, at least for a part of the trial). However, they are difficult to evaluate since the assumption that the data in both periods is equal and can be merged may be distorted by unblinding and conditioning effects.109
Compared to multi-strain studies, more investigations (15/29) were limited to 4 weeks (or 2 × 4 weeks in a cross-over design) and some extended up to 8 weeks or beyond (11/29). Similarly, the range of included IBS patients ranged from 12 to 362. Two trials included IBS-C99,101 and 3 trials only IBS-D patients,84,96,97 while 2 studies stratified the patients into IBS-C and IBS-D85 or into IBS-D and IBS-M.95 Only one study addressed another subgroup of patients who may benefit from probiotics: patients with post-infectious IBS.110 Unfortunately, the researchers did this post-hoc rather than prospectively (see below).92
Inclusion criteria also varied. Four studies did not specify which criteria were used defining IBS patients,56,88,105,107 which may indicate that data was collected before Rome criteria became effective and in 2 studies, this was expressed in the article (pre-Rome98,106). Whilst male patients were included, females were dominant in all trials; one study exclusively recruited female patients.102Bacterial strains
With respect to the bacterial strains studied, the heterogeneity continued: 6 studies used
A number of studies have used the yeast
Only one study compared 2 completely different probiotics and found that
As with multi-strain probiotics, the concentration of bacteria in these studies was either not reported,84,87,93 or the daily dosage varied by almost a factor of 1000, ranging from 1 × 108 (Whorwell et al102) to 8 × 1011 (Choi et al95).Global outcome
Again, we used the authors’ evaluation of their results and separated positive from negative studies. Studies that noted an improvement of QoL without changes in abdominal symptoms (pain, stool frequency, and bloating),84,95,101 were labelled as “negative except QoL,” since it is against all recommendations (Rome criteria and FDA/EMA) to use QoL as a primary endpoint in IBS treatment studies.
Three more studies were identified as negative, contrary to the evaluation of the respective authors: one found a significant effect on abdominal pain, but not on other IBS symptoms. Upon visual inspection of the data (ibid., see Figure 3 in Pineton de Chambrun et al83), the effect occurred during weeks 6 to 8 with a sudden change in pain severity, mimicking a “recruitment bias” of unknown origin; however, it lacks any rational discussion and explanation. A similarly surprising and unexplained change occurred in the responder data of another study (ibid., see Figure 2 in Kruis et al),92 during weeks 9 to 12, which may explain for the post-hoc significance in a subgroup of patients with post-infectious IBS. Finally, one study108 found significant changes in bloating scores, but not for any other core symptoms of IBS.
Balancing all studies revealed that 15 of the 28 studies (excluding the study comparing lactobacilli against bifidobacteria104) yielded a negative or at least partly negative outcome. Taking all patients in positive and negative studies into account, more patients had a benefit from probiotics than those who did not. This distribution is even more skewed if we allow the studies with limited positive outcome (see above) to count as “positive.” In this case, only 9 of the 29 studies had a clearly negative outcome.
With respect to individual bacterial strains, the ratio of positive to negative studies with
The 4 studies using the yeast
Our systematic review of 56 published RCTs of probiotics in IBS and of 10 systematic reviews and meta-analyses, including most of these trials, identified major flaws in the RCTs, obscuring the evidence for the efficacy of probiotics as a treatment option in IBS. Consequently, less evidence was produced with increasing numbers of RCTs added to meta-analyses. The latest and largest of these meta-analyses26 concluded that while across all studies moderate evidence exists for efficacy of probiotics in general (though restricted to single-strain preparations) on global symptoms, neither individual bacterial strain reached a sufficient level of evidence, nor are individual core IBS symptoms (except bloating) effectively treated by any of the tested bacteria. Multistrain probiotics appear to be of no value at all. Narrative reviews on the efficacy of probiotic treatment in IBS usually follow these systematic reviews and meta-analyses.111
In an attempt to understand why the picture has not improved but has rather developed into the opposite in recent years, we analysed all available studies for homogeneity in the applied drugs (or food supplements), in design and trial features and global outcomes and found that heterogeneity rather than homogeneity has increased over the years. Patient recruitment and selection especially, treatment duration, probiotic dosages and the choice of primary and secondary endpoints of the study have not followed the same route that pharmacology of GI drugs have paved through the FDA,112 EMA,113 and international consensus parties such as the Rome group.17 Whilst in GI pharmacology placebo rates have dropped and efficacy of drugs over placebo has increased,114 the development of probiotics has not followed the same trend. One reason for this might be that, investments in clinical research and trials are still the exception rather than the rule for the nutritional (dairy) industry, not least because of lower profit rates and higher turn-around of investments into marketing of products.
Instead of rigorously questioning the value of previously tested probiotics in IBS, we finally attempted to outline the requirements for future trials to overcome these limitations. Many of these are already clinical standards that have been set during the last 25 years in GI drug development, by drug companies, approval authorities, and expert consensus parties. These have not been adopted yet by the nutritional industry that is responsible for most of the compounds that have been put to the market. However, clearly there cannot be different standards for drug and nutrient testing, and the future common policy of EMA and European Food Safety Authority in Europe115 has underlined this.
Since GI health claims for probiotics have to assess efficacy with respect to patients with functional bowel disorders of IBS-type,115 they have to be measured with standards of drug testing in the same area, ie, according to FDA and EMA rules112,113 and the consensus of the gastroenterology community.17 The following provides a summary of putative paradigms to guide future trials.
First and foremost, any clinical study should be registered and it should be registered before the study start and not after the data evaluation.57,86,92 Only 6 of the 27 multi-strain probiotic studies were registered. Even more surprisingly, only 3 of the 29 studies with single-strain probiotics were registered despite the fact that the single-strain probiotics were used not as nutritional supplement, but as drug by the majority. Registration prevents (silent) deviations from the proposed protocol and statistical evaluation strategy and diminishes doubts as to whether the reported efficacy data was a re-interpretation of missing efficacy on the primary endpoints chosen (see above).
Secondly, the study would need to be adequately powered. As we have repeatedly discussed,114 studies with samples size lower than 100 patients are at increased risk of producing high and variable placebo response rates and therefore would need strong effects on the primary study endpoint to achieve significance. However, this cannot be expected with nutritional interventions (even when provided as approved drugs) which is confounded by the variability of daily nutrition including consumption of preand probiotics.116 In this respect, 19 of the 27 multi-strain and 21 of the 29 single-strain probiotic studies had a priori low chance of finding significant effects; however, to our knowledge only a few were based on a power calculation.
Thirdly, a probiotic study in IBS patients would preferentially not use a cross-over design; such designs have almost completely been abandoned in gastrointestinal pharmacology and elsewhere for obvious reasons. They carry the risk of unblinding,109 of cross-over effects when wash-out periods are too short,116 and of conditioning effects in the second (placebo) phase when the initial treatment is the probiotic.117 Consequently, the statistical evaluation should be restricted to the first treatment phase; however, these studies are usually not powered for such “quasi parallel design.” Only 4 of the single-strain studies and none of the others installed a cross-over design.
Furthermore, inclusion and exclusion criteria should closely follow the EMA and FDA guidelines for clinical trials in IBS,112,113 including the definition of minimal severity for inclusion, global primary endpoints, and adequate secondary endpoints (pain, bloating, and a clinically meaningful responder definition). It should incorporate at least 8 weeks of treatment, an adequate follow-up interval and restriction to one of the different IBS subtypes. Documentation of nutritional habits as well as stool and pain diaries should be mandatory.
Finally, either a single-strain probiotic should be tested against placebo, or 2 different probiotics in a comparative effectiveness research design118 and the compounds chosen should be based on the available evidence. This excludes multi-strain probiotics, dairy products that require maintenance of the cooling chain and probiotics which have not been able to demonstrate superiority over placebo despite frequent trials, as documented in our list above. Comparative effectiveness research trials need larger samples (for demonstration of non-inferiority and the choice of the comparator is essential.118 A 3-arm trial (such as O’Mahony et al104) testing 2 different probiotics against placebo is certainly an elegant alternative. Comparing probiotic treatment to other dietary or non-dietary therapies (prebiotics86 and simethi-cone87) or adding probiotics to other treatments (acupuncture46) may help in recruiting patients but does not support the search for evidence.
We are aware of the limitations of our own analysis. While we included as many published trials as possible into this systematic review, we excluded trials reported as abstracts only and those not published in English. For a meta-analysis of overall efficacy, this may create a publication bias: however, for a critical review of methodological flaws this may not be as relevant since detailed data needed for critical analyses is usually not included into congress abstracts. We also ignored differences in primary endpoints (and their statistical characteristics, continuous or binary) reported in these studies and referred to reported overall efficacy data as “true” -assuming that the authors have reported the best evidence they could gain from their own data, irrespective of what the initial evaluation plan was. Otherwise, the overall evidence may have been even poorer. Finally, since most of the multi-strain preparations were nutritional supplements from different suppliers worldwide, the same product may have been available in different countries under different brand names with different labelling and approved for different indications due to differences in national regulations for approval of nutritional supplements. We were not able to resolve this issue.
When evaluating the efficacy of the FODMAP diet in IBS on behalf of the Rome Foundation, Yao et al116 thoroughly discussed the methodological difficulties of dietary trials, with an emphasis on functional bowel disorders. Most importantly, dietary intervention trials need to address the collinearity between food, nutrients, and bioactive components which could obscure the relationship between food and their effects in the gut. The authors provided recommendations for such trials, ranging from assessment and documentation of baseline nutrients intake, via adequate masking of study food, to the design of adequate sham controls; however, the authors insist that for food trials, the randomized, double-blind and placebo-controlled study remain as the gold standard.
In summary we conclude, that the heterogeneity of the studies of probiotics in IBS questions the value of meta-analyses. The use of different bacterial strains and different mixtures of these strains, as well as different dosages, are the main contributors to this heterogeneity. Current data provides limited evidence for the efficacy of a small number of single-strain probiotics in IBS (mostly bifidobacteria) and sound studies following strict trial guidelines (FDA and EMA guidelines for clinical trials) are needed.