The functional luminal imaging probe (FLIP) measures the cross-sectional area of the esophageal lumen using impedance planimetry technology during controlled volumetric distension. When combined with the FLIP measure of distensive (ie, intra-FLIP) pressure, distensibility of the esophagus is also assessed. In 2014, our group developed the FLIP panometry technique that displays the esophageal diameter changes along a space-time continuum (ie, esophageal “diameter” topography) with associated pressure during a volume-controlled distension protocol.¹ By assessing esophagogastric junction (EGJ) opening and the contractile response to distension, ie, secondary peristalsis, FLIP panometry provides an evaluation of esophageal motility.^1,2

We recently reported a diagnostic approach to classify EGJ opening (that accurately identified EGJ obstruction as defined by high-resolution manometry (HRM) and the Chicago classification version 4.0 (CCv4.0) applying FLIP panometry metrics: the EGJ distensibility index (DI) and maximum EGJ diameter.^3,4 This approach accurately identified EGJ obstruction, as defined by high-resolution manometry (HRM) and the CCv4.0, when the FLIP metrics were both distinctly reduced (ie, reduced EGJ opening [REO]), or both distinctly normal (ie, normal EGJ opening). The promise of the FLIP panometry EGJ opening classification was further demonstrated by outperforming HRM to detect esophageal retention on timed barium esophagram (TBE).⁵ However, the FLIP panometry EGJ opening classification left a “middle-ground” classification of borderline EGJ opening (BEO) when the EGJ-DI or maximum EGJ diameter were reduced, but criteria for REO were not met; this was observed in 23% of the study cohort.³ The BEO classification was associated with less diagnostic certainty and it was recommended that additional complementary testing should be pursued to facilitate reaching an overall clinical impression.

Although applying dichotomous physiomarker thresholds to classify motility findings is conventional practice, doing so carries limitations. In particular, results close to the thresholds or falling in middle-range “gray zones” (ie, borderline or inconclusive classifications), may be associated with an unquantified degree of diagnostic uncertainty.^3,4,6 Statistical machine learning techniques can address these limitations by generating diagnostic “probabilities” to facilitate application of test data to clinical diagnosis. The present study aim to develop and test a clinical decision support tool using FLIP Panometry to determine probabilities for the presence of EGJ obstruction.

The study cohorts (patients and controls) were previously described.³ Consecutive, adult patients (ages 18-89 years) patients that completed FLIP during upper endoscopy and HRM for evaluation of esophageal symptoms between November 2012 and December 2019 were prospectively evaluated with data maintained in an esophageal motility registry. Additional clinical evaluation, TBE in particular, was obtained at the direction of the primary treating gastroenterologist. Criteria for exclusion included technically limited FLIP or HRM studies and suspected causes of secondary esophageal motor abnormalities such as previous foregut surgery (including previous pneumatic dilation) or esophageal mechanical obstructions including esophageal stricture, eosinophilic esophagitis, severe reflux esophagitis (Los Angeles-classification C or D), hiatal hernia > 3 cm (Supplementary Figure). As well, patients with inconclusive assignments of EGJ outflow (based on CCv4.0, independent of FLIP) were excluded from this analysis as the inconclusive label was not amenable to model training or testing.

A cohort of healthy, asymptomatic, adult research volunteers were also included (“controls”).⁷ Informed consent was obtained for subject participation; control subjects were paid for their participation. The study protocol was approved by the Northwestern University Institutional Review Board (STU00210130).

Functional Lumen Imaging Probe Study Protocol and Analysis

The FLIP study using 16-cm FLIP (EndoFLIP EF-322N; Medtronic, Inc, Shoreview, MN, USA) was performed during sedated endoscopy and analyzed as previously described.^2,7,8 With the endoscope withdrawn and after calibration to atmospheric pressure, the FLIP was placed transorally and positioned within the esophagus with 1-3 impedance sensors beyond the EGJ. Stepwise 10-mL FLIP distensions beginning with 40 mL and increasing to target volume of 70 mL were then performed; each stepwise distension volume was maintained for 30-60 seconds.

FLIP data were exported using a customized program (available free, open source at http://www.wklytics.com/nmgi) to generate FLIP Panometry plots for analysis.^9,10 FLIP analysis was performed blinded to clinical details. The EGJ analysis specifically focused on the EGJ-DI at the 60 mL FLIP fill volume and the maximum EGJ diameter that was achieved during the 60 mL or 70 mL fill volume.

High-resolution Manometry Protocol and Analysis, Including Labeling of “Esophagogastric Junction Obstruction”

Manometry studies were performed after a least a 6-hour fast. The HRM assembly comprised of 36 circumferential pressure sensors at 1-cm intervals (Medtronic Inc) was placed transnasally and positioned to record from the hypopharynx to the stomach with approximately 3 intragastric pressure sensors. After a 2-minute baseline recording, the HRM protocol was performed with ten, 5-mL liquid swallows in a supine position and five 5-mL liquid swallows in an upright, seated position.⁵

Manometry studies were analyzed according to the CCv4.0 (Supplementary Table).^4,11 A median integrated relaxation pressure (IRP) of > 15 mmHg was considered abnormal for supine swallows; a median IRP of > 12 mmHg was considered abnormal for upright swallows.⁴ TBE was applied when available to patients with an HRM classification of EGJ outflow obstruction (EGJOO) to reach an assignment of “conclusive” EGJOO. TBE was considered conclusively abnormal if the 5-minute column height was > 5 cm or a 1-minute column height was > 5 cm in addition to impaction of a 12.5-mm barium tablet.¹² Inconclusive-EGJOO was assigned in patients with an HRM classification of EGJOO that completed a TBE that was not conclusively abnormal or did not complete a TBE (as recommended by CCv4.0).^4,11,12

Normal EGJ (ie, not EGJ obstruction) was defined by normal median supine IRP, normal median upright IRP, or < 20% of supine swallows with intrabolus pressurization; ie, isolated elevated supine IRP, elevated upright IRP, or presence of intrabolus pressurization (but not all 3) were assigned as not EGJ obstruction.¹¹ As this study focused on EGJ obstruction, criteria for “inconclusive” disorders of peristalsis were not applied.⁴

Labels for model training/testing were assigned via CCv4.0 with conclusive disorders of EGJ outflow applied as “EGJ obstruction” and normal EGJ outflow as “not EGJ obstruction” (additional details in Supplementary Table). Hence, strict criteria for assignment of presence or absence of EGJOO as outlined by CCv4.0 that are independent of FLIP were applied.

To further assess the performance of the FLIP prediction model, TBE was also utilized as a secondary test measure when completed (n = 272 patients; 49% of the cohort). TBE results were categorized for analysis based on the findings of greatest severity by: (1) 5-minute column height > 5 cm, (2) 1-minute column height > 5 cm or inability of the barium tablet to pass, or (3) “normal” (ie, not meeting preceding severity criteria).

Statistical Methods and Model for Prediction of Esophagogastric Junction Obstruction

Summary statistics were computed as mean (standard deviation [SD]), or median (interquartile range [IQR]) depending on data distribution. Groups (by TBE) were compared using Mann-Whitney U tests. Statistical significance was considered at P < 0.05 and the Bonferroni correction was utilized to adjust for multiple comparisons.

The decision support tool was created using Bayesian additive regression trees (BART) to estimate the probability of EGJ obstruction versus normal EGJ outflow (defined by HRM/CCv4.0) given EGJ-DI and maximum EGJ diameter.¹³ BART is a Bayesian analog of standard tree-based algorithms (eg, random forests) that predicts outcomes using an average of several decision trees. Because BART posits a semiparametric probability distribution for the data based on probit regression, it can flexibly model the relationship between FLIP measures and the probability of EGJ obstruction, learning nonlinear and interaction terms from the data. For a given set of FLIP measures (EGJ-DI and maximum EGJ diameter), BART can be used to compute 2 distributions relevant to clinical decisions: the posterior distribution and posterior predictive distribution.

First, BART estimates a posterior distribution for the probability of EGJ obstruction. Given a maximum EGJ diameter (x₁) and EGJ-DI (x₂), clinical decisions may depend on the probability of obstruction P[obstruction | x₁, x₂] = p(x₁, x₂). Note that p(x₁, x₂) is an unknown quantity that could possibly range from 0 to 1. BART estimates a posterior distribution of p(x₁, x₂) that reflects the likelihood of p(x₁, x₂) taking various values in [0,1] given the data observed. Posterior distributions characterize uncertainty in knowledge of p(x₁, x₂) and can be summarized via measures of centrality (mode) as well as 95% credible regions that reflect areas in the [0,1] range that p(x₁, x₂) is most likely to fall.

Second, and perhaps most relevant to clinical decisions, is the posterior predictive distribution (PPD). The PPD quantifies the probability that some new or future patient has an EGJ obstruction given a maximum EGJ diameter and EGJ-DI. PPDs take into account uncertainty in the estimate of p(x₁, x₂) reflected in the posterior distribution. Thus, the posterior distribution tells how uncertain to be about p(x₁, x₂), while the PPD tells how to navigate that uncertainty when predicting if a new patient has an EGJ obstruction. Henceforth, the posterior predictive probability of an EGJ obstruction is referred to as a “diagnostic probability.”

To tune relevant parameters that govern model complexity including the number of trees, the probability of a tree growing a branch (base), and tree depth (power) 10-fold cross validation was used on the training set. Cross validation was run on a random subset of 80% of the data (n = 473), referred to as the “training set.”

Model performance was evaluated on a holdout test set as well as with a leave-one-out approach. The holdout “test set” included the remaining 20% of the cohort (ie, those not included in the training set), n = 118. For the leave-one-out approach, the model was fit on all observations (ie, training and test set observations) except one, and a prediction was made for the lone holdout observation. Model performance by both methods was summarized via predictive accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC). Posterior distributions were derived using Markov Chain Monte Carlo and Bayesian backfitting on the complete dataset (training + testing).¹³ Four chains (10 000 burn-in samples) were used. Analyses were implemented in the R programming language (version 4.1.0) using the dbarts library.

Subjects

Among the patients 557 patients (mean [SD]: age 53 (17) years, 56% female), 243 (44%) had a conclusive disorder of EGJ outflow, 314 (56%) had an HRM classification with normal EGJ outflow (Table). Dysphagia was the indication for esophageal motility testing in 89% of the patients. The median (IQR) supine and upright IRP values were 31 mmHg (23-41) and 29 mmHg (20-38), respectively, for patients with a conclusive disorder of EGJ outflow. The median (IQR) supine and upright IRP values were 10 mmHg (7-14) and 9 mmHg (5-12), respectively, for patients with normal EGJ outflow. All 35 controls (age 30 [6] years, 71% female) had normal EGJ outflow per CCv4.0 with median (IQR) supine and upright IRP of 10 mmHg (8-14) and 9 mmHg (4-10), respectively.

Model Development and Performance

Posterior distributions were derived using Markov Chain Monte Carlo and Bayesian backfitting on the complete dataset (training + testing).¹³ Four chains (10 000 burn-in samples) were used. The model with the lowest cross validation error rate (Gini index) involved 250 trees with a base of 0.75 and power 3.0.

On the holdout test set, the model accuracy for EGJ obstruction was 90%, with 85% sensitivity, 97% specificity, and 0.97 AUROC. Model performance estimated via leave-one-out validation demonstrated model accuracy of 89% with 87% sensitivity, 92% specificity, and 0.95 AUROC.

Decision Support Tool and Patient-case Examples

Diagnostic probabilities for EGJ obstruction relative to FLIP Panometry metrics, are illustrated in Figure 1. The decision support tool to calculate probability for EGJ obstruction using FLIP measures is available free online: https://www.wklytics.com/nmgi/prob_flip.html.

Figure 1. Probability for esophagogastric junction (EGJ) obstruction using functional lumen imaging probe (FLIP) panometry metrics. Four patients (A-D) from the study cohort are plotted based on the EGJ-distensibility index (DI) and maximum EGJ-diameter from FLIP panometry as described in the text. Figure used with permission from the Esophageal Center of Northwestern (available from URL: https://www.wklytics.com/nmgi/prob_flip.html).

Additionally, to illustrate application of the decision support tool, several examples from the patient cohort are also included (Fig. 1 and 2).

Figure 2. Case examples of functional lumen imaging probe (FLIP) panometry and high-resolution manometry (HRM). The FLIP panometry (left) and swallow from HRM study (right) from the 4 patients (A-D) labeled in Figure 1 are displayed. (A) Patient (HRM/Chicago classification version 4.0 (CCv4.0) diagnosis was type I achalasia) was treated with per-oral endoscopic myotomy (POEM) with significant symptomatic improvement; an Eckardt score was 1 at follow-up. (B) Patient (HRM/CCv4.0 diagnosis was absent contractility) was treated with proton pump inhibitor (PPI) for gastroesophageal reflux. (C) Patient (HRM/CCv4.0 diagnosis of type I achalasia), who also completed timed barium esophagram (TBE) with 13 cm column height at 5 minutes, was treated with POEM with symptom improvement; an Eckardt score was 0 at follow-up. (D) Patient (HRM/CCv4.0 diagnosis of normal motility) completed a TBE, which was normal, and was treated with PPI and dietary modification. Figure used with permission from the Esophageal Center of Northwestern.

• (1)

  Patient A with an EGJ-DI of 0.7 mm²/mmHg and maximum diameter 6.2 mm

• (2)

  Patient B with an EGJ-DI of 7.0 mm²/mmHg and a maximum EGJ diameter of 16.1 mm

• (3)

  Patient C with an EGJ-DI of 3.2 mm²/mmHg and maximum diameter 10.8 mm

• (4)

  Patient D with an EGJ-DI of 1.9 mm²/mmHg and maximum diameter 12.7 mm

The diagnostic probabilities for EGJ obstruction for the 4 patients are A: 97%, B: 4%; C: 68%; and D: 48%. Figure 3 shows the posterior distribution of the estimated probability of obstruction for all 4 patients. The HRM/CCv4.0 diagnosis for patients were (1) type I achalasia, (2) absent contractility, (3) type I achalasia, and (4) normal motility (Fig. 2).

Figure 3. Case examples for probability of esophagogastric junction (EGJ) obstruction. The posterior distributions from the Bayesian additive regression trees (BART) model for the 4 patient cases (A-D) labeled in Figure 1 are displayed. The probability for EGJ obstruction was determined as posterior predictive distribution. Figure used with permission from the Esophageal Center of Northwestern.

Diagnostic Probability for Esophagogastric Junction Obstruction Associated With Timed Barium Esophagram Findings

Two hundred and seventy-two patients completed TBE (Table). Patients with a 5-minute barium column height > 5 cm had a greater diagnostic probability for EGJ obstruction (based on FLIP and the BART model) than patients with a 1-minute barium column height > 5 cm or tablet impaction (P < 0.001), who had greater diagnostic probability for EGJ obstruction than patients with normal TBE (P < 0.001; Fig. 4).

Figure 4. Probability for esophagogastric junction (EGJ) obstruction associated with esophageal retention on timed barium esophagram (TBE). Patients were categorized by TBE results of greatest severity. The probability for EGJ obstruction was based on functional luminal imaging probe (FLIP) panometry and the Bayesian additive regression trees (BART) model. “○” and “*” represent outliers in the box-and-whisker plots. Figure used with permission from the Esophageal Center of Northwestern.

This study aimed to use machine learning to develop a diagnostic tool to apply to FLIP panometry data to determine the probability of EGJ obstruction, defined using HRM and CCv4.0. The major finding was that the resultant model’s accuracy in predicting EGJ obstruction was 89-90% depending on whether a “holdout test set” or “leave-one-out” approach was applied. The corresponding sensitivities, specificities and areas under the ROC curves were (87%, 92%, 0.95) and (85%, 97%, 0.97), respectively. The performance of the model was also supported by greater diagnostic probabilities for EGJ obstruction being associated with esophageal retention on TBE. Although conventional diagnostic performance metrics indicate that this model is very accurate, its additional contribution is that it applies a probabilistic, as opposed to dichotomous, framework to identify EGJ obstruction among patients evaluated for esophageal motility disorders.⁴ Conventional approaches focus on binary (ie, positive/negative) or categorical diagnoses rather than the probability of diagnoses. Such approaches either do not incorporate diagnostic probabilities, or compare them to some threshold (eg, > 50%) to determine diagnoses. In contrast, the model presented here is used explicitly to estimate the probability of a diagnosis. By focusing on probabilities instead of “yes/no,” the model quantifies the confidence or uncertainty of a diagnosis given the data.

The promise of FLIP to evaluate EGJ function and identify obstruction among patients evaluated for esophageal motility disorders, in particular achalasia, has been demonstrated in previous studies utilizing FLIP (among which are earlier reports of the patient cohort utilized for this study).^3,14-17 In a recent report of this patient cohort, we demonstrated that among patients with a conclusive CCv4.0 diagnosis, 86% of patients with REO on FLIP had a conclusive disorder of EGJ outflow per CCv4.0 and 99% of patients with normal EGJ opening on FLIP had normal EGJ outflow per HRM and CCv4.0.³ However, that also left a portion of patients with a BEO classification that considered inconclusive, but to an unspecified (ie, unquantified) degree. Further, the limitations of the applied fixed thresholds were noted recognizing that parameters falling near the proposed thresholds may be associated with less certainty than more extreme measures.

Probabilities, however, can be interpreted clinically and combined with insights derived from other available clinical data. For instance, in the case examples described (Fig. 1 and 2), the FLIP findings in patients A and B are associated with a high degree of certainty regarding EGJ obstruction. In the appropriate clinical context, high-probability findings such as these are likely sufficient to reach a confident clinical diagnosis. For FLIP findings associated with a greater degree of uncertainty (such as patients C or D), application of additional complementary testing (eg, HRM or TBE) may be necessary. However, instead of being limited to a classification with a non-specific impression of uncertainty (such as recently described with BEO on FLIP), the diagnostic probability provides guidance to the provider to weigh other data and pursue additional complementary testing if deemed necessary.³ If clinical, endoscopic, and/or TBE findings strongly support EGJ obstruction, a diagnostic probability for EGJ obstruction of 68% in case C may be sufficient to reach a diagnosis. Alternatively, a greater degree of uncertainty is reflected in case D with probability for obstruction of 48% such that pursuit of additional complementary testing (eg, HRM) is warranted.

While a strength of the study lies in the novel application of a machine learning-generated clinical decision support tool that utilizes a probabilistic approach to diagnosis of an essential esophageal motility parameter, there are also limitations to the study. Any model is subject to the limitations inherent to its supervised training labels. Here, while HRM and CCv4.0 represents the state of the art classification of esophageal motility disorders, it is not a perfect standard for obstruction. Incorporation of additional components to the training labels, such as those derived from provocative maneuvers with HRM or clinical outcomes/response to treatments could be utilized for future development of new or refined models.

Probabilistic machine learning offers a promising solution to alleviate the limitations associated with classification schemes using fixed rules and thresholds for clinical diagnoses. Future directions will seek to incorporate additional metrics or features (such as the contractile response patterns on FLIP Panometry) into the model to improve statistical precision and clinical performance.¹⁸ Similar models using other technologies (such as HRM) are also expected. Ultimately, incorporation of this machine learning decision support tool represents further evolution of the promise of FLIP panometry for evaluating esophageal function.

Note: To access the supplementary table and figure mentioned in this article, visit the online version of Journal of Neurogastroenterology and Motility at http://www.jnmjournal.org/, and at https://doi.org/10.5056/jnm21239.

This work was supported by P01 DK117824 (John E Pandofino) from the Public Health service, American College of Gastroenterology Junior Faculty Development Award (Dustin A Carlson), a grant from the Northwestern Digestive Health Foundation (Wenjun Kou), and gifts from Joe and Nives Rizza and The Todd and Renee Schilling Charitable Fund.

John E Pandofino, Peter J Kahrilas, and Northwestern University hold shared intellectual property rights and ownership surrounding FLIP panometry systems, methods, and apparatus with Medtronic Inc. Dustin A Carlson: Medtronic (speaking and consulting) and Phathom Pharmaceuticals (consulting). Peter J Kahrilas: Ironwood (consulting), Reckitt Benckiser (consulting), and Johnson & Johnson (consulting). John E Pandolfino: Sandhill Scientific/Diversatek (consulting, speaking, and grant), Takeda (speaking), Astra Zeneca (speaking), Medtronic (speaking, consulting, patent, and license), Torax (speaking and consulting), and Ironwood (consulting). Jacob M Schauer, Jacqueline E Prescott, and Wenjun Kou: nothing to disclose.

Jacob M Schauer contributed to study concept and design, data analysis, data interpretation, drafting of the manuscript, and approval of the final version; Wenjun Kou contributed to data analysis, programming, obtaining funding, editing the manuscript critically, and approval of the final version; Jacqueline E Prescott contributed to data analysis and approval of the final version; Peter J Kahrilas contributed to editing the manuscript critically and approval of the final version; John E Pandolfino contributed to study concept, obtaining funding, editing the manuscript critically, and approval of the final version; and Dustin A Carlson contributed to study concept and design, data acquisition, data analysis, data interpretation, drafting of the manuscript, obtaining funding, and approval of the final version.