Available on https://psyarxiv.com/4pf9j, this is a comment on a recent article in JAMA (Futier et al., 2020).
The multicenter FLASH trial1 concluded that “Among patients at risk of postoperative kidney injury undergoing major abdominal surgery, use of HES [hydroxyethyl starch] for volume replacement therapy compared with 0.9% saline resulted in no significant difference in a composite outcome of death or major postoperative complications within 14 days after surgery.” Indeed, the results were opposite to those expected: death or major complications were more prevalent in the HES group (139 of 389 patients: 35.7%) than in the saline group (125 of 386 patients: 32.4%). An associated editorial2 pointed out that “absence of evidence is not evidence of absence” and concluded that “The results of the FLASH trial corroborate the detrimental kidney effects of HES” suggested by recent meta-analyses.
Here we quantify the degree to which the data from the FLASH trial, considered in isolation, undercut the hypothesis that HES improves the primary outcome. We conducted Bayesian logistic regression3,4 and considered three rival models. First, under H0, (i.e., HES is ineffective) the log odds ratio equals ψ=0; second, under the positive-effect model H+ (i.e., HES helps, the authors’ hypothesis), ψ is assigned a positive-only normal prior distribution N+(μ,σ); third, under the negative-effect model H– (i.e., HES harms, the hypothesis suggested by recent meta-analyses), ψ is assigned a negative-only normal prior distribution N–(μ,σ).
A default analysis (i.e., μ=0, σ=1) shows that H0 receives the most support from the data; however, the data are only about 2.41 times more likely under H0 than under H–, a level of evidence that has been termed “not worth more than a bare mention”.5 In contrast, the data are about 12.30 times more likely to occur under H0 than under H+, a level of evidence that has been termed “strong”.5 If the three rival models were deemed equally plausible a priori (i.e., each 0.33), the corresponding posterior model probabilities for H0, H–, and H+ would be 0.67, 0.28, and 0.05, respectively.
In sum, the data from the FLASH trial mainly act to increase the plausibility that HES is ineffective, and decrease the plausibility that HES is helpful. These analysis, easily conducted in JASP (jasp-stats.org) or R (https://CRAN.R-project.org/package=abtest) arguably provide a more detailed perspective that supplements the statement that the results showed “no significant difference”.
References
1. Futier E, Garot M, Godet T, et al. Effect of Hydroxyethyl Starch vs Saline for Volume Replacement Therapy on Death or Postoperative Complications Among High-Risk Patients Undergoing Major Abdominal Surgery: The FLASH Randomized Clinical Trial. JAMA. 2020;323(3):225–236.
2. Zampieri FG, Cavalcanti AB. Hydroxyethyl Starch for Fluid Replacement Therapy in High-Risk Surgical Patients: Context and Caution. JAMA. 2020;323(3):217–218.
3. Kass RE, Vaidyanathan SK. Approximate Bayes factors and orthogonal parameters, with application to testing equality of two binomial proportions. Journal of the Royal Statistical Society: Series B (Methodological) 1992;54:129-44.
4. Gronau QF, Raj K. N. A., Wagenmakers EJ. (2019). Informed Bayesian inference for the A/B test. Manuscript submitted for publication and available on arXiv: http://arxiv.org/abs/1905.02068
5. Jeffreys, H. Theory of Probability. 1st ed. Oxford University Press, Oxford, UK, 1939.
About The Authors
Eric-Jan Wagenmakers
Eric-Jan (EJ) Wagenmakers is professor at the Psychological Methods Group at the University of Amsterdam.
Quentin F. Gronau
Quentin is a PhD candidate at the Psychological Methods Group of the University of Amsterdam.
