A Simple and Robust Way of Concluding Meta-Analysis Results Using Reported P values, Standardized Effect Sizes, or Other Statistics

Abstract

Meta-analysis is a powerful tool to estimate measures of associations/effects based on published/unpublished reports. However, problems exist in many meta-analyses, particularly related to study heterogeneity. This article proposes a way of concluding meta-analysis results using P-values, taking heterogeneity into account. There is little research focused on evaluating conclusiveness of summary results of reported meta-analyses. Generally, a P-value is directly linked to the test statistic z=b/s_b following a standard normal distribution with mean zero and unit variance, where b is an estimator of β and s_b is the estimated standard error of b for any study included in a meta-analysis. This forms the basis of the proposed method for deriving overall test statistics and corresponding P-values used for comparing results of meta-analyses. Two published meta-analyses were chosen and specific software was applied. Results are consistent with the two published meta-analysis reports in terms of P-values for significance and direction of summary measure of treatment effect. This proposed method can be utilized to safeguard against improper conclusions of published meta-analyses due to heterogeneity. Exploring more sophisticated statistical methods for situations when the key assumption applied to this proposed method is violated could be pursued and could expand the scope of applications beyond this method.