statsmodels.stats.multitest.local_fdr_correction

statsmodels.stats.multitest.local_fdr_correction(pvals, null_proportion=1.0, is_sorted=False)

Estimate local and tail false discovery rates for a list of p-values.

Fits a monotone (non-increasing) estimate of the marginal density of the p-values using the Grenander estimator. Combined with an estimate of the proportion of true null hypotheses, this yields, by Bayes’ rule, empirical Bayes estimates of the tail and local false discovery rates.

Parameters:

pvalsarray_like, 1d

List of p-values of the individual tests.

null_proportionfloat, optional

Estimate of \(\pi_0\), the proportion of true null hypotheses. Defaults to the conservative choice 1.0, i.e. all hypotheses are assumed null.

is_sortedbool, optional

If False (default), the p-values will be sorted, but the estimated FDR values are returned in the original order. If True, then it is assumed that the p-values are already sorted in ascending order.

Returns:

LocalFDRCorrectionResult

A namedtuple with the estimated tail false discovery rates (fdr) and estimated local false discovery rates (lfdr).

See also

local_fdr

Local FDR estimation for Z-scores using Poisson regression.

fdrcorrection

Benjamini-Hochberg/Benjamini-Yekutieli p-value correction.

Notes

Let \(t \in [0, 1]\) denote a p-value threshold and \(\hat{\pi}_0\) the null_proportion. Let \(\hat{F}\) be the Grenander estimate of the empirical distribution function of pvals, given by the least concave majorant (LCM) of the empirical cdf, and let \(\hat{f}\) be its density estimate, given by the left-hand slope of the LCM. The tail and local false discovery rates are then estimated by Bayes’ rule as

\[\widehat{Fdr}(t) = \min\left(1, \hat{\pi}_0 \frac{t}{\hat{F}(t)} \right) \approx \Pr(\text{null} \mid p \leq t)\]

\[\widehat{fdr}(t) = \min\left(1, \frac{\hat{\pi}_0}{\hat{f}(t)} \right) \approx \Pr(\text{null} \mid p = t)\]

Tied p-values are treated as repeated observations at a single support point of the empirical distribution function.

This method assumes that the p-values are independent, are uniformly distributed under the null, and have non-increasing densities under the alternative.

References

[*]

U Grenander (1956). On the theory of mortality measurement: part II. Scandinavian Actuarial Journal, 39, 125-153.

[†]

B Efron, R Tibshirani, J D Storey, and V Tusher (2001). Empirical Bayes analysis of a microarray experiment. Journal of the American Statistical Association, 96:456, 1151-1160.

[‡]

B Efron (2007). Size, Power and False Discovery Rates. The Annals of Statistics, 35:4, 1351-1377.

[§]

K Strimmer (2008). A unified approach to false discovery rate estimation. BMC Bioinformatics, 9, 303.

[¶]

J A Soloff, D Xiang, and W Fithian (2024). The edge of discovery: Controlling the local false discovery rate at the margin. The Annals of Statistics, 52:2, 580-601.

Examples

>>> from statsmodels.stats.multitest import local_fdr_correction
>>> import numpy as np
>>> pvals = np.random.rand(30)
>>> lfdr = local_fdr_correction(pvals).lfdr