FCalibrate {pCalibrate} | R Documentation |
Transforms two-sided p-values from the F-test of overall significance in the linear model to sample-size adjusted lower bounds on the Bayes factor for the point null hypothesis against the alternative.
FCalibrate(p, n, d, alternative="chi.squared", intercept=TRUE, transform="id")
p |
a vector of two-sided p-values |
n |
a scalar or a vector of positive integers. Specifies the sample size(s). May be a vector only if |
d |
a scalar or a vector of positive integers.
Specifies the dimension(s) of the vector(s) of regression coefficients,
i.e. the number(s) of explanatory variables in the linear model(s).
May be a vector only if |
alternative |
either |
intercept |
logical. If TRUE, the linear model contains an unknown intercept term, otherwise the intercept is fixed. Defaults to TRUE. |
transform |
either |
Note that under the point null hypothesis that all regression coefficients are equal to zero, the F-statistic F (which is the (1-p)-quantile of the F-distribution with d and n-d-1 degrees of freedom) has a central F-distribution with d and n-d-1 degrees of freedom if the linear model contains an unknown intercept term (otherwise F has a central F-distribution with d and n-d degrees of freedom). Under a simple point alternative, F has a non-central F-distribution with d and n-d-1 degrees of freedom.
To obtain the lower bound on the Bayes factor
for alternative="simple"
,
the likelihood under the alternative is then maximized numerically with respect to
the non-centrality parameter.
That calibration is described in Held & Ott (2018),
Section 3.1 (in the last two paragraphs).
For alternative="chi-squared"
, the calibration is proposed in Held & Ott (2016),
Section 3
and also described in Held & Ott (2018), Section 3.2.
The corresponding minimum Bayes factor has already been derived in Johnson (2005).
As described there, assigning a scaled chi-sqaured distribution to the
non-centrality parameter of the F-distribution corresponds to
assigning a (multivariate) normal prior distribution
centered around the null value to the vector of regression coefficients.
A matrix containing the lower bounds on the Bayes factors
as entries,
for all combinations of p-value and sample size n
or dimension d
(whichever is multidimensional).
The values for the k-th sample size or dimension (k-th entry in the vector n
or d
) and the different p-values
are given in the k-th row.
Computation may fail for alternative="simple"
if the p-value p
is extremely small
and min{n, n-d} is also small. Warnings will be given in this case and
the returned value is minBF=NaN
.
Held , L. and Ott, M. (2016). How the maximal evidence of P -values against point null hypotheses depends on sample size. American Statistician, 70, 335–341
Held, L. and Ott, M. (2018). On p-values and Bayes factors. Annual Review of Statistics and Its Application, 5, 393–419.
Johnson, V. E. (2005). Bayes factors based on test statistics. Journal of the Royal Statistical Society, Series B 67:689–701.
FCalibrate(p=c(0.05, 0.01, 0.005), n=20, d=2, alternative="simple") # chi-squared alternatives FCalibrate(p=c(0.05, 0.01, 0.005), n=20, d=2, intercept=FALSE) FCalibrate(p=c(0.05, 0.01, 0.005), n=20, d=c(2, 5, 10)) FCalibrate(p=c(0.05, 0.01, 0.005), n=c(10, 20, 50), d=2) # plot for chi-squared alternatives: d=2 and different sample sizes n # note that the minimum Bayes factor decreases with decreasing sample # size p <- exp(seq(log(0.0001), log(0.3), by=0.01)) n <- c(5, 10, 20) minBF <- FCalibrate(p, n, d=2) # compare to the bound for large n minTBF <- LRCalibrate(p, df=2) par(las=1) matplot(p, t(minBF), ylim=c(0.0003, 1), type="l", xlab="two-sided F-test p-value", ylab="Minimum Bayes factor", log="xy", lty=1, lwd=2, axes=FALSE, main="Local normal alternatives") lines(p, minTBF, col="gray", lty=2, lwd=2) axis(1, at=c(0.0001, 0.0003, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3), as.character(c(format(c(0.0001,0.0003), nsmall=4, digits=4, scientific=FALSE), c(0.001, 0.003, 0.01, 0.03, 0.1, 0.3)))) my.values <- c(3000, 1000, 300, 100, 30, 10, 3, 1) my.at <- 1/my.values my.ylegend <- c(paste("1/", my.values[-length(my.values)], sep=""), "1") axis(2, at=my.at, my.ylegend) box() legend("bottomright", legend=rev(c("n=5", "n=10", "n=20", "n large")), col=rev(c(1:3, "gray")), lty=c(2, rep(1, times=3)), lwd=2)