stability {stablelearner} | R Documentation |
Stability assessment of results from supervised statistical learning (i.e., recursive partitioning, support vector machines, neural networks, etc.). The procedure involves the pairwise comparison of results generated from learning samples randomly drawn from the original data set or directly from the data-generating process (if available).
stability(x, ..., data = NULL, control = stab_control(), weights = NULL, applyfun = NULL, cores = NULL, names = NULL)
x |
fitted model object. Any model object can be used whose class is
registered in |
... |
additional fitted model objects. |
data |
an optional |
control |
a list with control parameters, see |
weights |
an optional matrix of dimension n * B that can be used to
weight the observations from the original learning data when the models
are refitted. If |
applyfun |
a |
cores |
integer. The number of cores to use in multicore computations
using |
names |
a vector of characters to specify a name for each fitted model object. By default, the objects are named by their class. |
FIXME
For a single fitted model object, stability
returns an object of
class "stablelearner"
with the following components:
call |
the call from the model object |
learner |
the information about the learner retrieved from |
B |
the number of repetitions, |
sval |
a matrix containing the estimated similarity values for each
similarity measure specified in |
sampstat |
a list containing information on the size of the learning
samples ( |
data |
a language object referring to the |
control |
a list with control parameters used for assessing the stability, |
For several fitted model objects, stability
returns an object of
class "stablelearnerList"
which is a list of objects of class
"stablelearner"
.
Philipp M, Rusch T, Hornik K, Strobl C (2018). Measuring the “Stability of Results from Supervised Statistical Learning”. Journal of Computational and Graphical Statistics. Forthcoming
boxplot.stablelearnerList
, summary.stablelearner
## assessing the stability of a single result library("partykit") r1 <- ctree(Species ~ ., data = iris) stab <- stability(r1) summary(stab) ## assessing the stability of several results library("rpart") r2 <- rpart(Species ~ ., data = iris) stab <- stability(r1, r2, control = stab_control(seed = 0)) summary(stab, names = c("ctree", "rpart")) ## using case-weights instead of resampling stability(r1, weights = TRUE) ## using self-defined case-weights n <- nrow(iris) B <- 500 w <- array(sample(c(0, 1), size = n*B*3, replace = TRUE), dim = c(n, B, 3)) stability(r1, weights = w) ## assessing stability for a given data-generating process my_dgp <- function() dgp_twoclass(n = 100, p = 2, noise = 4, rho = 0.2) res <- ctree(class ~ ., data = my_dgp()) stability(res, data = my_dgp)