Predictive model scoring
scoring(
response,
...,
type = "quantitative",
metrics = NULL,
weights = NULL,
names = NULL,
messages = 1
)
Observed response
model predictions (continuous predictions or class probabilities (matrices))
continuous or categorical response (the latter is automatically chosen if response is a factor, otherwise a continuous response is assumed)
which metrics to report
optional frequency weights
optional names of models coments (given as ..., alternatively these can be named arguments)
controls amount of messages/warnings (0: none)
Numeric matrix of dimension m x p, where m is the number of different models and p is the number of model metrics
data(iris)
set.seed(1)
dat <- csplit(iris,2)
g1 <- NB(Species ~ Sepal.Width + Petal.Length, data=dat[[1]])
g2 <- NB(Species ~ Sepal.Width, data=dat[[1]])
pr1 <- predict(g1, newdata=dat[[2]], wide=TRUE)
pr2 <- predict(g2, newdata=dat[[2]], wide=TRUE)
table(colnames(pr1)[apply(pr1,1,which.max)], dat[[2]]$Species)
#>
#> setosa versicolor virginica
#> setosa 22 0 0
#> versicolor 0 25 3
#> virginica 0 2 23
table(colnames(pr2)[apply(pr2,1,which.max)], dat[[2]]$Species)
#>
#> setosa versicolor virginica
#> setosa 17 1 8
#> versicolor 1 15 10
#> virginica 4 11 8
scoring(dat[[2]]$Species, pr1=pr1, pr2=pr2)
#> brier -logscore
#> pr1 0.08719281 0.1419163
#> pr2 0.51025054 0.8547227
## quantitative response:
scoring(response=1:10, prediction=rnorm(1:10))
#> rmse mae
#> prediction 6.502139 5.757058