Evaluate the model runs produced by a call to alternate()
with
user-defined error functions.
Usage
evaluate_models(
x,
eval_funs = NULL,
eval_weights = 1,
param_cols,
eval_cols,
n_best = NULL,
f_best = 0.01,
scaling_fun = scale_min_median,
...
)
# S3 method for class 'cfp_altres'
evaluate_models(
x,
eval_funs = NULL,
eval_weights = 1,
param_cols = cfp_id_cols(cfp_layers_map(cfp_og_model(x))),
eval_cols = NULL,
n_best = NULL,
f_best = 0.01,
scaling_fun = scale_min_median,
...
)
Arguments
- x
A cfp_altres object, as returned by
alternate()
.- eval_funs
A named list of evaluation functions. Each function must accept the arguments
x
andparam_cols
that are passed from this function.- eval_weights
A vector of weights the same length of
eval_funs
or one. Alternatively adata.frame()
that specifies the weight for any wishederror_parameter
(names ofeval_funs
) andparam_cols
combinations. Provide the weights as a numeric in theparameter_weight
column.- param_cols
The columns that, together, define different parameters (e.g. different gases) for which NRMSEs should be calculated separately (e.g. "gas"). Defaults to the id_cols of layers_map. If no such distinction is wished, set to
character()
- eval_cols
A character vector of columns for which the model error should be returned separately. Must be a subset of
param_cols
and defaults to the complete set.- n_best
An integer number of runs to select as the best runs.
- f_best
A numeric between 0 to 1 as the fraction of runs to select as the best. Defaults to 0.01.
- scaling_fun
A scaling function. Defaults to min-median scaling.
- ...
Any arguments that need to be passed to the
error_funs
. Note that all matching arguments will be applied to each function!
Value
A list with components best_runs
the runs with the lowest model
error (ME), model_error
the model error for all runs,
models_evaluated
the raw values returned by error_funs and
best_runs_runmap
, a cfp_run_map()
which can be used to rerun the
best_runs
model configurations. Note, that for best_runs_runmap
the
value of run_id
is remapped to values 1:n_best
.
Examples
# \donttest{
PROFLUX <- pro_flux(base_dat |> filter(site == "site_a"))
run_map <-
cfp_run_map(
PROFLUX,
params = list(TPS = c(0.9, 1.1)),
type = "factor",
n_runs = 5)
PF_alt <- alternate(
PROFLUX,
\(x) complete_soilphys(x, DSD0_formula = "a*AFPS^b", quiet = TRUE),
run_map)
evaluate_models(
PF_alt,
eval_funs = list("NRMSE_conc" = error_concentration)
)
#> Joining with `by = join_by(error_parameter)`
#> $best_runs
#> # A tibble: 1 × 2
#> run_id ME
#> <dbl> <dbl>
#> 1 4 0
#>
#> $model_error
#> # A tibble: 5 × 2
#> run_id ME
#> <dbl> <dbl>
#> 1 4 0
#> 2 5 0.777
#> 3 2 1
#> 4 3 7.60
#> 5 1 830.
#>
#> $models_evaluated
#> # A tibble: 5 × 7
#> # Groups: error_parameter, site, gas [1]
#> error_parameter site gas NRMSE run_id sNRMSE parameter_weight
#> <chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 NRMSE_conc site_a CO2 0.276 1 830. 1
#> 2 NRMSE_conc site_a CO2 0.209 2 1 1
#> 3 NRMSE_conc site_a CO2 0.209 3 7.60 1
#> 4 NRMSE_conc site_a CO2 0.209 4 0 1
#> 5 NRMSE_conc site_a CO2 0.209 5 0.777 1
#>
#> $best_runs_runmap
#>
#> A cfp_run_map to be used in alternate().
#> number of runs: 1
#> parameters to alternate:
#> param param_id
#> 1 TPS 1
#> run_id param value type gas param_id
#> 1 1 TPS 1.07765 factor CO2 1
#>
# }