Skip to contents

Evaluate model runs for calibration

Source: R/evaluate_models.R
evaluate_models.Rd

Evaluate the model runs produced by a call to alternate() with user-defined error functions.

Usage

evaluate_models(
  x,
  eval_funs = NULL,
  eval_weights = 1,
  param_cols,
  eval_cols,
  n_best = NULL,
  f_best = 0.01,
  scaling_fun = scale_min_median,
  ...
)

# S3 method for class 'cfp_altres'
evaluate_models(
  x,
  eval_funs = NULL,
  eval_weights = 1,
  param_cols = cfp_id_cols(cfp_layers_map(cfp_og_model(x))),
  eval_cols = NULL,
  n_best = NULL,
  f_best = 0.01,
  scaling_fun = scale_min_median,
  ...
)

Arguments

x

A cfp_altres object, as returned by alternate().

eval_funs

A named list of evaluation functions. Each function must accept the arguments x and param_cols that are passed from this function.

eval_weights

A vector of weights the same length of eval_funs or one. Alternatively a data.frame() that specifies the weight for any wished error_parameter (names of eval_funs) and param_cols combinations. Provide the weights as a numeric in the parameter_weight column.

param_cols

The columns that, together, define different parameters (e.g. different gases) for which NRMSEs should be calculated separately (e.g. "gas"). Defaults to the id_cols of layers_map. If no such distinction is wished, set to character()

eval_cols

A character vector of columns for which the model error should be returned separately. Must be a subset of param_cols and defaults to the complete set.

n_best

An integer number of runs to select as the best runs.

f_best

A numeric between 0 to 1 as the fraction of runs to select as the best. Defaults to 0.01.

scaling_fun

A scaling function. Defaults to min-median scaling.

...

Any arguments that need to be passed to the error_funs. Note that all matching arguments will be applied to each function!

Value

A list with components best_runs the runs with the lowest model error (ME), model_error the model error for all runs, models_evaluated the raw values returned by error_funs and best_runs_runmap, a cfp_run_map() which can be used to rerun the best_runs model configurations. Note, that for best_runs_runmap the value of run_id is remapped to values 1:n_best.