| Type: | Package |
| Title: | Meta Fuzzy Functions for Aggregating Prediction Pools |
| Version: | 1.0.0 |
| Description: | Aggregates numeric candidate predictions using Meta Fuzzy Functions and membership-derived weights learned on validation predictions, following Tak (2018) <doi:10.1016/j.asoc.2018.08.009> and Tak (2021) <doi:10.1016/j.ins.2021.01.024>. Provides a common prediction-pool interface, fixed and tuned fits, explicit validation-based selection, and standard methods for inspecting weights, predicting and plotting results. Candidate pools may contain different regression learners, bootstrap fits, alternative hyperparameter configurations, or externally generated regression and time-aligned forecasting predictions. Resampling results of 'caret' and 'tidymodels' can be converted into pools, and forecasting pools are built by rolling-origin evaluation or from 'fable' forecasts. Supports fuzzy c-means, possibilistic fuzzy c-means following Pal et al. (2005) <doi:10.1109/TFUZZ.2004.840099>, Gustafson-Kessel clustering with the improved covariance estimation of Babuska et al. (2002) <doi:10.1109/FUZZ.2002.1006654> and k-means clustering. Optional helpers train built-in or user-defined regression learners on explicit data splits, cross-validation folds or bootstrap samples. |
| License: | MIT + file LICENSE |
| Encoding: | UTF-8 |
| LazyData: | true |
| Depends: | R (≥ 3.2.0) |
| Imports: | e1071, parallel |
| Suggests: | glmnet, randomForest, xgboost, lightgbm, caret, caretEnsemble, tune, workflows, parsnip, rsample, stacks, forecast, fable, fabletools, tsibble, knitr, rmarkdown, testthat (≥ 3.1.7) |
| Config/testthat/edition: | 3 |
| VignetteBuilder: | knitr |
| NeedsCompilation: | no |
| Config/roxygen2/version: | 8.1.1 |
| Packaged: | 2026-11-20 21:43:39 UTC; nihattak |
| Author: | Nihat Tak [aut, cre], Sadık Çoban [ctb] |
| Maintainer: | Nihat Tak <nihattak@gmail.com> |
| Repository: | CRAN |
| Date/Publication: | 2026-11-22 00:00:18 UTC |
MFF: Meta fuzzy functions for aggregating prediction pools
Description
Model-agnostic aggregation of numeric candidate predictions using weights derived from clustering validation prediction profiles.
Main workflow
mff_pool() constructs aligned inputs from any prediction source.
model_train() and boot_train() optionally generate regression pools;
mff_cv() makes model_train() use out-of-fold validation predictions, and
as_mff_pool() converts caret, tidymodels, stacks and fable results.
rolling_train() builds rolling-origin forecasting pools.
fit_mff() fits fixed configurations and tune_mff() searches a grid.
predict.mff_fit() applies validation-selected weights to new predictions.
evaluate() reports MAE, RMSE, MAPE, SMAPE, MSE and median absolute error.
Classes and access
Inputs use mff_prediction_pool. Fixed fits use mff_fit; tuned fits
inherit from mff_fit through mff_tuned.
Use validation_predictions(), weights.mff_fit(),
selected_configuration() and search_results() to inspect results.
All three object types have print and summary methods; fits have plot methods
including a weight heatmap. See mff_example() for a real-data pool.
External forecasting
Ordered external point forecasts can be supplied with
task = "forecasting" and validation/test indexes.
The package checks index alignment and ordering. External training and
information-leakage prevention remain the user's responsibility.
Acknowledgements
Supported by TUBITAK under project 125F138.
Author(s)
Maintainer: Nihat Tak nihattak@gmail.com
Authors:
Nihat Tak nihattak@gmail.com
Other contributors:
Sadık Çoban sadikc716@gmail.com [contributor]
References
Tak, N. (2018). Meta fuzzy functions: Application of recurrent type-1 fuzzy functions. Applied Soft Computing, 73, 1–13. doi:10.1016/j.asoc.2018.08.009. Tak, N. (2021). Forecast combination with meta possibilistic fuzzy functions. Information Sciences, 560, 168–182. doi:10.1016/j.ins.2021.01.024. Tak, N. and Ucan, A. (2026). Meta fuzzy feature-selection-based regression functions. Applied Soft Computing, 190, 114592. doi:10.1016/j.asoc.2026.114592.
Convert resampling results of other modeling frameworks to prediction pools
Description
as_mff_pool() turns out-of-fold predictions stored by another modeling
framework into an mff_pool(). The out-of-fold predictions and the
corresponding training outcomes form the validation part of the pool.
When newdata is supplied, test predictions are computed by the
framework's own prediction methods from models fitted on all training rows.
The frameworks are optional; their packages are needed only for the
corresponding method.
Usage
as_mff_pool(x, ...)
## Default S3 method:
as_mff_pool(x, ...)
## S3 method for class 'list'
as_mff_pool(x, newdata = NULL, test_observed = NULL, ...)
## S3 method for class 'caretList'
as_mff_pool(x, newdata = NULL, test_observed = NULL, ...)
## S3 method for class 'tune_results'
as_mff_pool(x, newdata = NULL, test_observed = NULL, ...)
## S3 method for class 'data_stack'
as_mff_pool(x, newdata = NULL, test_observed = NULL, ...)
## S3 method for class 'fbl_ts'
as_mff_pool(x, data, horizon = 1L, n_test, ...)
Arguments
x |
An object to convert; see Details. |
... |
Reserved for future use. |
newdata |
Optional data frame of test rows to predict. |
test_observed |
Optional observed test outcomes, used only for scoring. |
data |
For a fable, the tsibble holding the observed series. |
horizon |
For a fable, the forecast step to keep from every origin. |
n_test |
For a fable, the number of final target times used as test.
As in |
Details
Supported inputs:
- A list of caret models
A named list of regression models from
caret::train(), or acaretListfrom caretEnsemble, trained withsavePredictions = "final"or"all". Each model contributes the out-of-fold predictions of its selected tuning parameters, averaged over repeated resamples, and becomes one candidate (pool_type = "heterogeneous"). All models must have predicted the same training rows. Test predictions use the final models.- tidymodels tuning results
The result of
tune::tune_grid()for a regression workflow withcontrol_grid(save_pred = TRUE). Every parameter combination becomes one candidate (pool_type = "hyperparameter"), with out-of-fold predictions averaged over repeated resamples. Test predictions requiresave_workflow = TRUE: each combination is finalized and refitted on the full training data before predictingnewdata.- A stacks data stack
A
data_stackfromstacks::add_candidates(). Its candidate columns become candidates of a validation-only pool; test predictions are not available from a data stack.
Value
An mff_prediction_pool. pool_info(pool)$metadata$source names the
framework.
See Also
Examples
if (requireNamespace("caret", quietly = TRUE)) {
set.seed(1)
dat <- datasets::mtcars
ctrl <- caret::trainControl(method = "cv", number = 4,
savePredictions = "final",
index = caret::createFolds(dat$mpg[1:26], k = 4,
returnTrain = TRUE))
models <- list(
linear = caret::train(mpg ~ wt + hp + disp, dat[1:26, ], method = "lm",
trControl = ctrl),
knn = caret::train(mpg ~ wt + hp + disp, dat[1:26, ], method = "knn",
trControl = ctrl, tuneLength = 2),
linear_small = caret::train(mpg ~ wt, dat[1:26, ], method = "lm",
trControl = ctrl)
)
pool <- as_mff_pool(models, newdata = dat[27:32, ],
test_observed = dat$mpg[27:32])
fit <- fit_mff(pool, clusters = 2, seed = 1)
weights(fit)
}
Generate a bootstrap regression prediction pool
Description
Fit one learner to bootstrap samples of training rows and predict identical validation and test rows for every candidate.
Usage
boot_train(
formula,
data,
split,
times,
learner = "lm",
controls = list(),
control = mff_control()
)
Arguments
formula |
A formula with a single untransformed numeric response. Formula offsets are known additive response-scale contributions: learners fit the training response minus its offset, then the appropriate validation or test offset is added to predictions. Offsets must be finite. |
data |
A data frame. Missing values in used variables are rejected. |
split |
A list created by |
times |
Number of bootstrap candidates. |
learner |
A built-in learner name accepted by |
controls |
A named control list for the learner, as for one candidate
in |
control |
An |
Value
An mff_prediction_pool with uniquely named bootstrap candidates.
Examples
pool <- boot_train(Volume ~ Girth + Height, datasets::trees,
mff_split(1:15, 16:23, 24:31), times = 6,
control = mff_control(seed = 4))
dim(validation_predictions(pool))
fit <- tune_mff(pool, clusters = 2:3, method = "kmeans")
evaluate(predict(fit, pool), test_observed(pool))
Compute error metrics for predicted values or prediction matrices
Description
Compute error metrics for predicted values or prediction matrices by comparing them with the corresponding true response values.
Usage
evaluate(predicted, actual)
Arguments
predicted |
numeric vector or numeric matrix of predictions. If matrix, columns are evaluated separately. |
actual |
numeric vector of true target values. |
Details
The evaluate function is used to quantify the predictive performance of meta fuzzy function predictions by comparing them with the corresponding true response values. It supports both vector- and matrix-valued prediction inputs, allowing performance assessment of a single meta fuzzy function as well as simultaneous evaluation of multiple meta fuzzy functions. MAPE is 100 times the mean of abs(error) / (abs(actual) + 1e-8). SMAPE is 100 times the mean of 2 * abs(error) / (abs(predicted) + abs(actual) + 1e-8). MAPE can be unstable when actual values are near zero. When a prediction vector is provided, the function computes a single set of performance metrics; when a prediction matrix is provided, metrics are computed separately for each meta fuzzy function.
Value
A matrix with columns MAE, RMSE, MAPE, SMAPE, MSE, and MedAE, with one row per evaluated prediction vector.
See Also
fit_mff() and tune_mff(), which use these metrics for validation-based
selection, and predict.mff_fit() for test predictions.
Examples
x <- seq(100)
y <- 2*x + stats::rnorm(100)
m <- stats::lm(y ~ x)
pred <- stats::predict(m)
evaluate(pred, y)
Fit a fixed MFF configuration
Description
Cluster candidate validation prediction profiles, normalize their memberships into aggregation weights, and select one meta function using validation loss.
Usage
fit_mff(
pool,
clusters,
method = c("fcm", "pfcm", "gk", "kmeans"),
m = 2,
eta = 2,
metric = "RMSE",
standardize = FALSE,
reduction = c("none", "pca"),
nstart = 1L,
iter_max = NULL,
seed = NULL,
reltol = NULL,
gamma = 0,
beta = 1e+15
)
Arguments
pool |
An |
clusters |
A cluster count, or a vector of counts for tuning; each must
be between two and the number of candidates. For k-means and PFCM, it
must be smaller than the number of candidates: |
method |
Membership method: fuzzy c-means (fcm), possibilistic fuzzy c-means (pfcm), Gustafson-Kessel (gk), or crisp k-means (kmeans). PFCM follows Pal, Pal, Keller and Bezdek (2005) and GK the modified algorithm with improved covariance estimation of Babuska, van der Veen and Kaymak (2002); both are implemented in this package (see Details). |
m |
Fuzziness exponent greater than one; ignored for k-means. |
eta |
PFCM typicality exponent, greater than 1; ignored for other methods. |
metric |
Validation selection metric, using the formulas in |
standardize |
Standardize each validation-observation coordinate across candidate profiles before clustering. Aggregation uses original predictions. |
reduction |
Use original coordinates or all nonzero PCA components. PCA preserves Euclidean distances but need not preserve GK/PFCM solutions. |
nstart |
Number of independent random starts per configuration, used by all four methods. The start with the smallest clustering objective is kept; validation error is not used to choose the start. |
iter_max |
Iteration limit per start, used by all four methods. Defaults: kmeans 10, fcm 100, pfcm 100, gk 1000. |
seed |
Optional local seed; when supplied the caller's RNG state is restored. |
reltol |
A finite positive convergence tolerance, or NULL to retain
defaults: sqrt(.Machine$double.eps) for FCM, 1e-5 for PFCM and 1e-3 for
GK. FCM uses relative objective change (control$reltol), PFCM the mean
absolute change of the prototype coordinates, |
gamma |
GK weighting parameter in |
beta |
GK condition-number threshold for the fuzzy covariance
matrices, greater than one; ignored for other methods. When the number
of validation observations is at least the number of candidates, the
fuzzy covariance matrices are singular, and their zero eigenvalues are set
to the largest eigenvalue divided by |
Details
Test matrices and test outcomes do not enter fitting. Meta-function selection is supervised by validation outcomes; clustering candidate prediction profiles does not use the outcomes.
PFCM follows the PFCM-AO algorithm of Pal, Pal, Keller and Bezdek (2005,
Table XI) with a = b = 1: memberships, typicalities and prototypes
are updated by their equations (21)-(23), and the penalties
\gamma_i are computed by their equation (11) with K = 1 from
a terminal FCM partition started from the same randomly selected data
points. Defaults follow the paper's protocol (tolerance 1e-5, at most 100
iterations). Weights are derived from the memberships u.
GK follows the modified Gustafson-Kessel algorithm of Babuska, van der
Veen and Kaymak (2002, Section V and Appendix). Each fuzzy covariance
matrix F_i is replaced by (1 - \gamma) F_i + \gamma
\det(F_0)^{1/n} I, where F_0 is the covariance matrix of all
clustered profiles and n their dimension; if its condition number
exceeds \beta, every eigenvalue \lambda with
\beta \lambda < \lambda_{max} is set to \lambda_{max} /
\beta. Distances are (z - v_i)^T \det(F_i)^{1/n} F_i^{-1} (z - v_i)
with unit cluster volumes. Starts are initialized as in the authors'
implementation, and a start converges when no membership changes by more
than reltol between iterations. The eigenvalue constraint keeps the
matrices invertible when clusters contain few or linearly dependent
profiles.
Value
An mff_fit list with candidate names, method, parameters, weights,
validation scores, selected function, metric, preprocessing and diagnostics.
Accessors include weights.mff_fit(), validation_scores(),
selected_configuration() and fit_diagnostics().
References
Tak, N. (2018). Meta fuzzy functions: Application of recurrent type-1 fuzzy functions. Applied Soft Computing, 73, 1–13. doi:10.1016/j.asoc.2018.08.009. Gustafson, D. E. and Kessel, W. C. (1979). Fuzzy clustering with a fuzzy covariance matrix. Proceedings of the IEEE Conference on Decision and Control, 761–766. doi:10.1109/CDC.1978.268028. Babuska, R., van der Veen, P. J. and Kaymak, U. (2002). Improved covariance estimation for Gustafson-Kessel clustering. Proceedings of the IEEE International Conference on Fuzzy Systems, 1081–1085. doi:10.1109/FUZZ.2002.1006654. Pal, N. R., Pal, K., Keller, J. M. and Bezdek, J. C. (2005). A possibilistic fuzzy c-means clustering algorithm. IEEE Transactions on Fuzzy Systems, 13, 517–530. doi:10.1109/TFUZZ.2004.840099.
Examples
pool <- mff_example()
fit <- fit_mff(pool, clusters = 2, method = "fcm", seed = 12,
nstart = 3, reltol = 1e-6)
fit
weights(fit)
evaluate(predict(fit, pool), test_observed(pool))
Configure MFF tuning
Description
Configure MFF tuning
Usage
mff_control(
seed = 123L,
parallel = FALSE,
workers = 1L,
verbose = FALSE,
error_action = c("continue", "stop")
)
Arguments
seed |
An integer seed, or NULL to use the current random state. |
parallel |
Whether to use a socket cluster for grid evaluation. |
workers |
A positive integer number of worker processes. |
verbose |
Whether to report the grid size. |
error_action |
Continue past failed configurations, or stop. |
Value
A validated list of tuning controls.
Examples
mff_control(seed = 123, parallel = FALSE)
Cross-validation for out-of-fold validation predictions
Description
Describes how model_train() produces validation predictions from the
training rows when no separate validation rows are available. Each training
row is predicted once by candidates trained on the other folds; these
out-of-fold predictions and the training outcomes form the validation part
of the pool. Test predictions come from candidates refitted on all training
rows.
Usage
mff_cv(folds = 5L, fold_id = NULL)
Arguments
folds |
Number of folds, at least two. Ignored when |
fold_id |
Optional fold labels, one per training row in the order of
|
Value
An mff_cv object.
Examples
dat <- datasets::trees
pool <- model_train(Volume ~ Girth + Height, dat,
mff_split(train = 1:24, validation = NULL, test = 25:31),
resampling = mff_cv(folds = 4), seed = 1)
dim(validation_predictions(pool))
pool_info(pool)$metadata$resampling$fold_id
Candidate predictions of diamond prices
Description
Validation and test predictions of seven regression learners for the price
of diamonds. The data are 2000 rows sampled at random, with a fixed seed,
from the diamonds data of the ggplot2 package. These rows were randomly
split into 1000 training, 500 validation and 500 test rows. Each learner was
trained by model_train() on the training rows with the formula
price ~ carat + cut + color + clarity + depth + table (cut, color and
clarity as unordered factors) and its default settings: penalized
regressions tuned by train-only cross-validation, random forests with 100
trees, and boosters with 200 rounds and one thread. The predictions are
stored so that examples need neither ggplot2 nor the optional learner
packages. They were computed with ggplot2 4.0.3, glmnet 5.1, randomForest
4.7.1.2, xgboost 3.2.1.1 and lightgbm 4.7.0.
mff_example() builds the data into a prediction pool.
Usage
mff_diamonds
Format
A data frame with 1000 rows and 10 columns:
- row
Row of the diamond in
ggplot2::diamonds.- set
Factor with levels
"validation"and"test".- price
Observed price in US dollars.
- lm, lasso, ridge, elastic_net
Predictions of the linear learners.
- random_forest, xgboost, lightgbm
Predictions of the tree-ensemble learners.
Source
Computed from the diamonds data in the ggplot2 package
(H. Wickham, ggplot2: Elegant Graphics for Data Analysis, Springer, 2016).
Examples
head(mff_diamonds)
with(mff_diamonds, tapply(price, set, mean))
Prediction-pool example built from diamond prices
Description
Builds the mff_diamonds predictions into a prediction pool: 500 validation and 500 test rows for seven regression learners of diamond price. Candidate information records the learner family. The example is an interface demonstration, not a benchmark.
Usage
mff_example()
Value
An mff_prediction_pool with seven candidate columns.
Examples
pool <- mff_example()
pool
fit <- fit_mff(pool, clusters = 2, method = "fcm", seed = 12)
weights(fit)
evaluate(predict(fit, pool), test_observed(pool))
Define a user learner
Description
Wraps any regression method so that model_train() and boot_train() can
train it on explicit splits, cross-validation folds or bootstrap samples.
The learner receives the numeric model matrix built from the formula on the
training rows (factor levels and contrasts are learned there) and the
training response minus any formula offset; the offset is added back to
its predictions.
Usage
mff_learner(fit, predict, name = "custom", intercept = FALSE)
## S3 method for class 'mff_learner'
print(x, ...)
Arguments
fit |
A function |
predict |
A function |
name |
A label recorded as the candidate's algorithm. |
intercept |
Keep the intercept column of the model matrix in |
x |
An |
... |
Ignored. |
Value
An mff_learner object.
Examples
# Projection pursuit regression from base R.
ppr_learner <- mff_learner(
fit = function(x, y, nterms = 2) stats::ppr(x, y, nterms = nterms),
predict = function(object, newx) stats::predict(object, newx),
name = "ppr"
)
dat <- datasets::airquality[stats::complete.cases(datasets::airquality), ]
set.seed(1)
rows <- sample(nrow(dat))
pool <- model_train(Ozone ~ Solar.R + Wind + Temp, dat,
mff_split(rows[1:60], rows[61:85], rows[86:111]),
learners = list(lm = "lm", ppr2 = ppr_learner,
ppr3 = ppr_learner),
controls = list(ppr3 = list(nterms = 3)), seed = 1)
candidate_info(pool)
Create an aligned prediction pool
Description
Construct the shared input to fixed and tuned MFF aggregation. Rows are observations and columns are candidate predictions, irrespective of the software or algorithm that generated them.
Usage
mff_pool(
validation_predictions,
validation_observed,
test_predictions = NULL,
test_observed = NULL,
candidate_info = NULL,
task = c("regression", "forecasting"),
pool_type = c("external", "heterogeneous", "bootstrap", "hyperparameter"),
validation_index = NULL,
test_index = NULL,
metadata = list()
)
Arguments
validation_predictions |
A finite numeric matrix with unique column names. |
validation_observed |
A finite numeric validation response vector. |
test_predictions |
An optional numeric matrix with the same candidate names and order as the validation matrix. |
test_observed |
Optional observed test responses; used only for scoring. |
candidate_info |
Optional data frame with a |
task |
Either |
pool_type |
A descriptive pool type: external, heterogeneous, bootstrap, or hyperparameter. This label does not change the aggregation algorithm. |
validation_index, test_index |
Optional observation indexes. Forecasting
requires strictly increasing numeric, Date, or POSIXct indexes; validation
must precede test. This checks alignment, not how external models were trained.
For |
metadata |
A list of provenance or split information. |
Value
An mff_prediction_pool list. Its matrices, outcomes, candidate
information and provenance are accessed using validation_predictions(),
test_predictions(), candidate_info() and pool_info().
See Also
fit_mff(), tune_mff(), model_train(), boot_train()
Examples
pool <- mff_example()
pool
dim(validation_predictions(pool))
candidate_info(pool)
Define training, validation and test rows
Description
Define training, validation and test rows
Usage
mff_split(train, validation, test)
Arguments
train, test |
Nonempty vectors of positive integer row indexes. |
validation |
A nonempty vector of positive integer row indexes, or
NULL when validation predictions are produced by cross-validation on the
training rows (see |
Value
A list with three disjoint index vectors, used by model_train()
and boot_train(); validation has length zero when it was NULL.
Omitted data rows are not used.
Examples
split <- mff_split(1:15, 16:23, 24:31)
cv_split <- mff_split(train = 1:23, validation = NULL, test = 24:31)
Train regression candidates on explicit data splits
Description
Generate aligned predictions for a common pool. Candidate training is
optional: external predictions can be supplied directly to mff_pool().
Usage
model_train(
formula,
data,
split,
learners = "lm",
controls = list(),
seed = 123L,
resampling = NULL
)
Arguments
formula |
A formula with a single untransformed numeric response. Formula offsets are known additive response-scale contributions: learners fit the training response minus its offset, then the appropriate validation or test offset is added to predictions. Offsets must be finite. |
data |
A data frame. Missing values in used variables are rejected. |
split |
A list created by |
learners |
Unique learner names: lm, lasso, ridge, elastic_net,
random_forest, xgboost, or lightgbm; or a named list whose elements are
such names or |
controls |
A named list of named control lists, one per candidate name.
For an |
seed |
A local seed, or NULL. |
resampling |
NULL to predict the validation rows of |
Value
An mff_prediction_pool, including candidate settings and split
provenance. One candidate is allowed as a baseline, but fitting MFF needs
at least two. Unseen factor levels fail; encoding is learned on train rows.
With cross-validation, a factor level that occurs only in a held-out fold
also fails, and the fold assignment is stored in
pool_info(pool)$metadata$resampling.
Examples
pool <- model_train(Volume ~ Girth + Height, datasets::trees,
mff_split(1:15, 16:23, 24:31), learners = "lm")
summary(pool)
# Several candidates with learner settings given in controls; the settings
# are stored with the pool and printed by pool_info().
if (requireNamespace("glmnet", quietly = TRUE) &&
requireNamespace("randomForest", quietly = TRUE)) {
pool <- model_train(Volume ~ Girth + Height, datasets::trees,
mff_split(1:15, 16:23, 24:31),
learners = list(lm = "lm", ridge = "ridge",
forest_small = "random_forest",
forest_large = "random_forest"),
controls = list(ridge = list(nfolds = 3),
forest_small = list(ntree = 25),
forest_large = list(ntree = 500,
nodesize = 3)),
seed = 1)
pool_info(pool)
}
Plot an MFF fit
Description
Draw candidate weights, validation scores, tuning-search losses, or test diagnostics for a fixed or tuned MFF fit.
Usage
## S3 method for class 'mff_fit'
plot(
x,
type = c("weights", "weight_heatmap", "validation_scores", "test_scores",
"observed_predicted", "series", "search"),
metric = NULL,
newdata = NULL,
observed = NULL,
show_all = FALSE,
heatmap_values = TRUE,
heatmap_digits = 3L,
...
)
Arguments
x |
An |
type |
Plot weights (grouped bars), a weight heatmap, validation scores,
test scores, observed versus predicted test values, a test series, or
tuning-search losses ( |
metric |
A metric supported by |
newdata |
A candidate prediction matrix or an |
observed |
Observed test outcomes for test plots. For a pool, stored test
outcomes are used when |
show_all |
Whether test plots show all functions of the fitted configuration. The default shows only the function selected on validation data; test plots never reselect a function. |
heatmap_values |
Whether to print weights in heatmap cells. |
heatmap_digits |
Nonnegative integer number of decimal places. |
... |
Additional graphics arguments passed to the base plotting function; they replace the defaults of the plot, such as the axis labels. |
Value
The fitted object, invisibly.
Examples
pool <- mff_example()
fit <- tune_mff(pool, clusters = 2:3, method = "kmeans")
plot(fit, type = "weight_heatmap")
plot(fit, type = "observed_predicted", newdata = pool)
plot(fit, type = "search")
Predict from an MFF fit
Description
Apply validation-selected weights to aligned candidate predictions.
Usage
## S3 method for class 'mff_fit'
predict(object, newdata, type = c("selected", "all"), ...)
Arguments
object |
An |
newdata |
A finite numeric prediction matrix, or an |
type |
Use the validation-selected function or all functions in the
fitted configuration. |
... |
Reserved for future use. |
Value
A numeric vector for "selected" or a numeric matrix for "all".
Observed test outcomes are never used to select a function.
Examples
pool <- mff_example()
fit <- fit_mff(pool, clusters = 2, method = "kmeans", seed = 3)
predict(fit, pool)
predict(fit, test_predictions(pool), type = "all")
Display an MFF fit
Description
Displays the selected engine's convergence status, iteration count and
messages. Iteration-limit and backend-warning results are identified
explicitly; see fit_diagnostics() for the status values.
Usage
## S3 method for class 'mff_fit'
print(x, ...)
Arguments
x |
An |
... |
Reserved for future use. |
Value
The input object, invisibly.
Display a prediction pool
Description
Display a prediction pool
Usage
## S3 method for class 'mff_prediction_pool'
print(x, ...)
Arguments
x |
An |
... |
Reserved for future use. |
Value
The input, invisibly.
Build a forecasting pool by rolling-origin evaluation
Description
Produces aligned validation and test forecasts for a univariate series.
For every target time t of the validation and test blocks, each model
is fitted to the observations up to the origin t - h, where h
is horizon, and its h-step-ahead point forecast becomes the
candidate prediction for t. A model therefore never receives an
observation later than its origin. The test block consists of the last
n_test observations. The validation block ends at the origin of the first
test target, h steps before it, so that no validation outcome used to
learn and select the aggregation is observed after any test origin; for
h > 1 the h - 1 target times between the blocks are not used.
The earliest origin must leave at least min_train observations.
Usage
rolling_train(
y,
models,
horizon = 1L,
n_validation,
n_test,
window = c("expanding", "sliding"),
window_size = NULL,
min_train = 2L,
index = NULL,
control = mff_control()
)
Arguments
y |
A numeric vector or univariate |
models |
A named list of functions |
horizon |
Forecast horizon |
n_validation, n_test |
Numbers of validation and test target times. |
window |
|
window_size |
Window length for |
min_train |
Minimum number of observations a model may receive. |
index |
Optional strictly increasing numeric, Date or POSIXct index of
|
control |
An |
Value
An mff_prediction_pool with task = "forecasting". The origins
of the validation and test forecasts and the unused target times between
the blocks (all in the units of index) and the settings are stored in
pool_info(pool)$metadata.
See Also
as_mff_pool() for forecasts already produced with fable.
Examples
y <- datasets::AirPassengers
models <- list(
naive = function(y, h) rep(utils::tail(y, 1), h),
seasonal_naive = function(y, h) utils::tail(y, 12)[seq_len(h)],
drift = function(y, h) {
n <- length(y)
utils::tail(y, 1) + seq_len(h) * (y[n] - y[1]) / (n - 1)
}
)
pool <- rolling_train(y, models, horizon = 1, n_validation = 24, n_test = 12)
pool
fit <- fit_mff(pool, clusters = 2, seed = 1)
evaluate(predict(fit, pool), test_observed(pool))
Summarize an MFF fit
Description
The printed summary includes convergence status, iterations and engine messages, including an explicit note when the iteration limit was reached or the backend emitted warnings.
Usage
## S3 method for class 'mff_fit'
summary(object, ...)
## S3 method for class 'mff_fit_summary'
print(x, ...)
Arguments
object |
An |
... |
Reserved for future use. |
x |
An |
Value
An mff_fit_summary list. Fixed and tuned summaries share the same
fields; search is NULL for fixed fits and a data frame for tuned fits.
Summarize a prediction pool
Description
Summarize a prediction pool
Usage
## S3 method for class 'mff_prediction_pool'
summary(object, ...)
## S3 method for class 'mff_pool_summary'
print(x, ...)
Arguments
object |
An |
... |
Reserved for future use. |
x |
A pool summary. |
Value
A classed mff_pool_summary list with dimensions and candidate metadata.
Tune MFF configurations on validation predictions
Description
Search an explicit grid and return the selected fit with search diagnostics.
Usage
tune_mff(
pool,
clusters,
method = c("fcm", "pfcm", "gk", "kmeans"),
m = seq(1.5, 2.5, by = 0.1),
eta = 2,
metric = "RMSE",
standardize = FALSE,
reduction = c("none", "pca"),
nstart = 1L,
iter_max = NULL,
control = mff_control(),
reltol = NULL,
gamma = 0,
beta = 1e+15
)
Arguments
pool |
An |
clusters |
A cluster count, or a vector of counts for tuning; each must
be between two and the number of candidates. For k-means and PFCM, it
must be smaller than the number of candidates: |
method |
Membership method: fuzzy c-means (fcm), possibilistic fuzzy c-means (pfcm), Gustafson-Kessel (gk), or crisp k-means (kmeans). PFCM follows Pal, Pal, Keller and Bezdek (2005) and GK the modified algorithm with improved covariance estimation of Babuska, van der Veen and Kaymak (2002); both are implemented in this package (see Details). |
m |
Fuzziness exponent greater than one; ignored for k-means. |
eta |
PFCM typicality exponent, greater than 1; ignored for other methods. |
metric |
Validation selection metric, using the formulas in |
standardize |
Standardize each validation-observation coordinate across candidate profiles before clustering. Aggregation uses original predictions. |
reduction |
Use original coordinates or all nonzero PCA components. PCA preserves Euclidean distances but need not preserve GK/PFCM solutions. |
nstart |
Number of independent random starts per configuration, used by all four methods. The start with the smallest clustering objective is kept; validation error is not used to choose the start. |
iter_max |
Iteration limit per start, used by all four methods. Defaults: kmeans 10, fcm 100, pfcm 100, gk 1000. |
control |
A control list created by |
reltol |
A finite positive convergence tolerance, or NULL to retain
defaults: sqrt(.Machine$double.eps) for FCM, 1e-5 for PFCM and 1e-3 for
GK. FCM uses relative objective change (control$reltol), PFCM the mean
absolute change of the prototype coordinates, |
gamma |
One or more GK weighting parameters in |
beta |
GK condition-number threshold for the fuzzy covariance
matrices, greater than one; ignored for other methods. When the number
of validation observations is at least the number of candidates, the
fuzzy covariance matrices are singular, and their zero eigenvalues are set
to the largest eigenvalue divided by |
Details
Grid order follows expand.grid(m, clusters, eta) for PFCM and
expand.grid(m, clusters, gamma) for GK, with the method-irrelevant
dimensions removed. Exact ties select the first grid
row, then the first meta function. Failed rows are recorded; all-failed
searches stop. Only validation data are sent to workers.
Value
An object of class c("mff_tuned", "mff_fit"). All mff_fit
fields retain their meaning. Additional fields are a search data frame
(grid, seed, success, loss, elapsed time and messages) and tuning controls.
Examples
pool <- mff_example()
fit <- tune_mff(pool, clusters = 2:3, method = "fcm", m = c(1.5, 2),
nstart = 3, reltol = 1e-6, control = mff_control(seed = 12))
selected_configuration(fit)
search_results(fit)
Access prediction-pool components
Description
Retrieve aligned predictions, observed responses and candidate metadata without depending on the internal list layout.
Usage
validation_predictions(x)
validation_observed(x)
test_predictions(x)
test_observed(x)
candidate_names(x)
candidate_info(x)
pool_info(x)
## S3 method for class 'mff_pool_info'
print(x, ...)
Arguments
x |
An |
... |
Ignored. |
Value
Predictions are numeric matrices; responses are numeric vectors.
Missing test components return NULL. Candidate names are a character
vector, candidate information a data frame, and pool information a list
of class mff_pool_info containing task, type, indexes, and provenance.
Its print method shows a compact summary; the full components are
accessed with $ or [[.
Examples
pool <- mff_example()
head(validation_predictions(pool))
head(test_observed(pool))
pool_info(pool)
Access fitted MFF results
Description
Retrieve validation scores, the selected function or configuration, and diagnostics without accessing internal list components.
Usage
validation_scores(object)
selected_function(object)
selected_configuration(object)
## S3 method for class 'mff_configuration'
print(x, ...)
search_results(object)
fit_diagnostics(object)
Arguments
object |
An |
x |
An |
... |
Ignored. |
Value
validation_scores() returns a numeric matrix with one row per
meta function; selected_function() returns its selected integer index;
selected_configuration() returns a named list (class
mff_configuration) of the method, the metric and the parameters that
apply to the method; fit_diagnostics()
returns engine diagnostics and preprocessing metadata. Its convergence
element is "converged" when the selected start met the backend stopping
rule (the tolerance for FCM, PFCM and GK; no reassignment for kmeans)
before iter_max without warnings, "iteration_limit" when iter_max
was reached first, and "backend_warning" when the rule was met but the
backend emitted warnings, so convergence is not confirmed.
search_results() returns the tuning data frame and requires mff_tuned.
Examples
fit <- tune_mff(mff_example(), clusters = 2:3, method = "kmeans")
validation_scores(fit)
selected_function(fit)
selected_configuration(fit)
search_results(fit)
Extract MFF aggregation weights
Description
Extract MFF aggregation weights
Usage
## S3 method for class 'mff_fit'
weights(object, type = c("selected", "all"), ...)
## S3 method for class 'mff_fit'
coef(object, ...)
Arguments
object |
An |
type |
Extract selected-function weights or all function weights. |
... |
Reserved for future use. |
Value
A named numeric vector for "selected", or a candidate-by-function
numeric matrix for "all". coef() returns selected aggregation
coefficients, with no intercept.
Examples
fit <- fit_mff(mff_example(), clusters = 2, method = "kmeans", seed = 3)
weights(fit)
weights(fit, type = "all")
coef(fit)