FOCISelector#

class pyFOCI.FOCISelector(max_features=None, min_delta=0, standardize='normalize', nn_strategy='grouping', random_state=None)#

Feature selector using hierarchical forward selection based on the nonlinear Azadkia–Chatterjee T_n coefficient and its Fuchs form (see references).

At each step, among remaining features, we choose the feature that maximizes the cumulative T_n on the growing set S_k = S_{k-1} ∪ {j}.

Parameters:

max_featuresint or None, default=None

Maximum number of features to select. If None, no hard cap is applied and selection proceeds until early stopping (if min_delta is not None) or until all features are selected (if min_delta is None).

min_deltafloat or None, default=0

Minimum required improvement in the cumulative T_n to continue selecting. Behavior:

First step: select a feature only if best_Tn > min_delta; otherwise, select none.

Subsequent steps: continue only if best_Tn > previous_best + min_delta; otherwise, stop.

None disables early stopping (select up to max_features).

Notes:

min_delta can be negative to relax stopping, 0 to reproduce standard early stopping, and positive to require stricter improvement.

Compatibility with the reference implementation:

min_delta == 0 corresponds to stop=TRUE

min_delta is None corresponds to stop=FALSE

standardize{“normalize”, None}, default=”normalize”

If “normalize”, each column of X is standardized to zero mean and unit variance before computing nearest neighbors. If None, X is used as-is. Columns with zero variance are left unchanged.

nn_strategy{“grouping”, “radius”}, default=”grouping”

Strategy used to select nearest neighbors for computing \(T_n\).

random_stateint, RandomState instance or None, default=None

Controls the random tie-breaking among nearest neighbors. Pass an int for reproducible results across multiple calls. If None, the global NumPy random state is used.

Attributes:

n_features_in_int: Number of features seen during fit.
feature_names_in_ndarray of shape (n_features_in_,): Feature names seen during fit. Defined only when X has feature names.
support_mask_ndarray of shape (n_features_in_,), dtype=bool: Boolean mask of selected features determined during fit.
Tn_path_ndarray of shape (n_selected,): Values of the cumulative T_n along the selection path.

References

Mona Azadkia and Sourav Chatterjee. A simple measure of conditional dependence. The Annals of Statistics, 49(6):3070–3102, 2021. https://doi.org/10.1214/21-AOS2073

R FOCI package (reference implementation): https://cran.r-project.org/package=FOCI

Sebastian Fuchs. Quantifying directed dependence via dimension reduction. Journal of Multivariate Analysis 201 (2024): 105266. https://doi.org/10.1016/j.jmva.2023.105266

Methods

`fit`(X, y)	Fit the selector by hierarchical forward selection maximizing T_n over the growing feature set.
`fit_transform`(X[, y])	Fit to data, then transform it.
`get_feature_names_out`([input_features])	Mask feature names according to selected features.
`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Get parameters for this estimator.
`get_support`([indices])	Get a mask, or integer index, of the features selected.
`inverse_transform`(X)	Reverse the transformation operation.
`set_output`(*[, transform])	Set output container.
`set_params`(**params)	Set the parameters of this estimator.
`transform`(X)	Reduce X to the selected features.

fit(X, y)#

Fit the selector by hierarchical forward selection maximizing T_n over the growing feature set.

Parameters:

Xarray-like of shape (n_samples, n_features): Training input samples.
yarray-like of shape (n_samples,): Target values.

Returns:

self

fit_transform(X, y=None, **fit_params)#

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

Xarray-like of shape (n_samples, n_features): Input samples.
yarray-like of shape (n_samples,) or (n_samples, n_outputs), default=None: Target values (None for unsupervised transformations).
**fit_paramsdict: Additional fit parameters. Pass only if the estimator accepts additional params in its fit method.

Returns:

X_newndarray array of shape (n_samples, n_features_new): Transformed array.

get_feature_names_out(input_features=None)#

Mask feature names according to selected features.

Parameters:

input_featuresarray-like of str or None, default=None

Input features.

If input_features is None, then feature_names_in_ is used as feature names in. If feature_names_in_ is not defined, then the following input feature names are generated: ["x0", "x1", ..., "x(n_features_in_ - 1)"].
If input_features is an array-like, then input_features must match feature_names_in_ if feature_names_in_ is defined.

Returns:

feature_names_outndarray of str objects: Transformed feature names.

get_metadata_routing()#

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routingMetadataRequest: A MetadataRequest encapsulating routing information.

get_params(deep=True)#

Get parameters for this estimator.

Parameters:

deepbool, default=True: If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

paramsdict: Parameter names mapped to their values.

get_support(indices=False)#

Get a mask, or integer index, of the features selected.

Parameters:

indicesbool, default=False: If True, the return value will be an array of integers, rather than a boolean mask.

Returns:

supportarray: An index that selects the retained features from a feature vector. If indices is False, this is a boolean array of shape [# input features], in which an element is True iff its corresponding feature is selected for retention. If indices is True, this is an integer array of shape [# output features] whose values are indices into the input feature vector.

inverse_transform(X)#

Reverse the transformation operation.

Parameters:

Xarray of shape [n_samples, n_selected_features]: The input samples.

Returns:

X_originalarray of shape [n_samples, n_original_features]: X with columns of zeros inserted where features would have been removed by transform().

set_output(*, transform=None)#

Set output container.

Refer to the user guide for more details and Introducing the set_output API for an example on how to use the API.

Parameters:

transform{“default”, “pandas”, “polars”}, default=None

Configure output of transform and fit_transform.

"default": Default output format of a transformer
"pandas": DataFrame output
"polars": Polars output
None: Transform configuration is unchanged

Added in version 1.4: "polars" option was added.

Returns:

selfestimator instance: Estimator instance.

set_params(**params)#

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters: