FOCISelector#

class pyFOCI.FOCISelector(max_features=None, min_delta=0, *, method='r_foci', standardize='normalize', rank_method='max', nn_strategy='grouping', nn_tie_breaking='random', random_state=None, n_jobs=None)#

Feature selector using hierarchical forward selection based on the nonlinear Azadkia–Chatterjee T_n coefficient, using the FOCI R reference form by default and the Fuchs form as an alternative (see references).

At each step, among remaining features, we choose the feature that maximizes the per-step score on the growing set S_k = S_{k-1} ∪ {j}.

Parameters:

max_featuresint or None, default=None

Maximum number of features to select. If None, no hard cap is applied and selection proceeds until early stopping (if min_delta is not None) or until all features are selected (if min_delta is None).

min_deltafloat or None, default=0

Minimum required improvement in the selection score to continue selecting. Behavior:

First step: select a feature only if best_score > min_delta; otherwise, select none.

Subsequent steps: continue only if best_score > previous_best + min_delta; otherwise, stop.

None disables early stopping (select up to max_features).

Notes:

min_delta can be negative to relax stopping, 0 to reproduce standard early stopping, and positive to require stricter improvement.

Compatibility with the reference implementation:

min_delta == 0 corresponds to stop=TRUE

min_delta is None corresponds to stop=FALSE

method{“r_foci”, “fuchs”}, default=”r_foci”

Selection scoring method:

“r_foci” (default): Azadkia–Chatterjee \(Q_n/S(y)\) numerator/denominator form, matching the FOCI R reference implementation’s selection and stopping.
“fuchs”: Fuchs (2024) closed-form score.

standardize{“normalize”, None}, default=”normalize”

If “normalize”, each column of X is standardized to zero mean and unit variance before computing nearest neighbors. If None, X is used as-is. Columns with zero variance are left unchanged.

rank_method{“max”, “average”}, default=”max”

Method used to assign one-based ranks to the target values. If “max”, tied values receive the maximum rank in their tie group. If “average”, tied values receive the average rank in their tie group.

nn_strategy{“grouping”, “radius”}, default=”grouping”

Strategy used to compute nearest neighbor tie sets. Ignored when nn_tie_breaking="first".

nn_tie_breaking{“random”, “mean”, “first”}, default=”random”

How to resolve ties among equally-distanced nearest neighbors. If “random”, one tied neighbor is selected at random. If “mean”, the mean target rank across all tied neighbors is used. If “first”, a single nearest neighbor per sample is queried; the first one returned by the nearest-neighbor method is used without computing tie sets. This option is available to study its impact on the outcome, stability and speed of the algorithm.

random_stateint, RandomState instance or None, default=None

Controls the random tie-breaking among nearest neighbors. Only used for nn_tie_breaking="random". Pass an int for reproducible results across multiple calls. If None, the global NumPy random state is used.

n_jobsint or None, default=None

Number of parallel worker processes used to score candidate features in each forward-selection round. None and 1 score candidates sequentially. -1 uses all available processors; values below -1 follow joblib’s convention. A fixed integer random_state assigns deterministic random streams per candidate, so results are reproducible and identical across worker counts.

Attributes:

n_features_in_int: Number of features seen during fit.
feature_names_in_ndarray of shape (n_features_in_,): Feature names seen during fit. Defined only when X has feature names.
support_mask_ndarray of shape (n_features_in_,), dtype=bool: Boolean mask of selected features determined during fit.
score_path_ndarray of shape (n_selected,): Values of the per-step selection scores; their interpretation depends on method.

References

Mona Azadkia and Sourav Chatterjee. A simple measure of conditional dependence. The Annals of Statistics, 49(6):3070–3102, 2021. https://doi.org/10.1214/21-AOS2073

R FOCI package (reference implementation): https://cran.r-project.org/package=FOCI

Sebastian Fuchs. Quantifying directed dependence via dimension reduction. Journal of Multivariate Analysis 201 (2024): 105266. https://doi.org/10.1016/j.jmva.2023.105266

Methods

`fit`(X, y)	Fit the selector by hierarchical forward selection maximizing the per-step score (see `method`) over the growing feature set.
`fit_transform`(X[, y])	Fit to data, then transform it.
`get_feature_names_out`([input_features])	Mask feature names according to selected features.
`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Get parameters for this estimator.
`get_support`([indices])	Get a mask, or integer index, of the features selected.
`inverse_transform`(X)	Reverse the transformation operation.
`set_output`(*[, transform])	Set output container.
`set_params`(**params)	Set the parameters of this estimator.
`transform`(X)	Reduce X to the selected features.

fit(X, y)#

Fit the selector by hierarchical forward selection maximizing the per-step score (see method) over the growing feature set.

Parameters:

Xarray-like of shape (n_samples, n_features): Training input samples.
yarray-like of shape (n_samples,): Target values.

Returns:

self

fit_transform(X, y=None, **fit_params)#

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

Xarray-like of shape (n_samples, n_features): Input samples.
yarray-like of shape (n_samples,) or (n_samples, n_outputs), default=None: Target values (None for unsupervised transformations).
**fit_paramsdict: Additional fit parameters. Pass only if the estimator accepts additional params in its fit method.

Returns:

X_newndarray array of shape (n_samples, n_features_new): Transformed array.

get_feature_names_out(input_features=None)#

Mask feature names according to selected features.

Parameters:

input_featuresarray-like of str or None, default=None

Input features.

If input_features is None, then feature_names_in_ is used as feature names in. If feature_names_in_ is not defined, then the following input feature names are generated: ["x0", "x1", ..., "x(n_features_in_ - 1)"].
If input_features is an array-like, then input_features must match feature_names_in_ if feature_names_in_ is defined.

Returns:

feature_names_outndarray of str objects: Transformed feature names.

get_metadata_routing()#

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routingMetadataRequest: A MetadataRequest encapsulating routing information.

get_params(deep=True)#

Get parameters for this estimator.

Parameters:

deepbool, default=True: If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

paramsdict: Parameter names mapped to their values.

get_support(indices=False)#

Get a mask, or integer index, of the features selected.

Parameters:

indicesbool, default=False: If True, the return value will be an array of integers, rather than a boolean mask.

Returns:

supportarray: An index that selects the retained features from a feature vector. If indices is False, this is a boolean array of shape [# input features], in which an element is True iff its corresponding feature is selected for retention. If indices is True, this is an integer array of shape [# output features] whose values are indices into the input feature vector.

inverse_transform(X)#

Reverse the transformation operation.

Parameters:

Xarray of shape [n_samples, n_selected_features]: The input samples.

Returns:

X_originalarray of shape [n_samples, n_original_features]: X with columns of zeros inserted where features would have been removed by transform().

set_output(*, transform=None)#

Set output container.

Refer to the user guide for more details and Introducing the set_output API for an example on how to use the API.

Parameters:

transform{“default”, “pandas”, “polars”}, default=None

Configure output of transform and fit_transform.

"default": Default output format of a transformer
"pandas": DataFrame output
"polars": Polars output
None: Transform configuration is unchanged

Added in version 1.4: "polars" option was added.

Returns:

selfestimator instance: Estimator instance.

set_params(**params)#

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters: