Glossary and Notations

Notations

target

\(Y\), y

In the documentation, the target variable is denoted by a capital letter \(Y\) when referring to it as a random variable. In the API and code examples, the target variable is denoted by a lowercase letter y, following the scikit-learn convention.

index

\(X^{j}\)

The superscript index notation is used to denote the \(j^{th}\) feature of the feature vector \(X\).

minus index

\(X^{-j}\)

The minus index notation is used to denote all features except the one with the given index. For instance, \(X^{-j}\) denotes all features except the \(j^{th}\) one.

minus group

\(X^{-G}\)

Similar to the minus index notation for individual features, we use the minus index notation to denote the complement of a group of features. For instance, \(X^{-G}\) denotes all features except the ones in the group \(G\).

marginal permutation of a feature

\(X^{\pi (j)}\)

To denote a perturbed version of an input \(X\) where the \(j^{th}\) feature has been permuted, we use the superscript notation \(\pi (j)\). This permutation is marginal, meaning that the values of the \(j^{th}\) feature are shuffled across samples, independently from the other features.

conditional permutation of a feature

\(X^{\pi(j| -j)}\)

To denote a perturbed version of an input \(X\) where the \(j^{th}\) feature has been sampled from its conditional distribution given all other features, we use the notation \(X^{\pi(j| -j)}\). This means that the values of the \(j^{th}\) are drawn from the distribution \(P(X^{j} | X^{-j})\).

knockoff feature

\(\tilde X\)

The knockoff input \(\tilde X\) is a perturbed version of the original input \(X\) constructed such that each knockoff feature is pairwise exchangeable with the original feature and independent from the output \(Y\) conditionally on the original features.

Glossary

CFI

Conditional Feature Importance

Conditional Feature Importance (CFI) is a measure of feature importance that consists in sampling a feature of interest from the conditional distribution of that feature given all other features, and measuring the performance drop triggered by this perturbation.

CluDL

Clustered Desparsified Lasso

Clustered Desparsified Lasso (CluDL) is an extension of the Desparsified Lasso to clusters of features. It aims at overcoming the limitations of the Desparsified Lasso when the number of features is large and the correlations between them are strong.

D0CRT

Distilled Conditional Randomization Testing

Distilled Conditional Randomization Testing (dCRT) is a method for feature selection based on conditional independence testing, which uses a distillation step to reduce computational cost.

EnCluDL

Ensemble Clustered Desparsified Lasso

Ensemble Clustered Desparsified Lasso (EnCluDL) is an extension of the Clustered Desparsified Lasso that combines multiple clusterings to de-randomize the procedure and improve robustness.

FDP

False Discovery Proportion

The False Discovery Proportion (FDP) is the ratio between the number of false discoveries and the total number of discoveries. Denoting \(\hat S\) the estimated set of important features, and \(S^*\) the true set of important features, the FDP is defined as:

\[\text{FDP}(\hat S) = \frac{|\hat S \setminus S^*|}{\max(|\hat S|, 1)}.\]

where \(|\cdot|\) denotes the cardinality of a set.

FDR

False Discovery Rate

The False Discovery Rate (FDR) is the expected value of the False Discovery Proportion (FDP). For a selection set \(\hat S\) it is defined as \(\text{FDR}(\hat S) = \mathbb{E}[\text{FDP}(\hat S)]\).

LOCO

Leave-One-Covariate-Out

The Leave-One-Covariate-Out (LOCO) is a measure of feature importance that consists in retraining a predictive model without the feature of interest and measuring performance drop triggered by this ablation.

PFI

Permutation Feature Importance

The Permutation Feature Importance (PFI) is a measure of feature importance that consists in permuting the values of the feature of interest and measuring the performance drop triggered by this perturbation.