Automatic Degree Selection

Choosing the optimal degree for a Bézier simplex can be challenging. A degree that is too low may underfit and fail to capture the manifold’s complexity, while a degree that is too high may overfit to noise.

PyTorch-BSF provides an automated tool to select the best degree based on k-fold cross-validation.

How to Use

The select_degree() function iterates through multiple degrees and evaluates the mean squared error (MSE) using cross-validation.

import torch
import torch_bsf
from torch_bsf.model_selection.degree_selection import select_degree
from torch_bsf.sampling import simplex_random

# Your data
ts = simplex_random(n_params=3, n_samples=60, seed=0)  # parameters
xs = torch.sin(3 * ts[:, :2])  # values

# Select the best degree between 1 and 5
best_d = select_degree(
    params=ts,
    values=xs,
    min_degree=1,
    max_degree=5,
    num_folds=5,
    seed=0,  # shuffle the samples into the same folds every time
)

# Use the best degree for final fitting
bs = torch_bsf.fit(params=ts, values=xs, degree=best_d)
print(1 <= best_d <= 5)

True

How It Works

Iteration: The tool fits Bézier simplices with torch_bsf.fit() for degrees from min_degree to max_degree.
K-Fold CV: For each degree, it performs \(k\)-fold cross-validation: the samples are shuffled (unless shuffle=False) and split into num_folds folds, a model is fitted to all folds but one, and its MSE is measured on the held-out fold. The mean MSE across all validation folds is calculated.
Heuristic Search: If the CV error does not improve for patience consecutive degrees (1 by default), the search stops early to save time.
Optimal Selection: The degree with the lowest mean validation MSE is returned.

Configuration

Important parameters include:

min_degree / max_degree: Range of degrees to check.
num_folds: Number of cross-validation folds (capped at the number of samples).
patience: Number of consecutive degrees without improvement before stopping.
shuffle / seed: Whether to shuffle the samples before splitting them into folds, and the random seed for shuffling (also passed to fit()).

Any other keyword arguments are passed to torch_bsf.fit(), e.g., smoothness_weight, normalize or ridge. By default, each fold is fitted exactly by the "lstsq" solver. To train with the Lightning trainer instead (which needs the lightning extra; see Installation), pass solver="trainer" together with any Trainer arguments, such as max_epochs (smaller is faster, but might be less accurate for selection) and accelerator / devices.

To select the degree of a Bézier simploid (see Simploidal Sets), pass its simploid_type; every factor then gets the same degree.

best_d = select_degree(params=ts, values=xs, simploid_type=(2, 1))

When to Use

Exploratory Data Analysis: When you are unsure about the underlying manifold’s complexity.
Automated Processes: If you are building an automated pipeline for Pareto front approximation.
Model Optimization: When accuracy is more important than training time, and you want the most precisely tuned model.

Command-Line Interface

degree_selection can also be invoked directly from the command line as a Python module.

As a Python Module (CLI)

Run the module directly to search for the best degree and print it to stdout:

python -m torch_bsf.model_selection.degree_selection \
    --params params.csv \
    --values values.csv \
    --min_degree 1 \
    --max_degree 5 \
    --num_folds 5 \
    --seed 0

All options except --params and --values are optional and fall back to their defaults (--min_degree 1, --max_degree 5, --num_folds 5, --patience 1, --solver lstsq). Pass --simploid-type (e.g., --simploid-type 2,1) to select the degree of a Bézier simploid. The trainer options (--batch_size, --max_epochs, --accelerator and --devices) are used only with --solver trainer.

Example Output

With the default --loglevel INFO, the command prints per-degree progress to stderr and writes the selected degree to stdout:

INFO:__main__:Checking degree 1...
INFO:__main__:Degree 1: Mean MSE = 0.043644
INFO:__main__:Checking degree 2...
INFO:__main__:Degree 2: Mean MSE = 0.000454
INFO:__main__:Checking degree 3...
INFO:__main__:Degree 3: Mean MSE = 0.000177
INFO:__main__:Checking degree 4...
INFO:__main__:Degree 4: Mean MSE = 0.000115
INFO:__main__:Checking degree 5...
INFO:__main__:Degree 5: Mean MSE = 0.000278
INFO:__main__:No improvement for 1 consecutive degree(s) at degree 5, stopping.
Best degree: 4

The final line Best degree: <N> is the only output written to stdout. Pass --loglevel WARNING to suppress the per-degree progress messages.

Available Options

Run with --help to see all available options:

usage: python -m torch_bsf.model_selection.degree_selection
       [-h] --params PARAMS --values VALUES [--header HEADER]
       [--simploid-type A1,...,An] [--min_degree MIN_DEGREE]
       [--max_degree MAX_DEGREE] [--num_folds NUM_FOLDS] [--patience PATIENCE]
       [--seed SEED] [--solver {lstsq,trainer}] [--batch_size BATCH_SIZE]
       [--max_epochs MAX_EPOCHS] [--accelerator ACCELERATOR]
       [--devices DEVICES] [--loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}]

Automatic degree selection for a Bezier simplex (or simploid) via k-fold
cross-validation

options:
  -h, --help            show this help message and exit
  --params PARAMS       Path to the input parameters CSV file
  --values VALUES       Path to the output values CSV file
  --header HEADER       Number of header rows to skip (default: 0)
  --simploid-type, --simploid_type A1,...,An
                        Type of the Bezier simploid (default: a Bezier simplex
                        of dimension n_params - 1)
  --min_degree MIN_DEGREE
                        Minimum degree to search (default: 1)
  --max_degree MAX_DEGREE
                        Maximum degree to search (default: 5)
  --num_folds NUM_FOLDS
                        Number of cross-validation folds (default: 5)
  --patience PATIENCE   Consecutive non-improving degrees before stopping
                        (default: 1)
  --seed SEED           Random seed for the folds and training (default: none)
  --solver {lstsq,trainer}
                        'lstsq' solves the least-squares problem exactly,
                        'trainer' trains with Lightning (default: lstsq)
  --batch_size BATCH_SIZE
                        Batch size for --solver=trainer (default: full-batch)
  --max_epochs MAX_EPOCHS
                        Training epochs per fold for --solver=trainer
                        (default: 2)
  --accelerator ACCELERATOR
                        Accelerator for --solver=trainer (default: auto)
  --devices DEVICES     Devices for --solver=trainer, integer or 'auto'
                        (default: auto)
  --loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        Python logging level for degree selection progress
                        (default: INFO)

Via MLproject

The degree_selection entry point in MLproject calls the same module:

mlflow run https://github.com/opthub-org/pytorch-bsf \
    -e degree_selection \
    -P params=params.csv \
    -P values=values.csv \
    -P min_degree=1 \
    -P max_degree=5 \
    -P num_folds=5

The best degree is printed to the run’s stdout and can be reviewed in the MLflow UI. The simploid_type (default "", i.e., a Bézier simplex), solver (default lstsq), header (default 0), and, for solver=trainer, max_epochs (default 2), accelerator and devices (default "auto") parameters are also exposed. The seed and batch_size options are not exposed through the MLproject entry point; MLflow runs shuffle the folds at random and use full-batch loading.