Smoothness Regularization

Fitting a Bézier simplex to a noisy dataset can sometimes result in an “oscillation” or “unstable” surface. To produce a smooth, visually appealing manifold that truly represents the underlying structure, PyTorch-BSF provides Smoothness Regularization.

This technique adds a penalty term to the loss function based on the distance between adjacent control points in the simplex.

The Smoothness Penalty

The smoothness penalty \(\mathcal L_{smooth}\) is defined as:

\[\mathcal L_{smooth} = \sum_{\text{adjacent } i, j} \|\mathbf x_i - \mathbf x_j\|^2\]

where \(\mathbf x_i\) and \(\mathbf x_j\) are adjacent control points in the Bézier simplex. Two control points are considered adjacent if their multi-indices differ by 1 in two components and are identical in others.

How to Use

You can enable smoothness regularization by passing the smoothness_weight argument to the fit() function.

import torch
import torch_bsf
from torch_bsf.sampling import simplex_random

ts = simplex_random(n_params=3, n_samples=200, seed=0)  # parameters on a triangle
xs = torch.sin(3 * ts[:, :2]) + 0.05 * torch.randn(200, 2)  # noisy values

# Fit with a smoothness weight (e.g., 0.01)
bs = torch_bsf.fit(
    params=ts,
    values=xs,
    degree=3,
    smoothness_weight=0.01,
)

Since the penalty is quadratic, the default "lstsq" solver minimizes the regularized loss exactly. With \(n\) samples of \(m\) values, the design matrix \(A\), the targets \(Y\) and the Laplacian \(L\) of the adjacency graph, it solves

\[(A^\top A + nm\,w\,L)\,P = A^\top Y,\]

which is the minimizer of \(\frac{1}{nm}\|AP - Y\|^2 + w\,\mathcal L_{smooth}\), the loss minimized by solver="trainer".

Ridge Regularization

The "lstsq" solver also takes ridge, the weight \(r \ge 0\) of the Tikhonov penalty \(r\|P\|^2\) on the control points, which is added to the left-hand side above as \(r I\). It keeps the control points small when the samples do not determine them (e.g., too few samples for the degree); without it, such problems get the minimum-norm solution.

bs = torch_bsf.fit(params=ts, values=xs, degree=3, ridge=1e-6)

CLI / MLflow

You can also specify the weights via the command-line interface or MLflow using the --smoothness_weight and --ridge flags.

CLI Example:

python -m torch_bsf --params params.csv --values values.csv --degree 3 --smoothness_weight 0.1

MLflow Example:

mlflow run https://github.com/opthub-org/pytorch-bsf -P params=params.csv -P values=values.csv -P degree=3 -P smoothness_weight=0.1

Bézier Simploids

smoothness_weight and ridge work in the same way for a Bézier simploid (fit(..., simploid_type=...), see Simploidal Sets) and for fit_simploidal_set(). Two control points of a Bézier simploid are adjacent if their multi-indices differ in a single factor, by moving one unit between two components of that factor (see adjacency()); for a Bézier simplex, this is the definition above. In a simploidal Bézier space, the stored control points adjacent in the control net of some simploid are adjacent, and a pair on a shared face counts once (see adjacency()).

from torch_bsf.simploidal import sample_simploid

t = sample_simploid((2, 1), 200, seed=0)  # parameters on a prism
x = torch.sin(3 * t[:, :2]) + 0.05 * torch.randn(200, 2)  # noisy values
bs = torch_bsf.fit(t, x, degree=3, simploid_type=(2, 1), smoothness_weight=0.01)

Choosing the Weight

The optimal value for smoothness_weight depends on the noise level of your data:

0.0 (Default): No regularization. Best for noise-free data.
0.01 - 0.1: Light regularization to smooth out minor jitter.
1.0 or more: Strong regularization. Use this if your data is very noisy or you want a very “flat” manifold.

Benefits

Noise Reduction: Effectively filters out high-frequency noise in the training set.
Manifold Stability: Prevents the Bézier simplex from “overfitting” to individual scattered observations.
Visual Quality: Produces smoother surfaces that are better for visualization and Pareto front approximation.