ExpRegularization

phasic.svgd.ExpRegularization(first_reg=1.0, last_reg=0.0, tau=1000.0)

Exponential decay schedule: reg = first_reg * exp(-iteration/tau) + last_reg * (1 - exp(-iteration/tau)).

This schedule helps by starting with strong moment regularization to guide initial exploration, then gradually reducing regularization as optimization converges to allow fine-tuning.

Parameters

first_reg : float = 1.0

Initial (first) regularization strength at iteration 0

last_reg : float = 0.0

Final (last) regularization strength as iteration → ∞

tau : float = 1000.0

Decay time constant (larger = slower decay)

Examples

>>> schedule = ExpRegularization(first_reg=5.0, last_reg=0.1, tau=500.0)
>>> schedule(0)      # iteration 0
5.0
>>> schedule(500)    # iteration 500 (≈63% decay)
0.1925
>>> schedule(5000)   # iteration 5000 (full decay)
0.1