ExpStepSize

phasic.svgd.ExpStepSize(first_step=0.01, last_step=1e-06, tau=1000.0)

Exponential decay schedule: step_size = first_step * exp(-iteration/tau) + last_step * (1 - exp(-iteration/tau)).

This schedule helps prevent divergence with large datasets by gradually reducing the step size as optimization progresses.

Parameters

first_step : float = 0.01

Initial (first) step size at iteration 0

last_step : float = 1e-6

Final (last) step size as iteration → ∞

tau : float = 1000.0

Decay time constant (larger = slower decay)

Examples

>>> schedule = ExpStepSize(first_step=0.1, last_step=0.01, tau=500.0)
>>> schedule(0)      # iteration 0
0.1
>>> schedule(500)    # iteration 500 (≈63% decay)
0.037
>>> schedule(5000)   # iteration 5000 (full decay)
0.01