← Applet Gallery Optimizer Race Course notes: Chapter 27 →
Chapter 27 · Optimization

Optimizer Race

Select optimizers, configure learning rates, and watch them race to minimize a 2D loss function. Trajectories are drawn in real time on a contour plot.

Controls

Click or drag on contour plot to set start point

Loss Function

\(L = w_1^2 + 50\,w_2^2\)

Optimizers

SGD
0.010
Momentum
0.010
0.900
RMSProp
0.010
0.990
1e-8
Adam
0.010
0.900
0.999
1e-8

Metrics

Quadratic Bowl
Start: (-3.5, 3.5)
Loss vs. Iteration