Compare SGD with momentum, RMSprop, and Adam optimizers. How does Adam combine ideas from the other optimizers, and when would you choose one over another?

Sample answer preview

Optimizers determine how neural network weights are updated based on computed gradients. Different optimizers use different strategies to navigate the loss landscape efficiently, with tradeoffs in speed, stability, and final performance.