Gradient descent
An optimisation method that gradually changes parameters in the direction in which the error decreases most.
Think of it as walking downhill in fog: you feel the slope of the ground and take a step downwards. The step size is called the learning rate – too large and you overshoot; too small and training takes forever.
In practice, variants such as stochastic gradient descent or Adam are used, which work on small batches of data and reach the goal faster.
General information, not legal or professional advice.