next up previous
Next: Up: Previous:

Gradient Descent


\psfig{figure=figures/parabola-floor.ps} \psfig{figure=figures/parabola-slope-fat.ps}

Gradient


\begin{displaymath}\nabla E[\vec{w}] \equiv \left[\frac{\partial E}{\partial w_{... ...artial w_{1}}, \cdots \frac{\partial E}{\partial w_{n}}\right] \end{displaymath}

Training rule:

\begin{displaymath}\Delta \vec{w} = -\eta \nabla E[\vec{w}] \end{displaymath}

i.e.,

\begin{displaymath}\Delta w_{i} = -\eta \frac{\partial E}{\partial w_{i}}\end{displaymath}