Live neural network · 2 inputs · 10 hidden layers × 5 ReLU neurons · 1 output · hover a neuron, drag it and let go

1 · Forward propagation

z = (x₁·w₁) + (x₂·w₂) + b   a = ReLU(z) = max(0, z)

2 · Loss calculation

L = ½(ŷ − y)²   (mean squared error)

3 · Backpropagation

δ = ∂L/∂z (chain rule)   ∂L/∂w = δ · aprev

4 · Optimization (SGD)

w ← w − η · ∂L/∂w   b ← b − η · δ

Training loss

+w −w gradient