A hard decision passes through an office: front-desk staff catch simple details, pass them up to specialists who spot bigger patterns, up to a manager who makes the call. A neural network stacks layers of simple 'neurons' the same way to learn complex features.
From neurons to networks
An artificial neuron computes a weighted sum of its inputs, adds a bias, and passes the result through a nonlinear activation function. Stacking neurons into layers forms a neural network; early layers learn simple features and later layers combine them into complex ones. Training adjusts the weights via gradient descent and backpropagation.
Without a nonlinear activation, stacking layers just produces another linear function. Nonlinear activations (like ReLU) let networks approximate complex, curved relationships.
In image recognition, first layers detect edges, middle layers detect shapes, and deeper layers detect whole objects — features learned, not hand-coded.
- A neuron receives inputs x₁, x₂ with weights w₁, w₂ and bias b.
- Write the value it computes before activation.
- State what the activation function then does.
What you should see: It computes w₁x₁ + w₂x₂ + b, then the activation function applies a nonlinearity to that sum to produce the neuron's output.