Neural Networks: The Foundation
Layered functions trained by optimization. Used in many AI systems; not the definition of AI, and not a brain.
What is a Neural Network? (Simple Version)
A neural network is a stack of simple math steps. Each step takes numbers in, combines them, and sends numbers out.
Neural networks are one method family used in many AI systems. They are not the definition of AI. See What is AI?.
The computation
Inputs → weighted sum → nonlinearity → output
numbers in
→ multiply each input by a weight, then add a bias
→ apply a nonlinearity (for example ReLU: keep positives, zero the rest)
→ pass the result to the next layer
→ final numbers out (a class score, a next-token score, a pixel, a move, …)
Examples of that input–output map after training:
image pixels → [Neural Network] → class scores
email features → [Neural Network] → spam vs not-spam scores
board position → [Neural Network] → move scores
prompt tokens → [Neural Network] → next-token scores
Those are uses. They are not a claim that every AI system is a neural network.
One unit
Each unit (often called an artificial neuron) does the same small job:
- Takes in numbers
- Multiplies them by weights (how much each input counts)
- Adds them up, plus a bias
- Applies a nonlinearity so stacked layers can represent more than a straight line
Stack those units into layers. The whole network is still a function: parameters in, outputs out.
What “learning” means here
For a typical supervised net, training is adjusting parameters to reduce a loss on data:
- Forward pass: compute an output from the current weights
- Measure a loss: how far that output is from the target
- Update weights: change parameters so the loss tends to fall
- Repeat on many examples
That loop is how this method is trained. It is not a complete definition of AI, and it is not the only method used inside AI systems.
Neural Network Playground
Click to add points. Watch the network learn to classify them!
Keyboard: 0 / 1 to switch labels,T to train,C to clear
- Click to add points (2 classes)
- Network tries to separate them
- Background shows decision boundary
- Cyan = Class 1, Pink = Class 0
What's Happening?
The neural network is learning to draw a boundary between your two classes using gradient descent. The background color shows its confidence - brighter means more confident. Watch how the boundary evolves as it trains!
Try it. Draw simple patterns and watch a small network adjust its weights. The playground is a toy classifier, not a picture of intelligence.
References
Rosenblatt, F. (1958). The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain. Psychological Review, 65(6), 386-408.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533-536.
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989). Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1(4), 541-551.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems (NeurIPS).
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
Kingma, D. P., & Ba, J. (2014). Adam: A Method for Stochastic Optimization. ICLR 2015.
Citation Note: Some citations are open access (arXiv or DOI, where linked). Others are books or journal articles that may sit behind a publisher paywall. Use the linked DOI or arXiv when available. If you notice any citation errors, please let us know.