Hands-On Neural Networks with PyTorch

A free, hands-on course of 19 Jupyter notebooks that build neural networks from a single neuron up to Transformers, generative models, and retrieval-augmented generation — in PyTorch, with every mechanism implemented by hand before reaching for the library version.

Every notebook in this course has been executed end to end: what you see on this site is real output from real runs, not illustrative code that has never been checked. Running them was also how several claims in earlier drafts turned out to be wrong — Lab 11, for instance, measures five random seeds rather than trusting one, because two single runs of the same comparison gave opposite answers.

Get started :material-arrow-right: View on GitHub :fontawesome-brands-github:

What's covered

The 19 labs build in five stages.

1 · Foundations

The linear algebra and calculus neural networks are made of, a working Python/conda environment, and the PyTorch basics — tensors, autograd, and the training loop.

• 01 · Introduction to Neural Networks — overview of neural networks, their applications, and workflow.
• 02 · Linear Algebra for Neural Networks
• 03 · Calculus for Neural Networks
• 04 · Miniconda Installation & Environment Setup
• 05 · Python Refresher & PyTorch

2 · Core architectures

A single neuron and its activation function, forward and back propagation done by hand, then dense networks, CNNs for images, RNNs and LSTMs for sequences, and transfer learning to reuse what others have trained.

• 06 · Neurons and Activation Functions
• 07 · Forward and Back Propagation
• 08 · Dense Neural Networks
• 09 · Convolutional Neural Networks (CNN)
• 10 · Recurrent Neural Networks (RNN)
• 11 · Long Short-Term Memory (LSTM)
• 12 · Transfer Learning

3 · Advanced topics

The techniques that decide whether a network generalises, the attention mechanism that replaced recurrence, and two families of generative model.

• 13 · Regularization and Optimization
• 14 · Attention and Transformers
• 15 · Autoencoders and Variational Autoencoders
• 16 · Generative Adversarial Networks (GANs)

4 · Production

Everything that happens after the accuracy prints.

• 17 · Deployment and Model Optimization — saving and loading without the traps, TorchScript and ONNX export, quantization, honest benchmarking.

5 · Retrieval

Reusing the Transformer as a search engine, and grounding a language model in your own documents.

• 18 · Embeddings and Semantic Search
• 19 · Retrieval-Augmented Generation (RAG)

See About & References for the ideas that recur across all 19 labs, and further reading.

Why this course

• Build it by hand, then use the library. Every mechanism — a neuron, backpropagation, an LSTM gate, scaled dot-product attention, the reparameterization trick — is implemented from scratch, checked against PyTorch's own version, and only then replaced with it.
• Every claim is measured, not asserted. The labs were run, not just written; where a result was seed-dependent or a "textbook" claim didn't hold up, the notebook says so and shows the numbers.
• Diagrams and animations, not walls of text. Hand-authored SVG diagrams and generated GIFs sit alongside the code, in Misc/.
• Free and open source, under the MIT license.