How Modern AI Models Work
Look inside modern AI: the ideas that made deep learning work, how a model can decide without generating text, and how learning from rewards is built.
Start learning Open course bookAbout this course
It is easy to use an AI model without knowing what happens inside it. This course is for learners who want to open the box. It concentrates on mechanisms: how networks came to be trainable at great depth, what a model computes when it reads a prompt, and how a program can improve at a task through trial, error and reward.
The course has three steps. You begin with the history of deep learning told through its key research ideas, from backpropagation to dropout, distillation and beyond, each understood as the answer to a specific problem. You then study how language models process input, and how some decisions can be read directly from that processing without generating a reply. Finally you see reinforcement learning assembled from nothing, including the automatic differentiation that all neural network training depends on. Each lesson pairs a hands-on video course with a book chapter that works the central calculations by hand.
What you will learn
- Explain the obstacles that held deep networks back and the idea that removed each one
- Carry out dropout, layer normalization and softmax with temperature on small examples
- Distinguish the prefill and decoding phases of language model inference
- Explain causal attention masks and why cached work can be reused across requests
- Describe the reinforcement learning loop of state, action and reward, and calculate discounted returns
- Trace a forward and backward pass through a small computational graph
Who this course is for
Learners who have met the basics of neural networks, for example through the Neural Networks course on this site, and who are comfortable reading Python. The reinforcement learning lesson uses the C language, but its chapter can be followed without knowing C. School-level algebra is enough for the book; the videos go further.