Andrej Karpathy's LLM intro is still the best foundation video for understanding how ChatGPT actually works

What makes it different from most explainers: he covers training, not just inference. Most people understand that you type a prompt and get a response. Fewer people understand the training process that shapes how the model responds, what it has been optimised for and what patterns it was rewarded for reinforcing.

The limitations section is particularly useful for people who are frustrated by AI behaviour they find strange or inconsistent. When you understand why models hallucinate, why they can be confidently wrong, why they struggle with certain types of reasoning, the behaviour makes more sense even if it is still annoying.

This one is worth bookmarking as a resource to share when the forum gets into debates about whether AI "understands" or "reasons" or "knows" things. A lot of those debates get clearer once you understand what LLMs actually are at a technical level.

What is the most practically useful thing you learned from understanding how LLMs are trained rather than just how to use them?