What is Generative AI?
How generative AI works: from embeddings and the Transformer to foundation models. Scaling laws, LLMOps, and differences between LLMs, RAG, and agents.
Links containing ?t= open the video at a specific second.
Video summary
The ideas to retain
1. Embeddings: Turning text into numbers
ML models need numbers as input, and text is not naturally numerical.
2. The Transformer: The architecture that changes everything
Before the Transformer (2017), models processed text in order, token by token, with two practical consequences:
3. Scaling laws: How performance changes with scale
Training uses self-supervised learning: with no manual labels and a simple signal such as predicting the next token, the model learns from raw text at scale. The capabilities that emerged…
Next step
