What is Generative AI?
How generative AI works: from embeddings and the Transformer to foundation models. Scaling laws, LLMOps, and differences between LLMs, RAG, and agents.
Links containing ?t= open the video at a specific second.
Video summary
The ideas to retain
1. Embeddings: Translating text into numbers
ML models need numbers as input, and text is not naturally numerical.
2. The Transformer: The architecture that changes everything
Before the Transformer (2017), models processed text in order, token by token, with two practical consequences:
3. Scaling laws: when quantity becomes quality
Training uses self-supervised learning: without labels and with a simple signal (predicting the next token), the model learns from the raw volume of text, and that volume produces a result…
Next step
