From the Caves to AGI¶
This series follows the history of abstraction from humanity's earliest symbolic representations to massive language models that seem to understand the world.
We will trace how the need to count led to calculation, how attempts to mechanize reasoning led to computing, and how efforts to imitate the brain brought us into the era of generative AI.
That history matters because today's generative-AI systems did not appear in isolation. They are the direct result of five centuries of mathematical notation, eight decades of computing and more than half a century of machine learning. Following that chain from the beginning makes it easier to understand why AI works the way it does, where its real limits lie and what may come next. Without that context, recent advances can look like magic. With it, their development has a traceable logic.
Contents¶
1. Represent (≈ 40,000 BCE – 1700)¶
- Invent languages for describing the world: from physical counting—marks and notches—to numbers and manipulable symbols.
- Formalize truth: the Greeks establish proof and geometry as standards for verifiable knowledge.
- Do science with mathematics: zero and algebra make notation more expressive; mathematical models enable prediction; calculus introduces derivatives and equations for describing changing quantities.
2. Mechanize (≈ 1700 – 1956)¶
- Turn symbols into machinery: automate calculation mechanically and later electronically. This is where computation becomes an explicit engineering goal.
- Separate program from hardware: from Jacquard and Babbage to reusable instructions independent of a specific machine.
- Foundations of modern computing: Boolean logic, formal proof, computability and its limits through Turing, practical memory-based architecture, and Shannon's theory of information and compression.
3. Learn (≈ 1956 – 2012)¶
- From rules to data: move from hand-coding intelligence to fitting models from data.
- Engines of practical learning: probability and generalization, including Bayes; optimization through gradients and regularization; and parameter adjustment through backpropagation.
- Before the LLM era: the perceptron and its limits, the rise and fall of symbolic AI, statistical NLP with n-grams and HMM/CRF models, and the neural revival driven by data, GPUs, benchmarks and scale.
4. Scale (≈ 2012 – 2024)¶
- Scale as a multiplier: deep learning accelerates through more data and compute plus better representations; 2012 marks a regime change.
- Transformers and foundation models: attention enables massive pretraining and broad reuse. Scaling laws and human alignment make this a major inflection point that eventually leads to ChatGPT.
- From answering to acting, while unifying modalities: tools and agents extend models into action; multimodality brings text, image and audio into shared systems.
5. Beyond the Transformer (≈ 2022 – Q1 2026)¶
- The limits of pure scaling: why the Transformer alone is not enough to describe where the frontier is moving.
- Memory and active inference: long context, external memory and test-time compute as new levers beyond parameter count.
- Agents, tools and search: systems that act, plan and retrieve information from the real world.
- World models and foundation robotics: internal representations that simulate and predict, applied to physical control.
Recommended next series: Generative Multimodality — what it means to unify text, image, audio and video inside one system.