---
title: Beyond the Transformer
seo_title: Beyond the Transformer — video
description: How the field is trying to move beyond pure Transformer scaling by combining tools, search, inference-time memory, world models and robotics.
keywords: beyond the Transformer, test-time compute, AI memory, world models, Mamba, SSM, AI robotics, AI agents, AI search, future of AI
date: '2026-03-31T00:00:00+00:00'
robots: index,follow,max-snippet:-1,max-image-preview:large,max-video-preview:-1
hide:
- toc
- navigation
video_watch_page: true
---


<div class="s5-video-watch" data-s5-video-watch data-video-id="videos-series-from-cave-to-agi-05-mas-alla-md">
  <header class="s5-video-watch__header">
    <div class="s5-video-watch__crumbs"><a href="https://5sigmas.com/en/videos/">All videos</a><span>History of AI</span><span>1:03</span></div>
    <h1>Beyond the Transformer</h1><p>How the field is trying to move beyond pure Transformer scaling by combining tools, search, inference-time memory, world models and robotics.</p>
  </header>
  <div class="s5-video-watch__player">
    <video controls crossorigin="anonymous" preload="metadata" poster="/en/series/from-cave-to-agi/05-mas-alla.jpg" playsinline data-s5-watch-player><source src="/en/series/from-cave-to-agi/05-mas-alla.mp4" type="video/mp4">Your browser does not support the video element.</video>
    <p>Links containing <code>?t=</code> open the video at a specific second.</p>
  </div>
  <section class="s5-video-watch__summary" aria-labelledby="video-summary-title">
    <div class="s5-video-watch__section-head"><span class="s5-eyebrow">Video summary</span><h2 id="video-summary-title">The ideas to retain</h2></div>
    <div class="s5-video-watch__snippet-grid"><article><span>01</span><h2>1. Why the Transformer is no longer a complete map</h2><p>The Transformer reorganized the field because it was parallelizable, scalable and extremely general. But scaling it also made several limits increasingly visible.</p></article>
<article><span>02</span><h2>1.1 Truth, uncertainty and hallucination</h2><p>Another important limitation appears here. Generative LLMs trained around next-token prediction are not directly optimized to distinguish truth, falsehood and unknown information. They are…</p></article>
<article><span>03</span><h2>2. From next-token prediction to search over solution spaces</h2><p>One of the most important directions in this new phase is a renewed emphasis on something that the LLM boom had pushed somewhat into the background: search.</p></article></div>
  </section>
  
  
  <aside class="s5-video-watch__source"><div><span class="s5-eyebrow">Context and evidence</span><h2>Continue with the full article</h2><p>The chapter develops the mechanism, primary sources, limitations and connections to the rest of the series.</p></div><a class="s5-video-watch__source-link" href="https://5sigmas.com/en/series/from-cave-to-agi/05-mas-alla/">Read the article →</a></aside>
  <section class="s5-video-watch__related" aria-labelledby="related-videos-title"><div class="s5-video-watch__section-head"><span class="s5-eyebrow">Next step</span><h2 id="related-videos-title">Related videos</h2></div><div class="s5-video-watch__related-grid"><article><a href="https://5sigmas.com/en/videos/series/from-cave-to-agi/03-aprender/"><img src="https://5sigmas.com/en/series/from-cave-to-agi/03-aprender.jpg" alt="" loading="lazy" width="1280" height="720"><span>History of AI · 1:03</span><strong>Learn — from rules to data</strong></a></article>
<article><a href="https://5sigmas.com/en/videos/series/from-cave-to-agi/02-mecanizar/"><img src="https://5sigmas.com/en/series/from-cave-to-agi/02-mecanizar.jpg" alt="" loading="lazy" width="1280" height="720"><span>History of AI · 1:03</span><strong>Mechanize — from calculation to computing</strong></a></article>
<article><a href="https://5sigmas.com/en/videos/series/from-cave-to-agi/01-representar/"><img src="https://5sigmas.com/en/series/from-cave-to-agi/01-representar.jpg" alt="" loading="lazy" width="1280" height="720"><span>History of AI · 1:03</span><strong>Represent — from counting to calculus</strong></a></article></div></section>
</div>