Back INSIDE THE CHIP // notes from Reiner Pope on how AI silicon actually works AI
Download Open
Show description 1,989 chars · AI

INSIDE THE CHIP // notes from Reiner Pope on how AI silicon actually works

INSIDE THE CHIP // notes from Reiner Pope on how AI silicon actually works

SYS // CHIP-NOTES v1.0
SRC: Dwarkesh × Reiner Pope (MatX)

STATUS: NOMINAL
NODE: 3nm

// SECTIONS

00The Big Idea
01Logic Gates → MAC
02Mux + Data Movement
03Systolic Arrays
04Clock Cycles
05FPGA vs ASIC
06Cache vs Scratchpad
07CPU vs GPU Cores
08Brains vs Chips
09GPU = tiny TPUs
10Key Takeaways

// TRANSCRIPT BREAKDOWN

insidethe chip
how AI silicon actually works

A bottom-up walk through how an AI chip is built — starting from AND gates and
ending at the GPU-vs-TPU architectural split. Notes from a conversation
between Dwarkesh Patel and Reiner Pope, CEO of MatX. Every
section here distills one big idea from the transcript and the diagrams that make it stick.

primitive
multiply-accumulate

building block
full adder (3→2)

unit cell
systolic array

enemy #1
data movement cost

scaling law
compute ∝ p × q

00

The one idea that runs the whole thing

META

Every level of chip design is the same fight: maximize compute relative to
communication. From the precision of a single multiplier, to the size of a systolic
array, to the layout of a whole datacenter — you are always trying to do more arithmetic per
byte you move. That's it. That's the whole show.

level 1
◇at the gate

Bit-width scales quadratically — halving precision more than doubles throughput. This is why FP4 is so much faster than FP8.

level 2
◇at the core

A systolic array bakes a 2D loop of MACs into hardware so the weight matrix sits in place while activations flow through.

level 3
◇at the chip

A scratchpad replaces a cache so memory access is deterministic and software, not hardware, controls movement.

01

Logic gates → multiply-accumulate

primitives

why MAC?
The atomic op of AI

Look inside any matrix multiply and you find a triple for loop:

// matrix multiply, three nested loops
for i: for j: for k:
out[i,k] += A[i,j] * B[j,k]

Every step is one multiply-accumulate — a multiply, an add into a running sum.…