Semi writing goes bad when the vocabulary is familiar and the picture is not. People say cache, HBM, PCB, GPU fluently — then stall when asked what moves, what waits, and what the factory is actually stacking. This note is a short reset: three concepts, three Branch Education videos, no jargon for its own sake.
Pair this with the denser rack / board pieces — The PCB Stack Inside an AI Rack, Huawei / HBM, What AI Hardware Is Running Out Of— after the mental model is clean. Next shelf up (TPU + CPO, high-signal YouTube only): TPU & CPO watch list.
1 · Cache, memory, storage — three speeds, one lie
The lie: “memory” is one thing. The machine is a stack of different distances to data. Registers and L1/L2/L3 caches sit next to the cores — tiny, expensive, fast. DRAM (and on AI cards, HBM) is the working set you can afford to keep warm. Disk / SSD is where data lives when you are not using it. Latency and bandwidth are not the same constraint: a miss can stall a core even when peak bandwidth looks huge on a slide.
What to re-learn in five minutes: every “fast AI” story is partly a story about keeping useful bits close to the math. When people say the memory wall, they mean the cores are waiting on the stack, not that FLOPs disappeared.
Memory hierarchy · Branch Education
How does Computer Cache, Memory, and Storage Work?https://www.youtube.com/watch?v=TfhL5kBiQVI
2 · PCB — not “the green board,” a multi-layer factory object
A PCB is the city grid under the chips: copper traces, power and ground planes, vias that climb between layers, pads where packages land (including BGA under an SoC). Smartphones and AI racks both live or die on how those layers are stacked — signal integrity, power delivery, heat, and yield.
Factory angle (same Branch walk-through): boards are laminated, imaged, etched, drilled, plated, and tested — a manufacturing loop, not a single print. When AI racks talk about 22-layer HDI or mid-planes copper-sintered together, that is this loop pushed to extremes. If you only remember “PCB = green rectangle,” you will misread the supply chain articles.
Board stack + manufacturing walk · Branch Education
What are PCBs? How do PCBs Work?https://www.youtube.com/watch?v=Z2LgmIGE2nI
Treat this as the board-side factory tour: how layers become a product.
3 · GPU — parallel machine, not “CPU with more cores”
A GPU wins when thousands of simple lanes run the same kind of work on different data (SIMD / SIMT). It loses when the work is full of branches, tiny dependent steps, or constant trips to far memory. “More TFLOPS” only matters if the problem maps onto that parallel shape and the memory system can feed it.
What to re-learn: graphics cards and AI accelerators share the same basic bet — throughput over single-thread latency — and they inherit the same cache / HBM / PCB constraints from sections 1 and 2. Architecture diagrams are not decoration; they are where the bottlenecks hide.
GPU as parallel machine · Branch Education
How do Graphics Cards Work? Exploring GPU Architecturehttps://www.youtube.com/watch?v=h9Z4oGN89MU
Watch order · ~5 minutes of framing + the videos
- Skim this page once (concepts only).
- Play the three Branch Education videos above — cache/memory, PCB (+ manufacturing), GPU.
- Come back to the AI-rack / HBM essays when you need numbers, not definitions.
If you can explain, without slides: (1) why a cache miss hurts, (2) why a via and a power plane matter on a board, (3) why a GPU hates serial work — you are past “I already know this” and into usable semi literacy.