Major blocks
| Block | Role |
|---|---|
| Core | Executes instructions — fetch, decode, execute, retire |
| L1 instruction / data cache | Small, per-core, 1–2 cycle access |
| L2 cache | Larger per-core or shared pair, ~10 cycles |
| L3 / last-level cache (LLC) | Shared across cores on socket, tens of cycles |
| Memory controller | Interfaces to DDR DRAM (2–4 channels) |
| PCIe root complex | Connects to GPU, NVMe, network |
| Chipset / IO hub | USB, SATA, slower IO |
| Coherence fabric | Ring, mesh, or point-to-point between cores |
| Power / clock management | DVFS, C-states, P-states |
Pipeline stages (simplified)
| Fetch | Get instructions from I-cache; branch prediction |
|---|---|
| Decode | Convert to micro-ops (µops) |
| Rename | Map architectural → physical registers |
| Dispatch | Issue to reservation stations |
| Execute | Integer / FP / load-store units |
| Writeback | Store result in physical register |
| Retire | Commit to architectural state (in program order) |
Modern CPU features
- Superscalar: multiple instructions issue per cycle.
- Out-of-order: executes ready ops first; retires in order.
- SMT / hyperthreading: 2 threads share one core's front end.
- SIMD (AVX, NEON): operate on many data elements per instruction.
- Branch prediction: modern predictors reach >95% accuracy.
- Speculative execution: guess branches; rollback if wrong (Spectre/Meltdown mitigations).
- NUMA: multiple sockets with local memory — pin workloads to local CPU.
- Chiplets / tiles: AMD / Intel split cores and IO across dies.
Worked examples
Average memory access time. The cache levels above exist to hide DRAM latency. Using illustrative numbers: an L1 hit costs 2 cycles, an L2 hit about 10 cycles, and DRAM about 200 cycles. If 10 percent of accesses miss L1 and 20 percent of those also miss L2, the average is 2 + 0.10 × (10 + 0.20 × 200) = 2 + 0.10 × 50 = 7 cycles. Improving the L2 hit rate from 80 to 90 percent drops it to 2 + 0.10 × (10 + 0.10 × 200) = 5 cycles. Small changes in miss rate have an outsized effect because the DRAM penalty is so large.
Why more cores do not always help. Amdahl’s law gives speedup = 1 / (s + (1 − s) / n), where s is the serial fraction. With 20 percent serial work and 8 cores, speedup is 1 / (0.2 + 0.8 / 8) = 1 / 0.3 = 3.33, and even infinite cores cap it at 1 / 0.2 = 5.
Common misconceptions
- Clock speed equals performance. Instructions per cycle, cache size and memory latency matter as much as GHz.
- A hyperthread is a core. Two SMT threads share one core’s execution units, so the gain is typically a modest fraction, not double.
- More cores, more memory bandwidth. All cores share the memory controllers, so bandwidth-bound programs stop scaling early.
- Cache is invisible to the programmer. Access patterns decide whether data comes from L1 or DRAM; see the memory architecture page for stride examples.