16.4 Memory devices and organization

Memory devices store and retrieve the data that digital systems depend on. In the context of sequential logic, these are the circuits that give your designs the ability to remember. This guide covers the major memory types, how they differ at the circuit level, and how they're organized into a hierarchy that balances speed, capacity, and cost.

Memory Types

Volatile and Non-volatile Memory

The most fundamental way to classify memory is by what happens when you cut the power.

Volatile memory (like RAM) needs a constant power supply to hold its data. The moment power is removed, everything stored is lost. The tradeoff is speed: volatile memory offers much faster read and write access, which is why it's used for data the processor needs right now.

Non-volatile memory (like ROM and flash memory) retains its data even after power is removed. It's slower to access, but that persistence makes it essential for anything that needs to survive a shutdown, such as firmware, boot instructions, and long-term file storage.

Random Access Memory (RAM) and Read-Only Memory (ROM)

RAM lets you read from and write to any memory location. "Random access" means you can jump to any address directly rather than reading through sequentially. RAM is volatile, so it serves as temporary working memory for the processor.

ROM is non-volatile memory designed primarily for reading. The data is written during manufacturing or through special programming methods and isn't meant to be modified during normal operation. ROM stores things like firmware and boot instructions that the system needs every time it powers on. It provides fast read access but doesn't support casual writes.

Static RAM (SRAM) and Dynamic RAM (DRAM)

These are the two main circuit-level implementations of RAM, and the differences come down to how each bit is physically stored.

• SRAM stores each bit using a flip-flop (typically 6 transistors per cell). Because flip-flops hold their state as long as power is supplied, SRAM doesn't need refreshing. It's fast but expensive and takes up more chip area per bit.
  • Typical use: cache memory, where speed matters most and capacity requirements are small.
• DRAM stores each bit as charge on a tiny capacitor (1 transistor + 1 capacitor per cell). Capacitors leak charge over time, so DRAM must be periodically refreshed (re-read and re-written) to avoid losing data. This makes it slower than SRAM, but the simpler cell design means much higher storage density at lower cost.
  • Typical use: main memory (system RAM), where you need gigabytes of capacity at a reasonable price.

Memory Organization

Memory Cells and Buses

A memory cell is the smallest unit of storage, representing a single bit. In SRAM, that's a flip-flop; in DRAM, it's a capacitor-transistor pair. Cells are arranged into arrays, and groups of cells form larger units like bytes (8 bits) and words (the natural data width of the processor).

Two key buses connect the processor to memory:

• Address bus: Carries the address of the memory location the processor wants to access. The number of address lines determines how many locations can be addressed. For example, 16 address lines can address $2^{16} = 65{,}536$ locations, while 32 address lines can address $2^{32} \approx 4.3 \text{ billion}$ locations.
• Data bus: Carries the actual data being read or written. Its width (8, 16, 32, or 64 bits) determines how much data transfers in a single access. A wider data bus means more data per transaction.

Memory Hierarchy

Not all memory can be fast, large, and cheap at the same time. The memory hierarchy solves this by layering different memory types:

This works because of the principle of locality: programs tend to access the same data repeatedly (temporal locality) and access data near recently used addresses (spatial locality). By keeping the most-used data in faster memory, the system performs almost as if all memory were fast.

Memory Hierarchy

Cache Memory

Cache is a small, fast SRAM buffer sitting between the processor and main memory. Its job is to store copies of frequently accessed data so the processor doesn't have to wait for slower DRAM.

Most systems have multiple cache levels:

• L1 cache: Smallest and fastest, built directly into the processor core
• L2 cache: Larger and slightly slower, often per-core
• L3 cache: Largest cache level, typically shared among all cores

Two outcomes are possible on every memory access:

• Cache hit: The requested data is found in the cache. The processor gets it quickly, and performance stays high.
• Cache miss: The data isn't in the cache. The processor must fetch it from main memory (or a lower cache level), which takes significantly longer. Minimizing cache misses is a major goal in system design.

Main Memory

Main memory (primary memory) is where the system stores currently running programs, their data, and intermediate results. It's almost always built from DRAM because of the favorable balance between capacity and cost.

Main memory is volatile, so every time you power on a computer, the operating system must load programs from secondary storage into main memory before they can execute. It acts as the middle layer, faster than a disk but slower than cache.

Secondary Storage

Secondary storage provides large-capacity, non-volatile storage for files, programs, and data not actively in use.

• Hard disk drives (HDDs) store data magnetically on spinning platters. Read/write heads move across the platter surface to find data. HDDs offer large capacities at low cost, but the mechanical components (spinning disks, moving heads) make access times relatively slow.
• Solid-state drives (SSDs) store data in NAND flash memory chips with no moving parts. This gives them faster access times, lower latency, and better durability than HDDs. The tradeoff is higher cost per bit, though SSD prices have been dropping steadily.