1.2 Performance Metrics and Evaluation

Performance Metrics for Computer Systems

Performance metrics give you a quantitative way to evaluate and compare computer systems. Without them, claims like "this processor is faster" are meaningless. These metrics let you measure speed, efficiency, and processing capacity so you can assess architectural improvements and make informed design decisions.

Defining and Calculating Performance Metrics

Execution time is the total time a system takes to complete a specific task or program, measured in seconds or clock cycles. Three factors drive it: clock speed, instruction count, and average cycles per instruction (CPI). This relationship is captured by the CPU performance equation:

$\text{CPU Time} = \text{Instruction Count} \times \text{CPI} \times \text{Clock Cycle Time}$

Or equivalently:

$\text{CPU Time} = \frac{\text{Instruction Count} \times \text{CPI}}{\text{Clock Rate}}$

This equation matters because it separates the three independent contributors to execution time. Changing the ISA might reduce instruction count but increase CPI. A deeper pipeline might raise the clock rate but also raise CPI due to hazards. You need to consider all three terms together.

Throughput is the number of tasks or operations completed per unit of time. Common units include instructions per second (IPS), floating-point operations per second (FLOPS), and transactions per second (TPS). Throughput reflects a system's overall processing capacity and is especially relevant for server and batch-processing workloads.

Latency is the time delay between initiating a request and receiving the response, typically measured in seconds or clock cycles. It captures how responsive a system is to individual requests. A system can have high throughput but also high latency (think of a long, full pipeline), so these two metrics tell you different things.

Evaluating Performance Improvements

Speedup is the ratio of execution time on a reference system to execution time on an improved system:

$\text{Speedup} = \frac{T_{\text{old}}}{T_{\text{new}}}$

A speedup of 2.0 means the new system completes the task in half the time.

Amdahl's Law calculates the theoretical maximum speedup when only a portion of the system is improved:

$\text{Speedup} = \frac{1}{(1 - F) + \frac{F}{S}}$

where $F$ is the fraction of execution time that can be improved and $S$ is the speedup factor applied to that fraction.

For example, if 80% of a program's execution time can be parallelized ($F = 0.8$) and you speed that portion up by 4× ($S = 4$):

$\text{Speedup} = \frac{1}{(1 - 0.8) + \frac{0.8}{4}} = \frac{1}{0.2 + 0.2} = \frac{1}{0.4} = 2.5$

Even with a 4× improvement on 80% of the work, overall speedup is only 2.5×. The key takeaway: the unimproved fraction dominates. No matter how much you accelerate the improved portion, the serial remainder sets a hard ceiling. As $S \to \infty$, the maximum speedup approaches $\frac{1}{1 - F}$, which in this case is 5×.

Factors Influencing Architecture Performance

Instruction Set Architecture and Pipeline Design

The instruction set architecture (ISA) determines the complexity, granularity, and efficiency of available instructions, which directly affects all three terms in the CPU performance equation.

• RISC architectures use simpler instructions that typically execute in one cycle (low CPI), but may require more instructions for a given task (higher instruction count).
• CISC architectures offer more complex instructions that can do more per instruction (lower instruction count), but individual instructions often take multiple cycles (higher CPI).

Pipeline depth affects performance by allowing multiple instructions to overlap in different stages of execution. A deeper pipeline enables higher clock frequencies because each stage does less work. However, deeper pipelines come with tradeoffs: branch mispredictions become more expensive (more stages to flush), and data hazards require more forwarding or stalling. There's a practical sweet spot where the throughput gains from pipelining balance against these penalties.

Memory Hierarchy and Parallelism

Cache hierarchy design reduces the effective memory access latency by exploiting data locality. Key design parameters include cache size, associativity, block size, and replacement policy. Effective cache design minimizes miss rates and keeps frequently accessed data close to the processor. A single cache miss to main memory can cost hundreds of cycles, so even small improvements in hit rate can significantly affect overall performance.

Memory system architecture determines how fast data moves between the processor and memory. Memory bandwidth, latency, and interconnect topology all matter. High-bandwidth, low-latency memory systems are critical for data-intensive workloads where the processor would otherwise stall waiting for data.

Instruction-level parallelism (ILP) techniques allow multiple independent instructions to execute concurrently within a single core:

• Out-of-order execution dynamically reorders instructions to fill pipeline slots that would otherwise be idle
• Superscalar processing issues multiple instructions per clock cycle to parallel functional units
• Speculative execution predicts the outcome of branches and begins executing instructions along the predicted path

The effectiveness of ILP depends on the inherent parallelism in the code and the hardware's ability to detect and resolve dependencies.

Thread-level parallelism (TLP) and multi-core architectures improve performance by executing multiple threads simultaneously on separate cores. Realizing TLP gains requires appropriate workload partitioning, efficient synchronization, and low-overhead communication between cores. Amdahl's Law applies directly here: the serial portion of a workload limits how much benefit additional cores provide.

Clock Frequency and Power Constraints

Processor clock frequency determines the number of cycles executed per second, so higher frequencies generally mean faster execution. However, power consumption scales roughly with the cube of frequency (since $P \propto C \times V^2 \times f$ and voltage must increase with frequency). This creates a practical wall: beyond a certain point, the heat generated makes further frequency increases infeasible. This power wall is a major reason the industry shifted toward multi-core designs rather than continuing to push single-core clock speeds.

Benchmarking for Architecture Comparison

Types of Benchmarks

Benchmarking measures system performance using standardized workloads, enabling objective comparisons between architectures or configurations. The choice of benchmark matters enormously because different workloads stress different parts of the system.

• Synthetic benchmarks are artificial workloads designed to stress specific subsystems. LINPACK measures floating-point performance (and is used to rank the TOP500 supercomputers). STREAM measures sustainable memory bandwidth. These are useful for isolating specific capabilities but don't necessarily reflect real application performance.
• Application-specific benchmarks use real-world programs representative of actual usage scenarios. They give more realistic performance estimates for targeted domains:
  • SPEC CPU covers general-purpose integer and floating-point computing
  • TPC-C measures online transaction processing (database workloads)
  • MLPerf evaluates machine learning training and inference performance

Application benchmarks are generally preferred for architectural comparisons because they capture the complex interactions between ISA, memory hierarchy, and parallelism that synthetic benchmarks miss.

Performance Analysis Tools and Techniques

Microarchitectural simulators like gem5 and SimpleScalar model processor behavior at the instruction level. They let researchers study how specific design choices (e.g., changing cache associativity or adding a functional unit) affect performance before building hardware. The tradeoff is simulation speed: detailed cycle-accurate simulation can be orders of magnitude slower than native execution.

Performance profiling tools like perf (Linux) and Intel VTune identify bottlenecks in running code. They provide detailed data on CPU utilization, cache miss rates, memory access patterns, and function-level execution times. These tools are essential for understanding where time is actually spent.

Reproducibility is critical in benchmarking. Results are only meaningful if they can be reliably compared across systems. You need to carefully control and document system setup, compiler version and optimization flags, OS configuration, and runtime environment. A benchmark result without this context is difficult to interpret.

Performance Analysis and Design Choices

Interpreting Performance Results

Performance analysis goes beyond collecting numbers. It requires understanding how hardware components, software optimizations, and workload characteristics interact to produce the observed results.

Bottleneck identification pinpoints the components or resources limiting overall performance. Common bottlenecks include:

• Memory bandwidth saturation
• High cache miss rates (especially last-level cache misses)
• Instruction dependencies that limit ILP
• I/O latency in storage-bound workloads
• Branch mispredictions in control-flow-heavy code

Finding the actual bottleneck is the first step toward meaningful optimization. Improving a component that isn't the bottleneck yields little or no speedup.

Scalability assessment evaluates how performance changes as you increase workload size, core count, or problem complexity. Strong scaling measures speedup for a fixed problem size as you add cores. Weak scaling measures whether performance holds as both problem size and core count grow proportionally. These assessments reveal the practical limits of parallel architectures for a given workload.

Guiding Architectural Design Decisions

Sensitivity analysis varies architectural parameters (cache size, pipeline depth, branch predictor accuracy, etc.) and measures the impact on performance. This reveals which parameters matter most for a given workload and helps identify optimal design points. For instance, if doubling the L2 cache from 256 KB to 512 KB yields a 15% speedup but doubling it again to 1 MB yields only 2%, you've found the point of diminishing returns.

Comparative analysis evaluates different architectures, algorithms, or optimization techniques against each other. A thorough comparison considers not just raw performance but also power efficiency, cost, area, and compatibility with existing software ecosystems.

Workload characterization examines the properties of specific workloads: instruction mix, data access patterns, working set size, control flow behavior, and communication patterns. This information guides architecture optimization for targeted application domains. A processor designed for database workloads will make different tradeoffs than one designed for scientific computing.

Performance modeling uses analytical models, simulation, or machine learning-based approaches to estimate performance for architectures or workloads that don't yet exist. These projections help architects evaluate design alternatives early in the process, before committing to expensive hardware implementations.