C++ Memory Locality & CPU Cache Line Alignment

When optimizing low-latency C++ applications (such as game engines, high-frequency trading platforms, or database storage engines), hardware microarchitecture dictates performance far more than arithmetic operation count.

Modern x86_64 CPUs do not fetch single bytes from RAM - they load contiguous 64-byte Cache Lines into L1/L2 caches.

1. Summary & Key Takeaways

• Cache Line Granularity: Memory is transferred between main RAM and CPU caches in fixed 64-byte blocks.
• Row-Major vs. Column-Major: Traversing memory sequentially maximizes L1 cache line hits ($O(1)$ amortized cost per fetch), whereas stride-based traversal triggers constant L1 cache misses.
• False Sharing: Multiple threads modifying distinct variables on the exact same 64-byte cache line cause cache invalidation ping-ponging across CPU cores. Use alignas(64) to mitigate.

2. Memory Stride: Row-Major vs. Column-Major

Consider iterating over a $4096 \times 4096$ integer matrix.

Traversing memory sequentially (row-major order) allows the CPU hardware prefetcher to fetch adjacent 64-byte cache lines before they are requested, achieving maximum L1 cache hit rates.

Conversely, strided iteration (column-major order) forces a full 64-byte cache line load for every single element access, thrashing the L1 cache.

3. False Sharing & alignas(64) Optimization

In multi-threaded C++11 code, two threads writing to adjacent variables in memory can degrade performance if both variables reside on the same cache line.

Using alignas(64) guarantees that modifications by Thread 0 will never invalidate the L1 cache line of Thread 1 on a neighboring core.

4. Code Comparison: Memory Locality & Cache Alignment

Compare production cache-alignment and memory-locality implementations across C++, Rust, Java, Go, and Python below:

// C++ Memory Locality & CPU Cache Line Alignment
#include <iostream>
#include <vector>
#include <cstdint>

// 1. Row-Major Access (Sequential / Cache-Friendly)
void traverseRowMajor(int** matrix, int N) {
    long long sum = 0;
    // Contiguous memory reads -> Maximum L1 Cache Hit Rate
    for (int i = 0; i < N; ++i) {
        for (int j = 0; j < N; ++j) {
            sum += matrix[i][j]; 
        }
    }
}

// 2. Column-Major Access (Strided / Cache Thrashing)
void traverseColumnMajor(int** matrix, int N) {
    long long sum = 0;
    // Non-contiguous memory reads -> Constant L1 Cache Misses
    for (int j = 0; j < N; ++j) {
        for (int i = 0; i < N; ++i) {
            sum += matrix[i][j];
        }
    }
}

// 3. False Sharing & alignas(64) Struct Alignment
struct alignas(64) WorkerThreadMetrics {
    uint64_t processed_items;
    uint64_t error_count;
    // Guaranteed to occupy its own isolated 64-byte cache line
};