Cache-Friendly C++: Data-Oriented Design and AoS vs SoA

By 0xmote · Jun 16, 2026 · 22 min read · Updated Sep 27, 2026 Advanced tutorial

Why Memory Layout Decides Performance

Modern CPUs can execute billions of instructions per second, but main memory delivers data at a much slower rate. When the CPU needs data that isn’t in cache, it stalls and waits — a cache miss typically costs 100-300 cycles.

A well-tuned algorithm with poor memory layout often runs slower than a naive algorithm with cache-friendly layout. This is the core insight of data-oriented design (DoD): design your data structures for how the CPU accesses them, not just for logical convenience.

L1 cache hit:  ~4 cycles   (data already in L1)
L2 cache hit:  ~12 cycles
L3 cache hit:  ~40 cycles
RAM:           ~200 cycles  (50x slower than L1)

These figures are typical orders of magnitude for current desktop and server CPUs; exact values vary by microarchitecture and clock speed. What matters is the shape: each level is several times slower than the one above it, and a trip to DRAM costs as much time as hundreds of simple arithmetic instructions. Out-of-order execution can hide some of that latency by working on other instructions while a load is pending, and hardware prefetchers hide more by fetching ahead when they detect a predictable pattern. Data-oriented design is largely about making access patterns predictable enough for those mechanisms to work, and compact enough that fewer lines need to be fetched in the first place.

Cache Lines — The Unit of Memory Transfer

The CPU doesn’t fetch individual bytes from memory. It fetches cache lines — typically 64 contiguous bytes. When you access one byte, the surrounding 63 bytes come along for free.

This matters for data layout:

If the next data you need is in the same cache line, it’s free
If it’s in a different cache line, you pay the full cache miss penalty

// Each line: 64 bytes = 16 ints
int arr[1000];

// Sequential access — cache-friendly
// After loading arr[0], arr[1]..arr[15] are already in cache
for (int i = 0; i < 1000; i++) {
    process(arr[i]);  // hits L1 ~15 out of 16 accesses
}

// Strided access — cache-unfriendly
// Jumps 64 bytes each iteration — every access may miss
for (int i = 0; i < 1000; i += 16) {
    process(arr[i]);  // potential cache miss every time
}

Two caveats keep this example honest. A 1,000-element int array is 4 KB and fits comfortably in a typical 32–48 KB L1 data cache, so after the first pass both loops hit in cache; the difference only appears when the data is much larger than the cache. And a constant stride is exactly what hardware prefetchers are built to recognize, so the strided loop over a large array is still far faster than truly random access. The strided loop’s real cost is efficiency: it uses 4 of every 64 bytes it pulls from memory, so it needs sixteen times the memory bandwidth per useful element. That ratio — bytes used versus bytes fetched — is the quantity the rest of this article tries to improve.

Array of Structures vs Structure of Arrays

The most impactful layout decision for performance-critical code.

Array of Structures (AoS)

The natural object-oriented layout:

struct Particle {
    float x, y, z;      // position (12 bytes)
    float vx, vy, vz;   // velocity (12 bytes)
    float mass;          // (4 bytes)
    int id;             // (4 bytes)
    uint32_t color;     // (4 bytes)
    // total: 40 bytes per particle
};

std::vector<Particle> particles(100'000);

// Position-only update loop
for (auto& p : particles) {
    p.x += p.vx * dt;
    p.y += p.vy * dt;
    p.z += p.vz * dt;
    // mass, id, color are LOADED but NEVER USED — wasted bandwidth
}

With 40-byte structs, each cache line (64 bytes) holds ~1.6 particles. The position update loop accesses x, vx, y, vy, z, vz — which are all in the same struct. But it also loads mass, id, color from every cache line, wasting 12 of every 40 bytes — 30% of memory bandwidth.

In real code the waste is usually much larger than in this tidy example, because “particle” or “entity” structs accumulate fields over time: a name string, a pointer to a mesh, flags, debug data. A position update over a 200-byte struct uses 24 bytes of it. At 100,000 particles the AoS array is 4 MB, larger than many L2 caches, so each frame streams it from L3 or memory; the position and velocity arrays of the SoA version total 2.4 MB, and a loop touching only positions reads half of that.

Structure of Arrays (SoA)

struct Particles {
    std::vector<float> x, y, z;    // all x positions together
    std::vector<float> vx, vy, vz; // all velocities together
    std::vector<float> mass;
    std::vector<int> id;
    std::vector<uint32_t> color;
    std::size_t count;
};

Particles particles;
particles.count = 100'000;
particles.x.resize(100'000);
// ... initialize all arrays

// Position-only update loop — reads only what it needs
for (std::size_t i = 0; i < particles.count; ++i) {
    particles.x[i] += particles.vx[i] * dt;
    particles.y[i] += particles.vy[i] * dt;
    particles.z[i] += particles.vz[i] * dt;
}
// `mass`, `id`, `color` arrays are never touched — zero wasted bandwidth
// x[0]..x[15] fit in one cache line, prefetching ahead works perfectly

For SIMD (AVX2 processes 8 floats at once), SoA aligns perfectly — you can load 8 consecutive x values and 8 consecutive vx values into SIMD registers and update them all at once.

You rarely have to write intrinsics for this: with -O3 (or -O2 on recent GCC) plus a suitable -march, compilers auto-vectorize simple SoA loops like this one, and -fopt-info-vec (GCC) or -Rpass=loop-vectorize (Clang) reports whether they did. The same loop over AoS data usually does not vectorize well, because the x values are 40 bytes apart and would need gather instructions. One thing that can block vectorization in the SoA version is aliasing: the compiler cannot always prove that particles.x and particles.vx do not overlap, and may add runtime checks or give up. Taking the arrays as local float* pointers (optionally with the non-standard __restrict) often helps.

When AoS Is Better

SoA is not universally better:

Small counts (< ~1000 objects): cache behavior difference is negligible
Accessing many fields at once: if your loop touches x, y, z, mass every time, AoS keeps them together
Random access by object: if you frequently look up a single particle by index and use all its data, AoS has better locality for that use case
Code clarity: SoA code is more verbose and error-prone

Rule: profile first. If a loop touches fewer than half the struct’s fields on most iterations, SoA is likely faster.

A middle path that is often enough is hot/cold splitting: keep an AoS layout, but move rarely used fields (names, debug info, creation timestamps) into a separate struct referenced by index or pointer. The hot struct shrinks to what the frequent loops need, most of the cache benefit arrives, and the code keeps its object-like shape. It is also a smaller change to make in an existing codebase than a full conversion to parallel arrays.

Cache Line Alignment

Basic Alignment

#include <cstddef>
#include <new>  // for std::hardware_destructive_interference_size

// Align a struct to the cache line boundary
struct alignas(64) HotData {
    int counter;
    float accumulated;
    // ... rest of the fields
};

// C++17 portable cache line size
constexpr std::size_t CACHE_LINE = std::hardware_destructive_interference_size;

struct alignas(CACHE_LINE) ThreadLocal {
    long count;
    double sum;
};

Alignment matters most for data that multiple threads write to, or that is accessed in tight loops where cache line boundaries matter.

Over-aligning has costs, which is why it should be targeted rather than applied everywhere. alignas(64) rounds sizeof(HotData) up to 64 even though its fields need 8 bytes, so an array of them uses eight times the memory — the opposite of cache-friendly for anything iterated in bulk. Use it for a small number of per-thread or contended objects, not for elements of large arrays. std::hardware_destructive_interference_size also has practical wrinkles: GCC warns (-Winterference-size) when it is used in a header because its value can differ between compiler flags and would change struct layouts across translation units, and some standard library versions did not provide it at all until recently. Many codebases therefore define their own constexpr std::size_t kCacheLine = 64; (128 on some Apple and ARM designs, where adjacent-line prefetching makes 128 the safer spacing).

Checking Your Layout

#include <iostream>

struct AoS_Particle {
    float x, y, z;   // 12 bytes
    float vx, vy, vz; // 12 bytes
    int id;           // 4 bytes
};
// Total: 28 bytes — 2.28 particles per 64-byte cache line

int main() {
    std::cout << "Particle size: " << sizeof(AoS_Particle) << '\n';
    std::cout << "Particles per cache line: "
              << 64.0 / sizeof(AoS_Particle) << '\n';
    // Particles per cache line: 2.28
}

sizeof includes padding the compiler inserts to satisfy alignment, and field order changes it: a struct { char a; double b; char c; } is 24 bytes, while { double b; char a; char c; } is 16. Ordering members from largest to smallest alignment is a free size reduction. A non-integer “particles per line” also means some particles straddle two cache lines, so touching one of them can cost two misses; that is the case where padding the struct to 32 bytes, or restructuring, can actually help. Tools such as pahole (from the dwarves package) print the exact layout of a struct including padding holes, which beats reasoning about it by hand.

False sharing is the silent multi-threaded performance killer. Two threads write to different variables, but those variables share a cache line:

// PROBLEM: both counters on the same cache line
struct Counters {
    long thread0_count;  // bytes 0-7
    long thread1_count;  // bytes 8-15
    // Both in the same 64-byte cache line!
};

Counters shared;

// Thread 0 writes shared.thread0_count → invalidates the cache line on Thread 1
// Thread 1 writes shared.thread1_count → invalidates the cache line on Thread 0
// They fight over the same cache line even though they write different data

The mechanism is the cache coherence protocol: before a core can write to a line, it must own it exclusively, which invalidates copies in other cores’ caches. When two cores take turns writing to the same line, it bounces between them on every write, and each bounce costs on the order of a cross-core memory transfer. The program is still correct — the variables are logically independent — which is what makes false sharing hard to find. The symptom is a multithreaded loop that scales worse than expected, sometimes running slower on four threads than on one. The classic source is exactly this code: an array or struct of per-thread counters or accumulators written in a hot loop. The simplest fix is often not padding at all but accumulating into a local variable inside each thread and writing the shared slot once at the end.

Fix: pad to separate cache lines

struct alignas(64) PaddedCounter {
    long count;
    char padding[64 - sizeof(long)];  // fill the rest of the cache line
};

// Or simpler with alignas:
struct ThreadCounters {
    alignas(64) long thread0_count;
    alignas(64) long thread1_count;
    // Each is on its own cache line — no interference
};

// Or use hardware_destructive_interference_size:
struct SafeCounters {
    alignas(std::hardware_destructive_interference_size) long thread0_count;
    alignas(std::hardware_destructive_interference_size) long thread1_count;
};

Measuring false sharing:

# Linux: watch cache line invalidations with perf
perf stat -e cache-misses,cache-references,LLC-load-misses ./my_program

# Compare with padded vs unpadded version
# False sharing shows as high cache miss rate even with small working sets

Generic cache-miss counters are only a hint for false sharing, because they do not say why lines missed. On Linux, perf c2c record followed by perf c2c report is purpose-built for this: it reports cache lines with “HITM” events (loads that found the line modified in another core’s cache), along with the offsets within the line and the code addresses touching them — which usually points straight at the two fields that need separating. Intel VTune’s memory access analysis offers similar information. Always compare timings of the padded and unpadded versions under the real thread count, since false sharing disappears entirely when the threads happen to run on the same core.

Where SoA bites back

Index Mismatch on Delete

SoA stores parallel arrays — indices must stay consistent across all arrays:

// The swap-with-last delete — must apply to ALL arrays
void deleteParticle(Particles& p, std::size_t idx) {
    std::size_t last = p.count - 1;

    // Swap with last in EVERY array
    std::swap(p.x[idx],   p.x[last]);
    std::swap(p.y[idx],   p.y[last]);
    std::swap(p.z[idx],   p.z[last]);
    std::swap(p.vx[idx],  p.vx[last]);
    std::swap(p.vy[idx],  p.vy[last]);
    std::swap(p.vz[idx],  p.vz[last]);
    std::swap(p.mass[idx], p.mass[last]);
    std::swap(p.id[idx],   p.id[last]);
    std::swap(p.color[idx], p.color[last]);

    p.count--;
    // Forget one array → silent data corruption
}

An earlier version of this function swapped every array except color — which demonstrates the hazard better than any warning: the code compiled, and after a delete, one particle would silently render in another particle’s color. Two defenses help. Keep the list of arrays in one place (a helper that applies an operation to every member, or a std::tuple of vectors with std::apply), so adding a field means changing one line. And shrink the vectors too (pop_back on each, or a resize at the end), so count and each vector::size() cannot disagree; a debug assertion that all sizes match catches most mistakes early. Note also that swap-with-last changes the index of the moved element, so any external reference to “particle 99,999” by index is now wrong — ECS implementations keep a separate entity-id-to-index map for exactly this reason.

Random Access in SoA

If you frequently access all fields of a single object by index, SoA is slower — each field access may be in a different cache line:

// SoA — random access by index fetches from many cache lines
void printParticle(const Particles& p, std::size_t idx) {
    std::cout << p.x[idx] << ' '     // cache miss
              << p.y[idx] << ' '     // cache miss (different array)
              << p.z[idx] << ' '     // cache miss
              << p.mass[idx] << '\n'; // cache miss
}

For mixed access patterns, AoSoA (Array of Structures of Arrays) is sometimes the best compromise — groups of 8 or 16 objects packed together for SIMD, multiple groups forming the outer array.

The “cache miss” comments above describe the worst case, when idx is random and the particle has not been touched recently. For a handful of lookups per frame this does not matter at all; it becomes significant when code does many random single-object accesses — collision responses, network updates for specific entities — over data too large for the cache. That is the access pattern to measure before and after a layout change.

Entity Component System as SoA in practice

ECS is the mainstream game engine approach to data-oriented design:

// Each component type is stored in a contiguous array
// Systems iterate component arrays rather than game objects

struct PositionComponent { float x, y, z; };
struct VelocityComponent { float vx, vy, vz; };
struct RenderComponent { uint32_t mesh_id; uint32_t material_id; };

// Physics system touches only Position + Velocity (tight loop, great cache use)
void physicsSystem(
    std::vector<PositionComponent>& positions,
    const std::vector<VelocityComponent>& velocities,
    float dt)
{
    for (std::size_t i = 0; i < positions.size(); ++i) {
        positions[i].x += velocities[i].vx * dt;
        positions[i].y += velocities[i].vy * dt;
        positions[i].z += velocities[i].vz * dt;
    }
}

// Render system touches only Position + Render
void renderSystem(
    const std::vector<PositionComponent>& positions,
    const std::vector<RenderComponent>& renders)
{
    for (std::size_t i = 0; i < positions.size(); ++i) {
        submitDrawCall(renders[i].mesh_id, renders[i].material_id,
            positions[i].x, positions[i].y, positions[i].z);
    }
}

This sketch assumes every entity has every component and that index i means the same entity in each array, which real engines cannot assume — a static wall has a position and a mesh but no velocity. Production ECS libraries solve that in one of two ways. Archetype storage (used by Unity DOTS and flecs) groups entities with the same set of components into tables with one column per component, so iterating “all entities with Position and Velocity” walks dense, aligned arrays. Sparse set storage (used by EnTT) keeps each component in its own dense array plus an index from entity id to position, making adding and removing components cheap at the cost of some indirection during joins. Either way, the principle is the one this article started from: organize memory around the loops that run every frame, not around the objects a designer thinks in.

When I have seen DoD refactors disappoint, the cause was usually applying the layout change to code that was not memory-bound in the first place — a loop dominated by a branchy per-element function call or a lock gains little from better layout. Check with a profiler that the hot loop is actually waiting on memory (high cycles per instruction, many last-level cache misses) before restructuring data.

Data layout rules to measure against

Cache misses cost ~50-200x more than L1 hits — data layout is as important as algorithm choice
AoS is natural for OOP; SoA wins when loops access only a few fields from large arrays
Cache lines are 64 bytes — pack frequently co-accessed data within 64-byte boundaries
False sharing occurs when two threads write different variables that share a cache line — use alignas(64) to isolate hot written data
SoA pitfalls: maintain index consistency across all parallel arrays, and avoid random single-object lookups
Measure with perf stat -e cache-misses before optimizing — false sharing and AoS/SoA problems look similar in profiles
ECS (Entity Component System) in game engines is the mainstream application of SoA principles at scale

Frequently Asked Questions (FAQ)

Q. I converted my hot structs to SoA and some code got slower. Why?

A. SoA only wins when a loop touches a few fields across many objects. Code that reads every field of one object by index (printing, serializing, per-entity updates) now pulls each field from a different array and a different cache line, so it can get slower than AoS. For mixed access patterns, keep the rarely used (cold) fields out of the hot arrays or group objects into small SoA blocks (AoSoA), and measure with perf stat before and after instead of assuming the change helped.