C++ Performance Optimization: Measure First, Then Copies, Allocations, Cache and Compiler Flags

Start with a profile, not a guess

The single most reliable rule of performance work: the slow part is rarely where you expect. Before changing code, run a profiler on a realistic workload and look at where time actually goes.

# Linux: sample the whole program, with call stacks
g++ -O2 -g -fno-omit-frame-pointer app.cpp -o app
perf record -g ./app
perf report

Two flags matter here. -g adds debug info so the report shows function names and lines, without changing the generated code. -fno-omit-frame-pointer makes stack unwinding reliable, so time is attributed to the right callers; the cost is usually small. Profile an optimized build: an -O0 profile mostly measures code the compiler would have removed.

On Windows, the Visual Studio profiler (CPU Usage tool) gives the same view; Callgrind works anywhere Valgrind does but runs the program much slower, so it suits deterministic instruction counts rather than wall-clock timing. gprof still exists, but it needs -pg instrumentation, handles shared libraries and multithreaded programs poorly, and is rarely the best choice today.

Then fix things in roughly this order, because each level usually dwarfs the next:

1. Algorithm and data structure. O(n^2) to O(n log n), a linear search to a hash lookup.
2. Allocations and copies. Work the program does that nobody asked for.
3. Memory access patterns and layout. Cache misses cost far more than arithmetic.
4. Compiler flags, then SIMD and parallelism.

The measurements below come from one machine (g++ 10.3, -O2, Windows, best of five runs). They illustrate the size of effects, not numbers to expect on your hardware.

---

Remove copies you did not mean to make

Parameters

void process(std::vector<int> data);          // copies the caller's vector on every call
void process(const std::vector<int>& data);   // reads without copying
void process(std::vector<int>& data);         // modifies the caller's vector
void consume(std::vector<int> data);          // takes ownership: caller may std::move into it

Pass-by-value is not wrong in itself. It is the right signature when the function keeps its own copy anyway (a constructor storing a member, a setter), because callers can then std::move in and pay nothing. It is a bug when the function only reads. The classic hidden version is a range-for loop that copies each element:

for (auto item : items) { /* ... */ }         // copies every element
for (const auto& item : items) { /* ... */ }  // no copies

With std::map<std::string, Big>, even for (const std::pair<std::string, Big>& kv : m) copies, because the map’s element type is std::pair<const std::string, Big> and the mismatched reference binds to a converted temporary. const auto& avoids the trap entirely.

Returns

std::vector<int> makeData() {
    std::vector<int> v(1'000'000);
    return v;             // NRVO or, at worst, a move; never a deep copy since C++11
}

std::vector<int> result = makeData();

Returning a local by value is the fast path. Writing return std::move(v); is a pessimization: it prevents the compiler from constructing v directly in the caller’s storage, and GCC and Clang warn about it with -Wpessimizing-move.

---

Allocation

Reserve when you know the size

std::vector<int> v;
v.reserve(n);                  // one allocation
for (int i = 0; i < n; ++i) v.push_back(i);

Without reserve, std::vector grows geometrically, so the total copying is still amortized O(n), but each growth step allocates, moves every element, and frees the old buffer. For large vectors of cheap elements that is noticeable; for vectors of types without a noexcept move constructor it is worse, because the vector must copy elements on growth to keep its exception guarantee. Marking your move constructors noexcept is itself a performance fix.

String building

A common claim is that std::ostringstream is faster than repeated +=. On the machine above, building a string from one million numbers:

Method	Time
std::string with +=	about 12 ms
+= after reserve	about 11 ms
std::ostringstream	about 24 ms

+= on std::string already grows geometrically, and streams add locale and formatting machinery per insertion. Use streams for formatting convenience, not speed; for fast number-to-text conversion, std::to_chars (C++17) avoids both allocation and locale.

Small allocations in hot paths

Every new in a tight loop is a trip to the general-purpose allocator. The usual remedies, in increasing order of effort: reuse a container across iterations (clear() keeps the capacity), use std::pmr resources for arena-style allocation, or a custom pool for one object type. Measure before writing a pool; modern allocators are fast for common sizes, and a pool that hands out raw pointers adds ownership bugs.

---

Memory access patterns

Traversal order

std::vector<int> m(N * N);   // row-major: element (i, j) at i * N + j

for (int i = 0; i < N; ++i)          // inner loop walks contiguous memory
    for (int j = 0; j < N; ++j)
        m[i * N + j] += 1;

for (int j = 0; j < N; ++j)          // inner loop jumps N ints each step
    for (int i = 0; i < N; ++i)
        m[i * N + j] += 1;

For N = 4000 on the test machine, the row-order loop took about 7 ms and the column-order loop about 34 ms: the same arithmetic, five times slower, purely from cache misses. Row order uses every byte of each 64-byte cache line it loads and lets the hardware prefetcher run ahead; column order uses 4 bytes per line and defeats the prefetcher.

Layout: keep hot data together

// Every particle update loads the name and history it never reads
struct Particle {
    float x, y, z;
    float vx, vy, vz;
    std::string name;
    std::array<float, 64> history;
};

// Split hot and cold fields; the update loop touches only what it needs
struct ParticleHot  { float x, y, z, vx, vy, vz; };
struct ParticleCold { std::string name; std::array<float, 64> history; };
std::vector<ParticleHot>  hot;
std::vector<ParticleCold> cold;   // same index

When a loop touches a few fields of many objects, the size of each object determines how many useful values fit in cache. Splitting hot from cold fields, or going further to a struct-of-arrays layout (std::vector<float> x, y, z), often beats any instruction-level tweak. Pointer-heavy structures (std::list, std::map, trees of unique_ptr) are the opposite: each node is a separate allocation somewhere in memory, and traversal is a chain of cache misses. That is why a sorted std::vector with binary search frequently beats std::map for read-mostly data.

More on this in the cache optimization guide and alignment and padding.

---

Compiler flags

g++ -O2 app.cpp                     # the usual release baseline
g++ -O3 app.cpp                     # more inlining and vectorization
g++ -O2 -march=x86-64-v3 app.cpp    # allow AVX2 etc.; binary needs a matching CPU
g++ -O2 -flto a.cpp b.cpp           # link-time optimization across files

• -O3 is not automatically faster. It helps numeric kernels and can hurt large programs through code growth. Benchmark both.
• -march=native is fine for code that runs where it is built (benchmarks, HPC jobs). For anything you ship, it is how you get Illegal instruction crashes on older CPUs — the version of this I have seen most is a build server newer than some production machines. Choose an explicit level (x86-64-v2, -v3) or dispatch at runtime.
• -flto lets the optimizer inline across translation units, which matters when hot functions live in different .cpp files. It must be passed at link time too, and static libraries need gcc-ar so they keep the LTO data.
• -ffast-math allows reassociation and assumes no NaNs or infinities. It can speed up floating-point loops, and it can also silently break code that checks std::isnan, relies on exact summation order, or handles infinities. Enable it per file, knowingly, never globally by default.

inline is not an optimization switch

inline on a function mostly changes linkage rules: it allows the definition to appear in multiple translation units (as in headers). Whether a call is actually inlined is the optimizer’s decision, based on its own cost model; at -O2 small functions are inlined whether or not you write the keyword. constexpr is also not a speed hint for runtime calls; it only enables compile-time evaluation where a constant is required.

Clever code is not faster code

int a = x * 2 + y / 4;
int b = (x << 1) + (y >> 2);   // not equivalent for negative y

For y = -7, y / 4 is -1 (division truncates toward zero) but y >> 2 is -2 (arithmetic shift rounds down). The compiler already turns multiplication and division by powers of two into shifts where it is correct, adding a fix-up for signed division. Hand-written shifts are harder to read and, here, give a different answer.

---

Precomputation, correctly

Lookup tables can replace expensive computation, but the table must represent the same function:

// Precomputed sin for whole degrees
class SinTable {
    std::array<double, 360> table_;
public:
    SinTable() {
        for (int d = 0; d < 360; ++d)
            table_[d] = std::sin(d * 3.141592653589793 / 180.0);
    }
    double sinDegrees(int d) const {
        d %= 360;
        if (d < 0) d += 360;           // % keeps the sign of the dividend
        return table_[d];
    }
};

A version that stores sin(i * 0.01) for 360 entries and looks up x % 360 returns wrong values for any input past 3.59 radians, because the period of sin is not 360 steps of 0.01. The negative-index line matters too: -30 % 360 is -30 in C++, which indexes before the array. Tables also trade computation for memory traffic; for a function that is already cheap, a table that misses cache can be slower than recomputing.

---

SIMD

#include <immintrin.h>

// Compile with -mavx (or a -march that includes AVX) and call only on CPUs that support it
void add_avx(const float* a, const float* b, float* c, std::size_t n) {
    std::size_t i = 0;
    for (; i + 8 <= n; i += 8) {
        __m256 va = _mm256_loadu_ps(a + i);
        __m256 vb = _mm256_loadu_ps(b + i);
        _mm256_storeu_ps(c + i, _mm256_add_ps(va, vb));
    }
    for (; i < n; ++i) c[i] = a[i] + b[i];   // scalar tail for the remainder
}

Before writing intrinsics, check whether the compiler already vectorized the plain loop: -fopt-info-vec (GCC) or -Rpass=loop-vectorize (Clang) reports it, and Compiler Explorer shows the generated instructions. Simple element-wise loops like this one are usually auto-vectorized at -O3 (and at -O2 in GCC 12 and later). Intrinsics earn their complexity for things compilers do not vectorize well: shuffles, gathers, loops with early exits, or when you need a specific instruction. They also tie the code to one instruction set, which brings back the -march and dispatch question above.

---

The optimization steps and their traps

Step	What to do	Typical trap
Measure	Profile an optimized build with symbols	Optimizing an -O0 profile or a guess
Copies	const auto& loops, by-value only to take ownership	return std::move(local), mismatched pair type in map loops
Allocation	reserve, reuse buffers, noexcept moves	Assuming streams are faster than +=
Memory	Row-order traversal, hot/cold split, contiguous containers	Node-based containers on hot paths
Flags	-O2 or measured -O3, explicit -march, LTO	-march=native in shipped binaries, global -ffast-math
SIMD	Check auto-vectorization first	Intrinsics without a scalar tail or CPU check

Measure again after each change, one change at a time. Performance work without a before-and-after number is guesswork.

---

Related Articles

• C++ Profiling
• C++ Cache Optimization
• C++ Alignment and Padding
• C++ Benchmarking
• std::string vs std::string_view