Profiling & Optimization

Reviewed & published by Brayan K

By the end of this lesson you'll be able to time C++ code accurately with std::chrono, avoid the traps that make micro-benchmarks lie, reach for the right profiler to find real hotspots, and reason about whether an optimisation is even worth doing — so you optimise what matters instead of guessing.

Part of the free C++ course at LearnCodingFast — hands-on lessons with worked examples and the output they print, plus practice exercises and a quick quiz.

What You'll Learn

• Live by the golden rule: measure, don't guess
• Time code with std::chrono::high_resolution_clock
• Dodge micro-benchmark pitfalls (code optimised away, cold cache)
• Pick the right tool: perf, Valgrind/Callgrind, gprof, Google Benchmark
• Read a profile and find the true hotspot
• Apply Amdahl's law and weigh big-O against constant factors

💡 Real-World Analogy

Optimising without profiling is like a doctor prescribing surgery before running any tests. You feel the problem is the heart, so you operate on the heart — when the real issue was a vitamin deficiency. A profiler is the diagnostic scan: it shows you exactly which part of the body (your code) is consuming the time. Only after the scan do you pick the treatment. Programmers who skip the scan spend days "fixing" a function that runs for 0.1% of the total time, while the real culprit sits untouched. Measure first, then operate.

1. Measure, Don't Guess — Timing with std::chrono

The single most expensive habit in performance work is guessing. Your intuition about which line is slow is almost always wrong, because the CPU's caches, branch predictor, and the optimiser reshape your code in ways you can't see. The cure is to measure. The simplest tool is std::chrono: take a timestamp before the work, another after, and subtract. high_resolution_clock is the finest-grained clock the standard library offers — perfect for small benchmarks.

The pattern is always the same: auto start = Clock::now(); → do the work → auto end = Clock::now(); → duration_cast the gap into the unit you want (microseconds here). Read the worked example, run it, and notice it checks both answers match before comparing speed — a fast wrong answer is worthless.

#include <iostream>
#include <chrono>
#include <vector>
#include <numeric>   // accumulate, iota
using namespace std;

// chrono is the standard timing toolkit. high_resolution_clock is the
// finest-grained clock available, good for micro-benchmarks.
using Clock = chrono::high_resolution_clock;

// APPROACH A: sum with a hand-written loop.
long long sumLoop(const vector<int>& v) {
    long long total = 0;
    for (int x : v) total += x;   // visit every element once -> O(n)
    return total;
}

// APPROACH B: sum with the standard-library algorithm.
long long sumAccumulate(const vector<int>& v) {
    return accumulate(v.begin(), v.end(), 0LL); // 0LL = long long zero
}

int main() {
    const int N = 1000000;
    vector<int> data(N);
    iota(data.begin(), data.end(), 1);   // fill with 1, 2, 3, ... N

    // PATTERN: read the clock, do the work, read the clock again,
    //          then subtract to get the elapsed duration.
    auto start = Clock::now();               // timestamp BEFORE
    long long a = sumLoop(data);
    auto end = Clock::now();                  // timestamp AFTER
    auto loopUs = chrono::duration_cast<chrono::microseconds>(end - start).count();

    start = Clock::now();
    long long b = sumAccumulate(data);
    end = Clock::now();
    auto accUs = chrono::duration_cast<chrono::microseconds>(end - start).count();

    // Always confirm both approaches produce the SAME answer first.
    cout << "Manual loop result:     " << a << endl;
    cout << "std::accumulate result: " << b << endl;
    cout << "Results match: " << (a == b ? "yes" : "NO!") << endl;

    // Now compare the timings (microseconds = millionths of a second).
    cout << "Manual loop:     " << loopUs << " us" << endl;
    cout << "std::accumulate: " << accUs << " us" << endl;
    // Exact numbers vary by machine; the point is HOW you measure,
    // not the specific figure. Both are O(n), so they are similar.
    return 0;
}

// ⚠️ No expected-output panel for this one, on purpose.
// These numbers are stopwatch readings. They depend on the machine, the
// compiler and what else it is doing, so only the direction of the
// comparison is the lesson — never the figures.
//
// One real run on the machine that builds this site printed:
//    Manual loop result:     500000500000
//    std::accumulate result: 500000500000
//    Results match: yes
//    Manual loop:     11104 us
//    std::accumulate: 12015 us

Your turn. The program below times how long it takes to build a big vector — fill in the two timestamp blanks and the conversion, then run it.

#include <iostream>
#include <chrono>
#include <vector>
using namespace std;

using Clock = chrono::high_resolution_clock;

int main() {
    // 🎯 YOUR TURN — time how long it takes to build a big vector.
    // Replace each ___ then press "Try it Yourself".

    const int N = 500000;
    vector<long long> v;

    // 1) Take a timestamp BEFORE the work
    auto start = ___;        // 👉 Clock::now()

    long long sum = 0;
    for (int i = 1; i <= N; i++) { v.push_back(i); sum += i; }

    // 2) Take a timestamp AFTER the work
    auto end = ___;          // 👉 Clock::now()

    // 3) Convert the gap to microseconds (us)
    auto us = chrono::duration_cast<chrono::microseconds>(end - start).count();

    cout << "Sum: " << sum << endl;          // print the result so it
    cout << "Took: " << us << " us" << endl; // can't be optimised away

    // ✅ Expected output (numbers vary by machine):
    //    Sum: 125000250000
    //    Took: 4213 us
    return 0;
}

2. Micro-Benchmark Pitfalls

A micro-benchmark times a tiny isolated snippet — and tiny snippets lie more than any other kind of measurement. The most infamous trap: if you compute a result and never use it, an optimising compiler is allowed to delete the entire calculation. Your benchmark then reports near-zero time and you celebrate a speed-up that never happened. The fix is to make the result observable — print it, or store it in a volatile so the compiler must actually do the work.

#include <iostream>
#include <chrono>
using namespace std;

using Clock = chrono::high_resolution_clock;

int main() {
    const int N = 100000000;

    // PITFALL: this loop computes a sum that is NEVER USED.
    // With -O2 the optimiser deletes the whole loop -> ~0 us. The
    // "blazing fast" result is an illusion: nothing actually ran.
    {
        auto start = Clock::now();
        long long sum = 0;
        for (int i = 0; i < N; i++) sum += i;   // result discarded
        auto end = Clock::now();
        auto us = chrono::duration_cast<chrono::microseconds>(end - start).count();
        cout << "Result ignored: " << us << " us  (may be deleted!)" << endl;
    }

    // FIX: make the result OBSERVABLE so the compiler must keep the work.
    // Printing it (or feeding it to a volatile sink) prevents elision.
    {
        auto start = Clock::now();
        volatile long long sum = 0;             // volatile = "really store this"
        for (int i = 0; i < N; i++) sum += i;   // now the work survives
        auto end = Clock::now();
        auto us = chrono::duration_cast<chrono::microseconds>(end - start).count();
        cout << "Result kept:    " << us << " us  (sum=" << sum << ")" << endl;
    }
    return 0;
}

// ⚠️ No expected-output panel for this one, on purpose.
// These numbers are stopwatch readings. They depend on the machine, the
// compiler and what else it is doing, so only the direction of the
// comparison is the lesson — never the figures.
//
// One real run on the machine that builds this site printed:
//    Result ignored: 256372 us  (may be deleted!)
//    Result kept:    247130 us  (sum=4999999950000000)

The other three micro-benchmark traps

• Cold cache: the first run pays to pull data into the CPU cache; later runs are warm and far faster. Add a warm-up pass you don't time, then measure.
• One sample: a single run is noise. Run many times and report the median or minimum, not one figure.
• Debug build: timing a -O0 build tells you nothing about release performance. Always benchmark with -O2/-O3.

3. Profilers — Find the Real Hotspot

chrono answers "how long does this take?". A profiler answers the more important question: "across my whole program, where is the time going?". It runs your program and reports a breakdown by function. A hotspot is a function with a large share of the total — that's where optimisation pays off. There are two flavours: sampling profilers (like perf) interrupt the program thousands of times a second and record where it is, giving low overhead and realistic numbers; instrumenting profilers (like Callgrind) count every call exactly, giving precise call graphs but running much slower.

🔧 The toolbox at a glance

perf (Linux): the go-to sampling profiler. perf record ./app then perf report shows a sorted list of hotspots with almost no slowdown. First choice for real workloads.

Valgrind / Callgrind: valgrind --tool=callgrind ./app gives an exact call graph and per-line instruction counts. Visualise it with kcachegrind. Slow (10–50×) but deterministic — great for understanding why a function is hot.

gprof: the classic. Compile with g++ -pg, run, then gprof a.out gmon.out. Simple function-level timings; dated, but everywhere.

Google Benchmark: a micro-benchmark library, not a profiler. It runs each case enough times for stable numbers, handles warm-up, and offers benchmark::DoNotOptimize(x) to defeat exactly the "optimised away" trap from Section 2. Reach for it when you're comparing two implementations carefully.

Whatever the tool, reading a profile is the same skill: look at the top of the sorted list, ignore the long tail. A function eating 60% of the run time is your target; a hundred functions at 0.2% each are not worth touching. This is where your effort multiplies.

4. Is It Even Worth It? Amdahl's Law & Big-O

Before you optimise, ask what the payoff can be. Amdahl's law says your overall speed-up is capped by the part you don't improve. If a function is 90% of the run time, making it infinitely fast still only makes the whole program 10× faster — and optimising a 10% slice can never beat 1.11× no matter what. This is the maths behind "optimise the biggest slice the profiler shows you."

#include <iostream>
#include <iomanip>
using namespace std;

// Amdahl's law: if a fraction p of the program can be sped up by factor s,
// the WHOLE program speeds up by only:  1 / ((1 - p) + p / s)
// The (1 - p) part you did NOT optimise puts a hard ceiling on your gains.
double amdahl(double p, double s) {
    return 1.0 / ((1.0 - p) + p / s);
}

int main() {
    cout << fixed << setprecision(2);

    // Suppose a function is 90% of total run time (p = 0.90).
    // Even if you make THAT function infinitely fast (s -> huge)...
    cout << "Speed up 90% of code by 2x:  " << amdahl(0.90, 2)   << "x faster\n";
    cout << "Speed up 90% of code by 10x: " << amdahl(0.90, 10)  << "x faster\n";
    cout << "Speed up 90% by INFINITY:    " << amdahl(0.90, 1e9) << "x faster\n";

    // ...the program can never beat 10x, because the other 10% is untouched.
    // Lesson: optimise the BIGGEST slice (find it with a profiler), and
    // know the ceiling before you spend a week chasing it.
    cout << "Speed up the wrong 10% by 100x: " << amdahl(0.10, 100) << "x faster\n";
    return 0;
}

// ✅ Expected output:
//    Speed up 90% of code by 2x:  1.82x faster
//    Speed up 90% of code by 10x: 5.26x faster
//    Speed up 90% by INFINITY:    10.00x faster
//    Speed up the wrong 10% by 100x: 1.11x faster

The other lens is big-O vs constant factors. Big-O tells you how cost grows as the input grows — O(n) doubles when the input doubles, O(n²) quadruples. It is the most reliable performance lever you have, because a better big-O wins by ever-larger margins as data scales. But big-O hides the constant factor: for small inputs a "worse" algorithm with a tiny constant can win. Below, replace the blank so you can watch an O(1) hash lookup demolish an O(n) linear scan as soon as the data is non-trivial.

#include <iostream>
#include <chrono>
#include <vector>
#include <unordered_set>
using namespace std;

using Clock = chrono::high_resolution_clock;

int main() {
    // 🎯 YOUR TURN — big-O vs constant factors.
    // Looking a value up in a vector is O(n) (scan every element).
    // Looking it up in an unordered_set is O(1) average (hash jump).
    const int N = 20000;
    vector<int> vec;
    unordered_set<int> set;
    for (int i = 0; i < N; i++) { vec.push_back(i); set.insert(i); }

    int hits = 0;
    auto start = Clock::now();
    for (int q = 0; q < N; q++) {
        // 1) Search the VECTOR (slow O(n) linear scan)
        for (int x : vec) if (x == q) { hits++; break; }
    }
    auto vecUs = chrono::duration_cast<chrono::microseconds>(Clock::now() - start).count();

    start = Clock::now();
    for (int q = 0; q < N; q++) {
        // 2) Search the SET in O(1) average — replace ___
        if (___ ) hits++;   // 👉 set.count(q)  (returns 1 if present, else 0)
    }
    auto setUs = chrono::duration_cast<chrono::microseconds>(Clock::now() - start).count();

    cout << "Total hits: " << hits << endl;        // each loop finds N
    cout << "vector (O(n)):       " << vecUs << " us" << endl;
    cout << "unordered_set (O(1)): " << setUs << " us" << endl;

    // ✅ Expected output (numbers vary, set is DRAMATICALLY faster):
    //    Total hits: 40000
    //    vector (O(n)):       180000 us
    //    unordered_set (O(1)): 900 us
    return 0;
}

Pro Tips

• 💡 Profile a release build: measure -O2/-O3 code. Debug timings are meaningless for production.
• 💡 Fix the algorithm before the line: a better big-O beats any amount of micro-tuning. Change the data structure first.
• 💡 Report the median, not the mean: one slow outlier (a context switch) skews the average; the median ignores it.
• 💡 Defeat the optimiser deliberately: use volatile or benchmark::DoNotOptimize so the work you're timing actually runs.

Common Errors (and the fix)

• Optimising without measuring: you rewrite a "slow-looking" loop and the program is no faster — because that loop was never the hotspot. Fix: run a profiler (perf record) first and optimise only the top entries.
• Benchmark optimised away (reports 0 us): the result is unused so the compiler deleted the work. Fix: make it observable — print it, store it in a volatile, or use benchmark::DoNotOptimize(result).
• Cold-cache skew: the first timed run is far slower than the rest. Fix: run an untimed warm-up pass, then time several runs and take the median.
• Timing a debug build: a -O0 build is 10–100× slower and reorders nothing. Fix: always benchmark and profile a -O2/-O3 build.
• Chasing the wrong slice: you speed up a function that is 5% of run time by 50× and the program barely moves (Amdahl's law). Fix: spend effort on the biggest slice the profile shows.

📋 Quick Reference — Tools

Mini-Challenge: Benchmark Two String Builders

No blanks this time — just a brief and a blank canvas (with an outline to keep you on track). Time two ways of building a big string, then prove to yourself which one wins and by how much. Remember to print a result so the compiler can't delete your benchmark.

#include <iostream>
#include <chrono>
#include <string>
using namespace std;

using Clock = chrono::high_resolution_clock;

int main() {
    // 🎯 MINI-CHALLENGE: benchmark string building
    // Compare two ways to build one big string of 100000 "x" characters.
    //
    // 1. APPROACH A — naive concatenation in a loop:
    //      string a;
    //      for (...) a = a + "x";     // each += may COPY the whole string
    //    Time it with Clock::now() before and after.
    //
    // 2. APPROACH B — append in place:
    //      string b;
    //      for (...) b += "x";        // appends without rebuilding
    //    Time this one too.
    //
    // 3. Print both durations in microseconds, AND print a.size()
    //    so the optimiser can't delete the work.
    //
    // ✅ Example output (B should be much faster):
    //    Concat (a = a + x): 95000 us
    //    Append (b += x):    300 us
    //    Length: 100000

    // your code here
    return 0;
}

🎉 Lesson Complete

• ✅ The golden rule: measure, don't guess — your intuition about speed is usually wrong
• ✅ Time code with Clock::now() before and after, then duration_cast the gap
• ✅ Micro-benchmarks lie: code can be optimised away, a cold cache skews the first run, one sample is noise
• ✅ Profilers find hotspots: perf (sampling), Callgrind (exact), gprof (classic), Google Benchmark (micro)
• ✅ Read a profile by the top of the list; ignore the long tail
• ✅ Amdahl's law caps your gains; a better big-O beats constant-factor tuning at scale

Practice quiz

Continue this course

• Previous: Advanced Debugging: Valgrind, GDB, ASan, UBSan
• Next: Understanding Undefined Behavior & How to Avoid It — Common UB traps, how to detect them, and coding practices that prevent them
• Quick reference: C++ cheat sheet

Frequently asked questions

Why is everyone so insistent on 'measure, don't guess'?

Because human intuition about performance is almost always wrong. The line you think is slow is rarely the one the CPU spends time on — branch prediction, caches, and the optimiser change everything. A profiler tells you the truth in seconds; a guess can send you optimising code that runs 0.1% of the time.

What is the difference between profiling and benchmarking?

Benchmarking measures how long a specific piece of code takes (the 'how fast?' number). Profiling measures where a whole program spends its time across all its functions (the 'which part is slow?' map). You benchmark a candidate fix; you profile to find what to fix in the first place.

My timing numbers jump around between runs. Is something broken?

No — that is normal. The OS scheduler, other processes, CPU frequency scaling, and a cold cache all add noise. Run each benchmark several times, warm up the cache first, and report the median (or minimum) rather than a single run. For serious work use a library like Google Benchmark that handles this for you.

Should I always pick the lower big-O algorithm?

Usually, but not blindly. Big-O describes how cost grows as input grows; it hides the constant factor. For small inputs an O(n^2) routine with a tiny constant can beat an O(n log n) one with a big constant. Measure at your real input size — big-O tells you what wins eventually, the profiler tells you what wins today.

Why did my benchmark report 0 microseconds?

The optimiser almost certainly deleted your code. If a computed result is never used, the compiler is allowed to remove the whole calculation. Force the result to be observed — print it, accumulate it into a volatile, or feed it to a do-not-optimise sink — so the work cannot be elided.