False sharing in C++: cache line alignment and the MESI penalty

Scope: This post explains false sharing — the cache coherency problem that slows concurrent code without any data race, any lock contention, or any logical error. It covers the MESI cache coherency protocol, how struct layout causes false sharing, and how alignas eliminates it. All code is on GitHub.

The problem#

Two threads. Two logically independent variables. No shared state, no locks, no data races. And yet the code runs slower than it should — sometimes significantly slower, in a way that varies by CPU and disappears on single-core runs.

This is false sharing. It's a hardware problem, not a software one. The fix is a single line of C++, but understanding why the fix works requires understanding how CPU caches communicate.

Cache lines and coherency#

Modern CPUs don't read individual bytes from memory. They read in fixed-size blocks called cache lines — 64 bytes on virtually every x86 and ARM processor shipping today. When you access any byte in a cache line, the entire 64-byte block gets loaded into the L1 cache.

This matters for multi-core systems because each core has its own L1 and L2 caches. When two cores hold a copy of the same cache line and one of them writes to it, the hardware must ensure the other core doesn't read stale data. This is cache coherency.

The protocol that enforces it on x86 is called MESI. It tracks each cache line in one of four states:

The key transition: when a core writes to a cache line in S or E state, it broadcasts an invalidation message to all other cores. Any core holding a copy of that line transitions it to I. The next time those cores read from the line — even from a different byte on the same line — they must fetch the updated line from the writing core's L1/L2 or from L3.

That fetch is expensive. L1 access is ~4 cycles. A cross-core cache line transfer is 40–100 cycles depending on the CPU's cache topology.

False sharing: paying the coherency cost for nothing#

False sharing happens when two threads access logically unrelated variables that happen to sit on the same cache line.

Neither thread is accessing the other's variable. There's no data dependency, no race condition. But the hardware doesn't know that — it operates on cache lines, not variables. Every write by either thread invalidates the entire line on the other core, forcing a reload on the next access, even though the accessed byte didn't change.

The cost is paid silently. The code is correct. The profiler shows high cache miss rates. The effect is hard to spot without knowing what to look for.

A concrete example: the SPSC ring buffer#

A lock-free single-producer single-consumer ring buffer is the canonical example. Two threads, two indices:

Copytemplate <typename T, size_t N>
class RingBuffer {
    std::array<T, N> _Buffer;
    std::atomic<size_t> _Head;  // written by consumer, read by producer
    std::atomic<size_t> _Tail;  // written by producer, read by consumer
};

Without alignment hints, the compiler places _Head and _Tail adjacently in memory. They're 8 bytes each — they fit together in a single 64-byte cache line.

CopyCache line (64 bytes)
┌────────────────────────────────────────────────────┐
│ ... │ _Head (8B) │ _Tail (8B) │ padding            │
└────────────────────────────────────────────────────┘
           ↑               ↑
    consumer writes   producer writes

Every time the producer writes _Tail to publish a new item, it transitions the cache line to M on its core. The consumer's copy of the line is invalidated. When the consumer next reads _Head — which it owns and which didn't change — it must nonetheless fetch the entire line because the producer dirtied it.

The inverse happens on every Pop: the consumer writes _Head, invalidating the line on the producer's core. The producer reads _Tail next — its own variable, unchanged — but must still reload the line.

Every push/pop pair triggers a full cross-core cache line round-trip, even though the two threads are accessing logically independent variables. The throughput degrades proportionally to how fast those round-trips complete.

The fix: alignas#

64 bytes is the cache line size on every mainstream x86 CPU and most ARM chips — but not all. Some ARM implementations use larger values. std::hardware_destructive_interference_size exists precisely to handle this: it returns the correct value for the target platform rather than assuming 64. Use it with a fallback for older toolchains:

Copy#ifdef __cpp_lib_hardware_interference_size
    static constexpr size_t CacheLineSize = std::hardware_destructive_interference_size;
#else
    static constexpr size_t CacheLineSize = 64;
#endif

Force each atomic onto its own cache line:

Copytemplate <typename T, size_t N, bool Aligned = true>
class RingBuffer {
    std::array<T, N> _Buffer;
    alignas(Aligned ? CacheLineSize : alignof(size_t)) std::atomic<size_t> _Head {0};
    alignas(Aligned ? CacheLineSize : alignof(size_t)) std::atomic<size_t> _Tail {0};
};

alignas(CacheLineSize) tells the compiler to place the variable at a memory address that's a multiple of CacheLineSize. This guarantees _Head and _Tail each occupy their own cache line.

CopyCache line A (64 bytes)          Cache line B (64 bytes)
┌──────────────────────┐         ┌──────────────────────┐
│ _Head (8B) │ padding │         │ _Tail (8B) │ padding │
└──────────────────────┘         └──────────────────────┘
       ↑                                 ↑
 consumer writes                   producer writes

Now writes to _Tail don't touch the cache line containing _Head. The producer can keep _Tail's line in M state indefinitely — no invalidation. The consumer can keep _Head's line in M state indefinitely. Cross-core traffic only happens when one thread reads the other's index — which is necessary and unavoidable — not on every single operation.

One maintenance hazard: alignas on a member pads the space before it, not after. If someone later inserts a new member between _Head and _Tail, the alignment still holds for each member individually — but any new member that lands on the same cache line as _Head will re-introduce false sharing. Keep hot atomics in their own structs or at the end of the class where accidental neighbors are less likely.

Benchmarks#

Two benchmarks, measuring different things. All numbers on Apple M-series (10-core, ARM).

Ring buffer: realistic contention#

Producer and consumer running concurrently, consumer sleeping 50ns between pops — approximating a thread doing real work between iterations:

Copystd::jthread Reader([&Rb](std::stop_token St) {
    while (!St.stop_requested())
        if (!Rb.Pop())
            std::this_thread::sleep_for(std::chrono::nanoseconds(50));
});

size_t i = 0;
for (auto _: State)
    while (!Rb.Push(i++))
        std::this_thread::yield();

~29% penalty. The sleep softens the contention — the consumer is off the CPU most of the time, so the cache line isn't being fought over continuously.

Counter: raw false sharing cost#

Two threads in a tight loop, each incrementing its own atomic counter with no sleep. Nothing else happening — just two threads invalidating each other's cache line on every operation:

Copystruct CountersUnaligned {
    std::atomic<size_t> A {0};  // thread 1 writes
    std::atomic<size_t> B {0};  // thread 2 writes — same cache line as A
};

struct CountersAligned {
    alignas(CacheLineSize) std::atomic<size_t> A {0};
    alignas(CacheLineSize) std::atomic<size_t> B {0};
};

5.6× slower. This is the hardware cost in isolation — no work being done, no sleep, just the coherency round-trip on every increment. The counter never leaves its core's L1 cache when aligned; unaligned, every increment forces a cross-core invalidation and refetch of the shared line.

The counter shows the maximum possible false sharing penalty — pure coherency traffic with nothing else going on. The ring buffer shows what you actually pay under realistic load, where the consumer does work between pops and the cache line isn't contested on every single operation.

One thing worth noting: memory_order_relaxed does not avoid this cost. Relaxed ordering removes synchronization constraints, but the cache coherency protocol is a hardware mechanism — it operates regardless of the memory order you specify. A relaxed store to a shared cache line still invalidates other cores' copies.

When to apply this#

alignas(CacheLineSize) wastes memory — each padded variable consumes a full cache line regardless of its actual size. An atomic<size_t> is 8 bytes; with padding it occupies CacheLineSize bytes. For most structs, this is irrelevant. For structs in hot paths where multiple cores write to different fields simultaneously, it's worth profiling.

Apply it when:

• Two or more threads write to different fields of the same struct concurrently at high frequency
• A profiler shows unexpectedly high L1/L2 cache miss rates on a struct
• You're building a data structure explicitly designed for concurrent access (queues, work-stealing deques, sharded counters)

Don't apply it preemptively to every shared struct. The padding costs memory, and the benefit only materializes under genuine multi-core write contention. Measure first.

Other common patterns#

The ring buffer is the clearest example, but the same issue appears anywhere multiple threads write to logically independent data that happens to be adjacent in memory:

Sharded counters: a stats struct with per-operation counters updated by different threads. Without padding, all counters share a few cache lines and contend on every increment.

Thread-local state arrays: a std::array<ThreadState, N> where thread i writes exclusively to _State[i]. If sizeof(ThreadState) isn't a multiple of 64, adjacent entries share cache lines and threads contend on each other's state.

Producer/consumer indices in any queue: the pattern from the ring buffer applies to any structure where one thread advances a head pointer and another advances a tail pointer.

In all these cases, the fix is the same: ensure the independently-written data occupies separate cache lines.

The read-write trap#

False sharing doesn't require two writers. A reader can be a victim too.

Suppose you add a const size_t _Capacity to the ring buffer and it lands on the same cache line as _Head. The consumer writes _Head on every pop, transitioning that line to M. The producer reads _Capacity to check for space — it never writes it — but it still has to reload the line because the consumer dirtied it. A value that never changes still causes cache misses because of what it shares a line with.

The general rule: group data by access pattern, not by logical relationship.

Copy┌──────────────────────────────────┐
│ Read-only data (shared, stable)  │  ← one cache line
├──────────────────────────────────┤
│ Data written by thread A         │  ← padded to its own line
├──────────────────────────────────┤
│ Data written by thread B         │  ← padded to its own line
└──────────────────────────────────┘

Read-only data can share a cache line freely — multiple cores can hold it in S state simultaneously with no coherency traffic. The only groupings that need separation are between any two threads that write concurrently.

Finding false sharing in production#

The profiler shows high cache miss rates, but standard profilers can't tell you why a line is being missed or which other core is responsible. On Linux, perf c2c fills this gap.

perf c2c monitors HITM (Hit Modified) hardware events — these fire when a core tries to load a cache line that's currently in the M state on another core's L1. That's the exact hardware signature of false sharing.

Copyperf c2c record ./your-binary
perf c2c report

The report shows contended cache lines sorted by HITM count, with the source locations that access them. The top entries are your false sharing suspects. It turns "I think I have false sharing somewhere" into "it's this struct, this field, accessed from these two call sites."