Understanding SOH vs LOH Allocation Strategy for Performance

The Two Heaps That Shape .NET Memory Performance

Every object you allocate in .NET gets routed through one of two heaps. There’s no single amorphous block of memory behind the scenes. Instead, you’ve got the Small Object Heap (SOH) and the Large Object Heap (LOH), split by a hard boundary of 85,000 bytes. Objects smaller than that go to the SOH. Anything equal to or larger lands on the LOH. This isn’t some random cutoff. It’s a deliberate engineering bet—balancing the cost of moving data around against the risk of fragmenting your memory space. If you care about performance, you need to understand that trade-off in your bones.

Generational Collection and the SOH Compaction Guarantee

The SOH gets split into three generations: Gen0, Gen1, and Gen2. New objects start in Gen0. Once Gen0 fills up, the garbage collector kicks off an ephemeral collection. Objects that survive get promoted to Gen1, and if they hang around long enough, they eventually end up in Gen2. The real magic here is compaction. After the GC sweeps away the dead objects, it slides the survivors together. No gaps. A neat, contiguous block of free space sits at the end. Allocation stays fast—usually just a pointer bump—and fragmentation doesn’t get a foothold. But don’t kid yourself. Moving those survivors during compaction chews through CPU cycles. The generational hypothesis bets that most objects die young. Gen0 collections stay cheap because there’s hardly anything to move.

Take a method that creates a List<byte> with a few hundred elements. The internal array and the list itself land in Gen0. If the method finishes fast and the caller tosses the reference, those objects become garbage. They get cleaned up in the next Gen0 collection without ever seeing a promotion. The system hums along because it dodges the overhead of moving long-lived data.

Why 85,000 Bytes Defines a Performance Cliff

The LOH exists for one simple reason: copying big objects during compaction is painfully expensive. Shifting a 2 MB array around in memory burns memory bandwidth and can stall your managed threads long enough to notice. Microsoft’s decision to stick objects ≥ 85,000 bytes on a separate heap side-steps that cost. Instead of compacting, the LOH uses a free-list algorithm. When a large object dies, the GC notes the freed block in a list of available spaces. Future allocations can squeeze into those gaps if they fit. You avoid the copy cost, but fragmentation creeps in. Over time, the LOH turns into a patchwork of free and used blocks. You might have plenty of total free space, but a request for a contiguous 1 MB array fails because no single gap is big enough.

Picture an app that keeps allocating and discarding byte[100000] arrays. Each one hits the LOH. When they become unreachable, a Gen2 collection (which also sweeps the LOH) reclaims the memory. But without compaction, holes riddle the heap. A later allocation for a slightly larger array might not find a contiguous block. The runtime then begs the OS for more virtual memory. After hours of this, your process’s working set balloons even though the number of live objects hasn’t changed.

Allocation Strategy: Choosing the Right Heap for the Job

Performance tuning often means keeping objects off the LOH unless you’ve got no other choice. The first trick is obvious: break large arrays into smaller chunks. Instead of one byte[200000], use a List<byte[]> where each inner array stays comfortably under the 85K mark. Those chunks live on the SOH, getting compaction and generational collection. Indexing gets a little fiddly, but the payoff is better GC pause times and tighter memory density. Same logic applies to strings. Stringing together lots of small strings into one giant string can accidentally push the result onto the LOH. Using StringBuilder properly, or splitting the data into manageable pieces, keeps the pressure on the SOH where it belongs.

When you can’t dodge large buffers—think image processing, serialization, network I/O—pooling becomes your lifeline. ArrayPool<byte>.Shared lets you rent temporary buffers and return them when you’re done. The pool hangs onto a cache of arrays, so you’re not constantly allocating and freeing on the LOH. This cuts down fragmentation because you reuse the same blocks instead of asking for new ones. The catch? You have to be religious about returning rented arrays. Leak a buffer, and you’ve punched a permanent hole in the LOH.

There’s another, more subtle card to play: LOH compaction mode, available since .NET Framework 4.5.1. Set GCSettings.LargeObjectHeapCompactionMode to GCLargeObjectHeapCompactionMode.CompactOnce, and you’re telling the next blocking Gen2 collection to compact the LOH too. Think of it as an emergency lever. Compaction still forces that copying cost the LOH was built to avoid, so use it sparingly—maybe during a maintenance window or after you’ve spotted nasty fragmentation through ETW events.

Diagnosing LOH Fragmentation with Real-World Symptoms

Fragmentation doesn’t wave a red flag. You’ll notice memory usage creeping up, Gen2 collections dragging on, or an OutOfMemoryException getting thrown when the process’s private bytes are nowhere near the 32-bit or 64-bit ceiling. That’s the LOH screaming that it can’t find a single chunk big enough for the next allocation, even though free memory is all over the place. Tools like PerfView and dotMemory show you fragmentation ratios directly. In PerfView, the GCStats report lays out the LOH size and the fragmentation percentage after each collection. A ratio north of 50% after a few Gen2 collections is a blaring sign your allocation pattern needs a rethink.

ETW tracing gets you even closer to the metal. Events like Microsoft-Windows-DotNETRuntime/GC/AllocationTick for large objects tell you exactly which types and sizes are slamming the LOH. Pair those with GC/Triggered events, and you’ll see if LOH allocations are yanking the trigger on premature Gen2 collections. I’ve seen a logging system serialize huge XML or JSON strings onto the LOH thousands of times a minute, forcing Gen2 collections that freeze every managed thread for hundreds of milliseconds. Classic anti-pattern.

Pinning and the LOH: A Double-Edged Sword

LOH objects often get pinned for async I/O. Hand a big byte array to NetworkStream.ReadAsync, and the runtime pins the buffer so the GC can’t move it during native I/O. On the SOH, pinning shreds the heap because the GC can’t compact around a pinned object. The LOH’s free-list algorithm shrugs off pinning a bit better since compaction isn’t happening anyway. But pinning still gums up the works. If a pinned 1 MB array camps in the middle of the LOH for a long-running operation, the spaces before and after it might become too small for new requests. You’re effectively pouring memory down the drain.

You fight this with disciplined lifetime management. Use Memory<byte> and ArrayPool together to shorten how long you pin. Rent a buffer, pin it for the I/O call, and shove it back into the pool the instant the operation finishes. The pool’s internal arrays might still sit on the LOH, but they get reused instead of abandoned. The fragmentation sting gets contained.

When the LOH Is the Right Choice

Don’t get me wrong—the LOH isn’t a villain. It solves a real problem. Big, long-lived data structures like caches, precomputed lookup tables, or machine learning models dodge the generational promotion tax by living there. A 5 MB array that stays alive for the process’s entire lifetime shouldn’t be carved into SOH chunks. Doing that would force hundreds of small objects to slog their way into Gen2, bloating collection times and chewing up memory with object headers. Park it on the LOH, and it stays out of the ephemeral generations entirely. Gen0 and Gen1 collections stay snappy.

The decision boils down to lifetime and volatility. Allocate a large object once and never toss it? Fragmentation on the LOH doesn’t matter. The heap holds one big, happy contiguous block that never needs freeing. But if you allocate and free that object in a loop, chunking on the SOH almost always wins. A memory profiler like dotMemory helps you spot which large objects are truly static and which are just passing through.

FAQ

Q: What types of objects typically end up on the LOH in a web application?

A: Common culprits include large string builders that exceed 85,000 characters, byte arrays for file uploads or image processing, and serialized JSON payloads cached in memory. Diagnostic tools like dotMemory or ETW traces can pinpoint the exact allocation stacks that create these objects.

Q: How can I detect LOH fragmentation without third-party tools?

A: Use the built-in GC.GetGCMemoryInfo() method in .NET Core 3.0 and later. The returned GCMemoryInfo struct includes TotalCommittedBytes, HeapSizeBytes, and FragmentedBytes for each generation, including the LOH. Monitoring these values over time reveals fragmentation trends. You can also enable the gcTrimCommitOnLowMemory setting to let the runtime release fragmented pages back to the OS under memory pressure.

Q: Is it safe to call GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce in production?

A: It can be safe if used during a low-traffic period and on .NET Framework 4.5.1+ or .NET Core 2.0+. The compaction forces a full blocking Gen2 collection with LOH compaction, which pauses all managed threads. The pause duration depends on the LOH size and can be several seconds. Test thoroughly under realistic load before deploying this as an automated remediation strategy.

Understanding the SOH and LOH allocation strategy isn’t about memorizing thresholds. It’s about internalizing the performance model—the cost of copying versus the cost of fragmentation—and making deliberate choices for each allocation in your application’s critical paths.