When a production .NET app goes sideways for reasons that make no sense, the gap between blind trial-and-error and a solid fix usually comes down to how well you know Windbg. Plenty of developers treat the debugger as a last resort—something you fire up after logs and dashboards come up empty. But Windbg has a handful of advanced commands that can surface threadpool starvation, finalizer bottlenecks, and quiet memory corruption that Visual Studio’s managed debugger just glosses over. Here are the commands I actually reach for when the problem stops being a simple null reference.
Setting the Stage: Symbols, Extensions, and the Debugger Engine
Before you type a single diagnostic command, you need a debugger environment that won’t lie to you. Windbg depends on symbols—for Microsoft’s binaries and your own assemblies alike. Point your symbol path at the public Microsoft server with a local cache. The command .symfix+ c:\symbols handles that in one shot. For .NET work, load the SOS extension: .loadby sos coreclr if you’re on .NET Core, or .loadby sos clr for .NET Framework. Skip this step and managed commands will just fail silently, which confuses the hell out of people.
You also need to know what mode the debugger is in. Windbg can attach to a live process, chew on a crash dump, or do a non-invasive inspection. Each mode changes which commands actually work. If you’re hunting memory corruption, a full dump is non-negotiable—minidumps don’t carry the heap data that !dumpheap needs. I keep a scratch file with the exact bitness and .NET version of the target, because a mismatched SOS version will throw errors that make you doubt your own sanity.
Command 1: !dumpheap -stat and the Art of Memory Triage
!dumpheap -stat is usually the first thing I run on a memory-pressure dump. It sums up the managed heap by type—object count and total size. That output isn’t a leak diagnosis on its own; it’s a triage snapshot. You have to read it with context. Seeing System.String at the top with plausible numbers? That’s normal in most apps. But if a custom type like MyApp.Caching.UnboundedCacheEntry shows up with millions of instances, you’ve probably found your smoking gun.
One gotcha: !dumpheap -stat includes objects that are live and objects that are waiting for finalization. So if a type looks way too heavy, follow up with !finalizequeue to see whether those instances are stuck behind a blocked finalizer thread. I’ve lost count of how many times that two-step combo showed me a disposable type that wasn’t getting cleaned up, with the finalizer thread stalled and preventing the GC from reclaiming anything.
Command 2: !clrstack and the Native-Managed Boundary
If a thread looks stuck, !clrstack gives you the managed call stack. But it won’t show native frames, and those are frequently the real troublemakers. I’ve made it a habit to run !dumpstack right after !clrstack so I can see the full native-plus-managed picture. This matters a lot when you’re staring down a ThreadAbortException or an OutOfMemoryException in a multi-threaded mess. A pattern I see often: the managed stack says the thread is blocked on a WaitHandle, but the native stack reveals it’s sitting inside CoWaitForMultipleHandles—an STA re-entrancy problem. Without that native context, you’d waste time blaming a lock contention that isn’t there.
It’s also a lifesaver for async hangs. The managed stack for an async method might end at System.Threading.Tasks.Task.Wait, but !dumpstack can expose the underlying SyncBlock that never got signaled. Combine that with !syncblk and you can figure out which thread owns the lock and why it can’t let go.
Command 3: !analyze -v and the Exception Context
Plenty of developers run !analyze -v on a crash dump and just accept the first exception it spits out. The command is useful, but its value hinges on the dump’s integrity and whether the debugger can reconstruct the faulting context. For managed crashes, !analyze -v will often point at something like clr!SlowAllocateString or coreclr!AllocateObject as the faulting frame. That only tells you an allocation failed. The real question is what ate the memory. I grab the exception record and register state from !analyze -v, then manually inspect the managed objects those registers reference with !do (dump object).
Say you’re dealing with an AccessViolationException. !analyze -v shows the instruction that tried an invalid read. Check that address with !address <address> and see if it falls inside a managed heap segment. That tells you whether the corruption is GC-related or a pure native interop bug. You need that level of detail when the stack trace alone is leading you in circles.
Command 4: !pe and the Exception Object
!pe (print exception) is the most direct way to inspect a managed exception object from a dump. If you’ve got the exception’s address—from !dumpstack -ee or from !threads output—!pe <address> prints the message, stack trace, and inner exceptions. Two things trip people up here. One, the printed stack trace is the one captured when the exception was thrown; it might not match the thread’s current state. Two, if the exception was created but never thrown (a pattern I see in logging libraries all the time), the stack trace field is null, and !pe shows an empty trace. That’s confused many an engineer.
I often pair !pe with !dumpobj <address> to dig into custom properties on the exception. An AggregateException holds a list of inner exceptions. !pe will list them, but !dumpobj lets you walk the actual List<Exception> array and find a specific one that the basic command might not surface.
Command 5: !threadpool and Starvation Detection
Threadpool starvation is notoriously hard to spot from logs alone. !threadpool shows the current state of the .NET threadpool: worker threads, completion port threads, min and max settings, and the count of pending work items. In a healthy process, pending items are low and active threads sit well below the max. If you see a pile of pending items and a thread count stuck at the minimum, the threadpool isn’t injecting threads fast enough—classic starvation.
On .NET Core, you’ll see extra details like the hill-climbing algorithm’s current target. When I run into a high-throughput process that won’t scale threads, I check whether the app code set the minimum thread count too low with ThreadPool.SetMinThreads. A lot of teams ship with values that look fine under test loads but fall flat under real traffic. Windbg makes that misconfiguration obvious in seconds.
Command 6: !bpmd and Just-in-Time Breakpoints
Not all debugging is post-mortem. When you can attach Windbg to a live process, !bpmd (breakpoint on managed method) is pure gold. It sets a breakpoint on a managed method without needing the JIT-compiled address in advance. The syntax !bpmd MyAssembly.dll MyNamespace.MyClass.MyMethod plants a pending breakpoint that springs into action once the method is JIT-compiled.
This is a handy trick for tracing intermittent issues in environments where you can’t recompile with diagnostic code. I once used !bpmd to break every time a third-party library’s Dispose method was called. It confirmed that Dispose was being hit multiple times on the same object because of a race condition in the calling code. Without that breakpoint, I’d have been stuck instrumenting IL or chasing unreliable log lines.
Putting It Together: A Diagnostic Workflow
These commands aren’t a bag of random tricks; they fit into a workflow. When a memory dump from a production outage lands in my lap, I follow a set sequence. First, !analyze -v to snag the immediate exception context. Second, !dumpheap -stat to scan for obvious memory oddities. Third, !threads and !clrstack to see where threads are blocked. Fourth, !syncblk and !dumpstack on the suspicious ones. Finally, I drill into specific objects with !do and !pe to nail down the hypothesis. A structured approach keeps you out of the “random command syndrome” that burns time and leads to bad conclusions.
Windbg is a sharp tool. It will let you inspect bogus addresses or misinterpret corrupted data without a second thought. Always cross-check what you find. If !do says an object is some type, verify with !dumpmt on its method table. If a stack trace looks impossible, check for stack corruption with !k and compare it to the managed view. A healthy dose of skepticism is your best asset in the debugger.
FAQ
When should I use !analyze -v versus manual stack inspection?
Start with !analyze -v for any crash dump. It automates pulling out the exception record and faulting thread. But if the dump is truncated or the exception chain has custom inner exceptions that the automated analysis gets wrong, switch to manual inspection with !threads, !pe, and !clrstack on the relevant threads. The automated command can steer you wrong when the final exception is just a symptom, not the root cause.
Why does !dumpheap -stat show high memory usage for types I don’t recognize?
Weird types in the heap often come from dynamically generated assemblies—stuff from System.Reflection.Emit, Entity Framework query compilation, or serializers like Newtonsoft.Json. Use !dumpheap -type <partial type name> to look at an instance, then !gcroot to see what’s keeping it alive. These types usually live in caches with no size limits, so they grow without bound over time.
How can I detect a blocked finalizer thread without Windbg?
It’s tough to spot a blocked finalizer thread without Windbg because the symptoms—climbing memory, an unresponsive process—are pretty generic. Performance counters for “Finalization Survivors” and “Promoted Finalization-Memory” can hint at trouble, but they won’t point to the blocking code. Windbg’s !finalizequeue and !threads commands show the finalizer thread’s state and the objects stuck waiting, which is far more precise.
What’s the difference between !clrstack and !dumpstack in a deadlock scenario?
!clrstack shows only managed frames, which might end at a Monitor.Enter or WaitHandle.WaitOne call. !dumpstack gives you the full native and managed stack, exposing the underlying synchronization primitives and any native interop layers. In a deadlock, !dumpstack can reveal that a thread is waiting on a CRITICAL_SECTION rather than a managed monitor, and that changes the whole diagnostic path.