When a production .NET 6+ service stops responding, leaks memory, or spikes CPU, the first question is not “what changed?” It is “what evidence do we have?” Two tools produce that evidence without a debugger attached: dotnet-dump and dotnet-gcdump. They are part of the .NET diagnostics CLI family, alongside dotnet-trace, dotnet-counters, and dotnet-stack. This article is for engineers who must collect a dump from a Linux container, inspect managed heaps, and decide whether the problem is a leak, a pin, a finalizer queue, or a workload pattern. It assumes you already know that a memory dump is not a profiler and that a GC dump is not a heap dump.
The audience here runs .NET 6+ on Windows, Linux containers, Kubernetes, Azure App Service Linux, and serverless runtimes. The constraints are the same across those environments: no Visual Studio on the box, no interactive debugger, and often no shell history after the pod dies. The workflow that matters is capture, transfer, analyze, and preserve. Everything else is noise.
What dotnet-dump and dotnet-gcdump Actually Capture
dotnet-dump collects a full process dump: managed heap, native heap, thread stacks, module list, and environment. On Linux it uses the runtime’s diagnostics IPC channel, not gcore. On Windows it can produce a minidump or a full dump. The file is not portable across OS families by default; a dump taken on Linux must be analyzed on Linux or with a tool that understands Linux dumps. dotnet-gcdump collects a compact, GC-focused snapshot: object graph, roots, finalization queue, and GC heap segments. It is not a full memory dump. It is smaller, faster, and often enough for managed memory investigations.
Adjacent concepts you will meet immediately: EEHeap, GCHeap, LOH, POH, finalization queue, freachable queue, pinned objects, generation 2, memory pressure, and workstation vs server GC. If those terms are not familiar, the dump will not explain them to you. The tooling assumes you know what a root is and why a pinned byte[] on the LOH behaves differently from a short-lived string.
Installing the Tools Without Breaking the Box
Install as a global tool, not as a project dependency:
dotnet tool install --global dotnet-dump
dotnet tool install --global dotnet-gcdumpOn a locked-down production image, install into a sidecar or a diagnostics pod. Do not modify the running application image to add diagnostic tools. On Kubernetes, run a one-shot pod in the same namespace with the same UID and the same /tmp and /diag volume mounts. On Azure App Service Linux, use the SSH console and install into /tmp or a persistent storage mount. On serverless, you usually cannot attach; you must rely on startup hooks or pre-instrumented images.
Version alignment matters. The tool version must be compatible with the runtime version of the target process. A dotnet-dump built for .NET 8 can usually inspect a .NET 6 process, but the reverse is not guaranteed. Check the tool’s --version output and the target process’s runtime version before capture.
Capturing a Full Dump with dotnet-dump
Find the process ID first. On Linux, pgrep -f MyApp or ps aux | grep dotnet. On Windows, tasklist | findstr MyApp. Then:
dotnet-dump collect -p 1234 -o /diag/app_20250321_1410.dmpAdd --type Full on Windows if you need native memory. On Linux the default is a full dump. The process pauses briefly during collection. For a large heap, that pause can be seconds. Do not collect during a rolling deployment or while a health probe is about to fail. If the process is already unresponsive, the pause is irrelevant; collect immediately.
For a crashing process, use dotnet-dump collect --crashreport or configure DOTNET_DbgEnableMiniDump and DOTNET_DbgMiniDumpType to capture automatically on unhandled exceptions. That is the only reliable way to get a dump from a process that dies before you can attach.
Capturing a GC Dump with dotnet-gcdump
dotnet-gcdump collect -p 1234 -o /diag/app_20250321_1410.gcdumpThe GC dump is the better first move for managed memory issues. It is smaller by an order of magnitude, it does not require the same pause, and it can be analyzed in PerfView, Visual Studio, or dotnet-gcdump report. If the GC dump does not answer the question, then take a full dump. That order saves time and disk.
One common mistake: taking a GC dump after the process has already been restarted. The GC dump is a point-in-time snapshot. If the pod restarted, the evidence is gone. Automate capture on memory thresholds, not after the incident.
Analyzing a Full Dump with dotnet-dump analyze
Open the dump:
dotnet-dump analyze /diag/app_20250321_1410.dmpThe first commands are always the same:
clrstack
pe
dumpheap -stat
clrstack shows the managed stack of the current thread. pe prints the current exception, if any. dumpheap -stat gives a type-by-type object count and total size. That output is the triage. Look for the type with the largest total size that should not be there. A System.String at 40% of the heap is normal for a logging-heavy service. A MyApp.Cache.CacheEntry at 40% is a finding.
Then narrow the heap:
dumpheap -type MyApp.Cache.CacheEntry
Pick an address and trace roots:
gcroot 00007f8b1c00a1e0
If the root chain ends in a static field, a timer, or an event handler, you have the leak. If it ends in a pinned byte[], you have a pinning problem, not a managed leak. If it ends in the finalization queue, you have a finalizer that is not completing.
For large object heap issues:
dumpheap -stat -type System.Byte[]
dumpheap -type System.Byte[] -min 85000
The LOH threshold is 85,000 bytes. Objects above that are not compacted by default. Repeated LOH allocations and frees fragment the heap. The dump shows the fragmentation, not the cause. The cause is in the allocation pattern, which a GC dump or a trace can reveal.
Analyzing a GC Dump
Open the GC dump in PerfView or Visual Studio. The key views are Heap Stacks, GC Heap Alloc Ignore Free, and Finalizable Objects. Heap Stacks shows the allocation call stacks for the objects still on the heap. That is the direct answer to “what allocated this?” GC Heap Alloc Ignore Free shows the same for all allocations, including freed ones. Finalizable Objects shows objects waiting for finalization.
If the GC dump shows a single call stack responsible for 80% of the heap, the fix is in that code path. If it shows a flat distribution across many call stacks, the problem is likely a cache, a static collection, or a retention policy. The GC dump does not tell you which objects are reachable from where; it tells you what was allocated and by whom. For reachability, you need the full dump and gcroot.
Linux Container Specifics
On Linux, the diagnostics IPC channel is a Unix domain socket in /tmp. If the container runs as non-root and /tmp is not writable, the tools cannot connect. Set DOTNET_DiagnosticPorts or mount a writable /tmp. The default socket name includes the process ID and a random component. If you run the tool in a sidecar, it must share the same /tmp volume.
Memory limits matter. A container with a 512 MB limit and a 400 MB heap will produce a dump larger than the limit. Write the dump to a mounted volume, not the container’s writable layer. If the dump fills the layer, the kubelet will evict the pod and you lose the evidence.
For Kubernetes, the cleanest pattern is a diagnostics DaemonSet or a one-shot Job with hostPID: true and the same /tmp mount. The Job runs dotnet-dump collect against the target PID, writes to a PVC, and exits. The alternative is kubectl exec into the pod, but that requires the tool to be present in the image.
Windows and Azure App Service Linux
On Windows, run the tool from an elevated prompt. The process must be running under the same user or a user with debug privileges. For IIS-hosted apps, the process is w3wp.exe; identify the right instance by the application pool name. For Windows services, the process is the service executable.
On Azure App Service Linux, the SSH console gives you a shell. Install the tools into /tmp, find the dotnet process, and collect. The dump must be downloaded via FTP or the Kudu API before the instance is recycled. App Service recycles instances on configuration changes, scale operations, and platform maintenance. Do not assume the dump will be there tomorrow.
Common Failure Patterns and What They Look Like
Event Handler Leak
The heap is dominated by a type that should be short-lived. gcroot shows a chain ending in a static event. The fix is to unsubscribe or use a weak event pattern. The dump proves the root; it does not fix the code.
Pinned Object Fragmentation
The LOH is fragmented. dumpheap -stat shows many byte[] objects above 85,000 bytes. gcroot shows them pinned by a GCHandle or an async operation. The fix is to pool buffers or use ArrayPool<byte>. The dump shows the pinning; the allocation trace shows the source.
Finalizer Queue Backup
The finalization queue is long. dumpheap -type System.Object with the finalization queue root shows objects waiting. The fix is to implement IDisposable correctly and call Dispose, or to remove the finalizer if it is not needed. The dump shows the queue; the code review shows the missing using.
Large Object Heap Growth Without Leak
The LOH grows but the object count is stable. This is fragmentation, not a leak. The fix is to reduce LOH allocations or enable GCSettings.LargeObjectHeapCompactionMode for a one-time compaction. The dump shows the fragmentation; the GC settings show the tradeoff.
When Not to Use These Tools
If the process is CPU-bound, use dotnet-trace or dotnet-counters first. A dump is a point-in-time snapshot; it may catch the CPU spike or it may not. If the process is leaking slowly, a single dump may not show the trend. Take two dumps an hour apart and compare dumpheap -stat output. If the process is crashing on startup, use DOTNET_DbgEnableMiniDump and collect the crash dump automatically. If the problem is a deadlock, a dump is the right tool; clrstack on all threads shows the wait chain.
Preserving Evidence
Dumps are evidence. Store them with the incident record. Include the runtime version, the OS version, the container image digest, the environment variables, and the exact command used to collect. A dump without that context is a file, not evidence. If the dump contains sensitive data, treat it as sensitive. Do not upload it to a public issue tracker. Do not email it. Use a private storage account with access logging.
FAQ
What is the difference between dotnet-dump and dotnet-gcdump?
dotnet-dump captures a full process dump including native memory, thread stacks, and the managed heap. dotnet-gcdump captures only the GC-relevant data: object graph, roots, finalization queue, and heap segments. The GC dump is smaller and faster, but it cannot answer questions about native memory, thread stacks, or module state.
Can I analyze a Linux dump on Windows?
Not reliably. A dump taken on Linux must be analyzed on Linux or with a tool that explicitly supports Linux dumps. The reverse is also true. If you must analyze on a different OS, use a GC dump, which is platform-neutral, or collect the dump on the same OS family as the analysis machine.
How do I collect a dump from a crashing process?
Set DOTNET_DbgEnableMiniDump=1 and DOTNET_DbgMiniDumpType=4 in the process environment. On an unhandled exception, the runtime writes a dump to the current directory or the path in DOTNET_DbgMiniDumpName. This is the only reliable method for processes that die before you can attach.
Why does my dump show a huge System.String count?
Strings are the most common managed object in a logging-heavy service. A high string count is not automatically a leak. Compare the string count and total size across two dumps taken an hour apart. If the count grows without bound, trace the roots. If it is stable, it is a workload pattern, not a leak.
Next Step for This Site
This article is the first in a series on production .NET diagnostics. The next article will cover dotnet-trace and dotnet-counters for CPU and thread pool investigations. If you have a dump that resists analysis, send the dumpheap -stat output and the gcroot chain for the top type. I will use it as a case study in a future post.