Socket exhaustion is the point where a process can no longer create new outbound or inbound network connections because it has run out of available sockets, ephemeral ports, or file descriptors. In .NET, this usually shows up as SocketException with messages like “Only one usage of each socket address (protocol/network address/port) is normally permitted” or “An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.” Adjacent concepts include ephemeral port range exhaustion, TIME_WAIT accumulation, handle leaks, connection pool starvation, and async socket misuse. For production .NET services handling thousands of requests per second, socket exhaustion is rarely a single smoking gun. It is a slow-burn failure that starts with latency spikes, then intermittent timeouts, then cascading dependency failures. This article is for the engineer who is already seeing those symptoms and needs a methodical path to root cause.
First, Separate Client-Side from Server-Side Exhaustion
Socket exhaustion is not one failure mode. The first diagnostic split is whether the process is failing to accept inbound connections or failing to create outbound connections. Server-side exhaustion usually means the listening socket’s backlog is full or the process has hit its file descriptor limit. Client-side exhaustion usually means the process has consumed the entire ephemeral port range for a given destination IP and port, or it has leaked sockets through undisposed HttpClient instances, raw Socket objects, or broken connection pooling.
In high-throughput .NET services, the most common variant is client-side exhaustion caused by outbound HTTP calls. A service that calls ten downstream APIs per request at 2,000 requests per second is creating 20,000 outbound connections per second at peak. If any of those connections are not returned to the pool, the process burns through the ephemeral port range in minutes.
The Ephemeral Port Range Is a Hard Limit
Windows assigns outbound connections an ephemeral port from a configurable range. The default range on modern Windows Server is roughly 49,152 to 65,535, which gives about 16,384 ports per destination IP and port. Linux uses a different default range, often 32,768 to 60,999, controlled by net.ipv4.ip_local_port_range. When a .NET process opens a socket to a specific remote endpoint, the operating system binds a local ephemeral port. That port cannot be reused for the same remote endpoint until the socket is fully closed and the TIME_WAIT period has elapsed, unless SO_REUSEADDR or connection pooling is in play.
This is not a .NET-specific limit. It is a TCP stack constraint. But .NET applications make it easy to hit because HttpClient, HttpWebRequest, and raw Socket all have different lifetime rules. A single misused HttpClient per request is enough to exhaust the range under load.
Common .NET Causes of Socket Exhaustion
1. HttpClient Created Per Request
The classic failure. Creating a new HttpClient for every outbound call means each instance gets its own connection pool. The pool is discarded when the client is garbage collected, but the underlying sockets are not immediately closed. Under sustained load, the process accumulates sockets in TIME_WAIT or CLOSE_WAIT until the ephemeral port range is empty. The fix is to use a single static or long-lived HttpClient, or better, IHttpClientFactory with named or typed clients.
2. Sockets Not Disposed After Exceptions
Raw Socket and TcpClient usage is still common in high-performance services. If an exception is thrown between Connect and Close, and the code does not use a finally block or using statement, the socket leaks. One leaked socket per failed request is enough to kill a service during a dependency outage, because the failure rate spikes at exactly the moment the service is under the most stress.
3. Connection Pool Starvation
This is not the same as socket exhaustion, but it produces identical symptoms. HttpClient pools connections to the same endpoint. The default limit is ServicePointManager.DefaultConnectionLimit, which is 2 in some legacy .NET Framework configurations and 10 in others. If a service makes 100 concurrent calls to the same downstream API, only 10 connections are available. The other 90 requests queue. The queue grows, timeouts fire, and the service looks like it is out of sockets when it is actually out of pooled connections. In .NET Core and .NET 5+, SocketsHttpHandler.MaxConnectionsPerServer controls this, and the default is int.MaxValue, which is a different problem: unbounded connection growth.
4. Firewall or Load Balancer Idle Timeout Mismatch
A connection pooled by HttpClient can be silently closed by an intermediate network device. The client does not know the connection is dead until it tries to write to it. The write fails, the socket is discarded, and a new one is created. If the idle timeout on the load balancer is shorter than the client’s keep-alive interval, the client is constantly creating new connections. This is not a leak, but it looks like one in the metrics.
Diagnostic Sequence for a Production Incident
When a service starts throwing SocketException under load, do not guess. Run the sequence below in order. Each step either confirms or eliminates a cause.
Step 1: Capture the Exact Exception and Stack Trace
The exception message tells you which side is failing. “Only one usage of each socket address” is client-side ephemeral port exhaustion. “An operation on a socket could not be performed because the system lacked sufficient buffer space” is often a non-paged pool or buffer issue, not a port issue. “No connection could be made because the target machine actively refused it” is a server-side backlog or listener failure. The stack trace tells you which code path is creating the socket. If the stack trace points to HttpClient.SendAsync, the problem is in the outbound call path. If it points to Socket.Accept, the problem is inbound.
Step 2: Check the Process Handle Count
On Windows, use Performance Monitor or Get-Process -Name YourService | Select-Object Handles. On Linux, use ls /proc/<pid>/fd | wc -l. A process that is leaking sockets will show a steadily increasing handle count. A process that is simply hitting the ephemeral port limit may have a stable handle count but a high number of sockets in TIME_WAIT. The distinction matters: handle leak means undisposed objects; TIME_WAIT accumulation means high connection churn.
Step 3: Inspect Socket States
On Windows, netstat -ano | findstr <pid> shows all sockets owned by the process. Count the states. A large number of sockets in CLOSE_WAIT means the remote side closed the connection and the local process never called Close. That is a bug in the .NET code. A large number in TIME_WAIT is normal for high-churn services, but if the count is in the tens of thousands, the service is creating and destroying connections too quickly. On Linux, ss -tanp | grep <pid> gives the same view.
Step 4: Check the Ephemeral Port Range and Current Usage
On Windows, netsh int ipv4 show dynamicport tcp shows the current range. On Linux, cat /proc/sys/net/ipv4/ip_local_port_range. Then compare the number of outbound connections in netstat or ss to the range size. If the count is near the range limit, the service is either leaking sockets or churning connections too fast. Widening the range is a temporary mitigation, not a fix. The fix is to stop the leak or reduce churn.
Step 5: Capture a Dump and Inspect Socket Objects
If the handle count is rising but netstat does not show a matching number of sockets, the leak may be in a different handle type. If the handle count matches the socket count, capture a memory dump with dotnet-dump collect or procdump -ma. Load it in WinDbg or dotnet-dump and run !dumpheap -type System.Net.Sockets.Socket. The number of live Socket objects tells you whether the leak is managed or native. If the managed count is low but the native handle count is high, the leak is in a native library or a P/Invoke path.
Fixing the Root Cause, Not the Symptom
Once the diagnostic sequence identifies the cause, the fix is usually small. The hard part is resisting the urge to widen the ephemeral port range and move on. That buys hours, not days. The real fix is in the code.
Use IHttpClientFactory Correctly
In ASP.NET Core, register IHttpClientFactory and inject it into services. The factory manages handler lifetimes and connection pooling. For typed clients, use AddHttpClient<TClient>(). For ad-hoc calls, use CreateClient() and dispose the client when done. The factory reuses the underlying handler, so disposing the client does not close the pooled connections. This is the single highest-impact change for most .NET services.
Set Explicit Connection Limits
In .NET Core and .NET 5+, configure SocketsHttpHandler.MaxConnectionsPerServer to a value that matches the downstream service’s capacity. If the downstream API can handle 200 concurrent connections, set the limit to 200. This prevents the client from opening unbounded connections during a traffic spike and protects both sides. In .NET Framework, set ServicePointManager.DefaultConnectionLimit early in the process lifetime, before any outbound call is made.
Dispose Sockets in Finally Blocks
For raw socket code, wrap every Connect, Send, and Receive in a try/catch/finally that closes the socket. Better, use using declarations or await using for IAsyncDisposable socket wrappers. A socket that is not closed in an exception path is a leak that will only appear under failure conditions, which is exactly when you cannot afford it.
Align Keep-Alive and Idle Timeouts
If the service sits behind a load balancer or firewall, set the client’s keep-alive interval to less than the network device’s idle timeout. In SocketsHttpHandler, set PooledConnectionIdleTimeout and KeepAlivePingDelay. The goal is for the client to detect dead connections before the network device silently drops them. This reduces the number of failed writes and the resulting connection churn.
Monitoring for Early Warning
Socket exhaustion is predictable if you monitor the right counters. The key metrics are:
- Current outbound connections per process, compared to the ephemeral port range size.
- Socket count by state, especially CLOSE_WAIT and TIME_WAIT.
- Handle count for the process, watched for monotonic increase.
- Connection pool wait time from
HttpClientorSocketsHttpHandlertelemetry. - SocketException rate broken down by message and stack trace.
In .NET, System.Net.Http and System.Net.Sockets emit events through EventSource. The System.Net.Http source includes connection pool metrics. The System.Net.Sockets source includes socket connect and accept events. Wire these into your existing telemetry pipeline. The data is already there; most teams just do not collect it.
When Widening the Port Range Is Acceptable
There are legitimate cases where the ephemeral port range is too small for the workload. A service that makes 50,000 outbound connections per minute to a single destination will exhaust a 16,384-port range even with perfect connection pooling, because the pool cannot reuse a connection that is still in TIME_WAIT. In that case, widening the range on Windows with netsh int ipv4 set dynamicport tcp start=1025 num=64510 or on Linux with sysctl -w net.ipv4.ip_local_port_range="1024 65535" is a reasonable mitigation. But do this only after confirming that the connection churn is inherent to the workload, not caused by a leak or a pooling misconfiguration. Otherwise you are masking the bug.
Case Study: The 3 AM Pager Storm
A .NET 6 service handling payment callbacks started throwing SocketException every night at 3 AM. The on-call engineer restarted the service, and the errors stopped for 24 hours. The pattern repeated for a week. The team assumed a nightly batch job was overloading the service. The real cause was a scheduled database maintenance window that closed all idle connections to the downstream payment API. The service’s HttpClient pool held connections that the database server had closed. The next wave of requests tried to write to dead connections, failed, and created new ones. The connection churn spiked, the ephemeral port range filled, and the service started throwing SocketException. The fix was to set PooledConnectionIdleTimeout to 5 minutes and KeepAlivePingDelay to 1 minute. The service now detects dead connections before the write fails, and the 3 AM pager storm stopped.
FAQ
What is the difference between socket exhaustion and connection pool starvation?
Socket exhaustion means the operating system cannot allocate a new socket because the process has used all available ephemeral ports or file descriptors. Connection pool starvation means the HttpClient or SocketsHttpHandler has reached its configured maximum connections per server and is queueing requests. Both produce timeouts and SocketException, but the fixes are different. Pool starvation is fixed by raising the connection limit or reducing concurrency. Socket exhaustion is fixed by eliminating leaks or reducing connection churn.
How do I know if my .NET service is leaking sockets?
Watch the process handle count over time. If it increases monotonically under constant load and never drops, you likely have a leak. Then check netstat -ano | findstr <pid> on Windows or ss -tanp | grep <pid> on Linux. If the number of sockets in CLOSE_WAIT is high and growing, the remote side closed the connection and your code never called Close. That is a managed code bug. If the socket count is stable but the handle count is rising, the leak is in a different handle type.
Can I just increase the ephemeral port range to fix socket exhaustion?
You can, but it is a mitigation, not a fix. If the service is leaking sockets, a wider range only delays the failure. If the service is churning connections too fast, a wider range may hide the problem until traffic grows again. The correct sequence is to diagnose the cause, fix the leak or pooling issue, and then decide whether the workload genuinely needs a wider range. In high-throughput services, a wider range is sometimes necessary even with perfect code, but it should be the last step, not the first.
What is the best way to prevent socket exhaustion in .NET Core and .NET 5+?
Use IHttpClientFactory for all outbound HTTP calls. Configure SocketsHttpHandler.MaxConnectionsPerServer to a value that matches the downstream service’s capacity. Set PooledConnectionIdleTimeout and KeepAlivePingDelay to detect dead connections before writes fail. Dispose raw sockets in finally blocks or with using declarations. Monitor socket count by state and handle count per process. These practices eliminate the vast majority of socket exhaustion incidents in production .NET services.
Next up on this site: a deep look at CLOSE_WAIT vs TIME_WAIT in .NET services, including how to read netstat output without guessing and what each state tells you about the code that owns the socket.