Handling Azure Function (Host in Linux) IHttpClientFactory request cancel issue

Photo from: https://learn.microsoft.com/en-us/azure/architecture/hybrid/azure-functions-hybrid

We have a Azure Function that host in the platform Linux, and the function using hybrid connections that connection to on premise agents under the intranet of firewall.

We got suddenly request cancel problems (not often but sometimes) for firing request from cloud to local APIs. See below:

My colleague has been contact for IT support (from official) for the problem and got feedback that under the hybrid connections structure the IHttpClientFactory might reuse the connection that suppose to be closed but not aware due to if the service bus idled around 1 minute then bus will cut down the connection directly (TCP protocol within default TIME_WAIT: 2 minutes).

As per feedback that we are thinking if we could closed the connection as soon as possible or establish new connection per request might solve the problem for us? So here were the two methods we could try to apply:

• Manage the lifetime of HttpMessageHandler as shorter as possible, usually must smaller than the service bus idle time (1 minute). So we could reduce the risk that IHttpClientFactory reuse the connection that will dispose in the nearly future.

services.AddHttpClient<IXXXXXService, XXXXXService>()
    .SetHandlerLifetime(TimeSpan.FromMinutes(1));

• Control the HttpClient itself and setup the PooledConnectionLifetime and PooledConnectionIdleTimeout to make sure available connection from the pool before every time service bus drop the connection.

public class HttpClientHelper
{
  private static readonly Lazy<HttpClient> lazy = new(
  () =>
  {
   var handler = new SocketsHttpHandler()
   {
    AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate,
    PooledConnectionLifetime = TimeSpan.FromSeconds(60),
    PooledConnectionIdleTimeout = TimeSpan.FromSeconds(10)
   };
   var httpClient = new HttpClient(handler);
   return httpClient;
  });
  public static HttpClient Instance { get { return lazy.Value; } }
}

Both methods were workaround since they were tried to reduce reuse ability of HttpClient instance. As we know that create or new instance(s) might increase the cost of each transaction.

Reference

• https://stackoverflow.com/questions/71240555/using-ihttpclientfactory-to-immediately-close-connections
• https://www.siakabaro.com/how-to-manage-httpclient-connections-in-net/
• https://blog.vjirovsky.cz/azure-functions-and-forbidden-socket-exception/
• https://learn.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests