C# HttpClient cause header or cookie too large issue
Once upon a time, there was a happy service that exchanged useful data between the service and an API endpoint. Suddenly, one day, an evil spirit (X) came, and unexpected things (O) happened. The happy service stopped working but continued to reject requests, always responding with a complaint message like the one below:
A brave engineer then stepped forward to figure out why the happy service was rejecting requests. The engineer initially thought the problem might be with the NGINX side or caused by other issues such as networking or caching. After some time passed, an elite engineer — colleague of the brave engineer — suggested taking a closer look at the following code section:
public partial class XService
{
public async Task<ResultModel> InvokeActionAsync(string jsonData, string path)
{
var payload = new StringContent(jsonData, Encoding.UTF8, MediaTypeNames.Application.Json);
var verifyPath = $"{path}";
var url = new Uri("Https://xxxx/api/" + verifyPath);
var httpClient = _httpClientFactory.CreateClient(Constants.NAME_HTTP_CLIENT_API);
// ----- caution this code section ----- //
httpClient.DefaultRequestHeaders.Add("cache-control", "no-cache");
httpClient.DefaultRequestHeaders.Add("X-API-KEY", "_____Key_____");
using var response = await httpClient.PostAsync(url, payload);
var result = new ResultModel();
if (response.IsSuccessStatusCode)
{
var content = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
result.Status = "Success";
result.Data = content;
result.Message = string.Empty;
}
else
{
result.Status = "Error";
result.Data = string.Empty;
result.Message = $"X Server Status Code {response.StatusCode}.";
}
return result;
}
}The key point was that HttpRequestHeaders.Add always appends headers but does not override existing ones. So, if the Add method is called repeatedly using the same HTTP client factory, the headers or cookies could become too large, potentially causing issues.
Understanding the root cause, they created the extension method shown below:
internal static class HttpHeadersExtension
{
public static HttpRequestHeaders AddAlways(this HttpRequestHeaders headers, string key, string value)
{
if (headers.Contains(key))
headers.Remove(key);
headers.Add(key, value);
return headers;
}
public static HttpRequestHeaders AddIfNotExists(this HttpRequestHeaders headers, string key, string value)
{
if (!headers.Contains(key))
headers.Add(key, value);
return headers;
}
public static HttpRequestHeaders RemoveIfExists(this HttpRequestHeaders headers, string key)
{
if (headers.Contains(key))
headers.Remove(key);
return headers;
}
}The brave engineer then replaced HttpRequestHeaders.Add() with HttpRequestHeader.AddIfNotExists(). Finally, the happy service returned to normal and continued to serve requests happily. :)
