Welcome to today’s post.
In today’s post I will discuss what rate limiting is and how it is used within an ASP.NET 7.0 Core Web API.
The term rate limiting is often confused with rate throttling. They mean different things when dealing with API service requests. Throttling is where we slow down an already high frequency (or spike) of API requests in a short duration down to a lower frequency of requests. Rate limiting is where we set a limit on the number of API calls.
In the first section, I will explain the main reason for using the Rate Limiter API.
Reasons for using the Rate Limiter API
The feature of rate limiting solves one of the important problems that is encountered when we host a Web API service. The problem that we encounter is the allocation of resources when concurrent requests call endpoints within our API service. In many Web API methods that access server resources there is a limitation on how many requests can concurrently access a particular resource on the server. This limitation is governed by server limits on resources including CPU, disk I/O, disk space, network connections and so on.
For example, within a Web API method that executes a request to select data from a SQL database that is hosted on a server. Now imaging if there were many concurrent HTTP GET requests submitted to this Web API. Depending on the resources of the database server, which are dependent on read/write access to different objects, there will be a limitation on the number of select query requests on a specific SQL table that can be processed efficiently. Even shared reads on a database resource are constrained by server disk I/O, which can cause the Web API service method to hang.
Or the above reasons, we make use of the rate limiter to constrain concurrent requests to API endpoints to acceptable levels that do not overwhelm backend server resources.
In the next section, I will explain the important methods used within the Rate Limiter API.
Explaining the Rate Limiter API Library
There are four implementations of the RateLimiter abstract class, which is which is defined within the System.Threading.RateLimiting namespace. One of these, which is quite useful is the ConcurrencyLimiter class. There are three key properties that define concurrent rate limiting within the ConcurrencyLimiter class, and these are defined within the ConcurrencyLimiterOptions class, which is a constructor parameter of the ConcurrencyLimiter class.
These properties are:
PermitLimit
The permit limit is the maximum number of requests that are running concurrently against the API resource.
QueueLimit
The queue limit is the maximum number of requests that are queued concurrently when the maximum number of requests is still running.
QueueProcessingOrder
The queue processing order determines the priority order in which requests in the queue are processed. This is basically in either a first in first out order (QueueProcessingOrder.OldestFirst) or a last in first out order (QueueProcessingOrder.NewestFirst).
In the next section, I will explain how we setup the Rate Limiter service within a .NET Core Web API application.
Setup of the Rate Limiter Service within a .NET (Core) Web API application
To use the above API in a Web API application, we will first need to add the namespace Microsoft.AspNetCore.RateLimiting. You will also need to have .NET 7.0 Framework, and this means also having Visual Studio 2022.
Below is an excerpt of the ConfigureServices() startup method and the configuration of two of the rate limiter services, ConcurrencyLImiter and FixedWindowLImiter. This requires adding the rate limiter service collection using AddRateLimiter(). I have also configured some policies for each of the rate limiter services.
using Microsoft.AspNetCore.RateLimiting;
…
public void ConfigureServices(IServiceCollection services)
{
…
int fixedTimeWindow = Convert.ToInt32(
Configuration.GetSection("AppSettings:TimeWindow").Value
);
// Configure the Rate Limiter service
services.AddRateLimiter(opts =>
{
opts.AddConcurrencyLimiter(policyName: "concurrent-get",
options =>
{
options.PermitLimit = 2;
options.QueueProcessingOrder =
System.Threading.RateLimiting.QueueProcessingOrder.OldestFirst;
options.QueueLimit = 2;
})
.AddFixedWindowLimiter(policyName: "fixed-get", options =>
{
options.PermitLimit = 1;
options.Window = TimeSpan.FromMilliseconds(fixedTimeWindow);
options.QueueProcessingOrder =
System.Threading.RateLimiting.QueueProcessingOrder.OldestFirst;
options.QueueLimit = 0;
})
.AddFixedWindowLimiter(policyName: "fixed2-get", options =>
{
options.PermitLimit = 2;
options.Window = TimeSpan.FromMilliseconds(fixedTimeWindow);
options.QueueProcessingOrder =
System.Threading.RateLimiting.QueueProcessingOrder.OldestFirst;
options.QueueLimit = 0;
})
.OnRejected = (context, cancellationToken) =>
{
if (context.Lease.TryGetMetadata(MetadataName.RetryAfter,
out var retryAfter))
{
context.HttpContext.Response.Headers.RetryAfter =
((int)retryAfter.TotalSeconds)
.ToString(NumberFormatInfo.InvariantInfo);
}
context.HttpContext.Response.StatusCode =
StatusCodes.Status429TooManyRequests;
context.HttpContext.RequestServices
.GetService<ILoggerFactory>()?
.CreateLogger(
"Microsoft.AspNetCore.RateLimitingMiddleware"
)
.LogWarning("OnRejected: {GetUserEndPoint}",
GetUserEndPoint(context.HttpContext));
return new ValueTask();
};
});
…
}
In addition, to enable the rate limiter middleware you will need to add the routing and rate limiter middleware to the application pipeline using UseRouting() and UseRateLimiter() within the Configure() method as shown:
public void Configure(IApplicationBuilder app, IHostEnvironment env,
ILogger<Startup> logger)
{
…
// Enable Routing middleware
app.UseRouting();
// Enable Rate Limiter middleware.
app.UseRateLimiter();
…
…
app.UseEndpoints(endpoints =>
{
…
});
}
Note: The Rate Limit middleware setup call UseRateLimiter() must be made AFTER the call to UseRouting() to enable routing middleware. If the rate limiter middleware is setup before routing, then any rate limiting policies will be overwritten in the application builder pipeline.
I will now explain how to apply the above policies into your API controller classes.
Configuring Rate Limiter Policies in API Controllers
Configuring rate limiting policies to API controller classes and methods is done by declaring attributes at the controller class level or at the API method level.
There are two attributes that are used to enable or disable rate limiting policies in a controller and these are [EnableRateLimiting] and [DisableRateLimiting].
The declarative configuration of rate limiter policies is quite straightforward. Below is an example of how to apply one of the rate limiter fixed window policies to the class. When the attribute is applied at the class level, the policies are applied to all methods in the API controller unless we have a policy attribute that overrides the class attribute. The policy attributes can either enable or disable rate limiter policies.
namespace BookLoan.Controllers
{
[EnableRateLimiting("fixed-get")]
public class BookController : Controller
{
…
[HttpGet("api/[controller]/AllBooks")]
public async Task<List<BookViewModel>> AllBooks()
{
…
}
[HttpGet("api/[controller]/Details/{id}")]
public async Task<ActionResult> Details(int id)
{
…
}
…
}
}
Recall the ConfigureServices() method earlier when we configured the RateLimiter policies. We had a delegate method OnRejected() in the HTTP pipeline. When an API request is permitted to run within the API controller or method within the permitted lease limit, the HTTP response will be 200 (OK).
.OnRejected = (context, cancellationToken) =>
{
if (context.Lease.TryGetMetadata(MetadataName.RetryAfter,
out var retryAfter))
{
context.HttpContext.Response.Headers.RetryAfter =
((int)retryAfter.TotalSeconds)
.ToString(NumberFormatInfo.InvariantInfo);
}
context.HttpContext.Response.StatusCode =
StatusCodes.Status429TooManyRequests;
context.HttpContext.RequestServices
.GetService<ILoggerFactory>()?
.CreateLogger(
"Microsoft.AspNetCore.RateLimitingMiddleware"
)
.LogWarning("OnRejected: {GetUserEndPoint}",
GetUserEndPoint(context.HttpContext));
return new ValueTask();
};
When the API request exceeds the permitted lease limit, the rate limiter service will reject the request and return a 429 status (Rejected). Following rejection, the request is retried after time (equal to the permit window duration). The error is then logged.
In the next section, I will run some sample API requests to test the fixed window rate limiter.
Sample Testing Runs of the Fixed Window Rate Limiter
In the FixedWindowRateLimiter policy, I have configured the duration Window in milliseconds, TimeWindow from the appSettings file.
When rate limiting is not applied or is disabled, and then I run the following CURL command:
curl http://localhost:25138/api/Book/AllBooks & curl http://localhost:25138/api/Book/AllBooks & curl http://localhost:25138/api/Book/AllBooks & curl http://localhost:25138/api/Book/AllBooks & curl http://localhost:25138/api/Book/AllBooks
I get the following five responses with 200 (OK) status:
Request started: "GET" http://localhost:25138/api/Book/AllBooks
Response sent: http://localhost:25138/api/Book/AllBooks with HTTP status 200.0
[00:04:45 INF] HTTP GET /api/Book/AllBooks responded 200 in 24.1731 ms
Request started: "GET" http://localhost:25138/api/Book/AllBooks
Response sent: http://localhost:25138/api/Book/AllBooks with HTTP status 200.0
[00:04:46 INF] HTTP GET /api/Book/AllBooks responded 200 in 13.1740 ms
Request started: "GET" http://localhost:25138/api/Book/AllBooks
Response sent: http://localhost:25138/api/Book/AllBooks with HTTP status 200.0
[00:04:47 INF] HTTP GET /api/Book/AllBooks responded 200 in 68.2634 ms
Request started: "GET" http://localhost:25138/api/Book/AllBooks
Response sent: http://localhost:25138/api/Book/AllBooks with HTTP status 200.0
[00:04:47 INF] HTTP GET /api/Book/AllBooks responded 200 in 62.7656 ms
Request started: "GET" http://localhost:25138/api/Book/AllBooks
Response sent: http://localhost:25138/api/Book/AllBooks with HTTP status 200.0
[00:04:48 INF] HTTP GET /api/Book/AllBooks responded 200 in 29.0487 ms
After applying the fixed window rate limiter policy “fixed-get” at the controller class level as shown earlier, I run the API and post the following CURL command:
curl http://localhost:25138/api/Book/Details/1 & curl http://localhost:25138/api/Book/Details/2 & curl http://localhost:25138/api/Book/Details/3 & curl http://localhost:25138/api/Book/Details/4 & curl http://localhost:25138/api/Book/Details/5
I get the following responses to the file and console logs:
Request started: "GET" http://localhost:25138/api/Book/Details/1
Response sent: http://localhost:25138/api/Book/Details/1 with HTTP status 200.0
[01:55:23 INF] HTTP GET /api/Book/Details/1 responded 200 in 2108.8117 ms
Response sent: http://localhost:25138/api/Book/Details/1 with HTTP status 200.0
Request started: "GET" http://localhost:25138/api/Book/Details/2
[01:56:04 WRN] OnRejected: User Anonymous endpoint:/api/Book/Details/2 127.0.0.1
[01:56:04 INF] HTTP GET /api/Book/Details/2 responded 429 in 564.6747 ms
Response sent: http://localhost:25138/api/Book/Details/2 with HTTP status 429.0
Request started: "GET" http://localhost:25138/api/Book/Details/3
Response sent: http://localhost:25138/api/Book/Details/3 with HTTP status 200.0
[01:56:04 INF] HTTP GET /api/Book/Details/3 responded 200 in 140.7482 ms
Request started: "GET" http://localhost:25138/api/Book/Details/4
[01:56:07 WRN] OnRejected: User Anonymous endpoint:/api/Book/Details/4 127.0.0.1
[01:56:07 INF] HTTP GET /api/Book/Details/4 responded 429 in 1870.9649 ms
Response sent: http://localhost:25138/api/Book/Details/4 with HTTP status 429.0
Request started: "GET" http://localhost:25138/api/Book/Details/5
[01:56:07 INF] HTTP GET /api/Book/Details/5 responded 200 in 19.7255 ms
Notice that the permit limit was set to 1 and allowed one request for every 5000 millisecond (5 second) window. The above log showed the first, third and fifth requests were allowed, but the second and fourth were rejected as they exceeded the window duration bounds.
When we configure the API controllers to use the “fixed2-get” fixed window policy, which has the permit limit set to 2, then run the following command:
curl http://localhost:25138/api/Book/AllBooks & curl http://localhost:25138/api/Book/AllBooks & curl http://localhost:25138/api/Book/AllBooks
The rate limiter logs show the output:
Request started: "GET" http://localhost:25138/api/Book/AllBooks
Response sent: http://localhost:25138/api/Book/AllBooks with HTTP status 200.0
[02:35:40 INF] HTTP GET /api/Book/AllBooks responded 200 in 810.1158 ms
Response sent: http://localhost:25138/api/Book/AllBooks with HTTP status 200.0
Request started: "GET" http://localhost:25138/api/Book/AllBooks
Response sent: http://localhost:25138/api/Book/AllBooks with HTTP status 200.0
[02:35:41 INF] HTTP GET /api/Book/AllBooks responded 200 in 281.1704 ms
Request started: "GET" http://localhost:25138/api/Book/AllBooks
Response sent: http://localhost:25138/api/Book/AllBooks with HTTP status 429.0
[02:35:42 WRN] OnRejected: User Anonymous endpoint:/api/Book/AllBooks 127.0.0.1
[02:35:42 INF] HTTP GET /api/Book/AllBooks responded 429 in 13.1528 ms
The first and second requests succeeded, but the third request was rejected. In some cases, all requests may succeed if they all reach the API controller within the time window.
In the above policies I have tested, the queue limit was set to zero. When a request fails to obtain a permit lease, it is rejected if it cannot be placed onto the queue. In the next run I configured a policy with a queue limit of 1 as shown below:
.AddFixedWindowLimiter(policyName: "fixed2q1-get", options =>
{
options.PermitLimit = 2;
options.Window = TimeSpan.FromMilliseconds(fixedTimeWindow);
options.QueueProcessingOrder = System.Threading.RateLimiting.QueueProcessingOrder.OldestFirst;
options.QueueLimit = 1;
})
We then run the following CURL command to post 5 GET requests to the web API service:
curl http://localhost:25138/api/Book/Details/1 & curl http://localhost:25138/api/Book/Details/2 & curl http://localhost:25138/api/Book/Details/3 & curl http://localhost:25138/api/Book/Details/4 & curl http://localhost:25138/api/Book/Details/5
The response file and debug log outputs are as shown:
Request started: "GET" http://localhost:25138/api/Book/Details/1
Response sent: http://localhost:25138/api/Book/Details/1 with HTTP status 200.0
[23:22:08 INF] HTTP GET /api/Book/Details/1 responded 200 in 24.7956 ms
Request started: "GET" http://localhost:25138/api/Book/Details/2
Response sent: http://localhost:25138/api/Book/Details/2 with HTTP status 200.0
[23:22:09 INF] HTTP GET /api/Book/Details/2 responded 200 in 10.7602 ms
Request started: "GET" http://localhost:25138/api/Book/Details/3
Response sent: http://localhost:25138/api/Book/Details/3 with HTTP status 200.0
[23:22:13 INF] HTTP GET /api/Book/Details/3 responded 200 in 3385.4315 ms
Request started: "GET" http://localhost:25138/api/Book/Details/4
Response sent: http://localhost:25138/api/Book/Details/4 with HTTP status 200.0
[23:22:13 INF] HTTP GET /api/Book/Details/4 responded 200 in 18.6782 ms
Request started: "GET" http://localhost:25138/api/Book/Details/5
Response sent: http://localhost:25138/api/Book/Details/5 with HTTP status 200.0
[23:22:18 INF] HTTP GET /api/Book/Details/5 responded 200 in 4269.3229 ms
Do you notice a pattern here?
There are two responses where the duration is quite high, close to or over 4000ms. The other responses have a much lower duration, all under 30ms. What this tells us is that the requests with lower response duration were within the permit limit, and those with much higher response durations exceeded the permit limit and were pushed onto the queue. The queuing operations (pushing and popping) and subsequent execution of the request from the queue was quite an expensive set of operations.
What we notice here is that even though increasing the queuing permit guarantees more requests are submitted to the API controllers, the drawback is that the duration can increase considerably when the number of permits is quite low.
As we have seen from the above policy configurations within a .NET 7 web API application, and sample test runs against a Web API service, the rate limiter is a useful service within the ASP.NET 7 Core library that is used to protect a web API service against excessive HTTP REST calls that might bring down the service.
That is all for today’s post.
I hope you have found this post useful and informative.
Andrew Halil is a blogger, author and software developer with expertise of many areas in the information technology industry including full-stack web and native cloud based development, test driven development and Devops.
