Enterprise Documentation
Rate Limit Tiers
Document updated on Aug 9, 2024
The rate limit based on tiers allows you to have multiple sets of service and endpoint rate limits that apply differently to users depending on their tier or sometimes called subscription plans.
For example, Mary has a Gold plan that entitles her to make more requests per second than John, who is in an inferior Silver plan:
The tiered rate limit component allows you to get a header from the requests that set the tier a user belongs to. Then, the limits apply depending on the plan.
Tiered rate-limiting configuration
The tiers configuration uses the extra_config’s namespace qos/rate limit/tiered and can be done either at the service level (root) or the endpoint level. When you set the tiers at the service level, you define the tiers for all API endpoints. On the endpoint level, you set the tiers for that specific endpoint.
The tiers configuration object is an array with all the different tiers available, which are evaluated in order. This means that if a user can match more than one plan, the first plan matching in the list is the rate limit applied to them.
The configuration of the tiers is as follows:
Fields of Tiered Rate Limit
| The header name containing the tier name. The string you provide is case-insensitive. If you need to take the value from a place that is not a header (a token, an API key), you must use propagate functions in the components that convert values to internal headers. | ||||||
| The list of all tier definitions and limits for each. Each item in the list is a tier object.
Each item is an object with the following properties:
|
As you can see, all tier definitions come with an associated ratelimit object, which works the same as a regular service rate-limit:
Configuration for each `ratelimit` in the tier
max_rate
, or
client_max_rate
| Defines the maximum number of tokens a bucket can hold, or said otherwise, how many requests will you accept from all users together at any given instant. When the gateway starts, the bucket is full. As requests from users come, the remaining tokens in the bucket decrease. At the same time, the max_rate refills the bucket at the desired rate until its maximum capacity is reached. The default value for the capacity is the max_rate value expressed in seconds or 1 for smaller fractions. When unsure, use the same number as max_rate.Defaults to 1 |
| The cleanup period is how often the routine(s) in charge of optimizing the memory dedicated will go iterate all counters looking for outdated TTL and remove them. A low value keeps the memory slightly decreasing, but as a trade-off, it will increase the CPU dedicated to achieving this optimization. This is an advanced micro-optimization setting that should be used with caution. Specify units using ns (nanoseconds), us or µs (microseconds), ms (milliseconds), s (seconds), m (minutes), or h (hours).Defaults to "1m" |
| These are the number of routines that search for and remove outdated rate limit counters. The more routine(s) you add, the faster the memory optimization is completed, but the more CPU it will consume. Generally speaking, a single thread is more than enough because the delete operation is very fast, even with a large number of counters. This is an advanced micro-optimization setting that you should use with caution. Defaults to 1 |
| Defines the maximum number of tokens a bucket can hold, or said otherwise, how many requests will you accept from each individual user at any given instant. Works just as capacity, but instead of having one bucket for all users, keeps a counter for every connected client and endpoint, and refills from client_max_rate instead of max_rate. The client is recognized using the strategy field (an IP address, a token, a header, etc.). The default value for the client_capacity is the client_max_rate value expressed in seconds or 1 for smaller fractions. When unsure, use the same number as client_max_rate.Defaults to 1 |
| Number of tokens you add to the Token Bucket for each individual user (user quota) in the time interval you want (every). The remaining tokens in the bucket are the requests a specific user can do. It keeps a counter for every client and endpoint. Keep in mind that every KrakenD instance keeps its counters in memory for every single client. |
| Time period in which the maximum rates operate. For instance, if you set an every of 10m and a rate of 5, you are allowing 5 requests every ten minutes.Specify units using ns (nanoseconds), us or µs (microseconds), ms (milliseconds), s (seconds), m (minutes), or h (hours).Defaults to "1s" |
| Available when using client_max_rate and you have set a strategy equal to header or param. It makes no sense in other contexts. For header it is the header name containing the user identification (e.g., Authorization on tokens, or X-Original-Forwarded-For for IPs). When they contain a list of space-separated IPs, it will take the IP from the client that hit the first trusted proxy. For param it is the name of the placeholder used in the endpoint, like id_user for an endpoint /user/{id_user}.Examples: "X-Tenant" , "Authorization" , "id_user" |
| Sets the maximum number of requests all users can do in the given time frame. Internally uses the Token Bucket algorithm. The absence of max_rate in the configuration or a 0 is the equivalent to no limitation. You can use decimals if needed. |
| All rate limit counters are stored in memory in groups (shards). All counters in the same shard share a mutex (which controls that one counter is modified at a time), and this helps with contention. Having, for instance, 2048 shards (default) and 1M users connected concurrently (same instant) means that each user will need to coordinate writes in their counter with an average of under 500 other users (1M/2048=489). Lowering the shards might increase contention and latency but free additional memory. This is an advanced micro-optimization setting that should be used with caution. Defaults to 2048 |
| Available when using client_max_rate. Sets the strategy you will use to set client counters. Choose ip when the restrictions apply to the client’s IP address, or set it to header when there is a header that identifies a user uniquely. That header must be defined with the key entry.Possible values are: "ip" , "header" , "param" |
Rate Limit tier at the service level
The following configuration sets four different tiers at the service level, which would apply to all endpoints defined in the configuration simultaneously:
{
"version": 3,
"host": [
"http://localhost:8080"
],
"endpoints": [],
"extra_config": {
"qos/ratelimit/tiered": {
"tier_key": "X-Plan",
"tiers": [
{
"tier_value": "admin",
"tier_value_as": "literal",
"ratelimit": {
"client_max_rate": 100,
"client_capacity": 100,
"every": "1m",
"max_rate": 10000,
"strategy": "header",
"key": "X-Account-Id"
}
},
{
"tier_value": "user",
"tier_value_as": "literal",
"ratelimit": {
"client_max_rate": 20,
"client_capacity": 20,
"every": "1m",
"max_rate": 10000,
"strategy": "header",
"key": "X-Account-Id"
}
},
{
"tier_value": "value.matches('Account-[a-Z]+')",
"tier_value_as": "policy",
"ratelimit": {
"client_max_rate": 20,
"client_capacity": 20,
"every": "1m",
"strategy": "header",
"client_key": "X-USER-ID"
}
},
{
"tier_value": "",
"tier_value_as": "*",
"ratelimit": {
"client_max_rate": 2,
"client_capacity": 2,
"every": "1m",
"strategy": "ip"
}
}
]
}
}
}
In the configuration above, we have defined the following behavior:
- When a request comes with an
X-Planheader with the valueadmin, each user can do 100 reqs per minute. The user uniqueness comes in theX-Account-Idheader. In addition, all users of the admin tier can make up to 10000 requests together. - When a request comes with an
X-Planheader with the valueuser, each user can do 20 reqs per minute. - When a request comes with an
X-Plan, its value is matched against the policyAccount-[a-Z]+, e.g., if it containsAccount-abcdef, then it rates to 20 per minute. - A final special tier
*(which you must place at the end) matches the remaining cases and sets to 2 req/s based on their IP address. This would rate limit unknown accounts.
Tier evaluation
When there are multiple rate limits in the configuration (tiered rate limit is just one of them), the qos/ratelimit/tiered evaluate in the first place. If the user has not reached the usage limit, the rest are checked, having a behavior of consistently applying the most restrictive rate limit.
Speaking of the tiered rate limit alone, when the tiers evaluate, it’s done sequentially. The first tier matching its definition applies, and no additional tiers are checked. Be careful when using the special tier * that matches any request, and always set it in the last position of the array.
Pay attention to similar attributes like tier_key and the inner ratelimit attributes key and client_key. The tier_key tells which header contains the tier name, while the others set wh
Extracting the tier from a JWT token
As the tiers work with a header, if you want to extract them from a JWT, you will need to use the propagate_headers attribute, specifying which claim contains the tier. As JWT validation is only available at the endpoint level, the tier definition must go inside each endpoint. We recommend using Flexible Configuration to reduce the repeat code in your configuration.
For instance, a token containing a claim plan could be configured as this:
{
"endpoint": "/foo",
"extra_config": {
"auth/validator": {
"propagate_claims": [
[
"plan",
"X-Plan"
]
]
},
"qos/ratelimit/tiered": {
"tier_key": "X-Plan",
"tiers": [
{
"tier_value": "admin",
"tier_value_as": "literal",
"ratelimit": {
"client_max_rate": 100,
"client_capacity": 100,
"every": "1m",
"max_rate": 10000,
"strategy": "header",
"key": "X-Account-Id"
}
}
]
}
}
}
Using the API Key role as tier definition
Similarly, API Keys work with propagated headers too. You can use the role of an API key as the tier definition, both at the service and endpoint levels. To do so, you only need to use the propagate_role attribute in the API keys:
{
"version": 3,
"extra_config": {
"auth/api-keys": {
"strategy": "header",
"identifier": "Authorization",
"propagate_role": "X-Krakend-Role",
"keys": [
{
"key": "4d2c61e1-34c4-e96c-9456-15bd983c5019",
"roles": [
"user"
],
"@description": "ACME Inc."
},
{
"key": "58427514-be32-0b52-b7c6-d01fada30497",
"roles": [
"admin",
"user"
],
"@description": "Administrators Inc."
}
]
},
"qos/ratelimit/tiered": {
"tier_key": "X-Krakend-Role",
"tiers": [
{
"tier_value": "admin",
"tier_value_as": "literal",
"ratelimit": {
"client_max_rate": 100,
"client_capacity": 100,
"every": "1m",
"max_rate": 10000,
"strategy": "header",
"key": "Authorization"
}
},
{
"tier_value": "",
"tier_value_as": "*",
"ratelimit": {
"client_max_rate": 1,
"client_capacity": 1,
"every": "1m",
"max_rate": 10,
"strategy": "header",
"key": "Authorization"
}
}
]
}
}
}
Notice that the key used in the ratelimit of each plan matches the identifier of the API Key. Notice that there is also a special tier * for nonmatching tiers that is used as the default, but keep in mind that the API key component can pass as tier_value the string ANY when the endpoint does not require roles, so you might want to add a tier with this value as well.
