An API Gateway is not the new Unicorn

The more we talk to people about API gateways, the more we have come to realize that the term means different things to different people, and more specifically we see a growing trend where developers (but mostly managers) expect an API gateway to be able to perform miracles, usually a long way away from its intended use.

It’s as if the API Gateway is the new unicorn that will solve all problems and make everything wonderful in the microservices scene.

We wrote this post to start a conversation around the API gateway pattern and highlight some parts that are usually omitted, leading to misunderstanding or errors in the pattern implementation. This is because many of the definitions around are biased and trying to sell you a one-size-fits-all solution.

Let’s start from the beginning: Since the API Gateway only makes sense in an environment with several services, here’s a reminder of the microservice pattern.

The microservice pattern

At the dawn of the microservices era, Martin Fowler set out the common characteristics of a Microservice Architecture. Not all microservice architectures have all these characteristics, but they tend to have most of the following:

1. Componentization via Services
2. Organized around Business Capabilities
3. Products not Projects
4. Smart Endpoints and Dumb Pipes
5. Decentralized Governance
6. Decentralized Data Management
7. Infrastructure Automation
8. Design for Failure
9. Evolutionary Design

The problem an API Gateway tries to solve

One of the most referenced API Gateway pattern descriptions available on the net is on Chris Richardson’s microservices.io website. The article introduces the problem of the microservice pattern with the following question:

How do the clients of a Microservices-based application access the individual services?

Both the definition of the problem and the proposed solution presented in the article are a perfect introduction point - but they can lead to a major misconception. Let’s analyze the definition of the problem first and then test it against the referred microservice definition.

1. Who are the clients of a Microservices-based application? By definition, they should be the applications themselves: mobile apps, websites, SDK users etc.
2. What is a Microservices-based application? It is not a service, that’s for sure. No company adopts the microservice pattern in order to build a single service. It’s more likely to be an ecosystem of services, each one responsible for a small, limited set of features
3. Why would the clients access any service? They’d rather consume use cases. Of course, each use case could be a composition of N services, but this has nothing to do with the client itself.

We would like to think that a better definition of the problem is broader and could go like this:

How do the clients of a microservice-based ecosystem consume their use cases?

And any kind of solution to this problem provided by any API gateway in the market should be also checked against the common characteristics of microservice-based ecosystems.

The API gateway solution

The API Gateway solution is not an answer for any specific product in the market and therefore it should fit this abstract definition:

The API gateway is a way to solve the problem of how clients consume their use-cases in a microservice-based ecosystem within the microservice pattern

Going back to the microservices.io article we mentioned earlier, it proposes this solution to the problem:

[An API gateway] is the single entry point for all clients. The API gateway handles requests in one of two ways. Some requests are simply proxied/routed to the appropriate service. It handles other requests by fanning out to multiple services.

If we accepted the abstract definition of an API gateway, this assertion would become incomplete and some parts are wrong. Let’s start with the ones that are wrong:

1. By definition, an API Gateway should not be a single entry point in a microservice ecosystem as it would enforce centralization, coordination and have a single point of failure. The API gateway should be the entry point to some set of use cases, and you should analyze how to group these use cases in different API gateway services. As with any other microservice, the scope of the service matters. Otherwise we are moving the monolith from the API to the gateway!
2. Proxying is not in the scope of the API gateway layer, because it means clients are totally coupled to the service implementation rather than to the use case they are trying to cover. There were tons of good proxies out there before the microservice pattern was popular and they solve a very different problem.

And the same document makes a variation putting one API Gateway per client type and calls it the Backend For Frontend as

A variation of this pattern is the Backend for Front-End pattern. It defines a separate API gateway for each kind of client.

This model is closer to the initial abstract definition we accepted before. But what is missing in the previous definitions is:

1. The API gateway decouples clients from the actual services involved in the use cases the clients are consuming (no matter how many of them are involved in a single use case).
2. The API gateway has a dedicated endpoint per use case. Each use case can be consumed by multiple clients.
3. Use cases should be idempotent if possible.
4. A use case can not contain a sequential transaction between external sources/services.

Finally, the biggest objection to that approach is that it opens the door to a lot of bad practices: the single entry point concept and this final statement:

The API gateway might also implement security, e.g. verify that the client is authorized to perform the request

CAUTION! This implies that the API gateway is a good place to implement all the transversal features required in our microservice environment. It is simply not possible to do this without breaking some of the characteristics expected on every microservice.

So, what can be expected from an API gateway?

These are some of the things you could expect from an API Gateway:

• Server-side request validation: It is possible to inject some server-side request validation before reaching the backends, but these validations should not require any external request. You can validate a JWT signature but you shouldn’t request that validation from a 3rd party before serving it.

• QoS optimization: There are several strategies available for the quality of the service and improving the user experience. All are based on key concepts:

  • Grouping requests to the backends from a single client as a use case
  • Abuse of the idempotency of the backend services by sending several requests concurrently. There’s more on that in the concurrent-requests documentation.
  • Shrink responses to contain the indispensable data, as it improves the response times and the bandwidth consumption of the clients.
  • Reduce the number of requests to send from the client (http header caching)

• Request and response manipulation: The API gateway could and should do some sort of request and response manipulation in order to fulfil each use case.

• Interaction with existent SD: Non-blocking interactions with existent Service Discovery are totally sound. Coordinating those interactions is not.

• Distributed rate limit and throttling: It’s ok to introduce features that can run on isolated, distributed instances of a service, but imposing hard-limits per cluster is just not a realistic expectation (if we are concerned about the overall performance).

• Circuit breaker: To detect failures and prevent stressing your ecosystem.

• Multiple encodings and protocols: As your services might be diverse.

• Metric collection: In order to have visibility on what is going on in this layer, as long as this is non blocking and not dependent on a centralized place.

• Clean API versioning contributor: As you might want to keep a clean and consistent versioning for your gateway that is completely unrelated to the versioning of each of your internal services.

What you shouldn’t expect from an API Gateway

The most recent Technology Radar from Thoughtworks (PDF) made us smile as they see API Gateways in the same way we do:

We remain concerned about business logic and process orchestration implemented in middleware, especially where it requires expert skills and tooling while creating single points of scaling and control. Vendors in the highly competitive API gateway market are continuing this trend by adding features through which they attempt to differentiate their products. This results in OVERAMBITIOUS API GATEWAY products whose functionality — on top of what is essentially a reverse proxy — encourages designs that continue to be difficult to test and deploy. API gateways do provide utility in dealing with some specific concerns - such as authentication and rate limiting - but any domain smarts should live in applications or services.

For us, the boundary of which functionalities can go inside an API Gateway and which should not can be summarized in a simple sentence:

Anything that you can validate or create within the API Gateway without the help of other services or shared states is suitable. Anything else, move it away.

For example, if you receive ciphered roles and you want to decipher them and validate against an ACL INSIDE the gateway configuration, DO IT. If you want to query an ACL using an external database, DON’T.

So, with that simple rule of thumb and the list of characteristics already discussed:

• the API gateway should not be a Single Point of Failure
• the API gateway should not be centralized or coordinated synchronously
• the API gateway should not be used as a centralized configuration point
• the API gateway should not depend on state
• the API gateway should not be anything other than just another microservice
• the API gateway should not know anything about the business logic encapsulated in any of its backends

Summary

The microservices pattern proposes a solution for dealing with the massive complexity of some environments by breaking down the system into more simple, cohesive components and delegating part of the responsibility to the clients, making a huge leap towards decentralization.

The API gateway pattern is a way to reduce the complexity delegated to the clients in a microservice-based ecosystem by adding some more microservices responsible for dealing with some of the complexity without breaking any of the characteristics of the microservices. It’s a service that ensures the pipes are still dumb while making life much easier for the client developers. It doesn’t help backend services in exposing their resources, encoding the input/output or checking if the user has enough privileges to consume the requested resource.

On a final thought, if we take into account CAP theorem and we are thinking of adding a new layer between the clients and the resource servers they consume, why on earth would anyone add features banking on the system consistency ( C ) to that new microservice layer? Do we want to move from the fat database to a fat API gateway? If so, maybe we should use the names that already exist for those components: Edge Service, Enterprise Service Bus, API Manager… and we should also ask the companies that adopted them: why did you abandon them years ago?

Thanks for reading!