We at Full 360 do API modernization. That means, we take your ancient services and bring it up to date, whether its serverless or cloud or brokers, depends on what fits the need.
We had a use case in which a SOAP service went from having a small set of clients to have a lot more, it was part of a major initiative that involved integration with tons of third parties.
That meant that not only we wanted those third parties not to have to deal with outdated WSDL, code generation, and VPN access, we also wanted to improve big time the amount of request the service could handle.
But sometimes is just not that simple, sometimes you cannot just remove/update some pieces of the puzzle.
So we build a solution that included:
That allowed us to take the heat from third parties with ease and then export to the original service, but we had one issue, the export was too fast! We needed to tune down the TPS on the export API
We spent some time searching for some ways to apply throttling to an akka-http based API with akka streams for handling requests. Each approach presented its own issues:
- Each request materializes its own stream: that makes it complicated to throttle the whole API.
- You could do some config around it to trim connections down, but what’s the point on doing that?
- There are also some libraries around, but they are mostly for spray or they are too bloated.
- You could also use the Low Level API of akka-http itself which in theory allows you to do throttling using akka-streams own flows.
I tried all of that, with no luck, so I grew tired of fighting someone else’s code and decided to fight my own, and found out its actually pretty easy to do custom directives for akka-http.
Since we were using Google’s Guice for dependency injection we already had in our dependencies a Rate Limiter, so I used it and worked pretty well.
And that is actually pretty large, you can do smaller directive if you don’t mind getting the TPS value from an environment variable. Also you can use the ip value we are extracting to do per ip throttling.
The way you’ll use that in your Route declaration would be something like this:
And that’s it! Now you can throttle an akka-http API or even build your own directive in which you can do all sorts of things
Please be sure to read akka-http docs as a reference on what directive you can use to extract attributes out of the request!
Note: This is not intending to probe this way as the best solution, but to share the way you can build your own custom directives on akka-http. The throttling aspect of this use case came up late in the game, so that’s why a throttling directive was used. If it would have known from the begining, maybe a broker would have been more appropriate on this use case.
