Spend enough time in IT, and you start to recognize the cycles in the conversation. A new thing is proposed, everyone gets excited; someone points out that the new thing can’t yet do everything, everyone gets disillusioned; people work out what the new thing is good for — and what it isn’t — and the argument moves on to the next new thing.
Right now in the barrel we have serverless architectures.
The instigator of the current round of serverless criticism is David Heinemeier Hansson, better known as DHH, of Basecamp fame. DHH posted a polemic titled Even Amazon can’t make sense of serverless or microservices, in which he cited a case study by the Amazon Prime Video team about their infrastructure.
The report goes into some detail about the internals of the Prime Video service. DHH quotes a passage about hard limits to scaling, but this note here was the most informative to me:
The two most expensive operations in terms of cost were the orchestration workflow and when data passed between distributed components. To address this, we moved all components into a single process to keep the data transfer within the process memory, which also simplified the orchestration logic.
DHH
Here’s the thing: as Adrian Cockroft (of Netflix fame) pointed out, nothing in the original report actually called the serverless model into question. All that the Amazon team said was that the service, and their understanding of it, had matured and their needs had changed:
When you are exploring how to construct something, building a prototype in a few days or weeks is a good approach. Then they tried to scale it to cope with high traffic and discovered that some of the state transitions in their step functions were too frequent, and they had some overly chatty calls between AWS lambda functions and S3. They were able to re-use most of their working code by combining it into a single long running microservice […]
Adrian Cockroft
When you start building a new service, by definition you do not understand its internal performance characteristics or the load patterns that users will subject it to. Therefore, it is a good idea to err on the side of caution and greater flexibility. This was after all the original promise of the cloud, which it has emphatically delivered on. Where the original dot-com boom required promising startups to spend millions of dollars of investment capital buying server hardware, software licenses, colo space, and so on before they could even really begin developing their proposed service, in the cloud era you can get started with a credit card. It’s also much faster, with infrastructure build-outs that used to take weeks or months of experts working around the clock, now happening automatically in minutes.
Serverless is simply the next step in that evolution. Cloud is much more flexible than owning your own hardware, but its traditional pricing model is still rigid, requiring users to estimate their needs up front, and scaling might not be as responsive or granular as you might want. If your provider only offers t-shirt sizing, and you fall between a small and a medium, but might need a large if things go well, you may find that you have some tough decisions to make. Serverless instead promises to let your service scale from zero to however big it needs to get, and lets you pay as you go.
Here is where the caveats come in: many things that claim to be serverless… aren’t. True serverless scales to zero: if nobody uses your service, you pay nothing. That’s an important guarantee when you are launching a new service: if users aren’t on your service yet, you have no revenue, so knowing that your costs would also be zero in that scenario gives you some peace of mind that you’re not going to be on the hook for a ton of capacity that you’re not using.
The other side of that particular coin is that the pure consumption-based nature of serverless pricing scales to infinity — and so there is a cross-over point where it makes more sense to commit to a defined amount of capacity rather than continuing on the pay-as-you-go model. That financial flexibility comes at the cost of architectural complexity, too, which also factors into where the cross-over point will be for each service architecture.
That is all that happened to the Amazon Prime Video team: they hit that cross-over point, and found that they no longer needed to pay the cost of that extreme flexibility. Being inside Amazon, there was presumably not a direct cost in the same way as there would be for an external AWS customer, although it’s probably a safe assumption that Amazon does account for that capacity in a pretty granular fashion. However, what the team describes in their report is the cost to them in terms of architectural complexity and service performance. Having taken the time to understand the specific needs of their service, they no longer need to pay that toll: they simplified their architecture and got better performance by hard-wiring the specific pathways that they needed.
The key take-away here, though, is that we are seeing the end-stage of the process. If the Amazon Prime Video architects had had to work all this out without the flexibility of a distributed architecture, they might never have got to this point. Instead, they were able to get their service off the ground quickly, refine it rapidly in response to evolving usage patterns, and now that the rate of change has cooled off to a manageable level, they have reviewed their architectural requirements in light of their hard-earned expertise.
This is how things should be done. This story is not a failure of serverless; this is literally what it’s for!
SnapLogic enables something very similar, with an additional twist: you can connect anything to everything, and everyone can get involved. Domain experts can take advantage of our graphical low-code/no-code interface to design data and app integrations without needing programming knowledge. If that integration takes off and becomes popular, it can be refactored by experts to remove performance bottlenecks, avoid duplication, and ensure compliance. That’s a high bar for a project to clear in its early stages, though, when the focus is on enabling people to get started and build the functionality that they need.
On this note, SnapLogic can also integrate with all sorts of serverless services and stitch them together in a unified whole. Our business and technology approach is about working with our users’ architectures, not mandating or constraining their choices. That’s why we work with both on-premises and cloud infrastructure, whether self-managed or delivered as a service — or indeed, fully serverless for maximum flexibility.
The benefit to the business of both serverless architectures and low-code/no-code development is the ease of getting a new project off the ground. If you over-index on architectural purity in the early stages, you’ll never get anything done. Vice versa, once you have a good idea of what you need to do, you should absolutely double down on that and incorporate what you have learned. Amazon seems to be doing okay on that basis. We should all learn from their example.
