Development

AWS Lambda can now be invoked directly from SQS

By Graham 28 June 2018
AWS Lambda can now be invoked directly from SQS

While quietly perusing Twitter this evening, I happened to notice one from the official AWS account with a link to a blog post from Amazon tech hero Randall Hunt describing the newly available capability for AWS Lambda: SQS as an event source!

This is functionality that I, personally, have been wanting for a while now. While Simple Notification Service (SNS) is absolutely brilliant for a fan-out architecture, and provides immense flexibility with a wide range of supported subscriber types, controlled, serverless polling of SQS wasn’t really a viable option. While you could run a Lambda for a few minutes doing long-polling on SQS, and then terminate before exhausting the 5-minute execution duration cap, it really felt a bit dirty. To properly implement a queue-polling architecture, you really had to deploy an application on EC2, which meant managing servers etc. Not that there’s anything wrong with that of course, it just seemed like there was a big glaring hole in the Serverless model.

Native SQS to Lambda event integration though really patches this omission and then some. Randall’s blog post explains it in full, but it seems like Amazon have implemented some really nice intelligent scaling mechanisms to adjust Lambda concurrency (up to a defined limit) in response to queue depth. This should really help constrain costs and ensure consistent throughput regardless of spiky traffic.

I’ve not yet had a chance to observe this in the wild though, so best tested for your workload before betting the farm on it, but this looks like yet another long-awaited piece of functionality that Amazon have knocked out the park. Have you explored SQS as an event source for Lambda yet? Any observations or gotchas so far? Let me know in the comments below!

comments powered by Disqus