AWS Lambda function with SQS as an event trigger

AWS Lambda can now be invoked directly from SQS

While quietly perusing Twitter this evening, I happened to notice one from the official AWS account with a link to a blog post from Amazon tech hero Randall Hunt describing the newly available capability for AWS Lambda: SQS as an event source!

This is functionality that I, personally, have been wanting for a while now.  While Simple Notification Service (SNS) is absolutely brilliant for a fan-out architecture, and provides immense flexibility with a wide range of supported subscriber types, controlled, serverless polling of SQS wasn’t really a viable option.  While you *could* run a Lambda for a few minutes doing long-polling on SQS, and then terminate before exhausting the 5-minute execution duration cap, it really felt a bit dirty.  To properly implement a queue-polling architecture, you really had to deploy an application on EC2, which meant managing servers etc.  Not that there’s anything wrong with that of course, it just seemed like there was a big glaring hole in the Serverless model.

Native SQS to Lambda event integration though really patches this omission and then some.  Randall’s blog post explains it in full, but it seems like Amazon have implemented some really nice intelligent scaling mechanisms to adjust Lambda concurrency (up to a defined limit) in response to queue depth.  This should really help constrain costs and ensure consistent throughput regardless of spiky traffic.

I’ve not yet had a chance to observe this in the wild though, so best tested for your workload before betting the farm on it, but this looks like yet another long-awaited piece of functionality that Amazon have knocked out the park.  Have you explored SQS as an event source for Lambda yet?  Any observations or gotchas so far?  Let me know in the comments below!

Solutions Architect at Indicia. More years than I care to remember wrangling SQL Server and .NET, now mainly AWS. Data geek, football fan, and Xbox gamer. Check out my gaming blog if you're into that sort of thing.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.