Serverless computing and event-driven functions are what it’s all about at the moment. But what happens when the event trigger fires, and your process then encounters an error? How do you recover from this given the event has since passed and may never happen again? This is a common question in AWS when working with their serverless, event-driven Lambda Functions.

Fortunately, AWS lets you define Dead Letter Queues for this very scenario. This option allows you to designate either an SQS queue or SNS topic as a DLQ, meaning that when your Lambda function fails it will push the incoming event message (and some additional context) onto the specified resource. If it’s SNS you can send out alerts or trigger other services (maybe even a retry of the same function - although watch out for infinite loops), or any combination of the above, given its fanout nature. If it’s SQS you can persist the message and process it with another service.

So let’s look at both options in a little more detail.

AWS is knocking it out of the park at the moment with loads of new services and features coming out every week. Indeed, it can be hard to keep up with the degree of change. But, while working on one of our Redshift clusters today we spotted a potential scoop that would remove a key blocker for one extremely useful service, Redshift Spectrum.

Up until now it’s only been possible to use Spectrum if you don’t have Enhanced VPC Routing enabled on your Redshift cluster. There are so many benefits to using Enhanced VPC Routing (reduced data transfer cost, control, security) that it’s hard to see why anyone wouldn’t be using it, especially if you move data between Redshift and S3 a lot.

But we spotted a new parameter being applied to one of our clusters when we made some maintenance changes to a parameter group. There’s now a parameter named spectrum_enable_enhanced_vpc_routing showing, which hints that Amazon may be about to remove this crucial limitation.

Late last night, Amazon announced that their proprietary AWS data visualisation tool, Quicksight was now generally available in the US and Ireland. Quicksight aims to be a PowerBI-esque drag and drop visualisation tool, that allows you to access your data from AWS (and other sources) in seconds, regardless of scale. I’ve had a very quick go this morning, and visualised some data from a modest 1TB Redshift cluster after just a few minutes. The biggest challenge was finding out the correct IP range for Quicksight to enable access to my VPC (Thank you serverfault).