UPDATE 2022-01-12: So it seems AWS backtracked on this slightly, having changed to simply preventing the creation of new or changing of existing ds2 clusters post 2021-12-31.

It’s rampant speculation time. AWS have released a number of nice features for Redshift over the last few months, from maintenance improvements like auto-vacuum and auto-analyze, to time savers like the in-browser query editor and new cluster configuration recommendation tool, that helps you find the right cluster configuration for your needs. It’s these two features that lead me to my wild assumption in the title, for one main reason - neither of them support the ds2.xlarge node type.

One of the major pain points for me with Amazon Redshift has always been the coupling between storage and compute. Competitors like Snowflake and Google’s BigQuery offer independent compute and storage, which means easier (and quicker) scaling in times of increased load. Redshift’s main drawback in the scalability sense has been that it can take up to 24 hours to resize your cluster (during which it’s in read-only mode), meaning there’s a lot of pressure to get your cluster configuration spot on before you go into production. Redshift’s provision of elasticity is just not up to par with most of Amazon’s other services. While Redshift Spectrum helps with this, it’s not a solution to the issue of scalability for an existing cluster.

In the lead up to re:Invent, Amazon last night dropped a load of really neat announcements (server-side encryption for DynamoDB as standard, SSE support for SNS), among which was the reveal of Elastic resize for Redshift. As an aside, if this is the stuff they’re announcing now, there should be some really nice announcements at re:Invent.

What seems like an age ago, I spotted a setting on one of our Redshift clusters that suggested Enhanced VPC routing support for Redshift Spectrum might be on the way. After waiting a while, and waiting some more, and then waiting some more, it seems that Amazon have finally released this into the wild, and Redshift Spectrum now works with clusters that have Enhanced VPC routing available!

As of Build 1.0.4349 or Build 1.0.4515, this functionality will be available in Redshift. It hasn’t made it into the official announcements yet, but it has popped up on the Redshift forums here: https://forums.aws.amazon.com/ann.jspa?annID=6197

One of the things that I really like about Google BigQuery is the ability to write queries right there in the web browser without having to install a hefty IDE. Sure, there are times when having the full power of something like JetBrains DataGrip comes in handy (source control integration, customisation, formatting), but sometimes you just want to dive in and write a quick query without any messing around. Amazon did this for Athena, which was really handy, but strangely never did so for Redshift…until now!

AWS is knocking it out of the park at the moment with loads of new services and features coming out every week. Indeed, it can be hard to keep up with the degree of change. But, while working on one of our Redshift clusters today we spotted a potential scoop that would remove a key blocker for one extremely useful service, Redshift Spectrum.

Up until now it’s only been possible to use Spectrum if you don’t have Enhanced VPC Routing enabled on your Redshift cluster. There are so many benefits to using Enhanced VPC Routing (reduced data transfer cost, control, security) that it’s hard to see why anyone wouldn’t be using it, especially if you move data between Redshift and S3 a lot.

But we spotted a new parameter being applied to one of our clusters when we made some maintenance changes to a parameter group. There’s now a parameter named spectrum_enable_enhanced_vpc_routing showing, which hints that Amazon may be about to remove this crucial limitation.

Last year, Microsoft added a preview connector enabling Power BI to query Amazon Redshift. This wasn’t publicised as an “official” data source, and took some steps in order to be able to even see the connector in Power BI Desktop. Crucially, you could only use this connector in Power BI Desktop, not when workbooks are deployed to the cloud. Yesterday, Microsoft announced the connector is now available within the Power BI Service, which means that workbooks containing Redshift data connections can now be deployed to the cloud. I’ve been working a lot with Redshift over the past year or so, and Power BI’s still my go-to data-viz solution, so I’m delighted to see the this announcement, as it means that Redshift-based workbooks can now be shared with others via powerbi.com.