One of the major pain points for me with Amazon Redshift has always been the coupling between storage and compute. Competitors like Snowflake and Google’s BigQuery offer independent compute and storage, which means easier (and quicker) scaling in times of increased load. Redshift’s main drawback in the scalability sense has been that it can take up to 24 hours to resize your cluster (during which it’s in read-only mode), meaning there’s a lot of pressure to get your cluster configuration spot on before you go into production. Redshift’s provision of elasticity is just not up to par with most of Amazon’s other services. While Redshift Spectrum helps with this, it’s not a solution to the issue of scalability for an existing cluster.
In the lead up to re:Invent, Amazon last night dropped a load of really neat announcements (server-side encryption for DynamoDB as standard, SSE support for SNS), among which was the reveal of Elastic resize for Redshift. As an aside, if this is the stuff they’re announcing now, there should be some really nice announcements at re:Invent. Continue reading…
What seems like an age ago, I spotted a setting on one of our Redshift clusters that suggested Enhanced VPC routing support for Redshift Spectrum might be on the way. After waiting a while, and waiting some more, and then waiting some more, it seems that Amazon have finally released this into the wild, and Redshift Spectrum now works with clusters that have Enhanced VPC routing available!
Serverless computing and event-driven functions are what it’s all about at the moment. But what happens when the event trigger fires, and your process then encounters an error? How do you recover from this given the event has since passed and may never happen again? This is a common question in AWS when working with their serverless, event-driven Lambda Functions.
Fortunately, AWS lets you define Dead Letter Queues for this very scenario. This option allows you to designate either an SQS queue or SNS topic as a DLQ, meaning that when your Lambda function fails it will push the incoming event message (and some additional context) onto the specified resource. If it’s SNS you can send out alerts or trigger other services (maybe even a retry of the same function – although watch out for infinite loops), or any combination of the above, given its fanout nature. If it’s SQS you can persist the message and process it with another service.
SSMS 2012 Express is available as a standalone download
Something I see online all the time, is people trying to find a standalone download for SQL Server Management Studio. Until recently, it seems, Microsoft certainly didn’t make it easy to just install the client tools without having to install the entire SQL Server database engine as well. The are plenty of reasons why you might want to install just the client tools and not the database engine on your computer, after all, in this world of self-service business intelligence, it’s no longer just developers and DBAs who are playing with data directly.
Fortunately, with SQL 2012 Express, you have a wide variety of download options, ranging from a standalone version of SSMS Express, to SQL Express (including SSMS), with Reporting Services and Full Text Search.
As if renaming the accurately titled Business Intelligence Development Studio (BIDS) to the rather ambiguous SQL Server Data Tools (SSDT) wasn’t bad enough, in December, Microsoft’s latest SSDT release only brought half the expected capabilities to Visual Studio 2012. Yep, the December 2012 SSDT download was missing a key component: the project and item templates for developing MS BI projects in Visual Studio. Thankfully, the newest release (5th March, 2013) has finally added all of the MS BI templates to SSDT, so you can now develop SSIS packages, SSAS cubes and SSRS reports in the Visual Studio 2012 environment.
Unfortunately, they’ve not made the whole process easy. Searching for “SQL Server Data Tools” will likely lead you to a download which, upon installation, will add connectivity and server management tools to VS 2012 – making it like an up-to-date version of SQL Server Management Studio (SSMS), but without the BI project templates.
The latest release (with the BI templates) is actually called:
Microsoft SQL Server Data Tools – Business Intelligence for Visual Studio 2012
So make sure that if you’re trying to get SSDT for BI development work, that you download the correct version. Unfortunately, that wasn’t the end of the issues, as I had a bit of trouble with installation that I felt needed sharing.
Fig. 1: On a 64-bit SQL instance, be sure to select “Perform a new installation of SQL Server 2012”
Once you execute it, the installer will unpack and run the SQL Server 2012 SP1 setup wizard. Don’t worry about this, remember that SSDT, like BIDS before it, is actually a component of SQL Server based upon the Visual Studio shell, NOT actually an extension to Visual Studio itself.
The trick with the installation is when you reach the Installation Type step (see Fig 1.).
If you’re running an x64-based SQL instance (64-bit), make sure to select “New Instance” on the Installation Type page, and NOT “Add features to an existing instance”.
This is because although the SQL Server instance is 64-bit, the Visual Studio 2012 shell is actually 32-bit. If you attempt to upgrade a 64-bit instance with a 32-bit component, it fails the Installation Rules checks and won’t allow you to proceed.
Choosing “New Instance” will work but don’t worry, it doesn’t actually require creation of a new SQL instance, it just allows the installer to get past the pre-installation checks.
If you’ve got a 32-bit instance of SQL Server, it doesn’t matter what option you choose here.
The new Business Intelligence Project templates in action
Once the installation has completed (may require a restart), you can open Visual Studio 2012 (or the new SQL Server Data Tools 2012 item on your start menu) and get developing. Click “New Project” in the File menu and check for the “Business Intelligence” templates to confirm that it’s worked.
I’ve yet to find any real differences between the Visual Studio 2012 based SSDT and the Visual Studio 2010 based version that shipped with SQL Server 2012. At the moment, the main advantage of using this release seems to be to take advantage of the improved features of Visual Studio 2012 over its 2010 counterpart, rather than any advancements in the Business Intelligence templates/tools themselves.
They might be there, however, I just haven’t come across them yet. Let me know in the comments below if you’ve spotted any improvements over SSDT 2010 and what they are.