Query 1 Million Terabytes of Data in 155 Seconds - Amazon Redshift Spectrum
I was watching a Keynote for Amazon Web Services the other day, and whilst watching this I saw a new service they were bringing to the market to make queries into large (I mean massive) datasets were taking to long and AWS have created a new product to speed these up. The most interesting part was timesaving that they created in undergoing these massive queries.
So for a SQL Query (quite complex)on a large dataset (1 exabyte, yes that’s 1 million terabytes) that until 2 weeks ago, would take 1000 nodes (equivalent to a 1000 low spec computers), would take 5 years to return the result.
Now AWS introduced a new service called Redshift Spectrum, which can do the equivalent in 155 seconds. Quite an improvement, he definitely deserved a louder clap than he got.