By using AWS, Pinsight protected its margin and impressed its customer by shipping within the project's tight deadline. The keys to beating that deadline were on-demand scalability and the ease of replicating a critical on-premises workflow on AWS.

“We were banking on the fact that we would be able to not only translate our workflows to AWS, but that the AWS services we were going to use would also be able to scale enough to get the backfill done in time,” says Chris Swanda, a senior DevOps engineer at Pinsight. "Using AWS, we were able to quickly replicate a critical product delivery workflow at scale."

Controlling costs was also crucial to project success, so Pinsight was glad for the transparent pricing structure and easy-to-use cost tools available on AWS. “AWS provides a clear set of tools so you can see per-unit prices,” Swanda says. “Then it’s just a matter of manipulating spreadsheets to estimate the costs of different solution approaches.”

An approach that relied on Amazon EC2 Spot Instances was important to the project's overall cost picture. "We knew that judicious use of Spot Instances would play a big part in keeping our costs under control, so we used the AWS Spot pricing tool to analyze Spot prices and set our ceilings," says Swanda. “It was rare for us to not get capacity on a cluster, but when we did, Amazon EMR just picked up and carried on with the workflow without problems.”

The easy scalability of AWS has transformed what Burcham and his team can accomplish—on this project, and others like it. “Ten years ago, the lead time and cost of achieving such a massive compute expansion at scale and on-demand just did not exist,” he says. “On AWS, accomplishing the eightfold increase in capacity and throughput we needed for this project was just another day at the office.”