AWS Step Functions Customers

Coinbase is a secure digital currency exchange platform that makes it easy to buy, sell, and store cryptocurrency like Bitcoin, Ethereum, and Litecoin. As of 2017, Coinbase is the largest bitcoin broker in the world and serves users in 33 countries. Coinbase has a high performing engineering organization. They deploy hundreds of times per day across hundreds of projects, release new features from idea to launch in under an hour, and sustain low failure rates. This is possible because most of their change management and deployment processes are automated and their engineers have adopted a DevOps culture.

A key part of that automation is their secure, automated deployment pipeline, Odin. Odin takes a description of a project release and then safely and securely launches it into AWS. Coinbase automated Odin using AWS Step Functions, allowing them to deploy 12-factor-applications to AWS easily, reliably and securely. Step Functions provided the tools for reliability and end-to-end visibility out of the box, with a serverless operating model that doesn’t require any infrastructure to be provisioned or managed. Designing Odin as a Step Functions state machine lets Coinbase visually monitor the progress of the deployment, and audit each deployment step by step. With new insight into the reasons why services were failing, Coinbase was able to improve their rate of successful mission critical deployments from 90% to 97%. The visibility provided to engineers allowed them to diagnose and resolve their own issues quickly, and reduced the number of support tickets related to unsuccessful deployments.

"If a team focuses on building quality automation into their workflows, stability follows. By building quality automation with services like AWS Step Functions, Coinbase avoids trading speed for stability, and instead we get both."

Graham Jenson, Infrastructure Engineer – Coinbase

Cox Automotive Inc. makes buying, selling, owning and using cars easier. To support digital advertising, they wanted to increase the volume and quality of their machine learning prediction models. However, they were concerned that their models might drift over time, leading to poor decisions. Injecting Data Scientist oversight into the model retraining process was the best solution, but the challenge was to to do so without slowing down the Decision Science team.

Cox Automotive decided to automate their ML model delivery workflow with AWS, including a model approval pipeline coordinated by AWS Step Functions. Once models are retrained using Amazon SageMaker, the output artifacts and diagnostics are uploaded to Amazon S3, which triggers Step Functions to run a model approval workflow. An email is sent to a Data Scientist, who reviews the model and clicks either “Approve” or “Reject”. If approved, Step Functions sends the artifacts to a destination S3 bucket along with an Amazon SNS notification indicating that the new model is ready for consumption.

Investing in automation has given both teams time to work on higher value activities. Cox Automotive now has a reusable, automated process that allows them to spend more time making more accurate models, and less time building deployment pipelines.

"Using services like Amazon SageMaker and AWS Step Functions to automate model deployment has enabled us to deliver recurring quality, mostly-automated models to the business with minimal human intervention and overhead."

Jeremy Irwin, Lead Solutions Architect – Cox Automotive

CyberGRX creates a risk management platform that pinpoints, measures, and prioritizes third-party cyber risks, transforming third-party cyber risk management for enterprises all over the world. It chose to build on Amazon Web Services (AWS) due to the simplicity of deployment and how the design mapped easily to the company's existing workflows.

"We needed to run massive parallel computations on our Bayesian network algorithm and decided to use the AWS Step Functions distributed map state because it provided the capability to scale in a way that was unattainable before. Using Distributed Map, we were able to process the entire exchange, 227,000 companies worth of data—57 billion data points—in under an hour, which previously took us over 8 days of processing. Not only did it save us a massive amount of manpower time, it also made it far less expensive."

Charles Burton, Director, Senior Software Engineer – CyberGRX

Xylem is a leading global water technology company committed to developing innovative technology solutions to the world’s water challenges. To perform analytics, Xylem was using a Hadoop-based, on-premises data lake to store massive amounts of electric, water, and gas metering data. However, this solution limited their data analysis capabilities and was not flexible enough to allow for the addition of new projects. Data processing was taking four to five hours each day, and if they added new customers, up to several days.

By moving the Xylem Data Lake to an AWS-based platform leveraging AWS Batch, AWS Step Functions, and AWS Lambda, the Xylem data team was able to dramatically expand their data analytics capabilities. Xylem feeds each input file to a Step Functions state machine, which then validates the file, tracks metadata and finally, launches and monitors an AWS Batch job that processes the data. Each state machine runs independently and in parallel. The AWS Batch jobs allocate containers automatically, and Xylem can now parse thousands of files simultaneously, without having to worry about jobs competing for resources. As a result, Xylem has reduced data processing time from 20 hours to only two hours – regardless of the size of the dataset.

"Using AWS Step Functions as part of the Xylem Data Lake has transformed our pace of innovation. We can now develop internal and external analytics projects with fewer handoffs to other engineers, fewer interrupts with DevOps, and with more technologies than we ever thought possible."

Mitchell Hensley, Vice President, Software Strategy – Xylem