Cloud data movement made easy.

Here’s the challenge of data warehousing in the “API economy”: developers and their services work more efficiently, but it’s harder for architects and analysts to access data behind APIs. That’s why we created the Simple Data Pipe. We want to make it easier to connect to JSON data encapsulated in Web APIs, and land it all in one place, in its native form. Where you go with it next is up to whatever your business reporting requires. The Simple Data Pipe is an open-source project, built by IBM developer advocates who’ve dealt with the same problems. The code comes with pre-built API connectors to pipe data from sources such as Salesforce.com, Stripe, and Reddit into a Cloudant JSON document store. Simple Data Pipe also comes with JavaScript frameworks for

creating your own connectors via OAuth. (Let us know if you create your own. Contributions are welcome!) Once data lands in Cloudant, the analytics services on the IBM Cloud have you covered. It’s easy to move the data to a relational data warehouse — and visualize it in Chartio, Looker, Tableau, and other tools you know and love — or explore data in a Jupyter notebook. Finally, report on all the right data from the Web services that run your business, all thanks to the Simple Data Pipe. While this roster will expand, here is the current set of options for moving data out of Cloudant:

• dashDB, via native Cloudant integration with dashDB. Finish movement using Cloudant’s web dashboard.
• Apache Spark, via native Cloudant integration with Bluemix’s Spark service. Finish movement by calling the Cloudant connector in a Spark Scala Notebook.
• Transporter, the open source ETL pipeline by Compose.io. Finish movement by configuring package info and associated JavaScript code.
• DataWorks, enterprise-grade APIs for data shaping & movement. A paid service on Bluemix as of February 2016. Provision DataWorks on Bluemix first, before deploying the new Simple Data Pipe.

Watch a Deployment

Next Steps

Out of the box you can use the Simple Data Pipe to load CRM data from Salesforce and billing information from Stripe. Click the Deploy to Bluemix button to test drive the application or visit the Github page to explore other deployment options. Learn more about how to load data from other data sources on our connector home page.