Announcing source{d} Engine beta for Code Retrieval and Analysis and source{d} Lookout Alpha for Assisted Code Review

Today we’re excited to announce the public beta of source{d} Engine and public alpha of source{d} Lookout. Combining code retrieval, language agnostic parsing, and git management tools with familiar APIs parsing, source{d} Engine simplifies code analysis. Our second announcement is about source{d} Lookout, a service for assisted code review that enables running custom code analyzers on GitHub pull requests.

source{d} Engine, our solution to “Code as Data” challenges, offers advanced code and architecture analysis to developers and, for C-level executives, engineering analytics and business intelligence. The Engine also lays the foundation for a full suite of Machine Learning on Code applications for assisted code review as you can see in the diagram below.

Code as Data tools like source{d} Engine provide insights from your source code and the datasets necessary for ML on Code applications such as source{d} Lookout.

source{d} Engine beta

For the past 3 years, the source{d} team has been hard at work building a suite of popular open source tools enabling Machine Learning for large scale code Analysis including:

• go-git, Rovers and Borges for Code Retrieval and Unification: Retrieve and store the code history of both open-source and private repositories as a dataset.
• Enry and Babelfish for Language Agnostic Code Analysis: Automatically identify languages, parse code into universal Abstract Syntax Trees, and extract the pieces that matter in a completely language-agnostic way.
• Gitbase for easy code repositories querying with familiar APIs: Use Standard SQL to obtain insights from your source code and version control history, while classifying languages and parsing code through simple custom functions.

In an effort to make all these tools plus others available in an easy and unified way and provide users with the best user experience possible, we’ve decided to build source{d} Engine. Check out the documentation to get started or learn more about its architecture.

source{d} Lookout Alpha

This early Alpha release of source{d} Lookout is our first step towards a full suite of Machine Learning on Code applications. It provides assisted code review on GitHub pull requests and includes the following key features:

• Easily extensible: source{d} Lookout provides a framework to develop and deploy new code analyzers. These analyzers benefit from language agnostic representations of source code (UASTs), avoiding the need for multiple parsing steps.
• Inferred code style: Recognize source code patterns related to programming style from a given organization or particular project and warn developers about inconsistencies related to style in the code being contributed. Right now Lookout only works for Javascript but support for other languages is coming soon.

Check out the Lookout GitHub repository for more information on how to get started or how to develop your own code analyzers.

Learn More about source{d} products