II. AlphaGo Zero

Varun was one of the first readers to write a comment below my post. Here’s his comment:
Hi Louis, fantastic article but what do you think about the AlphaGo Zero announcement today in relation to reinforcement learning, which started learning from scratch, and beat the crap out of prior version of Alpha Go (which was based on supervised learning/learning from data)? Sorry I am prodding you here, but in the hopes of a v2 article, which factors in reinforcement learning which could be a game changer and this idea of a “data moat ” and data network effects being no match for reinforcement learning-driven systems in the near future.

Great point! Here’s what I think:

For those who missed it, Deepmind released a paper in Nature presenting a new version of AlphaGo — its algorithm which learns how to play Go — called AlphaGo Zero. The novelty of the algorithm is that it uses reinforcement learning (RL). One of the specificities of the algorithm is that it learns by itself without needing any initial dataset. In (very) short, the algorithm tries to find the optimal move to do at each steps according to a computed probability to win at any given time. After much less iterations and using much less computing power (smaller by an order of magnitude according to both variables), AlphaGo Zero outperformed its precedent supervised version. Certain industry

Because this new version required NO initial dataset to outperform those trained on years of data.

Hence, in this setup, the equation of my post

Success in AI = Data (* Data)+ ML Talent + Algorithms

is fundamentally wrong. It’s wrong. But it might be wrong only if we are in a situation where the entire set of possibilities can be set in the definition of the problem, so that we can compute a probability to win at any given time.

It’s the case in the game of Go, but this is not the case in many real world problems.

Think about predicting sales for example. The world changes after any given action undertaken by a sales person, hence I don’t see how we could add all the possible outcomes of a given action in the existing model. Or at least this representation might be very partial because there are many hidden variables.

AlphaGo Zero is an amazing breakthrough in the field of AI, but I am still wondering how this could apply in industrial setups.

One use of reinforcement learning I have seen consists in training a ML model to predict the reward function, before learning the impact of each of the variables on the reward function based on historical data. But this still requires historical data. If you have hints on how you could train a reinforcement learning model without any initial dataset in an industrial setup, send me a note, I’d love to learn more.

Part III. Will incumbents build AI products which are not core to their main business?

Abishek is a Product Manager at Clarifai, a leading NYC-based computer vision startup, that has raised $40M with leading VCs such as USV.

Here is in short his reaction:

ML could be used for a lot of different applications (building domain understanding, recommendation engine, computer vision, language, etc.). If a company (non tech/large data) finds applications for more than one of those, but that isn’t necessarily its core competency (not their business), should they be encouraged to build that capability in house to achieve the 80% accuracy/satisfaction level?

I think this is a very interesting question, and I obviously don’t have the answer.

What I am thinking is the following:

• If this is core business, the likelihood that they will build it is high. A good example is the motorway company I mentioned in my post that tries to detect truck’s sizes at the toll based on computer vision. Collecting payment is clearly core business for them. They have much more initial data than a new startup entering the field, and with many more generic ML API (such as Clarifai) available, I think it might not be too difficult to build an algorithm with 80+% accuracy.
• If it’s not core business, I think it’s more likely that a startup could win because the ROI on kickstarting such a project might not be that evident for the incumbent. I think photo moderation for real estate portal or social media is a good example here.

In a less recent post, I reviewed the number of successful Applied AI companies that were tech enabler (vs. full stack), and then looked at those that were horizontal (in this situation, think non-core business for the customer) vs. vertical (core business for the customer). I found out that most of them were horizontal: