AI Diary (#2) — PyTorch Developer Conference 2018 Edition

In this entry I share all the important announcements made at the PyTorch developer conference and the interactions I had with some of the folks at the conference. Even if you are not an avid user of PyTorch you will still be able to appreciate the changes and developments since they reflect a wide and general demand from the deep learning and machine learning community.

Destination: San Francisco ✈️

Event: PyTorch Developer Conference 2018 (PTDC-2018)

MISSION

Last week I decided to attend the first PyTorch developer conference (PTDC-2018 for short); I traveled from Taipei to San Francisco — a whopping 10,351 km (13+ hour flight) to find out more about the culture and community fueling PyTorch — an extraordinary deep learning framework that has grown tremendously over the last 2 years since its inception.

EXPECTATIONS

Upon landing in San Francisco, where the conference was to be held, I didn’t know what to expect. Prior to the conference, I had only engaged with a few PyTorch developers but no one from the core team or even the companies that actually use PyTorch internally. I was interested to hear what everyone had to say or announce.

I was very excited for this conference unlike any other developer or even research conference I had attended before. The reason being that I am really interested to see how PyTorch keeps growing and where it’s headed — something that is basically dictated by industry leaders, researchers, and community developers like us. So it’s good to hear what they had to say in their presentations and in person.

FIRST IMPRESSION

In fact, in the panel interview, someone asked Soumith Chintala what type of features do he intend to push forward in the next coming years. He simply answered “I do what the community wants me to do.” The most sincere and humble answer any person can give, and rightly so because that is what the community deserves for committing so much time and effort to this project.

I arrived at the venue, and I was already impressed with the great organization and theme of the conference. I checked in and the first person that I saw was Soumith Chintala — the founder of PyTorch. I noticed that he was engaging with a lot of people and my first thought was, “seems like a friendly guy.” This was further validated by his humble gesture to decide to approach and greet me. He was the first person I spoke to at the conference upon my arrival at the venue, and in the short conversation we had, he seemed like the down to earth guy that I assumed he would be — that’s comforting, as the success of projects depend a lot on the people that lead projects. He is the right guy for it in my opinion.

MY FUTURE CONTRIBUTION

He also introduced me to their documentation engineer as I have a deep interest in helping out with the education aspect of PyTorch and other deep learning frameworks. I also do the same with TensorFlow, which I also love. I am excited for this and I hope that I can help the community in this regard.

CONFERENCE BEGINS

After a couple cups of decaf, the main conference finally began. The morning session mostly contained talks by internal developers and project managers of Facebook and how they are using PyTorch to build, improve and enhance their products. Some speakers also discussed their research projects. This gave me the impression that PyTorch is not only for the industry but also for the research community — a nice balance that can only bring good things to the PyTorch team in the future, in my own opinion.

If feels like the entire industry wants a say in the development and growth of PyTorch — that gives me the impression that PyTorch has a bright future. For instance, I heard that Microsoft is dedicating entire internal teams to help accelerate the growth of PyTorch. I am excited to see how this plays out. Obviously, every company has their own interest taken care of first before dedicating efforts to an open source software. However, I will give Microsoft and the other players the benefit of the doubt — I think they mean good for the community and I expect great contributions from them in the future.

ANNOUNCEMENTS

Other notable announcements related to the PyTorch 1.0 preview release include:

• PyText, a library that helps launch NLP projects into production mode, will be released soon (in the next month or so).
• A Google team is working hard to integrate Tensorboard with PyTorch.
• You will eventually be able to use PyTorch on TPUs. How awesome is that!
• ONNX has been integrated as the main export format, which means PyTorch 1.0 models will be inter-operable with other deep learning frameworks. This is perhaps the most important announcement from the PyTorch developer team.
• Big cloud service providers for deep learning tools, such as Microsoft Azure, Google Cloud, and Amazon’s SageMaker, now support the latest stable version of PyTorch.
• The just-in-time (JIT) compiler will allow PyTorch models to efficiently be exported and be used in production mode. In addition, the JIT compiler will allow models to exported in a C++ runtime based on Caffe2 bits. PyTorch offers the computation graph in static mode, similar to TensorFlow, in production since it speeds up computations significantly while the eager execution (dynamic graph) is left as the default mode.
• GPyTorch, a tool used to build efficient Gaussian processes in PyTorch, has been released in beta mode.
• Microsoft announced that they have fully integrated the accessibility of PyTorch docs directly on VS Code.

TAKEAWAYS IN A TWEET

Overall, I am deeply impressed by this vibrant and young community, and I can only say positive things about what I saw and personally experienced throughout the duration of the event. Great organization. Impressive projects. Amazing Leadership! And most importantly, phenomenal community. So much that I hope that I can make it to the upcoming PTDC.

OTHER USEFUL LINKS, RESOURCES, AND ANNOUNCEMENTS

NEXT UP

In the next upcoming entry of the AI Diary, I will discuss some of the recent advancements in NLP related to dialogue systems. I discuss the future of NLP and AI, and some interesting ML papers, including works that explore and test the capabilities of word embeddings in very unnatural and interesting ways.