Room: Zoom

8:30-9:00 Arrivals

9:00-9:10 Welcome, 

9:10-10:00 Overview of GPyZhenwen Dai, Spotify [slides]

10:00-10:30 Coffee Break

10:30-11:20 Challenges in building widely applicable GP softwareAki Vehtari, Aalto University [slides]

11:20-12:10 Overview of Secondmind's opensource toolboxesHenry Moss, Secondmind, John McLeod, Secondmind, Nicolas Durrande, Secondmind

12:10-14:00 Lunch

14:00-14:50 Reliable Deep Learning with Edward2 & Uncertainty BaselinesZachary Nado, Google Brain, Dustin Tran, Google Brain [slides]

14:50-15:40 Bayesian Optimization: From Research to Production with BoTorch & AxMax Balandat, Facebook [slides]

15:40-16:00 Tea Break

16:00-16:30 DiscussionJavier Gonzalez, Microsoft Research Cambridge

Abstracts

Reliable Deep Learning with Edward2 & Uncertainty Baselines

Abstract

Deep learning models are applied to complex, real-world scenarios and yet we have little handle of how well they work in those scenarios. This is problematic in safety-critical areas such as healthcare, and it’s more broadly important when the model is deployed to serve predictions on data very different from what the model was trained on. In this talk, we’ll describe our team’s progress at Google toward reliable deep learning: its challenges, advances, and how we designed infrastructure to accelerate research in the area. First, we’ll discuss how we use the Edward2 probabilistic programming language to design uncertainty models. Second, we’ll talk about Uncertainty Baselines, a library for managing experiments related to uncertainty and robustness. We have used these libraries internally and externally across dozens of papers and several product launches.

Bayesian Optimization: From Research to Production with BoTorch & Ax

Abstract

Black-box optimization problems are ubiquitous in many practical settings: At Facebook, they include AutoML, optimizing ranking policies in large scale online A/B tests, tuning backend infrastructure, AI hardware/software co-design, simulation optimization, and many others. The Adaptive Experimentation team at Facebook maintains BoTorch and Ax, two open-source python libraries for Bayesian Optimization that are widely used both internally and externally. In this talk I will explain the origins and the respective goals of BoTorch and Ax, their features and capabilities, design principles, and how they enable the team to conduct novel research and effectively translate it to production. I will discuss the tension between catering to different target audiences (researchers as well as practitioners), and how software engineering is done in practice in an applied research team. Finally, I will highlight a recent example use-case that leverages both methodological and systems innovations to efficiently perform automated latency-aware neural architecture search with multi-objective Bayesian optimization.