Date
Share
alain_riou

Alain Riou

Sony CSL and Télécom Paris

Alain Riou is a PhD student at Télécom Paris and Sony CSL. His studies have mostly been about computer science and machine learning and is also very interested in music-related topics. He is currently working on deep learning methods for controllable music generation, with a strong focus on self-supervised learning approaches.

Automatic and Distributed Hyperparameter Optimization

Deep learning algorithms often involve a lot of hyperparameters that can strongly influence the final performance of the network. Even worse, some approaches (e.g. GANs, reinforcement learning…) may completely fail with a wrong set of hyperparameters. Properly selecting hyperparameters may therefore be critical, but manual hyperparameter optimization is in practice extremely costly, boring and time-consuming. The purpose of this presentation is to give an overview of methods that can be used to automate and facilitate this task. It will introduce hyperparameter search algorithms and early-stopping mechanisms that can strongly accelerate hyperparameter optimization, as well as tools for performing efficient and distributed optimization over multiple GPUs or nodes without the need of human monitoring.