Deep learning algorithms often involve a lot of hyperparameters that can strongly influence the final performance of the network. Even worse, some approaches (e.g. GANs, reinforcement learning…) may completely fail with a wrong set of hyperparameters. Properly selecting hyperparameters may therefore be critical, but manual hyperparameter optimization is in practice extremely costly, boring and time-consuming. The purpose of this presentation is to give an overview of methods that can be used to automate and facilitate this task. It will introduce hyperparameter search algorithms and early-stopping mechanisms that can strongly accelerate hyperparameter optimization, as well as tools for performing efficient and distributed optimization over multiple GPUs or nodes without the need of human monitoring.