Random hyperparameter optimization

Aarni Koskela / December 03, 2018

Valohai now supports random search for hyperparameter optimization (which we call the Tasks feature), which has been proven in the aptly named paper Random search for hyper-parameter optimization to be an efficient way to find “neighborhoods” of likely-to-be-optimal hyperparameter values, which can then be iterated further to find the really good values.

Random hyperparameter search for machine learning experiments

This is a valuable tool to add to the previously existing linear, logarithmic and multiple-value hyperparameter optimizers. Valohai uses a seeded Mersenne Twister random number generator to generate values in a given range and supports both an uniform distribution as well as a truncated normal distribution of values.

You can try this feature out today; simply create a Task for your parameter-enabled step and choose the Random option.

MLOps Ebook

Free eBook

Practical MLOps

Learn what MLOps is all about and how MLOps helps you avoid the deadlock between machine learning and operations. This eBook gives an overview of why MLOps matters and how you should think about implementing it as a standard practice.