Random hyperparameter optimizationAarni Koskela
Valohai now supports random search for hyperparameter optimization (which we call the Tasks feature), which has been proven in the aptly named paper Random search for hyper-parameter optimization to be an efficient way to find “neighborhoods” of likely-to-be-optimal hyperparameter values, which can then be iterated further to find the really good values.
This is a valuable tool to add to the previously existing linear, logarithmic and multiple-value hyperparameter optimizers. Valohai uses a seeded Mersenne Twister random number generator to generate values in a given range and supports both an uniform distribution as well as a truncated normal distribution of values.
You can try this feature out today; simply create a Task for your parameter-enabled step and choose the Random option.