Valohai, the MLOps platform company, is collaborating with Twitter and Facebook to launch a competition for the annual The Neural Information Processing Systems (NeurIPS) conference to advance the optimization of machine learning models towards more accurate AI solutions . The goal is to find better optimization algorithms for machine learning.
Our joint challenge is called Black-Box Optimization for Machine Learning where competitors are trying to find the best method for optimizing the black-box functions arising from tuning machine learning models. Competitors input the configuration of the search space for the algorithms, but everything else about the objectives remains hidden.
The competitors are encouraged to elaborate on the open-source optimization algorithm such as Bayesian optimization and evolutionary algorithms. One of the fundamental challenges in training a machine learning model is choosing the right hyperparameters. The estimation of hyperparameters is supposed to improve based on the knowledge gained during the several training iterations.
The competition runs on top of the Valohai API
Valohai, together with Twitter, built a dedicated website for participating and seeing how competitors fare against others throughout the competition. The platform is built on top of the Valohai Machine Learning platform to trigger the black box each time the submission is sent.
The Black-Box Optimization competition website is a custom-built lightweight layer on top of the Valohai platform and its API . While the user interface for competitors is simple and straightforward, what happens under the hood is a more complicated story. Every submission injects hundreds of jobs into an execution queue consumed by several cloud worker instances orchestrated by the Valohai auto-scaling system.
Each job runs a slightly different experiment to measure the optimizer's performance, and the results are read constantly back to the competition website for real-time scoring. Participants can stop the runs at any time midway. All the inputs, logs, metrics, and outputs for every new submission are automatically stored for the post-competition analysis, and each run is reproducible.
Do you want to participate?
The competition lasts from July 1st to October 15th. Individuals and teams can take part and send the submissions at any time between the competition dates. There will be a $10 000 total in prizes divided between the TOP 3 teams or individuals. The winner will receive a $5000 prize, second place is rewarded with $3000, and third place with $2000.
Go to the competition website bbochallenge.com
What is NeurIPS?
The Neural Information Processing Systems (NeurIPS) is a research-focused annual conference gathering together with the brightest minds in the machine learning (ML) industry to share the research on neural information processing systems in their biological, technological, mathematical, and theoretical aspects. The conference was first held in 1987 and, for the last few years, there has been a Competition Track where ML practitioners can join to solve the most pressing current challenges in ML development.