Valohai Developer Core
Juha KiiliThe Wright brothers - the inventors of modern aviation - created the first wind tunnels out of an old bicycle and scrap metal tubes. Linus Torvalds - the creator of Linux - couldn't make version-control products scale for his project and gave birth to Git. Léon Bottou and Yann LeCun - the fathers of deep learning - got tired of writing C++ and developed the programming language Lush which later evolved into Torch and PyTorch.
They were all pioneers.
A pioneer is someone who builds something unique, a first mover. But there is another more profound commonality. While pioneering aviation, operating systems, and deep learning, they are also toolsmiths. The bicycle wind tunnel, the Git version control system, and the Lush programming language are not the work they are best known for but tools and prerequisites to their pioneering success.
Linus Torvalds with Linux. Photo: Jim Sugar / Corbis Documentary
The tradeoff between exploitation and exploration is as old as life itself. Exploit what is already there or explore for something better? Settle for the off-the-shelf tool or innovate a new one? Buy it or build it?
In a pioneering organization, the answer is often to build some of it out of necessity. The pioneers ask computer questions no one dared to ask before, finding themselves with unique combinations of new technologies where the off-the-shelf tools come short.
Valohai Developer Core: Agnostic and Unopinionated
Valohai is the only genuinely agnostic and unopinionated MLOps platform tailored for the pioneering enterprise. Whatever environments, technologies, hardware, cloud, languages, libraries, or tools need to be glued together, Valohai is built to be ready for it.
Firstly, Valohai is not tied to cloud providers, data sources, deep learning libraries, or programming languages. We are fully unopinionated by design. Want to run a hybrid of on-premise and cloud? Need to synthesize images using a 3D engine like Unity? Want to hook into a 3rd party labeling tool like Labelbox or Sama? Need to trigger pipelines from a CI/CD tool like Jenkins? With Valohai, everything goes!
Secondly, agnosticism is achieved by being an API-first product. The development of every feature in Valohai follows the same recipe: A fully functional REST API is crafted first, and then everything else follows. Our internals (Web UI and CLI) always use the same API as our customers, which ensures throughout dogfooding but also makes the platform customizable and integration with 3rd party systems straightforward.
In an extreme case, our entire UI can be replaced if needed. It's why Twitter and Meta chose Valohai as their infrastructure partner for organizing the NeurIPS Black-Box competition to train a mind-blowing 277M models in a few months. Users uploaded, monitored, and scored experiments with a web UI custom-built for the competition, while the organizers monitored the competition behind the scenes with our classic UI.
Check out our blog post to dig deeper.
The architecture of the Valohai-powered NeurIPS Black-Box Competition 2020
The Pioneers’ way is unpredictable
The world of state-of-the-art deep learning is moving fast, and today's approaches will surely be obsolete in a few years, if not sooner.
The same can be said for many platforms and tools out there. A pioneering organization should avoid locking itself into a single cloud provider, technology, or idea. The smart pioneer avoids anyone claiming all the answers upfront and instead looks for someone prepared for the unknown.
The pioneers' journey will be full of sidestepping, u-turns, and zigzags. No organization can exactly know where they'll end up, but we will help you to get there. Valohai developer core will keep all paths open, just in case you need to change course.
Want to know more about Valohai?
This article is part of a series of blog posts: