Our eBook Practical MLOps is coming soon. Sign up to get a preview edition today.
MLOps Platforms Compared
In this article, we compare popular MLOps platforms, both managed and open-source.
Comparing machine learning and MLOps platforms is incredibly tricky as these products are complex. Generally, the differences between platforms can only be fully realized with real-world testing with an actual use-case. The marketing messaging for these platforms is very similar so getting clear differentiation is difficult (note to self, think different 😉).
A common way to compare platforms is to compare features. Many of these platforms are quite identical in top-line features, but how the features work in practice vary wildly. Therefore we’ve chosen to compare the platforms in how they position themselves.
Traditional machine learning focus vs. deep learning focus
Products that focus on traditional machine learning are built for structured data (SQL, Excel, etc.) and efficient processing through, for example, Spark. These platforms may also offer additional capabilities for data analysis and data manipulation in visual tools.
On the other hand, MLOps platforms for deep learning are built to handle massive amounts of unstructured data such as images, videos, or audio. Significant emphasis is placed on utilizing powerful GPU machines as these models may take days to train even on the most powerful hardware.
On the traditional machine learning side, we have products like MLFlow built by Databricks – authors of Spark – while Valohai sits on the deep learning side with a heavy focus on machine orchestration.
Exploration focus vs. productization focus
Generally, MLOps, as a concept, is focused on machine learning production.
Exploration focused platforms emphasize data analytics, experiment tracking, and working in notebooks, while productization focused platforms primarily concentrate on machine learning pipelines, automation, and model deployment.
Algorithmia, Flyte, and Metaflow are most strictly production focused. Flyte and Metaflow focus on building production pipelines while Algorithmia is for model versioning and model deployment only, not training models. MLFlow, on the other hand, is more focused on experiment tracking, and Dataiku has a lot to offer on the data analysis side of things.
Citizen data scientist focus vs. expert data scientist
Citizen data scientists are more subject matter experts rather than technical experts. Some MLOps platforms are focused on this category of users as they offer capabilities for teams with less engineering expertise to build and deploy machine learning models. These platforms focus on visual tools and access through a Web UI.
On the other end, we have platforms focused on expert data scientists with engineering expertise or teams that contain significant data science and engineering expertise. These platforms tend to avoid proprietary code as much as possible and focus on supporting as many existing languages and frameworks as possible. Web UI may not be as big of a focus as expert users tend to opt for command-line interface (CLI) or API when integrating the platform with existing tools.
Datarobot roots for the citizen data scientist with its heavy emphasis on AutoML, while open-source platforms like Flyte, Metaflow, and Kubeflow are more suited for large teams of expert data scientists with deep engineering/DevOps skills. Most managed platforms fall somewhere between without requiring the same DevOps as the open-source platforms. Valohai and cnvrg.io, however, put a heavier emphasis on remaining technology agnostic and interoperable.
Specialized approach vs. end-to-end approach
Most of the MLOps platforms listed in this article approach MLOps from an end-to-end perspective, meaning the user should be able to automatically train, evaluate, and deploy a model in a single platform. For a real apples to apples comparison, those with a specialized approach should be excluded, but we included a few that often come up in MLOps platform evaluations.
There is a place for specialized products, but you’ll generally need to complement them with other products to form an end-to-end MLOps platform.
Algorithmia, Flyte, and Metaflow standout in this comparison as they are more narrowly focused in either pipelines or deployment. Datarobot is not genuinely end-to-end except in AutoML use cases, and MLFlow is only starting to move to an end-to-end approach.
|Algorithmia||Managed||We help enterprise companies develop an optimal path to machine learning operational maturity.||Enterprise, deployment|
|Allegro AI||Managed, Open-source||End-to-end enterprise-grade platform for data scientists, data engineers, DevOps and managers to manage the entire machine learning & deep learning product life-cycle.||Enterprise, Data management|
|cnvrg.io||Managed, Open-source||An end-to-end machine learning platform to build and deploy AI models at scale||Technology agnostic|
|Dataiku||Managed||Dataiku is the platform democratizing access to data and enabling enterprises to build their own path to AI in a human-centric way.||Enterprise, Data Analysis, Business Intelligence|
|Datarobot||Managed||DataRobot is the leading end-to-end enterprise AI platform that automates and accelerates every step of your path from data to value.||AutoML, Enterprise|
|Iguazio||Managed, Open-source||The Iguazio Data Science Platform automates MLOps with end-to-end machine learning pipelines, transforming AI projects into real-world business outcomes.||Structured data|
|Valohai||Managed||Train, Evaluate, Deploy, Repeat. Valohai is the MLOps platform that can automate everything from data extraction to model deployment.||Deep Learning, API-first, Technology agnostic|
|Flyte||Open-source||Lyft’s Cloud Native Machine Learning and Data Processing Platform, Now Open Sourced||Pipelines|
|Kubeflow||Open-source||The Kubeflow project is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable, and scalable.||Community, Extendability|
|Metaflow||Open-source||A framework for real-life data science||Pipelines|
|MLFlow||Open-source||MLflow is an open-source platform for managing the end-to-end machine learning lifecycle.||Experimentation, Spark|
Which platform works best for your use-case?
The right MLOps platform, in the end, comes down to your specific use case and also your particular strengths. If you evaluate MLOps platforms, you may want to consider the scales presented above and figure out which end you lean on each. You should also figure out if there is an essential scale for you, which we didn’t consider. There are 1001 different ways of making comparisons.
Bear in mind, this comparison was compiled in October 2020, and all of these platforms are continually evolving in features and market positioning. For any feedback, don’t hesitate to email us at firstname.lastname@example.org.
For more comparisons between Valohai and specific platforms, see: