Kubeflow

The Kubeflow project is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable. Our goal is not to recreate other services, but to provide a straightforward way to deploy best-of-breed open-source systems for ML to diverse infrastructures. Anywhere you are running Kubernetes, you should be able to run Kubeflow.

Why is this technology in trial?

Google backs Kubeflow, and pipelines can be deployed to GCP Vertex AI via CLI. Kubeflow can be deployed on any kubernetes cluster, so you can lift and shift your pipelines to Azure AKS or AWS EKS. Kubeflow Pipeline component inputs & outputs are traced and cached automatically on GCS or any other cloud storage system - better default behaviour for ML Engineering than Gitlab build pipelines.