- Kale – Aims at simplifying the Data Science experience of deploying Kubeflow Pipelines workflows.
- Flyte – Easy to create concurrent, scalable, and maintainable workflows for machine learning.
- MLRun – Generic mechanism for data scientists to build, run, and monitor ML tasks and pipelines.
- Prefect – A workflow management system, designed for modern infrastructure.
- ZenML – An extensible open-source MLOps framework to create reproducible pipelines.
- Argo – Open source container-native workflow engine for orchestrating parallel jobs on Kubernetes.
- Kedro – Library that implements software engineering best-practice for data and ML pipelines.
- Luigi – Python module that helps you build complex pipelines of batch jobs.
- Metaflow – Human-friendly lib that helps scientists and engineers build and manage data science projects.
- Couler – Unified interface for constructing and managing workflows on different workflow engines.
- Valohai – Simple and powerful tool to train, evaluate and deploy models.
- Dagster.io – Data orchestrator for machine learning, analytics, and ETL.
- Netflix Genie – Genie developed by Netflix is an open-source distributed workflow/task orchestration framework.
https://github.com/pditommaso/awesome-pipeline
https://neptune.ai/blog/best-workflow-and-pipeline-orchestration-tools
