Kestra

Kestra is an open-source, event-driven orchestration platform for building, scheduling and operating workflows using a declarative YAML model. It provides an API-first experience and a web UI that keep workflows as code while enabling visual inspection, iterative testing and execution.

Key Features

• Declarative YAML workflows with inputs, variables, subflows, conditional branching, retries, timeouts and backfills
• Event-driven and scheduled triggers (webhooks, message buses, file events, CRON/advanced schedules) with millisecond latency support
• Rich plugin ecosystem and task runners to run code in any language (Python, Node.js, R, Go, shell, custom containers) and connect to databases, cloud services and message brokers
• Built-in web UI with code editor (syntax highlight, autocompletion, topology/DAG view), execution logs, dashboards and a Playground mode for iterative task testing
• API-first design, Git/version-control integration and Terraform provider for Infrastructure-as-Code and CI/CD workflows
• Scalable, fault-tolerant architecture with workers, executors and support for containerized and Kubernetes deployments

Use Cases

• Data pipeline orchestration: scheduled ETL/ELT, batch and streaming data workflows, integration with databases and cloud storage
• ML/AI and model pipelines: orchestrate preprocessing, training, validation and deployment steps across compute runners
• Infrastructure and business automation: orchestrate provisioning, service orchestration, webhooks and event-driven automation across teams

Limitations and Considerations

• Advanced governance features (SSO, RBAC, multi-tenant enterprise controls) are provided in commercial/Enterprise offerings rather than the core open-source distribution
• Frontend editing capabilities (interactive drag-and-drop flow editing) are evolving; some UI graph editing features are currently limited and under active development
• Plugin coverage varies by integration; teams building uncommon integrations may need to implement or maintain custom plugins

Kestra combines an Everything-as-Code approach with a feature-rich UI and extensible plugin model to unify orchestration across data, infra and application workflows. It is designed for teams that need both developer-grade reproducibility and operational observability in workflow automation.