kestra Documentation

Event Driven Orchestration & Scheduling Platform for Mission Critical Applications

27,789 stars Java Java 1 file · ~28,354 tokens #ai-agents#automation#control-plane#data-engineering#data-orchestration#data-orchestrator#devops#etl
RAW Doc

Docker & Self-Hosted Deployment Verified OCI

Deploy kestra on any server, homelab, or cloud container platform.

1-Click Terminal Command
bash (docker run) 
docker run -d --name kestra -p 8080:8080 --restart always kestra/kestra:latest
Configured Services & Ports
⚙️ postgres
image: postgres:18
⚙️ kestra
image: kestra/kestra:latest
🔌 Ports: 8080:8080
docker-compose.yml
yaml (docker-compose.yml) 
volumes:
  postgres-data:
    driver: local
  kestra-data:
    driver: local

services:
  postgres:
    image: postgres:18
    volumes:
      - postgres-data:/var/lib/postgresql
    environment:
      POSTGRES_DB: kestra
      POSTGRES_USER: kestra
      POSTGRES_PASSWORD: k3str4
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
      interval: 30s
      timeout: 10s
      retries: 10

  kestra:
    image: kestra/kestra:latest
    pull_policy: always
    # Kestra, by default, has a termination grace period of 5m. We need to wait a little more to be sure no active tasks are running.
    stop_grace_period: 6m
    # Note that this setup with a root user is intended for development purpose.
    # Our base image runs without root, but the Docker Compose implementation needs root to access the Docker socket
    user: "root"
    command: server standalone
    volumes:
      - kestra-data:/app/storage
      - /var/run/docker.sock:/var/run/docker.sock
      - /tmp/kestra-wd:/tmp/kestra-wd
    environment:
      KESTRA_CONFIGURATION: |
        datasources:
          postgres:
            url: jdbc:postgresql://postgres:5432/kestra
            driverClassName: org.postgresql.Driver
            username: kestra
            password: k3str4
        kestra:
          # server:
          #   basic-auth:
          #     username: [email protected] # it must be a valid email address
          #     password: Admin1234! # it must be at least 8 characters long with uppercase letter and a number
          repository:
            type: postgres
          storage:
            type: local
            local:
              base-path: "/app/storage"
          queue:
            type: postgres
          tasks:
            tmp-dir:
              path: /tmp/kestra-wd/tmp
          url: http://localhost:8080/
    ports:
      - "8080:8080"
      # The following is the management port, not exposed by default.
      # Note that exposing the management port is intended for development purpose as this port is un-authenticated by default
      # and expose sensitive information and actions.
      # - "8081:8081"
    depends_on:
      postgres:
        condition: service_started

1. Project Overview & Quickstart (kestra-io/kestra)

File: README.md

Open-source orchestration platform for data, AI, and infrastructure workflows

🌟 What is Kestra?

Kestra is an open-source, event-driven orchestration platform for data, AI, and infrastructure workflows. It unifies scheduled and event-driven automation behind a declarative, language-agnostic interface. By bringing Infrastructure as Code best practices to your data, process, and microservice pipelines, you can build reliable workflows directly from the UI in just a few lines of YAML.

📖 Table of Contents

Key Features:

  • Everything as Code and from the UI: keep workflows as code with a Git Version Control integration, even when building them from the UI.
  • Event-Driven & Scheduled Workflows: automate both scheduled and real-time event-driven workflows via a simple trigger definition.
  • Declarative YAML Interface: define workflows using a simple configuration in the built-in code editor.
  • Rich Plugin Ecosystem: hundreds of plugins built in to extract data from any database, cloud storage, or API, and run scripts in any language.
  • Intuitive UI & Code Editor: build and visualize workflows directly from the UI with syntax highlighting, auto-completion and real-time syntax validation.
  • Scalable: designed to handle millions of workflows, with high availability and fault tolerance.
  • Version Control Friendly: write your workflows from the built-in code Editor and push them to your preferred Git branch directly from Kestra, enabling best practices with CI/CD pipelines and version control systems.
  • Structure & Resilience: tame chaos and bring resilience to your workflows with namespaces, labels, subflows, retries, timeout, error handling, inputs, outputs that generate artifacts in the UI, variables, conditional branching, advanced scheduling, event triggers, backfills, dynamic tasks, sequential and parallel tasks, and skip tasks or triggers when needed by setting the flag disabled to true.

🧑‍💻 The YAML definition gets automatically adjusted any time you make changes to a workflow from the UI or via an API call. Therefore, the orchestration logic is always managed declaratively in code, even if you modify your workflows in other ways (UI, CI/CD, Terraform, API calls).

🚀 Quick Start

Launch on AWS (CloudFormation)

Deploy Kestra on AWS using our CloudFormation template:

Launch on Google Cloud (Terraform deployment)

Deploy Kestra on Google Cloud Infrastructure Manager using our Terraform module.

Get Started Locally in 5 Minutes

Launch Kestra in Docker

Make sure that Docker is running. Then, start Kestra in a single command:

bash
docker run --pull=always -it -p 8080:8080 --user=root \
  --name kestra --restart=always \
  -v kestra_data:/app/storage \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /tmp:/tmp \
  kestra/kestra:latest server local

If you're on Windows and use PowerShell:

powershell
docker run --pull=always -it -p 8080:8080 --user=root `
  --name kestra --restart=always `
  -v "kestra_data:/app/storage" `
  -v "/var/run/docker.sock:/var/run/docker.sock" `
  -v "C:/Temp:/tmp" `
  kestra/kestra:latest server local

If you're on Windows and use Command Prompt (CMD):

cmd
docker run --pull=always -it -p 8080:8080 --user=root ^
  --name kestra --restart=always ^
  -v "kestra_data:/app/storage" ^
  -v "/var/run/docker.sock:/var/run/docker.sock" ^
  -v "C:/Temp:/tmp" ^
  kestra/kestra:latest server local

If you're on Windows and use WSL (Linux-based environment in Windows):

bash
docker run --pull=always -it -p 8080:8080 --user=root \
  --name kestra --restart=always \
  -v kestra_data:/app/storage \
  -v "/var/run/docker.sock:/var/run/docker.sock" \
  -v "/mnt/c/Temp:/tmp" \
  kestra/kestra:latest server local

Check our Installation Guide for other deployment options (Docker Compose, Podman, Kubernetes, AWS, GCP, Azure, and more).

Access the Kestra UI at http://localhost:8080 and start building your first flow!

Your First Hello World Flow

Create a new flow with the following content:

yaml
id: hello_world
namespace: dev

tasks:
  - id: say_hello
    type: io.kestra.plugin.core.log.Log
    message: "Hello, World!"

Run the flow and see the output in the UI!

🧩 Plugin Ecosystem

Kestra's functionality is extended through a rich ecosystem of plugins that empower you to run tasks anywhere and code in any language, including Python, Node.js, R, Go, Shell, and more. Here's how Kestra plugins enhance your workflows:

  • Run Anywhere:

    • Local or Remote Execution: Execute tasks on your local machine, remote servers via SSH, or scale out to serverless containers using Task Runners.
    • Docker and Kubernetes Support: Seamlessly run Docker containers within your workflows or launch Kubernetes jobs to handle compute-intensive workloads.

  • Code in Any Language:

    • Scripting Support: Write scripts in your preferred programming language. Kestra supports Python, Node.js, R, Go, Shell, and others, allowing you to integrate existing codebases and deployment patterns.
    • Flexible Automation: Execute shell commands, run SQL queries against various databases, and make HTTP requests to interact with APIs.

  • Event-Driven and Real-Time Processing:

    • Real-Time Triggers: React to events from external systems in real-time, such as file arrivals, new messages in message buses (Kafka, Redis, Pulsar, AMQP, MQTT, NATS, AWS SQS, Google Pub/Sub, Azure Event Hubs), and more.
    • Custom Events: Define custom events to trigger flows based on specific conditions or external signals, enabling highly responsive workflows.

  • Cloud Integrations:

    • AWS, Google Cloud, Azure: Integrate with a variety of cloud services to interact with storage solutions, messaging systems, compute resources, and more.
    • Big Data Processing: Run big data processing tasks using tools like Apache Spark or interact with analytics platforms like Google BigQuery.

  • Monitoring and Notifications:

    • Stay Informed: Send messages to Slack channels, email notifications, or trigger alerts in PagerDuty to keep your team updated on workflow statuses.

Kestra's plugin ecosystem is continually expanding, allowing you to tailor the platform to your specific needs. Whether you're orchestrating complex data pipelines, automating scripts across multiple environments, or integrating with cloud services, there's likely a plugin to assist. And if not, you can always build your own plugins to extend Kestra's capabilities.

🧑‍💻 Note: This is just a glimpse of what Kestra plugins can do. Explore the full list on our Plugins Page.

📚 Key Concepts

  • Flows: the core unit in Kestra, representing a workflow composed of tasks.
  • Tasks: individual units of work, such as running a script, moving data, or calling an API.
  • Namespaces: logical grouping of flows for organization and isolation.
  • Triggers: schedule or events that initiate the execution of flows.
  • Inputs & Variables: parameters and dynamic data passed into flows and tasks.

🎨 Build Workflows Visually

Kestra provides an intuitive UI that allows you to interactively build and visualize your workflows:

  • Drag-and-Drop Interface: add and rearrange tasks from the Topology Editor.
  • Real-Time Validation: instant feedback on your workflow's syntax and structure to catch errors early.
  • Auto-Completion: smart suggestions as you type to write flow code quickly and without syntax errors.
  • Live Topology View: see your workflow as a Directed Acyclic Graph (DAG) that updates in real-time.

🔧 Extensible and Developer-Friendly

Plugin Development

Create custom plugins to extend Kestra's capabilities. Check out our Plugin Developer Guide to get started.

Infrastructure as Code

  • Version Control: store your flows in Git repositories.
  • CI/CD Integration: automate deployment of flows using CI/CD pipelines.
  • Terraform Provider: manage Kestra resources with the official Terraform provider.

🌐 Join the Community

Stay connected and get support:

  • Slack: Join our Slack community to ask questions and share ideas.
  • LinkedIn: Follow us on LinkedIn — next to Slack and GitHub, this is our main channel to share updates and product announcements.
  • YouTube: Subscribe to our YouTube channel for educational video content. We publish new videos every week!
  • X: Follow us on X if you're still active there.

🤝 Contributing

We welcome contributions of all kinds!

📄 License

Kestra is licensed under the Apache 2.0 License © Kestra Technologies.

⭐️ Stay Updated

Give our repository a star to stay informed about the latest features and updates!

Thank you for considering Kestra for your workflow orchestration needs. We can't wait to see what you'll build!

File: jmh-benchmarks/README.md

JMH-Benchmarks module

This module contains benchmarks written using JMH from OpenJDK.

Running Benchmarks

To run all benchmarks

bash
./gradlew jmh

To run a specific benchmark

bash
./gradlew jmh -Pjmh.include=io.kestra.core.utils.MapUtilsBenchmark

File: ui/src/assets/docs/basic.md

Flow properties

Kestra lets you define and automate flows using simple YAML syntax. Each flow requires an id, namespace, and tasks. Other fields like triggers, labels, inputs, and more are optional.

Expand to read more about each of these flow properties.

Property Description
id The flow identifier which represents the name of the flow. This ID must be unique within a namespace and is immutable (you cannot rename the flow ID later, but you can recreate it with a new name).
namespace Each flow lives in one namespace. Namespaces are used to group flows and provide structure. Some concepts in Kestra, such as Namespace Files or KV Store are tied to a namespace. You cannot change a flow’s namespace after creation; create a new flow with the desired namespace and delete the old one.
tasks The list of tasks to be executed. Tasks are atomic actions in your flows. By default, they will run sequentially one after the other. However, you can use additional Flowable tasks to run some tasks in parallel.
inputs The list of strongly-typed inputs that allow you to make your flows more dynamic and reusable. Instead of hardcoding values in your flow, you can use inputs to run multiple Executions of your flow with different values determined at runtime. Use the syntax {{ inputs.your_input_name }} to access specific input values in your tasks.
outputs Each flow can produce outputs that can be consumed by other flows. This is a list property, so that your flow can produce as many outputs as you need. Each output needs to have an id (the name of the output), a type (the same types you know from inputs e.g. STRING, URI or JSON) and value which is the actual output value that will be stored in internal storage and passed to other flows when needed.
labels Key-value pairs that you can use to organize your flows based on your project, maintainers, or any other criteria. You can use labels to filter Executions in the UI.
description The description of the flow.
checks checks are flow-level assertions evaluated when validating inputs and before creating a new execution.
variables The list of variables (such as API endpoints, table names, download URLs, etc.) that you can access within tasks using the syntax {{ vars.your_variable_name }}. Variables help reuse some values across tasks.
sla The list of SLA conditions specifying an execution behavior if the workflow doesn't satisfy the assertion defined in the SLA.
errors The list of error tasks that will run if there is an error in the current execution.
finally The list of finally tasks that will run after the workflow is complete. These tasks will run regardless of whether the workflow was successful or not.
afterExecution The list of afterExecution tasks that will run after the execution finishes, regardless of the final state. These tasks will run after execution of the workflow reaches a final state, including the execution of the tasks from the finally block.
disabled Set it to true to temporarily disable any new executions of the flow. This is useful when you want to stop a flow from running (even manually) without deleting it. Once you set this property to true, nobody will be able to create any execution of that flow, whether from the UI or via an API call, until the flow is re-enabled by setting this property back to false (default behavior) or by deleting this property.
revision The flow version, managed internally by Kestra, and incremented upon each modification. You should not manually set it.
triggers The list of triggers which automatically start a flow execution based on events, such as a scheduled date, a new file arrival, a new message in a queue, or the completion event of another flow's execution.
pluginDefaults The list of default values, allowing you to avoid repeating the same plugin properties. Using values, you can set the default properties. The type is a full qualified Java class name, e.g. io.kestra.plugin.core.log.Log, but you can use a prefix e.g. io.kestra to apply some properties to all tasks. If forced is set to true, the pluginDefault will take precedence over properties defined in the task (the default behavior is forced: false).
concurrency Control the number of concurrent executions of a given flow by setting the limit key. Executions beyond that limit will be queued by default — you can customize that by configuring the behavior property which can be set to QUEUE (default), CANCEL or FAIL.
retry Set a flow-level retry policy to restart the execution if any task fails. The retry behavior is customizable — you can choose to CREATE_NEW_EXECUTION or RETRY_FAILED_TASK (default). Only with the CREATE_NEW_EXECUTION behavior, the attempt of the execution is incremented. Otherwise, only the failed task run is restarted (incrementing the attempt of the task run rather than the execution). Apart from the behavior property, the retry policy is identical to task retries.

Plugin documentation

To inspect properties of a specific plugin, click anywhere in that task, trigger or plugin code within the editor. The task documentation will load in this view.

Task properties

Each flow consists of tasks. The following core properties are available to all tasks.

Property Description
id A unique identifier of the task
type A full Java class name that represents the type of the task
description Your custom documentation of what the task does
retry How often should the task be retried in case of a failure, and the type of retry strategy
timeout The maximum time allowed for the task to complete
when Skip a task if the provided condition evaluates to false
disabled A boolean flag indicating whether the task is disabled or not; if set to true, the task will be skipped during the execution
workerGroup The group of workers (EE-only) that are eligible to execute the task; you can specify a workerGroup.key and a workerGroup.fallback (the default is WAIT)
allowFailure A boolean flag allowing to continue the execution even if this task fails
allowWarning A boolean flag allowing to mark a task run as Success despite Warnings
logLevel The level of log detail to be stored.
logToFile A boolean that lets you store logs as a file in internal storage. That file can be previewed and downloaded from the Logs and Gantt Execution tabs. When set to true, logs aren’t saved in the database, which is useful for tasks that produce a large amount of logs that would otherwise take up too much space. The same property can be set on triggers.

Input types

Inputs in Kestra are strongly typed. Each type defines how values are entered, validated, and rendered in the UI. Below are all supported input types and their validation rules.

Property Description
STRING Any string value. Values are passed without parsing. You can add a validator regex for custom validation.
INT Integer without decimals. Supports min and max validation to enforce numeric ranges. Example: 42.
FLOAT Floating-point number with decimals. Supports min and max validation. Example: 3.14.
BOOL Boolean flag, must be true or false. Avoid scalar equivalents such as yes/no, as the API and UI expect true or false.
FORM Groups related inputs under a shared displayName in the UI. Child inputs are defined in inputs and resolved as nested paths — e.g. a region child of an environment form is accessed as {{ inputs.environment.region }}. Use full dotted paths in dependsOn conditions. Forms cannot be nested and cannot declare defaults or prefill.
SELECT Single value chosen from a predefined list, either static values or a dynamic list defined via an expression, which can render values using kv() or http() functions. Supports allowCustomValue to let user enter a custom value and autoSelectFirst to preselect the first item.
MULTISELECT One or more values chosen from a predefined list, either static values or a dynamic list defined via an expression, which can render values using kv() or http() functions. Supports allowCustomValue to let user enter a custom value and autoSelectFirst to preselect the first item.
DATE ISO-8601 date (YYYY-MM-DD). Supports after and before validation to enforce valid ranges. Example: 2042-12-28.
TIME ISO-8601 time (HH:MM:SS) without timezone. Supports after and before validation. Example: 10:15:30.
DATETIME ISO-8601 datetime with timezone in UTC format (YYYY-MM-DDTHH:MM:SSZ). Supports after and before validation. Example: 2042-04-02T04:20:42.000Z.
DURATION ISO-8601 duration string, e.g. PT5M6S. Supports min and max duration bounds.
FILE File upload via multipart/form-data. Uploaded files are stored in Kestra’s internal storage and exposed as kestra:///... URIs. You can pass Namespace Files as defaults using the protocol: defaults: nsfile:///hello.txt.
JSON Valid JSON string, parsed into a typed object. Example: {"name": "kestra"}.
YAML Valid YAML string, parsed into a typed object. Example: - user: John.
URI Valid URI string. Example: https://kestra.io/docs.
SECRET Encrypted string stored in the database. Decrypted at runtime and masked in UI/execution context. Requires an encryption key.
ARRAY JSON array or YAML list. Must define itemType (e.g., INT, STRING, DATETIME). Each item is validated against its type. Example defaults: [1, 2, 3] or YAML list.

Input properties

All input types share a common set of properties. These define how inputs behave, whether they are required, and how they appear in the UI.

Property Description
id Unique identifier for the input. Used to access the input at runtime e.g. {{ inputs.myInput }}.
type The input type, chosen from the supported list e.g. STRING, INT, FLOAT, etc.
required Whether the input must be provided. If required: true, the execution must have a value, either provided at runtime or via defaults.
defaults Default value to use if no value is provided at runtime. Must match the declared type.
dependsOn Makes this input dependent on an array of inputs with optional condition to show inputs in the UI only when needed.
displayName Human-readable label for the input in the UI.
description Markdown description rendered in the UI when executing the flow.
expression Pebble expression to fetch dynamic values e.g. using kv() or http() functions. Commonly used with SELECT or MULTISELECT.
autoSelectFirst If true, auto-selects the first option from a dropdown when no default is set (applies to SELECT and MULTISELECT).

Flow example

Here's an example flow showing how to use labels, inputs, variables, triggers and description.

text
/* Detailed source-code truncated for AI context efficiency. */

You can document flows, tasks, inputs, or triggers with the description property. These descriptions are rendered in the UI using Markdown.

Pebble templating

Kestra has a Pebble templating engine allowing you to dynamically render variables, inputs and outputs within the execution context using Pebble expressions. For example, the {{ flow.namespace }} expression allows accessing the namespace of the current flow and the {{ printContext() }} function allows you to print the entire context of the execution, which is useful for debugging.

Pebble functions, filters and tags

Filter Example and Description
abs {{ -7 | abs }} — Returns the absolute value of -7, resulting in 7.
abbreviate {{ "this is a long sentence." | abbreviate(7) }} — Shortens a string using an ellipsis. The length includes the ellipsis. Can be chained with other filters: {{ "apple" | upper | abbreviate(3) }}.
appLink {{ appLink('yourAppId') }} — Fetch the URL of the App (EE-only) linked to the current Execution. To get the base URL of the app allowing to create new Executions, add baseUrl=true e.g. {{ appLink('yourAppId', baseUrl=true) }}. If there is only one App linked to the flow, you can skip the App ID argument e.g. {{ appLink() }}.
base64Decode {{ "aGVsbG8=" | base64Decode }} — Decodes the base64 string, resulting in "hello".
base64decode {{ "dGVzdA==" | base64decode }} — Decodes a base64-encoded string into UTF-8.
base64Encode {{ "hello" | base64Encode }} — Encodes the string in base64, resulting in "aGVsbG8=".
base64encode {{ "test" | base64encode }} — Encodes a string to base64.
block {{ block("post") }} — Renders the contents of the "post" block in a template {% block header %} Introduction {% endblock %}
capitalize {{ "hello" | capitalize }} — Capitalizes the first letter, resulting in "Hello".
for {% for user in users %}{{ loop.index }} - {{ user.id }}{% endfor %} — Iterates over a list of values in a template. Supports else block for empty collections: {% for user in users %} ... {% else %} ... {% endfor %}.
format Hello %s | format('World') — format a string.
if {% if users is empty %} ... {% endif %} — Conditional block based on an expression in a template. Supports multiple branches: {% if category == "news" %} ... {% elseif category == "sports" %} ... {% else %} ... {% endif %}.
macro {% macro input(type, name) %} ... {% endmacro %} — Creates a reusable template fragment. Invoke like a function: {{ input("text", name="Mitchell") }}.
raw {% raw %}{{ user.name }}{% endraw %} — Writes a block of syntax that won't be parsed in a template.
set {% set header = "Test Page" %} — Defines a variable in the current context in a template.
chunk {{ [1, 2, 3, 4] | chunk(2) }} — Splits the list into chunks of size 2, resulting in [[1, 2], [3, 4]].
className {{ "12.3" | number | className }} — Returns the class name of an object.
contains {{ ["apple", "pear", "banana"] contains "apple" }} — Checks if a collection contains a particular item.
date {{ execution.startDate | date("yyyy-MM-dd") }} — Formats the date as "yyyy-MM-dd". Can use named arguments: {{ stringDate | date('yyyy/MMMM/d', existingFormat='yyyy-MMMM-d') }}. Can format timestamps with timezone: {{ 1378653552123 | date(format="iso_milli", timeZone="Europe/Paris") }}.
dateAdd {{ execution.startDate | dateAdd(1, "DAYS") }} — Adds 1 day to the date. Can be used with null-coalescing: {{ trigger.date ?? execution.startDate | dateAdd(-1, 'DAYS') }}.
dayOfMonth {{ dayOfMonth('2025-01-15') }} — Returns the day of the month as an integer (1–31). Also accepts datetime strings: {{ dayOfMonth('2025-01-15T10:00:00Z') }}.
dayOfWeek {{ dayOfWeek('2025-01-06') }} — Returns the day-of-week name as an uppercase string (e.g. MONDAY, SUNDAY). Locale-independent alternative to date | date('EEEE').
default {{ myVar | default("default value") }} — Returns "default value" if myVar is null or empty.
distinct {{ ['1', '1', '2', '3'] | distinct }} — Returns a list of unique elements, resulting in [1, 2, 3].
escapeChar {{ "Can't be here" | escapeChar('single') }} — Escapes special characters in a string.
env {{ env('ENV_NAME', 'default') }} — Returns an environment variable exposed in the execution context, or the optional default value when it is missing or empty.
errorLogs {{ errorLogs() }} — Prints all error logs from the current execution.
fileExists {{ fileExists(output.download.uri) }} — Returns true if file is present at the given uri location.
fileSize {{ fileSize(output.download.uri) }} — Returns the size of the file present at the given uri location.
filter {% filter upper %}hello{% endfilter %} — Applies a filter to a chunk of template content.
first {{ [1, 2, 3] | first }} — Returns the first element of the list, resulting in 1.
flatten {{ [[1, 2], [3, 4]] | flatten }} — Flattens a nested list, resulting in [1, 2, 3, 4].
fromIon {{ fromIon(read(someItem)).someField }} — Converts an ION string to an object and accesses its properties.
fromJson {{ fromJson('{"foo": [666, 1, 2]}').foo[0] }} — Converts a JSON string to an object and accesses its properties.
hourOfDay {{ hourOfDay('2025-01-06T14:30:00Z') }} — Returns the hour of day as an integer (0–23). Accepts ISO 8601 datetime strings with or without timezone offset. The hour is taken from the local time as written; no UTC normalization is performed.
http {{ http(uri = 'https://dummyjson.com/products/categories') }} — Fetches data from an external API directly.
indent {{ "Hello\nworld" | indent(4) }} — Adds 4 spaces before each line except the first, resulting in "Hello\n world".
isDayWeekInMonth {{ isDayWeekInMonth('2025-01-06', 'MONDAY', 'FIRST') }} — Returns true if the given date is the Nth occurrence of the specified weekday in its month. position accepts FIRST, SECOND, THIRD, FOURTH, or LAST. Example for last Friday: {{ isDayWeekInMonth('2025-01-31', 'FRIDAY', 'LAST') }}.
isFileEmpty {{ isFileEmpty(output.download.uri) }} — Returns true if file present at the given uri location is empty.
isIn {{ execution.state isIn ['SUCCESS', 'KILLED', 'CANCELLED'] }} — Returns true if the value on the left is present in the list on the right. Useful for conditions such as runIf.
isLastWorkingDay {{ isLastWorkingDay('2025-01-31') }} — Returns true if the given date is the last working day of its month (Monday–Friday by default). An optional second argument overrides which days count as working days: {{ isLastWorkingDay('2025-01-30', 'MONDAY,TUESDAY,WEDNESDAY,THURSDAY') }}.
isPublicHoliday {{ isPublicHoliday('2024-12-25', 'FR') }} — Returns true if the given date is a public holiday in the specified country (ISO 3166-1 alpha-2 code). Backed by the Jollyday library (70+ countries). An optional third argument narrows the check to a sub-division: {{ isPublicHoliday('2024-12-25', 'FR', 'IDF') }}.
isWeekend {{ isWeekend('2025-01-04') }} — Returns true if the given date falls on Saturday or Sunday.
is defined {% if missing is not defined %} ... {% endif %} — Checks if a variable is defined in a template.
is empty {% if user.email is empty %} ... {% endif %} — Checks if a variable is empty in a template.
is even {% if 2 is even %} ... {% endif %} — Checks if an integer is even in a template.
is iterable {% if users is iterable %} ... {% endif %} — Checks if a variable is iterable in a template.
is json {% if '{"test": 1}' is json %} ... {% endif %} — Checks if a variable is a valid JSON string in a template.
is map {% if {"apple":"red", "banana":"yellow"} is map %} ... {% endif %} — Checks if a variable is a map in a template.
is null {% if user.email is null %} ... {% endif %} — Checks if a variable is null in a template.
is not even {% if 3 is not even %} ... {% endif %} — Negates a boolean expression in a template.
is odd {% if 3 is odd %} ... {% endif %} — Checks if an integer is odd in a template.
join {{ ["a", "b", "c"] | join(",") }} — Joins the list into a string, resulting in "a,b,c".
jq {{ myObject | jq(".foo") }} — Applies JQ expression to extract the "foo" property from myObject.
keys {{ {"a": 1, "b": 2} | keys }} — Returns the keys of the map, resulting in ["a", "b"].
kv By default, retrieves a KV pair from the current namespace: {{ kv('MY_KEY') }}. If a namespace is provided as second argument, retrieves from that namespace: {{ kv('MY_KEY', 'company.team') }}. Can also use named parameters to specify error handling: {{ kv(key='KEY_ID', namespace='NAMESPACE_ID', errorOnMissing=false) }} returns null instead of error when key is missing.
last {{ [1, 2, 3] | last }} — Returns the last element of the list, resulting in 3.
length {{ "Hello" | length }} — Returns the length of "Hello", which is 5.
loopOutputs {{ loopOutputs(outputs.myLoop.outputs, 'uri') }} — Extracts the value of a named output from every iteration of a Loop task, returning a list in iteration order. Useful to collect a specific output (e.g. a file URI or a row count) produced by a task inside the loop across all iterations. If an iteration has no value for the given name, null is included at that position.
lower {{ "HELLO" | lower }} — Converts the string to lowercase, resulting in "hello".
max {{ max(user.age, 80) }} — Returns the largest of its numerical arguments.
md5 {{ "hello" | md5 }} — Computes the MD5 hash of the string.
merge {{ [1, 2] | merge([3, 4]) }} — Merges two lists, resulting in [1, 2, 3, 4].
min {{ min(user.age, 80) }} — Returns the smallest of its numerical arguments.
monthOfYear {{ monthOfYear('2025-06-01') }} — Returns the month of the year as an integer (1–12). Also accepts datetime strings: {{ monthOfYear('2025-06-01T10:00:00Z') }}.
nindent {{ "Hello\nworld" | nindent(4) }} — Adds a newline and then 4 spaces before each line, resulting in "\n Hello\n world".
number {{ "123" | number }} — Parses the string "123" into the number 123.
numberFormat {{ 12345.6789 | numberFormat("###,###.##") }} — Formats the number 12345.6789 as "12,345.68".
now {{ now() }} — Returns the current datetime. Can specify timezone: {{ now(timeZone='Europe/Paris') }} or format: {{ now(format='sql_milli') }}.
parent {{ parent() }} — Renders the content of the parent block.
printContext {{ printContext() }} — Prints the entire context of the execution, useful for debugging.
randomInt {{ randomInt(1, 10) }} — Generates a random integer from a specified range.
randomPort {{ randomPort() }} — Generate a random available port.
range {{ range(0, 3) }} — Generates a list from 0 to 3.
read {{ read('subdir/file.txt') }} — Reads an internal storage file and returns its content as a string.
render {{ render(namespace.github.token) }} — Recursively renders the variable containing Pebble expressions.
renderOnce {{ renderOnce(expression_string) }} — Renders nested Pebble expressions only once.
replace {{ "Hello world!" | replace({'world': 'Kestra'}) }} — Replaces "world" with "Kestra", resulting in "Hello, Kestra!".
reverse {{ [1, 2, 3] | reverse }} — Reverses the list, resulting in [3, 2, 1].
rsort {{ [3, 1, 2] | rsort }} — Sorts the list in reverse order, resulting in [3, 2, 1].
secret {{ secret('MY_SECRET') }} — Retrieves secret MY_SECRET. Pass namespace= to read a secret from another namespace, e.g. {{ secret('MY_SECRET', namespace='other.namespace') }}.
sha1 {{ "hello" | sha1 }} — Computes the SHA-1 hash of the string.
sha256 {{ "hello" | sha256 }} — Computes the SHA-256 hash of the string.
sha512 {{ "hello" | sha512 }} — Computes the SHA-512 hash of the string.
slice {{ "Hello, world!" | slice(0, 5) }} — Extracts a substring, resulting in "Hello".
slugify {{ "Hello World!" | slugify }} — Converts a string to a URL-friendly format.
sort {{ [3, 1, 2] | sort }} — Sorts the list in ascending order, resulting in [1, 2, 3].
split {{ "a,b,c" | split(",") }} — Splits the string into a list, resulting in ["a", "b", "c"].
startsWith {{ "hello world" | startsWith("hello") }} — Checks if a string starts with a given prefix.
string {{ 123 | string }} — Converts 123 into a string.
subflow {{ subflow(namespace='company.team', id='my_subflow', inputs={'k': 'v'}).outputs.my_output }} — Synchronously runs a subflow and returns its terminal execution, so you can read its outputs (e.g. to populate a SELECT input's values).
substringAfter {{ "a.b.c" | substringAfter(".") }} — Extracts the substring after the first occurrence of a separator.
substringAfterLast {{ "a.b.c" | substringAfterLast(".") }} — Extracts the substring after the last occurrence of a separator.
substringBefore {{ "a.b.c" | substringBefore(".") }} — Extracts the substring before the first occurrence of a separator.
substringBeforeLast {{ "a.b.c" | substringBeforeLast(".") }} — Extracts the substring before the last occurrence of a separator.
tasksWithState {{ tasksWithState('failed') }} — Returns a map of tasks and their states.
timestamp {{ execution.startDate | timestamp }} — Converts the date to a Unix timestamp in seconds.
timestampMicro {{ execution.startDate | timestampMicro }} — Converts the date to a Unix timestamp in microseconds.
timestampMilli {{ execution.startDate | timestampMilli }} — Converts the date to a Unix timestamp in milliseconds.
timestampNano {{ execution.startDate | timestampNano }} — Converts the date to a Unix timestamp in nanoseconds.
title {{ "article title" | title }} — Capitalizes the first letter of each word.
toIon {{ myObject | toIon }} — Converts myObject into an ION string.
toJson {{ myObject | toJson }} — Converts myObject into a JSON string.
trim {{ " Hello " | trim }} — Trims leading and trailing whitespace, resulting in "Hello".
upper {{ "hello" | upper }} — Converts the string to uppercase, resulting in "HELLO". Can be chained with other filters: {{ "When life gives you lemons, make lemonade." | upper | abbreviate(13) }}.
urlDecode {{ "a%20b" | urlDecode }} — URL decodes the string, resulting in "a b".
urlEncode {{ "a b" | urlEncode }} — URL encodes the string, resulting in "a%20b".
uuid {{ uuid() }} — Generates a UUID in the Kestra format (i.e., a UUID encoded in Url62).
values {{ {'foo': 'bar', 'baz': 'qux'} | values }} — Retrieves the values from a map.
decrypt {{ decrypt(key='MY_KEY', encrypted=outputs.cipher.value) }} — Decrypts an AES-encrypted value using the provided key.
encrypt {{ encrypt(key='MY_KEY', plaintext='hello') }} — Encrypts a plaintext value using the provided key (AES).
endsWith {{ "hello world" | endsWith("world") }} — Checks if a string ends with a given suffix.
fetchContext {{ fetchContext() }} — Returns the entire execution context as a map. Alias of printContext.
fileURI {{ fileURI(path='subdir/file.txt') }} — Returns the internal-storage URI of a namespace file. Optional namespace and version arguments.
ksuid {{ ksuid() }} — Generates a K-Sortable Unique IDentifier (timestamp-prefixed, base62-encoded).
nanoId {{ nanoId() }} — Generates a compact URL-friendly unique ID. Default length is 21 characters.
nl2br {{ "line1\nline2" | nl2br }} — Converts newline characters into HTML `` tags.
regexExtract {{ "order-12345-done" | regexExtract(regex="\\d+") }} — Returns the first substring matching the regex, or null if no match. Optional group argument selects a capture group (0 = whole match).
regexMatch {{ "abc-123" | regexMatch(regex="\\d+") }} — Returns true if the input contains a substring matching the regex (find() semantics, not full-string match).
regexReplace {{ "hello world" | regexReplace(regex="world", replacement="kestra") }} — Replaces every regex match. Replacement string can reference capture groups via $1, $2, ….
urldecode {{ "a%20b" | urldecode }} — URL decodes the string. Lowercase alias of urlDecode.
urlencode {{ "a b" | urlencode }} — URL encodes the string. Lowercase alias of urlEncode.
yaml {{ yaml('foo: [666, 1, 2]').foo[0] }} — Converts a YAML string to an object and accesses its properties.

Common Pebble expressions

Expression Description
{{ flow.id }} The identifier of the flow.
{{ flow.namespace }} The name of the flow namespace.
{{ flow.tenantId }} The identifier of the tenant.
{{ flow.revision }} The revision of the flow.
{{ execution.id }} The execution ID, a generated unique id for each execution.
{{ execution.startDate }} The start date of the current execution, can be formatted with {{ execution.startDate | date('yyyy-MM-dd HH:mm:ss.SSSSSS') }}.
{{ execution.endDate }} The end date of the current execution, can be formatted with {{ execution.endDate | date('yyyy-MM-dd HH:mm:ss.SSSSSS') }}.
{{ execution.originalId }} The original execution ID, this id will never change even in case of replay and keep the first execution ID.
{{ execution.outputs }} The outputs of the execution as defined in the flow outputs, only populated when the execution is terminated (finally or afterExecution block).
{{ execution.state }} The current execution state (e.g. RUNNING, SUCCESS, FAILED, KILLED).
{{ task.id }} The current task ID.
{{ task.type }} The current task Type (Java fully qualified class name).
{{ taskrun.id }} The current task run ID.
{{ taskrun.startDate }} The current task run start date.
{{ taskrun.parentId }} The current task run parent identifier. Only available with tasks inside a Flowable Task.
{{ taskrun.value }} The value of the current task run, only available with tasks inside in Flowable Tasks.
{{ taskrun.iteration }} The index of the current task run iteration, only available with tasks inside in Flowable Tasks.
{{ taskrun.attemptsCount }} The number of attempts for the current task (when retry or restart is performed).
{{ parent.taskrun.value }} The value of the closest (first) parent task run, only available with tasks inside a Flowable Task.
{{ parent.task.id }} The task id of the closest (first) parent task run Flowable Task, only available with tasks wrapped in a Flowable Task.
{{ parents }} The list of parent tasks, only available with tasks wrapped in a Flowable Task.
{{ item }} The current iteration item, only available within ForEach/EachParallel/EachSequential flowable tasks.
{{ item.value }} The value of the current iteration item.
{{ item.key }} The key of the current iteration item (when iterating over a map).
{{ item.index }} The zero-based index of the current iteration item.
{{ item.parent }} The parent iteration item, only available when nested inside another ForEach-style flowable.
{{ item.parent.value }} The value of the parent iteration item.
{{ item.parent.key }} The key of the parent iteration item.
{{ item.parent.index }} The zero-based index of the parent iteration item.
{{ item.parents }} The list of all ancestor iteration items, ordered from outermost to innermost.
{{ labels }} The executions labels accessible by keys, for example: {{ labels.myKey1 }} returns the value of the myKey1 label.
{{ trigger.date }} The date of the current schedule.
{{ trigger.next }} The date of the next schedule.
{{ trigger.previous }} The date of the previous schedule.
{{ trigger.executionId }} The ID of the execution that triggers the current flow.
{{ trigger.namespace }} The namespace of the flow that triggers the current flow.
{{ trigger.flowId }} The ID of the flow that triggers the current flow.
{{ trigger.flowRevision }} The revision of the flow that triggers the current flow.
{{ trigger._context.id }} The ID of the trigger that executes the current flow.
{{ trigger._context.type }} The type of the trigger that executes the current flow.
{{ envs.foo }} Accesses environment variable ENV_FOO (by default prefixed with ENV_).
{{ kestra.environment }} Accesses Environment variables such as kestra.environment.name. Must be set in your configuration to be accessible.
{{ kestra.url }} Accesses Environment URL variable. Must be set in your configuration to be accessible.
{{ globals.foo }} Accesses global variable foo.
{{ vars.my_variable }} Accesses flow variable my_variable.
{{ inputs.myInput }} Accesses flow input myInput.
{{ namespace.env.name }} Accesses namespace variable env.name.
{{ outputs.taskId.data }} Accesses task output attribute called data.
{{ tasks.taskId.state }} Accesses the runtime state of a sibling task by id, e.g. SUCCESS, FAILED, SKIPPED.
{{ files.myFile }} Accesses a file made available by the execution (e.g. file inputs, internal storage files).
{{ "apple" ~ "pear" ~ "banana" }} Concatenates multiple strings, resulting in "applepearbanana".
{{ foo == null ? bar : 42 }} Returns bar if foo is null, otherwise returns 42.
{{ foo ?? bar ?? baz }} Returns the first non-null value: foo if defined, otherwise bar, otherwise baz.
{# THIS IS A COMMENT #} Adds a comment that won't appear in the output.
{{ foo.bar }} Accesses a child attribute of a variable, e.g., if foo = {"bar": "value"}, returns "value".
{{ foo['my-key'] }} Uses subscript notation to access attributes with special characters, e.g. use foo['my-key'] instead of foo.my-key.
{{ "Hello #{who}" }} String interpolation within a literal, e.g., if who = "World", returns "Hello World".

File: ui/src/assets/docs/dashboard_home.md

Custom Dashboards as Code

Build custom dashboards to track workflow executions, logs and metrics filtered by specific namespaces or labels and showing data you care about.

The declarative syntax allows you to manage dashboards as code — you can version control that dashboard definition alongside your flows.

Example

Below is an example dashboard definition that displays:

  • time series of executions over time
  • table with specific metrics aggregated per namespace
  • table with logs aggregated per log level and namespace.
yaml
title: Getting Started
timeWindow:
  default: P7D
  max: P365D
charts:
  - id: executions_timeseries
    type: io.kestra.plugin.core.dashboard.chart.TimeSeries
    chartOptions:
      displayName: Executions
      legend:
        enabled: true
      column: date
      colorByColumn: state
    data:
      type: io.kestra.plugin.core.dashboard.data.Executions
      columns:
        date:
          field: START_DATE
          displayName: Date
        state:
          field: STATE
        total:
          displayName: Executions
          agg: COUNT
          graphStyle: BARS
        duration:
          displayName: Duration
          field: DURATION
          agg: SUM
          graphStyle: LINES

  - id: table_metrics
    type: io.kestra.plugin.core.dashboard.chart.Table
    chartOptions:
      displayName: Sum of sales per namespace
    data:
      type: io.kestra.plugin.core.dashboard.data.Metrics
      columns:
        namespace:
          field: NAMESPACE
        value:
          field: VALUE
          agg: SUM
      where:
        - field: NAME
          type: EQUAL_TO
          value: sales_count
        - field: NAMESPACE
          type: IN
          values:
            - dev_graph
            - prod_graph
      orderBy:
        - column: value
          order: DESC

    - id: table_logs
    type: io.kestra.plugin.core.dashboard.chart.Table
    chartOptions:
      displayName: Log count by level for filtered namespace
    data:
      type: io.kestra.plugin.core.dashboard.data.Logs
      columns:
        level:
          field: LEVEL
        count:
          agg: COUNT
      where:
        - field: NAMESPACE
          type: IN
          values:
            - dev_graph
            - prod_graph

For more examples, check our GitHub repository and explore Dashboard Blueprints.

Querying data

The data property of a chart defines the type of data that is queried. The type determines which columns are displayed.

Dashboards can query data from these source types:

  • type: io.kestra.plugin.core.dashboard.data.Executions: data related to your workflow executions
  • type: io.kestra.plugin.core.dashboard.data.Logs: logs produced by your executions
  • type: io.kestra.plugin.core.dashboard.data.Metrics: metrics emitted during executions.

After defining the data source, specify the columns to display in the chart. Each column is defined by the field and may include additional optional properties.

Property Description
field The only required field, specifies the name of the column in the data source to use
displayName Sets the label displayed in the chart
agg Defines the aggregation function applied to the column: supported aggregations include AVG, COUNT, MAX, MIN, SUM
graphStyle Indicates the style of the graph displayed: supported styles include LINES, BARS, POINTS
columnAlignment Specifies the alignment of the column in the table: supported alignments include LEFT, RIGHT, CENTER

You can also use the where property to set conditions that filter the result set before displaying it in the chart. Filters can apply to any column in the data source. For shared logic, use the AND operator in the where property to combine several conditions. If multiple conditions are needed with a different logic, use the type: OR property.

Available filter types include:

  • CONTAINS
  • ENDS_WITH
  • EQUAL_TO
  • GREATER_THAN
  • GREATER_THAN_OR_EQUAL_TO
  • IN
  • IS_FALSE
  • IS_NOT_NULL
  • IS_NULL
  • IS_TRUE
  • LESS_THAN
  • LESS_THAN_OR_EQUAL_TO
  • NOT_EQUAL_TO
  • NOT_IN
  • OR
  • REGEX
  • STARTS_WITH.

Available field types include the following columns:

  • ATTEMPT_NUMBER
  • DATE
  • DURATION
  • END_DATE
  • EXECUTION_ID
  • FLOW_ID
  • FLOW_REVISION
  • ID
  • LABELS
  • LEVEL
  • MESSAGE
  • NAME
  • NAMESPACE
  • START_DATE
  • STATE
  • TASK_ID
  • TASK_RUN_ID
  • TRIGGER_ID
  • TYPE
  • VALUE.

Note that some of the above are reserved only for specific types of data e.g., the LEVEL column is only available for type: io.kestra.plugin.core.dashboard.data.Logs.

File: ui/src/components/no-code/README.md

Guide on How to Handle the `NoCode` Editor

Code within Vue components must be written using the <script setup> block, as enforced by the ESLint rule.

This is a brief guide on working with the NoCode editor component.

File: ui/README.md

Kestra UI

Kestra UI is running using Vite.

INSTRUCTIONS

Development:

  • (Optional) By default, your dev server will target localhost:8080. If your backend is running elsewhere, you can create .env.development.local under ui folder with this content:
text
VITE_APP_API_URL={myApiUrl}

  • Navigate into the ui folder and run npm install to install the dependencies for the frontend project.

  • Now go to the cli/src/main/resources folder and create a application-override.yml file.

Now you have two choices:

Local mode:

Runs the Kestra server in local mode which uses a H2 database, so this is the only config you'd need:

yaml
micronaut:
  server:
    cors:
      enabled: true
      configurations:
        all:
          allowedOrigins:
            - http://localhost:5173

You can then open a new terminal and run the following command to start the backend server: ./gradlew runLocal

Standalone mode:

Runs in standalone mode which uses Postgres. Make sure to have a local Postgres instance already running on localhost:

yaml
kestra:
    type: postgres
  storage:
    type: local
    local:
      base-path: "/app/storage"
  queue:
    type: postgres
  tasks:
    tmp-dir:
      path: /tmp/kestra-wd/tmp
  anonymous-usage-report:
    enabled: false

datasources:
  postgres:
    # It is important to note that you must use the "host.docker.internal" host when connecting to a docker container outside of your devcontainer as attempting to use localhost will only point back to this devcontainer.
    url: jdbc:postgresql://host.docker.internal:5432/kestra
    driverClassName: org.postgresql.Driver
    username: kestra
    password: k3str4

flyway:
  datasources:
    postgres:
      enabled: true
      locations:
        - classpath:migrations/postgres
      # We must ignore missing migrations as we may delete the wrong ones or delete those that are not used anymore.
      ignore-migration-patterns: "*:missing,*:future"
      out-of-order: true

micronaut:
  server:
    cors:
      enabled: true
      configurations:
        all:
          allowedOrigins:
            - http://localhost:5173

If you're doing frontend development, you can run npm run dev from the ui folder after having the above running (which will provide a backend) to access your application from localhost:5173. This has the benefit to watch your changes and hot-reload upon doing frontend changes.

Translations

The UI is translated into thirteen languages, with English as the source of truth and every other locale generated from it. How the pipeline works - generation, fingerprints, checks, CI - is documented in scripts/translations/README.md.

2. Official Technical Reference & Guides (kestra-io/docs)

Event-Driven Declarative Orchestrator

Kestra Documentation

Kestra is an open-source infinitely-scalable orchestration platform enabling all engineers to manage business-critical workflows declaratively in code.

Documentation

The official Kestra documentation can be found under kestra.io/docs.

Local development of the Kestra Docs

To run the docs locally, you must have Node.js and NPM installed. Then, run the following commands:

bash
npm install
npm run dev

You can access the docs at localhost:4321.

Contributing Tips

Ensure that all links, including images, are relative.

Troubleshooting tips

Depending on your Node.js and OS version, you may encounter the following error: Error message "error:0308010C:digital envelope routines::unsupported".

To resolve this issue, you will need to switch to the OpenSSL legacy provider by executing the following command: export NODE_OPTIONS=--openssl-legacy-provider.

If you are using an Apple Silicon Mac, please ensure that you are using Node.js version 20 or higher.

License

Apache 2.0 © Kestra Technologies

Stay up to date

We release new versions on a monthly basis. To stay informed about the latest releases and receive notifications for future updates, please consider starring our main repository.

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

#ai-agents Browse ai-agents catalog → #automation Browse automation catalog → #control-plane Browse control-plane catalog → #data-engineering Browse data-engineering catalog → #data-orchestration Browse data-orchestration catalog → #data-orchestrator Browse data-orchestrator catalog → ⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →