Deployment¶

Overview¶

In this section we provide guides for different deployment methods; your choice will depend on a range of factors.

If you decide to deploy your Kedro project onto a single machine, you should consult our guide to single-machine deployment, and decide whether to:

• use Docker for container-based deployment
• use package-based deployment
• use the CLI to clone and deploy your codebase to a server

If your pipeline is sizeable, you may want to run it across separate machines, so will need to consult our guide to distributed deployment.

flowchart TD
    A{Can your Kedro pipeline run on a single machine?} -- YES --> B[Consult the single-machine deployment guide]
    B --> C{Do you have Docker on your machine?}
    C -- YES --> D[Use a container-based approach]
    C -- NO --> E[Use the CLI or package mode]
    A -- NO --> F[Consult the distributed deployment guide]
    F --> G["What distributed platform are you using?<br/><br/>Check out the guides for:<br/><br/><li>Airflow</li><li>Amazon SageMaker</li><li>AWS Step functions</li><li>Azure</li><li>Dask</li><li>Databricks</li><li>Kubeflow Pipelines</li><li>Prefect</li><li>Vertex AI</li>"]

Deployment methods¶

The following pages provide information for deployment to, or integration with, the following:

• Airflow
• Amazon SageMaker
• Amazon EMR Serverless
• AWS Step functions
• Azure
• Dagster
• Dask
• Databricks
• Kubeflow Pipelines
• Prefect
• Vertex AI

Effective node grouping for deployment¶

Effective node grouping makes deployments easier to manage and update. It also improves performance by using resources more efficiently and enables pipelines to handle larger datasets as they scale across different environments. To learn more about grouping nodes using pipelines, tags, and namespaces, follow our detailed guide:

• Node Grouping in Kedro