Package an entire Kedro project¶
This section explains how to build your project documentation, and how to bundle your entire project into a Python package.
Kedro also has an advanced feature which supports packaging on a pipeline level allowing you share and reuse pipelines across projects! To read more about this please look at the section on micro-packaging.
Add documentation to your project¶
Kedro uses the Sphinx framework and creates a
docs directory that builds a basic template for project-specific documentation. We recommend that you add your project-specific documentation as markdown in
If you want to customise your documentation beyond the basic template, refer to the Sphinx documentation for details of how to extend
Once you have added any documentation you need, run the following from the project root directory:
kedro build-docs --open
The HTML documention is built to
docs/build/html and opens automatically in a browser tab.
build-docs command creates documentation based on the code structure of your project. Documentation includes any
docstrings defined in your code.
Package your project¶
To package your project, run the following in your project root directory:
Kedro builds the package into the
dist folder of your project, and creates one
.egg file and one
.whl file, which are Python packaging formats for binary distribution.
The resulting package only contains the Python source code of your Kedro pipeline, not any of the
logs subfolders. This means that you can distribute the project to run elsewhere, such as on a separate computer with different configuration information, dataset and logging locations.
We recommend that you document the configuration required (parameters and catalog) in the local
README.md file for any project recipients.
Recipients of the
.whl files need to have Python and
pip on their machines, but do not need to have Kedro installed.
A recipient can install the project by calling:
pip install <path-to-wheel-file>
kedro-tutorial, is placed in the
bin subfolder of the Python install folder, so the project can be run as follows:
python -m kedro_tutorial
The recipient will need to add a
conf subfolder. They also need to add
logs if the pipeline loads/saves local data or uses logging.
Once your project is installed, to run your pipelines from any Python code, simply import it:
from kedro_tutorial.__main__ import main main( ["--pipeline", "__default__"] ) # or simply main() if you don't want to provide any arguments
This is equivalent to running
kedro run, and you can provide all the parameters described by
kedro run --help.
Docker, Airflow and other deployment targets¶
There are various methods to deploy packaged pipelines via Kedro plugins: