Define testable, modular Python dataflows with built-in lineage and metadata.
Build modular ETL, ML workflows, and LLM pipelines using standard Python functions and automatic DAG creation.
Apache Hamilton is a lightweight Python library for building directed acyclic graphs of data transformations using standard functions. It suits data scientists and engineers who want to organize code into testable, self-documenting pipelines that run anywhere Python runs.
It automatically builds a DAG from regular Python functions where parameters specify dependencies. Function modifiers keep code dry, and data adapters handle schema validation for pandas, polars, and Ibis.
It suits data teams transitioning proof-of-concept models into maintainable production pipelines without rigid orchestrator lock-in. It does not suit those looking for an orchestrator to schedule jobs or users who need loops and conditional agent logic.
Separates DAG definition and execution so data scientists and engineers can collaborate effectively.
Uses @config.when() to modify DAGs between environments instead of error-prone if/else flags.
Decorate functions to validate output properties and automatically inspect dataframe schemas.
Automatically populates a data catalog with lineage, tracing, and execution observability.
Free under the Apache-2.0 licence, with the only cost being your own hosting resources if you run the Hamilton UI server.
880 open remote AI-training roles, updated November 15, 2026; most pay $20–$175 an hour. Zekai lists them free; the application goes to the employer’s platform.
No, Hamilton is an asset-layer library for organizing data transformation code, not an operational system for creation like Airflow.
You can use @config.when() decorators to swap out logic based on configuration parameters instead of writing inline conditional statements.
See Zekai first in Google