GenAI Dynamic Task Builder for Airflow
Python · Apache Airflow · GenAI · LLM
View source →Idea
Data pipelines are typically static: DAGs are written, reviewed, and deployed. This project asks what changes when an LLM sits in the loop — generating and executing Airflow tasks dynamically from natural language descriptions.
What it demonstrates
- Translating natural language intent into executable Airflow task definitions
- Guardrails around LLM-generated code execution in an orchestrator
- Agentic orchestration patterns applied to data engineering workflows