How to Install Apache Airflow on Windows using Docker Container
1. create a directory named airflow and inside that create three sub directories - logs, dags, plugins and place docker-compose.yaml file inside airflow directory
2. install docker desktop for windows
3. download docker-compose.yaml file of airflow latest release
airflow.apache...
Airflow releases: airflow.apache...
4. docker-compose up airflow-init
5. docker-compose up
Cleanup - below command will delete all the conainer and free up the volumes and memory
`Run docker-compose down --volumes --remove-orphans`
command in the directory you downloaded the docker-compose.yaml file
airflow-scheduler - The scheduler monitors all tasks and DAGs, then triggers the task instances once their dependencies are complete.
airflow-webserver - The webserver is available at localhost:8080.
airflow-worker - The worker that executes the tasks given by the scheduler.
airflow-triggerer - The triggerer runs an event loop for deferrable tasks.
airflow-init - The initialization service.
postgres - The database.
redis - The redis - broker that forwards messages from scheduler to worker.
Reference: airflow.apache...
For any queries, you can connect with me using below links:
LinkedIn: / ashutoshtripathiai
Instagram: / ashutoshtripathi_ai
Twitter: / ashutosh_ai
Website: ashutoshtripat...
If you want to message me directly, then connect with me on LinkedIn and send a direct message.
If you want to discuss any topic around ML or have queries related to job switch, please feel free to schedule a 30 min call with me, I am available on topmate: topmate.io/ash...
#airflow #mlops #ashutosh_ai
2 окт 2024
