Тёмный

What's new in Apache Airflow 2.10? 

Data with Marc
Подписаться 28 тыс.
Просмотров 2 тыс.
50% 1

🚨 Join me at the Airflow Summit between Sept 10 and Sept 12, 2024, in San Francisco. Use the code 20DISC_MARC for a 20% discount
airflowsummit.... 🚨
Here are the significant updates:
✅ Dynamic Dataset definition
💻 Create datasets dynamically from tasks! No more static datasets
✅ Big update on the Datasets view
💻 Better track which dataset triggers which DAG runs and more!
✅ Hybrid Executors!
💻 Configure multiple executors to efficiently run your tasks
✅ Dark mode
💻 That's right! Save your eyes (still experimental, though)
✅ New button to re-parse DAGs
💻 Stop wasting time, hit the button to get your DAG updates right away
✅ New concat() method
💻 Combine multiple outputs from upstream tasks
And more!
🏆 BECOME A PRO: www.udemy.com/...
👍 Smash the like button to become an Airflow Super Hero!
❤️ Subscribe to my channel to become a master of Airflow

Опубликовано:

 

1 окт 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 8   
@Empusas1
@Empusas1 Месяц назад
I like your content and have your courses in Udemy. What I am searching for, which is not included in your courses, is how to properly size airflow in a docker environment. If I need more workers, is it better to increase the celery workers or add new worker containers. What are the limiting factors of the host and how to monitor them. I see that my host, which is a vm, is mostly idle, but I have only 16 parallel celery workers.
@surawutjirasaktavee
@surawutjirasaktavee Месяц назад
Thank you for sharing! Can you have the course that explains how to set a standalone dag-processor for multi-tenant teams using the official helm chart?
@MarcLamberti
@MarcLamberti Месяц назад
As of today, you can’t with the Helm chart. The feature hasn’t been implemented yet
@rembautimes8808
@rembautimes8808 29 дней назад
Just joined your Channel. Very good content
@MarcLamberti
@MarcLamberti 28 дней назад
Thank you!
@monkshandilya
@monkshandilya Месяц назад
Trying to run airflow scheduler on both ec2 nodes in a Load Balancer setup .but it's not working as expected , any suggestions?
@kamranalisyed5553
@kamranalisyed5553 Месяц назад
Wow love it. Especially multi executors.
@andriifadieiev9757
@andriifadieiev9757 Месяц назад
Thank you for sharing!
Далее
How AI was Stolen
3:00:14
Просмотров 943 тыс.
# Rural Funny Life Wang Ge
00:18
Просмотров 760 тыс.
🛑самое главное в жизни!
00:11
Просмотров 67 тыс.
Everything New in Airflow 2.10!
11:50
Просмотров 236
What's new in Apache Airflow 2.8?
9:58
Просмотров 3 тыс.
Airflow DAG Factory: Create DAGs dynamically with YAML
16:00
FREE Hacks to Create PPT using Chat GPT in 8 Mins!
8:46