Focus On: The Apache Airflow PythonOperator, all you need in 20 mins!

Подписаться 5 тыс.

Просмотров 21 тыс.

50% 1

Focus On: The Apache Airflow PythonOperator, all you need in 20 mins!
In this video you are going to discover everything about the PythonOperator in Airflow 2.0
At the end of the video you will be able to:
⏺ Execute a Python function (Amazing isn't it?)
⏺ Pass positional arguments to your Python function
⏺ Pass keyword arguments to your Python function
⏺ Evaluate your arguments at runtime and pass variables
⏺ Reduce the number of calls to your database
⏺ Make your DAG cleaner
⏺ Get the current execution date in Airflow 2.0
⏺ Leverage the TaskFlow API to transform your PythonOperator in a mind-blowing way!
Materials:
www.notion.so/The-PythonOpera...
The NEW WAY OF CREATING DAGS:
• TaskFlow API in Airflo...
Enjoy ❤️

Развлечения

Опубликовано:

15 апр 2021

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 29

@steliosioannides7128 2 года назад

Thanks Marc for this very useful video! High appreciated ! Keep it up !!

@andrikramer686 3 года назад

Awesome. Thanks a lot, Marc.

@Leonardo-jv1ls 2 года назад

Thank you. You've helped me a lot.

@amitjain000 3 года назад

Awesome.. thanks for such a beautiful knowledge share.

@JosePla 2 года назад

This is exactly what I was looking for

@chris0628 Год назад

To the point, no fluff 👍🏽

@AbhishekAmeria 3 года назад

That's was very useful. Thanks :)

@vladimirobellini6128 3 года назад

this is great. thanks!

@FranVarVar Год назад

Amazing. Thanks Marc!

@Astronomer Год назад

My pleasure!

@shivangitomar5557 2 года назад

Very good vidoe!!!

@averychen4633 5 месяцев назад

you are the best

@manjunathmani3678 3 года назад

nice man

@SpiritOfIndiaaa 3 года назад

Excellent thanks a lot...I have to check oracle table if a record exists for current date ,if exists then only call downdown stream task ,how to achieve this ?

@bullandrooster 2 года назад

Can I run 2 dags with different python version and different pandas version? For now I see airflow offers only one python version at the time, and only predefined required dependencies. And all my dags must match that specs.

@sridhar43299 2 года назад

How to send object from gcs to email bu airflow. If possible send me code

@brendoaraujo9110 2 года назад

instead of calling the script function, is there an operator that I can call the script completely to be executed?

@GeandersonLenz 3 года назад

How to turn ON this module includes?

@leomax87 2 года назад

did you resolve it?

@eunheechoi3745 8 месяцев назад

whenever I first run the airflow test command, it throws an error saying 'sqlalchemy.exc.NoReferenceTableError: Foreign key associated with column 'dag_run_note.user_id' could not find table 'ab_user' with which to generate a foreign key to target column 'id' do you what it is about? and how to fix it? Second run it doesn't occur....

@Astronomer 8 месяцев назад

Hmmmmm interesting, if it works on the second run maybe it needs to generate the table first?

@valetta6789 2 года назад

I am trying to pull the value from another task in a different folder, however, I see only the current task instance. Is this new behaviour, or am I unable to pass between different directories?

@Astronomer Год назад

So your DAG's are in different folders?

@valetta6789 Год назад

@@Astronomer I have no idea what I was talking about😄

@BalvanshHeerekar 2 года назад

Hey, while using includes I get the error - ModuleNotFoundError: No module named 'includes'

@leomax87 2 года назад

I'm facing the same error. How Can I resolve it?

@eldrigeampong8573 Год назад

@@leomax87 you can import the sys module in python after which you will navigate a number of levels either up or down into the directory your file is located. import sys sys.path.append(".") # that's move a level up into the dags folder from includes.my_dags.functions import process