Python Async Web Scraping - Day 27 - 30 Days of Python

Подписаться 348 тыс.

Просмотров 22 тыс.

50% 1

Day 27 - Async Web Scraping in Python
In 30 Days of Python, I'll teach you the fundamentals of Python. We created this series as an introduction to programming with Python.
Why Python? It's a very popular way to writing computer programs and automations with board applications: SpaceX & NASA use it to launch rockets, Tesla with autonomous driving, Instagram as a web app backend, Intel to pilot drones/UAVs, and many many others.
What you can do with Python is nearly endless:
Run powerful web applications
Fly drones and launch rockets
Run autonomous robots like drones and self-driving cars
Build connected systems with Arduino and Raspberry PI
Automate business intelligences and perform financial analysis
Machine learning including deep learning
And many more ways
That list is fancy but don't let it scare you. Python is just a tool for writing a series of step-by-step automation rules and that's exactly what software is: a collection of rules for a computer to follow.
The right tools in the right hands can impact the world in such amazing ways.
Shakespeare had very few tools for writing plays and yet he continues to influence our entertainment despite being long gone. Creativity mixed with the right tools can transform how we do everything or why we do it.
That's why I want to teach you this tool. The world needs your creatively, your perspective, your project.
Let's do this!
Install Python on Windows: cfe.sh/blog/install-python-dj...
Install Python on macOS: cfe.sh/blog/install-django-on...
Code: github.com/codingforentrepren...
Subscribe: cfe.sh/youtube
Suggested next project: www.codingforentrepreneurs.co...
00:00:00 - Welcome
00:00:26 - Sync vs Async
00:14:37 - Async Web Scraping
00:22:32 - Async Scrape Multiple URLs
00:30:50 - Asyncio Semaphore
00:37:12 - Thank you & next steps

Опубликовано:

1 авг 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 39

@CodingEntrepreneurs 3 года назад

The code!! github.com/codingforentrepreneurs/30-Days-of-Python/tree/master/tutorial-reference/Day%2027

@alah9153 3 года назад

Thanks for the great tutorial! Upon running the scipt, I got 2 errors: 1) UnicodeEncodeError: 'charmap' codec can't encode character in position : character maps to SOLUTION: add following parameter to `output_file.write_text(...)`: `encoding="utf-8"` 2) RuntimeError: Event loop is closed Did anyone resolve this error?

@tarasmarusov2306 3 года назад

The best video of asyncio + aiohttp modules I've seen so far

@bojadev 3 года назад

Thank you from Argentina! You make so amazing videos!

@judeleon8485 3 года назад

Wow! great channel. I wonder why it took me so long to find this channel that has such great contents despite how vigorously I have been searching

@anastasiabarkova7124 3 года назад

Thank you SO MUCH for this tutorial on using asyncio/aiohttp for web scraping!! I was struggling with the asyncio documentation, and you explained it very simply and very efficiently. I have a code that is running thanks to you!

@aashayamballi 3 года назад

Thank you Justin... 🙏🏼

@ulvidamirli2758 3 года назад

Thank you for great tutorial! 👍👍👍👍👍

@vighneshkum 3 года назад

Thank you so much, its help alot. God bless you.

@subhakantasatapathy1976 Год назад

Great content!! 👍👍 Just the thing I needed for my project...

@fuad471 3 года назад

perfect video with practical examples which was defiinetly helpful especailly multi-url scraping part

@fakhrirobi5156 3 года назад

Thank you very much..apreciate it 🔥🔥

@DonLarryTT 3 года назад

YOU'RE BACK!!! :DDD

@freekeys 3 года назад

I found ur YT channel few weeks ago but ur content 🙏 amazing...... ❤️

@daddy7931 3 года назад

thank you

@gambomaster 3 года назад

Hello Justin. I have been following this series from the start. Just want to say thank you and one request. It will be good if you make one video on tkinter and desktop apps. May be a small budget calculator type app. I am sure others would also want to see that. 👍

@jahangircs 3 года назад

after long time 😍

@vinodkumar-py2dw 3 года назад

I love 2 Justins... One is one of the kings of Pop Music... Another one is one of the kings of Python, it's none other you sir... With love, from 🇮🇳 India

@CodingEntrepreneurs 3 года назад

hahahahaha love it. Thank you. love you too

@megoboom17 3 года назад

thank you, man! from Russia

@atefehz6606 3 года назад

I subscribed on your chanel for django stuff becuse you teach everything so good and practical but now i'm also learning web scraping ? Dude your chanel worths more than my cs degree 😂👌 i like how you just go for the meat of the code ! Can you make more videos on web scraping and maybe in advanced mode how to combine it with machine learning ? That would be a dream come true 😍 thanks for your good work ^_^

@CodingEntrepreneurs 3 года назад

Thank you!!

@ivanzorin7293 3 года назад

Thank you so much! I really appreciate it! But I wanted to refactor my parser of restaurants, made it asynchronous and you said in the video that it wouldn't work with js web sites, you didn't show it after and didn't explain why we couldn't do that (

@gavrilka 3 года назад

Thanks for your videos, Justine! I have a question about Chromedriver as service and Chrome as browser. Webpage i am trying to parse, is javascript code generated content. How can i scrape such webpage? My non async version of code is using Selenium + PhantomJS browser. But here with arsenic i dont know how to get full page contect filled with javascript content. Also, i am using BeautifulSoup4 for parsing page source. Should i think about some other framework for async code?

@piusijachi2763 3 года назад

Hi Justine can you explain how implement this into a django website...also u never did private chat section of your django social media project(i kw off topic just stuck).... You are awesome love from #🇳🇬Nigeria

@CodingEntrepreneurs 3 года назад

That’s a great idea. Thanks for the suggestion

@dalecornplant6073 3 года назад

thing is I don't understand why over complicate the base concept by adding in file handling as oposed to holding a variable and displaying manipulating a variable, especially when trimming it down to the important part of the html / body etc is going to be more optimal for business purpose / use case long term...

@ranaatul7599 3 года назад

Sir can you please upload the video on Rest API..

@benlien6021 3 года назад

i guess if you already how to do async. it is clear logic. but for me it is super confusing where your flow is going, and why it is.

@carrieyuan 3 года назад

Do we have to use asyncio.create_task() together with asyncio.gather()? I use only asyncio.gather() , it also works. I'm confused now.

@leonardobarrios1509 3 года назад

Thak you good content, how could it be implemented in django? . what would be other options without using celery?

@CodingEntrepreneurs 3 года назад

It should be able to in Django 3.1 as most certainly able using Channels 2 (in Django)

@nachoeigu 2 года назад

Thank you for your amazing content. Is it possible to render the dynamic content of some pages with aiohttp?

@CodingEntrepreneurs 2 года назад

Check out arsenic

@nachoeigu 2 года назад

@@CodingEntrepreneurs thank you very much for the info :)

@ruchitmicro8409 3 года назад

Yo can I run async functions inside a Django view function?

@ifeanyindukwe3086 3 года назад

Please can someone help, I get this error msg when I run asyncio.run(main()): ClientConnectorCertificateError: Cannot connect to host www.boxofficemojo.com:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)')]