Python Web Scraping with Beautiful Soup and Regex 

I like your tutorials because u go directly to the content, something rare in youtube these days...

Dude i watched over 15 videos+ that was recommended and after hours i found this FULLY EXPLAINED.

I felt so powerful as soon as I pulled an array of strings from a random website. Thank you for your great tutorial

Love this presentation. Straight to the point with short and specific explanation. Keep it coming! :)

I was able to do this on another leaderboard site, appreciate your work here.

This is remarkable, very informative and down to the earth - I really love this concise format of yours which is rather contradictory to what most of ppl on yt are providing

Hey man! I watched this and applied the concepts to a weather site and made a csv of all the sunset / sunrises in 2019! Thank you! Please I love the way you explain things keep making videos sir! I have applied your teaching in a couple videos and it’s great! Learning so much!

This is THE best beautifulsoup tutorial on the internet.

Your videos are fast-paced and straight to the point. Thanks!

Thanks for the video. I like how you take the basics and break it down with really good and practical examples.

Been doing something similar for a while but in a much more complicated way. Looking forward to making my life much easier. Thank you!

It helps so much to have someone that matches your personality when learning stuff. I can't stand when asking someone for instructions on how to do something and they tell me everything that I can expect and every once i a while throw in the thing I'm supposed to do next. None of the fluff here. Just context. Every other creator would/has made this subject a 45min+ video but here I am feeling proficient after just 14 minutes with EM. Thank you, Sir!

Your videos are fast but glorious! Love your content man! Thank you! Bless 🙏

in a sea of youtube tutorials yours is the pearl. thanks!

wow.. thats the crisp explnation,,, you're the man!!

I was struggling to navigate to iterate through second span tag in multiple td tags I.e. (tr[1:]/td[0]/span[1]) I was trying it the whole day. This is the best tutorial I have seen. Thank youuuuu.

These videos are brilliant. Thanks for all your hard work.

Real Nice content! Straight to the point. I’ve played around with beautiful soup a few years ago for an small project, and I just wish this video was around at the time....

Simply brilliantly explained. I have seen few of your videos and I like your style, therefore You have earned another subscriber.

If only I had this tutorial a few years back. Good stuff.

Some of the best tutors are always straight-forward: down and dirty!

Wow these videos are awesome! Direct and concise but understandable!! Well done!

Dude you're helping me out immensely with computer science courses

was trying to scrape modarchive for my project, this is way better than writing the name and id down by hand lmao, thank you!

Really nice content! You explain just enough to be clear but not too much that's perfect. Please continue to remind the type of the elements you create, it's very important for beginners.

This video is really a good intro to web scraping.

These were really good examples, thank you!

You are the coolest tech guy I have ever seen on RU-vid

You're definitely skilled! For anyone watching these videos, don't get discouraged, this takes time. @Engineer Man , Can you talk about the experience of learning how at the beginning of your videos?

I spent forever teaching myself this last week and now you post this. Kill me now

your videos are a life saver man. keep up the great content

Exactly what I needed thank you very much!!

EM you're awesome. i was studying web scraping and this come up. subscribed yesterday to your channel! lol

Excellent! Thank you!! Great vid!

This content is absolute gold.

Wow, I liked this video so much! It was very useful! 😄 You really have helped me a lot, it was well and fully explained, with real life examples Thank you so much for this tutorial! 👏👏

Awesome! Kudos! Very helpful. Thanks man!

Cheers man i used this tutorial to sort through 310000 embed links you’re so awesome

this is perfect for what I needed thank you!

he makes it so quick and simple

Beautiful video man! Really valuable information here. As a sysadmin with over 10 years experience, I can state its really clean method of scraping. I was used to use bash scripts for everything but using libraries in python is sooo helpful. It would be a pain in the as in bash with awk, grep, etc. I hope to see more soon

Keep up the good work man!

love your tutorials man

Excellent tutorial. Thanks.

Using CSS selectors is the way to go, gets rid of nested for loops and is more robust.

Wow just found your channel by accident and I’m loving it. Awesome content!! Do you think you can do a vid on Elastic Stack?

You go straight to the point. Obviously, you know a lot more in-depth about this topic. Anyway, I like it.

This is excellent! Well explained! :D

Super helpful, thank you.

Great video. Stuff like that makes me want to program and develop software until i die.

Awesome video! Can you do one on the requests module please?

3 searches in the loop - very dirty ) "The speed of software halves every 18 months"

great stuff, thank you for this.

Great content

Great vid with fantastic content. Would love to see this where you first login in order to get content. Getting the headers set is a challenge.

😎👍🏼 Thank you, Sir!

I love your page man.

This was so freaking helpful

Amazing. Thanks a lot.

Love your content!

Thank you for your video. Awesomesauce

ty, saved me an hour of time from all these other videos. holy shit

Top explanation, tnx!

13:52 Happiest man!

Good video. Personally, i like using node with dom module and write css queries to extract what i want.

Whaou! Amazing!!! many thanks

Thank you thank you!

Great video. Great channel

Thank you for a very clear

carefully he's a hero

Happy that you've chosen this topic. I've been exploring web scraping and have a script that works pretty well on a site that I frequent. Another awesome tool that can be used to also automate web navigation is the selenium package. But on more of a question-related note, I know the script you just made was pretty simple, and the one I have isn't that complicated, but I've been wondering how one would go about writing an object-oriented script for scraping?

Love this vid

Awesome tutorial man! Can I suggest scraping dynamic pages as the next tutorial. Would be a sweet follow up

That was beautiful not to say absolutely excellent. Man ,,,,,

thanks man

This is an awesome video, I actually enjoy the in depth walk through of what your reasoning behind writing code is, step by step. Versus just saying " I did this" and not really explaining anything. On a separate note , I'm looking to get into python, and I have previous code development experience, but It's been a little while, and setting up an environment to start doing some coding is a bit daunting. I'm looking to do more on the machine learning , neural networks side of things. I don't struggle with any of the logic, mathematics, but I know there are many pros/cons of various IDE's . Some have better support for various packages , etc.. I was wondering if you could either make a video on some of this information, or maybe throw a few pointers my way. I would really appreciate that. Otherwise, keep up the great content!!!

Awesome

@Engineer Man , what is your day job? And how did you get into coding? Do you have a CS degree? and.... well instead of more questions, rather just ask whats your background (ito what lead to you adding so much value with these vids)?

keep up the hard work :)

loved it ! Efficiency and very clear for a beginner. Would be great to have the login part, and why not sending the extraction into a csv file ;) I subscribe ;)

Thank you..!!

Went through a 30 second hillshire farms ad. Great match youtube

It would be cool to see how to deal with pagination. So you want data from /page=1, /page=2 etc. Etc.

I use requests and bs4. Never thought of using regex besides pattern searching

so practical tutorial.. I love how you teach people

Teach me your ways

Hey, I really love your videos man! How about elements within elements, so for example `this link is in a paragraph`. How would you approach that?

I love you man

How does he knows a lib for everything? i'am addicted to his videos.

great tutorials! do you have a tutorial on scraping with a function( ) using beautiful soup

Can you go over an example that first requires you to login and then requires you to use a query string with a hash token that changes after every login?

thank you so much for this amazing tutorial, i would like to ask what do we do if the site i want to scrap require to be logged in btw this got recap

you are the best so i subscribed your chanel

Cool

Great video again. I've been playing web scraping a while and I find that most of websites nowadays using javascript rendering quite heavy. Will you make a part 2 for talking about how to web scrape javascript rendering website? And what do you think about another web scraping package, Scrapy? thanks Man

Génial. Dieu sait que je galère à utiliser BSoup . Et là j'ai l'impression d'avoir enfin compris.

Hey great video! Lets say in your last leaderboard example, I would like to get notified when the leaderboard has changed, so to say when something changed on the site. I have built a script where I can see the HASH change, but I cant output what actually changed on to website, do you have any tips to how to monitor what actually changed on the website?

awesome vid! :D how would you do it with a site you need to be logged in to?

Can you do a video showing the interactions between sessions and bs4?

Hey E-man Great video! I wanted to ask you this. You said that "you have to know the structure of html before doing scraping from websites" which is true but I need to create a process where I can search a keyword or basically a word on multiple websites and get the content out of those websites. Now from this content, I will create a sentiment value and generate a newsfeed in my application with a sentiment value in it. Can you suggest a way I can do that? Thanks in Advance