Python Selenium Tutorial #2 - Locating Elements From HTML

Подписаться 1,6 млн

Просмотров 699 тыс.

50% 1

In this python selenium tutorial I cover how to locate elements from a webpage. This will show you how to find elements from a selenium script using their ID, class name, name and other features. In this video we use selenium to search for something on a website then list out all of the search results.
Selenium Documentation: selenium-python.readthedocs.io/
Playlist: • Python Selenium Tutori...
Subscribe to my second RU-vid channel for weekly podcasts: / @timeoutwithtim
◾◾◾◾◾
💻 Enroll in The Fundamentals of Programming w/ Python
tech-with-tim.teachable.com/p...
📸 Instagram: / tech_with_tim
🌎 Website techwithtim.net
📱 Twitter: / techwithtimm
⭐ Discord: / discord
📝 LinkedIn: / tim-ruscica-82631b179
📂 GitHub: github.com/techwithtim
🔊 Podcast: anchor.fm/tech-with-tim
💵 One-Time Donations: www.paypal.com/donate/?token=...
💰 Patreon: / techwithtim
◾◾◾◾◾◾
⚡ Please leave a LIKE and SUBSCRIBE for more content! ⚡
⭐ Tags ⭐
- Tech With Tim
- Python Tutorials
- Selenium Tutorial
- Selenium Python
- Locating Elements Selenium
- Find Elements Selenium
⭐ Hashtags ⭐
#python #selenium #webscraping

Опубликовано:

27 апр 2020

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 579

@longdidier 2 года назад

Great tutorial! for anyone following on 2022, the method find_element_by_class_name was deprecated, gotta use find_element(by=By.CLASS_NAME, value="element to look for")

@utku5713 2 года назад

oh thanks :)

@indicgamer2907 2 года назад

man finding elements by class is returning the nosuchelement exception no matter how much i try

@januszkonarski1274 2 года назад

@@indicgamer2907 Try this: article.find_element(by=By.CSS_SELECTOR, value="div.entry-summary")

@adthn 2 года назад

im getting an "unresolved reference" for that "By" in "By.CLASS_NAME"

@rushilkulkarni7206 2 года назад

@@adthn WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@name='search_query']"))).send_keys("youtube test") driver.find_element_by_xpath("//button[@id='search-icon-legacy']/yt-icon").click() try this

@loznero6297 4 года назад

You don't even know how useful these tutorials are... I've tried so many times to learn selenium, but there's no proper tutorials out there... until now!

@markdillon9588 4 года назад

Agreed ! And even less with python and selenium 👍

@johanjim4940 4 года назад

ya same very few tutorial on selenium i wanted to learn selenium 2 months back but due to less tutorials i dropped it but now i am learning from Tim.

@thomasalberto613 4 года назад

This channel is simply fucking great

@vasanthbalaji1528 4 года назад

Check out in Udemy i have created a f/w using python and selenium

@janicejose5809 4 года назад

so hell true i was struggling trying to learn selenium until i found this channel

@rustams7502 3 года назад

How is this FREE CONTENT so much better than the course I'm following on Udemy or any tutorial I've seen on selenium out there? The answer: Tim is an outrageously excellent communicator, an aberration

@tigron1530 2 года назад

He doesnt has to make this content paid since he wins money from visits..

@kshau_ 2 года назад

every coding tutorial i see, people are just criticizing udemy lol

@xilllllix 2 года назад

@@kshau_ it's bec the quality is poor compared with the price of the course

@shabba194 2 года назад

Which course are you taking?

@sidneyw.mathiasdeoliveira8621 2 года назад

Ansewer: free lessons are not worried of being super complete and well explained

@MahaKhanMK 4 года назад

These tutorials are exceptional so clear and precise. Thank you Tim! I need to make an entire project and submit it by the end of May and this tutorial is helping me understand things tremendously! Please if you can finish the entire series within the initial weeks of May. And keep being awesome. Much Love.

@chasemedsker 4 года назад

You teach better than any teacher I have ever seen. For some reason, it is extremely easy to follow along with everything you're saying. You earned a new sub forsure.

@Helpmecry 4 года назад

If i have a moral to tell anyone about learning tutorials: "Don't just pause the video and go faster than Tim i literally spent half an hour trying to figure out an error that Tim was just about to explain 😂 " : 8:40

@smh0110 3 года назад

i did the same lol

@shravanasati9631 3 года назад

haha same

@subhanshareef939 3 года назад

you are not the only one

@midnightdev73 3 года назад

That's relatable

@nahuelgarcia9223 2 года назад

I actually did that even AFTER reading your comment and got stuck for like 10 minutes without understanding why it wasn't working xD.

@MandolinSashaank 2 года назад

I would literally invest a fortune to get any of your courses. The way you explain stuff just blows my mind! I can never be thankful enough for all of this!

@sonatartica2590 3 года назад

I am struggling to become an hybrid tester, being a functional QA for over 8 years but your videos are really helping me a lot to achieve my goal in order to grow further in my IT career! THANKS!!!

@TheAInfinity 4 года назад

Great tutorial for Selenium. It's important to note that Selenium is very slow, and for any large scale project, it would take minutes (sometimes hours) to finish running, so this web scraping approach should be the last option to be used for websites with heavy JavaScript. The practical approach would be to send a GET request to the URL of the website and adding the search query to the end of the URL and then parsing the response. You can see at 5:30, the URL has the query add "?s=test". This is the same approach but maybe 100 times faster than using Selenium.

@danfog7370 3 года назад

What is the name of this practical approach, or are there any tutorials you can refer me to? I've realised this selenium tutorial is great but not practical for a site with large scale data

@alenjose3903 3 года назад

@@danfog7370 regular webscraping using requests

@playstationnextgenil491 3 года назад

That can be very hard at times though. As most websites tend to obfuscate these kind of things.

@YourMom-du5ud 2 года назад

Yep. It's easy to use with websites that deliberately obfuscate how they work to prevent automatic interaction, though. For example, I wrote a simple API to upload and download photos from and to Amazon Photos, which really doesn't work by using normal requests.

@drygordspellweaver8761 Год назад

@user-bg2pl1we7x you mean that you wrote a simple *script*. Not an API. An API is literally what requests work through.

@alecouto 3 года назад

Several times I got myself stucked in minor mistakes using Selenium, sometimes mine, some from the books, and here made it so easy to follow. Have my subscription and my Like, Sir.

@KrispyKruiser 3 года назад

Your the best python tutor ive seen. Seriously great detail, the use cases are helpful as well, everything. top notch.

@caioglech 4 года назад

Congratulations on the video. Keep bringing us more of those on Selenium. Especially regarding most common errors such as Stale Element Reference Exception and No Such Element. It could be very helpful.

@arjunprasadsarkhel7814 4 года назад

Thank you Tim for this, looking forward fof more

@TechWithTim 4 года назад

Thank you!

@felipeleon5647 3 года назад

I consider that Tom is one of the best teachers in computer science that I´ve watched. Please continue by explaining different topics in python. Thanks

@theradicalramblr6504 3 года назад

tom?

@heathengaming8461 4 года назад

Looking forward for more selenium tutorials 👏

@kennethgan1333 4 года назад

www.udemy.com/course/learn-selenium-automation-in-easy-python-language/learn/lecture/16983696?start=1#overview

@santiagogaborovmilich7887 3 года назад

This is exactly what I was looking to understand web scraping. Amaizing tutorial! Thanks so much for sharing it.

@KhalilYasser 4 года назад

Awesome Tim. As for the headers I used this line header = article.find_elements_by_tag_name("a")[0] then print(header.get_attribute('innerText'))

@nczioox1116 4 года назад

Probably one of the best Python tutorial channels on RU-vid

@user-dy5ho4sj2w 4 года назад

I went through this video twice because the material has been so useful. Thank you again!

@qusek6446 2 года назад

NEVER STOP MAKING THIS CONTENT IT IS EXTREMELY VALUABLE THANK YOU SO MUCH FOR EXISTING

@artyom9499 2 года назад

Thank you Tim for complicating things in the end of the video, making it even more challenging to understand.

@chairjacker Месяц назад

Ive been trying to use chatgpt for a selenium project, but looking the descriptions on this series I'm delighted to go throughly through all these tutorials so i may actually get this project of mine done. Thanks so much for being consistent and thorough in your tutorials.

@fabu324 4 года назад

Great new tutorial! You're just awesome man and I've earned so many coding skills from you!!!

@sephirothu1290 2 года назад

Hey man I just wanted to say: you have the gift of simplifying complex things. God Bless You.

@heidisdiy6888 4 года назад

Thank you, thank you, thank you for this tutorial! You explain things very clearly! I hope you keep on making these videos!

@andrewromano6351 3 года назад

Dear Tim, OMG WOW! This is a life changer!!! With this, anything is possible. THANK YOU!

@danaraujo7870 3 года назад

Tim is the GOAT. Easily on par with the UMich courses on coursera (but where they teach you the language, he gives these awesome tutorials to show what you can really do with it).

@fmostream 4 года назад

Love your tutorials, this one really helped me out starting up with selenium

@atbox7535 2 года назад

Hello Tech with Tim, your Selenium python tutorial is amazing it gave me an way better understanding then documentation. Thanks for the tuturial.

@sohansoharab1670 4 года назад

The difference betwn "find_element" and "find_elements" is single item vs list

@Jamesandrews1997 4 года назад

Bro, I had no idea why my code wasn't working. Thank you!

@debajyotidas4549 3 года назад

Thnx for mentioning, couldn't find the send_keys().

@Lahmeinthehouse 3 года назад

THANK you!!! Great clarification :)

@stark2461 3 года назад

thank you man, i was getting error like 'list' object has no attribute 'send_keys'

@j43fura71 3 года назад

thank you brother

@rikittu 4 года назад

Thank you for this. Im looking forward to all the parts of this series

@BM-vz2nb 3 года назад

you are doing a great job Tim! Thanks a lot for your efforts!

@danielmonteleone5850 3 года назад

appreciate you man I have so many ideas for how I'm going to use what I learned, so much practical info.

@2c0bb61 4 года назад

Good content, thanks for putting your time and effort into this. Definitely worth a like

@apratimghosh109 4 года назад

Amazing tutorial, everything worked perfectly. Thank you!

@enoch5267 3 года назад

Best Programming RU-vidr Ever!!!! Thanks for your Tutorial Tim.. Very Understandable Thanks!!!

@oseiryth 2 года назад

i rarely subscribe but your contents are very concise and technical, that i had to subscribe; more power to your channel.

@user-vw3pr5gq9g 3 года назад

frankly speaking ,Tim is just an amazing teacher

@TopBagon 4 года назад

Amazing job. Learning from you is surprisingly easy

@sivaaj8504 4 года назад

It's been a while I came across a good tutorial. Thanks !!

@vonesha 4 года назад

Please keep up the good work. Following this series!

@ramizquliyev394 3 года назад

You are amazing!!! Normally I don't like to write comment but I just want to support you. Thanks a lot

@DanNorris81 4 года назад

THANK YOU for these videos! Highly appreciated videos!

@pierrevvr 3 года назад

this playlist is dope! thanks bro, cheers from france

@DrSebastianFranciscoAntonio 4 года назад

Incredibly helpful, thank you tim

@Kaloris24 4 года назад

Great job Tim! Recent subscriber and happened to see the start of this series recently. Trying to watch 1 video of yours before work (2nd shift ugh). I love Python as well. I was wondering if you could do a Go tutorial series if you get time please? Especially since it's newer and that you are good at teaching. Thank you.

@TechWithTim 4 года назад

Ya id actually love to do go! Definitely on my list

@vaibhavjain1914 3 года назад

man I love your video gonna make a bot to attend my online classes with this, thanks dude 😉😉

@RubyPiec 4 года назад

Only subbed cause you came in my reccomendations, love programming and am named Tim

@annaburshtyko5714 3 года назад

Very useful tutorial, thanks a lot!

@gloryness 4 года назад

Btw, there is a "main" ID on the first page too, and for me it got that instead of the second page. It's an easy fix, just put time.sleep(5) a few lines before the try statement.

@duke4279 4 года назад

Awesome explanation and great contents, liked and subscribed!!

@morty6224 4 года назад

Thank you for teaching this on free. Love you😍😍😍😍

@fatimansarizadeh 4 года назад

Many tnx Tim. U r doing perfect work. Can you please consider talking about preprocessing datasets like NELL.

@julian3833 4 года назад

You rock Tim, great work!

@namesarnav 3 года назад

All I wanna say is...THANK YOU SO MUCH FOR THESE VIDS

@yaagikmaurya4843 3 года назад

Thanks for such a great tutorial it's simple though very useful

@climax_set_bird464 3 года назад

Just remember that you have to type find_elements_by_class_name etc. because there could be more than one of the same class name. Typing find_element_by_class_name etc. will throw an error.

@arshadedris Год назад

Thank you so much Tim I really Appreciate your time, help and effort. I have seen all of your tutorials in every section of SDET that was amazing and understandable. I just wanna know about the name of application that you are working in this video, wanna start practicing by watching your video and try with myself. Thank you !!!

@mv848 Год назад

Amazing, thank you so , so much! It works like a charm every time.

@mikerope5785 2 года назад

i think the titles weren't printing because they were nested inside an anchor tag. you would have to add a line to find by tag name "a" for each "entry-title" and it would have printed. You are a great teacher Tim.

@jamesenrique648 5 месяцев назад

Wow, this video provided the help I´ḿ looking for to resolve some work issues. Thanks!

@sohansoharab1670 4 года назад

Nice.. Please by the end of the series, discuss about the difficulties and dependencies on collecting data from dynamic websites and SPA.. Please that would be really beneficial.. And I appreciate your effort.. Thank you so much! Beacuse for example: sometimes some websites shows valid src in img tag but after scrapping with selenium they become dummy like DATA:jsoockepskfuoe......== something I didn’t find the issue yet

@samchan45 2 года назад

starting a new job, and you are a godsend!

@aryanyekrangi7093 4 года назад

I love your videos! so detailed!

@kheshavlg 4 года назад

AWESOME video waiting for the next one !!!!!!!!!!!!!!!!!!!!

@cuteypatootie 2 года назад

Hi Tim! Thanks for the awesome videos I’m a new subscriber and love the content. I have a question: is it more efficient to grab a section like this and iterate through it, or would it just be better to select all of those articles headers text using 1 line xpath? Perhaps you showed it this way in case we want to extract more data than just the headers? Also, I find myself using web driver wait on almost all element location to ensure I am waiting for DOM. Is this correct? Seems like code redundancy. (Figured out how to solve this part in part 6!) Thank you so much!

@phonylattice2743 4 года назад

Thank you so much for this tutorial!

@chiragchhillar2958 4 года назад

please bro i am waiting for you to arrange the playlist. U teaches very well bro. I am from india and i loved the way u explain

@saadaddine6422 4 года назад

This is great. You're the best

@mandyssugarshack6414 2 года назад

You are an amazing teacher!

@goku64100 Год назад

fun fact you no longer need to import keys in selenium 4.3 they got rid of the find_element_by function and instead all you need to do is driver.find_element("whatever your searching by (name, xpath, ect)","value")

@xscitobor1233 Год назад

That seems harder

@mariorossi1456 4 года назад

Really helpful! Keep it up!

@mpower3435 3 года назад

Thanks for your efforts Tim

@CochDot 3 года назад

Amazing tutorials for selenium .... Thanking you With regards, Indian Tubers

@lawrencedoliveiro9104 3 года назад

3:27 Notice we have both “find_elements_by_class_name” and “find_element_by_class_name”. I suspect what you said earlier about only finding the first instance of an element with a specified class applies to the latter call, not to the former; the former one would let you iterate through them all.

@BobbyTomMathew Год назад

This tutorial is incredible.

@jean.lucasls 22 дня назад

This video is better than so many paid courses!

@paulsamuel9649 4 года назад

Really helpful. THANKS👍

@fabiiic-ops8600 3 года назад

Nice Tutorial!, For the Problem with the Headerst (class = "entry-title") just take the opacity out of your styles. Then you gonna see it again :) And as I can see, you have more css stlyes with the same tag like a { color: blue;} a {color: orange} Keep the code nice and steady and dont repeat the tags for better performance and less errors. Cheers

@paweszczepanski6738 3 года назад

Tim, You are the best.

@hermanyu9451 2 года назад

For those in 2022, change main.find_elements_by_tag_name() to main.find_elements(By.TAG_NAME, "article"). Same thing with header, use article.find_element(By.CLASS_NAME, "entry-summary")

@dangieebitz 2 года назад

Thanks!!!

@lakaka4567 Год назад

thanks

@luisbarbosa5597 3 года назад

awesome tutorial. tyvm bro!

@user-uk8we4bx9o 4 года назад

Thank you for your series

@atulram8206 3 года назад

@Tech With Tim, The glitch you are talking about (7:00) is the opacity for the entry-title class is 0% .entry-title { color: orange; opacity: 0%; }

@nhc772 4 года назад

Thank you. I've got my second interview on Thursday for QA! Wish me luck :D

@erichuang5971 4 года назад

how did it go brother

@nhc772 4 года назад

@@erichuang5971 it turned out to be a phony interview... it was really brief and misleading. they told me to expect an email the following week asking for a third interview. i got an email the next week saying they regret to inform me they were going with someone else! the whole process was messed up and stretched out with COVID, but it was still kinda crappy to experience. hope there are better cards for you!

@zeroxx131 3 года назад

Thank you so much! Subscribed!

@freezaki7302 4 года назад

Wow, great tutorial

@bluedragonn1 4 года назад

Men I wish all of my teachers explain my classes as you

@paulohsgoes1959 3 года назад

Excellent job. Congrats

@clasifiedman2000 Год назад

Just as an fyi to anyone struggling with using"find_element_by_name" please note that selenium has removed this feature and replaced it with driver.find_element("name", "x"

@sandismole5893 Год назад

Thank you Sir!

@jiaofeng6639 Год назад

Super super awesome and helpful 😊😊😊

@stephenduff1917 Год назад

Please update the video for the new code

@markdillon9588 4 года назад

Cheers Tim! Been wrecking my head trying to do this. Python and selenium get less love than java. I work with testing for SharePoint and CRM which heavily uses drop downs which use JavaScript, would love if you did a vid on how to test that maybe as it makes navigation via selenium difficult ? Also multiple tests with different test variable inputs would be amazing !

@dnetne5508 4 года назад

Excellent!

@estherm6464 3 года назад

I don't know how else to say this, I love you.

@rafj9883 4 года назад

Thanks Tim!

@beta8 4 года назад

good work tim

@jumtmeamjusthere 4 года назад

Thanks, Tim your tutorials are really great, I have a question if i may, how to use a second profile with google chrome in selenium it doesn't seem to work

@colednicholas88 4 года назад

You are the absolute man . . .

@omarbenjellountouimi4693 3 года назад

Thank you for these amazing tutorials ! I have a question: how do you proceed if the HTML in the source page is different from the one in the inspect element ? Is there a way to get the HTML from the inspect element ? I know it has something to do with javascripts that are not run in the source page. But I don't how to get the data from the javascripts. :)