Bro anyway to scrap google search result links??? There is a great google search module available only problem is that it doesn't support proxy... Can u make a video....
John, Your videos are quite interesting and you do a great job explaining as you go. I did some searching around but did not see if you have already tackled vrbo or a site which is setup similarly. I have had rather poor results overall (but of course I am a bit of a newbie). I am curious to compare rates for a given location.
Thanks John! I would love to see a video where you identify reasons why automated clicks from various browser automation tools (playwright, selenium) are not clicking something on the page. I have run into jquery/ajax/bot blocking issues where it just won't click it. Love the videos!
Can you do a video about scrapy inline request method ? , I didn't find a video about using scrapy inline request , it will be very helpful to everyone...
Hii. i thought it would be nice. If you make video on how to test a big scrapy project without running the whole script. I tend to replace return with yield. So control my script. But i am guessing, you might have other ways to do it. Thank you
Great video, I would like to ask you pne question. How to know the amount of request you can make for second to a server in order to avoid break it? I mean, I have to scrape 30k pages and if I use time.sleep(2) is so slow! But if I make all in once, I am scared to break the server
Hi! THANK YOU! VERY COOL TUTORIAL AND YOU'VE DONE IT IN JUST less than 10 minutes. THANKS! Just a question, does the Rule objects must be in a variable named "rules"? and the order of those Rule classes but be accordingly right? Because if the 2nd Rule object which contains the callback is the first element of the rules list. It wouldn't work as expected right? THANKS!
Seems like you're on a different OS in every video. Looks like we have standard Ubuntu here instead of WSL? Jokes aside, another super helpful video! Every time I get stuck it seems you've already made a video to address the issue. Many thanks for your channel!
Hah my older videos are WSL, but I made the switch to Linux around a year ago, so going forward all will be Ubuntu/whatever I move too to get away from snap when I’m ready
@@JohnWatsonRooney Nice! I think we had this conversation in one of your live streams. I did a bunch of distro hopping but finally landed on openSUSE Tumbleweed with the Gnome desktop - you get a rolling release that still has very good stability, and great software compatibility since it's derived from an enterprise OS. Anyways happy scraping, thanks again for the content!
@@stephenwilson0386 Yes I remember. I've also done my fair share of distrohopping but before it was always more of just curiosity - I learned a lot... Main thing was how much I love i3wm and tiling!
@@JohnWatsonRooney I tried tiling for awhile and mostly enjoyed it (i3 and bspwm), but I'd always end up missing some of the creature comforts of a full desktop. I've thought about going back maybe as a secondary install though. It's one of the great things about Linux, there's no "one-size-fits-all" forced on you like Windows or Mac.
Do you have spesific course that explain Scrapy in advanced/intermediate level? Sounds good if you create one on learning platform like Udemy, bcz i like your way to guide us through this channel. 🥲
