How to store data with Python and SQLite3 

SQLite is very easy as compared to other databases. A short but very useful video.👌👍

An absolute pillar of society.

Very nice overview of creating and viewing a database.

Thank you, John! You're doing an amazing job by sharing your knowledge with others in such a great form. Wish you all the best!

Really helpful and great stuff. Thanks John.

Thanks

Thank you for this! Keep up the great work.

would be cool to see this as part of a scrapy project e.g. in a pipeline

Could you please make some video about using microsoft graph API to access outlook or sharepoint with some steps to Register the app

Another great one John!

Thank you for the videos. I've learnt a lot from you. Can you please make a video about handling captchas without using selenium?

As an extension to this tutorial would be really cool to see a way of hosting this sqlite3 database instance in a docker instance w/redis or something. Nevertheless excellent video, super practical. Love your content.

hey that was wonderful , could u make a video on when amazon blocks from scrapping or shows captcha . because the way you explains and teaches the things are really really easy to copup with . Thanks

wow this vid is so clean

Cool thanks

great practical stuff !

Thanks John! Really helpful again. A while back you mentioned a video for deploying to Heroku, is that still in works?

John I think mongodb is better than sqlite for crawling multiple spiders. In sqlite we have to write more codes and gives unnecessary errors related to pipelines and database connection. For complex and large project mongodb is better.

The best of the best

Thanks for this!

Thank you very much.

Great work.

Amazing video! Could you do a video on web scraping using a VPN network as a proxy? For example, using Protonvpn or Nordvpn for scraping data from amazon? Thanks!

Hi, Can you guide us how to create new sheet in workbook everyday for data update.

late comment but how about checking if data is found ?? i could loop through but depending on data that'd be slow

Hello John.Thanks for your job.Can you make video how to send email once a day automatclly?

Great Video once again. Can you please finish the fastapi video? Thanks again

Great video. Can you talk a bit about using sql "create table as" in python?

Hello, great content.I wanted to ask a question that i have looked in Google so long couldn't find answer. Is it possible to use selium in headless mode and then when a get 200 response open that same request in gui mode? I would appreciate if you respond to this.

At no point do you close any of the connections. Does this mean you have a load of open connections in the background or is there some sort of (out of scope?) clean-up?

What if the "price real " changes the value ? How would you update the entry than ?

it would be great if you go from scraped API json to NOSQL without duplicates, deduping so to speak

I need your advice regarding SQL... Is there any advantage to learning it, if I am pretty comfortable with pandas? So basically the question is, what do I get out of learning SQL ... what are its advantages over pandas? Something that pushed me to learn SQL (I keep saying SQL because I see a lot of python programmers say SQLite, I wonder how it is different from SQL?!) ... the thing that pushed me to want to learn it was seeing pandas not able to deal with large data frames, so how is SQL doing int that regard? pandas is great, but once the pickle file exceeds 2 GB, handling data becomes extremely difficult.. is it the same with SQL?

lets go !!! first comment. keep up the work john

Hi John, Could you please share your VSC settings which you were usingin previous videos? The theme looks pretty good and terminal also!

Hello John thank you for amazing, really amazing video again. You lessons are awesome and it looks so easy. 😊😊😊👏👏👏 P. S. Please, Please, make a course on Udemy about all the Python stuff and I buy it immediately 😎😎

❤

Can you scrap Udemy website ?

so I did this using pycharm and jupyter and neither didn't created my .db database, and it created an error alert stating that the name I gave my .db; NameError: (name) is not defined. Did you define your example.db off video?

Moving too fast

Please work on your speaking style. I often come on your channel and try to learn something from you. But the way you speak destroys the learning passion. 😢

Keep getting this error, can anyone help? Traceback (most recent call last): File "/Users/noah/Documents/Database/main.py", line 7, in cur.execute('''CREATE TABLE IF NOT EXISTS patientnameandage sqlite3.OperationalError: near "in": syntax error

How does this compare to storing the data in a pandas dataframe and then exporting the pandas dataframe to sql format? Is this just a better way to save the scrapped data into a db in real time, line by line? I need to get better at extract transform load processes for my scrapers that I want to run consistently to build a time series picture. Would be interested to see some videos on simple ways to set up a scraper to run say once per week, and push data to a sql database on aws, that can then be queried via a graphql api using something like hasura.io, and then how to monitize that dataset on rapid api or a make your own site for it with something like blobr.io.