Python Pandas Tutorial: Importing Bulk CSVs Data Into SQL Server Using Python #17

Подписаться 16 тыс.

Просмотров 24 тыс.

50% 1

Importing Bulk CSV Data Into SQL Server Using Python
In this lecture your will learn or understand how we can import the bulk CSVs or data into SQL Server using Python and SQLAlchmey package.
Using Python and SQL scripts is a powerful technical combination to help developers and database administrators to do data analytics activities. Python provides many useful modules (Pandas, NumPy etc) to perform data computation and processing of data efficiently.
Follow the below steps to import the bulk data or CSVs into SQL server using Python SQLAlchemy.
1.) What is your HostName?
2.) What is DataBase Name?
3.) What is name of DataBase's Table?
4.) What is ODBC Driver Name?
5.) How to establish the connection between SQL Server and Python using SQLAlchemy?
6.) What is the connection parameter?
7.) How to use if_exists parameter?
8.) Learn to develop the entire script using Python and SQL Alchemy.
Recommended Tutorial:
Auto Export Data into Excel from SQL using Python Pyodbc | Python SQL Automation | Task Scheduler : • Auto Export Data into ...
Python Excel Automation: • Excel Automation Using...
Python Teaser: • A Beautiful Python Pro...
Python Pandas Tutorial: • Python Pandas Tutorial...
Python Playlist: • Python Tutorial for Be...
Python Data Structure Playlist: • Python Data Structure
Python OOPs Playlist: • Object Oriented Progra...

Опубликовано:

6 окт 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 38

@xactindices1781 2 года назад

I've been looking everywhere to find a good resource for importing pandas df to sql server and vice versa. Best tutorial ever. Thanks a mill.

@alshukriai 2 года назад

Thank you very much, best tutorial on how to append data to sql.

@Sundar_Tenkasi 3 месяца назад

Nice explanation and this short code only I want to load file to table

@ProgrammingIsFunn 3 месяца назад

I am glad this video helps you.

@anjurawat5713 3 года назад

Wonderful concept 👌

@seriouscat6429 7 месяцев назад

this works, thanks

@Himansh29 3 года назад

Wonderfull 👍👍

@funnyfunnes2978 3 года назад

Amazing

@acertech1550 2 года назад

Thanks a lot, very good tutorial

@ProgrammingIsFunn 2 года назад

Glad to hear. 😊

@rakeshjangid48 3 года назад

Valuable resource easy to available, on easy to access medium in the simple easy definition. What else do we need more?

@vivekkumarsingh5664 3 года назад

This is Awesome tutorial...

@krishnachandraroy3199 3 года назад

Nice 😌😌

@hemantaborah9967 2 года назад

Your imparting skills are quite impressive. I find it easy to understand. Nice video. How we can insert data from pandas by pd.to_sql method in a user defined column of a particular table?? Say i want to insert live dataframe (with changing source values, and with no timestamp) in a 2nd column of a sql table where the 1st column record the timestamp of storing the dataframe??? Valuable solution/ reply will be helpful for me. Thanks in advance

@AnujVerma-kp3md Год назад

please please check my code i am not able to execute showing error import socket import sqlalchemy import pyodbc import os import pandas as pd import platform print(platform.node()) os.chdir(r"C:\\Users\\AM552QS\\Downloads\\excel") os.getcwd() print(os.listdir()) pyodbc.drivers() conn = sqlalchemy.create_engine(f'mssql+pyodbc://{socket.gethostname()}//SMARTDB?trusted_connection=yes&ODBC Driver 17 for SQL Server') for fl in os.listdir(): df = pd.read_csv(fl) df.to_sql("Test", con=conn, if_exists="append", index=False)

@BharatPatel-u4b Месяц назад

when I run my Batch file, I get this error "ImportError: No module named pyodbc"

@ProgrammingIsFunn Месяц назад

Create virtual env and install pyodbc using pip install pyodbc

@fra2847 Год назад

Please show how to import many CSVs files into separate tables (1 CSv = 1 Table) with DBForge or simpler. Ty.

@cdrosero 2 года назад

Nice video, Does the table structure should match the CSV files?

@dikshyakasaju7541 2 года назад

Getting the error: InterfaceError: (pyodbc.InterfaceError) ('28000', '[28000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Login failed for user Although i followed the same exact steps

@ProgrammingIsFunn 2 года назад

which driver you are getting when you are printing pyodbc.drivers() ?

@dikshyakasaju7541 2 года назад

@@ProgrammingIsFunn ['SQL Server', 'SQL Server Native Client 11.0', 'SQL Server Native Client RDA 11.0', 'ODBC Driver 17 for SQL Server', 'ODBC Driver 18 for SQL Server']

@alicevaleina3479 Год назад

me too

@clovisstanford6515 6 месяцев назад

Cell In[5], line 6 df.to_sql('Paint',con=conn,if_exists = "append",index=False) ^ IndentationError: unexpected indent GIVING ME THIS ERROR .

@khaliljamil4558 Год назад

it is a nice video and walk through I'm get stuck at the last part to be error " OS ERROR: Initializing from file failed

@AnujVerma-kp3md Год назад

can some one help please please check my code i am not able to execute showing error import socket import sqlalchemy import pyodbc import os import pandas as pd import platform print(platform.node()) os.chdir(r"C:\\Users\\AM552QS\\Downloads\\excel") os.getcwd() print(os.listdir()) pyodbc.drivers() conn = sqlalchemy.create_engine(f'mssql+pyodbc://{socket.gethostname()}//SMARTDB?trusted_connection=yes&ODBC Driver 17 for SQL Server') for fl in os.listdir(): df = pd.read_csv(fl) df.to_sql("Test", con=conn, if_exists="append", index=False)

@alicevaleina3479 Год назад

I keep getting InterfaceError. Any advice? Already checked everything twice, no mistakes.

@Gautam-l3t 10 месяцев назад

I found operational error , how I fix that ??

@ProgrammingIsFunn 10 месяцев назад

On which line of statement and please share the error

@sanketwade4465 2 года назад

is it the last video of series?

@ProgrammingIsFunn 2 года назад

No Sanket, soon we will try to upload videos on Pandas…

@acertech1550 2 года назад

is there a way to check if record is already exist before inserting data?

@ProgrammingIsFunn 2 года назад

Yes we can check. Simply you have to setup primary key in your table then you can not insert duplicate data.

@AnujVerma-kp3md Год назад

@@ProgrammingIsFunn please please check my code i am not able to execute showing error import socket import sqlalchemy import pyodbc import os import pandas as pd import platform print(platform.node()) os.chdir(r"C:\\Users\\AM552QS\\Downloads\\excel") os.getcwd() print(os.listdir()) pyodbc.drivers() conn = sqlalchemy.create_engine(f'mssql+pyodbc://{socket.gethostname()}//SMARTDB?trusted_connection=yes&ODBC Driver 17 for SQL Server') for fl in os.listdir(): df = pd.read_csv(fl) df.to_sql("Test", con=conn, if_exists="append", index=False)

@keooudaybouasy7864 2 года назад

can you Share you code because i follow you but code is error

@avinashkendre711 2 года назад

it is giving error for permission denied

@shobhasahu1030 2 года назад

OperationalError: (pyodbc.OperationalError) ('08001', '[08001] [Microsoft][ODBC Driver 18 for SQL Server]Named Pipes Provider: Could not open a connection to SQL Server [2]. (2) (SQLDriverConnect); [08001] [Microsoft][ODBC Driver 18 for SQL Server]Login timeout expired (0); [08001] [Microsoft][ODBC Driver 18 for SQL Server]A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. getting error