Informatics Practices · Ch 3 — Data Handling using Pandas – II
Import and Export of Data between Pandas and MySQL
Import and Export of Data between Pandas and MySQL
To work with real-world data, you will rarely type every value into a DataFrame by hand. Instead, data usually lives in files (like CSV or text files) or inside a database. The process of bringing data from a database into a Pandas DataFrame is called importing data. After you have finished analysing that data, you will often need to send the results back to the database for storage or further use — this is called exporting data.
Pandas can both read data from a MySQL database and write data to it. To make this possible, you first need a connection between your Python environment and the MySQL server. This connection is established using a database driver called pymysql. You must install this driver in your Python environment before you can proceed.
Install the driver using the command: pip install pymysql
Along with the driver, you also need a library called SQLAlchemy. SQLAlchemy is a toolkit that helps Python interact with SQL databases. It handles the details of connecting, sending queries, and receiving results. Install it with:
pip install sqlalchemy
Creating the Connection with create_engine()
Once SQLAlchemy is installed, you use its create_engine() function to establish a connection. This function takes a special string called a connection string as its argument. The connection string contains all the information needed to locate and log in to your database. It is built from several parts:
- Driver: The database driver you are using. For MySQL with pymysql, this is written as
mysql+pymysql. - Username: Your MySQL username (commonly
root). - Password: The password for that MySQL user.
- Host: The address of the server (usually
localhostif the database is on your own machine). - Port: The port number MySQL listens on. The default port is
3306. - Name of the database: The specific database you want to connect to.
The function returns an engine object, which is the handle you will use to perform database operations.
engine = create_engine('driver://username:password@host:port/name_of_database', index=False)
For example, if your MySQL username is root, your password is mypassword, and you want to connect to a database called CARSHOWROOM on your local machine, the call would look like: …
| CarId | CarName | Price | Model | YearManufacture | Fueltype |
|---|---|---|---|---|---|
| D001 | Car1 | 582613.00 | LXI | 2017 | Petrol |
| D002 | Car1 | 673112.00 | VXI | 2018 | Petrol |
| B001 | Car2 | 567031.00 | Sigma1.2 | 2019 | Petrol |
| B002 | Car2 | 647858.00 | Delta1.2 | 2018 | Petrol |
| E001 | Car3 | 355205.00 | 5 STR STD | 2017 | CNG |
| E002 | Car3 | 654914.00 | CARE | 2018 | CNG |
| S001 | Car4 | 514000.00 | LXI | 2017 | Petrol |
| S002 | Car4 | 614000.00 | VXI | 2018 | Petrol |
8 rows in set (0.00 sec) …