Skip to content

Informatics Practices · Ch 3 — Data Handling using Pandas – II

Import and Export of Data between Pandas and MySQL

3.9

Import and Export of Data between Pandas and MySQL

To work with real-world data, you will rarely type every value into a DataFrame by hand. Instead, data usually lives in files (like CSV or text files) or inside a database. The process of bringing data from a database into a Pandas DataFrame is called importing data. After you have finished analysing that data, you will often need to send the results back to the database for storage or further use — this is called exporting data.

Pandas can both read data from a MySQL database and write data to it. To make this possible, you first need a connection between your Python environment and the MySQL server. This connection is established using a database driver called pymysql. You must install this driver in your Python environment before you can proceed.

Note

Install the driver using the command: pip install pymysql

Along with the driver, you also need a library called SQLAlchemy. SQLAlchemy is a toolkit that helps Python interact with SQL databases. It handles the details of connecting, sending queries, and receiving results. Install it with:

pip install sqlalchemy

Creating the Connection with create_engine()

Once SQLAlchemy is installed, you use its create_engine() function to establish a connection. This function takes a special string called a connection string as its argument. The connection string contains all the information needed to locate and log in to your database. It is built from several parts:

  • Driver: The database driver you are using. For MySQL with pymysql, this is written as mysql+pymysql.
  • Username: Your MySQL username (commonly root).
  • Password: The password for that MySQL user.
  • Host: The address of the server (usually localhost if the database is on your own machine).
  • Port: The port number MySQL listens on. The default port is 3306.
  • Name of the database: The specific database you want to connect to.

The function returns an engine object, which is the handle you will use to perform database operations.

engine = create_engine('driver://username:password@host:port/name_of_database', index=False)

For example, if your MySQL username is root, your password is mypassword, and you want to connect to a database called CARSHOWROOM on your local machine, the call would look like: …

Table 3.54mysql> SELECT * FROM INVENTORY -- the CARSHOWROOM database's INVENTORY table
CarIdCarNamePriceModelYearManufactureFueltype
D001Car1582613.00LXI2017Petrol
D002Car1673112.00VXI2018Petrol
B001Car2567031.00Sigma1.22019Petrol
B002Car2647858.00Delta1.22018Petrol
E001Car3355205.005 STR STD2017CNG
E002Car3654914.00CARE2018CNG
S001Car4514000.00LXI2017Petrol
S002Car4614000.00VXI2018Petrol

8 rows in set (0.00 sec) …