File handling using pickle module

Why Pickle? The Problem with Binary Files

When you write data to a text file, everything becomes a string. Numbers, lists, dictionaries — all get converted to text. That works fine for simple output, but what if you want to store a Python list or dictionary as a Python object and read it back exactly as it was? A text file cannot do that. When you read back "32600" from a text file, you get a string, not an integer.

Binary files solve this, but they come with their own challenge: how do you write a complex Python object (like a list containing an integer, a string, and more integers) into a stream of bytes and then reconstruct it perfectly? That is exactly what the pickle module does.

What is Pickling?

Pickling is the process of converting a Python object into a byte stream. The byte stream can then be written to a binary file. The reverse process — reading the byte stream from a file and reconstructing the original Python object — is called unpickling.

Think of it like this: you have a Python object (say, a list of employee details). Pickling "freezes" that object into a sequence of bytes. You write those bytes to a file. Later, you read those bytes back and "thaw" them into the exact same Python object, with the same data types, structure, and values.

The Two Key Functions: dump() and load()

The pickle module provides two primary functions for working with binary files.

pickle.dump() — Writing (Serialization)

This function takes a Python object and writes its pickled representation to a binary file.

Syntax:

pickle.dump(object, file_object)

• object: The Python object you want to store (list, dictionary, tuple, etc.).
• file_object: The file handle of a binary file opened in write or append mode ('wb' or 'ab').

pickle.load() — Reading (Deserialization)

This function reads a pickled byte stream from a binary file and reconstructs the original Python object.

Syntax:

object = pickle.load(file_object)

• file_object: The file handle of a binary file opened in read mode ('rb').
• Returns: The reconstructed Python object.

How It Works: A Complete Example

The textbook walks through Program 2-8, which manages employee records in a binary file. Here is the complete program, followed by its logic broken down step by step.

Program 2-8: To perform basic operations on a binary file using pickle module

# Program to write and read employee records in a binary file
import pickle
print("WORKING WITH BINARY FILES")
bfile=open("empfile.dat","ab")
recno=1
print ("Enter Records of Employees")
print()
while True:
    print("RECORD No.", recno)
    eno=int(input("\tEmployee number : "))
    ename=input("\tEmployee Name : ")
    ebasic=int(input("\tBasic Salary : "))
    allow=int(input("\tAllowances : "))
    totsal=ebasic+allow
    print("\tTOTAL SALARY : ", totsal)
    edata=[eno,ename,ebasic,allow,totsal]
    pickle.dump(edata,bfile)
    ans=input("Do you wish to enter more records (y/n)? ")
    recno=recno+1
    if ans.lower()=='n':
        print("Record entry OVER ")
        print()
        break
print("Size of binary file (in bytes):",bfile.tell())
bfile.close()
print("Now reading the employee records from the file")
print()
readrec=1
try:
    with open("empfile.dat","rb") as bfile:
        while True:
            edata=pickle.load(bfile)
            print("Record Number : ",readrec)
            print(edata)
            readrec=readrec+1
except EOFError:
    pass

Output of Program 2-8:

>>>
 RESTART: Path_to_file\Program2-8.py
WORKING WITH BINARY FILES
Enter Records of Employees
RECORD No. 1
	Employee number : 11
	Employee Name : D N Ravi
	Basic Salary : 32600
	Allowances : 4400
	TOTAL SALARY :  37000
Do you wish to enter more records (y/n)? y
RECORD No. 2
	Employee number : 12
	Employee Name : Farida Ahmed
	Basic Salary : 38250
	Allowances : 5300
	TOTAL SALARY :  43550
Do you wish to enter more records (y/n)? n
Record entry OVER
Size of binary file (in bytes): 216
Now reading the employee records from the file
Record Number :  1
[11, 'D N Ravi', 32600, 4400, 37000]
Record Number :  2
[12, 'Farida Ahmed', 38250, 5300, 43550]
>>>

Writing Records (Appending to the File)

1. Open the file in append-binary mode. The mode 'ab' is used so that new records are added to the end of the file without erasing existing data.

   bfile = open("empfile.dat", "ab")
   

2. Create a list for each employee record. The program collects employee number, name, basic salary, and allowances from the user. It calculates the total salary and packs all five pieces of data into a single list.

   edata = [eno, ename, ebasic, allow, totsal]
   

3. Dump the list into the file. The entire list is pickled and written to the binary file in one call.

   pickle.dump(edata, bfile)
   

4. Repeat for multiple records. The program loops until the user chooses to stop.

5. Check the file size. Before closing, bfile.tell() returns the current position of the file object, which equals the total size of the file in bytes. In the example output, after two records, the file size is 216 bytes.

6. Close the file. This is essential after writing to ensure all data is flushed to disk.

Reading Records (Displaying the Data)

1. Open the file in read-binary mode. The mode 'rb' is used.

   with open("empfile.dat", "rb") as bfile:
   

2. Load records in a loop. Each call to pickle.load(bfile) reads the next pickled object from the file and reconstructs it as a Python list.

   edata = pickle.load(bfile)
   

3. Handle the end of file. When there are no more records to read, pickle.load() raises an EOFError. The program catches this exception using a try...except block and exits the loop gracefully.

   try:
       while True:
           edata = pickle.load(bfile)
           print(edata)
   except EOFError:
       pass
   

4. Display each record. Since each record was stored as a list, the output shows a list like [11, 'D N Ravi', 32600, 4400, 37000].

The try...except Block: Handling EOFError

Binary files do not have a natural "end-of-line" marker like text files do. When you read a text file, readline() returns an empty string when the file ends. But with pickle.load(), there is no such signal. Instead, Python raises an EOFError (End of File Error) when there is no more data to read.

The standard pattern is to wrap the reading loop in a try...except block:

try:
    with open("empfile.dat", "rb") as bfile:
        while True:
            edata = pickle.load(bfile)
            # process the record
except EOFError:
    pass   # end of file reached, exit gracefully
``` …