Review

1. File Handling in Python

File Handling allows Python programs to interact directly with the computer's storage system to read, write, append, and manage external files. Whether you are parsing server logs, saving application configuration settings, processing CSV datasets, or persisting user state, mastering file input/output (I/O) is a core skill for building production-grade software.

Learning Objectives

• Understand file modes (r, w, a, x, b) and text encoding principles.
• Use the with context manager to guarantee safe file opening and automatic resource cleanup.
• Master efficient file reading patterns (line-by-line streaming vs. full buffer reading).
• Perform modern path manipulations safely using Python’s standard pathlib module.
• Safely handle I/O exceptions such as FileNotFoundError and PermissionError.

2. Opening and Closing Files Safely

Python provides the built-in open() function to establish a stream connection between your program and a target file. Historically, developers manually called open() and later close(). However, if an error occurred before reaching close(), the file handle remained open in memory, causing potential data loss or memory leaks.

The modern, Pythonic standard is using the with statement—known as a Context Manager. It automatically handles closing the file context as soon as execution exits the block, even if an exception is raised inside.

Manual vs. Context Manager Approach:

# ❌ BAD: Manual file handling (Prone to resource leaks if errors occur)
file = open("data.txt", "r", encoding="utf-8")
content = file.read()
# If an exception happens here, file.close() never runs!
file.close()


# ✅ GOOD: Pythonic Context Manager (Auto-closes file upon exit)
with open("data.txt", "r", encoding="utf-8") as file:
    content = file.read()
    # File is automatically closed when exiting this indented block!

3. File Operations & Modes Matrix

When calling open(filename, mode, encoding), the mode argument determines what operations are permitted and how Python treats existing data:

Mode	Operation	If File Exists?	If File Missing?	Primary Use Case
'r'	Read Only	Opens at start	Raises FileNotFoundError	Reading text content without modifying.
'w'	Write Only	Truncates (Overwrites completely!)	Creates new file	Creating new files or overwriting existing contents.
'a'	Append Only	Appends to end of file	Creates new file	Adding log lines or activity history without overwriting.
'x'	Exclusive Creation	Raises FileExistsError	Creates new file	Preventing accidental overwrite of critical files.
'b'	Binary Mode	Appended to modes (e.g., 'rb', 'wb')	Depends on base mode	Handling images, PDFs, audio, or compiled binaries.

4. Reading and Writing Strategies

Reading Large Files Efficiently

While .read() loads the entire file contents into memory as a single string, doing this on multi-gigabyte log files will crash your application. Instead, use Python file objects as iterators to stream content line by line with minimal memory consumption.

# ❌ Memory Inefficient (Loads entire 2GB file into RAM at once)
with open("huge_system.log", "r", encoding="utf-8") as file:
    lines = file.readlines()  # Creates a massive list in memory
    for line in lines:
        print(line.strip())

# ✅ Memory Efficient Streaming (Loads ONE line into RAM at a time)
with open("huge_system.log", "r", encoding="utf-8") as file:
    for line in file:  # Iterates line by line
        if "ERROR" in line:
            print(f"Log Alert: {line.strip()}")

Writing and Appending Content

Use .write() for individual strings or .writelines() to output a list of pre-formatted string entries.

log_entries = [
    "2026-07-23 10:00:01 INFO User logged in\n",
    "2026-07-23 10:02:15 WARNING High CPU usage detected\n",
    "2026-07-23 10:05:30 ERROR Database connection timeout\n"
]

# Write mode overwrites file
with open("app.log", "w", encoding="utf-8") as file:
    file.writelines(log_entries)

# Append mode adds to existing content
with open("app.log", "a", encoding="utf-8") as file:
    file.write("2026-07-23 10:10:00 INFO Service restored\n")

5. Modern File Handling with pathlib

While the traditional os.path module relies on raw string operations, standard Python modern code uses pathlib.Path. It provides object-oriented path manipulation that works cross-platform seamlessly (handling Windows \ vs Unix / differences automatically).

from pathlib import Path

# Create a Path object
data_dir = Path("data")
config_file = data_dir / "config.json"  # Overloaded / operator for paths!

# Create directory if it doesn't exist
data_dir.mkdir(parents=True, exist_ok=True)

# Quick read/write helper methods without explicit open context managers!
config_file.write_text('{"theme": "dark", "version": "1.2"}', encoding="utf-8")

if config_file.exists():
    content = config_file.read_text(encoding="utf-8")
    print(f"File Contents:\n{content}")

6. Common Beginner Mistakes

7. Code Examples

Here are practical production patterns for working with structured files and exception-safe file operations:

import json
from pathlib import Path

# 1. Safe JSON Data Storage Pattern
def save_user_profile(user_data, filepath):
    path = Path(filepath)
    
    # Ensure parent directory exists
    path.parent.mkdir(parents=True, exist_ok=True)
    
    try:
        with open(path, "w", encoding="utf-8") as file:
            json.dump(user_data, file, indent=4)
        print(f"Successfully saved profile to {path}")
    except PermissionError:
        print(f"Error: Insufficient permission to write to {path}")

# 2. Robust File Reading with Exception Handling
def load_configuration(filepath):
    path = Path(filepath)
    
    try:
        with open(path, "r", encoding="utf-8") as file:
            return json.load(file)
    except FileNotFoundError:
        print(f"Config file '{filepath}' missing! Returning default configuration.")
        return {"theme": "default", "notifications": True}
    except json.JSONDecodeError:
        print(f"Config file '{filepath}' is corrupted! Failed to parse JSON.")
        return {}

# Executing code
profile = {"name": "Alex", "role": "Developer", "level": 5}
save_user_profile(profile, "settings/user_profile.json")

config = load_configuration("settings/user_profile.json")
print("Loaded Profile:", config)

Interactive Practice

Mini Practice: Exception-Safe Log Counter

Write a Python function count_log_errors(filename) that opens a log file using a context manager, iterates line by line, counts how many lines contain "ERROR", and returns the total. Handle FileNotFoundError by returning 0.

0% complete

Your code

Check when done

I used a with open(..., encoding='utf-8') context manager I iterated over lines efficiently without loading the entire file at once I caught FileNotFoundError to prevent runtime crashes

Quiz

Test Your Understanding

1. What is the primary advantage of opening files using the `with open(...)` statement?

It automatically converts all written text to uppercase string format. It guarantees the file is automatically closed when execution leaves the block, preventing resource leaks. It prevents any exception from ever being raised inside the block.

2. What happens if you open an existing text file with mode `'w'`?

New content is appended to the bottom without altering existing text. Python raises a FileExistsError exception immediately. The existing file contents are instantly erased (truncated to 0 bytes).

3. Which method is most memory-efficient for processing a multi-gigabyte log file?

Calling file.read() and splitting by newline characters. Iterating directly over the file object in a loop (`for line in file:`). Calling file.readlines() to parse a list of strings.

4. Why is explicitly passing `encoding='utf-8'` strongly recommended when opening text files?

It ensures consistent cross-platform behavior and avoids OS-dependent decoding errors. It allows Python to automatically compress text files on disk. It is required by Python syntax or a SyntaxError is raised.

Submit Answers

8. Challenge Project

Log File Audit & Summary Generator: Build a command-line script that analyzes a system log file and writes a summary report:

1. Prompt the user or accept a file path using pathlib.Path.
2. Safely open the target log file using exception handling for missing files.
3. Stream through the file line by line to count occurrences of "INFO", "WARNING", and "ERROR" log levels.
4. Extract all "ERROR" lines and write them into a separate report file named error_summary.txt.
5. Append a timestamped execution log to an audit.log file using append mode ('a').

9. Key Takeaways

• Always use the with open(...) context manager to ensure files are safely closed.
• Specify encoding="utf-8" explicitly for cross-platform compatibility.
• Stream large files line by line using for line in file: to maintain low memory usage.
• Use pathlib.Path for clean, cross-platform path creation and management.
• Differentiate between write mode ('w' - truncates) and append mode ('a' - preserves).