How Do You Read and Write Files in Python?

The open function returns a file object you read from or write to, and the with statement closes it when the block ends.

import os
import tempfile

path = os.path.join(tempfile.gettempdir(), "tutorsbot_demo.txt")

with open(path, "w", encoding="utf-8") as handle:
    handle.write("first line\n")
    handle.write("second line\n")

with open(path, "r", encoding="utf-8") as handle:
    content = handle.read()

print(content)
print(len(content.splitlines()))

os.remove(path)

Writing with the w mode creates the file if it does not exist and truncates it if it does. Reading it back with read gives the whole contents as one string, and splitlines confirms there are two lines.

The three reading methods are worth knowing separately. read returns the entire file as a single string, readline returns just the next line, and readlines returns a list of all the lines. Each also has an optional size argument for reading a fixed number of characters or lines, which is how you peek at the beginning of a large file without loading it.

Two habits from the snippet are worth keeping. Always pass encoding explicitly, because the default depends on your operating system's locale and can differ between a developer machine and a server. And always use the with statement, so the file is closed even if an exception happens in the middle of the block.

There are three families of API, and the built-in open is the one to use by default.

The built-in open function returns a file object with read, write, readline, readlines and iteration support. It handles buffering, encoding and newline translation for you, and combined with the with statement it gives you a block that always closes the file.

The pathlib module is the modern, recommended way to express the same thing. Path objects carry the path around with them, so you write path.read_text() and path.write_text(data) without repeating the filename, and they have a rich set of methods for joining paths, checking existence and globbing. It is the same machinery underneath — pathlib.Path.open returns a file object just like open does.

The os module offers the low-level functions: os.read, os.write, os.open and os.fdopen, which work with file descriptors. These are what you need when you are dealing with pipes, sockets or non-blocking I/O, and they are the wrong tool for ordinary file handling.

Two more options for special cases. The csv module reads and writes delimited files correctly, handling quoting and embedded newlines in a way that splitting on commas never will. And the shutil module has copy, move and copyfile for whole-file operations.

The guidance to give in an interview is: use pathlib for paths, the csv module for delimited data, and the built-in open with the with statement and an explicit encoding for everything else.

Iterating the file object directly reads one line at a time, so memory use stays flat no matter how large the file is.

import os
import tempfile

path = os.path.join(tempfile.gettempdir(), "tutorsbot_lines.txt")

with open(path, "w", encoding="utf-8") as handle:
    for number in range(1, 10001):
        handle.write(f"line {number}\n")

total = 0
count = 0
with open(path, "r", encoding="utf-8") as handle:
    for line in handle:
        count += 1
        total += int(line.split()[1])

print("lines:", count)
print("sum:", total)

os.remove(path)

The loop reads 10,000 lines and adds up the numbers, and the memory used is one line at a time. If you had used read instead, the whole file would have been loaded as a single string first.

The mechanism is that a file object is an iterator. Calling next on it returns the next line, so the for loop pulls one line at a time and the file object's internal buffer does the reading in fixed-size chunks. The file position is kept between calls, which is also why you can mix a for loop with readline and the position stays where you left it.

Two variants to know. If you want the lines without the trailing newline, use a generator over the file that strips each line, and if the lines are huge you can read fixed-size blocks with a while loop and read(size). For CSV data, the csv module already streams row by row when you iterate it, which is the pattern to use for very large files.