Data and reporting

How to Read a Text File in Python

Read a small UTF-8 file with pathlib, process a large file line by line, and troubleshoot missing paths, decoding errors, or empty reads.

Use Path("notes.txt").read_text(encoding="utf-8") to read a small text file into one string. For a large file, open it with a with block and loop over the lines. Both approaches need the file’s actual encoding and a path Python can find from its current working directory.

Read a small file as one string

from pathlib import Path

path = Path("notes.txt")
text = path.read_text(encoding="utf-8")
print(text)

Run this from a folder with a UTF-8 file named notes.txt. The result includes any final newline. The method opens and closes the file internally. This is handy for a small template or short notes where you need the complete string, and it is easy to replace the input path in a test.

A relative name starts at the process’s current working directory. It does not necessarily start at the directory where the script lives. If the file is bundled alongside a script, you can construct it relative to Path(__file__).resolve().parent in that script. If the user selects an input, keep that path an argument and document whether relative paths start in the current folder. __file__ is not a reliable starting point in an interactive notebook.

Count lines without holding the file in memory

from pathlib import Path

path = Path("notes.txt")
with path.open(encoding="utf-8") as handle:
    nonblank = sum(1 for line in handle if line.strip())
print(nonblank)

Python yields a line from the text stream as you iterate. The with block closes the handle when control leaves the block, including when an exception occurs. line.strip() is used here only to decide whether the line contains non-whitespace. If you later want the content exactly as delivered apart from a line terminator, avoid indiscriminately using strip(), which also removes meaningful spaces and tabs.

Typical line-by-line processing uses memory proportional to the current lines and whatever summaries you keep. An unusually long line can still be large. If the file is a CSV export, a quoted field can contain embedded line breaks, so a file line and a CSV record are different things. Use the CSV parser to count records. If a format defines records by bytes, open in binary mode and implement that format’s boundary rules.

Choose between read, readline, and iteration

An open text stream gives you several ways to advance. read() reads the rest of the stream into one string in the ordinary no-limit form. readline() reads one line and usually retains the newline when one is present. It returns an empty string at the end, while an empty line inside a file is represented by a newline. readlines() collects lines in a list. It is convenient when you need to revisit a small set but has a memory cost similar to materializing an iterable with list().

Reading advances the stream position. If a debug print calls handle.read() before the main loop, the following loop is already at end of file. Reopen the file or deliberately seek to the start where that makes sense. When you only need a first-line preview, calling readline() means the main loop will begin on the next line. Write that choice down rather than letting a debug statement change the analysis.

Fix a missing path or decoding error

FileNotFoundError means the selected path could not be found in that context. Print Path.cwd() and the path you attempted, then check the file name and extension. PermissionError means the operating system denied the action. Check ownership and access rules. UnicodeDecodeError occurs when the chosen codec cannot decode the bytes. Confirm the producer’s encoding instead of discarding the failing bytes automatically.

If the file opens but shows unexpected characters, compare a few names or identifiers with the original export. A wrong codec can produce readable text with corrupted values. Use the file encoding guide to distinguish a bad decode from a terminal display problem.

Check whether the file changed during the read

An exported report may come from another program that is still writing it. Your iteration can finish successfully and still see only a prefix of a large file. Agree on a readiness signal if the file arrives while the program is running. For example, a producer can put completed exports into the input directory only after closing them. If the file format includes a record count or footer, verify that count before trusting the results. Watch for the same issue when a batch process reuses an input name every day.

When your code makes a second pass through a file, ask whether the second open reads the same generation. Make a stable input copy using the workflow’s approved mechanism, or combine the calculations into a single pass. A one-pass count and classification can often use little memory while avoiding an accidental second read of a different file.

Decide whether you need the original bytes

Text mode may normalize common line endings while decoding. That is usually desirable when counting lines across Windows and Linux. If you need to compare exact file bytes or produce a checksum, use a binary stream or Path.read_bytes() for small files. Compute the digest from the original bytes rather than a re-encoded text view. Use the text view for human-readable processing and keep the binary source for byte-level verification when the distinction matters.

Sources and further reading

Related reading

Browse Data and reporting · Explore project tutorials

Search PythonProjects.org