Manage temporary files¶
The question. You need a scratch file or directory — for intermediate output, test fixtures, caching, or the 'write to temp then rename' trick for atomic file updates. You want it cleaned up when you're done, even if an exception slips out.
The answer: tempfile.TemporaryDirectory() or tempfile.NamedTemporaryFile(), both used with with. The with block guarantees cleanup; the system temp directory gets chosen for you; naming is collision-free.
# A temporary directory with guaranteed cleanup — the most flexible pattern.
# Hands you back a string path; you build any number of files inside.
import tempfile
from pathlib import Path
with tempfile.TemporaryDirectory(prefix='demo_') as tmpdir:
tmp = Path(tmpdir)
(tmp / 'data.txt').write_text('hello', encoding='utf-8')
(tmp / 'config.txt').write_text('setting=1', encoding='utf-8')
print(f'tmpdir: {tmp}')
print(f'files: {sorted(p.name for p in tmp.iterdir())}')
# Outside the with block: directory and everything in it are gone.
print(f'exists after: {Path(tmpdir).exists()}')
Variant: NamedTemporaryFile for a single file¶
When you need exactly one file and want its name immediately, NamedTemporaryFile is more direct. Remember to flush() before reading back via path (in-process file handles share a buffer; out-of-process readers don't).
import tempfile
from pathlib import Path
with tempfile.NamedTemporaryFile(
mode='w', suffix='.txt', encoding='utf-8', delete=True
) as f:
f.write('temporary data\n')
f.flush() # so Path().read_text() below sees it
print(f'name: {f.name}')
print(f'round-tripped: {Path(f.name).read_text(encoding="utf-8")!r}')
print(f'exists after with: {Path(f.name).exists()}') # False
Variant: atomic-write with delete=False + rename¶
The standard pattern for 'overwrite this file, but don't leave it half-written if the process dies'. Write to a temporary file in the same directory, then rename atomically over the target. Same-directory rename is atomic on all major filesystems.
import tempfile
from pathlib import Path
def safe_write(target: str | Path, content: str) -> None:
'''Atomic overwrite: write to a temp file, rename over the target.'''
target = Path(target)
target.parent.mkdir(parents=True, exist_ok=True)
with tempfile.NamedTemporaryFile(
mode='w', dir=target.parent, suffix='.tmp',
delete=False, encoding='utf-8',
) as f:
f.write(content)
tmp = Path(f.name)
tmp.replace(target) # atomic on same filesystem
safe_write('/tmp/important.txt', 'critical information\n')
print(Path('/tmp/important.txt').read_text(encoding='utf-8'))
Path('/tmp/important.txt').unlink()
Why this works¶
TemporaryDirectory creates a new, unique directory in the system temp location (/tmp on Linux/macOS, %TEMP% on Windows) and yields its path. On with-exit it recursively deletes the directory — success, exception, early return, all the same. That's the behaviour you want for a scratch area: no leaks, no stale files between test runs, no collisions with another process because the name is unique.
The path is a str, so wrap it in Path(tmpdir) if you want the pathlib API for joining and reading. Inside the block, write as many files as you like; they all disappear together on exit.
For a single short-lived file (write, read back, done), NamedTemporaryFile is more direct — see the extra cells. For the atomic-write pattern (write to temp, rename to target), delete=False + explicit rename is the canonical approach.
Trade-offs¶
Use TemporaryDirectory when you need more than one file, or when the code under test might create files itself and you just need a clean slate. Use NamedTemporaryFile (see extras) when you need exactly one file and want its name immediately.
Reach for mkdtemp() (no with) when the lifetime is longer than a single function — you'll clean up manually with shutil.rmtree inside your own try/finally. Reach for delete=False on NamedTemporaryFile when you need the file to survive the with block so you can rename it, hand the path to a subprocess, etc.
Tests especially benefit from tempfile. Each test gets its own scratch directory, passes, and cleans up after itself — no lingering artifacts between runs, no race between tests running in parallel.
Related reading¶
- Process large files — the atomic-write pattern for outputs bigger than you want to hold twice in memory.
- Avoid common file-handling mistakes — including the 'forget to clean up' trap this recipe sidesteps.
- pathlib quick reference — the
PathAPI for joining and reading the temp file's contents.