Day 2: Python idiom II: comprehensions, generators, typing, context managers
The idiom that makes Python read like Python
You'll read far more Python this roadmap than you write from scratch — library source, other people's training scripts, Stack Overflow answers. Four idioms account for most of what makes Python code look different from the languages you already know: comprehensions, generators, type hints, and context managers.
Comprehensions
A list/dict/set comprehension is a single-expression loop that builds a collection — Python's answer to .map()/.filter() chains, but read left-to-right as 'for each x, keep it if condition, transform it'.
# imperative
result = []
for path in image_paths:
if path.suffix in (".jpg", ".png"):
result.append(path.stem)
# idiomatic
result = [p.stem for p in image_paths if p.suffix in (".jpg", ".png")]
# dict comprehension
sizes = {p.stem: p.stat().st_size for p in image_paths}Generators: lazy, one item at a time
A generator ((x for x in ...) or a function with yield) produces values on demand instead of building the whole collection in memory up front. For a catalog of 50,000 garment photos, [load(p) for p in paths] loads all 50,000 into RAM before you touch any of them; a generator loads one at a time as your pipeline consumes it.
def load_images(paths):
for p in paths:
yield cv2.imread(str(p))
# nothing is loaded yet — this just builds the generator object
images = load_images(catalog_paths)
# images are loaded one at a time as the loop pulls them
for img in images:
process(img)Payoff, later
PyTorch's DataLoader (Stage 1) is a generator under the hood — it yields one batch at a time instead of materializing an entire dataset in memory. Same idea, applied to training data instead of catalog photos.
Type hints
Python is dynamically typed, but function signatures can carry optional type hints, checked by a separate tool (mypy, or your editor) rather than the interpreter itself. Treat them as documentation your IDE enforces — cheap insurance in a codebase you'll be reading more than writing.
from pathlib import Path
import numpy as np
def remove_background(image: np.ndarray) -> np.ndarray:
"""Returns an image with the background masked to transparent."""
...
def process_catalog(paths: list[Path]) -> list[np.ndarray]:
return [remove_background(cv2.imread(str(p))) for p in paths]Context managers: the with statement
with guarantees cleanup runs even if the block raises — the Python equivalent of try/finally, or a using block. File handles, and later model/GPU-memory contexts (torch.no_grad() in Stage 1), all lean on this pattern.
# file is guaranteed to be closed even if reading raises
with open("catalog.json") as f:
data = json.load(f)
# Stage 1 preview: same pattern, disabling gradient tracking for inference
with torch.no_grad():
prediction = model(image_tensor)Key terms
- Comprehension
- A single-expression syntax for building a list/dict/set from an iterable, optionally filtered and transformed.
- Generator
- An iterable that produces values lazily, one at a time, instead of materializing the whole collection in memory.
- Type hint
- Optional, non-enforced-at-runtime type annotation on variables/parameters, checked by external tools like mypy.
- Context manager
- An object implementing __enter__/__exit__, used via
with, that guarantees cleanup code runs even on exception.
You need to process 50,000 catalog images one at a time without ever holding all 50,000 in memory. Which tool fits?
