Day 62: SAM/SAM2: promptable segmentation
The segmentation foundation model
SAM (Segment Anything Model) and its successor SAM2 are foundation models for segmentation: trained on billions of masks, they segment almost anything given a prompt — a point, a box, or text. This is a leap over Stage 0's GrabCut: where GrabCut used color statistics and often failed on similar-colored backgrounds, SAM has learned actual object structure. Prompt it with your YOLO person box and it returns a precise person mask.
from ultralytics import SAM
sam = SAM("sam2_b.pt")
# prompt with the YOLO person box -> precise mask
result = sam("customer.jpg", bboxes=[[x1, y1, x2, y2]])
person_mask = result[0].masks.data[0].cpu().numpy() # boolean HxW maskCallback to Stage 0 Day 11
Remember GrabCut's failure modes — garment/background color overlap, cluttered scenes? SAM is precisely the neural upgrade that lesson promised. You felt where the classical method broke; now you see why a model trained on billions of masks doesn't break the same way. That contrast — classical vs learned — is worth articulating in an interview.
Key terms
- SAM / SAM2
- Segment Anything Model — a promptable segmentation foundation model trained on billions of masks.
- Segmentation prompt
- The input (point, box, or text) that tells SAM which object to segment.
- Foundation model
- A large model pretrained on massive data, adaptable to many tasks — here, segmenting almost any object.
Why does SAM handle a garment matching its background color far better than Stage 0's GrabCut?
