Day 82: Designing the validation protocol: 5 people, tape measure
Designing an honest test
The Stage 2 exit criterion is validation against 5 real people with a tape measure, per-dimension error recorded honestly. Today you design that protocol so the results are meaningful. Decide: which dimensions, how you'll take ground-truth tape measurements consistently, what photo conditions (and whether to include deliberately imperfect ones), and how you'll record the paired (predicted, true) values for analysis.
- Ground truth: define exactly how each tape measurement is taken (shoulder point to shoulder point, chest at armpit level) so it's repeatable.
- Conditions: capture each person in good conditions *and* a couple of realistic imperfect ones (slight angle, casual pose) to test degradation.
- Recording: log predicted cm, true cm, condition, and the engine's reported confidence for every dimension — so you can check whether confidence tracks actual error.
- Sample: 5 people is small; note it as a limitation. Vary body types where you can.
Test whether confidence is calibrated
The most valuable analysis isn't just 'how accurate?' but 'does the confidence predict the error?'. If high-confidence measurements are consistently accurate and low-confidence ones are consistently off, your confidence is *calibrated* and trustworthy. That's a sophisticated result most portfolio projects never check — and a memorable interview point.
Key terms
- Validation protocol
- A predefined, repeatable procedure for measuring ground truth and comparing it to predictions.
- Ground truth
- The true, independently-measured value (here, tape-measure dimensions) against which predictions are scored.
- Confidence calibration
- Whether reported confidence actually predicts accuracy — high confidence tracking low error.
Beyond accuracy, what sophisticated question should the validation check?
