AI vision models don't fail suddenly — they drift. The defect detection accuracy that was 97.3% at deployment becomes 91.2% six months later as product formulations change, packaging materials switch suppliers, and lighting conditions shift with the seasons. The dangerous part is that model drift is invisible without a structured performance review programme — the line keeps running, the model keeps passing units, and the quality escapes accumulate until a customer complaint or an audit makes the problem visible. This checklist covers every step of an AI vision model performance review cycle — from weekly accuracy monitoring through to full model retraining and validation — built for direct use with Oxmaint's AI Vision Inspection Integration and importable as a scheduled work order package.
1. Weekly Performance Monitoring
Weekly monitoring is the early warning system for model drift. The key metrics — defect detection rate, false reject rate, and confidence score distribution — should be reviewed every week by the quality engineer responsible for the vision system. A single week of degraded metrics may be noise. Two consecutive weeks signals an investigation. Three consecutive weeks triggers the retraining assessment in Section 3.
2. Monthly Performance Review
The monthly review is a structured assessment that goes deeper than weekly monitoring — comparing performance across product variants, shifts, and camera positions to identify patterns that weekly spot checks miss. A model that performs at 96% on the day shift but 88% on the night shift has a lighting problem, not a model problem. The monthly review is where that distinction is made.
3. Retraining Trigger Assessment
Retraining triggers are the defined conditions that escalate from monitoring to action. Every FMCG vision system should have documented thresholds — and when those thresholds are breached, the retraining assessment in this section determines whether the cause is a model problem (requires retraining), a hardware problem (requires maintenance), or a process change (requires new training data). Getting this diagnosis right before starting retraining saves weeks of work.
4. Model Retraining Process
Model retraining is a structured engineering process — not a one-click operation. The quality of the retraining outcome depends entirely on the quality of the training data: how many samples are in each defect class, how representative they are of current production conditions, and whether the annotations are accurate. Garbage in, garbage out applies to retraining as much as to initial model development.
5. Validation, Release, and Documentation
Model validation is the formal sign-off process that confirms the retrained model performs to specification before it takes control of production quality decisions. In regulated FMCG environments — those operating under FSMA preventive controls, SQF, or BRC — the validation record is a controlled document that must be retained and available for audit. The validation process below produces that record.
