How to meet it
OWASP AISVS v1.01
Training Data Integrity & Traceability · Training Data Quality and Security Assurance
Check auto-generated labels
Verify that automatically generated labels are subject to confidence thresholds and consistency checks to detect misleading or low-confidence labels.
OWASP AISVS v1.01, requirement 1.3.2
In plain English
When labels come from a model or a script, keep the ones it is sure about and send the uncertain or inconsistent ones to a person.
What good looks like
- Auto-labels below a confidence threshold are excluded or reviewed.
- Consistency checks compare auto-labels with a sample of human labels.
- Thresholds are set from measured error rates.
How teams usually do it
- Confidence scores from the labelling model stored with each label.
- Regular agreement checks between auto-labels and a gold-standard set.
Evidence an auditor would accept
- Threshold settings and how they were chosen.
- Recent agreement statistics.
Common mistakes
- Treating model-generated labels as ground truth for the next model.
- Never re-checking thresholds after the labelling model changes.
Not sure which requirements apply to your system? Take the assessment.
Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.