AI Training Data Provenance Checker
Regime

ISO/IEC 42001:2023: what it asks of a training dataset list

The AI management system standard. Annex A carries the data controls a dataset list is read against: data resources (A.4.3), data for development (A.7.2), acquisition of data (A.7.3), quality of data (A.7.4), data provenance (A.7.5) and data preparation (A.7.6).

Cited for every list.

Findings that cite it

FindingClause
Origin not recordedISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
ISO/IEC 42001 A.4.2
Terms not recordedISO/IEC 42001 A.7.3
Terms that need review for this useISO/IEC 42001 A.7.3
Personal data with no lawful basis recorded, or reused from another purposeISO/IEC 42001 A.7.3
Article 9, criminal offence or children's data declaredISO/IEC 42001 A.5.4
Labelled by a vendor, a crowd or a model with no quality check recordedISO/IEC 42001 A.7.4
ISO/IEC 42001 A.7.6
No version or snapshot dateISO/IEC 42001 A.7.5
ISO/IEC 42001 A.6.2.3
Declared potential overlap between train and testISO/IEC 42001 A.6.2.4
ISO/IEC 42001 A.7.4
High-risk use with no bias examination recordedISO/IEC 42001 A.7.4
ISO/IEC 42001 A.5.4
Older than your N-year policy threshold (a threshold you set, not a legal deadline)ISO/IEC 42001 A.7.4
ISO/IEC 42001 A.7.2
Content where a label belongsISO/IEC 42001 A.4.3
Vendor data with no provider namedISO/IEC 42001 A.10.3
ISO/IEC 42001 A.7.3

Source classes anchored here

42
Source classClause
Claims system extractISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
Policy and underwriting systemISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
CRM exportISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
ERP and finance systemISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
HR records and staff surveysISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
Service desk and ticketingISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
Application logs and telemetryISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
Internal document storeISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
Email archiveISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
Data warehouse or lake extractISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
Internal, system not namedISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.4.3
Support and call transcriptsISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Chat logsISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Reviews on your own siteISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Survey and complaint free textISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Call recordingsISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Licensed vendor datasetISO/IEC 42001 A.7.3
ISO/IEC 42001 A.10.3
Data broker listISO/IEC 42001 A.7.3
ISO/IEC 42001 A.10.3
Annotation vendor outputISO/IEC 42001 A.7.3
ISO/IEC 42001 A.10.3
Licensed image or media libraryISO/IEC 42001 A.7.3
ISO/IEC 42001 A.10.3
Credit reference extractISO/IEC 42001 A.7.3
ISO/IEC 42001 A.10.3
Government open dataISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Academic or research datasetISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Open benchmarkISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Public register or recordsISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Open source codeISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Forum postsISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
News articlesISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Social media postsISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
General web crawlISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Specific website scrapeISO/IEC 42001 A.7.3
ISO/IEC 42001 A.7.5
Synthetic data, generated in houseISO/IEC 42001 A.7.4
ISO/IEC 42001 A.7.6
ISO/IEC 42001 A.7.5
Synthetic data from a vendorISO/IEC 42001 A.7.4
ISO/IEC 42001 A.7.6
ISO/IEC 42001 A.7.5
ISO/IEC 42001 A.10.3
Simulation outputISO/IEC 42001 A.7.4
ISO/IEC 42001 A.7.6
ISO/IEC 42001 A.7.5
Labels produced by a modelISO/IEC 42001 A.7.6
ISO/IEC 42001 A.7.4
Text generated by a modelISO/IEC 42001 A.7.6
ISO/IEC 42001 A.7.4
Distillation outputsISO/IEC 42001 A.7.6
ISO/IEC 42001 A.7.4
Pre-trained model or embeddingsISO/IEC 42001 A.7.6
ISO/IEC 42001 A.7.4
Data shared under an agreementISO/IEC 42001 A.7.3
ISO/IEC 42001 A.10.3
Joint venture dataISO/IEC 42001 A.7.3
ISO/IEC 42001 A.10.3
Reinsurer, broker or bank feedISO/IEC 42001 A.7.3
ISO/IEC 42001 A.10.3
Industry consortium dataISO/IEC 42001 A.7.3
ISO/IEC 42001 A.10.3

ISO/IEC 42001:2023: every clause cited

11 of the 38 held

The requirement text is our statement of each clause, read against the copy we hold and cited to it; it is not the instrument verbatim.

ISO/IEC 42001 A.4.2Resource documentation

Resources needed for the AI life cycle (data, tooling, compute, human) shall be identified and documented.

What an auditor asks to see: Resource inventory; Capacity planning records; Per-system resource documentation; Compute, storage, data, human expertise listed
What an auditor will probe: Is resource documentation maintained per AI system or only generic?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.4.3Data resources

As part of identifying resources, the organization shall document information about the data resources utilized for the AI system.

What an auditor asks to see: Data inventory; Data lineage records; Datasheets; Data sources, types, volumes, sensitivity; Provenance and licensing; Data quality metadata
What an auditor will probe: Is data lineage maintained from source to model?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.5.4Assessing AI system impact on individuals or groups

The organization shall assess and document the potential impacts of AI systems to individuals or groups of individuals throughout the system's life cycle.

What an auditor asks to see: Per-system impact assessments; Fairness analysis; Privacy impact (link to PIA where relevant); Autonomy and rights considerations
What an auditor will probe: Are demographic-specific impacts analyzed where relevant?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.6.2.3Documentation of AI system design and development

Documentation of AI system design and development activities shall be maintained.

What an auditor asks to see: Design documents; Architecture diagrams; Model documentation; Algorithm choices and rationale; Model cards or equivalent; Code repositories with provenance
What an auditor will probe: Is documentation sufficient to reconstruct decisions later?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.6.2.4AI system verification and validation

AI systems shall be verified and validated, and the results documented. Verification confirms requirements are met; validation confirms intended use is achieved.

What an auditor asks to see: V&V plans; Test results; Acceptance reports; Test plans covering accuracy, robustness, bias, security; Independent verification where required; Documented sign-off
What an auditor will probe: Are V&V results signed off independently from developers?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.7.2Data for development and enhancement of AI systems

The organization shall define, document, and implement processes to determine data requirements and ensure data quality for AI system development and enhancement.

What an auditor asks to see: Data requirements specification; Data quality procedure; Defined data requirements per AI system; Quality criteria (accuracy, completeness, timeliness, representativeness); Quality measurement records
What an auditor will probe: Is data quality measured or assumed?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.7.3Acquisition of data

The organization shall determine and document details about the acquisition and selection of data used in AI systems, including provenance and consent where applicable.

What an auditor asks to see: Data acquisition records; Provenance documentation; Consent records; Source identification; Licensing or consent evidence; Selection criteria and rejection rationale
What an auditor will probe: Is data provenance traceable to lawful sources?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.7.4Quality of data for AI systems

The organization shall define and document quality requirements for data and ensure they are met.

What an auditor asks to see: Data quality standards; Quality assessment reports; Quality dimensions defined (accuracy, completeness, representativeness, bias); Measurement evidence; Remediation records
What an auditor will probe: Is representativeness and bias assessed for training data?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.7.5Data provenance

The organization shall document the provenance of data used in AI systems to enable evaluation and traceability.

What an auditor asks to see: Provenance records; Lineage diagrams; End-to-end data lineage from source to model; Transformations documented; Datasheets
What an auditor will probe: Is lineage maintained automatically or relies on manual updates?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.7.6Data preparation

The organization shall define and document criteria for selecting data preparations and the data preparation methods used.

What an auditor asks to see: Data preparation procedures; Transformation scripts; Approval records; Preparation methods (cleaning, labeling, augmentation, balancing); Decisions and rationale; Reproducibility evidence
What an auditor will probe: Are labeling decisions reviewed for bias?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.10.3Suppliers

Establish a process ensuring that the organization's use of services, products or materials provided by suppliers aligns with its approach to the responsible development and use of AI systems.

What an auditor asks to see: Supplier assessment criteria covering responsible AI; Completed assessments for AI suppliers including model, dataset and component providers; Contract terms binding suppliers to the organization's AI requirements
What an auditor will probe: Standard security due diligence used with nothing AI specific; Datasets and pre-trained models sourced with no assessment; No reassessment when the supplier changes its model
Source: ISO/IEC 42001:2023