AI Training Data Provenance Checker
Synthetic · source class

Synthetic data from a vendor

Bought synthetic data carries two provenances: the vendor's generator and the real data it learned from.

What to record for it

The vendor, the licence, what the generator was fitted on, and the version.

Personal data is unlikely: where your personal data column is blank the checker reads no, and says it assumed so.

A line that places here

example

Synthetic records | synthetic data vendor | internal

Check this line

What the checker reads on these lines

7 of the 12 findings

Clauses

3 regimes
RegimeClause
ISO/IEC 42001ISO/IEC 42001 A.7.4 Quality of data for AI systems
ISO/IEC 42001 A.7.6 Data preparation
ISO/IEC 42001 A.7.5 Data provenance
ISO/IEC 42001 A.10.3 Suppliers
NIST AI RMFNIST AI RMF MP-2.3 Data collection, selection and TEVV
NIST AI RMF MN-3.1 Third-party resources monitored
EU AI ActEU AI Act Art. 10 Data and data governance

The clauses, set out

ISO/IEC 42001 A.7.4Quality of data for AI systems

The organization shall define and document quality requirements for data and ensure they are met.

What an auditor asks to see: Data quality standards; Quality assessment reports; Quality dimensions defined (accuracy, completeness, representativeness, bias); Measurement evidence; Remediation records
What an auditor will probe: Is representativeness and bias assessed for training data?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.7.6Data preparation

The organization shall define and document criteria for selecting data preparations and the data preparation methods used.

What an auditor asks to see: Data preparation procedures; Transformation scripts; Approval records; Preparation methods (cleaning, labeling, augmentation, balancing); Decisions and rationale; Reproducibility evidence
What an auditor will probe: Are labeling decisions reviewed for bias?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.7.5Data provenance

The organization shall document the provenance of data used in AI systems to enable evaluation and traceability.

What an auditor asks to see: Provenance records; Lineage diagrams; End-to-end data lineage from source to model; Transformations documented; Datasheets
What an auditor will probe: Is lineage maintained automatically or relies on manual updates?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.10.3Suppliers

Establish a process ensuring that the organization's use of services, products or materials provided by suppliers aligns with its approach to the responsible development and use of AI systems.

What an auditor asks to see: Supplier assessment criteria covering responsible AI; Completed assessments for AI suppliers including model, dataset and component providers; Contract terms binding suppliers to the organization's AI requirements
What an auditor will probe: Standard security due diligence used with nothing AI specific; Datasets and pre-trained models sourced with no assessment; No reassessment when the supplier changes its model
Source: ISO/IEC 42001:2023
NIST AI RMF MP-2.3Data collection, selection and TEVV

Scientific integrity and TEVV considerations are identified and documented, including those related to experimental design, data collection and selection (e.g., availability, representativeness, suitability), system trustworthiness, and construct validation. The evaluation design is documented as a scientific claim: what was measured, on what data, and whether the measure validly stands for the property claimed.

What an auditor asks to see: Documented experimental design for evaluation of the system; Data collection and selection decisions, with availability, representativeness and suitability addressed; Construct validation showing the metric stands for the property claimed; The TEVV considerations identified and how each is handled
What an auditor will probe: Benchmark accuracy reported with no argument that it measures the property claimed; Data selection undocumented, so representativeness cannot be assessed; Evaluation designed by the same people optimising against it
Source: NIST AI Risk Management Framework
NIST AI RMF MN-3.1Third-party resources monitored

AI risks and benefits from third-party resources are regularly monitored, and risk controls are applied and documented. Third-party data, model, software and hardware dependencies are monitored on an ongoing basis, not assessed once at procurement, and the controls applied are recorded.

What an auditor asks to see: The inventory of third-party resources the AI system depends on; Monitoring records showing regular review of those dependencies; The risk controls applied to each and evidence they are in place; The route by which a third-party change reaches the risk owner
What an auditor will probe: Third parties assessed at onboarding and not monitored afterwards; Model or API version changes by the provider go unnoticed; Controls documented in the contract with no operational evidence
Source: NIST AI Risk Management Framework
EU AI Act Art. 10Data and data governance applies if this system is high-risk under Annex III

High-risk AI systems that make use of techniques involving the training of AI models shall use training, validation and testing data that meet the quality criteria in Art.10(2)-(5): appropriate data governance, examination for possible biases, identification of data gaps/shortcomings, statistically relevant datasets to the intended purpose, and considerations specific to the geographical, contextual, behavioural or functional setting of intended use.

What an auditor asks to see: Data governance procedures; Bias examination records and remediation; Data-quality assessment per dataset
What an auditor will probe: Training data used without bias examination; Datasets not representative of the deployment context
Source: EU AI Act

Other sources in synthetic