AI Training Data Provenance Checker
Web-scraped · source class

General web crawl

A general crawl collects whatever it reaches: copyright text, personal pages and opt-out signals it may or may not have honoured.

What to record for it

The crawl, its date, the opt-out signals honoured, the filters applied and what was kept.

Personal data is likely: the checker assumes it where your personal data column is blank, labels the assumption, and words any finding that rests on it as a question.

A line that places here

example

Web text | web crawl | none recorded

Check this line

What the checker reads on these lines

9 of the 12 findings

Clauses

5 regimes
RegimeClause
ISO/IEC 42001ISO/IEC 42001 A.7.3 Acquisition of data
ISO/IEC 42001 A.7.5 Data provenance
NIST AI RMFNIST AI RMF MP-4.1 Legal risks of components and third-party data
NIST AI RMF GV-6.1 Third-party and intellectual property risk policy
EU AI ActEU AI Act Art. 10 Data and data governance
EU AI Act Art. 15 Accuracy, robustness and cybersecurity
EU AI Act Art. 53 general-purpose model providers only Obligations for providers of general-purpose AI models
GDPRGDPR Art. 14 Information where personal data have not been obtained from the data subject
GDPR Art. 6 Lawfulness of processing
UK GDPRUK GDPR Art. 14 Information to be provided where personal data have not been obtained from the data subject
UK GDPR Art. 6 Lawfulness of processing

The clauses, set out

ISO/IEC 42001 A.7.3Acquisition of data

The organization shall determine and document details about the acquisition and selection of data used in AI systems, including provenance and consent where applicable.

What an auditor asks to see: Data acquisition records; Provenance documentation; Consent records; Source identification; Licensing or consent evidence; Selection criteria and rejection rationale
What an auditor will probe: Is data provenance traceable to lawful sources?
Source: ISO/IEC 42001:2023
ISO/IEC 42001 A.7.5Data provenance

The organization shall document the provenance of data used in AI systems to enable evaluation and traceability.

What an auditor asks to see: Provenance records; Lineage diagrams; End-to-end data lineage from source to model; Transformations documented; Datasheets
What an auditor will probe: Is lineage maintained automatically or relies on manual updates?
Source: ISO/IEC 42001:2023
NIST AI RMF MP-4.1Legal risks of components and third-party data

Approaches for mapping AI technology and legal risks of its components – including the use of third-party data or software – are in place, followed, and documented, as are risks of infringement of a third-party’s intellectual property or other rights. There is a followed approach for mapping the technology and legal risk carried by each component, including data and software obtained from third parties and the rights position attached to them.

What an auditor asks to see: The documented approach for mapping component technology and legal risk; Component inventory identifying third-party data, models and software; Intellectual property and rights analysis for each third-party component; Evidence the approach was followed for the components actually in use
What an auditor will probe: Approach documented but not applied to components adopted since; Pre-trained models used with no analysis of the provenance of their training data; Rights reviewed for commercial components only, not for freely obtained ones
Source: NIST AI Risk Management Framework
NIST AI RMF GV-6.1Third-party and intellectual property risk policy

Policies and procedures are in place that address AI risks associated with third-party entities, including risks of infringement of a third party’s intellectual property or other rights. Third-party AI risk is addressed by policy covering data, models, software and services obtained externally, including the rights position on training data and model outputs.

What an auditor asks to see: Third-party AI policy covering data, pre-trained models, software and services; Due diligence records for third-party AI components in use; Contract terms addressing intellectual property, data rights and liability for AI components; The intellectual property position recorded for training data and model outputs
What an auditor will probe: Standard vendor due diligence applied with no AI-specific questions; Open source models adopted with no review of the licence or the training data provenance; Policy addresses suppliers but not freely obtained models and datasets
Source: NIST AI Risk Management Framework
EU AI Act Art. 10Data and data governance applies if this system is high-risk under Annex III

High-risk AI systems that make use of techniques involving the training of AI models shall use training, validation and testing data that meet the quality criteria in Art.10(2)-(5): appropriate data governance, examination for possible biases, identification of data gaps/shortcomings, statistically relevant datasets to the intended purpose, and considerations specific to the geographical, contextual, behavioural or functional setting of intended use.

What an auditor asks to see: Data governance procedures; Bias examination records and remediation; Data-quality assessment per dataset
What an auditor will probe: Training data used without bias examination; Datasets not representative of the deployment context
Source: EU AI Act
EU AI Act Art. 15Accuracy, robustness and cybersecurity a provider duty

High-risk AI systems shall be designed and developed in such a way that they achieve an appropriate level of accuracy, robustness, and cybersecurity, and shall perform consistently in those respects throughout their lifecycle. Resilience to errors, faults and inconsistencies; protection against attempts by unauthorised third parties to alter use, output or performance (incl data poisoning, model poisoning, adversarial examples and confidentiality attacks).

What an auditor asks to see: Accuracy/robustness measurements relevant to the intended purpose; Adversarial/data-poisoning threat modelling and mitigation; Cybersecurity controls aligned with state-of-the-art
What an auditor will probe: No adversarial-attack threat modelling; Accuracy claims not supported by test evidence
Source: EU AI Act
EU AI Act Art. 53Obligations for providers of general-purpose AI models providers of general-purpose models

Providers of general-purpose AI models must draw up and keep up to date the technical documentation of the model, including its training and testing process and the results of its evaluation, containing at least the Annex XI information, for provision on request to the AI Office and the national competent authorities; draw up, keep up to date and make available to providers who intend to integrate the model information and documentation containing at least the Annex XII elements and sufficient to let them understand the model's capabilities and limitations and meet their own obligations; put in place a policy to comply with Union law on copyright and related rights, including identifying and complying, through state of the art technologies, with a reservation of rights expressed under Art.4(3) of Directive (EU) 2019/790; and draw up and make publicly available a sufficiently detailed summary of the content used for training, following the template provided by the AI Office. The two documentation duties do not apply to models released under a free and open source licence meeting the stated conditions, unless the model has systemic risk. Providers must cooperate with the Commission and the national competent authorities, and where they neither adhere to an approved code of practice nor comply with a European harmonised standard they must demonstrate alternative adequate means of compliance for assessment by the Commission.

What an auditor asks to see: Model technical documentation checked against every Annex XI element, with training, testing and evaluation results included; The downstream integrator pack checked against Annex XII, with evidence it is actually supplied to integrators; The copyright compliance policy, naming the technologies used to identify and honour text and data mining reservations of rights; The published training content summary following the AI Office template, with its publication location and date; Where the open source exemption is claimed, evidence the licence and the published parameters, architecture and usage information meet the Art.53(2) conditions; Where neither an approved code of practice nor a harmonised standard is followed, the documented alternative adequate means of compliance
What an auditor will probe: A training content summary published at a level of generality that is not sufficiently detailed against the template; A copyright policy that states intent but names no technology for identifying reservations of rights; The open source exemption claimed for a model with systemic risk, where it does not apply; Downstream documentation limited to an interface reference, omitting the capability and limitation information integrators need to meet their own duties
Source: EU AI Act
GDPR Art. 14Information where personal data have not been obtained from the data subject

Where personal data has not been obtained from the data subject, provide the same identity, contact, purpose, legal basis, recipient and transfer information as Article 13, plus the categories of personal data concerned and the source the data came from including whether it was a publicly accessible source. Provide it within a reasonable period and at the latest within one month of obtaining the data, or at the latest at the first communication with the data subject if the data is used to communicate with them, or at the latest when the data is first disclosed to another recipient. The obligation does not apply where the data subject already has the information, where provision proves impossible or would involve disproportionate effort in which case appropriate protective measures including making the information publicly available must be taken, where obtaining or disclosure is expressly laid down by Union or Member State law with appropriate safeguards, or where the data must remain confidential under an obligation of professional secrecy.

What an auditor asks to see: A source register showing, per dataset, where the data came from and whether the source was publicly accessible; Evidence of the notification sent with its date, tested against the one month, first communication and first disclosure triggers; The disproportionate effort assessment where the exemption is relied on, showing what was weighed rather than only that a conclusion was reached; The alternative protective measures put in place under that exemption, including where the information was made publicly available; Supplier contract terms requiring the source of the data and the lawfulness of its collection to be disclosed
What an auditor will probe: Purchased or enriched marketing data used with no Article 14 notification at all, which is the most common finding in this area; Disproportionate effort claimed because notifying is inconvenient or costly rather than genuinely disproportionate; The source recorded as a vendor name with no indication of where the vendor itself obtained the data; Notification sent at the first marketing contact months after the data was obtained, missing the one month limit
Source: GDPR
GDPR Art. 6Lawfulness of processing

Process personal data only where at least one lawful basis applies: the data subject's consent, necessity for a contract with the data subject or pre-contractual steps at their request, compliance with a legal obligation, protection of vital interests, performance of a public interest task or exercise of official authority, or legitimate interests that are not overridden by the data subject's interests, rights and freedoms. Public authorities cannot rely on legitimate interests for processing carried out in performance of their tasks. Where the basis is legal obligation or public task, that basis must be laid down in Union or Member State law and the purpose must be determined in it. Before processing for a purpose other than the one collected for, without consent or a legal mandate, assess compatibility against the link between the purposes, the context of collection, the nature of the data, the consequences for the data subject and the safeguards in place.

What an auditor asks to see: A lawful basis recorded per processing activity, not per system or per department; Legitimate interests assessments showing the interest pursued, the necessity test and the balancing against the data subject's rights; The Union or Member State provision cited where the basis is legal obligation or public task; Compatibility assessments for each secondary use, covering the five factors Article 6(4) names; Evidence that the basis stated to the data subject in the privacy information matches the one recorded internally
What an auditor will probe: Consent recorded as the basis where the processing would happen regardless of the answer, which makes it neither free nor the real basis; Legitimate interests asserted with no balancing test on file, or a balancing test that never reaches an adverse conclusion for any activity; One lawful basis applied to a whole system that covers several distinct processing purposes; The basis switched after the fact when the first one fails, rather than settled before processing began
Source: GDPR
UK GDPR Art. 14Information to be provided where personal data have not been obtained from the data subject

Where data come from elsewhere the controller must give the Article 13 information plus the categories of data and the source (including whether publicly accessible), within a reasonable period and at the latest one month after obtaining the data, or at first communication with the data subject, or when first disclosing to another recipient, and must tell the data subject before further processing for a new purpose. The duty falls away where the data subject already has the information, where obtaining or disclosure is expressly required by domestic law with appropriate protections, where professional secrecy requires confidentiality, where providing the information is impossible or would involve disproportionate effort (judged by the number of data subjects, the age of the data and the safeguards), or where it would render impossible or seriously impair the purposes; a controller relying on the last two must protect the data subject's interests, including by publishing the information.

What an auditor asks to see: Notices sent to data subjects whose data came from third parties, with dates; Disproportionate effort assessments and the published information; Record of data sources per data set
What an auditor will probe: Enriched or purchased data used with no notice to the people concerned; Disproportionate effort claimed without a written assessment; Notice sent later than one month after receipt
Source: UK GDPR
UK GDPR Art. 6Lawfulness of processing

Processing is lawful only if at least one basis applies: consent for specific purposes, contract with the data subject, legal obligation, vital interests, a public task laid down in domestic law or relevant international law (section 9A of the 2018 Act), a recognised legitimate interest, or legitimate interests not overridden by the data subject's interests, rights and freedoms (particularly where the data subject is a child). Neither legitimate-interest basis is open to public authorities performing their tasks. A recognised legitimate interest (Article 6(1)(ea)) applies only where a condition in Annex 1 is met: disclosure on request to a body that states it needs the data for a public task, national security, public security or defence, responding to an emergency, detecting or preventing crime or prosecuting offenders, and safeguarding a vulnerable individual (under 18, or 18 or over and at risk); no balancing test is required for these. Article 6(11) gives direct marketing, intra-group transmission for internal administration and network and information security as examples of processing that may be necessary for legitimate interests, which still need the balancing test.

What an auditor asks to see: Lawful basis recorded per processing purpose; Legitimate interests assessments for Article 6(1)(f) processing; Annex 1 condition and the requesting body's written statement kept for each recognised legitimate interest disclosure
What an auditor will probe: Treating the Article 6(11) examples as automatically lawful without a balancing test; Public authorities relying on legitimate interests for their core tasks; Recognised legitimate interest claimed where no Annex 1 condition fits
Source: UK GDPR

Other sources in web-scraped