Data Quality and Data Governance for AI in GxP Environments

AI applications can detect defective products, predict process deviations or provide quality-related information. Their ability to perform these tasks reliably depends to a large extent on the data on which they rely. The key question is whether those data are suitable for the intended use.

Data quality, data governance and data qualification are closely connected: Data quality describes how well data meet the requirements of their intended use. Data governance defines responsibilities and how data are managed throughout their lifecycle. Data qualification assesses and documents whether a specific dataset is suitable for its defined purpose. Quality by Design (QbD) supports early planning of the data quality required.

The regulatory examples in this article relate mainly to medicinal products and GMP. Applicability to other GxP uses must be assessed separately.

Data quality depends on the intended use

A dataset may be adequate for its original purpose yet have significant gaps when used in a new AI application.

Temperature data from a temperature-controlled cabinet, such as an incubator, illustrate this point: hourly averages may be sufficient for a monthly trend report. If an AI application is intended to detect changes in temperature control behaviour, such as gradual temperature drift, brief temperature spikes or increasing fluctuations, then the dataset must adequately represent those changes. Even correctly calculated and fully archived hourly averages can obscure important temporal patterns. The original readings therefore need a temporal resolution appropriate for the task and must cover a sufficiently long observation period.

A validated incubator temperature-monitoring system does not automatically establish that its exported data are suitable for the AI application. The assessment must consider whether the export contains the required values and metadata and whether the data adequately represent the intended operating conditions.

The starting point is therefore the intended use: What task does the AI perform? What decision does it influence? For which products, processes and operating conditions is it intended? What would be the consequences of incorrect results?

These questions determine the data requirements. For visual inspection of packaging, relevant factors include defect types, packaging variants and image acquisition conditions. For process prediction, measurement frequency and the temporal alignment of measured values may be critical.

Consider data integrity and fitness for use together

In GxP environments, data integrity is fundamental to reliable records. The ALCOA++ principles address attributability, legibility, contemporaneous recording, original records and accuracy. They also cover completeness, consistency, enduring records, availability and traceability throughout the data lifecycle. (EMA/INS/GCP/112288/2023, Section 4.5)

Data integrity alone does not demonstrate fitness for a specific AI application. Even records that meet data integrity requirements may contain no examples of rare product defects or may inadequately represent operating conditions at other sites.

Data also need their business and scientific context. A measured value such as “pH 7.0” cannot be assessed reliably without information about the sample, time, method and result status. Such descriptive information is called metadata. If an export omits metadata needed to establish these relationships and interpret the values, the exported dataset is incomplete for that purpose and cannot be assessed unambiguously. (PIC/S PI 041-1, Section 9.1.5.2 and glossary entry “Metadata”)

The following data quality attributes are particularly relevant when assessing a dataset:

Which quality attributes are most important, and which acceptance criteria are set for them, depend on the task and its associated risks.

Representativeness requires process understanding

An AI-based visual inspection system is intended to distinguish conforming packaging from defective packaging. If its dataset consists mainly of well-lit, standard packaging, challenging cases may be missing: reflections, small defects or acceptable variants that resemble defects. Understanding the inspection process is essential to determining which cases matter. Process and inspection subject matter experts should therefore be involved early in assembling and assessing the dataset.

More images of the same standard situation do not close these gaps. There must be a justified basis for concluding that the data cover the intended use, including, for example, different packaging formats and inspection conditions. Rare, critical product defect types and relevant subgroups need to be considered separately. (Draft Annex 22, Sections 3.2 and 5.1)

A simplified example shows how an unsuitable acceptance criterion can lead to an incorrect assessment: of 1,000 test cases, 990 are conforming and ten are defective. An AI system that classifies all packaging as conforming achieves an overall accuracy of 99 per cent. If the only acceptance criterion were an accuracy of at least 99 per cent, the model would meet that criterion without detecting a single defect. The criterion would therefore be unsuitable for the intended purpose. Performance must consequently be assessed separately for relevant product defect types and subgroups. The test dataset and its relevant subgroups must be large enough to allow statistically robust estimation of the performance metrics.

The reference assessment by subject matter experts is equally important. In supervised learning, individual images are assigned categories such as “conforming” or “crack”. These assignments are called labels. The underlying reference is often referred to as ground truth. A robust process is needed to establish and verify it, including how borderline cases and differences between expert assessments are handled. Section 5.3 of Draft Annex 22 also addresses this for test data. (Draft document)

Data volume, coverage and reference quality must therefore be assessed together. There is no universal minimum number of images or test cases that can be specified independently of the application. The required number depends on the task, relevant subgroups and the statistical robustness required.

Quality by Design: plan quality from the outset

QbD is an established approach in pharmaceutical development. ICH Q8(R2) describes a systematic approach to development that begins with predefined objectives and combines scientific product and process understanding with quality risk management. (ICH Q8(R2), Part II, Section 4)

This thinking can be applied methodologically to AI data: requirements for data generation, selection and processing are derived early from the task the AI will perform. Applying QbD principles to AI data is a methodological approach presented here, rather than a regulatory requirement in its own right.

For visual inspection, for example, relevant product defect classes and acceptable product variants are defined before the dataset is assembled. Image acquisition conditions, metadata and reference assessments by subject matter experts are considered from the outset.

Rules for data cleaning also belong in this early planning. An unusual value may be a transcription error, but it may equally reflect a genuine process deviation. Removing it indiscriminately may eliminate a case that is important for the AI application. Data cleaning and exclusions therefore need a sound justification based on domain expertise. For test data, Draft Annex 22 addresses predefined preprocessing and the documentation and justification of cleaning and exclusions. (Sections 5.4–5.5)

QbD thus focuses attention on how data quality is established. Recurring gaps should, where possible, be addressed at the point where data are captured and processed.

Data governance defines responsibilities and decision-making

Quality rules remain effective only if responsibilities and decision-making processes are clear. This is where data governance comes in. PIC/S defines it as the sum of the arrangements that provide assurance of data integrity throughout the lifecycle. This includes assigning responsibility for the data. (PIC/S PI 041-1, Sections 5.1–5.2)

For AI data, this also includes maintaining fitness for the intended use. A practical allocation of responsibilities might be:

• Data owner: responsible for the meaning of the data within the process, their intended use and quality rules.
• Data steward: maintains definitions, monitors data quality and coordinates the resolution of data issues.
• IT and system owners: responsible, within their defined remits, for data flows, access permissions, data security and system operation.
• QA and process owners: perform the review, assessment and approval tasks specified in the quality management system.

These functions may be organised differently. What matters is appropriate competence and clear decision-making authority.

Governance also includes data lineage, the traceable chain of data origin and processing. It links initial capture with selection, transformations and subsequent use. If a data error is discovered, this chain helps identify affected records and datasets and, where applicable, model versions.

Data qualification provides documented evidence of fitness for use

Whether a dataset is suitable for the intended AI use can be systematically assessed and documented. Data qualification is used here to describe a methodological approach comprising test planning, execution and a final assessment.

A useful documentation structure consists of a Data Qualification Plan, a Data Qualification Protocol and a Data Qualification Report. These specific document titles are not a universal regulatory requirement. Their content can be incorporated into existing validation and quality processes.

The Data Qualification Plan defines requirements and tests

The plan defines which dataset will be assessed and for what use. It describes relevant quality attributes, risks, test methods and acceptance criteria, as well as responsibilities and the handling of deviations.

For models developed in-house or adapted for the application, this also includes distinguishing between training data, tuning or ML validation data, and independent test data. ML validation data are used for model selection and optimisation. This use of the term “validation” must not be equated with GxP validation of the overall system. (EMA Reflection Paper, Section 2.5.2)

Independent test data are intended to demonstrate how the specified model handles cases not previously used in development. Separate folders alone do not ensure independence: images of the same physical package may be closely related. Repeatedly adapting a model based on the same test dataset can also compromise the validity of that dataset as independent evidence. This direct or indirect influence of test information on development is called data leakage. (Draft Annex 22, Sections 6.1 and 6.4)

One acceptance criterion for data quality might, for example, require each test image to be correctly and traceably linked to a reference test object, such as a package with a defined defect, and to its verified reference assessment. Detection performance for a defect is, by contrast, a model performance criterion. A suitable dataset enables performance assessment; it does not predetermine its outcome.

The Data Qualification Protocol records execution

The protocol records which tests were actually performed and their results. This includes the dataset versions assessed, the test tools used, references to test evidence and any deviations identified.

For visual inspection, the tests may examine, for example, the links between test images, reference test objects and their reference assessments; unacceptable overlap between test data and data used for training or tuning; or coverage of relevant product defect classes.

Traceability is essential. A statement such as “labels checked” is not sufficient on its own. The subject, method, scope and findings of the checks must be identifiable. Excluded data and their effects on the assessment must also be documented.

The Data Qualification Report assesses fitness for use

The report brings the results together and assesses whether the predefined criteria have been met. It identifies remaining gaps and records the final conclusion on whether the dataset is suitable for the intended AI use and what restrictions, if any, apply.

The conclusion should relate to a specific data version and use. A blanket statement such as “data quality is adequate” leaves unclear which products, conditions or tasks have actually been covered.

Data qualification contributes to the documented evidence of fitness for use of the overall system. Model performance, interfaces, error handling and fitness within the intended process require further, coordinated assessments.

Data for pretrained models and knowledge assistants

Even with a pretrained model, it remains necessary to establish which inputs and test data are suitable for the specific use. Available supplier evidence and how to address missing information also need to be assessed.

A knowledge assistant likewise needs suitable data. With retrieval-augmented generation (RAG), relevant content is retrieved from a knowledge base and supplied to the language model to help generate its response.

For an assistant working with work instructions, relevant considerations include source approval, effective versions, site applicability, and the complete and accurate transfer into the knowledge base of the content needed for the use case. A well-maintained knowledge base alone does not demonstrate that the application retrieves appropriate content and generates factually correct responses.

For generative AI and RAG-based knowledge assistants, regulatory classification also depends on the specific use. We explain which regulations apply and when validation is required in AI in GMP and MedTech Companies: Which Regulations Apply and When Is Validation Required?.

Maintain data quality throughout the lifecycle

Documented evidence of fitness for use relates to a defined baseline and its operating conditions. New products, changed measurement methods or additional sites may alter that basis. A change in the distribution of input data during operation, for example in image brightness or packaging variants, is referred to as data drift. This can change performance even when the model itself remains unchanged. (Draft Annex 22, Sections 10.3–10.4)

Operational arrangements must define which changes or data issues trigger reassessment and who is responsible. Data governance responsibilities should be linked to change management and periodic evaluation. Data quality metrics and model performance should be considered together.

Distinguish regulatory requirements from methodological approaches

EU GMP Annex 11 sets out risk-based requirements for validation, documentation and data processing. The consultation draft of Annex 22 addresses additional AI-specific topics, such as representative test data and their independence. Based on the official publication status checked on 15 October 2026, Annex 22 remains a draft. (EudraLex Volume 4, Annex 22 consultation)

Whether data are suitable for an AI application depends on their intended use. QbD supports early planning of data quality; data governance defines responsibilities and decisions throughout the lifecycle. The data qualification approach proposed here structures the assessment and documentation of fitness for use. Its scope depends on the intended use, risk and available evidence.

Would you like to assess the data underpinning a planned AI application? Contact QMVP to discuss your use case, the available data and the evidence required.

Sources and references

• ICH Q8(R2), Pharmaceutical Development, August 2009: Part II, Section 4, glossary entry “Quality by Design”, printed page 16 / PDF page 20. Applying the concept to AI data is a methodological proposal. Original document
• PIC/S PI 041-1, Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments, 1 July 2021: Sections 5.1–5.2 on data governance; 7.4–7.5 on ALCOA+. Inspection guidance. Original document
• EMA/INS/GCP/112288/2023, Guideline on computerised systems and electronic data in clinical trials, 9 March 2023, final version: Section 4.5, pages 12–13, on ALCOA++, including traceability. Scope: clinical trials (GCP). Original document
• EU GMP Chapter 4, Documentation, consultation draft 2025: Section 4.63, Table 1, and glossary entry “ALCOA++” on data integrity, including traceability. Draft; not an adopted final version. Draft document
• EU GMP Annex 11, Computerised Systems, revision January 2011, in operation since 30 June 2011: Sections 1–7 on risk management, responsibilities, suppliers, validation and data processing; Sections 10–12 on change management, periodic evaluation and access restrictions; glossary entry “System owner” on data security. Original document
• EMA Reflection Paper: Reflection paper on the use of Artificial Intelligence (AI) in the medicinal product lifecycle, EMA/CHMP/CVMP/83833/2023, final version September 2024: Sections 2.5.1–2.5.2 and 2.6 on data preparation, data separation and governance. Adopted reflection paper. Original document
• EU GMP Annex 22, consultation draft 2025: Sections 3.2 and 5.1–5.5 on subgroups and test data, 6.1 and 6.4 on test data independence, and 10.3–10.4 on monitoring during operation. Not an adopted final version. Draft document