AI in GMP and MedTech Companies: Which Regulations Apply and When Is Validation Required?

Authors: Eugen Betche and Dr Franz Müller, QMVP GmbH. Last updated: 16 October 2026

In QA, a chatbot helps draft a deviation report. In production, AI summarises information or supports a quality-related assessment. Some applications have been approved. Others are used as “shadow AI”, without an agreed approval process.

The regulatory assessment depends on what the AI is actually used for. The influence of its outputs on records, assessments and decisions is equally important. Calling a tool an “assistant” or approving it internally does not answer these questions.

This article brings together insights and lessons learned from the “2nd Annex 22 / AI Conference”, held in Copenhagen on 7 and 8 October 2026, with the applicable regulatory requirements and relevant guidelines. It focuses on the presentations by Ib Alstrup and Victor Bechmann, who discussed how to assess AI applications from a GMP inspection perspective.

The focus is on GMP activities in the pharmaceutical industry and AI applications in the medical device industry within the EU and German legal frameworks. GCP, GLP, GDP and pharmacovigilance are not covered in depth. These areas, and the US market, require a separate regulatory assessment. The statements on Annex 11 relate to its GMP scope.

Sources reviewed as of 14 October 2026.

What do we mean by AI applications?

AI applications can perform very different tasks. A model may, for example, analyse images and assign defect categories. Generative AI produces content such as text or images. It includes large language models, or LLMs, which are often accessed through a chat interface.

Machine learning is a field within AI. Today's LLMs also rely on machine learning, so these terms do not describe mutually exclusive categories. In this article, “AI application” means the software in use, including its AI functionality. The applicable requirements depend on that functionality and its use in the regulated process.

When an AI application becomes GxP-relevant

The first question in assessing GxP relevance is whether the AI application supports or performs a regulated activity. Its potential impact on product quality, patient safety and data integrity must then be assessed. Use within a pharmaceutical or medical device company alone is not enough to establish GxP relevance.

For GMP activities, Annex 11 is particularly relevant, as it covers computerised systems within this scope. In the medical device industry, product requirements must be considered separately from the requirements for software used in production, inspection and testing, and QMS processes. The relevant requirements are explained in the following sections. Annex 11, Principle and Section 1, MDR, Article 10(9) and Annex I, ISO 13485:2016

The importance of intended use was also discussed at the conference. Ib Alstrup and Victor Bechmann, both inspectors at the Danish Medicines Agency, focused on the influence of AI on technical decisions. Alstrup is Chair and Rapporteur of the Annex 11 and Annex 22 drafting group; Bechmann is Co-Rapporteur.

The intended use and the actual use must be considered together. AI software may be approved for general office tasks yet still be used for a GMP-relevant assessment. It is therefore also essential to consider how its outputs are used in the work process.

Four guiding questions for assessing an AI application can be derived from the speakers' presentations:

• What activity does the AI support or perform?
• How much does the technical decision depend on the AI output?
• Are the outputs used for GxP-relevant records, assessments or process decisions?
• What consequences could an incorrect output have for product quality, patient safety or data integrity?

When considering criticality, Alstrup and Bechmann focus particularly on how strongly the model influences a decision and on the potential consequences of an incorrect decision. This helps assess the significance of the AI functionality within a GMP process. It does not, however, replace the assessment of which regulatory requirements apply to the specific application.

Human approval does not automatically remove GxP relevance. Even a signed report may rely substantially on an AI output for its technical conclusions. What matters is the assessment users perform themselves and the extent to which they rely on the AI.

Here, risk refers to potential adverse effects on product quality, patient safety and data integrity. Both the possible consequences of an error and the probability of resulting harm must be considered. Where validation is required, this risk influences the necessary scope of validation and the supporting evidence. A low risk does not automatically remove the validation requirement. ICH Q9(R1), Sections 3 and 4

AI in pharmaceutical GMP processes

When AI is used for a GMP-regulated activity, the existing GMP requirements apply. In Germany, Good Manufacturing Practice is governed in particular by the German Ordinance on the Manufacture of Medicinal Products and Active Substances (AMWHV). The EU GMP Guidelines provide further detail on these requirements. German Federal Ministry of Health on GMP

Annex 11 already applies

Annex 11 applies to computerised systems used as part of GMP-regulated activities. It requires the application to be validated. The current version does not explicitly mention AI. However, AI functionality within such a system forms part of the assessment.

The Commission continues to list the January 2011 version as the applicable Annex 11. The validation requirement therefore does not depend on Annex 22 being finalised or on classification as high-risk AI under the AI Act. The AI Act is the EU regulation on artificial intelligence. It governs the placing on the market, putting into service and use of AI systems, following a risk-based approach. European Commission on the AI Act Annex 11, Principle and Section 1, EudraLex Volume 4

The subject of validation is the AI-enabled application within the overall computerised system. Its fitness for the intended GMP process is what matters. The AI model is one component of that system.

Verification of an AI model does not replace validation of the AI-enabled application for its intended GMP use. This also applies to models from external providers and to cloud-based applications. A model verified by its provider does not, by itself, demonstrate that the specific application is fit for use in the company's own GMP process. The existing Annex 11 requirements for suppliers and service providers must also be considered.

Annex 11 is currently being revised. However, the draft revision published in 2025 does not yet change the applicable requirements. The fundamental validation requirement is already included in the 2011 version. Consultation on Annex 11 and Annex 22

Example 1: Text assistance in a deviation report

A text assistant in the form of a chatbot summarises publicly available technical information solely to help an individual understand a topic. If the outputs are not used for a GMP activity, this use does not give rise to a GMP validation requirement.

In a deviation report, however, the same label, “text assistance”, may cover different functions:

• Language editing: The facts and technical assessment have already been established. The AI improves the wording. Even this may unintentionally change the meaning of statements.
• Summarisation: The AI condenses quality-relevant records. Its selection, omissions and emphasis may influence the subsequent assessment.
• Technical support: The AI proposes causes, assessments or corrective actions. Any proposals adopted feed directly into the technical assessment.

The label “text assistance” is therefore insufficient for the regulatory assessment. What matters is whether the software edits wording, summarises information or supports technical assessments, and how its outputs are then used.

Human review must also be considered specifically. A plausibility check of the text differs from an independent assessment of its statements against the underlying records and sources. A final signature alone does not show what review has taken place.

There is therefore no blanket exemption for text assistance. Its regulatory status depends on its function in the GMP process and the consequences of potential errors.

Additional GMP requirements depend on the task

Annex 15 is also relevant to equipment and manufacturing processes. If AI affects a process subject to qualification or validation, this relationship must also be assessed. System validation and process validation address different subjects. Annex 15 explicitly refers to Annex 11 for computerised systems. Annex 15, Principle

Annex 11 addresses the computerised application. Depending on the task, requirements for the manufacturing process, inspection process or analytical procedure may apply in addition.

For active substance manufacture, EU GMP Part II, Section 5.40, explicitly requires GMP-related computerised systems to be validated. The use of AI does not change this starting point. Additional requirements may also apply when AI is used. EU GMP Part II, Section 5.4

Annex 1 contains specific requirements for automated visual inspection processes for sterile medicinal products. Section 8.32 requires validation of the inspection process. AI functionality must be assessed according to its contribution to that process. Annex 1, Section 8.32

Example 2: AI-enabled visual inspection

A model analyses images solely in a development experiment. Its outputs are used neither for GMP decisions nor as GMP evidence. No validation requirement under Annex 11 can be derived for this use.

If the model later controls the rejection of vials during automated visual inspection, the regulatory assessment changes. Both the requirements for the computerised system and those for the inspection process then apply.

If an AI or ML model forms part of an analytical procedure, the requirements for that procedure must also be considered. EU GMP Chapter 6, Section 6.15, requires analytical methods to be validated. ICH Q2(R2) provides further detail on this requirement. ICH Q14 addresses development and the lifecycle, including multivariate procedures. Their scope includes, in particular, procedures for release and stability testing. This does not create a general additional validation requirement for every AI-enabled text assistant, such as a chatbot, used in QC. EU GMP Chapter 6, Section 6.15, ICH Q2(R2), ICH Q14, Sections 1.1 and 1.2

Data integrity remains part of the assessment

If an AI application creates, changes or summarises GMP-relevant data, its impact on data integrity must be considered. Summaries may omit information or alter the relationships between facts. Generated statements may also appear to be substantiated findings even when the underlying records do not support them.

Data provenance, completeness and the traceability of processing steps therefore remain essential. The ALCOA++ principles also apply to GMP data processed by AI: records must be attributable, legible, contemporaneous, original and accurate, as well as complete, consistent, enduring, available and traceable. Which inputs, outputs and processing steps constitute GMP records depends on their function in the process.

PIC/S PI 041-1 explains the existing GMP/GDP expectations in this context, including those relating to data provenance, metadata and audit trails. The guidance does not create a separate new legal obligation. The AI-specific additions in the draft Annex 22 build on these existing requirements. PIC/S PI 041-1, Sections 5, 7 and 9

Annex 22 remains a draft

Annex 22 is intended to add a dedicated annex on artificial intelligence to the EU GMP Guidelines. The published draft addresses data-trained ML models in critical GMP applications in medicinal product and active substance manufacture. Its scope is limited to static models with deterministic outputs.

The draft covers, among other things, intended use, acceptance criteria, the suitability and independence of test data, the assessment of model outputs and subsequent use. It is intended to supplement Annex 11 with AI-specific considerations. The existing requirement to validate the computerised application does not, however, depend on its finalisation. Draft Annex 22, Sections 1 to 10

Static models do not continue to learn independently during use. Deterministic models produce identical outputs for identical inputs.

Dynamic and probabilistic models, generative AI and LLMs are outside the stated scope of the current draft. The draft advises against their use in critical GMP applications. For non-critical GMP applications of generative AI, it emphasises qualified users' responsibility for the suitability of the outputs. These statements have draft status. Draft Annex 22, Section 1

The conference presentations, however, show that adaptive and probabilistic models are also part of the regulatory discussion. Alstrup and Bechmann refer to the EMA workshop held in June 2026. The final scope of Annex 22 has therefore not yet been settled.

The speakers explicitly identify their statements as personal professional views. These statements are neither final requirements nor a general authorisation to use particular AI applications.

Bechmann also emphasises that the model label alone is insufficient. The application's intended use, capabilities and risks remain decisive.

AI in the medical device industry

In the medical device industry, the first distinction is between product functionality and use in internal processes.

AI-enabled software may itself be a medical device because of its medical intended purpose, for example when it analyses medical images for diagnostic assessment. AI functionality may also form part of a medical device or IVD, where it may, for example, analyse measurement signals.

This must be distinguished from AI applications used in the manufacturer's production, inspection and testing, or quality management processes. Such applications may assess test results, summarise quality records or support audit planning. The decisive questions are what task the software performs and how its outputs are used.

AI as a medical device or product component

Where AI has a medical or IVD intended purpose, the MDR or IVDR applies, as appropriate. The specific product functionality determines the regulatory assessment. The regulations contain explicit requirements for software validation in MDR Annex I, Section 17.2, and IVDR Annex I, Section 16.2. MDR, IVDR, MDCG 2019-16 rev.1, Section 3.7

MDCG 2019-11 rev.1 supports the qualification and classification of medical device software. MDCG 2025-6 explains the interplay between the MDR, IVDR and AI Act. Both documents are guidance, rather than separate binding legislation. MDCG 2019-11 rev.1, MDCG 2025-6

For these AI systems, high-risk classification under the AI Act must also be assessed. Article 6(1) covers AI that is itself a product or a safety component of a product covered by the legislation listed in Annex I. In addition, that product must be required to undergo conformity assessment by a notified body. A safety function alone is insufficient for this classification. AI Act, Article 6(1) and further provisions on classification, Annex I

Additional technical references can support this assessment. IEC 62304 describes lifecycle requirements for medical device software. It does not, however, cover the final validation and release of the medical device. IEC 82304-1 addresses health software products on general computing platforms without dedicated hardware. These standards have different scopes. Whether they are binding must be determined for the specific case. IEC 62304, IEC 82304-1

The IMDRF guiding principles for Good Machine Learning Practice concern AI-enabled medical devices. They are a technical reference and do not create a separate general GxP validation requirement. They are not automatically the governing framework for internal QA or QMS assistants. IMDRF GMLP, N88 FINAL:2025

AI in production, inspection and testing, and QMS processes

An internal AI application does not require validation solely because it is used by a medical device manufacturer. However, the software validation requirements of ISO 13485 may apply to these applications.

The standard distinguishes three areas of application:

• Clause 4.1.6 addresses software applications used in the QMS.
• Clause 7.5.6 addresses software applications used in production and service provision.
• Clause 7.6 addresses software applications used in monitoring and measurement.

Where ISO 13485 applies and the software application falls within the relevant requirement, the application must be validated. The scope of validation depends on the risk associated with its use. ISO 13485:2016, European Commission on harmonised standards

Example 3: AI-enabled audit planning

A company with a QMS based on ISO 13485 uses AI-enabled software to analyse previous audit reports and the status of corrective actions. The software proposes topics and processes for more detailed examination in the next internal audit. Those responsible for quality management review the proposals and incorporate some of them into the audit plan. The selection of audit focus areas therefore relies in part on the AI analysis.

The AI application thus influences which processes and topics receive more detailed examination during an audit. Errors in selecting or prioritising the areas or processes to be audited may result in quality-critical topics being overlooked. The AI application must therefore be considered a software application used in the QMS. Where ISO 13485 applies, Clause 4.1.6 governs its validation.

Which additional requirements apply?

The assessment under GMP or medical device legislation does not resolve every regulatory question concerning AI use in a company. The AI Act and, where applicable, data protection must also be considered.

AI Act: Role and application are decisive

For the AI Act, the first step is to determine its applicability and the company's role. The distinction between provider and deployer is particularly relevant. A company using AI is not automatically subject to the same obligations as the system's provider.

The AI Act distinguishes prohibited AI practices, high-risk AI systems and other applications to which specific obligations may apply depending on their function. These include, in particular, transparency obligations. Low-risk applications are not subject to the specific high-risk requirements solely because they use AI. This does not, however, mean they are exempt from all provisions of the AI Act. European Commission on the risk-based approach

Article 4 requires providers and deployers to take measures to promote AI literacy among the people working with AI. This means the knowledge and skills needed to use the AI competently and to assess its capabilities, limitations and risks in the relevant work context. Article 4, Article 3(56)

Article 5 prohibits certain AI practices. Article 50 contains transparency obligations for certain applications and content. These depend on functionality, the company's role and statutory exceptions. They do not create a blanket labelling requirement for every internal document produced with AI assistance. Article 5, Article 50

Whether an AI system is classified as high-risk AI is determined by Article 6. In addition to the product-related cases described above, the intended purposes listed in Annex III may be relevant. GMP criticality must be assessed separately: an AI application does not become high-risk AI solely because it performs a critical GMP task. AI Act, Article 6

Chapter V contains additional obligations for providers of general-purpose AI models, known as GPAI models. Merely using another provider's model API does not automatically make a company a GPAI model provider. AI Act, Article 53

The application dates also differ. AI literacy requirements and the original prohibitions have applied since 2 February 2025. Article 50 has generally applied since 2 August 2026. For the high-risk requirements, the Commission's current timeline gives 2 December 2027 for Annex III and 2 August 2028 for Annex I. Article-specific transitional provisions must also be considered. European Commission on the application dates

These dates do not postpone existing GMP obligations. An AI application may already require validation under GMP without being classified as high-risk AI. Conversely, the AI Act does not create a GMP validation requirement for an application outside GMP.

Data protection when personal data are processed

Where an AI application processes personal data, the GDPR also applies. Data protection obligations apply independently of GxP relevance and high-risk status. The GDPR does not create a general GxP validation requirement. GDPR

Understanding the status of guidance and voluntary guides

The EMA Reflection Paper explains regulatory considerations for the use of AI and machine learning throughout the medicinal product lifecycle, from drug discovery to post-authorisation use. It provides regulatory guidance and does not create a separate general validation requirement. EMA Reflection Paper

GAMP 5 and the GAMP AI Guide are ISPE industry guides. Their status must be distinguished from legal requirements and applicable GMP requirements. GAMP 5, GAMP AI Guide

Contracts, quality agreements or internal quality management requirements may nevertheless make the use of guidance mandatory. Approved product and process documentation may also establish additional obligations. It is therefore necessary to distinguish a useful reference from a specific requirement that must be met. EMA Reflection Paper, GAMP 5, GAMP AI Guide

Key regulatory requirements and references at a glance

The table summarises the key assessments relevant to the focus of this article. It is not a list of requirements that every company must meet for every AI application.

When no GxP validation requirement can be derived for an AI application

Each validation requirement described has a specific scope. Annex 11 covers computerised systems used as part of GMP-regulated activities. The ISO 13485 requirements cited relate to specific software applications used in the QMS, in production, or in monitoring and measurement. Annex 11, Principle, ISO 13485:2016, Clauses 4.1.6, 7.5.6 and 7.6

The decisive questions are therefore what task the AI application actually performs, which records or decisions it influences, and whether that use falls within one of the relevant requirements. If its use remains outside these scopes, no corresponding validation requirement can be derived from the provisions discussed. Requirements under product legislation, other standards or contracts must be assessed separately.

This assessment applies to the specific use considered. If a general-purpose assistant is later used for GMP assessments or quality decisions, its regulatory status must be reassessed.

Whether an AI application requires validation follows from its specific use and the requirements applicable to that use. Its intended use, actual use and the influence of its outputs on regulated processes, records and quality decisions are decisive. Human approval or the label “assistant” alone does not answer this question.

Where validation is required, it addresses whether the AI-enabled application is fit for its intended use in the process. Verification of the underlying model alone is insufficient. Classification under the AI Act must be assessed additionally and separately.

QMVP supports companies in assessing the regulatory requirements applicable to AI applications in GxP processes.

Sources

GMP, quality risk management and data integrity

• German Federal Ministry of Health: Good Manufacturing Practice (GMP); overview of the German and European GMP framework.
• European Commission: EudraLex Volume 4; overview of GMP requirements and the applicable chapters and annexes.
• EU GMP Annex 11: Computerised Systems, January 2011 revision; in particular, the Principle and Sections 1 and 3.
• EU GMP Part II: Basic Requirements for Active Substances used as Starting Materials, August 2014 version; in particular, Section 5.4 and the validation requirement in Section 5.40.
• EU GMP Annex 15: Qualification and Validation, 2015 version; in particular, the Principle and the distinction from Annex 11.
• EU GMP Annex 1: Manufacture of Sterile Medicinal Products, 2022 version; in particular, Section 8.32 on automated visual inspection.
• EU GMP Chapter 6: Quality Control, 2014 version; in particular, Section 6.15 on the validation of analytical methods.
• ICH Q9(R1): Quality Risk Management; in particular, Sections 3 and 4. Regulatory guideline on quality risk management.
• ICH Q2(R2): Validation of Analytical Procedures; regulatory guideline on the validation of analytical procedures.
• ICH Q14: Analytical Procedure Development; in particular, Sections 1.1 and 1.2 on scope. Regulatory guideline.
• PIC/S PI 041-1: Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments, 1 July 2021; in particular, Sections 5, 7 and 9. Regulatory guidance on the interpretation of existing GMP/GDP requirements.

Medical devices, IVDs and relevant standards

• Regulation (EU) 2017/745 on medical devices (MDR); in particular, Article 10(9) and Annex I, Section 17.2.
• Regulation (EU) 2017/746 on in vitro diagnostic medical devices (IVDR); in particular, Annex I, Section 16.2.
• ISO 13485:2016: Medical devices — Quality management systems — Requirements for regulatory purposes; in particular, Clauses 4.1.6, 7.5.6 and 7.6. A standard whose binding status must be determined for the specific case.
• IEC 62304:2006+AMD1:2015: Medical device software — Software life cycle processes; standard on medical device software lifecycle processes. It does not cover the final validation and release of the medical device.
• IEC 82304-1:2016: Health software — Part 1: General requirements for product safety; standard for health software products within its stated scope.
• MDCG 2019-16 rev.1: Guidance on Cybersecurity for medical devices; in particular, Section 3.7 on verification and validation. Non-binding guidance.

AI Act and data protection

• AI Act: EU regulation on artificial intelligence. References used in the European Commission's official AI Act Service Desk: Article 3(56), Article 4, Article 5, Article 6, Article 50, Article 53 and Annex I.
• European Commission: AI Act – Regulatory Framework and AI Act Implementation Timeline; additional explanations of the risk-based approach and application dates.
• Regulation (EU) 2016/679: General Data Protection Regulation (GDPR); requirements for the processing of personal data within the regulation's scope.

Drafts and conference materials

• European Commission: Consultation on the revision of Chapter 4 and Annex 11 and the new Annex 22, 7 July to 7 October 2025; includes the draft revision of Annex 11. Drafts must be distinguished from applicable requirements.
• Annex 22: Artificial Intelligence, 2025 consultation draft; in particular, Section 1 on scope and Sections 2 to 10. A draft, not an applicable final requirement.