ADAAS
ADAASADAAS
No. 024 · Healthcare & Regulation

AI-Generated Code in Healthcare and Regulated Systems: The Traceability Problem No One Is Talking About

Healthcare regulation requires traceability from clinical requirement to validated implementation. AI code generation breaks this chain by default.

AI-Generated Code in Healthcare and Regulated Systems: The Traceability Problem No One Is Talking About

Healthcare software is governed by regulatory frameworks — HIPAA, GDPR Article 9, FDA SaMD guidance, MDR, IEC 62304 — that pre-date AI code generation by decades. None of them anticipated a world where the implementation could be regenerated faster than the validation evidence could be produced.

The Regulatory Frameworks Healthcare Engineers Already Live With

Engineering leaders in healthcare already operate under a complex regulatory environment. Before discussing the AI-specific problem, it is worth being precise about the frameworks that shape every software decision in the sector.

IEC 62304: Medical Device Software Lifecycle

The dominant international standard for software in medical devices. It requires documented traceability between user requirements, software requirements, software architecture, and verified implementation. Each layer must link to the layers above and below it. The framework predates Agile, predates DevOps, and absolutely predates AI code generation; yet it remains binding.

FDA Pre-market Guidance (US)

For software-as-medical-device (SaMD) products, the FDA requires documented evidence that the software performs as intended for its declared clinical use. The recent guidance on AI/ML-based SaMD adds requirements for documenting model behaviour, but does not relax the underlying traceability requirements for the surrounding code.

How AI Code Generation Breaks the Chain

The traceability requirement assumes a stable relationship between a requirement and the code that satisfies it. AI generation severs that relationship by default: the implementation can be regenerated in seconds, but the evidence that it still satisfies the requirement cannot.

IEC 62304 requires an unbroken chain from clinical requirement to verified implementation. AI regeneration invalidates the last link without touching the ones above it.
The implementation can be regenerated faster than the validation evidence can be produced. That asymmetry is the whole problem.

The gap is not that AI writes bad code. It is that AI writes code faster than the traceability evidence around it can be regenerated — so the evidence silently goes stale.

How Executable Architecture Maps to Healthcare Requirements

Executable architecture languages like AIS were not designed for healthcare regulation, yet their structural properties map unexpectedly well onto what IEC 62304, FDA, and MDR require. The same machine-readable definition that solves the AI traceability problem also produces the structured evidence regulation expects.

entity('PatientRecord') {
  feature('Calculate Dosage') {
    ref:calculateDosage
    classification(clinical_impact:HIGH)
    satisfies(requirement:SRS_3_4_2)
    satisfies(requirement:URS_2_1_clinical_decision)
  }
}

container('clinical-engine') {
  realize(ref:calculateDosage) {
    constraint(ref:doseRangeValidation) { satisfies(requirement:SRS_3_4_3) }
    validation(ref:doseCalcUnitTests)   { verifies(requirement:SRS_3_4_2) }
    audit(ref:clinicalAuditTrail)
  }
}

AI generation against this definition cannot produce an implementation that skips the constraints, skips the validation hooks, or breaks the requirement linkage. The structural properties healthcare regulation requires become enforced by construction.

Every AI-generated implementation is bound to the same executable definition, so constraints, validation and audit hooks are derived rather than remembered.

Where to Start

  • Formalize the highest clinical-impact features in an executable architecture model first.
  • Restrict AI generation on clinical-impact features to the architectural framework.
  • Integrate the architecture model into technical-file generation so documentation stays current because it is derived, not maintained.

Healthcare organizations that solve the traceability problem structurally will navigate the next decade with the speed advantages of AI and the regulatory posture of careful engineering. The ones that adopt AI tools without solving the gap will discover the problem the expensive way.