Article
The pattern: on-premise clinical RAG
The architectures emerging in healthcare IT departments share one underlying choice: keep clinical data inside the perimeter. The recurring pattern is an on-premise clinical RAG, where three layers stay distinct and are governed separately — interoperability, the language model and data governance.
The interoperability layer rests on FHIR. As of today, the version in production in real implementations is R4: it is the standard everything is built on. R6 is still in ballot and should not be treated as available for clinical environments. On R4 you model the resources (Patient, Observation, DocumentReference, DiagnosticReport) that feed the context retrieved by the retrieval system. For an overview of the mechanism, see our piece on clinical RAG with open-weight LLMs; for a comparison of standards, openEHR, FHIR and CDA.
Open-weight biomedical LLMs
On the model side, the open-weight model ecosystem now offers mature options for the biomedical domain. BioMistral is built on a Mistral base with pretraining on PubMed Central. Meditron-7B and Meditron-70B, developed by EPFL and Yale, are adapted from Llama-2. Alongside them sit numerous Llama and Mistral variants fine-tuned on clinical corpora.
A necessary clarification: open weight means the weights can be downloaded, not that all licences are the same. Some models are released under permissive licences such as Apache 2.0, others under licences with use restrictions, and fine-tuned variants inherit the terms of their base model. Google’s MedGemma is an interesting option, but its weights are distributed under the HAI-DEF terms (Health AI Developer Foundations), which do not match an approved open source licence. The licence therefore has to be read model by model when assessing redistribution, modification and use inside a regulated device.
Data governance
The third layer — the one often deferred — is governance. In a clinical context this means control over which data enters the model’s context, redaction of identifying data where it is not needed, traceability of queries and data residency. Keeping the model local solves only the transfer problem: the rest — who accesses it, what gets retrieved, how you demonstrate it after the fact — requires explicit policies applied bidirectionally, on request and on response.
The EHDS picture: preparing towards 2027
On the regulatory side, the reference is the European Health Data Space (EU Regulation 2025/327), in force since early 2025. The first deadlines land two years later, in 2027: they mostly concern primary use, the obligation to join MyHealth@EU, and secure processing environments with their data permits. It is best to read 2027 as a preparation phase and first deadlines, not as the operational start of everything.
In particular, secondary use on electronic health record data does not start in 2027: it is expected around 2029, with remaining categories later still. Anyone planning a clinical infrastructure today is therefore working against a phased horizon, not a single date. We covered the secondary-use dimension in our article on EHDS and secondary use.
AI Act and medical devices
There is a second regulatory track. An AI system that is a medical device, or a safety component of a device regulated under MDR/IVDR with a Notified Body assessment, falls into the high-risk category of the EU AI Act. The compliance deadlines for these systems are evolving: the provisional agreement on the Digital Omnibus (reached in early May, pending formal adoption) shifts high-risk systems embedded in regulated products towards August 2028, and stand-alone ones towards December 2027. The realistic horizon is therefore 2027-2028, not a single hard date to assume today. For the detail of the measure, see our analysis of the Digital Omnibus.
What this means in practice
For a healthcare organisation, the operational message is that the three layers must be designed together from the start. A local LLM without FHIR upstream produces answers that cannot be traced back to structured data; FHIR without governance exposes the data to untracked uses; and neither, on its own, covers the MDR path if the system becomes a device.
This is the direction of AgenticHealth, noze’s on-premise AI for clinical research: today source-cited research on protocols, literature and regulations, with local LLMs; read-only FHIR connectors, on pseudonymised data from a research cohort, are on the roadmap. It is not medical software: it does not support diagnosis or patient-care decisions.
Links: European Health Data Space (EHDS) — EU Commission · FHIR v6.0.0-ballot