The pattern: on-premise clinical RAG
The architectures emerging in healthcare IT departments share one underlying choice: keep clinical data inside the perimeter. The recurring pattern is an on-premise clinical RAG, where three layers stay distinct and are governed separately — interoperability, the language model and data governance.
The interoperability layer rests on FHIR. As of today, the version in production in real implementations is R4: it is the standard everything is built on. R6 is still in ballot and should not be treated as available for clinical environments. On R4 you model the resources — Patient, Observation, DocumentReference, DiagnosticReport — that feed the context retrieved by the retrieval system. For an overview of the mechanism, see our piece on clinical RAG with Open Source LLMs; for a comparison of standards, openEHR, FHIR and CDA.
Open Source biomedical LLMs
On the model side, the Open Source ecosystem now offers mature options for the biomedical domain. BioMistral is built on a Mistral base with pretraining on PubMed Central. Meditron-7B and Meditron-70B, developed by EPFL and Yale, are adapted from Llama-2. Alongside them sit numerous Llama and Mistral variants fine-tuned on clinical corpora.
A necessary clarification: Google’s MedGemma is an interesting option, but it is not Open Source in the full sense. It is an open-weights model distributed under the HAI-DEF licence (Health AI Developer Foundations): the weights are accessible, but the licence terms do not match an approved Open Source licence. The distinction matters when assessing redistribution, modification and use inside a regulated device.
Data governance
The third layer — the one often deferred — is governance. In a clinical context this means control over which data enters the model’s context, redaction of identifying data where it is not needed, traceability of queries and data residency. Keeping the model local solves only the transfer problem: the rest — who accesses it, what gets retrieved, how you demonstrate it after the fact — requires explicit policies applied bidirectionally, on request and on response.
The EHDS picture: preparing towards 2027
On the regulatory side, the reference is the European Health Data Space (EU Regulation 2025/327), in force since early 2025. The first deadlines land two years later, in 2027: they mostly concern primary use, the obligation to join MyHealth@EU, and secure processing environments with their data permits. It is best to read 2027 as a preparation phase and first deadlines, not as the operational start of everything.
In particular, secondary use on electronic health record data does not start in 2027: it is expected around 2029, with remaining categories later still. Anyone planning a clinical infrastructure today is therefore working against a phased horizon, not a single date. We covered the secondary-use dimension in our article on EHDS and secondary use.
AI Act and medical devices
There is a second regulatory track. An AI system that is a medical device, or a safety component of a device regulated under MDR/IVDR with a Notified Body assessment, falls into the high-risk category of the EU AI Act. The compliance deadlines for these systems are evolving: the provisional agreement on the Digital Omnibus (reached in early May, pending formal adoption) shifts high-risk systems embedded in regulated products towards August 2028, and stand-alone ones towards December 2027. The realistic horizon is therefore 2027-2028, not a single hard date to assume today. For the detail of the measure, see our analysis of the Digital Omnibus.
What this means in practice
For a healthcare organisation, the operational message is that the three layers must be designed together from the start. A local LLM without FHIR upstream produces answers that cannot be traced back to structured data; FHIR without governance exposes the data to untracked uses; and neither, on its own, covers the MDR path if the system becomes a device.
This is the approach behind AIHealth, noze’s on-premise clinical platform: local LLMs, RAG over FHIR and DICOM, data governance and the MDR path treated as parts of the same build.
Links: European Health Data Space (EHDS) — EU Commission · FHIR v6.0.0-ballot