---
title: "FHIR and open-weight LLMs on-premise: clinical RAG and health-data governance"
description: "On-premise clinical RAG built on FHIR as the interoperability layer, open-weight biomedical LLMs and data governance. EHDS roadmap towards 2027 and the AI Act for medical devices, with a link to AgenticHealth."
url: "https://www.noze.it/en/insights/fhir-llm-digital-health-on-premise/"
lang: "en"
alternate: "https://www.noze.it/insights/fhir-llm-digital-health-on-premise/"
published: "2026-05-25"
author: "noze S.r.l."
keywords: "Digital Health, AI, Open Source, Compliance, Governance, FHIR, LLM, RAG, Open weight, BioMistral, Meditron, MedGemma, DICOM, EHDS, EU AI Act, MDR, DigitalHealth"
---

# FHIR and open-weight LLMs on-premise: clinical RAG and health-data governance

On-premise clinical RAG built on FHIR as the interoperability layer, open-weight biomedical LLMs and data governance. EHDS roadmap towards 2027 and the AI Act for medical devices, with a link to AgenticHealth.

25 May 2026

![Three-panel diagram inside a dashed green perimeter: FHIR R4 interoperability, a local open-weight LLM and data governance, with EHDS and AI Act deadlines along the bottom.](https://www.noze.it/images/insights/clinical-rag-onprem-layers-en.svg)

Keeping the model local solves data transfer, not the traceability of who queried what: the three layers have to be designed together. The EHDS (EU Regulation 2025/327) has been in force since early 2025, but the first deadlines land two years later.

## The pattern: on-premise clinical RAG

The architectures emerging in healthcare IT departments share one underlying choice: keep clinical data **inside the perimeter**. The recurring pattern is an **on-premise clinical RAG**, where three layers stay distinct and are governed separately — interoperability, the language model and data governance.

The interoperability layer rests on **FHIR**. As of today, the version in production in real implementations is **R4**: it is the standard everything is built on. `R6` is still in ballot and should not be treated as available for clinical environments. On `R4` you model the resources (`Patient`, `Observation`, `DocumentReference`, `DiagnosticReport`) that feed the context retrieved by the retrieval system. For an overview of the mechanism, see our piece on [clinical RAG with open-weight LLMs](https://www.noze.it/en/insights/clinical-rag-open-source-llm/); for a comparison of standards, [openEHR, FHIR and CDA](https://www.noze.it/en/insights/openehr-fhir-cda-comparison/).

## Open-weight biomedical LLMs

On the model side, the **open-weight** model ecosystem now offers mature options for the biomedical domain. **BioMistral** is built on a **Mistral** base with pretraining on **PubMed Central**. **Meditron-7B** and **Meditron-70B**, developed by **EPFL** and **Yale**, are adapted from **Llama-2**. Alongside them sit numerous **Llama** and **Mistral** variants fine-tuned on clinical corpora.

A necessary clarification: open weight means the weights can be downloaded, not that all licences are the same. Some models are released under permissive licences such as Apache 2.0, others under licences with use restrictions, and fine-tuned variants inherit the terms of their base model. Google’s **MedGemma** is an interesting option, but its weights are distributed under the **HAI-DEF** terms (Health AI Developer Foundations), which do not match an approved open source licence. The licence therefore has to be read model by model when assessing redistribution, modification and use inside a regulated device.

## Data governance

The third layer — the one often deferred — is **governance**. In a clinical context this means control over which data enters the model’s context, **redaction** of identifying data where it is not needed, traceability of queries and data residency. Keeping the model local solves only the transfer problem: the rest — who accesses it, what gets retrieved, how you demonstrate it after the fact — requires explicit policies applied bidirectionally, on request and on response.

## The EHDS picture: preparing towards 2027

On the regulatory side, the reference is the **European Health Data Space** (EU Regulation **2025/327**), in force since early 2025. The **first deadlines** land two years later, **in 2027**: they mostly concern _primary use_, the obligation to join **MyHealth\@EU**, and secure processing environments with their _data permits_. It is best to read 2027 as a **preparation phase and first deadlines**, not as the operational start of everything.

In particular, _secondary use_ on electronic health record data **does not start in 2027**: it is expected around **2029**, with remaining categories later still. Anyone planning a clinical infrastructure today is therefore working against a phased horizon, not a single date. We covered the secondary-use dimension in our article on [EHDS and secondary use](https://www.noze.it/en/insights/ehds-secondary-use-2027/).

## AI Act and medical devices

There is a second regulatory track. An AI system that **is** a medical device, or a safety component of a device regulated under **MDR/IVDR** with a **Notified Body** assessment, falls into the **high-risk** category of the **EU AI Act**. The compliance deadlines for these systems are **evolving**: the provisional agreement on the **Digital Omnibus** (reached in early May, pending formal adoption) shifts high-risk systems _embedded_ in regulated products towards **August 2028**, and _stand-alone_ ones towards **December 2027**. The realistic horizon is therefore **2027-2028**, not a single hard date to assume today. For the detail of the measure, see our analysis of the [Digital Omnibus](https://www.noze.it/en/insights/ai-act-digital-omnibus-2026/).

## What this means in practice

For a healthcare organisation, the operational message is that the three layers must be designed together from the start. A local LLM without FHIR upstream produces answers that cannot be traced back to structured data; FHIR without governance exposes the data to untracked uses; and neither, on its own, covers the **MDR** path if the system becomes a device.

This is the direction of [**AgenticHealth**](https://www.noze.it/en/products/agentichealth/), noze’s on-premise AI for clinical research: today source-cited research on protocols, literature and regulations, with local LLMs; read-only **FHIR** connectors, on pseudonymised data from a research cohort, are on the roadmap. It is not medical software: it does not support diagnosis or patient-care decisions.

**Links**: [European Health Data Space (EHDS) — EU Commission](https://health.ec.europa.eu/ehealth-digital-health-and-care/european-health-data-space-regulation-ehds_en) · [FHIR v6.0.0-ballot](https://build.fhir.org/ballot-intro.html)
