TotalSegmentator: automatic segmentation of 104 anatomical structures in whole-body CT

AIHealth

On-premise clinical platform with local LLMs, RAG on FHIR/DICOM data, diagnostic support, remote follow-up. Architecture designed for the MDR pathway.

Discover AIHealth →

Digital Health

Medical software development compliant with CE and MDR regulatory standards. Clinical decision support systems, AI integration in clinical workflows.

Discover →

One model to segment everything

Most published medical segmentation models segment one specific structure — liver, prostate, brain tumours — trained on dedicated datasets. A radiologist wanting to quantify multiple structures in the same CT has to use different models, each with its own dependencies, formats, input expectations.

TotalSegmentator — published by Jakob Wasserthal and collaborators from the University of Basel in Radiology: Artificial Intelligence (RSNA) in 2023 (2022 arXiv preprint) — answers this fragmentation with a single model segmenting 104 anatomical structures in whole-body CT images with one call.

The project is coordinated by Shan Yang and Martin Segeroth’s group at University Hospital Basel with collaborations with University of Zurich and DKFZ Heidelberg (combining Basel expertise with nnU-Net state of the art). Weights are distributed under Apache 2.0 and the software is installable via pip install totalsegmentator.

The training dataset

The crucial point of the work is the dataset. The authors annotated 1228 CTs at Basel hospital, following a rigorous protocol:

• Different acquisitions (with/without contrast, various enhancement phases)
• Different scanners
• Different protocols
• Heterogeneous population (age, sex, pathologies)

The 104 structures include:

• Bones (numbered vertebrae T1-L5, ribs, pelvis, femur, scapula, humerus, …)
• Parenchymal organs (liver, spleen, pancreas, kidneys, adrenal glands, thyroid, stomach, colon, bladder, uterus, prostate, …)
• Major vessels (aorta, vena cava, portal vein, carotid arteries, renal arteries, …)
• Muscles (iliopsoas muscle, rectus abdominis, …)
• Lungs (separate lung lobes)
• Heart (atria, ventricles, myocardium, ascending aorta)
• Other structures (trachea, oesophagus, spinal cord, brain, subcutaneous/visceral fat)

The dataset was published as an open dataset under CC BY 4.0, giving the community a reference resource for anatomical segmentation.

The architecture

TotalSegmentator uses nnU-Net as underlying framework. Practical choices:

• 3D fullres model for each subgroup of structures (grouping to avoid excessive GPU memory)
• Resampling to 1.5 mm isotropic as default (the “total” version)
• “fast” version with 3 mm resampling for quick inference
• Multi-fold inference with ensemble

Model sizes run on consumer GPUs (RTX 3090, A6000); inference times on the order of 30 seconds for a reasonable-resolution CT.

Use pipeline

The user interface is deliberately minimal:

# Installation
pip install totalsegmentator

# Inference
TotalSegmentator -i input.nii.gz -o output_folder

Output is a set of NIfTI files (one per structure) or a single multi-class volume, at choice. Each structure has a standardised label enabling automated downstream pipelines.

3D Slicer integration is immediate via an extension that allows execution from the graphical interface. MONAI Label integration available for manual correction scenarios.

Validated performance

The Radiology AI 2023 paper documents performance on 57 external test cases from different institutions, with comparisons against:

• Published specialist models (for individual structures)
• Qualified human annotators (radiologists)

The central result: TotalSegmentator produces segmentations comparable to radiologists on many structures, with average Dice > 0.95 for well-defined main organs (liver, kidneys, lungs, heart) and > 0.90 for more complex structures. Lower but acceptable performance for small variable structures (small vessels, thin muscles).

Community impact

TotalSegmentator had rapid uptake:

• Significant downloads on PyPI since the first release
• Citations in the literature growing exponentially through 2023
• Integration in clinical workflows for retrospective studies
• Basis for derivative research — extensions to uncovered structures, adaptations to MR, radiomics work on automatically produced segmentations

As a practical example: a body composition analysis project (computing lean mass, visceral/subcutaneous fat) that previously required manual annotation on thousands of cases can now be executed in bulk on any CT dataset, producing quantitative metrics per patient.

TotalSegmentator vs. standard nnU-Net

TotalSegmentator is nnU-Net-based but not simply “nnU-Net on a big dataset”. Additional contribution:

• Curated dataset of 1228 CTs with rigorous multi-structure annotations
• Training protocol optimised for multi-structure coverage
• Packaging in an immediately usable open source tool
• Cross-site validation
• Documentation and examples

TotalSegmentator’s value is not architectural (it is nnU-Net) but availability of a ready product meeting a real clinical need.

Recent versions

As of November 2023, TotalSegmentator v2 is rolling out, with:

• Extension to 117 structures (vs. initial 104)
• New structures on cardiac CT (detailed cardiac chambers, coronaries)
• MR support in alpha — separate model for MR body segmentation
• Improved performance on hard structures (small vessels, lymph nodes)

Clinical applications

TotalSegmentator enables clinical workflows that previously required extensive manual annotation:

Radiotherapy

Automatic segmentation of organs at risk for radiotherapy planning. The medical physicist verifies and corrects instead of tracing from scratch — time saving of 5-10x per patient.

Surgical planning

Patient-specific 3D reconstruction for abdominal, thoracic, orthopaedic surgery. Structured segmentations ready for 3D printing or VR visualisation.

Body composition

Sarcopenia, visceral/subcutaneous fat mass studies from routine CTs. Relevance in oncology (sarcopenia as prognosis predictor), endocrinology, rehabilitation.

Radiomic quantification

Standardised anatomical segmentations as ROIs for radiomic pipelines — texture, shape, intensity feature extraction on a fixed structure set, with cross-site reproducibility.

COVID-19 pandemic and sequel monitoring

Longitudinal studies on post-COVID lung sequelae used TotalSegmentator for automatic lung segmentations on thousands of follow-up CTs.

Epidemiological research

Imaging studies on biobanks (UK Biobank, German National Cohort, Italian imaging via CINECA) can apply TotalSegmentator across hundreds of thousands of CTs for quantitative anatomical measures.

Certification and clinical use

Like its open source ancestors, TotalSegmentator is not a certified medical device. Its use in clinical production requires:

• Integration into an IEC 62304-qualified product
• CE marking under MDR (EU) 2017/745 — typically Class IIa (Rule 11 for diagnostic-support software)
• ISO 14971 risk management
• Clinical evaluation
• Post-market surveillance

Some companies are building certified products based on TotalSegmentator or embedding it in broader solutions. The Apache 2.0 licence allows this without impediment.

In the Italian context

Italian adoption in research projects:

• IRCCS and Italian universities use TotalSegmentator in retrospective radiomic studies
• Italian radiotherapy — centres like Pavia, Milan, Padua, Turin explore its use for OAR autosegmentation
• Body composition studies — used in oncology prevention and research programmes

The availability of a reliable open source pipeline dramatically lowers the entry threshold for Italian clinical projects, without commercial licence dependency.

Outlook

Expected future directions:

• Broader anatomical coverage — integration of uncovered structures (fine lymphatic, fine vascular)
• Multi-modal modalities — whole-body MR, PET/CT
• Pediatric pipelines — anatomically different from adults
• Pathological pipelines — anomaly recognition beyond simple anatomical identification
• Integration with predictive AI — anatomical segmentation as input to downstream predictive models (age estimation, mortality, cardiovascular risk)
• Cross-site standardisation — attestation of cross-scanner, cross-contrast, cross-population robustness

TotalSegmentator in 2023 represents a maturation point for open source medical AI: a tool that works on real data, immediately usable, with published validation. The combination of annotated dataset + solid training + accessible packaging is a recipe becoming the standard for upcoming open source clinical AI projects.

References: Jakob Wasserthal et al., “TotalSegmentator: Robust Segmentation of 104 Anatomic Structures in CT Images”, Radiology: Artificial Intelligence (2023). University Hospital Basel, University of Zurich, DKFZ. Apache 2.0 licence, public weights. Dataset open licence CC BY 4.0. pip: totalsegmentator. Technical base: nnU-Net.