nnunetv2
medical-imaging
radiology
3d
self-supervised
foundation-model
nnunet
nnssl
consti95 commited on
Commit
8edba04
·
verified ·
1 Parent(s): f8964be

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ nnFoundation_banner.webp filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,129 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-sa-4.0
3
+ library_name: nnunetv2
4
+ tags:
5
+ - medical-imaging
6
+ - radiology
7
+ - 3d
8
+ - self-supervised
9
+ - foundation-model
10
+ - nnunet
11
+ - nnssl
12
+ - arxiv:2609.26924
13
+ ---
14
+
15
+ ![nnFoundation](nnFoundation_banner.webp)
16
+
17
+ ## nnFoundationCNN
18
+ <sub>Copyright German Cancer Research Center (DKFZ) and contributors. Please make sure that your usage of these models is in compliance with their license.<sub>
19
+
20
+ [![arXiv](https://img.shields.io/badge/arXiv-2609.26924-b31b1b.svg)](https://arxiv.org/abs/2609.26924)
21
+
22
+ `nnFoundationCNN` is one of the two **nnFoundation** 3D radiology foundation models, pre-trained
23
+ with the [`nnssl`](https://github.com/MIC-DKFZ/nnssl) self-supervised learning framework.
24
+
25
+ ## Model pair
26
+
27
+ | Model | Architecture | Details | Params | Repository |
28
+ |---|---|---|---|---|
29
+ | **nnFoundationCNN** | [ResEnc](https://arxiv.org/abs/2404.09556) | 6 stages, features 32–64–128–256–320–320 | 102M | this repository |
30
+ | nnFoundationCNN | [Primus](https://arxiv.org/abs/2503.01835) | 40 layers, embedding dim 1056, 16 heads, 8³ patch tokens | 674M | [MIC-DKFZ/nnFoundationViT](https://huggingface.co/MIC-DKFZ/nnFoundationViT) |
31
+
32
+ ## Using nnFoundation
33
+
34
+ To fine-tune nnFoundation on your own downstream tasks, use one of our dedicated repositories:
35
+
36
+ - **Segmentation:** [nnU-Net -- Fine-tuning from nnssl checkpoints](https://github.com/MIC-DKFZ/nnUNet/blob/master/documentation/finetuning_from_nnssl_checkpoints.md)
37
+ - **Detection:** TBA
38
+ - **Classification:** TBA
39
+ - **Report generation:** TBA
40
+
41
+ ### Planning and preprocessing straight from this repository
42
+
43
+ [nnU-Net](https://github.com/MIC-DKFZ/nnUNet/blob/master/documentation/finetuning_from_nnssl_checkpoints.md) can pull these weights itself — pass the repository URL where a checkpoint path is
44
+ expected. Set `nnssl_pretrained_models` first; that is where the download is cached.
45
+
46
+ ```bash
47
+ export nnssl_pretrained_models=/path/to/pretrained_models
48
+
49
+ nnUNetv2_preprocess_like_nnssl \
50
+ -d <DATASET_ID> \
51
+ -n <UniquePretrainingName> \
52
+ -pc https://huggingface.co/MIC-DKFZ/nnFoundationCNN \
53
+ -am like_pretrained
54
+ ```
55
+
56
+ Or download the file yourself and pass a local path:
57
+
58
+ ```python
59
+ from huggingface_hub import hf_hub_download
60
+
61
+ ckpt = hf_hub_download("MIC-DKFZ/nnFoundationCNN", "checkpoint_final.pth")
62
+ ```
63
+
64
+ ## Repository contents
65
+
66
+ | File | Purpose |
67
+ |---|---|
68
+ | `checkpoint_final.pth` | the pre-trained weights (102M parameters) |
69
+ | `adaptation_plan.json` | architecture + preprocessing plan; nnU-Net reads this to confirm compatibility |
70
+ | `config.json` | placeholder so the Hub records download counts |
71
+
72
+ ## Expected input
73
+
74
+ Single-channel 3D volumes, **Z-score normalised**, with **no resampling**.
75
+ Recommended downstream patch size: **192 × 192 × 192**.
76
+
77
+ ## Checkpoint format
78
+
79
+ `checkpoint_final.pth` is a `torch.save` dictionary that loads safely with `weights_only=True`:
80
+
81
+ | Key | Contents |
82
+ |---|---|
83
+ | `network_weights` | the pre-trained `state_dict` |
84
+ | `nnssl_adaptation_plan` | same content as `adaptation_plan.json` |
85
+ | `citations` | the reference(s) to cite when using these weights |
86
+
87
+ > **Note:** this checkpoint stores a number of `state_dict` entries as aliases of the same
88
+ > underlying tensor (the `all_modules.*` keys mirror the named conv/norm modules). This is
89
+ > expected for ResEnc and is why the file is 410 MB rather than ~1.5 GB — do not deduplicate
90
+ > these keys, or loading will fail.
91
+
92
+ ## Citation
93
+
94
+ If you use the **nnFoundation** models or the `nnssl` framework, please cite:
95
+
96
+ <details>
97
+ <summary>nnFoundation BibTeX</summary>
98
+
99
+ ```bibtex
100
+ @misc{harsy2026nnfoundation3dfoundationmodels,
101
+ title={nnFoundation: 3D Foundation Models for Radiology},
102
+ author={Constantin Ulrich Harsy and Tassilo Wald and Karol Gotkowski and Yannick Kirchhoff and Marcel Knopp and Maximilian Rokuss and Elisa Stegmeier and Philipp Schader and Dasha Trofimova and Raphael Stock and Kim-Celine Kahl and Stephen Schaumann and Selen Erkan and David Zimmerer and Stefan Denner and Moritz Langenberg and Sebastian Ziegler and Katharina Eckstein and Maximilian Fischer and Jonathan Suprijadi and Bálint Kovács and Benjamin Hamm and Anand Deshpande and Dimitrios Bounias and Nico Disch and Shuhan Xiao and Jessica Kächele and Jan Sellner and Rajesh Baidya and Jeremias Traub and Lars Krämer and Maximilian Zenk and Tim Rädsch and Stefan Dvoretskii and Robin Peretzke and Jonathan Deissler and Alexandra Ertl and Partha Ghosh and Kris Dreher and Stefan Dinkelacker and Annika Reinke and Evangelia Christodoulou and Numan Saeed and Yoland Savriama and Santiago Estrada and David Kügler and Laura Alexandra Daza Barragan and Cristina Isabel Gonzalez Osorio and Jan Peeken and Michael Baumgartner and Marvin Teichmann and Guillaume Chabin and Matthias Kirchler and Valentin Koch and for the ALFA study and Markus Hohenhaus and Dimitri Koslov and Nina Decker and Mohammad Yaqub and Arnd Heuser and Martin Reuter and Julia A. Schnabel and Tobias Heimann and Florin Ghesu and Paul Brachmann and Claus P. Heußel and Alexander Radbruch and Gianluca Brugnara and Aditya Rastogi and Martha Foltyn-Dumitru and Heinz-Peter Schlemmer and Ignaz Reicht and Julius C. Holzschuh and Michael Bach and Bram Stieltjes and Kai Schlamp and Lena Maier-Hein and Marco Nolden and Ralf Floca and Paul F. Jäger and Philipp Vollmuth and Fabian Isensee and Klaus H. Maier-Hein},
103
+ year={2026},
104
+ eprint={2609.26924},
105
+ archivePrefix={arXiv},
106
+ primaryClass={cs.CV},
107
+ url={https://arxiv.org/abs/2609.26924},
108
+ }
109
+ ```
110
+
111
+ </details>
112
+
113
+ If you use the `nnssl` framework, the **OpenMind dataset**, or the **OpenMind checkpoints**, please cite:
114
+
115
+ <details>
116
+ <summary>OpenMind BibTeX</summary>
117
+
118
+ ```bibtex
119
+ @InProceedings{Wald_2025_ICCV,
120
+ author = {Wald, Tassilo and Ulrich, Constantin and Suprijadi, Jonathan and Ziegler, Sebastian and Nohel, Michal and Peretzke, Robin and Kohler, Gregor and Maier-Hein, Klaus},
121
+ title = {An OpenMind for 3D Medical Vision Self-supervised Learning},
122
+ booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
123
+ month = {October},
124
+ year = {2025},
125
+ pages = {23839-23879}
126
+ }
127
+ ```
128
+
129
+ </details>
adaptation_plan.json ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architecture_plans": {
3
+ "arch_class_name": "ResEncL",
4
+ "arch_kwargs": null,
5
+ "arch_kwargs_requiring_import": null
6
+ },
7
+ "pretrain_plan": {
8
+ "dataset_name": "Dataset000_Pretrain",
9
+ "plans_name": "nnsslPlans",
10
+ "original_median_spacing_after_transp": [
11
+ 1,
12
+ 1,
13
+ 1
14
+ ],
15
+ "image_reader_writer": "SimpleITKIO",
16
+ "transpose_forward": [
17
+ 0,
18
+ 1,
19
+ 2
20
+ ],
21
+ "transpose_backward": [
22
+ 0,
23
+ 1,
24
+ 2
25
+ ],
26
+ "configurations": {
27
+ "noresample": {
28
+ "data_identifier": "nnsslPlans_noresample",
29
+ "preprocessor_name": "DefaultPreprocessor",
30
+ "spacing_style": "noresample",
31
+ "normalization_schemes": [
32
+ "ZScoreNormalization"
33
+ ],
34
+ "use_mask_for_norm": [
35
+ false
36
+ ],
37
+ "resampling_fn_data": "resample_data_or_seg_to_shape",
38
+ "resampling_fn_data_kwargs": {
39
+ "is_seg": false,
40
+ "order": 3,
41
+ "order_z": 0,
42
+ "force_separate_z": null
43
+ },
44
+ "resampling_fn_mask": "resample_data_or_seg_to_shape",
45
+ "resampling_fn_mask_kwargs": {
46
+ "is_seg": true,
47
+ "order": 1,
48
+ "order_z": 0,
49
+ "force_separate_z": null
50
+ },
51
+ "spacing": null,
52
+ "patch_size": [
53
+ 192,
54
+ 192,
55
+ 192
56
+ ]
57
+ }
58
+ },
59
+ "experiment_planner_used": "ExperimentPlanner"
60
+ },
61
+ "pretrain_num_input_channels": 1,
62
+ "recommended_downstream_patchsize": [
63
+ 192,
64
+ 192,
65
+ 192
66
+ ],
67
+ "key_to_encoder": "encoder.stages",
68
+ "key_to_stem": "encoder.stem",
69
+ "keys_to_in_proj": [
70
+ "encoder.stem.convs.0.conv",
71
+ "encoder.stem.convs.0.all_modules.0"
72
+ ],
73
+ "key_to_lpe": null,
74
+ "citations": [
75
+ {
76
+ "type": "Model",
77
+ "name": "nnFoundation",
78
+ "bibtex_citations": [
79
+ "@misc{harsy2026nnfoundation3dfoundationmodels,\n title={nnFoundation: 3D Foundation Models for Radiology}, \n author={Constantin Ulrich Harsy and Tassilo Wald and Karol Gotkowski and Yannick Kirchhoff and Marcel Knopp and Maximilian Rokuss and Elisa Stegmeier and Philipp Schader and Dasha Trofimova and Raphael Stock and Kim-Celine Kahl and Stephen Schaumann and Selen Erkan and David Zimmerer and Stefan Denner and Moritz Langenberg and Sebastian Ziegler and Katharina Eckstein and Maximilian Fischer and Jonathan Suprijadi and Bálint Kovács and Benjamin Hamm and Anand Deshpande and Dimitrios Bounias and Nico Disch and Shuhan Xiao and Jessica Kächele and Jan Sellner and Rajesh Baidya and Jeremias Traub and Lars Krämer and Maximilian Zenk and Tim Rädsch and Stefan Dvoretskii and Robin Peretzke and Jonathan Deissler and Alexandra Ertl and Partha Ghosh and Kris Dreher and Stefan Dinkelacker and Annika Reinke and Evangelia Christodoulou and Numan Saeed and Yoland Savriama and Santiago Estrada and David Kügler and Laura Alexandra Daza Barragan and Cristina Isabel Gonzalez Osorio and Jan Peeken and Michael Baumgartner and Marvin Teichmann and Guillaume Chabin and Matthias Kirchler and Valentin Koch and for the ALFA study and Markus Hohenhaus and Dimitri Koslov and Nina Decker and Mohammad Yaqub and Arnd Heuser and Martin Reuter and Julia A. Schnabel and Tobias Heimann and Florin Ghesu and Paul Brachmann and Claus P. Heußel and Alexander Radbruch and Gianluca Brugnara and Aditya Rastogi and Martha Foltyn-Dumitru and Heinz-Peter Schlemmer and Ignaz Reicht and Julius C. Holzschuh and Michael Bach and Bram Stieltjes and Kai Schlamp and Lena Maier-Hein and Marco Nolden and Ralf Floca and Paul F. Jäger and Philipp Vollmuth and Fabian Isensee and Klaus H. Maier-Hein},\n year={2026},\n eprint={2609.26924},\n archivePrefix={arXiv},\n primaryClass={cs.CV},\n url={https://arxiv.org/abs/2609.26924}, \n}"
80
+ ]
81
+ }
82
+ ]
83
+ }
checkpoint_final.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac262d3e8c226c79f9567fddc34d38e011284730ddf2294b5c6c319ce2f22bbd
3
+ size 409579442
config.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "description": "Dummy config to allow tracking HF downloads."
3
+ }
nnFoundation_banner.webp ADDED

Git LFS Details

  • SHA256: 686b06d9221e868f0e0453cbeef1ffa6da4c70c15d76b6fc1f23a20afd9d5a5d
  • Pointer size: 132 Bytes
  • Size of remote file: 6.31 MB