BEN_tutorial_Ⅰ
Note: to replicate experiments conveniently, we did not package key functions as ‘.exe’ files here. Preprocessing is also simplified.
# 1.0 Set up
## 1.1 Verify Runtime Settings
** IMPORTANT **
In the “Runtime” menu for the notebook window, select “Change runtime type.” Ensure that the following are selected: * Runtime Type = Python 3 * Hardware Accelerator = GPU
See your Hardware Accelerator in Edit/Notebook settings in Colab Menu(On the upper left usually).
## 1.2 Set up environment and install all necessary packages
After installation, you might ** need to restart ** the runtime in order to use newly installed versions (Tensorflow 1.15).
[ ]:
!pip install SimpleITK tensorflow-gpu==1.15.4 keras==2.2.4 h5py==2.10 scikit-image==0.16.2
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Requirement already satisfied: SimpleITK in /usr/local/lib/python3.7/dist-packages (2.1.1.2)
Requirement already satisfied: tensorflow-gpu==1.15.4 in /usr/local/lib/python3.7/dist-packages (1.15.4)
Requirement already satisfied: keras==2.2.4 in /usr/local/lib/python3.7/dist-packages (2.2.4)
Requirement already satisfied: h5py==2.10 in /usr/local/lib/python3.7/dist-packages (2.10.0)
Requirement already satisfied: scikit-image==0.16.2 in /usr/local/lib/python3.7/dist-packages (0.16.2)
Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (1.15.0)
Requirement already satisfied: numpy<1.19.0,>=1.16.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (1.18.5)
Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (1.1.0)
Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (1.15.0)
Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (1.14.1)
Requirement already satisfied: tensorflow-estimator==1.15.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (1.15.1)
Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (1.1.2)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (0.37.1)
Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (1.47.0)
Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (1.0.8)
Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (3.3.0)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (1.1.0)
Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (0.2.0)
Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (3.17.3)
Requirement already satisfied: gast==0.2.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (0.2.2)
Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-gpu==1.15.4) (0.8.1)
Requirement already satisfied: scipy>=0.14 in /usr/local/lib/python3.7/dist-packages (from keras==2.2.4) (1.7.3)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from keras==2.2.4) (3.13)
Requirement already satisfied: matplotlib!=3.0.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image==0.16.2) (3.2.2)
Requirement already satisfied: imageio>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image==0.16.2) (2.4.1)
Requirement already satisfied: PyWavelets>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image==0.16.2) (1.3.0)
Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image==0.16.2) (2.6.3)
Requirement already satisfied: pillow>=4.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image==0.16.2) (7.1.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image==0.16.2) (0.11.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image==0.16.2) (3.0.9)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image==0.16.2) (1.4.3)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image==0.16.2) (2.8.2)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib!=3.0.0,>=2.0.0->scikit-image==0.16.2) (4.1.1)
Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.4) (1.0.1)
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.4) (57.4.0)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.4) (3.3.7)
Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.4) (4.12.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow-gpu==1.15.4) (3.8.0)
1.3 Download code from github.
[ ]:
!git clone https://github.com/yu02019/BEN.git
Cloning into 'BEN'...
remote: Enumerating objects: 162, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 162 (delta 2), reused 10 (delta 0), pack-reused 143
Receiving objects: 100% (162/162), 6.47 MiB | 31.56 MiB/s, done.
Resolving deltas: 100% (46/46), done.
[ ]:
cd BEN
/content/BEN
## 1.4 Download MRI data.
Note: todo: add
[ ]:
!gdown --id 1fmyBtxNJAYFtlwVBY5IiFsmUixc2MsU9
!sh download.sh
/usr/local/lib/python3.7/dist-packages/gdown/cli.py:131: FutureWarning: Option `--id` was deprecated in version 4.3.1 and will be removed in 5.0. You don't need to pass it anymore to use a file ID.
category=FutureWarning,
Downloading...
From: https://drive.google.com/uc?id=1fmyBtxNJAYFtlwVBY5IiFsmUixc2MsU9
To: /content/BEN/download.sh
100% 273/273 [00:00<00:00, 510kB/s]
/usr/local/lib/python3.7/dist-packages/gdown/cli.py:131: FutureWarning: Option `--id` was deprecated in version 4.3.1 and will be removed in 5.0. You don't need to pass it anymore to use a file ID.
category=FutureWarning,
Downloading...
From: https://drive.google.com/uc?id=1z-VMfPvFILNcaEc9jSYzcnub7t4ZcrfJ
To: /content/BEN/cross_domain/cross_domain.zip
100% 26.8M/26.8M [00:00<00:00, 144MB/s]
Archive: cross_domain.zip
creating: 7T/
creating: 7T/src/
inflating: 7T/src/20171226_185416T1RAREs30001a001.nii.gz
inflating: 7T/src/20180321_133106T2TurboRAREs20001a001.nii.gz
inflating: 7T/src/20180510_143551T1RAREs30001a001.nii.gz
creating: epi/
creating: epi/src/
inflating: epi/src/ASD_Xinhua_SJL_20191215_con06__E3_P1.nii.gz
inflating: epi/src/ASD_Xinhua_SJL_20191215_tcell_0__E3_P1.1.nii.gz
creating: rat/
creating: rat/label/
inflating: rat/label/42d_cas_27_T2.nii.gz
creating: rat/src/
inflating: rat/src/42d_cas_27_T2.nii.gz
inflating: rat/src/42d_cas_40_T2.nii.gz
inflating: rat/src/42d_cas_44_T2.nii.gz
/usr/local/lib/python3.7/dist-packages/gdown/cli.py:131: FutureWarning: Option `--id` was deprecated in version 4.3.1 and will be removed in 5.0. You don't need to pass it anymore to use a file ID.
category=FutureWarning,
Downloading...
From: https://drive.google.com/uc?id=1K5qIe6o8uG7cw_Uu0H3aB0FE40YXb9Hp
To: /content/BEN/weight/weight.zip
100% 69.3M/69.3M [00:00<00:00, 72.9MB/s]
Archive: weight.zip
creating: unet_fp32_all_BN_NoCenterScale_polyic_epoch15_bottle256_04012051/
inflating: unet_fp32_all_BN_NoCenterScale_polyic_epoch15_bottle256_04012051/.data-00000-of-00002
inflating: unet_fp32_all_BN_NoCenterScale_polyic_epoch15_bottle256_04012051/.data-00001-of-00002
inflating: unet_fp32_all_BN_NoCenterScale_polyic_epoch15_bottle256_04012051/.index
inflating: unet_fp32_all_BN_NoCenterScale_polyic_epoch15_bottle256_04012051/checkpoint
creating: unet_fp32_all_BN_NoCenterScale_polyic_epoch15_bottle256_04012056/
inflating: unet_fp32_all_BN_NoCenterScale_polyic_epoch15_bottle256_04012056/.data-00000-of-00002
inflating: unet_fp32_all_BN_NoCenterScale_polyic_epoch15_bottle256_04012056/.data-00001-of-00002
inflating: unet_fp32_all_BN_NoCenterScale_polyic_epoch15_bottle256_04012056/.index
inflating: unet_fp32_all_BN_NoCenterScale_polyic_epoch15_bottle256_04012056/checkpoint
The MRI data look like:
(src: raw MRI scans; label:ground truth)
[ ]:
import os
import logging
import warnings
import tensorflow as tf
warnings.filterwarnings("ignore")
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2' # tf log errors only
logging.getLogger('tensorflow').setLevel(logging.ERROR)
print(tf.__version__)
1.15.4
# 2.0 Cross species (Mouse -> Rat) In this case, only ONE label is required.
## 2.1 Declare key function and path
[ ]:
from utils.update_model import update_weight
# input (3 or more MRI scan)
input_folder = r'cross_domain/rat/src'
# label (use 1 label in this task)
label_folder = r'cross_domain/rat/label'
output_folder_base = os.path.dirname(input_folder) # save pred nii files.(time id based on weight)
cvs_result_base = r'cross_domain/rat' # folder path. Evaluation metric in csv. (path of csv, time id)
need_rotate = True # always use Ture
source_domain_weight = 'weight/unet_fp32_all_BN_NoCenterScale_polyic_epoch15_bottle256_04012056/' # load and adjust BN base on this old model(fixed/source domain usually)
new_model_name_prefix = 'Rat-42d-2022' # The prefix of new weight filename will use this string
[ ]:
def run_main(
BN_list=None,
droprate=None,
new_model_name='unet_temp',
weight=None,
max_num=-1,
freeze=True,
momentum=0.99,
epochs=15,
):
new_weight = update_weight(input_folder, label_folder, need_mkdir=True, weight=weight, need_rotate=need_rotate,
model_name=new_model_name, BN_list=BN_list, droprate=droprate, freeze=freeze,
momentum=momentum, epochs=epochs, max_num=max_num)
print('Finetune done.\n New model path is : ', new_weight)
''' setting filename '''
import re
num_extract = re.findall(r'\d+', new_weight)
output_folder = output_folder_base + '/' + 'pred' + '-' + str(new_model_name) # e.g. 'pred-04061444'
cvs_result = cvs_result_base + '/' + os.path.basename(output_folder) + '.csv'
new_weight = new_weight + '.hdf5'
''' infer and compute csv'''
''' infer whole folder '''
from utils.inference import inference_pipeline
import tensorflow.keras.backend as K
K.clear_session() # release GRAM
inference_pipeline(input_folder, output_folder, is_mkdir=True, weight=new_weight, need_rotate=need_rotate,
BN_list=BN_list, droprate=droprate)
# avoid OOM
''' evaluation output metrics '''
pass
return output_folder
2.2.1 Run baseline methods domain transfer/adaptation
[ ]:
num_list = [1]
# ft at 0 label (finetune with 0% cross_domain / zero-shot )
zeroshot_out_folder = run_main(BN_list=[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],
new_model_name='{}-%{}-ft'.format(new_model_name_prefix, 0),
weight=source_domain_weight, max_num=1, freeze=False, epochs=0)
print('Done zeroshot') # zeroshot_out_folder : 'cross_domain/rat/pred-Rat-42d-2022-%0-ft'
# ft
for num_id in num_list:
ft_out_folder = run_main(BN_list=[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],
new_model_name='{}-%{}-ft'.format(new_model_name_prefix, num_id),
weight=source_domain_weight, max_num=num_id, freeze=False)
print('Done ft')
Finding 3 nii.gz format files.
Reading: 42d_cas_27_T2.nii.gz
(50, 256, 256)
Done.
len of nii_path: 1
Reading: 42d_cas_27_T2.nii.gz
(50, 256, 256)
Done.
********** Inferring CT/MRI scans: **********
finetune all layers on target domain!
Using TensorFlow backend.
Train on 45 samples, validate on 5 samples
New model has trained and saved as: weight/Rat-42d-2022-%0-ft_07170753/
Finetune done.
New model path is : weight/Rat-42d-2022-%0-ft_07170753/
Makedir: cross_domain/rat/pred-Rat-42d-2022-%0-ft
Object will save in (Existing) folder/print(save_filename): cross_domain/rat/pred-Rat-42d-2022-%0-ft
Finding 3 nii.gz format files.
Reading: 42d_cas_27_T2.nii.gz
(50, 256, 256)
Reading: 42d_cas_40_T2.nii.gz
(50, 256, 256)
Reading: 42d_cas_44_T2.nii.gz
(50, 256, 256)
Done.
********** Inferring CT/MRI scans: **********
********** 3 file(s) to save: **********
Saving: (50, 256, 256)
Saving: (50, 256, 256)
Saving: (50, 256, 256)
********** Done. **********
Done zeroshot
Finding 3 nii.gz format files.
Reading: 42d_cas_27_T2.nii.gz
(50, 256, 256)
Done.
len of nii_path: 1
Reading: 42d_cas_27_T2.nii.gz
(50, 256, 256)
Done.
********** Inferring CT/MRI scans: **********
finetune all layers on target domain!
Train on 45 samples, validate on 5 samples
Epoch 1/15
45/45 [==============================] - 8s 185ms/sample - loss: 0.4148 - dice_coef: 0.7091 - val_loss: 1.0412 - val_dice_coef: 0.4951
Epoch 2/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0839 - dice_coef: 0.9088 - val_loss: 1.9823 - val_dice_coef: 0.3484
Epoch 3/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0989 - dice_coef: 0.8991 - val_loss: 2.5603 - val_dice_coef: 0.3236
Epoch 4/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0789 - dice_coef: 0.9288 - val_loss: 2.6299 - val_dice_coef: 0.3225
Epoch 5/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0551 - dice_coef: 0.9428 - val_loss: 2.6021 - val_dice_coef: 0.3239
Epoch 6/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0556 - dice_coef: 0.9451 - val_loss: 2.5453 - val_dice_coef: 0.3218
Epoch 7/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0505 - dice_coef: 0.9485 - val_loss: 2.5122 - val_dice_coef: 0.3172
Epoch 8/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0517 - dice_coef: 0.9507 - val_loss: 2.5609 - val_dice_coef: 0.3143
Epoch 9/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0435 - dice_coef: 0.9565 - val_loss: 2.5965 - val_dice_coef: 0.3155
Epoch 10/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0375 - dice_coef: 0.9538 - val_loss: 2.5762 - val_dice_coef: 0.3164
Epoch 11/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0334 - dice_coef: 0.9588 - val_loss: 2.4212 - val_dice_coef: 0.3249
Epoch 12/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0326 - dice_coef: 0.9586 - val_loss: 2.2402 - val_dice_coef: 0.3401
Epoch 13/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0307 - dice_coef: 0.9602 - val_loss: 2.1123 - val_dice_coef: 0.3578
Epoch 14/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0284 - dice_coef: 0.9638 - val_loss: 2.0613 - val_dice_coef: 0.3721
Epoch 15/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0262 - dice_coef: 0.9665 - val_loss: 2.0853 - val_dice_coef: 0.3787
New model has trained and saved as: weight/Rat-42d-2022-%1-ft_07170754/
Finetune done.
New model path is : weight/Rat-42d-2022-%1-ft_07170754/
Makedir: cross_domain/rat/pred-Rat-42d-2022-%1-ft
Object will save in (Existing) folder/print(save_filename): cross_domain/rat/pred-Rat-42d-2022-%1-ft
Finding 3 nii.gz format files.
Reading: 42d_cas_27_T2.nii.gz
(50, 256, 256)
Reading: 42d_cas_40_T2.nii.gz
(50, 256, 256)
Reading: 42d_cas_44_T2.nii.gz
(50, 256, 256)
Done.
********** Inferring CT/MRI scans: **********
********** 3 file(s) to save: **********
Saving: (50, 256, 256)
Saving: (50, 256, 256)
Saving: (50, 256, 256)
********** Done. **********
Done ft
## 2.2.2 Run BEN’s domain transfer/adaptation
[ ]:
# DA (domain adaptation using BEN's AdaBN)
for num_id in num_list:
DA_out_folder = run_main(BN_list=[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],
new_model_name='{}-%{}-DA'.format(new_model_name_prefix, num_id),
weight=source_domain_weight, max_num=num_id, freeze=True, momentum=0.8)
print('Done BEN AdaBN')
Finding 3 nii.gz format files.
Reading: 42d_cas_27_T2.nii.gz
(50, 256, 256)
Done.
len of nii_path: 1
Reading: 42d_cas_27_T2.nii.gz
(50, 256, 256)
Done.
********** Inferring CT/MRI scans: **********
Trainable lay: batch_normalization_10
Trainable lay: batch_normalization_11
Trainable lay: batch_normalization_12
Trainable lay: batch_normalization_13
Trainable lay: batch_normalization_14
Trainable lay: batch_normalization_15
Trainable lay: batch_normalization_16
Trainable lay: batch_normalization_17
Trainable lay: batch_normalization_18
Trainable lay: batch_normalization_19
Only finetune BN on target domain!
Train on 45 samples, validate on 5 samples
Epoch 1/15
45/45 [==============================] - 1s 18ms/sample - loss: 0.5112 - dice_coef: 0.5646 - val_loss: 0.2656 - val_dice_coef: 0.3957
Epoch 2/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5138 - dice_coef: 0.5647 - val_loss: 0.2340 - val_dice_coef: 0.4507
Epoch 3/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5185 - dice_coef: 0.5481 - val_loss: 0.2176 - val_dice_coef: 0.4763
Epoch 4/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5314 - dice_coef: 0.5663 - val_loss: 0.2249 - val_dice_coef: 0.4757
Epoch 5/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5189 - dice_coef: 0.5637 - val_loss: 0.3050 - val_dice_coef: 0.4505
Epoch 6/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5162 - dice_coef: 0.5628 - val_loss: 0.4617 - val_dice_coef: 0.3992
Epoch 7/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5173 - dice_coef: 0.5556 - val_loss: 0.6839 - val_dice_coef: 0.3411
Epoch 8/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5139 - dice_coef: 0.5701 - val_loss: 0.8865 - val_dice_coef: 0.3118
Epoch 9/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5120 - dice_coef: 0.5594 - val_loss: 1.1239 - val_dice_coef: 0.2921
Epoch 10/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5365 - dice_coef: 0.5736 - val_loss: 1.3558 - val_dice_coef: 0.2784
Epoch 11/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5155 - dice_coef: 0.5663 - val_loss: 1.4377 - val_dice_coef: 0.2702
Epoch 12/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5157 - dice_coef: 0.5644 - val_loss: 1.6448 - val_dice_coef: 0.2610
Epoch 13/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5407 - dice_coef: 0.5651 - val_loss: 1.8445 - val_dice_coef: 0.2540
Epoch 14/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5063 - dice_coef: 0.5780 - val_loss: 1.9374 - val_dice_coef: 0.2474
Epoch 15/15
45/45 [==============================] - 0s 4ms/sample - loss: 0.5150 - dice_coef: 0.5603 - val_loss: 1.8927 - val_dice_coef: 0.2562
Train on 45 samples, validate on 5 samples
Epoch 1/15
45/45 [==============================] - 2s 38ms/sample - loss: 0.4016 - dice_coef: 0.7272 - val_loss: 2.4019 - val_dice_coef: 0.3657
Epoch 2/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0843 - dice_coef: 0.9059 - val_loss: 3.3542 - val_dice_coef: 0.2951
Epoch 3/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0953 - dice_coef: 0.9027 - val_loss: 3.1302 - val_dice_coef: 0.2940
Epoch 4/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0794 - dice_coef: 0.9251 - val_loss: 2.2256 - val_dice_coef: 0.3383
Epoch 5/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0558 - dice_coef: 0.9380 - val_loss: 1.6000 - val_dice_coef: 0.4353
Epoch 6/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0604 - dice_coef: 0.9379 - val_loss: 1.2561 - val_dice_coef: 0.5171
Epoch 7/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0565 - dice_coef: 0.9434 - val_loss: 1.0971 - val_dice_coef: 0.5479
Epoch 8/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0438 - dice_coef: 0.9487 - val_loss: 1.0271 - val_dice_coef: 0.5501
Epoch 9/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0412 - dice_coef: 0.9557 - val_loss: 0.9009 - val_dice_coef: 0.5736
Epoch 10/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0359 - dice_coef: 0.9613 - val_loss: 0.7670 - val_dice_coef: 0.6122
Epoch 11/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0329 - dice_coef: 0.9575 - val_loss: 0.6865 - val_dice_coef: 0.6298
Epoch 12/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0326 - dice_coef: 0.9606 - val_loss: 0.5558 - val_dice_coef: 0.6676
Epoch 13/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0325 - dice_coef: 0.9568 - val_loss: 0.5060 - val_dice_coef: 0.6803
Epoch 14/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0268 - dice_coef: 0.9681 - val_loss: 0.4324 - val_dice_coef: 0.7097
Epoch 15/15
45/45 [==============================] - 0s 11ms/sample - loss: 0.0271 - dice_coef: 0.9664 - val_loss: 0.3877 - val_dice_coef: 0.7293
New model has trained and saved as: weight/Rat-42d-2022-%1-DA_07170754/
Finetune done.
New model path is : weight/Rat-42d-2022-%1-DA_07170754/
Makedir: cross_domain/rat/pred-Rat-42d-2022-%1-DA
Object will save in (Existing) folder/print(save_filename): cross_domain/rat/pred-Rat-42d-2022-%1-DA
Finding 3 nii.gz format files.
Reading: 42d_cas_27_T2.nii.gz
(50, 256, 256)
Reading: 42d_cas_40_T2.nii.gz
(50, 256, 256)
Reading: 42d_cas_44_T2.nii.gz
(50, 256, 256)
Done.
********** Inferring CT/MRI scans: **********
********** 3 file(s) to save: **********
Saving: (50, 256, 256)
Saving: (50, 256, 256)
Saving: (50, 256, 256)
********** Done. **********
Done BEN AdaBN
## 2.3 Visualize the output ### For fair comparision, we download ground truth here after finishing training for plotting. Ground truth will be saved at ‘/content/BEN/cross_domain/rat/gt/’ (click to jump to this folder after running the following cell)
[ ]:
# download ground truth
!gdown --id 1aV3cZBYAwtdiDcW04q1Z2lz7AelobJqb
!sh download_gt.sh
/usr/local/lib/python3.7/dist-packages/gdown/cli.py:131: FutureWarning: Option `--id` was deprecated in version 4.3.1 and will be removed in 5.0. You don't need to pass it anymore to use a file ID.
category=FutureWarning,
Downloading...
From: https://drive.google.com/uc?id=1aV3cZBYAwtdiDcW04q1Z2lz7AelobJqb
To: /content/BEN/download_gt.sh
100% 146/146 [00:00<00:00, 264kB/s]
/usr/local/lib/python3.7/dist-packages/gdown/cli.py:131: FutureWarning: Option `--id` was deprecated in version 4.3.1 and will be removed in 5.0. You don't need to pass it anymore to use a file ID.
category=FutureWarning,
Downloading...
From: https://drive.google.com/uc?id=1EHNors0X7C-BoY2OgrSeV-OFAluN8YhS
To: /content/BEN/cross_domain/gt.zip
100% 49.1k/49.1k [00:00<00:00, 58.2MB/s]
Archive: gt.zip
inflating: 42d_cas_27_T2.nii.gz
inflating: 42d_cas_40_T2.nii.gz
inflating: 42d_cas_44_T2.nii.gz
[ ]:
''' cross species '''
from glob import glob
from utils.load_data import get_itk_array
from utils.visualization import load_slice_cross_species, plot_segmentation_cross_species
raw, zeroshot, ft, DA, gt = load_slice_cross_species(raw_path=r'cross_domain/rat/src',
zeroshot_path=r'cross_domain/rat/pred-Rat-42d-2022-%0-ft',
ft_path=r'cross_domain/rat/pred-Rat-42d-2022-%1-ft',
DA_path=r'cross_domain/rat/pred-Rat-42d-2022-%1-DA',
gt_path=r'cross_domain/rat/gt', scans_num=2)
plot_segmentation_cross_species(raw, zeroshot, ft, DA, gt, task='species', hspace=-0.3, figsize=(16,9))
From top raw to the fifth raw: ** Raw image **, ** Zero-shot ** (0 label used), ** finetune ** (1 label used), ** BEN’s ** result (1 label used), and ** Ground truth **.
## 2.4 (Optional) Simple postprocessing Just do some simple postprocessing here, e.g., only save the top-K largest connected regions. (K=1 in brain extraction)
[ ]:
from utils.postprocess import remove_small_objects
# ft postprocessing
remove_small_objects(input_path=r'cross_domain/rat/pred-Rat-42d-2022-%1-ft', output_path=r'cross_domain/rat/pred-Rat-42d-2022-%1-ft-post')
# BEN postprocessing
remove_small_objects(input_path=r'cross_domain/rat/pred-Rat-42d-2022-%1-DA', output_path=r'cross_domain/rat/pred-Rat-42d-2022-%1-DA-post')
Makedir: cross_domain/rat/pred-Rat-42d-2022-%1-ft-post
Object will save in (Existing) folder/print(save_filename): cross_domain/rat/pred-Rat-42d-2022-%1-ft-post
matrix shape: (50, 256, 256)
matrix shape: (50, 256, 256)
matrix shape: (50, 256, 256)
Makedir: cross_domain/rat/pred-Rat-42d-2022-%1-DA-post
Object will save in (Existing) folder/print(save_filename): cross_domain/rat/pred-Rat-42d-2022-%1-DA-post
matrix shape: (50, 256, 256)
matrix shape: (50, 256, 256)
matrix shape: (50, 256, 256)
[ ]:
''' cross species - postprocessing '''
raw, zeroshot, ft, DA, gt = load_slice_cross_species(raw_path=r'cross_domain/rat/src',
zeroshot_path=r'cross_domain/rat/pred-Rat-42d-2022-%0-ft',
ft_path=r'cross_domain/rat/pred-Rat-42d-2022-%1-ft-post',
DA_path=r'cross_domain/rat/pred-Rat-42d-2022-%1-DA-post',
gt_path=r'cross_domain/rat/gt', scans_num=2)
plot_segmentation_cross_species(raw, zeroshot, ft, DA, gt, task='species', hspace=-0.3, figsize=(16,9))
From top raw to the fifth raw: ** Raw image **, ** Zero-shot ** (0 label used), ** finetune ** (1 label used), ** BEN’s ** result (1 label used), and ** Ground truth **.
Compared with other methods, it further shows BEN’s advantages.
# 3.0 Conclusion
BEN shows the capacity to generalize well to other domains with minimal or even without additional labeled data, while the baseline approaches lead to abundant errors.
# 4.0 (Optional) Download new weight
After updating weight [1], BEN has already deployed on this new domain. You can download this weight to use next time without the abovementioned steps.
[1] In practice, we recommend using 4~6 target domain labels for domain adaptation. We also provide a semi-supervised or human-in-the-loop manner to deploy BEN.
[ ]:
import os
from google.colab import files
new_weight = '{}-%{}-DA'.format(new_model_name_prefix, num_id)
new_weight_folder = glob(r'/content/BEN/weight/' + new_weight + '*')[0]
print(new_weight_folder)
os.system(f"zip -r -j download.zip {new_weight_folder}/*")
files.download("/content/BEN/weight/weight.zip")
/content/BEN/weight/Rat-42d-2022-%1-DA_07170754
[ ]: