Backbone

Interface

Local Net

class deepreg.model.backbone.local_net.LocalNet(*args: Any, **kwargs: Any)

Build LocalNet for image registration.

Reference:

Init.

Image is encoded gradually, i from level 0 to D, then it is decoded gradually, j from level D to 0. Some of the decoded levels are used for generating extractions.

So, extract_levels are between [0, D].

Parameters

  • image_size – such as (dim1, dim2, dim3)

  • num_channel_initial – number of initial channels.

  • extract_levels – from which depths the output will be built.

  • out_kernel_initializer – initializer to use for kernels.

  • out_activation – activation to use at end layer.

  • out_channels – number of channels for the extractions

  • depth – depth of the encoder. If depth is not given, depth = max(extract_levels) will be used.

  • use_additive_upsampling – whether use additive up-sampling layer for decoding.

  • pooling – for down-sampling, use non-parameterized pooling if true, otherwise use conv3d

  • concat_skip – when up-sampling, concatenate skipped tensor if true, otherwise use addition

  • name – name of the backbone.

  • kwargs – additional arguments.

build_bottom_block(filters: int, kernel_size: int, padding: str) → Union[tensorflow.keras.Model, tensorflow.keras.layers.Layer]

Build a block for bottom layer.

This block do not change the tensor shape (width, height, depth), it only changes the number of channels.

Parameters

  • filters – number of channels for output

  • kernel_size – arg for conv3d

  • padding – arg for conv3d

Returns

a block consists of one or multiple layers

build_output_block(image_size: Tuple[int, …], extract_levels: Tuple[int, …], out_channels: int, out_kernel_initializer: str, out_activation: str) → Union[tensorflow.keras.Model, tensorflow.keras.layers.Layer]

Build a block for output.

The input to this block is a list of tensors.

Parameters

  • image_size – such as (dim1, dim2, dim3)

  • extract_levels – number of extraction levels.

  • out_channels – number of channels for the extractions

  • out_kernel_initializer – initializer to use for kernels.

  • out_activation – activation to use at end layer.

Returns

a block consists of one or multiple layers

build_up_sampling_block(filters: int, output_padding: Union[Tuple[int, …], int], kernel_size: Union[Tuple[int, …], int], padding: str, strides: Union[Tuple[int, …], int], output_shape: Tuple[int, …]) → Union[tensorflow.keras.Model, tensorflow.keras.layers.Layer]

Build a block for up-sampling.

This block changes the tensor shape (width, height, depth), but it does not changes the number of channels.

Parameters

  • filters – number of channels for output

  • output_padding – padding for output

  • kernel_size – arg for deconv3d

  • padding – arg for deconv3d

  • strides – arg for deconv3d

  • output_shape – shape of the output tensor

Returns

a block consists of one or multiple layers

get_config() → dict

Return the config dictionary for recreating this class.

Global Net

class deepreg.model.backbone.global_net.GlobalNet(*args: Any, **kwargs: Any)

Build GlobalNet for image registration.

GlobalNet is a special UNet where the decoder for up-sampling is skipped. The network’s outputs come from the bottom layer from the encoder directly.

Reference:

Image is encoded gradually, i from level 0 to E. Then, a densely-connected layer outputs an affine transformation.

Parameters

  • image_size – tuple, such as (dim1, dim2, dim3)

  • num_channel_initial – int, number of initial channels

  • extract_levels – list, which levels from net to extract, deprecated. If depth is not given, depth = max(extract_levels) will be used.

  • depth – depth of the encoder. If given, extract_levels is not used.

  • name – name of the backbone.

  • kwargs – additional arguments.

build_output_block(image_size: Tuple[int, …], extract_levels: Tuple[int, …], out_channels: int, out_kernel_initializer: str, out_activation: str) → Union[tensorflow.keras.Model, tensorflow.keras.layers.Layer]

Build a block for output.

The input to this block is a list of length 1. The output has two tensors.

Parameters

  • image_size – such as (dim1, dim2, dim3)

  • extract_levels – not used

  • out_channels – not used

  • out_kernel_initializer – not used

  • out_activation – not used

Returns

a block consists of one or multiple layers

U-Net

class deepreg.model.backbone.u_net.UNet(*args: Any, **kwargs: Any)

Class that implements an adapted 3D UNet.

Reference:

  • O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,”, Lecture Notes in Computer Science, 2015, vol. 9351, pp. 234–241. https://arxiv.org/abs/1505.04597

Initialise UNet.

Parameters

  • image_size – (dim1, dim2, dim3), dims of input image.

  • num_channel_initial – number of initial channels

  • depth – input is at level 0, bottom is at level depth.

  • out_kernel_initializer – kernel initializer for the last layer

  • out_activation – activation at the last layer

  • out_channels – number of channels for the output

  • extract_levels – list, which levels from net to extract.

  • pooling – for down-sampling, use non-parameterized pooling if true, otherwise use conv3d

  • concat_skip – when up-sampling, concatenate skipped tensor if true, otherwise use addition

  • encode_kernel_sizes – kernel size for down-sampling

  • decode_kernel_sizes – kernel size for up-sampling

  • encode_num_channels – filters/channels for down-sampling, by default it is doubled at each layer during down-sampling

  • decode_num_channels – filters/channels for up-sampling, by default it is the same as encode_num_channels

  • strides – strides for down-sampling

  • padding – padding mode for all conv layers

  • name – name of the backbone.

  • kwargs – additional arguments.

build_bottom_block(filters: int, kernel_size: int, padding: str) → Union[tensorflow.keras.Model, tensorflow.keras.layers.Layer]

Build a block for bottom layer.

This block do not change the tensor shape (width, height, depth), it only changes the number of channels.

Parameters

  • filters – number of channels for output

  • kernel_size – arg for conv3d

  • padding – arg for conv3d

Returns

a block consists of one or multiple layers

build_decode_conv_block(filters: int, kernel_size: int, padding: str) → Union[tensorflow.keras.Model, tensorflow.keras.layers.Layer]

Build a conv block for up-sampling

This block do not change the tensor shape (width, height, depth), it only changes the number of channels.

Parameters

  • filters – number of channels for output

  • kernel_size – arg for conv3d

  • padding – arg for conv3d

Returns

a block consists of one or multiple layers

build_decode_layers(tensor_shapes: List[Tuple], image_size: Tuple, num_channels: Tuple, depth: int, extract_levels: Tuple[int, …], decode_kernel_sizes: Union[int, List[int]], strides: int, padding: str, out_kernel_initializer: str, out_activation: str, out_channels: int)

Build layers for decoding.

Parameters

  • tensor_shapes – shapes calculated in encoder

  • image_size – (dim1, dim2, dim3).

  • num_channels – number of channels for each layer, starting from the top layer.

  • depth – network starts with d = 0, and the bottom has d = depth.

  • extract_levels – from which depths the output will be built.

  • decode_kernel_sizes – kernel size for up-sampling

  • strides – strides for down-sampling

  • padding – padding mode for all conv layers

  • out_kernel_initializer – initializer to use for kernels.

  • out_activation – activation to use at end layer.

  • out_channels – number of channels for the extractions

build_down_sampling_block(filters: int, kernel_size: int, padding: str, strides: int) → Union[tensorflow.keras.Model, tensorflow.keras.layers.Layer]

Build a block for down-sampling.

This block changes the tensor shape (width, height, depth), but it does not changes the number of channels.

Parameters

  • filters – number of channels for output, arg for conv3d

  • kernel_size – arg for pool3d or conv3d

  • padding – arg for pool3d or conv3d

  • strides – arg for pool3d or conv3d

Returns

a block consists of one or multiple layers

build_encode_conv_block(filters: int, kernel_size: int, padding: str) → Union[tensorflow.keras.Model, tensorflow.keras.layers.Layer]

Build a conv block for down-sampling

This block do not change the tensor shape (width, height, depth), it only changes the number of channels.

Parameters

  • filters – number of channels for output

  • kernel_size – arg for conv3d

  • padding – arg for conv3d

Returns

a block consists of one or multiple layers

build_encode_layers(image_size: Tuple, num_channels: Tuple, depth: int, encode_kernel_sizes: Union[int, List[int]], strides: int, padding: str) → List[Tuple]

Build layers for encoding.

Parameters

  • image_size – (dim1, dim2, dim3).

  • num_channels – number of channels for each layer, starting from the top layer.

  • depth – network starts with d = 0, and the bottom has d = depth.

  • encode_kernel_sizes – kernel size for down-sampling

  • strides – strides for down-sampling

  • padding – padding mode for all conv layers

Returns

list of tensor shapes starting from d = 0

build_layers(image_size: tuple, num_channel_initial: int, depth: int, extract_levels: Tuple[int, …], encode_kernel_sizes: Union[int, List[int]], decode_kernel_sizes: Union[int, List[int]], encode_num_channels: Optional[Tuple], decode_num_channels: Optional[Tuple], strides: int, padding: str, out_kernel_initializer: str, out_activation: str, out_channels: int)

Build layers that will be used in call.

Parameters

  • image_size – (dim1, dim2, dim3).

  • num_channel_initial – number of initial channels.

  • depth – network starts with d = 0, and the bottom has d = depth.

  • extract_levels – from which depths the output will be built.

  • encode_kernel_sizes – kernel size for down-sampling

  • decode_kernel_sizes – kernel size for up-sampling

  • encode_num_channels – filters/channels for down-sampling, by default it is doubled at each layer during down-sampling

  • decode_num_channels – filters/channels for up-sampling, by default it is the same as encode_num_channels

  • strides – strides for down-sampling

  • padding – padding mode for all conv layers

  • out_kernel_initializer – initializer to use for kernels.

  • out_activation – activation to use at end layer.

  • out_channels – number of channels for the extractions

build_output_block(image_size: Tuple[int, …], extract_levels: Tuple[int, …], out_channels: int, out_kernel_initializer: str, out_activation: str) → Union[tensorflow.keras.Model, tensorflow.keras.layers.Layer]

Build a block for output.

The input to this block is a list of tensors.

Parameters

  • image_size – such as (dim1, dim2, dim3)

  • extract_levels – number of extraction levels.

  • out_channels – number of channels for the extractions

  • out_kernel_initializer – initializer to use for kernels.

  • out_activation – activation to use at end layer.

Returns

a block consists of one or multiple layers

build_skip_block() → Union[tensorflow.keras.Model, tensorflow.keras.layers.Layer]

Build a block for combining skipped tensor and up-sampled one.

This block do not change the tensor shape (width, height, depth), it only changes the number of channels.

The input to this block is a list of tensors.

Returns

a block consists of one or multiple layers

build_up_sampling_block(filters: int, output_padding: Union[Tuple[int, …], int], kernel_size: Union[Tuple[int, …], int], padding: str, strides: Union[Tuple[int, …], int], output_shape: Tuple[int, …]) → Union[tensorflow.keras.Model, tensorflow.keras.layers.Layer]

Build a block for up-sampling.

This block changes the tensor shape (width, height, depth), but it does not changes the number of channels.

Parameters

  • filters – number of channels for output

  • output_padding – padding for output

  • kernel_size – arg for deconv3d

  • padding – arg for deconv3d

  • strides – arg for deconv3d

  • output_shape – shape of the output tensor

Returns

a block consists of one or multiple layers

call(inputs: tensorflow.Tensor, training=None, mask=None) → tensorflow.Tensor

Build compute graph based on built layers.

Parameters

  • inputs – image batch, shape = (batch, f_dim1, f_dim2, f_dim3, ch)

  • training – None or bool.

  • mask – None or tf.Tensor.

Returns

shape = (batch, f_dim1, f_dim2, f_dim3, out_channels)

get_config() → dict

Return the config dictionary for recreating this class.