Layer¶

Activation¶

AdditiveUpSampling¶

Conv3d¶

Conv3dBlock¶

class deepreg.model.layer.Conv3dBlock(*args: Any, **kwargs: Any)¶

A conv3d block having conv3d - norm - activation.

Init.

Parameters

name – name of the layer
kwargs – additional arguments.

Conv3dWithResize¶

Deconv3d¶

Deconv3dBlock¶

class deepreg.model.layer.Deconv3dBlock(*args: Any, **kwargs: Any)¶

A deconv3d block having conv3d - norm - activation.

Init.

Parameters

name – name of the layer
kwargs – additional arguments.

Dense¶

DownSampleResnetBlock¶

IntDVF¶

class deepreg.model.layer.IntDVF(*args: Any, **kwargs: Any)¶

Integrate DVF to get DDF.

Reference:

integrate_vec of neuron https://github.com/adalca/neurite/blob/legacy/neuron/utils.py

Init.

Parameters

fixed_image_size – tuple, (f_dim1, f_dim2, f_dim3)
num_steps – int, number of steps for integration
name – name of the layer
kwargs – additional arguments.

call(inputs: tensorflow.Tensor, **kwargs) → tensorflow.Tensor¶

Parameters

inputs – dvf, shape = (batch, f_dim1, f_dim2, f_dim3, 3)
kwargs – additional arguments.

Returns

ddf, shape = (batch, f_dim1, f_dim2, f_dim3, 3)

get_config() → dict¶: Return the config dictionary for recreating this class.

LocalNetResidual3dBlock¶

LocalNetUpSampleResnetBlock¶

MaxPool3d¶

Norm¶

Residual3dBlock¶

UpSampleResnetBlock¶

Warping¶

class deepreg.model.layer.Warping(*args: Any, **kwargs: Any)¶

Warps an image with DDF.

Reference:

https://github.com/adalca/neurite/blob/legacy/neuron/utils.py where vol = image, loc_shift = ddf

Init.

Parameters

fixed_image_size – shape = (f_dim1, f_dim2, f_dim3) or (f_dim1, f_dim2, f_dim3, ch) with the last channel for features
name – name of the layer
kwargs – additional arguments.

call(inputs, **kwargs) → tensorflow.Tensor¶

Parameters

inputs –
(ddf, image)
- ddf, shape = (batch, f_dim1, f_dim2, f_dim3, 3)
- image, shape = (batch, m_dim1, m_dim2, m_dim3)
kwargs – additional arguments.

Returns

shape = (batch, f_dim1, f_dim2, f_dim3)

get_config() → dict¶: Return the config dictionary for recreating this class.

Util¶

Module containing utilities for layer inputs

deepreg.model.layer_util.deconv_output_padding(input_shape: Union[Tuple[int, …], int], output_shape: Union[Tuple[int, …], int], kernel_size: Union[Tuple[int, …], int], stride: Union[Tuple[int, …], int], padding: str) → Union[Tuple[int, …], int]¶

Calculate output padding for Conv3DTranspose in any dimension.

Parameters

input_shape – shape of Conv3DTranspose input tensor, without batch or channel
output_shape – shape of Conv3DTranspose output tensor, without batch or channel
kernel_size – kernel size of Conv3DTranspose layer
stride – stride of Conv3DTranspose layer
padding – padding of Conv3DTranspose layer

Returns

output_padding for Conv3DTranspose layer

deepreg.model.layer_util.gaussian_filter_3d(kernel_sigma: Union[Tuple, List]) → tensorflow.Tensor¶

Define a gaussian filter in 3d for smoothing.

The filter size is defined 3*kernel_sigma

Parameters: kernel_sigma – the deviation at each direction (list) or use an isotropic deviation (int)
Returns: kernel: tf.Tensor specify a gaussian kernel of shape: [3*k for k in kernel_sigma]

deepreg.model.layer_util.get_n_bits_combinations(num_bits: int) → List[List[int]]¶

Function returning list containing all combinations of n bits. Given num_bits binary bits, each bit has value 0 or 1, there are in total 2**n_bits combinations.

Parameters

num_bits – int, number of combinations to evaluate

Returns

a list of length 2**n_bits, return[i] is the binary representation of the decimal integer.

Example

>>> from deepreg.model.layer_util import get_n_bits_combinations
>>> get_n_bits_combinations(3)
[[0, 0, 0], # 0
 [0, 0, 1], # 1
 [0, 1, 0], # 2
 [0, 1, 1], # 3
 [1, 0, 0], # 4
 [1, 0, 1], # 5
 [1, 1, 0], # 6
 [1, 1, 1]] # 7

deepreg.model.layer_util.get_reference_grid(grid_size: Union[Tuple[int, …], List[int]]) → tensorflow.Tensor¶

Generate a 3D grid with given size.

Reference:

volshape_to_meshgrid of neuron https://github.com/adalca/neurite/blob/legacy/neuron/utils.py

neuron modifies meshgrid to make it faster, however local benchmark suggests tf.meshgrid is better

Note:

for tf.meshgrid, in the 3-D case with inputs of length M, N and P, outputs are of shape (N, M, P) for ‘xy’ indexing and (M, N, P) for ‘ij’ indexing.

Parameters: grid_size – list or tuple of size 3, [dim1, dim2, dim3]
Returns: shape = (dim1, dim2, dim3, 3), grid[i, j, k, :] = [i j k]

deepreg.model.layer_util.pyramid_combination(values: list, weight_floor: list, weight_ceil: list) → tensorflow.Tensor¶

Calculates linear interpolation (a weighted sum) using values of hypercube corners in dimension n.

For example, when num_dimension = len(loc_shape) = num_bits = 3 values correspond to values at corners of following coordinates

[[0, 0, 0], # even
 [0, 0, 1], # odd
 [0, 1, 0], # even
 [0, 1, 1], # odd
 [1, 0, 0], # even
 [1, 0, 1], # odd
 [1, 1, 0], # even
 [1, 1, 1]] # odd

values[::2] correspond to the corners with last coordinate == 0

[[0, 0, 0],
 [0, 1, 0],
 [1, 0, 0],
 [1, 1, 0]]

values[1::2] correspond to the corners with last coordinate == 1

[[0, 0, 1],
 [0, 1, 1],
 [1, 0, 1],
 [1, 1, 1]]

The weights correspond to the floor corners. For example, when num_dimension = len(loc_shape) = num_bits = 3, weight_floor = [f1, f2, f3] (ignoring the batch dimension). weight_ceil = [c1, c2, c3] (ignoring the batch dimension).

So for corner with coords (x, y, z), x, y, z’s values are 0 or 1

weight for x = f1 if x = 0 else c1
weight for y = f2 if y = 0 else c2
weight for z = f3 if z = 0 else c3

so the weight for (x, y, z) is

W_xyz = ((1-x) * f1 + x * c1)
      * ((1-y) * f2 + y * c2)
      * ((1-z) * f3 + z * c3)

Let

W_xy = ((1-x) * f1 + x * c1)
     * ((1-y) * f2 + y * c2)

Then

W_xy0 = W_xy * f3
W_xy1 = W_xy * c3

Similar to W_xyz, denote V_xyz the value at (x, y, z), the final sum V equals

  sum over x,y,z (V_xyz * W_xyz)
= sum over x,y (V_xy0 * W_xy0 + V_xy1 * W_xy1)
= sum over x,y (V_xy0 * W_xy * f3 + V_xy1 * W_xy * c3)
= sum over x,y (V_xy0 * W_xy) * f3 + sum over x,y (V_xy1 * W_xy) * c3

That’s why we call this pyramid combination. It calculates the linear interpolation gradually, starting from the last dimension. The key is that the weight of each corner is the product of the weights along each dimension.

Parameters

values – a list having values on the corner, it has 2**n tensors of shape (*loc_shape) or (batch, *loc_shape) or (batch, *loc_shape, ch) the order is consistent with get_n_bits_combinations loc_shape is independent from n, aka num_dim
weight_floor – a list having weights of floor points, it has n tensors of shape (*loc_shape) or (batch, *loc_shape) or (batch, *loc_shape, 1)
weight_ceil – a list having weights of ceil points, it has n tensors of shape (*loc_shape) or (batch, *loc_shape) or (batch, *loc_shape, 1)

Returns

one tensor of the same shape as an element in values (*loc_shape) or (batch, *loc_shape) or (batch, *loc_shape, 1)

deepreg.model.layer_util.resample(vol: tensorflow.Tensor, loc: tensorflow.Tensor, interpolation: str = 'linear', zero_boundary: bool = True) → tensorflow.Tensor¶

Sample the volume at given locations.

Input has

volume, vol, of shape = (batch, v_dim 1, …, v_dim n), or (batch, v_dim 1, …, v_dim n, ch), where n is the dimension of volume, ch is the extra dimension as features.

Denote vol_shape = (v_dim 1, …, v_dim n)
location, loc, of shape = (batch, l_dim 1, …, l_dim m, n), where m is the dimension of output.

Denote loc_shape = (l_dim 1, …, l_dim m)

Reference:

neuron’s interpn https://github.com/adalca/neurite/blob/legacy/neuron/utils.py

Difference
1. they dont have batch size
2. they support more dimensions in vol
TODO try not using stack as neuron claims it’s slower

Parameters

vol – shape = (batch, *vol_shape) or (batch, *vol_shape, ch) with the last channel for features
loc – shape = (batch, *loc_shape, n) such that loc[b, l1, …, lm, :] = [v1, …, vn] is of shape (n,), which represents a point in vol, with coordinates (v1, …, vn)
interpolation – linear only, TODO support nearest
zero_boundary – if true, values on or outside boundary will be zeros

Returns

shape = (batch, *loc_shape) or (batch, *loc_shape, ch)

deepreg.model.layer_util.warp_grid(grid: tensorflow.Tensor, theta: tensorflow.Tensor) → tensorflow.Tensor¶

Perform transformation on the grid.

grid_padded[i,j,k,:] = [i j k 1]
grid_warped[b,i,j,k,p] = sum_over_q (grid_padded[i,j,k,q] * theta[b,q,p])

Parameters

grid – shape = (dim1, dim2, dim3, 3), grid[i,j,k,:] = [i j k]
theta – parameters of transformation, shape = (batch, 4, 3)

Returns

shape = (batch, dim1, dim2, dim3, 3)