Summary:
This entry deals with groupwise registration, that is a registration of multiple images at once without explicit reference frame.

Hypothesis:
Groupwise registration can be efficiently extended by B-spline deformable transform.

Evidence:
The authors provide output of the algorithm, that is test images before and after registration along with mean and standard deviation images. Sharper mean images and zero standard deviation images are interpreted as successful alignment.

Open Science:
Source code as well as input images are provided in the entry. Images used in the paper are not included, although one can expect these are selected from the input images as a result of the included tests. All the details needed to understand the algorithm are contained in the paper.

Reproducibility:
Downloaded, compile and run the tests was a matter of minutes. I did not verify if the results are the same as the tests with 3D volumes are more time expensive than I presumed.

Use of Open Source Software:
Only the Insight toolkit is used.

Open Source Contributions:
Provided source code is clearly described in the appendix to the article and accompanied by numerous tests.

Code Quality:
Source code is properly commented, cleanly divided into separate classes. It is possible to use it without any problem with current ITK, as far as I can tell the code does not involve any non-portable mechanism.

Applicability to other problems:
There are many problems that can benefit from a deformable groupwise registration, specifically where there is a need to bring a set of images into alignment without choosing a pivot/reference frame coordinate system.

Suggestions for future work:
Test the algorithm not only with volume data with synthetic deformations but also test alignment of 2D image sets and non-synthetic deformations.

Requests for additional information from authors:
Script that produces the output images used in the paper should be added.

Additional Comments:
Although B-splines transform used is have been optimized still the speed is not quick,
it takes over a minute to align two exactly identical 480x480 images(?).

I encountered a bug worth patching, GroupwiseRegistration executable causes memory
corruption when registration->GetLastTransformParameters() is called after resetting the
length of parameters array registration->SetTransformParametersLength(int), this call
should be moved to line 1123, after using the old transform parameters.

Overall Review:
The algorithm implemented through this paper is of great utility to the medical imaging community. It is implemented in ITK, which means that it will be accessible to a very large user base. A great amount of effort was placed to provide an efficient implementation of the algorithm along with two different transformation models. Their experimental results, while limited, provide sufficient evidence for the correctness of their implementation.

Summary:
This paper experiments with a new transformation model based on B-splines for an existing joint alignment algorithm. Using the Jacobian of the deformation field within a stochastic optimization framework (in a multi-resolution setting) an efficient implementation of this algorithm is derived. The authors provide an open source implementation of the algorithm and their transformations using ITK. They provide basic experimental results on synthetic data to validate the correctness of their implementation and the similarity in results between affine registration and B-spline registration.

Hypothesis:
B-splines have not yet been tested as a deformation model within existing Groupwise Registration algorithms. The implicit hypothesis in this work is that the use of B-splines is advantageous, at least for the type of data tested here.

However, the major contribution of this work is providing an efficient open source implementation of the Groupwise Registration algorithm that can be used with either affine transformations or B-spline based transformations.

Evidence:
The authors did not explicitly state that B-spline registration is better or worse than affine registration. No direct, quantitative analysis was performed to show which type of registration worked better. Visually, the results are very similar and it is not clear if B-spline registration provides any advantages over affine registration. Also, the experimental results are somewhat weak since they are based on synthetic data. Especially since the synthetic volumes are created by randomly transforming the same original volume.

However, their results do confirm that their implementation of the algorithm using either transformation model is correct. In all 4 experiments, the algorithm produced significantly sharper mean images and an almost black STD image.

I disagree that a black STD image is sufficient evidence for a perfect registration (as stated in page 7). One needs to also compare the mean image to the original image used before any random transformations were applied. That way we can ensure that the algorithm is not creating data. The extreme case is if the algorithm changed all the images to purely white, then the STD image will be black but the mean image will be far from reality.

Open Science:         
This paper adheres strongly to the concept of open science. The authors provide an open source implementation of the algorithm and do a good job at describing it. They also provide all experiment parameters and the data used for their experiments. The experiments can easily be replicated by following the steps in Section 5 of the paper.

Reproducibility:
I downloaded and viewed the provided code but did not attempt to compile it. By viewing the code and my basic understanding of ITK it seems as though one can reproduce the results by simply installing the necessary packages and running a provided script (as outlined in Section 5).
 
Use of Open Source Software:
The authors used ITK to implement their algorithm. They did not provide any specific information regarding ITK with respect to its advantages, disadvantages or hints for users. However, ITK is a widely used open source package for registration, I am not sure that such information was necessary. Of course any important information specific to the classes they implemented might have been worth mentioning.
 
Open Source Contributions:
The authors provide their code along with a helpful description on how to use it. Since it was implemented using ITK it is in usable form. I have not attempted to use it.
 
Code Quality:
Their code is easy to read and since it was implemented with ITK, they followed all the guidelines and good code design enforced by ITK.
 
Applicability to other problems:
Groupwise image registration is a very important problem in medical imaging. A large number of researchers will find this implementation of great value.

Suggestions for future work:

1.     More extensive experiments using real data
2.     A study to compare B-spline registration to affine registration
3.     Provide run time information for the experiments
 
Requests for additional information from authors:
None.
 
Additional Comments:
The authors did not give a reason for extending previous work to include B-spline registration. It is not clear from their results if B-spline registration is better or even faster than affine registration. Filling in these gaps would make the paper much more complete.