Reviews

Martin Urschler

Sunday 2 September 2007

Summary:
In this paper a nonlinear image registration algorithm is described which uses a block matching strategy. The algorithm uses an entropy based similarity measure, i.e. it is suitable for inter-modality registration. The authors claim that the scheme is very fast due to the avoidance of an optimization step. Results are presented for registering a synthetically transformed 2D image of a person.

Hypothesis:

The authors claim that a nonlinear registration algorithm based on block matching is a very efficient way for registration. In their opinion many standard algorithms which require optimization of a certain kind are prone to difficult choices of parameter settings, lead to large run-times due to the optimization and are hard to parallelize. Further, they claim that methods based on so-called differential similarity measures (the second group of approaches for nonlinear registration) can be fast and accurate but in practice are complex to implement for inter-modality similarity measures, require a large number of iterations, require derivatives whose computation leads to smoothing and have problems with boundaries.

Starting from these problems and due to the fact that the MPEG standard performs block matching as well the authors propose that block matching is a well-suited nonlinear registration method. They claim that block matching is simple to formulate and implement, is numerically stable, easy to parallelize (i.e. suitable for hardware implementation), is robust against noise and requires only few parameters.

Evidence:

The authors categorization of nonlinear registration into global optimization, differential similarity measures and block matching schemes is in my opinion not very useful. To use the term global optimization in the context of nonlinear registration is dangerous. IMHO there are no reported global optimization techniques, since the parameter space of the nonlinear registration problem is far to large for that. All of the reported algorithms are local optimization based and require good initializations and pyramid schemes to work and to reach a local optimum as registration result. Further, to me it is not quite clear what is meant by methods based on differential similarity measures. If this is the Demons-like approach or the B-spline based approach, then they can also be categorized as optimization-based, since their formulation also seeks for a local optimum over a certain parameterization of the space of unknowns.

The claim that nonlinear registration algorithms are hard to parallelize is contradicted by the implementations of the algorithms in ITK and by another contribution of this workshop from Aylward et al. The speed differences could not be reproduced by the author of this review. Although the performance of the fast block matching scheme is a lot faster than e.g. B-spline, its run-time is similar to Demons on real-world 3D medical volumes. This fact can be seen from our own evaluation of the method in the evaluation paper we submitted to this workshop (Urschler et al). Last, the authors criticize standard algorithms for requiring derivatives leading to smoothing. Without knowing the exact details of the block matching scheme, I think that there are also some implicit assumptions in each block matching that lead to an implicit smoothing of the data.

It would be very useful if this paper would explain a little more of the details of the block matching scheme and if there would be a better explanation of the parameters that were used.

Open Science:

Source Code of their method and the output of their algorithm on the test image are provided for comparison. It would be easily possible to replicate the authors claims if the code would build.

Reproducibility:

I tried to reproduce the authors work directly under Windows using MSVC 8 compiler in the free edition. However, I didn't manage to compile the code since it has a hard-coded requirement to use ITK 2.8.1. I'm using the current release of ITK (3.2) and didn't try to fix this problem. I suggest that this flaw to require exactly ITK2.8.1 should be replaced by requiring AT LEAST ITK 2.8 or higher!

So the direct reproduction of the algorithm's performance on the example 2d image was not possible. However, I took the code and put it into the evaluation framework which was contributed to this same workshop in Urschler et al. There the results for intra-modality registration using several synthetic data sets and a number of comparison measures on real-world 3D medical volumes are presented. The results are not very promising as can be seen from the numbers and the visual outcome. I understand that this contribution is an early work in progress. Note, that I used the same parameters as in the authors source code. It is also perhaps important to note that I am solely providing intra-modality data while the contribution claims to be an inter-modality algorithm due to the entropy based similarity measure. I do not see why it should not work on intra-modality data, however, this might be a possible problem and might not have been tested yet by the authors. I invite the authors to look at my wrapper class around their source code and check if I made any coding mistake. Further, they can reproduce my evaluation and perhaps fine-tune their algorithm to perform better on the 3D intra-modality data.

Use of Open Source Software:
Not Applicable

Open Source Contributions:
The source code is provided and easy to use as it is - if you plug it out of the contributed package (ITK 2.8 restriction).

Code Quality:

The coding style of ITK is supported, the code is working under Windows MSVC 8 and Linux gcc 4.1 (64 bit). The authors claim in the paper that some refactoring is necessary to split the main class into modules.

Applicability to other problems:

A working code could be applied to any nonlinear registration problem.

Suggestions for future work:

For future work a more thorough preparation of the related work might be interesting. Also there was a publication by Bostjan Likar and Franjo Pernus in 2001 in Image and Vision Computing (A hierarchical approach to elastic registration based on mutual information) who have shown a block matching related nonlinear registration scheme which would be interesting to compare to. Some of the claims that lead to the implementation do not sound convincing to me. Further work is necessary in the implementation of the algorithm and in testing it on real-world medical applications.

Requests for additional information from authors:
As I already mentioned a more detailed explanation of the algorithm and the parameters would be useful. Although one could look it up in the mentioned additional papers, it would be good to be a little more self-contained here.

Additional Comments:

Fast block matching is an interesting direction however in this contribution its derivation and its advantages are not yet presented very well. Further, some additional evaluation of our own does not yet show the claimed key contributions. This might be an issue due to the work-in-progress state of the paper, however, the authors have yet to prove that.

Richard Beare

Friday 7 September 2007

Summary:
The authors describe some implementation details of a block matching, mutual information, based scheme for non linear registration. Theoretical details of the scheme are not covered.

Additional Comments:
The previous reviewer has done a thorough job, and I agree with most of the comments. This paper would be much more useful if the theoretical aspects of the method and details of the deformation field were included. Evidence of applicability ot 3D images as well as images with more serious initial non-registration (i.e some form of rotation and translation rather than local distortion) would be more convincing.

Most significantly though the survey of previous related literature isn't thorough. The use of block matching in mpeg was noted, although I'm not convinced it is particularly relevant since the aim in that application is compression. Block matching using MI and other metrics has been presented before - in particular:

S. Ourselin, A. Roche, G. Subsol, X. Pennec, and N. Ayache. Reconstructing a 3D Structure from Serial Histological Sections. Image and Vision Computing, 19(1-2):25--31 , January 2001.

S. Ourselin, C. Sattonnet, A. Roche, and G. Subsol. Automatic alignement of histological sections for 3D reconstruction and analysis. Analytical Cellular Pathology, 18(3):123, 1999 .

There is also a degree of similarity to the non linear registration of mni_autoreg (which isn't MI based)

DL Collins, CJ Holmes, TM Peters and AC Evans, Automatic 3D model-based neuro-anatomical segmentation. Human Brain Mapping, 3(3) p 190-208, 1995

Conclusion: 

I think there are a number of deficiencies in the paper that need to be corrected. However the development of classes that support alternative registration approaches is very welcome. One of the reasons why block based schemes haven't received much attention in the registration field is the lack of publicly available implementations. Hopefully this code will be the start of such a set of classes.

Luis Ibanez

Wednesday 12 September 2007

Summary:
This paper presents an implementation of a block-matching registration algorithm suitable for performing deformable registration.

This contribution offers a great alternative to those involved in deformable-registration, and in particular those who have concerns with the computation time that these registration involved

Hypothesis:
N/A

Evidence:
N/A

Open Science:

The paper is a good example of Open Science practices.

The authors provide all the material needed for reproducing their work:

• source code
• input image data
• tests and parameters
  

The paper clearly describes the principles of the method, has appropriate references to previous publications, and provide a clear the examples illustrating the result of the method.


Reproducibility:

It was easy to reproduce the work of the authors, thanks to the organized material they provided.

After downloading the code, only two changes were necessary (for me):

1. Remove from their CMakeLists.txt file the requirement for ITK 2.8 (the code actually works fine with today's ITK CVS version - Sept 12 2007)
2. Comment out a concept check in itkVectorDivideImageFilter.
   

After these two changes, the code compiled fine.

The authors organized examples in Ctest, making it very easy to replicate their results.

By following the command that they configured in their test, it was straight forward to run the code with variations of the parameters.

Use of Open Source Software:

The authors used the Insight Toolkit version 2.8, but the code still works fine with ITK CVS (Sept 12 2007).

They provide comments about improvements that could be made in existing ITK classes that they were using. Their source code contribution includes the modified version of these classes.


Open Source Contributions:

The authors provided their source code.

The code was usable (after very minor modifications).

It took me 15 minutes to reproduce the work that the authors describe in the paper.


Code Quality:

The code is well organized. They provide the a class that computes the registration. These class follow the ITK Coding style for the most part. Some editing will be needed to make it pass the KWStyle checking.

The use of NaN as part of the algorithm is a bit worrisome, since it is hard to anticipate how this will work on different platforms.

Applicability to other problems:

The authors work can be applied to a wide range of problems.

Indeed their contribution should probably be split in to two or three different objects. In particular code that manages block-matching could be separated from the code that computes the Mutual Information metric. The authors are aware of this possibility and address it as an item for future work.

Suggestions for future work:

Splitting the class into a block-matching manager and a selectable image metric will extend the range of problems where this code could be used.

Performing a comparison with other methods would have been interesting, but just form the anecdotal point of view. Since a fair comparison will require to demonstrate that the corrections performed by multiple algorithms are equivalent, and we know that the variety of parameter values that can be when executing a deformable registration method make difficult to establish what combination of parameters in method A should be equivalent to what other combination of parameters in method B.

Requests for additional information from authors:

It would have been interesting to run the algorithms for a variety of combinations of parameters.

Once you have the ADD_TESTS lines in your CMakeLists.txt file, exploring variations of the parameters is relatively easy.

For example, to play with the values of regularization and to show how they affect the final result.

It would have been interesting too to write out the deformation field. (to a metaimage for example), so it can be viewed with applications such as ParaView.

Additional Comments:

The file itkVectorDivideImageFilter.h line 179 has a concept check for "OutputVectorDivisionOperatorCheck". The terms of this concept check don't seem to be defined. The lines of this concept check were commented out in order to be able to compile the code.

It is likely that the authors were developing and testing thei code using an ITK build where the concept checking was turned off. (CMake option).

----

Minor comment : Your files have TABS in some lines. You may want to set your editor to use 2 spaces instead of tabs.