Reviews

Xenophon Papademetris

Tuesday 5 September 2006

Summary: The paper presents a robot assisted needle placement system based on IGSTK. The platform has been tested and encouraging results are presented.

Open Science: The authors provided source code and a data set, unfortunately one also needs a robot and the intra-operative scanner to replicate the findings both of which are unfortunately not so easily downloadable or available.

Reproducibility: I downloaded and looked at the code which is of good quality and looks very similar to code in ITK. I did not attempt to compile it as it requires VTK 5 and ITK 2.6 which I did not have installed on my system, however the Insight Journal dashboard suggests that it compiled just fine.

Use of Open Source Software: The authors use the usual combination of ITK/VTK and the IGSTK toolkit (as implied by the title). The comment about IGSTK being a safe platform for researchers and clinicians to use needs to be qualified, especially since any additional code (extensions, user applications) is perfectly capable of crashing even the most stable of systems.

Open Source Contributions: The source code is made available (together with IGSTK and FLTK), see reproducibility above.

Code Quality: See reproducibility above.

Applicability to other problems: The methodology developed for the application is fairly crude in terms of what constitutes state of the art image analysis. Having said that, the results suggest that it is suitable for the application and perhaps use of more sophisticated methods might be overkill.

The use of a CT scanner for registration is uncommon though, intra-operative scanners are not as easily available and use of CT scanner in a patient setup is kept to a minimum for radiation purposes. This makes the method less applicable to other problems. Point fiducials are time consuming but often easier to implement in most common situations.

Suggestions for future work:

1. Figure 4: The word ERROR appears in the textbox below the figure. This is a dangerous think to publish for anything related to intervention! While in this case, this is not a serious issue, it should be qualified in the caption. I assume this is the output from the state machine.
2. The validation study is not sufficiently well described. What was the experimental setup? Did the authors test the amount of time for which communication with the robot remained stable under stress (also leaving the connection on for a few hours without actually invoking it). What is the affect of accidental disconnection of the robot on the software (cables do get loose in real situations). In general testing of this type needs to assume that the worst will happen as it often unfortunately does in the case of real interventions.
3. I assume that the authors mean \\\\\\\"more sophisticated\\\\\\\" as opposed to \\\\\\\"more relaxed\\\\\\\" registration methods. Iterative closest point methods might be appropriate, the authors might also consider robust methods which are able to do outlier rejection and hence handle any overdetection (the Robust Point Matching framework was pioneered by Anand Rangarajan and his collaborators comes to mind here, this code is available in various forms).

Requests for additional information from authors: The validation study needs to be better described. What CT scanner was used what was the environment in which the robot was placed etc. Reproducibility is also critical, how do the results vary if the robot is imaged twice in the same position, or if the robot is moved back to the same position and reimaged. This is really the weak point of this paper.

David Holmes

Tuesday 5 September 2006

Summary: The authors describe an application of the IGSTK software for robot-guided needle placement. While the paper is loosely about the entire robot-guided application, the method focuses on the registration component of the procedure largely because the is the most critical aspect of the software. The background briefly describes both the hardware and the IGSTK software. The method and results are in regards to the fiducial registration method.

Hypothesis: Although not a hypothesis, the premise of the paper is that robot-guided needle placement is an important clinical application and requires effective software. Specifically, the software must be effective at registering the tool to the patient. Once achieved, visualization is important.

Evidence: The evidence is through a small validation study on a phantom. The details of the validation study are lacking a bit, but the results suggest that the fiducial registration method is effective.

Open Science: This work is completely open science and includes source code which can be used in conjunction with the paper to evaluate and test the method.

Reproducibility: The source code allows for an evaluation of the method and reproducing the study. The authors also include data in their submission which is great. That being said, there is not much information on the validation study (including the type of phantom) and how the ground truth was determined. Nevertheless, the method is reproducible and would allow another investigator to conduct an independent evaluation.

Use of Open Source Software: Everything is open source

Open Source Contributions: Code is available; however, I did not have the opportunity to review it thoroughly or run it. It does seems sparse on documentation in the code.

Code Quality: Same as above.

Applicability to other problems: One would hope and expect that the application could be applied to other IGS applications. This provides a nice example of how to use IGSTK in a clinical application.

Suggestions for future work: The authors appropriately suggest future work for themselves. I look forward to following this project into the future. Get going on the clinical trials to show the work that open-source can and should be used to treat patients.

Requests for additional information from authors: The paper would benefit from including more references appropriate to the registration method. There are several fiducial based registration papers in the literature.

The validation study could be more detailed as well.

Additional Comments: [This is a free-form field]