Reviews

David Holmes

Tuesday 5 September 2006

Summary: The authors provide an “update” to their current efforts in developing the BioImage Suite. The BioImage Suite fully utilizes several readily available software packages to provide a “user-friendly user interface” for medical image analysis. The paper describes the various algorithms in the package as well as some of the biomedical applications have been developed with the suite. The discussion includes a description of licensing and availability.

Hypothesis: N/A

Evidence: As the paper title suggests, this is suppose to be an “update” to the previous paper. The evidence should be in the “updating” of the textual description of the package. While I think that the suite is both interesting and potentially useful, I found the paper to be lacking a bit. I found much the the text to be the same (almost word for word) with the previous submissions which the authors are updating.

Open Science: The package will be open-source (and available under the GPL) soon. The package can be downloaded now following registration. The authors note that there is one library which is closed. The issue becomes whether this hinder the open-science movement to build upon their success. As the authors point out, the single library is a small component and their hands are tied due to other licensing. However, even with the restrictions (by third-parties), the authors still have the right to publish the methods included within the code. (See next section)

Reproducibility: Since the package will be largely open source, reproducibility should be straightforward (although that is only a best guess as I have not reviewed the source). With regards to the single closed library, the authors will not be able to provide source code, but they should be publishing (in great detail) the methods within the library as that is not restricted. As such, anyone who has a valid licence of numerical recipes (which many do) can recreate the functionality. In the same manner, anyone with the appropriate Vector Vision Link licence should be able to write the appropriate connection code.

Use of Open Source Software: The authors take full advantage of open source software from the programming languages used (tcl/tk) to the graphics drivers (mesa), when needed. The exciting aspect of this work is the combination of all of these tools into a useful package. As already noted, there are some minor restrictions with the Vector Vision Link software and the Numerical Recipes software.

Open Source Contributions: I do not have the source code. It was not included with submission. The website has a mechanism to get the software.

Code Quality: Ditto

Applicability to other problems: This suite has great application to many different clinical problems. I look forward to exploring its potential.

Suggestions for future work: If not available, I suggest that the authors post or publish the methods within the closed library so others can properly reproduce the work.

Requests for additional information from authors:

Additional Comments:

Gavin Baker

Thursday 31 August 2006

Summary:

This paper describes BioImage Suite, an software package for medical image analysis. It is has been used in a wide variety of applications, including fMRI, DTI, and segmentation. The aim is to provide easy-to-use tools, with a strong clinical application focus.

The BioImage Suite is in a similar solution space to Slicer, Brainsuite and Analyse. It comprises a GUI and a series of command-line tools for batch processing, and has been developed internally at Yale over the past 10 years, being used on a variety of projects.

This paper is an updated version of a previous submission which described the BioImage Suite, and the authors have fulfilled their promise to publish the source of the package. The content of the paper itself appears to be virtually the same as to last years\’ submission, with some changes to reflect the beta release of the source code.

Hypothesis:

That the BioImage Suite can support clinical image analysis applications, and that it is user-friendly and well-documented.

Evidence:

The project website includes quite a comprehensive set of online documentation, which describes how to use the various components of the system in some detail, including screenshots and tutorial-style information. The system appears to be reasonably user-friendly, based on studying the available screenshots.

While the system has obviously seen considerable use and wide application over the years, the paper did not discuss clinical applications or present a case-study.

Open Science:

The authors provide access to the full source code (and nearly all the dependent libraries). The paper shows examples and screenshots of the application running. The binary distribution also includes some example image data.

Reproducibility:

I followed the instrucions on the project website to obtain the software. This involved a quick registration process, and was then able to download the software. I obtained the latest source (bioimagesuite_1.995_src_507.zip dated 24th August 2006) as well as the pre-built binaries for Linux.

I was unable to completely build the software from source on my machine (Debian GNU/Linux Etch, i586). While I do have LAPACK, BLAS, ATLAS and all such packages installed (including clapack.h), I do not have CLAPACKConfig.cmake and could find no reference via Google. I managed to hack past this problem, only to encounter other difficulties which prevented further progress.

I untarred them into a private directory, adjusted the path settings according to the documentation. (The install documentation says that untarring will result in three directories, but it omits mention of the bioimage_extra directory.) The system assumes that the software is installed under /usr/local, which was not an option in this instance. While the scripts to run BioImage have a BASE directory, it is not used by the scripts, and changing the paths is most cumbersome.

Some issues: the bioimagesuite/setpaths.sh script refers to FSL, which is not included. The setextra.sh script refers to minc20, which is under the extra directory but the script assumes it is under the base directory. The configuration tries to set TCL_LIB_PATH, but the correct environment variable (according to the documentation I consulted) specifies TCLLIBPATH, ie. without the underscores). Also, since this package is distributing its own copy of Tcl, the TCL_LIBRARY environment variable needs to be set to the base path of init.tcl.

With my custom script (see below) I was able to successfully run the suite. I loaded some test data (mni_305.hdr), and carried out some common tasks: preprocessing, bias-field correction, smoothing, histogram segmentation, morphology and others. I was able to perform these common operations without needing to refer to the documentation, supporting the ease of use claim.

Use of Open Source Software:

The BioImage Suite makes extensive use of existing FLOSS projects, including ITK, VTK, LAPACK, Mesa 3D, Tcl/Tk, IWidget, and Metakit. It leverages these libraries to provide a high-level interface. (It also appears to include the HDF5 package, which doesn\'t seem to be mentioned in the documentation.) The paper does not discuss in any detail how these libraries were used.

Open Source Contributions:

The complete source code is made available on the BioImage website, which is available after a brief registration step. The source comprises some 300,000 lines of source files, representing a very significant contribution to the community.

This website also features user forums, encompassing support, discussion on fMRI, DTI, integrating with other software, news and discussion. several relevant papers and resources are provided, including tutorials and user manual.

As the project is making the transition from an internal project to its first public release, various environmental issues have become apparent, as noted above. This is due to the inevitable dependencies and assumptions that one is able to make in a controlled or homogeneous environment, such as the presence of dependent libraries or tools, and the locations and paths of components, and so on. As the project progresses in its new public phase, these issues will surely be ironed out and the project should become more easily deployable on external systems.

Code Quality:

The code is broken up into sensible modules (such as GUI, Imaging, and so on). The layout of the source code is generally readable and consistent, but often uses extremely long lines (>80 characters). Most classes have small, cohesive methods. The class interfaces seem to be generally well-defined. There is still a certain amount of debug code, such as cout and printf calls sprinkled about (while others use the VTK debug macros). There are quite a few instances of literal constants and magic numbers, such as IDs for menu items which must be the same between modules, or processing modes, colour constants, lookup tables, etc. The coding conventions are fairly similar to the ITK conventions (making the transition of contributions easier to manage). (A few dozen source files were chosen semi-randomly across the various directories in the source code for inspection.)

There is very little documentation within the code. Using a tool such as Doxygen and providing inline documentation of classes and methods would improve the maintainability of the code, and enable external developers to get up to speed and contribute and enhance the code base.

The code is generally designed for portability, as evidenced by the fact that it runs on Windows, Linux, Solaris and Mac OS X. This is obviously in no small part thanks to the portability of the underlying libraries, however the authors have gone to great lengths to not only support the major platforms in their own code, but to provide pre-built binaries for each also.

Applicability to other problems:

The suite includes implementations of several useful algorithms, which would be very useful in their own right. Some of these may be appropriate to make as contributions to the ITK or VTK core. While the suite contains tools for specific applications such as angiography, it provides general segmentation and registration tools, which may be equally used in other areas such as thoracic imaging or brain sementation.

Suggestions for future work:

Building

Making the codebase easier to build on different systems is the biggest hurdle to adoption by external developers, and to a lesser extent users. The paths configuration could be streamlined, to make it easier to install in a custom root (for example, /opt/bioimage).

Distribution

The binary distributions are broken up into four differnent tarballs for download. Since they are all required, and are built to be run together, it would be helpful to have a single tarball to download all the required binaries.

Documentation

The system currently has minimal developer documentation. Information such as the build and runtime dependencies, minimum versions of packages required (eg. CLAPACK/LAPACK), CMake, and so on would be very useful. Since the system provides much code that could be re-used elsewhere, providing an API reference (much like ITK does with the doxygen-generated docs) would be extremely valuable.

Startup Scripts

The scripts included assume the system is installed in /usr/local, and are difficult and tedious to update.

Due to the difficulty of changing all the paths in all the startup scripts, I wrote a single script that sets all the paths at once, which are all relative to a single base directory. This avoids the problems of requiring BioImage Suite be installed in /usr/local, and makes it easier to configure and customize, since it is all relative to one single base directory variable. The script is below:

#!/bin/sh

BASE=`pwd`
ITK_DIR=$BASE/itk241_yale
VTK_DIR=$BASE/vtk44_yale
EXTRA_DIR=$BASE/bioimagesuite_extra
BIS_DIR=$BASE/bioimagesuite

MAIN_DIR=${BIS_DIR}/main
BIN_DIR=${BIS_DIR}/bin
APPS_DIR=${BIS_DIR}/apps
MJACK_DIR=${BIS_DIR}/mjack
VTKNR_DIR=${BIS_DIR}/vtknr
DATATREE_DIR=${BIS_DIR}/datatree

PATH=$BIS_DIR/bin:$EXTRA_DIR/bin:$VTK_DIR/bin:$BIN_DIR:$MAIN_DIR:$APPS_DIR:$MJACK_DIR:$VTKNR_DIR:$DATATREE_DIR:$PATH
LD_LIBRARY_PATH=$ITK_DIR/lib/InsightToolkit:$VTK_DIR/lib:$VTK_DIR/lib/vtk:$EXTRA_DIR/lib:$LD_LIBRARY_PATH

export PATH
export LD_LIBRARY_PATH

TCL_LIBRARY=$VTK_DIR/lib/tcl8.4
TK_LIBRARY=$VTK_DIR/lib/tk8.4
TCLLIBPATH=$VTK_DIR/lib/tcllib1.8

export TCL_LIBRARY
export TK_LIBRARY
export TCLLIBPATH

pxmenu.tcl

Requests for additional information from authors:

It would be very interesting to have not just an overview of the system, but a discussion of the design of the package. How do the components fit together? How did they design the architecture, with its blend of C++ and Tcl code? How can some of the core algorithms be reused in other projects?

Cross-platform development is very important, and the developers seem to have an interesting approach involving VMWare Server. A discussion of their approach, for building, testing, deployment, etc would also be of interest to many developers.

Given the fact that the BioImage Suite has apparently been in widespread use within Yale for many years, it would have been most useful to present some case studies to show how it is being used, what benefits it confers to the users, and how it is employed in clinical diagnosis and interventions. The core of the paper is a general overview of the features of the system, and doesn\'t provide much detail on how the code was implemented.

Of particular interest to many members of the community would be a discussion of how the system interfaces to the Brainlab IGS system (a suggestion from a reviewer in last year\'s submission). Of course, the publication of this module\'s source would be very useful.

Additional Comments:

The suite is built upon a large number of other libraries, most of which appear to be published under a BSD-style license. The authors are publishing this suite under the GPL, which has implications on the other libraries whose source is distributed alongside. It is not a foregone conclusion that all these licenses are GPL-compatible, and some legal advice may be appropriate to ensure no terms are being bent. It is somewhat of a concern that the authors state in the paper that a core library will remain proprietary, as this is obviously not compatible with the GPL (depending on the linkage between the module and the rest of the system).

The Numerical Recipes code is obviously proprietary and will remain so, free/open alternatives to a significant number of these algorithms are available elsewhere (eg. GSL, etc) which may overcome this particular limitation, so it should be possible to replace this with code and free the affected module.

Since the authors use VMWare extensively for their multi-platform development, perhaps they would consider providing a Virtual Server Appliance with BioImage Suite ready to run. This would enable people to try the software after a single download, and would avoid potential compatability issues with DLLs, configuring paths, and so on.

Conclusion

In summary, the project includes full source code, documentation and sample data. The suite clearly addresses the need to have productive tools and interfaces for working with medical image data. The paper provides a good overview of the suite, but leaves the reader wanting to know more detail about how it was designed and how it is being used in practice. This suite is of general and broad interest, for researchers to analyse and explore image data. This package should prove very popular and useful for a wide variety of users, and anyone interested should download it to evaluate it for their application.

Peter Kazanzides

Wednesday 6 September 2006

Summary: This paper describes the BioImage Suite, which is a collection of image analysis tools, including GUI-based and command-line programs, developed at Yale University.

Hypothesis: N/A

Evidence: N/A

Open Science: The authors are releasing the bulk of the source code under the GPL license. One library cannot be released as open source due to licensing issues. A pre-release version (binary and source) is available from the web site, though it currently requires a user name and password (easy to obtain after registering).

Reproducibility: I downloaded the Windows binary and installed it on Windows XP. I used the suggested root directory (C:\Yale). The installation consisted of unzipping 4 ZIP files (not 3 as mentioned in the documentation). The program is starting by running a batch file. This was very easy to do and worked. I also downloaded the source code, but did not try to compile it myself.

Use of Open Source Software: The paper mentions that BioImage Suite uses VTK, ITK, and TCL/TK (with extensions such as IWidgets). In addition, by reviewing the downloaded files, it is clear that it also uses other open source packages such as Xerces, MINC, and CLAPACK.

The project uses the CMake and Subversion open source tools.

Open Source Contributions: This is a significant amount of software that is being made available to the research community. The authors are working to replace some of the closed source libraries. The authors provide binary versions for many different operating systems.

Code Quality: I downloaded, but did not review, the source code. The design of the web site is very good and appears to have ample documentation.

Applicability to other problems: There are a lot of image analysis tools, which should be applicable to a wide range of medical applications.

Suggestions for future work: The authors state that they make extensive use of VMWare for testing the software on different operating systems, but do not describe the testing framework. For a software package of this size, I would strongly urge the developers to create an automated testing framework (e.g., using Dart), if they have not already done so.

Requests for additional information from authors: The authors mention similar open source packages, such as Slicer, Brainsuite, and SPM, but do not offer a comparison to BioImage Suite. Given that they recently received an NIH grant for “continued development and maintenance”, I assume that they must have already described (at least in the proposal) the new features that their package could offer.

Additional Comments: I find the GUI to be less “sexy” than the 3D Slicer package (though I am more familiar with 3D Slicer). The availability of command line tools is a nice feature in BioImage Suite and the generation of Makefiles to manage the batch processing is a good idea.

The VVLink capability is interesting (I also read the ISBI06 paper available on the website).

Eigil Samset

Wednesday 23 August 2006

Summary: A medical image processing suite is described. The system packages existing algorithm in an easy-to-use application. The software is in wide use at Yale and local collaborating laboratories.

Hypothesis: Non applicable

Evidence: The authors give a clear description of the software, and examplifies the capability of the software in clinical applications with several screenshots.

Open Science: The BioImage Suite is publically available for download. Usename and password is given on request. The source-code was not readily available (it would requre a separate request). The authors give a good description of core algorithms that are being used in the software and give referenes to these.

Reproducibility: Downloading and running the precompiled version of the software went smoothly. When not installing to the default location, the file start_bioimagesuite had to be modified (in addition to files mentioned in the webpage) No attempt was made to get source code or compile it.

Use of Open Source Software: The authors use existing opensource software such as VTK, ITK and CLapack.

Open Source Contributions: The code was not easily available, and was not reviewed. The authors claim that the source code will be made available under GPL (except for a part of the software). The authors do not discuss the choice of licence.

Code Quality: Non applicable

Applicability to other problems: The software system that is presented can have a wide use in image processing problems. It seems that it can provide a good platform to package image processing algorithms to end users. The authors empasize that the system is easy to learn and use, which is often not the case for academic software. There is no discussion of how BioImage Suite compares to other systems like Slicer, Brainsuite, SPM or Analyze.

Suggestions for future work: For this reviewer the software related to VectorVision link would have the highest interest. This cannot currently be opensources. The suggested work to enable this would therefor be very usefull for the Image-Guided-Therapy community.

Requests for additional information from authors: It would be interesting to know more about the data model used in the system.

]