An open, clinically-validated database of 3D+t cine-MR images of the left ventricle with associated manual and automated segmentation

Please use this identifier to cite or link to this publication: http://hdl.handle.net/1926/550
In this paper, we describe a database of cine-MR (3D+t) images of the
left ventricle. This database contains the voxel data, one automated
and two manual segmentations for each sequence of images. The
segmentations are validated from a clinical point of view. We detail
how the images were obtained, as well as how the associated
segmentations were performed. We also provide the data clinical
validation process.

This database, including tools to compute quantitative measures and
the software package used to obtain the automated segmentation, is
freely available for research purposes.
Data
minus 1 File (74Kb)
Code
There is no code review at this time.

Reviews
minus MICCAI workshop review by Kevin Cleary on 09-11-2007 for revision #1
starstarstarstarstar expertise: 2 sensitivity: 4.8
yellow

Summary:

Assessment of cardiac cine images is a challenging task.  A standard reference database of images will improve evaluation of methods for these types of images.  The authors provide a valuable open database both manual/automatic segmentation references.

 

Hypothesis:

n/a

 

Evidence:

The paper is less about evidence to prove a hypothesis than description of a database and segmentation methods.  The choice of methods used for comparing manual and automatic segmentation seems appropriate, although other methods could have been used as well. 

 

Open Science:

In general the methods and approach are easy to follow.  However, more detail could be provided on the following

a)  How many slices can be acquired per heartbeat?  Is slice mismatch a major problem (they refer to manually adjusting misaligned slices) or only present in very few cases?

b) How exactly is the oversampling achieved?  Is the final resolution of the volume after oversampling 256x160x?? slices?

c) If manual segmentation is not available for every time step, how many time steps is it available for?

 

The section on comparing EF and MM calculations is very confusing due to the use of the abbreviation “resp.” which I take to mean “respectively”.  This should be rewritten, and perhaps a table included to make it more clear.

 

Reproducibility:

I did not reproduce the work.

 

Use of Open Source Software:

Only Analyze is mentioned in the paper, which is not open source.

 

Open Source Contributions:

n/a

 

Code Quality:

n/a

 

Applicability to other problems:

The creation of standard databases such as this is very applicable to many problems in medical imaging, and the authors are to be commended for this effort.  It seems likely that this database of images will be useful to other investigators, even more so if it can be added to in the future.

 

Requests for additional information from authors:

In the conclusion:  I do not understand how 216 LV slices convert into 4752 “borders” of endocardium and epicardium.  What exactly is a border in this context, and why are there so many of them?  Does it really produce statistical or population diversity?

 

This work is significant by itself and does not need to be artificially inflated through the use of such obtuse terms.

 

Additional Comments:

This review was written by Kenneth Wong in our lab and submitted by Kevin Cleary.

minus important contribution for standardizing segmentation evaluation by Martin Urschler on 09-02-2007 for revision #1
starstarstarstarstar expertise: 3 sensitivity: 4.8
yellow

Summary:
This paper presents an open-science segmentation evaluation database accompanied by evaluation criteria and two manual and an automatic reference segmentations for comparison. The segmentation is dedicated to cardiac MR images showing the left ventricle in 4D (3D+time).

Hypothesis:
NA

Evidence:
NA

Open Science:
The paper is an excellent example for the power of open-science. Unfortunately the link to download all contributed material does not work at the time of this writing, so this review will solely be based on looking at the paper (which I took from the MIDAS & Kitware collection using the papers DSpace handle) and on the presented material on the accompanying webpage.

Reproducibility:
I did not reproduce the mentioned experiments or download any code.

Use of Open Source Software:
NA

Open Source Contributions:
As already mentioned I was not able to use any provided code.

Code Quality:
NA

Applicability to other problems:
The basic idea of providing evaluation databases and criteria along with some results from other methods for comparison is in the review authors opinion (and I dare to say also in the opinion of many more people in the medical imaging community) an extremely important one. Successes from other disciplines like computer vision (stereo reconstruction, multi-view 3D reconstruction -> Middlebury) or pattern recognition (face recognition databases) prove this importance despite the additional legal, ethical and of course commercial problems such an effort has in the medical community.

Suggestions for future work:
The only suggestion I currently can think of is to contribute this web-site to the list of open-science databases proposed in the workshop paper of Holms et al (Data, data everywhere).



Additional Comments:

Overall this paper is an important open-science contribution and IMHO should definitely be accepted for the workshop to further discuss its ideas. It is very well and clearly written, shows a good over-view of the related work, although there are some segmentation evaluation efforts like e.g. the VALMET project (Gerig et al @ MICCAI 2001) or Fenster and Chu (Evaluation of Segmentation Algorithms for Medical Imaging, IEEE-EMBS 2005) which could also be mentioned. One can of course think about the advantages and drawbacks of a watershed type segmentation to get the automatic results, but this is not the main point of this contribution. Overall very promising and I am looking forward to discussions at the workshop.

minus Open Database of Heart MRI datasets + segmented results by Alexandre Gouaillard on 07-31-2007 for revision #1
starstarstarstarstar expertise: 3 sensitivity: 4.8
yellow

Summary:
This paper introduce a database of MRI scans of the heart along with segmentations made both by experts and by the authors 4D watershed algorithm (described in a different paper).

Hypothesis:
Non Applicable

Evidence:
Non Applicable

Open Science:
Tha original datasets, along with the segmented results and source code are available from the authors' webpage mentioned in the paper.

Reproducibility:
Non Applicable

Use of Open Source Software:
I did not check the code as it is not the purpose of this paper.

Open Source Contributions:
See above.

Code Quality:
Non Applicable

Applicability to other problems:
I guess that availability of the data along with the segmented results is a good step toward Open Source, allowing others to try their algorithms against the same dataset and compare their results directly.

Suggestions for future work:
Authors could suggest users to upload their results and make a database of algorithms and their results on that kind of datasets. That kind of database becomes more and more interesting as the number of comparable results grows.

Requests for additional information from authors:
Did you try anything else than linear interpolation to extend your neighborhood in time?
In your illustration of the results, the shapes seems pretty smooth. Smoothing the surfaces is known to have side effect (shrinking, modifying the spatial position of some points resulting in anatomical inconcsistency when several disconnected shapes are represented). Did you have such problem? If yes, how did you deal with them?

Additional Comments:
This is off topic here but I feel like the algorithm paper lacks of illustrations.
Having data/src code/additional papers in a different location make the review difficult. I would suggest that the authors upload more material on IJ, or make a big tarball accessible from their webpage, instead of 4 to 5 different files.

minus A good contribution for the research community by Miguel Angel Rodriguez-Florido on 07-04-2007 for revision #1
starstarstarstarstar expertise: 3 sensitivity: 4.3
yellow

Summary:
This work describe a database of cine-MR images (3D+t) of the left ventricular of the heart. The paper presents the acquisition, the segmentation methodologies used, and the validation of the results, comparing automatic software-based segmentation with manual. The database is public an available on the web.
Hypothesis:
NA

Evidence:
This is a very empirical work that does not suppose any opinion.

Open Science:
This work is a great contribution to Open Science. It provides a clinical valued database very useful for other researchers. They describe their methods and provide the software-based tools that they have used. The results can be replicated.

Reproducibility:
Although I haven't reproduced the authors work, they explain in their paper how to do it. The database is public and their results can be used for similar research lines.

Use of Open Source Software:
For manual segmentation they use Analyze Software, but they don't say anything in the text about the kind of license for the software tools/scripts that they provide in the web site. I haven't downloaded this software, and I don't know if the code has a license warning or file.

Open Source Contributions:
NA

Code Quality:
NA

Applicability to other problems:
This open clinically-validated database is useful for all researchers that work with various cardiomyopathies.

Suggestions for future work:
NA

Requests for additional information from authors:
No comments on this item

Additional Comments:
Nice work and nice proposal: "such work is only possible if cardiologists and computer scientists are working together in close partnership".

 

Add a new review
Quick Comments


Resources
backyellow
Download All

Statistics more
backyellow
Global rating: starstarstarstarstar
Review rating: starstarstarstarstar [review]
Code rating:
Paper Quality: plus minus

Information more
backyellow
Categories: Feature extraction, Programming, Registration, Segmentation, Watersheds
Keywords: Cardiac magnetic resonance, left ventricular function, left ventricular mass, myocardial infarction, image processing
Toolkits: VTK
Export citation:

Share
backyellow
Share

Linked Publications more
backyellow
N-Dimensional Phase Unwrapping N-Dimensional Phase Unwrapping
by Vigneault D., Wang W., Tee M., Bluemke D., Noble A.
Anisotropic Diffusion in ITK Anisotropic Diffusion in ITK
by Mirebeau J., Fehrenbach J., Risser L., Tobji S.

View license
Loading license...

Send a message to the author
main_flat
Powered by Midas