A quantitative measure for retinal blood vessel segmentation evaluation

Analysis of retinal blood vessels allows us to identify individuals with the onset of cardiovascular diseases, diabetes and hypertension. Unfortunately, this analysis requires a specialist to identify specific retinal features which is not always possible. Automation of this process will allow the analysis to be performed in regions where specialists are non-existent and also large scale analysis. Many algorithms have been designed to extract the retinal features from fundus images. However, to date, these algorithms have been evaluated using generic image similarity measures without any justification of the reliability of these measures. In this article, we study the applicability of different measures for retinal vessel segmentation evaluation task. In addition, we propose an evaluation measure, F1, which is based on precision, recall and F-measure concept to deal with this evaluation task. An important property of F1 is its tolerance of small localization errors which often appear in a segmented image, but do not affect the desired retinal features. The performances of different measures are tested on both real and synthetic datasets which take into account the important properties of retinal blood vessels. The results show that F1 provides the greatest correlation to the desired evaluation measure in all experiments. Thus, it is the most suitable measure for retinal segmentation evaluation task.