Color-texture image segmentation based on multistep region growing

Color-texture image segmentation based on multistep region growing Irene Fondón Carmen Serrano Begoña Acha University of Seville 共Spain兲 Escuela Superior de Ingenieros Departamento de Teoría de la Señal y Comunicaciones Camino de los Descubrimientos s/n 41092 Sevilla, Spain E-mail: [email protected] Abstract. A new method for color image segmentation is proposed. It is based on a novel region-growing technique with a growth tolerance parameter that changes with step size, which depends on the variance of the actual grown region. Contrast is introduced to determine which value of the tolerance parameter is taken, choosing the one that provides the region with the highest contrast in relation to the background. Color and texture information are extracted from the image by means of a novel idea: the construction of a color distance image and a texture energy image. The color distance image is formed by calculating CIEDE2000 distance in the L*a*b*color space. The texture energy image is extracted from some statistical moments. Then, a novel texture-controlled multistep region-growing process is performed for the segmentation. One advantage of the method is that it is not designed to work with a particular kind of images. This method is tested on 80 natural color images of the Corel photo stock collection with excellent results. Numerical evidence of the quality of these results is provided by comparing them with the manual segmentation of five experts and with another color and texture segmentation algorithm. ©2006 Society of Photo-Optical Instrumentation Engineers. 关DOI: 10.1117/1.2205900兴 Subject terms: color image processing; segmentation; texture analysis. Paper 050256RR received Apr. 4, 2005; revised manuscript received Oct. 14, 2005; accepted for publication Oct. 15, 2005; published online May 31, 2006. 1 Introduction Automatic image segmentation is one of the fundamental problems of early computer vision. It is often the first and most vital step in an image analysis task. If any error occurs during this step, the whole process will fail irremediably, that is, an effective segmentation can usually dictate the eventual success of the analysis.1 Until a few years ago, segmentation techniques were proposed mainly for gray-level images due to acquisition devices and computer resources available.2Today with the development of color television or Internet, color image processing has become a crucial subject of study. Although color is the most important information in natural images, it is not the only source of knowledge available. In reality, natural scenes are rich in color and texture, so, those segmentation processes that take into account only color information will probably fail in giving us a proper result. That is why texture information must be included. However, texture is in general forgotten in most proposals, probably due to the difficulty in obtaining accurate boundary information when texture, which is a nonlocal image property, is considered. 共see Ref. 3, pp. 119– 130兲. In this paper, a new image region-growing segmentation method, which we refer to as CTREG 共color and texture region-growing algorithm兲, based on color and texture information is proposed. In the literature, there are many different region-growing algorithms. Most of them applied to gray-scale images and some of them are extended to color images. Hojjatoleslami and Kittler4presented a region-growing method based on two different contrast measures. The pixel with the highest gray-level value among the neighbors of the current pixel is added. The region-growing procedure will continue until a maximum size of the region, specified by the user, is reached, selecting the final boundary as the one with the highest contrast. In this sense, the main disadvantage of the method is its poor efficiency: only one pixel is added in each iteration, the contrast parameters must be calculated each time a new pixel is added and the growing does not stop until the maximum size of the region is reached. Adams and Bischof5proposed a method for gray-scale images, where the seeds are selected manually. They analyzed the eight-neighborhood of the pixels in the border, adding the neighbors that have the lowest difference to the mean of any of the existing regions. The method depends on the order in which the pixels are analyzed. Fan et al.6extended the previously mentioned technique to color images. Then, they improved the algorithm, automatically selecting the seeds, and proposed a new method for pixel labeling.7 Cheng8published a region-growing approach to color segmentation using 3-D clustering and relaxation labeling. The algorithm starts by smoothing the image to remove noise. Then the region-growing procedure based on relaxation labeling is performed. As final step, it merges regions if they are too small. The three methods mentioned take into account only the color information, but not the texture, and for many natural scenes it is very important to consider both characteristics to get a good segmentation. Finally, 0091-3286/2006/$22.00 © 2006 SPIE Optical Engineering 45共5兲, 057002 共May 2006兲 Optical Engineering May 2006/Vol. 45共5兲057002-1 Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/journals/optice/21903/ on 03/30/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx Maeda et al.9proposed a region-growing algorithm that joins color and texture information by applying fuzzy sets, performing a region-growing procedure based on a fixed homogeneity parameter. The proposed method 共CTREG兲calculates a color distance image in the L*a*b*color space to extract the color information. The value of a pixel in this image is the difference between its color in the original one and the color taken as a reference. To obtain texture information, statistical moment images and their corresponding energy images are calculated. Once we have color and texture information, a novel texture-controlled multistep regiongrowing process is performed for the segmentation. In an ordinary region-growing procedure, the merging condition is always the same, that is, the algorithm grows a region with a determined condition. With the multistep technique proposed in this paper, the merging condition automatically changes to find its optimum value. Shen et al.10 presented a method where the concept of multistep region growing was introduced, but it has two main disadvantages: 共1兲a very high computational cost because the tolerance parameter grows from a minimum to a maximum value for each seed, and a region-growing procedure is performed for each value of the tolerance step; and 共2兲the criteria to determine which tolerance step parameter gives the best region is defined for very specific images 共grayscale mammograms兲, and it cannot be used with other types of images. There are other adaptive region-growing algorithms,11,12 but most of them are specifically designed to work with a particular kind of image. CTREG has the advantage that its parameters are not preselected according to the kind of image, and therefore, it can be applied more successfully on various kinds of images. The proposed method in this paper introduces the following novelties: 共1兲a new texture analysis method to control the seed selection and the merging condition of the subsequent region growing, 共2兲a contrast criterion for the stop condition of the region growing, and 共3兲an automatically adaptive merging condition. In this way, it covers the lacks the preceding region-growing algorithms present: either the exclusion of texture information or a fixed merging condition. 2 Algorithm Description Figure 1 shows the scheme of the algorithm. The blocks of the scheme are described next. 2.1 Selecting a Region of Interest In the first step, the user selects an area 共selection box兲of the image with the mouse. The average color of the selection box is calculated and it is considered as the reference color. The algorithm segments all areas in the image with color and texture similar to the selected ones. 2.2 Color Information 2.2.1 L*a*b*color space In a further step, the color of each pixel is substituted by its distance to the reference color. Therefore, a perceptually uniform color space is required so that Euclidean distances between colors measured in this space are correlated with color differences according to human perception. We chose the L*a*b*color space, which is a perceptually uniform orthogonal Cartesian coordinate system.13 2.2.2 Anisotropic diffusion Original images may include some noise that could change the result of the segmentation. To solve this problem we use anisotropic diffusion for the denoising step. This is a nonlinear filtering method stronger in the homogeneous parts of the image and weaker in the edges.14 In other words, it has the property of eliminating noise while preserving edges, which is a desirable characteristic in color segmentation. Moreover, as we are processing color images, the most effective method to perform the diffusion is to apply it to the intensity and chromaticity separately.15 Other techniques of noise removal, such as median or Gaussian filtering, can be seen as isotropic diffusions because they try to make a pixel more similar to its neighbors. Instead of that, anisotropic diffusion tries to make a pixel more similar to the pixels of the neighborhood to which it was previously similar. Anisotropic diffusion gives us a good result for our purpose: noise removal with edge preserving. 2.2.3 Distance image As previously noted, the query region and the original image are transformed to the L*a*b*color space. The reference color is defined as the average color of the query region in the L*a*b*color space, that is, the centroid of the L*a*b*color coordinates in the selection box. Once the reference color is obtained, 共Lref *,aref *,bref *兲, the distance between every single pixel of the image and this reference color is calculated. We chose CIEDE2000 as the distance metric. This measure has been extensively tested Fig. 1 Scheme of the algorithm. Fondón, Serrano, and Acha: Color-texture image segmentation¼ Optical Engineering May 2006/Vol. 45共5兲057002-2 Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/journals/optice/21903/ on 03/30/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx and outperformed other existing color difference formulas.16 As a consequence, it has been approved by CIE and adopted as the CIE 2000 color difference formula. Then, a new image is built, where each pixel value will be the CIEDE2000 color difference to the reference color. To obtain a better visualization, we invert this image, that is, those pixels whose values are similar to the reference ones, will appear light in a dark background. This inverted image is called the distance image. Figure 2, presents some examples for different reference color selections. 2.3 Texture Information Texture information plays a significant role in image interpretation. Human beings are able to distinguish in an image those regions that are equal in color but different in texture. Our goal is to take advantage of this additional information, treating it not as a problem but as a useful tool. The proposed method extracts texture features only from the luminance component 共L*兲of the original image and not from the chrominance ones 共a*,b*兲. This assumption is based on previous works: the psychophysical studies of Poirson and Wandell suggest that color and pattern information in the human visual system are processed separately.17 Mojsilovic et al.18 suggest that the overall perception of color patterns is formed through the interaction of a luminance component, a chrominance component, and an achromatic pattern component. The luminance and chrominance components are used in extracting color-based information, while the achromatic pattern component is utilized as texture pattern information. Thus, they state that human perception of pattern is unrelated to the color content of an image. Mäenpää and Pietikäinen19 conclude that it seems that texture information should be extracted from the luminance component, whereas color is more a regional property. The texture features employed in this method are based on some local low statistical moments.20 To justify the choice of first-order statistics for texture feature extraction, Zamperoni21 states that although one can construct some patterns for which the choice of first-order statistics does not work, the converse is true for a surprisingly high number of real images representing natural scenes of a given type, as confirmed by the experiments reported in Lowitz22 and in Kim and Yaroslavskii.23 Furthermore, as this algorithm is applied to generalpurpose images, the texture present in this kind of images represents properties such as smoothness, coarseness, regularity, etc. rather than the arrangement of image primitives. In textures where a primitive is repeated, structural methods, spectral methods, or techniques based on second-order statistics, such as the coocurrence matrix could work better. But this does not happen in images where patterns are not present. The algorithm calculates for every pixel, four statistical moments mpq with p,q=兵0,1其by processing the L*component with local masks expressed in a normalized coordinate system. A formal expression of these moments is mpq =1 W2兺 m=i−W/2 i+W2 兺 n=j−W/2 j+W/2 f共m,n兲xm pyn q, xm=m−i W/2 ,yn=n−j W/2 , i,j苸image, p,q=0,1, 共1兲 where Wis the window width, 共i,j兲are the pixel coordinates for which the moments are computed, 共m,n兲are the coordinates of another pixel which falls within the window, 共xm,yn兲are the normalized coordinates for 共m,n兲, and f共m,n兲is the value of the L*component at the pixel with coordinates 共m,n兲. This normalized expression leads us to compare among pixel moments and it is equivalent to the finite convolution of the image with a mask. The sizes of these masks were fixed to the size of the selection box. Usually, for each segmentation this size will be different, so CTREG is automatically adapted to the texture we want to isolate. With all these parameters, we can build four new images Mpq with p,q=兵0,1其corresponding to each statistical parameter. To this purpose we assign to each pixel a value equal to the previously calculated moment mpq. For example, in the case of pixel 共30,20兲if we want to build the image M11 we define the value at position 共30,20兲as the moment m11, calculated with a window centered in that pixel. Note that we have included the moment m00, because it is an important feature to characterize the texture. The mean of the luminance in a texture is very significant. The presence of shaded areas in an object will cause this moment to vary, but this is not a problem in terms of perception: as we are separating different appearances, when the same object is partly in shade, it is perceived as two different regions. After an analysis of the moments, we defined new images calculated from the energy of the moments. We call them energy images E00,E01,E10, and E11 and they represent the strength of each moment around the pixel location. The computation of the energies follows Fig. 2 From the same original image 共a兲, we can obtain different color distance images. For example, if the reference color is yellow we obtain image 共b兲. Images 共c兲,共d兲,共e兲, and 共f兲correspond to pink, green, bright white, and dark blue reference colors, respectively 共color online only兲. Fondón, Serrano, and Acha: Color-texture image segmentation¼ Optical Engineering May 2006/Vol. 45共5兲057002-3 Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/journals/optice/21903/ on 03/30/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx Epq共i,j兲=1 W2兺 m=i−W/2 i+W/2 兺 n=j−W/2 j+W/2 Mpq 2共m,n兲,共2兲 where Epq共i,j兲is the energy corresponding to the pixel with coordinates 共i,j兲in the image Mpq,Wis the window width, Mpq共m,n兲is the value of the pixel with coordinates 共m,n兲 in the moment image Mpq, and p,q=兵0,1其 Each pixel is now characterized with four values, one from each energy image. They are considered as coordinates in a 4-D space. Subsequently, to assign each pixel to one texture in the image, we apply a clustering procedure in the 4-D space. The approach we have chosen is the k-means algorithm. To obtain the value kof numbers of clusters automatically, we use Dunn’s coefficient24 D= min 1ⱕiⱕk 冉 min 1ⱕjⱕk,j⫽i 再 d共ci,cj兲 max 1ⱕnⱕk 关d⬘共cn兲兴 冎 冊 ,共3兲 where d共ci,cj兲is the Euclidean distance between cluster i and cluster j, that is, the intercluster distance; d⬘共cn兲is the intracluster distance for cluster n; and kis the number of clusters. We assume that in natural scenes the number of different textures is less or equal to 16. Thus, we perform 16 clusterings beginning from k=1 to k=16. Each time, the Dunn’s coefficient 关Eq. 共3兲兴 is computed and stored. When the last clustering is done, the obtained Dcoefficients are compared, selecting the kvalue that provides the highest value of D, which leads to a maximum intercluster distance and a minimum intracluster distance. Once each pixel in the image has been classified, we select only those pixels whose texture is equal to the desired one, obtaining a black and white image in which white pixels are those with the desired texture, as shown in Fig. 3. This image will be used afterward in the regiongrowing process. 2.4 Multistep Region Growing Once the distance image and the texture information are obtained, the region-growing process begins. As explained, region-growing techniques have two critical aspects: the seed selection and the choice of the merging condition. 2.4.1 Seed Selection To identify possible seeds we take advantage of the knowledge about how the distance image has been built. Those pixels more similar to the reference color have been assigned a high value 共note that we have inverted the distance image兲. To select the seeds, the next three steps are followed. 共1兲First is selection of the local maxima of the image, which represent the candidates to seeds. Not all these candidates will be seeds for the region growing, because these local maxima do not belong necessarily to the region of interest. Second is application of a threshold to these candidate seeds. The threshold is determined from the histogram of the distance image, more specifically, the threshold will be the position of the peak closest to the right part of the histogram, as seeds should have high values as already explained. In images where different objects are present, the histogram typically presents different modes, each representing an object or the background, and each mode contains at least a local maximum. In the distance image, the rightmost mode corresponds to the object to be segmented. Therefore, any pixel with values belonging to this mode is a valid seed for the region growing. One way of ensuring that seeds belong to the rightmost mode is to choose pixels with values on the right of the rightmost peak in the histogram, because this peak will definitely correspond to this mode. Obviously, the only thing that matters for the good performance of the algorithm is that seeds belong to the region to be segmented, and provided this condition is met, the choice of a particular threshold for the seeds is not decisive in the success of the segmentation. The procedure to find significant peaks and valleys and the threshold follows the algorithm developed by Acha et al.25 Finally, texture information is applied to reject some of the seed candidates: the final seeds must have not only the desired color but also the desired texture. That is, among the group of color seeds, only those pixels that appear white in the texture image are selected. An example that summarizes the seed selection procedure is shown in Fig. 4. 2.4.2 Multistep region growing We use a dynamic region-growing method to segment the distance image. The structure is shown in Fig. 5. The original idea introduced by this method is that in an ordinary region growing, the merging condition is always the same. For each seed, the algorithm grows a region with a determined condition. With this multistep technique, the merging condition automatically changes to find its optimum value, which will correspond to the highest value of the contrast parameter explained later on in this subsection. Let us take a particular seed. The process begins with a region growing with three conditions: 1. It does not belong to another previously grown region. 2. The texture of the pixel must be the desired one. This means that a pixel will be added to the region only if it has a value equal to one 共for normalized values兲in the texture image. 3. The new pixel must be similar to the pixels that already are in the region. This similarity is measured according to Fig. 3 Original image 共a兲is processed to isolate the mushrooms 共circled in red兲.共b兲Result of the k-means algorithm for k=2, the value of Dis 0.5908; 共c兲result for k=6 and D=0.2620; 共d兲corresponds to k=10 and D=0.2888; and 共e兲was obtained with k=16 and D=0.2423. Finally, the maximum Dvalue is obtained for k=2, which leads to the texture information shown in image 共f兲共color online only兲. Fondón, Serrano, and Acha: Color-texture image segmentation¼ Optical Engineering May 2006/Vol. 45共5兲057002-4 Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/journals/optice/21903/ on 03/30/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx Fmax +Fmin 2− ␶ ⱕFij ⱕFmax +Fmin 2+ ␶ ,共4兲 where Fmax and Fmin are the maximum and minimum values of the pixels in the distance image inside the region; iand jare the coordinates of the pixel Fin the distance image; and ␶ is the tolerance step, which will be iteratively increased. Note that the region growing does not depend on the position of the seed within the region. Although a boundary is encountered on one side, the algorithm will continue growing with the same parameter of tolerance in the other directions until no other pixel can be included in the region; and only then, the contrast is calculated to determine if we should increase the tolerance and continue growing. Once a region is grown with a particular ␶ , the next step is to verify whether the region obtained is optimal. If it is not optimal, the region growing will be repeated with a more relaxed condition, that is, ␶ is increased. More specifically, ␶ follows the expression ␶ = ␣ ␴ ,共5兲 where ␴ is the standard deviation of the region grown before, and ␣ is a variable with an initial value experimentally fixed to 0.1. For each iteration, to relax the condition and make the merging interval larger, we increase ␣ by 0.1. Then, the region growing is repeated with this more relaxed condition. The optimality criterion to choose ␶ , i.e., to choose the best region during the region-growing process, consists in maximizing a contrast parameter. This contrast parameter is calculated as: contrast = 兩inside edge − outside edge兩 inside edge + outside edge ,共6兲 where inside edge and outside edge represent the mean values of the pixels belonging to the inner border and outer border of a region respectively, as shown in Fig. 6. At the beginning, the region growing has a very restrictive merging condition. This will lead us to obtain a small region. While repeating the process, the contrast parameter of Eq. 共6兲is calculated. While the grown region is inside the object, the contrast parameter increases its value in a smooth way, because pixels belonging to the inner border and to the outer border of the region are similar. When the region whose contrast is being calculated matches the object, the contrast parameter has a high value because pixels surrounding the region will differ from those inside the region. If we continue growing, the contrast parameter will be low again because both the inner border and the outer border are similar. Therefore when the contrast parameter reaches its maximum we have obtained the best region. An example of the region-growing process is shown in Fig. 7. Figure 8 shows the evolution of the contrast parameter Fig. 4 Seed selection process: 共a兲the original image with the selection box in green, 共b兲image corresponding to the color distance image from which local maxima are extracted 共c兲. After the thresholding process 共d兲, with a threshold equal to 221, we obtain the color seeds 共e兲. Using texture information 共f兲, the final seeds are obtained 共g兲. Final result is shown in 共h兲共color online only兲. Fig. 5 Scheme of the multistep region-growing process. The flowchart is applicable to any merging criterion and optimality test. Fondón, Serrano, and Acha: Color-texture image segmentation¼ Optical Engineering May 2006/Vol. 45共5兲057002-5 Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/journals/optice/21903/ on 03/30/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx while the region is growing for two different images. As we can observe in this figure, the contrast parameter changes softly while no boundary is reached. A steep slope in the contrast parameter evolution corresponds to those values of ␣ for which boundaries are reached. This increase may be either because the whole boundary of the object has been reached or because the boundary of the region grown matches in part the boundary of the object. In the second situation, the tolerance will continue to be increased until the whole boundary is reached. During this increase of the parameter ␣ , the contrast parameter never decreases, and as a consequence, the stop condition is not reached, because the increase of the tolerance is not high enough to overcome a boundary. Once the whole boundary is reached, if the tolerance is being enlarged again, the region will exceed the limits of the object and, therefore, the contrast will decrease. In such a situation, the region growing will stop because the stop condition has been attained. 3 Experimental Results As a first step to test CTREG algorithm, we constructed a database of 10 known ground-truth images that are artificial compositions of geometrical objects of different colors and textures. Some examples can be seen in Figs. 9共a兲–9共c兲. These images are called “ground-truth” because, as they are formed by geometrical pieces, the boundary of each piece is known accurately. To further validate CTREG, we used 80 images taken from the commercial Corel stock photo collection. The validation test was performed by comparing the results of the algorithm with Carson et al.’s work26 and with the manual segmentation carried out by a group of five experts. Carson’s work is an algorithm that segments natural images based on texture and color information and its results are available on the Internet 共http:// elib.cs.berkeley.edu/ photos/blobworld兲. To analyze the performance of the algorithm in an objective way, two parameters were calculated. The first parameter is the precision or positive predictive value, which measures the ratio between the number of pixels segmented by the algorithm that fit the segmentation gold standard and the total number of pixels segmented. The second parameter is called recall or sensitivity, and it is the ratio between the number of pixels segmented by the algorithm that fit the Fig. 6 共a兲Possible result of the growing for one seed, 共b兲the inside edge shown in red, and 共c兲the outside edge of that region shown in green 共color online only兲. Fig. 7 Illustration of the region-growing process: 共a兲original image; 共b兲Seeds for the region growing, the one to be grown in this example is marked with a red circle; 共c兲distance image and desired region circled in red; 共d兲and 共e兲grown regions with increasing ␶ parameters; and 共f兲final region 共color online only兲. Fig. 8 Evolution of the contrast parameter while iteratively increasing parameter ␣ for two different figures: 共a兲and 共f兲original images; 共b兲and 共g兲region grown with ␣ =2.7 and 0.7, respectively; where pixels belonging to the grown region have gray-level values corresponding to the distance image and the rest of the pixels are in black; 共c兲and 共h兲grown region with ␣ =3.8 and 0.9, respectively, which corresponds to the highest contrast value; 共d兲and 共i兲grown region with ␣ =3.9 and 1.4, respectively, where we can observe that it is larger than the desired one; and 共e兲and 共j兲evolution of the contrast parameter versus ␣ 共color online only兲. Fondón, Serrano, and Acha: Color-texture image segmentation¼ Optical Engineering May 2006/Vol. 45共5兲057002-6 Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/journals/optice/21903/ on 03/30/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx segmentation gold standard and the total number of pixels in the segmentation gold standard. Intuitively, we can see that the first parameter measures the oversegmentation, which would be null if precision were 1. Likewise, recall measures the undersegmentation. The segmentation gold standard was obtained by applying an expectation maximization validation algorithm.27 It estimates the ground truth from a group of five experts, who segmented manually the images of the database. Tables 1 and 2 present the average results for the 10 known ground-truth images and the 80 Corel images. We can observe the good performance of CTREG when it is tested with the known ground-truth images achieving a recall and a precision values above 0.9. In Fig. 9 some examples of these segmentations are shown. Regarding to the 80 Corel images, we can also say that the obtained results are very good, particularly when we compare CTREG results with the manual segmentation of the group of experts. The algorithm achieves a recall of 0.69 for a precision of 0.81. Note that, although the sensitivity is 0.69, this is because experts tend to oversegment. Therefore, this should not be interpreted as a poor performance of the algorithm. Carson’s algorithm achieves a recall of 0.68 and a precision of 0.63. We can observe that although in terms of recall both methods are very similar, regarding to precision our algorithm outperforms Carson’s one. Figure 10 presents some images segmented with both algorithms. Figures 10共e兲–10共h兲show segmentation results of the proposed algorithm and Figs. 10共i兲–10共l兲show segmentation results of Carson’s method. Carson’s method segments the whole image into regions with the same color and texture, and in the Fig. 9 Examples of segmentations performed by CTREG. 共a兲to 共c兲 original known ground-truth images with the selected color and texture marked by the user in green and 共d兲to 共f兲images segmented by CTREG. Table 1 Quantification of segmentation results for the 10 known ground-truth images. CTREG Recall mean 0.98 Recall standard deviation 0.0003 Precision mean 0.94 Precision standard deviation 0.004 Table 2 Quantification of segmentation results for the 80 Corel images. CTREG Carson’s Algorithm Recall mean 0.69 0.68 Recall standard deviation 0.19 0.21 Precision mean 0.81 0.63 Precision standard deviation 0.21 0.31 Fig. 10 Examples of segmentations performed by CTREG and Carson’s algorithm. 共a兲to 共d兲original images from Corel photo stock collection with the selected color and texture marked by the user in green; 共e兲to 共h兲images segmented by CTREG, 共i兲to 共l兲images segmented by Carson’s algorithm 共color online only兲. Fondón, Serrano, and Acha: Color-texture image segmentation¼ Optical Engineering May 2006/Vol. 45共5兲057002-7 Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/journals/optice/21903/ on 03/30/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx resulting image, each region is shown in a different color. In CTREG, only the region with the same texture and color as the selection box is colored in pink. Note that in Figs. 10共f兲and 10共g兲the front part of the airplanes, the snout of the bear, and a small part of its back are not segmented because they have different color and texture than the selection box specified by the user, shown as a green square in the figures. The same happens in Fig. 10共h兲. Another remark is that in Fig. 10共k兲Carson’s method does not consider all the airplanes as belonging to the same region. A similar problem occurs in Fig. 10共i兲. 4 Conclusions A new method for natural color image segmentation was proposed based on a multistep region-growing technique, which has an automatic adaptable step. Contrast was introduced to determine whether or not a region is the best, and the step that provides a region with the highest contrast in relation to the background was chosen. Color and texture information were used to fulfill the requirements of real color images. These two pieces of information were extracted from the image by means of a novel idea: the construction of a color distance image and a texture energy image. One advantage of the method is that it is not designed to work with a particular kind of images, but we have demonstrated its good performance with images that present high segmentation difficulty. As already discussed, the algorithm was validated with 10 known ground-truth images. Additionally, its performance was quantified by comparing the result of segmenting 80 natural images with the manual segmentation of a group of five experts, obtaining very good results. Furthermore, the algorithm was compared with a previously published and extensively validated algorithm, outperforming its results. Acknowledgments The authors would like to thank the experts for segmenting the images. This work is financially supported by project TIC-2002-01401 共CICYT, Spain兲. References 1. R. González and R. E. Woods, Digital Image Processing, 2nd ed., p. 567, Prentice Hall, Upper Saddle River, NJ 共2002兲. 2. L. Lucchese and S. K. Mitra, “Unsupervised low frequency driven segmentation of colour images,” in Proc. IEEE Int. Conf. on Image Processing, pp. 240–244 共1999兲. 3. X. Muñoz, “Image segmentation integrating colour, texture and boundary information,” PhD Thesis, Universitat de Girona 共Spain兲, 共2002兲. 4. S. A. Hojjatoleslami and J. Kittler, “Region growing: a new approach,” IEEE Trans. Image Process. 7共7兲, 1079–1084 共1998兲. 5. R. Adams and L. Bischof, “Seeded region growing,” IEEE Trans. Pattern Anal. Mach. Intell. 16共6兲, 641–647 共1994兲. 6. J. Fan, D. K. Y. Yau, A. K. Elmagarmid, and W. G. Aref, “Automatic image segmentation by integrating color-edge extraction and seeded region growing,” IEEE Trans. Image Process. 10共10兲, 1454–1466 共2001兲. 7. J. Fan, G. Zeng, M. Body, and M.-S. Hacid, “Seeded region growing: an extensive and comparative study,” Pattern Recogn. Lett. 26, 1139– 1156 共2005兲. 8. S.-C. Cheng, “Region-growing approach to colour segmentation using 3-D clustering and relaxation labelling,” IEE Proc. Vision Image Signal Process. 150共4兲, 270–276 共2003兲. 9. J. Maeda, S. Novianto, S. Saga, Y. Suzuki, and V. V. Anh, “Rough and accurate segmentation of natural images using fuzzy regiongrowing algorithm,” in Proc. Int. Conf. on Image Processing, Vol. 3, pp. 227–231 Kobe, Japan 共1999兲. 10. L. Shen, R. Rangayyan, and J. E. L. Desautels, “Detection and classification of mammographic calcifications,” Int. J. Pattern Recognit. Artif. Intell. 7共6兲, 1403–1416 共1993兲. 11. X. Hao, C. Bruce, C. Pislaru, and J. F. Greenleaf, “A novel region growing method for segmenting ultrasound images,” in Proc. IEEE Int. Ultrasonics Sympo., pp. 1717–1720 共2000兲. 12. R. Pohle and K. D. Toennies, “Segmentation of medical images using adaptative region growing,” Proc. SPIE 4332, 1337–1346 共2001兲. 13. K. N. Plataniotis and A. N. Venetsanopoulos, Color Image Processing and Applications, pp. 35–37, Springer, Berlin 共2000兲. 14. P. Perona and J. Malik, “Scale-space and edge detection using anisotropic diffusion,” IEEE Trans. Pattern Anal. Mach. Intell. 7, 629–639 共1990兲. 15. L. Lucchese and S. K. Mitra, “Color segmentation based on separate anisotropic diffusion of chromatic and achromatic channels,” IEE Proc. Vision Image Signal Process. 3, 141–150 共2001兲. 16. M. R. Luo, G. Cui, and B. Rigg, “The development of the CIE 2000 colour-difference formula: CIEDE2000,” Color Res. Appl. 26共5兲, 340–350 共2001兲. 17. B. Poirson and B. Wandell, “Pattern-color separable pathways predict sensitivity to simple colored patterns,” Vision Res. 36共4兲, 515–526 共1996兲. 18. A. Mojsilovic, J. Kovacevic, D. Kall, R. Safranek, and S. Ganapathy, “Matching and retrieval based on the vocabulary and grammar of color patterns,” IEEE Trans. Image Process. 9共1兲, 38–54 共2000兲. 19. T. Mäenpää and M. Pietikäinen, “Classification with color and texture: jointly or separately?,” Pattern Recogn. 37, 1629–1640 共2004兲. 20. M. Tuceryan, “Moment based texture segmentation,” Pattern Recogn. Lett. 15共7兲, 659–668 共1994兲. 21. P. Zamperoni, “Model-free texture segmentation based on distances between first-order statistics,” Digit. Signal Process. 5, 197–225 共1995兲. 22. G. Lowitz, “Can a local histogram really map texture information?,” Pattern Recogn. 2, 141–147 共1983兲. 23. V. Kim and L. P. Yaroslavskii, “Rank algorithms for picture processing,” Comput. Vis. Graph. Image Process. 35, 234–258 共1986兲. 24. U. Maulik and S. Bandyopadhyay, “Performance evaluation of some clustering algorithms and validity indices,” IEEE Trans. Pattern Anal. Mach. Intell. 24共12兲, 1650–1654 共2002兲. 25. B. Acha, C. Serrano, J. I. Acha, and L. M. Roa, “CAD tool for burn diagnosis,” Lect. Notes Comput. Sci. 2732, 294–305 共2003兲. 26. C. Carson, S. Belongie, H. Greenspan, and J. Malik, “Blobworld: image segmentation using expectation-maximization and its application to image querying,” in Proc. Third Int. Conf. on Visual Information and Information Systems, LNCS, Vol. 1614, 509–516, Amsterdam 共1999兲. 27. S. K. Warfield, K. H. Zou, and W. M. Wells, “Validation of image segmentation and expert quality with an expectation-maximization algorithm,” in Proc. MICCAI 2002 Fifth Int. Conf. on Medical Image Computing and Computer Assisted Intervention, pp. 298–306, Tokyo 共2002兲. Irene Fondón received her MS degree in telecommunication engineering from the University of Seville, Spain, in 2004. She has been working as an associated professor since 2005 in the Signal Processing and Communications Department of the University of Seville. She is currently working toward her PhD degree. Her current research activities include works in the field of image processing and its medical applications. She is author of papers both in journals and international conferences. Carmen Serrano received her MS degree in telecommunication engineering from the University of Seville, Spain, in 1996 and her PhD degree in January 2002. In 1996, she joined the Signal Processing and Communication Department at the same university, where she is currently an associate professor. Her research interests concern image processing and color image segmentation, classification and compression, mainly with biomedical applications. She is author of numerous papers both in journals and international conferences. Fondón, Serrano, and Acha: Color-texture image segmentation¼ Optical Engineering May 2006/Vol. 45共5兲057002-8 Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/journals/optice/21903/ on 03/30/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx Begoña Acha received her PhD degree in telecommunication engineering in July 2002. She has been working as an associate professor since 1996 in the Signal Processing and Communications Department of the University of Seville. Her current research activities include works in the field of color image processing and its medical applications, in particular in segmentation, classification and compression. She is author of numerous papers both in journals and international conferences. Fondón, Serrano, and Acha: Color-texture image segmentation¼ Optical Engineering May 2006/Vol. 45共5兲057002-9 Downloaded From: http://spiedigitallibrary.org/pdfaccess.ashx?url=/data/journals/optice/21903/ on 03/30/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx