Model of vessels for 3D reconstruction

MODEL OF VESSELS FOR 3D RECONSTRUCTION Luis Álvarez León, Karina Baños Rodríguez, Carmelo Cuenca Hernández, Julio Esclarín Monreal, Javier Sánchez Pérez. Address: University of Las Palmas de Gran Canaria Edificio de Informática y Sistemas. Campus Universitario de Tafira 35017 Las Palmas de Gran Canaria. Spain Telephone: +3428458708/09 Fax: +3428458711 e-mail: {lalvarez,kbaños,ccuenca,jesclarin,jsanchez}@dis.ulpgc.es Presenting author: Julio Esclarín Monreal Abstract In this paper, we present a vascular tree model made with synthetic materials and which allows us to obtain images to make a 3D reconstruction. In order to create this model, we have used PVC tubes of several diameters and lengths that will let us evaluate the accuracy of our 3D reconstruction. We have made the 3D reconstruction from a series of images that we have from our model and after we have calibrated the camera. In order to calibrate it we have used a corner detector. Also we have used Optical Flow techniques to follow the points through the images going and going back. Once we have the set of images where we have located a point, we have made the 3D reconstruction choosing by chance a couple of images and we have calculated the projection error. After several repetitions, we have found the best 3D location for the point. Keywords: Angiography, Stereo Vision, Disparity Map, Scale Space, 3D and Range Data Analysis. 1. Introduction. Given the difficulties to obtain medical images in good conditions, due to their privacy and diverse technical problems that these can present, such as: artifacts, occlusions, poor definition of the image, etc; we have developed a model of vascular tree using PVC. This way we can obtain a series of images following the same technology of a rotational angiography, planning an arch of fixed radius about the model and obtaining the images with an angular separation of about 3 degrees. Once obtained, they were manipulated in order that the final result is as similar as possible to the angiographies, but without the problems previously mentioned. Besides, by having the 3D model it is possible to test the quality of the results obtained, knowing exact values such as: distances between bifurcations, the diameters of the glasses, etc; which will allow us to know the kindness of our results. This work has developed in the following way: 1. Obtaining the images. 2. Calibration of cameras. 3. Follow-up of points. 4. Reconstruction in 3D. 2. Rotational Angiography Features. We want to reconstruct images to apply in the Rotational Angiography field that it has the next features: The series of images in Rotational Angiography is acquired while the imaging assembly rotates in a continuous arc around the patient. The whole acquisition is rather fast, so that the complete series can be acquired with a single injection of contrast agent. The quality of the individual images is generally fully adequate for diagnosis, with the following added advantages: wide range of projections, optimum views of vascular structures. For the 3D reconstruction, it is essential for the images to precisely match each other. This requires an extremely stable and reproducible image geometry. The system is calibrated to compensate for distortion in the image intensifier such us pincushion and the varying distortion caused by movement through the magnetic field of the earth. The images are acquired in the rotational angiography mode over an angle of 180 degrees. The run maybe carried out in one of three different angulations: -30 degrees cranial, 0 degrees axial, 30 degrees caudal. Images are acquired at a frames rate of 12.5 frames by seconds, and a rotation speed of up to 30 degrees per second, the whole acquisition takes 8 seconds resulting in an average of 100 images per run. We have made the images with an angle of 3.6 degrees and approximately three meters of distance from the model using a of 70 mm focal distance with a digital camera and because the characteristics of its digitizer then we have a focal distance 105 mm with an arc of 270 degrees that allows around 70 images. We have also obtained images of a calibrator in order to obtain the intrinsic parameters of the camera. Figure 1: images acquired 3. Multiscale Analisis and Calibration Camera. One of the main concepts of vision theory and image analysis is multiscale analysis. A Multiscale Analysis Tt associates, with an original image u(0) = u0 a sequence of smoothed images u(t,x,y) which depend upon an abstract parameter t > 0, the scale. ),(),,0( ),,(),( yxfyxu yxtuyxf t T = → The datum of u0(x,y) is not absolute in perception theory, but can be considered as the element of an equivalence class. If A is any affine map to the plane, u0(x,y) and u0(A(x,y)) can be assumed equivalent from a perceptual point of view. Last but not least, the observation of u0(x,y) does not generally give any reliable information about the number of photons sent by any visible place to the optical sensor. Therefore, the equivalence class in consideration will be g(u0(A(x,y))), where g stands for any contrast function depending on the sensor. These considerations lead us to focus on the only multiscale analyses which satisfies these invariance requirements : The Affine Morphological Scale Space (AMSS). This multiscale analyses can be defined by a simple Partial Differential Equation: () 3 1 22 3 1 2yyxxyyxxxyt uuuuuuutu +−= where u(t,x,y) denotes the image analyzed at the scale t and the point (x,y). In order to calibrate a camera system we need to corner detection and this is very sensitive to noise. The AMSS multiscale analysis present the advantage that we know, analytically, the displacement of the corner location across the scales. Then we can search it in at the scale tn = t0 +n∆t, for n=1,..,N, where ∆t represents the discretization step for the scale and t0 represents the initial scale that we use to begin to look for corners. We compute for the scale t0 the location of the extreme of the curvature that we denote by ( ) ii yx 00 , ,for i=1,.., M, these points represent the initial candidates to be corners. We follow across the location ( ) i n i nyx , of the curvature extreme. For each sequence ( ) i n i nyx , n=1,..,N, we compute in a robust way(using orthogonal regression and eliminating outliers) the best line which fit the sequence of points, this line corresponds to the bisector line of the corner, and we can represent it as a straight line which equation: () ()()() () yxo bbtyxtytx , 2 tan,, 2 1 0 −       += α where α is the angle of the corner and () yx bbb ,=, is the unit vector in the direction of the bisector line of the corner, and t is the scale. Then we can find the corner doing t = 0 in this equation. In order to calibrate the cameras system, we extract the characteristics of the sequence of views with a morphologic corner detector. This detector gives us sub pixel information. When the views are taken from very close positions, the conventional methods of calibration can be unstable, to solve this problem we divide the sequences of views into several sub sequences (in this way the optical centre displacements are bigger). Now, we calibrate every subsequence of view in an independent way. Figure 2: Motion parameters derived from point matches. In the last step, we make the calibration between the different sub sequences to obtain only one calibration. The method we have used is very stable, even when there are noise and small displacements between the optical centres. xworl d zworl d x y worl d x y y z 2 z 0 x y z 1 M j m2 j m0 j m1 j C 0 C 1 C 2 Oworl d O0 O1 O2 3. Optical Flow and 3D Reconstruction Once we have calibrated the camera system we propose a method for the recovering of disparity maps between pair of stereoscopic images. Disparity maps are obtained through a matching process in where we have to find for the pixels in the left image their correspondent on the right image. There are some methods, like correlation-based techniques, that estimate good matching points but do not generate smooth disparity maps for the whole image, so the solutions in this case are not continuous. To improve the accuracy of the matching process we make use of the so-called epipolar geometry. This geometry represents the relation that exists between stereoscopic images. Thanks to this geometry, the method is able to look for correspondences in straight lines only. Figure 3: Epipolar Geometry and corresponding points The method we propose for the computation of disparity maps is based on an energy minimization approach: Where I and I´ are the stereoscopic images, λ is a parameter that gives us the distance between the point that it is the projection of m over the it epipolar straight line and the responsive point of m in I’, m’ (see image 3). The second term in the equation is used to regularize. D is a diffusion tensor that diffuses in one or another way depending where the point is placed. If gradient of I is high then the regularization is over the contour line and if it is low, we make regularization. This energy consists of an attachment term that enables the process to find similar pixels in both images and a regularization term that is necessary to constraint the number of possible solutions and to generate smooth solutions. This method is a dense method in the sense that for every pixel on one image we obtain its correspondent on the other image. When we minimize this energy, we obtain the Euler-Lagrange equations, which are represented by means of partial differential equations. This is a diffusion-reaction equation that behaves anisotropically at contours with high values for the gradient of the images and isotropically at homogeneous regions where the image gradient is low. The diffusion part is formulated in such a way that the discontinuities of the images are preserved. We use a scalespace and pyramidal strategy to allow the method to locate large displacements. Thanks to this energy minimization approach the resulting disparity maps that we may obtain are smooth by regions. To search the corresponding points in every image, we start with a point in one image, and using the optical flow techniques, we search the corresponding point in the next image. Once this point is obtained, we go back and we search if the corresponding point in the first image is the start point. If it is true, we continue searching other point in the next image. Once we have this new point we come back until the first image verifying that the points calculates are the same points that we have with a small error. We finish this process when we search a point bad placed. m=(x,y) m’=(x’,y’) 0)','( =yxRe ),( yx λ () () ∫∫ ∇∇∇++−= λλλλ IDhxIxIE 2 ))((')()( G G G When we have a set of images, then we take a couple of cameras by chance and we reconstruct the 3D point. We project this point to the plane of the whole cameras and we calculate the projection error, the distance between both points, the real point and the projected point in every camera. Later we take another couple of cameras, always by chance, and we repeat the same steps several times, up to 8 times if it is possible. We keep with the point that minimizes the projection error. We shown some results in the figure 4, we have used 25 images to obtain the 3D reconstruction with different projection error: 1.5 in the left image and 1.0 in the right image, and we can observe that in the right image there are less points than the left image, but this points are better placed. Figure 4: some results REFERENCES 1. L. Álvarez, K. Baños, C. Cuenca, J. Esclarín and J. Sánchez. 3D reconstruction from a vascular tree model, EUROCAST, Las Palmas, 2003. 2. L. Álvarez, C. Cuenca. Calibración de multiples cámaras utilizando objetos de calibración esféricos. Caepia 2001. 3. L. Álvarez, R. Deriche, J. Sánchez, and J. Weickert. Dense disparity mapestimation respecting image derivatives: a pde and scale-space based approach. Journal of Visual Communication and Image Representation, 13:3–21, January 2002. Also published as Inria Research Report no 3874. 4. L. Álvarez, J. Weickert, and J. Sánchez. Reliable estimation of dense optical flow fields with large displacements. International Journal of Computer Vision, 39(1):41–56, 2000. 5. O. Faugeras. Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press, 1993. 6. Hans-Hellmut Nagel and W. Enkelmann. An investigation of smoothness constraints for the estimation of displacement vector fields from images sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(5):565–593, 1986. 7. Olivier Faugeras and Renaud Keriven. Complete dense stereovision using level set methods. Proceedings of Fifth European Conference on Computer Vision, 1998. 8. J. Weickert. Anisotropic Diffusion in Image Processing, Teubner, Stuttgart, 1998. 9. P. Peene, P. Cleeren, B. D’Herde, L. Storme, J. Vanrusselt and G. Souverijns. Non/substracted Rotational Angiography on a Multipropose Digital C/arm Radiography System. Medica Mundi, 1999. 10. J. Moret, R. Kemkers, J. Op de Beek, R. Koppe, E. Klotz and M. Grass. 3D Rotational Angiography: Clinic value in Endovascular treatment. Medicamundi. 1998 11. L. Robert, R. Deriche. Dense depth map reconstruction: A minimization and regularization approach which preserves discontinuities. B. Buxton, R. Cipolla (Eds.), Computer vision - ECCV'96, Volume I, Lecture Notes in Computer Science, Vol. 1064, Springer, Berlin, 439-451, 1996.