In laparoscopic surgery, soft tissue deformations substantially change the surgical site, thus impeding the use of preoperative planning during intraoperative navigation. Extracting depth information from endoscopic images and building a surface model of the surgical field-of-view is one way to represent this constantly deforming environment. The information can then be used for intraoperative registration. Stereo reconstruction is a typical problem within computer vision. However, most of the available methods do not fulfill the specific requirements in a minimally invasive setting such as the need of real-time performance, the problem of view-dependent specular reflections and large curved areas with partly homogeneous or periodic textures and occlusions.Methods:
In this paper, the authors present an approach toward intraoperative surface reconstruction based on stereo endoscopic images. The authors describe our answer to this problem through correspondence analysis, disparity correction and refinement, 3D reconstruction, point cloud smoothing and meshing. Real-time performance is achieved by implementing the algorithms on thegpu. The authors also present a new hybrid cpu-gpu algorithm that unifies the advantages of the cpu and the gpu version.Results:
In a comprehensive evaluation usingin vivo data, in silico data from the literature and virtual data from a newly developed simulation environment, the cpu, the gpu, and the hybrid cpu-gpu versions of the surface reconstruction are compared to a cpu and a gpu algorithm from the literature. The recommended approach toward intraoperative surface reconstruction can be conducted in real-time depending on the image resolution (20 fps for the gpu and 14fps for the hybrid cpu-gpu version on resolution of 640 × 480). It is robust to homogeneous regions without texture, large image changes, noise or errors from cameracalibration, and it reconstructs the surface down to sub millimeter accuracy. In all the experiments within the simulation environment, the mean distance to ground truth data is between 0.05 and 0.6 mm for the hybrid cpu-gpu version. The hybrid cpu-gpu algorithm shows a much more superior performance than its cpu and gpu counterpart (mean distance reduction 26% and 45%, respectively, for the experiments in the simulation environment).Conclusions:
The recommended approach for surface reconstruction is fast, robust, and accurate. It can represent changes in the intraoperative environment and can be used to adapt a preoperative model within the surgical site by registration of these two models.
The present research was conducted in correlation with the “Research training group 1126: Intelligent Surgery—Development of new computer-based methods for the future workplace in surgery” founded by the German Research Foundation. Furthermore, it is sponsored by the European Social Fund through the state of Baden-Württemberg. The authors thank the Visual Information Processing Group at the Imperial College in London for their help with evaluating data sets. The authors also thank FA Richard Wolf and the NVidia Academic Partnership program for their support.
II. MATERIALS AND METHODS
II.B. Correspondence analysis
II.B.1. Hybrid recursive matching (cpu version)
II.B.2. Hybrid recursive matching (gpu version)
II.C. Disparity correction
II.D. Bilateral disparity smoothing
II.E. Point cloud reconstruction, smoothing and meshing
II.F. Hybrid cpu-gpu algorithm
III.A. Evaluation on virtual image sequences
III.B. Evaluation on stereo endoscopic images
III.B.1. Image sequences of a silicone phantom
III.B.2. Image sequences from daVinci interventions
III.C. Runtime evaluation
- Medical imaging
- Surface reconstruction
- Endoscopic imaging
- Image reconstruction
- Medical image reconstruction
Data & Media loading...
Article metrics loading...
Full text loading...