Published on Sat Mar 10 2018

A Deep Learning Approach for Pose Estimation from Volumetric OCT Data

Nils Gessert, Matthias Schlüter, Alexander Schlaefer

OCT is suitable for accurate pose estimation due to its micrometer range resolution and volumetric field of view. We design a new 3D convolutional neural network architecture to directly predict the 6D pose of a small marker geometry. We show that exploiting volume information for pose estimation yields higher accuracy than relying on 2Drepresentations.

0
0
0
Abstract

Tracking the pose of instruments is a central problem in image-guided surgery. For microscopic scenarios, optical coherence tomography (OCT) is increasingly used as an imaging modality. OCT is suitable for accurate pose estimation due to its micrometer range resolution and volumetric field of view. However, OCT image processing is challenging due to speckle noise and reflection artifacts in addition to the images' 3D nature. We address pose estimation from OCT volume data with a new deep learning-based tracking framework. For this purpose, we design a new 3D convolutional neural network (CNN) architecture to directly predict the 6D pose of a small marker geometry from OCT volumes. We use a hexapod robot to automatically acquire labeled data points which we use to train 3D CNN architectures for multi-output regression. We use this setup to provide an in-depth analysis on deep learning-based pose estimation from volumes. Specifically, we demonstrate that exploiting volume information for pose estimation yields higher accuracy than relying on 2D representations with depth information. Supporting this observation, we provide quantitative and qualitative results that 3D CNNs effectively exploit the depth structure of marker objects. Regarding the deep learning aspect, we present efficient design principles for 3D CNNs, making use of insights from the 2D deep learning community. In particular, we present Inception3D as a new architecture which performs best for our application. We show that our deep learning approach reaches errors at our ground-truth label's resolution. We achieve a mean average error of \SIµ\metre and \SI for position and orientation learning, respectively.

Thu Jul 02 2020
Computer Vision
4D Spatio-Temporal Convolutional Networks for Object Position Estimation in OCT Volumes
Optical coherence tomography (OCT) can be employed as an optical tracking system. OCT allows for a temporal stream of OCT image volumes capturing the motion of an object.
0
0
0
Tue Mar 03 2020
Computer Vision
Single-Shot Pose Estimation of Surgical Robot Instruments' Shafts from Monocular Endoscopic Images
Surgical robots are used to perform minimally invasive surgery. A new method to estimate the pose of the surgical instruments' shafts using a monocular endoscope.
0
0
0
Mon Oct 22 2018
Computer Vision
Two-path 3D CNNs for calibration of system parameters for OCT-based motion compensation
optical coherence tomography (OCT) is an imaging modality which is used in medical interventions. However, performing motion compensation with OCT is problematic due to its small field of view. We propose a novel deep learning-based approach that directly learns input parameters of motor.
0
0
0
Mon Feb 26 2018
Computer Vision
i3PosNet: Instrument Pose Estimation from X-Ray in temporal bone surgery
i3PosNet infers position and orientation of instruments from images using a pose estimation network. It outperforms conventional image registration-based approaches reducing average and maximum errors by at least two thirds. i3Pos net generalizes to real x-rays without any further adaptation.
0
0
0
Thu Apr 26 2018
Computer Vision
Force Estimation from OCT Volumes using 3D CNNs
Estimating the interaction forces of instruments and tissue is of interest, particularly to provide haptic feedback during robot assisted interventions. Different approaches based on external andintegrated force sensors have been proposed. We propose a novel image-based force estimation method using optical coherence tomography.
0
0
0
Thu Mar 30 2017
Computer Vision
Concurrent Segmentation and Localization for Tracking of Surgical Instruments
Real-time instrument tracking is a crucial requirement for various computer-assisted interventions. In order to overcome problems such as specular reflections and motion blur, we propose a novel method that takes advantage of the interdependency between localization and segmentation.
0
0
0