Published on Mon Oct 19 2020

Learning to Reconstruct and Segment 3D Objects

Bo Yang

To endow machines with the ability to perceive the real-world in a three dimensional representation as we do as humans is a fundamental and long-standing topic in Artificial Intelligence. We aim to understand scenes and the objects within them by learning general and robust representations using deep neural networks.

0
0
0
Abstract

To endow machines with the ability to perceive the real-world in a three dimensional representation as we do as humans is a fundamental and long-standing topic in Artificial Intelligence. Given different types of visual inputs such as images or point clouds acquired by 2D/3D sensors, one important goal is to understand the geometric structure and semantics of the 3D environment. Traditional approaches usually leverage hand-crafted features to estimate the shape and semantics of objects or scenes. However, they are difficult to generalize to novel objects and scenarios, and struggle to overcome critical issues caused by visual occlusions. By contrast, we aim to understand scenes and the objects within them by learning general and robust representations using deep neural networks, trained on large-scale real-world 3D data. To achieve these aims, this thesis makes three core contributions from object-level 3D shape estimation from single or multiple views to scene-level semantic understanding.

Sat Aug 04 2018
Computer Vision
A survey on Deep Learning Advances on Different 3D Data Representations
3D data is a valuable asset the computer vision filed as it provides rich information about the full geometry of sensed objects and scenes. With the availability of both large 3D datasets and computational power, it is now possible to consider applying deep learning to learn specific tasks on 3D data.
0
0
0
Mon Sep 18 2017
Computer Vision
Wide and deep volumetric residual networks for volumetric image classification
3D shape models that directly classify objects from 3D information have become more widely implementable. Current state of the art models rely on deep convolutional and inception models that are resource intensive. Residual neural networks have been demonstrated to be easier to optimize.
0
0
0
Thu Dec 08 2016
Computer Vision
3D Shape Segmentation with Projective Convolutional Networks
This paper introduces a deep architecture for segmenting 3D objects into their labeled semantic parts. The architecture combines image-based Fully Convolutional Networks (FCNs) and surface-based Conditional Random Fields (CRFs)
0
0
0
Fri Sep 03 2021our pick
Computer Vision
Representing Shape Collections with Alignment-Aware Linear Models
In this paper, we revisit the classical representation of 3D point clouds as linear shape models. Our key insight is to leverage deep learning to represent a collection of shapes as affine transformations of low-dimensional linear models.
0
0
0
Sun Jun 22 2014
Computer Vision
3D ShapeNets: A Deep Representation for Volumetric Shapes
3D shape is a crucial but heavily underutilized cue in today's computer vision systems. We propose to represent a geometric 3D shape as a probability distribution of binary variables on a 3D voxel grid.
0
0
0
Tue Mar 24 2020
Machine Learning
Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction
Deep Local Shapes (DeepLS) is a deep shape representation that enables encoding and reconstruction of high-quality 3D shapes without prohibitive memory requirements. It replaces the dense volumetric signed distance function (SDF) representation used in traditional surface reconstruction systems.
0
0
0