Published on Fri Dec 20 2019

JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds

Lin Zhao, Wenbing Tao

feature fusion module is proposed to fuse different layer features of the backbone network. A joint instance semantic segmentation module is developed to transform semantic features into instance embedding space, and further fused with instance features to facilitate instance segmentation.

0
0
0
Abstract

In this paper, we propose a novel joint instance and semantic segmentation approach, which is called JSNet, in order to address the instance and semantic segmentation of 3D point clouds simultaneously. Firstly, we build an effective backbone network to extract robust features from the raw point clouds. Secondly, to obtain more discriminative features, a point cloud feature fusion module is proposed to fuse the different layer features of the backbone network. Furthermore, a joint instance semantic segmentation module is developed to transform semantic features into instance embedding space, and then the transformed features are further fused with instance features to facilitate instance segmentation. Meanwhile, this module also aggregates instance features into semantic feature space to promote semantic segmentation. Finally, the instance predictions are generated by applying a simple mean-shift clustering on instance embeddings. As a result, we evaluate the proposed JSNet on a large-scale 3D indoor point cloud dataset S3DIS and a part dataset ShapeNet, and compare it with existing approaches. Experimental results demonstrate our approach outperforms the state-of-the-art method in 3D instance segmentation with a significant improvement in 3D semantic prediction and our method is also beneficial for part segmentation. The source code for this work is available at https://github.com/dlinzhao/JSNet.

Tue Feb 26 2019
Computer Vision
Associatively Segmenting Instances and Semantics in Point Clouds
A 3D point cloud describes the real scene precisely and intuitively. To date, how to segment diversified elements in such an informative 3D scene is rarely discussed. In this paper, we introduce a simple and flexible framework to segment instances and semantics in point clouds simultaneously.
0
0
0
Sun Mar 01 2020
Computer Vision
3DCFS: Fast and Robust Joint 3D Semantic-Instance Segmentation via Coupled Feature Selection
We propose a novel fast and robust 3D point clouds segmentation framework via a coupled feature selection module. Our 3DCFS outperforms state-of-the-art methods in terms of accuracy, speed and computational cost.
0
0
0
Mon Apr 01 2019
Computer Vision
JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds with Multi-Task Pointwise Networks and Multi-Value Conditional Random Fields
Deep learning techniques have become the to-go models for most vision-related tasks on 2D images. However, their power has not been fully realised on several tasks in 3D space. In this work, we jointly address the problems of semantic and instance segmentation of 3D point clouds.
0
0
0
Wed Apr 03 2019
Computer Vision
3D-BEVIS: Bird's-Eye-View Instance Segmentation
Recent deep learning models achieve impressive results by operating directly on unstructured point clouds. Current point-based methods scale linearly with the number of points by processing local sub-parts of a scene individually. To perform instance segmentation by clustering, global consistent features are required.
0
0
0
Mon Jul 27 2020
Computer Vision
Self-Prediction for Joint Instance and Semantic Segmentation of Point Clouds
We develop a novel learning scheme named Self-Prediction for 3D instance and semantic segmentation of point clouds. Our method designs a new learning scheme to enhance point relation exploring for better segmentation. We use label propagation to predict labels of one subset when given labels of the other.
0
0
0
Fri Sep 27 2019
Machine Learning
Rethinking Task and Metrics of Instance Segmentation on 3D Point Clouds
Instance segmentation on 3D point clouds is one of the most extensively researched areas toward the realization of autonomous cars and robots. Certain existing studies have split input point clouds into small regions such as 1m x1m.
0
0
0