Published on Fri Nov 30 2018

TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes

Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Nießner, Leonidas Guibas

TextureNet is a neural network architecture designed to extract features from high-resolution signals associated with 3D surface meshes. The key idea is to utilize a 4-rotational symmetric field to define a domain for convolution on a surface.

0
0
0
Abstract

We introduce, TextureNet, a neural network architecture designed to extract features from high-resolution signals associated with 3D surface meshes (e.g., color texture maps). The key idea is to utilize a 4-rotational symmetric (4-RoSy) field to define a domain for convolution on a surface. Though 4-RoSy fields have several properties favorable for convolution on surfaces (low distortion, few singularities, consistent parameterization, etc.), orientations are ambiguous up to 4-fold rotation at any sample point. So, we introduce a new convolutional operator invariant to the 4-RoSy ambiguity and use it in a network to extract features from high-resolution signals on geodesic neighborhoods of a surface. In comparison to alternatives, such as PointNet based methods which lack a notion of orientation, the coherent structure given by these neighborhoods results in significantly stronger features. As an example application, we demonstrate the benefits of our architecture for 3D semantic segmentation of textured 3D meshes. The results show that our method outperforms all existing methods on the basis of mean IoU by a significant margin in both geometry-only (6.4%) and RGB+Geometry (6.9-8.2%) settings.

Tue Dec 01 2020
Computer Vision
DiffusionNet: Discretization Agnostic Learning on Surfaces
We introduce a new approach to deep learning on 3D surfaces. A simple diffusion layer is highly effective for spatial communication. The resulting networks automatically generalize across different resolutions.
6
0
2
Wed Aug 15 2018
Computer Vision
PFCNN: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames
Surface meshes are widely used shape representations and capture finer data than point clouds or volumetric grids. They are challenging to apply CNNs directly due to their non-Euclidean structure. We use parallel frames on surface to define PFCNNs that enable effective feature learning.
1
0
1
Thu Apr 02 2020
Computer Vision
DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes
DualConvMesh-Nets (DCM-Net) is a family of deep hierarchical convolutional networks over 3D geometric data. It combines two types of convolutions: geodesic and Euclidean.
0
0
0
Sun Sep 16 2018
Machine Learning
MeshCNN: A Network with an Edge
Polygonal meshes provide an efficient representation for 3D shapes. They leverage non-uniformity to represent large flat regions as well as sharp, intricate features. This inhibits mesh analysis efforts using neural networks that combine convolution and pooling operations.
0
0
0
Fri Jun 04 2021
Computer Vision
Subdivision-Based Mesh Convolution Networks
Convolutional neural networks (CNNs) have made great breakthroughs in 2D computer vision. But the irregular structure of meshes makes it hard to exploit the power of CNNs directly. This paper introduces a novel and flexible CNN framework, named SubdivNet.
2
0
1
Tue Jun 09 2020
Machine Learning
MeshWalker: Deep Mesh Understanding by Random Walks
Most attempts to represent 3D shapes for deep learning have focused on volumetric grids, multi-view images and point clouds. This paper proposes a very different approach, termed MeshWalker, to learn the shape directly from a given mesh.
0
0
0