Published on Wed May 05 2021

Learning Feature Aggregation for Deep 3D Morphable Models

Zhixiang Chen, Tae-Kyun Kim

3D morphable models are widely used for the shape representation of an object in computer vision and graphics applications. We focus on deep learning on 3D mesh data with a hierarchical structure to capture information at multiple scales.

1
0
0
Abstract

3D morphable models are widely used for the shape representation of an object class in computer vision and graphics applications. In this work, we focus on deep 3D morphable models that directly apply deep learning on 3D mesh data with a hierarchical structure to capture information at multiple scales. While great efforts have been made to design the convolution operator, how to best aggregate vertex features across hierarchical levels deserves further attention. In contrast to resorting to mesh decimation, we propose an attention based module to learn mapping matrices for better feature aggregation across hierarchical levels. Specifically, the mapping matrices are generated by a compatibility function of the keys and queries. The keys and queries are trainable variables, learned by optimizing the target objective, and shared by all data samples of the same object class. Our proposed module can be used as a train-only drop-in replacement for the feature aggregation in existing architectures for both downsampling and upsampling. Our experiments show that through the end-to-end training of the mapping matrices, we achieve state-of-the-art results on a variety of 3D shape datasets in comparison to existing morphable models.

Wed Aug 07 2019
Machine Learning
Mesh Variational Autoencoders with Edge Contraction Pooling
3D shape analysis is an important research topic in computer vision and computer graphics. While existing methods have generalized image-based deep learning tosheets using graph-based convolutions, the lack of an effective pooling operation restricts the learning capability of their networks.
0
0
0
Sat Oct 17 2020
Machine Learning
MeshMVS: Multi-View Stereo Guided Mesh Reconstruction
Deep learning based 3D shape generation methods generally utilize latent features extracted from color images to encode the semantics of objects. These color image semantics only implicitly encode 3D information, potentially limiting the accuracy of the generated shapes. In this paper we propose a multi-view mesh generation method
0
0
0
Thu May 27 2021
Machine Learning
MeshCNN Fundamentals: Geometric Learning through a Reconstructable Representation
Mesh-based learning is one of the popular approaches nowadays to learn shapes. In this paper, we propose infusing MeshCNN with geometric reasoning to achieve higher quality learning.
7
0
1
Mon Aug 05 2019
Computer Vision
Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation
We study the problem of shape generation in 3D mesh representation from a few color images with known camera poses. Instead of building a direct mapping function from images to 3D shape, our model learns to predict series of deformations to improve a coarse shape.
0
0
0
Fri Sep 22 2017
Artificial Intelligence
Hierarchical Detail Enhancing Mesh-Based Shape Generation with 3D Generative Adversarial Network
The study addresses the needs for automatic generation of mesh-based geometries. It proposes a novel framework that utilizes signed distance function representation. The framework generates detail preserving three-dimensional surface mesh by a deep learning based approach.
0
0
0
Wed Aug 12 2020
Computer Vision
DSG-Net: Learning Disentangled Structure and Geometry for 3D Shape Generation
DSG-Net is a deep neural network that learns a disentangled structured and geometric mesh representation for 3D shapes. Two key key aspects of shapes, geometry, and structure, are encoded in a synergistic manner to ensure plausibility of the generated shapes.
2
0
0
Mon Jun 12 2017
NLP
Attention Is All You Need
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms. Experiments on two machine translation tasks show these models to be superior in
51
215
883
Mon Oct 30 2017
Machine Learning
Graph Attention Networks
Graph attention networks (GATs) are novel neural network architectures that operate on graph-structured data. GATs leverage masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions.
5
99
432
Sun Jun 30 2019
Machine Learning
Spectral Clustering with Graph Neural Networks for Graph Pooling
Spectral clustering (SC) is a popular clustering technique to find strongly connected communities on a graph. SC can be used in Graph Neural Networks (GNNs) to implement pooling operations that aggregate nodes belonging to the same cluster.
1
5
31
Fri Sep 09 2016
Machine Learning
Semi-Supervised Classification with Graph Convolutional Networks
We present a scalable approach for semi-supervised learning on graph-structured data. The approach is based on an efficient variant of convolutional neural networks. We motivate the choice of our architecture via a localized first-order approximation of graph convolutions.
3
1
2
Sun Mar 03 2019
Computer Vision
3D Hand Shape and Pose Estimation from a Single RGB Image
Most current methods in 3D hand.analysis from monocular RGB images only focus on estimating the 3D locations of keypoints. We propose a Graph Convolutional Neural Network (Graph CNN) based method to construct a full 3D mesh of hand surface.
0
0
0
Sat Dec 21 2013
Neural Networks
Spectral Networks and Locally Connected Networks on Graphs
Convolutional Neural Networks are extremely efficient architectures in image and audio recognition tasks. We show through experiments that for low-dimensional graphs it is possible to learn convolutional layers with a number of parameters independent of the input size.
0
0
0