Published on Sun Aug 16 2020

Neural Descent for Visual 3D Human Pose and Shape

Andrei Zanfir, Eduard Gabriel Bazavan, Mihai Zanfir, William T. Freeman, Rahul Sukthankar, Cristian Sminchisescu

We present deep neural network methodology to reconstruct the 3d pose and shape of people, given an input RGB image. We rely on a recently introduced,expressivefull body statistical 3d human model, GHUM, trained end-to-end. Central to our methodology, is a learning

0
0
0
Abstract

We present deep neural network methodology to reconstruct the 3d pose and shape of people, given an input RGB image. We rely on a recently introduced, expressivefull body statistical 3d human model, GHUM, trained end-to-end, and learn to reconstruct its pose and shape state in a self-supervised regime. Central to our methodology, is a learning to learn and optimize approach, referred to as HUmanNeural Descent (HUND), which avoids both second-order differentiation when training the model parameters,and expensive state gradient descent in order to accurately minimize a semantic differentiable rendering loss at test time. Instead, we rely on novel recurrent stages to update the pose and shape parameters such that not only losses are minimized effectively, but the process is meta-regularized in order to ensure end-progress. HUND's symmetry between training and testing makes it the first 3d human sensing architecture to natively support different operating regimes including self-supervised ones. In diverse tests, we show that HUND achieves very competitive results in datasets like H3.6M and 3DPW, aswell as good quality 3d reconstructions for complex imagery collected in-the-wild.

Fri Jun 26 2020
Machine Learning
Making DensePose fast and light
DensePose estimation task is a significant step forward for enhancing user experience. Existing neural network models capable of solving this task areavily parameterized. To enable Dense Pose inference on the end device with current models, one needs to support an expensive server-side infrastructure.
0
0
0
Thu Jun 17 2021
Computer Vision
THUNDR: Transformer-based 3D HUmaN Reconstruction with Markers
ThUNDR is a transformer-based deep neural network methodology. It reconstructs the 3d pose and shape of people, given monocular RGB images. We show state-of-the-art results on Human3.6M and 3DPW.
1
0
3
Mon Mar 25 2019
Computer Vision
DenseBody: Directly Regressing Dense 3D Human Pose and Shape From a Single Color Image
Recovering 3D human body shape and pose from 2D images is a challenging task due to high complexity and flexibility of human body. Previous methods addressing these issues typically rely on body part segmentation and 2D/3D joints. In this paper, we propose to directly regress the 3
0
0
0
Sat Jun 15 2019
Computer Vision
Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era
3D reconstruction is a longstanding ill-posed problem, which has been explored for decades. Since 2015, image-based 3D reconstruction using convolutional neural networks (CNN) has attracted increasing interest. This article provides a comprehensive survey of the recent developments in this field.
1
5
38
Wed Apr 01 2020
Computer Vision
PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
0
0
0
Thu Mar 14 2019
Computer Vision
Learning to Reconstruct People in Clothing from a Single RGB Camera
We present a learning-based model to infer the personalized 3D shape of people from a few frames (1-8) of a monocular video. The model learns to predict the parameters of a statistical body model and instance displacements that add clothing and hair to the shape.
1
43
132
Sat Apr 11 2020
Machine Learning
Meta-Learning in Neural Networks: A Survey
The field of meta-learning, or learning-to-learn, has seen a dramatic rise in interest in recent years. Contrary to conventional approaches to AI where tasks are solved from scratch using a fixed learning algorithm, meta- learning aims to improve the learning algorithm itself.
1
90
462
Tue Jun 14 2016
Neural Networks
Learning to learn by gradient descent by gradient descent
The move from hand-designed features to learned features in machine learning has been wildly successful. In spite of this, optimization algorithms are still designed by hand. In this paper we show how the design of an optimization algorithm can be cast as a learning problem.
1
0
1
Mon Dec 17 2018
Machine Learning
On the Continuity of Rotation Representations in Neural Networks
In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. We show that the 3Drotations have continuous representations in 5D and 6D, which are more suitable for learning.
2
0
1
Tue Sep 11 2018
Computer Vision
3D Human Body Reconstruction from a Single Image via Volumetric Regression
This paper proposes the use of an end-to-end Convolutional Neural Network for direct reconstruction of the 3D geometry of humans via volumetric regression. The proposed method does not require the fitting of a shape model.
0
0
0
Wed May 08 2019
Computer Vision
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction
This paper addresses the problem of 3D human pose and shape estimation from a single image. Previous approaches consider a parametric model of the human body. Instead of predicting model parameters, we directly regress the 3D location of the mesh vertices. This architecture allows us to explicitly encode the template mesh structure within the network.
0
0
0
Thu May 01 2014
Computer Vision
Microsoft COCO: Common Objects in Context
The dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement.
1
0
0