Published on Mon May 10 2021

HuMoR: 3D Human Motion Model for Robust Pose Estimation

Davis Rempe, Tolga Birdal, Aaron Hertzmann, Jimei Yang, Srinath Sridhar, Leonidas J. Guibas

HuMoR: a 3D Human Motion Model for Robust Estimation of temporal pose and shape. We demonstrate that our model generalizes to diverse motions and body shapes after training on a large motion capture dataset.

1
10
46
Abstract

We introduce HuMoR: a 3D Human Motion Model for Robust Estimation of temporal pose and shape. Though substantial progress has been made in estimating 3D human motion and shape from dynamic observations, recovering plausible pose sequences in the presence of noise and occlusions remains a challenge. For this purpose, we propose an expressive generative model in the form of a conditional variational autoencoder, which learns a distribution of the change in pose at each step of a motion sequence. Furthermore, we introduce a flexible optimization-based approach that leverages HuMoR as a motion prior to robustly estimate plausible pose and shape from ambiguous observations. Through extensive evaluations, we demonstrate that our model generalizes to diverse motions and body shapes after training on a large motion capture dataset, and enables motion reconstruction from multiple input modalities including 3D keypoints and RGB(-D) videos.

Mon Jun 07 2021
Computer Vision
Task-Generic Hierarchical Human Motion Prior using VAEs
A deep generative model that describes human motions can benefit a wide range of computer vision and graphics tasks. It can provide robustness to video-based human pose estimation, predicting complete body movements for motion capture systems during occlusions, and assisting key frame animation.
3
8
52
Tue Dec 01 2020
Computer Vision
We are More than Our Joints: Predicting how 3D Bodies Move
A key step towards understanding human behavior is the prediction of 3D human motion. Successful solutions have many applications in human tracking, HCI, and HCI. Most previous work focuses on predicting a time series of future 3Djoint locations given a sequence 3D joints from the past.
0
0
0
Wed Dec 18 2019
Machine Learning
Contextually Plausible and Diverse 3D Human Motion Prediction
We tackle the task of diverse 3D human motion prediction. We condition the sampling of latent variable that acts as source of diversity on the representation of the past observation. Our approach yields motions of higher quality while retaining diversity.
0
0
0
Sun Aug 09 2020
Computer Vision
3D Human Motion Estimation via Motion Compression and Refinement
Method decomposes a temporal sequence of human motion into a smooth motion representation. Two-step encoding of human motion captures human motion in two stages. Method produces both smooth and accurate 3D human pose and motion estimates.
0
0
0
Wed Dec 11 2019
Computer Vision
VIBE: Video Inference for Human Body Pose and Shape Estimation
Video Inference for Body Pose and Shape Estimation (VIBE) makes use of an existing large-scale motion capture dataset. VIBE leverages AMASS to discriminate between real human motions and those produced by our temporal pose and shape regression networks.
1
0
0
Mon Nov 30 2020
Machine Learning
Forecasting Characteristic 3D Poses of Human Actions
We propose the task of forecasting characteristic 3D poses. From a monocular video observation of a person, we can predict a future 3D pose of that person in a likely action-defining, characteristic pose. Our experiments with this dataset suggest that our proposed probabilistic approach outperforms
0
0
0
Wed Jul 22 2020
Computer Vision
Contact and Human Dynamics from Monocular Video
Existing deep models predict 2D and 3D kinematic poses from video that are approximately accurate, but contain visible errors that violate physical constraints. We present a physics-based method for inferring 3D human motion from video sequences.
1
140
514
Tue Jun 19 2018
Machine Learning
Neural Ordinary Differential Equations
We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is thencomputed using a black-box differential equation solver.
3
4
29
Sat Apr 06 2019
Computer Vision
Context-aware Human Motion Prediction
The problem of predicting human motion given a sequence of past observations is at the core of many applications in robotics and computer vision. Current state-of-the-art formulate this problem as a sequence-to-sequence task, in which a historical of 3D skeletons feeds a Recurrent Neural
1
5
8
Thu Nov 06 2014
Machine Learning
Conditional Generative Adversarial Nets
Generative Adversarial Nets [8] were recently introduced as a novel way to train generative models. In this work we introduce the conditional version of these nets, which can be constructed by simply feeding the data. We show that this model can generate MNIST digits conditioned on class labels.
2
0
3
Thu Nov 19 2015
Machine Learning
Generating Sentences from a Continuous Space
The standard recurrent neural network language model (RNNLM) generates sentences one word at a time and does not work from an explicit global sentence representation. In this work, we introduce and study an RNN-based variational autoencoder generative model that incorporates distributed
1
0
1
Mon Dec 22 2014
Machine Learning
Adam: A Method for Stochastic Optimization
Adam is an algorithm for first-order gradient-based optimization of stochastic objective functions. The method is straightforward to implement and has little memory requirements. It is well suited for problems that are large in terms of data and parameters.
2
0
1