Published on Fri Mar 05 2021

Real-time RGBD-based Extended Body Pose Estimation

Renat Bashirov, Anastasia Ianina, Karim Iskakov, Yevgeniy Kononenko, Valeriya Strizhkova, Victor Lempitsky, Alexander Vakhitov
0
0
0
Abstract

We present a system for real-time RGBD-based estimation of 3D human pose. We use parametric 3D deformable human mesh model (SMPL-X) as a representation and focus on the real-time estimation of parameters for the body pose, hands pose and facial expression from Kinect Azure RGB-D camera. We train estimators of body pose and facial expression parameters. Both estimators use previously published landmark extractors as input and custom annotated datasets for supervision, while hand pose is estimated directly by a previously published method. We combine the predictions of those estimators into a temporally-smooth human pose. We train the facial expression extractor on a large talking face dataset, which we annotate with facial expression parameters. For the body pose we collect and annotate a dataset of 56 people captured from a rig of 5 Kinect Azure RGB-D cameras and use it together with a large motion capture AMASS dataset. Our RGB-D body pose model outperforms the state-of-the-art RGB-only methods and works on the same level of accuracy compared to a slower RGB-D optimization-based solution. The combined system runs at 30 FPS on a server with a single GPU. The code will be available at https://saic-violet.github.io/rgbd-kinect-pose

Tue Nov 29 2016
Computer Vision
Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision
CNN-based approach for 3D human body pose estimation from single RGB images. We show state-of-the-art performance on established benchmarks through transfer of learned features. We also contribute a new benchmark that covers outdoor and indoor scenes.
0
0
0
Thu Apr 11 2019
Computer Vision
Expressive Body Capture: 3D Hands, Face, and Body from a Single Image
We create a new, unified, 3D model of the human body, SMPL-X, that extends SMPL with fully articulated hands and an expressive face. We evaluate 3D accuracy on a new curated dataset comprising 100 images with pseudo ground-truth. This is a step toward automatic expressive human capture from monocular RGB data.
0
0
0
Sat Nov 29 2014
Computer Vision
Egocentric Pose Recognition in Four Lines of Code
We tackle the problem of estimating the 3D pose of an individual's upper thighs (arms+hands) from a chest mounted depth-camera. Our method provides state-of-the-art hand pose recognition performance from egocentric RGB-D images in real-time.
0
0
0
Tue Dec 18 2018
Computer Vision
OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
OpenPose is the first open-source system for multi-person 2D pose detection. It uses a nonparametric representation to learn to associate body parts with individuals in the image. The bottom-up system achieves high accuracy and realtime performance.
0
0
0
Sat Dec 09 2017
Computer Vision
Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB
We propose a new single-shot method for multi-person 3D pose estimation in general scenes from a monocular RGB camera. Our approach uses novel occlusion-robust pose-maps (ORPM) which enable full body pose inference even under strong partial occlusions.
1
0
1
Sun Oct 16 2016
Computer Vision
Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input
Real-time simultaneous tracking of hands manipulating and interacting with external objects has many potential applications in augmented reality, tangible computing, and wearable computing. Jointly tracking hand and object pose is more challenging than tracking either of the two separately.
0
0
0