Published on Tue Jul 30 2019

SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition

Carlos Caetano, Jessica Sena, François Brémond, Jefersson A. dos Santos, William Robson Schwartz

SkeleMotion is a novel skeleton image representation to be used as input of Convolutional Neural Networks (CNNs) The proposed approach encodes the temporal dynamics by explicitly computing the magnitude and orientation values of the skeleton Joints. Different temporal scales are employed to compute motion values.

0
0
0
Abstract

Due to the availability of large-scale skeleton datasets, 3D human action recognition has recently called the attention of computer vision community. Many works have focused on encoding skeleton data as skeleton image representations based on spatial structure of the skeleton joints, in which the temporal dynamics of the sequence is encoded as variations in columns and the spatial structure of each frame is represented as rows of a matrix. To further improve such representations, we introduce a novel skeleton image representation to be used as input of Convolutional Neural Networks (CNNs), named SkeleMotion. The proposed approach encodes the temporal dynamics by explicitly computing the magnitude and orientation values of the skeleton joints. Different temporal scales are employed to compute motion values to aggregate more temporal dynamics to the representation making it able to capture longrange joint interactions involved in actions as well as filtering noisy motion values. Experimental results demonstrate the effectiveness of the proposed representation on 3D action recognition outperforming the state-of-the-art on NTU RGB+D 120 dataset.

Thu Mar 09 2017
Computer Vision
A New Representation of Skeleton Sequences for 3D Action Recognition
This paper presents a new method for 3D action recognition with skeleton sequences (i.e., 3D trajectories of human skeleton joints) The proposed method transforms each skeleton sequence into three clips each consisting of several frames for spatial temporal feature learning.
0
0
0
Wed Dec 26 2018
Computer Vision
Learning to Recognize 3D Human Action from A New Skeleton-based Representation Using Deep Convolutional Neural Networks
A new skeleton-based representation for 3D action recognition in videos. The proposed representation is able to represent spatio-temporal evolutions of complex 3D motions. It outperforms previous state-of-the-art approaches.
0
0
0
Wed Jul 18 2018
Computer Vision
Skeletal Movement to Color Map: A Novel Representation for 3D Action Recognition with Inception Residual Networks
We propose a novel skeleton-based representation for 3D action recognition in videos using Deep Convolutional Neural Networks. The SPMF (Skeleton Pose-Motion Feature) is built from two of the most important properties of a human action: postures and motions.
0
0
0
Thu Jul 06 2017
Computer Vision
Skeleton-based Action Recognition Using LSTM and CNN
Convolutional Neural Networks (CNN) and Long Short Term Memory (LSTM)-based learning methods have achieved promising performance for action recognition. Our method achieved state-of-the-art results on NTU RGB+Ddatasets for 3D human action analysis.
0
0
0
Tue May 02 2017
Computer Vision
Investigation of Different Skeleton Features for CNN-based 3D Action Recognition
CNN-based approaches can jointly capture spatio-temporal information from texture color images. The proposed method achieved state-of-the-art performance on NTU RGB+D dataset for 3D human action analysis.
0
0
0
Wed Sep 11 2019
Machine Learning
Skeleton Image Representation for 3D Action Recognition based on Tree Structure and Reference Joints
The proposed representation has the advantage of combining the use of use of reference joints and a tree structure skeleton. While the former incorporates different spatial relationships between the joints, the latter preserves important spatial relations by traversing a skeleton tree. Experimental results demonstrate the effectiveness of the proposed representation for 3D action
0
0
0