Published on Thu Mar 24 2016

Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks

Wentao Zhu, Cuiling Lan, Junliang Xing, Wenjun Zeng, Yanghao Li, Li Shen, Xiaohui Xie

Skeleton based action recognition distinguishes human actions using the co-occurrences of skeleton joints. We propose an end-to-end fully connected fully connected LSTM network. Experimental results on three human action recognition datasets consistently demonstrate the effectiveness of the proposed model.

0
0
0
Abstract

Skeleton based action recognition distinguishes human actions using the trajectories of skeleton joints, which provide a very good representation for describing actions. Considering that recurrent neural networks (RNNs) with Long Short-Term Memory (LSTM) can learn feature representations and model long-term temporal dependencies automatically, we propose an end-to-end fully connected deep LSTM network for skeleton based action recognition. Inspired by the observation that the co-occurrences of the joints intrinsically characterize human actions, we take the skeleton as the input at each time slot and introduce a novel regularization scheme to learn the co-occurrence features of skeleton joints. To train the deep LSTM network effectively, we propose a new dropout algorithm which simultaneously operates on the gates, cells, and output responses of the LSTM neurons. Experimental results on three human action recognition datasets consistently demonstrate the effectiveness of the proposed model.

Mon Jun 26 2017
Computer Vision
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates
Skeleton-based human action recognition has attracted a lot of research attention during the past few years. The proposed work extends this idea to spatial domain as well as temporal domain to better analyze the hidden sources of action-related information within the human skeleton sequences.
0
0
0
Tue Jul 18 2017
Computer Vision
Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networks
Human action recognition in 3D skeleton sequences has attracted a lot of research attention. Long Short-Term Memory (LSTM) networks have shown promising performance in this task. Not all skeletal joints areformative for action recognition, and the irrelevant joints often bring noise.
0
0
0
Fri Nov 18 2016
Computer Vision
An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data
Human action recognition is an important task in computer vision. We propose an end-to-end spatial and temporal attention model for human action recognition. We build our model on top of the Recurrent Neural Networks (RNNs) with Long Short-Term Memory (LSTM)
0
0
0
Sun Apr 09 2017
Computer Vision
Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks
Traditional approaches based on handcrafted features are limited to represent the complexity of motion patterns. Recent methods that use Recurrent Neural Networks (RNN) to handle raw skeletons only focus on the contextual dependency in the temporal domain and neglect the spatial configurations of skeletons.
0
0
0
Tue Jul 30 2019
Machine Learning
SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition
SkeleMotion is a novel skeleton image representation to be used as input of Convolutional Neural Networks (CNNs) The proposed approach encodes the temporal dynamics by explicitly computing the magnitude and orientation values of the skeleton Joints. Different temporal scales are employed to compute motion values.
0
0
0
Mon May 07 2018
Computer Vision
Relational Network for Skeleton-Based Action Recognition
Skeleton-based action recognition has attracted much attention recently. Most existing methods use Convolutional Neural Network (CNN) to extract spatio-temporal information embedded in the skeleton sequences for action recognition. We introduce the Recurrent Relational Network to learn the spatial features in a single skeleton.
0
0
0