Published on Mon Jul 05 2021

SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework for Person Re-Identification

Haocong Rao, Xiping Hu, Jun Cheng, Bin Hu

Person re-identification via 3D skeletons is an emerging topic with great potential in security-critical applications. Existing methods typically learn body and motion features from the body-joint trajectory. In this paper, we propose a Self-supervised Multi-scale Skeleton Graph Encoding.

2
0
0
Abstract

Person re-identification via 3D skeletons is an emerging topic with great potential in security-critical applications. Existing methods typically learn body and motion features from the body-joint trajectory, whereas they lack a systematic way to model body structure and underlying relations of body components beyond the scale of body joints. In this paper, we for the first time propose a Self-supervised Multi-scale Skeleton Graph Encoding (SM-SGE) framework that comprehensively models human body, component relations, and skeleton dynamics from unlabeled skeleton graphs of various scales to learn an effective skeleton representation for person Re-ID. Specifically, we first devise multi-scale skeleton graphs with coarse-to-fine human body partitions, which enables us to model body structure and skeleton dynamics at multiple levels. Second, to mine inherent correlations between body components in skeletal motion, we propose a multi-scale graph relation network to learn structural relations between adjacent body-component nodes and collaborative relations among nodes of different scales, so as to capture more discriminative skeleton graph features. Last, we propose a novel multi-scale skeleton reconstruction mechanism to enable our framework to encode skeleton dynamics and high-level semantics from unlabeled skeleton graphs, which encourages learning a discriminative skeleton representation for person Re-ID. Extensive experiments show that SM-SGE outperforms most state-of-the-art skeleton-based methods. We further demonstrate its effectiveness on 3D skeleton data estimated from large-scale RGB videos. Our codes are open at https://github.com/Kali-Hac/SM-SGE.

Sun Jun 06 2021
Computer Vision
Multi-Level Graph Encoding with Structural-Collaborative Relation Learning for Skeleton-Based Person Re-Identification
Skeleton-based person re-identification (Re-ID) is an emerging open topic providing great value for safety-critical applications. Existing methods typically extract hand-crafted features or model skeleton dynamics from the body joints. We construct graphs to model human skeletons from different levels, and for the first time propose a Multi-level Graph
1
0
0
Sat Sep 05 2020
Computer Vision
A Self-Supervised Gait Encoding Approach with Locality-Awareness for 3D Skeleton Based Person Re-Identification
Person re-identification (Re-ID) via gait features within 3D skeleton sequences is a newly-emerging topic with several advantages. Existing solutions rely on hand-crafted descriptors or supervised gait representationlearning. This paper proposes a self-supervised gait encoding approach that can leverage unlabeled skeleton data to learn
2
0
0
Fri Aug 21 2020
Computer Vision
Self-Supervised Gait Encoding with Locality-Aware Attention for Person Re-Identification
Gait-based person re-identification (Re-ID) is valuable for safety-critical applications. Existing methods eitheropt hand-crafted features or learn gait features by traditional supervised learning paradigms. We for the first time propose a generic gait encoding approach that can utilize unlabeled skeleton data.
0
0
0
Thu Apr 15 2021
Computer Vision
Spatial-Temporal Correlation and Topology Learning for Person Re-Identification in Videos
0
0
0
Wed Apr 14 2021
Computer Vision
Graph-based Person Signature for Person Re-Identifications
0
0
0
Wed Jan 27 2021
Computer Vision
GaitGraph: Graph Convolutional Network for Skeleton-Based Gait Recognition
Gait recognition is a promising video-based biometric for identifying individual walking patterns from a long distance. At present, most gait recognition methods use silhouette images to represent a person in each frame. We propose GaitGraph that combines skeleton poses with Graph Convolutional Network.
2
0
0
Mon Oct 30 2017
Machine Learning
Graph Attention Networks
Graph attention networks (GATs) are novel neural network architectures that operate on graph-structured data. GATs leverage masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions.
5
99
432
Mon Mar 09 2015
Machine Learning
Distilling the Knowledge in a Neural Network
A new type of ensemble composed of one or more full models and many specialist models. Unlike a mixture of experts, these specialist models can be trained rapidly and in parallel.
5
4
21
Wed Apr 04 2018
Computer Vision
Learning Discriminative Features with Multiple Granularities for Person Re-Identification
The Multiple Granularity Network (MGN) is a multi-branch deep network. It consists of one branch for global feature representations and two branches for local feature representations. The method has robustly achieved state-of-the-art performances and outperformed any existing approaches.
0
0
0
Tue Mar 17 2020
Machine Learning
Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction
0
0
0
Sat Sep 05 2020
Computer Vision
A Self-Supervised Gait Encoding Approach with Locality-Awareness for 3D Skeleton Based Person Re-Identification
Person re-identification (Re-ID) via gait features within 3D skeleton sequences is a newly-emerging topic with several advantages. Existing solutions rely on hand-crafted descriptors or supervised gait representationlearning. This paper proposes a self-supervised gait encoding approach that can leverage unlabeled skeleton data to learn
2
0
0
Tue Mar 28 2017
Computer Vision
Robust Depth-based Person Re-identification
Person re-identification (re-id) aims to match people across non-overlapping camera views. When people appeared in extreme illumination or changedclothes, the RGB appearance-based re-id methods tended to fail. To overcome this problem, we propose to exploit depth information
0
0
0