Published on Fri Apr 12 2019

Generative Hybrid Representations for Activity Forecasting with No-Regret Learning

Jiaqi Guan, Ye Yuan, Kris M. Kitani, Nicholas Rhinehart

Automatically reasoning about future human behaviors is a difficult problem. Part of this difficulty stems from learning systems' inability to represent all kinds of behaviors. Human behavior is generally not fixed: people can change their habits and routines. This suggests systems must be able to learn and adapt continuously.

0
0
0
Abstract

Automatically reasoning about future human behaviors is a difficult problem but has significant practical applications to assistive systems. Part of this difficulty stems from learning systems' inability to represent all kinds of behaviors. Some behaviors, such as motion, are best described with continuous representations, whereas others, such as picking up a cup, are best described with discrete representations. Furthermore, human behavior is generally not fixed: people can change their habits and routines. This suggests these systems must be able to learn and adapt continuously. In this work, we develop an efficient deep generative model to jointly forecast a person's future discrete actions and continuous motions. On a large-scale egocentric dataset, EPIC-KITCHENS, we observe our method generates high-quality and diverse samples while exhibiting better generalization than related generative models. Finally, we propose a variant to continually learn our model from streaming data, observe its practical effectiveness, and theoretically justify its learning efficiency.

Thu Dec 22 2016
Computer Vision
First-Person Activity Forecasting with Online Inverse Reinforcement Learning
We address the problem of incrementally modeling and forecasting long-term goals of a first-person camera wearer. In contrast to prior work in trajectory forecasting, our algorithm, DARKO, goes further to reason about semantic states and future goal states.
0
0
0
Mon Apr 10 2017
Computer Vision
Learning Human Motion Models for Long-term Predictions
We propose a new architecture for the learning of predictive spatio-temporal motion models from data alone. Our approach, dubbed the Dropout Autoencoder, is capable of synthesizing natural looking motion sequences over long time horizons without catastrophic drift or motion degradation.
0
0
0
Tue Aug 11 2020
Computer Vision
Adversarial Generative Grammars for Human Activity Prediction
The adversarial generative grammar is designed so that it can learn stochastic production rules from the data distribution. The proposed adversarial grammar outperforms the state-of-the-art approaches, being able to predict much more accurately and further in the future.
0
0
0
Fri Jun 12 2020
Machine Learning
Learning-to-Learn Personalised Human Activity Recognition Models
Human Activity Recognition (HAR) is the classification of human movement. State-of-the-art methods of HAR rely on having access to a considerable amount of labelled data. We present a meta-learning methodology for learning to learn personalised HAR models for HAR.
0
0
0
Tue Dec 01 2020
Computer Vision
We are More than Our Joints: Predicting how 3D Bodies Move
A key step towards understanding human behavior is the prediction of 3D human motion. Successful solutions have many applications in human tracking, HCI, and HCI. Most previous work focuses on predicting a time series of future 3Djoint locations given a sequence 3D joints from the past.
0
0
0
Wed Mar 18 2020
Machine Learning
DLow: Diversifying Latent Flows for Diverse Human Motion Prediction
Deep generative models are often used for human motion prediction. How to efficiently produce diverse samples from a deep generative model after it has been trained is still an under-explored problem. We propose a novel sampling method, Diversifying Latent Flows, to produce a diverse set of samples.
0
0
0