Published on Wed Feb 01 2017

Visual Saliency Prediction Using a Mixture of Deep Neural Networks

Samuel Dodge, Lina Karam

Most existing models predict saliency using local mechanisms limited to the receptive field of the network. We propose a model that incorporates global scene semantic information in addition to local information gathered by a convolutional neural network. Our model is formulated as a mixture of viewpoints.

0
0
0
Abstract

Visual saliency models have recently begun to incorporate deep learning to achieve predictive capacity much greater than previous unsupervised methods. However, most existing models predict saliency using local mechanisms limited to the receptive field of the network. We propose a model that incorporates global scene semantic information in addition to local information gathered by a convolutional neural network. Our model is formulated as a mixture of experts. Each expert network is trained to predict saliency for a set of closely related images. The final saliency map is computed as a weighted mixture of the expert networks' output, with weights determined by a separate gating network. This gating network is guided by global scene information to predict weights. The expert networks and the gating network are trained simultaneously in an end-to-end manner. We show that our mixture formulation leads to improvement in performance over an otherwise identical non-mixture model that does not incorporate global scene information.

Fri Jan 12 2018
Computer Vision
Deep saliency: What is learnt by a deep network about saliency?
This article examines the specific problem. of saliency detection. It investigates the properties of the learnt representation by visualizing the artificial neurons' receptive fields.
0
0
0
Wed Mar 02 2016
Machine Learning
Shallow and Deep Convolutional Networks for Saliency Prediction
The prediction of salient areas in images has been traditionally addressed with hand-crafted features based on neuroscience principles. This paper addresses the problem with a completely data-driven approach by training a convolutional neural network.
0
0
0
Mon Oct 08 2018
Computer Vision
Saliency Prediction in the Deep Learning Era: Successes, Limitations, and Future Challenges
Visual saliency models have enjoyed a big leap in performance in recent years, thanks to advances in deep learning and large scale annotated data. Despite enormous effort and huge breakthroughs, models still fall short in reaching human-level accuracy.
0
0
0
Mon Jul 06 2015
Neural Networks
End-to-end Convolutional Network for Saliency Prediction
The convolutional network in this paper, named JuntingNet, won the LSUN 2015 challenge on saliency prediction with a superior performance in all considered metrics. The learning process is formulated as a minimization of a loss function.
0
0
0
Tue Aug 25 2020
Machine Learning
FastSal: a Computationally Efficient Network for Visual Saliency Prediction
This paper focuses on the problem of visual saliency prediction, predicting regions of an image that tend to attract human visual attention. We modify and test various efficient convolutional neural network architectures like EfficientNet and MobileNetV2.
0
0
0
Mon Sep 05 2016
Computer Vision
A Deep Multi-Level Network for Saliency Prediction
This paper presents a novel deep architecture for saliency prediction. The model is composed of three main blocks: a feature extraction CNN, a feature encoding network, and a prior learning network. Results show that our model outperforms under all evaluation metrics.
0
0
0