Published on Wed Jan 20 2016

QUOTE: "Querying" Users as Oracles in Tag Engines - A Semi-Supervised Learning Approach to Personalized Image Tagging

Amandianeze O. Nwana, Tsuhan Chen

A common trend in image tagging research is to focus on visually relevant tags. This tends to ignore the personal and social aspect of tags, especially on photoblogging websites such as Flickr. Previous work has identified that many of the tags that users provide on images are not visually relevant.

0
0
0
Abstract

One common trend in image tagging research is to focus on visually relevant tags, and this tends to ignore the personal and social aspect of tags, especially on photoblogging websites such as Flickr. Previous work has correctly identified that many of the tags that users provide on images are not visually relevant (i.e. representative of the salient content in the image) and they go on to treat such tags as noise, ignoring that the users chose to provide those tags over others that could have been more visually relevant. Another common assumption about user generated tags for images is that the order of these tags provides no useful information for the prediction of tags on future images. This assumption also tends to define usefulness in terms of what is visually relevant to the image. For general tagging or labeling applications that focus on providing visual information about image content, these assumptions are reasonable, but when considering personalized image tagging applications, these assumptions are at best too rigid, ignoring user choice and preferences. We challenge the aforementioned assumptions, and provide a machine learning approach to the problem of personalized image tagging with the following contributions: 1.) We reformulate the personalized image tagging problem as a search/retrieval ranking problem, 2.) We leverage the order of tags, which does not always reflect visual relevance, provided by the user in the past as a cue to their tag preferences, similar to click data, 3.) We propose a technique to augment sparse user tag data (semi-supervision), and 4.) We demonstrate the efficacy of our method on a subset of Flickr images, showing improvement over previous state-of-art methods.

Tue Dec 06 2016
Computer Vision
Tag Prediction at Flickr: a View from the Darkroom
Automated photo tagging is one of the most compelling applications of deep learning. We present our efforts in developing a large-scale photo tagging system for Flickr photo search.
0
0
0
Tue Jul 07 2020
Computer Vision
Location Sensitive Image Retrieval and Tagging
LocSens is a model that learns to rank triplets of images, tags and coordinates by plausibility. LocSens learns to fuse textual and location information at different levels of granularity.
0
0
0
Tue May 31 2016
Computer Vision
Fast Zero-Shot Image Tagging
The word vectors of relevant tags for a given image rank ahead of the irrelevant tags, along a principal direction in the word vector space. We arrive at a quite versatile tagging model that runs fast given a test image, in constant time w.r.t.
0
0
0
Thu Apr 12 2007
Artificial Intelligence
Personalizing Image Search Results on Flickr
Flickr allows users to upload photos, annotate them with tags, submit them to groups, and also to form social networks by adding other users as contacts. We show how to exploit this metadata to personalize search results for the user, thereby improving search performance.
0
0
0
Sat Mar 28 2015
Computer Vision
Socializing the Semantic Gap: A Comparative Survey on Image Tag Assignment, Refinement and Retrieval
A comprehensive treatise of three closely linked problems, i.e., image tag assignment, refinement, and tag-based image retrieval is presented. The paper introduces a taxonomy to structure the burgeoning literature, understand the ingredients of the main works, and recognize their merits and limitations.
0
0
0
Sat Aug 29 2015
Computer Vision
Image Annotation Incorporating Low-Rankness, Tag and Visual Correlation and Inhomogeneous Errors
Tag-based image retrieval (TBIR) has drawn much attention in recent years due to the explosive amount of digital images and crowdsourcing tags. However, TBIR still suffering from the incomplete and inaccurate tags provided by users.
0
0
0