Learning Mixtures of Submodular Functions for Image Collection Summarization

Learning Mixtures of Submodular Functions for Image Collection Summarization

Abstract

We address the problem of image collection summarization by learning mixtures ofsubmodular functions. Submodularity is useful for this problem since it naturallyrepresents characteristics such as fidelity and diversity, desirable for any summary.Several previously proposed image summarization scoring methodologies, in fact,instinctively arrived at submodularity. We provide classes of submodular compo-nent functions (including some which are instantiated via a deep neural network)over which mixtures may be learnt. We formulate the learning of such mixtures as asupervised problem via large-margin structured prediction. As a loss function, andfor automatic summary scoring, we introduce a novel summary evaluation methodcalled V-ROUGE, and test both submodular and non-submodular optimization(using the submodular-supermodular procedure) to learn a mixture of submodularfunctions. Interestingly, using non-submodular optimization to learn submodularfunctions provides the best results. We also provide a new data set consisting of14 real-world image collections along with many human-generated ground truthsummaries collected using Amazon Mechanical Turk. We compare our methodwith previous work on this problem and show that our learning approach outper-forms all competitors on this new data set. This paper provides, to our knowledge,the first systematic approach for quantifying the problem of image collection sum-marization, along with a new data set of image collections and human summaries.

Grafik Top
Authors
  • Tschiatschek, Sebastian
  • Iyer, Rishabh
  • Wei, Haochen
  • Bilmes, Jeff
Grafik Top
Shortfacts
Category
Paper in Conference Proceedings or in Workshop Proceedings (Paper)
Event Title
Neural Information Processing Systems (NIPS)
Divisions
Data Mining and Machine Learning
Event Location
Montreal, Canada
Event Type
Conference
Event Dates
08.-13.12.2014
Series Name
Advances in Neural Information Processing Systems 27 (NIPS 2014)
Date
8 December 2014
Official URL
https://www.tschiatschek.net/files/tschiatschek14s...
Export
Grafik Top