Advanced

Combining Text Semantics and Image Geometry to Improve Scene Interpretation

Medved, Dennis LU ; Jiang, Fangyuan LU ; Exner, Peter LU ; Oskarsson, Magnus LU ; Nugues, Pierre LU and Åström, Karl LU (2014) 3rd International Conference on Pattern Recognition Applications an Methods (ICPRAM 2014) In Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods p.479-486
Abstract
Inthispaper,wedescribeanovelsystemthatidentifiesrelationsbetweentheobjectsextractedfromanimage. We started from the idea that in addition to the geometric and visual properties of the image objects, we could exploit lexical and semantic information from the text accompanying the image. As experimental set up, we gathered a corpus of images from Wikipedia as well as their associated articles. We extracted two types of objects: human beings and horses and we considered three relations that could hold between them: Ride, Lead, or None. We used geometric features as a baseline to identify the relations between the entities and we describe the improvements brought by the addition of bag-of-wordf eatures and predicate–arguments tructures we... (More)
Inthispaper,wedescribeanovelsystemthatidentifiesrelationsbetweentheobjectsextractedfromanimage. We started from the idea that in addition to the geometric and visual properties of the image objects, we could exploit lexical and semantic information from the text accompanying the image. As experimental set up, we gathered a corpus of images from Wikipedia as well as their associated articles. We extracted two types of objects: human beings and horses and we considered three relations that could hold between them: Ride, Lead, or None. We used geometric features as a baseline to identify the relations between the entities and we describe the improvements brought by the addition of bag-of-wordf eatures and predicate–arguments tructures we derived from the text. The best semantic model resulted in a relative error reduction of more than 18% over the baseline.
(Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
in
Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods
pages
479 - 486
publisher
SciTePress
conference name
3rd International Conference on Pattern Recognition Applications an Methods (ICPRAM 2014)
external identifiers
  • scopus:84902308102
ISBN
978-989-758-018-5
language
English
LU publication?
yes
id
3ecf3d61-eac1-4bb9-9de8-6a4411dabe6f (old id 4352695)
date added to LUP
2014-03-07 13:05:09
date last changed
2017-01-15 04:44:55
@misc{3ecf3d61-eac1-4bb9-9de8-6a4411dabe6f,
  abstract     = { Inthispaper,wedescribeanovelsystemthatidentifiesrelationsbetweentheobjectsextractedfromanimage. We started from the idea that in addition to the geometric and visual properties of the image objects, we could exploit lexical and semantic information from the text accompanying the image. As experimental set up, we gathered a corpus of images from Wikipedia as well as their associated articles. We extracted two types of objects: human beings and horses and we considered three relations that could hold between them: Ride, Lead, or None. We used geometric features as a baseline to identify the relations between the entities and we describe the improvements brought by the addition of bag-of-wordf eatures and predicate–arguments tructures we derived from the text. The best semantic model resulted in a relative error reduction of more than 18% over the baseline.<br/>},
  author       = {Medved, Dennis and Jiang, Fangyuan and Exner, Peter and Oskarsson, Magnus and Nugues, Pierre and Åström, Karl},
  isbn         = {978-989-758-018-5 },
  language     = {eng},
  pages        = {479--486},
  publisher    = {SciTePress},
  series       = {Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods },
  title        = {Combining Text Semantics and Image Geometry to Improve Scene Interpretation},
  year         = {2014},
}