Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Privacy-preserving visual content tagging using graph transformer networks

Vu, Xuan-Son LU ; Le, Duc-Trong ; Edlund, Christoffer ; Jiang, Lili and Nguyen, Hoang D (2020) p.2299-2307
Abstract
With the rapid growth of Internet media, content tagging has become an important topic with many multimedia understanding applications, including efficient organisation and search. Nevertheless, existing visual tagging approaches are susceptible to inherent privacy risks in which private information may be exposed unintentionally. The use of anonymisation and privacy-protection methods is desirable, but with the expense of task performance. Therefore, this paper proposes an end-to-end framework (SGTN) using Graph Transformer and Convolutional Networks to significantly improve classification and privacy preservation of visual data. Especially, we employ several mechanisms such as differential privacy based graph construction and... (More)
With the rapid growth of Internet media, content tagging has become an important topic with many multimedia understanding applications, including efficient organisation and search. Nevertheless, existing visual tagging approaches are susceptible to inherent privacy risks in which private information may be exposed unintentionally. The use of anonymisation and privacy-protection methods is desirable, but with the expense of task performance. Therefore, this paper proposes an end-to-end framework (SGTN) using Graph Transformer and Convolutional Networks to significantly improve classification and privacy preservation of visual data. Especially, we employ several mechanisms such as differential privacy based graph construction and noise-induced graph transformation to protect the privacy of knowledge graphs. Our approach unveils new state-of-the-art on MS-COCO dataset in various semi-supervised settings. In addition, we showcase a real experiment in the education domain to address the automation of sensitive document tagging. Experimental results show that our approach achieves an excellent balance of model accuracy and privacy preservation on both public and private datasets. (Less)
Please use this url to cite or link to this publication:
author
; ; ; and
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
host publication
MM '20 : Proceedings of the 28th ACM International Conference on Multimedia - Proceedings of the 28th ACM International Conference on Multimedia
editor
Wen Chen, Chang and Cucchiara, Rita
pages
9 pages
publisher
Association for Computing Machinery
external identifiers
  • scopus:85106902322
ISBN
978-1-4503-7988-5
DOI
10.1145/3394171.3414047
language
English
LU publication?
no
id
b555f5ee-496e-4e6e-a235-ffb9bd9d1ef6
date added to LUP
2026-02-10 23:54:39
date last changed
2026-03-09 10:27:17
@inproceedings{b555f5ee-496e-4e6e-a235-ffb9bd9d1ef6,
  abstract     = {{With the rapid growth of Internet media, content tagging has become an important topic with many multimedia understanding applications, including efficient organisation and search. Nevertheless, existing visual tagging approaches are susceptible to inherent privacy risks in which private information may be exposed unintentionally. The use of anonymisation and privacy-protection methods is desirable, but with the expense of task performance. Therefore, this paper proposes an end-to-end framework (SGTN) using Graph Transformer and Convolutional Networks to significantly improve classification and privacy preservation of visual data. Especially, we employ several mechanisms such as differential privacy based graph construction and noise-induced graph transformation to protect the privacy of knowledge graphs. Our approach unveils new state-of-the-art on MS-COCO dataset in various semi-supervised settings. In addition, we showcase a real experiment in the education domain to address the automation of sensitive document tagging. Experimental results show that our approach achieves an excellent balance of model accuracy and privacy preservation on both public and private datasets.}},
  author       = {{Vu, Xuan-Son and Le, Duc-Trong and Edlund, Christoffer and Jiang, Lili and Nguyen, Hoang D}},
  booktitle    = {{MM '20 : Proceedings of the 28th ACM International Conference on Multimedia}},
  editor       = {{Wen Chen, Chang and Cucchiara, Rita}},
  isbn         = {{978-1-4503-7988-5}},
  language     = {{eng}},
  pages        = {{2299--2307}},
  publisher    = {{Association for Computing Machinery}},
  title        = {{Privacy-preserving visual content tagging using graph transformer networks}},
  url          = {{http://dx.doi.org/10.1145/3394171.3414047}},
  doi          = {{10.1145/3394171.3414047}},
  year         = {{2020}},
}