Advanced

Polysemiotic Communication vs. Multimodality : a conceptual and terminological distinction applied in street art

Stampoulidis, Georgios LU (2019) Street art and Urban Creativity Conference In Street Art & Urban Creativity Scientific Journal 5(2). p.26-31
Abstract
Multimodality is in fact a polysemous word, which is tightly related to the notions of modality, and (semiotic) mode and is used in conceptually different ways across different disciplines (for a review see Adami, 2016; Devylder, 2019; Green, 2014). As cognitive semiotics (Zlatev et al., 2016) aims to integrate concepts and methods from semiotics, cognitive science and cognitive linguistics, we endeavor to offer a coherent terminology, in line with the proposals of Green (2014), Stampoulidis et al. (2019) and Zlatev (2019), which distinguishes the notions of perceptual (sensory) modalities (sight, hearing, smell, touch and taste) and semiotic systems (language, depiction and gesture).


For example, using this... (More)
Multimodality is in fact a polysemous word, which is tightly related to the notions of modality, and (semiotic) mode and is used in conceptually different ways across different disciplines (for a review see Adami, 2016; Devylder, 2019; Green, 2014). As cognitive semiotics (Zlatev et al., 2016) aims to integrate concepts and methods from semiotics, cognitive science and cognitive linguistics, we endeavor to offer a coherent terminology, in line with the proposals of Green (2014), Stampoulidis et al. (2019) and Zlatev (2019), which distinguishes the notions of perceptual (sensory) modalities (sight, hearing, smell, touch and taste) and semiotic systems (language, depiction and gesture).


For example, using this polysemiotic/multimodal distinction would allow us to describe the work of street art displayed in Figure 1 consisting of verbal text (language) and pictorial elements (depiction) as clearly a form of polysemiotic communication, instantiated in the particular socio-cultural medium of street art, whereas the street artwork displayed in Figure 2 might be considered as an example of unisemiotic communication (only the semiotic system of depiction is present). Nevertheless, both artworks may be considered monomodal since at least one perceptual modality is involved: sight. On the other hand, artworks such as these displayed in Figure 3 and Figure 4 can be both polysemiotic and unisemiotic, respectively, and (potentially) multimodal (if) they trigger multiple senses in the viewer, such as sight and touch, for example. It is important to note that the terminological distinction and conceptual dichotomy between the semiotic systems of language and depiction are not always clear-cut, especially in the case of street art (and graffiti), as has been argued in a certain literature (Bal, 1991; Neef, 2007). Therefore, we would like to stress that street art is typically a form of polysemiotic communication, and thus, we restrict the term unisemiotic either to the case of primarily depiction-dominant or primarily language- dominant graphic representations.



In our study on street art, marking this polysemiotic/ multimodal distinction – terminological and conceptual – would help us toward a synthetic analysis of the interaction between language and depiction, and that of language, depiction, vision, and (potentially) smelling, touching or even hearing, into a whole communicative situation. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
street art, metaphor, multimodality, polysemiotic communication, Athens
in
Street Art & Urban Creativity Scientific Journal
volume
5
issue
2
pages
6 pages
publisher
Urban Creativity
conference name
Street art and Urban Creativity Conference
conference location
Lisbon, Portugal
conference dates
2019-07-04 - 2019-07-06
external identifiers
  • scopus:85104002228
ISSN
2183-3869
DOI
10.25765/sauc.v5i2.156
language
English
LU publication?
yes
id
518ca013-570d-4502-9d38-9a361507eb12
date added to LUP
2020-01-28 11:53:06
date last changed
2021-04-23 08:40:44
@article{518ca013-570d-4502-9d38-9a361507eb12,
  abstract     = {Multimodality is in fact a polysemous word, which is tightly related to the notions of modality, and (semiotic) mode and is used in conceptually different ways across different disciplines (for a review see Adami, 2016; Devylder, 2019; Green, 2014). As cognitive semiotics (Zlatev et al., 2016) aims to integrate concepts and methods from semiotics, cognitive science and cognitive linguistics, we endeavor to offer a coherent terminology, in line with the proposals of Green (2014), Stampoulidis et al. (2019) and Zlatev (2019), which distinguishes the notions of perceptual (sensory) modalities (sight, hearing, smell, touch and taste) and semiotic systems (language, depiction and gesture).<br/><br/><br/>For example, using this polysemiotic/multimodal distinction would allow us to describe the work of street art displayed in Figure 1 consisting of verbal text (language) and pictorial elements (depiction) as clearly a form of polysemiotic communication, instantiated in the particular socio-cultural medium of street art, whereas the street artwork displayed in Figure 2 might be considered as an example of unisemiotic communication (only the semiotic system of depiction is present). Nevertheless, both artworks may be considered monomodal since at least one perceptual modality is involved: sight. On the other hand, artworks such as these displayed in Figure 3 and Figure 4 can be both polysemiotic and unisemiotic, respectively, and (potentially) multimodal (if) they trigger multiple senses in the viewer, such as sight and touch, for example. It is important to note that the terminological distinction and conceptual dichotomy between the semiotic systems of language and depiction are not always clear-cut, especially in the case of street art (and graffiti), as has been argued in a certain literature (Bal, 1991; Neef, 2007). Therefore, we would like to stress that street art is typically a form of polysemiotic communication, and thus, we restrict the term unisemiotic either to the case of primarily depiction-dominant or primarily language- dominant graphic representations.<br/><br/> <br/><br/>In our study on street art, marking this polysemiotic/ multimodal distinction – terminological and conceptual – would help us toward a synthetic analysis of the interaction between language and depiction, and that of language, depiction, vision, and (potentially) smelling, touching or even hearing, into a whole communicative situation.},
  author       = {Stampoulidis, Georgios},
  issn         = {2183-3869},
  language     = {eng},
  number       = {2},
  pages        = {26--31},
  publisher    = {Urban Creativity},
  series       = {Street Art & Urban Creativity Scientific Journal},
  title        = {Polysemiotic Communication vs. Multimodality : a conceptual and terminological distinction applied in street art},
  url          = {https://lup.lub.lu.se/search/ws/files/75574843/Stampoulidis_SAUC2019.pdf},
  doi          = {10.25765/sauc.v5i2.156},
  volume       = {5},
  year         = {2019},
}