Semiotic resources and affordances in social interaction - Report on the Creativity Cluster Lecture by Professor Shaun Gallagher (University of Memphis).
Thursday 21 September, 2017
Professor Gallagher is a renowned contemporary philosopher whose research focuses in the area of cognitive science, especially topics related to embodiment and inter-subjectivity. In his ASC Creativity Cluster talk, he spoke about the semiotic resources and affordances in social cognition.
Image: Professor Shaun Gallagher (left) with Creativity Cluster leader Dr Lambros Malafouris.
A semiotic resource refers to a means to make meaning. Van Leeuwen (2004) defined them as the actions, materials and artefacts used with communication purposes, either produced physiologically (e.g. the vocal apparatus, the muscles used in facial expressions), or technologically (e.g. a pen and ink, the software and hardware of a computer). The semiotic resources are at the same time a material, social and cultural resource, and have a potential meaning based in their past use and a group of affordances based in their possible uses. Source: WordPress
Professor Gallagher began his talk with a fascinating discussion of people’s reactions to ‘first encounters’ giving as an example what happened when the first platypus1The platypus (Ornithorhynchus anatinus), also known as duck-billed platypus, is a semiaquatic egg-laying mammal endemic to eastern Australia, including Tasmania. Photographs by Joel Sartore National Geographic Photo Ark.
was shipped to Britain from Australia in 1799. He reported that many people thought that it was a joke, that the animal had somehow been constructed or created by sewing together different parts of other animals. The discussion of the epistemological implications of this story about the platypus can be drawn from the work of Umberto Eco, who suggests that the works of Kant and Peirce are inadequate to explain our initial perception of something new and unexpected. Professor Gallagher argued that first encounters of this kind an embodied account of cognition2Embodiment usually refers to how the body and its interactive processes, such as perception or cultural acquisition through the senses, aid, enhance or interfere with the development of human functioning. Source: WordPress.
which takes into consideration what our senses perceive. Using such a philosophy in the context of social interactions, Professor Gallagher introduced Interaction Theory3A term originated by Professor Gallagher himself in 2001 due to the necessity of a better explanation of social cognition.
(IT) which tries to explain how people understand others according to their bodily movement (e.g. facial expressions, gestures, body posture, vocal intonation, etc.) in the context of their surroundings. To explain this he gave as an example the work of Colwyn Trevarthen who focussed on the importance of the infant’s sensory-motor processes when interacting with their carers. Professor Gallagher showed a video of neonate imitation where an adult is interacting with a newborn and sticking out his tongue. The reaction of the newborn is soon to stick out his tongue in response. Professor Gallagher explained that the baby’s reaction demonstrates that from an extremely early age infants are drawn into interaction with others. It tunes out that there are two types of intersubjective interactions: 1) those early developing interactions based on sensory-motor processes (primary intersubjectivity, e.g. gaze following, gestures, facial expressions, neonatal imitation, vocal intonation, proto-conversations) and 2) contextualized interactions where the individual starts to pay attention to the behaviour of others in pragmatic situations where they engage with artifacts and where joint action is possible (secondary intersubjectiviy).
The platypus (Ornithorhynchus anatinus). Photographs by Joel Sartore. National Geographic Photo Ark.
Verbal and non-verbal elements in conversation are semiotic resources, that is to say ways to communicate and produce meaning, but there are others ways to do so in social interactions. Interaction Theory proposes that a more sophisticated understanding of others is based on the implicit and explicit uses of ‘narrative’. This means that besides primary and secondary intersubjectivities, and the contribution of the social interaction itself, our personal and cultural narratives provide us with a background knowledge to be able to understand others. Prof. Gallagher claimed that we live in a world where we perceive others instinctively and intuitively; we can know what other wants by their gestures, body movements, etc. Drawing from the work of Charles Goodwin, he showed a picture of two girls playing hopscotch and having a discussion; the way in which both girls expressed themselves verbally and bodily demonstrated the complexity and multi-faceted aspects of ‘interaction’ even in a relatively simple situation. He described the events in the picture as taking place within a pragmatic and social context, supplemented with narrative practices and this was an example of the use of multiple semiotic resources.
Finally, Prof. Gallagher pointed out that in semiotics, besides interaction and narrative, there are also affordances. James J. Gibson coined the noun affordance in 1977 and explained it pertains to the environment providing the opportunity for action. For example, when we perceive an object we observe the object's affordances (i.e. possible uses) such as lifting or grasping and not just its particular qualities. In social interactions, Gibson believed affordances were important to understanding social behaviours as our behaviour depends on what we perceive the other person’s intentions to be. Although the exact definition of the term ‘affordances’ in a philosophical context is a matter of debate, Professor Gallagher referred to a definition that included physical, social and cultural resources occurring in an individual’s affordance space and defined by evolution, social development as well as social and cultural practices.
Prof. Gallagher explaining the term affordances.