Navigation – Plan du site

The case for an embodied approach to listening. Bodies, technologies and perception

Andrea Giomi
Cet article est une traduction de :
Pour une approche de l’écoute incarnée. Corps, technologies et perception

Résumé

In Western philosophical and scientific tradition, aural processes have been subordinated, since antiquity, to rational knowledge and, more recently, to higher cognitive mechanisms. Within this ontological system, sound and listening are represented as essentially disembodied phenomena. However, in recent years, the emergence of an embodied perspective in music studies allows scholars to rethink listening in relation to corporeality. This article aims at presenting a review of the most important approaches – such as motor-mimetic theory, embodied music cognition, musical gesture studies and ecological theories – within the embodied paradigm. By paying particular attention to the close link between sensorimotor patterns and perception theses approaches provides a model to investigate the auditory process in a situated, multimodal and enactive perspective.

Haut de page

Texte intégral

Introduction

  • 1    Cf. Marc Leman et al., “What is embodied music cognition?” in Rolf Bader (dir.), Springer Handbo (...)
  • 2  The notion of “enaction” was introduced by the Chilean biologist and philosopher Francisco Varela (...)
  • 3    These characters are listed under the “4E cognition” label (embodied, embedded, extended, enacti (...)

1In recent years, a growing number of studies have examined the way in which the perception of music can be interpreted as an essentially embodied phenomenon. These theories, often collected under the umbrella of embodied musical cognition,1 implicitly presuppose an ecological approach to listening. They are based on the idea that the experience of the world is determined by the mutual interaction between the sensory-motor capacities of an organism and the environment in which it operates.  This mutual and reciprocal relationship forms the basis of a form of enactive knowledge.2 In this article, we wish to expose the main elements of these theories by taking an interest, in particular, in the epistemological and ontological implications this has for listening. Even though on the description of the auditive process, the cognitive and disembodied approach remains dominant, new theories, based on an embodied perspective, seem to be emerging as the result of a major epistemological overhaul. The convergence between the phenomenological tradition, ecological approaches within psychology, notions of enaction or autopoiesis in biology, theories of perception/ action in philosophy, as well as the discovery of mirror neurons in neuroscience, allows us today to radically rethink the way we perceive reality. These different scientific approaches tend toward an embodied take on listening, where cognition is defined as an enactive, non-dualist phenomenon, linked to the body and situated in an environment.3 According to this perspective, previous categories of body and mind are integrated in a holistic vision that interlaces cognitive functions, bodily movement and multi-sensory activity. Over the past ten years, this paradigm was applied to the study of the perception of sound. In this framework, listening is interpreted as a multi-modal experience, oriented toward action and, in the final analysis, embodied.

1. The role played by technology as a mediator

  • 4    See Eduardo R. Miranda and Marcelo M. Wanderley, New digital musical instruments: control and in (...)

2Technology as a mediator has played a major role in the adoption of the paradigm outlined above. The widespread use of interactive technologies (motion capture systems, physical computing techniques, machine learning algorithms applied to the study of movement, bio-feedback, etc.) has notably allowed for more in-depth study of the relationship between the body and music. This input can be split into two categories: on the one hand, technologies of movement analysis have experimentally shown the bi-univocal connection between acoustic phenomena and the sensory-motor system; on the other, the development of new musical interfaces4 has allowed us to come up with new study models for bodily movement and gestures in relation to auditive processes.

2. Embodied musical cognition

  • 5  According to computational and cognitive approaches, sound perception is often transmitted by pure (...)
  • 6    See Micheline Lesaffre, Pieter-Jan Maes and Marc Leman (dir.), The Routledge Companion to Embodi (...)
  • 7    Marc Leman, Embodied music cognition and mediation technology, Cambridge (Mass.), MIT Press, 200 (...)

3Instead of approaching listening according to a purely computational model,5 the embodied perspective considers that the listener is involved in an ‘interactive loop’ with the surrounding sound. According to this paradigm, the action/ prediction duo – that is to say the relationship between the musical action (the movement of the musician, the dance movements set to music, or the physical and spontaneous reaction of the listener) and the cognitive patterns of the representation of action, channelled by listening – is co-determined by the constraints of the sound surroundings and the body of the living organism. However, music is a phenomenon with which humans interact using their sensory-motor, cognitive and emotive capacities in order to optimize this interaction.6 This perspective constitutes the field that, since Marc Leman, we have referred to as “embodied musical cognition”.7 This theory, in placing much weight on sensory-motor and perception interactions, sees cognition as a collection of processes located at the crossroads of the listener’s environment and embodiment. Such an approach gives us a model for studying cognition, that goes beyond the traditional dichotomy of a purely neurophysiological vision and a purely psychological vision of listening.

3. Movement as a mediator in the auditive process

  • 8    According to Jackendoff, for example, perception and the cognitive processing of music and langu (...)

4According to the cognitivist, disembodied paradigm, we must consider listening as a unidirectional process, in which perception and action are separate: the action is the product of the cognitive treatment of information obtained thanks to perception.8 In other words, when we are carrying out an action, we are continually treating data that is coming from the sensory system.

  • 9    Marc Leman, Embodied music cognition and mediation technology, Cambridge (Mass.), MIT Press, 200 (...)
  • 10    Pieter-Jan Maes et al., “The coupling of action and perception in musical meaning formation”, Mu (...)

5However, from an embodied perspective, the activities linked to music come from the mutual interaction between perception and action. The sensory-motor mechanisms activated by music play a direct role in facilitating the perception of sound. The central nervous system does not need to ‘treat information’ to establish a link between sensations and actions – it just has to find the appropriate signals in the environment that must be associated with the right motor response. Embodied cognition theories support the idea that there is no real separation between mental processes and the body, by describing cognition in terms of the dynamics between agent and environment rather than in terms of information calculation. From this angle, perception and action become interdependent and occur conjointly in all sensory-motor activities. Bodily movement becomes the mediator between the subjective experience of music through listening (the mental representation or the internal sensation produced by the sound) and the sound surroundings (as much as a physical environment as a system for mediating cultural behaviours).9 According to what Leman refers to as ‘ontology oriented toward action’, the movement is then not only a means of expression but, more generally, a mediator for the formation of musical meaning.10

4. Intrasensoriality and intersensoriality of embodied listening

  • 11    See Michel Bernard, De la création chorégraphique, Pantin, CND, 2001, p. 90-93, and Maurice Merl (...)

6This notion of mediator-gesture is based on two key points. The first aspect is the active and passive nature of hearing: while in the disembodied tradition, sound perception is seen as a purely passive activity within listening (the reception of acoustic signal by the auditive membrane), embodied approaches underline the fact that listening always provokes an activity on the gravitational-muscular level as well as the activation of the selection and inhibition process in the sensory nervous system. In addition, this back and forth reflects what phenomenology defines as the intra-sensorial, or the first chiasma of perception.11

7The second question comes from the fact that listening is not made up of solely auditory sensations. It also involves visual stimuli, proprioceptive sensations, tactile/ haptic perceptions and, of course a kinaesthetic response. The interaction between these different sensorial elements is evidence of the intersensorial dimension of listening. The acoustic information interlaces in a wide range of multimodal exteroceptive stimuli (the movement of the musician’s hands, the tactile sensation from sound vibrations, etc.) and proprioceptive stimuli (the internal variations in muscle tone in reaction to music, kinaesthetic and postural articulation, etc.) that combine by resonating within one another.

5. The mimetic theory

  • 12    Marc Leman and Pietr-Jan Maes, “The role of embodiment in the perception of music”, Empirical Mu (...)

8Then, on a purely perceptive level, listening is embodied spontaneously in the gesture, even with regard to the intentionality of musical interaction, the link with the sound environment is established through the imitation of the physical information detected through the bodily articulations. In other words, it is a question of supposing that the gesture also intervenes in the process of sound understanding and meaning.12

  • 13    Rolf I. Godøy, “Motor-Mimetic Music Cognition”, Leonardo, MIT Press, n° 36, vol. 4, 2003, p. 317 (...)
  • 14    Marc Leman, Embodied music cognition and mediation technology, Cambridge (Mass.), MIT Press, 200 (...)
  • 15    Arnie Cox, “Embodying music: Principles of the mimetic hypothesis”, Music Theory Online, n° 17, (...)
  • 16    See Marc Leman et al., “Sharing musical expression through embodied listening: A case study base (...)
  • 17    See, among others, Jonna K. Vuoskoski et al., “Crossmodal interactions in the perception of expr (...)
  • 18    Kristian Nymoen et al., “Searching for Cross-Individual Relationships between Sound and Movement (...)
  • 19    Pierre Schaeffer, Traité des objets musicaux [1966], Éditions du Seuil, Paris, 1977.
  • 20    Kristian Nymoen et al., “Analyzing correspondence between sound objects and body motion”, ACM Tr (...)

9Rolf Godøy proposes that we interpret such a level of interaction using what he refers to as “the motor-mimetic theory”.13 The premise of this theory comes from a fairly obvious empirical statement: that music literally ‘moves’ people and, that their bodies resonate with the sound energy according to how immersed they feel. The main hypothesis affirms that people often respond to music with bodily movements that reflect the structural aspects of the sound or by reproducing the moves that produced the sounds that they have seen or that they imagine. Leman refers to this as “music-driven expressive movements”.14 Arnie Cox uses the term “mimetic motor imagery”.15 These theoretical hypotheses are backed up by empirical studies that show that there are fundamental analogies between listeners’ intentional movements and those of the musicians.16 These analogies are confirmed by the fact that the performers’ movements contribute significantly to the perception of the expressive intention.17 Other studies show that certain of the main characteristics of western music such as melody, harmony, tone and rhythm are interpreted in a way that incorporates morphological elements.18 From this angle, Godøy explored the link between embodiment (incorporation) and sound object by trying to promote a synthesis between the embodied approach and Pierre Schaeffer’s typo-morphology approach.19 Experiments have been carried out in this area: participants were asked to move according to the perceptive characteristics of the sonic objects. The data acquired through a motion capture system were analysed and showed that there is a morphological correlation with a collection of both quantitative and qualitative sound parameters.20

  • 21    Pietr-Jan Maes et al., “Action-Based Effects on Music Perception”, Frontiers in Psychology, n° 4 (...)

10By acting as mediator, the body builds up a repertoire of idiomatic gestures and actions, that work in patterns that allows it, first of all, to code the expressive intentionality contained in the music, and secondly to represent the morphology of sonic objects in a coherent way. This repertoire can be considered as a collection of movements carried out to reach a particular objective, linked to the experiences and sensations that result from these actions. The coupling of action and listening thus, forms a motor that guides our understanding of music. This mechanism means that listening to music can evoke an embodied knowledge of bodily movements associated with a certain expression of sound. This continuous and bi-directional process allows the listener not only to attribute intentions and feelings to the music, but also to predict the results of the actions and project them on to the music.21

6. Enaction and the ecological perspective

  • 22    Evan Thompson and Mog Stapleton, “Making sense of sense-making: Reflections on enactive and exte (...)

11The theories we have presented so far provide models for an ecological approach to listening. They are based, in fact, on the idea that the experience of the world is determined by the mutual interaction between an organism’s sensory-motor capacities and the environment in which it operates. This mutual and reciprocal relationship forms the basis for a form of enactive knowledge. According to the enaction paradigm, living organisms participate actively in the generation of feelings through their bodily involvement as well as through their transforming interactions with the environment. Mental processes are therefore, always embodied, integrated with the environment, connected to the action. The basis for these approaches comes from the fact that the living organism is ‘autonomous’, meaning capable of auto-transformation due to interaction with the environment.22 While such an approach is based on the philosophical writings of Maurice Merleau-Ponty, and the biological research of Maturana and Varela, the ecological perspective takes its lead more from the seminal work of James J. Gibson. The central aspect of this perspective is to affirm the direct nature of the perceptive act, by refusing the idea of mediation by mental representation. The environment, therefore, takes on a major role in the structuring of human perception. The enactive and ecological approaches are substantially different in as much as the former valorises the organism’s auto-generative process in interaction with the environment, and the latter lends major importance to the role played by the environment.

  • 23    Jakub R. Matyja and Andrea Schiavio, “Enactive music cognition: Background and research themes”, (...)

12According to the enactive approach to music,23 terms of reciprocal resonance between the sound environment and the listener, at times we hear the action/ perception loop and at times the emphatic phenomenon. Ecological approaches add the idea that music is experienced directly, without any intermediary mental representations, revealing itself through modes of sensorial action and perception – an idea that obviously differs from purely cognitive approaches that consider the movement of the body to be simply the sub-product of internal processes that involve a system of symbolic representation. According to the ecological approach, organisms extract immediate perceptive meaning by detecting information in their environment. As such, the world is not perceived as a collection of objective qualities, but as a network of relevant meanings.

  • 24    James J. Gibson, “The Theory of Affordances”, in Robert Shaw and John Bransford (dir), Perceivin (...)
  • 25    Eric F. Clarke, Ways of Listening: An Ecological Approach to the Perception of Musical Meaning, (...)
  • 26    Rolf I. Gødoy, “Gestural Affordances of Musical Sound”, in Rolf I. Gødoy and Marc Leman (dir.), (...)
  • 27    Michael Schutz and Scott Lipscomb, “Hearing gestures, seeing music: vision influences perceived (...)

13The notion of affordance plays a central role within this perspective. This is the notion that denotes ‘possibilities of action’ available to organisms by the objects present in their environment.24 These affordances are specified in relational terms, which means both by the environment’s particular structural characteristics and by the repertoire of sensory-motor capacities that the organism uses to detect and respond to these structural characteristics. This concept was integrated in theories about musical cognition,25 by highlighting the fact that the actions set off by affordances are the product of a cultural mediation process and not only the structural properties of the object. For example, Gødoy notes that traditional instruments possess a repertoire of specific musical gestures linked to the idiomatic language of each instrument.26 In fact, instruments present quite explicit, well-known properties, based either on the instrument’s ergonomics or on cultural codes, that, one the one hand, outline the possibilities of action between instrument and musician, and on the other inform listeners’ expectations. In addition, evoking the kinaesthetic elements of sound production while listening to music, just goes to confirm the multimodal essence of perception. As we have already shown briefly, perception can thus be described as a collection of connected sensations in which a piece of sensory information evokes a sensation in another modality. Some empirical proof confirms this hypothesis with added nuances: while the movements of the performer can modify the sensation of sound for the listener, the sounds listened to also come with kinaesthetic sensations and representations. 27

7. From embodied cognition to technological performance

  • 28    Federico Visi et al., “Gesture in performance with traditional musical instruments and electroni (...)
  • 29    Pieter-Jan Maes et al., “A Conceptual Framework for Music-Based Interaction Systems” in Rolf Bad (...)

14In recent years, theoretical and experimental advances in enactive and embodied approaches seem to also be informing the conception of sound interfaces as well as artistic creation. In particular, certain studies suggest ways to integrate the conceptual aspects of embodied musical cognition in the elaboration of mapping strategies28 as well as in the development of interactive systems to support musical practice and sensory-motor learning processes.29

15Closely linked to the above, I will now give a brief rundown of a piece of research-creation that I carried out between 2017 and 2019. “Feedback Loop Driver” is a musical performance that relies heavily on improvisation, and that uses both motion capture and bio-signal technologies. All of the sound manipulations are generated in real time by the physical movements of the performer on stage. The interactive system, using Max/Msp software, allows us to detect three main body descriptors: the electric activity of muscles, the qualities of the movement and the position in space. The central metaphor of this work is the notion of feedback. On the one hand, feedback is considered to be an auditory response to the gestures that are codified by the machine. On the other, feedback also influences the way in which my physical movement (posture, gestures, movements in space) adapt to the sound environment. The bi-univocal relationship between these two types of gesture, and the feedback loop that goes along with them, guides the internal evolution of the improvisation. From a sound design perspective, I worked on exploring sounds that could elicit a feeling of effort on my part. I explored the metaphorical analogy between the accumulation of physical energy (in reference to the huge physical effort required to contract a muscle) and the accumulation of sound energy (saturation and distortion). These two elements are linked on a sensory level: if the bodily effort generates audio distortion, the physical tension expressed by the performer reflects the incorporation of the surrounding soundscape. This approach to interactive sound design allows me to connect the internal perception of movement to the sensations produced by sound feedback.

16The sensation of effort is indeed felt on two levels. One the one hand, the physical effort (linked to muscle contraction) needed to generate the sound; on the other, the secondary ‘feedback’ provoked by the dynamic and spectral accumulation of sound energy and that resonates with the primary sensation provoked by the contraction of muscles. The tensions expressed by the sound are thus incorporated by becoming the object of a figurative gesture that takes root in the effort felt as well as in the imaginative valorisation of sound material. A multimodal integration of the different sensory stimuli occurs to reinforce the precision in the execution of sound gestures as well as the expressiveness of the physical presence. Listening, in this case, allows us to interpret the surrounding soundscape both in relation to the musical composition processes envisaged, and in relation to the bodily changes experienced. In doing so, the technological devices become part of the sensible body by stimulating a higher degree of resilience and enactive knowledge.

Conclusion

17The idea of listening as an enactive, multimodal and embodied phenomenon is an essential step toward a philosophical repositioning that will allow us to get beyond the epistemological dualism that exists in the study of music and the way we experience sound. Recent studies based on an embodied perspective propose that we move listening to a more bodily dimension, where our sensory-motor system plays a central role. Various experimental studies have, for example, shown how spontaneous bodily movements, responding to music, can be linked to the metaphorical descriptions used to evoke the music’s expressive qualities. In the same way, the ecological and enactive approaches suggest a situated and embodied definition of listening in which the different actors (listeners, musicians, music) are integrated. In this framework, activities linked to music and the formation of musical meaning emerge from the mutual interaction between perception and musical action. By listening attentively to music, the listener continually recreates mental simulations of the gestures of sound production, by adding coherent kinaesthetic elements. At the same time, the musician’s bodily movements seem to have a direct influence on the way in which we perceive sound, highlighting the multimodal nature of listening.

18A number of studies integrate motion capture technologies not only to record and analyse the link between movements and listening, but also to develop new strategies for designing sound interfaces and interactive musical systems. The central mechanism in which interactive technologies can intervene is the action/ perception loop. Indeed, a number of studies have examined how the use of sound feedback in real time allows for reinforced emotional engagement and bodily involvement by the interactors in a multitude of musical activities such as performance, listening and dance.

19The positive function of sound feedback reveals itself to be decisive in two ways. Firstly, the auditory system is capable of recognizing time variations on a more precise level relative to other perceptive channels by soliciting major awareness of the nuances at the basis of the composition of the gesture. Secondly, in harmony with the principles of multimodal integration and moto-mimetic theory, the perceptive coherence between sound and movement reinforces our capacity to accomplish a motor-based and gestural task while at the same time encouraging active participation in musical phenomena. As such, the use of technology as a mediator seems to support an embodied approach to listening and sound experience, in an increasingly effective manner.

Haut de page

Bibliographie

BADER Rolf (dir.), Springer Handbook of Systematic Musicology, Cham (Switzerland), Springer-Verlag, 2018.

BERNARD Michel, De la création chorégraphique, Pantin, CND, 2001.

BREGMAN Albert S., Auditory scene analysis: The Perceptual Organization of Sound, Cambridge (Mass.), MIT Press, 1990.

CLARKE Eric F., Ways of Listening: An Ecological Approach to the Perception of Musical Meaning, New York, Oxford University Press, 2005.

COX, Arnie, « Embodying Music: Principles of the Mimetic Hypothesis », Music Theory Online, n° 17.2, 2011, p. 1-24. [En ligne] http://www.mtosmt.org/issues/mto.11.17.2/mto.11.17.2.cox.html [consulté le 3 septembre 2019].

GIBSON James J., “The Theory of Affordances”, in Robert Shaw and John Bransford (dir), Perceiving, Acting, and Knowing: Toward an Ecological Psychology, Lawrence Erlbaum Associates, New Jersey, 1977, p. 67-82.

GODØY Rolf I., “Motor-Mimetic Music Cognition”, Leonardo, MIT Press, n° 36, vol. 4, 2003, p. 317–319.

GODØY Rolf I. and Leman Marc (dir.), Musical Gestures: Sound, Movement and Meaning, London/New York, Routledge, 2010.

JACKENDOFF Ray, Languages of the mind: essays on mental representation, Cambridge (Mass.), MIT Press, 1992.

NYMOEN Kristian, JENSENIUS Alexander R., TØRRESEN Jim, GLETTE Kyrre H., SKOGSTAD Ståle A., “Searching for Cross-Individual Relationships between Sound and Movement Features using an SVM Classifier”, NIME ’10, Sydney, Australia, 2010, p. 259–262.

LEMAN Marc, Embodied music cognition and mediation technology, Cambridge (Mass.), MIT Press, 2008.

LEMAN Marc, DESMET Frank, STYNS Frederik, VAN NOORDEN Leon, MOELANTS Dirk, “Sharing musical expression through embodied listening: A case study based on Chinese Guqin music”, Music Perception, n° 26, vol. 3, 2009, p. 263–278.

LEMAN Marc and MAES Pietr-Jan, “The role of embodiment in the perception of music”, Empirical Musicology Review, n° 9, vol. 3/4, 2015, p. 236-246.

LESAFFRE Micheline, MAES Pieter-Jan and LEMAN Marc (dir.), The Routledge Companion to Embodied Music Interaction, New York, Routledge, 2017.

MATYJA Jakub R. and SCHIAVIO Andrea, “Enactive music cognition: Background and research themes”, Constructivist Foundations, n° 8, vol. 3, 2013, p. 351-357.

MAES Pietr-Jan, VAN DYCK Edith, LESAFFRE Micheline, LEMAN Marc, KROONENBERG Pieter M., “The coupling of action and perception in musical meaning formation”, Music Perception, n° 32, vol. 1, 2014, p. 67-84.

MAES Pietr-Jan, LEMAN Marc, PALMER Caroline et WANDERLEY Marcelo M., “Action-Based Effects on Music Perception”, Frontiers in Psychology, n° 4, 2014.

MENARY Richard, “Introduction to the special issue on 4E cognition”, Phenomenology and the Cognitive Sciences, n° 9, vol. 4, 2010, p. 459-463.

MERLEAU-PONTY Maurice, Le visible et l’invisible, Paris, Gallimard, 1961.

Miranda Eduardo R. et Wanderley Marcelo M., New digital musical instruments: control and interaction beyond the keyboard, Middleton (Wisconsin), A-R Editions, 2006.

Nymoen Kristian, Godøy Rolf I., Jensenius Alexander R. et Tørresen Jim, “Analyzing correspondence between sound objects and body motion”, ACM Transactions on Applied Perception, n° 10, vol. 2, 2013, p. 1-22.

SCHAEFFER Pierre, Traité des objets musicaux, Paris, Éditions du Seuil, 1977.

SCHUTZ Michael et LIPSCOMB Scott, “Hearing gestures, seeing music: vision influences perceived tone duration”, Perception, n° 36, vol. 6, 2007, p. 888–897.

SLOBODA John A., Exploring the Musical Mind: Cognition, Emotion, Ability, Function, Oxford, Oxford University Press, 2005.

THOMPSON Evan et STAPLETON Mog, “Making sense of sense-making: Reflections on enactive and extended mind theories”, Topoi, n° 28, vol. 1, 2009, p. 23-30.

VARELA Francisco, THOMPSON Evan and ROSCH Eleanor, The Embodied Mind: Cognitive Science and Human Experience, Cambridge (Mass.), MIT Press, 1991.

VISI Federico, SCHRAMM Rodrigo and MIRANDA Eduardo R., “Gesture in performance with traditional musical instruments and electronics: Use of embodied music cognition and multimodal motion capture to design gestural mapping strategies”, MOCO 14: Proceedings of the International Workshop on Movement and Computing, Paris, 2014.

VUOSKOSKI Jonna K., THOMPSON Marc R., CLARKE Eric F. and SPENCE, Charles, “Crossmodal interactions in the perception of expressivity in musical performance”, Attention, Perception, & Psychophysics, n° 76, vol. 2, 2014, p. 591-604.

Haut de page

Notes

1    Cf. Marc Leman et al., “What is embodied music cognition?” in Rolf Bader (dir.), Springer Handbook of Systematic Musicology, Cham (Switzerland), Springer-Verlag, 2018, p. 747-760.

2  The notion of “enaction” was introduced by the Chilean biologist and philosopher Francisco Varela in the early nine-ties to define a new approach to cognition based on the dynamic interaction between the living organism and its envi-ronment. See Francisco Varela, Evan Thompson and Eleanor Rosch, The Embodied Mind: Cognitive Science and Hu-man Experience, Cambridge (Mass.), MIT Press, 1991.

3    These characters are listed under the “4E cognition” label (embodied, embedded, extended, enactive). See Rich-ard Menary, “Introduction to the special issue on 4E cognition”, Phenomenology and the Cognitive Sciences, n° 9, vol. 4, 2010, p. 459-463.

4    See Eduardo R. Miranda and Marcelo M. Wanderley, New digital musical instruments: control and interaction be-yond the keyboard, Middleton (Wisconsin), A-R Editions, 2006.

5  According to computational and cognitive approaches, sound perception is often transmitted by purely mental operations taking place in the brain. By insisting on just the cognitive dimension, listening is reduced either to prior symbolic structures – see John A. Sloboda, Exploring the Musical Mind: Cognition, Emotion, Ability, Function, Oxford, Oxford University Press, 2005 – or to high-level cognitive processes like language – see Ray Jackendoff, Languages of the mind: essays on mental representation, Cambridge (Mass.), MIT Press, 1992. I refer, in particular to the generative theory of music – or human physiology’s only mechanical reactions – see, among others, Albert S. Bregman, Auditory scene analysis: The Perceptual Organization of Sound, Cambridge (Mass.), MIT Press, 1990.

6    See Micheline Lesaffre, Pieter-Jan Maes and Marc Leman (dir.), The Routledge Companion to Embodied Music In-teraction, New York, Routledge, 2017, p. 1-10.

7    Marc Leman, Embodied music cognition and mediation technology, Cambridge (Mass.), MIT Press, 2008.

8    According to Jackendoff, for example, perception and the cognitive processing of music and language are run by simi-lar functions based on a specific grammar: both categories of stimuli imply the building of abstract structures from purely auditory events. This means that a musical grammar includes certain rules that define the musical structures that are accessible to the listener and links these structures to their acoustic equivalent: see Ray Jackendoff, op. cit., p. 125-156.

9    Marc Leman, Embodied music cognition and mediation technology, Cambridge (Mass.), MIT Press, 2008, p. 13, 43-45 et 137-138.

10    Pieter-Jan Maes et al., “The coupling of action and perception in musical meaning formation”, Music Perception, n°32.1, 2014, p. 67–84.

11    See Michel Bernard, De la création chorégraphique, Pantin, CND, 2001, p. 90-93, and Maurice Merleau-Ponty, Le visible et l’invisible, Paris, Gallimard, 1961, p. 170-201.

12    Marc Leman and Pietr-Jan Maes, “The role of embodiment in the perception of music”, Empirical Musicology Re-view, n° 9, vol. 3/4, 2015, p. 236-246.

13    Rolf I. Godøy, “Motor-Mimetic Music Cognition”, Leonardo, MIT Press, n° 36, vol. 4, 2003, p. 317–319.

14    Marc Leman, Embodied music cognition and mediation technology, Cambridge (Mass.), MIT Press, 2008, p. 95-97.

15    Arnie Cox, “Embodying music: Principles of the mimetic hypothesis”, Music Theory Online, n° 17, vol. 2, 2011, p. 1-24. [En ligne] http://www.mtosmt.org/issues/mto.11.17.2/mto.11.17.2.cox.html [consulté le 4 novembre 2018].

16    See Marc Leman et al., “Sharing musical expression through embodied listening: A case study based on Chinese Guqin music”, Music Perception, n° 26, vol. 3, 2009, p. 263–278.

17    See, among others, Jonna K. Vuoskoski et al., “Crossmodal interactions in the perception of expressivity in musical performance”, Attention, Perception, & Psychophysics, n°76.2, 2014, p. 591-604.

18    Kristian Nymoen et al., “Searching for Cross-Individual Relationships between Sound and Movement Features using an SVM Classifier”, NIME ’10, Sydney, Australia, 2010, p. 259–262.

19    Pierre Schaeffer, Traité des objets musicaux [1966], Éditions du Seuil, Paris, 1977.

20    Kristian Nymoen et al., “Analyzing correspondence between sound objects and body motion”, ACM Transactions on Applied Perception, n° 10, vol. 2, 2013, p. 1-22.

21    Pietr-Jan Maes et al., “Action-Based Effects on Music Perception”, Frontiers in Psychology, n° 4, 2014.

22    Evan Thompson and Mog Stapleton, “Making sense of sense-making: Reflections on enactive and extended mind theories”, Topoi, n° 28, vol. 1, 2009, p. 23-30.

23    Jakub R. Matyja and Andrea Schiavio, “Enactive music cognition: Background and research themes”, Constructivist Foundations, n° 8, vol. 3, 2013, p. 351-357.

24    James J. Gibson, “The Theory of Affordances”, in Robert Shaw and John Bransford (dir), Perceiving, Acting, and Knowing: Toward an Ecological Psychology, Lawrence Erlbaum Associates, New Jersey, 1977, p. 67-82.

25    Eric F. Clarke, Ways of Listening: An Ecological Approach to the Perception of Musical Meaning, New York, Oxford University Press, 2005.

26    Rolf I. Gødoy, “Gestural Affordances of Musical Sound”, in Rolf I. Gødoy and Marc Leman (dir.), Musical Ges-tures: Sound, Movement and Meaning, London/New York, Routledge, 2010, p. 103-125.

27    Michael Schutz and Scott Lipscomb, “Hearing gestures, seeing music: vision influences perceived tone duration”, Perception, n° 36, vol. 6, 2007, p. 888–897.

28    Federico Visi et al., “Gesture in performance with traditional musical instruments and electronics: Use of embodied music cognition and multimodal motion capture to design gestural mapping strategies”, MOCO 14: Proceedings of the International Workshop on Movement and Computing, Paris (France), 2014.

29    Pieter-Jan Maes et al., “A Conceptual Framework for Music-Based Interaction Systems” in Rolf Bader (dir.), Springer Handbook of Systematic Musicology, Cham (Suisse), Springer-Verlag, 2018, p. 793-804.

Haut de page

Pour citer cet article

Référence électronique

Andrea Giomi, « The case for an embodied approach to listening. Bodies, technologies and perception », Hybrid [En ligne], 06 | 2019, mis en ligne le 01 mars 2021, consulté le 23 septembre 2021. URL : http://www.hybrid.univ-paris8.fr/lodel/index.php?id=1398

Haut de page

Auteur

Andrea Giomi

He is a sound artist, performer and researcher. He is currently a postdoctoral fellow at the Performance Laboratory of the Université Grenoble-Alpes where he is developing research on the sonification of movement. Holder of a doctorate in music and performing arts from the Côte d'Azur University in Nice and the Alma Mater University in Bologna, his research focuses on embodied musical cognition, human-machine interaction, motion capture technologies, digital arts aesthetics, phenomenology and the theory of perception. He is also a performer and producer of electronic music. As an artist, his interests focus on the emerging relationships between body, sound and technological environments.

Haut de page