• 1  For an account of the evolution of digital comics from the 1990s to today, see Julien Baudry, “Gén (...)

1Compared to audiovisual creation (cinema, digital art, etc.), which produces moving images, or even interactive ones like with digital narrations, comics appear as a technologically rudimentary medium, whose sole means of expression are text and image. Indeed, the image in comics is necessarily fixed (i.e., not moving) and mute, insofar as the words and noises that populate the diegetic universe are reproduced in a purely visual and graphic mode. Of course, the impossibility of dubbing a story is closely linked to the materiality of the paper medium. Intrinsically noiseless, paper prohibits a priori the use of audible content. Hence the stabilization of a set of iconotextual conventions (bubbles, onomatopoeia, etc.) to account for sounds and thus compensate for the absence of audio clips. As one would expect, the appearance of “digital” comics, first on CD-ROM in the late 1990s, then on the web in the 2000s,1 changed this purely graphic approach to sound. Digital technology allows the use of sound effects, music and even prerecorded voices (narrator intervention, dialogue pronunciation, etc.), so that the semantic content is not only conveyed by the text and the image, but also by sound elements. The digital comic therefore invites readers to keep their ears open, while its paper predecessor depended exclusively on the visual channel. Listening is added to seeing, as Thierry Groensteen rightly notes:

• 2  Thierry Groensteen, Bande dessinée et narration, Paris, PUF, 2011, p. 74.

2[…] Paper comics were polysemiotic, since they usually associate text and image, but they are monosensory, soliciting sight exclusively. Creative digital comics can make use of a wider range of semiotic possibilities; they are plurisensory as soon as sound is introduced.2

• 5  Daniel Bougnoux, Introduction aux sciences de la communication, Paris, La Découverte, 1998.

5In our opinion, Fresnault-Deruelle’s attribution of “unrepresentability” to sound and movement merits discussion. Neither sound nor movement is in itself “unrepresentable.” At most, they test the expressive potential of the medium and its capacity to make referential illusion. This means that comics, like any other medium, comes with constraints (area of ​​the page, mute images, etc.) to which authors must adapt to give substance to their fictional project. All of these constraints have recently been reinterpreted in an interesting way by the researcher Pascal Robert. The latter sees comics as a medium inhabited by paradox. In his introductory book on the sciences of communication, Daniel Bougnoux defines paradox as the result of a cleavage between a statement (E) and its conditions of enunciation (CE).5 In other words, there is a paradox when there is an imbalance between the content of the message and the modalities of its enunciation. Pascal Robert takes up this definition of paradox to argue that the representation of sound and movement in comics is based on a tension between the conditions of enunciation (CE) and the statement (E), insofar as

• 7  Pierre Fresnault-Deruelle, « Le Fantasme de la parole », Europe, n° 720, 1989, p. 54-64.

6As early as 1989, Fresnault-Deruelle saw in comic strips a “rustle of voiceless signs” and “a swarm of the motionless,” which the reader apprehends by developing a “visual ear.”7 Such formulations emphasize the deeply paradoxical nature of the medium. Because of their material characteristics, paper comics expose the author to a series of cleavages between statement and enunciation. But what about digital media? Are the aforementioned paradoxes still active in a technological space that uses audiovisual content?

• 9  Olivier Jouvray is a comics scripwriter  and cofounder of Revue Dessinée, which specialises in rep (...)

10In other words, homochrony occurs when the user follows a predefined time sequence (watching a movie, listening to a piece of music, etc.). On the other hand, heterochrony allows the reader to browse inscriptions (texts, images, etc.) at his own pace, outside of any temporal dictates (ex. when reading a novel, a comic book, etc.). That said, it should be noted that for some authors of comics, the use of sounds can only lead to a homochronous reception and is therefore hardly reconcilable with the reading mode of comics. On this subject, we subscribe to Olivier Jouvray’s9 declaration that

• 11  Needless to say, these categories correspond to Michel Chion’s work on sound in cinema.  For examp (...)

11Our position will be more nuanced. While the addition of sound clips can effectively divest the reader of the privilege of choosing his reading rhythm — and thus come into conflict with a certain representation of reading - there are many examples where the sound fits in perfectly with the narrative. Moreover, it is important to emphasize that the use of sound material does not necessarily lead to a temporal dictate. It all depends on the type of sound used by the author. With regards to digital comics, we must first make a distinction between intradiegetic and extradiegetic sounds. The first are produced at the level of diegesis and are constitutive of the fictional universe. The second, on the other hand, are external to diegetic events and have reality only for the reader (for example the music of a film credits). Added to this is the need to distinguish between durative sounds (lasting for several seconds) and momentary sounds (not meant to stretch out over time). We thus obtain four categories of sounds, which are listed and explained in the following table:11

• 12  [Online] http://hobolobo.net/ [consulted on October 16, 2018].
• 13  [Online] http://professeurcyclope.arte.tv/revues/22/chapter/vide_ordures.html [consulted on Octobe (...)

13Of course, of the four types of sounds referenced above, intradiegetic sounds come closest to homochrony. In this case the author has two possibilities: either to remain in the heterochrony by creating a general sound environment that envelops the reading without constraining it (urban noises, bird songs, etc.), or he makes dialogues audible, forcing the reader to follow a homochronous flow of speech. The rest of this article aims to analyze these two cases. To do this, it seems necessary to link the use of sounds with the other two sensory modalities called upon when reading a digital creation, namely sight and touch (through an interactive interface). Our first analysis, which focuses on Stefan Živadinović’s Hobo Lobo of Hamelin,12 will show that the maintenance of heterochrony makes it possible to reinforce the reader’s immersion in the fictional space by establishing a semantically relevant intersensoriality between the eye, hand and ear. As for homochrony, its introduction into the narrative marks the loss of any gestural/interactive control over the work. Since digital technology is essentially a manipulable medium, the programming of a homochronous sequence implies a temporary suspension of any interactive participation. Stéphane Oiry’s comic Vide-Ordures13 will allow us to analyze the evacuation of the reader’s gesture by the sound component — an evacuation which, as we shall see, opens up interesting possibilities at the enunciative level.   

• 14  Anthony Rageul, « Le corps et le récit », Les carnets de la bande dessinée, 2012. [online] https:/ (...)

14Stefan Živadinović’s Hobo Lobo of Hamelin, a free adaptation of the Grimm brothers’ tale The Pied Piper of Hamelin, is an unusual creation in more ways than one. The work’s originality is due not only to its sound environment, which will be discussed below, but also to its parallax effects. Indeed, during the reading the user is invited to use the web browser to scroll horizontally through a long strip composed of three superimposed layers. But as researcher Anthony Rageul says, these “are offset from each other, moving at different speeds.” Thus the elements in the foreground scroll before our eyes faster than the elements of the layer behind it, and so on until the furthest background layer, the slowest of all. This involves a parallax effect that simulates the respective movements of objects toward and away from each other when the viewer is looking at them while moving.”14 Figure 1 attempts to illustrate the process put in place by the author. It shows an image divided into three layers: the foreground shows a character and a sign bearing the inscription “Hamelin” (on the right side of the image); the second layer shows the little village below; lastly, the third layer is mountains visible in the background. Each of these layers refers to a stratum programmed to scroll at a pre-defined speed.

• 15  We borrow the metapohor of the dive from Pascal Robert (cf. Pascal, Robert, La bande dessinée : un (...)

15Involving minimal interactivity (that of the simple scrolling of the horizontal elevator), Stefan Živadinović’s strata are set in motion by the reader and, due to the variability of their scrolling speed, generate a striking effect of perspective and depth. This leads to a powerful feeling of immersion, as if the reader sank into the fictional space and entered the strange city of Hamelin. And yet sound is used precisely to reinforce this “dive” into the diegetic space.15 To this end, the author has developed a sound environment composed of various sound effects: harmonica music, the muddled murmur of villagers, etc. As mentioned above, such sound effects are heterochronous by nature: their primary purpose is not to prescribe the rhythm of reading, but to accentuate the immersive experience that results from the parallax effect. The reading of Hobo Lobo of Hamelin thus relies on a subtle interaction between optics (the perception of parallax), haptics (the parallax mechanism is activated by the reader’s gest) and the auditory (the sound accompanying the eye and the hand). Sight, touch and hearing act in concert to give rise to an intersensory experience that is difficult to conceive of in paper form.

• 17  Edmond Couchot and Norbert Hillaire, L’art numérique : comment la technologie vient au monde de l’ (...)
• 18  To our knowledge, only few comics based on a scrolling reading integrate an automatized progressio (...)

17As Edmond Couchot and Norbert Hillaire point out,17 the programming of a digital work presupposes a dialogue between an “upstream author” (the creator of the work) and a “downstream author” (the receiver). The first programs and anticipates the doing of the second. However, the scroll speed of a scrollbar is potentially variable depending on the individual, so it can be difficult to get a perfect coordination between the gesture of the reader and the activation of the sound. A radical way to overcome this pitfall is to divest the reader of any control of the scrollbar. The latter is then handed over to the computer, advancing automatically for a more or less long period of time during which a sound fragment is heard. Of course, the direct consequence of such a process is to turn the reading into homochrony, since the reader literally loses his grip on the sequence of narrative units.18 It appears that interactive mechanisms — in this case the use of scrolling — lead authors to favor certain types of sounds (for example, ambient sounds in Zivadinović’s case) and to abandon others. The analysis of a second comic strip, whose reading is based not on scrolling but on clicking, will allow us to further develop this interrelation between interactivity and the sound dimension.  

• 19  Ferdinand (de) Saussure, Course in General Linguistics, trans. Roy Harris, New York, Bloomsbury, 2 (...)

18In certain works, the presence of intradiegetic sounds sometimes leads to a homochronous temporality. This is the case, for example, when the author chooses to make dialogues or monologues audible. The protagonists’ lines of dialogue are therefore no longer the purview of bubbles, but rather that of sound clips. On the reader’s side, the assertion of a vocalized utterance ipso facto signals the triumph of homochrony over heterochrony. Since Saussure we know that “auditory signals have available to them only the linearity of time. The elements of such signals are presented one after another: they form a chain. This feature appears immediately when they are represented in writing, and a spatial line of graphic signs is substituted for a succession of sounds in time.”19 By giving the characters a voice that is listened to (and no longer read), the digital medium reintroduces the linearity of the oral signifier in the reading experience. The spatialization of bubbles on the page’s surface is substituted for a phonic flow that the reader is invited to follow, without the possibility of escaping it (except by deliberately skipping over the passage in question, with the risk of missing important diegetic information).

• 20  [Online] http://professeurcyclope.arte.tv/ [consulted on October 16, 2018]. (...)

20A significant example of this fruitful encounter between sound and click is provided in Stéphane Oiry’s comic work, Vide-ordures. This work, published in the online journal Professor Cyclope20in September 2015, has the peculiarity of being punctuated by many of the author’s own lines of dialogue. The story opens with a strange confession of the main character, who declares aloud, “I have a strange relationship with garbage chutes.” This short sentence, not even three seconds long, introduces a homochronous listening time. The sound clip imposes its linearity and divests the reader of any interactive control over the course of diegetic actions. It should also be noted that the use of the first person, together with the indexical relationship that unites the speaking subject (Oiry himself) with his own voice, gives this story an autobiographical aura. Yet the retrospective character is constantly challenged by the display of heterochronous textual intertitles. Like subtitles in silent films, they intervene between passages with sound, commenting on them with a tone tinged with humor. Take for example the third caption, interposed between two pre-recorded lines:

• 21  André Gaudreault and Philippe Marion, La fin du cinéma ? Un média en crise à l’ère du numérique, P (...)

22We can now ask for ourselves the question raised by Philippe Marion and André Gaudreault: “Do comics loses their ‘epiphanic soul’ when digital possibilities allow them to open up [...] to homochrony?”21 We understand “epiphanic soul” to refer to the enunciative and semiotic characteristics of the medium as they are constituted on paper. As we have seen, the narrative of comics is based on a series of paradoxes, that is to say, cleavages between the conditions of enunciation imposed by the materiality of the page (the impossibility of resorting to the sound channel) and imperatives related to the construction of the statement (for example, to account for sounds emitted in the diegesis). This paradoxical situation has two consequences: on the one hand, it leads the author to construct a narrative on the basis of an exclusively graphic statement (bubbles, onomatopoeia, etc.), while on the other hand the graphic nature of the medium leads to a heterochronous type of reception, with the reader himself adjusting his reading rhythm. By authorizing the perception of sounds, digital technology evidently breaks with this paradoxical logic and consequently upsets the material, enunciative and reading properties of the medium.

• 22  On this subject, see Thierry Groensteen, Bande dessinée et narration, Paris, PUF, p.75.
• 23  Pascal, Robert (dir.), Bande dessinée et numérique, Paris, CNRS Éditions, 2016, p.17.
• 24  Jacques, Dürrenmatt, Bande dessinée et littérature, Paris, Classiques Garnier, 2013, p. 141.

23While this invitation to be listened to, and not only to be seen, has provoked some reticence in academic literature,22 it is clear that it opens up interesting possibilities in terms of immersion and reflexivity. Thus, in Hobo Lobo of Hamelin, the sound elements are part of an intersensorial apparatus that aims to plunge the reader (through sight, gest, and hearing) into the diegetic space — all the while, let us note, retaining a heterochronous reception. That said, it should be remembered that this projection into the fictional world is already at work in the paper medium. On this point Pascal Robert emphasizes that paper comics have “this ability to really project through the page itself [...] a 3D space to which the page gives access through the boxes and which the reader participates in exploring and rebuilding.”23 The sound environment developed by Živadinović, when thought together with the parallax effect, only increases the sense of immersion already present with printed sheets. Similarly, the polyphonic narrative observable in Stéphane Oiry’s Vide-ordures, which is based on a fruitful dialogue between sound and text, is part of what Jacques Dürrenmatt calls “complex enunciation techniques,” which were asserted “in the field of comics at the end of the 70s”24 (among these techniques, we can mention the intermingling of voices, the variations of points of view, etc.). While Oiry’s creation stands out from paper works, especially because of its use of homochronous sounds, it is also an extension of enunciative experiments that are well established in the history of the ninth art. Thus a dialectic is made, in the work of both Živadinović and Oiry, between rupture (use of sounds, resolution of paradoxes) and continuity (reinforcement of immersive feeling, enunciative complexity, etc.). The digital fate of the comics — of its “epiphanic soul,” as Marion and Gaudreault would say — undoubtedly resides in this articulation between continuity and rupture, between heritage and novelty; an articulation certainly difficult to manage for authors, but which proves to be most stimulating on the artistic and narrative levels.