1From the oldest to the most innovative (notably in the digital field), many technological processes allow for plays on the boundaries between the real and the virtual. In this article, I will examine some of these processes from the perspective of creation and reception, so as to enrich the definition of what I would call “the illusions between the real and the virtual” and provide creation with a new resource. I will quickly set out the theoretical founding principles of my field of study, before I introduce my research-creation approach. A single discipline is not sufficient to fuel the exchange between practice and analysis. As a consequence, I will resort to a complex methodology, drawing from various disciplines and fields of expertise, in order to sort out the links between art, technology and psychology, and ultimately establish a suitable ground for the emergence of new artistic forms.
2Before we explain what IRVs consist in, let us start by observing a simple optical illusion, in this case Müller-Lyer’s, representing two lines having the same length, contrary to appearances.
fig. 1
Müller-Lyer’s illusion.
- 1 Jacques Ninio, La Science des illusions, Paris, Odile Jacob, 1998, p. 45.
- 2 Jacques Ninio, La Science des illusions, Paris, Odile Jacob, 1998, 4e page de couverture.
3This illusion directly challenges sight, as well as our way of interpreting spatial dimensions. Indeed, as stated by Jacques Ninio in Les Sciences de l’illusion: “Scientists are interested in illusions because they hope to pull out the logic behind the normal functioning of perception.”1 In this sense, illusion is not only a deception of the senses, as numerous dictionaries remind us, it also “reveals how the brain processes sensory data.”2
4Besides, through the astonishment illusion provokes, it allows us to take a break and stand back from our own perceptions. Moreover, these effects also involve a pleasurable aspect: we like retry and be astonished again by the effects of illusion, as simple as they may be.
- 3 Cf. Michael Edwards, De l’émerveillement, Paris, Fayard, 2008.
- 4 Ernst H. Gombrich, L’Art et l’Illusion, Paris, Phaidon, 2002, p. 7. Also published in English: Art (...)
5Illusion places amazement3 within easy reach: our habits are turned upside down for a short moment, forcing us to take a fresh look at our senses and question our perception of reality. To quote Gombrich: “amazement is at the root of knowledge.”4
- 5 Jacques Ninio, La Science des illusions, Paris, Odile Jacob, 1998, p. 67.
- 6 Ernst H. Gombrich, L’Art et l’Illusion, Paris, Phaidon, 2002, p. 7.
6In L’Art et l’Illusion, Gombrich examines this tool with regards to artists who wish to represent reality. As far as art is concerned, many processes can be used to create an illusion. Painters have long since been aware that “the appearance of a color in a painting depends on the surrounding colors and lights, which influence each other.”5 According to him, illusions in the field of arts are not only the result of the artist’s work, but also “necessary instruments for the analysis of appearances.”6
- 7 Here, we are referring to innovation technologies, especially the “latest” technologies developed. (...)
7It is possible to categorize several types of illusions when it comes to those created with new7 technologies. In this sense, cinema is a striking example: the release of the Lumière brothers’ Arrival of a train at La Ciotat in 1895 provoked strong reactions, as this new medium simultaneously disturbed the audience and provided it with a new way of seeing and hearing. However, because of an adaptation phenomenon, this illusion becomes less surprising from a technical point of view, although it keeps on moving people through various narrations and creations.
- 8 An expression used in marketing and referring to the “fact that a product or service may cause the (...)
8Other processes will be created, reviving the “Wow effect,”8 such as stereoscopic processes, sound spatialization, high-frequency images. The transfer of these technologies from laboratories to various applications throughout the world, in particular in the field of art, paves the way for artists.
9Consequently, my research focus on different technologies contributing to the creation of a range of illusions, both by using new devices (like the ones hitting the market every year in various fields, notably in the field of virtual reality) and older technologies that may broaden contemporary researches.
- 9 Cf. Maurice Merleau-Ponty Phénoménologie de la perception, Paris, Gallimard, 1945, and Francisco V (...)
10To understand the locus of illusions between the real and virtual that I will explore, let us first note a few theoretical elements to help further explain this topic and my approach, which fall within a phenomenological approach.9
11Our perceptions occur in an unstable reality, which is the constantly updated result of our sensory-motor interactions with our environment. I do not consider illusion as an “error” of the senses. On the contrary, I am convinced that it rests upon a particular perceptive basis.
- 10 Denis Berthier, Méditations sur le réel et le virtuel, Paris, L’Harmattan, 2004.
- 11 Olivier Nannipieri, Les Paradoxes de la présence dans les environnements immersifs: de la réalité (...)
- 12 Cf. Paul Milgram et al., “Augmented Reality: A class of displays on the reality-virtuality continu (...)
- 13 Cf. Philippe Fuchs (ed.) et al, Le Traité de la réalité virtuelle, vol. 1-2, Paris, École des Mine (...)
- 14 Patrice Bouvier, La Présence en réalité virtuelle, une approche centrée utilisateur, PhD thesis de (...)
- 15 Among the artists in keeping with this trend, let us mention: Myron Krueger (Vidéoplace, 1970), Je (...)
12The virtual I am interested in refers to rebuilt, simulated or reflecting spaces, yet producing “real effects.”10 Following on from Olivier Nannipieri’s analyses, our approach tries to “focus on this phenomenon as the confluence of a technical device […] and a perceiving subject.”11 At this boundary between the real and the virtual, perceptions may spread out in several fields: in the real world, in the virtual world, or in a mixed real/virtual world.12 These perceptions are studied and exploited in the field of virtual reality (noted VR from now on) and mixed reality13 and represent a whole range of “creations of illusions”. One of the essential “illusions” specific to these fields is that of presence,14 which gives subjects the impression that they are in a real environment, that they are immersed in this environment and that their interactions are “natural,” while they are in fact in a virtual or mixed environment.15 Consequently, it is exactly at this point between the real and the virtual, and with the help of the “screen” provided by the illusion of presence and inherent to these devices, that I propose to study, experiment with and find the illusions that will serve as the tools for my artistic expression.
13Through the analysis of three installations, we will study how new illusions in virtual reality (VR) develop as artistic forms. These three experiments, conducted with different people, show similarities from the perspective of the perceptive device, while differing in terms of protocol, tools and objectives. The first one was carried out in the field of video games, the second was conducted in order to establish a experimental laboratory and the third one was part of a theatrical research in a trans-media concept. I will introduce these creations in chronological order, to show how some of these experiments on sensations have been confirmed and some analyses verified, thus resulting in the emergence of new artistic possibilities. Throughout this process, I will demonstrate how a creation dynamic may be developed, relying on the constant exchange between research and creation.
14In the same way as the choice of a canvas (dimensions, material, texture…) is crucial before one starts to paint, the choice of the medium (the VR device and a real/virtual scenery which we shall describe hereafter) is essential in the case of these three installations. Consequently, we will highlight the inherent characteristics of this medium, allowing for a certain number of variations on artistic explorations.
- 16 A Razer Hydra sensor mounted on the headset gives the position.
15• VR headset, Oculus Rift DK1: these headphones allow for a total immersion in the surrounding virtual world. It is equipped with orientation sensors: the movements of the head can be recorded and synchronized in real time with the virtual camera.16
- 17 Cf. James J. Gibson, “The Theory of Affordances” in Robert Shaw and John Bransford (ed.), Perceivi (...)
16• An interface object: connecting the real world with the virtual one. Reproduced in the same dimensions as the virtual surroundings, this object will be equally real and virtual. The user will be able to handle this object in one hand and will interact “naturally” with the virtual world. By means of a mounted position and orientation sensor (Razer Hydra), the movements of the real object will be synchronized in real time with the movements of its virtual copy. The objects that are used are everyday objects. Therefore, they offer “affordances,” which means that they express all the possible actions and hint at their own use.17
17• Headphones.
18• The software used for real-time creation is Unity 3D and its MiddleVR plug-in that allows the use of VR interfaces.
fig. 2
Oculus Rift DK1 and Razer Hydra.
19• The real space contains at least a table, a chair and the interface object.
20• The virtual space represents a closed room containing at least these same table, chair and interface object in the same dimensions, for the sake of coherence.
21It is technically essential to precisely calibrate the location of the aforementioned real objects, with their virtual counterparts.
22Each of these creations resorts to a different kind of object and element.
23I will not be able to explain in details all the research-creation process for every installation. Therefore, I propose to only give an overview of each of them, in order to highlight their development and outcomes.
- 18 Clarifying interviews by Pierre Vermersch and attendance questionnaires by Patrice Bouvier and Sté (...)
24In the course of the descriptions, we will notice that taking the spectator’s subjectivity into account is crucial. We wanted to answer the following question: how to make spectators interact with the installation and stay? In this perspective, we have analyzed their reactions. It should be noted that these analyses draw on methods falling within such disciplines as psychology or VR: questionnaires, measures or interviews18 likely to result in further studies and processing. My objective first consisted in collecting sensations, before experimenting on several places and contexts, comparing them following a more elaborate and global method, and going through experiments so as to identify some of their characteristics likely to be a source of inspiration in terms of creation.
- 19 Sébastien Kuntz, Judith Guez, Florian Costes, Charles Kaing, Florian Reneau and François Gutherz, (...)
25VRLux was created in 48 hours at the 2013 Global Game Jam (GGJ) by the VR association VRGeeks, which focuses, among other things, on all the forms of creation and exploitation of technologies related to VR. This experiment, more akin to a video game, invites the spectator, sitting before a table in a small room with a candle as the only object with which to interact, to solve riddles in order to proceed to the next level.
26The physical table and chair of the installation are typical desk furniture. In this first experiment, the interface object is a cylindrical candle, identically reproduced in the virtual world. The virtual space shows a tiny cubic room. Various objects are represented on the table, according to the riddles (every time a riddle is solved, the environment turns black before new objects are displayed on the table). The candle is lit in the virtual world, and all the characteristics associated with this object are used to enable the aforementioned natural interaction: for instance, the candle can be used to explore the dimly-lit virtual environment and burn some of the elements (a rope, for instance). The first issue was related to the effect of presence of the real and the virtual elements. Do real tangible elements convey more presence in this VR environment? Do they revive the effect of presence by further clouding the boundary between the real and the virtual?
fig. 3
VRLux, 2013.
© Sébastien Kuntz, Judith Guez, Florian Costes, Charles Kaing, Florian Reneau and François Gutherz.
27This installation was exhibited several times, in front of heterogeneous audiences possessing various degrees of VR knowledge. Despite these differences, users would always understand how to interact by turning their head and handling the candle. After the experiment, they would most often be very enthusiastic and express a strong desire to tell what they had just experienced.
28— At the first GGJ 2013 exhibition, spontaneous reactions showed us that the experiment seemed indeed realistic and that the impression of presence bore fruit. We heard people exclaiming “I’m totally into this!” or “this is gripping.” When asked: “As you saw the candle, did you know what to do with it?”, many people answered that it just came naturally. Some people even felt the heat as they reached for the candle (let me just remind you that the candle was only lit in the virtual world). The sound and sound effects were also said to contribute to the immersion.
29— Following from these reactions, I have drafted a questionnaire, in order to sharpen the results at the Laval Virtual show (March 2013) and the Press Start event held in the Bibliothèque publique d’information (BPI) of the Centre Pompidou in Paris (May 2013). The respondents described the experiment as “gripping,” “enjoyable,” “funny,” “surprising.” As a whole, the presence was well perceived and most of the time, it was the outside noise that would “draw” people out of the experiment.
30In conclusion, this experiment allowed us to confirm the real/virtual hybridization, both at the visual and tactile levels: the subject is “naturally” introduced into this mixed environment through known points of reference (the chair, the table and the candle allowing for an intuitive interaction). The familiar aspect of the scenery is reassuring and the homothetic dimension of the virtual place compared to the real one gives the experiment stability.
31At first, subjects are surprised by the fact that they cannot see their hand in the virtual world, but as they hold the candle, they realize it serves as a hand and allows them to focus on the interaction with the environment. The feeling conveyed by the closed space was decisive. We should probably improve the real environment and conditions of presentation, so that spectators are less often “drawn away” from their immersion.
- 20 Judith Guez, Guillaume Bertinet and Kevin Wagrez, Lab’surd, VR installation, Laval Virtual, 2014. (...)
- 21 Philippe Fuchs (ed.) et al., Le Traité de la réalité virtuelle, vol. 1-2, Paris, École des Mines d (...)
32VRlux offered a simulation of the real designed to immerse people in a virtual world with realistic “perceptive codes.” Is it possible to go beyond this realistic setting? The virtual indeed allows to simulate, but also to put forward imaginary worlds, as Philippe Fuchs reminds us in his VR treaty.21 Consequently, how are we to access these imaginary worlds? How to ensure the acceptance of new perception codes? Is it possible to overcome the illusion and simulation of reality that are inherent to the VR process, so that one accepts other perception modes? Thinking about the candle again, I wished it could transform, change shape or color once the adaptation done. The idea behind Lab’surd—designed as an experimental laboratory—seemed appropriate for the testing, adding and collecting of reactions. The objective was to offer a glass to a person sitting at a table and experiment with various kinds of illusion. A crucial question forced itself on us: how to create the disruptions allowing for the transition from the realistic simulation world to a more imaginary one and back? All magicians have their bag of tricks. Is there a way we can also fill our “bag of sensations” that is characteristic of these perceptions in mixed real/virtual worlds? One of our hypotheses consisted in considering that the visual illusion created with this VR device would result in these disruptions, allowing the spectator to switch from a realistic simulation world to a more imaginary one.
33Consequently, during the development of Lab’surd, we tried to create illusions likely to work in virtual worlds, with the help of the headset. We wanted to play with perceptions and check the extent to which the distortion of reality was still effective. In this perspective, we used an interface to program the order of effects, while illusions could be triggered at will via a keyboard. This flexibility gave us a modifiable ground for experimentation.
- 22 [Online] http://les-algoristes.org [accessed 15 September 2014]. (...)
34The installation was first designed to be exhibited in the Paris ACM SIGGRAPH stand at the Laval Virtual 2014 VR show. We were allotted a personal space, which we used to narrow the boundary between the real and the virtual. In the virtual world, we built a copy of the real stand in the same dimensions. Only the posters were different: they represented a few paintings inspired by the surrealists and Algorists,22 as well as a few optical illusions. The virtual table and chair were identical to the real ones and placed in a corner of the stand, in front of a wall. However, in the virtual world, we put a mirror in front of the user and a small red ball on the table. Besides, a small black box and our interface object—a big black plastic glass—were placed on this table, both in the real and the virtual worlds. The interaction simply consisted in using the glass to play with the virtual ball. After a while, we would trigger a few tricks where the table was. We have tried several methods to make the spectators move their head and distract them: using sound spatialization to have them believe something had just happened behind them (the crumpling of a piece of paper, a breaking glass, etc.); using the properties of the reflection in the mirror (balls falling down in the mirror’s background, hence in our back?); animating the posters so as to progressively give them life (by switching from 2D to 3D, for instance).
35To create “magical” moments, we have chosen a scenario that would become more and more explicit. We would start with hardly perceptible effects, instilling a subtle uncanny feeling, which we shall call “small changes” illusions: the ball in the reflection changes color, the movements in the reflection are reversed, the box on the table turns into an anamorphose. Once the disruption operated, we would trigger “big changes” illusions, or in other words, more perceptible illusions: mirrors becoming endless, the box placed in front of the spectator displaying a view of the stand, the table turning into a mirror, etc. We have also tested progressive changes on the interface object, the glass becoming limp or bigger.
fig. 4
Lab’surd, 2014.
© Judith Guez, Guillaume Bertinet and Kevin Wagrez.
36During the exhibition, we noticed that the interplay of the visual and tactile dimensions allowed for a better introduction of the experiment, like in VRLux. This is all the truer in this case, as the virtual environment was a faithful copy of the real environment, with a few minor exceptions. The sound environment was similar (background noise of a living-room), which even seemed to solve the problems induced by outside noise. The contextual environment (the stand) yet comprised one more modeled wall than the real scenery, which led several people to exclaim as they took the headset off: “Oh right! I forgot that there was no wall here in reality!” The sameness of the real and the virtual sceneries then facilitates the blurring of these two spaces of perception.
37Some effects seem indeed to be effective, such as the softness of the glass, the anamorphose (the posters), the balls in the mirror, etc. Even if the subject does not notice all of the effects (this is not the purpose, by the way), they seem disconcerted at first, trying to find answers and to understand what is going on (“I had the impression that something was going to happen behind me”). Then, as they spot some effects, they try to understand, until they give up and accept this environment and its unusual codes just to have fun. There is a rupture point that changes the subject’s interpretation of his/her experience, a moment when everything becomes possible (“I first saw the mirror—nothing special. Then I noticed the ball in the distance—I must have really felt the effect of illusion at this point, because I turned around and I couldn’t see the balls. Then I was completely into it! At one moment, I even thought I saw the flattened box…”).
38Repeated, this experiment enabled us to confirm the hypotheses on “illusions of rupture,” through which we are able to operate a transition from a virtual environment—quite similar to a simulation and with realistic perceptive codes—to a world based on largely unknown perceptions and leading to another imaginary, unrealistic world. The simulation > rupture > imaginary world scenario thus allows, through simulation, for the intuitive immersion of the spectator in the real/virtual environment and, through multi-sensory ruptures, to their progressively accepting to enter a world they did not expect. Therefore, I suggest that spectators adopt this approach to question their own reality and perception of the surrounding world.
- 23 Judith Guez, Émilie-Anna Maillet and Guillaume Bertinet, La Chambre de Kristoffer, VR installation (...)
- 24 In the “Laboratoire Européen Spectacle vivant et Transmédia,” at the Granit, scène nationale de Be (...)
- 25 This trend was initiated by Cie 14:20. As stated by Raphäel Navarro, this is an art form “whose la (...)
39This last experiment was created during a residency,24 as part of the Kant trans-media project, which follows on from a meeting with Émilie-Anna Maillet, director of the Ex voto à la Lune company. After her latest creation Hiver (Winter) by Jon Fosse, for which she used the Pepper’s ghost technique to generate sorts of holograms onstage, among real actors, she wanted to use this process again in her play for young audiences, entitled Kant. This play tells the story of an eight-year-old child named Kristoffer, who wonders about his life and the universe before sleep. The old Pepper’s ghost technique, still used today by the magie nouvelle25 movement, raises doubts by mixing real and virtual elements on stage. We wanted to make the most of this ambiguity, so as to play on sensations and hybridize the child’s imaginary visions and reality. In this perspective, we decided to complement the play, in which the actor plays the child on stage, with another perspective on this text, using an immersive and interactive installation. The question here is: is it possible to make the spectator feel the child’s experiences in a more subjective way, to shift his/her point of view through a change in temporality, to make him/her experience sensations from the child’s standpoint?
40I have chosen to take the “table/chair/interface object” device again, with a view to prompt this other vision. Together with set designer Céline Diez, we have created the real scenery showing a corner of the room—the virtual scenery representing the same room, in its entirety, so as to facilitate the spectator’s immersion in the mixed environment, as formerly confirmed. We have selected a bigger table and a bigger chair, to give the spectators the impression that they were smaller, with their legs dangling above the floor. The flashlight seemed to be a relevant interface object, according to the results of the other experiments: a tool likely to arouse the desire to explore and having an influence on the virtual environment, while being coherent with the narration of the play.
41Following on from our work on Lab’surd, we have based our scenario on the same structure: simulation > transition > imaginary world. Starting with the simulation of the room, we wanted the spectator to slowly slip into the child’s imaginary world. The transition was operated through a very progressive change in the objects found in the room (toy cars driving by themselves, toy planes taking off, globe turning…) and, little by little, the spectator would be transported into the child’s fantastic world. Then, the walls would start to decay, objects would become weightless and drift away, giving way to a space landscape: the spectator would be hanging in the “middle” of the Milky Way, surrounded by stars, planets and all the aforementioned objects levitating around him/her.
fig. 5
Kristoffer’s room, 2014
© Judith Guez, Émilie-Anna Maillet and Guillaume Bertinet.
42 After the residency, we had the opportunity to present the entirety of this installation to around fifteen people—most of them having absolutely no knowledge of VR and no experience with a VR headset. We were astonished by the ease with which that audience slipped into the real/virtual environment. Once seated, headset on and flashlight in their hand, spectators would observe, explore their surroundings, turn their heads. By creating a genuine theatre set, representing a real place (a child’s room), we were not as dependent on the representation codes of the real world, we had more freedom to create the dual real/virtual space and choose the most relevant level of similarity between the two sets, real and virtual. Induced by progressive changes in the scenario, the transition from the realistic room to its deconstruction in the universe was incredibly effective and had the spectator carried away by the narration.
43The three experiments we have described have revealed certain characteristics of that boundary between the real and the virtual that are of particular interest to me in terms of creation. These characteristics have allowed me to play with the feeling of presence inherent to these VR devices, so that I could design new illusions.
44Let us get back to our original analogy with painting. Like the colors of a painter’s palette, these illusions form:
-
the illusion of the play on perception between the visual and tactile dimensions,
-
the superimposition of virtual elements on the real ones,
-
the ruptures through progressive or sudden illusions,
-
the transition from the simulation to the imaginary world, via a transition phase.
45I use, combine, modify and transform these colors by applying them onto my canvas. In this case, the canvas was the VR device, consisting of the table, chair and interface object, while the effect of presence represented its texture, its inherent property.
46Finally, the way I use my palette on this canvas defines my relation to spectators: here, every one of the described installations—through their styles, contexts and objectives—allowed me to fine-tune the forms of illusion, by helping me to imagine particular relations with the spectators and design new ways for them to access their own perceptions.
47Through this overview of “a part of my palette,” the very instrument of my research-creation, I wish to give other canvases, as well as other art forms, the colors of the illusion between the real and the virtual.
