Home / Issues / 11.2014 / Connecting Interactive Arts and Virtual Reality with Enaction
Document Actions

VRIC 2010

Connecting Interactive Arts and Virtual Reality with Enaction

  1. Pierre De Loor Lab-STICC - UEB - ENI
  2. Kristen Manac'h Lab-STICC - UEB - ENI
  3. Charlie Windelschmidt Compagnie Dérézo
  4. Frédéric Devillers Lab-STICC - UEB - ENI
  5. Pierre Chevaillier Lab-STICC - UEB - ENI
  6. Jacques Tisseau Lab-STICC - UEB - ENI


This paper reports on a Virtual Reality theater experiment named Il était Xn fois, conducted by artists and computer scientists working in cognitive science. It offered the opportunity for knowledge and ideas exchange between these groups, highlighting the benefits of collaboration of this kind. Section 1 explains the link between enaction in cognitive science and virtual reality, and specifically the need to develop an autonomous entity which enhances presence in an artificial world. Section 2 argues that enactive artificial intelligence is able to produce such autonomy. This was demonstrated by the theatrical experiment, "Il était Xn fois" (in English: Once upon Xn time), explained in section 3. Its first public performance was in 2009, by the company Dérézo. The last section offers the view that enaction can form a common ground between the artistic and computer science areas.

  1. published: 2014-02-26


1.  Introduction

Cognitive science offers insights into human knowledge, producing models of use to artificial intelligence (AI). Within cognitive science, the area called enaction addresses the relationship between human sense-making and interaction, with one aim being the proposal of guidelines to improve virtual reality (VR) systems. Artists are also involved in sense-making, but not in a theoretical way: they intuitively make sense of the human experience by utilizing various interactive art systems. This paper shows that collaboration between artists and artificial intelligence researchers (in particular, enactionnists), is beneficial to both the theoretical and practical forms of VR.

Enaction is closely linked to phenomenology [ Hus91, VTR93, Noe04 ], which conceives, among other things, that sense comes foremost from the interaction between an agent and the way he perceives his environment [ MP90 ]. In addition, participation and co-ordination are important as a basis for meaning, as identified by [ DD07 ]. It is also necessary to realize that the actions of the subject, and others, contribute equally to enactive knowledge, since this affects the functional and dynamical properties of a VR system. This point is fundamental to the development of interplay between interactive virtual agents and humans.

All these features contribute toward the notion of "enactive artificial intelligence" which emphasizes autonomy, resistance, agentivity, identity, openness, and interaction in artificial systems [ FZ09, DMT09 ]. One of the many challenges is to find relevant use-cases that fully exploit the enactive AI model because of the degrees of freedom and evolutive capabilities exhibited by such models. Difficulties arise because the models are not based on classical computer science approaches, such as rules, facts and deduction, but on concepts borrowed from artificial life such as self-organization, self-adaptation, and co-evolution.

Virtual reality technologies have been utilized in various studies devoted to the theater and, more generally, interactive arts [ HG03, Cal04, CLC05, Ani11 ]. In the theater, work ranges from the use of virtual sets to the immersion of the audience within virtual worlds where the spectators become the actors [ Shy03 ]. Maintaining the illusion, the emotional context, and the existence of virtual elements are major challenges for artists [ Rea00 ]. These challenges become even more complex when the virtual scenery is populated by virtual actors. In the interactive arts, the system allow an observer to interact with an artistic representation while watching it [ Pau08, Ric09 ], and ideally this interaction should allow the observer to become a creator. This interaction is, sometimes, very reactive and easily understood by the user so that only a few movements can guide the system into a given state [ Lev06 ]. But the understanding of the interaction may be more difficult, especially when the virtual entity has some autonomy [ Sta10 ], causing it to become a virtual actor. We will argue that the user's engagement can be improved if the virtual actor has behavioral characteristics defined through enaction. To summarize, the successful co-presence of real and virtual actors is vital to interactive arts such as virtual theater, and can be achieved through the use of the "enactive AI" paradigm.

The paper is composed as follows: Section 2 argues for a refocusing of VR principles toward an enactive viewpoint, and explains the difference between this view and the more common VR approach. Section 3 introduces enactive AI, and describes how VR applications may be developed from this enactive perspective. Section 4 explains how these conceptual stances were utilized by the theater company Dérézo to create an enactive inspired theater production called "Il était Xn fois". Section 5 details the free ontogenetic behavior of an enactive virtual actor. Such an entity forms the basis for the reciprocal needs of artists and computer scientists in enactive AI, and also highlights why collaboration between artists and scientists is so important. These common interests are summarized in section 6.

2.  The links between VR and Enaction

Cognitive science researchers sometimes express surprise at the lack of epistemic foundations for VR. For example, physics is formalized as theories and models about the world. These models can be used to develop technologies, e.g. the combustion engine or particle accelerator, or to understand technologies that were developed before the complete establishment of relevant theory. The epistemic problem for VR lies in providing better definitions for suitable models and theories. This is an ambitious issue, and will likely take years.

Cognitive science researchers are interested in VR because of the intuitive links between human knowledge and reality. "Virtual" reality may be, at minimum, a means to gain a better insight into these links. Most VR system developers make the assumption that there is a separation between our bodies and our environment. They assume that our bodies perceive an external and unique environment, analyses it, and then acts on it to achieve a predefined goal. According to this view, VR consists in replicating bodily perception and action [ WCP07, MN99, CC07, LAVK06 ]. From a cognitive science perspective, this approach is "debatable". One alternative is the enactive paradigm, created in the 1980s by two biologists, Humberto Maturana and Francisco Varela, building on the notion of the autonomy of life [ Var79, MV80, VTR93 ]. According to enaction, cognition is "the history of structural coupling that brings out a world", and it is illusory to try to understand an entity by separating it from its surroundings. Merleau-Ponty [ MP90 ] illustrates this point with the example of a blind man with a stick which he uses to "touch" and "see" the world. For him, the stick is not a tool separate from his body, but part of him, used as a kind of eye.

The separation of body and environment can be questioned in most forms of human technology use, such as the intuitive car driver, pilot, or machinist. And what of a book reader, who starts to shudder and feel emotions because she is so immersed in the story? According to enaction, an understanding of knowledge must involve the history of the coupling between the entity and the world. This coupling also implies reciprocity because as the agent modifies the environment so in turn does it modify the agent. This dynamic interplay is more fundamental than a predefined and perceptible reality, unchanging over time. Enaction introduces the notion of the co-evolution of the agent and its environment. There are close links between constructivism, phenomenology, and enaction since we must interact in order to gain feedback. This information helps us to create representations of the world in terms of sensorimotor invariants. Other important enaction-related elements are: i) a person's first experience is considered fundamental, and ii) the awareness associated with a first experience is necessary for explaining cognition; this necessity is ignored by other paradigms, such as cognitivism or connectionism. For an in-depth explanation of the paradigm, the following studies should be consulted [ VTR93, McG05, McG06, Gap11 ]. Enaction is highly relevant to VR because it places interaction and dynamics at the center of the knowledge construction process.

Three areas of research illustrate how enaction may contribute to the theory missing from VR.

  1. Realism/credibility debate. In VR, there is no consensus among researchers about the way to evaluate "presence" [ ZJ98, SSKM01, WN02, Sla04, SVS05 ]. For example, the skills of the novel writer may allow a reader to feel more "present" inside the text than in front of a boring movie using the most recent rendering technologies. This indicates that realism has little to do with objective measures of contextualization. However, enaction allows us to re-focus the debate. Enactionnists study the history of the individual rather than objective measures of behavior. They look at the coupling between humans and their environment, and examine the invariants found between an individual's actions and what the environment offers in return. Therefore, VR should also focus on this coupling, and on the variability in scenarios/stories, that forms the VR application's interactions. Users may not feel presence in terms of objective realism, but they find themselves "engaged" with a story or an interaction. This generates motivation, interest, and a desire to return to the narrative at a later date. These considerations are a source of concern not only for the entertainment industry, but also for successful training and serious games.

  2. Technical/science debate. It seems that humans have always had the ability to distinguish the real from the virtual. Human plasticity allows us to adapt to new technologies and to "detect" their quirks and drawbacks. As a consequence, every new technology, such as immersive VR rigs, high-definition stereoscopic displays, and device-less interacting systems, is met with enthusiasm initially, but is gradually relegated to the status of "still imperfect". A meaningful category are the sensory substitution devices that show how humans can adapt to modes of perception different from those they naturally use [ AHLO05, Gap11 ]. One example is the "projection" of a camera image onto stimulators on a user's tongue which show how the brain can adapt to "seeing with the tongue" [ BTK03 ]. Of note is the fact that although we have the ability to adapt to new sensory inputs, very few humans use this ability to create different "representations" of the world. This indicates that the challenge of such systems is not purely technical, but also epistemic. Beyond the contribution of technology, what ultimately matters for the successful design of VR devices is an understanding of the knowledge created and the new forms of sense-making. The enaction stance which acknowledges that knowledge comes from sensorimotor invariance, is well-suited to the analysis of the results from sensory substitution experiments. Moreover, it is the only cognitive science paradigm that addresses sense-making [ DRJ11 ]. Hence, enaction is a good way to examine VR as something more than purely a technical challenge.

  3. Automatic/Autonomy debate. How can a virtual world be more that just a place that reproduces the perceptions and possibilities for action? According to phenomenology and enaction, sense-making arises from the interaction between an agent and an object or another agent. The key notions are the dynamics and expandability of the coupling between humans and their environment. Thus, the VR user must be able to construct invariants, and progressively evolve them through interaction. In this way, she constructs a world, understands it, and believes in it. In a virtual environment populated by virtual entities, this has a significant impact on the required features: the entities must be interactivly autonomous and be able to evolve. Unfortunately, the usual computational models employed by autonomous entities cannot easily account for the dynamics and uncertainty related to interaction with humans. Some recent studies have tried to introduce coupling properties into human interactions with a virtual host to better encourage social resonance [ BPD09, vWRZ10, Kop10 ]. However, these approaches face the problem that although the virtual entities can adapt, they cannot evolve during the course of the interaction. Consequently, the coupling between the user and such entities is weak because it is impossible for the user to evolve the link toward a unique dynamic. Indeed, the entity should not only evolve according to its interactions with the user, but also show resistance, which is a sign of greater autonomy. Evolution needs to be progressive, passing through phases of varying complexity, and requires collaboration between the sensorimotor and the cognitive aspects. The integration of both aspects is a prerequisite for permiting a more coherent co-evolution between reality and virtual worlds.

    We will address these topics through the notion of enactive AI.

3.  Interactive autonomy for sense-making in VR

This section sets out our arguments for the use of enaction to address the construction of artificial entities capable of co-evolution with users in a VR application. Such entities need to be resistant, while exhibiting invariances that support the constitution elements of sense. To build inter-subjectivity between users and an entity, we will employ models based on phenomenological and enactive considerations. Enactive AI must have the capability to actively regulate its structural coupling in relation to a viability constraint [ FZ09 ]. In artificial life, existing approaches insist on the autonomy of artificial entities, which address the simulation of artificial autopoiesis [ McM04, ED09 ]. Even though the clarification of autonomy is fundamental in the enactive viewpoint, it is insufficient to create virtual agents that can sense and interact with humans. Interestingly, the problem of self-organization, within an enactive perspective, is also addressed in robotics [ SF08 ]. Theoretical principles have been developed that lead to the creation of sensorimotor invariants in virtual entities.

This has been achieved using dynamical models, such as recurrent neural networks, able to approximate complex systems and maintain a dynamic without input from control-like devices. In these systems, the environment is nothing more than an element that disturbs the dynamic.

Our analysis of numerous enactive AI studies in [ DMT09 ], led us to a formulation of enactive based AI. It is less attached to the notion of identity because of the uncertainty about the direct link between biological and artificial properties. Nevertheless, we believe that VR applications should push further against some of the artificial limits imposed in previous work on enactive based AI. Our argument relies on: i) the lack of behavioral ontogenesis in current enactive based artificial agents, and ii) the need to add humans into the loop to further develop a relationship between human sense-making and enactive based artificial agents.

3.1.  Behavioral Ontogenesis

Behavioral ontogenesis is related to the capability of an enactive based entity to evolve in response to its environment. Although current models of autopoiesis and enactive robotics use complex notions of emergence to characterize their general behavior, their ontogenesis is relatively simple. Autopoiesis and stability are the typical foci of attention in artificial life, to the detriment of evolution. However, agent behaviour in evolutionary robotics is defined using an evolutionary approach, but its Darwinian inspiration (often a fitness criterion) creates a bottleneck for behavioral ontogenesis. Namely, since high fitness corresponds to a good behavior, then an existing good-enough behaviour may prevent the agent from evolving. This indicates that the main challenge in evolutionary robotics is to find fitness functions which accommodate a behavioral ontogenesis capability. This problem was addressed in [ ML09 ] which proposed an incremental evolutionary approach that allows enactive based agents to learn behavior from environmental interactions.

3.2.  Sense-making

Despite the issues detailed in the previous section, let us assume the existence of a virtual enactive entity capable of co-evolution, and of VR techniques and computational models that allow us to simulate its environment. Under these conditions, the virtual enactive entity and the environment will co-evolve and engage in "uncontrollable natural derivations". Enaction views the subject world as the result of an agent's actions upon its senses. As a consequence, the sensorimotor invariants evolving at the center of the artificial system can be equated to "virtual sense-making" in the virtual world.

It is worth considering what this type of sense-making means for an artificial system co-evolving with another artificial system. We must be wary of anthropomorphism, which is inappropriate here because the construction of meaning and sense for such entities cannot be compared to that of humans. In our opinion, human interaction with artificial entities which evolve through co-operation, can only be assigned a meaning that emerges from interactions with a human observer. We are not questioning the value of experiments in evolutionary robotics aimed at understanding fundamental cognitive principles, but rather addressing the sense-making problem. Nevertheless, one must be cautious of the potential impossibility of attaining such knowledge, as argued for the notion of autonomy in [ RS08 ]. Our aim is to move closer toward one of the themes of AI: the confrontation between a human user and an artificial agent [ Tur50 ]. We are following an enactive approach, which explores the sensorimotor confrontation between Man and artificial agent.

Man must feel the "presence" of the artificial agent, which expresses its meaning as sensorimotor resistance. This notion of presence can be evaluated subjectively, in a similar manner to the TURING test, as illustrated by various VR studies, such as [ AHLO05, SVS05 ]. In this way, we can link phenomenology and enaction-based AI,which further indicates that the involvement of humans in this co-evolution will enable all the participants (human and artificial agent) to create meaning. If Man is not a part of this loop, then there is no intelligent system, from his point of view. But if Man is involved, then the emergence of a user-world is favored by the coupling. This raises the issue of the interaction mode employed between Man and the artificial agent.

3.3.  A new enactive based A.I. perspective

Enactive based AI lets us define a perspective formed from more than just the regulation of a structural coupling with a viability constraint. In addition, we can design ontogenetic mechanisms for complex dynamical systems controlled by people. This objective is illustrated in Figure 1.

Figure 1. An artificial entity based on the enaction metaphor.

An artificial entity based on the enaction metaphor.

Artificial entities become complex systems enriched with ontogenetic mechanisms, which supervise their evolution via an "en habitus deposition" of their interactions [ Hus91 ]. This supervision can be conducted via a simulated environment, and must include human interaction. Our view is that the user's body has to be involved in this sensorimotor and cognitive coupling achieved through VR interfaces. In an enactive perspective, this leads to an understanding of the co-emergence of the senses during the course of an interaction.

Our approach echoes interactive art concerns, which were revealed to us during the development of the theatrical experiment "Il était Xn fois", in collaboration with members of the Dérézo company, which is described in Section 4.

4.  Enactive-inspired theater

The "Il était Xn fois" (in English: "Once upon Xn time") staging forms an element of the "Virthea"  [1] research project, which promotes a creative synergy of scientific and technological inputs. It also points to deeper questions about the evolution of technologies and their uses, and especially how they may become instruments for submission rather than tools (or even companions) for personal development. One aim of the Dérézo theatre company is to defend the re-appropriation of these tools by the artist, raising the possibility of using technology to preserve free will, emotion and creativity.

‘‘Il était Xn fois" has been performed three times in public, and a video showing its main phases is available at the Virthea site  [2]. The staging begins with an Internet-based session, where actresses interact via video with an Internet user. Only at the end of this part, is a venue indicated for the second session.

Figure 2. Part 1 of "Il était Xn fois", in real time on the web.

Part 1 of "Il était Xn fois", in real time on the web.

The second part of "Il était Xn fois" associates light, sound, and video with body movement and space. The narrative speaks of "a dream-like tale that evokes imagination and the unconscious, through the story of an individual's inner adventure, a self-initiation, wrought by the interrogation of multiple representations of the self". "Il était Xn fois" involves an actress in digital writing, whereby she randomly acts upon and learns from machines, and they learn in return. The performance highlights the phenomenology-inherited concept of co-constitution between reality and virtual worlds.

Figure 3. Part 2 of "Il était Xn fois", utilizing three interactive screens.

Part 2 of "Il était Xn fois", utilizing three interactive screens.

One goal is to surpass technological, letting it be "forgotten" for the benefit of theatrical and emotional aspects. This is linked to the notions of presence and engagement which occur during interaction, and are sources of concern for VR researchers. The artists highlight these concepts by showing that they relate to techniques for forgetting. Emphasis is placed on the emotional dimension and the suggestion that forgetting is not necessarily synonymous with realism.

This was made possible thanks to a collaboration with VR specialists. Figure 5 describes the technical setup we used: A tri CCD camera captures bursts from a light pen which are converted into the drawing of shapes on large display screens (Figure 3).

Figure 4. The actress dressed in the motion capture suit and her avatar.

The actress dressed in the motion capture suit and her avatar.

Another tri CCD camera detects the color of the hairs and teh hands of the actress. A video processing programmed with jitter/Max/MSP [ Lyo12 ] enables to trigger sounds and to vary their speed and/or intensity in relation with the position of the head and the hand of the actress. A central element is the use of a motion capture suit (moven system) to embody interaction between the actress and her "double" (or avatar), which is rendered onto the screens (Figure 4). The stage director introduces in real time some perturbations and modifications. This was achieved through the AReVi tool (a virtual reality software developed by scientists [ RHM98 ]) in addition with video traitment by jitter.

Figure 5. Technical aspect of the show.

Technical aspect of the show.

The body/avatar link addresses a principal concern in enactive embodied cognition: the importance of the body in the search for meaning. For us, this point is far more important for users to feel their presence into the virtual environment than to place them into a full immersive system offering a first-person point-of view using 3D tracking. Moreover, such systems do not suit the constraints of theatrical live shows with a public audience, since very few persons can be tracked.

An artistic challenge is to present both the avatar's appropriation by the actress and its empowerment. These elements echo the scientific problems described in Section 3 concerning the supervised ontogenesis of autonomous entities. Our hope is that the artificial model experiments reported in the next section will be used by scientists and artists to address the key issue of co-evolution.

Figure 6. The avatar becomes autonomous.

The avatar becomes autonomous.

5.  Free ontogenetic behavior of an enactive based virtual entity

As explained in Section 3.1, the empowerment of enactive based artificial agents is challenged by problems linked to the evolutionary approach. The underlying question is: can we address ontogenetic behavior without predefining behavior? One possibility is to start with a freely evolving behavior, called a free ontogenetic behavior, which does not require any precise character to be defined for the entity. The only prerequisite is that the behavior must evolve through interactions. This approach should appeal to artists, since a free behaviour is one they can appropriate, and use intuition and feeling to guide the changes to the agent's abstract representation and mood.

In Section 3, we mentioned how the dynamic models used in robotics suffer from limitations in the evolutionary approach. Nevertheless, they do offer a way to create complex dynamics. In this spirit, we employed Continuous Time Recurrent Neural Networks (CTRNNs) [ Bee95 ], due to their utility for approximating dynamical systems [ FN93 ]. A network is composed of fully-connected units, with the the evolution of each unit computed by using a differential equation dependent on both its own state and the states of all the other units.

The dynamic properties of CTRNNs make them a good candidate for modeling the behavior of interactive and evolving artificial entities. In particular, their ability to retain internal dynamics without input enhances their suitability for enactive based artificial intelligent entities. Nevertheless, their internal dynamics can be disturbed bythe application of input signals to some of the network units. This may lead to a shift in the network's dynamic, which can then be shaped through interaction with the overall entity.

Figure 7. Oscillatory activity of one CTRNN unit due to three different input profiles: (a) no input to the CTRNN; (b) the application of a constant input; (c) the application of a sequence of various inputs. Each color corresponds to a specific value in the input signal.

Oscillatory activity of one CTRNN unit due to three different input profiles: (a) no input to the CTRNN; (b) the application of a constant input; (c) the application of a sequence of various inputs. Each color corresponds to a specific value in the input signal.

These properties are illustrated in Figure 7 by plotting the activity of a single unit from a 5-unit CTRNN under three different conditions. The network parameters were chosen from randomly generated sets based on the rich internal dynamics they exhibit. The comparison of plot (b) with (a) show how change is induced in the oscillation frequency by external perturbations. Of particular note is how activities are not systematically alike for the same inputs; indeed, over the two grey-colored periods, the dynamics are totally different, despite receiving similar inputs.

This illustrates the sensitivity of CTRNNs to the interaction process, and shows that perturbations induced by interaction can lead the activity toward different attractors.

A user interacting with an entity governed by such a VR-based model might be able to discover some regularity in the enactive entity's responses to her actions. However, she might not immediately understand the interacting behavior because of the complexity of the entity's internal dynamics. For that reason, we ensure that the interaction process exploits the evolving abilities of the model, which leads to an ontogenesis of the artificial behavior.

We carried out another experiment to illustrate the impact of an interaction process on an enactive entity when coupled to an environment shared with a user. An artificial entity was placed in a 3D environment, and controlled by the same CTRNN as in the previous experiment, thereby allowing the user to control an avatar in the environment. The enactive entity was able to "perceive" the avatar through a perturbation of its CTRNN generated by changes to its position. It could also move in the environment by computing a displacement using the unit's activity. Figure 8 illustrates the trajectory of the avatar in 3D space in response to the user's activity. Figure 9 is a plot of the simultaneous trajectory of the enactive entity in the same space. The same color code is used in both figures to show the evolution of the enactive entity and the user's avatar over time: black corresponds to the time t = 0 and yellow to the end of the simulation. This information allows us to formulate a dynamic for the two entities' coupling.

Figure 8. 3D trajectory of the user's avatar coupled with the enactive entity (see Figure 9)

3D trajectory of the user's avatar coupled with the enactive entity (see Figure 9)

Figure 9. 3D trajectory of the enactive entity coupled with the user's avatar (see Figure 8)

3D trajectory of the enactive entity coupled with the user's avatar (see Figure 8)

In this experiment, the activity of the enactive entity was modified by the user's moves. These were controlled by the coupling so the user could carry out sense-making. In this way, the evolution of the virtual enactive entity was affected by the user's behavior, which led to a form of co-evolution. This experiment constitutes a first step toward behavioral ontogenesis. The result would be an assimilation of user interaction into the internal dynamics of the entity, so that moves by the user would progressively affect the entity's moves.

6.  Enaction as a common ground for artists and scientists

There are many reasons why it is worth establishing collaborations between artists and scientists working at the boundaries between computer science and cognitive science. For example, the Dérézo company previously relied on simple numerical models, controlled by a stage director, to manage the relationship between an actress and her avatar. Our collaboration meant they were able to utilize enaction theory to build superior artificial interactive models. The current situation in the rest of the interactive arts is similar to the previous approach of the the Dérézo company - there is little attempt to address the ontogenesis of autonomous entities, and so user interactions always have the same result upon a model.

We believe that the Dérézo theater company case stands as a good argument for the utility and benefits of our proposition. For example, the "Il était Xn fois" experiment allowed us to identify several links between VR research from phenomenological and enactive perspectives. Indeed, the play gives prominence to sense-making arising from a system-human coupling, which is the basis of enaction theory. It addresses the issue of credibility while disregarding the virtual, and examines empowerment. All of these topics have been the focus of scientific research in artificial enactive systems.

The Dérézo virtual theater considers how humans can "learn from machines, and how the machines can learn in return". This clearly parallels the concepts of co-evolution and co-constitution in phenomenology.

The Dérézo theatrical strives to represent the unrepresentable, which matches the concerns of enactionnists. The notion of representation as a simple theoretical tool is unfounded and limits our understanding of cognition.

Theater addresses many other dimensions, among them the idea of creative engagement [ BEC08 ], which is also important for interactive art. This attribute is poorly integrated into paradigms such as enaction, and suggests a fruitful direction to explore for a better understanding of VR models. Conversely, the inherent complexity of enactive autonomous entities is not necessarily suitable for a theatrical aesthetic. This suggests the prospect of finding compromises between the principles of enactive based AI and VR. Since artists are primarily involved in sense-making, and familiar with the manipulation of abstract representations, they are ideal interactive users to test and develop ontogenetic behaviors for enactive based entities.

7.  Conclusion

Enaction greatly benefits VR applications that utilize autonomous entities. The justification lies in the observation that human sense-making is a key point rarely addressed in VR, and enaction brings an understanding of sense-making to the cognitive sciences. Enaction also highlights how interaction between an agent and its environment is the basis of sense-making. With enaction, challenges such as the issue of presence are placed into perspective as the dynamic of interaction between a user and a VR system. Artificial entities inside virtual environments must exhibit behavioral properties that allow the human/entity coupling to evolve toward a specific dynamic. It must include enaction with a subjective world which means that the entities cannot be pure automata; they must exhibit behavioral ontogenesis. These cognitive science topics have close parallels with artists' concerns, in essence because both disciplines must address the issues of human sense and subjectivity. This paper used the Dérézo company as a case-study of how interactive arts are interested in the evolution of the coupling between an actress and her avatar. Artists need their virtual entities to exhibit the same kind of behaviors as the entities relevant to enaction and VR. Enaction can act as a basis for incorporating behavioral properties into virtual agents, the key being enactive AI for which intrinsic autonomy is ontology. However, this research field faces a dilemma: a byproduct of developing truely autonomous models is that the behavior of these models can no longer be mastered. This problem can be addressed with enactive based AI and the collaborative help of artists who are familiar with the concerns of the abstract world and sense-making. We described an example enactive-based agent able to interact with a user's avatar by means of free ontogenetic behavior. It maintains a history of the interactions in the internal agent dynamic, which guides the evolution of its coupling. In summary, a strong collaborative link between artists, enactionnists, and AI researchers could result in a serious enhancement to the foundations of VR.

8.  Acknowledgments

This work has been supported in part by French National Research Agency (ANR) through CONTINT program (project INGREDIBLE no. ANR-12-CORD-0001).


[AHLO05] Malika Auvray Sylvain Hanneton Charles Lenay Kevin O'Regan There is something out there: distal attribution in sensory substitution, twenty years later Journal of Integrative Neuroscience,  4 2005 4 505—521 0219-6352

[Ani11] Animação Animação: Marionnettes Virtuelles Mario.net/IRIT 2011 http://www.numericircus.netLast visited January 29th, 2014.

[BEC08] Zafer Bilda Ernest Edmonds Linda Candy Designing for creative engagement Design Studies,  29 2008 6 525—540 DOI 10.1016/j.destud.2008.07.0090142-694X

[Bee95] Randall D. Beer On the dynamics of small continuous-time recurrent neural networks Adaptive Behavior,  3 1995 4 471—511 DOI 10.1177/1059712395003004051059-7123

[BPD09] Elisabetta Bevacqua Ken Prepin Etienne De Sevin Radoslaw Niewiadomski Catherine Pelachaud Reactive behaviors in SAIBA architecture Proc. of the 8th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2009),  2009 Keith S. Decker Jaime Simão Sichman Carles Sierra Cristiano Castelfranchi(Eds.) pp. 9—12 Budapest, Hungary May 10-15th, 2009.

[BTK03] Paul Bach-y-Rita Mitchell E. Tyler Kurt A. Kaczmarek Seeing with the brain International Journal of Human-Computer Interaction,  15 2003 2 285—295 DOI 10.1207/S15327590IJHC1502_61044-7318

[Cal04] Jørgen Callesen Virtual puppets in performance Proceedings of the International Symposium Marionette: Metaphysics, Mechanics, Modernity,  2004 Copenhagen University.

[CC07] Biswarup Choudhury Sharat Chandran A Survey of Image-based Relighting Techniques Journal of Virtual Reality and Broadcasting,  2007 4 7 GRAPP 2006 Special Issueurn:nbn:de:0009-6-212081860-2037

[CLC05] Marc Cavazza Jean-Luc Lugrin Sean Crooks Alok Nandi Mark Palmer Marc Le Renard Causality and Virtual Reality Art Fifth International Conference on Creativity and Cognition (C&C'05),  New York, NY, USA ACM Press 2005 pp. 4—12.

[DD07] Hanne De Jaegher Ezequiel Di Paolo Participatory Sense-Making. An Enactive Approach to Social Cognition Phenomenology and the Cognitive Sciences,  6 2007 4 485—507 DOI 10.1007/s11097-007-9076-91568-7759

[DMT09] Pierre De Loor Kristen Manac'h Jacques Tisseau Enaction-Based Artificial Intelligence: Toward Co-evolution with Humans in the Loop Minds and Machine,  19 2009 3 319—343 DOI 10.1007/s11023-009-9165-30924-6495

[DRJ11] Ezequiel Di Paolo Marieke Rohde Hanneke De Jaegher Horizons for the Enactive Mind: Values, Social, Interaction, and Play Enaction: Towards a New Paradigm for Cognitive Science,  John Stewart Olivier Gapenne Ezequiel Di Paolo(Eds.) MIT Press Cambridge, MA, USA 2011 33—87 0262014602

[ED09] Matthew D. Egbert Ezequiel Di Paolo Integrating Autopoiesis and Behavior: An exploration in computational chemo-ethology Adaptive Behavior,  17 2009 5 367—386 DOI 10.1177/10597123093438211059-7123

[FN93] Ken-ichi Funahashi Yuichi Nakamura Approximation of dynamical systems by continuous time recurrent neural networks Neural Networks 6 1993 6 801—806 DOI 10.1016/S0893-6080(05)80125-X0893-6080

[FZ09] Tom Froese Tom Ziemke Enactive artificial intelligence: Investigating the systemic organization of life and mind Artificial Intelligence,  173 2009 3-4 466—500 DOI 10.1016/j.artint.2008.12.0010004-3702

[Gap11] Olivier Gapenne Kinaesthetics and the construction of perceptual objects Enaction: Toward a new paradigm for cognitive science,  John Robert Stewart Olivier Gapenne Ezequiel A. Di Paolo(Eds.) MIT Press 2011 0262014602

[HG03] Martin Hachet Pascal Guitton Using Virtual Reality for "New Clowns" Proceedings of second International Conference on Virtual Storytelling (ICVS),  2003 LNCS 2897 pp. 211—219 Springer-Verlag DOI 10.1007/978-3-540-40014-1_26978-3-540-20535-7

[Hus91] Edmund Husserl Expérience et jugement: Recherches en vue d'une généalogie de la logique Presses Universitaires de France 1991 2-13-943855-5

[LAVK06] Tobias Lentz Ingo Assenmacher Michael Vorländer Torsten Kuhlen Precise Near-to-Head Acoustics with Binaural Synthesis Journal of Virtual Reality and Broadcasting,  3 2006 2 urn:nbn:de:0009-6-58901860-2037

[Lev06] Golan Levin The Table is The Score : An Augmented-reality Interface for Real-Time, Tangible, Spectrographic Performance Proceedings of 2006 International Computer Music Conference,  2006 http://www.flong.com/texts/publications/pub_icmc06/, Last visited January 29th, 2014.

[Lyo12] Eric Lyon Designing Audio Objects for Max/MSP and Pd A-R Editions 2012 The computer music and digital audio series9780895797155

[McG05] Kevin McGee Enactive Cognitive Science. Part 1: History and Research Themes Constructivist Foundations,  1 2005 1 19—34 1782-348X

[McG06] Kevin McGee Enactive Cognitive Science. Part 2: Methods, Insights, and Potential Constructivist Foundations,  1 2006 2 73—82 1782-348X

[McM04] Barry McMullin Thirty Years of Computational Autopoiesis: A Review Artificial Life,  10 2004 3 277—295 DOI 10.1162/10645460412555481064-5462

[MD09] Kristen Manac'h Pierre De Loor Guiding for Associative Learning: How to Shape Dynamical Cognition Advances in Artificial Life, Darwin Meets von Neumann,  2009 George Kampis Istvan Karsai Eors Szathmary (Eds.) LNCS/LNAI 5777,5778189—196 Springer DOI 10.1007/978-3-642-21283-3_24978-3-642-21282-6

[MP45] Maurice Merleau-Ponty Phénoménologie de la Perception Éditions Gallimard 1990 Collection "Tel"Paris, France.

[MV80] Humberto R. Maturana Francisco J. Varela Autopoiesis and Cognition: The realization of the living Reidel Dordrecht 1980 90-277-1015-5

[Noe04] Alva Noë Action in Perception MIT Press Cambridge, Mass 2004 0262140888

[Pau08] Christiane Paul Digital Art Thames & Hudson, Ltd  London 2008 2nd Edition World of Art Series978-0-500-20398-9

[Rea00] Mark Reaney Art in Real-Time : Theater and Virtual Reality Séminaire CIREN,  2000 Université Paris 8, Saint-Denis, France http://www.ku.edu/~ mreaney/reaney/ciren, Last visited October 29th, 2013.

[RHM98] Pierre Reignier Fabrice Harrouet Serge Morvan Jacques Tisseau Thierry Duval ARéVi: a virtual reality multiagent platform Virtual Worlds Jean-Claude Heudin Springer Berlin, Heidelberg Lecture Notes in Computer Science1434 1998 229—240 DOI 10.1007/3-540-68686-X_22978-3-540-64780-5

[Ric09] Francisco Ricardo Literary Art in Digital Performance: Case Studies and Critical Positions Continuum Press New York, NY, USA 2009 978-0-8264-3600-9

[RS08] Marieke Rohde John Stewart Ascriptional and 'genuine' autonomy Biosystems Special Issue on Modeling Autonomy,  91 2008 2 424—433 DOI 10.1016/j.biosystems.2007.05.0170303-2647

[SF08] Mototaka Suzuki Dario Floreano Enactive Robot Vision Adaptive Behavior,  16 2008 2-3 122—128 DOI 10.1177/10597123080891831059-7123

[SSKM01] Martijn J. Schuemie Peter Van Der Straaten Merel Krijn Charles Van Der Mast Research on Presence in Virtual Reality: a Survey CyberPsychology & Behavior,  4 2001 2 183—201 DOI 10.1089/1094931013001178841094-9313

[Sta10] Iain Stasukevich Camille Utterback Shifts Time in Airport Installation 35 Videography,  9 September 29, 2010 16 http://camilleutterback.com/wordpress/wp-content/uploads/2010/10/Utterback_Videography2010.pdf, Last visited January 29th, 2014.

[SS05] Maria V. Sanchez-Vives Mel Slater From presence to consciousness through virtual reality Nature Reviews Neuroscience,  6 2005 332—339 DOI 10.1038/nrn16511471-0048

[Tur50] Alan Turing Computing machinery and intelligence Minds: A quarterly Review of Psychology and Philosophy,  59 1950 236 433—460.

[Var79] Francisco J. Varela The Principles of Biological Autonomy North Holland New York 1979 0-4444-00321-5

[VTR93] Francisco J. Varela Evan T. Thompson Eleanor Rosch The Embodied Mind: cognitive science and human experience MIT Press Cambridge 1993 0-262-72021-3

[vWRZ10] Herwin van Welbergen Dennis Reidsma Job Zwiers A Demonstration of Continuous Interaction with Elckerlyc MOG 2010: 3rd Workshop on Multimodal Output Generation,  2010 Ielka van der Sluis Kirsten Bergmann Charlotte van Hooijdonk Mariët Theune (Eds.) WP 10-02 02 CTIT Workshop Proceedings Series51—57 Enschede CTIT 0929-0672

[WCP07] D. Weidlich L. Cser T. Polzin D. Cristiano H. Zickner Virtual Reality Approaches for Immersive Design CIRP Annals - Manufacturing Technology,  56 2007 1 139—142 DOI 10.1016/j.cirp.2007.05.0340007-8506

[WN02] John R. Wilson Sarah C. Nichols Measurement in virtual environments: another dimension to the objectivity/subjectivity debate Ergonomics,  45 2002 14 1031—1026 DOI 10.1080/001401302101667800014-0139

[ZJ98] Pavel Zahorik Rick L. Jenison Presence as Being-in-the-World Presence,  7 1998 1 78—89 DOI 10.1162/1054746985655411054-7460



Any party may pass on this Work by electronic means and make it available for download under the terms and conditions of the Digital Peer Publishing License. The text of the license may be accessed and retrieved at http://www.dipp.nrw.de/lizenzen/dppl/dppl/DPPL_v2_en_06-2004.html.

  1. Deutsch
  2. English