Home / Issues / 6.2009 / Considering Stage Direction as Building Informed Virtual Environments
Document Actions

VRIC 2008

Considering Stage Direction as Building Informed Virtual Environments

  1. Alain Bonardi IRCAM, UMR STMS
  2. Francis Rousseaux LASC, Ecole Centrale d’Electronique


This article begins with some recent considerations about real-time music, inspired by the latest contribution of French composer Philippe Manoury. Then, through the case study of the scenic performance La Traversée de la nuit, we analyse some perspectives for designing an Informed Virtual Environment dedicated to live show artistic domain.

  1. published: 2010-03-19


1.  Introduction

Since its creation by Pierre Boulez about thirty years ago, the Institute for Music/Acoustic Research and Coordination (ircam, http://www.ircam.fr/) has been at the centre of what composer Philippe Manoury calls in an article published in September 2007 "the split that electronics provoked in the world of music".

This split immediately led the Parisian institute of research to the heart of the issue of Informed Virtual Environments [ ATLO07, HGH07 ], as many publications by research teams (http://www.ircam.fr/equipes.html) illustrate, whether they are about Instrumental acoustics or about Room acoustics [ Cau06, Cor07, VDWS06 ], about Perception and sound design [ SSM08 ], Analysis/synthesis [ VT06 ], Music representation [ DCA05 ], Analysis of musical practices [ Don06 ], or about Real-time musical interactions [ dB07 ].

The so-called Informed Virtual Environments approach is different from immersion in virtual worlds because it targets to model interactions rather than the world itself (objects, characters, simulation models). The aim is to enable users to live an experience that thus includes experiment, simulation, collaboration and creation in the virtual world.

If the ircam particularity is to associate research and musical creation within the project teams, it is the merit of the composer Philippe Manoury to have written the most synthetic recent paper about the real-time contemporary music state of the art and about the relationship between Informed Virtual Environments and musical composition support tools.

In this article, we shall build on the analysis of this composer to present the intellectual elaboration and the technical performance that make possible the live show named La Traversée de la nuit. We shall show how this work constitutes a strong scientific proposition for Informed Virtual Environments dedicated to musical and artistic contemporary stage creation.

2.  Considerations on the state of real time music


If we have chosen to explore in more details Philippe Manoury's singular contribution, it is because it provides an exemplary overview of all the questions that ircam researchers have been asking about subjects closely related to Virtual Reality.

In his article [ Man07 ], Philippe Manoury relates his fascination and his frustration facing the first attempts to bring together experimental electronic music and instrumental music. That was in the late 1970's and the experiment was limited by the set time imposed by magnetic tape on instrumentalists who were dependent on its inexorable progress. With ircam's first successes in the 1980's, Manoury believed that real time synchronization of electronic music and instrumental playing would give the power of interpretation back to instrumentalists. He explains how as a musician himself he searched for an interactive interpretation of synthesised music to use his own instrumental playing.

At the time, the automatic tools for following scores were still missing. The interactivity between acoustic instruments and machines was only partially completed, "fragment by fragment", during limited and sporadic synchronizations between pre-composed parts of music that were unable to evolve during the performance.

According to Philippe Manoury, the conception of a reliable score follower in fact raises difficult questions: how to detect, recognize and analyze instrumental sounds and gestures in real time. Even though the situation is better for those who improvise without exactly trying to reproduce or notate their interactive works  [2], three main obstacles exist to achieve a score follower.

  1. Notation of electronic music:

    Writing composers write down on a score the sounds in their minds; a kind of "virtual real time" guides their intuition and imagination. To have the same effect with electronic music, not only must their calculations be done in real time, but relevant graphic interfaces must also exist for the expression and dynamic control of the composer to be expressed. Metaphors inspired by instrumental gestures (for instance variation of pressure of mouth or on the bow), or other sources out of the scope of the learned tradition have been developed over the years. But much remains to be done.

  2. Capturing relevant information at the musical source:

    Sensors have been used to detect a flautist's fingering; others have been put under the keys of a vibraphone. Video systems can analyze the gestures of a percussionist, and aerials can measure the position of a bow. These methods combined can analyze the pressure on a string. Systems of captation were at first mechanical and then used audio, video or radar methods, along with sensors, gravitation and acceleration. And we must not forget such ad hoc instruments as midi (Musical Instrument Digital Interface) keyboards.

  3. Polyphonic analysis:

    Superimposed sounds mix their harmonics without the computer being able to discern their respective fundamentals. With the help of the techniques of analysing acoustic signals, these systems must become more "musical" even though this goal still remains, as a whole, out of reach.

Philippe Manoury makes two more important points:

  1. When they compose, composers have to distinguish between instrumental interpretation and what cannot be changed by the musician. The composer might give more importance to improvisation by renouncing complex retroactive strategies. Computerized systems also have their own specific place in interpretation, as long as the composer renounces strict control over interpretation except if he gives more importance to improvisation by renouncing using strategies that are complex and retroactive in their mode of elaboration. This is the case when we ask synthesisers to receive information from the outside world and to process this information under partially defined rules. Interactive interpretation is thus the complex product of two indeterminate forces.

  2. In conclusion, Philippe Manoury underlines the importance of the visual process used to predict the movement of musical time. This process still remains widely inaccessible to machines since it must combine two sensitive registers, putting them on an equal footing with human beings. In contrast to a machine that splits time into very small clock cycles, musicians keep time through bodily movements: nodding the head, tightening the diaphragm, tapping the foot. An orchestra conductor does all these and moves to communicate with the musician-interpreters. That's why it is so difficult for musicians and machines to share music in real time.

3.  Placement for an exploratory research

Since they met at ircam in 1997, the authors of this article have regularly collaborated in the field of Informed Virtual Environments [ RB07, BBR05, RB03, BR02 ]. Their goal: to layout the foundations of cooperative musical interaction between performers in a broad meaning (musicians, actors, dancers, etc.) and 'intelligent' artificial systems. Their purpose is to bind theoretical knowledge and practical stage approach in the exploration of Informed Virtual Environments for performance arts. They would like to explore all consequences of the paradigms of realtime computation in the field of performing arts, as initialized by Philippe Manoury. Contrary to a priori modelling of characters and situations implied by narrative aspects of performance, they are interested in progressive emergence of knowledge based on human-to-machine interaction to shape forms, structures or similarities.

Their collaboration is based on the common conception of stage 'scenographies' using computers (Virtualis, Norma, Alma Sola,...) addressing research issues; the final realization being achieved by Alain Bonardi (http://www.alainbonardi.net/). These productions represent a real size experiment bringing closer to reality future, more responsive stage devices. They extend previous stage direction approaches trying no longer to put the emphasis on the reification of characters and situations, for instance Reaney's or Wild's ones. Stage director Mark Reaney [ Rea99 ] has explored for years the intervention of augmented reality in theatre shows. In some of his shows, members of the audience wear 3D vizualisation glasses where elements of virtual scenographies are projected. In his staging of Orgia by Pasolini in 2001 (Belfort, France), Jean-Lambert Wild has elaborated a set from the representation of a kind of 'life game' computed from captors giving physiological information about the actors: blood pressure, temperature, etc. More generally, they try to handle computerized device and artefacts on stage to arouse common artistic knowledge in situation of interaction?

The most emblematic collaboration was La Traversée de la Nuit, created in Enghien-les-Bains in November 2003  [3] and performed several times since then. La Traversée de la Nuit involved a multimodal communication stage device that created interaction between the audience, a character on stage and a complex artificial system. The device was centred on emotional communication. The austerity of the scenography contrasted with the intensity of the dramatic situation: after fifty years of silence, a 1944 deportee from Ravensbruck, Geneviève de Gaulle Anthonioz, tells her story.

The idea was to play on duelling temporalities of sound and visual effects, working on different layers of memory to saturate the space between communication and liberty and forcing the actors and spectators to carve out, every night, a space where hope might live.

The stage directions limit the play's artistic interpretation. For example, to show the schizophrenia of the central character, she is played by two actors, one is an actress who speaks the lines, the other a dancer who moves with her and sometimes for her. And although the spoken lines themselves are to be rendered as neutrally as possible, a metaphorical representation of the narrator's mind is created in real time on a large screen at the back of the stage, in the actress' voice. The dancer and the actress both respond to the image, especially the dancer who, being freer, can vary her movements more.

Figure 1. General scheme of interaction (performers, audience, computers) in La Traversée de la Nuit.

General scheme of interaction (performers, audience, computers) in La Traversée de la Nuit.

Those dramatic choices echo Philippe Manoury's axiom about the composer's visual role in the conducting of a score. The device in La Traversée de la Nuit, for instance, involves a neural network that lets the voice of the actress be processed in real time while "recognizing" emotional states. Those emotions in turn, thanks to a multi-agent system, command the movements of mobiles arranged and projected on the wide screen at the back of the stage. Thus, in way mysterious both to the audience and the actors, the voice's emotional variation produces a visual avatar.

The interaction between the audience and the doubly portrayed character creates a complex perspective. Each player can see his or her emotions and react to them in space and time, even to the point of the actors who play with the projected mobiles. Manoury's processes of premonition, memorization, combining shapes, forming strategies, transitions, proportions and short-circuits can be achieved following narrative time, made visually concrete, then deconstructed through multiple interactions.

One thinks of Husserl in his Leçon pour une phénoménologie de la conscience intime du temps, but also of Whitehead trying to describe his naturalist cosmos [ Hus64, Whi04 ]. The entire theory of reduced listening is brought into question (Pierre Henri, Pierre Schaeffer, Michel Chion), since it must be hybridized with modern theories of visual perception. Thus we are asked to read Maurice Merleau-Ponty, but also Marie-José Mondzain and Laurence Hansen-Løve [ Mon96, HL07 ], within the new perspective of an originary multimodal theory of perception.

4.  The Flagship Project: La Traversée de la Nuit

4.1.  Multi-modal interactions

The stage direction in La Traversée de la Nuit is based on an autarkic human-to-machine system: an actress, Valérie Le Louédec, tells the whole text. A dancer, Magali Bruneau, performs a certain number of gestures inspired by Nô theatre and there is a multimedia computer, as an artificial actor. However the two actresses on stage represent the two sides of the same character - conscious and unconscious, according to the traditional shite and waki in Nô theatre. The computer projects images onto a very large screen at the back of the stage (the actress and the dancer can see part of it at any time), provoking the reaction of the comedians who may adapt their play. At the beginning of the loop, the computer grasps the emotional states of the actress's voice (figures 1 and 2).

Figure 2. Example of image generation on the backstage screen in La Traversée de la Nuit (actress Valérie Le Louédec on the left, dancer Magali Bruneau on the right; photography: Julien Piedpremier).

Example of image generation on the backstage screen in La Traversée de la Nuit (actress Valérie Le Louédec on the left, dancer Magali Bruneau on the right; photography: Julien Piedpremier).

4.2.  Technical description of the human-machine system

The technical implementation (see fig. 3) of the human-machine system is based on a neural network to analyse the actress's voice as an input and a multi-agent system to generate projected images as an output. The whole system was coded using the real-time Max/MSP/Jitter platform.

Let us precise at once that the neural network used for voice analysis has nothing to do with techniques in the field of speech recognition. The only purpose is to classify and recognize the various status of the voice, not what is said. The neural network was trained in supervised mode during several months. The actress would impose herself a list of emotional states and read the whole text using one of them. The input voice is computed one sentence after another. A twelve-component vector is computed from each of them: four components represent vowel pronunciation (formants), four of them have to do with rhythm (durations), and four of them have to do with its amplitudes. For each vector presented as an input, the neural network provides an emotional state "recognized" at each drop of the sound level (for instance every end of sentence).

Figure 3. Functional scheme of the software implied, related to a neural network and a multi-agent system.

Functional scheme of the software implied, related to a neural network and a multi-agent system.

The multi-agent system [ VMdO03, PP02 ] enables the real-time generation of images projected on a screen at the back of the stage. The agents may be compared to dynamic "billstickers" that would build together images always new (each of them carries a small part of image, still or animated):

  • Each agent has a simple psychological model of sensitivity (positive or negative), that reacts to the emotional states provided by the neural network, according to the text sequences (that affect sensitivity weights). The result is a "mood" that conditions the agent willingness to achieve its goals.

  • The agents cooperate towards one goal by optimizing a utility function computed from the appearance qualities of the images generated. There is one different utility function per text sequence. Agents may move, twist, enlarge, reduce, and make more or less transparent their own small images.

  • Agents are coordinated in the execution of these common goals in relationship with the emotional state acknowledged by the neural network by a "mood compensation" mechanism: the ones who are in a very good "mood" (high positive value) grant part of their willingness to the ones who are in a very bad "mood" (high negative value).

  • Agents communicate together peer-to-peer by exchanging their "moods" at fixed periods.

The environment of agents includes the emotional states acknowledged by the neural network, the event number indicating the current position inside the text, the loaded values for each text sequence, and the indications of an observer agent indicating the qualities of the global image generated.

We show below on two following video screenshots (figures 4 and 5) how the 'climate' of the voice can influence the movement of the agents. The same text ('[...] avant que retentisse la deuxième sirène de l'appel. Nous sommes le 29 octobre, et il ne fait pas encore très froid'  [4]) is told with two opposite expressions; we have voluntarily isolated two agents out of the total population and shown step by step their evolution. For the first video, the voice sounds quite 'soft' and 'intimate': the two agents remain quite close. In the second video, the video sounds 'loud' and 'cold': the two agents move away from each other.

Figure 4. Video snapshot of a first simulation with two visual agents driven by a 'soft' and 'intimate' voice.

Video snapshot of a first simulation with two visual agents driven by a 'soft' and 'intimate' voice.

Figure 5. Video snapshot of a second simulation with two visual agents driven by a 'loud' and 'cold' voice.

Video snapshot of a second simulation with two visual agents driven by a 'loud' and 'cold' voice.

5.  Turning Stage Direction Practices into Virtual Environment Information

Facing a theatre text, every stage director would like to propose his/her interpretation. Let us remind that the interpretation is not immanent to the text, in spite of the numerous indications (playwright foreword, stage indications, etc.). A text cannot exist on stage without an exegesis completed by the stage director.

This interpretation is always an effort to create forms. Let us try to understand this approach in a computer science viewpoint. The stage director process starts by setting a synthetic ontology of the drama: it describes the characters as types (19th century light comedies work with a small number of types, mainly three: husband, wife, and lover) and instanciations, by telling the name of the character, his/her situation at the beginning of the play, his/her costume, etc. During the play, there are instanciation variations  [5]: the spectator discovers that such or such character is far different from what he/she imagined at the beginning. These instanciation variations may lead to ontology revisions. It is for instance the challenge, as well dramatic as metaphysical, in the play El Burlador de Sevilla by the Spanish playwright Tirso de Molina (1630), inaugurating the Don Juan myth: may this character be saved by acknowledging his sins in extremis before his death? May the character's ontology be modified at the very end of the play? Let us notice that this approach may be applied to other forms of dramaturgical, no more based on characters but on situations (in this case, ontologies of situations would be considered).

In this traditional approach of theatre, the concept of similarity by the ontologies is essential. The director stages every scene considering it as an example in a set of cases provided by theatre literature. Explaining a character to an actor consists in pointing in the proposed ontology and link this ontology other ontologies, either of the same play or other ones. Sometimes two stage directions by two different directors can be considered as instances of the same ontology.

In a more general point of view, this is a formal approach, where the example is represented as an instance of a general structure grasping all cases, and one looks for similarities by varying the instanciation. The obvious advantage is to provide an explication of the "similar to the example" side of the proposition, and even a distance measure — this is the way to create a recapitulate concept in intension. Ontologies enable to look for the similarities of an example by staying inside a concept, even if it means to move to the immediate more general concept when the search is unsuccessful. This kind of approach has to do with pre-organization necessities evoked by Manoury. In the field of digital arts, artistic installations as well as virtual worlds usually implement highly structured and completely predefined ontologies that model the environments, the objects, the users, etc.

In a computer science viewpoint, there is another way to deal with the question of similarity [ Per94 ]: it is the interactive data-mining approach, where the example is represented as a specialization of a set of cases. One looks for other neighbouring specializations, but without having at disposal a pre-defined ontology. The user accepts to shape an ad hoc ontology at his/her hand with the interactive help of computers. This is an approach by extension: shaping a similarity consists in shaping a list of contents of similar form by successive amending actions. It uses computing in an interpretative and interactive way, with amending actions on contents and their forms (on the user's side, who is provoked by the proposals of the computer).

6.  Tools for Collaborative Informed Virtual Environments

In the case of the staging of La Traversée de la Nuit, we have built a dramaturgical Informed Virtual Environment system. It first means we have designed a virtual world with its multi-modal interfaces and its feedback loop: voice (actress) to image (population of kinds of sprites on screen), image to movement (dancer), image to voice (feedback to the actress), movement to voice (dancer to actress). We come to an Informed Virtual Environment, since dramaturgical knowledge is handled by the actor/machine relationship in our system. This knowledge is not a set of predefined ontologies that would "translate" the text into static structures. It is centred on a set of interactions that let the performers play with the system as an instrument but providing at the same time some surprise. If we come back to necessities expressed by Manoury:

  • We deal with writing and interpreting strategies developed by the author and the stage director: memory, premonition, etc. are overexposed by the association of voice, movement and image.

  • We keep separate the two levels of predetermination (text, music, banks of images pre-selected, indications of staging) and what belongs to interpretation (voice, part of movements, images).

  • We do not have such precise time requirements as musicians, but we have organized time coordination thanks to voice inputs and visualisation that provide important synchronisation or trigger information, both for performers and to the audience (telling there is an important change).

Last but not least, we are currently working on a theatre creation named Les petites absences  [6] that is also a research in collaboration with the Real-time Musical Interactions Team at ircam, managed by Frédéric Bévilacqua, and the actors of the Théâtre du Signe in Caen. During this project, we put the emphasis on testing ideas about a future possible dramaturgical working station for performers and stage directors. We name it 'Virtual Assistant for Performers and Stage Directors'. Figure 6 shows a screenshot of the main patch: different layouts of processes are implemented as successive waves starting at the video and sound captation of a performer. After the acquisition, the first step is the analysis layout, the second one the classification level, and the last one the collection level (this level is developed in [ RB07 ]).

Figure 6. Screenshot of the main window of the virtual assistant.

Screenshot of the main window of the virtual assistant.

Figure 7 shows a screenshot of the gesture and voice analysis module.

Figure 7. Screenshot of the gesture and voice analysis window.

Screenshot of the gesture and voice analysis window.

We start from audio and video capture of performers (dressed in black in contrast with a white backstage wall). Their silhouettes are surrounded with bounding boxes. Descriptors about gesture (quantity of movement, surface of contraction, etc.) and about voice (pitch, loudness, brightness, etc.) are computed for each frame. Then aggregators can be computed for each descriptor: for instance, average value, variance, minimum and maximum values. This gives a 70-component vector that characterizes each sequence.

We therefore constitute a stack of vectors. The most important aspect for us is to arouse the emergence of dramaturgical knowledge thanks to man to machine dialogue. We therefore let the user (for instance a stage director) organize collections of sequences that seem quite close to him/her. From this intuitive personal collection, we infer thanks to a Principal Component Analysis the most significant aggregators (among the 70 present) for the collection. This enables to avoid pre-defined modelling of performing situations, and to shape a kind of emerging interaction.

Second step, we are developing a real-time (for Max/MSP) fuzzy logic  [7] engine able to handle uncertain reasoning [ BTA06 ]. This gives us the opportunity of thinking of semantic mapping between the inputs and the outputs of the scenography. For instance, how can we link the presence of the actor on stage (corresponding to the surface of contraction he/she occupies on the screen capture) to the level of red lightning? First idea would be to write in Max/MSP (that can handle all device for stage command) a predefined mathematical function giving the level of red according to the presence of the actor. On the contrary, our idea is to let some ambiguity in the mapping thanks to fuzzy logic, and to be able to build a feedback loop using fuzzy modifiers. For instance, let us consider a fuzzy rule like 'ifthe actor is very present, the level of red should be high'. From the first empirical observations, we can adjust the rule using fuzzy modifiers, to remain on a semantic level. This semantic level is an important thing when building up an Informed Virtual Environment and considering interaction not only as a numerical mapping.

In Les petites absences fuzzy rules were mainly used to control a generator of crowd sounds - that would fit the behavior of the main actor - or to trigger some events. Stage director M. Bataille-Testu wrote by himself the fuzzy rules.

The outcome is very positive since the play has been performed 25 times since December 2008 giving satisfying results. However, there are several interactive processes that cannot be specified using fuzzy rules: for instance, it is very difficult to program (with fuzzy rules) contrapuntic imitation of an instrumentist.

We are interested in driving scenographies thanks to fuzzy rules. As they enable to work out relationships between input linguistic variables (on the performer side) and output ones (on the electronic side), they constitute interesting ways of stating elements of interactive scenography. As the users may enter their own vocabulary, they provide original ways for artists to specify interactions for stage shows.

During our experiments one difficulty was the understanding of fuzzy rules. First of all, it is not obvious for a non-specialist that the result of a fuzzy implication will give a fuzzy subset and not a crisp value. But with the help of words, this point becomes less problematic. That is why the linguistic counterpart is so important, especially here. Second point is the implicit reciprocity in the rules stating that the reverse is true: e.g. 'if the movement is nearly still, then the frequency is high' may implicitly mean that 'if the movement is not nearly still, then the frequency is not high'.

Figure 8. Screenshot of a sketch of the real-time Max/MSP fuzzy logic engine to link an input (up: quantity of movement, elongation) and an output (down: performer status) of scenographic and dramaturgical 'systems'.

Screenshot of a sketch of the real-time Max/MSP fuzzy logic engine to link an input (up: quantity of movement, elongation) and an output (down: performer status) of scenographic and dramaturgical 'systems'.

7.  Conclusion

This research provides sketches of tools to implement various aspects of Informed Virtual Environment systems dealing with stage scenography and dramaturgy. Starting from the concept of similarity in extension (opposed to similarity in intension), we have looked for combinations of human practices and computerized artifacts that no longer try to model staging a priori. Our recent research and its application to such theatre shows as La Traversée de la Nuit and Les petites absences confirms the necessity according to us to continue working in the directions set by composer Philippe Manoury in the field of interactive music. Coming to semantic and macroscopic approaches of the actors's status can hopefully enable further exploration of high level human-to-machine interactions on stage.


[ATLO07] S. Aubry I. Thouvenin D. Lenne, and S. Okawa Knowledge Integration for Annotating in Virtual Environments International Journal of Product Development,  4 (2007)no. 6533—546issn 1477-9056.

[BBR05] A. Bonardi T. Bouaziz, and F. Rousseaux A Style of Theatre Production Inspired by Interactive Content Data Mining
Human System Learning Who is in Control?,  Editions Europia, Paris 2005isbn 2-909285-33-2.

[BR02] A. Bonardi and F. Rousseaux Composing an Interactive Virtual Opera: The Virtualis Project Leonardo,  MIT Press,  35 (2002)no. 3315—318issn 0024-094X.

[BTA06] A. Bonardi I. Truck, and H. Akdag Building Fuzzy Rules in an Emotion Detector 11th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU 2006), Paris  Vol. 12006pp. 540—547.

[Cau06] R. R. Caussé Instruments de musique virtuels et modélisations physiques [virtual musical instruments and physical modelling],  Découverte,  no. 3402006, In French.

[Cor07] E. Corteel Synthesis of Directional Sources using Wave Field Synthesis, Possibilities and Limitations EURASIP Journal on Advances in Signal Processing, special issue on Spatial Sound and Virtual Acoustics,  (2007)no. 11issn 1687-6172.

[dB07] S. deLahunta and F. Bevilacqua Sharing Descriptions of Movement International Journal of Performance and Digital Media,  1 (2007)no. 33—16issn 1479-4713.

[DCA05] M. Desainte-Catherine and A. Allombert Interactive scores: A Model for Specifying Temporal Relations between Interactive and Static Events Journal of New Music Research,  34 (2005)no. 4361—374issn 0929-8215.

[Don06] N. Donin Vers l'annotation multimédia d'informations musicales [Towards multimedia annotation of musical information] L'inouï, revue de l'Ircam,  no. 22006, In French.

[HGH07] J. Herder K. Garbe, and I. Herbst Spatial Audio for Augmented Reality 10th International Conference on Human and Computer,  Duesseldorf/Aizu-Wakamatsu/Tokyopp. 53—582007.

[HL07] L. Hansen-Love Les images peuvent-elles mentir? [Can images lie?] PhiloSophie, Grenoble,  2007, In French.

[Hus64] E. Husserl Leçons pour une phénoménologie de la conscience intime du temps [Lectures on the phenomenology of inner time-consciousness] PUF Epiméthée 1964, In French.

[Man07] Philippe Manoury Considérations (toujours actuelles) sur l'état de la musique en temps réel [Considerations (always present) on the state of music in real time] revue l'étincelle, IRCAM,  no. 32007, In French.

[Mon96] M.-J. Mondzain Image, icône, économie: Les Sources byzantines de l'imaginaires contemporain [Image, Icon, Economy: the byzantine roots of contemporary imagery]Paris Seuil, Paris,  L'Ordre philosophique,  1996 isbn 978-2020208673, In French.

[Per94] J.-F. Perrot Des objets aux connaissances [Objects of awareness] Journée: Méthodes Objets et Intelligence Artificielle Frontières, Ponts et Synergies,  1994Paris RIA, In French.

[PP02] C. Pelachaud and I. Poggi Multimodal Embodied Agents The Knowledge Engineering review,  17 (2002)no. 2181—196issn 0269-8889.

[RB03] F. Rousseaux and A. Bonardi Music-ripping : des pratiques qui provoquent la musicologie [Music-Ripping: practices that provoke musicology] Musicae Scientiae,  8 (2003)111—124issn 1029-8649, In French.

[RB07] F. Rousseaux and A. Bonardi Is knowledge emerging in the Secrecy of our Musical Collections? 4th Sound and Music Computing Conference,  Lekfada, Greecepp. 238—2452007.

[Rea99] M. Reaney Virtual Reality and the Theatre: Immersion in Virtual Worlds Digital Creativity,  10 (1999)no. 3183—188, issn 1462-6268.

[SSM08] C. Suied P. Susini, and S. McAdams Evaluating warning sound urgency with reaction times Journal of Experimental Psychology,  14 (2008)no. 3201—212issn 1076-898X.

[VDWS06] I. Viaud-Delmon O. Warusfel A. Seguelas E. Rio, and R. Jouvent High Sensitivity to Multisensory Conflicts in Agoraphobia Exhibited by Virtual Reality European Psychiatry,  21 (2006)no. 7issn 0924-9338.

[VMdO03] M. Becker Villamil S. Raupp Musse, and L. Paulo Luna de Oliveira A Model for Generating and Animating Groups of Virtual Agents Intelligent virtual agents: 4th International Workshop, IVA 2003, Kloster Irsee, Germany, September 15-17, 2003,  Lecture notes in artificial intelligenceVol. 2792,  pp. 164—169,  2003isbn 3-540-20003-7.

[VT06] C. Vergez and P. Tisserand The BRASS Project - From Physical Models to Virtual Musical Instruments: Playability IssuesLecture Notes in Computer ScienceVol. 3902,  pp. 24—33,  2006isbn 3-540-34027-0.

[Whi04] A.-N. Whitehead Modes de pensée [Modes of thinking]VRIN, Analyse et philosophie2004isbn 978-2-7116-1667-1,  In French.

[1] This excerpt is almost word for word what Philippe Manoury developed in his eponymous article.

[2] According to Philippe Manoury, composing music consists in elaborating a complex musical discourse. The journey from invention to presentation involves memory, premonition, combining shapes, forming strategies, transitions, proportions and short-circuits. None of these can be improvised (cf. his own pieces Pluton, Neptune, En écho, Partita I) And since nothing can be predetermined in this relationship between instrument and computer, the interaction may oscillate between complexity and a disarming simplicity. Spontaneous creation produces formal and standardized archetypes.

[3] This theatre play was performed on november, 2003, in the Centre des Arts, in Enghien-les-Bains (France). Stage direction: Christine Zeppenfeld; actress: Valrie Le Loudec ; danser : Magali Bruneau; multimedia design: Alain Bonardi and Nathalie Dazin; music: Stéphane Grémaud; lightning: Thierry Fratissier.

[4] A translation of this text could be "[] before the second call alarm rings out. It is October, 29th, and still not too cold"

[5] Instanciation is a word often used by computer scientists, that comes from instance, and means example, case; an instanciation generalizes in a way the operation of affectation of a numeric value to a variable: to describe the real world, computer scientists instanciate abstract classes, stating that such or such entity is a particular case of a class, which is itself linked to other classes by generalization hierarchies and/or formal properties. This builds a set sometimes named ontology. Ontologies are claimed to describe parts of knowledge about the world and are very used in artificial intelligence. This can also be considered as an object design (an object design is based on heritage graphs in order to generate programs by simple instanciation of key-parameters).

[6] This play was created in December 2008 at the Comédie de Caen.

[7] Project lead with the collaboration of researchers of the Laboratory of Advanced Computing of Paris 8 University: Isis Truck and Nicolas Jouandeau.



Any party may pass on this Work by electronic means and make it available for download under the terms and conditions of the Digital Peer Publishing License. The text of the license may be accessed and retrieved at http://www.dipp.nrw.de/lizenzen/dppl/dppl/DPPL_v2_en_06-2004.html.