The History of the Interface in Interactive Art

The History of the Interface in Interactive Art
Söke Dinkla, 1994
At the moment the catch word "interactivity" is common talk. Most often it is mentioned in connection with a revolution in television. Techno-prophets anticipate more than 200 TV channels for the near future in each home. Thus, viewers will not only be able to choose from an almost unlimited offer, they will also be able to determine the course and outcome of individual programs[1]. Proponents of these new opportunities are already praising interactivity as a means to change the passive reception of the viewer into an active one[2]. Thus, it seems as if Bertolt Brecht's Radio Theory [3], which he developed in the late twenties, is now to become reality. Brecht envisioned the transformation of broadcasting from a distribution machine into a communication device that offers listeners the opportunity to help create its content. And actually this development has been actively persued for years by groups such as the Ponton Media Art Lab, by persons such as Myron Krueger, and by the communication structure of the internet.

This slightly anarchistic approach was notably absent from this year's Siggraph computer trade show in Orlando, Florida. The trade show showed that besides interactive TV games the US-American entertainment industry is concentrating on the employment of interactive technologies in the scope of big theme parks. While the well-known Virtuality games by W-lndustries individualize the player, the theme parks stress cooperation and team spirit. The company Evans & Sutherland, for example, presented at Siggraph the game Virtual Adventures, in which six players search together for the eggs of the Loch Ness monster. The game appeals to typically athletic characteristics such as ambition and team spirit. It offers alternative experiences of pleasure and frustration, that is the classic features of a game.

Computer games like this have a more than 20 year old history of technologicaldevelopment, which remarkably took place at the same time in military research and in art. In these years Interactive Art supplied many alternatives to theabove mentioned Loch Ness game and is essentially characterized by the attempt to "humanize" the interface between system and player. On top of this, the use of technologies that address the viewer directly and involve her or him in adialogue, constitutes a decisive change in the tradition of the image. Therefore my main attention focuses in the following on the reciprocal dialogue between user and system and on the design of the interface. In the following text I will distinguish six important implications of interactivity.

0. Historical Background and some Conflicts of Interactive Art

Its background in art consists of participational art forms from the late sixties like for example Happenings and reactive kinetic environments. Theoretical works like Umberto Ecos Opera aperta (1962) [4] contributed to are interpretation of the part played by the spectator. In German aesthetics this view was developed further especially by Wolfgang Kemp in the middle ofthe eighties. His book Der Betrachter ist im Bild (The Viewer is Inside the Picture) [5], in which he describes the method of receptional aesthetics (Rezeption. . . sthetik), seems to anticipate the perception principle we areexperiencing today in virtual reality. But this line of tradition is not unbroken, as will be shown later on in this paper.

In a way Interactive Art builds on the traditions of participational art formsby allowing the viewer to intervene in the action. However, in most works, unlike in Happenings, this interaction is not meant as an attack against the established art audience. Instead, it meets the needs of a media educated public. The implications of Interactive Art, though, go even further: this art also reflects the role played by computer technology [6]. This may seem complicated, because Interactive uses the same technology it comments upon, meaning, there is a certain lack of distance. The situation of Interactive Art is therefore comparable with Video Art, which had to gain certain independence from the language of television. Both art forms demonstrate that today the role of the artist is changing significantly. Instead of being a commentator standing outside society, the artist nowdecides to take part in the socio-technological change and judge from within.
1. Power and Play

With the American Myron Krueger the development of computer-controlled Interactive Art started. He began as early as 1969 to conceive spaces in which actions of visitors set off effects. In co-operation with Dan Sandin, Jerry Erdman and Richard Veneszky he conceived the work Glowflow in 1969. Glowflow is a space with pressure sensitive sensors on its floor, loudspeakers in thefour corners of the room and tubes with coloured suspensions on the walls. Th evisitor who steps on one of the sensors sets off either sound or light effects. In the scope of the Art & Technology movement in the late sixties artists like Robert Rauschenberg and James Seawright created similar'responsive environments'. But at that time no one in the 'art world' thoughtof creating a more complex computer-controlled dialogue and focussing the interaction itself.

In the computer sciences this situation was different. Almost simultaneously with Glowflow Ivan Sutherland developed at the Universiy of Utah the precursor of today's head-mounted-display (HMD). This display was worn like apair of glasses and contained two small monitors, each of which showed a stereoscopical sight to the eyes. Sensors registrate the head movements and transmit the information to a computer which then calculates the perspective and gives the viewer the impression to move within the image.

Thus, at the end of the sixties two trends emerged independently of each other, which have significantly influenced the present situation of Interactive Art and computer technology in general:
the development of 'responsive environments' in the scope of the US-American Art & Technology movement and
the development of the head-mounted-display

Krueger's work cannot be assigned to either of these trends. Neither did he participate in the projects of the Art & Technology movement, nor did he regard the head-mounted-display as a suitable interface. He thus used a different variant, which was also developed at the end of the sixties, but in the scope of Video Art: it was the closed-circuit installation in which visitors are confronted by their own camera image. Krueger now combined this principle with computer technology.

In Videoplace, a work Krueger has been constantly developing since 1974, the visitors find themselves faced with their own projected video image that canbe changed by the computer program. In Videoplace there are a number of different interactions, in which Krueger subverts the rules of narcissistic self-reflection and self-control of the traditional video closed-circuit and lets the user play with constantly changing versions of themselves. In the most famous interaction called Critter a green figure appears on the screenand tries to make contact with the visitor. It steers towards an exposed partof the visitor's body and lands there. Then Critter begins to climb up the arm, shoulder and neck until it reaches the highest point of the head. Once there, it performs a joyful dance. Since Critter is programmed to reach the highest point of the visitor's outline, the aim of the players is to outwitCritter, that is, to subvert the program and develop their own rules. Thus, the interactions of Videoplace are not only a joyful game but are also concerned with the probing of power distribution between user and system.

Krueger's attitude towards the interface shows that he is opposed to the isolation of the user caused by the head-mounted-display. Instead, he creates an open space where it is the interaction and not the instrument that causes the proximity to the system. This has important consequences for the understanding of the interface. The technical interface - in this case the video camera - is, in a way, invisible and loses significance. It is substituted by the application itself.

In Europe the approach to Interactive Art and also the use of the interface was quite different. The situation at the beginning of the 80's could be characterized by the catch words "Participation versus Interaction".
2. Participation versus Interaction

In Amsterdam in 1983 the Australian Jeffrey Shaw produced his first interactive installation. He transferred his participational concept of art, which he developed during the sixties to computer installations. In his first interactive installation Points of View Shaw takes up the joystick, the interface that is still customary for video games. Sitting on a chair the spectator can move the projected video image of a stage with Egyptian Hieroglyphs. With a second joystick she or he can steer sound traces. InPoints of View the spectator turns into the director who individually selects the picture and sound material. The intended reception of Points of View is described by Shaw as following: "It is the particular audio visual journeymade by a spectator who operates the joystick which constitutes a'performance' of this work. For the other spectators that performance becomes'theater'. "[7]

Although in Points of View Shaw dispenses with the physical performance ofthe spectator, he still keeps his familiar terminology. The term movement does not any longer signify the movement of the performer in space, like in the former Happenings, but the movement of the image caused by the joystick. The projected scene can be changed in its perspective with only very small physical expenditure. Thus, the computer-controlled system inverts the reception situation of the earlier Happenings. Formerly the spectator had to change her or his position to perceive differently; now she or he induces the computer image to change its perspectives. Thus, the movement of the spectator is substituted by the movement of the image.

By means of the development in Shaw's oeuvre the above mentioned break in the tradition reaching from participational art forms to Interactive Art becomes clear. New points of view are not formed by physical experience but with the help of new interactive media strategies. As I presumed at the beginning, artists like Shaw adress in their Interactive Art a media-educated audience, but nevertheless formulate an opposite position to the passive reception of technically produced moving images. At the same time Shaw is also criticizing certain potentials of interactive technology itself. He decides against the video camera as the interface with the system, probably because he considers it too unvisible. Instead, he uses a bicycle in his most famous work called The Legible City, begun in 1988. With the familiar pedaling and steering movements the cyclist can move through a projected city of letters. The choice of this specific interface on the one hand aims at providing the visitors withfamiliar patterns of behaviour, on the other hand the bicycle as interface constitutes a refusal to do without physical activity altogether.
3. Proximity and Manipulation

At the same time as Points of View - in 1983 - the Canadian David Rokeby began to develop his interactive sound installation Very Nervous System, whichin the beginning he exhibited with changing titles and changing technical equipment [8]. After Rokeby had experimented for a short time with light sensors as interface and with analogous electronics, he decided - without knowing the earlier works of Krueger - to use the video camera as interface. Rokeby's Very Nervous System has a much more suggestive effect than the worksof Krueger and Shaw because he works with nonvisual system effects as well aswith an invisible interface. If one reacts intuitively to the sound, aclosed-circuit is created, in which music and movement are slowly becoming aunity.

There is, however, a basic restriction: Krueger's Videoplace requires a contrasted background to absorb distinguishably the persons in space; Rokebyon the other hand is working only with a strong spotlight to achieve the same effect. Therefore, the causal relations between an actual movement and the sound are ambiguous. Although Rokeby employs the same interface as Krueger, their positions differ from each other. Krueger's dissatisfaction with the'responsive environment' Glowflow was caused primarily by the fact that the visitors interpreted chance events as the response to their actions. While Krueger attempts a precise attribution of cause and effect to reveal the reactions of the system, Rokeby is playing with the irritation of the visitor. He hugely reduces the distance between visitor and system.

This is shown by his newest installation so far, titled Silicon remembers Carbon from 1993. In this installation the visitor is even allowed to enter the image that is projected on the floor and change it with her or his movements. Infrared sensors and cameras are used as interface With this concept of reducing the distance Rokeby attempts a tightrope walk: on the one hand the visitor assumes that she or he can control the image or the sound, on the other hand the visitor is manipulated by these effects. This suggestive power of interactive correlation is only disturbed by the fact that Rokeby, aswell as Shaw and Krueger, creates environments which allow the presence ofmore than one visitor.

The works of Shaw, Rokeby and Krueger are conceived as environments. This is not the case with the works which were created in the United States at thesame time or a bit later. Most of them are conceived as installations, thatis, the surrounding space is involved less strongly and the user often hasdirect access to the input instruments. The most common input instruments are the touchscreen and the mouse. As the works of Krueger, Shaw and Rokeby have shown, the description of the interface is not restricted to its technology. The same holds true for videodisc installations.
4. Strategies of Seduction

Nearly around the same time as similar works by the group around Glorianna Davenport at the Media Lab at MIT [9] Lynn Hershman from San Francisco developed her first interactive installation Lorna , finished in 1984. Lorna as well as Hershman's second installation Deep Contact (1990) both work withverbal requests like "Help Lorna Leave Her Home!". The picture sequences and the texts depict women in the world of media as passive objects of maledesire. In Deep Contact changes in a projected video image are triggered by touching a screen. The touching of body parts of the character Marion on the touch-screen sets off different strands of narration and, according to Hershman 'entangles the viewers into meeting their own voyeurism'[10. ]

Her newest work so far, A Room of One's Own (1992), also attributes this part to the spectator: the visitor looks through a little periscope into a small bedroom on whose back wall sequences of images are projected. The interactionin Hershman's work is being sexualized by the tactility of the touch screen (in Deep Contact ) as well as by the intimacy of the observed situation (in A Room of One's Own ). At the same time a fatal situation ensues. As soon as the spectator acts he or she is caught in his or her role as voyeur. Hershman does not use interactivity to free the user from passivity, but to expose him orher as a voyeur. Put differently, the desires of the audience are the cause for the repressive depiction of women in media. Not even interactive technology can change that fact.
5. Nonlinear Narration

The New Yorker Grahame Weinbren produced his first interactive installation The Erlking in 1986. In this installation the interaction is mainly initiated and born by mysterious, almost static images. Weinbren - in co-operation with Roberta Friedman - works with distinctly cinematographic sequences.

The first picture shows the soprano Elisabeth Arnold who sings Schubert's song Der Erlkönig. This picture functions as a leitmotif and guideline assistance to which the user can return again and again. The other pictures are partly based an Goethe's ballad, in which an old man narrates the saga of the Erlking. Originating from the basic sequence the structure of the narration branches out. It goes not only into detail but also into additional aspects which are only loosely associated with the main plot or the backup picture. In addition to this storyline Weinbren uses Freud's 1918 case study "From the History of an Infantile Neurosis" to try out a nonlinear access to the sequence of images.

Narration and song in The Erlking are being quoted as historical examples of oral tradition and are confronted with the nonlinear interactive form of narration. As a result the interactive system takes over the role of pictorial memory. The user occupies the role of the director and cutter respectively, similar to Shaw's Points of View. Weinbren hopes that interactive technologies are a more appropriate means to tell these old stories [11]. This hope is problematic, since with originally linear storylines the fragmentation of content doesn't necessarily lead to a better understanding. Only if the stories were very well known today - that means if they had a kind of socialsignificance - the interactive access could possibly add new points of view.
6. Remembering, Forgetting, Reconstructing - The 'Surrogate Travel'

The New Yorker Ken Feingold is the first who uses a touchscreen as interface without integrating a second monitor. In his first interactive installation The Surprising Spiral from 1991 the surface susceptible to touch is set in the cover of a book. Fingerprints and two open hands inside the book indicatethat this object may be touched. Thus, the book functions as interface to the pictorial action of The Surprising Spiral. A second contact point, depicting a mouth with a light source, makes sound manipulations possible.

On the videodisc of The Surprising Spiral pictures and sound are stored that Feingold recorded in India, Japan, Argentina, Thailand, Scotland and theUnited States over a period of 12 years. The documentary pictures are contrasted with fast-moving Japanese TV advertisings and coloured computer animations. Feingold makes a collage out of disparate film material from different contexts, such as ethnographical, cultural, historical, religious, aesthetic and medial contexts. Thus he combines nearly all the approaches thatuntil now made the reconstruction of historical facts possible. It becomes clear that despite the partly documentary film material and the mostly photorealistic video pictures the aspect of documentary truth in the TheSurprising Spiral is of no importance [12].

Because of the missing mise-en-sc&eagrave;ne it is only the interaction, or to be more precise, the filling up of empty positions, which creates a new context for the user. Thus, her or his part in the reconstruction of reality seems to be autonomous to a large extent Although Feingold - with the book as interface -is quoting the reading culture, his position differs fundamentally from Shaw's, who in his Legible City tries to mediate between reading culture andi nteractive perception. In Feingold's Surprising Spiral the book is a relict of times past - auratically charged, but nevertheless hollow and robbed of its original function. What today is preserved or forgotten as history, does not follow the laws of written culture anymore, but is determined by the technological memory media. The reconstruction of the stored material is determined by the perception strategies of these new media.

The Surprising Spiral does not allow the purposeful approach of certain places, which is still possible in Shaw's Legible City. Its place is taken by the non directional, intuitive exploration of images and texts. This gliding through the picture sequences is similar to the images of Feingold's travel impressions - short moments which are unrepeatable, which are always remembered, or reconstructed differently or sometimes even forgotten.

On the basis of this sketch showing the beginnings of Interactive Art one cans ee that critical concepts about the role of interactivity in society are not missing. By discussing the works of Myron Krueger, Jeffrey Shaw, David Rokeby, Lynn Hershman, Grahame Weinbren and Ken Feingold I have distinguished six important implications of interactivity:
Power and Play
Participation versus Interaction
Proximity and Manipulation
Strategies of Seduction
Nonlinear Narration and
Remembering, Forgetting, and Reconstructing
7. The Second Generation

In the past years the second generation of interactive artists has emerged. Like with every second generation things are both easier and more difficult for them. On the one hand the artists are able to build on what has already been achieved, on the other hand they have to fulfill expectations of new developments. This young generation shows a clear geographical separation concerning the technologies used. While North-American and Canadian artists like Bill Seaman and Luc Courchesne are working with interactive installations and are using a touchscreen as interface, in Europe and especially in Germany the environment is asserting itself. The group Supreme Particles from Germany, for example, is working with the video camera as interface like Krueger and Rokeby. In Architexture the recorded image of the visitor is reproduced as a metallic-organic colour pattern on a moving projection screen. The computer graphical alienation of the image is so pronounced that a recognition is not easy. The fascination of the game is created primarily by the inner life of the image that pulsates between its own morphology and the representation of the visitor.

The sea animals in A-Volve by Christa Sommerer and Laurent Mignonneau have anautonomous existence, too. In A-Volve the visitors create little sea creatures with which they can then interact in a large water basin. The individual virtual creatures react very differently to the hand movements of the visitors. Some can be attracted, others try to flee from the hands. As their modes of behaviour are very difficult to find out, they create free play for the visitors who start to ascribe individual characteristics to thevarious animals. The interface Sommerer and Mignonneau worked with has completely lost its technoid character. This idea was already employed by the artists in their 1992 work Interactive Plant Growing. Here the reaching forreal plants causes the growth of computer-generated plants on a projection screen. Sommerer and Mignonneau draw the consequences from the increasing control computer technology has over our environment. To them the so-called artificial and the natural world do not oppose each other, but are closely interconnected areas. In dealing with these areas a sensibility is required that has to be partly re-learned, partly found anew.

Agnes Hegedfs' work Handsight requires a similar sensibility. The externalised eye - as the interface with the system - gives the viewer access to the virtual world which in the end is to be explored by using the sense oftouch. In Joachim Sauter's and Dirk Lysebrink's "Zerseher", too, the eye acquires tactile qualities. Only through the eye movements that are recorded by an eye tracker can a monitor image be destroyed and newly generated.

These few examples already show that the concepts for designing the interface and with it the design of the interaction are getting more and more subtle and diverse. The feedback-loop, which was most conspicuous in David Rokeby's Very Nervous System, is getting closer in the works of the young generation. The group Otherspace - that by the way, like the Supreme Particles and Sommerer & Mignonneau worked at the Institute for New Media in Frankfurt - uses brainwaves' to set little beetle-like beings into motion. Only if the test person manages to relax, do the solar-powered beetles start to move. Their movement in turn soothes the visitor so much that the result is a very intimate relationship. The debate on Artificial Life - or A-Life - that took place at last year's Ars Electronica seems to have created a sort of 3. Frankfurt School that is decisively influencing the development of Interactive Art. The question about the crucial differences between the first and the second generation of interactive artists makes clear various aspects:
Through institutions such as the Institute for New Media in Frankfurt, theMedia Art Academy in Cologne and the Karlsruhe Centre of Art and MediaTechnologies SGI workstations are available to young artists especially in Germany. This is one reason for the fact that the second generation favours interactive environments (and invisible or 'natural' interfaces) overinstallation work.
While in the work of the first generation a story or metaphors often influenced the content of the work, the content of the newer works is the interaction itself, which works without any form of traditional narration. Because of this new meaning of the interaction the design of the interface becomes increasingly important.
At the same time the antagonism between computer system and human being isovercome. It is not so much the antagonism but the forms of futureco-existence that are being reflected. That is, to put it shortly, the affirmation of interactive technology prevails over a critical distance, butthis does not result in an unreflected use of technology. Concerning thisgeneral affirmation of technology the first generation does not differ greatlyfrom the second.

All in all the multi-layered, encoded levels of meaning in early interactiveworks, which disclose their actual content only after a sort of decoding, contributed to a certain acceptance of Interactive Art in the 'art world'. However, this strategy had its price: the narrational contents often do notcome from contemporary social contexts, but from the safe context of history. With this, some artists of the first generation adressed the 'reading-habits' of the artcritic's establishment. They negated the achievements of the avantgarde, which clearly saw that art only has a chance when talking to the masses and not only to a small bourgeois elite.

This trend is starting to change with the new generation. If they will pursue this direction Interactive Art will fulfill its promise of being the beginning of a new dialogue between the two ideologically separated sections of art and technology.
References
[1] Uwe Jean Heuser: Der Computer Sbernimmt (The Computer Takes Over), in:Die Zeit, 29. 10. 1993, pp. 41 -42
[2] See for example Lynn Hershman: Art-ificial Sub-versions, Inter-action, and the New Reality, in: Camerawork. A Journal of Photographic Arts, Vol. 20, Nr. 1, 1993, pp. 20-25, p. 22
[3] Bertolt Brecht, Radiotheorie (Radio Theory), in: Gesammelte Schriften, Vol. 18, Frankfurt/M. 1967, pp. 119-134
[4] Umberto Eco, Das offene Kunstwerk (The Open Work, Opera apena), Frankfurt/M. [2]. 21973
[5] Wolfgang Kemp, Der Betrachter ist im Bild. Kunstwissenschaft und Rezeptionsästhetik (The Viewer is Inside the Picture. Sciences of Art and Receptional Aesthetics), Köln 1985
[6] See Erkki Huhtamo: Commentaries on Metacommentaries on Interactivity, in: CAD Forum, 4th International Conference on Development and Use of ComputerSystems, MediaScape, Zagreb 1993, pp 229-236
[7] Jeffrey Shaw, information on Points of View 1, 2, 3, dated 1983, withletter to the author, 24. 6. 1991
[8] For a detailed discussion of the development of Rokeby's oeuvre and itsdating see Süke Dinkla: Interaktive computergestützte Installationen. Eineexemplarische Analyse (Interactive Computer-Controlled Installations. A Studyof Some Examples), unpublished MA thesis, University of Hamburg, 1992, pp. 73-78. Here you will find a more detailed discussion of Shaw's Legible City and Krueger's Videoplace, too. A kind of summary of the MA thesis is published under the title 'Interactive Computer-Supported Installations. Examples of a New Art Form', in: CAD Forum, 5th International Conference on Development and Use of Computer Systems, MediaScape, Zagreb 1994, pp. 29-36 (originally published in: Künstlerischer Austausch. Artistic Exchange. Conference Proceedings of the XXVIII. International Congress for Art History, Berlin 1992, pp. 283-294, ill. , german)
[9] The former Film/Video Group (now the Interactive Cinema Group) produced the Videodisc Elastic Movies Disc with pieces by Bill Seaman, Luc Courchesne, Russell Sasnett and Rosalyn Gerstein in 1984. They worked at that time withBenjamin Bergery and Glorianna Davenport in the work-shop in Elastic Movie Time. This information is based on interviews with Bill Seaman in Karlsruhe(8. /9. 2. 1994) and with Glorianna Davenport (11. 7. 1994) in Cambridge and on theviewing of the Elastic Movies Disc at the Media Lab at MIT.
[10] See Lynn Hershman: (note 2), pp. 23, 24 and Lynn Hershman: Some Thoughts on Deep Contact, unpublished statement, 1991
[11] Interview with Grahame Weinbren, 18. 7. 1994, New York City
[12] Timothy Druckrey made a similar observation concerning Feingold's work Childhood/Hot and Cold Wars/The Appearance of Nature in his article 'Revisioning Technology', in: Iterations. The New Image, ed. by Timothy Druckrey, International Center of Photography New York City, Cambridge/London 1993, pp. 17-38, p. 35