Panel presentation: Aesthetics and Tools in the Virtual Environment; The Art of Virtual Reality
Governor’s Conference on the Arts VII: Arts, Entertainment and Technology; The Role of the Artist in the Digital Age
Los Angeles, 1996


Absolute Animation
and
Immersive VR

Michael Scroggins
California Institute of the Arts
School of Film/Video

aka@emsh.calarts.edu

http://emsh.calarts.edu/~aka



Opening Comments

While critical discourse has long been an important factor in evaluating art production, it has come to dominate mainstream art practice for a number of years. Writers are naturally attracted to forms of art which are directly engaged in discursive reason because they are readily accessible to analytic language. The art and language movement of the early seventies serves as a vivid example of the result of this particular bias amplified through the incestuous feedback loop of theory and practice. The superheated outcome of this compression of roles was invigorating to many (including myself), but the perception that this school (as all previous schools) rendered all other approaches passé should be clearly seen as absurd. The ephemerality of intellectual fashion is well documented.

Current theories of culture and the specific effects of electronic media upon culture inform much of the exciting work being done in computer (involved) art making today. It is important to recognize that this type of art making is not necessarily the only valuable work being done, but simply the most seductive to the engaged intellect and thus in danger of marginalizing other practices by pushing them to the periphery of serious consideration --even characterizing them as naive and/or frivolous. I am clearly exaggerating these tendencies here because I want to throw the bias of my own art practice into relief. I'm interested in reinforcing the validity of the type of non-discursive art practice exemplified by absolute music, and am painfully aware of the paradox of using language to do so. Laurie Anderson put the dilemma most succinctly when she said:

"Talking about music is like dancing about architecture."1


Abstract

The ability to shape temporal experience through the manipulation of a set of simultaneous and successive events is a power which sound producing instruments have afforded the aural composer/performer since pre-history.

The development during the last decade of realtime videographic devices capable of instantaneous generation and manipulation of absolute (or abstract) images has given the visual artist the same power. In this decade, the rapid advancements being made in realtime computer graphics technology promise even more powerful visual instruments, particularly in the totally enveloping new medium of immersive Virtual Reality.


Introduction

The idea that one might compose event streams with light much as one composes event streams with sound is an old one, and the last century saw many experiments with the newly evolving technologies for manipulating light through the development of various forms of color organ. The evolution of abstract and absolute painting in the early decades of this century provided for a new surge of interest in the possibilities of a visual art of pure movement.2

Writing in the Paris of 1914, Leopold Survage proclaimed:

"Painting, having liberated itself from the conventional forms of objects in the exterior world, has conquered the terrain of abstract forms. It must get rid of its last and principle shackle --immobility-- so as to become as supple and rich a means of expressing our emotions as music is. Everything that is accessible to us has its duration in time, which finds its strongest manifestation in rhythm, action, and movement, real, arranged, and unarranged.

I will animate my painting. I will give it movement. I will introduce rhythm into the concrete action of my abstract painting, born of my interior life; my instrument will be the cinematographic film, this true symbol of accumulated movement. It will execute the 'scores' of my visions, corresponding to my state of mind in its successive phases. I am creating a new visual art in time, that of colored rhythm and rhythmic color."3

Survage made a substantial series of paintings for his first absolute film, but was unable to get the financial backing to have it shot; however other artists, such as Viking Eggeling, Walther Ruttmann, and Oskar Fischinger, were fortunate enough to realize their ambitions, producing a strong body of work in absolute animation that inspired generations of artists including: Len Lye, Harry Smith, Jordan Belson, the Whitney brothers, Larry Cuba, and many others, including myself.

Wassily Kandinsky developed theories based on relationships of color, shape, and sound, but never attempted to make an animated film, apparently preferring to work in the medium of spatial/temporal light play and kinetic theater --as did several of his colleagues at the Bauhaus: Laszlo Moholy-Nagy, Ludwig Hirschfeld-Mack, et al. In 1928, Kandinsky produced a performance in Dessau, consisting of the movement of colored lights and elemental shapes manipulated within the void of a black draped stage.4

In 1921, Theo van Doesburg visited Viking Eggeling and Hans Richter in their studio and wrote enthusiastically in the journal De Stijl about the potential of their absolute animation work, expressing caution about the unavoidable danger for misunderstanding which can be involved in comparing it to music.5 The problem of the musical model which concerned van Doesburg is still with us today. It results from a confusion between thinking of a comparison based upon a direct synaesthetic correspondence rather than one based upon general principles of temporal composition.

The scientific investigation of the periodic wave nature of both sound and light has led to many attempts to find a direct correlation between the octave structure of audible vibration and the structure of the visible light spectrum. Many investigators, including Newton and Goethe, considered the idea of a direct relationship between sound and light, but nothing truly universal has ever been determined.6

Apart from the fundamental physical differences in electromagnetic and air pressure waves, the sensory input from the organs of hearing and sight appear to be mapped onto the brain, and thus consciousness, in quite different ways. It seems that any universal correspondences remain elusive, and ultimately we are left with only very subjective relationships.

In searching for relationships, it is probably best to avoid looking for direct correspondences and concentrate on finding the essential parameters characteristic of each medium, and how, as event streams in time, these basic parameters may be composed into patterns of architectonic structure having affect on the perceiver.

I would like to identify three main forms of non-verbal time-based artforms and some of their basic parameters:

Music

Dance

Animation

a. Rhythm

a. Rhythm

a. Rhythm

b. Pitch

b. Posture

b. Color

c. Timbre

c. Movement

c. Shape

d. Harmony

d. Pantomime

d. Texture


These parameters are obviously arbitrary. They are not intended as direct corollaries (i.e., while pitch might be compared to color, it is certainly not a corollary to posture!) and each might be applied to another form in a slightly different way. For example, harmony is not necessary in all forms of music, and harmony can also be understood in terms of a particular type of spatial relationship, physical motion, or relationship of two or more colors.7

It can be seen that rhythm is the parameter which they all have in common, and it is in fact the rhythmic structuring of the other parameters that gives each medium its shape in time. It is of course important to realize that it is possible to identify rhythms occurring simultaneously at different scales of time, from the minute and quick events which make up the most discrete of units, to the larger and slower overall temporal structures which give form to the entire composition.


Performance

Musical instruments can be seen as a technological extension of the will to voice sound. A vocalist produces sound through a direct act of will, unmediated by technology. An instrumentalist produces sound through an act of will mediated by a technology which allows for an expanded range of aural expression. Each kind of musical instrument has unique capacities for generating sound and this effects the range of affect a musician can elicit with them . The potential of a trombone is different from that of a violin, an oboe, a piano, or timpani. The design of an instrument presents both opportunities and constraints in the composition of music. A composition for harpsichord is unlikely to resemble a composition for sitar.

Visual instruments can be seen as a technological extension of the will to move. A dancer produces movement through a direct act of will, unmediated by technology. An animator produces movement through an act of will mediated by a technology which allows for an expanded range of kinesthetic expression. The concept of an instrumental form of dance is relatively new and carries with it some particular problems regarding the human form as sign; but one need only note the vast difference between singing and playing a piano to begin to see the implications.8


Stepping into Virtual Reality

Aside from my awareness of the difficulties in comparing absolute music to absolute animation (including the different ways in which hearing and seeing are mapped onto consciousness), I still aspired in my videographic animation work to create an affect as engaging as that possible through a musical experience. Even though I did achieve some success toward that end, I was never entirely satisfied. I realized that one of the problems which faced me was the limitation of my visual compositions to the same relatively small flat window in space to which painting and drawing are usually confined. While I enjoyed playing with the familiar compositional devices which the rectilinear border provides, I was envious of the totally enveloping nature of music. I yearned for a way to break free of the boundary of the frame and become as totally immersed in image as in the ocean of air pressure which constitutes sound. That has now become possible. The new medium of immersive VR has dissolved the boundary of the frame, allowing us to virtually enter into the image for the first time in history.9

In the realm of immersive VR it will be possible to manifest and choreograph a wide variety of form in motion. Softly pulsing aurora-like color fields may ebb and flow through the space. Shapes may be made to coalesce out of the void or fly in and out of proximity from afar. Independent shapes may be caused to orbit one another in hierarchical constellations of complex and shifting interdependence. Shapes may be made to flow around, move through, or bounce off of participants' body forms. Sweeping gestures of the arms and legs may leave an intricate calligraphy of dissolving trails, or the gestures of one performer may create solid trails, while those of another carve out voids in a virtual pas de deux of additive and subtractive sculpting.10

Procedural animation programs that mimic the flocking behavior of birds or the schooling behavior of fish, may allow a performer to swing swirling clusters of tiny scintillating triangles around at the end of an invisible line radiating from a hand; open the hand, and the line grows longer; close the hand, and the line grows shorter.11

The development of particular instrumentality's will depend upon the understanding and desires which evolve from direct experience in the virtual world. As in improvisation with videographic animation, the complete range of compositional possibilities will only be revealed through the directed play of the will into kinesthetic expression. The full potential of absolute animation in Virtual Reality remains to be seen.


Concluding Remarks

The drive to aesthetic motion has been a part of human culture from time immemorial. The universal existence of the art of dance bears witness to the importance of this drive. From deeply felt rituals of cosmic union to mere decorative entertainment, dance has played, and continues to play, an active role in nearly all societies. Absolute animation instruments have extended the will to motion beyond the constraints of the human physique, thus allowing for the creation of aesthetic experiences beyond that which is possible through the direct movement of the unmediated body.

The capacity for total immersion and interaction in VR make it an ideal medium for the continued development of absolute animation. This development will yield experiences as far beyond our abilities to imagine them today as were the abilities of the inventors of the first primitive musical instruments to imagine the awesome experience of a late nineteenth century symphony. The work I have produced in realtime absolute animation leads me directly to the threshold of VR, and I look forward with great excitement to extending my work into this new realm of expanded possibilities.


Notes

1. Laurie Anderson, Home of the Brave, Warner Reprise Video, New York, 1986.
return to text


2. An excellent historical overview of early color music is given by William Moritz in the exhibition catalog for The Spiritual in Art: Abstract Painting 1890 - 1985, Los Angeles County Museum of Art, Abbeville Press, Inc., New York, 1986, pp. 297-311.
return to text

3. Text of a sealed document, No. 8182, deposited on June 29, 1914, at the Academy of Sciences of Paris, as quoted by Robert Russet and Cecile Starr in Experimental Animation, Van Nostrand Rheinhold Company, New York, 1976.
return to text

4. Ludwig Grote, "Kandinsky's Stage Compositions", i 10, International Review, Vol. No. 13, Amsterdam, 1928 p. 4 f as quoted by Wulf Herzogenrath in the exhibition catalog, Film as Film: Formal Experiment in Film, 1910-1975, Hayward Gallery, London, 1979, p. 28.
return to text


5. De Stijl, Vol. IV, No. 5, 1921, pp. 71-75.
return to text


6. Some of Newton's thoughts on the correspondences between light and sound may be found in an edition of his 1730 treatise, Opticks, Dover Publications Inc., New York, 1952, pp. 154-158, 346.

Goethe's thoughts on the problem are expanded upon in Charles Lock Eastwood's notes to his 1840 translation of the Theory of Colours, M.I.T. Press, Cambridge, 1970, pp. 417-418.
return to text


7. For an example of the difficulties in this area see John Whitney's book, Digital Harmony: On the Complementarity of Music and Visual Art, McGraw-Hill, New York, 1980.
return to text


8. Experiments by Oskar Schlemmer, Laszlo Moholy-Nagy, and others at the Bauhaus could be seen as an attempt to create a form of "instrumental dance". See The Theatre of the Bauhaus, Edited by Walter Gropius, Wesleyan University Press, Middletown, 1961.
return to text


9. An early comprehensive look at the history and future of the field of Virtual Reality can be found in the quarterly publication, Mondo 2000, Issue Number 2, Summer 1990.

Jaron Lanier, then CEO of VPL Research, Inc., discusses his thoughts on Virtual Reality in, "An Interview with Jaron Lanier: Virtual Reality", Whole Earth Review, Fall 1989, pp. 108-119.

A good overview of early work in "artificial reality" can be found in James D. Foley's, "Interfaces for Advanced Computing", Scientific American, Vol. 257, No. 4, October, 1987, pp. 126-135.
return to text


10. My ideas for working with trails in VR grow out of the film work of Norman McLaren, the stereoscopic timelapse light sculptures of Ed Emshwiller, my own experiments in analog and digital video trails, and discussions with Vibeke Sorensen about her proposal to the National Science Foundation (NSF) for a project at the San Diego Super Computer Center researching and developing tools for three dimensional "fingerpainting".
return to text


11. Craig Reynolds describes his work modeling behavioral systems in, "Flocks, Herds, and Schools: A Distributed Behavioral Model", Computer Graphics, ACM SIGGRAPH Conference Proceedings , Vol. 21, No. 4, July 1987, p. 25
return to text