STEWART DICKSON
http://emsh.calarts.edu/~mathart
The Internet, specifically the World-Wide Web, is a radically new model of human communication and the recorded word in that the content is now directly accessible to machine interpretation. As significant as this is also that machine-readable representations of three- and higher-dimensional objects may now be processed as easily as words and two-dimensional images.Keywords: Language Communication Internet HyperText MIME Three-Dimensional VRMLThe challenge is now to better integrate higher-dimensional objects into communication streams to improve the power of person-to-person communication.
This paper will survey some current applications and resources for communication of three-dimensional shape information over the Internet, and suggest some new applications. We will take as motivation the symbolic processing of the "naturally-evolved" human language of mathematics, and processing "natural language" queries to on-line "dictionary" or "library" databases.
1. Background and Motivation
The Internet [1] World-Wide Web (WWW)
[2] Uniform Resource Locator (URL)
[3]
http://emsh.calarts.edu/~mathart is the repository of my artist's portfolio
and the on-line presentation of my sculptural work since 1978.
While presenting my work in Virtual Reality [4]
at the 4th Conference on Cyberspace at the Banff Centre for the Arts
[5], 1994, I determined that my
work is about the relationship between Cyberspace and physical experience and
therefore my work should be presented on-line. Cyberspace is where I belong,
therefore I resolved to "hang out my shingle" at http://www.mathart.com.
[6]
As early as 1986 I was able to have work accepted in a juried art exhibition
based not upon the finished work, but upon a computer-rendered proposal --
a photorealistic rendering of how the work might appear -- but the actual
work itself did not yet exist. I then produced the actual work in time for
the exhibition. [7]
In 1989 I created a proposal for an exhibit of physical sculptures from
mathematics.
[8] This work could be called under the blanket name,
"artifacts from cyberspace" -- but it is much more. The objects are the
products of famous and historical breakthroughs or "quantum leaps" in
understanding which were brought about through computer visualization.
This is part of the historical legacy of Cyberspace.
In 1975, Benoit Mandelbrot [9] used a computer at IBM
to make a graph of a dynamical procedure which was known to be chaotic.
Complex dynamics was a "monstrous" topic of inquiry, because there were no
known methods for dealing with such systems - least of which was simply
graphing the behavior, because so many - billions - of calculations are
required.
Fig. 1 -- The Mandelbrot Set
What Benoit found was a graph with symmetry, hierarchical self-similarity and infinite detail. No graph of this kind had ever been seen before. He had to invent a new branch of mathematics - fractal mathematics - to describe what he found. In 1982, Alan Norton - also of IBM - extended the mathematics of the Julia set (a "complimentary" graph to the Mandelbrot set) to Quaternion complex 4-space, which can be viewed in 3-space. [10]
The Seventeenth-Century French mathematician Pierre de Fermat wrote in the margin of his copy of Arithmetica by Diophantus, near the section on the Pythagorean Theorem (a squared plus b squared equals c squared),
"x ^ n + y ^ n = z ^ n - it cannot be solved with non-zero integers x, y, z for any exponent n greater than 2. I have found a truly marvelous proof, which this margin is too small to contain."This was left as an enigmatic riddle after Fermat's death and it became a famous, unsolved problem of number theory for over 350 years.
Andrew Hanson [11] has made some pictures, and I have in turn made sculpture (Figure 4), of a system analogous to Fermat's last theorem - a superquadric surface parameterized in complex four-space. We think that the mathematics of the n=3 case are similar to Fermat's own proof of the n=3 special case. Our pictures (Figure 2) have lent some visual concreteness to the recent news of Andrew Wiles' proof of the Taniyama-Weil conjecture, which implies the proof of Fermat.
Fig. 2 -- x^5 + y^5 = z^5
Image by Andrew Hanson and Stewart Dickson
By the 1890's the study of minimal surfaces was thought to be exhausted - no new surfaces could be described mathematically which were non-self-intersecting in three-space and which had vanishing mean curvature. However, in 1983 a graduate student in Rio de Janeiro named Celsoe Costa wrote down an equation for what he thought might be a new minimal surface, but the equations were so complex, they obscured the underlying geometry.
David Hoffman [12] at the University of Massachusetts at Amherst enlisted James Hoffman to make computer-generated pictures of Costa's surface (Figure 3). The pictures they made suggested first, that the surface was probably embedded - which gave them definite clues as to the approach they should take toward proving this assertion mathematically - and second, that the surface contained straight lines, hence symmetry by reflection through the lines.
Fig. 3 -- Costa's Minimal Surface
Image by David Hoffman and James Hoffman
The symmetry led Hoffman and William Meeks, III [13] to extrapolate that the surface was radially periodic and that new surfaces of the same class could be achieved by increasing the periodicity. They did so by altering the mathematical description of the surface to be the solution to a boundary-value problem constrained by the behavior of a minimal surface at the periodic lines of symmetry. The result: Hoffman and Meeks proved that Costa's surface was the first example of an infinitely large class of new minimal surfaces which are embedded in three-space.
The technique Hoffman and Meeks used was to make a picture which caused them to modify their mathematical theory and discover something totally unexpected about that theory. [14] They later extended their techniques to find minimal surfaces of more complex geometry and they also created pictures of them. This is a new kind of experimental mathematics and a procedure not far from creative visual art.
However, the pictures which were made in many cases were merely two-dimensional renderings of three-(or higher)-dimensional objects. The real discoveries were objects in three-(or higher)-dimensional spaces, but because of conventions in computer imaging technology (primarily the cathode-ray-tube) the observations of the discoveries were made through two-dimensional "windows" into cyberspace.
The real discoveries remained in the vacuum of abstract spaces, cyberspace and the cathode-ray tube.
I sent a copy of my exhibit proposal to Stephen Wolfram, author of the Mathematica System for Doing Mathematics by Computer. [15]
Mathematica implements a large subset of the "natural" language of mathematics in a machine-executable form. When making a graph of a mathematical expression, the machine can process symbolic language into a three-dimensional representation.
When the three-dimensional representation of a mathematical expression is cast into physical materials -- a sculpture in-the-round (Figure 4) -- via automated fabrication technologies, [16] I term this operation the direct concretization of mathematical language. [17]
Fig. 4 -- x^5 + y^5 = z^5
Stereolithograph by Stewart Dickson
The power of visualization has been proven to a fairly convincing degree by the history I have cited, but can similar arguments be made for tactilization? Clearly the segment of our population who cannot use computer pictures because their eyes do not work well enough to see the pictures have a real need for concrete artifacts from cyberspace.
I argue further that the physical presence provides a degree of immediacy missing from computer graphic displays, which in turn supplies more information than a two-dimensional picture. However, for the human observer, the meaning of the mathematical language which originates the form can become disconnected from the form when it becomes physical. I propose that improvement can be made by mapping descriptive text onto the three-dimensional object (See Figure 5).
For the purpose of discussion of the Internet, I believe mathematical objects may be of less importance to general human communication than are other types of three-dimensional shape information.
My on-line portfolio, in its current implementation, is no more than a traditional catalogue -- a published representation of artworks which might be purchased by a collector, or exhibited by a curator. Unlike a traditional catalogue, a Hypertext document on the Internet may be located by search engines and autonomous "Web Crawler" information-gathering agents. [18] Regardless of this feature, nothing about an Internet catalogue is intrinsically new, but the immediacy of information over distance. In general, publication on the Internet can be much more than this.
2. An Example in the Virtual Reality Modeling Language (VRML)
[19]
[20]
The World-Wide Web was born with the invention of the network-extended
HyperText Document, written in the the Hypertext Markup Language (HTML),
[21] and the desktop computer "Web browser"
application, NCSA Mosaic. [22]
The ability of Web server and browser applications to support formatting and
mixing of text and graphics is due to their support of the
Multi-Media Internet Mail Extensions ('MIME') [23]
standards.
VRML - the Virtual Reality Modeling Language
- is to HTML of the WWW - as 3-D information is to 2-D information.
The VRML "document" is a 3-D "world", containing embedded text and images.
Furthermore, object surfaces in the world, text and images serve as "hot-links"
to other documents -- of any type. Activating one of these objects in a
VRML world transports the viewer to another world.
Figure 5 illustrates how a VRML object comprises an integrated
entity which can communicate information about itself:
Fig. 5 -- Annotated Hyperbolic Paraboloid
Image by Stewart Dickson
A portion of a hyperbolic paraboloid has been mapped with text information indicating special features of the surface -- in particular, the curves which result when one of the parameters in the equation is held to a constant: The Parabolas at X=0 and Y=0, the Hyperbolas parallel to the X-Y plane, achieved by setting Z to a non-zero constant, and the degenerate hyperbola, or two straight lines which lie in the X-Y plane at Z=0.
The regions of the surface in Figure 5 mapped with text could be active regions in a VRML application. Activating ("touching" with a computer cursor) one of these regions would supply the viewer with auxiliary information about that region, or could transport the user to another VRML "world".
Note that a similar integration of information could be achieved by translating the text into Braille on a physical model of the Hyperbolic Paraboloid.
Hoffman and Hoffman did in fact encode the Gaussian curvature of the surface as a color map in the images they made, of the type shown in Figure 3.
3. Automatic Agents in the Noosphere
[24]
[25]
The Internet is the electronically-assisted social connection and collaboration
of the global population. This electronic assistance takes on an unprecedented
significance when it is in the form of networked, graphical computer
workstations. For the first time, with the advent of the
World-Wide Web, the content of human knowledge -- the no-o-sphere -- becomes
directly accessible by automatic agents, and the information density of
person-to-person communication explodes.
There have long existed Internet repositories of geometrical models, available
via the Internet File Transfer Protocol (FTP), [1]
such as avalon.chinalake.navy.mil. [26]
The models at this site are supplied in
various standard file formats, and it is left to the viewer to have
client-side software for viewing or manipulating the models.
To this capability to access and examine 3-D data objects, the VRML viewer adds
the ability to activate 3-D "worlds" with WWW-style Hyper-links. There is the
possibility of client-server interaction within a 3-D world the viewer
navigates, peering through the window of the CRT. Networked, client-server
game-style applications can be developed using VRML.
In 1995 I participated in INTERSCULPT'95 [27] --
a simultaneous and joint exhibition of sculpture in the Silicon Gallery,
[28] Philadelphia and Galerie
Graphe, Paris. During this event, I executed a work of 3-D FAX -- a 3-D work
created numerically in one place, transmitted to two other places via
the Internet File Transfer Protocol (FTP) then rendered physical in two
different remote locations via Rapid Prototyping technologies.
I believe this demonstration was a little closer to telepresence
[29] than VRML is at this point.
This demonstration, while novel, still did not fully utilize all the potential
available in the Internet.
With the availability of this technology, we must begin to think how to develop
applications to utilize the power of increased information density and
automatic content analysis.
In terms of three-dimensional information, there is clearly no precedent.
Therefore, I took the motivation for this development from the world of
two-dimensional images.
If the "literature" of the two-dimensional no-o-sphere is the image, then might
not the definition of the "language" be found in an image "dictionary"? The
"lexicon" used by the print designer is typically the stock photography
catalog; [30] A designer of moving images may often
consult a stock film catalogue; [31] And in the
dawning era of three-dimensional graphic design and
animation, there are beginning to appear vendors of "3-D Clip-Art" and
pre-built models for various purposes. [32]
The on-line versions of these catalogues are furthermore searchable.
The idea of an image dictionary implemented in a symbolic processing system
was the motivation behind my On-Line Interactive Poetry,
[33] Visual Poetry [34]
and corresponding Visual Dictionary project. [35]
I proposed using Natural-Language Processing [36]
to formalize the idea of an image dictionary, and in the process I discovered
several interesting things.
The system I built is a client communications application -- as simple as a
Java-enhanced Web Browser, [37] and a dictionary
server application, similar to an extension of the existing 'Webster' server.
[38]
The application I can demonstrate at the time of this writing is a networked,
client-server-based system of "automatic visual poetry" and "automatic
philosophy", similar to the famous "RACTER". RACTER composes prose from
a dictionary, a library of given grammatical skeletons and inference rules.
[39] [40]
My addition is that image 'icons' or 'ideograms' corresponding to the meaning
of the generated words, are in-lined in the returned HTML object.
Fig. 6 -- Web-Browser View of MathArt Visual Poetry
Image meaning has a level of detail in its context which can far outstrip verbal precision. I believe that there is much imprecision evident in current stock photography catalogues, if one views them strictly as image "Dictionaries". And the same is true of three-dimensional objects and worlds.
Given this difficulty in assigning an image to a verbal symbol, I determined that the dictionary system should be evolutionary -- dynamically re-assigning images to verbal contexts according to their frequency of use.
Furthermore, whereas MIME types are self-identifying objects to the extent that communications applications can decode and display them, any means for identifying the philosophical content of the objects is not standardized.
Identifying an image on the Internet per se, is probably limited at this point to the URL of the MIME object. This is a fundamental problem, of particular interest to those wishing to retain copyright over images published on the Internet. The ownership of an image itself is problematic in the age of post-mechanical reproduction.
At the present stage of the Internet, pictures of sufficient resolution to be valuable tend to occupy more data than can be economically transmitted for casual viewing. Therefore, the images which are routinely transmitted for informational viewing tend to be of a resolution sufficiently low to be of little commercial value in themselves.
But, bandwidth will not be a limiting factor for long. Owners of copyrighted material are beginning to use so-called "digital watermarking" [42][43] to identify images.
Similar issues also exist for 3-D object data, being often viewed as equivalent to the set of moulds for an original sculpture, over which the author typically wishes to keep edition control. I know of no method for similarly "watermarking" a 3-D object file, outside of "signing" it with geometrical text, embedded in the object's surface.
In the testing of the prototype Visual Dictionary server system, I have developed a HTML-based user interface from which anyone can add word and image definitions to the dictionary. Thus, I can potentially harness the collaborative power of the Internet to build the dictionary. The system is intended to eventually use a combination of natural language processing and evolutionary programming to automatically update word and image definitions from recorded frequencies of contextual word usage.
Clearly, in this age of electronic communication, the language of our civilization is evolving faster than published dictionaries can keep pace. Given, however, than a dictionary can produce information on the way people think about the ideas words describe -- I believe that an automatic, evolutionary dictionary can produce unexpected new information on the culture which engages in discourse over the Internet.
The system is designed to be extensible to support any MIME type, including VRML objects. I believe such a system could serve as a tool for studying ideas regarding the philosophical content of images and three-dimensional shapes.
4. Conclusion: Future Extensions
I have shown how the Internet can be a searchable catalog and repository of
three-dimensional shapes. I have also presented an overview of the current
technology of the World-Wide Web -- the active Hypertext document or VRML world
which combines words, pictures and three-dimensional objects -- with special
emphasis on the content as philosophical symbols in a logical universe of
discourse.
I have demonstrated that a VRML entity can be a multi-dimensional,
self-describing, integrated information object. If translated into Braille
on a physical model, in-the-round of a mathematical surface, something
interesting in addition occurs: As the reader's hand passes over the text,
describing the surface at that region, the hand is constrained to the surface
under all six degrees of freedom -- that is, the hand not only occupies a
three-space position determined by the mathematics of the surface, but it is
also oriented according to the normal vector to the surface and the line along
which the text lies in the surface. I believe there occurs in such a case an
element of corporeal awareness of the object which should not be discounted.
Where can we go from here? There are a host of improvements and developments
which can and will be made to the visual dictionary system.
In the present form of the visual dictionary, verbal contexts could be
translated into rules of two-dimensional image composition, which could in turn
produce HTML 4.0 extensions to add further expression to the existing
"automatic poetry" system. Similarly, rules of three-dimensional composition
could be applied to generate 3-D VRML constructions from text input.
I have proposed a two-way, person-to-person application, which utilizes the
visual dictionary server, to act as a semiotic amplifier: To multiply "by 1000"
the meaning of words in a typed conversation, by automatically in-lining
image icons into the conversational stream. In the virtual 3-D world of
cyberspace, it is not unreasonable to speak of similar "conversational" modes
in three dimensions as well.
What could three-dimensional conversation mean? There exist formal languages
which can describe three-dimensional form and gesture.
American Sign Language is a commonly used language which is expressed in the
three-dimensional form of the hands.
Fig. 7 -- The Laban notation of the classical ballet
Laban notation [44] has been formalized in a computer-based lexical analysis system, which can parse symbolic dance 'language' and produce the corresponding movement in a computer-rendered figure. [45] This is in effect a translator from two-dimensional, symbolic language into time-based, three-dimensional expressive form. It is known that Asian and Polynesian dance has an even more literal correspondence between gesture and meaning than does the European ballet.
The role of the computer network is to span distances and serve as an expert translator and an amplifier of meaning. My goal is to synthesize the shape-describing power of mathematics with higher-level philosophical symbols to produce a generalised language of three-dimensional form and gesture, which could be used to enhance person-to-person communication.
References
This article is reprinted in:
"Immersed in Technology: Art and Virtual Environments",
Diana Augaitis, Douglas MacLeod and Mary Anne Moser, eds.;
MIT Press, 1996, ISBN 0-262-13314-8;
http://www-mitpress.mit.edu/book-home.tcl?isbn=0262133148.
The VRML specification is basically the OpenInventor
[20] 3-D computer
graphics exchange standard with extensions in the form of "custom" node types
unique to VRML -- in particular, nodes such as AsciiText, Switch, WWWAnchor and
WWWInline. This allows the VRML author the ability to build objects in three
dimensions which behave interactively in a three-dimensional VRML browser in
the same manner as HTML documents do in a two-dimensional HTML browser.
Open Inventor;
http://www.sgi.com/Technology/Inventor.html.
HTML includes features which support electronic documents containing mixed
text and graphics. Furthermore, "anchors" in the document (either word
strings or images) may serve as "hyper-links" (or "hot-links") to other
documents. The Web browser allows the reader to select a hot-link by the
keyboard and mouse interface, and by so doing, a new hypertext document,
referenced in the hyper-link, is loaded into the browser and becomes the new
focus.
The authorship is attributed to RACTER, "written in compiled BASIC on a Z80
with 64k of RAM." Racter (the program) was co-authored by Chamberlain and
Thomas Etter.' [41]
"...connection need not presuppose domination, or mediation, control. The
ecology of souls, together behaving as one organism, has in its form the
embedded understanding that each part is important, and none dominant. The
center is everywhere, the circumference nowhere. Pierre Teillard de Chardin
[25]
called this nexus of connection noosphere; studies of connective mediation are
equally studies in noospherics."
'The introduction claims: "With the exception of this introduction, the
writing in this book was all done by computer."