light
screen/
projection/
refl ection
spectator
from: Jean-Louis Baudry,
Idological Effects of the Basic
Cinematographic Apparatus
. 1970 Cinéthique... translation
1974 by the Regents of the
University of California... Film
Quarterly 1974
Cinema Apparatus
in AR
Let me take several examples to outline some territory in augmented or mixed reality within
the art gallery. David Rokeby is an artist whose interactive work made an early entrance to
the art gallery. His famous 1988 piece, Very Nervous System used motion tracking to create
an invisible interface to computer sound creation. His subsequent gallery work has been
mainly visual, relying on live camera, and frequently incorporating this live-camera image
into the gallery space. His 2002 installation, Seen, installed at the Canadian Pavilion of the
Venice Biennale, used what could be termed a screen-as-window arrangement. The intended
留學(xué)生dissertationconstruction was four live-feed video cameras whose footage would be altered in real time with
temporal, spatial, and chroma fi lters, and then projected into four augmented-image screens.
In actual fact, he was not able to arrange the hookup for the live cameras, and used instead 30
minutes of pre-recorded footage. But so important to Rokeby—and, in fact, to the apparatus
defi nition of AR—is the live nature of this work that he still describes the apparatus as livecamera
based in his description on his website, and only notes in parenthesis that this was not,
in fact, true. These ‘live camera feeds’ are then presented in four augmented screens, each
deconstructing the temporal image in different ways: the middle two screens display only the
elements of the image that are in motion or static, and the other two screens provide algorythmic
changes of the image, based on combining multiple frames in time, tracing the paths of motion
that have taken place in the image. A second work by Rokeby, the 2002 Cheap Imitation, is#p#分頁標(biāo)題#e#
confi gured as a ‘magic mirror’—a technological constellation that is fairly common within AR in
the gallery as well as performance. The monitoring camera is pointed out at the viewer from the
screen, and instead of projecting the analogous mediated image of the viewer, the screen presents
an augmented image... in this case the viewer/ fi gure of the art work is presented, through motion
tracking, as distorted segments of Marcel Duchamp’s Nude Descending a Staircase... or, more
acurately, the subject’s observed motion triggers the revealing of segments of the Duchamp
Seen, David Rokeby, 2002
Cheap Imitation (screen view), David Rokeby, 2002.
Be Careful, Fragile, Clara Boj and Diego Diaz, 2006
Level Head, Julian Oliver, 2008
image directly in front of their motion, so that if they move their hand, directly in front of this
movement that part of the image is revealed.... if they move their whole body, a larger section of
the image is revealed, corresponding to their body.

In a totally different aesthetic attitude, there is a screen-as-mirror piece by Clara Boj and Diego
Diaz, Be Careful, Fragile which was created for the 2006 ARCO exhibition in Madrid, Spain.
This magic mirror does not provide an affected mediated image, as Cheap Imitation does, instead
it uses a more functional aesthetic of AR and motion tracking, overlaying a 3 dimensional virtual
object, in this case a vase, on top of the mediated image using a fi ducial marker tracking system.
The gallery visitor is presented with an empty pedestal, identical to many of the pedestals at the
exhibition holding real artwork, behind which is a screen that shows the gallery space, as if in
a mirror, except the pedestal holds a virtual-looking vase, it’s contours and color synthetically
regular. When the view passes their hand over the top of the pedestal, crossing the visual path
between the live camera and a black-and-white printed marker on top of the pedestal, the screen
shows the animated image of the vase falling to the fl oor and breaking. Then, in the screen, the
viewer sees the two artists arrive and place another identical vase on top of the pedestal. Boj and
Diaz, in their description, say the work “questions its own presence, as digital artwork, in the art
market... showing that there are no “unique digital artworks.” Unlike much of Rokeby’s work,
Be Careful, Fragile could be described as sheerly conceptual, lacking any effort for an aesthetic#p#分頁標(biāo)題#e#
sublimity, but within the context of conceptual art, it succeeds, successfully representing a
concept within a gallery art frame. A fi nal AR piece I want to note, is Julian Oliver’s Level Head
(which he, in fact, terms a “spatial memory game,” but at the same time lists the art galleries and
festivals it has been exhibited within). Like Be Careful, Fragile, Level Head uses fi ducial marker
systems—tracking embedded in an object—but here the object is also the site of what will be
replaced with the virtual, so it becomes a type of magic object within the mirroring screen.
Unlike Be Careful, Fragile, the object embedded with the AR markers is the art object itself.
The cube is meant to be handled and shown to the camera, which gives back an image, or partial
image, of the user, along with an augmented image of the cube, where each side is replaced
with relative, relating animations: a digitally drawn human character that walks labrynthine
paths through a confusing, Escher-like cubic world. Like Rokeby’s Seen, it’s interesting to
note the language of his description, where he states the only interface is the marker cube...
because the projection as mediated reality is so common, it is suppressed in the description of the
apparatus—not considered as part of the interface.
If we take these three orientations of cameras and screens in the art gallery, what can we say
about the elements of the apparatus? How do they differ from those of cinema in the structuring
of circuits between objective reality, camera, projector, and spectator, and those things that come
in between, and can we fi nd within this process of creating a mixed reality what is suppressed
in the circuit, and what is presented as the representation or art. If we take the basic AR circuit,
and combine the apparatus diagrams of Baudry with the diagrams of Azuma, we might have
components like those drawn here, containing objective reality, subject/ spectator, camera/
capture, mediated reality, computation/ tracking, process / scene generator, render/ video
compositor, and screen. I have also included in brackets within these chains aknowledgement
of the input that comes from outside the electrical apparatus; such things as the framing of
participation, framing of reception, and the produced content added as augmentation. We can
make some generalizations about what is suppressed and what used to express within these
different types of AR artworks.
All areas of AR seek to suppress the computation/tracking and render... for all AR a system
crash, rasterizing issues, registration errors are considered against the goal. The interface
should be “invisible,” and the computer should not reveal its inhuman character, displaying the
sharp edges of computation, fundamental errors of representation, or, most repressed of all, the#p#分頁標(biāo)題#e#
digital code that lies underneath (such as a ‘screen of death’ when not only a program, but the
underlying operating system kernel crashes). In fact many artists, such as Rokeby, emphasize
their desire to subvert the “digital” nature of the machine, and produce, instead, image that is
evocative of the human or organic. But within this wide generalization of suppression that is in
almost all of new media art, there is a diversity of how the perception of objective reality and
computation, mediation (or ‘virtuality’... a bad term) should co-exit. We can see a spectrum,
where at one end there is Virtual Reality, and at the other end, sheerly industrial applications.
In Virtual Reality the goal is to completely suppress Objective Reality and Mediated Reality
(that is, suppress any perception of mediation within the media) and achieve an absolute cinema
of fi rst person perspective. At the other end are the AR proto-typical Boeing wiring diagram
interfaces, which used see-through displays over which were overlayed tracking diagrams, with
the specifi c goal to neither suppress objective reality, nor to convolve the virtual and the real. In
mediated reality objective reality
capture/
surveillance/
camera
Processor/
scene generator/
fi lter
Difference /
Augmentation
video compositer/
render
screen
spectator
Augmented Reality Apparatus: Window
tracking/
computing
[framing]
objective reality/
Installation
Subject/
Figure
mediated reality
capture/
surveillance/
Processor/
scene generator/
fi lter
video compositer/
render
screen
Augmented Reality Apparatus: Mirror/ POV
tracking/
computing
Artist/
Figure/
Objective Reality
screen/
mediated reality
Video Art Apparatus: Acconci
spectator
camera
frame/
mis-en-scen
Video Art Apparatus: Campus
objective reality/
Installation
Subject/
Figure
mediated reality
capture/
surveillance/
camera
Processor/
scene generator/
fi lter
Difference/ Augmentation
video compositer/
render
screen
Augmented Reality Apparatus: expanded with dispositif
tracking/
computing
limits of allowed
performance/ activity
framing
objective reality/
Installation
Subject/
Figure
capture limitations/
camera
screen limitations/
installation THE
WALL/
mediated reality
projector
between these two are the projects of the art gallery which we have been looking at, which seek
different degrees of suppression and convolvement between virtual and real.
In the Magic Mirror type display, such as Rokeby’s Cheap Imitation, the object of representation#p#分頁標(biāo)題#e#
is the mediated subject or viewer and his or her surrounding objective reality. In a site-specifi c
mode, objective reality is not suppressed, but instead the area of expression is within the
juxtaposition of the real, the live mediation, and the illusion. So that the experience of viewing
Cheap Imitation is one of checking and inspection... the viewer presents themselves to the live
camera and checks for their own representation in the mirroring display. The difference between
the imagined refl ection/mediation of a live mirroring camera, and the observed augmentation
displaying fragments of Duchamp—the lack and excess that occurs between ‘expected’ and
found—is the space of expression. In the case of the Window displays, such as Rokeby’s Seen,
the mediated is presented as real (thus Rokeby’s mendacious insistence that the pre-recorded
footage was live)... the mediated piazza San Marco is to be perceived as the real object being
effected, and, as in fi lm, to be taken as objective reality—though the suppression is even greater
than in fi lm, because it is to be taken as a circuit from objective reality... the live broadcast...
the analogue of the thing itself. So again, the area of expression is the difference between the
image of the piazza that the viewer imagines the camera to be seeing, and the augmented images
playing with duration on the screens. But here there is no checking, as in the mirror, instead
perhaps a backwards analysis... by searching the augmented image of the piazza and drawing on
their own experience and knowledge, the viewer imagines what the non-augmented image might
be, and thereby locates at least the area and nature of the augmentation. Similarly we can see
different goals in the computational processing of the different AR art works. Rokeby’s fi lter
based augmented projections I would call ‘Affected Reality’. The mediated is not suppressed,
instead everything is rendered to this fi nal product. It is like what Brenda Laurel has described
as a stage, where all the behind the scenes capture, mediation,and computation, are rendered to
a single digital representation, with no other artifact [Laurel 2001, p110]. In this type of work,
the computational nature of the processing is frequently extremely suppressed. Rokeby is an
exemplar in this respect. In a lecture for Info Art, he describes his aesthetic goals in computation:
“the computer as a medium is strongly biased and so my impulse while using the computer
was to work solidly against these biases. Because the computer is purely logical, the language
of interaction should strive to be intuitive. Because the computer removes you from your
body, the body should be strongly engaged. Because the computer’s activity takes place on the
microscopic scale of silicon wafers, the encounter with the computer should take place in humanscaled#p#分頁標(biāo)題#e#
physical space. And because the computer is objective and disinterested, the experience
should be intimate.” But, at the same time that the nature of computation is suppressed, the fact
of augmentation—the fact of the mediation being manipulated and affected—is emphasized as
exactly the site of representation and expression. The discontinuity between real mediation and
illusionistic augmentation is not suppressed—as in cinema when the CG dinosaurs are made
to combine realistically—but played upon. We as receivers are looking within the art work for
what is affected in the image and what remains in order to interpret the representation. With the
the two examples of object-based AR in the gallery—Be Careful, Fragile and Level Head—the
role of the mediated is different. The mediated is to a larger extent suppressed. Instead the
mediated image is to be taken as a valid reality, and it is within the mediation that the borders
between virtual and real are presented. The juxtaposition between illusion and objective (albeit
mediated) reality is emphasized as metonymy and addition within the screen... an augmentation
of space. In a sense, within these types of augmented reality, the point of entrance to the work
is in the edges between real and illusion—these sharp edges of addition. It is not an affected
mediation. This is why, at least for me, the fi ducial marker based work carries a special
fascination: the marker as a material object becomes a frame for this edge... a materiality of the
digital code, similar to the aesthetic preciousness of the fi lm sprockets for structuralist fi lm.
In all these works I think you could argue that the ground of representation is the circuit itself...
the mediation taking place—like Rokeby’s insistence on the live nature of the cameras. The
basis of representation is the live nature of the transmission—the mediated as a component...
a live analogue of reality, and exactly this circuit which is being augmented. In many ways
the body—either the body as subject-object as is the case with mirror displays, or the body
as depicted and showing presence and scale as in the Piazza San Marco—is the delineating
edge of augmentation. In the case of the magic mirror, part of the game is that our bodies
dually inhabit the non-augmented space and an augmented space. In a sort of Christian Metz,
cinematic identifi cation schema, it is like a tertiary stage of identifi cation (beyond his secondary
narcissism) where we must have embodied mediation itself in our perception to identify. This
circuit as a material for expression is not essentially digital, but instead, electric... based off of
the live nature of the electric image, but taken to its height in digital augmentation. We can, in
fact, see its relationship to the criticism of video art by Rosalind Krauss in the 70’s.#p#分頁標(biāo)題#e#
In her 1976 essay, Video and the Aesthetic of Narcissism, Krauss defi nes her critical search in
the new medium of video art as for “an object-state, separate from the artist’s own being, through
which his intentions must pass” ...like the pigment bearing substances of painting, and the
matter through space of sculpture. She defi nes the crucial element of the electric image in this
regard as the instantaneity of the communication from notion to message: “This is why it seems
inappropriate to speak of a physical medium in relation to video. For the object (the electronic
equipment and its capabilities) has become merely an appurtenance. And instead, video’s real
medium is a psychological situation, the very terms of which are to withdraw attention from
an external object—an Other—and invest it in the Self.” The object is bracketed out, she says,
and instead the artist is creating within a psychological state created by the mapping of the
mind onto this network, in a very McLuhanesque the-medium-becomes-the-nervous-system
way. In her analysis, this self-gazing video art is, for the viewer in the art gallery, like the
viewing of an electronic loop of the artist, camera, and screen. AR, instead, is this loop itself
installed in the gallery, now including the apparatus within the gallery. The actual apparatus of
the electric image medium are installed. Lars Qvortrub, in an essay on digital poetics, proposes
that all of new media art could be described as ready-mades, where instead of R. Mutt’s urinal,
there is now a computer, a projector, a screen, a camera... “digital materials have realized the
intentions of the avant-garde” he says [p241]. But AR art has complicated the ready-made; it
is an art of apparatus, but not of the objects themselves, but the objects plugged in to eachother,
broadcasting to eachother and constantly looping in real live time... art of circuits. The apparatus
is released from the linear narrative of cinema because it does not have to be linearly productive
(in the sense of Leotard’s acinema), because it is constantly in movement. If we take this
electric circuit as the fundamental material of expression within AR art, then I think the next
step is to come to grapple with how this circuit augments the viewer. Like Krauss, we can try to
留學(xué)生dissertationunderstand what this means for a psychological state. If video art of the 70’s was appropriately
placed to deal with issues of surveillance, celebrity, and mediation coming out the tradition of
pop art, then what is AR art poised to express? We could begin by saying it is a state of postmediation.
We are no longer seeking to grapple with mediation, but have embodied it, and are,
instead, grappling with our relationship to the interface and machine. And, if we have succeeded#p#分頁標(biāo)題#e#
in creating an acinema—breaking narrative but retaining movement—then what are the new
ideological limits of the new apparatus.