Hand, C. "Mixed Arcadian Reality: Exploiting Synergies Between the Real and the Virtual". Workshop on Arcadian Virtuality - Ecological Information Spaces, at i3 Spring Days Conference, Sitges, Spain, 7-10 March 1999.
Mixed Arcadian Reality: Exploiting Synergies Between the Real and the Virtual
Chris Hand
HCI Group, School of Computing, Napier University,
219 Colinton Road, Edinburgh, Scotland. EH14 1DJ
The idea of building a user experience by creating a synthetic "world" containing objects, as opposed to "writing programs", is central to the evolution of artificial realities. Environments such as ARK (Smith, 1986) took Papert' Logo microworlds (Papert, 1980) a step further and created systems which could develop over time according to what the user did in the world. This persistence continued with the introduction of multi-user worlds using fledgling networking technology: MUDs. Unlike their text-based cousins the chat worlds, MUDs and MOOs include a persistent database of objects representing buildings, landscapes, users and their possessions/contents. These means that the artificial world can evolve over time.
Multi-user VEs can be considered to represent an ecology of objects, an organic collection of entities which evolves over time. We identified the opportunity for capitalising on this during the CRCMOO project, which created a text-based VE to support the CRC cards collaborative OO design review technique:
... creating a customised "Corporate Virtual Workspace" ... an evolving, design-oriented ecology in which virtual teams of software designers use tele-working and virtual worlds technologies to create new forms of software.
(Hand, Skipper & Dinsdale, 1995; p62)
Since the problem domain was software engineering, we saw no reason why users and objects alike should not be embodied in the VE at the same representational level, with the result that any of the objects participating in "run-time" message-passing could be (at one extreme) a completed and working software component, or (at the other end of the spectrum) a human user role-playing the object in order to gain insight into its collaborations and responsibilities, or any of the points in between. Thus a software system would evolve from a "live design" being role-played by the design team, to a fully functional object system.
The VR experience will never be purely virtual, despite our best attempts to pretend otherwise. Our only purely virtual experience is dreams. Whether driven by sci-fi ideas of disembodied minds and direct neural interfaces, or a weak attempt at achieving immortality driven by post-enlightenment faith in technology, the abandoning of physicality is a mirage. Within the VR community, this school of thought manifests itself in those who equate sensory immersion with cognitive immersion, and believability with display resolution.
Put another way, virtuality and physicality comprise a mutually-supportive duality, but this is often ignored in the rush to become totally virtual. A good example of the folly of abandoning physicality is the long-standing fallacy of the "paperless office", which has consistently failed to appear: paper in its many forms has a useful and important role to play in the office.
This real aspect is often the neglected part of the equation in building the electronic communities which have been appearing and growing and dying for some time (Morningstar, 1990; Rheingold, 1993). While they sport many of the features of a real community, and their virtual benefits are numerous, they still lack important features of real-world groups, which ultimately marginalises their participants. As Robert Cailliau has famously pointed out, "there's no such thing as a virtual beer", and in fact many of the successful and longer-lived e-communities have survived by organising regular face-to-face social gatherings, which strengthen the groups rather than undermining their on-line aspect.
Therefore the design of modern interactive environments should accept the mutually-supportive duality of the physical and the virtual. Moreover this then allows us to exploit the synergies between them.
Another evolutionary strand of VR has recognised the importance of the physical, and here we can identify the influence of Myron Krueger, one of the early VR pioneers. Early installations such as Videoplace (Krueger, 1991) used pressure pads in the floor and video techniques to detect movements of users in a special room. From the beginning, Krueger's approach was to design the real and the virtual as a whole. Krueger's influence can be followed through the strand of VR which includes instrumented props (e.g. Sachs, 1991), augmented reality interfaces such as Wellner's DigitalDesk (Wellner, 1991), through mixed realities (e.g. Brown et al, 1996) to Tangible media (Ishii & Ullmer, 1997). In other words the strand in which the balance between real and virtual is being redressed.
Nevertheless, I'm not suggesting that we should abandon the virtual in favour of the real; rather that we should strike a balance and give the real-world aspects of system design equal consideration. This means that we need to look for design principles that will allow us to combine, and get the best of, both worlds.
Alongside the evolution of VR it is interesting to observe the recent developments in hypermedia, particularly where it looks as though the two might merge. In his ECHT'92 keynote address, Boulter described the approaches of "hypertextualizing the space" or "spatializing the hypertext". While the former resulted in projects such as Harmony (Andrews, 1993) and VRML, the latter approach is exemplified by Aquanet (Marshall & Rogers, 1992), and VIKI (Marshall et al, 1994). These last two systems, being multi-user environments explicitly structured around euclidean space, are moving very close to CVEs.
Arcadian metaphors have already made some progress in these hypermedia VEs. Bernstein's analysis (1993) of the Aquanet paper interprets the collaborative user activity as farming. He describes how farming (which he also calls gardening) differs from mining or manufacturing of information: the emphasis being on "continuous cultivation and community" rather than extraction or stockpiling. In another hybrid Hypermedia/VE system supporting a virtual trade exhibition (Hand, 1995) we provided a way for users to create web pages by collecting objects in a MOO -- a process that could be described as harvesting.
Permaculture is a design system, "one of the most holistic, integrated systems analysis and design methodologies" (Pilarski, 1994; p450), which may be able to help us to design integrated, holistic, multi-purpose spaces. The term "permaculture" - a contraction of permanent and agriculture, or permanent and culture - is attributed to Bill Mollison, the Australian ecologist, who defines it thus:
It is the harmonious integration of landscape and people providing their food, energy shelter, and other material and non-material needs in a sustainable way. [...] The philosophy behind permaculture is one of working with, rather than against, nature, of looking at systems in all their functions, rather than asking only one yield of them, and of allowing systems to demonstrate their own evolutions.
(Mollison, 1997)
Although permaculture's emphasis is on designing and creating agricultural environments, as a holistic approach it encompasses all users' "material and non-material needs" (ie. real and virtual) which includes work and leisure as well as food and shelter.
As well as being founded on people-centered and Earth-centred ethics, Permaculture includes many guiding principles such as "Each element performs multiple functions", "Each function is supported by many elements" and "Polyculture and diversity of species" which may be transported over into the design of virtual as well as real Arcadian spaces. Space precludes any detailed contemplation of how these principles might be applied, but perhaps we are already beginning to realise that everything we design and build should be in harmony with nature. We recognise that buildings, electronic devices, VCRs etc. need to be designed in harmony with human nature (Norman, 1988), as well as incorporating ecological features which make them less discordant with mother nature. User-centered system design has already begun to address the issue of technology aiming to be in harmony with human nature to some extent, but if we leave the "rest" of nature out of the equation then we can't expect to produce an holistic solution. Recognising that we will never be wholly virtual requires more commitment to designing the real.
Rather than just using Arcadian metaphors in the purely virtual world, there are benefits to be gained from adopting a holistic, permaculture approach to designing the whole experience, in both the real and the virtual domains. Creating a real garden -- be it indoor, outdoor, or a hybrid -- for the user to inhabit while interacting with (and to compliment) its virtual equivalent allows us to enrich the user experience along more than one dimension:
Taking our cues from Tangible Media type projects such as Ishii's ambientROOM (Ishii & Ullmer, 1997) we can imagine using the real arcadian space as a way of providing ambient displays for its occupants -- for instance an arcadian equivalent to the Live Wire device (Weiser & Brown, 1995) might use a small garden wind-chime hanging on a stepper motor, with the level of virtual wind being related to (say) level of remote user activity in a VE, and causing more or less background tinkling accordingly as a way of promoting peripheral awareness. Foreground information might also be presented using projections or custom displays.
A simple example of such an arcadian mixed reality is Stream of Consciousness (White & Small, 1998), an interactive, augmented reality garden described by its creators as a "harmonious environment". Although only small (6 feet square) it includes many features of a classical garden, such as a fountain with water flowing over stones, and a bench for sitting on. The virtual aspect of this environment is interactive poetry: words are projected onto the surface of the water as if they were floating downstream, and using a "liquid haptic" pad it is possible to change the movement of the words and to change the words themselves.
We could envisage further combinations of real/virtual, biology/technology, such as cyborganisms combining plants with small mobile robots and environmental sensors, so the plants (which may be food-yielding) would be able to move around the space to find their optimum growing positions, moving out of strong sunlight to avoid drying out, or seeking water when necessary, or acting as foreground/ambient displays. As with VR, this is all just a system integration exercise -- there is no technological reason why we can't start building these environments immediately.
Many thanks to Mark Skipper for his valuable comments on an earlier version of this paper.
Andrews, K. (1993) "Constructing Cyberspace: Virtual Reality and Hypermedia". In Proceedings of Virtual Reality Vienna '93.
Bernstein, M. (1993) "Enactment in Information Farming", Proceedings of Hypertext'93, pp242-249, Seattle, ACM Press.
Brown, C. C., Reynard, G. T., and Benford, S. D., "The Internet Foyer: An Example of Mixed Reality", in Hand, C (ed.) Proceedings of the 3rd UK VR-SIG Conference, Leicester, UK, July, 1996, pp. 113-119.
Hand, C. (1995) "Creating hypermedia documents by doing: an alternative to authoring". IEE Colloquium on The Authoring and Application of Hypermedia-Based User Interfaces, London, 14th November 1995. Digest No: 95/202. pp3/1-3/3.
Hand, C., Skipper, M. and Dinsdale, A. (1995) "Collaborative Object-Oriented Design using Role-Playing in Shared Virtual Spaces". Proceedings of the 4D Dynamics Conference, De Montfort University, Leicester. September 21 1995. pp.59-62.
Ishii, H. and Ullmer, B. (1997) "Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms". Proceedings of CHI'97.pp234-241.
Krueger, M. (1991) Artificial Reality II. Addison-Wesley.
Marshall, C C, Shipman, F M and Coombs, J H. (1994) "VIKI: Spatial Hypertext Supporting Emergent Structure". in ECHT'94: Proceedings of the 6th European Conference on Hypermedia Technology, Edinburgh, Scotland, 18-23 September 1994.
Mollison, B. (1997) Permaculture: A Designers' Manual. Ten Speed Press.
Morningstar, C. and Farmer, F. R. (1990), "The Lessons of Lucasfilm's Habitat" in Cyberspace: First Steps, Michael Benedikt (ed), 1990, MIT Press, Cambridge, Mass.
Norman, D. A. (1988). The psychology of everyday things. New York: Basic Books, Inc.
Papert, S. (1980) Mindstorms: Children, Computers and Powerful Ideas. New York: Basic Books, Inc.
Pilarski, M. (ed.) (1994). Restoration Forestry. Kivaki Press, USA.
Rheingold, H. (1993). The Virtual Community: Homesteading on the Electronic Frontier. Addison-Wesley.
<URL: http://www.rheingold.com/vc/book/>
Sachs, E., Roberts, A. and Stoops, D. (1991) "3-Draw: A Tool for Designing 3D Shapes". IEEE Computer Graphics and Applications, Nov 1991. pp18-26.
Smith, R. B., (1986) "The Alternate Reality Kit: an animated environment for creating interactive simulations", Proceedings 1986 IEEE Computer Society workshop on visual languages. Dallas, June 1986. pp99-106.
Weiser, M. and Brown, J. S. (1995) "Designing Calm
Technology". Electronic Publication, Dec 1995.
<URL: "http://www.ubiq.com/hypertext/weiser/calmtech/calmtech.htm>
Wellner, P. (1991) "The DigitalDesk Calculator: Tactile Manipulation on a Desk Top Display". Proceedings of ACM Symposium on User Interface Software and Technology (UIST '91). Nov 11-13 1991. pp27-33
White, T and Small, D. (1998) "An Interactive Poetic Garden" CHI'98 late-breaking results, pp335-336.