1. Powerhouse Museum, Sydney

Conversations is composed of three electronically linked installations that are positioned at remote locations within the museum. The visitor at each installations wears a head mounted display (HMD) whose twin LCD screens present an immersive stereoscopic display. A gyroscopic tracking device, (Intersense) attached to each HMD, monitors the movements of the viewer’s head, so that it can generate images which correspond to the direction and angle of view. Microphones and headphones attached to the HMD enable the viewers at the three remote locations to hear and speak to each other. The microphones are also linked to a software based voice recognition system that enables the viewer to also interact with the environment by means of spoken commands. Each of the installations is equipped with a PC and high performance real time computer graphics hardware (nVidia). Custom application graphics software enables the construction of a shared distributed virtual environment in which the three viewers are simultaneously immersed and can individually navigate and interact with the features of this hybrid space of representation.

Installation at the Powerhouse Museum, Head Mounted Display

Video documentation

Its main attributes are:

A contemporary stereoscopic panoramic photograph of the Pentridge Prison and its environs. The viewers, wearing the HMDS, find themselves located within this scene and can freely look about it in any direction. The viewers also move between two different renditions of this scene – one is a daytime setting where the sequence of Ryan’s escape from prison is enacted, and the other is a nighttime setting inhabited by the ghosts of the protagonists in Ryan’s story.

Panoramic view, Pentridge in Daytime

Actors, who are the central characters in Ryan’s escape and subsequent murder trial, are three dimensionally placed within this photographic setting. These narrative agents are video-graphic entities that have been previously recorded in a Chromakey studio. In the Pentridge daytime setting, these narrative agents have been scripted and composited into the virtual environment and present the viewers with a linear narrative experience. In the nighttime setting, where Pentridge has become a ghost world, these ethereal narrative agents float about and individually respond to each of the three visitors, both when they are purposely looked at, as detected by the tracker on the HMD, and when they are talked to, as interpreted by the voice recognition system.

Panoramic view, Ghost world – Pentridge at Nighttime

Computer generated avatars of the three visitors, who become aware of one another, are positioned within the Pentride setting. Each is identified by a different color, with which each corresponding installation is also coded. The real time behavior of these avatars depicts the viewers’ navigation of the virtual environment, their interactions with the narrative agents, and with one other. The trackers on the visitors’ heads control the corresponding behavior of their avatars’ heads, and when voice detection is activated, these avatars are animated by typical conversational gestures.

2. iCinema Scientia Laboratory

In this configuration, an innovative new component has been added to the three HMD installations. It is a large immersive hemispherical projection, set within a three-meter diameter half dome, mounted vertically in front of the viewer. Whereas each of the HMD viewpoints expresses the natural point of view of a visitor within the shared virtual environment, the hemispherical projection presents a global view of this environment, where one can simultaneously see all the multimedia assets – the Pentridge prison setting, the videographic narrative agents and the three visitor avatars. Using simple trackball control the user can rotate this scene and move back and forth between distant to close-up viewing. This immersive visualisation system provides a highly effective technique for rendering a complete audio-visual observatory of the Conversations multi-user, multi-modal, virtual environment with all of its narrative and interactive components, and is a significant advance on the multi-screen monitor setups that have been used until now to elucidate the workings of such complex mixed reality situations.