Project Overview

Project Directors: Jeffrey Shaw, Dennis Del Favero
Programmers: Ardrian Hardjorno, Volker Kuchelmeister, Matthew McGinity
(Additional Software and Hardware Engineering: Jared Berghold, Marc Chee, Robin Chow, Alex Kuptsov, Alex Ong & Xin Guan)
Project Funding: ARC DP0209550, ARC DP0345547, ARC LE0453517

AVIE exterior view

AVIE: exterior view showing Nebula

AVIE is the world’s first mobile and scalable 360-degree 3D immersive and interactive visualisation system. It measures 7–10 metres in diameter and 3–3.5 metres in height. It includes a motion tracking system and a multi-channel audio system. Driven by an experimental framework, it uses a suite of cutting-edge hardware and software resources that enable the development of applications in AI visualisation, immersive environments and networked collaboration. With a number of these systems available, it is deployable on-site with industry and project partners for research development and application.

tvispg1 AVIE: interior view showing T_Visionarium II

AVIE has been installed as core visualisation infrastructure at: ArtLAB ÉPFL (Switzerland); Chengdu Research Institute, Chengdu (China); CityU Hong Kong (Hong Kong); China Coal Technology & Engineering Group, Fushun (China); Experimental Media Performing Arts Centre, Rensselaer University, New York (USA); ZKM Media Museum (Germany); Mines Rescue Pty Ltd; Museums Victoria; Museum of Applied Arts & Sciences; UTS Sydney (all Australia).

It has been commercialised for the mine safety training industry by: Mines Rescue Pty Ltd at its Newcastle, Wollongong, Lithgow and Singleton facilities, Australia; Shenyang Research Institute of China Coal Technology & Engineering Group (CCTEG), Fushun, China.

AVIE has been commissioned as a curated visualisation platform by numerous premier international venues for biennales, exhibitions and festivals, including:

  1. Fo Guang Shan Buddha Museum, Dashu, Taiwan, 2020
  2. École Polytechnique Fédérale de Lausanne, Switzerland, 2020
  3. CityU Galleries, City University of Hong Kong, 2020
  4. Maritime Museum 360, Hong Kong, 2020
  5. Centre for Applied Computing & Interactive Media, Chengdu, 2020
  6. Art of Immersion: Virtual Frontiers, ZKM, Karlsruhe, 2017
  7. Chengdu Research Institute, Chengdu, 2017
  8. Digitalife, Palazzo delle Esposizione, Rome, 2017
  9. Sydney Film Festival, Sydney, 2016
  10. Bonlieu Scène Nationale, Annecy, 2016
  11. International Symposium of Electronic Art 2016 (ISEA16), Hong Kong, 2016
  12. GLOBALE, ZKM, Karlsruhe, 2016.
  13. Seoul Mediacity Biennale, Seoul Museum of Art, Seoul, 2016
  14. Montreal Museum of Fine Art, Montreal, 2016
  15. Chronus Art Center, Shanghai, 2014-15
  16. Child, Nation and World Cinema Symposium, Sydney, 2014
  17. 20th International Conference on Virtual Systems and Multimedia, Hong Kong, 2014
  18. Data Arena, UTS, Sydney, 2014
  19. International Symposium of Electronic Art 2013 (ISEA13), Sydney, 2013
  20. 10th Shanghai Biennale, Power Station of Art, Shanghai, 2013
  21. SIGGRAPH Asia, Hong Kong, 2013
  22. Hong Kong Book Fair, Hong Kong, 2013
  23. Limited Knowledge, Hong Kong, 2013
  24. Festival d’Avignon, Avignon, 2013
  25. Business of Design Week, Hong Kong, 2012
  26. NODEM, Hong Kong, 2012
  27. Grand Opening of Run Run Shaw Creative Media Centre, Hong Kong, 2012
  28. Smithsonian Institution, Washington DC, 2012
  29. Matter-Light, Bethune, 2011
  30. Sydney Film Festival, Sydney, 2011
  31. Festival of Avignon, Gymnase Paul Giera, Avignon, 2011
  32. National Institute of Experimental Arts double conference, Sydney, 2011
  33. ALiVE – Applied Laboratory for Interactive Visualization & Embodiment, CityU, Hong Kong, 2011
  34. Digitalife, La Pelanda, Rome, 2010
  35. 15th Biennial Film & History Conference, UNSW, Sydney
  36. Dynamic Earth, Museum Victoria, Melbourne, 2010-2020
  37. Inaugural Exhibition, ALiVE – Applied Laboratory for Interactive Visualization & Embodiment, Hong Kong, 2010
  38. STRP Festival, Strijp R Klokgebouw, Eindhoven, 2010
  39. APPEA Conference and Exhibition, Convention and Exhibition Centre, Brisbane, 2010
  40. Energy Field Days, Showground, Sydney, 2010
  41. Un Volcan Numérique, Le Volcan Scène Nationale du Havre, Le Harve, 2009
  42. Seconde Nature, Espace Vasarély, Aix-en-Provence, 2009
  43. International Documentary Film Festival Amsterdam, Zuiderkerk, Amsterdam, 2009
  44. International Architecture Biennale, Erasmus Bridge Tower, Rotterdam, 2009
  45. Imagining Media at ZKM, Karlsruhe, 2009
  46. Immigration Museum, Melbourne, 2009
  47. Sydney Festival, Sydney, 2008
  48. Biennale of Seville, Carlos V Alambra Palace, Granada, 2008
  49. eARTS Festival: eLandscapes, Zendai Museum of Modern Art, Shanghai, 2008
  50. Mines Rescue Pty Ltd Australian Facilities in Woonona, Argenton, Lithgow & Singleton, 2008
  51. YOUser: the Century of the Consumer, YOU_niverse & Panorama Festival, ZKM Media Museum, Karlsruhe, 2007
  52. EMPAC, Rensselaer University, New York, 2007
  53. i.Future Festival, Singapore Science Centre, 2007
  54. VSMM Conference, UNSW, Sydney, 2007
  55. Australian Council of University Art & Design Schools Conference, UNSW, Sydney, 2007
  56. AIMEX Australian Mining Industry Conference, Sydney, 2007
  57. From Spark to Pixel, Martin Gropius Bau, Berlin, 2007
  58. Kaladham Museum, Karnataka, (permanent installation since 2006)
  59. La Rivoluzione Algoritmica, Artescienza: Spazio Deformato, Casa dell’Architettura, Rome, 2006
  60. Lille3000 Festival, Bombaysers de Lille, Rotonde de L’Opera, Lille, 2006
  61. Festival d’Avignon, Avignon, 2005
  62. ZKM, Karlsruhe, 2004
  63. Cinémas du Futur, Lille 2004 Capitale Européenne de la Culture, Centre Euralille, Lille, 2004

Video AVIE

AVIE has been developed though three iterations, with further development on-going. AVIE1 (2004-07) facilitated audio-visual data to respond kinaesthetically to the user's body across the full breadth of the projected space; AVIE2 (2008-2010) has augmented this environment by delivering the ability to recompose digital files in real-time; AVIE3 (2011-2017) has delivered a base-line AI Database system that enables digital agents to interact with users,based on their ability to interpret and respond to the user’s behaviour in space. AVIE provides a platform in which 3D audio-visual experiences are created by a combination of multiple projectors and multiple sound sources, driven by a cluster of computers, running a highly versatile real-time 3D VR engine. It comprises custom designed technologies: a Motion Tracking system, a Mixed Reality system and an AI Reasoning system. The AI system coordinates the actions of the visualisation system by allowing it to autonomously reason about its response to user behaviour. The Motion Tracking system tracks the physical location of users in AVIE, while the Mixed Reality system allows the creation of 'scenarios' that exhibit a feedback loop between the world of the user and of the audio-visual data.

AVIE visualisation forms:

  • High-resolution panoramic stereo images with augmented 3D graphics overlays
  • High-resolution panoramic videos, including stereo videos
  • Immersive multimedia data visualisation
  • Real-time interactive 3D environments supported by artificial intelligence systems

AVIE interaction platforms

AVIE utilises two types of interaction:

  1. a) remote device: a wireless tablet device allowing real-time selection and navigation within interactive applications
  1. b) motion tracking: real time software that detects, captures, tracks and interprets human movement and gestures, allowing capture and audience participation in the direct cinematic experience, interplay between users and digital characters and analysis of user behaviour

mining_in_AVIE_1 mining_in_AVIE_2 AVIE: interior views showing Mining VR

AVIE Components

avie_top AVIE: external view

Panoramic Immersive Stereoscopic 3D Visualisation System

AVIE consists of a cylindrical silvered screen 4 metres high and 10 metres in diameter on which 3D panoramic images sequences and videos are projected. The 6 projectors use active shuttering technology to deliver stereoscopic 3D to end users, and the images from the individual projectors are seamlessly blended by custom software developed in-house at iCinema. 6 high-performance image generators, capable of processing images and videos as well as rendering large virtual environments in real time, are linked to the projectors to
deliver the full immersive 3D experience. It accommodates up to 30 people at any one time.