Breaking Boundaries:
A Live Motion-Capture Performance Between Two Cities

An Interview with Andrew Crowe and Ashley Lee Wong from MetaObjects, Production and Development (Hong Kong) for Lu Yang’s DOKU, The Binary World

Lu Yang’s DOKU, The Binary World is a live audiovisual performance consisting of a real-time collaboration between dancers in Freespace of Hong Kong and Sydney Opera House, who will be interacting through the bodies of specially designed avatars in a shared virtual environment.

Hong Kong based digital studio MetaObjects, provides the production and technical development backbone of DOKU, The Binary World’s Hong Kong performance. They have been working with Lu Yang since their Electromagnetic Brainology performance at the Hong Kong Visual Arts Centre in 2018. Lu Yang’s live motion capture performances are a way to explore ideas around neurology, brain control and digital reincarnation, which have been reoccurring themes in the artist’s works.

Digital technologies always play a significant part in Lu Yang’s work. In DOKU, The Binary World, MetaObjects see the digital realm as the most important part of the performance, for it is “not just adding digital elements to a live performance, but the performance itself is based around the virtual environment.”

DOKU, the avatar, has a complex existence. Each avatar is a high-detail recreation of Lu Yang using facial capture techniques. Lu Yang sees these avatars as their digital reincarnation, as ideal versions of the self that would never exist in the “real” world. They are genderless avatars, with each of them representing the binary elements of the six realms of Buddhist reincarnation. Conceptually, the performance itself seeks to challenge binaries and binary thinking. “At the end, both the heaven and hell avatars, with the mixing of the different aesthetic characteristics of both, in terms of colour, dance styles or other elements, will end up combining into one.”

Lu Yang’s many works are about the control of body through external means. The DOKU series pushes that further by seeing the digital realm as a way of representing an afterlife. Ashley Wong, Artistic Director of MetaObjects explains that “the internet or the virtual world is actually a space where you can be reincarnated over and over again, and you can reinvent the self.” Lu Yang therefore sees the digital realm as a space where one reincarnates and how one may “experience the cyclic existence of learning and suffering that you endure through a Buddhist practice in life.”

While the movements of the avatars are controlled by that of dancers in real-time, the virtual existence allows them to co-exist in two spaces (Hong Kong and Sydney) simultaneously. In this live event, the motion data of the movements of dancers are streamed between Hong Kong and Sydney simultaneously so that the avatars can interact virtually. MetaObjects explains, “In each venue, there will be a physical dancer, and then the audience will see both the avatars of the local dancer and the dancer in the other venue who interact with each other in the virtual space.”

The dancers’ physical bodies and how their motions animate their corresponding avatars in the virtual space breaks not only the physical boundaries between the venues on different sides of the Earth, but the boundaries between the “real” and the “virtual” environment. MetaObjects and Lu Yang have worked together on a number of performances and with each performance, the technology has evolved with greater complexity both conceptually and technically.

MetaObjects works collaboratively with Lu Yang to create a live 3D performance programme using a game engine. “We take the 3D models for characters, the environment, and all the textures that Lu Yang created and convert them into the format that can be used within the game engine. We then use game engines to create virtual 3D environments with motion capture data from live dancers.” A VJ in Hong Kong and Sydney respectively will also be triggering the visual effects and camera movements in the virtual environment using a Xbox game controller live. Audiences will have a sense of liveness as the visual effects respond dynamically to the music in a spontaneous nature.

Technology is nowadays often perceived in terms of usefulness with a utilitarian sense, but tracing back to its Greek origin “techne” also embodies aesthetics. “All technology has a certain aesthetic quality to it even in the way it operates and the way that it looks, and we have to remember the origins of aesthetics aspect of how we live our lives, and how technology is always a part of human society and civilisation”. MetaObjects sees Lu Yang’s work as an attempt to “think and exist through Buddhist concepts to bring into conversation traditional Chinese aesthetic qualities together with practices in modern science and technology, such as neurology and motion capture.”