Experiments

Early experiments in AR and body tracking led to the creation of the technology stack.


Technology Stack

Zed Mini + LeapMotion + Vive + Unity combine to track the wearer's hands and position, process depth and occlusion data in VR, and are deployed to the player using the Unity engine.

Zed Depth View
Zed Stereoscopic View
LeapMotion Skeleton View

Zed Depth View Zed Stereoscopic View LeapMotion Skeleton View

Workshops

Exploratory workshops were mounted with performers and improvisers. Participants generated prompts inspired by the technology to create improvised scenes exploring unusual use-cases. Insights from a wide range of perspectives and disciplines inspired further exploration and sketching.


Unity

Developed in Unity to take advantage of the strong support for the technology stack and VR, the performance uses virtual elements calibrated to the performance space to imbue the real-world with augmented properties.

Test footage of augmented lightning effect, demonstrating how the walls and floor of the space have been augmented with colliders.

Code structure of the narrative version of Playtime for Punctum.
Code structure of the gesture-based fingerpainting interaction.

Code structure of the narrative version of Playtime for Punctum. Code structure of the gesture-based fingerpainting interaction.

Summary

Augmented role-play is suitable for play and exploration in the unusual context of real-world architecture augmented with virtuality. It suggests methods of interaction outside of common screen-based ones, and affords exploration with the body. The interaction between real and virtual objects offers players bodily and narrative agency, and players can make experiences for themselves regardless of the designers’ intention. Augmented role-play is an opportunity for exploration and experimentation of the possibilities inherent in the body, and in the relationship between actual and virtual environments.

The flexibility of the toolset allows a distinct experience to be created for specific sites, as virtual primitives stand in for real-world architecture, endowing them with physical properties. It is a simple matter to calibrate an augmented role-play for a new location. This affords exciting opportunities when designing with specific locations in mind, as real-world characteristics, properties, and historical contexts can be brought to bear on the design of virtual elements, creating an embodied digital experience that lets participants create their own unique experiences.


Using Format