Disney’s ‘Magic Bench’ takes augmented reality to greater heights

Disney Research latest innovation on augmented reality now allows people to have a multi-sensory immersive experience. This means a user can occupy the same space and directly interact with an animated character.

Dubbed as “Magic Bench”, Disney Research's newest technology eliminates the need for head-mounted display or handheld devices. Magic Chair works using instruments properly set in the surroundings of the users, letting people share the whole augmented reality experience as a group.

"Our mantra for this project was: hear a character coming, see them enter the space, and feel them sit next to you,” shared Moshe Mahler, principal digital artist at Disney Research.

The magic takes place in the bench

Mahler noted that the bench plays a crucial role for the whole experience. The whole immersive experience does not begin until user/s is seated on the bench. It contains haptic actuators and serves as a controller. With the help of the bench, the developers were able to determine the number of participants, where they are located and what they are looking at. The bench also creates a suitable stage for the mixed reality.

With the seated participants acting as the middle ground, the bench separates the foreground from the background.

Users seated in the Magic Bench may receive a glowing orb from an elephant, or feel a small donkey kicking the bench or rained on. It might feel magical for the users, but the whole experience is the product of a simple, yet complicated technological process.

The science behind the magical experience

A large screen is set in front of the bench. The screen creates a mirrored image of the participants and their surroundings, making a third person point of view. Color camera and depth sensor are strategically placed in the platform to collect data. An algorithm aligns the data gathered by the color camera with the information from the depth sensor and creates a real-time, 3D reconstruction of the participants, the bench and the surroundings.

Disney Research also used a modified algorithm to create a 2D backdrop in areas where depth sensor has no corresponding line of sight with the color camera. By doing so, the developers were able to eliminate the depth shadows in these areas. The 3D and 2D reconstructions are positioned together in a virtual space. Users could now see themselves on the large screen together with computer-generated 3D characters and effects.

The Magic Bench is set to be presented at the SIGGRAPH 2017, the Computer Graphics and Interactive Techniques Conference, in Los Angeles starting on July 30.