Tangible Dynamics

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/c30b35cd-05e8-4812-b5c8-f1fe085db023/IMG_4582.jpg

A real-time 3D tabletop experience controlled by motion tracked light-emitting orbs.

Collaboration between: Alvaro Lacouture & Nicole Ginelli

We are inspired by new ways to visualize and bring physicality to realtime/ user driven 3D. We want to bring liveliness to 3D animation, allowing users to control details and parameters of a living animation.

This piece is a physical 3D viewing experience/ table projection that reacts according to the position of two illuminated objects and outputs real-time 3D particle animations. Tracking the control objects is done with a Kinect. Visual rendering is done by a real-time game engine (Unity), while data communication and video mapping is controlled with Touchdesigner. LEDs within the control objects are controlled via Arduino.

Motion tracking and video mapping are tools that have been used countless times before. For the motion tracking, we tried using a camera with IR sensors on the control spheres, color tracking with openCV to track their color, and even used the HTC Vive controllers + sensors to get an accurate tracking. Each of these had their pros and cons. However we ultimately found that combining the kinect's depth tracking in addition to luminance detection was the best solution for a real-time movement and reaction effect.

Sections

Final Proposal

Documentation

Visual Research

Technical Details

Materials