An interactive promotional experience developed for Coca-Cola, where two participants stood in front of a screen and saw animated thought bubbles appear above their heads, creating a short branded conversation.

I developed the Unity application and implemented the player-tracking logic using Kinect.

This included testing different tracking approaches, moving from regular-camera face detection to depth-based body tracking, and creating the logic that kept Player One and Player Two consistent throughout the experience.

This was an early commercial project and involved several technical challenges, especially around reliable user tracking during a live event.

The main challenges included:

• Avoiding hardware-heavy solutions that would require more powerful computers.
• Tracking two people at the same time in front of the screen.
• Keeping each participant assigned correctly as Player One and Player Two.
• Finding a tracking method that worked reliably during the event.

• Depth-based tracking – Unlike traditional cameras, it could distinguish real faces from printed images.
• Consistent head tracking – It accurately recognized users regardless of their facial orientation..

My initial approach was to use face-tracking algorithms through OpenCV in Unity. This would have allowed the experience to work with a regular camera, but it created two major issues.

First, face tracking was not reliable enough when participants moved, turned their heads, or were not looking directly at the camera. Second, real-time image processing required more powerful computers than the ones available for the event.

To solve this, I switched to Kinect, which was available from the client. Kinect’s depth-based tracking made the experience more reliable because it could detect users based on body and head position rather than only facial features.

The biggest technical issue was keeping the same two participants assigned as Player One and Player Two. Kinect could refresh the detected bodies during runtime and return them in a different order. This caused the two players to occasionally swap positions in the game logic.

To solve this, I used the unique ID of each detected body and locked each participant to a specific player slot, maintaining continuity throughout the interaction.