The project

The goal of the project was to track two people standing in front of a screen and display their thoughts. Naturally the thoughts revolved around Coca-Cola, emphasizing why purchasing Coca-Cola products was a great idea. The thoughts appeared as a short sequence, creating a mini conversation between the two characters.

Technical Approach & Challenges

My initial approach was to use face-tracking algorithms via OpenCV in Unity. While this method had the advantage of working with a regular camera, it also presented significant challenges:

  • Face recognition complexity – The algorithm could detect faces looking directly at the camera, but accuracy dropped when faces were at an angle or partially obscured.
  • Hardware limitations – Real-time face tracking and image processing required powerful computers, which were not available for this project.

To overcome these obstacles, I switched to using the Kinect, a 3D tracking camera that was available from the client. The Kinect offered several advantages:

  • Depth-based tracking – Unlike traditional cameras, it could distinguish real faces from printed images.
  • Consistent head tracking – It accurately recognized users regardless of their facial orientation..

Biggest Challenge: Consistent Player Tracking

One of the toughest challenges was ensuring that the system consistently tracked the same two people throughout the experience. The Kinect frequently refreshed detected bodies and assigned them in a random order. To solve this, I leveraged the unique ID of each detected body, locking them as Player One and Player Two to maintain continuity.