As he played, a little Processing sketch used the infra red cameras in the Xbox Kinect to create a semi 3D image of him (this is called ‘point cloud’), the cloud moved as he moved, and changed color as guitar’s sound changed. We captured the sound and video from the processing, and mashed them together on editing software (Final Cut Pro and After Effects) to add a little more texture to the video. This bit was handled by Sowmya.
In the end what we have is a music video for MellowGhost’s ‘How to be the rain’.
I hope you enjoy it!