For my final project I wanted to create some techniques for utilizing AI tools within the program TouchDesigner. I had been working in TouchDesigner for another class I had been taking this semester, and I absolutely loved using it and seeing how many different techniques that can be implemented in it. After seeing a special effects clip shown in class, I wanted to try my hand at creating interesting visual effects using AI as well.
My first attempt was trying my hand at audio reactive GANs but I found that they were pretty limited in their scope, as it would just cycle through the same couple of images for the duration of the whole song.
So my next idea was to use a real time image detection model that would be fed into TouchDesigner - this worked somewhat well, but unfortunately my computer couldn’t run the program in real time.
What I ended up landing was to use Runwayml’s depth map generator to create faux 3D effects on 2D images. The technique I used was taking the depth map, which had its white to black gradient replaced with a rainbow gradient. After this I used a chroma key to select specific depths in the image to create greenscreens, scanning effects and much more.
The gif below is a snippet of the first project I made using this technique, I used the depth map image to create a pixel sorting effect that starts from the tops of the buildings instead of being overlaid on top of them in a straight line like traditional methods.
The project linked below was also made using this technique, this time I had the chroma key on a timer that creates a scanning effect on the foreground, and seperately isolated the sky to add visual there.
https://drive.google.com/file/d/1hbCy-SEwkPip8DMpPtRIOCWIlzlihmHn/view?usp=sharing
I created this as way for myself to create art, but I purposely designed it in a way that it would be plug and play for other people. So below I linked the .tox components that can allow anyone to use these effects with a video and the depth imagery from it.