https://editor.p5js.org/Valerie-Lin/sketches/y7ftJSecB

For my final project, I made a digital string puppet that you control with your hand.

The visual inspiration came from Nick in Zootopia, wearing his suit. But the real core idea is actually from a very old traditional Chinese art form: string puppet theater.

The most interesting part, I think, is that I used digital technology to bring this traditional performance onto the computer screen. Real string puppets are actually quite hard to operate—there are lots of strings, the movements aren’t easy to control, and it takes a long time to learn. But in this project, I wanted to let more people experience what it feels like to "control a puppet" in an easy and fun way. So I turned it into a digital version that you can control remotely with hand gestures. All you have to do is raise your hand, and the puppet will dance and move along with it.

For the tech part, I used the HandPose model from ml5.js for hand recognition. It can find 21 key points on a hand. I only used the five fingertips of the hand, treating them like the five "strings" controlling the puppet.

Also, I ran into a little problem at first: the puppet's movements were too sensitive. It just followed the hand mechanically, which looked strange. To make the motion more natural, I added this line of code: 56

let pull = constrain((targetBase.y - tip.y) / 200, -0.6, 0.6) * 0.35;

In simple terms, it simulates how tight or loose the string is pulled, based on how high the fingertip is. This gives the puppet's limbs a kind of "being-pulled" feeling, which looks much more realistic.

And that's actually why I chose to code this instead of using animation software. If I used software, the movements would be pre-set, like I’m playing an animation. But with code, the puppet can responds to your own movements in real time. It’s not performing for you; it’s waiting for you to control it. It's a kind of live performance that you complete together with the puppet.

This whole process made me realize: code isn't just a tool here—it actually changes what the artwork is. It shifts it from something you watch to something you do, from presentation to interaction.