Hand Tracking Water Canvas - Reflection

In this updated version of SenseSpace, we experimented with a real-time hand and pen tracking system that lets participants draw directly on a “water canvas” inside the browser. The effect works by capturing hand movements through a webcam, processing them with computer vision, and then mapping that motion onto a web interface that simulates animated water ripples and ink-like strokes. The result is an immersive interaction where gestures feel like they’re leaving traces on the surface of a digital pond.

Why This New Effect?

When we were discussing how to move the project forward, we split into two directions:

• Tactility & Environment (covered by my teammate)
• Projections and Computer vision (my focus)

My part was about finding new visual interactions that still connected back to nature and calmness. The inspiration came from simple but satisfying moments with water, like writing with a stick on the thin water layer over sand at a beach or the feeling of dragging your finger across a pool surface and seeing patterns ripple out.

Both of us realized that the refreshing feeling didn’t come from anything complex or “high-tech.” It came from this surprising little challenge-play element in water: the resistance and fluidity combined, where you can see both the water and the sand beneath. It’s a space where control and randomness meet and that made the experience relaxing and engaging. Since we can’t bring actual water into our installation (too much complexity and maintenance), we thought: why not simulate it digitally as part of the projection? That way we preserve the biophilic vibe without adding a literal water element.

How It Works in the Experience

• Visitors enter the setup and notice two beanbags facing the projections.
• In front of them, there’s a small “pond-like” digital canvas (think aquarium-style surface).
• They can pick up a stick (or use their hand/pen in front of the camera) and start drawing.
• As they draw, their marks ripple across the pond and get projected onto a larger canvas or side wall, expanding their personal sketch into a bigger visual pattern.

The whole point is not to make “perfect drawings” but to give people a way to play with water-like surfaces in a calm, controlled way, making spontaneous shapes and patterns.

Why It Fits

This effect works because:

• It draws directly from real sensory experiences with water and sand.
• It avoids irrelevant, artificial visuals (like random AI-generated patterns).
• It stays aligned with our goal of building a biophilic, calming environment.
• It provides a natural sense of control and satisfaction while still keeping the interaction playful and surprising.