Draw in the Air

Overview

Pinch your thumb and finger together and draw glowing neon in the air; hold an open palm to clear it. Your hand is the brush, the camera is the canvas, and the live picture of you sits beside the drawing so you can see what your hand is doing.

Type

Gesture experiment · web

Input

Camera, hand, a clap

Runs on

The browser, on device

Built with

Canvas, Web Audio, MediaPipe hand tracking

Making a pinch feel like a pen

• A pinch with a threshold either side. The pen goes down when thumb and index come within 0.32 of the palm's length and only lifts again past 0.48, so a shaky pinch never breaks the line.
• Steady when slow, snappy when fast. The fingertip is smoothed with an adaptive lerp - calm for careful strokes, quick for big ones.
• A cursor that shows the pinch. A hollow ring that tightens as you pinch and fills in while you draw.
• Clearing on purpose. An open palm has to be held - four fingers out, thumb spread - and a ring fills while it does, so waving at the camera never wipes the drawing.
• Colours by dwelling. Hold your fingertip over a colour on the palette to pick it up.

A clap for the sky

Clap and the sky behind the drawing switches between morning and night, with stars. A clap is told apart from talking by its shape: a sudden, loud, bright burst with lots of energy above about 1.5 kHz, where speech sits mostly low.

Keys for the rest: C clears, S saves the drawing, G greets, H hides the interface.