Ziploc bag, blue dye, water: the “$2 touchscreen” | Touchscreen Monitors

At the 2006 TED Talks, technologist Jeff Han made a lot of jaws drop with his demonstration of a multitouch touchscreen, and the implications it could have for computing. "This is the end of the point-and-click interface," the blogosphere seemed to say, and the recent announcement of the multitouch-based Microsoft Surface computer proved that, if nothing else, the software giant that taught most of the world how to use a mouse might be getting serious about touch as tomorrow’s preferred interface method.

But it was an unassuming two-minute video, posted on the blog of a Menlo Park, Calif.-based software company called Medallia, that truly made our jaws hit the floor. Filmed and posted by Erling Ellingsen, the video shows a Ziploc bag being filled with water that has been dyed blue.

Using a simple webcam mounted under the table, the system is able to recognize different colors in the dyed liquid — at multiple points on the "screen."

By placing the bag atop a clear tabletop with a webcam mounted underneath, Ellingsen is able to control a Mac computer — surfing the Web, playing chess, panning and zooming in a Google Earth-like app — by touching multiple points on the bag. The webcam recognizes different colors in the bag — blue is areas where nothing is being touched, while dark blue indicates that there is something behind it (presumably a hand) casting a shadow. An absence of blue from any given point indicates that a touch is being made there.

Ellingsen followed up the post with a bit of explanation on how the low-tech device works:

- The main idea is that you threshold the image into three areas: background (light blue), fingers (dark blue; these are shown as an overlay on-screen) and pressure points (not blue).

- I used a bag of dye for now, since that was easy to make. It might be feasible to tape LEDs to the edges of the table, and use FTIR-like scattering; I'd like to try that later. Actually, if you have one of those cheesy engraved-perspex-plate-with-blue-LEDs-in-the-base things lying around, you might be able to use that.

The phantom of the user's hand can be seen hovering over this Google page. A "click" results from a finger pressing down on the bag, dispersing all blue color from that point.
- Large areas of non-blue are interpreted as fingers. There is a mouse mode, where every touch immediately moves the mouse to that point, and a multi-touch mode which sends an NSNotification with a list of points for each frame. These will of course only be understood by programs that understand this protocol -- of which there currently exist only one (the rotozoomer at the end).

- The on-screen display is just a regular transparent OSX window. The background pixels are 100% transparent (alpha=0), and the hands show up as black with alpha 0.1 or so.

(Watch the full video demonstration.)

Is anybody going to use a Ziploc bag full of blue water to control any sort of real-world application? No. Of course not.

But is this a reminder that true ingenuity sometimes requires throwing all preconceived ideas out the window and allowing creativity room to breathe? Oh, yes, it most certainly is.