Sketch practically anywhere with SPARK

Jeff Browne demonstrates SPARK, a system that uses computer vision to capture and interact with physical sketches on whiteboards.

Part of my PhD dissertation project at UC Santa Barbara. Jeff Browne: cs.ucsb.edu/~jbrowne Tim Sherwood: cs.ucsb.edu/~sherwood When faced with complex design, analysis, or engineering tasks, novices and professionals alike attempt to better understand problems through diagrams, and the natural first step in this process is working on a whiteboard. Through their drawings, people can gain valuable insights into subtleties of design and analysis tasks, but once a diagram gains sufficient complexity, further progress becomes tedious (or even intractable) without the aid of a computer. The goal of this work is to utilize popular consumer hardware (webcams, smartphones, and projectors when available) to enable sketch recognition where people are already drawing: whiteboards, chalkboards, and even on loose paper. Our system enables a person to interact with her real world drawings by recognizing meaning from images of hand drawn diagrams that are captured via a smartphone or a webcam, and by providing an interface for interacting with that meaning through augmenting projectors and/or the phone's display. In service of this goal, we make contributions to the three operational phases of the system's architecture: capturing users' marks, recognizing the drawn diagrams, and enabling interaction with the recognized structures. MyScript equation recognition is provided courtesy of VisionObjects (visionobjects.com).