Marc Levoy : 2011 Plenary Session


Tuesday, April 12, 2011
Location: Fisher Conference Center, Arrillaga Alumni Center

"Computational Photography and the Stanford Frankencamera "
9:15am - 9:45am


Computational photography refers broadly to sensing strategies and algorithmic techniques that enhance or extend the capabilities of digital photography. Representative techniques include high dynamic range imaging, flash-noflash imaging, coded aperture and coded exposure imaging, photography under structured illumination, multi-perspective and panoramic stitching, digital photomontage, all-focus imaging, and light field imaging.

Although interest in computational photography has been increasing, progress has been retarded by the lack of a portable, programmable camera with enough image quality and computing power for everyday photography. To address this problem, we have designed an open architecture for such cameras. Our architecture consists of a hardware specification, a software stack based on Linux, and an API with bindings for C++. Our architecture permits control and synchronization of the sensor and image processing pipeline at the microsecond time scale, as well as the ability to incorporate and synchronize external hardware like lenses and flashes. We have also built two reference implementations: a Nokia N900 smartphone with a modified software stack, and a camera of our own design that accommodates SLR lenses and SLR-quality sensors. The latter is big and ugly - hence the name Frankencamera.

In this talk, I'll briefly survey the field of computational photography, I'll enumerate the characteristics we believe any open-source camera should have, and I'll describe our architecture, API, reference implementations, and first round of applications. Our immediate goal is to distribute this platform to computational photography researchers and students worldwide. Our long-term goal is to spur the growth of an open-source camera community, leading eventually to commercial cameras that accept plugins and apps.


Marc Levoy is a Professor of Computer Science and (jointly) Electrical Engineering at Stanford University. He received a Bachelor's and Master's in Architecture from Cornell University in 1976 and 1978, and a PhD in Computer Science from the University of North Carolina at Chapel Hill in 1989. In the 1970's Levoy worked on computer animation, developing a cartoon animation system that was used by Hanna-Barbera Productions to make The Flintstones, Scooby Doo, and other shows. In the 1980's Levoy worked on volume rendering, a technique for displaying three-dimensional functions such as computed tomography (CT) or magnetic resonance (MR) data. In the 1990's he worked on 3D laser scanning, culminating in the Digital Michelangelo Project, in which he and his students spent a year in Italy digitizing the statues of Michelangelo. Outside of academia, Levoy co-designed the Google book scanner and launched Google's Street View project. His current interests include light fields. optical microscopy, and computational photography - meaning computational imaging techniques that extend the capabilities of digital photography.

Awards: Charles Goodwin Sands Medal for best undergraduate thesis (1976), National Science Foundation Presidential Young Investigator (1991), ACM SIGGRAPH Computer Graphics Achievement Award (1996), ACM Fellow (2007).