Zongfu Yu hopes to enhance machine vision by creating light detectors that process much more information than human eyes can comprehend.
That plan has earned him a National Science Foundation CAREER award, providing $500,000 of support over five years.
Zongfu Yu
“It’s difficult to see now, but, down the road, we could have new ways to visualize the world,” says Yu, the Dugald C. Jackson assistant professor of electrical and computer engineering at the University of Wisconsin-Madison.
Such enhanced light detectors could greatly improve automated navigation systems or other applications where algorithms need to make decisions based on visual information. Boosting the vision of machine eyes, however, isn’t as simple as modifying existing cameras.
Although cameras have improved substantially over the years, with innovative new lenses that can render images with breathtaking clarity, individual photographic pixels still only contain information about light intensity.
Cameras contain photodetectors that record the intensity of incoming light—and whether that information is captured on film or by a charge-coupled device, the systems were designed to create pleasing images for human eyes.
“Yet cameras today are more often used by machines than people,” says Yu.
Human vision merely perceives color and brightness, whereas a computer algorithm could intuit information about the world based on other details of an incoming light beam that are imperceptible to human eyes.
For example, computers could use the angle at which light encounters a detector, or another property called the phase, to infer how far away an object is.
The researchers have already created nanoscale structures composed of pairs of tiny wires capable of detecting the angle of incoming light. In addition to the photodetectors, Yu’s group now is building new computational tools to this unique image information.
“The information from these new photodetectors is rather unconventional, so you cannot use old algorithms,” says Yu.
To develop their new algorithms, Yu and his students are training neural networks to work with the detectors. Developing new photodetector hardware and software at the same time represents a substantial shift from historical approaches.
“We’re rethinking how the pixels of cameras should be built,” says Yu.
Although scaling up the production of one or two proof-of-concept nanoscale photodetectors could be technically challenging, Yu envisions collaborating with industry when it comes time to mass-produce the devices. At this point, the project’s focus is to shine new light onto what types of information can be detected.
The scientists are working with the Wisconsin Alumni Research Foundation to patent the new photodetectors.