An interdisciplinary team of researchers at Northeastern have built a device that can recognize “millions of colors” using new artificial intelligence techniques—a massive step, they say, in the field of machine vision, a highly specialized space with broad applications for a range of technologies. 

The machine, which researchers call “A-Eye,” is capable of analyzing and processing color far more accurately than existing machines, according to a paper detailing the research published in Materials Today. The ability of machines to detect, or “see,” color is an increasingly important feature as industry and society more broadly becomes more automated, says Swastik Kar, associate professor of physics at Northeastern and co-author of the research.

“In the world of automation, shapes and colors are the most commonly used items by which a machine can recognize objects,” Kar says.

The breakthrough is twofold. Researchers were able to engineer two-dimensional material whose special quantum properties, when built into an optical window used to let light into the machine, can process a rich diversity of color with “very high accuracy”—something practitioners in the field haven’t been able to achieve before. 

Additionally, A-Eye is able to “accurately recognize and reproduce ‘seen’ colors with zero deviation from their original spectra” thanks, also, to the machine-learning algorithms developed by a team of AI researchers, helmed by Sarah Ostadabbas, an assistant professor of electrical and computer engineering at Northeastern. The project is a result of unique collaboration between Northeastern’s quantum materials and Augmented Cognition labs. 

The essence of the technological discovery centers on the quantum and optical properties of the class of material, called transition metal dichalcogenides. Researchers have long hailed the unique materials as having “virtually unlimited potential,” with many “electronic, optoelectronic, sensing and energy storage applications.” 

“This is about what happens to light when it passes through quantum matter,” Kar says. “When we grow these materials on a certain surface, and then allow light to pass through that, what comes out of this other end, when it falls on a sensor, is an electrical signal which then [Ostadabbas’s] group can treat as data.”

As it relates to machine vision, there are numerous industrial applications for this research tied to, among other things, autonomous vehicles, agricultural sorting and remote satellite imaging, Kar says. 

“Color is used as one of the principle components in recognizing ‘good’ from ‘bad,’ ‘go’ from ‘no-go,’ so there’s a huge implication here for a variety of industrial uses,” Kar says. 

Machines typically recognize color by breaking it down, using conventional RGB (red, green, blue) filters, into its constituent components, then use that information to essentially guess at, and reproduce, the original color. When you point a digital camera at a colored object and take a photo, the light from that object flows through a set of detectors with filters in front of them that differentiate the light into those primary RGB colors. 

You can think about these color filters as funnels that channel the visual information or data into separate boxes, which then assign “artificial numbers to natural colors,” Kar says. 

“So if you’re just breaking it down into three components [red, green, blue], there are some limitations,” Kar says. 

Instead of using filters, Kar and his team used “transmissive windows” made of the unique  two-dimensional material. 

“We are making a machine recognize color in a very different way,” Kar says. “Instead of breaking it down into its principal red, green and blue components, when a colored light appears, say, on a detector, instead of just seeking those components, we are using the entire spectral information. And on top of that, we are using some techniques to modify and encode them, and store them in different ways. So it provides us with a set of numbers that help us recognize the original color much more uniquely than the conventional way.”

As the light pass through these windows, the machine processes the color as data; built into it are machine learning models that look for patterns in order to better identify the corresponding colors the device analyzes, Ostadabbas says. 

“A-Eye can continuously improve color estimation by adding any corrected guesses to its training database,” the researchers wrote. 

Davoud Hejazi, a Northeastern physics Ph.D. student, contributed to research.

For media inquiries, please contact media@northeastern.edu.