AI System Scores Better Than 75% Of Americans In Visual Intelligence Test

Northwestern University professors created a new computational model that performs as well or better than 75% of American adults in a standard intelligence test. The team believes this is an important step in making artificial intelligence (AI) that sees and understands the world as humans do.

“The model performs in the 75th percentile for American adults, making it better than average,” said Northwestern Engineering’s Ken Forbus. “The problems that are hard for people are also hard for the model, providing additional evidence that its operation is capturing some important properties of human cognition,” he added.

CogSketch AI Platform

The computation model was built on CogSketch, a sketch-understanding system developed in Forbus’ laboratory at Northwestern University. Sketching is a natural activity that people do while thinking or trying to communicate an idea, especially when spatial content is involved. Sketching is also heavily used in engineering and geoscience. CogSketch is used to model spatial understanding and reasoning, making it suitable for research based on sketches, but also for testing against a standardized visual intelligence test such as the Raven’s Progressive Matrices test.

The computational model developed by Forbus builds on the idea that analogical reasoning is at the heart of visual problem solving. In other words, if we want our AI systems to solve complex visual problems, they need to be able to make analogies and compare one object to another. Images are compared via structure mapping, which aligns common structures found in two images, in order to identify commonalities and differences. The structure-mapping theory was developed by psychology professor Dedre Gentner, who also works at Northwestern.

The Raven’s Progressive Matrices Test

The 60-item Raven’s Progressive Matrices test measures a person's reasoning ability by showing them a matrix of images with one missing image. The test-taker is then supposed to select which image is missing from a set of six to eight options.

Forbus and former colleague Andrew Lovett, who was a postdoctoral researcher in psychology at Northwestern, said their AI system did better in the test than the average American.

"The Raven’s test is the best existing predictor of what psychologists call ‘fluid intelligence, or the general ability to think abstractly, reason, identify patterns, solve problems, and discern relationships,’" said Lovett, now a researcher at the US Naval Research Laboratory. "Our results suggest that the ability to flexibly use relational representations, comparing and reinterpreting them, is important for fluid intelligence," he said.

The professors said that understanding sophisticated relational representations is key to higher-order cognition. It can help connect entities and ideas such as “the clock is above the door” or “pressure differences cause water to flow.”

Today’s AI systems are mainly good at recognizing objects, a task that not too long ago was also quite difficult for computers. However, the professors said that object identification without subsequent reasoning isn’t all that useful, which is why they subjected their AI system to the Raven's Progressive Matrices Test. They also believe their research is an important step towards gaining a better understanding of computer vision.