Software has come a long way from the idea of identifying letters and numbers that are drawn on a touchscreen, but it is helpless when confronted with sketches. Researchers from the University of Applied Sciences in Berlin, Germany, however have found a way to train software to recognize drawings of simple objects such as rabbits, pizzas, a telephone, or zebra.
The difficult task in recognizing these objects is that sketches and drawings may not resemble the look of the actual shape of the objects. "It might be that we only recognize it as a rabbit because we all grew up that way," said James Hays, assistant professor of computer science at Brown University, who developed the new program with Mathias Eitz and Marc Alexa from the Technical University in Berlin. "Whoever got the ball rolling on caricaturing rabbits like that, that’s just how we all draw them now."
To train their software, the scientists used crowdsourcing platform Mechanical Turk to acquire 20,000 sketches of objects in 250 categories that were fed to the software to enable a form of semantic understanding of sketches. As a result, their system was able to recognize a sketch correctly in 56 percent of cases, while humans recognized 73 percent of the drawings correctly.
"The gap between human and computational performance is not so big, not as big certainly as it is in other computer vision problems," Hays said.
Key to improving the software are more drawings, which are currently provided via an iPhone/iPad app that the researchers think could be turned into a game. "The game could ask you to sketch something and if another person is able to successfully recognize it, then we can say that must have been a decent enough sketch," Hays said. “You could collect all sorts of training data that way."
Stay on the Cutting Edge
Join the experts who read Tom's Hardware for the inside track on enthusiast PC tech news — and have for over 25 years. We'll send breaking news and in-depth reviews of CPUs, GPUs, AI, maker hardware and more straight to your inbox.
Now onto the real challenge, create a program that can read my handwriting. Won't bother talking about my drawings it's significantly worse, there is no hope for that.Reply
Beyond that, pretty neat tech that is coming out these days.
Need a program to recognize doctors writingReply
"The game could ask you to sketch something and if another person is able to successfully recognize it, then we can say that must have been a decent enough sketch,"Reply
So, I guess these guys have never heard of "Draw Something"?
Row 2 column 2 is a goatReply
Row 2 column 4 is an std
Row 3 column 2 is a manticore
Whoever got the ball rolling on caricaturing rabbits like that, that’s just how we all draw them now."Reply