Monday, 17 Sep, 2012 Technology

Latest Invention: System that Can Match Rough Drawings to Real Items


A group of researchers at Carnegie Mellon University are working on a system that makes it possible for computers to match reasonably good drawings made by the user with photos of the same things.

But researchers from Rhode Island's Brown University and the Technical University of Berlin managed to come up with software that examines users' crude, cartoony sketches, and attempts to understand their drawings.

In order to create the system, the team began with a database composed of 250 categories of annotated photographs. Afterwards, with the help of Amazon's Mechanical Turk crowd-sourcing service, researchers hired people to draw rough sketches of items from each of those categories.

The next step involved subjecting the 20,000 sketches to recognition and machine learning algorithms. Thus the system was taught what general type drawings could be attributed to which categories.

After examining a large number of sketches of a rabbit, for example, made by different people, the system was able to learn that combinations of certain shapes usually meant "rabbit".

The final stage was the development of an interface in which people draw pictures using a stylus and the system tried to figure out what they're drawing as users were drawing it.

Currently, as long as the item is part of one of the 250 categories, the system can correctly guess it about 56 percent of the time. Humans, on the other hand, managed to register a success rate of about 73 percent.

The system could be used in the future to help perform sketch-based internet searches. Thus, when you saw an object but did not know what it's called and does, but you remember its shape, then this technology could help you by analyzing a simple drawing.

But before that happens, researchers still have to complement the database with other categories. One of possible ways of doing that is by engaging people in playing an online game in which one player is challenged to draw a certain object, and other players have to guess what it is. A myriad of sketches would then be submitted to the database to improve the system.

[via Brown University]

Powered by

Add your comment:

antispam code