Illinois-Princeton team takes first place in robot vision competition

An Illinois-Princeton team won first place in the Software League for the Association for the Advancement of Artificial Intelligence 2007 Semantic Robot Vision Challenge (SRVC) competition on July 26. Team members include ECE graduate students Juan Carlos Niebles and Jia Li; Rahul Mehta, ECE undergraduate; Fei-Fei Li, former ECE assistant professor who is now at Princeton University; and Princeton students Richard Socher and Brendan Collins.

The SRVC, held from July 22 to July 25, is a new research competition that began this year. It is “designed to push the state-of-the-art in image understanding and automatic acquisition of knowledge from large unstructured databases of images,” according to the SRVC Web site.

“The competition was about developing computer algorithms that can learn visual models of objects from images that are obtained from the Web,” said Niebles. “The algorithms had to be able to recognize such objects in the images taken by the organizer's robot.”

Jia Li said research to improve the technology in the field of computer vision could eventually produce general object recognition software that may play a vital role in helping visually impaired people in everyday tasks.

As a part of the Mobile Robot Competition and Exhibition at the 22nd National Conference on Artificial Intelligence in Vancouver, British Columbia, Canada, the competition was divided into two separate categories: Software League and Robot League.

“[For the Software] League, the teams are provided a textual list of objects which were unknown until the start of the competition. Then each team was given 30 minutes in what was called the ‘Internet search phase’,” said Niebles. “During this time, each team's software had to autonomously search the Internet for images related to the objects in the list. Thus, the algorithms had to build a database of what the objects look like using the images from the Web.”

Mehta said the team split its code into three parts; one for downloading branded objects, such as ‘Dell Monitor,’ general objects such as ‘eyeglasses,’ and CDs, posters, and books.

The six teams that qualified to compete then placed their camera-equipped robot into an arena filled with the 20 objects on the list—everything from household items like an iron, Tide detergent, and a vacuum, to food, books, and DVDs. Each robot had 15 minutes to navigate the space to collect images of the items on the floor and furniture. During the ‘environment search phase,’ each team’s software had to analyze the images taken by the robot and differentiate between the objects that were provided on the list, from those that were simply ‘distractor’ objects, Niebles said. The Illinois-Princeton team was able to identify seven objects with its robot and software.  

Scoring was based on the “quality of fit and the correctness of the bounding box over the object in the image.” After the event, participants were required to attend a workshop to discuss results and share details about their software and approach, something Niebles said he found very helpful and a highlight of the competition.

The Illinois-Princeton team won first-place in its league and also received the highest score in the entire competition. The team was the only one to find a ‘general,’ unbranded, object without the use of template matching techniques, said Fei-Fei Li.    

The team was well prepared for the event. At the VisionLab, which transferred from Illinois to Princeton with Fei-Fei Li’s move, researchers are interested in computer vision problems that involve semantic interpretation of images and video sequences.

“Our lab has developed algorithms that can categorize scenes in images like ‘beach,’ ‘mountain,’ ‘office,’ as well as algorithms that can learn and recognize human actions, such as walking, running, and hand waving, in video sequences,” said Niebles. “Thus, we found this competition a great opportunity to apply some of the computer vision techniques we have developed at the lab, as well as techniques that have been developed in other research groups that we have had a chance to experiment with. Without doubt, our research experience in computer vision problems was a big plus for us during the competition.”

The team applied a broad range of electrical engineering techniques, including image processing, computer vision, pattern recognition, machine learning, and object recognition, Niebles said.

“I got to learn about all the tools and development that goes on in computer vision,” said Mehta. “It was a great opportunity to learn more about image processing and electrical engineering in general, outside of class.”

In fact, the team would consider participating again if given the opportunity. “We hope the event will continue to take place, with more challenging tasks, such that researchers will continue to find ideas and methods that help to improve the state-of-the-art in computer vision technology,” Niebles said.

Semantic Robot Challenge Web site: http://semantic-robot-vision-challenge.org/
Illinois-Princeton team Web site: http://vision.cs.princeton.edu/srvc/
VisionLab Web site: http://visionlab.ece.illinois.edu/datasets.html