Researchers build frameworks that help machines learn how to learn

Ask a human being to count how many people are in a picture, and we automatically and effortlessly start to pick out the silhouettes of humans.

Ask a machine to do this, and it’s a different story—the machine must draw on a set of pre-coded computer vision routines to survey the entire image and extract only the needed information, which is not always an easy or seamless process.

With a $500,000 grant from the National Science Foundation, Assistant Professor Maxim Raginsky and Professor Pierre Moulin will work to understand the theoretical framework of the algorithms that machines use to extract the most relevant data to complete a task, from image identification to processing video streams.

“Machines can be given a lot of data these days, but not all of it is going to be useful for the task at hand. They have to learn to extract the relevant bits,” said Raginsky, who, like Moulin, is affiliated with the Coordinated Science Laboratory. “We as humans constantly try things and learn through trial and error. We experiment, receive feedback, and that becomes an experience that changes us."

Their research wants to help them figure out: "How do we do that with machines? How do we get a machine to tell that this part of the raw pixels in the image is not as relevant for what you want as this other part?"

“The traditional approach to this was to hand-engineer the features the machine extracts from the image, but recently there has been a lot of interest in developing automated ways of discovering these features," Raginsky said. "In other words, we want to develop machines that learn to learn.”

Automated inference from images and other types of data is a big business—companies like Facebook and Google are employing what is called deep neural networks to automate feature extraction from images, helping improve the accuracy of tasks like image searches and picture tagging.

However, the state of the practice is far ahead of the state of the theory. To create these deep neural networks, practitioners have been employing “hacks” to get their desired results. Raginsky and Moulin plan to investigate the theory behind why the algorithms work, with the hope of providing more robust and new algorithms for future use.

“Even though the hacks have turned out to be incredibly successful, there is a chance that some resources may not have been used as efficiently as they could have,” Raginsky said. “If you want to use something, ultimately you want to understand why it works, and more importantly, you want to understand where it breaks down. You want to see all the ways it can be applied.”

The researchers will work to understand the problem of how machines learn to extract useful information from signals and how they can make efficient use of that information across multiple tasks.

“With deep neural networks, machines can extract information that is universally helpful across a broad range of tasks," Raginsky said. "We’re interested in understanding the limits of when the same information would be used for several seemingly different learning tasks."

With this greater theoretical basis, the team hopes to build and test stronger algorithms.

“We hope that we can improve our understanding of the process and explain why it works as opposed to just getting results,” he said. “If we understand the more general theoretical framework behind it, that gives us a recipe for devising new algorithms and using them in unexplored settings.”