Wang develops font recognition system for Adobe

ECE News

Ashish Valentine, ECE ILLINOIS
12/3/2015

Story Highlights

  • Grad student Atlas Wang has developed a platform called DeepFont for Adobe products, which can scan pictures and determine the fonts of text inside them.
  • Wang developed the product as part of a team working at Adobe in the summer of 2014, and continued his research afterward at Illinois. It has recently been shipped with the latest editions of Adobe's Photoshop and Typekit, and is already popular with the design and academic communities.
  • The software works using a deep learning algorithm, which is designed to learn to accomplish tasks like the human brain does, continually learning and refines its own recognition skills.

A designer working for a boutique ad firm is stumped, trying to decide on the perfect font to complement her new banner. Ruminating on the problem on her bus ride home, she suddenly spies a concert poster on the street with featuring just the font.

The bus stops for a few seconds to let out a passenger, and she whips out her phone, snapping a photo of the poster. An app immediately responds with the name of the font, and she works it into her ad that very evening.

Wang (8th from the left) at an Adobe summer outing
Wang (8th from the left) at an Adobe summer outing
This software, described by Adobe principal scientist Hailin Jin at the company’s MAX conference as “Shazam for fonts,” is DeepFont. What’s more: this isn’t a hypothetical technology - it’s already been shipped with Adobe’s latest editions of Photoshop and Typeset.

DeepFont has been grad student Zhangyang “Atlas” Wang’s project since he worked an internship with Adobe Research in 2014, and he’s proud to finally have it shipped with the company’s flagship products.

Screen captures from DeepFont, identifying the font that handwritten text comes closest to. The user photographs text, then the app analyzes it and proposes a list of fonts that possibly match the text.
Screen captures from DeepFont, identifying the font that handwritten text comes closest to. The user photographs text, then the app analyzes it and proposes a list of fonts that possibly match the text.
Before its release, a demo of the software was presented to an audience of design professionals at the MAX conference to roaring applause. Since then, DeepFont has amassed several endorsements from designers and firms around the world, just a few of which are visible on its rapidly expanding hashtag on Twitter.

Font recognition is a huge need for designers, who traditionally rely upon professionals who charge high rates and take an average of 45 minutes to an hour to reliably determine fonts.

“I’m happy to resolve a need that the design community has been feeling for so long,” Wang said. “There’s definitely a sense of accomplishment when people like what you’ve developed, that’s a great feeling.”

The software itself works using a new type of machine learning called deep learning, which aims to create algorithms that mimic the human brain by continuously learning, recognizing patterns, and improving their performance.

Atlas Wang
Atlas Wang
Wang and his team started by building a database of real-world images with text, and teaching their algorithm the basics of distinguishing fonts. They designed the algorithm to work from this training by feeding it new examples and allowing it to recognize patterns. It will continuously refine its recognition abilities.

In a recently released paper, Wang’s team also demonstrated its ability to account for text being photographed at an angle, or being partially covered with shadows. The algorithm is an add-on to Adobe products like Photoshop, and is small enough in size to work on mobile phones.

“The chance to work with Adobe on DeepFont was incredible, it was definitely one of the best internship experiences I’ve had,” Wang said. “Besides the game-changing nature of what we were developing, the people at Adobe were always very friendly and passionate about their work. The teams were small and specialized enough that I could have regular lunch meetings with the development head of Photoshop. She could tell me exactly how she wanted DeepFont to look and feel, and I could implement her specifications the same day.”

With DeepFont released, Wang hopes for even greater adoption of his font recognition technology, and eventually to finish his education and pursue a career in academia. His adviser, Professor Emeritus and Research Professor Thomas S Huang, is impressed with his performance and research abilities, especially his ability to balance developing DeepFont while simultaneously conducting other streams of research.

Thomas S. Huang
Thomas S. Huang
“Atlas is amazingly good at multitasking,” Huang said. “In addition to DeepFont, Atlas is also deeply involved in studying image aesthetics, such as how the brain evaluates images, and building computational models of the brain, and he’s also working on a project with Statistics professor Michelle Wang on using image recognition to determine the nutritional value of images of food. He is a brilliant young man, making many contributions to science and technology.”

Media Contact

Julia Sullivan

Assistant Director of Communications
1064 ECE Building
(217) 300-3731
juliams@illinois.edu

Todd Sweet

Director of Communications
1066 ECE Building
(217) 333-5943
tmsweet@illinois.edu