2/28/2025 Cassandra Smith
Written by Cassandra Smith
Two teams of Electrical and Computer Engineering researchers were recognized with awards from the IEEE Computer Society.
Professor Rakesh Kumar said he is proud of his student, David Chen, for winning an IEEE Micro Top Pick for a paper in the first semester of his graduate career. The paper covers wafer-scale integration techniques for building large-scale network switches. These techniques could lead to dramatic cost savings and performance improvements in data centers.
“Today’s data centers have huge amounts of servers, GPUs, CPUs, devices you need to connect together,” said Chen. He continued to say sometimes as many as 10,000 or even 100,000. There are a couple of ways to connect them. Sometimes they are connected using lots of cables. Not only are those cables expensive---think around $4,000 each, but the connectivity can be unreliable.
So, what if you ditch the cables and take the indirect route using switches? Yeah, you’ve lowered the cost but now you risk your computers jumping through several hoops which could take a lot of time and time is money, as the saying goes.
Chen is answering the question: How can I build a switch that can connect an extremely large number of computers? That answer is.... make a bigger switch! Kumar said David’s idea was to create that larger switch, which would only require a small number of switches to connect all of the computers. That would save money. It would also allow for a fewer number of hoops that need to be jumped through which could lead to higher performance.
Today’s switches are very small—think chip-sized. Chen expanded that switch by using wafer-scale integration. “This new technology essentially allows you to build a chip that’s much bigger,” said Kumar. “So, David has been working with this technology that allows you to build chips that are at least 50 times bigger than today’s biggest chips, which means that he can build a switch that can connect 30 to 50 times more the number of computers. More than the conventional switches.”
Chen said he knew this work was going to mean something for the industry. “It was not surprising to me that it got recognition, but I didn’t expect it to be Micro Top Pick! I was so happy when I heard the news!”
Kumar said he was proud of Chen and that he expected great things from him in the future.
Neural Processing Unit Virtualization
Another awarded paper came from the team of ECE professor Jian Huang. It was on neural processing unit (NPU) virtualization. NPUs were deployed in cloud platforms to meet the increasing demand for machine learning services. The Ph.D. student Yuqi Xue from Huang’s lab observed that NPUs deployed in the cloud have low resource utilization with a variety of popular deep neural network workloads. “We think we should utilize NPUs by virtualizing them. Based on virtualized NPUs, we can create many instances so they can share the physical devices.” said Huang. He continued to say that is how the hardware is best utilized. However, hardware architecture as well as system software do not support NPU virtualization, causing a barrier.
So, how do you solve such a dilemma? Huang and his Ph.D. student Yuqi Xue were working to modify hardware architecture and operating systems to enable NPU virtualization. “We discussed these problems and figured out how we can change the hardware architecture to support this, to support NPU virtualization and how we can change the operating system to support the virtualization.”
Huang said this work is gaining traction. “We think is it very important and maybe the industry is still considering whether to virtualize NPUs now, but we also believe they will have to do so in the future because that’s the way they can improve their utilization and simplify their hardware management and then lower their operating costs.” This is why he thinks their paper, “Hardware-Assisted Virtualization of Neural Processing Units for Cloud Platforms,” was among those awarded papers for IEEE Micro Top Picks.
Huang worked on this project with PhD student Yuqi Xue, who focuses on low-cost AI infrastructure as well as a collaborator from Google. Collaboration with Google made sure their design was practical for real data centers. Their team utilized Google’s education program to access TPU (Tensor Processing Unit) cloud services for conducting experiments. Huang said this collaboration with Google provided valuable insights into how to transcribe their work to real-world industry. They also used open data sets to make sure their research was transparent and reproducible.
Illinois Grainger Engineering Affiliations
Rakesh Kumar is a Professor and John Bardeen Faculty Scholar in the Electrical and Computer Engineering Department at the University of Illinois at Urbana Champaign with research and teaching interests in computer architecture and system-level design automation. He is also an affiliate of the Coordinated Science Laboratory.
Jian Huang is an Associate Professor and Y.T. Lo Faculty Fellow in the Electrical and Computer Engineering Department at the University of Illinois Urbana-Champaign. He is also an affiliate of the Coordinated Science Laboratory and the Siebel School of Computing and Data Science. His work focuses on computer systems and architecture. He is leading a team to build sustainable AI infrastructures and platforms to accelerate the AI-driven knowledge discovery and retrieval.