Seventy years ago, when Claude Shannon developed a new area of mathematics—information theory—to analyze data compression and communication, people could not have anticipated it would have an impact on topics such as machine learning or economics.
Varun Jog
But today, that’s exactly what’s happening; the mathematics behind information theory has developed so much that it now crosses lines into many other disciplines.
And that’s exactly what Varun Jog, an assistant professor of electrical and computer engineering at the University of Wisconsin-Madison, hopes to explore. Recipient of a 2020 National Science Foundation CAREER Award, Jog hopes to use the five-year grant to help connect information theory with other topics in mathematics.
The first part of his research, he says, is to focus on the similarities between information theory and geometry. “There happen to be a lot of analogues of geometry problems on the information theory side,” he says.
In particular, he hopes to better understand the concept of entropy—which captures the level of uncertainty in a system—by analogy with concepts such as volume and surface area in geometry.
He also plans to carry that cross-pollination a bit further by applying tools from both disciplines to one another to establish new information theoretic and geometric inequalities. “I want to level up new mathematical tools to study problems in information theory and geometry,” Jog says. “I think there might be some ideas I can borrow from geometry to prove things in information theory and vice versa.”
The third thrust of his research is to use some of those mathematical tools to probe machine learning. In particular, he wants to examine deep-learning neural networks. “Deep learning is popular and works really well, but it’s not super clear why,” he says. “There’s not a lot of theory backing it.”
To look under the hood of deep learning, he plans to apply some of the mathematical tools from information theory as well as those derived from optimal transport theory, a complex branch of math developed in economics to figure out the best way to transport goods from one place to another at minimal cost.
While most of the work funded by Jog’s CAREER award is theoretical, he says there are real-world applications for his work. For instance, he’s collaborating with faculty across the College of Engineering and at the UW School of Medicine and Public Health on a project called Machine Learning for Medical Imaging, which combines UW-Madison’s medical expertise, engineering skill and cutting-edge machine-learning theory to improve on medical diagnoses.
He’s also working with colleagues at the UW-Madison Waisman Center to create machine learning algorithms to determine which patients are at a high risk of hearing loss before the problem reaches an advanced stage. “So, I work on very applied and very theoretical problems,” Jog says.
He is also engaged with the broader mathematics community via the Wisconsin Math, Engineering and Science Talent Search competition, where he helps in devising challenging problem-sets for high-school math enthusiasts in Wisconsin.
He’s glad, however, that the theoretical side of his work has received recognition. “The CAREER award is like feedback from the community that this is good work and we want you to keep it up,” he says. “That’s positive feedback that I appreciate.