Smart cameras help us care for our homes, pets and loved ones, but what’s looking out for our privacy beneath the gaze of so many artificial eyes?
It’s an increasingly important question, especially as advances in cloud computing and machine learning make it easier than ever for people—possibly with ill intentions—to track our every move.
“There’s a huge amount of data being uploaded to the cloud,” says Jingjie Li, a second-year electrical and computer engineering PhD student at the University of Wisconsin-Madison. “We want to ensure that private data can be protected at the user end instead of being uploaded.”
The Smart Cities team. Submitted photo.
That’s why Li and fellow graduate students Setareh Behroozi, Tianen Chen, Di Wu and Zongshen Wu, all in the electrical and computer engineering department, developed a concept called Nightshift, which would empower smart cameras to preserve people’s anonymity.
“This has been a great opportunity to engage with a product that really needs my knowledge,” says Behroozi, whose research focuses on low-power design, which will be crucial for smart cameras to perform advanced computations with limited battery resources.
Nightshift was one of the winners in the final round of Foxconn Technology Group’s inaugural “Smart Cities-Smart Futures” competition, earning the students a $5,000 cash prize, for a total of $7,000 in awards over the duration of the entire competition. (Since the contest’s start in 2018, final round winners have advanced through first and second rounds, where the prizes were $500 and $1,500, respectively.)
Each round of the competition pushed the students to showcase their technical knowledge, as well as develop their entrepreneurial abilities.
“One of the benefits of the competition was thinking outside the box to do market research and write a business plan,” says Chen. “We’re all technical people, so we didn’t have much startup experience.”
They’ll use the prize as seed money to push forward with algorithm development for their product.
Final round winners of the Foxconn Smart Cities: Smart Futures Competition Setareh Behroozi (left), Jingjie Li (middle) and Zongshen Wu (right) developed a technology that enables privacy protections for smart cameras. Photo: Sam Million-Weaver.
Nightshift will deploy machine learning algorithms to make cameras smart—or rather, smart enough to recognize when they need to turn a blind eye.
“Cameras have different privacy requirements depending on the context of the environment,” says Li.
For example, cameras monitoring a college classroom late at night might take advantage of facial recognition to identify intruders—whereas during the day, while class is in session, students have a reasonable expectation that their faces won’t be recorded and uploaded to a server without their consent.
“We are thinking about how to preserve privacy at the edge while allowing algorithms to do analysis,” says Li.
Privacy at the edge is what sets Nightshift apart. Instead of asking people to trust that their private information will be safe in remote servers (which have been shown to be all-too vulnerable to leaks and hacks), Nightshift adds privacy protections before data reaches the cloud.
To help protect privacy, Nightshift enables cameras to capture useful information yet also automatically scrub identifying details from footage of people going about their daily lives.
In practice, that means separating faces from their identifiable features.
Smart cameras recognize faces based on measurements called biomarkers, which can include such parameters as the distance between someone’s eyes, the angles at the corners of their mouth or the furrows in their forehead.
Nightshift would allow cameras to quantify those biomarkers, but disconnect the measurements from a person’s face before uploading data for analysis. Currently, the students are testing prototypes and optimizing algorithms to separate the two.
“We plan to hide the identities of faces by transporting only certain biomarkers to the cloud,” says Li.
Although the product is separate from their academic research, the students have taken full advantage of the ecosystem of experts in the Department of Electrical and Computer Engineering at UW-Madison. Among them are some of their faculty advisors, Assistant Professor Younghyun Kim and Professor Parmesh Ramanathan, who are trailblazers in the fields of secure computing and ultrafast networks, as well as Assistant Professor Kassem Fawaz, an expert in privacy.
But the students are also taking a big-picture view of the smart city concept, seeking input outside of the traditional engineering realm.
“Smart cities aren’t only about technology,” says Li. “We need to engage with residents in the surrounding environment. It’s cross-disciplinary research engaging engineering, computer science and even the humanities for user studies.”