Walking through the world, we leave a trail of bits and bytes in our wake. People generate an astonishing amount of data every day — by some estimates that number will reach 44 zettabytes by 2020, a figure so large it doesn’t even seem plausible. Tweets, emails, interactions with our voice assistants; heck, simply owning a phone and turning it on can paint an unnervingly accurate picture of how we spend our time.
For all the hand wringing around big data and its problematic privacy implications, the truth is, the average person doesn’t fully understand what kind of data their online activities generate and how that information is being used — and monetized. We’re all guilty of this apathy; we scroll through binding user agreements, quickly signing away all rights to our privacy. We opt into location sharing that’s later sold to data brokers looking to further optimize the targeted ads we’re served.
It’s not entirely our fault, says Norman Sadeh, a professor at Carnegie Mellon University who researches personal privacy. In fact, privacy shortcomings are often a fundamental design flaw that he and a number of designers and engineers believe can be fixed by introducing yet another piece of technology: a personal privacy assistant.
“By themselves, people are not going to engage with these settings; they’re not going to read privacy policies,” he says. “So what can we do to read privacy policies on their behalf and tell them about the things they care about?”
A graphic illustration of a patient using a medication dispensing machine that verifies the patient’s identity through facial recognition. Image credit Artefact.Nudge nudge
Sadeh leads the Personal Privacy Assistant project at CMU, for which he and his team of researchers have been exploring how an intelligent digital assistant could step in to help manage the growing amount of data people produce. The app they created is like a digital maid-meets-teacher that aims to clean up privacy settings and prompts people to think more critically about how their personal data is being used.
Right now, the app for mobile phone privacy settings is only available on jailbroken Android phones, but Sadeh’s team recently released an app for iOS and Android that applies the same logic to the Internet of Things. Both apps are based around the idea of “privacy nudging,” or gentle reminders that encourage people to look twice at their settings to see if they want to change anything.
The apps are built on machine learning algorithms that have studied the privacy preferences of individuals and determined what they might consider a problematic setting based on past behaviors. Sadeh describes the nudges, which come in the form of notifications, as distant relatives of Clippy, the paperclip Microsoft once used to dole out advice to people using its writing software.
“It was a great idea from a conceptual standpoint, but it would only tell you about things that you already knew,” he says. “These assistants have to be very smart about being helpful without being annoying.” The trick is to use machine learning to make informed predictions about what any given person might want more information about regarding their privacy.
“We found that by asking you three to five questions enables us to predict about 80 percent of the privacy settings you have,” Sadeh explains. Those are questions as simple as, “How comfortable are you with sharing your location?” or “How comfortable are you sharing access to your camera?”
Screenshots from Carnegie Mellon University’s Privacy Assistant app which asks the user screening questions to benchmark their comfort with degrees of data sharing, then provides personalized privacy settings recommendations. Image credit CMU.Earning trust
The irony of entrusting an app to advocate on behalf of people that are already skeptical of technology’s role in the erosion of their personal privacy isn’t lost on John Rousseau, executive creative director at Seattle design studio Artefact. He and his team recently released a new project called Kagi, a conceptual app that’s based on many of the same ideas behind Sadeh’s Personal Privacy Assistant.
“I think there’s an argument to be made that something like this could function more like a public utility and less like the for-profit products that we know today,” Rousseau says. In other words, Facebook or Google can’t be in charge of making sure your data is protected.
Artefact describes Kagi as an intermediary between the platforms that collect your data and the services that want to use it. Like the Personal Privacy Assistant, Kagi would work in the background, quietly monitoring app activity to make sure no questionable decisions are being made without the user’s consent. Getting people to trust an app requires that a personal assistant like Kagi earns their trust. Rousseau likens it to the process of getting used to autonomous driving features. Most people won’t automatically hand over complete trust to a vehicle; it’s only after a slow onboarding process and repeated usage that trust is given.
“We’ll have to design all kinds of different moments where an artificial agent is essentially building trust by virtue of proving its capability and proving its efficacy,” Rousseau says.
A hint of what the Kagi interface could look like. Motion graphic credit Artefact.For Sadeh, trust comes down to control. Machine learning can reduce the burden for someone who might not have the time or inclination to keep up with new privacy policies, but ultimately control must rest with the people using the app.
“The idea that you’re going to rely on machine learning to make all your privacy decisions on your behalf is incompatible with the very notion of privacy,” he says. “You have to configure this technology in such a way that consumers retain control.”
Kagi takes this idea a step further by imagining a world in which people have control over not just what data is collected, but what they do with it.
“When people protect their personal data from automatic access, we expect new services to emerge that allow them to monetize or extract value from their data in new ways,” Artefact writes. This far-off vision centers around the idea that people should own their data, and that they should be able to leverage that information in a marketplace to either make money or give it away. It’s a bluesky idea that Sadeh, and even Rousseau, admits isn’t realistic given the current regulations around data.
“We’re making an assumption about this future, which is that the incentive structures are better aligned to the needs of all stakeholders,” Rousseau says. “We’re assuming that the rules have changed.”