Anthropic’s ‘anonymous’ interviews cracked by professor with an LLM
A Northeastern University professor used a large language model to de-anonymize a subset of interviews conducted by Anthropic’s Interviewer tool.
In December, the artificial intelligence company Anthropic unveiled its newest tool, Interviewer, used in its initial implementation “to help understand people’s perspectives on AI,” according to a press release. As part of Interviewer’s launch, Anthropic publicly released 1,250 anonymized interviews conducted on the platform.
A proof-of-concept demonstration, however, conducted by Tianshi Li of the Khoury College of Computer Sciences at Northeastern University, presents a method for de-anonymizing anonymized interviews using widely available large language models (LLMs) to associate responses with the real people who participated.
Scientists exposed
When Anthropic released its 1,250 interviews, the company also grouped them into three categories: general workforce with 1,000 interviews, creatives with 125 interviews and scientists with another 125.
Anthropic says that Interviewer uses Claude, the company’s proprietary LLM, to automatically conduct interviews and send the results to human researchers.
Tianshi Li, an assistant professor of computer science at Northeastern, focused on the scientist subset. First, she filtered them down to 24 interviews that mentioned specific scientific studies. From those, using a publicly available LLM, she was able to de-anonymize 25% of the interviews, associating the interview with a specific paper, and, in some cases, identifying the specific scientist participant.
“It’s surprisingly easy” to pull off this kind of de-anonymization effort, Li says. “I got some initial positive results within one day” of Anthropic releasing its dataset, she continues.
When asked for comment, an Anthropic spokesperson said that those interviewed were “external professionals who first agreed to participate and were informed that their responses to the interview questions would be made publicly available.” They noted that none of the responses were attributed to any individual.
Further, the spokesperson said, “We take data privacy seriously and in standard research contexts where anonymization is expected, we follow rigorous de-identification protocols.”
Li says that she decided to work with the scientist subset almost arbitrarily, because she is a scientist herself and found herself wondering how she would have answered the questions asked by Interviewer.
“People can really be easily influenced by the way AI is asking a question,” Li notes, “and they can share a lot of information about themselves gradually, not in one shot, and they may not fully realize what they have shared.”
This slow, progressive disclosure of information is one way that people reveal more than they might have intended, she says, leaving their data potentially exposed to bad actors.
Is any data anonymous anymore?
Editor’s Picks
Li describes LLMs as a kind of microscope. Data that appears irrelevant or abstract to an untrained human eye often contains valuable information that, under an LLM’s magnification, yields opportunities for inference and connection.
And when the entire internet is available to an LLM as a kind of public dataset to draw from, Li continues, those inferences and connections can be surprising and quick.
She declined to identify which LLM she used to de-anonymize the results.
To make matters worse, not a lot of research has been done yet on the use of LLMs for cyberattacks like the kind Li studies. She says this most recent paper is a proof of concept, demonstrating the ease with which “off-the-shelf LLMs,” as she calls them, can be used to breach supposedly private or anonymous data.
Whether any data can really be anonymized anymore, given the inferential power of LLMs, remains a new and open question, Li says. “We might need something else,” she says, in addition to the tried-and-true methods of anonymization used in the past.
She also says that it’s an issue of education, that the public needs to be aware of the exposure that comes with making something public.
The moment you post something online, she continues, whether that’s a scientific paper, a social media post or a piece of journalism, it becomes information that can be parsed and used by an LLM to make inferences about the human behind the data — you.
