ChatGPT chats exposed in Google search results spark privacy concerns

OpenAI has deactivated a controversial ChatGPT feature following revelations that it inadvertently exposed private user conversations in Google search results. The move comes after a report by Fast Company highlighted that thousands of highly personal ChatGPT chats were publicly discoverable, potentially impacting millions of users.

Fast Company’s investigation found numerous conversations indexed by Google, some containing intimate details such as specific descriptions of interpersonal relationships. While user identifying information was not directly included in the indexed chats, the deeply personal nature of some content raised concerns about potential indirect identification.

According to Dane Stuckey, OpenAI’s Chief Information Security Officer, the exposed chats belonged to users who had “opted in” to indexing. This occurred when users chose to share a chat and subsequently clicked a checkbox labeled “Make this chat discoverable.” Below this option, in smaller, lighter text, was a caveat explaining that the chat could then appear in search engine results. Fast Company noted that users often share chats for simple purposes like sending them via WhatsApp or saving a link for later access, and may have overlooked or misunderstood the indexing implication due to the subtle formatting.

Initially, OpenAI defended the clarity of its labeling. However, Stuckey later confirmed that the company ultimately decided the feature “introduced too many opportunities for folks to accidentally share things they didn’t intend to.” The exposed conversations included highly sensitive discussions about drug use, sex lives, mental health, and traumatic experiences.

The revelation drew strong criticism from AI ethicist Carissa Veliz of the University of Oxford, who expressed “shock” at the indexing of such “extremely sensitive conversations.” Veliz criticized the broader practice of tech companies, stating they often “use the general population as guinea pigs” by deploying new AI products and waiting to gauge user reactions to potentially invasive design choices.

Stuckey characterized the feature as a “short-lived experiment” launched “to help people discover useful conversations.” He confirmed that OpenAI is actively working to remove the indexed content from relevant search engines. Google has not commented on the matter, leaving it unclear whether all indexed chats have been fully removed or what role the search giant played in their display.

For users concerned about their past chats, Fast Company suggested that those who still have access to their shared links could input parts of these links into Google to check for discoverable conversations.

This incident adds to existing privacy concerns surrounding OpenAI. It follows a recent legal battle where OpenAI vowed to fight a court order requiring it to preserve all deleted chats indefinitely, a stance that worried users who believed their temporary and deleted chats were not being saved. While OpenAI CEO Sam Altman previously acknowledged the “screwed up” nature of users’ most private chats being searchable in a legal context, Fast Company noted a perceived lack of similar transparency regarding OpenAI’s own practices potentially exposing chats via search engines.

OpenAI has reiterated its commitment to user privacy. “Security and privacy are paramount for us, and we’ll keep working to maximally reflect that in our products and features,” Stuckey stated, indicating the company’s intent to rebuild trust after this misstep.