When you click the 👍 or 👎 to rate a response, you’re unwittingly handing over your conversation to be used for training. Even if you turn that setting off in your privacy settings!

Only a week or so ago, OpenAI, the company that runs ChatGPT, announced a mathematical breakthrough.

Shortly after, two groups of mathematicians working on the same problems claimed their unpublished research had found its way to OpenAI through the conversations they’d been having with ChatGPT.

OpenAI said no researcher had seen their work, but admitted it “cannot rule out” that data from their use of ChatGPT had helped improve its models.

This got me thinking and researching how your AI conversations could end up in AI training data.

What I discovered is that even if you’ve disabled the use of your conversations for AI training in your settings, they can still be used via a technicality that’s included in every response!

Next time you have a conversation with an AI, take a look, and near the bottom of every response from the AI, there will be an option to send feedback with the thumbs-up 👍 or thumbs-down 👎 icon.

It turns out that after reading the privacy policies for Claude, ChatGPT, and Gemini, these feedback icons sit outside your account’s privacy settings. Clicking them hands over your conversation and any files you’ve uploaded in that conversation for AI training, and they can be retained for many years.

In the business versions, ChatGPT and Claude will still use your feedback for training. However, with Claude, administrators can disable thumbs-up and thumbs-down ratings. With Gemini in Google Workspace, this feedback isn’t used for AI training.

My advice is simple: don’t give any feedback to the AI companies based on the response you’ve had.

The rating helps them, but doesn’t do much for you. Your next answer doesn’t get better just because you clicked thumbs down on the last one. All you’re doing is giving these companies more training data.

I’ve written up the full details on the Kimbley IT blog, including what ChatGPT, Claude, and Gemini each say on their own privacy pages and how the business plans differ.

Which leads me back to the mathematicians' research data. Could one of those mathematicians have rated a response from ChatGPT with a thumbs-up or thumbs-down, unknowingly handing over their research data to the AI, which in turn helped OpenAI make its mathematical breakthrough?

James