Skip to content

Beware of AI’s fake confidence

Published by Kevin Nix in AI Share


AI remains a locomotive traveling at high speeds through a dark tunnel without a conductor. We don’t know yet how things will shake out on the other side. The train may not even stay on the tracks at this speed and no guardrails. 

Wherever happens, the single most important thing to remember right now is to tread with caution. If you use AI tools, like Claude, ChatGPT, Gemini or other “advisors,” “bots,” or “tools,” there’s a right and really bad mindset in how you use them for information retrieval. I’m not talking about coding or any number of specialized tasks.

In The Atlantic, Nick Dothée writes about how what keeps him coming back to LLMs is not their accuracy. We all know they hallucinate and give wrong, nuance-free, incomplete answers.

But what keeps him coming back is ChatGPT’s confidence. The platform replies with such authority and swagger and even keeled tone, sounding like it knows what it’s talking about when it comes to search. 

That’s a problem because it starts to corrode our own confidence. 

Dothee writes that, “Confidence changes how people evaluate information…[P]eople tend to use confidence as a shortcut for assessing credibility, especially when accuracy is hard to judge. The effect persists even when people know a system or person can be wrong.”

The same thing is true with human beings. We all know someone who thinks they’re always right and never shows doubt or qualifies what they say. Who is constantly in the fake-it-til-you-make it mode. But as you get older, with life experience, you can often detect such fake confidence.  

So don’t turn off your B.S. detectors when you go on AI platforms. That little voice inside your head should be constantly questioning how the AI bot answered your prompt. Make the bots provide their sources. You are the fact checker in control—not the AI.

Levers of Power is delivered to your inbox once a week.

Get a single theme-driven weekly briefing, the best curated political content, previews for new live learning sprints, and new tiny book announcements. Above all, get inspiration and know-how.

Personal information