Sounding like you know vs actually knowing

Using AI to learn, not just answer

AI can give anyone the words to sound like an expert in a few seconds. That’s useful, but it’s not the same as understanding something, and at work the difference shows up quickly. Here’s the test I use and the questions I ask AI so I end up knowing it, not just sounding like I do.

  1. 1
    Sounding like you knowYou ask the AI, take the answer and repeat it
  2. 2
    Knowing where your understanding stopsThe test shows you, so you know what to go and learn next
  3. 3
    Actually knowingYou can explain it in your own words, answer questions about it and have a deeper conversation

Admitting you don’t know feels riskier now

A recent survey by Use.AI of 10,417 working adults found that 56% feel admitting they don’t know something at work is riskier than it was before AI. 37% said they had pretended to know something in the last six months, and 58% of those went and checked privately afterwards. It’s higher for younger people too: 71% of 18 to 29 year olds, against 44% of people over 45.

CautionIt’s worth saying that Use.AI sells an AI assistant, and the survey doesn’t show that AI caused any of this.

But it matches what I see, and I think the reason is fairly simple. When everyone has an assistant that can explain anything, “I don’t know” sounds like you didn’t bother to ask, so people ask the AI, take the answer and repeat it.

The good news is that the same tool that makes it easy to sound like you know is also a very good way to actually learn, if you use it a bit differently.

What sounding like you know looks like

I’ve seen this many times. You can usually tell because the person explains things very mechanically, like they’re repeating something they read. It sounds fine at first. The issues come up when the conversation goes a bit deeper, either because you question them on their thinking, or because you add something from your own knowledge and they can’t follow it or don’t know what you’re talking about.

Let’s say someone asks an AI about programming languages and comes back with “I chose PHP because of X”. You ask whether they considered Y, and if not, why not, and they can’t answer or don’t know what Y is, when someone who works with it every day would. Or someone with no experience or training in programming is suddenly explaining why React is the best choice.

That could just be a gap in their education, and that’s fine, nobody knows everything. But either way it comes down to whether the person can show they understand the topic: explain it in their own words, answer questions about it and have a deeper conversation. The AI’s answer can’t do that part for you.

The difference only shows once the conversation goes past the first answer.
Sounds right Knows it
How it’s explainedMechanically, like something they readIn their own words
“Why?” or “What about Y?”Can’t answer, or doesn’t know what Y isAnswers, and can talk through the alternatives
You add something you knowCan’t follow itHas the deeper conversation

The test I use on myself

Before I rely on something I’ve learned from AI, I ask myself three things:

  1. 3
    If someone asked me why, or asked about the alternatives, could I have that discussion?
  2. 2
    Does it actually sound logical when I say it out loud, not just when I read it?
  3. 1
    Could I explain this to someone in my own words?

If the answer to any of them is no, I don’t know it yet. I just have an answer.

One caveat, and it’s an honest one: to really judge whether an answer is right, you need to understand the subject yourself. The test doesn’t replace that. What it does is show you where your understanding stops, so you know what to go and learn next.

How I use AI to learn, not just to answer

This is the routine I follow, and it’s not complicated.

  1. 1
    Read the answer and do a sanity check.

    That sounds obvious, but a lot of people skim it and move on. If it’s a long output, I ask it to summarise the key points first, so I know what I’m actually checking.

  2. 2
    Ask it how it got there.

    How did it reach that conclusion or that proposal? This often shows an assumption I didn’t know it had made.

  3. 3
    Ask about the alternatives.

    Were there other options, and if so, why weren’t they chosen? This is the question that turns “use X” into understanding why X, and it’s exactly the question the person repeating an answer can’t handle.

  4. 4
    Ask it to explain anything I couldn’t explain myself.

    If there’s a word or idea in the answer I’m unfamiliar with, or couldn’t explain in more detail, I ask it to go into more detail on that part.

Then I go back to the test. Could I explain it in my own words now? Usually, after a few rounds, yes.

Questioning the AI catches its mistakes too

This isn’t only about learning. Asking follow-up questions is also one of the simplest ways I catch the AI getting something wrong.

It happened again on 12 November. I was discussing some new requirements with an AI before it started building, and something it said made me think it had misunderstood what I meant. So I asked why it had asked that, and checked it didn’t mean X, because if it did, that’s not what I wanted. It admitted it had misread my meaning and corrected itself before it went any further. If I’d just let it carry on, I’d have found out much later, after it had built the wrong thing.

That happens to me a lot, and it’s the same questions as above, just pointed at the AI’s understanding instead of mine.

What I teach people

When I help people start using AI at work, this is a big part of it. Partly it’s mindset: the AI’s answer is a starting point to question, not a finished thing to repeat. Partly it’s habits, like checking the data yourself instead of trusting the summary, and for anything that matters, having a second agent review the first one’s work. A second reviewer asks the questions you might forget to ask, and it doesn’t have the same blind spots as the one that wrote the answer.

None of this slows you down much. A couple of follow-up questions takes a minute, and it’s the difference between being able to use what you learned and only being able to repeat it.

Try it this week

Next time you ask AI something you’ll rely on at work, don’t stop at the first answer. Paste these in as follow-ups, one at a time:

  1. 1
    Summarise the key points of your answer in a few lines.
  2. 2
    How did you get to that conclusion? What did you assume?
  3. 3
    What were the other options, and why didn’t you pick them?
  4. 4
    Explain [the part I don’t understand] in more detail, in plain words.

Then try explaining the answer to a colleague, or just out loud to yourself. If you get stuck on “why”, that’s the next question to ask.

Source: the Use.AI survey of 10,417 working adults, as reported by The Agile Brand Guide.