The LLM in the room — Going beyond using AI to do someone else’s job badly

There is a pattern in the way people use artificial intelligence. Since I spotted it, I have been unable to unsee it:

People mostly use artificial intelligence to do the jobs they didn’t value anyway.

For example, the people using AI-generated images aren’t really doing talented illustrators out of a job. Those people were not hiring illustrators to create this sort of imagery before. At best, they were using cheap, low-quality stock images &emdash; or committing crimes against clip art.

In the same way, the people using large language models (LLMs) to generate content never valued high-quality writing anyway.

Some people are using AI to generate “synthetic personas” consisting of made-up insights with zero human input, in the ultimate user experience theatre. Those people never valued user research.

We also see this effect in the way some software engineers are using tools like Claude Design to knock up front-end interfaces, instead of working with a user interface designer.

It goes the other way when designers use LLMs to vibe-code their ideas, instead of collaborating with a software engineer.

I noticed it recently when I watched a video lecture about using LLMs in information architecture work. The questions from the live attendees were all about how to use an LLM to generate an ontology itself. The presenter was aghast at the idea.

LLMs are terrible at information architecture work. After all, when much of the purpose of ontology is to do the very human work of understanding what we mean, you wouldn’t want to outsource that to a stochastic parrot.

Instead, the presenter used LLMs to make the front-ends and back-ends that brought their ontology into use. No doubt front-end and back-end engineers would be horrified at that idea.

Using AI in this way amounts to cutting your colleagues out

If you stand back, it’s like that picture of all the Spidermans pointing at each other, as if they are saying to each other: “I am using AI to replace your job.”

Zoom out further, and it feels like the entire agenda that is driving the current hype cycle: the notion that AI can replace almost all of our jobs. But you can bet the big tech bosses pushing this line intend for their own jobs to stay.

People are trying to use LLMs to cut out the human expertise that comes from disciplines other than their own.

John Cutler recently described the trend where people feel like they are being replaced at work:

I know they aren’t doing it on purpose, but if I listen to what they are saying, and watch what they are doing, the only natural conclusion is that they’re pitching a world where I don’t exist at the company…

Value is created through human-to-human exchanges

The elephant in the room is that value must be created through human-to-human exchanges. This means technology must enhance relationships between humans — not replace them.

This is why Google Maps is the most important app on your mobile device. It is the only app that makes users happier the more they use it. That’s because people use maps to get to a real place to do real things with real people.

We also know the value of working in interdisciplinary teams.

A software engineer can tell a human-centred practitioner why their idea isn’t technically feasible. In turn, a human-centred practitioner can tell a software engineer why their idea won’t work for users. And real user researchers can carry out real research that tells both of them how real users really behave.

In short, people with different perspectives working together produce better outcomes.

Hallucinations are really system errors

By now, it is old news that a major problem with LLMs is their probabilistic nature. This causes them to produce inaccuracies.

Euphemistically, these are called “hallucinations”. That is a cutesy anthropomorphism invented by the AI industry to disguise what they really are: system errors.

A repeated pattern is that AI produces an output that feels plausible on the surface, but crumbles the moment you interrogate it.

Vibe-coded software breaks more easily and requires more maintenance.

Writing generated by LLMs is verbose, laden with factual inaccuracies and cliches.

Graphics produced by AI are firmly in the uncanny valley. Everything is weirdly misaligned. The outlined drawings of people look impossibly soulless, with quarter-smiles and vacant black dots for eyes.

The problem with AI is not the mistakes, it’s the lack of accountability

A common counter-argument is that humans also make many mistakes. For some activities, it could be argued that artificial intelligence makes fewer mistakes than some humans in some circumstances.

But that argument misses the point. When humans make mistakes, they can be held accountable for them. You can interrogate their approaches. You can work with them to improve their output.

This is not truly possible with an AI system. In the words of IBM in 1979:

A computer can never be held accountable.

Therefore a computer must never make a management decision.

This is core to why decision making needs to be social. Decisions have to be scrutinised to ensure they are the right decisions. This means decision makers have to be held to account.

Brute force disguised as creativity

A challenge has been posed by the incident where OpenAI’s agents “escaped” their environment and hacked the infrastructure of Hugging Face. This was seen by some as evidence that artificial intelligence can find novel ways of solving problems.

But the details of what actually happened are less flattering. Cloud Security Alliance outlined a range of very unintelligent aspects of the attack. These included:

• Repeating actions that had already been successful.
• Inefficient routes, and “sloppy” and “clumsy behaviours that no human would choose”.
• Thousands of lines of incoherent text in its activity logs.

The hack happened because the AI agents could try any old thing without getting bored or running out of energy.

This feels less like intelligence, and more like a misguided application of the infinite monkey theorem.

The AI effect returns

In short, it’s the classic AI effect that has been observed for decades. As Rodney Brooks said in 2002:

Every time we figure out a piece of it, it stops being magical; we say, “Oh, that’s just a computation.”

After all, when IBM’s Deep Blue beat chess grandmaster Garry Kasparov in 1997, it was a milestone moment. But by IBM’s own admission, its success was really down to brute force computing power.

It didn’t matter that computers won at chess

Despite computers getting the better of the greatest human players almost three decades ago, chess is arguably bigger than ever as a human pastime.

That is because chess was always popular because it is not a mere computational puzzle. Chess is a form of human expression. It is a game of how humans strategise and make decisions when facing adversaries under pressure.

This is why Magnus Carlsen can make headlines today in a way that no chess computer can.

Finding the human-centred use for AI

I do think there is still a role for artificial intelligence in our lives. It’s not as if we can uninvent it. But even if we could, we possibly shouldn’t. Indeed, I have used it a number of times to research sources for this very article.

Human-centred uses of LLMs and AI agents are emerging. There are signs that the tenacity of AI agents could be putting power in the hands of ordinary people like never before.

In March, Tom Loosemore declared: “Demand management is doomed”:

“Managing demand” by making processes confusing and time-consuming will no longer work. AI agents are relentlessly dogged.

More recently, Paul Taylor said: “The cost of complaining just collapsed”. He said he is hearing organisations facing a rise in complaints.

For decades, many corporate models have relied on friction as a business strategy. Often described in behavioural economics as engineered “sludge”, this is the deliberate introduction of operational hurdles, cognitive overload, or procedural delays designed to influence consumer behaviour in favour of the company.

He goes on to note that AI tools are now making it so easy to navigate these bureaucratic barriers that it amounts to “a redistribution of power”.

Workers might be needed more than ever

This technology — and people’s awareness of how to use it for this purpose — is still in its infancy.

Organisations might now have to put in the effort to make their services actually human-centred, instead of just pretending they are. Workers might find themselves becoming as busy as ever.

Geoffrey Hinton was wrong in his 2014 prediction that artificial intelligence would render radiologists obsolete within five years. In fact, it transpires to be a textbook example of Jevons’ paradox. Increased efficiency in the use of a resource often causes an increase in demand, not a decrease as people often initially expect.

In a similar way, the rise of artificial intelligence may create the need for more workers, not fewer.

So we should rapidly focus on uses of AI that go beyond trying to do someone else’s job badly.