An Embarrassing Climbdown
KPMG was forced into an embarrassing climbdown recently. It had to withdraw a report on AI adoption because some of its sources were made up. At least four prominent companies made public statements saying that the claims about how they used AI were wrong.
I use a lot of sources in my writing. I often start by having AI extract the key arguments from a paper I’ve come across. I ask it to tell me whether the claims are conjecture, modelled or supported by facts. It appears that I now need to ask it to check the sources are real, or go through them all myself.
The Department of Science, Innovation and Technology published a report on AI adoption in the UK earlier this year. It noted that natural language processing and text generation were by far the most popular uses of AI. 85% of adopters use LLMs to generate emails, reports and presentations. They are at risk of similar mistakes to KPMG.
The problem at KPMG was that no one checked the links that AI provided as examples of successful adoption. Having another human reviewer would not necessarily have helped, especially if they waited till close to the deadline to skim-read the report. Who clicks through the references in an article to check they are all real? Someone should.
I’ll bet no one used to click through the sources when they knew articles were entirely researched and written by humans. Using that same process when the tools of the trade change causes issues. This set me thinking about the type of AI training we should be giving employees.
The lesson is not that AI hallucinates, but that businesses must change how they train people now that AI has become part of everyday work.
How to Train Your Workforce
The DSIT report was a comprehensive look at AI adoption across the whole economy. That includes plumbers and newsagents, as well as research labs and derivatives traders. The results show far less adoption than studies by the likes of Lloyds Banking Group and Microsoft, which focus on larger companies.
The DSIT found that only 16% of businesses use AI. 80% do not. The most important reasons for this were no need for it and a lack of skills. They are very different objections and can be in conflict. How would you know you have no use for AI if you lack the skill to use it?
Firm-specific barriers to adoption include messy data, old systems, poor process documentation and no measurement framework. These sound like real reasons hiding behind a catch-all such as we have no use for AI. They are also fertile areas for training.
Not every problem requires an AI solution. Simple automation is the best way to address repetitive tasks, such as ordering the same materials each month from a supplier. But you still need clean data and a clear process to make it work.
Thereafter, you need to test and evaluate a workflow. Testing an automation asks whether it works, while evaluating asks whether it does a better job at a lower cost than what it is set to replace.
This is where you figure out if AI research is inventing sources and whether catching them takes more time than doing the research yourself. In most cases AI will still be worth it.
There’s been a lot in the media of late about how having humans review AI does not work. Humans make mistakes, find it too easy to trust the answers and get in the way of AI systems that can now work in 24-hour loops, finding mistakes, improving processes and delivering outcomes.
The DSIT report makes clear this is not how most businesses view AI. Humans remain a valuable part of the process, especially when output is being sent to clients. Training people to understand how AI makes different types of decisions becomes a valuable part of an overall AI education programme.
The overall programme is important because even when you know what you want AI to do in your business, you do not just start. Training people how to use AI to do tasks is fine, but someone needs to own the design of the project to deliver precise outcomes, which will be carefully planned and prepared.
Then choose a small team to be responsible for testing and evaluation. Make someone responsible for governance, meaning monitoring how AI performs in production. You won’t be able to test every possible scenario ahead of release and will need to adapt systems to changing market conditions.
There are a lot of horror stories about AI trainers who did little more than teach people how to make a markdown file. If you are going to employ a training firm, make sure they understand that what and why are as valuable lessons as how to do something.
There are AI products that can sit on your systems and tell you what could and should be automated. There is automated validation and evaluation that will accelerate adoption. There is software for governance.
Comprehensive teaching will cover all of this. But the most important thing it will do is impart an understanding of the strengths and weaknesses of automation and provide a framework for deciding when you should do it.
Questions to Ask and Answer
Do I know who is using an LLM for comprehension and writing?
Is this writing being published or sent to clients?
Have the authors received training on best practice for using LLMs?