The spread of systems known as large language model (LLM), ChatGPT above all, has caused a real revolution in the relationship between man and machine.
Put to the test, in fact theArtificial intelligence it proves excellent at carrying out tasks in just a few seconds that can take humans hours. Yet, in some contexts, the chatbot still show all its limitations.
Apparently, in fact, i accountants they don’t have to (for now) fear competition from ChatGPT. According to some recent tests, in fact, the AI created by OpenAI it is decidedly denied in the calculations that characterize this sector.
The results of some experiments, carried out by David Woodprofessor of accounting at Brigham Young University (BYU), have exposed a high level of AI incompetence when it comes to dealing with certain bills.
Wood decided to harness the power of the global accounting community through a social media presentation soliciting help to put ChatGPT to the test on some sort of accounting exam.
They participated in the study 327 co-authors coming from 186 schools located in 14 countries. These have put together more than 2,000 questions for the accounting exam to submit to the chatbot.
Typical of a comprehensive accounting exam, the questions spanned all major topics. which financial accounting, audit, accounting management, tax and not only. The questions presented were of different types (multiple choice, short answers, true/false) and with different levels of difficulty.
ChatGPT failed in accounting: here’s why
The results were unequivocal: ChatGPT recorded a result of 47.4% of correct answers which, in itself, wasn’t that disastrous. By involving some students engaged in the subject, however, they proved to be much better than the AI, obtaining an average of 76,7%.
According to Wood’s paper on the test, ChatGPT demonstrated difficulty getting its artificial neurons to tackle problems involving tax, financial and management evaluation, which are areas that require high mathematical skills.
These results can be explained in several ways. First, AI has been fed a huge amount of data but, of this information, only a small part is related to numbers and mathematics.
The same structure as ChatGPT is based on linguistic modelstherefore focused on providing colloquial answers to the questions. In this regard, it is easy to understand why AI is not comfortable with accounting.
It should then be considered that the result of these tests may be partly conditioned by the chance. In other contexts and with other questions, albeit on similar topics, ChatGPT could provide different results.
