ChatGPT Passes CPA Exam on Second Attempt. Is It Coming For Accounting Jobs?

An upgraded and better-trained version of ChatGPT retook a practice CPA exam and passed, not unlike the roughly half of human candidates who fail on their first try. But does this experiment mean ChatGPT is coming for accounting jobs?

So, Is ChatGPT Coming For Accounting Jobs?

Researchers said the latest version of ChatGPT successfully took the CPA exam, just weeks after an earlier version of the chatbot failed.

"This calls into question the 'competitive advantage' of the human accountant relative to the machine. To our knowledge, for the first time, AI has performed as well as a majority of human accountants on a real-world accounting task. This raises important questions of how machine and accountant will work together in the future," the study's authors said.

The fear around artificial intelligence (AI) is that it's coming for human jobs. There are some jobs that a machine just can't do, but then there are those that will change to accommodate AI efficiencies. The question is, do accounting jobs fall into that category? This latest test shows that might be the case, but not without some tweaking to the AI systems.

How ChatGPT Passed The CPA Exam

ChatGPT's success wasn't without help.

Better Equipped With Learning Tools

While ChatGPT has proven its skills in a number of applications, its performance so far in student assessments like the CPA exam has been unimpressive, something the researchers wanted to change.

To tweak the chatbot, the researchers used "chain of thought" prompting and a "10-shot" scenario—where it was trained with 10 examples—to better prepare the AI on the subject matter and improve its retention of key concepts.

Upgraded Version

The version that passed the CPA exam was ChatGPT 4.0, an upgrade from the original version. The researchers found newer models of ChatGPT, which are equipped with better training and reasoning capabilities, outperformed older versions by an average of 16.5%.

The earlier Version 3.5 averaged a score of just 53.1 out of 100 and was unable to pass any section of the exam, while the latest Version 4.0 averaged a score of 85.1 and passed all four sections. In its best section, auditing and attestation (AUD), the chatbot got a score of 87.5.

Earlier this month, trade magazine Accounting Today had Version 3.5 of ChatGPT take the CPA Exam. The chatbot scored just 48% in business environment and concepts (BEC), its strongest category, and a paltry 35% in financial accounting and reporting (FAR), its weakest category.

The results from the researchers' test prove the latest versions of ChatGPT, which comes equipped with the most advanced reasoning capabilities yet, can handle just about any assessment, which could have wide-ranging implications for major industries like accounting and auditing.

ChatGPT has also proven its capabilities in several high-profile exams including the Wharton MBA exam, the bar exam, and various AP exams.