Specific data on the performance impact of AI in the Accounting Industry is scarce, but I did come across a 2023 Harvard Business Review study* that tested 758 Management Consultants from Boston Consulting Group (BCG) on 18 "Realistic consulting tasks". One group used AI (GPT-4) to help them complete the tasks while the control group had no access to AI tools.
The Consultants that used GPT-4 on these tasks completed 12% more tasks, were 25% faster in completing tasks, and also produced results that were 40% "higher quality" than the control group who did not use an AI tool.
Even more interesting to me was the fact that these AI-based improvements were not evenly spread across the consultants. Those who were in the bottom half of performers (judged by performance on a baseline test conducted before the experimental task) improved by 43%. While the top half of performers only improved by 17% when using AI.
Which I have oversimplified into this equation:
Q: What would it mean for your firm if you could raise the quality of work of your bottom 50% of performers to the level of your top 50% of performers?
Three Key Takeaways
Individual Consultants using AI (GPT-4) to help them on "realistic consulting tasks" were 25% faster and produced 40% higher quality work than those not using AI.
The bottom half of performers improved by 43% when given an AI tool which brought them very close the level of the top half of performers who only improved by 17%.
We are still learning which complex tasks AI is better at. And... there are still tasks on which we humans perform better WITHOUT the help of AI.
For the longer version and the data behind this working paper click below.