Dr GPT or: How I Learned to Stop Worrying and Love the AI Part 4
Part 4: Google to the rescue?
Introducing Bard
Having been underwhelmed by my first three forays into the application of AI to actuarial functions, all hopes were pinned on Google’s new offering. Google entered the arena later than its rivals, with “Bard AI” earlier this year. Unlike Bing and ChatGPT, both of which are based on GTP3.5 and GPT4, Bard uses Google’s proprietary model “LaMDA”. In the fourth part of this series, I’ll report my thoughts on Bard AI, having had the opportunity to test the capabilities of the beta version.
Most of Google’s promotional material around the launch of Bard highlighted its uses for helping with lifestyle tasks such as camping and gardening. For this reason, I thought it would be unfair to compare it directly with ChatGPT which is marketed as a tool capable of assisting with professional work and code writing. At least that is what I thought until I asked Bard what it could do for actuaries. Interestingly, Bard informed me it could help calculate reserves and premiums. Great … here we go again!
The Bermuda Triangle
Following Bard’s suggestion that it could help with actuarial calculations, I asked it to project triangles (a fundamental actuarial method for calculating reserves), which resulted in a detailed explanation of how it could project a triangle from a 3-D space to a 2-D space. So, while it seemed Google’s bot could speak well and had fast response times, context clues still seemed to be a struggle, resulting in a pointless response completely unrelated to the actuarial work it claimed to offer help with.
Trying a different approach, I sent Bard a triangle and asked it to project it to ultimate. After several messages, explaining how it could project it, Bard finally did project … an unrelated triangle. Bard’s understanding of the chain ladder method (an actuarial loss reserving technique) seemed to involve nonsense logic, including multiplying by a seemingly arbitrary factor of 0.8 and calling it a day.
A wild goose chase averted
While this result may seem like Bard is less useful than ChatGPT for actuaries, we could still suggest that the outcome of this investigation was a positive one. Bard’s lack of comprehension and competence was immediately apparent, unlike that of ChatGPT which tried to persuade me it could produce useful results but never delivered a correct answer despite countless back-and-forth message exchanges. I therefore saved a lot of wasted time in trying to coax a correct answer out of Bard.
In conclusion, while Bard turned out to be next to useless for professional purposes, it was a far less painful experience than ChatGPT, if only due to it being openly less knowledgeable and therefore not getting my hopes up of delivering any useful results. Google’s own promotional material rightly doesn’t describe Bard as a tool for mathematical professionals.
It’s worth noting that at the time of writing, I had only had access to Bard for a few days. Time will tell whether Bard or any of its competitors become more useful to anyone other than campers and gardeners.
Your thoughts?
What are your impressions of AI for professional applications? Please comment below - I'd love to hear your thoughts.
- Jack Kelly, Actuarial Analyst at Marcuson Consulting
In case you missed the rest of the series, please click below for Parts 1, 2, and 3.
