Taken from this LinkedIn post by Dean Casey:
After 25 years of building rock-solid systems for 50+ wealth managers, I thought AI would be a breeze.
💥Wow, was I wrong! 💥
I have built many successful projects using AI. It is amazing what you can do with it. I was finding it easier and easier to develop some really great products.
But it lead to a lot of over-confidence and I was recently humbled.
My goal with this newsletter is to help you learn from my mistakes and hopefully laugh with me 😀
Here’s why you need to be cautious with AI in your business:
1️⃣ Inconsistent outputs
The same input can yield different results each time, which is a nightmare for metric testing.
Let me share a real-world example from our recent project:
We built a mobile app and AI system to automate face-to-face client meetings.
It works like this:
👉 You record the meeting using your mobile phone and our app,
👉 It uses an AI model to transcribe the meeting – a fancy way of saying it converts voice to text.
👉 A separate AI model then summarises the meeting and extracts the action items.
👉 The extracted data is uploaded to the CRM system and also to the Task Management system for further handling
Sounds simple, right?
💥 Wrong! 💥
Here’s what happened:
✅ We built the app very quickly
We are so good I arrogantly thought
✅ The meeting recordings worked well even in less than ideal environments.
I was quietly impressed with our own brilliance
✅ We started testing. At face value, the transcription seemed fine. A couple of minor words were incorrect but the context was perfect
This is going to be a piece of cake!
Then 💥
We uploaded the same meeting recording to the same AI model five times (actually more like 500 times). Each time, the model returned different results:
😢 First run: 4 action items
😢 Second run: 5 action items.
😢 Third run: 4 action items again, but worded differently
😢 Fourth run: …. you get the idea
Wow, how could I be humbled so badly.
We tried everything, changing prompts, better example outputs, testing a range of models. The list went on…
This is unusable!
Imagine sending incorrect action items to your team after each meeting? Or even worse, imagine that this was fully automated and sent the meeting notes directly to your clients.
2️⃣ Black box problem
You can’t see how AI makes decisions. Debugging becomes very challenging and time consuming.
In traditional programming, if something goes wrong, you can trace the logic step-by-step. With AI, it’s like trying to fix a car engine while blindfolded.
For example:
When our AI system misinterpreted a client’s request, we couldn’t pinpoint why. Was it the audio quality? The prompts? The AI model selected? A gap in the AI’s training data? Or something else entirely?
This lack of transparency is especially risky in wealth management, where decisions impact people’s financial futures.
3️⃣ Overconfidence trap
I use AI tools all day, every day. From ChatGPT to software development copilots (AI helpers). They are really brilliant and save me huge amounts of time.
However, it’s easy to think AI can do everything. I forgot that, in reality, it’s just a mathematical approximation engine with potentially many outputs to a prompt or question.
We fell into this trap ourselves. We thought,
“Great! AI will automate our client’s daily lives and save many hours of manual labour.”
But here’s the reality check:
❌ AI struggled with people talking over each other.
❌ AI struggled with background noise
❌ Was inconsistent with accurately extracting action items (which was worse than none at all)
The lesson?
AI is a powerful tool, but it’s not a magic wand. Know its strengths and limitations.
4️⃣ Time sink
What was quick to build took weeks to test and validate.
Our initial AI system took just 2 weeks to build. Sounds efficient, right?
But then came testing:
❌ 6+ weeks (and counting) of running multiple scenarios
❌ Countless hours comparing AI outputs to actual meeting notes
❌ Variable outputs
❌ Endless frustrating attempts to improve accuracy
In the end, we spent huge amounts of time testing and tweaking with no certainty or confidence that we were moving in the right direction
Here’s my hard-earned advice:
✅ Have a well defined goal (with timelines and budget) before starting any project.
✅ Make sure that the project is suitable for AI and the AI model is suitable
Pro Tip: Many wealth management tasks are metric calculation based. Do not use AI for these (or at least not generative AI).
✅ Always extensively test AI outputs before implementation.
For example: In early stages build confidence by having experienced personnel review AI-generated outputs before sending to clients
Pro Tip: Looks can be deceiving. You have been warned
✅ Finally, be prepared to walk away if it isn’t working!
I forgot this rule and poured huge amounts of wasted time into this black hole. It often felt like we were almost there – tweaking prompts, changing models. In reality there were too many variables and unknowns. We needed to reset, re-plan and start again.
Remember
AI is powerful but it can be dangerous. Make sure you understand the consequences to your business of incorrect outputs.
Don’t let the hype cloud your judgment. Know its limitations.
Your business deserves better than inconsistent results.
Stay safe on your AI journey and have a great weekend!
With 25 years of experience helping 50+ wealth managers grow through automation and superior client experiences, we can guide you through the AI minefield.
PS. Want to hear more about this story and laugh with me (or at me)?
DM me “AI sucks” to setup a chat.