My name is Teagan, and I’m a Managing Consultant at ISL Talent. I specialise in product and have worked hard to build a solid network over the years. I’ve created this blog series to gather insights from high level product people and help others in the community with advice to take their careers to the next level.
In this interview, I chat to Product Manager Brian Joyce. We talk about everything from leveraging GenAi and LLMs in product management to the ethics of AI and future industry predictions.
Tell me a bit about yourself
"I’m a Product Manager/Owner with 7+ years and have been responsible for taking ideas/problems from inception to delivery. I’ve worked on various SaaS solutions for an innovation team called The Lab for O2. We were responsible for leading innovation for O2 with a mantra of, 'fail early, fail fast, fail cheap'. Not everything we worked on was a success, but we learned from our failures and built many innovative successful SaaS solutions for the business."
Can you tell me about your career journey to get where you are now?
"I started my tech career as a Database Administrator (DBA) on Microsoft SQL Server. It’s a great way into tech and I learnt a lot working as a DBA for over 5 years. As time went on, I yearned to do more beyond databases so started to learn how to code. At the time Ruby on Rails was popular, so I went deep into the framework and landed a job for O2 as a software engineer in The Lab. Many of the projects we were taking on seemed to have a feel for being solved with big data/ML/AI. I put together a case for hiring and leading some data scientists and software engineers to tackle these projects. I then implemented standards such as the Jobs To Be Done (JTBD) framework in order to put a bit of structure around asking the all-important question of, 'why do they need this problem solved?'.
Over the years the team was hugely successful in building many groundbreaking solutions, bringing new tech and ideas to O2 and ensuring we were able to stay on top of an ever-changing landscape. More recently I’ve started freelancing for various customers in a similar vein to the work I was doing in The Lab. I look at how new tech can help businesses solve problems and help lead the product lifecycle process. It’s a rewarding job with many challenges."
What is one piece of advice you can offer product people in the current climate?
"Specialise if you can. Don’t be afraid to be wrong, testing a hypothesis is more important and often more valuable than confirming an assumption. Customers change, people change, products change - being able to adapt in a robust and clear way will serve you well."
How can leveraging GenAI and LLMs provide a competitive edge in product development and management?
"As someone that built and led data science teams and has been working in the AI/ML space for years the landscape has changed massively since LLMs became popular. A few years ago, if you asked anyone what a Large Language Model was, they probably wouldn’t have a clue, now they’ve become mainstream.
The idea that you can pop a few bullet points into a LLM with an instruction and it can come back with a coherent story, sentence, paragraph or blog post about that topic is obviously highly beneficial.
The most useful thing I have found is using LLMs as a tool with the understanding they can get things wrong. When most people use LLMs today they are putting in a single prompt and expecting perfection, but even we as humans can’t do that, so why expect the LLMs to?
To get a competitive edge learn to agentify your workflows (yes agentify is a real word). Use agents to help the LLMs come up with answers to your questions or tasks. The reason behind this is because it gives the LLM a chance to evaluate and correct itself. Essentially going back thinking, analysing its last answer and trying to improve upon it, something we do as humans all the time. By using agents in your workflows, the quality of the output from an LLM will be much higher, and by using agents you will also get a better understanding of the limitations of LLMs."
What industries do you think are adopting these technologies and succeeding and which do you think are behind the curve?
"Adoption has been widespread. Most of the companies I’ve been helping over the last few years have been using LLMs in one form or another. In one company it’s been used to fix a bug in production in minutes instead of hours, saving the company millions. In areas such as content creation there has obviously been massive uptake, something that I think Google is struggling with.
I’ve seen huge progress in fintech, legal, biotech and telco. If there was an industry which I think could benefit from disruption from this, I’d suggest construction. Various aspects of the construction industry such as a deep technical understanding of various standards could be better understood and simplified using LLMs. I also think there is a massive opportunity in combining knowledge graphs with LLMs. Many companies are falling into the trap of using LLMs exclusively with fine-tuned models, but in the real world I’ve seen a higher standard of output a combination of knowledge graphs with LLMs."
How can we ensure our teams are trained properly and using AI in an ethical manner?
"The ethical landscape for Gen AI and LLMs has changed dramatically in the last few years. OpenAI has recently been in some turbulence because it disbanded its entire team dedicated to keeping AI safe. Companies building and using LLMs are charged with keeping them safe, but it’s not as easy as it seems. Two months ago, it was possible for someone with just a basic understanding of how to use LLMs to circumvent the safety protocols OpenAI had put in place.
When we think about training teams, keeping humans in the loop and holding organisations responsible for the output is still the best way to ensure safety, but it comes at the cost of full automation. Gemini has been criticised for producing historically inaccurate images of people and events. Models still hallucinate, they may sound authoritative and correct, but may not be. Verifying output with different LLMs is a step some of my clients have taken to try to maintain automation while ensuring high quality output from an LLM."
Are there any specific tools or platforms you recommend for product managers looking to integrate these technologies?
Some tools I like include:
Research
Agents and coding
Running different LLMs
For starting LLM use
To structure data
For control of big data
Any final advice or predictions?
"Just remember that not all information is up to date. Context windows are one of the main limitations of LLMs, although the issue will likely be solved in a year or two.
We can already see reductions in costs for most LLMs which will continue. I think Biotech and construction are going to change dramatically in the next few years, with improved materials in construction and personalised medicines in biotech. Quantum computing is also going to increase the rate of acceleration of change. The companies that survive will be those investing in innovation and adapting faster than anyone else."
Get in touch
In this interview, we’ve explored the importance of specialisation, adaptability, and the value of hypothesis testing in product management. We've also delved into the transformative potential of GenAI and LLMs in product development, emphasising the benefits of using agents to enhance workflow efficiency and accuracy.
In summary, continuous learning, embracing innovation, and adapting to new technological advancements are key to staying competitive and successful in the ever-evolving landscape of product management.
These blogs are all about connecting people, so if you liked what Brian had to say, connect with him here. Alternatively, if you’d like to feature in a future blog or would like to chat to me about product roles, drop me a message on LinkedIn here.
What are you doing to combat proximity bias in your startup?
How to let your personality shine through in your CV
Corporate social responsibility for tech SME's