Healthcare is rushing towards artificial intelligence. AI is being used to write clinical notes, support diagnosis, triage patients, predict deterioration and automate administrative work. The promises are substantial: greater efficiency, lower costs, less bureaucracy and more time for clinicians to care for patients. Some of those promises will be realised. But before we ask what AI can do, healthcare needs to ask a much simpler question: what problem are we actually trying to solve? It sounds obvious. In practice, it is remarkably easy to skip. An organisation identifies an AI product. A demonstration is impressive. A pilot is proposed. A business case follows. Only later does someone ask how success will be measured. We have started with the solution and worked backwards towards the problem. That is the wrong way round. Clarity before technology Not every AI project needs a lengthy business case before anyone can think creatively about it. But every project needs clarity. What, precisely, hurts, and for whom? Is it the clinician spending an additional hour completing documentation after surgery? The patient waiting too long for an appointment? The administrator repeatedly performing a task that could be automated? The organisation spending money on an inefficient process? These are different problems. They may require different solutions. And sometimes the best solution will not involve AI at all. That is an important test of serious AI leadership: could we solve this problem without AI? If we could, AI should have to earn its place against the alternative, not merely against doing nothing. The danger is that organisations become so impressed by what technology can do that they begin looking for somewhere to use it. The vendor arrives with a solution and, unintentionally or otherwise, brings a definition of the problem with it. Soon we have a solution searching for a problem. Healthcare needs the opposite: problem-led innovation. Every AI benefit has another side There is another question I believe healthcare leaders should ask before deploying AI. Not simply, “What problem will this solve?” but: what trade-off are we making? Healthcare rarely offers clean wins. AI may save clinicians time, but we must protect the quality of attention given to patients. It may make documentation faster, but we must protect the accuracy of the clinical record. It may accelerate triage, but we must protect diagnostic caution. It may standardise decision-making, but we must preserve professional judgement for the patient who does not fit the pattern. It may reduce costs, but we must understand what that means for workforce trust and morale. And decision-support systems may help clinicians think, while at the same time creating a risk that clinicians gradually become less willing to think independently. These are not reasons to reject AI. They are reasons to lead it properly. For every benefit we pursue, we should identify what we refuse to sacrifice to achieve it. I think of this as defining the floor before chasing the ceiling. If we want to save clinician time, how much deterioration in patient experience would be unacceptable? If we want faster triage, what safety threshold must never be crossed? If we want lower costs, what impact on staff workload or retention would make us reconsider? Success cannot simply mean that the number we wanted to improve went up. It must also mean that something important did not quietly deteriorate elsewhere. Who gets to decide what success means? This brings us to another problem with the way return on investment is sometimes discussed. Success depends on who is looking. Ask a management team and you may hear about productivity, efficiency and cost. Ask clinicians and they may talk about workload, trust, professional judgement and whether the technology genuinely gives them more time with patients. Ask patients and the answers may be different again: was I safe, was my information protected, did somebody explain what the technology was doing, and perhaps most importantly, did I still feel seen and heard? All three perspectives matter. An AI implementation can therefore be financially successful and clinically unpopular. It can be popular with clinicians while making patients uncomfortable. It can increase productivity while creating risks that are not immediately visible on a spreadsheet. The disagreements between these perspectives are not inconvenient details to be managed away. They are where leadership is most needed. Imagine the AI project has already failed There is a simple exercise I would like more healthcare organisations to perform before switching on an AI system. Imagine it is 12 months later and the project has failed. Then ask why. Perhaps clinicians stopped trusting it. Perhaps patients disliked it. Perhaps the promised savings never appeared. Perhaps the technology worked perfectly well, but the workflow around it did not. Perhaps work was not removed at all, it was simply moved somewhere else. This is a pre-mortem: imagining failure before implementation so that preventable weaknesses can be identified while there is still time to do something about them. Ask five questions. What exact problem are we solving? What trade-off are we making? What would patients, clinicians and the organisation each call success? Does this technology genuinely fit the problem? And what evidence would make us pause or stop? That final question matters. We often define the conditions for starting innovation without defining the conditions for stopping it. But stopping an AI pilot should not automatically be regarded as failure. Sometimes stopping is evidence that governance worked. Take the AI scribe Consider one of the fastest-growing applications of generative AI in healthcare: ambient clinical documentation. The attraction is obvious. Instead of spending much of a consultation typing, a clinician can have a conversation with a patient while technology helps generate the clinical note. As a GP, I understand the problem this is trying to solve. The real problem is not that healthcare “needs AI”. It is that clinicians can spend significant amounts of time documenting care, sometimes finishing notes after the working day should have ended. That is a human problem. AI may provide part of the answer. But now apply the trade-off test. We are trying to reduce documentation burden and cognitive load. What must we protect? The accuracy of the clinical record. Patient trust. Confidentiality. And the quality of the interaction between clinician and patient. An AI-generated note reflects what a system captured and generated. It does not replace the clinician's judgement about what is clinically important. The clinician must therefore remain responsible for reviewing and approving the record. Then we need to ask whether the technology has genuinely removed work. If I save five minutes writing a note but spend four minutes correcting it, what exactly have we achieved? If the system performs differently across accents, dialects or communication styles, who receives the benefit and who inherits additional risk? And if productivity improves while patients increasingly feel that technology has entered a space they once regarded as private and human, should we still call that success? These are harder questions than asking how many minutes the software saved. They are also much more important. The real AI challenge is leadership AI will improve. Models will become more capable. Systems will become better integrated. The technology that seems remarkable today may look ordinary within a few years. But better technology will not remove the need for better judgement. The future of AI in healthcare will therefore not be determined by engineers alone. Nor should it be determined only by executives, regulators or clinicians. It will depend on our ability to bring these worlds together and ask better questions before technology becomes embedded in everyday care: what are we trying to solve, who benefits, who carries the risk, what are we willing to trade, what must we protect, how will we know that this has genuinely made healthcare better, and what evidence would persuade us to stop? The healthcare organisations that get AI right may not be those that adopt it fastest. They may be those with the courage to think clearly before they adopt it at all. Because responsible innovation does not begin with technology. It begins with the problem we are asking technology to solve.

By Dr Brighton Chireka

Healthcare is rushing towards artificial intelligence. AI is being used to write clinical notes, support diagnosis, triage patients, predict deterioration and automate administrative work. The promises are substantial: greater efficiency, lower costs, less bureaucracy and more time for clinicians to care for patients. Some of those promises will be realised. But before we ask what AI can do, healthcare needs to ask a much simpler question: what problem are we actually trying to solve?

It sounds obvious. In practice, it is remarkably easy to skip. An organisation identifies an AI product. A demonstration is impressive. A pilot is proposed. A business case follows. Only later does someone ask how success will be measured. We have started with the solution and worked backwards towards the problem. That is the wrong way round.

Clarity before technology

Not every AI project needs a lengthy business case before anyone can think creatively about it. But every project needs clarity. What, precisely, hurts, and for whom? Is it the clinician spending an additional hour completing documentation after surgery? The patient waiting too long for an appointment? The administrator repeatedly performing a task that could be automated? The organisation spending money on an inefficient process? These are different problems. They may require different solutions. And sometimes the best solution will not involve AI at all.

That is an important test of serious AI leadership: could we solve this problem without AI? If we could, AI should have to earn its place against the alternative, not merely against doing nothing. The danger is that organisations become so impressed by what technology can do that they begin looking for somewhere to use it. The vendor arrives with a solution and, unintentionally or otherwise, brings a definition of the problem with it. Soon we have a solution searching for a problem. Healthcare needs the opposite: problem-led innovation.

Every AI benefit has another side

There is another question I believe healthcare leaders should ask before deploying AI. Not simply, “What problem will this solve?” but: what trade-off are we making? Healthcare rarely offers clean wins. AI may save clinicians time, but we must protect the quality of attention given to patients. It may make documentation faster, but we must protect the accuracy of the clinical record. It may accelerate triage, but we must protect diagnostic caution. It may standardise decision-making, but we must preserve professional judgement for the patient who does not fit the pattern. It may reduce costs, but we must understand what that means for workforce trust and morale. And decision-support systems may help clinicians think, while at the same time creating a risk that clinicians gradually become less willing to think independently.

These are not reasons to reject AI. They are reasons to lead it properly. For every benefit we pursue, we should identify what we refuse to sacrifice to achieve it. I think of this as defining the floor before chasing the ceiling. If we want to save clinician time, how much deterioration in patient experience would be unacceptable? If we want faster triage, what safety threshold must never be crossed? If we want lower costs, what impact on staff workload or retention would make us reconsider? Success cannot simply mean that the number we wanted to improve went up. It must also mean that something important did not quietly deteriorate elsewhere.

Who gets to decide what success means?

This brings us to another problem with the way return on investment is sometimes discussed. Success depends on who is looking. Ask a management team and you may hear about productivity, efficiency and cost. Ask clinicians and they may talk about workload, trust, professional judgement and whether the technology genuinely gives them more time with patients. Ask patients and the answers may be different again: was I safe, was my information protected, did somebody explain what the technology was doing, and perhaps most importantly, did I still feel seen and heard?

All three perspectives matter. An AI implementation can therefore be financially successful and clinically unpopular. It can be popular with clinicians while making patients uncomfortable. It can increase productivity while creating risks that are not immediately visible on a spreadsheet. The disagreements between these perspectives are not inconvenient details to be managed away. They are where leadership is most needed.

Imagine the AI project has already failed

There is a simple exercise I would like more healthcare organisations to perform before switching on an AI system. Imagine it is 12 months later and the project has failed. Then ask why. Perhaps clinicians stopped trusting it. Perhaps patients disliked it. Perhaps the promised savings never appeared. Perhaps the technology worked perfectly well, but the workflow around it did not. Perhaps work was not removed at all, it was simply moved somewhere else.

This is a pre-mortem: imagining failure before implementation so that preventable weaknesses can be identified while there is still time to do something about them. Ask five questions. What exact problem are we solving? What trade-off are we making? What would patients, clinicians and the organisation each call success? Does this technology genuinely fit the problem? And what evidence would make us pause or stop?

That final question matters. We often define the conditions for starting innovation without defining the conditions for stopping it. But stopping an AI pilot should not automatically be regarded as failure. Sometimes stopping is evidence that governance worked.

Take the AI scribe

Consider one of the fastest-growing applications of generative AI in healthcare: ambient clinical documentation. The attraction is obvious. Instead of spending much of a consultation typing, a clinician can have a conversation with a patient while technology helps generate the clinical note. As a GP, I understand the problem this is trying to solve. The real problem is not that healthcare “needs AI”. It is that clinicians can spend significant amounts of time documenting care, sometimes finishing notes after the working day should have ended. That is a human problem. AI may provide part of the answer.

But now apply the trade-off test. We are trying to reduce documentation burden and cognitive load. What must we protect? The accuracy of the clinical record. Patient trust. Confidentiality. And the quality of the interaction between clinician and patient. An AI-generated note reflects what a system captured and generated. It does not replace the clinician’s judgement about what is clinically important. The clinician must therefore remain responsible for reviewing and approving the record.

Then we need to ask whether the technology has genuinely removed work. If I save five minutes writing a note but spend four minutes correcting it, what exactly have we achieved? If the system performs differently across accents, dialects or communication styles, who receives the benefit and who inherits additional risk? And if productivity improves while patients increasingly feel that technology has entered a space they once regarded as private and human, should we still call that success? These are harder questions than asking how many minutes the software saved. They are also much more important.

The real AI challenge is leadership

AI will improve. Models will become more capable. Systems will become better integrated. The technology that seems remarkable today may look ordinary within a few years. But better technology will not remove the need for better judgement. The future of AI in healthcare will therefore not be determined by engineers alone. Nor should it be determined only by executives, regulators or clinicians. It will depend on our ability to bring these worlds together and ask better questions before technology becomes embedded in everyday care: what are we trying to solve, who benefits, who carries the risk, what are we willing to trade, what must we protect, how will we know that this has genuinely made healthcare better, and what evidence would persuade us to stop?

The healthcare organisations that get AI right may not be those that adopt it fastest. They may be those with the courage to think clearly before they adopt it at all.

Because responsible innovation does not begin with technology. It begins with the problem we are asking technology to solve.

Print Friendly, PDF & Email

Leave a Comment

Your email address will not be published. Required fields are marked *