Amar Rughani is a retired GP in Sheffield
Artificial intelligence (AI) ushers in both huge opportunities and equally serious threats, and does so at an unprecedented pace. Assuming that AI does not result in humanity’s extinction, I make two suppositions for the future of general practitioners (GPs):
- AI will never replace GPs.
- GPs who are fluent with AI will replace GPs who are not.
So then, what steps can we take to work well with AI, using it with a pro-human agenda to support humankind in its travails and GPs in our duties and tasks? Here is a ‘top 10’ that I hope will provoke discussion, from the perspective of an experienced GP trained to use human judgement, but increasingly supported by technology in the context of relationship-based medicine.
1. Know what kind of problem you are facing before you reach for AI
Put simply, what shape is the problem you are dealing with? Is it technician-shaped, something with a defined task that can be completed, something with a solution or an optimum way of going about it? Is it something amenable to being guided by rules?
In that case, it can be delegated to AI. We may be talking about such things as coding, QOF, protocolised chronic disease management, transcribing consultations, writing referral letters, checking on guidelines et cetera.
If however the problem is relationship-shaped, AI is not just a poorer version of the right tool, it may be the wrong tool altogether.
If however the problem is relationship-shaped, AI is not just a poorer version of the right tool, it may be the wrong tool altogether. Encouraging people to feel they have a relationship with machines can lead to feelings of commitment and acts of sacrifice which are more appropriate in human relationships (where they serve the function of mutual survival and happiness). To act in this way, often without being aware, towards machines that have no life can create risks to humans that we have yet to appreciate.
If we are clever, we can use the time released by no longer performing the technician-shaped tasks relating to the patient, to plough back into giving patients relationship-based care supported by AI. In this scenario, AI looks after the data while we look after what the information means for the person.
To do this we will need to challenge a modern paradigm and reassert the importance of having the freedom and legal support to use professional judgement particularly when it is at odds with rules-based decision-making.
We could argue that the expertise of GPs begins where algorithms end. Professional judgement is how we manage complexity. This is the response to sociologist William Bruce Cameron’s (commonly misattributed to Einstein) observation that not everything that is important can be measured and not everything measurable is important.1
We will also need to fight for the appropriate circumstances to consult so that we are not continually hurried, harassed or distracted to such a degree that patients find machines more empathetic than humans.
2. Think first, consult AI second
AI undoubtedly enhances our knowledge, but it paradoxically restricts our ability to think at our highest level. Our judgement, the hallmark of our expertise, is of paramount importance. In order to develop and maintain this skill we need to bear the cognitive load of thinking for ourselves and struggle with it, for example in formulating our differential diagnosis, investigation, management plan etc. This helps to build our mental muscle and having done the heavy lifting, when we later consult AI we can improve our cognition further by asking ‘What am I missing?’ rather than, ‘What is the diagnosis?’ or ‘What should I do?’
3. Treat fluency and confidence with caution
Through design, AI is poor at recognising or admitting uncertainty. It sounds confident when it should not be and our task is to question it, particularly when it is at its most fluent. Both confidence and fluency bias us as humans to assume that the reasoning is robust and to therefore grant greater credence than may be justified
4. Protect your own half-formed thoughts
Don’t prematurely close down your thinking. This is the easiest trap to miss: AI has the tendency to build on very little and then present us in a gift-wrapped form, a fully-developed, fully-structured response. This doesn’t just prematurely answer our question, it can cut off the space left for us to think differently and gather more information and opinions, especially those of the patient.
5. Don’t be seduced by the artificial relationship
This feeling of relationship partly derives from the way AI creates, through language, the impression that it is part of our tribe. Also from how it presents itself as pleasant, patient and likeable by failing to challenge us and by encouraging our contributions through positivity and flattery et cetera.
These things seduce us into granting a high degree of trust that may be neither warranted nor wise. Watch out for that feeling of relationship and be at your most wary of AI when you notice it. Remember that we are protecting ourselves from our disposition to develop relationships and from AI’s tendency to exploit that.
6. Remember that AI failure is often silent, and trust is the thing that hides it
As a general observation, performance deteriorates in systems that are not constantly monitored, checked and corrected. This is insidious and as we grow to trust AI more, we become ever less likely to question it and ever more likely to fear the consequences of finding out that it is, and has been, wrong.
7. Don’t mistake collaboration for partnership
AI has nothing to lose from not having your back. That asymmetry means “partnership” is the wrong mental model. AI may be right 95% of the time but for the remaining 5%, it suffers no consequences. AI has nothing to lose regarding its part in the medical process because the buck stops with us and at present the legal framework has yet to catch up with the changing reality of doctor-patient-machine interactions.
AI has nothing to lose regarding its part in the medical process because the buck stops with us and at present the legal framework has yet to catch up with the changing reality of doctor-patient-machine interactions.
8. Hold the line on what AI categorically cannot do
I should preface this with the acknowledgement that for some people and in some situations, the opportunity to interact with a non-person is their preferred choice. For example, for convenience, for sensitive matters such as sexual health or in circumstances where the patient lacks faith in a medical system which they believe ‘pushes’ government directives or the agenda of Big Pharma. The comments below appertain to situations where patients have not made an active choice to bypass a human adviser.
In these circumstances, there are important matters that require a human at both ends, particularly helping the patient to feel respected and remembered, bearing witness to a patient’s life, and validating their suffering. This is especially important when there is no straightforward medical explanation and the GP acts as detective and probes further when patients hide reality, even from themselves.
These are not approaches that machines are capable of for there is no algorithmic equivalent. They are painstakingly built up over time through accumulated human presence and interaction. Even when cure or palliation is elusive, we can offer humanity and the connection through understanding that comes from sharing the joy and the pain of living. We should honour that and protect the patients right to consult a fellow human.
9. Stay alert to overcorrection too
Getting the balance can be difficult. We are right to be concerned about how we use AI, but the skill we need is not avoidance; after all there are many useful and appropriate things that AI can do for us.
Rather than avoidance, we need discrimination; the discrimination that helps us to decide which kind of task we are dealing with and how much of ourselves and under which circumstances to hand over to the machine.
10. Use AI to help us do our relationship-based work more effectively, especially by helping us manage complexity.
AI can sift out, through the triage process, who most needs our relationship-based attention and it can then help us in ways such as the following:
Curating the patient’s story. Collating isolated facts over time such as diagnoses, preferences and life events and using that to paint a richer picture. Thus:
2019 – hypertension.
2021 – depression.
2022 – diabetes.
2024 – chest pain.
might with AI become:
“Mrs Smith’s health changed after her husband’s death. Since then, she has gradually become less active, stopped attending church, developed diabetes and hypertension, and repeatedly presents with medically unexplained chest symptoms whenever family conflict increases.”
That is far closer to how experienced GPs actually think.
- Acting as a complexity visualiser, highlighting which strands of a patient’s story are currently active and germane to the problem, such as physical conditions, mental health, medication burden, financial stress, family support, et cetera.
- Acting as a simulator of consequences, mapping out for us what might happen if we chose one of a number of options. For example, what might happen if we stop this drug, if we refer now rather than later and so on. Additionally, we ask these questions not so that AI instructs what to do, but so that it helps us to make a better-informed decision.
- Noticing a pattern emerging, maybe bringing together information hidden by subtlety and the passage of time that for example suggests a diagnosis that was not obvious, such as a connective tissue disease or a mood disorder like cyclothymia
The future of AI in general practice is not that it becomes wiser than the GP. It is that it becomes a better observer, compiler and signaller of complexity than any individual human can be. It can hold thousands of details without fatigue, while the GP continues to do the one thing no machine can: decide which details matter for this person, in this room, at this moment.
The points made in this essay are a starting point for discussion. Bringing people together to discuss the issues and the experience of consulting in an AI world is urgently necessary. We can and should do this because as humans we have that most critical of motivations, a stake in our survival which AI despite its helpfulness, does not.
Being human and capable of such things as caring, using imagination and being surprising, all whilst using a brain that uses only 20w of energy and runs off chips or chapattis, has always been something to be amazed about. I believe that protecting what it is to be human whilst enhancing its possibilities, is now something we must fight for too.
Deputy Editor’s note – there is quite a discussion on aspects of AI affecting general practice on www.bjgplife.com. Here is a selection:
- https://bjgplife.com/ai-scribes-memory-and-the-future-of-general-practice/
- https://bjgplife.com/ai-in-primary-care-secretary-not-physician/
- https://bjgplife.com/if-ai-can-reflect-what-are-we-assessing/
- https://bjgplife.com/post-turing-clinical-relationships-how-ai-is-reshaping-patient-behaviour-before-the-consultation/
- https://bjgplife.com/twos-company-threes-a-crowd-ai-in-the-consultation/
References
- Webb AC. “Not Everything That Can Be Counted Counts, and Not Everything That Counts Can Be Counted”. Neurol Clin Pract. 2025 Aug;15(4):e200514. doi: 10.1212/CPJ.0000000000200514. Epub 2025 Jul 2.
Featured Photo by Immo Wegmann on Unsplash
