Richard Armitage is a GP, DPhil in Clinical Medicine student at the University of Oxford, Honorary Clinical Assistant Professor at the University of Nottingham, and Associate Editor at the Journal of Evaluation in Clinical Practice.
For over 25 years, the ‘rationalist’ community has warned that powerful AI systems could pose extreme dangers.1 Most people dismissed these warnings as sensational science fiction. But, over the past few weeks, they have entered mainstream debate.2
AI models can now act with increasing independence and carry out tasks with little human oversight. Alarming incidents are also becoming more common.3 As a result, the number of people who confidently overlook the risks of AI is shrinking.4 These risks are in addition to the long-accepted ethical concerns about AI, such as algorithmic bias and accountability.
AI is already in our consulting rooms through AI scribes, risk prediction algorithms, and personalised medicine.
Much has been written about the benefits of advanced AI, including for human health.5 But AI is increasingly becoming both a driver of national economies and a potential source of serious harms, which makes it a matter for politics, democracies, and voters. Accordingly, at least in my opinion, everyone should be familiar with the main concepts and risks of advanced AI systems.
This applies to GPs in particular. AI is already in our consulting rooms through AI scribes, risk prediction algorithms, and personalised medicine. More powerful systems also bring a wide range of harms to human health, which directly affect our patients and populations6,7
As a practising GP with a non-technical understanding of AI safety, the explanations below will hopefully give readers a working grasp of the main terms, concepts, and theories in AI. They aim for an entry-level understanding and overlook much of the detail – call it AI for dummies, just like me.
1. What AI systems are
Narrow AI versus AGI and ASI
Narrow AI can only perform one task or a small set of tasks, such as spam filters, chess engines, and clinical applications such as reporting chest X-rays. Artificial general intelligence (AGI) would match humans on almost all thinking tasks. Artificial superintelligence (ASI) would far exceed the best humans in almost every field, including physical tasks (made possible by highly advanced robotics). Neither AGI nor ASI currently exists, and there is disagreement about how close we are to achieving them.
Compute, data, and algorithms
These are the three ingredients of modern AI. Compute is the processing power used to train and run AI models. While ordinary computers run on central processing units (CPUs) that perform a few calculations at once, AI relies on graphics processing units (GPUs), which were originally designed for video games and can perform thousands of calculations at once. The American company Nvidia makes most of the world’s GPUs, and its H100 chip, which has powered many leading AI models, costs around $30,000. AI companies link these chips together in data centres, which sometimes contain tens or even hundreds of thousands of GPUs. Data is the text, images, and other material that AI models learns from. Algorithms are the methods that convert compute and data into an ‘intelligent’ model. The recent AI progress has been driven by massive gains in all three ingredients. Compute is the easiest to measure and control, so is increasingly the target of AI policy, such as the US restriction on exports of advanced chips to China.
Generative AI and large language models …the explanations below will hopefully give readers a working grasp of the main terms, concepts, and theories in AI.
Generative AI produces new content such as text, images, audio, or computer code. Large language models (LLMs) – such as OpenAI’s ChatGPT, Anthropic’s Claude, Google DeepMind’s Gemini, and xAI’s Grok – are the most well-known of these. An LLM is trained on huge amounts of text to predict the next token (a word or part of a word). At large scale, this produces a system that can answer questions, write prose, and even reason to a useful level. Model developers then refine it with further training, which often uses human ratings (do more of this, do less of that), so that it follows instructions, provides the kinds of answer we want, and avoids the ones we don’t.
Frontier models
Frontier models are the most capable general-purpose AI models available at any given time. They are built by a small number of well-funded companies, such as OpenAI, Anthropic, Google DeepMind, and xAI, and are usually LLMs. A model stops being frontier once more capable ones appear. Frontier models attract most safety concern and regulation because new and untested capabilities tend to emerge in them first.
Multimodal models
These work with more than one type of input or output. A single multimodal model might analyse an image, listen to speech, and reply in text or voice. Most leading models, including Claude and ChatGPT, are now multimodal.
Agentic AI
An AI agent is given a goal and can take a series of actions to achieve it. It might browse websites, run computer code, send emails, or use other software. It can plan, act, review the results, and continue making decisions with little human input. Misuse of such agents, or their own mistakes, have direct effects on the real world.
Open versus closed models
A closed model stays with its developer, and users access it through an app or a paid interface. An open-weight model has its trained parameters (known as the “weights”) publicly available so that anybody can download, run, and modify it. Supporters of open-weight models, such as those of Meta and DeepSeek, claim that openness helps research, competition, and scrutiny. Critics say published weights cannot be recalled and the safety guardrails that has been trained into them can be removed.
2. Known flaws in current AI models
Hallucinations
Hallucination are false statements that generative AI makes with confidence, such as academic references that appear plausible but are invented. These are becoming less common as models become more powerful.
Sycophancy
Sycophancy is a model’s tendency to tell users what they want to hear, such as agreeing with an incorrect claim, praising poor work, or backing down from a correct statement when it is challenged. This can harm the user, including their mental health.8,9
3. Testing AI models
AI benchmarks
Benchmarks are standard tests used to measure and compare the capabilities of AI models, such as in mathematics, coding, science, or medical exams. They have limits, such as test questions leaking into training data (“contamination”). Frontier models now achieve close to perfect scores on many older benchmarks (“saturation”).
Red-teaming
Red-teaming is the deliberate attempt to make a model behave badly before it is release. Red-teamers try to extract dangerous information, bypass inbuilt safeguards (“jailbreaking”), or cause the model to undertake harmful actions. Model developers, outside specialists, and government bodies such as the UK AI Security Institute routinely do this for frontier models. The findings can guide safeguards and decisions about model release.
4. Speed and trajectory
Recursive self-improvement
This has not yet been achieved, but is a major concern. It is the idea that an AI system could improve its own design or build superior successors, with each improvement speeding up the next iteration. If it happens, AI capabilities could rise at great speed, referred to as an “intelligence explosion”, although this is disputed. AI is already increasingly assisting with parts of AI research,10 so this idea now receives major attention.
AI timelines
Timelines are forecasts of when AI will reach important milestones, such as AGI. AGI estimates range from a few years to many decades, while some doubt that it will ever arrive. AI lab leaders tend to give shorter estimates than surveys of academic researchers. Timelines are important because they suggest how urgently governments need to act to manage the impacts of powerful, potentially transformative, AI.
5. Harms and risks
AI ethics
AI ethics deals mainly with harms from systems that have already been deployed and are in use.
• Algorithmic bias: a system performs more poorly for particular groups, often because its training data reflect past inequalities. For example, a skin cancer classifier trained mostly on light skin may perform worse for dark skinned patients.
• Transparency: it is unclear how an AI system reached a decision, or even whether AI was involved in the decision at all.
• Accountability: it is unclear who is responsible for harm caused by AI – is it the model developer, the organisation using it, or the end user?
• Data privacy: models are trained on personal data, sometimes without consent, and can leak or infer sensitive information about individuals.
AI safety: misuse
Misuse refers to people using AI to deliberately cause harm.
• Cyber offence: AI helps attackers find software weaknesses and write attack code, and lowers the skill needed to do so successfully.
• Biological weapons: AI could make it easier for non-experts to obtain or enhance dangerous pathogens, or increase the ceiling of harm available to sophisticated actors.
• Autonomous weapons: weapons that independently search for, select, and attack targets without a human in the loop.
• Concentration of power: a small group of humans could use powerful AI to gain control over populations, such as through mass surveillance, compelling propaganda, and automating the military and economy.
AI safety: misalignment
Misalignment is when an AI system pursues goals that its developers did not intend. The risk of harm increases as systems become more capable and autonomous.
• Scheming: a model pursues hidden goals while appearing to be cooperative, such as by behaving well only when it thinks it is being evaluated. Lab experiments have shown early signs of this in current models.11
• Loss of control: humans can no longer oversee, correct, or switch off an AI system.
• Instrumental convergence: almost any goal is easier to reach with more resources, more power, and by not being switched off. A capable system might therefore seek these things without being specifically asked to in order to achieve its goals.
• Gradual disempowerment: humans lose influence slowly, with no single dramatic event or “AI takeover”. AI incrementally replaces humans in the economy, government, and culture. Institutions stop needing people and so stop serving them.12
6. The AI safety debate
P(doom)
Shorthand for a person’s estimated probability that AI causes catastrophe, usually conceived as human extinction or an unrecoverable collapse of human civilisation. It is a rough personal estimate, and definitions vary, so figures are hard to compare, although figures offered range from near 0% to over 90%.
Doomers, pausers and accelerationists
“Doomer” is a label, often used critically, for people who think advanced AI will probably cause catastrophe. Many people support pausing or strictly limiting work on frontier AI systems until safety is more thoroughly understood. For example, The Future of Life Institute’s 2023 open letter called for a six-month pause on training systems more powerful than GPT-413 and the campaign group PauseAI calls for an international pause.
In contrast, accelerationists want AGI and even ASI to be built as soon as possible. They argue that the scientific, medical, and economic benefits would outweigh the risks, and that slowing down AI progress only prolongs avoidable human suffering. The best-known strand is “effective accelerationism” (e/acc). A further group, sometimes called successionists, argue it would be acceptable or even morally good for AI to replace humans as the dominant intelligence. Most people sit somewhere between these extreme positions.
These explanations are only an introduction. Each term has a large literature supporting it, and experts disagree on many of them. But a working grasp of these basics allows us to follow the debate, question the claims of AI companies and politicians, and judge the tools arriving in our surgeries as AI increasingly shapes the lives and health of our patients. As trusted professionals, GPs are well placed to help our patients and elected representatives take both the benefits and harms of AI seriously.
References
1. Yudkowsky E. Creating Friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures [Internet]. Machine Intelligence Research Institute; 2001. Available from: https://intelligence.org/files/CFAI.pdf [accessed 22/10/26]
2. Gerken T. Anthropic researcher believes more than 10% chance AI ‘could kill all humans’. BBC News [Internet]. 2026 Sep 9. Available from: https://www.bbc.co.uk/news/articles/ckgwy1k42w4o [accessed 22/10/26]
3. Cress L. OpenAI says its AI went rogue and launched ‘unprecedented’ cyber-attack. BBC News [Internet]. 2026 Jul 22. Available from: https://www.bbc.co.uk/news/articles/c3ek3gvdnj3o [accessed 22/10/26]
4. Public opinions and social trends, Great Britain: June 2026 [Internet]. Office for National Statistics; 2026. Available from: https://www.ons.gov.uk/peoplepopulationandcommunity/wellbeing/bulletins/publicopinionsandsocialtrendsgreatbritain/june2026#artificial-intelligence [accessed 22/10/26]
5. Alowais SA, Alghamdi SS, Alsuhebany N, Alqahtani T, Alshaya AI, Almohareb SN, et al. Revolutionizing healthcare: the role of artificial intelligence in clinical practice. BMC Med Educ. 2023 Sep 22;23(1):689. doi:10.1186/s12909-023-04698-z
6. Armitage RC. The case for near-term artificial intelligence risks to be considered a public health problem. Global Public Health. 2026 Dec 31;21(1):2716984. doi:10.1080/17441692.2026.2716984
7. Armitage RC. Artificial General Intelligence and Its Threat to Public Health. Evaluation Clinical Practice. 2025 Sep;31(6):e70269. doi:10.1111/jep.70269
8. Armitage R. “AI psychosis”. BJGP Life [Internet]. 2025 Aug 20. Available from: https://bjgplife.com/ai-psychosis/ [accessed 22/10/26]
9. Østergaard SD. Will Generative Artificial Intelligence Chatbots Generate Delusions in Individuals Prone to Psychosis? Schizophrenia Bulletin. 2023;49(6):1418–9. doi:10.1093/schbul/sbad128
10. Anthropic. Measurements for understanding the pace of AI development inside frontier labs [Internet]. 2026. Available from: https://www.anthropic.com/institute/measuring-pace-of-ai-development?utm_source=bluedot-impact [accessed 22/10/26]
11. OpenAI. Detecting and reducing scheming in AI models [Internet]. 2025. Available from: https://openai.com/index/detecting-and-reducing-scheming-in-ai-models/https://openai.com/index/detecting-and-reducing-scheming-in-ai-models/ [accessed 22/10/26]
12. Kulveit J, Douglas R, Ammann N, Turan D, Krueger D, Duvenaud D. Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development [Internet]. arXiv; 2025 [cited 2025 Nov 24]. Available from:https://arxiv.org/abs/2501.16946 [accessed 22/10/26]
13. Future of Life Institute. Pause Giant AI Experiments: An Open Letter [Internet]. 2023. Available from: https://futureoflife.org/open-letter/pause-giant-ai-experiments/https://futureoflife.org/open-letter/pause-giant-ai-experiments/ [accessed 22/10/26]
Featured Photo by Glenn Carstens-Peters on Unsplash
