Artificial intelligence is no longer a distant prospect in healthcare — it is already embedded in radiology workflows, clinical decision support tools, medicines management platforms, and patient-facing digital therapeutics. For those of us who have spent careers at the intersection of clinical practice and digital health governance, this acceleration is both exciting and sobering.
As a pharmacist, I understand what is at stake at the point of care. Medicines optimization, allergy checking, dose calculation, and drug interaction screening are high-stakes, time-pressured activities where errors carry real consequences. As a healthcare AI strategist, I can see the transformative potential of machine learning to augment clinical judgement, reduce cognitive load, and surface insights that no individual clinician could process alone. But I also recognize that the speed of AI adoption has, in many settings, outpaced the governance frameworks designed to protect patients.
The central challenge facing healthcare organizations today is not whether to adopt AI, that debate is largely settled. The real question is: how do we adopt it responsibly? How do we move from enthusiasm to evidence, from deployment to accountability, and from innovation to sustained safety?
The answer, I would argue, lies in structured governance, and specifically in frameworks such as ISO/IEC 42001, the international standard for AI management systems.
The Promise and the Peril
The case for AI in healthcare is well-documented. Predictive algorithms are identifying sepsis earlier than conventional screening tools. AI-assisted diagnostics are matching or exceeding specialist accuracy in detecting diabetic retinopathy, skin cancer, and pulmonary nodules. Natural language processing is extracting structured intelligence from unstructured clinical notes, enabling risk stratification at scale. In pharmacy, AI is being piloted for automated medicines reconciliation, clinical trial matching, and the prediction of adverse drug reactions in complex polypharmacy cases.
Yet alongside these advances sit uncomfortable realities. Algorithmic bias, where AI systems trained on non-representative datasets perpetuate inequities in care, has been documented across multiple healthcare settings. Opacity in AI decision-making creates accountability gaps that are inconsistent with the duty of candor expected of healthcare professionals. And the regulatory landscape, while evolving, has not kept pace with the volume or velocity of AI being embedded into clinical practice.
From a pharmacist’s perspective, the risk is particularly acute in medicines management. AI tools advising on dosing or flagging interactions must not become a source of false reassurance. The clinical professional remains the accountable practitioner. Any AI system that erodes that professional judgement, rather than augmenting it, is a patient safety risk, not a solution.
This is precisely why governance matters. Not as bureaucratic overhead, but as the structural scaffolding that allows innovation to proceed safely.
ISO/IEC 42001: A Framework Built for This Moment
ISO/IEC 42001:2023 is the world’s first international standard for artificial intelligence management systems (AIMS). Published by the International Organization for Standardization (ISO), it provides a comprehensive framework for organizations that develop, provide, or use AI systems; a category that now encompasses virtually every significant healthcare provider.
The standard draws on the familiar architecture of ISO management system standards — Plan, Do, Check, Act — and applies it specifically to the governance of AI. Its scope covers AI policy, risk management, impact assessment, transparency, human oversight, and continual improvement. For healthcare organizations navigating the dual pressures of innovation and regulation, ISO/IEC 42001 is not simply another compliance checkbox. It is a practical tool for building AI governance that is proportionate, auditable, and aligned with patient safety obligations.
AI Risk Assessment as a Clinical Safety Discipline
One of the most consequential contributions ISO/IEC 42001 makes to healthcare is its treatment of AI risk assessment as a structured, ongoing process. The standard requires organizations to identify and evaluate the risks and opportunities presented by their AI systems, considering both the likelihood and impact of potential harms. In a healthcare context, this maps directly to the clinical risk management principles already embedded in DCB0129 and DCB0160 — the NHS Digital standards governing clinical safety of health IT systems.
As a Clinical Safety Officer, I have seen organizations deploy AI tools without understanding their failure modes, their training data provenance, or their behavior at edge cases. ISO/IEC 42001 provides a disciplined approach to asking these questions before deployment, not after an incident has occurred.
Human Oversight and the Role of the Clinician
ISO/IEC 42001 places significant emphasis on human oversight; the principle that AI systems, particularly in high-risk domains, must operate under meaningful human control. For pharmacists and other prescribing clinicians, this is not a new concept. It is the professional and regulatory expectation that has always existed. What ISO/IEC 42001 adds is a formal governance requirement for organizations to design and document the mechanisms by which human oversight is exercised, maintained, and reviewed.
This is more nuanced than it may first appear. As AI systems become more sophisticated and autonomous, the quality of human oversight can paradoxically decline. Alert fatigue, automation bias, and inadequate training all erode the effectiveness of the clinician as a safety backstop. ISO/IEC 42001 challenges organizations to ensure that human oversight remains substantive, not performative.
Transparency and Explainability
ISO/IEC 42001 also mandates that organizations address the transparency of their AI systems — that those affected by AI-assisted decisions can understand, at an appropriate level, how those decisions were reached. In a clinical setting, this has direct implications for informed consent, shared decision-making, and professional accountability. A pharmacist advising a patient on a treatment option informed by AI must be able to explain that recommendation in clinically meaningful terms. Black-box outputs are not acceptable at the point of care.
Continual Improvement and Post-Market Surveillance
Perhaps most pertinent to healthcare, ISO/IEC 42001 requires organizations to monitor AI system performance continuously and to act on what they find. This mirrors the post-market surveillance requirements already applied to medical devices and software as a medical device (SaMD) under the Medical Device Regulation framework. For AI tools operating in clinical environments, performance degradation, whether due to dataset drift, changes in clinical practice, or evolving patient populations, is a patient safety issue. ISO/IEC 42001 provides the governance structure to detect and address it systematically.
Connected Devices, Data Governance, and ISO/IEC 27400
The deployment of AI in healthcare does not occur in isolation. Increasingly, AI systems are embedded within or dependent upon connected health technologies; wearable devices, remote patient monitoring platforms, smart infusion pumps, and telehealth infrastructure. This creates a data governance dimension that no healthcare AI strategy can responsibly ignore.
ISO/IEC 27400:2022 — the standard for IoT security and privacy — provides guidance on the protection of personal data generated, transmitted, and processed across connected health devices. In a healthcare context, where IoT endpoints may be generating continuous streams of sensitive patient data that feed directly into AI systems, the integrity of that data pipeline is a patient safety issue as much as it is a cybersecurity concern. Corrupted, intercepted, or manipulated sensor data could produce clinically dangerous AI outputs. ISO/IEC 27400, deployed alongside ISO/IEC 42001, strengthens the end-to-end governance architecture within which healthcare AI operates, ensuring that the data quality and security underpinning AI decision-making meets the standard that clinical environments demand.
Governance as an Enabler, Not a Barrier
A concern I encounter regularly in conversations with NHS trusts, digital health startups, and HealthTech investors is that governance frameworks will slow down innovation. The argument runs that in a sector where AI could save lives at scale, the cost of regulatory delay is measured in patient outcomes.
I understand the urgency. But I would challenge the premise. Governance frameworks, properly implemented, are not the enemy of innovation, they are its precondition. An AI tool deployed without structured risk assessment, human oversight mechanisms, and performance monitoring is not an innovation. It is a liability. The organizations that will sustain long-term impact in healthcare AI are those that can demonstrate to regulators, commissioners, clinicians, and patients that their systems are safe, effective, and continuously improving.
ISO/IEC 42001 provides the language and the structure to make that demonstration. It creates a common vocabulary for AI governance that spans developers, deployers, and users — bridging the gap between the technical teams building AI systems and the clinical professionals deploying them at the bedside. In a sector characterized by fragmented supply chains, complex procurement relationships, and multi-stakeholder accountability, that shared framework has genuine practical value.
For healthcare organizations seeking to build trust with their clinical workforce, their patients, and their regulators, ISO/IEC 42001 certification is increasingly a credible signal of intent. It is not a guarantee of safe AI. But it is evidence of a serious commitment to the governance processes from which safe AI emerges.
A Call to Action for Healthcare Leaders
The maturation of AI in healthcare will not be determined by the quality of the algorithms alone. It will be determined by the quality of the governance surrounding them. Clinical leaders, digital health officers, and healthcare executives all have a role to play in ensuring that AI is adopted not just rapidly, but responsibly.
For pharmacists and clinical safety professionals, this means engaging actively with AI governance; not as passive end-users, but as informed participants who can interrogate training data, challenge algorithmic outputs, and advocate for transparency on behalf of their patients. For organizations, it means treating ISO/IEC 42001 not as a documentation exercise, but as a living governance commitment embedded in operational practice.
Innovation and patient safety are not competing values in healthcare AI. Pursued with the right frameworks, the right oversight, and the right professional culture, they are mutually reinforcing. The standards exist. The expertise exists. The question now is whether healthcare organizations will choose to lead on governance, or wait until a serious incident compels them to.
For the patients whose lives depend on these systems, that choice cannot come soon enough.
