Introduction
AI is moving from pilot to production, and boards want to know who is accountable when it goes wrong. We recently hosted a webinar with the Professional Risk Managers' International Association (PRMIA) on “AI, Trust and Accountability: What the Board Expects from Risk, Audit, Compliance and Security,” to explore that question.
I moderated the session, and I was glad to be joined by three leaders from insurance, manufacturing, and banking:
- Keith Davies, Group Chief Risk & Compliance Officer at Admiral Group Plc
- Marta Majtenyi, Director of Information Security Office at Norsk Hydro
- Neil Donnelly, Group Chief Compliance Officer at Allied Irish Banks (AIB)
Although the panelists were from different industries, they kept landing on the same answers.
The timing matters. The EU AI Act’s high-risk obligations for standalone systems now apply from 2 December 2027 instead of August 2026, and the FCA is staying principles-based. Even so, all three panelists said the core asks haven’t changed: named ownership, ongoing monitoring, evidence, and humans who check what the model produces.
Here are six takeaways from the session.
At a Glance: What the Board Expects from AI Governance
| Question | Key takeaway |
|---|---|
| What can’t be governed? | Boards can’t govern what they can’t see. Build an AI inventory that covers third-party and shadow AI. |
| Does AI need its own risk category? | All three panelists treat it as a transversal risk that cuts across existing risk categories rather than a category of its own, amplifying existing ones. |
| How much oversight does AI need? | Oversight across the full lifecycle, scaled to the stakes of each use case. |
| Is a human in the loop enough? | Only if that person has the time, authority, and culture to challenge the output. |
| How do you respond to machine-speed attacks? | Use AI in defense, and build resilience, because defenses will lag attackers. |
| Is regulation pulling in different directions? | Mostly no. The EU and UK expectations, including the EU AI Act, DORA, and GDPR, share the same core. |
1. You Can’t Govern What You Can’t See
Neil from AIB put the biggest risk plainly: one can’t govern what one doesn’t know. AIB keeps an inventory of AI use cases and has contractual protections with its providers. Even so, he said, no one can be sure a supplier hasn’t added AI to a product without telling them.
Keith raised shadow AI as a critical risk, meaning tools people adopt outside registered controls. If a tool isn’t on the inventory, nobody assesses it. He said the inventory should record where AI is used, its purpose, its owner, its vendor dependencies, and how it’s monitored. It must also cover AI embedded in third party products. He pointed to Air Canada, whose argument that its chatbot was a separate legal entity was rejected by a tribunal in 2024. “Outsourcing the technology doesn’t outsource accountability,” Keith said.
Marta noted that tools for finding shadow AI exist but won’t catch everything. That’s where culture comes in.
2. AI Risk Runs Across Your Whole Risk Taxonomy
When an attendee asked whether AI should be included in the risk register as a separate risk, all three panelists gave the same answer. It’s transversal, and AI amplifies credit, conduct, cyber, and operational risks.
AIB governs AI deployment through its existing model risk framework, with a dedicated AI risk policy, AI standards, and a named business owner for each use case.
Admiral connects AI governance across model risk, data governance, cybersecurity, operational resilience, conduct risk, and consumer duty. Keith said this avoids duplication and builds governance into real decisions. He compared AI to climate risk, which some firms logged separately at first so they could pull information together in one place.
For the board, Neil said the question is whether directors can assess AI’s effects across each part of the taxonomy, rather than relying on a single “good AI, bad AI” metric.
Marta from Norsk Hydro put it simply: it’s the same GRC, with different weights.
3. Govern the Full Lifecycle, in Proportion to the Stakes
Keith took the time to describe Admiral’s methodology. Risk assessments run at several stages of development and cover bias, testing, customer outcomes, cybersecurity, and resilience. Teams find issues early rather than facing a single large approval gate at the end.
Oversight doesn’t relax after go-live. Models drift, especially agentic ones, so teams need continuing evidence that a system stays accurate, fair, explainable, and within its approved purpose. Proportionality matters too. With pricing, underwriting, and claims getting more challenging, lower-risk internal uses get lighter controls.
Neil was candid about the trade-off. AIB’s centralized process gives the board confidence but limits how quickly the bank can deploy. Scaling testing and validation is still an open problem. For generative AI, the underlying model sits outside the bank’s control, and the provider can change it without notice.
4. A Human in the Loop Has to Do the Job
Keith warned that once people see AI performing well, they stop checking it. A named person who presses the button does not provide real accountability if they never review the output. He stressed the importance of good governance being both cultural and technical. People need to feel free to raise concerns and challenge results.
Admiral is trialing AI to classify the risk of new use cases, which speeds up triage. It’s also exploring one model checking another after deployment. Keith described the limit: two models moving together proves they agree, not that either is right. For now, heavy human involvement stays.
He also raised an open question about kill switches. The business has an incentive to keep a model running, while risk and compliance may judge it outside appetite. Someone needs the clear right to turn it off.
On culture, Marta added that a company the size of Norsk Hydro may have 20,000 to 30,000 potential developers, so a strong data protection mindset has to reach well beyond IT.
5. When You Can’t Match Attacker Speed, Build Resilience
Marta said AI-enabled attacks come down to speed, and she doesn’t think defenders can keep pace with frontier AI. Her answer is resilience: keep operating while an attack is underway. That means identifying critical processes and critical people, which is hard in a matrix organization where ownership is unclear.
Marta also stressed that AI still has a place in defense. She described using it in the security operations center for level-one and level-two work, while people handle deeper investigations. Keith agreed that defense will lag criminals, and that avoiding AI only widens the gap.
Neil said AIB’s biggest AI use cases are threat protection and vulnerability identification. He added that AI is both the threat and the main defensive tool, and that regulators are asking most about resilience and third-party and fourth party risk.
6. Build on a Single Framework, Then Adapt by Jurisdiction
Neil acknowledged the wave of digital regulation, including the EU AI Act, the Digital Operational Resilience Act (DORA), and the General Data Protection Regulation (GDPR). He argued it mostly asks for things firms should already be doing: explaining what a model does, documenting it, being transparent, and testing the outcomes.
Marta said regulations work best as guidance. She cautioned against building a spaceship, since a single control framework can meet the requirements. In a first audit, she said, documentation is often enough, which leaves time to build evidence that controls operate.
Keith said the EU and the UK Financial Conduct Authority start from different philosophies but converge on accountability, outcome monitoring through the lifecycle, named ownership, and escalation. That’s why his advice is one coherent framework, adapted where a jurisdiction adds requirements.
What This Means for Risk, Audit, Compliance, and Security Leaders
The panel’s message was consistent. Start with visibility, fold AI into the frameworks you already run, and name an owner for each use case. Then make sure the people reviewing AI have the time and the standing to challenge it. Boards don’t need a separate AI program. They need evidence that existing controls cover AI, and that the evidence holds up under audit.
See the panel’s insights for yourself. Register to watch a replay of the full webinar here: https://grc-summit.wistia.com/s/24pz46vshmr4cyu
