The UK Research and Innovation (UKRI) agency has opened a £12 million funding opportunity for speculative, high-risk research into making artificial intelligence systems more explainable, but financial institutions are raising a critical warning: a convincing explanation does not guarantee a safe or well-governed AI model.

The funding initiative, announced on August 11, 2026, represents the first grant opportunity under UKRI's flagship AI Programme and targets early and mid-career researchers at UK institutions. Projects can receive up to £602,500 and must launch by February 1, 2027, lasting up to 24 months. Yet even as governments invest heavily in making AI more transparent, the banking sector is grappling with a harder truth: explainability has become one of the most misunderstood safeguards in financial AI governance.

Why Is AI Explainability Suddenly a Priority for Governments?

UKRI's investment reflects a global recognition that artificial intelligence is transforming research, industry and public services, but progress is uneven. The agency argues that increasing AI explainability could unlock innovation and advance safety, reliability and responsible use. The funding explicitly welcomes applications that "disrupt existing areas of explainable AI research or create new ones, delivering research with the potential to lead to radically new and disruptive technologies".

The opportunity is intentionally designed to be speculative and high-risk. UKRI encourages researchers to test unconventional approaches and even expects that some projects may not succeed as intended. The agency states it will "avoid failure; approaches that do not work as intended can still generate important learning and valuable knowledge for the wider research community".

Applications are particularly welcomed from early and mid-career researchers, and joint leadership of projects is allowed to encourage diversity. The closing date for applications is October 20, 2026, at 4:00 PM UK time.

What's the Problem With Relying on Explainability Alone?

While UKRI invests in making AI more interpretable, banking regulators and model-risk experts are sounding an alarm about a dangerous misconception: that a clear explanation proves a model is controlled and safe. The distinction between interpretability and explainability matters enormously. Interpretability refers to how directly a human can understand a model's inputs, structure and outputs. Explainability is broader and may include post-hoc techniques that produce a human-readable account of why a model made a decision, even when the model itself remains opaque.

The problem is fidelity. An explanation that sounds coherent is not necessarily an accurate description of what actually happened inside the model. This becomes especially critical with large language models (LLMs), which are exceptionally skilled at generating fluent rationales that can create the false impression of introspection.

"An explanation can be plausible without being complete, stable without being correct, or easy to read without faithfully describing the mechanism that produced the decision," according to analysis of banking AI governance.

Global Banking and Finance Review

For banks, this creates a governance trap. If a system denies a transaction, recommends closing an account or flags a customer as high risk, a natural-language explanation may make the decision appear auditable. But the real governance question is whether the institution can demonstrate that the explanation is stable, supported by the actual decision process, consistent with policy and legally sufficient for the use case.

How Should Banks and Regulators Approach AI Explainability?

The emerging supervisory view treats explainability as one control among many, not a standalone solution. The Bank for International Settlements (BIS), Financial Stability Board (FSB) and National Institute of Standards and Technology (NIST) all emphasize that trustworthy AI requires a multidimensional approach. Consider the key dimensions of responsible AI governance:

• Governance and Accountability: Institutions must establish clear roles, responsibilities and oversight mechanisms for AI systems, not just rely on transparency features.
• Data Quality and Validation: A perfectly understandable model can still be trained on biased data, rely on stale relationships or perform badly during market stress.
• Resilience and Security: AI systems must be tested for vulnerability to manipulation, cyber attacks and unexpected market conditions.
• Fairness and Non-Discrimination: A model can produce disparate outcomes even when no explicitly protected characteristic is used in the decision.
• Transparency and Explainability: While important, this is one dimension of trustworthiness, not a substitute for the others.

NIST's AI Risk Management Framework explicitly treats explainability as one characteristic of trustworthy AI alongside validity, reliability, safety, security, resilience, accountability, privacy and fairness. The FSB similarly places model risk alongside data quality, governance, cyber risk, third-party dependence and market correlations, emphasizing that transparency may help diagnose some problems but does not remove them.

Traditional banking model-risk frameworks, such as the Federal Reserve's SR 11-7 guidance, were built for bounded models with known inputs, specified objectives and defined outputs. But generative and agentic AI weaken each of those assumptions. A general-purpose model can respond differently to small changes in prompts, incorporate external tools, generate unstructured outputs and operate across tasks for which it was not explicitly trained.

What Does This Mean for the Future of AI Governance?

The gap between explainability and actual control is now visible in formal supervisory policy. In 2026, the U.S. Office of the Comptroller of the Currency (OCC) revised its model-risk guidance to explicitly exclude generative AI and agentic AI from scope because of their novelty and rapid evolution, while indicating that banking agencies intend further work on banks' use of AI. This exclusion is not a regulatory exemption; it is evidence that older model-risk frameworks do not map neatly onto newer systems.

The European Union's AI Act follows a proportionality logic by applying heightened obligations to high-risk uses, including certain systems used to evaluate individual creditworthiness. The regulatory architecture links risk level to requirements for governance, documentation, human oversight, robustness and traceability rather than treating explanation as a stand-alone solution.

As agentic AI systems become more sophisticated, the explainability problem becomes more serious. An agent can interpret an objective, call tools, retrieve information, make intermediate decisions and execute actions. The risk therefore moves beyond "why did the model say this?" to "what was the model allowed to do because it said this?".

UKRI's £12 million investment in fundamental research on AI explainability is timely, but the banking sector's cautionary message is equally important: governments and institutions must resist the temptation to treat a clear explanation as proof of control. True AI governance requires explainability as one component of a much broader framework that includes data quality, validation, resilience, accountability and human oversight.