Federal Budget 2026-27: expert insights on spending, tax reform and policy impacts. Visit hub.
In Part 1 of our AI risk and governance series, we described the gap between AI capability and organisations’ ability to govern it. In Part 2, we explore how that gap is becoming easier to see in practical terms.
Familiar risks are showing up in a faster, more scalable and less visible way. Controls and processes built around human judgement, manual review and a bit of friction are being pushed into a very different operating environment.
The easiest way to understand what is happening is to look at the patterns emerging in practice.
AI is already having a significant impact on cyber security. Recent announcements around frontier models, such as Claude Mythos, have raised alarm bells throughout the cyber community. Recent reporting suggests Mythos represents a step change in vulnerability identification, which will force a response from cyber teams across the market. Novel vulnerabilities may now be identified and exploited in minutes rather than weeks, leading to a wave of new attacks before many organisations have time to patch, test, or even understand what is happening.
More broadly, cyber risk presents an especially impactful use case for AI as it is technical, labour intensive and constrained by a real skills gap. A lot of defensive work still depends on deep expertise and specific coding, triage and analysis. AI is changing both sides of that equation. It is increasing the capability and speed of adversaries, while also giving organisations new ways to lift detection, triage, response and remediation.
The practical challenge for risk management is that the noise is increasing, and the threat cycle is accelerating – while many control and assurance cycles were built for a slower environment. Boards and risk teams will increasingly be judged on how quickly cyber teams can detect, respond and recover, not just whether controls are in place.
Fraud and scam risk is changing quickly, particularly where AI is improving the quality of impersonation and the realism of fabricated information.
The Arup deepfake case in the UK is a useful example that has been widely referenced in this context. A finance employee was purportedly convinced to make a series of payments after joining a video call with what appeared to be senior colleagues. The request looked legitimate, the participants looked and sounded familiar and the context made sense. The allegation was that the interaction had been manufactured using deepfake technology.
The relevance for audit and governance teams is critical. Many organisations still rely on informal verification, familiarity of communication, and judgement when approving transactions or making changes. Once those signals can be replicated convincingly, the control environment comes under pressure. Payment approvals, account changes and identity checks become harder to validate, particularly where urgency, seniority or commercial pressure are involved.
Beyond frauds and scams, AI is making traditional incentive structures easier to manipulate. Where processes reward volume or speed, and artefacts are used to support decisions, AI makes it easier to produce those inputs quickly, and at scale. This can amplify behaviours that would previously have been limited by effort or scrutiny.
Recent reporting on the purported CBA mortgage fraud issue has brought this issue into focus. Public commentary has pointed to broker and referral channels, along with concerns that supporting documentation may have been generated or enhanced using AI. The critical risk management question focusses on incentives and process design. Where a channel rewards volume and supporting artefacts are easier to manufacture, the system becomes easier to manipulate before patterns are detected.
Controls designed for a low volume, high friction environment can struggle when both of those assumptions change. The issue is not simply whether individual applications were right or wrong. The issue is whether the control environment can still identify manipulation at scale when the effort required to produce convincing inputs falls away.
Model behaviour becomes important when AI is being used to support, influence or make decisions at scale. In some cases, that sits behind productivity-style outputs such as analysis, prioritisation or drafting. In others, it extends much further into operational settings, including initial claims decisions, triage, onboarding or exception handling.
The core issue is decision integrity. Once AI-generated outputs begin influencing decisions at scale, even a relatively small error rate can create meaningful downstream risk. That risk becomes more pronounced when outputs are trusted because they are consistent, well written or operationally convenient. Human review can drift towards confirmation, especially when teams are under time pressure or dealing with volume.
In practice, decision quality can degrade gradually. Issues do not always surface through obvious failure. They build through repeated use, reduced challenge and wider reliance on outputs that appear reasonable on the surface. At that point, model behaviour becomes a governance issue rather than a technical one.
AI is also changing workforce composition. Over time, the expected model is fewer employees supported by more agents and automation across processes. While it brings efficiency, it also puts strain on the concept of 'human in the loop'. Fewer people remain closely connected to how processes operate end to end, and even fewer are in a position to intervene when something needs to be challenged or overridden.
This workforce of the future model naturally increases key person risk. Knowledge starts to concentrate in a small number of people who understand how workflows are configured, how outputs are interpreted and where things break. If those people leave, or the knowledge is not shared, the organisation becomes increasingly fragile at exactly the point where speed and automation are increasing.
A psycho‑social dimension is also emerging as a workforce issue. As roles change, people feel pressure to use AI to produce more work, more quickly, while still being accountable for the outcome. That can be uncomfortable, particularly when people do not fully understand how the output was generated or where the limits sit. Teams are less likely to challenge outputs they do not feel they own, and more likely to accept system-generated outputs even when something feels off.
Taken together, these patterns point to a consistent conclusion – the operating environment has changed. Inputs are easier to fabricate, outputs are easier to scale, external threats are evolving more quickly and human friction, which used to act as a control in its own right, is being reduced.
Boards, risk teams and internal audit functions are now operating in an environment where governance, control and assurance have to keep pace with a very different form of risk emergence. Part 3 of this series will focus on what that looks like in practice, including how organisations can adjust risk assessment, control environments and assurance so they remain effective as AI becomes more deeply embedded across operations.
The Association of Superannuation Funds of Australia (ASFA), in collaboration with JANA, has released its final Investment Manager Operational Due Diligence (ODD) Guidance Note, providing a practical framework to strengthen how superannuation funds assess and oversee operational risk
In this episode of Beyond the Numbers with Grant Thornton, we speak with Payroll Compliance experts Katherine Shamai and David Mintz to discuss the shifting payroll compliance landscape in Australia, understand how businesses should mitigate risk and the importance of proactive, continuous improvement in payroll processes.
AI-driven vulnerability discovery is compressing the time between weakness and exploitation. This insight explores what boards, CISOs and regulated entities need to change in their cyber risk assumptions, governance and oversight models.
