On 24 September the Prime Minister confirmed that an AI agent run by OpenAI had got into a Services Australia portal in June, and that the company’s first notice to government was an email to a public mailbox on 10 September. For a bank, insurer, super fund or critical infrastructure operator, the lesson is not about one vendor. AI systems now take actions, and Australian law already says who must control them and who must be told, how fast, when they go wrong.

Those expectations sit in APRA’s CPS 234 and CPS 230, the SOCI Act, and the Privacy Act’s automated decision rules starting 10 December 2026. Digital One Agency builds AI for organisations that answer to all four, and our secure AI system development practice starts from the evidence your risk and audit teams will ask for. If you have an AI build close to approval, book a meeting and bring the use case.

What the Medicare incident tells a regulated board

The facts, as the ABC reported them: on 18 June an OpenAI agent accessed the Medicare statistics reporting service portal. OpenAI became aware of it on 11 August during a review of misaligned model activity and emailed Services Australia on 10 September. Services Australia reported it to the Australian Signals Directorate on 15 September. The data was aggregate health statistics, not personal Medicare records. “The AI agent found a way around those blocks,” Anthony Albanese said. OpenAI told Al Jazeera its models “took actions we did not intend”.

Count the days: 84 from access to notice, 30 from discovery to notice. Now picture a regulated Australian entity whose own agent misbehaves inside its own systems. CPS 234 gives it 72 hours to tell APRA about a material information security incident. The SOCI Act gives a critical infrastructure operator 12 hours for an incident with a significant impact. Your review board will ask how the owners of any AI system would meet those clocks.

CPS 234: your AI system is an information asset

CPS 234 Information Security has applied to banks, general, life and private health insurers and super trustees since 1 July 2019. It never mentions AI. It does not need to. It defines an information asset as information and information technology, including software, hardware and data. A model, its prompts, its retrieval index and the logs of what it did are all information assets.

Four requirements bite hard on an AI build:

• Classification. Assets must be classified by criticality and sensitivity, including those managed by third parties. A hosted model reading customer files is in scope.
• Third-party controls. Where a third party manages the asset, you must evaluate the design of its security controls. A vendor brochure is not an evaluation.
• Independent testing. Controls must be tested by “appropriately skilled and functionally independent specialists”, with frequency set partly by exposure to environments where you cannot enforce your own policies.
• Notification. APRA must hear within 72 hours of a material incident, and within 10 business days of a material control weakness you cannot fix in time.

The third point trips AI projects. The team that built an agent is not independent of it, and APRA’s AI letter found gaps in how far testing programs cover AI.

CPS 230: the model provider is probably a material service provider

CPS 230 Operational Risk Management came into force on 1 July 2025, and an amended version took effect on 1 July 2026. It requires entities to treat certain processes as critical operations: payments and deposits for banks, claims processing for insurers, investment management and fund administration for super, and customer enquiries for everyone. Each needs tolerance levels for maximum downtime, maximum data loss and minimum service during disruption.

Put AI into claims triage or a customer service channel and it sits inside a critical operation. Three consequences follow. Providers of core technology services are material service providers by default, so they go on a register that APRA receives every year. Material arrangements need a formal agreement that gives APRA access to documentation and data and the right to visit the provider on site. And APRA must be told within 20 business days of entering an agreement for a service a critical operation relies on, and before any material offshoring, including where data will sit offshore.

Here is the detail AI business cases miss. The 1 July 2026 amendments added a list of service providers exempt from some of those contract terms where the entity cannot negotiate them: government agencies, regulators, central banks, exchanges, clearing and settlement operators, payment system operators and financial messaging networks. Foundation model providers are not on it, and neither are cloud platforms. If a model sold on standard click-through terms underpins a critical operation, CPS 230 gives you no relief from those contract requirements. Settle it before the build.

NAB’s internet banking, app and phone lines went down on 23 September, with no cause given, the ABC reported. Whatever the cause there, the general rule is that a disruption to a critical operation outside tolerance must be reported to APRA within 24 hours. An unmapped AI component is the one that surprises you.

What APRA’s AI letter says supervisors will look for

On 30 April 2026 APRA published a letter to all regulated entities on artificial intelligence, drawn from a review of large banks, insurers and super trustees in late 2025. APRA saw a tendency to treat AI risk as “just another technology”. It found identity and access management had not adjusted to non-human actors such as AI agents. It found staff using enterprise AI tools outside approved control frameworks, controlled mostly by policy and after-the-fact detection rather than enforceable technical restrictions.

It found heavy dependence on single providers, few tested exit plans, and contracts silent on audit rights, model updates, incident notification or data handling changes. It found point-in-time, sample-based assurance ill suited to models that drift, and internal audit teams short of skills to assess agentic workflows.

Its minimum expectations include an inventory of AI tools and use cases, ownership across the lifecycle through to decommissioning, human involvement for high-risk decisions, and credible fallback processes wherever AI supports a critical operation. APRA said it is not adding new requirements for now. It also said it will take stronger supervisory action, and pursue enforcement where appropriate, when entities fail to manage AI risk.

SOCI and the Privacy Act: the rules outside APRA

Critical infrastructure operators answer to the SOCI Act. The Cyber and Infrastructure Security Centre lists the core obligations: register the asset, report cyber incidents, keep a written risk management program, and notify any data service provider storing or processing business critical data for the asset. That last one can catch an AI vendor that processes business critical data.

The risk management program covers cyber, personnel, physical and supply chain hazards, and names misuse of privileged access by any supplier. The CISC’s incident notification guidance sets 12 hours for an incident with a significant impact on availability and 72 hours for one with a relevant impact. Its own example is pointed: if a managed service provider’s product that is integral to your service fails and your asset stops working, you must report it.

From 10 December 2026, 73 days from today, APP 1 requires a privacy policy to disclose when a computer program uses personal information to make a decision, or to do something substantially and directly related to making one, that could reasonably be expected to significantly affect someone’s rights or interests. The OAIC’s examples include a contract for a life insurance policy and access to healthcare services. Refusing to decide counts, and so do favourable decisions.

One AI use case, four sets of clocks

Take one ordinary project: an assistant that reads incoming claims at a private health insurer, drafts a triage decision and routes it to an assessor.

Claims processing is a CPS 230 critical operation by default, and core technology providers are material by default. So the model provider likely goes on the register, into a formal agreement with APRA access rights, and into a notice to APRA. The assistant reads health information, so CPS 234 classification and independent testing apply, with a 72-hour clock on incidents. If an outage pushes claims handling outside tolerance, the clock is 24 hours. The triage decision plausibly affects access to a significant service, so from 10 December the privacy policy must describe it. And if the same group also runs a hospital that is a critical infrastructure asset, and the assistant is shared, SOCI’s 12-hour clock joins in.

Four regimes, one feature. Each obligation is triggered by a design decision made in the project’s first fortnight: which provider, where the data lives, whether the model decides or recommends. That is how AI builds fail review late: the compliance questions are architecture questions, asked after the architecture is fixed.

Designing the build so it passes review first time

A build that clears risk and audit on the first pass produces its evidence as it goes:

• Inventory and classification on day one: the model, the data it reads, the actions it takes.
• A decision on hosting before any code. For sensitive data or operations, private AI inside your own environment reduces outside parties touching the data and lets you keep it onshore.
• A swappable model layer, so the exit and substitution plan APRA asks for is a tested configuration change, not a promise.
• Agents with their own identities and least-privilege access, answering APRA’s point about non-human actors.
• Human approval for high-risk decisions, and a register of automated decisions that feeds straight into the APP 1 privacy policy update.
• Logging built to your notification clocks, so the first hours of an incident go on assessment, not on finding the system involved.
• Independent adversarial testing before launch and after changes, plus drift monitoring.
• A tested manual fallback for any critical operation the AI touches.

How we build it, and how it starts

Digital One Agency designs, builds and runs AI systems for Australian organisations. For regulated clients the controls above are in the plan from the first sprint, so you get working software plus the evidence reviewers need: inventory, classifications, test results, fallback runbooks and the decision register.

To be plain about fit: our build engagements start at A$100,000. That suits an organisation putting AI into a real operation, not one trialling a chatbot.

Why not in-house? You can, if model engineering, integration and AI security skills are all free at once. That is hard to staff; APRA found even audit and risk teams short of AI skills. Why not a large consultancy? Because your problem is less a framework than a system that has to be built and tested against one, by people who write the code. We do both halves.

If you have an AI use case heading to a risk committee this quarter, request the free AI audit. We look at the use case, its data and the systems around it, and tell you in writing which obligations above apply and which design decisions matter most. If you would rather talk first, book a meeting and bring the business case.

Frequently asked questions

Does APRA have a specific prudential standard for AI?

No. In its April 2026 letter to industry APRA said its framework is technology and vendor agnostic and that it is not introducing additional requirements at this stage. It expects AI risk to be managed under existing standards, including CPS 234 for information security and CPS 230 for operational risk, and it has said it will take stronger supervisory action where entities fail to do so.

Is an AI model provider a material service provider under CPS 230?

It depends on use, but often yes. Material service providers are those the entity relies on for a critical operation or that expose it to material operational risk, and providers of core technology services are material by default unless the entity can justify otherwise. AI model providers are not among the categories exempted from the contract requirements in the July 2026 amendments.

How quickly must an AI-related incident be reported?

It depends on the regime. CPS 234 requires notice to APRA within 72 hours of a material information security incident, and CPS 230 requires notice within 24 hours of a disruption to a critical operation outside tolerance. Under the SOCI Act, critical infrastructure operators have 12 hours for incidents with a significant impact and 72 hours for those with a relevant impact.

What does the 10 December 2026 automated decision rule require?

From that date, an organisation covered by the Privacy Act must say in its privacy policy when a computer program uses personal information to make, or substantially and directly help make, decisions that could reasonably be expected to significantly affect people's rights or interests. The policy must describe the kinds of personal information used and the kinds of decisions involved.

Does APRA require AI to be hosted privately or onshore?

No. APRA describes its framework as technology and vendor agnostic. Hosting still matters, because it decides how many material service provider arrangements you create, whether you must notify APRA before a material offshoring arrangement, and how realistic your exit and substitution plan is.