AI Governance and Cybersecurity Are About to Collide
I’ve been doing some work and study in cybersecurity recently, and one thing has really started to sit with me…
A few weeks ago, a colleague and I built an MCP server, clipped it into Claude, and used it to run authorised intrusion-style testing against our own servers.
Not someone else’s system. Ours. The idea was simple enough.
Could we connect AI to tools we would normally use manually, and see whether it could adapt those tools to test the strength of our own defences?
Then we pushed the idea further.
We redeveloped some of that thinking into our own AI testing environment, just to see whether the system could take a set of tools, reason through the barriers in front of it, adjust its approach, and continue testing for weaknesses.
And that is where the reflection starts. Because the thing that stood out was not just that AI could assist with cybersecurity testing.
It was the speed of adaptation.
The system could take feedback, adjust, try a different path, interpret the result, and then keep moving.
That changes the nature of the problem.
Traditional cybersecurity often assumes that we build controls, harden systems, run testing, patch issues, and then repeat the process periodically.
But AI does not necessarily work periodically.
It works continuously.
And if bad actors are using adaptive AI systems, then static defence starts to look very fragile.
Anthropic has already reported cases where AI was misused in cyber operations, including situations where AI performed a substantial amount of the tactical work while humans remained more in a supervisory role. Anthropic’s own framing is important: the same capabilities that create cyber risk are also needed for cyber defence.
That, to me, is the real shift.
We are not just talking about better cyber tools.
We are talking about systems that can reason, adapt and operate at a tempo humans cannot match. And that raises a much bigger governance question.
I’ve been talking for a while about the need for living AI governance — not a static policy document, not a PDF sitting in a folder, not a once-a-year risk review.
A living system.
A system that monitors, tests, records, escalates, learns and adapts.
I think cybersecurity now needs the same treatment.
We need to start thinking seriously about living cyber defence systems.
- Not just firewalls.
- Not just endpoint protection.
- Not just audits.
- Not just penetration testing once every quarter or once every year.
But defence systems that are constantly assessing:
- what has changed;
- what new attack paths may exist;
- what tools are being connected;
- what permissions are being granted;
- what abnormal behaviour is occurring;
- what AI agents are doing;
- and whether existing controls are still defensible.
Because the question is no longer simply:
“Did we have a cybersecurity framework?”
The question is becoming:
“Was that framework alive enough to respond to the threat environment it was operating in?”
That is a very different standard. And it matters because the next wave will not just be about AI. It will also be about the acceleration of the compute.
As quantum computing develops, some current assumptions around encryption, identity, access and security architecture will be placed under pressure. I don’t think every system becomes irrelevant overnight, but I do think organisations need to stop pretending that today’s static controls will automatically be enough for tomorrow’s threat environment.
This is where defensibility becomes critical.
In law, governance and cybersecurity, it is not enough to say:
“We had a policy.”
The better question is:
“Could we prove the system was being monitored, tested, challenged and improved?”
That is the part I think many organisations are not ready for.
AI-enabled attacks are not just faster versions of old attacks.
- They are more adaptive.
- They can test.
- They can iterate.
- They can chain tools together.
- They can learn from failure.
And if the attacker becomes adaptive, the defence cannot remain static.
That is the real shift.
We are moving from cybersecurity as a control environment…to cybersecurity as a living intelligence system.
And that is going to require money, architecture, governance, evidence trails, human oversight and a very different mindset.
The future of cyber defence will not just be about blocking threats. It will be about proving that your organisation had a living, adaptive system capable of responding to them. That is the conversation I think we need to start having more seriously. Because the bad actors are already thinking this way.
The question is whether the defenders are moving fast enough.