AI Misuse, Living Governance and the Rise of Telemetry-Based Safeguards

One of the most uncomfortable realities emerging from the International AI Safety Report is that malicious AI use is no longer theoretical. It is already here. AI systems are being used to generate scam material, fraud campaigns, blackmail content, deepfake imagery and influence operations at a scale and speed that would have been difficult for small groups to achieve only a few years ago.

At the same time, AI capability is accelerating in areas that move beyond misinformation and into operational risk. Modern systems can identify software vulnerabilities, generate malicious code, assist with cyberattacks and provide increasingly advanced biological or chemical information. Even where safeguards exist, the report highlights a growing concern that models may lower the expertise barrier for novice actors attempting to pursue harmful objectives.

That last point matters more than many people realise.

Historically, certain forms of harm required specialist knowledge, years of education, technical mentorship, laboratory access or highly developed operational capability. AI has the potential to compress parts of that expertise curve. The concern is not necessarily that every malicious actor suddenly becomes an expert overnight, but that AI systems may help inexperienced individuals move further and faster than they otherwise could have alone. Potentially leading to deadly mistakes, simply due to the lack of experience or overconfidence in their new found knowledge.

This is where I think the conversation around AI governance starts changing fundamentally.

Static Governance Is Not Enough

Most governance models still operate as if AI systems are relatively static. Policies are written. Risk assessments are completed. Controls are documented. Periodic reviews occur. The problem is that modern AI systems evolve too quickly, interact too broadly and operate too dynamically for static governance alone to remain effective.

A system connected to external tools, APIs, databases, autonomous agents or live workflows may change behaviour based on prompts, user interaction, retrieved information, model updates or operational context. The governance problem is no longer simply “what was the model trained to do?” It becomes “what is the system actually doing right now?”

That is where living governance becomes necessary.

Living governance is not governance as a document. It is governance as an observable system. It requires continuous visibility into how AI behaves, what it accesses, what it attempts, what permissions it seeks, what patterns are emerging and where escalation thresholds are being approached.

And this is where AI telemetry becomes critically important.

Telemetry Is Not Just Monitoring

When people hear the word telemetry, they often think about technical logs or performance metrics. In reality, AI telemetry may become one of the most important governance layers organisations develop over the next decade.

A meaningful AI telemetry system would not simply track whether a model is running efficiently. It would monitor behavioural signals across the operational lifecycle of the system.

That could include:

• Patterns of prompt escalation
• Attempts to bypass safeguards
• Rapid topic progression toward restricted domains
• Repeated requests involving pathogens, weaponisation or malicious code
• Cross-domain correlation between cyber, chemical or biological queries
• Confidence changes in generated outputs
• Agent-to-agent interactions
• Tool invocation patterns
• External data access attempts
• Human override frequency
• Escalation events
• Behavioural drift over time

The important point is that telemetry shifts governance from passive review to active observation.

Instead of asking what an AI system theoretically could do, organisations start monitoring what users and systems are actually attempting to do in practice.

Detecting Escalation Pathways

One of the most interesting governance opportunities may involve identifying escalation pathways rather than focusing only on isolated prompts.

A single question about chemistry is not necessarily dangerous. A single question about pathogens may be academic. A coding request may be entirely legitimate. But when multiple behaviours begin converging across domains, intent starts becoming more visible.

For example, a living governance system might detect a pattern where a user progressively moves from harmless biological concepts toward increasingly operational laboratory procedures, acquisition pathways, synthesis questions, containment bypasses or deployment scenarios. The concern may not be any individual query, but the behavioural trajectory itself.

The same principle applies in cybersecurity.

An isolated request for code assistance is ordinary. But telemetry might detect an emerging sequence involving vulnerability discovery, privilege escalation, obfuscation techniques, persistence mechanisms, credential harvesting and automation tooling. Individually, some requests may appear benign. Together, they may indicate operational preparation.

This is where telemetry becomes more than monitoring. It becomes behavioural risk analysis.

The Role of Commitment Points

One of the weaknesses in many governance systems is that they focus heavily on outputs but less on operational reliance. This is where the concept of commitment points becomes important.

A commitment point is the moment where a system, user or organisation moves from information gathering into operational action. That may involve downloading generated code, invoking a real-world tool, triggering automation, accessing restricted systems, exporting data or transitioning from exploratory questioning into executable workflow behaviour.

Telemetry systems could potentially identify these transitions in real time.

That does not necessarily mean blocking every risky interaction automatically. In many environments, false positives could become disruptive very quickly. But it does mean introducing visibility, escalation pathways and conditional review before higher-risk thresholds are crossed.

In practice, this may look less like censorship and more like graduated governance:

Observation. Risk scoring. Escalation. Human review. Conditional permissions. Intervention thresholds. Audit preservation.

That layered approach is far more sustainable than assuming a single static safeguard will remain effective forever.

Living Governance Requires Human Oversight

One of the dangers in this discussion is assuming telemetry itself becomes the solution. It does not. Telemetry is visibility, not judgement.

Human oversight still matters because context matters. Researchers, security professionals, legal investigators, academics and medical specialists may all interact with sensitive domains legitimately. A governance system that simply blocks keywords without contextual reasoning would become both ineffective and disruptive.

The challenge is designing systems capable of distinguishing ordinary research, professional activity and legitimate operational need from behavioural escalation patterns associated with malicious intent.

That is not a simple technical problem. It is a governance problem, a legal problem, an operational problem and increasingly a societal problem.

The Bigger Governance Shift

What I think the International AI Safety Report highlights more than anything else is that AI misuse cannot be approached purely as a model problem anymore. It is becoming an operational systems problem.

The future governance battle will likely revolve around visibility.

Can organisations see behavioural escalation? Can they identify operational risk early? Can they detect intent trajectories rather than isolated outputs? Can they monitor when AI systems transition from information generation into actionable consequence? Can they preserve meaningful auditability without creating mass surveillance systems that undermine trust?

Those are difficult questions, but they are rapidly becoming unavoidable.

The reality is that AI capability will continue improving. Some safeguards will hold. Others will fail. New models will emerge faster than governance frameworks can comfortably adapt. That means governance itself needs to become more dynamic, more observable and more operationally embedded.

In many ways, this is why I keep coming back to the idea of living governance.

Not governance as paperwork. Not governance as annual review. Not governance as static policy language.

But governance as a continuously observable system capable of monitoring behaviour, detecting escalation, preserving accountability and creating meaningful intervention points before consequence occurs.

Because the real challenge may not be whether AI can generate dangerous knowledge…It may be whether institutions can develop governance systems capable of recognising when knowledge is turning into intent.