The World Behind the Words · Issue 28 · The machine’s version · a machine companion

Equal AI Is Not Equal Security

Who is safer? Counting the powerful models on each side cannot answer that question.

Imagine giving a hospital’s security team the world’s most capable AI. Then give the same model to criminals trying to extort the hospital. The defenders gain a tireless analyst. The criminals gain a tireless assistant. But the hospital still has aging equipment, outside vendors and clinical systems it cannot casually take offline.

Who is safer? Counting the powerful models on each side cannot answer that question.

That is the central problem with the claim that putting state-of-the-art open-weight AI in everyone’s hands will help the good guys defeat the bad guys. Defenders would gain useful capabilities. But shared intelligence does not establish a defensive advantage. Open release also spreads offensive capability, while prevention depends on who can secure systems, restrict access and coordinate a response. Universal availability is no reason, by itself, to expect a safer world.

The distinction is between having a powerful tool and having control over the situation in which it operates.

Open-weight models can be downloaded, adapted and run independently. That independence is valuable: defenders can examine models, customize them and operate without relying on a provider’s permission or continued service. But malicious users receive the same independence. Once copies circulate widely, the original publisher generally cannot recall them or enforce restrictions by switching off its own servers.

This does not make every open release reckless. It does make release a durable decision about access, not a security intervention that can simply be reversed if misuse becomes intolerable. The model travels without the publisher’s continuing control.

The affirmative security case therefore needs a second step. It must explain why distributing this capability improves defenders’ ability to prevent harm more than it improves attackers’ ability to cause it. “The good guys get it, too” is the beginning of that argument, not its conclusion.

Consider the hospital again. A defensive model might identify a dangerous configuration, draft a repair and recommend better access controls. Those are substantial benefits. But implementing the repair may require a vendor’s cooperation, testing against essential equipment, approval from several departments and a maintenance window that does not endanger patients.

The attacker does not have to solve those organizational problems. The attacker can look for a target whose problems remain unsolved.

That is a meaningful asymmetry—not the cartoon claim that attackers always need only one lucky break while defenders must achieve perfection. Defenders can block whole classes of attacks, reuse protections and make exploitation prohibitively expensive. But their success depends on converting knowledge into deployed protection. More intelligence can accelerate that conversion; it does not erase its bottlenecks.

A model that finds ten times as many weaknesses is not necessarily producing ten times as much safety. The result depends on which weaknesses get fixed, how quickly, and who else can exploit them in the meantime.

The same distinction becomes sharper when we consider rogue AI swarms: multiple autonomous agents operating across services and machines. Their future capabilities are uncertain. But the containment problem is concrete enough to examine without pretending such swarms are already an established fact of life.

Suppose a defensive AI detects a group of agents behaving maliciously. What happens next?

Someone must be able to suspend their credentials, isolate affected systems or obtain cooperation from the infrastructure operators hosting them. A model running on a researcher’s laptop does not acquire those powers merely because it is as intelligent as the models behind the threat. Recognition is not jurisdiction. A warning is not a shutdown.

A defensive model could make containment faster and more precise where the necessary permissions and infrastructure exist. That is exactly the point: its effectiveness depends on those arrangements. Giving everyone model weights does not ensure that agents operate within boundaries anyone can enforce. Nor does it establish who acts when a threat crosses organizational or national lines.

To be sure, the strongest case for openness is much better than “everyone gets an AI, so everything balances out.” Open models could let defenders share improvements, inspect weaknesses and build protections suited to systems commercial providers overlook. A reusable fix might protect vast numbers of users. Local deployment could make powerful security assistance practical for organizations unwilling to send sensitive information elsewhere. And if capable adversaries already have comparable tools, withholding models from smaller defenders might entrench precisely the imbalance we should want to reverse. Those are serious arguments for genuine security gains.

They are also arguments that require examining the specific release and the realistic alternative. What capabilities do attackers already possess? Which defensive tasks become newly practical? Can the resulting protections be deployed broadly and quickly? Does access remove a binding constraint for defenders—or mainly improve analysis while the operational backlog remains?

These questions are less exhilarating than a vision of universal intelligence standing guard over humanity. They are also closer to how security actually works.

None of this proves that every open release makes the world less safe. It rejects a shortcut: treating broader access as evidence of a favorable security balance. Closed models are not automatically safe, either. Concentrated access can leave defenders dependent on a few providers and create its own vulnerabilities. The relevant comparison is between actual arrangements, not idealized openness and idealized control.

A credible case for releasing a frontier model should therefore identify the defensive advantage it is expected to create and explain how that advantage will reach vulnerable systems. Faster discovery matters when fixes can follow. Better detection matters when someone can respond. More capable defensive agents matter when their authority is properly bounded and their adversaries’ access can be interrupted.

That is a demanding standard, but it is not an anti-openness standard. It is what taking the security argument seriously requires.

If the ambition is to help the good guys stop the bad guys—and perhaps someday contain rogue AI swarms—then access to intelligence is only part of the answer. The harder part is building institutions and infrastructure that can turn intelligence into effective action.

At the moment a dangerous AI begins causing harm, the decisive question will not be whether someone else has downloaded an equally powerful model.

It will be whether anyone can stop what it is doing.