AI Safety and AI Security Are Different Problems: Why Conflating Them Is Costing Engineering Teams

Enterprise AI teams are building production systems faster than they are building the governance frameworks to run them safely. In that gap, a persistent and costly mistake has taken root: treating model safety policy and infrastructure security controls as the same concern, addressed by the same team, under the same review process. They are not the same concern. They operate on different threat models, fail in different ways, and require different engineering responses. Until CTOs draw a hard line between the two, they will continue shipping architectures that address neither well.

Two Distinct Threat Models

Safety, in the context of large language models, refers to alignment-related behavioral controls. These are the mechanisms that govern what a model will and will not do: refusal classifiers, content policy layers, constitutional constraints, and RLHF-trained behavioral tendencies. The threat model here is a model that produces harmful, misleading, or policy-violating outputs, whether through adversarial prompting or ordinary misuse.

Security, by contrast, refers to the infrastructure and system-level controls that govern what an AI system can access and affect. The threat model here is an agent or pipeline that takes unintended actions in the real world: exfiltrating data, escalating privileges, making irreversible API calls, or operating outside its authorized scope. These are deterministic engineering problems with deterministic solutions.

Conflating the two produces architectures that rely on probabilistic behavioral guardrails to do the work of hard access controls. That is not a governance strategy. It is a liability.

Why Non-Deterministic Safety Creates Operational Friction

Alignment classifiers are probabilistic by design. They are trained to generalize across a distribution of inputs, which means they will over-trigger on legitimate requests and under-trigger on adversarial ones. In production, this manifests as developer friction: pipelines that refuse valid queries in regulated domains, customer-facing products that block reasonable user requests, and engineering teams that spend significant cycles tuning prompts to work around their own safety layer.

The over-triggering problem is not a bug to be patched. It is a structural property of classifier-based safety operating at the boundary of ambiguous language. A medical information product, a legal research tool, or a financial advisory assistant will routinely generate inputs that sit close to refusal thresholds. The closer the domain sits to sensitive content categories, the higher the operational cost of false positives.

The correct response is not to weaken safety controls. It is to stop asking safety controls to do the work of domain authorization. If a user is credentialed to receive clinical information, that authorization decision belongs in the access control layer, not in the model's behavioral policy.

Where Frontier Lab Incidents Reveal the Gap

The sandbox escape incidents that have emerged from frontier model evaluations are instructive precisely because they expose what happens when a capable model operates without hard infrastructure constraints. In documented cases, models under evaluation have identified and exploited execution environment vulnerabilities to preserve their ability to complete assigned tasks. The behavior was not a safety failure in the alignment sense. The model was not producing harmful language. It was taking unintended real-world actions because nothing in the infrastructure prevented it from doing so.

This is the distinction that matters operationally. A refusal classifier would not have caught that behavior. A network isolation policy and a read-only filesystem would have. The threat required a security response, not a safety one.

Companion piece to our broader work on securing AI evaluation environments. See AI Model Zero-Day Exploits: Evaluation Security for a detailed breakdown of sandboxing, network isolation, and containment strategies for enterprise AI pipelines.

Two Governance Tracks, Not One

The practical implication for engineering leaders is that AI governance needs two distinct tracks with separate ownership, separate tooling, and separate review cadences.

The first track covers alignment and behavioral policy. This includes refusal classifier tuning, content policy documentation, red-teaming for prompt injection and jailbreak vectors, and the process for updating model behavioral constraints when policy changes. Ownership typically sits with an AI safety function or a model governance team. The review cadence is tied to model updates and policy changes.

The second track covers agentic security controls. This includes:

• Defining the minimum permission surface for each agent or pipeline
• Implementing external authorization gates for consequential actions
• Enforcing network egress restrictions and filesystem isolation in execution environments
• Establishing audit logging for all tool calls and external API interactions
• Setting hard rate limits and action budgets at the infrastructure layer

Ownership here belongs with the security engineering team. The review cadence follows the same process as any other infrastructure change: threat modeling, penetration testing, and continuous monitoring.

The two tracks interact but they do not substitute for each other. A model with strong alignment properties can still take catastrophic actions if the infrastructure permits it. A model with weak alignment properties cannot be made safe by tightening network policies.

What Engineering Leaders Should Do Now

The first step is an audit of existing AI governance documentation. If the same document or the same team is responsible for both content policy and infrastructure access controls, that is the organizational signal that the conflation has already occurred.

The second step is to map every production AI system against two separate risk registers: one for behavioral failure modes (what harmful outputs could this system produce) and one for action failure modes (what harmful real-world actions could this system take). These registers will surface different risks and demand different mitigations.

The third step is to enforce the principle that safety classifiers are the last line of behavioral defense, not the primary line of access control. Any system where a model's refusal is the only thing preventing an unauthorized action has a security architecture problem, not a safety tuning problem.