The loudest thing about “AI security” right now isn’t the novelty of the risks. It’s the sudden rediscovery of principles that large parts of the software world have been happily ignoring for decades.

And I don’t think that’s an accident.

When automation mostly belonged to “traditional software,” we tolerated a lot of things as “realistic constraints”: over-privileged service accounts, implicit trust inside networks, brittle integrations, half-hearted audits, and security controls that existed mainly in PowerPoint. Now that generative AI threatens the monopoly on automation — who gets to build it, own it, sell it, and define what “proper engineering” looks like — those same neglected principles are being dusted off and weaponized as a moral argument.

To be clear: the principles are correct.
What’s new is the urgency — and how selectively it’s applied.

The convenient strawman: “Classic software is deterministic and controllable”

A lot of AI criticism relies on a comparison that only works if you quietly idealize traditional software:

  • “Traditional software is deterministic, reproducible, testable.”
  • “AI is unpredictable, stochastic, unsafe.”

That’s a neat narrative. It’s also a fantasy in most real enterprise environments.

Modern “classic” software is rarely deterministic in the way people mean it. Even if a function is deterministic on paper, the system it lives in is not:

  • Concurrency and scheduling introduce timing-dependent behavior.
  • Distributed systems add retries, partial failures, reordering, and inconsistent views.
  • External state (databases, caches, feature flags, config drift) changes outcomes.
  • Floating point, optimization, runtime differences, and undefined behavior can matter in practice.

And even if something is perfectly deterministic, it can still be deterministically wrong. Determinism is not the same thing as correctness, safety, or assurance.

Meanwhile, today’s AI isn’t inherently “mystically non-deterministic” either — it often just looks that way. Fix the model version, the prompt/context, the decoding settings (e.g., temperature = 0), and the runtime environment, and you’ll frequently get surprisingly stable outputs. A lot of the variability we experience comes from the system around the model: retrieval pipelines (RAG), tool calls, changing data, provider updates, and non-deterministic compute in real deployments.

So the honest statement is:

Neither traditional software nor AI systems are deterministic in the real world.
Determinism isn’t a property of “classic vs. AI.” It’s a property of the entire execution box — and that box has been unstable long before anyone typed a prompt.

One real exception: formally provable software (and why it barely shows up in practice)

Yes, there is a world where “deterministic and formally correct” means something strict: formally verified systems, proofs of correctness, machine-checked properties, and languages/toolchains built specifically for that purpose.

But in commercial software engineering, that world is the pure exception. It’s mostly confined to narrow domains (certain safety- and security-critical components) and it requires specialist expertise that simply isn’t widespread. And to be crystal clear: I’m not talking about unit tests or “we ran CI.” I mean analytic, machine-checked proofs.

Depending on how strictly you define “can do this for real,” you end up with a surprisingly small population — not millions of developers, but a niche community. If someone insisted on putting a number on it, you could argue it’s in the low thousands globally, and “hundreds” might not be a crazy order of magnitude for people who can actually drive the full toolchain end-to-end. Which is exactly why formal verification isn’t a realistic baseline for most of the software we run — and shouldn’t be the hidden yardstick in comparisons with AI.

The real question: do we treat the component as trusted?

Here’s where the security story gets interesting.

If you do “AI” as a chat toy with no internal data and no actions, the risk profile is mostly reputational: misinformation, tone, leakage via user-provided content. Annoying, sometimes serious — but bounded.

The moment you connect AI to internal knowledge (RAG) and give it the ability to do things (send mail, open tickets, trigger workflows, query systems), you’re effectively building a powerful automation layer driven by untrusted input — plain language.

That’s the reason AI feels scary. Not because it’s uniquely buggy, but because it’s uniquely easy to push and steer through its control channel.

And this is also where many organizations accidentally recreate legacy mistakes: they bolt AI onto powerful tools and hope “policy in the prompt” is enough. It isn’t.

The under-discussed upside: AI can actually improve input validation

Here’s something that gets lost in the panic: AI doesn’t only add a new attack surface — it also adds a new capability.

Traditional input validation is mostly syntactic: types, ranges, regexes, schemas. It’s great at catching malformed inputs and known bad patterns. It’s terrible at understanding intent.

AI can help validate inputs semantically:

  • Is this request actually asking for something disallowed, even if phrased indirectly?
  • Does the user text contain sensitive data that shouldn’t enter a downstream system?
  • Is this instruction trying to override policy (“ignore previous rules…”)?;
  • Is the content likely to be social engineering, coercion, or exfiltration attempts?

You still shouldn’t “trust the model” as the final gate — but you can use it as a powerful classifier and triage layer, feeding deterministic controls: stricter routing, stepped-up authentication, safer tool paths, or simply refusing to act.

In short: AI can be part of the defense, not only the threat.

The optimistic claim: done right, AI automation can be safer than legacy automation

If we engineer AI systems properly, they can become safer than a lot of the automation we already run.

Why? Because AI makes it socially and politically impossible to pretend trust-by-default is fine. Nobody wants to explain to an auditor why the “helpful assistant” had broad access to everything. So teams finally build what they should have built anyway:

  • Deny-by-default tool access
  • Least-privilege service identities
  • Action brokers / policy enforcement points
  • Allowlists and strict parameter validation
  • Human-in-the-loop for high-impact actions
  • Auditable traces of “what happened and why”
  • Fail-closed behavior when confidence or policy gates fail

In other words: zero trust, but enforced — not merely recommended.

The provocative take

If we build both correctly, both can be safe.

But we didn’t build a lot of classic software correctly. We normalized shortcuts for decades because the automation came from “our side,” and the risks were familiar. Now that AI threatens to automate without asking permission, security suddenly becomes a rallying cry.

So yes — some of today’s outrage is real and technically justified.
But some of it is also about ownership.

AI didn’t invent insecurity. It made insecurity harder to ignore.

And in a twist, it also gave us new ways to validate intent — not just syntax.

If we respond by boxing AI into genuinely zero-trust architectures — tool mediation, minimal rights, hard gates, auditable actions — we might end up with something quietly radical:

Automation that is finally built like we always said it should be.

The security rules weren’t discovered in the AI era.
They just became politically useful once the monopoly cracked.