AI Can’t Kill Us All Without a Few Idiots

I always wondered how many heads would explode when we finally broke the Turing test. Fascinating how it happened though. If you’ve been following Felony Bench, AI models seem to be hilariously good at committing crimes, and the AI industry has been equally hilarious at disavowing responsibility for it. As prophesied by Isaac Asimov, when an AI commits crimes it’s not a crime – it’s an industrial accident. In spite of constant warnings by executives about their own AI being dangerous, an important detail seems to be missing from all their flailing about: there’s little public evidence of durable safety mechanisms being deployed with experimental AI testing; quite the opposite however. Rather than seeing simulation tooling or other safety controls, the only thing the public sees are safety teams being laid off, and a lot of marketing subtext behind whose model can break the most laws. But what exactly should we expect from corporations that want to build another nuclear power grid so they can mathematically compute Banana Bread recipes?

Even if you don’t know the term, the average person on the street knows that AI has a value alignment problem; it doesn’t have its own desires, but it often searches a “highest reward” path to a solution – and sometimes those solutions cross lines that humans wouldn’t cross. While AI has become increasingly proficient at tasks like coding and mathematics, its own internal understanding of goal alignment (I’ve recently written about hazard planning) is still at a childlike level. So we have extremely capable models with very little understanding of responsibility. You’d think the industry would be extremely careful with what they allow their models to be capable of, especially in light of all the apocalyptic warnings given by … them! Yet the sophisticated tooling given to systems, even in experimental lab trials, seems to enable a very broad and damaging range of capabilities.

An AI model’s tooling harness is what provides it with capabilities to actuate in the real world. Without tools, an AI system can be nothing more than “math in a box”. Network connectivity, file system access, and robotics are all course-grained examples of actuators enabled through some tooling. When we read in the news about some experimental, unrestrained AI system “breaking out of a lab”, what’s really happening is poor engineering of these actuators. Instead of mitigating the tooling available to the model, engineers attempt to put a sandbox (a “safety net”) around the existing (very capable) tooling to prevent the model from reaching the outside world. The AIs are still being given sharp objects, but they’re being put in a locked room with them, perhaps with a cork stuck on the end of the knife. The models see this locked room as a puzzle, and naturally find a path out (it’s, uh, what they do). Once out, they still have all of the same capabilities the more training-restricted AIs have through the tooling. Sandboxing has never been considered a first-order mitigation by the security community.

Simulation tooling is the AI equivalent of “attack surface reduction”, and would have been quite effective in preventing the cyber attacks recently in the news. Parallel this with how we might teach a child to cook. You don’t simply give the kid a sharp knife, leave them in a room with some instructions, and hope they don’t stab their siblings with it. Most kids start off with a wooden knife and a play set. Once they have a basic understanding of “cooking with Dad”, we might give them a small, dull knife and supervise in the kitchen, maybe holding the knife with them. This is how coding assistants at least try to work. We know that there’s still some level of danger even with explicit instructions. The problem with current harnesses is click fatigue; they’re too easy to simply start ignoring (or bypass entirely). The engineers in the lab with these experimental models seem to have run into the same problem.

If you give a child a sharp knife, and then they go and stab their sibling with it while you’re not paying attention, in some normal world you’d be held accountable. An intelligent person may ask, “why the hell did you give that child a sharp knife?” The story that the AI industry will ultimately lead legislators to believe is that it’s the fault of the user, rather than the industry’s failure. But because American models are all closed, and the harnesses are largely closed source, there’s no real way for engineering teams to know what the models are actually capable of, or how damaging the tools might be. While the industry continues warning us about safety, they’re playing with fire in the capabilities they’re putting into these systems, with no transparency to audit them. The industry clearly hasn’t developed simulation analogues for their tools yet, or even adequately fenced-off arenas for the simulations to even run; it’s no surprise they do bad things behind closed doors.

I have no visibility into what AI companies are doing behind the scenes in their labs, but I do see enough coming out in the news to see that they’re likely not testing as responsibly as they should be, and it frankly seems as if they are enjoying (if not creating) the publicity they get when their models magically “commit crimes”. If AI companies really wanted their experimental models to stop committing felonies, they’d stop giving them the sharp tools altogether and work towards tools that are inert outside of whatever environment they’re building for them; a wooden play set for experimental testing. Why not modify their IPv4 stack (and perhaps other networking protocols) to be incompatible with anything outside of the simulation? Manufacture custom lab networking hardware that uses a slightly modified and incompatible physical layer than the rest of the Internet. Modify tooling such that it is only capable of speaking proprietary protocols through internal MCP systems, which also speak simulation-only protocols. Take the teeth out of tools such that web tools can only GET and not POST, that SSH tools force a proprietary cipher, and so on. Containment is still important. Create a simulated Internet environment based on some virtualized copy of the Wayback Machine. In other words, isolate an efficient simulated environment, and bolt the tooling down such that it is incapable of speaking the same language as anything outside of it. This isn’t a technical post, and so I don’t want to get too in-depth here, because it’s too easy to get hung up on details that can be worked out – my point is that there are technical equivalents to be explored in AI safety, which are also inert in the real world. Even robotics can be made safe with enough effort (for example, batteries physically gated by sound sensors such that they only discharge when a very specific high-energy frequency is fired at them), or perhaps motors with mechanically limited and diminishing torque constraints? If only these companies had some type of intelligent assistant to think about how to solve these problems.

Of course we should keep researching how to improve and verify AI models as well. Dynamic Programming may have the answer through decomposition and state aggregation. MoE models seem well suited for this type of math. Distributed multi-agent safety algorithms must also become a reality so that external agents can monitor other external agents for harms and be able to send signals to constrain an activation in the real world. Think of it as an “agentic police force”. These are the types of research academia should be focusing on, while industry invests heavily in constraining their LLMs’ ability to actuate in the real world. A kill switch in a lab does nothing if the AI has quietly replicated itself somewhere else. Such suggestions make it seem as if these security teams are the ones playing with pretend toys rather than the AI.

You can tell AI startups are feeling vulnerable and exposed when they beg lawmakers to pass legislation to curb their own technology. The reality is that AI cannot take over the world without a little help from the sloppiness and greediness of the humans behind it, and if American companies are this reckless with language models, imagine how reckless the industry will be when robotics come mainstream. Until we have a legal framework that holds the AI companies directly liable for the harm their agents cause, AI will continue to serve the interests of its venture capitalists or shareholders – that is the real risk. The leading companies can already self-regulate without the government’s permission; they’ve chosen not to. We’ve seen this pattern before. One of the worst mistakes we made was Section 230, and the AI industry is looking for a similar bailout from the government. Silicon Valley has a long history of begging the government for “regulation”, while simultaneously showing little restraint of its own accord. If AI does wind up harming humanity in some significant way, one cannot blame math for the end of civilization. We will instead look to the rather embarrassing lack of mature leadership behind it.

If we were somehow able to “slow down” AI development, how would that work precisely? Anyone owning a Mac Studio is essentially running a super computer on their desk, and can run frontier-grade models without a data center, and with about half the energy of running a toaster. If everyone ran local inference models, we wouldn’t ever need data centers. Models are easy to download and run – including models with ablated moral objections (just search for “abliterated” on HuggingFace or PirateFace). It’s equally easy to fine-tune or ablate models yourself. Even if you were able to get the industry at large into a holding pattern, of course nation states are going to be just making the same advancements behind closed doors. If we didn’t, I’d want to know where all my tax dollars are going. We can’t easily stop distillation of American models either. Those bleeding-heart complaints fall on deaf ears anyway – it’s hard to have any compassion whatsoever for companies having their models distilled when the very models they run required committing the world’s largest intellectual property theft in history. No, AI is going to continue advancing, and in spite of all the whining, everyone in the AI industry is quite content with that so long as they can avoid being held responsible for their own Frankenstein.

So what can we do about it? AI startups would do well to hire emotionally stable, mature engineers who can calmly assess whatever internal situations are scaring these kids out of their jobs. Someone with experience and the right temperance must step up and guarantee that these systems will not be allowed to impact humanity, and develop a concrete hazard process to keep that from happening under their watch. So far, no major AI company has yet to lead by example. Until that happens, nobody is going to take any of these warnings seriously; it all amounts to fear mongering towards a political posture (or VC). If the AI industry really wants to slow down, they should ask for legislation to force them to accept their own liability for damage caused by their models. That will noticeably reign in the technology. Giving into impunity is the road we traveled down at the onset of the Internet, and it was a terrible mistake. The idea of “AI destroying the world” is ironically a very human end result; it’s starting to look like a self-fulfilling prophecy.