Senko Rašić

Life is full of trade-offs, and some of the trade-offs involve relinquishing (part of) control for convenience.

In tech, this can often be seen in using services instead of downloading software and using it locally. For example, I use Netflix or Spotify instead of maintaining my own media library. It’s much more convenient, even if it doesn’t have all the content I’d have, or isn’t laid out exactly as I’d prefer.

A sneaky problem with relying more on services—thus relinquishing the choices and control over many details to them—is that we transform from active (human) agents to passive users who have to play by the providers’ rules.

AI says “no”

The frontier AI providers are under a lot of pressure to stop people from using their models for offensive security work: hacking, malware, exploits. Fair enough. The problem is that the filters can’t tell an attacker from a developer fixing their own code, so they catch a lot of people doing perfectly normal things.

Since June, Claude Code has been switching from Fable 5 to the older Opus 4.8 whenever a classifier decides a message is about cybersecurity. One developer got switched while fixing a user-enumeration bug in their own password manager.

Low-level code looks a lot like security code to a classifier. Salvatore Sanfilippo (antirez), the creator of Redis, complained a few days ago that “working at Redis with agents is becoming a real struggle because of the many security stops I get while working at non security things”.

Armin Ronacher asked his agent to download PHP 8.5.11 and compile it. The request was blocked for “violative cyber content”, presumably because of a recently disclosed PHP security bug. Earlier in October, he wanted to finish setting up a Windows PC without creating a Microsoft account, and also ran into the safety filter.

That last one is a double whammy: Windows itself has turned into a service that pushes you to sign in, and the AI tool you’d use to work around it refuses to help.

Writing small games with Claude is a lot of fun, and plenty of people do it (I use small games for my vibe-coding benchmark). Anthropic has now taken steps to block Claude from reverse-engineering and cloning video games. Users also report blocks on generic, non-malicious reverse engineering.

An AI researcher got tagged for “training a competitor” even though they never distilled Claude: the training data came from an open-source model, and Claude only submitted jobs and ran the analysis. The Hacker News discussion has more examples: custom firmware for the commenter’s own USB devices, a home LAN server, browser tests for their own codebase.

None of this is illegal, immoral or harmful. In most of these cases, the person wasn’t even breaking Anthropic’s usage policy. A system decided they shouldn’t be doing it, and that was that.

Tools vs. services

A kitchen knife can cut bread or hurt someone. It’s not the knife maker’s job to make sure you only cut bread. The author of a disk copy tool doesn’t have to make sure you’re not copying pirated movies. If you misuse a tool, that’s on you. That’s how tools work: you buy them, you use them any way you like, and you carry the responsibility for how you use them.

With services, it’s the other way around. The provider is responsible for what you do with the service (legally, or at least in the eyes of the press and regulators), so the provider polices you. You stop being a customer who makes their own decisions and owns the consequences. You become a user who has to stay inside the lines someone else drew.

It’s understandable why the service providers do this (government pressure, copyright lawsuits, bad press), but the reasons don’t change the result: a tool I pay for decides what work I’m allowed to do with it.

This isn’t an AI-specific problem. We use Gmail instead of running our own mail servers. I wrote about the developer side of this in Developer convenience, infrastructure cost, and losing the open web.

Often that’s a fair trade. I don’t want to maintain a mail server or a media library, and I’m happy to pay someone else to do it. Likewise, most people don’t have the hardware, or the interest, to run a large LLM locally, so outsourcing it to Anthropic or OpenAI makes sense.

The trade is convenience for control. It works while the service does what you need and stays out of your way. When the provider also decides what you can and can’t do with it, it turns into a devil’s bargain.

Free software, open models

The free software movement started by addressing a very similar problem: empowering the user to do what they want, instead of being beholden to the software (or service) providers. Although open source “won” in terms of acceptance and importance in modern tech, the underlying problem reappeared as a consequence of the software-as-a-service (SaaS) approach becoming ubiquitous.

With AI specifically, this is why I’m excited about open models. Yes, they’re not true “open source”, but they are close in spirit to free software: let the user do whatever they want with their tools. Once you’ve downloaded the model and have the hardware to run it, nobody can change its rules, take it away, or decide your work looks suspicious.

In How I want to use AI, I wrote that I want to be the active operator, not merely a clueless user. That’s hard to do when the tool decides what you’re allowed to operate on.

I don’t think we should give up on services. But as hardware gets better, more of our work should move back to our own machines, to tools that we own and control. So it’s funny when the boss of OpenAI says buying hardware to run local models is not optimal. True: it is inefficient from a global coordination point of view, but you can ask the Soviet Union how the (national) coordination of resources worked out for them.

Using third-party service providers introduces an unavoidable principal-agent problem. What’s best for you and what’s best for your service provider are not necessarily the same. Sometimes the difference is small and worth ignoring. Sometimes you do want to keep your personal sovereignty, and you do want the option to walk away.