Microsoft is cutting Claude Code licences inside its core product teams. The reason is not strategy. It is the bill. The end of the AI coding experiment has arrived?
I actually prefer gpt 5.5 to opus 4.7 for coding (haven’t tried 4.8 yet because what am I made of money?) and it is way way more token efficient.
When Claude says it’s “bloviating” it really means it
A big part of the problem is the harness which you allude to. Claude locks you into Claude code which is bloated and absolute ass at context management.
Once the Chinese models reach parity with the current generation models it’ll be a race to the bottom. Deepseek v-4 pro is right there but not quite. The models now are strong enough to be generalist problem solvers. Anything stronger will only benefit niche applications.
I’d like to see something like the gpt chat interface within a coding harness where the model is capable of selecting what to delegate to based on the task. This is where we will go in the future. A lot of enterprises incinerate tokens with Claude because people are using opus to write emails or whatever. It’s like taking a Ferrari to the grocery store.
These will become commodities though I think at which point it’s all about the integrations. You can already see the big providers pivoting into these value added services with GPT leaning into the consumer market with apps and Claude going heavy on business/enterprise
It’s been a few months but I gave the then latest Claude Opus and GPT Max the same task and they both used the same absurd amount of tokens just to translate some curl commands to some other code. Gemini did about the same with like 10% of the tokens, so I’m a little impressed by them. I believe Claude is burning so many tokens just reading the entire code base to match house style.
I’d like to see something like the gpt chat interface within a coding harness where the model is capable of selecting what to delegate to based on the task.
You can do this with multimodal harnesses like opencode or pi by restricting the model for each agent you’ve configured. You’ve picked the model per agent, not the AI. But you could probably define duplicates per model per agent if you wanted.
I personally think Apple has the right long term idea with making consumer hardware that can run adequate local models. All the data centers can get fucked. I haven’t experimented with AI writing emails, but I suspect Gemma E2 Flash can do it just fine.
Yeah probably a better pi setup would be to create more granular agent types. I tried creating a worker powered by deepseek but found it wasn’t really that good. Something like an agent team spec (use a technical product manager agent to coordinate modules, review the worker output, and to make necessary corrections) then you could use a frontier model to plan and issue work instructions to the TPM agent
Not sure if this would be more economical in practice but would be interesting to try
Long run I think you will always need the data centers to handle big training loads but we might go back to on-prem computing for enterprises to run frontier models. Or even edge nodes that have enough muscle that folks can subscribe to locally.
Something in your city you can sub to for $30 a month that lets you run any flagship open source model could be very compelling
I don’t use AI for anything except translating language. I just use the local models. I have no fucking idea how an entire language (and the info needed to properly transform to another language) can be contained in a file of 10-70mb. But it works adequately on my trashy old hardware.
Also tried a bit of text to speech, transcription and similar. Those were more of a pain in the ass and needed a lot more stuff to download and set up but it isn’t clear to me if that is the nature of it or maybe the front end user experience just hasn’t been polished yet. But my laptop can handle it anyway. It seems like these companies really want the SAAS model and that is directing how they build. More than it being an inherent requirement.
To me it kind of seems like most individual people have a limited set of tasks they want to accomplish so you could just have whatever you need locally. Probably could even generate a custom local model and send it updates from an interface of some kind. If you want something to write emails or whatever you have it pull and digest what it needs once and then by incremental upgrades after that.
Its hard for me to imagine individual users really need so much power from a data center because its not like you are using the whole thing yourself. Probably data centers are really for huge comprehensive tasks that humans could never do unaided like analyzing massive datasets.
Yeah you’re kind of getting where I think or at least am hoping the market will go. You can already retrain public models locally but the tooling is either way too low level, or completely vibe coded.
I think the main contradiction in the market is that AI models are too big and expensive to run profitabily. But if they make them more efficient then the models can run on consumer hardware and that fucks up the SaaS AI business.
Don’t worry, all the data centers will be put back to use mining crypto as god intended.
Too expensive to run profitably + efficient models undermine the SaaS model + Chinese tech is always threatening to surpass them for a fraction of the cost. Even if they try to gatekeep efficient and local models, deepseek is right there.
Deepseek can be blocked trivially on national security grounds. Trivial in comparison to gatekeepibg local models because that’s just data that’s already out there.
I actually prefer gpt 5.5 to opus 4.7 for coding (haven’t tried 4.8 yet because what am I made of money?) and it is way way more token efficient.
When Claude says it’s “bloviating” it really means it
A big part of the problem is the harness which you allude to. Claude locks you into Claude code which is bloated and absolute ass at context management.
Once the Chinese models reach parity with the current generation models it’ll be a race to the bottom. Deepseek v-4 pro is right there but not quite. The models now are strong enough to be generalist problem solvers. Anything stronger will only benefit niche applications.
I’d like to see something like the gpt chat interface within a coding harness where the model is capable of selecting what to delegate to based on the task. This is where we will go in the future. A lot of enterprises incinerate tokens with Claude because people are using opus to write emails or whatever. It’s like taking a Ferrari to the grocery store.
These will become commodities though I think at which point it’s all about the integrations. You can already see the big providers pivoting into these value added services with GPT leaning into the consumer market with apps and Claude going heavy on business/enterprise
It’s been a few months but I gave the then latest Claude Opus and GPT Max the same task and they both used the same absurd amount of tokens just to translate some curl commands to some other code. Gemini did about the same with like 10% of the tokens, so I’m a little impressed by them. I believe Claude is burning so many tokens just reading the entire code base to match house style.
You can do this with multimodal harnesses like opencode or pi by restricting the model for each agent you’ve configured. You’ve picked the model per agent, not the AI. But you could probably define duplicates per model per agent if you wanted.
I personally think Apple has the right long term idea with making consumer hardware that can run adequate local models. All the data centers can get fucked. I haven’t experimented with AI writing emails, but I suspect Gemma E2 Flash can do it just fine.
Yeah probably a better pi setup would be to create more granular agent types. I tried creating a worker powered by deepseek but found it wasn’t really that good. Something like an agent team spec (use a technical product manager agent to coordinate modules, review the worker output, and to make necessary corrections) then you could use a frontier model to plan and issue work instructions to the TPM agent
Not sure if this would be more economical in practice but would be interesting to try
Long run I think you will always need the data centers to handle big training loads but we might go back to on-prem computing for enterprises to run frontier models. Or even edge nodes that have enough muscle that folks can subscribe to locally.
Something in your city you can sub to for $30 a month that lets you run any flagship open source model could be very compelling
The admins at db0 have some sort of AI service mesh running already so thats way more viable than you’d think.
I don’t use AI for anything except translating language. I just use the local models. I have no fucking idea how an entire language (and the info needed to properly transform to another language) can be contained in a file of 10-70mb. But it works adequately on my trashy old hardware.
Also tried a bit of text to speech, transcription and similar. Those were more of a pain in the ass and needed a lot more stuff to download and set up but it isn’t clear to me if that is the nature of it or maybe the front end user experience just hasn’t been polished yet. But my laptop can handle it anyway. It seems like these companies really want the SAAS model and that is directing how they build. More than it being an inherent requirement.
To me it kind of seems like most individual people have a limited set of tasks they want to accomplish so you could just have whatever you need locally. Probably could even generate a custom local model and send it updates from an interface of some kind. If you want something to write emails or whatever you have it pull and digest what it needs once and then by incremental upgrades after that.
Its hard for me to imagine individual users really need so much power from a data center because its not like you are using the whole thing yourself. Probably data centers are really for huge comprehensive tasks that humans could never do unaided like analyzing massive datasets.
Yeah you’re kind of getting where I think or at least am hoping the market will go. You can already retrain public models locally but the tooling is either way too low level, or completely vibe coded.
I think the main contradiction in the market is that AI models are too big and expensive to run profitabily. But if they make them more efficient then the models can run on consumer hardware and that fucks up the SaaS AI business.
Don’t worry, all the data centers will be put back to use mining crypto as god intended.
Too expensive to run profitably + efficient models undermine the SaaS model + Chinese tech is always threatening to surpass them for a fraction of the cost. Even if they try to gatekeep efficient and local models, deepseek is right there.
Deepseek can be blocked trivially on national security grounds. Trivial in comparison to gatekeepibg local models because that’s just data that’s already out there.