Critics argue frontier model training costs under $1 billion.
Can someone explain to me how open source models can keep up if ... - pre-training isn't saturated - it costs $2-4B to train a current gen model - distillation is increasingly hard as access to the most powerful models gets blocked ..
?
> - it costs $2-4B to train a current gen model
I'd like to see the mafs on that far as I can tell, "current gen models" are at most (90th percentile) ≈6X DeepSeek V4 Pro in M(active) and 10x in D. That's maaaybe $1B. And I mean Mythos, not Opus/5.5, those are 2-3x cheaper.
Can someone explain to me how open source models can keep up if ... - pre-training isn't saturated - it costs $2-4B to train a current gen model - distillation is increasingly hard as access to the most powerful models gets blocked ..
?
@martin_casado there are two large, capable, and well-resourced entities with clear strategic interests in ensuring open models keep up: China and Nvidia
preventing distillation and capturing market share are in tension. it'll be hard to distill GPT-7-BioChem, easy to distill Default Claude.
Can someone explain to me how open source models can keep up if ... - pre-training isn't saturated - it costs $2-4B to train a current gen model - distillation is increasingly hard as access to the most powerful models gets blocked ..
?
The debate on if open or closed models win comes down to if there is disproportionate value to marginally better intelligence.
The believers of this sit across from the open models will be good enough camp.
Closed models will stay slightly smarter. Open models will be cheaper.
Critics argue frontier model training costs under $1 billion.
Can someone explain to me how open source models can keep up if ... - pre-training isn't saturated - it costs $2-4B to train a current gen model - distillation is increasingly hard as access to the most powerful models gets blocked ..
?
Many users defended open source AI models as customizable, cheap, and good enough for privacy-focused tasks, while others dismissed their competitiveness with frontier systems and criticized labs for conspiring on high margins.
