4 ms·

I am migrating my company to private open source AI and building custom tooling, orchestration, an IDE and harness around it. The commercial labs have show it’s

I am migrating my company to private open source AI and building custom tooling, orchestration, an IDE and harness around it. The commercial labs have show it’s far too risky to build on top of them. With the labs moving into the app layer every interaction with the API related to product development or innovation is data they will steal and use to compete against you by adding those features to their everything apps. No one would hire a knowledge firm (law firm, accountants, management consultants) if they could steal your information and give it to your competitors. Their actions have remained me that we can’t tolerate that with AI systems. With spaceX buying cursor and Google buying windsurf, and gathering all the developer interaction as training data, there is too much risk that development process itself will be rug pulled.

aspenmartin 3mo ago
This makes sense but I am not comfortable with the open source picture either. It depends on the use case and long term strategy. If you're handling customer support tickets or something, beyond some capability I can't imagine the ROI would be worth needing the absolute frontier. You are then in a wonderfully lovely place: absolutely full control over the model and inference stack (if you want). But - Plenty of businesses are stuck between a rock and a hard place: make 3p models load bearing, or sacrifice performance to the competitors that are willing to swallow that risk - I just cannot imagine a viable equilibrium where OSS models compete on capabilities with the frontier without a hidden payer and shaky economics. Quant funds, some sovereign AI effort, cloud business, sanctioned distilled frontier models; all of these are demonstrably viable vehicles for OSS development. But the optimal position for these purposes is not to be the best, it's to be good enough (which I think is the position they find themselves in). I can imagine temporary points where open models pull ahead but not sustainably. I agree there are many many use cases where the optimal choice is to reach for open weight models (I assume yours is one of them). But the economics of frontier model development necessitates that these models are behind and thats a problem for other cases.
- eueeu 3mo ago
  You’ve been boosting frontier model development for months and talking aloud about escape velocity lmao. Take your ball and go home
  - aspenmartin 3mo ago
    I don't have a ball, where's my ball? Did you take my ball? I want my ball.
- digitaltrees 3mo ago
  I am not sure I understand your arguments. You seem to be taking both positions. I suspect that is because there are a few confounding variables and diverse needs AI solves and that makes a cohesive analysis hard to reach. I have a clear view that cuts through the complexity, whether you find it useful or persuasive is another matter but I’ll lay it out. 1. All intelligence has value. A Harvard MBA or Stanford CS engineer have value. Haiku has value, Opus has value. 2. Sometimes it makes sense to buy, build or rent capabilities. But lack of ownership of something your business requires to operate is an existential risk which requires ownership. 3. Current OSS models are good enough for substantial automation and productivity gains with the right orchestration and harness. Especially with domain experts in the loop. 4. Frontier models while better, introduce existential risk; so they are a temptation that should be ignored if you care about your business being able to continue to exist. Let’s invert this conclusion. Imagine you build a workflow around Fable 5 and the workflow going down would harm your business. Well the government has demonstrated ability and willingness to impose a multi day / week / month interruption in your business with no legally required notice, right to appeal or compensation. And further demonstrates a willingness to give preferential access to your competitors. For these reasons, the only viable path is a model you have on servers you control and can replace if needed. That means cloud providers are fine if you have the model archived somewhere.
  - aspenmartin 3mo ago
    I agree with everything you're saying. And I would also agree if you can run your business adequately on OSS models you should absolutely do that for exactly the reasons you say. All I mean is: frontier models will always have their place and I don't see a way for OSS to ever change that picture. I also worry that the economics that makes meaningful open weights model development possible may break someday but I don't really see a mechanism for this at this point. I don't actually have a use case in mind where you would _need_ or get a substantial competitive advantage of the absolute frontier in some sort of workflow or whatever load bearing DAG there is for your business. But if there are cases like this, that will suck.
    - digitaltrees 3mo ago
jmartrican 3mo ago
Sounds expensive
- digitaltrees 3mo ago
  It is. But cash expenditures can be modeled in the business economics. But depending on an unrelated party for a core capability is an existential threat. Imagine if every single computer your business runs on could be bricked without notice or a fallback. You’d go bankrupt within a day. That’s the signal this fiasco has sent. That being said, I am working on a federated / coop model that will allow smaller companies to pool resources.
  - jmartrican 3mo ago
    The concern would be the ever increasing cost of GPUs. I imagine to help mitigate the risk, having a setup where more GPU's can be added on, and the old ones can still be used (as new ones get added on) would make the endeavor more palatable. Vs. having to swap out old GPUs for new ones, because the money spent on the old ones is lost (or at least some of it is lost if you can find a way to resell them).
    - digitaltrees 3mo ago
      I was spending $3000 per month on AI and my team was restrained from using it because we don’t have that type of budget per person. Even $200 per person was a stretch. But we could buy enough hardware that it made sense per person

aspenmartin 3mo ago

This makes sense but I am not comfortable with the open source picture either. It depends on the use case and long term strategy. If you're handling customer support tickets or something, beyond some capability I can't imagine the ROI would be worth needing the absolute frontier. You are then in a wonderfully lovely place: absolutely full control over the model and inference stack (if you want). But - Plenty of businesses are stuck between a rock and a hard place: make 3p models load bearing, or sacrifice performance to the competitors that are willing to swallow that risk - I just cannot imagine a viable equilibrium where OSS models compete on capabilities with the frontier without a hidden payer and shaky economics. Quant funds, some sovereign AI effort, cloud business, sanctioned distilled frontier models; all of these are demonstrably viable vehicles for OSS development. But the optimal position for these purposes is not to be the best, it's to be good enough (which I think is the position they find themselves in). I can imagine temporary points where open models pull ahead but not sustainably. I agree there are many many use cases where the optimal choice is to reach for open weight models (I assume yours is one of them). But the economics of frontier model development necessitates that these models are behind and thats a problem for other cases.

eueeu 3mo ago
You’ve been boosting frontier model development for months and talking aloud about escape velocity lmao. Take your ball and go home
- aspenmartin 3mo ago
  I don't have a ball, where's my ball? Did you take my ball? I want my ball.
digitaltrees 3mo ago
I am not sure I understand your arguments. You seem to be taking both positions. I suspect that is because there are a few confounding variables and diverse needs AI solves and that makes a cohesive analysis hard to reach. I have a clear view that cuts through the complexity, whether you find it useful or persuasive is another matter but I’ll lay it out. 1. All intelligence has value. A Harvard MBA or Stanford CS engineer have value. Haiku has value, Opus has value. 2. Sometimes it makes sense to buy, build or rent capabilities. But lack of ownership of something your business requires to operate is an existential risk which requires ownership. 3. Current OSS models are good enough for substantial automation and productivity gains with the right orchestration and harness. Especially with domain experts in the loop. 4. Frontier models while better, introduce existential risk; so they are a temptation that should be ignored if you care about your business being able to continue to exist. Let’s invert this conclusion. Imagine you build a workflow around Fable 5 and the workflow going down would harm your business. Well the government has demonstrated ability and willingness to impose a multi day / week / month interruption in your business with no legally required notice, right to appeal or compensation. And further demonstrates a willingness to give preferential access to your competitors. For these reasons, the only viable path is a model you have on servers you control and can replace if needed. That means cloud providers are fine if you have the model archived somewhere.
- aspenmartin 3mo ago
  I agree with everything you're saying. And I would also agree if you can run your business adequately on OSS models you should absolutely do that for exactly the reasons you say. All I mean is: frontier models will always have their place and I don't see a way for OSS to ever change that picture. I also worry that the economics that makes meaningful open weights model development possible may break someday but I don't really see a mechanism for this at this point. I don't actually have a use case in mind where you would _need_ or get a substantial competitive advantage of the absolute frontier in some sort of workflow or whatever load bearing DAG there is for your business. But if there are cases like this, that will suck.
  - digitaltrees 3mo ago

jmartrican 3mo ago

Sounds expensive

digitaltrees 3mo ago
It is. But cash expenditures can be modeled in the business economics. But depending on an unrelated party for a core capability is an existential threat. Imagine if every single computer your business runs on could be bricked without notice or a fallback. You’d go bankrupt within a day. That’s the signal this fiasco has sent. That being said, I am working on a federated / coop model that will allow smaller companies to pool resources.
- jmartrican 3mo ago
  The concern would be the ever increasing cost of GPUs. I imagine to help mitigate the risk, having a setup where more GPU's can be added on, and the old ones can still be used (as new ones get added on) would make the endeavor more palatable. Vs. having to swap out old GPUs for new ones, because the money spent on the old ones is lost (or at least some of it is lost if you can find a way to resell them).
  - digitaltrees 3mo ago
    I was spending $3000 per month on AI and my team was restrained from using it because we don’t have that type of budget per person. Even $200 per person was a stretch. But we could buy enough hardware that it made sense per person