7 ms·

Oh this is great, one more competitor with top model which will be available via API. I wonder what the pricing will be. OpenAI was slashing prices multiple tim

by ssijak 2y ago

Oh this is great, one more competitor with top model which will be available via API. I wonder what the pricing will be. OpenAI was slashing prices multiple times in the last year and a half I was using it.

mcintyre1994 2y ago
I can't imagine anyone would want to build on top of their APIs after they completed destroyed the Twitter API and its whole ecosystem.
- perbu 2y ago
  I assume it'll be a paid API so the "contract" is a lot more clear. Twitter never understood what to do with its API so pulling that particular rug makes sense. But I too wouldn't use this. X is playing fast and loose with ... everything, so having a business rely on their product seems risky.
- bboygravity 2y ago
  If they have the best model, everybody will use it. With LLMs (and AGI) it's really that simple: the company with the best model wins regardless of all else.
  - threeseed 2y ago
    It isn't that simple at all. It's going to be a combination of price, performance, quality, reliability, availability etc. And since the prompts need to be optimised for each model there is a degree of vendor lock-in.
    - bboygravity 2y ago
      I wasn't really talking about the marginal differences we see right now in August 2024. I'm talking about the next huge step forward that only 1 company will achieve, because it simply has the most GPUs (in limited supply) + energy source first and keeps that advantage. At some point this becomes a run-away self amplifying differentiator and it will make that company win regardless of all else. My money is on xAI in 2025. PS: the only reason prompts need to be optimized for each model is a symptom of models simply not being that good yet. This need will vanish in the near future as you get way better models. A recent hint of what I mean: mid-journey needed very elaborate prompt (and even loras) to get what you want. In flux that prompt can be much shorter (without loras) and it still gets closer to what you want. Same will happen with LLMs. Another example: with ChatGPT 4 you need to literally beg a model to only return what you ask for (for example JSON) or put it in a certain mode (JSON mode), in Claude Sonnet 3.5 it will simply just listen to what you ask for. So again: that's not "because every model needs model-specific fine-tuning" that's because previous models where simply not as good.
  - autonomousErwin 2y ago
    Best in what sense? Intelligence, speed, cost? Sometimes having a fast enough model at a low enough price makes you the obvious choice e.g. I know Claude is better than gpt-4o-mini but I use the latter for a lot more data processing because it's significantly cheaper and faster and the gains I'd get out of Claude seem somewhat marginal for my use case
    - andsoitis 2y ago
      > Best in what sense? Intelligence, speed, cost? Best at product / market fit. And that space is very very wide. Does the GenAI serve as a feature in a larger product (like realtime “reasoning” on X or in Apple’s case in iOS)? Is it a standalone product that general public or enterprises use? Does it play in a niche area? Etc.
- threeseed 2y ago
  Musk apparently lied about the DDOS that caused the X Trump stream failure. https://www.theverge.com/2024/8/12/24219121/donald-trump-elon-musk-interview-x-twitter-crashes https://www.theverge.com/2024/8/12/24219121/donald-trump-elo... If that's true it's not exactly the sort of behaviour you want from an API you're depending on.
  - chrisco255 2y ago
    This article is hearsay trash. It quotes an anonymous source saying "there was a “99 percent” chance Musk was lying about an attack.
    - mdhb 2y ago
      Let’s review the evidence then shall we: Evidence for DDOS: - Elon said so - the event in question very clearly had huge technical issues Evidence Against DDOS: - Elon said so - People who worked at Twitter said it was bullshit - every other spaces event that was run at the same time was unaffected. - no other part of the website was impacted in any way whatsoever.
      - hayd 2y ago
        - every other spaces event that was run at the same time was unaffected. - no other part of the website was impacted in any way whatsoever. Aren’t these last two an argument FOR a ddos attack? It seems reasonable to assume we’re there a ddos attack at that time it would be against the Elon/Trump stream explicitly.
        mdhb 2y ago
        I’d like to see an explanation of how that is even possible to get that level of targeting without knowing the connection details of either Elon or Trump. The rest of the attack surface is surely shared infrastructure with the rest of the website. So no I think it was just a straight up technical failure on their end.
- tgsovlerkhgsel 2y ago
  The nice thing with LLMs is that the API is relatively simple - for the most basic case, it's string in, string out. While you may need to redesign your prompt a bit, I bet for many use cases, LLMs are reasonably interchangeable, and the integration work required for an API change should be minimal.
  - vincnetas 2y ago
    Or like with ORM's you can start using intermediary library which unifies access to AI engines like Langchain4j (for java) and hides API details.
- miki123211 2y ago
  LLMs are pretty easy to switch, though. From a black box perspective, LLMs are pretty simple, you put text or images in, (possibly structured) text comes out, maybe with some tool invocations. If you use a good library for this, like Python's litellm for example, all it takes is changing one string in your code or config, as the library exposes most APIs of most providers under a simple, uniform interface. You might need to modify your prompt and run some evals on whatever task your app is solving, but even large companies regularly deprecate old models and introduce vastly better ones, so you should have a pipeline for that anyway. These models have very little "stickiness" or lock-in. If your app is a Twitter client and is built around the Twitter API, turning it into a Mastodon client built around the Mastodon API would take a lot of work. If your app uses Grok and is designed properly, switching over to a different model is so simple that it might be worth doing for half an hour during an outage.
  - createaccount99 2y ago
    From my experience, the system prompt matters a lot, and so it's not as simple as just switching.
  - cosmosgenius 2y ago
    Prompt to Output quality vary by a large amount between models IMO. The equivalent analogy would "lets switch programming language for this solved problem".
    - kloop 2y ago
      Sure, but to be consistent with the analogy, we're evoking the program from bash and it's been solved in several languages already. Trying it isn't exactly locking you into anything
  - maeil 2y ago
    The models are still of a level where for less common/benchmarked tasks, there's often only one model that's very good at it, and whichever is 2nd best is markedly worse, possibly to a degree where it's unusable for anything serious.
- bitshiftfaced 2y ago
  Those who would build on top of the API might be considering a couple of past changes that are significant, but not necessarily a reason to think they'll be further pain in the future: the company ownership changed, and those who train LLMs all of a sudden want all the human-created text on the internet.
- lopatamd 2y ago
  [dead]

by ssijak 2y ago

mcintyre1994 2y ago

I can't imagine anyone would want to build on top of their APIs after they completed destroyed the Twitter API and its whole ecosystem.

perbu 2y ago
I assume it'll be a paid API so the "contract" is a lot more clear. Twitter never understood what to do with its API so pulling that particular rug makes sense. But I too wouldn't use this. X is playing fast and loose with ... everything, so having a business rely on their product seems risky.
bboygravity 2y ago
If they have the best model, everybody will use it. With LLMs (and AGI) it's really that simple: the company with the best model wins regardless of all else.
- threeseed 2y ago
  It isn't that simple at all. It's going to be a combination of price, performance, quality, reliability, availability etc. And since the prompts need to be optimised for each model there is a degree of vendor lock-in.
  - bboygravity 2y ago
    I wasn't really talking about the marginal differences we see right now in August 2024. I'm talking about the next huge step forward that only 1 company will achieve, because it simply has the most GPUs (in limited supply) + energy source first and keeps that advantage. At some point this becomes a run-away self amplifying differentiator and it will make that company win regardless of all else. My money is on xAI in 2025. PS: the only reason prompts need to be optimized for each model is a symptom of models simply not being that good yet. This need will vanish in the near future as you get way better models. A recent hint of what I mean: mid-journey needed very elaborate prompt (and even loras) to get what you want. In flux that prompt can be much shorter (without loras) and it still gets closer to what you want. Same will happen with LLMs. Another example: with ChatGPT 4 you need to literally beg a model to only return what you ask for (for example JSON) or put it in a certain mode (JSON mode), in Claude Sonnet 3.5 it will simply just listen to what you ask for. So again: that's not "because every model needs model-specific fine-tuning" that's because previous models where simply not as good.
- autonomousErwin 2y ago
  Best in what sense? Intelligence, speed, cost? Sometimes having a fast enough model at a low enough price makes you the obvious choice e.g. I know Claude is better than gpt-4o-mini but I use the latter for a lot more data processing because it's significantly cheaper and faster and the gains I'd get out of Claude seem somewhat marginal for my use case
  - andsoitis 2y ago
    > Best in what sense? Intelligence, speed, cost? Best at product / market fit. And that space is very very wide. Does the GenAI serve as a feature in a larger product (like realtime “reasoning” on X or in Apple’s case in iOS)? Is it a standalone product that general public or enterprises use? Does it play in a niche area? Etc.
threeseed 2y ago
Musk apparently lied about the DDOS that caused the X Trump stream failure. https://www.theverge.com/2024/8/12/24219121/donald-trump-elon-musk-interview-x-twitter-crashes https://www.theverge.com/2024/8/12/24219121/donald-trump-elo... If that's true it's not exactly the sort of behaviour you want from an API you're depending on.
- chrisco255 2y ago
  This article is hearsay trash. It quotes an anonymous source saying "there was a “99 percent” chance Musk was lying about an attack.
  - mdhb 2y ago
    Let’s review the evidence then shall we: Evidence for DDOS: - Elon said so - the event in question very clearly had huge technical issues Evidence Against DDOS: - Elon said so - People who worked at Twitter said it was bullshit - every other spaces event that was run at the same time was unaffected. - no other part of the website was impacted in any way whatsoever.
    - hayd 2y ago
      - every other spaces event that was run at the same time was unaffected. - no other part of the website was impacted in any way whatsoever. Aren’t these last two an argument FOR a ddos attack? It seems reasonable to assume we’re there a ddos attack at that time it would be against the Elon/Trump stream explicitly.
      - mdhb 2y ago
        I’d like to see an explanation of how that is even possible to get that level of targeting without knowing the connection details of either Elon or Trump. The rest of the attack surface is surely shared infrastructure with the rest of the website. So no I think it was just a straight up technical failure on their end.
tgsovlerkhgsel 2y ago
The nice thing with LLMs is that the API is relatively simple - for the most basic case, it's string in, string out. While you may need to redesign your prompt a bit, I bet for many use cases, LLMs are reasonably interchangeable, and the integration work required for an API change should be minimal.
- vincnetas 2y ago
  Or like with ORM's you can start using intermediary library which unifies access to AI engines like Langchain4j (for java) and hides API details.
miki123211 2y ago
LLMs are pretty easy to switch, though. From a black box perspective, LLMs are pretty simple, you put text or images in, (possibly structured) text comes out, maybe with some tool invocations. If you use a good library for this, like Python's litellm for example, all it takes is changing one string in your code or config, as the library exposes most APIs of most providers under a simple, uniform interface. You might need to modify your prompt and run some evals on whatever task your app is solving, but even large companies regularly deprecate old models and introduce vastly better ones, so you should have a pipeline for that anyway. These models have very little "stickiness" or lock-in. If your app is a Twitter client and is built around the Twitter API, turning it into a Mastodon client built around the Mastodon API would take a lot of work. If your app uses Grok and is designed properly, switching over to a different model is so simple that it might be worth doing for half an hour during an outage.
- createaccount99 2y ago
  From my experience, the system prompt matters a lot, and so it's not as simple as just switching.
- cosmosgenius 2y ago
  Prompt to Output quality vary by a large amount between models IMO. The equivalent analogy would "lets switch programming language for this solved problem".
  - kloop 2y ago
    Sure, but to be consistent with the analogy, we're evoking the program from bash and it's been solved in several languages already. Trying it isn't exactly locking you into anything
- maeil 2y ago
  The models are still of a level where for less common/benchmarked tasks, there's often only one model that's very good at it, and whichever is 2nd best is markedly worse, possibly to a degree where it's unusable for anything serious.
bitshiftfaced 2y ago
Those who would build on top of the API might be considering a couple of past changes that are significant, but not necessarily a reason to think they'll be further pain in the future: the company ownership changed, and those who train LLMs all of a sudden want all the human-created text on the internet.
lopatamd 2y ago
[dead]