3 ms·

At current prices, and considering these OS Models' performance, investing in local inference sounds like a bad idea.

by taytus 3mo ago

matheusmoreira 3mo ago
Current prices are insane but at this point I'm starting to feel like it's an existential issue. I'm not a US citizen. At any point the USA could come up with some arbitrary export controls. Not having a computer capable of running at least Qwen is starting to actually seem risky to me. At least it's going to be usable as a very high end gaming PC.
- awakeasleep 3mo ago
  Why would you buy and build everything before the low probability catastrophe strikes, though? You don’t get any benefit from switching early and you pay a big opportunity cost.
  - inigyou 3mo ago
    because as soon as it strikes computer hardware will be completely unavailable to buy?
    - CamperBob2 3mo ago
      Also, there's a nontrivial learning curve involved in running your own inference server, once you move past the casual-goofing-around-with-llama-server stage. If you care about not being a sharecropper on Sam's or Dario's plantation, you should consider learning the ropes. Even if you don't put these skills to immediate use in your day job. I didn't appreciate this until I started down that road myself.
      - matheusmoreira 3mo ago
        > If you care about not being a sharecropper on Sam's or Dario's plantation Couldn't have put it better myself. That's what all this comes down to. Owning the hardware, owning the inference. Not perpetually renting them out on a meter like in the dystopian future they're envisioning.
        inigyou 3mo ago
        You also have the option to not use AI
        CamperBob2 3mo ago
        Not in the future, not if you want to get paid.
        matheusmoreira 3mo ago
        Yeah but the truth is I don't want to go back to the pre-LLM world. I've been programming alone for over ten years. Having a coding buddy to talk to, collaborate with or just bounce ideas off of quite literally changed my life. I don't want to go back to solo programming, and my projects aren't exactly swimming in a sea of active contributors.
  - Lapel2742 3mo ago
    > low probability catastrophe There is also a low probability that someone enters peace negotiations solely to threaten the negotiators with death, yet here we are. With these guys it is: Better safe than sorry.
  - OtomotO 3mo ago
    Because you will not be the only one struggling to get the hardware in the "unlikely" case the POTUS blurts out another fart.
- alfiedotwtf 3mo ago
  > At any point the USA could come up with some arbitrary export controls lol his already happened with Fable!
jrm4 3mo ago
At current "proprietary inference company behavior," investing in local inference sounds like the exceedingly far more rational option. Long term predictability ought to far outweigh a few more cycles of performance.

At current prices, and considering these OS Models' performance, investing in local inference sounds like a bad idea.

by taytus 3mo ago

At current prices, and considering these OS Models' performance, investing in local inference sounds like a bad idea.

Current prices are insane but at this point I'm starting to feel like it's an existential issue. I'm not a US citizen. At any point the USA could come up with some arbitrary export controls. Not having a computer capable of running at least Qwen is starting to actually seem risky to me. At least it's going to be usable as a very high end gaming PC.

awakeasleep 3mo ago
Why would you buy and build everything before the low probability catastrophe strikes, though? You don’t get any benefit from switching early and you pay a big opportunity cost.
- inigyou 3mo ago
  because as soon as it strikes computer hardware will be completely unavailable to buy?
  - CamperBob2 3mo ago
    Also, there's a nontrivial learning curve involved in running your own inference server, once you move past the casual-goofing-around-with-llama-server stage. If you care about not being a sharecropper on Sam's or Dario's plantation, you should consider learning the ropes. Even if you don't put these skills to immediate use in your day job. I didn't appreciate this until I started down that road myself.
    - matheusmoreira 3mo ago
      > If you care about not being a sharecropper on Sam's or Dario's plantation Couldn't have put it better myself. That's what all this comes down to. Owning the hardware, owning the inference. Not perpetually renting them out on a meter like in the dystopian future they're envisioning.
      - inigyou 3mo ago
        You also have the option to not use AI
        CamperBob2 3mo ago
        Not in the future, not if you want to get paid.
        matheusmoreira 3mo ago
        Yeah but the truth is I don't want to go back to the pre-LLM world. I've been programming alone for over ten years. Having a coding buddy to talk to, collaborate with or just bounce ideas off of quite literally changed my life. I don't want to go back to solo programming, and my projects aren't exactly swimming in a sea of active contributors.
- Lapel2742 3mo ago
  > low probability catastrophe There is also a low probability that someone enters peace negotiations solely to threaten the negotiators with death, yet here we are. With these guys it is: Better safe than sorry.
- OtomotO 3mo ago
  Because you will not be the only one struggling to get the hardware in the "unlikely" case the POTUS blurts out another fart.
alfiedotwtf 3mo ago
> At any point the USA could come up with some arbitrary export controls lol his already happened with Fable!

jrm4 3mo ago

At current "proprietary inference company behavior," investing in local inference sounds like the exceedingly far more rational option. Long term predictability ought to far outweigh a few more cycles of performance.