5 ms·

Is it possible to get non-thinking only now, though? If not, why would that matter, since it's irrelevant?

jjani 1y ago
Yes, by setting the thinking budget to 0. Which is very common when a task doesn't need thinking. In addition, it's also relevant because for the last 3 months people have built things on top of this.
- amazingamazing 1y ago
  interesting - why wouldn't you use dynamic thinking? and yeah, sucks when the price changes.
  - dcre 1y ago
    It makes responses much slower with zero benefit for many tasks. Flash with thinking off is very fast.
- Workaccount2 1y ago
  To be fair, the point of preview models and stable releases is so you know what is stable to build on.
  - Aeolun 1y ago
    The moment you start charging for preview stuff I think you give a tacit agreement that you can expect the price to not increase by a factor of 4.
    - woleium 1y ago
      that’s a somewhat naïve viewpoint.
      - Aeolun 1y ago
        I think the fact that everyone is like ‘wtf’ now kind of reinforces my viewpoint? Doesn’t mean you can’t do it, but people won’t be happy.
        woleium 1y ago
        Who cares about happy (in the short term), as long as they continue to pay.
  - jjani 1y ago
    Gmail was in beta for what, 2 decades? Did you never use it during that time? They've been using these "Preview" models on their non-technical user facing Gemini app and product for months now. Like, Google themselves has been using them in production, on their main apps. And gemini-1.5-pro is 2 months from depreciation and there was no production alternative. They told everyone to build their stuff on top of it, and then jacked up the price by 4x. Just pointing to some fine print doesn't change that.
  - vardump 1y ago
    I'd be more worried about Google just discontinuing another product. For example Stadia was similarly high profile, but it's gone now. More examples here: https://killedbygoogle.com/ https://killedbygoogle.com/
drag0s 1y ago
one example where non-thinking matters would be latency-sensitive workflows, for example voice AI.
- jjani 1y ago
  Correct, though pretty much anything end-user facing is latency-sensitive, voice is a tiny percentage. No one likes waiting, the involvement of an LLM doesn't change this from a user PoV.
  - eru 1y ago
    I wonder if you can hide the latency, especially for voice? What I have in mind is to start the voice response with a non-thinking model, say a sentence or two in a fraction of a second. That will take the voice model a few seconds to read out. In that time, you use a thinking model to start working on the next part of the response? In a sense, very similar to how everyone knows to stall in an interview by starting with 'this is a very good question...', and using that time to think some more.

Yes, by setting the thinking budget to 0. Which is very common when a task doesn't need thinking. In addition, it's also relevant because for the last 3 months people have built things on top of this.

amazingamazing 1y ago
interesting - why wouldn't you use dynamic thinking? and yeah, sucks when the price changes.
- dcre 1y ago
  It makes responses much slower with zero benefit for many tasks. Flash with thinking off is very fast.
Workaccount2 1y ago
To be fair, the point of preview models and stable releases is so you know what is stable to build on.
- Aeolun 1y ago
  The moment you start charging for preview stuff I think you give a tacit agreement that you can expect the price to not increase by a factor of 4.
  - woleium 1y ago
    that’s a somewhat naïve viewpoint.
    - Aeolun 1y ago
      I think the fact that everyone is like ‘wtf’ now kind of reinforces my viewpoint? Doesn’t mean you can’t do it, but people won’t be happy.
      - woleium 1y ago
        Who cares about happy (in the short term), as long as they continue to pay.
- jjani 1y ago
  Gmail was in beta for what, 2 decades? Did you never use it during that time? They've been using these "Preview" models on their non-technical user facing Gemini app and product for months now. Like, Google themselves has been using them in production, on their main apps. And gemini-1.5-pro is 2 months from depreciation and there was no production alternative. They told everyone to build their stuff on top of it, and then jacked up the price by 4x. Just pointing to some fine print doesn't change that.
- vardump 1y ago
  I'd be more worried about Google just discontinuing another product. For example Stadia was similarly high profile, but it's gone now. More examples here: https://killedbygoogle.com/ https://killedbygoogle.com/

drag0s 1y ago

one example where non-thinking matters would be latency-sensitive workflows, for example voice AI.

jjani 1y ago
Correct, though pretty much anything end-user facing is latency-sensitive, voice is a tiny percentage. No one likes waiting, the involvement of an LLM doesn't change this from a user PoV.
- eru 1y ago
  I wonder if you can hide the latency, especially for voice? What I have in mind is to start the voice response with a non-thinking model, say a sentence or two in a fraction of a second. That will take the voice model a few seconds to read out. In that time, you use a thinking model to start working on the next part of the response? In a sense, very similar to how everyone knows to stall in an interview by starting with 'this is a very good question...', and using that time to think some more.