5 ms·

As https://twitter.com/QuinnyPig/status/1729558866520658376 https://twitter.com/QuinnyPig/status/1729558866520658376 notes: > Amazon Q is launching in preview

by btown 3y ago

As https://twitter.com/QuinnyPig/status/1729558866520658376 https://twitter.com/QuinnyPig/status/1729558866520658376 notes: > Amazon Q is launching in preview for only $20 a month per user with a 10 user minimum. The road to "Go build!" increasingly has a tollbooth.

BiteCode_dev 3y ago
Given amazon's reputation, as a geek I'm not going to build anything on this. Or bard. Even if it's free. Open ai has the benefit of having a fresh track record.
- awsanswers 3y ago
  Ummmm
- vkou 3y ago
  Surely, you must be aware that Microsoft, who now runs OpenAI has a bit of a history of Embrace, Extend, Exterminate? Building on top of any of these platforms provided by trillion dollar companies is a sucker's game. The moment they decided your business looks tasty, they'll eat your lunch.
  - zavertnik 3y ago
    > Building on top of any of these platforms provided by trillion dollar companies is a sucker's game. Until local models reach the fidelity and speed that these megacorps offer, what choice does anyone actually have with respect to AI? I was under the impression that even if you get over the initial cost of hardware to achieve speed, the fidelity of your outputs would still be of a lower overall quality relative to GPT/Claude/Bard(maybe?). I could be 100% wrong though.
    - idonotknowwhy 3y ago
      The gap is closing. I'm finding goliath-120b does better than chat gpt 3.5 Nothing comes close to gpt4 though
      - zavertnik 3y ago
        For me, the gap between 3.5 and 4 is massive. If I'm stuck between using 3.5 and doing the work myself, more often than not, I'm choosing to do it myself. Not to imply 3.5 is unusable, its just my bar for minimum fidelity is closer to 4 than 3.5 with respect to tasks that I'm comfortable offloading onto an AI. What are you running goliath-120b on? Is it costly to run all day every day? How long does it take to complete an output? I've thought about building a multi GPU node for local LLMs but I always decide against it on the premise that the tech is so new I figure in the next 3-4 years we'll see specialized hardware combined with efficiency improvements that would make my node obsolete.
        idonotknowwhy 3y ago
        I run it on 2xRTX3090. I bought them used (probably ex-miners). > I always decide against it on the premise that the tech is so new I figure in the next 3-4 years we'll see specialized hardware combined with efficiency improvements that would make my node obsolete. You're probably right, this happened back in the day with bitcoin mining.
      - kristianp 3y ago
        How does Goliath-120b improve on llama2-70b by just combining two of them? https://huggingface.co/alpindale/goliath-120b?text=Hi https://huggingface.co/alpindale/goliath-120b?text=Hi. > An auto-regressive causal LM created by combining 2x finetuned Llama-2 70B into one.
        idonotknowwhy 3y ago
        I.. don't know. Even the creator of the model doesn't know why it worked out so well. It really is better (at reasoning) than the 70b models when I use it. Though some people reported that it makes spelling mistakes. P.S. This doesn't always work out well, people have tried swapping different layers randomly and it makes the models incoherent.
- rstupek 3y ago
  What reputation are you referencing?
  - jrockway 3y ago
    Maybe third-parties commingle their counterfeit knockoff AI models with Q in the fulfillment centers, and when you boot it up you have a chance of getting one of those instead of the real AI model you wanted (even though you made sure you selected the one that was "sold by and ships from amazon.com"). I am kidding. AWS has a reputation of being expensive and complicated, that's about it.
    - rstupek 3y ago
      Of course he said even if its free so probably not what he was referencing?
- Jtsummers 3y ago
  > Given amazon's reputation, as a geek I'm not going to build anything on this. Or bard. Bard is not Amazon's, which you may know but your comment implies is part of Amazon's portfolio. Bard is a Google product. Amazon, however, has a better track record compared to Google with respect to keeping services around. The main issues will be around cost effectiveness (versus self-hosting or alternate services).
- balls187 3y ago
  I was of the opposite opinion--does OpenAI's paid services prevent your queries and data from being used internally?
  - 93po 3y ago
    Yes
candiddevmike 3y ago
Would've been nice to see per request pricing as an option too.
zavertnik 3y ago
I'll take a tollbooth over something passive like ad injection every single day of the week.
- seydor 3y ago
  is that guaranteed?
  - zavertnik 3y ago
    The tollbooth? I would imagine so unless until the compute cost comes down and the hardware becomes more accessible/integrated at the consumer level. If you mean my preference for subscription over ads, that is guaranteed. I'm fine with an ad model for consuming content (like watching YouTube) but never with content generation (like using Photoshop). Plus, I really like these technologies and want to see them go further and I'm more than happy to pay for my product when the deal is good, which AI costs currently are relative to the hardware cost. Having to pay for these services + having big tech compete with each other for the best cutting edge release = a lot of money, time, and focus in that area to win the consumers on the merits of their products, whether that consumer is an enterprise customer or not. I don't see this kind of competition in any most other marketplaces for content generation tools, that's partially by virtue of AI being new tech but also because the race for dominating the AI marketplace has only just begun.
    - seydor 3y ago
      I mean , is it certain that advertising won't be injected?
collegeburner 3y ago
does quinn want companies to run large, expensive servers to do inference with no compensation? half the reason you're using services is because the hardware to do it locally isn't cheap. idk why he's kvetching about this when you also have to pay to host a web site, run a compute workload, whatever. but "muh bigcorp bad" ig

by btown 3y ago

BiteCode_dev 3y ago

Given amazon's reputation, as a geek I'm not going to build anything on this. Or bard. Even if it's free. Open ai has the benefit of having a fresh track record.

awsanswers 3y ago
Ummmm
vkou 3y ago
Surely, you must be aware that Microsoft, who now runs OpenAI has a bit of a history of Embrace, Extend, Exterminate? Building on top of any of these platforms provided by trillion dollar companies is a sucker's game. The moment they decided your business looks tasty, they'll eat your lunch.
- zavertnik 3y ago
  > Building on top of any of these platforms provided by trillion dollar companies is a sucker's game. Until local models reach the fidelity and speed that these megacorps offer, what choice does anyone actually have with respect to AI? I was under the impression that even if you get over the initial cost of hardware to achieve speed, the fidelity of your outputs would still be of a lower overall quality relative to GPT/Claude/Bard(maybe?). I could be 100% wrong though.
  - idonotknowwhy 3y ago
    The gap is closing. I'm finding goliath-120b does better than chat gpt 3.5 Nothing comes close to gpt4 though
    - zavertnik 3y ago
      For me, the gap between 3.5 and 4 is massive. If I'm stuck between using 3.5 and doing the work myself, more often than not, I'm choosing to do it myself. Not to imply 3.5 is unusable, its just my bar for minimum fidelity is closer to 4 than 3.5 with respect to tasks that I'm comfortable offloading onto an AI. What are you running goliath-120b on? Is it costly to run all day every day? How long does it take to complete an output? I've thought about building a multi GPU node for local LLMs but I always decide against it on the premise that the tech is so new I figure in the next 3-4 years we'll see specialized hardware combined with efficiency improvements that would make my node obsolete.
      - idonotknowwhy 3y ago
        I run it on 2xRTX3090. I bought them used (probably ex-miners). > I always decide against it on the premise that the tech is so new I figure in the next 3-4 years we'll see specialized hardware combined with efficiency improvements that would make my node obsolete. You're probably right, this happened back in the day with bitcoin mining.
    - kristianp 3y ago
      How does Goliath-120b improve on llama2-70b by just combining two of them? https://huggingface.co/alpindale/goliath-120b?text=Hi https://huggingface.co/alpindale/goliath-120b?text=Hi. > An auto-regressive causal LM created by combining 2x finetuned Llama-2 70B into one.
      - idonotknowwhy 3y ago
        I.. don't know. Even the creator of the model doesn't know why it worked out so well. It really is better (at reasoning) than the 70b models when I use it. Though some people reported that it makes spelling mistakes. P.S. This doesn't always work out well, people have tried swapping different layers randomly and it makes the models incoherent.
rstupek 3y ago
What reputation are you referencing?
- jrockway 3y ago
  Maybe third-parties commingle their counterfeit knockoff AI models with Q in the fulfillment centers, and when you boot it up you have a chance of getting one of those instead of the real AI model you wanted (even though you made sure you selected the one that was "sold by and ships from amazon.com"). I am kidding. AWS has a reputation of being expensive and complicated, that's about it.
  - rstupek 3y ago
    Of course he said even if its free so probably not what he was referencing?
Jtsummers 3y ago
> Given amazon's reputation, as a geek I'm not going to build anything on this. Or bard. Bard is not Amazon's, which you may know but your comment implies is part of Amazon's portfolio. Bard is a Google product. Amazon, however, has a better track record compared to Google with respect to keeping services around. The main issues will be around cost effectiveness (versus self-hosting or alternate services).
balls187 3y ago
I was of the opposite opinion--does OpenAI's paid services prevent your queries and data from being used internally?
- 93po 3y ago
  Yes

candiddevmike 3y ago

Would've been nice to see per request pricing as an option too.

zavertnik 3y ago

I'll take a tollbooth over something passive like ad injection every single day of the week.

seydor 3y ago
is that guaranteed?
- zavertnik 3y ago
  The tollbooth? I would imagine so unless until the compute cost comes down and the hardware becomes more accessible/integrated at the consumer level. If you mean my preference for subscription over ads, that is guaranteed. I'm fine with an ad model for consuming content (like watching YouTube) but never with content generation (like using Photoshop). Plus, I really like these technologies and want to see them go further and I'm more than happy to pay for my product when the deal is good, which AI costs currently are relative to the hardware cost. Having to pay for these services + having big tech compete with each other for the best cutting edge release = a lot of money, time, and focus in that area to win the consumers on the merits of their products, whether that consumer is an enterprise customer or not. I don't see this kind of competition in any most other marketplaces for content generation tools, that's partially by virtue of AI being new tech but also because the race for dominating the AI marketplace has only just begun.
  - seydor 3y ago
    I mean , is it certain that advertising won't be injected?

collegeburner 3y ago

does quinn want companies to run large, expensive servers to do inference with no compensation? half the reason you're using services is because the hardware to do it locally isn't cheap. idk why he's kvetching about this when you also have to pay to host a web site, run a compute workload, whatever. but "muh bigcorp bad" ig