4 ms·

I think this is where people will be disappointed when “AI” is brought to mass consumer. The level of results does not scale well down to mass consumer hardwar

by dagmx 3y ago

I think this is where people will be disappointed when “AI” is brought to mass consumer. The level of results does not scale well down to mass consumer hardware. And yes I know people can buy an NVIDIA GPU and run these models, but the phone like you said is the most common computer and where this will be hardest to scale too. It’s why I’m bearish on AI, and I think the pop will be due to being unable to scale down sufficiently

klyrs 3y ago
> The level of results does not scale well down to mass consumer hardware. For now. I'm pretty bearish on all things "AI" but of all things one can say about the future, today's hardware is yesterday's news. And in this case, I'd say the same goes for algorithms.
- dagmx 3y ago
  I guess I should clarify: I think it’ll get better on mass consumer hardware. I just don’t think it will live up to the hype people have. People are seeing Sora and StableDiffusion when they think of AI. And yes I can run SD on my iPhone, but it’s a poor experience that’s difficult to productize. The first real products will be so underwhelming compared to what people expect. Eventually the hardware and algorithms will improve and meet in the middle, but I think it’s so far out, that people will have moved on
czl 3y ago
With time mass consumer hardware scales up. You expect that trend to stop?
- dagmx 3y ago
  I think it’s not really relevant though to the current timeframe. The expectation of the current bubble is that something will be delivered soon (in the next year or two) I don’t see mass consumer hardware scaling up quickly enough in that time to have a product that will match the hype that everyone is showing with cloud based tools.
  - exe34 3y ago
    > The expectation of the current bubble is that something will be delivered soon (in the next year or two) Where have you seen this? And what is the claim that will be delivered? I'm absolutely certain something will be delivered in the next year or two. It's probably not AGI.
tavavex 3y ago
> The level of results does not scale well down to mass consumer hardware. I think that it does, though. A few years ago, running any remotely complex (and decent) generative AI on anything that wasn't a large compute server was out of the question. Nowadays, you can run a very respectable image or text generation algorithm on a middle-of-the-road gaming PC. Local models may not always be as good as the enormous things that companies run, but they're putting up a fight. This isn't founded on research, but given how much people were able to scale all of it down, I feel like there's going to be more of this "fat" to trim - data that takes up a lot of space but isn't very important to the final result. Add onto that the constant improvement of hardware, and the lines are going to intersect eventually.
imtringued 3y ago
Strix Point isn't even out yet. Sure that is a laptop form factor and you won't be able to run those 130b models because of RAM constraints, but things like mixtral 8x7b will easily run at 20 tokens per second locally on your laptop within this year.
- dagmx 3y ago
  And what will 20t/s give the user as a user experience? I don’t mean to sound combative, but my point is that the disconnect between what customers will get and what they see is very high. Will your metric meaningfully change that aspect?
  - exe34 3y ago
    That's faster than you can read, isn't it? Or at least fast enough to deliver through voice (which is even slower than reading).

by dagmx 3y ago

klyrs 3y ago

> The level of results does not scale well down to mass consumer hardware. For now. I'm pretty bearish on all things "AI" but of all things one can say about the future, today's hardware is yesterday's news. And in this case, I'd say the same goes for algorithms.

dagmx 3y ago
I guess I should clarify: I think it’ll get better on mass consumer hardware. I just don’t think it will live up to the hype people have. People are seeing Sora and StableDiffusion when they think of AI. And yes I can run SD on my iPhone, but it’s a poor experience that’s difficult to productize. The first real products will be so underwhelming compared to what people expect. Eventually the hardware and algorithms will improve and meet in the middle, but I think it’s so far out, that people will have moved on

czl 3y ago

With time mass consumer hardware scales up. You expect that trend to stop?

dagmx 3y ago
I think it’s not really relevant though to the current timeframe. The expectation of the current bubble is that something will be delivered soon (in the next year or two) I don’t see mass consumer hardware scaling up quickly enough in that time to have a product that will match the hype that everyone is showing with cloud based tools.
- exe34 3y ago
  > The expectation of the current bubble is that something will be delivered soon (in the next year or two) Where have you seen this? And what is the claim that will be delivered? I'm absolutely certain something will be delivered in the next year or two. It's probably not AGI.

tavavex 3y ago

> The level of results does not scale well down to mass consumer hardware. I think that it does, though. A few years ago, running any remotely complex (and decent) generative AI on anything that wasn't a large compute server was out of the question. Nowadays, you can run a very respectable image or text generation algorithm on a middle-of-the-road gaming PC. Local models may not always be as good as the enormous things that companies run, but they're putting up a fight. This isn't founded on research, but given how much people were able to scale all of it down, I feel like there's going to be more of this "fat" to trim - data that takes up a lot of space but isn't very important to the final result. Add onto that the constant improvement of hardware, and the lines are going to intersect eventually.

imtringued 3y ago

Strix Point isn't even out yet. Sure that is a laptop form factor and you won't be able to run those 130b models because of RAM constraints, but things like mixtral 8x7b will easily run at 20 tokens per second locally on your laptop within this year.

dagmx 3y ago
And what will 20t/s give the user as a user experience? I don’t mean to sound combative, but my point is that the disconnect between what customers will get and what they see is very high. Will your metric meaningfully change that aspect?
- exe34 3y ago
  That's faster than you can read, isn't it? Or at least fast enough to deliver through voice (which is even slower than reading).