4 ms·

... but it's really just linear regression.

by sieste 3y ago

Maken 3y ago
With lots of data.
bee_rider 3y ago
And matrix multiplications. Premium GEMMs.
troelsSteegin 3y ago
So, "We'll call it AI to sell it, Machine Learning to Build it", and regression to make it work.
raverbashing 3y ago
No, it's like saying "computers are just switches turning off and on". It's a naive take And forgetting all the abstractions that make it work If it was we'd have LLMs 20 years ago. Not even mnist is "just linear regression" (because neither sigmoid nor relu are linear)
- JackFr 3y ago
  No, it's not a linear regression, but it is at its heart optimization. And yes, that is indeed like saying a computer is just a bunch of zeroes and ones and logic gates. It's true and beautiful and profound but nearly useless from a practical perspective. And the wonder, like computers when you scale the numbers of transistors to the billions, is that when you scale the number of parameters to the billions, you end up with something amazing.
  - Uehreka 3y ago
    Yeah but the people making these comments aren’t trying to point to the wonder of how simple mathematics can underpin large complex systems, they’re doing the opposite: Trying to trivialize the system and its immense potential for good and bad by pointing out that it uses simple mathematics under the hood (and thus can’t be that amazing).
    - TeMPOraL 3y ago
      No, they're not trying to deny the system is amazing. They're saying that it's not magic, and that we will soon be able to create comparably amazing systems ourselves.
- VHRanger 3y ago
  sigmoid is logistic regression ReLu is truncated regression It's all just GLMs!
Uehreka 3y ago
I don’t get what people are trying to say when they say these kinds of things about AI. That human-level writing is as simple as a linear regression? That we could’ve had computer programs capable of human-level writing decades ago? Have they not used these AIs enough to see how powerful they are? Are they seeing the bad outputs and thinking that AIs are always doing that poorly? Like seriously, if you’re telling me that it was obvious that a “linear regression” could pass the LSAT I’ve got a macvlan to sell you. Edit: they’re also literally not linear regression! https://youtu.be/Ae9EKCyI1xU?feature=shared https://youtu.be/Ae9EKCyI1xU?feature=shared
- VHRanger 3y ago
  Formally it's a generalized linear model with a constructed feature set. A "kitchen sink" regression with enough polynomial terms (x^2, x^3, etc.) and interaction terms (ab, (ab)^2, etc.) will be a function approximator the same way a neural net is. The computational mechanics are different (there's a reason we dont use it) but in the land of infinite computational power it can be made equivalent.
  - hcks 3y ago
    What's the point of this comment seriously. Making universal function approximators is trivial. It's not where the value lies.
  - tsroe 3y ago
    >in the land of infinite computational power it can be made equivalent I.e. they are not equivalent and the original comment was wrong.
  - Roark66 3y ago
    >The computational mechanics are different (there's a reason we dont use it) but in the land of infinite computational power it can be made equivalent. In the land of infinite computational power every computation is just a series of 1s and 0s added and subtracted. You can implement everything with just few more operations. But we don't live in a land of infinite computational power and it took us (as humanity) quite a while to discover things like transformer models. If we had the same hardware 10 years ago would we have discovered them back then? I very much doubt it. We didn't just need the hardware, we needed the labelled data sets, prior art in smaller models etc. Personally I think current AI/ML (LLMs, ESRGANs, and diffusion models) have huge capability to increase people's productivity, but it will not happen overnight and not for everyone. People have to learn to use AI/ML. This brings me to the "dangers of AI". I laugh at all these ideas that "AI will become sentient and it will take over the world", but I'm genuinely fearful of the world where we've became so used to AI delivered by few "cloud providers" that we cannot do certain jobs without it. Just like you can't be a modern architect without Cad software, there may be time when you'll not be able do any job without your "AI assistant". Now, what happens when there is essentially a monopoly on the market of "AI assistants"? They will start raising prices to the point in future paying off your "AI assistant" bill may be higher than your taxes and you'll have a choice of paying or not working at all. This is why we have to run these models locally and advance local use of them. Yes (not at all)OpenAI will give you access to a huge model for a fraction of the cost, but it's like with the proverbial drug dealers that gives you the first hit for free, you'll more than make up for the cost of it once you get hooked up. The "dangers of AI" is that it becomes too centralised, not "uncontrollable"
deleted 3y ago
[deleted]
extrememacaroni 3y ago
It's basically just math.
- viewtransform 3y ago
  high-dimensional non-linear math

... but it's really just linear regression.

by sieste 3y ago

... but it's really just linear regression.

Maken 3y ago

With lots of data.

bee_rider 3y ago

And matrix multiplications. Premium GEMMs.

troelsSteegin 3y ago

So, "We'll call it AI to sell it, Machine Learning to Build it", and regression to make it work.

raverbashing 3y ago

No, it's like saying "computers are just switches turning off and on". It's a naive take And forgetting all the abstractions that make it work If it was we'd have LLMs 20 years ago. Not even mnist is "just linear regression" (because neither sigmoid nor relu are linear)

JackFr 3y ago
No, it's not a linear regression, but it is at its heart optimization. And yes, that is indeed like saying a computer is just a bunch of zeroes and ones and logic gates. It's true and beautiful and profound but nearly useless from a practical perspective. And the wonder, like computers when you scale the numbers of transistors to the billions, is that when you scale the number of parameters to the billions, you end up with something amazing.
- Uehreka 3y ago
  Yeah but the people making these comments aren’t trying to point to the wonder of how simple mathematics can underpin large complex systems, they’re doing the opposite: Trying to trivialize the system and its immense potential for good and bad by pointing out that it uses simple mathematics under the hood (and thus can’t be that amazing).
  - TeMPOraL 3y ago
    No, they're not trying to deny the system is amazing. They're saying that it's not magic, and that we will soon be able to create comparably amazing systems ourselves.
VHRanger 3y ago
sigmoid is logistic regression ReLu is truncated regression It's all just GLMs!

Uehreka 3y ago

I don’t get what people are trying to say when they say these kinds of things about AI. That human-level writing is as simple as a linear regression? That we could’ve had computer programs capable of human-level writing decades ago? Have they not used these AIs enough to see how powerful they are? Are they seeing the bad outputs and thinking that AIs are always doing that poorly? Like seriously, if you’re telling me that it was obvious that a “linear regression” could pass the LSAT I’ve got a macvlan to sell you. Edit: they’re also literally not linear regression! https://youtu.be/Ae9EKCyI1xU?feature=shared https://youtu.be/Ae9EKCyI1xU?feature=shared

VHRanger 3y ago
Formally it's a generalized linear model with a constructed feature set. A "kitchen sink" regression with enough polynomial terms (x^2, x^3, etc.) and interaction terms (ab, (ab)^2, etc.) will be a function approximator the same way a neural net is. The computational mechanics are different (there's a reason we dont use it) but in the land of infinite computational power it can be made equivalent.
- hcks 3y ago
  What's the point of this comment seriously. Making universal function approximators is trivial. It's not where the value lies.
- tsroe 3y ago
  >in the land of infinite computational power it can be made equivalent I.e. they are not equivalent and the original comment was wrong.
- Roark66 3y ago
  >The computational mechanics are different (there's a reason we dont use it) but in the land of infinite computational power it can be made equivalent. In the land of infinite computational power every computation is just a series of 1s and 0s added and subtracted. You can implement everything with just few more operations. But we don't live in a land of infinite computational power and it took us (as humanity) quite a while to discover things like transformer models. If we had the same hardware 10 years ago would we have discovered them back then? I very much doubt it. We didn't just need the hardware, we needed the labelled data sets, prior art in smaller models etc. Personally I think current AI/ML (LLMs, ESRGANs, and diffusion models) have huge capability to increase people's productivity, but it will not happen overnight and not for everyone. People have to learn to use AI/ML. This brings me to the "dangers of AI". I laugh at all these ideas that "AI will become sentient and it will take over the world", but I'm genuinely fearful of the world where we've became so used to AI delivered by few "cloud providers" that we cannot do certain jobs without it. Just like you can't be a modern architect without Cad software, there may be time when you'll not be able do any job without your "AI assistant". Now, what happens when there is essentially a monopoly on the market of "AI assistants"? They will start raising prices to the point in future paying off your "AI assistant" bill may be higher than your taxes and you'll have a choice of paying or not working at all. This is why we have to run these models locally and advance local use of them. Yes (not at all)OpenAI will give you access to a huge model for a fraction of the cost, but it's like with the proverbial drug dealers that gives you the first hit for free, you'll more than make up for the cost of it once you get hooked up. The "dangers of AI" is that it becomes too centralised, not "uncontrollable"

deleted 3y ago

[deleted]

extrememacaroni 3y ago

It's basically just math.

viewtransform 3y ago
high-dimensional non-linear math