3 ms·

It is worth doing some ML courses to understand what is going on. I am beginning this journey, and it takes a lot of the magic out of it. Really what is happe

It is worth doing some ML courses to understand what is going on. I am beginning this journey, and it takes a lot of the magic out of it. Really what is happening is just mathematical operations: a lot of matrix multiplications, some simple non-linear functions (because linear algebra alone cannot represent all logic), some statistical stuff to stop the numbers getting out of control. Importantly there are millions or billions of "magic numbers" that get updated as the model learns called parameters. Whether you train the network on representation A of the image, or representation B of the image, doesn't change much philosophically. It is just a function f(x) applied before the function g(x, parameters), so you get g(f(X), parameters). Now you could argue, well it might mean "something". After all you can reduce our brains to being a big mathematical functions. Possibly. But I think in this case it is too simple. The AI is just looking at a slightly different representation. Similar to changing VS code from/to dark mode for humans, or something like that. And the modellers might be changing representations all the time anyway as they tinker with things.

raverbashing 3y ago
Yeah but this is an ML 101 view of the thing, so let me put it this way The topology of the (real) input data is part of the dimensionality reduction needed for the identification to work, especially at a given number of parameters Understanding how the data is connected in the input (there are image lines and they form a "shape") helps a lot and CNNs used this to their advantage Maybe a very powerful RNN could deduce this by itself, but it is wasted processing. GPT also uses the input topology to its favour when it uses tokens, process text sequentially, etc
- quickthrower2 3y ago
  If this is the case, and I believe you, then this implies: "bytes are NOT all you need". You need a good representation of your image for the network to work well. It might so happen that one of the popular formats, maybe JPEG does a good job. But a dedicated ML person could do better most likely. This is probably what you would classically call "feature engineering"? And this is all hyperparameter search stuff in a way.
chpmrc 3y ago
I took Andrew Ng’s famous Coursera ML course and I still find this stuff extremely fascinating and as close to magic as you can get (in the digital realm). I agree that a higher degree of understanding lessens the emotional impact but often you just need to pause for a second, look back and appreciate what we’ve accomplished as a species. Sometimes I get on a plane and mid flight I realize “wait a second, I’m in a metal box flying in the sky at 800 km/h and I can breathe, eat, drink and watch tv” and I get goosebumps, even though I’m kind of familiar with the physics of it. Maybe some things just have an inherent “awe factor” and no matter how well you understand them you still get those butterflies. Or maybe I’m just blabbering :)
pmoriarty 3y ago
"I am beginning this journey, and it takes a lot of the magic out of it. Really what is happening is just mathematical operations: a lot of matrix multiplications, some simple non-linear functions (because linear algebra alone cannot represent all logic), some statistical stuff to stop the numbers getting out of control." And all that happens in the human brain is "just" chemical reactions. All that happens on the Earth is "just" chemistry.
- quickthrower2 3y ago
  Also I said "Now you could argue, well it might mean "something". After all you can reduce our brains to being a big mathematical functions. Possibly."
  - hombre_fatal 3y ago
    Well, it’s not just an aside. It takes all the wind out of your sails. What’s interesting here is the end result, not pointing out the basic elements the same reason reducing love or consciousness to chemical reactions doesn’t take away the mystique and wonder of those things. “Heh sorry to ruin it for you all :b but it’s all just chemical reactions” just isn’t as interesting as you might have been thinking when writing that comment.
- deleted 3y ago
  [deleted]

raverbashing 3y ago

Yeah but this is an ML 101 view of the thing, so let me put it this way The topology of the (real) input data is part of the dimensionality reduction needed for the identification to work, especially at a given number of parameters Understanding how the data is connected in the input (there are image lines and they form a "shape") helps a lot and CNNs used this to their advantage Maybe a very powerful RNN could deduce this by itself, but it is wasted processing. GPT also uses the input topology to its favour when it uses tokens, process text sequentially, etc

quickthrower2 3y ago
If this is the case, and I believe you, then this implies: "bytes are NOT all you need". You need a good representation of your image for the network to work well. It might so happen that one of the popular formats, maybe JPEG does a good job. But a dedicated ML person could do better most likely. This is probably what you would classically call "feature engineering"? And this is all hyperparameter search stuff in a way.

chpmrc 3y ago

I took Andrew Ng’s famous Coursera ML course and I still find this stuff extremely fascinating and as close to magic as you can get (in the digital realm). I agree that a higher degree of understanding lessens the emotional impact but often you just need to pause for a second, look back and appreciate what we’ve accomplished as a species. Sometimes I get on a plane and mid flight I realize “wait a second, I’m in a metal box flying in the sky at 800 km/h and I can breathe, eat, drink and watch tv” and I get goosebumps, even though I’m kind of familiar with the physics of it. Maybe some things just have an inherent “awe factor” and no matter how well you understand them you still get those butterflies. Or maybe I’m just blabbering :)

pmoriarty 3y ago

"I am beginning this journey, and it takes a lot of the magic out of it. Really what is happening is just mathematical operations: a lot of matrix multiplications, some simple non-linear functions (because linear algebra alone cannot represent all logic), some statistical stuff to stop the numbers getting out of control." And all that happens in the human brain is "just" chemical reactions. All that happens on the Earth is "just" chemistry.

quickthrower2 3y ago
Also I said "Now you could argue, well it might mean "something". After all you can reduce our brains to being a big mathematical functions. Possibly."
- hombre_fatal 3y ago
  Well, it’s not just an aside. It takes all the wind out of your sails. What’s interesting here is the end result, not pointing out the basic elements the same reason reducing love or consciousness to chemical reactions doesn’t take away the mystique and wonder of those things. “Heh sorry to ruin it for you all :b but it’s all just chemical reactions” just isn’t as interesting as you might have been thinking when writing that comment.
deleted 3y ago
[deleted]