9 ms·

Why things should dry up when contrary to fusion ai is already usable by millions daily ? Even if prpgress should stall a bit the product or fine-tunes or norma

by fvv 3y ago

Why things should dry up when contrary to fusion ai is already usable by millions daily ? Even if prpgress should stall a bit the product or fine-tunes or normal progress will still be super supeful , the "too soon" point has been surpassed

actionfromafar 3y ago
Depends on what you are looking for. I have this hesitation too. What we have and are on track for is useful and cool, but how far will we come in this spurt until we are back at slight incremental gains? Implementation wise in business, we are very early though. It feels like email in 1995, we have barely scratched the surface of what LLMs can mean for business and everyday life.
blagie 3y ago
A lot of previous plateaus in AI are usable and used by billions daily, for example, giving good navigation routes on your phone, managing NPCs in a video game, showing ads, or recommending movies. It's not that they don't have value -- they do, and in the trillions of dollars -- but once understood, they move from "AI" to "algorithms" and stop being exciting. The current progress feels different to me, though. The current step in capability is much higher than previous ones, as is the potential disruption.
- TheDudeMan 3y ago
  Yes. The thing that makes the current generation of AI different is that the architectures scale. Another $10 million in training effort WILL yield improvement. And Moore’s law pairs nicely with scaling behavior. In other words, there is currently no end in sight. Plus, algo advancements like this make things happen ever faster. Plus, increased VC money means more money to throw at hardware and more folks trying new things in software. Soon we’ll be replaced :(
- Al-Khwarizmi 3y ago
  I think what makes the current iteration of AI different is that we don't understand how the emerging abilities work. A map navigation algorithm: we understand it, we know where the limit is (basically it cannot do anything that isn't map navigation), so it stops being exciting. GPT: we don't understand it, we don't know where the limit is... And it doesn't seem it will stop being exciting until we do.
  - kybernetikos 3y ago
    People say this a lot - that we don't understand this or that, but I'm not really sure what they mean. We know exactly how these algorithms work. We know every calculation - the maths is not particularly difficult, we understand how the training process leads to information being stored in the weights, we know how inference works. What more would you want to understand before you would agree we understood it?
    - blagie 3y ago
      > We know exactly how these algorithms work. We have no idea how these algorithms work. > We know every calculation - the maths is not particularly difficult, We do know that. > we understand how the training process leads to information being stored in the weights, we know how inference works. We do not know that. > What more would you want to understand before you would agree we understood it? Let me give an analogy: We have an almost perfect understanding of transistors. If you hand me a Qualcomm mobile chipset in a black box, I'll have little or no understanding of how that allows me to make phone calls. Back in the day, I understood the x86 instruction set very well. However, if you gave me the binary of a video game, I'd have no idea how it worked. Neuroscientists understand the mathematics of how neurons work, imperfectly (but pretty well). For the sake of argument, we can pretend the models are perfect. We understand the neural wiring of simple organisms perfectly. We still have very little idea of how the human brain works. The algorithms in deep learning are evolved and have billions of parameters. We understand the general topology, and the math of individual neurons, but we have absolutely no idea how the things work as a system. Anyone who tells you they do is lying (very likely with no ill intent; they're probably deluding themselves as well). The people doing deep learning are, by and large, not brilliant mathematicians, of the type who did earlier AI. The math is simple compared to most of the convex optimization algorithms which came before (and could probably be made much better if those were applied). Even at the human level, a lot of work in deep learning is - randomly tweaking parameters, topologies, and algorithms - developing intuition (NOT theory) for which ones work better, and - bullshitting explanations for why that might be (which would, at best, pass for hypothesis in any scientific process) It's hard for me to emphasize how little we know about how or why these things work. We just set up a general framing which evolves well, and evolved it. An analogy would be if we set up a random number generator to write a piece of code, ran it 10^10^10 times, and picked the result which made the best wavelet transforms. We'd have no clue how it works. The only difference is (1) we have algorithms which are more tractable than randomly picking algorithms (2) we set up neural networks which evolve better than code, largely by virtue of being continuous rather than discrete.
ReptileMan 3y ago
Because suddenly the tech moves from world transformative to world enhancing. The potential profits from trillions to mere billions. From immortality to slightly longer lifespan.

by fvv 3y ago

actionfromafar 3y ago

Depends on what you are looking for. I have this hesitation too. What we have and are on track for is useful and cool, but how far will we come in this spurt until we are back at slight incremental gains? Implementation wise in business, we are very early though. It feels like email in 1995, we have barely scratched the surface of what LLMs can mean for business and everyday life.

blagie 3y ago

A lot of previous plateaus in AI are usable and used by billions daily, for example, giving good navigation routes on your phone, managing NPCs in a video game, showing ads, or recommending movies. It's not that they don't have value -- they do, and in the trillions of dollars -- but once understood, they move from "AI" to "algorithms" and stop being exciting. The current progress feels different to me, though. The current step in capability is much higher than previous ones, as is the potential disruption.

TheDudeMan 3y ago
Yes. The thing that makes the current generation of AI different is that the architectures scale. Another $10 million in training effort WILL yield improvement. And Moore’s law pairs nicely with scaling behavior. In other words, there is currently no end in sight. Plus, algo advancements like this make things happen ever faster. Plus, increased VC money means more money to throw at hardware and more folks trying new things in software. Soon we’ll be replaced :(
Al-Khwarizmi 3y ago
I think what makes the current iteration of AI different is that we don't understand how the emerging abilities work. A map navigation algorithm: we understand it, we know where the limit is (basically it cannot do anything that isn't map navigation), so it stops being exciting. GPT: we don't understand it, we don't know where the limit is... And it doesn't seem it will stop being exciting until we do.
- kybernetikos 3y ago
  People say this a lot - that we don't understand this or that, but I'm not really sure what they mean. We know exactly how these algorithms work. We know every calculation - the maths is not particularly difficult, we understand how the training process leads to information being stored in the weights, we know how inference works. What more would you want to understand before you would agree we understood it?
  - blagie 3y ago
    > We know exactly how these algorithms work. We have no idea how these algorithms work. > We know every calculation - the maths is not particularly difficult, We do know that. > we understand how the training process leads to information being stored in the weights, we know how inference works. We do not know that. > What more would you want to understand before you would agree we understood it? Let me give an analogy: We have an almost perfect understanding of transistors. If you hand me a Qualcomm mobile chipset in a black box, I'll have little or no understanding of how that allows me to make phone calls. Back in the day, I understood the x86 instruction set very well. However, if you gave me the binary of a video game, I'd have no idea how it worked. Neuroscientists understand the mathematics of how neurons work, imperfectly (but pretty well). For the sake of argument, we can pretend the models are perfect. We understand the neural wiring of simple organisms perfectly. We still have very little idea of how the human brain works. The algorithms in deep learning are evolved and have billions of parameters. We understand the general topology, and the math of individual neurons, but we have absolutely no idea how the things work as a system. Anyone who tells you they do is lying (very likely with no ill intent; they're probably deluding themselves as well). The people doing deep learning are, by and large, not brilliant mathematicians, of the type who did earlier AI. The math is simple compared to most of the convex optimization algorithms which came before (and could probably be made much better if those were applied). Even at the human level, a lot of work in deep learning is - randomly tweaking parameters, topologies, and algorithms - developing intuition (NOT theory) for which ones work better, and - bullshitting explanations for why that might be (which would, at best, pass for hypothesis in any scientific process) It's hard for me to emphasize how little we know about how or why these things work. We just set up a general framing which evolves well, and evolved it. An analogy would be if we set up a random number generator to write a piece of code, ran it 10^10^10 times, and picked the result which made the best wavelet transforms. We'd have no clue how it works. The only difference is (1) we have algorithms which are more tractable than randomly picking algorithms (2) we set up neural networks which evolve better than code, largely by virtue of being continuous rather than discrete.

ReptileMan 3y ago

Because suddenly the tech moves from world transformative to world enhancing. The potential profits from trillions to mere billions. From immortality to slightly longer lifespan.