3 ms·

I'm curious as to what engineering solutions you think are available to solve this problem? All the solutions I've heard up until now is "we just need more dat

I'm curious as to what engineering solutions you think are available to solve this problem? All the solutions I've heard up until now is "we just need more data" which isn't particularly sustainable or something a SWE would be in charge of doing

kcplate 2y ago
It’s not my expertise. However, if it’s a barrier to adoption it’s a problem to solve and will be. AI is exponentially improving and there is no reason to believe that this won’t continue.
- lelanthran 2y ago
  > AI is exponentially improving Not for the last two years at least. Over the last year it's been obvious that there are diminishing returns already... We increase the computational power by a factor of ten, and see a factor of 1.5 improvement.
- randomdata 2y ago
  > AI is exponentially improving The theoretical research may have improved exponentially. Implementations available for use seem to be on the decline – at best, stagnant. I find less and less utility as time goes on. Things LLMs shined at a couple of years ago now produce garbage. And for creative work, the output is much too formulaic, which was all well and good initially while still novel, but one has to keep pushing new boundaries and the current crop of tools really seems to struggle with that.
- PleasureBot 2y ago
  That's not how technology typically advances. Technology advances in waves. There's an initial breakthrough that leads to rapid progress in certain areas. But after some time there are diminishing returns as the the applications of that breakthrough are fully explored, and progress slows or stops until the next breakthrough. Predicting that the trajectory of progress for the next 20 years will match the progress that has been made in the first 2 years after the LLM breakthrough seems naive at best. I would also disagree with the idea that LLM tech is "exponentially improving" now. There was the initial release of ChatGPT which was an enormous step forward, but since then its been small iterations on the same fundamental technology. We are already seeing significantly diminishing returns in our ability to improve these models. Most available training data has already has already been fed in, 10xing the compute results in only marginal improvements in performance, etc.
  - kcplate 2y ago
    I guess we will see, but I can tell you after nearly 40 years of being in technology, I have never seen us go backwards, only forward and in the last 5 years forwards faster than ever. And with that experience I am advising my grandkids to look at trades for their future careers. Because there is zero doubt in my mind that nearly every job that requires a college education today is going to be gone in 20 years.

kcplate 2y ago

It’s not my expertise. However, if it’s a barrier to adoption it’s a problem to solve and will be. AI is exponentially improving and there is no reason to believe that this won’t continue.

lelanthran 2y ago
> AI is exponentially improving Not for the last two years at least. Over the last year it's been obvious that there are diminishing returns already... We increase the computational power by a factor of ten, and see a factor of 1.5 improvement.
randomdata 2y ago
> AI is exponentially improving The theoretical research may have improved exponentially. Implementations available for use seem to be on the decline – at best, stagnant. I find less and less utility as time goes on. Things LLMs shined at a couple of years ago now produce garbage. And for creative work, the output is much too formulaic, which was all well and good initially while still novel, but one has to keep pushing new boundaries and the current crop of tools really seems to struggle with that.
PleasureBot 2y ago
That's not how technology typically advances. Technology advances in waves. There's an initial breakthrough that leads to rapid progress in certain areas. But after some time there are diminishing returns as the the applications of that breakthrough are fully explored, and progress slows or stops until the next breakthrough. Predicting that the trajectory of progress for the next 20 years will match the progress that has been made in the first 2 years after the LLM breakthrough seems naive at best. I would also disagree with the idea that LLM tech is "exponentially improving" now. There was the initial release of ChatGPT which was an enormous step forward, but since then its been small iterations on the same fundamental technology. We are already seeing significantly diminishing returns in our ability to improve these models. Most available training data has already has already been fed in, 10xing the compute results in only marginal improvements in performance, etc.
- kcplate 2y ago
  I guess we will see, but I can tell you after nearly 40 years of being in technology, I have never seen us go backwards, only forward and in the last 5 years forwards faster than ever. And with that experience I am advising my grandkids to look at trades for their future careers. Because there is zero doubt in my mind that nearly every job that requires a college education today is going to be gone in 20 years.