3 ms·
You could have poured billions into developing space flight in the 1800s, but it doesn't mean you'd get a working spaceship once the funds ran out. I suspect w
by heavyset_go 2y ago
You could have poured billions into developing space flight in the 1800s, but it doesn't mean you'd get a working spaceship once the funds ran out.
I suspect we're in a similar situation with AGI.
- goethes_kind 2y agoWe got so far with transformers and huge amounts of compute and huge amounts of data. If we can get an architecture that is able to extract a slightly higher order of reasoning it will have a cascading effect when we apply the same level of compute and data. I see lots of potential for progressive improvement in this direction. Problem is, it's quite expensive to develop and test new architectures, but that's were the financial capital comes in.
- Jensson 2y agoIssue with AI is that most human tampering with the algorithms makes the end result worse, not better, it is really hard to get that tiny improvement.
- jasfi 2y agoYou can think of each of those as a bottleneck. The architecture (LLMs, transformers) was once the bottleneck, as was the amount of compute. From what I know the new bottleneck is the amount of quality data. Actually there was a breakthrough there too, because GPTs don't need supervised training.