3 ms·
I feel very frustrated with these takes because instead of grappling with what we're going to do about it (like having a conversation) it's a flat, dismissive d
by blurbleblurble 4y ago
I feel very frustrated with these takes because instead of grappling with what we're going to do about it (like having a conversation) it's a flat, dismissive denial, and it isn't even grounded in the science, which says that "memory augmented large language models are computationally universal". So at the very least we're dealing with algorithms that can do anything a hand written program can do, except that they've been trained to do it using natural language in extremely flexible ways. I'm having a hard time seeing how "fancy autocomplete" is the right description for this.
- maxdoop 4y agoI agree 100%. I don't understand why we can't look at the potential, or even current, capabilities of these LLMs and have a real conversation about how it might impact things. Yet so many folks here just confidently dismiss it. "It doesn't even think!" -- OK, define thinking? "It doesn't create novel ideas!" OK -- what do most devs do every day? "It is wrong sometimes!" OK -- is it wrong more or less often than an average dev?
- spacemadness 4y agoIn case we forget the amazing predictive capabilities of HN there’s always https://news.ycombinator.com/item?id=9224 https://news.ycombinator.com/item?id=9224
- enord 4y ago«Computationally universal»? Are you quite sure? Hoo boy pass the nitroglycerin hooooo aaah ohh after 80 years Turing was proven right after all ghasp
- tel 4y agohttps://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore...
- blurbleblurble 4y agoWell I don't understand the nitroglycerin reference and of course "computationally universal" doesn't mean "sentient" but the point is when you add the external memory they (Flan-U-PaLM 540B to be specific) have been demonstrated to be capable of simulating a specific, well defined Turing machine without special training. There are some other papers out there arguing this from a theoretical angle too, but this is the one whose title I quoted: https://arxiv.org/abs/2301.04589 https://arxiv.org/abs/2301.04589
- enord 4y ago5 thermostats in the right comfiguration can be a universal computer. Approximation, in the computational sense, is relative to some well defined function, which gives the approximation an ontology. The feasability of approximation is not disputed, but the content and nature of the ontology is a battlefield of motivated rhetoric. Yes, this is arguing semantics but in this particular case, semantics is the whole point.
- blurbleblurble 4y ago5 thermostats sounds like a very complex system, by the way!
- YeGoblynQueenne 4y ago>> I feel very frustrated with these takes because instead of grappling with what we're going to do about it (like having a conversation) it's a flat, dismissive denial, and it isn't even grounded in the science, which says that "memory augmented large language models are computationally universal" That's not what "the science says", it's the title of an article that someone put on arxiv. The article has no theoretical results, just a shoddy empirical demonstration of... something. The author claims that the something is an LLM simulating a Turing machine. But, is it? Really? Well, here's how the article concludes: >> Hopefully the reader has been convinced by this point. "Hopefully" is not how you show computational universality of a neural net architecture. This is how: https://www.sciencedirect.com/science/article/pii/S0022000085710136 https://www.sciencedirect.com/science/article/pii/S002200008... i.e. with maths. That article you link to is a bunch of hooey.
- blurbleblurble 4y agoThat's a good paper! Thanks for sharing.