4 ms·
>most weights in a LLM are significantly overdetermined. Once you have ingested several terabytes of natural language, you know how to generate natural language
by mathisfun123 3y ago
>most weights in a LLM are significantly overdetermined. Once you have ingested several terabytes of natural language, you know how to generate natural language.
more than anything, AI/ML/DL enables people to make claims that cannot be refuted by any application of the scientific method. it's literally "not even wrong" in exactly the same way as people claim string theory is.
- jjoonathan 3y agoString theory wishes it had (unique) claims as easily falsifiable as "Once you have ingested several terabytes of natural language, you know how to generate natural language." Obviously you need a page of fine print to make a claim strictly falsifiable, but complaining about the absence of fine print in casual discussion is absurdly uncharitable unless you have reason to believe that agreeable fine print couldn't be drawn up and I'm 99% sure that in this case you have so such reason.
- mathisfun123 3y ago>String theory wishes are you a string theorist? are you sure that you understand the claims made by string theory and how falsifiable they are? conversely >fine print couldn't be drawn up and I'm 99% sure... where does your 99% surety derive from? As someone trained both in mathematical physics and DL, i'm here to burst your bubble: the falsifiability of both disciplines is exacty the same because they're both premised on knowledged derived of models with an astronomical number of free parameters. just to be clear: i'm not poopooing LLN here, i.e. the models themselves are fine, i'm talking about absurd meta-claims (like ops about generating language) derived of observing many such models.