3 ms·
Stochastic parrot is way too simplistic a dismissal. It's hand wavy. But I don't get what's so impressive about the nuanced language of a language model that h
by namaria 2y ago
Stochastic parrot is way too simplistic a dismissal. It's hand wavy.
But I don't get what's so impressive about the nuanced language of a language model that has been given datacenter amounts of compute and virtually all of written word ever put digitalized. Yeah it's the first actually functioning natural language interface. At what cost though. It's completely out of proportion with the benefits and only bubble level 'investing' can justify this.
- lxgr 2y agoAre you referring to the (one-time) training cost or the cost per inferred token? The latter is pretty acceptable these days, especially with smaller models.
- namaria 2y agoSmall models can't do impressive language corner cases. And the benefits of the models that can scale linearly while their cost grows geometrically. Also claiming training costs are 'one off' when there's so much resources being poured into training new and bigger models is disingenuous.
- mensetmanusman 2y agoThe resources pouring in still pale in comparison to capital expenditures in the semi conductor industry.
- namaria 2y agoThe inflation in semi conductor capital expenditure is the problem caused by the AI hype.
- mensetmanusman 2y agoNah, tsmc spending tens of billion annually on capital before OpenAI.