3 ms·

I think the article nails it when they talk about requirements docs being useless: people don't know what they want/need. Same in "the old days" of software de

I think the article nails it when they talk about requirements docs being useless: people don't know what they want/need. Same in "the old days" of software development, same now with LLMs. It certainly gives me pause for thought - while LLMs right now are worse than useless due to hallucinations meaning you cannot trust a single thing they say, I am sure that will improve over the years. But if people cannot even express or articulate what they want (even well-educated, highly-literate people) then I can't see LLMs replacing too many jobs in the near to mid term.

perryizgr8 3y ago
> while LLMs right now are worse than useless due to hallucinations meaning you cannot trust a single thing they say This is absolutely not true. You should try Bing copilot (or whatever they are calling it this week). It provides extensive citations for everything it asserts. You can easily verify facts by following the hyperlinks. I agree that the original Chatgpt interface is very hard to trust on factual matters due to the lack of citations.
Falkon1313 3y ago
>LLMs right now are worse than useless due to hallucinations meaning you cannot trust a single thing they say I think this supports the article quite well too. This idea that LLMs 'hallucinate' or that you can't 'trust' them is a clear illustration of the underlying problem. I've never seen an LLM hallucinate or give an untrustworthy response. But I've seen many many people claim that they do. Those people are usually attempting to use a language model to do mathematical calculations, instead of using a calculator. Or to lookup facts, instead of using a database. Or to get an overview of a topic, instead of using an encyclopedia. So naturally, they have about as much luck as someone trying to chop down a tree with a phillips head screwdriver or trying to drive a screw into the wall with a chainsaw. Or even, trying to use a phillips head screwdriver to drive a phillips head screw, but doing so by holding the tip and banging the handle against the screw as if it were a hammer. Too many people aren't using the right tool for the job, and even if they are, they don't know how to use it correctly. That requires some understanding of the tool, some aptitude, and in the case of a language model, some literacy. But these LLMs are being marketed as if they are general purpose artificial intelligences that can do anything, when they're obviously not. And their users don't understand that and aren't using them properly. However, if you use them to pseudorandomly generate natural sounding text, given their training and context, then they absolutely do in fact do that correctly. No 'hallucinations', no 'wrong answers'. They randomly generate natural enough text which fits the topic, pretty much every time. The issue is marketing them as something they're not, to people who aren't literate enough on the topic to know the difference. And if, as the article claims, half the people aren't even literate enough to use them, let alone understand when and how to use them, then it's not a good idea to use them as a standard interface nor to make promises about what they can do, when the people that would be using them wouldn't be able to make them do those things. Back in the day, you pretty much had to be able to understand and program a computer to use one. But imagine if your standard cellphone or laptop nowadays just booted up to a BASIC prompt or even a DOS prompt. Most people would be lost. These days we have UIs such that almost anyone can use them, without that understanding and without having to write their own programs. LLMs are still in that earlier stage. Where you need to understand them to get value out of them. But they're being dolled up and advertised as if you don't. I agree with you, they can't successfully replace jobs when people don't even know what they want or how to use them.
faeriechangling 3y ago
See the thing is I rarely get tripped up by hallucinations but it’s in part because I’m good at identifying when somebody is making crap up because my two strengths are verbal intelligence and pattern recognition. Just noticing logical inconsistencies in outputted code, idiosyncratic syntax, sloppiness, and so on is very natural to me. I thus benefit strongly from LLMs despite their untrustworthy unreliable nature as they paper over some of my other weaknesses. Yet if an essential quality to using LLMs is dealing with hallucinations your ability to detect them really limits your ability to use the tool. Some people believe the IRS wants them to buy Google Play gift cards from Walmart and redeem them.

perryizgr8 3y ago

> while LLMs right now are worse than useless due to hallucinations meaning you cannot trust a single thing they say This is absolutely not true. You should try Bing copilot (or whatever they are calling it this week). It provides extensive citations for everything it asserts. You can easily verify facts by following the hyperlinks. I agree that the original Chatgpt interface is very hard to trust on factual matters due to the lack of citations.

Falkon1313 3y ago

>LLMs right now are worse than useless due to hallucinations meaning you cannot trust a single thing they say I think this supports the article quite well too. This idea that LLMs 'hallucinate' or that you can't 'trust' them is a clear illustration of the underlying problem. I've never seen an LLM hallucinate or give an untrustworthy response. But I've seen many many people claim that they do. Those people are usually attempting to use a language model to do mathematical calculations, instead of using a calculator. Or to lookup facts, instead of using a database. Or to get an overview of a topic, instead of using an encyclopedia. So naturally, they have about as much luck as someone trying to chop down a tree with a phillips head screwdriver or trying to drive a screw into the wall with a chainsaw. Or even, trying to use a phillips head screwdriver to drive a phillips head screw, but doing so by holding the tip and banging the handle against the screw as if it were a hammer. Too many people aren't using the right tool for the job, and even if they are, they don't know how to use it correctly. That requires some understanding of the tool, some aptitude, and in the case of a language model, some literacy. But these LLMs are being marketed as if they are general purpose artificial intelligences that can do anything, when they're obviously not. And their users don't understand that and aren't using them properly. However, if you use them to pseudorandomly generate natural sounding text, given their training and context, then they absolutely do in fact do that correctly. No 'hallucinations', no 'wrong answers'. They randomly generate natural enough text which fits the topic, pretty much every time. The issue is marketing them as something they're not, to people who aren't literate enough on the topic to know the difference. And if, as the article claims, half the people aren't even literate enough to use them, let alone understand when and how to use them, then it's not a good idea to use them as a standard interface nor to make promises about what they can do, when the people that would be using them wouldn't be able to make them do those things. Back in the day, you pretty much had to be able to understand and program a computer to use one. But imagine if your standard cellphone or laptop nowadays just booted up to a BASIC prompt or even a DOS prompt. Most people would be lost. These days we have UIs such that almost anyone can use them, without that understanding and without having to write their own programs. LLMs are still in that earlier stage. Where you need to understand them to get value out of them. But they're being dolled up and advertised as if you don't. I agree with you, they can't successfully replace jobs when people don't even know what they want or how to use them.

faeriechangling 3y ago

See the thing is I rarely get tripped up by hallucinations but it’s in part because I’m good at identifying when somebody is making crap up because my two strengths are verbal intelligence and pattern recognition. Just noticing logical inconsistencies in outputted code, idiosyncratic syntax, sloppiness, and so on is very natural to me. I thus benefit strongly from LLMs despite their untrustworthy unreliable nature as they paper over some of my other weaknesses. Yet if an essential quality to using LLMs is dealing with hallucinations your ability to detect them really limits your ability to use the tool. Some people believe the IRS wants them to buy Google Play gift cards from Walmart and redeem them.