3 ms·

I think the fundamental premise of this article is wrong, because I don't think the kind of thing chat GPT does is the same kind of thing that knowledge workers

I think the fundamental premise of this article is wrong, because I don't think the kind of thing chat GPT does is the same kind of thing that knowledge workers do. It isn't just a difference in reliability and talent in knowing what knowledge is relevant and understanding of niche knowledge, although those are also important factors that he doesn't weigh highly enough, it's also that there is a fundamental difference in kind between what's being done, making his comparison essentially a category error. Even if we assume that chat GPT doesn't hallucinate, it is an information retrieval system alone with some simple synthesis capabilities, whereas knowledge work is not just having the knowledge but having a full conceptual understaning of it and solving problems with creative application of thay knowledge and conceptual understanding — it's not just about being able to regurgitate a couple paragraphs of synthesized text or take a couple snippets from Stack Overflow and put them together, it's about actually understanding the meaniny and concepts of things and the principles behind them, and having a good understanding of the methods of reasoning and problem solving that you can self critically apply to your own thought processes in a self-correcting manner (which is something the structure of chatGPT precludes), and being able to creatively apply the knowledge you have, through the semantic lens of the things I just listed, in order to creatively solve a specific problem within a specific context.

logicprog 3y ago
It concerns me when people insist that chat GPT is doing the same thing humans are, or even the same thing knowledge workers are doing, because it indicates to me that a lot of people aren't aware of conceptual understanding and reasoning and critical thinking and problem solving and applying those things to your own thinking through metacognition in order to self-improve and self-correct, and are instead just acting by rote and regurgitating things essentially from memory with a little bit of synthesis, when they are capable of so much more and absolutely should be doing so much more. So perhaps the thing we've learned is that chat GPT has taken over the realm of some things only humans used to be able to do, but even though it fundamentally is not structured in a way to allow it to do the more advanced things humans do, so many people were just doing those basic things that it took over, that they can't tell the difference between a human and a large language model, and I think that's really kind of sad and upsetting.
mojomark 3y ago
"whereas knowledge work is not just having the knowledge but having a full conceptual understaning of it and solving problems [...] with creative application of thay knowledge and conceptual understanding" I think the point the author is making is that models (services) like ChatGPT are doing the former in your statement and it's up to the human user to do the latter. Application and allocation here are synonyms in this context. I use ChatGPT multiple times every day now, mostly to sift through knowledge and ask questions to make sure I understand that retrieved knowledge and also identify sources so that I can verify that knowledge if the truth isn't obvious. I mean, you can say whatever you want, but I personally find it tremendously useful productivity enhancement tool right now - otherwise I wouldn't waste my time with it.
- logicprog 3y ago
  First, a quick note: good God the number of typos in my messages is becoming unacceptable lol. Anyway, to your point: that's a fair enough interpretation of what he might be saying. So that's a fair point and maybe I shouldn't have bailed on the article so early. On a related note, I do fundamentally question the real helpfulness of using chat GPT to find and summarize knowledge for you, as opposed to the perceived helpfulness. Every time I've attempted to use it, or for instance someone has attempted to use it to summarize essays I've written, it has turned out data that is subtly wrong in a multitude of ways, often ones that are not individually super obvious, but which subtly pervert the meaning or import of whatever knowledge was being summarized and can very quickly add up to getting the wrong impression of something. So you might be receiving data that you think is helpful and useful, but is actually far less helpful than you think it is. And yes, you can double check things yourself, but that depends on knowing what to double check, which means that you either double check absolutely everything — and then it's equivalent to doing the research yourself! — or you only double check the things that jump out at you, in which case you're probably missing a whole lot. And that may have turned out fine so far, but that seems like a ticking time bomb to me. Even with asking for sources, that's a dangerous game to play, because it isn't actually giving you the actual sources of data used to generate the answers, because it has no memory or knowledge of that: links to the sources used were probably not embedded in the sources themselves, and so are probably not associated with them in the training data. So it's really just generating what it thinks a plausible answer to the question of sources might be, with no necessary relation to what it actually used. I often compare talking to chat GPT to talking to some kind of slick narcissist, who has only a passing knowledge of a huge range of subjects, gained through osmosis and half remembered details, but who isn't even really committed to giving you accurate knowledge, and is more committed to just saying whatever sounds right and is likely to make you happy. Or in this case, is likely to make the reward function go up. Not that I think there couldn't be a computer system that does what chat GPT does in theory, I just don't think this approach is correct for generating a reliable knowledge summarizing machine.

logicprog 3y ago

It concerns me when people insist that chat GPT is doing the same thing humans are, or even the same thing knowledge workers are doing, because it indicates to me that a lot of people aren't aware of conceptual understanding and reasoning and critical thinking and problem solving and applying those things to your own thinking through metacognition in order to self-improve and self-correct, and are instead just acting by rote and regurgitating things essentially from memory with a little bit of synthesis, when they are capable of so much more and absolutely should be doing so much more. So perhaps the thing we've learned is that chat GPT has taken over the realm of some things only humans used to be able to do, but even though it fundamentally is not structured in a way to allow it to do the more advanced things humans do, so many people were just doing those basic things that it took over, that they can't tell the difference between a human and a large language model, and I think that's really kind of sad and upsetting.

mojomark 3y ago

"whereas knowledge work is not just having the knowledge but having a full conceptual understaning of it and solving problems [...] with creative application of thay knowledge and conceptual understanding" I think the point the author is making is that models (services) like ChatGPT are doing the former in your statement and it's up to the human user to do the latter. Application and allocation here are synonyms in this context. I use ChatGPT multiple times every day now, mostly to sift through knowledge and ask questions to make sure I understand that retrieved knowledge and also identify sources so that I can verify that knowledge if the truth isn't obvious. I mean, you can say whatever you want, but I personally find it tremendously useful productivity enhancement tool right now - otherwise I wouldn't waste my time with it.

logicprog 3y ago
First, a quick note: good God the number of typos in my messages is becoming unacceptable lol. Anyway, to your point: that's a fair enough interpretation of what he might be saying. So that's a fair point and maybe I shouldn't have bailed on the article so early. On a related note, I do fundamentally question the real helpfulness of using chat GPT to find and summarize knowledge for you, as opposed to the perceived helpfulness. Every time I've attempted to use it, or for instance someone has attempted to use it to summarize essays I've written, it has turned out data that is subtly wrong in a multitude of ways, often ones that are not individually super obvious, but which subtly pervert the meaning or import of whatever knowledge was being summarized and can very quickly add up to getting the wrong impression of something. So you might be receiving data that you think is helpful and useful, but is actually far less helpful than you think it is. And yes, you can double check things yourself, but that depends on knowing what to double check, which means that you either double check absolutely everything — and then it's equivalent to doing the research yourself! — or you only double check the things that jump out at you, in which case you're probably missing a whole lot. And that may have turned out fine so far, but that seems like a ticking time bomb to me. Even with asking for sources, that's a dangerous game to play, because it isn't actually giving you the actual sources of data used to generate the answers, because it has no memory or knowledge of that: links to the sources used were probably not embedded in the sources themselves, and so are probably not associated with them in the training data. So it's really just generating what it thinks a plausible answer to the question of sources might be, with no necessary relation to what it actually used. I often compare talking to chat GPT to talking to some kind of slick narcissist, who has only a passing knowledge of a huge range of subjects, gained through osmosis and half remembered details, but who isn't even really committed to giving you accurate knowledge, and is more committed to just saying whatever sounds right and is likely to make you happy. Or in this case, is likely to make the reward function go up. Not that I think there couldn't be a computer system that does what chat GPT does in theory, I just don't think this approach is correct for generating a reliable knowledge summarizing machine.