4 ms·

This condescension is very common and very irritating: > Q. You say that the best experts of the future will be those who make the most use of AI. Are people w

This condescension is very common and very irritating: > Q. You say that the best experts of the future will be those who make the most use of AI. Are people who are waiting to use AI making a mistake? > A. I get it, it’s an unnerving technology. People are freaking out. They’re getting a sense of three sleepless nights and running away screaming. It feels like an essential threat to a lot of careers. I think if you’re a good journalist, the first time you think, “oh no.” But then you start to see how this could help you do things better than before. There are a lot of white-collar jobs where LLMs do more harm than good because a 1/4 hallucination rate means you waste too much time on wild goose chases. I briefly thought GPT-4 was useful for finding papers given a description of the results - I “kicked the tires” with some AI research and was very impressed. But when I tried to find papers on animal cognition, about 75% of the results were fictional, though supposedly authored by real animal cognition experts. And GPT-4o is even worse! The tools are just not good enough for my use case; Google Scholar is far more reliable. I just don’t understand the childish motivated reasoning behind assuming the skeptics are scared. Maybe if I spent “three sleepless nights” talking to ChatGPT I would be more enlightened.

simonw 2y ago
> I briefly thought GPT-4 was useful for finding papers given a description of the results. That’s one of the many poorly documented traps of LLMs: trying to use them to find papers like that is a fast-track to worthless hallucinations. If that was one of your first experiments I can’t blame you for thinking this tech is “more harm than good”. LLMs are terrible search engines… except for the times when they are great search engines! Learning when and what to use them for continues to be a significantly under-appreciated challenge.
- aithrowawaycomm 2y ago
  No, it was not my first experiment - again with this unbelievable condescension! I have been playing around with this stuff since GPT-3. That was my first practical use case where GPT wasn't a totally useless waste of money. It works very well with AI-related papers, maybe a 1% hallucination rate. I got the idea from an AI researcher and I was intrigued that ChatGPT might actually be useful for me. But it was not. The problem, as always with ANNs, is that I went slightly off the happy path. Even a skeptic like me assumed the AI papers successes was evidence that GPT-4 was better at remembering its pretraining data; instead I think it's evidence that a data contractor RLHFed the answers and GPT had it memorized. > LLMs are terrible search engines… except for the times when they are great search engines! But note that what you said about finding papers was wrong, it works extremely well for AI research. The reason LLMs are useless to me across the board is that these unpredictable and arbitrary limitations apply to everything, not just search. "Learning when and what to use them for" is pure trial-and-error because it seems to amount to guessing what tasks the 3rd-party data contractors trained the LLM to solve. I am not a Python or JavaScript developer, nor do I write code for extremely well-known libraries. I use F# for oddball projects (often analytics), and GPT-4 was utterly useless for F# codegen. My first experiments with GPT-3.5 showed that it would plagiarize hundreds of lines of public F# projects, including from my own GitHub, without any prompt engineering or trial-and-error - it was just blind plagiarism. GPT-4 isn't quite that bad, but it's still not even close to being good enough to help me - in particular it has no understanding of high-performance F#. I would be spending far more time auditing and optimizing its crappy code. And time spent writing code has never been the limiting factor in my F# development. I also do some recreational mathematics on finite geometry and combinatorial group theory; GPT-4 was utterly useless here, even with CoT prompting, and even though it solved more complex graduate-level algebra problems without any difficulty. Of course, those problems were repeated and solved in dozens of graduate textbooks. My cute little groups, not so much. I believe CoT prompting is theoretically incapable of helping GPT here since the computational complexity is too high. What CoT prompting gives you is a bunch of insidious errors that take time and effort to unravel. Otherwise there's nothing I do that would even conceivably benefit from an LLM: it can't play guitar, it can't play with my cats, and I would never use it to communicate with friends or family. I guess I could fill my brain with shallow subject knowledge about something, a few choice sentences. But I'd much rather understand something in depth by reading a book. I'm not too busy to read a book. Otherwise... maybe I could use LLMs to write polite no-thank-yous to unsolicited recruiters. This tech truly has nothing to offer me. I think you are failing to understand that, as a Python developer who maintains one of the biggest Python web frameworks and writes a popular blog for general tech audiences, LLMs are unusually well-suited towards your use cases, due to reasons that will not extend to people working in more isolated corners of the world.
  - simonw 2y ago
    "... LLMs are unusually well-suited towards your use cases" I'm very aware of that, and it's something I've been telling other people as well. If you are a software engineer who works primarily in languages that are well-represented in the training data (for me that's JavaScript and Python) you have a HUGE advantage available to you thanks to these tools, and you're possibly better positioned than any other profession to make the most of them. I just dumped Whisper transcripts from three recent podcast appearances I gave into a Claude project and asked it "Find direct quotes where I emphasize how useful LLMs are if you write Python and JavaScript" and it found this one: > "And that's great for me because the languages I use every day are Python and JavaScript and SQL. And those are the three languages that language models are best at." Running these kinds of fuzzy searches against transcripts is one of the many non-programming uses I have for this stuff now. Here's that actual quote in the video: https://www.youtube.com/watch?v=uRuLgar5XZw&t=757s https://www.youtube.com/watch?v=uRuLgar5XZw&t=757s
lolinder 2y ago
> There are a lot of white-collar jobs where LLMs do more harm than good because a 1/4 hallucination rate means you waste too much time on wild goose chases. I briefly thought GPT-4 was useful for finding papers given a description of the results - I “kicked the tires” with some AI research and was very impressed. But when I tried to find papers on animal cognition... This is less a question of which jobs benefit from AI in general and which don't than it is a question of tasks and specific tools. ChatGPT is not a search engine, so if you're looking for existing documents it's a very bad choice. But I've found myself using Perplexity—an LLM-powered search engine—more and more often because it reliably turns up results that Google fails to turn up. I suspect Perplexity is still also the wrong tool for scholarly articles, but that's not a fundamental limitation of the tech, it's just a question of the focus of the tools so far.
deleted 2y ago
[deleted]
dyauspitr 2y ago
What are you talking about? I haven’t had a hallucinated link since 3.5 to paper sources. They must have some sort of post processing in place to remove made up links.
rsynnott 2y ago
> I just don’t understand the childish motivated reasoning behind assuming the skeptics are scared. Maybe if I spent “three sleepless nights” talking to ChatGPT I would be more enlightened. A lot of people are very invested, whether emotionally or financially or both, in this stuff not being a flop. There’s a lot of motivated reasoning going on. It is necessary to believe that the heretics simply haven’t seen the light yet - to question that gets too close to questioning whether there’s any light to see.

simonw 2y ago

> I briefly thought GPT-4 was useful for finding papers given a description of the results. That’s one of the many poorly documented traps of LLMs: trying to use them to find papers like that is a fast-track to worthless hallucinations. If that was one of your first experiments I can’t blame you for thinking this tech is “more harm than good”. LLMs are terrible search engines… except for the times when they are great search engines! Learning when and what to use them for continues to be a significantly under-appreciated challenge.

aithrowawaycomm 2y ago
No, it was not my first experiment - again with this unbelievable condescension! I have been playing around with this stuff since GPT-3. That was my first practical use case where GPT wasn't a totally useless waste of money. It works very well with AI-related papers, maybe a 1% hallucination rate. I got the idea from an AI researcher and I was intrigued that ChatGPT might actually be useful for me. But it was not. The problem, as always with ANNs, is that I went slightly off the happy path. Even a skeptic like me assumed the AI papers successes was evidence that GPT-4 was better at remembering its pretraining data; instead I think it's evidence that a data contractor RLHFed the answers and GPT had it memorized. > LLMs are terrible search engines… except for the times when they are great search engines! But note that what you said about finding papers was wrong, it works extremely well for AI research. The reason LLMs are useless to me across the board is that these unpredictable and arbitrary limitations apply to everything, not just search. "Learning when and what to use them for" is pure trial-and-error because it seems to amount to guessing what tasks the 3rd-party data contractors trained the LLM to solve. I am not a Python or JavaScript developer, nor do I write code for extremely well-known libraries. I use F# for oddball projects (often analytics), and GPT-4 was utterly useless for F# codegen. My first experiments with GPT-3.5 showed that it would plagiarize hundreds of lines of public F# projects, including from my own GitHub, without any prompt engineering or trial-and-error - it was just blind plagiarism. GPT-4 isn't quite that bad, but it's still not even close to being good enough to help me - in particular it has no understanding of high-performance F#. I would be spending far more time auditing and optimizing its crappy code. And time spent writing code has never been the limiting factor in my F# development. I also do some recreational mathematics on finite geometry and combinatorial group theory; GPT-4 was utterly useless here, even with CoT prompting, and even though it solved more complex graduate-level algebra problems without any difficulty. Of course, those problems were repeated and solved in dozens of graduate textbooks. My cute little groups, not so much. I believe CoT prompting is theoretically incapable of helping GPT here since the computational complexity is too high. What CoT prompting gives you is a bunch of insidious errors that take time and effort to unravel. Otherwise there's nothing I do that would even conceivably benefit from an LLM: it can't play guitar, it can't play with my cats, and I would never use it to communicate with friends or family. I guess I could fill my brain with shallow subject knowledge about something, a few choice sentences. But I'd much rather understand something in depth by reading a book. I'm not too busy to read a book. Otherwise... maybe I could use LLMs to write polite no-thank-yous to unsolicited recruiters. This tech truly has nothing to offer me. I think you are failing to understand that, as a Python developer who maintains one of the biggest Python web frameworks and writes a popular blog for general tech audiences, LLMs are unusually well-suited towards your use cases, due to reasons that will not extend to people working in more isolated corners of the world.
- simonw 2y ago
  "... LLMs are unusually well-suited towards your use cases" I'm very aware of that, and it's something I've been telling other people as well. If you are a software engineer who works primarily in languages that are well-represented in the training data (for me that's JavaScript and Python) you have a HUGE advantage available to you thanks to these tools, and you're possibly better positioned than any other profession to make the most of them. I just dumped Whisper transcripts from three recent podcast appearances I gave into a Claude project and asked it "Find direct quotes where I emphasize how useful LLMs are if you write Python and JavaScript" and it found this one: > "And that's great for me because the languages I use every day are Python and JavaScript and SQL. And those are the three languages that language models are best at." Running these kinds of fuzzy searches against transcripts is one of the many non-programming uses I have for this stuff now. Here's that actual quote in the video: https://www.youtube.com/watch?v=uRuLgar5XZw&t=757s https://www.youtube.com/watch?v=uRuLgar5XZw&t=757s

lolinder 2y ago

> There are a lot of white-collar jobs where LLMs do more harm than good because a 1/4 hallucination rate means you waste too much time on wild goose chases. I briefly thought GPT-4 was useful for finding papers given a description of the results - I “kicked the tires” with some AI research and was very impressed. But when I tried to find papers on animal cognition... This is less a question of which jobs benefit from AI in general and which don't than it is a question of tasks and specific tools. ChatGPT is not a search engine, so if you're looking for existing documents it's a very bad choice. But I've found myself using Perplexity—an LLM-powered search engine—more and more often because it reliably turns up results that Google fails to turn up. I suspect Perplexity is still also the wrong tool for scholarly articles, but that's not a fundamental limitation of the tech, it's just a question of the focus of the tools so far.

deleted 2y ago

[deleted]

dyauspitr 2y ago

What are you talking about? I haven’t had a hallucinated link since 3.5 to paper sources. They must have some sort of post processing in place to remove made up links.

rsynnott 2y ago

> I just don’t understand the childish motivated reasoning behind assuming the skeptics are scared. Maybe if I spent “three sleepless nights” talking to ChatGPT I would be more enlightened. A lot of people are very invested, whether emotionally or financially or both, in this stuff not being a flop. There’s a lot of motivated reasoning going on. It is necessary to believe that the heretics simply haven’t seen the light yet - to question that gets too close to questioning whether there’s any light to see.