4 ms·

Anyone proposing that we use ChatGPT to write programs or query databases should read and ponder this and pretty much any other Djikstra papers they come across

Anyone proposing that we use ChatGPT to write programs or query databases should read and ponder this and pretty much any other Djikstra papers they come across.

jiggawatts 3y ago
What is the SQL query for: “What was the name of the guy, you know from that consulting company like Deloitte but the other one, who worked on the design for some authentication system last year with Dick?” I’ve seen LLMs find matches for queries like this, even if “Dick” is a nickname for “Richard”. An awful lot of business questions are in this category and neither relational theory nor full text search is useful to solve such problems. An LLM coupled with a vector database could solve this, and provide entire new categories of products instead of a sub-standard substitute for an existing product.
- photochemsyn 3y ago
  An LLM might also create a meta-analysis questioning the value of the basic assumptions involved in the desirability of creating 'entirely new categories of products' - which, if you look at the history of financial products invented in the United States over the past 40 years, has some validity. When Elon Musk et al. talk about 'slowing down AI' that's the kind of thing that seems to concern them. What if AI could make a convincing argument for unionization at Tesla and the institution of a VW-style system of corporate management? Very frightening, I guess...
  - dragonwriter 3y ago
    > When Elon Musk et al. talk about 'slowing down AI' that's the kind of thing that seems to concern them. No, Musk’s concern is just that his AI venture is starting out behind, so slowing down AI lets him catch up.
    - cubefox 3y ago
      How come you make such a disingenuous statement? He has warned about AI risk for 10 years. https://www.cnet.com/science/elon-musk-we-are-summoning-the-demon-with-artificial-intelligence/ https://www.cnet.com/science/elon-musk-we-are-summoning-the-... https://www.lesswrong.com/posts/kzHJ5BRhgkj9CSQ3N/musk-on-agi-timeframes https://www.lesswrong.com/posts/kzHJ5BRhgkj9CSQ3N/musk-on-ag... https://www.forbes.com/sites/ericmack/2015/01/15/elon-musk-puts-down-10-million-to-fight-skynet/ https://www.forbes.com/sites/ericmack/2015/01/15/elon-musk-p... https://www.youtube.com/watch?v=9jkRcrM6XKA https://www.youtube.com/watch?v=9jkRcrM6XKA https://twitter.com/elonmusk/status/1647601906087329794 https://twitter.com/elonmusk/status/1647601906087329794
      - cyphereal 3y ago
        Sure he warned of dangers, but also invested heavily in AI for autonomous driving by hiring DL researchers ("a million robot taxis by 2020"). He also invested in OpenAI until he had a falling out about who would be in charge of the company. He recently founded X.AI, which is supposedly also building generative AI. Everyone sees danger, but many, including Musk, are forging ahead.
      - kaba0 3y ago
        He is a scam artist with zero knowledge on anything. Just ignore him entirely.
        cubefox 3y ago
        Sure, zero knowledge, and Tesla and SpaceX were just accidents. Got it.
        kaba0 3y ago
        Both were existing companies bought up by him, with the "founder" title included. Also, paying salaries for talent is very different from having said talent oneself. He has no expert knowledge on anything, one might have thought that at least he is good at programming (paypal), but look at twitter, he ain't good even at that.
- moffkalast 3y ago
  > LLM coupled with a vector database Speaking of that, has anyone shown a convincing demo of that working at all yet, or is it just hype? I've seen one short clip of allegedly working in long term memory with it, but the reliability could've been anything from zero to one.
- drpixie 3y ago
  > I’ve seen LLMs find matches for queries like this, even if “Dick” is a nickname for “Richard”. An awful lot of business questions are in this category and neither relational theory nor full text search is useful to solve such problems. Question - have you seen an LLM that can answer queries like this correctly and consistently, for a range of natural inputs? The ones I've seen often produce correct answers, but likely often produce apparently correct but wrong answers, and often produce garbage. For those systems, we've just changed the human's work from "write a query" to "filter the correct answers from the garbage" :(
  - idopmstuff 3y ago
    Yes, provided the answer for the question is in context, not training. If you feed GPT4 a document and ask questions, it will virtually always answer those questions right. If you ask it for info in its training set, it will hallucinate sometimes. With Anthropic's 100k token context window, you can do a whole lot with this.
- codebje 3y ago
  Sure - once the hallucination problem is solved. Otherwise it'll just tell you the guy's name is Harry whether that's supported by the data available or not.
- kaba0 3y ago
  > provide entire new categories of products instead of a sub-standard substitute for an existing product. The question is, do people really want to give up deterministic execution or not. Sure, it might find that “Dick”, but will it reliably find the plain simple “Richard, order by date” query? Or will it sometimes omit/add a few additional records?
- deleted 3y ago
  [deleted]
rqtwteye 3y ago
I had some conversations with ChatGPT about python coding. I am not so sure that an AI that’s specialized on code couldn’t write and modify code with the user having a dialog in regular language. I think that’s where it will be going.
- galaxyLogic 3y ago
  When I write code it's usually not working the way I want it on the first try. There are errors and it also doesn't do exactly what I and users would want it to do. So I iterate. I compile and test and debug and modify. But how could an LLM do that kind of thing if it can not actually run the code and inspect the results, and compare them to some criteria it has in its "head"? Does any LLM do such a thing? Of yes you say surely next year but let's focus on what we know to be possible now.
  - CharlesW 3y ago
    > But how could an LLM do that kind of thing if it can not actually run the code and inspect the results… https://openai.com/blog/chatgpt-plugins#code-interpreter https://openai.com/blog/chatgpt-plugins#code-interpreter
  - williamcotton 3y ago
    I’ve been working on a tool called autorails. It is a GitHub application that reads issues and create PRs. The ReAct tool in LangChain with a custom DockerGitGitHub tool results in this PR: https://github.com/williamcotton/autorails-alpha/pull/38 https://github.com/williamcotton/autorails-alpha/pull/38 The log of the work done by the LLM is in the PR description.
deleted 3y ago
[deleted]
userbinator 3y ago
Or just try it --- and quickly realise that LLMs can tell lies confidently, and mislead you in ways that no human would do (except perhaps the best --- or worst --- of politicians.) To paraphrase a memorable comment I saw here recently: AI combines the speed of machines with the imprecision of humans.

Anyone proposing that we use ChatGPT to write programs or query databases should read and ponder this and pretty much any other Djikstra papers they come across.

jiggawatts 3y ago

What is the SQL query for: “What was the name of the guy, you know from that consulting company like Deloitte but the other one, who worked on the design for some authentication system last year with Dick?” I’ve seen LLMs find matches for queries like this, even if “Dick” is a nickname for “Richard”. An awful lot of business questions are in this category and neither relational theory nor full text search is useful to solve such problems. An LLM coupled with a vector database could solve this, and provide entire new categories of products instead of a sub-standard substitute for an existing product.

photochemsyn 3y ago
An LLM might also create a meta-analysis questioning the value of the basic assumptions involved in the desirability of creating 'entirely new categories of products' - which, if you look at the history of financial products invented in the United States over the past 40 years, has some validity. When Elon Musk et al. talk about 'slowing down AI' that's the kind of thing that seems to concern them. What if AI could make a convincing argument for unionization at Tesla and the institution of a VW-style system of corporate management? Very frightening, I guess...
- dragonwriter 3y ago
  > When Elon Musk et al. talk about 'slowing down AI' that's the kind of thing that seems to concern them. No, Musk’s concern is just that his AI venture is starting out behind, so slowing down AI lets him catch up.
  - cubefox 3y ago
    How come you make such a disingenuous statement? He has warned about AI risk for 10 years. https://www.cnet.com/science/elon-musk-we-are-summoning-the-demon-with-artificial-intelligence/ https://www.cnet.com/science/elon-musk-we-are-summoning-the-... https://www.lesswrong.com/posts/kzHJ5BRhgkj9CSQ3N/musk-on-agi-timeframes https://www.lesswrong.com/posts/kzHJ5BRhgkj9CSQ3N/musk-on-ag... https://www.forbes.com/sites/ericmack/2015/01/15/elon-musk-puts-down-10-million-to-fight-skynet/ https://www.forbes.com/sites/ericmack/2015/01/15/elon-musk-p... https://www.youtube.com/watch?v=9jkRcrM6XKA https://www.youtube.com/watch?v=9jkRcrM6XKA https://twitter.com/elonmusk/status/1647601906087329794 https://twitter.com/elonmusk/status/1647601906087329794
    - cyphereal 3y ago
      Sure he warned of dangers, but also invested heavily in AI for autonomous driving by hiring DL researchers ("a million robot taxis by 2020"). He also invested in OpenAI until he had a falling out about who would be in charge of the company. He recently founded X.AI, which is supposedly also building generative AI. Everyone sees danger, but many, including Musk, are forging ahead.
    - kaba0 3y ago
      He is a scam artist with zero knowledge on anything. Just ignore him entirely.
      - cubefox 3y ago
        Sure, zero knowledge, and Tesla and SpaceX were just accidents. Got it.
        kaba0 3y ago
        Both were existing companies bought up by him, with the "founder" title included. Also, paying salaries for talent is very different from having said talent oneself. He has no expert knowledge on anything, one might have thought that at least he is good at programming (paypal), but look at twitter, he ain't good even at that.
moffkalast 3y ago
> LLM coupled with a vector database Speaking of that, has anyone shown a convincing demo of that working at all yet, or is it just hype? I've seen one short clip of allegedly working in long term memory with it, but the reliability could've been anything from zero to one.
drpixie 3y ago
> I’ve seen LLMs find matches for queries like this, even if “Dick” is a nickname for “Richard”. An awful lot of business questions are in this category and neither relational theory nor full text search is useful to solve such problems. Question - have you seen an LLM that can answer queries like this correctly and consistently, for a range of natural inputs? The ones I've seen often produce correct answers, but likely often produce apparently correct but wrong answers, and often produce garbage. For those systems, we've just changed the human's work from "write a query" to "filter the correct answers from the garbage" :(
- idopmstuff 3y ago
  Yes, provided the answer for the question is in context, not training. If you feed GPT4 a document and ask questions, it will virtually always answer those questions right. If you ask it for info in its training set, it will hallucinate sometimes. With Anthropic's 100k token context window, you can do a whole lot with this.
codebje 3y ago
Sure - once the hallucination problem is solved. Otherwise it'll just tell you the guy's name is Harry whether that's supported by the data available or not.
kaba0 3y ago
> provide entire new categories of products instead of a sub-standard substitute for an existing product. The question is, do people really want to give up deterministic execution or not. Sure, it might find that “Dick”, but will it reliably find the plain simple “Richard, order by date” query? Or will it sometimes omit/add a few additional records?
deleted 3y ago
[deleted]

rqtwteye 3y ago

I had some conversations with ChatGPT about python coding. I am not so sure that an AI that’s specialized on code couldn’t write and modify code with the user having a dialog in regular language. I think that’s where it will be going.

galaxyLogic 3y ago
When I write code it's usually not working the way I want it on the first try. There are errors and it also doesn't do exactly what I and users would want it to do. So I iterate. I compile and test and debug and modify. But how could an LLM do that kind of thing if it can not actually run the code and inspect the results, and compare them to some criteria it has in its "head"? Does any LLM do such a thing? Of yes you say surely next year but let's focus on what we know to be possible now.
- CharlesW 3y ago
  > But how could an LLM do that kind of thing if it can not actually run the code and inspect the results… https://openai.com/blog/chatgpt-plugins#code-interpreter https://openai.com/blog/chatgpt-plugins#code-interpreter
- williamcotton 3y ago
  I’ve been working on a tool called autorails. It is a GitHub application that reads issues and create PRs. The ReAct tool in LangChain with a custom DockerGitGitHub tool results in this PR: https://github.com/williamcotton/autorails-alpha/pull/38 https://github.com/williamcotton/autorails-alpha/pull/38 The log of the work done by the LLM is in the PR description.

deleted 3y ago

[deleted]

userbinator 3y ago

Or just try it --- and quickly realise that LLMs can tell lies confidently, and mislead you in ways that no human would do (except perhaps the best --- or worst --- of politicians.) To paraphrase a memorable comment I saw here recently: AI combines the speed of machines with the imprecision of humans.