4 ms·

Today I asked ChatGPT4 (openai.com Version) how I can create a copy of a typescript object containing only the fields of a specific type. This is not possible,

Today I asked ChatGPT4 (openai.com Version) how I can create a copy of a typescript object containing only the fields of a specific type. This is not possible, as typescript (still, unfortunately) doesn't support runtime code generation for its type information. Nevertheless, ChatGPT tried various approaches to "make it work". It reminded me very much of a brainstorming session with Junior Engineers. I couldn't get it to simply state "Not possible". Even after four iterations. Could share the conversation, but would prefer to only share if someone wants me to. TL;DR: LLMs, even the most sophisticated ones, continue to disappoint and underdeliver¹. Posting this as a former ChatGPT4-Enthusiast. I still think we're using LLMs in the wrong way and do not exploit the full potential yet, and there is space (lots of!) to improve, or alternatively, there are limits to accept. ¹ in this case, it's really easy to pinpoint why the answer is wrong because it requires just a little bit of knowledge of the specific technology. But a so-easy-to-spot error, produced by the most expensive, hardest-to-train LLM model we currently have is a telltale sign that something is off. And will continue to be off, even for ChatGPT5.

nomel 3y ago
> ... we're using LLMs in the wrong way ... What do you see as the "right way"?
- firebaze 3y ago
  Tell me. I don't know. Some extremely easy questions will be answered with confidence, but the answers are wrong, while other, hard questions, will be answered correctly. But to tell the difference what is right or wrong, you need to rely on your own judgment or research. In any case, the conversation I referred to today: https://chat.openai.com/share/2dbaeea8-e1dc-4b72-9ae0-01ea5341b8ec https://chat.openai.com/share/2dbaeea8-e1dc-4b72-9ae0-01ea53... As I said, I knew there was no correct answer to begin with aside from "Not possible".
  - nomel 3y ago
    I think the expanse of few-error uses (which could be considered one aspect of "right way") has increased dramatically, with each GPT version, as this article shows. Your example may be trivial in a few years, or with some hyper specific LLM. So, the "right way", to me, is just "what can this LLM do better than the alternatives". I don't think that means we're doing things "wrong" now, but some fact checking peripherals bolted on could definitely help.
trekkie1024 3y ago
As a TypeScript dev, I’d be curious to see the conversation!

nomel 3y ago

> ... we're using LLMs in the wrong way ... What do you see as the "right way"?

firebaze 3y ago
Tell me. I don't know. Some extremely easy questions will be answered with confidence, but the answers are wrong, while other, hard questions, will be answered correctly. But to tell the difference what is right or wrong, you need to rely on your own judgment or research. In any case, the conversation I referred to today: https://chat.openai.com/share/2dbaeea8-e1dc-4b72-9ae0-01ea5341b8ec https://chat.openai.com/share/2dbaeea8-e1dc-4b72-9ae0-01ea53... As I said, I knew there was no correct answer to begin with aside from "Not possible".
- nomel 3y ago
  I think the expanse of few-error uses (which could be considered one aspect of "right way") has increased dramatically, with each GPT version, as this article shows. Your example may be trivial in a few years, or with some hyper specific LLM. So, the "right way", to me, is just "what can this LLM do better than the alternatives". I don't think that means we're doing things "wrong" now, but some fact checking peripherals bolted on could definitely help.

trekkie1024 3y ago

As a TypeScript dev, I’d be curious to see the conversation!