3 ms·

Yeah they will work, but they will also be unnecessary. You will also bake all sorts of logic into your application that will be solved natively on the stronger

Yeah they will work, but they will also be unnecessary. You will also bake all sorts of logic into your application that will be solved natively on the stronger model. If you believe in the scaling theory, then writing LLM applications is non sensical.

deleted 3y ago
[deleted]
mlyle 3y ago
> Yeah they will work, but they will also be unnecessary. You will also bake all sorts of logic into your application that will be solved natively on the stronger model. I think that a whole lot of what I do in prompt engineering is what's necessary to fully specify the output that I want. A newer model may be less finicky, so I have a higher chance of getting it to work on the first try (and it's more reliable afterwards), but it's hard for me to imagine it needing a whole lot less prompt.
Xenoamorphous 3y ago
How will a more powerful model be a substitute for RAG, which is usually used with private data that won’t be present in any training dataset?
- ldjkfkdsjnv 3y ago
  I just think that the capability of the model could radically change, such that however you structured your RAG pipeline, might need to be rewritten. More general problems could be solved by the model, that you were solving with some complicated contraption of prompts.
- kergonath 3y ago
  One of the idea is to just stuff all the documents in the prompt, which still keeps them private but avoids having to faff around with chunking, embedding, and vector stores. That’s not really the end of RAG as a concept, but it would change all the current tooling and infrastructure we built for it. I don’t think RAG is going away, at least not because of this. But I expect new techniques to become available fairly regularly.
kergonath 3y ago
> If you believe in the scaling theory, then writing LLM applications is non sensical. But not doing it is an opportunity cost. You don’t built skills, tooling and experience, and you don’t get feedback on what works and where you should go. It’s like computers in the 1990s: there’s always a better one 6 months away, so if you wait for it to stabilise, then you don’t do anything for a decade. Just enjoy the ride, bearing in mind that things change very fast and some things will be obsolete next year.

deleted 3y ago

[deleted]

mlyle 3y ago

> Yeah they will work, but they will also be unnecessary. You will also bake all sorts of logic into your application that will be solved natively on the stronger model. I think that a whole lot of what I do in prompt engineering is what's necessary to fully specify the output that I want. A newer model may be less finicky, so I have a higher chance of getting it to work on the first try (and it's more reliable afterwards), but it's hard for me to imagine it needing a whole lot less prompt.

Xenoamorphous 3y ago

How will a more powerful model be a substitute for RAG, which is usually used with private data that won’t be present in any training dataset?

ldjkfkdsjnv 3y ago
I just think that the capability of the model could radically change, such that however you structured your RAG pipeline, might need to be rewritten. More general problems could be solved by the model, that you were solving with some complicated contraption of prompts.
kergonath 3y ago
One of the idea is to just stuff all the documents in the prompt, which still keeps them private but avoids having to faff around with chunking, embedding, and vector stores. That’s not really the end of RAG as a concept, but it would change all the current tooling and infrastructure we built for it. I don’t think RAG is going away, at least not because of this. But I expect new techniques to become available fairly regularly.

kergonath 3y ago

> If you believe in the scaling theory, then writing LLM applications is non sensical. But not doing it is an opportunity cost. You don’t built skills, tooling and experience, and you don’t get feedback on what works and where you should go. It’s like computers in the 1990s: there’s always a better one 6 months away, so if you wait for it to stabilise, then you don’t do anything for a decade. Just enjoy the ride, bearing in mind that things change very fast and some things will be obsolete next year.