3 ms·
I've seen a lot of papers recently tackle the needle-in-a-haystack problem wrt LLMs, and I think this approach (and more generally, any in-context solution) is
by dvt 2y ago
I've seen a lot of papers recently tackle the needle-in-a-haystack problem wrt LLMs, and I think this approach (and more generally, any in-context solution) is a mistake.
Imo the best way to handle this is RAG + multi-shot prompting (+ symbolic mapping to an actual data structure). For example, a pre-processing step where you partition the context by "records," another step where you insert (and potentially split up the records) in a RAG database, and another step where you make fuzzy queries. So, if you ask for record 1234 you get an exact match on that line (or set of lines, or record, or whatever) of the original context. And if you ask for "elephant" but there's no "elephant" in the context, you might get the "hippo" record because of the RAG reranking.
This is a lot of work, and is essentially a data pipeline, but the results are much better-curated than just fine-tuning and hoping that generalized needle-in-a-haystack search will work reliably as part of a language model.
- pilooch 2y agoNo. Because the model should and will replace any piece of code. This is what happened already for other tasks, in computer vision, text (entity recognition, etc...), audio, etc. RAG will go away, decision multimodal models / LLMs will take over. No here yet, but inevitable I believe.
- vatsadev 2y agonot nesc. like I would rather use code to find apriltags than a vit or other model
- snovv_crash 2y agoEnd to end networks can sometimes have higher performance, but the failure mechanisms aren't explainable, and are unintuitive to humans. If you're building something that needs to be easy to work with, and that humans can understand the limitations of, splitting the network up into stages and having human-interpretable intermediate values is a good architecture choice.
- jimmySixDOF 2y agoNo so sure these are even mutually exclusive positions. The BYO data needs will far exceed the longest contexts for a long time to come. LLMs might integrate RAG to the point it is hard to talk about one without the other (DSPy is close) but there will still be some kind of private knowledge base graph feeding incontext learning so improvements in either area is positive.
- achierius 2y agoThis seems like a nonsensical position. Computer vision models have not replaced every piece of code -- there's still harness, formatting, even old-fashioned classical vision processing that goes on both before and after the model runs. It's perfectly reasonable to couch AI models inside other, classical code.
- imtringued 2y agoWe have end to end robot learning models that take nothing but camera input and instructions and directly produce the robot motions. There is literally no code left except for the sensors and actuators. It's all in the model.