3 ms·
Are language models really the best choice for this? Seems to me that the outcome would be near random because they are so poorly suited. Which might manifest
by Havoc 1y ago
Are language models really the best choice for this?
Seems to me that the outcome would be near random because they are so poorly suited. Which might manifest as
> We also found that the models were highly sensitive to seemingly trivial prompt changes
- baq 1y agothey're tools. treat them as tools. since they're so general, you need to explore if and how you can use them in your domain. guessing 'they're poorly suited' is just that, guessing. in particular: > We also found that the models were highly sensitive to seemingly trivial prompt changes this is as much as obvious for anyone who seriously looked at deploying these, that's why there are some very successful startups in the evals space.
- rob_c 1y ago> guessing 'they're poorly suited' is just that, guessing I have a really nice bridge to sell you... This "failure" is just a grab at trying to look "cool" and "innovative" I'd bet. Anyone with a modicum of understanding of the tooling (or hell experience they've been around for a few years now, enough for people to build a feeling for this), knows that this it's not a task for a pre-trained general LLM.
- baq 1y agoI think you have a different idea of what I'm saying than what I'm actually saying.
- infecto 1y agoI agree in sentiment but if you spent any amount of time in finance, even outside of equity markets, you would have a pretty quick mental model that LLMs are a weird fit for this space.
- kqr 1y agoNo, LLMs are not a good choice for this – as the results show! If I had to guess, they're experimenting with LLMs for publicity.
- Libidinalecon 1y agoExactly. This is a performance by a really bad method actor.