2 ms·
No? This is just giving up and hoping.
by iandanforth 3mo ago
No? This is just giving up and hoping.
- chrisjj 3mo agoNo change from regular chatbot coding, then.
- motoxpro 3mo agoIs there a solution you are using to solve this that is more accurate and cost effective? I'm working through it now so would be curious
- runarberg 3mo agoIs scraping and putting this in a structured format too inaccurate or expensive?
- motoxpro 3mo agoThat's the whole problem. If you have tons (100s of thousands or more) of labels, then you have "structured" data, but how do you correctly classify that scraped item into the correct label? Putting all the labels into the LLM is super expensive per call when you have millions of items to classify. You can't reduce the number of labels becasue they are correctly organizes/structured. This class of problem exists in many different domains.
- runarberg 3mo ago100s of thousand? In that case I would label about a 100 by hand and train a supervised learning model. This problem has also been solved for 3 decades now.
- Terr_ 3mo agoAhhh, but the LLM processing is the kind corporate leadership and investors will actually agree to buy! Orginally I started writing that as sarcasm, and now I'm not quite so sure.