3 ms·
As with most software development, modern AI work is all about knowing your tools and when it's appropriate to use them. If you have tabular data, even good LLM
by minimaxir 3y ago
As with most software development, modern AI work is all about knowing your tools and when it's appropriate to use them. If you have tabular data, even good LLMs will have trouble beating a gradient-boost tree algorithm, but if you're working with anything involving text data, you'll save yourself a lot of hassle and likely get better results if you go directly to a pretrained-LLM-generated text embeddings.
- cuuupid 3y agoI've found the best LLMs still fall a bit short when it comes to being truly best in class, SOTA in most useful tasks is still dominated by smaller models with focused approaches and innovative loss mechanisms.
- dartos 3y agoYeah, but you can talk to an LLM. I think the most compelling use for LLMs is “tool use.” Imagine if everyone could write simple automation scripts in literal English. We’re there.
- tensor 3y agoUsing fasttext as a first pass is probably the best idea for production work.
- minimaxir 3y agofasttext/word2vec/training-your-own-word-embeddings-from-scratch requires very optimistic assumptions about your data that rarely hold well in practice, such as not having idiosyncratic syntax from user input. I've learnt this the hard way. Transformers-based LLMs with a BPE tokenizer compensate for this well.
- miven 3y agoWell, LLMs don't consistently beat purpose-trained BERT-type models just yet, for example in [1] the authors show RoBERTa (fine-tuned for each individual task separately of course) going basically toe-to-toe with GPT-4 on quite a few somewhat conventional NLP tasks, while open-source LLaMA 2 models are getting severely outclassed. [1] https://arxiv.org/abs/2308.10092 https://arxiv.org/abs/2308.10092
- minimaxir 3y agoYou'll get far more than 80% of the way with just running a pretrained text embeddings model. There's always room for optimization (e.g. finetuning on your own data) but it's an incredible baseline.
- specproc 3y agoYeah, embedding are where it's at for text. Most problems are really about clustering or classification, when you like, get down to it, man. Semantic similarity is one hell of a drug.
- reexpressionist 3y ago"As with most software development, modern AI work is all about knowing your tools and when it's appropriate to use them." 100% agree. Ditto with just using the easiest to access models as initial proof-of-concept/dev/etc. to get started. (I do agree with the overall sentiment of the TC article, although as noted by others below, there's some mashing of terminology in the article. E.g., I, too, associate GOFAI with symbolic AI and planning.) There's another dimension, too, not mentioned in the article: Even with general purpose LLMs, for production applications, it's still required to have labeled data to produce uncertainty estimates. (There's a sense in which any well-defined and tested production application is a 'single-task' setting, in it its own way.) One of the reasons on-device/edge AI has gotten so interesting, in my opinion, is that we now know how to derive reliable uncertainty estimates with the neural models (more or less independent of scale). As long as prediction uncertainty is sufficiently low, there's no particular reason to go to a larger model. That can lead to non-trivial cost/resource savings, as well as the other benefits of keeping things on-device.
- wiricon 3y agoCan link to any methods for deriving reliable uncertainty estimates? Sounds useful.
- reexpressionist 3y ago[dead]