3 ms·
There's still lot's of low hanging fruit in data preparation for these models it seems. I wonder if we will reach insane quality when we will be able to allow
by machiaweliczny 3y ago
There's still lot's of low hanging fruit in data preparation for these models it seems.
I wonder if we will reach insane quality when we will be able to allow models to generate not exact same text but something similar (thus allowing for more compression). I think math leans a lot to this type of training as one can super easily verify if generated result is fine or not.
We probably should do similar thing for language when other model judges if generation is ok or not (but doesn't require exact generation). This probably can be done as fine tuning? Anyone know if this has been tested already?
- threatripper 3y agoCould this be implemented by making the loss zero if the output for the expected token is above some threshold? E.g. if the threshold is 0.1 it can output 10 likely tokens with 0.1 probability each and if any of them is correct it will not modify the weights.
- wongarsu 3y agoOne issue is that this doesn't punish the model of the top prediction produces complete nonsense. I can imagine getting good results when fine tuning an existing model though
- wongarsu 3y agoSo something like having a smaller model generate embeddings, and use the embedding distance between the predicted text and the expected text for the loss computation? That sounds like it would be incredibly expensive. But maybe it allows you to fine tune on less data, offsetting the cost a bit (and allowing you to fine tune on just your very best data)
- zitterbewegung 3y agoYou could derive similar data by looking at the embedding and then synthesize similar data and then add it to the training dataset. Right now fine tuning a model before synthesizing the similar data could be done.
- llm_trw 3y ago> There's still lot's of low hanging fruit in data preparation for these models it seems. Anecdotally I've been using gpt4 and claude3 to generate and rank question answer pairs for bert model fine tuning. I'm getting performance better than anything I've seen in the papers about them and I can train them on a desktop.
- jerojero 3y agoNot gonna tell on you but I thought you couldn't use open ai's models to train other LLMs?
- rspoerri 3y agoI still love the idea to you steal tools to create something and prohibit others from creating more tools from those products. This idea should be patentable. /s
- woleium 3y agoKinda like when William Fox stole the designs for a camera, moved to holywood and started fox studios, only to chase down unauthorized copies of his work later
- llm_trw 3y agoI'm not training an LLM I'm just training a LM.