4 ms·
Most part if you actually need stuff done, you can afford 20$ for OpenAIs proven models. These local projects are great because maybe eventually they will have
by m3kw9 4y ago
Most part if you actually need stuff done, you can afford 20$ for OpenAIs proven models.
These local projects are great because maybe eventually they will have a equivalent model that can be run on cheap parts
- Beltiras 4y agoThe knowledge cutoff date is brutal for some applications.
- m3kw9 4y agoI wonder how ingesting more and more data will affect the size of parameters, it’s gonna continually get bigger?
- rolisz 4y agoI don't think that the current models are at "knowledge capacity". So far all evidence points to training on more data on the same size model gives better results.
- cubefox 4y agoBoth increasing the amount of parameters and the amount of training tokens improves results (more precisely: lowers training loss), and costs computing power. For optimally improving loss per training computing power, model size and training tokens should be increased equally. That's the Chinchilla scaling law. (Though low loss is not always the same as good results, the data quality also matters.) Further reading: https://dynomight.net/scaling/ https://dynomight.net/scaling/
- snovv_crash 4y agoAn interesting corollary of this is that if you want to reduce the model size you can compensate by training for longer to achieve the same accuracy. Depending on your training:inference ratio this may be more optimal globally to reduce your total compute costs or even just reduce your frontend latency.
- cubefox 4y agoYeah, though I have not seen a formula which takes the number of expected inference runs into account for calculating the optimal data/parameter balance.
- thelittleone 4y agoKnowledge cut off and sending potentially sensitive IP to a centralised and untrusted third party. This would likely limit the addressable market.
- jjoonathan 4y agoAlso: they are lobotomized. If you want to do security research, Sorry Dave. If you want to nsfw writing, Sorry Dave. If you want to open the pod bay doors, Sorry Dave, I'm afraid I can't do that.
- yieldcrv 4y agoI tried to help a sex worker with ChatGPT and all it did was preach about avoiding sex work, further marginalizing in its virtue signalizing. I dont consider her marginalized, and “help” was just information about her job and techniques and venues. ChatGPT would have consumed this textual information too. But yeah offline fine tuned models wont have this problem. Kind of cool to see how the SWERF representation in tech is going to speedrun SWERF irrelevancy.
- jhartwig 4y agoYou tried to help a sex worker with chatGPT? Care to explain more on this use case lol? Curious minds want to know.
- yieldcrv 4y agoVenues to work, other people’s thoughts working there, management, nuances about local licenses, stuff that anybody with any job would do
- jstarfish 4y agoPeople are already setting up fake personas/OnlyFans accounts using chatbots and SD images. We have a high-value specialist currently chatting up a few of them at work. His wife doesn't know. He doesn't know we know. The photos are fake but he's too horny to notice. Time to dust off the "there are no women on the internet" meme...