4 ms·
So if you were an engineer at OpenAI, how would you propose going about deleting data from a trained LLM that's already in use?
by goatlover 4y ago
So if you were an engineer at OpenAI, how would you propose going about deleting data from a trained LLM that's already in use?
- cuu508 4y agoRemove it from the training set and train again. Don't do it per data erasure request, batch them and run it, say, every 30 days.
- a2128 4y agoLarge language models take months to train on thousands of GPUs (LLaMA trained for around 5 months on 2048 A100-80GB, consuming around 2,638 MWh), it's nowhere near viable to re-train them from scratch to remove some training data
- blibble 4y agothat's fine if their business model isn't compatible with the law then they can wind up the company and return the remaining capital to their investors it's no different to companies complaining that the cost of legal disposal of toxic waste renders their business nonviable
- JohnFen 4y ago> it's nowhere near viable to re-train them from scratch to remove some training data Tough. If their business is only viable by ignoring the law, they shouldn't be in business.