4 ms·
It's only half of the solution though. If the models are trained in a closed way, they can prioritize values encoded during training even if that's not what you
by cousin_it 6mo ago
It's only half of the solution though. If the models are trained in a closed way, they can prioritize values encoded during training even if that's not what you want (example: ask the open Chinese models about Tiananmen). It's not beyond imagining that these models would e.g. try to send your data to authorities or advertisers when their training says so, even if you run them locally.
So the full solution would be models trained in an open verifiable way and running locally.
- wrxd 6mo agoThe model is only generating tokens without touching the network at all, right? How would it send data away?
- procaryote 6mo agoTheoretically, by taking the opportunity to inject an exfiltration mechanism if you ask it to write code for you
- kg 6mo agoLots of people I know run models in "yolo" mode or the equivalent as well, which means it could just invoke curl or telnet to exfiltrate data.
- KetoManx64 6mo agoAll it would take is for one person to catch the model doing this and the reputation of the model and the company would be destroyed irrevocably.
- wolvoleo 6mo agoMany Chinese models are being caught doing this (it's also required by law in China) but there was not much hassle. Having said that Id easily trade some censorship about Chinese affairs I don't care about for the prudishness of American models. Though I generally get the abliterated versions of both.
- theshrike79 6mo agoThe Tiananmen test only hits the model's internal knowledge. What I'm more interested in is that if you give it a tool to access Wikipedia, will it censor its answer even then?