3 ms·
Publishing a zero knowledge proof rather than the solution is pretty clever.
by Genbox 5mo ago
Publishing a zero knowledge proof rather than the solution is pretty clever.
- coherentpony 5mo agoIs it? Nobody else can really build on their work.
- riffraff 5mo agoAIU the intent of this publication is not to further research but to make it clear to anyone that we need to move to post quantum cryptography ASAP.
- QuaternionsBhop 5mo agoIf only AI safety research had a mechanism this clear. "We have proof that building the machine will kill everybody, so get to work making a provably safe version."
- AnthonyMouse 5mo ago"AI safety" is essentially incoherent. It's like trying to build an all-purpose chemistry lab that can't produce explosives.
- cubefox 5mo agoNeat, an ontological argument against AI safety. Similar argument: "God doesn't exist" is essentially incoherent. God is the perfect being, and if he didn't exist, he wouldn't be perfect. I think the logical mistake is obvious.
- AnthonyMouse 5mo agoExcept that you have the logic backwards. It's an argument that something ("safe" general purpose AI) can't exist rather than that it has to. People want AI to be able to do every good thing but no bad thing, which is impossible twice. First because false positives and false negatives trade against each other, so a general purpose AI which can do anything approximating all the good things is going to have the bias leaning heavily towards being able to do things in general and therefore being able to do many things that are bad. And second because "good" and "bad" aren't things that anybody can agree on and then some people will demand that it must do X while others demand that it not do X (e.g. "help the rebels win the war"), which means someone is inherently going to be unsatisfied and it's not a thing that can be sensibly regarded as everyone working towards a common goal.
- cwillu 5mo agoYou've made a great argument for calling a general halt to AI development, but I'm not sure that was your intent.
- AnthonyMouse 5mo agoOnly that doesn't work either, because what people want is for themselves to have it but not their opponents, and you not building it while your opponents do is the opposite of that. It's like calling for a general halt to the production of military equipment. How do you expect that to actually happen?
- cubefox 5mo agoThe first one is a difficult balance but not really impossible. The second is basically utilitarianism: Of course you can't maximize all wishes because they often contradict each other, but there can be a reasonable trade-off. Some tradeoffs are clearly better than others.
- AnthonyMouse 5mo ago> The first one is a difficult balance but not really impossible. It's a direct trade off. If you want it to do more "good" things you make it able to do more "bad" things. > Of course you can't maximize all wishes because they often contradict each other, but there can be a reasonable trade-off. Some tradeoffs are clearly better than others. The easy tradeoffs are the ones nobody disputes and everybody is already trying to do. There is no lobby for having it hallucinate more or give you ingredients that will combine to make poison when you ask for a tasty recipe.
- adastra22 5mo agoThat may be the intent, but it is very anti-science.
- Nevermark 5mo agoPartial disclosures are actively pro-science. Evidence that a hard problem is solvable, and information on solution characteristics, are a big help to others. Even non-disclosure is just science-neutral, not anti-science. Partial disclosures are common where disclosures involve risky things, or where a problem was solved as part of an economic concern. But there are non-conflicting opportunities to partially inform others.
- rwmj 5mo agoBut the algorithm still isn't practical on existing quantum computers, or ones that are going to be around any time soon, so there's no reason not to publish in full.
- cubefox 5mo ago> See, some of the most reputable people in quantum hardware and quantum error-correction—people whose judgment I trust more than my own on those topics—are now telling me that a fault-tolerant quantum computer able to break deployed cryptosystems ought to be possible by around 2029. https://scottaaronson.blog/?p=9718 https://scottaaronson.blog/?p=9718
- mikelitoris 5mo agoCould be one of the intents, but the main intent is reputation building.
- LtWorf 5mo agoWake me up when there's an actual working machine.
- pseudohadamard 5mo agoThat's the whole point. And it's not "build on their work", it's "question their work", because so far every time someone's announced some magic quantum thing it's been followed up shortly afterwards by people poking holes on it, a famous recent example being the "quantum computer" that was replaced by /dev/random and it produced the same results. So the magic here isn't the quantum, it's coming up with a way to publish a claim in a way that it can't be refuted.
- adrian_b 5mo agoThis has been used for centuries. It is not a new invention. Hundreds of years ago, it was not unusual to publish an encrypted solution of some mathematical problem, in order to establish priority without disclosing the algorithm that was used. Of course, at that time very simple encryption methods were used, for instance an anagram of the solution was published (i.e. encryption by letter transposition).
- blamestross 5mo agoSeems like a wonderful use case for a one time pad! Every time you offer up the encryption key, it shows you were right!
- aleph_minus_one 5mo ago> Every time you offer up the encryption key, it shows you were right! Only if you have a pre-commitment.