7 ms·

Great example of the danger of talking about a solution before you’ve truly defined the problem. What problem is Fred trying to solve? The example he gives of

Great example of the danger of talking about a solution before you’ve truly defined the problem. What problem is Fred trying to solve? The example he gives of writing he would sign is a blog post. The blog post is already authenticated because it’s on avc.com. Fred Wilson has the tokens, social authority etc to ensure what shows up there under his name is from him. Technically speaking, it was already signed via TLS when sent to my web browser. It is also unclear how ChatGPT potentially breaks this sort of authentication. Again, what is the threat model, what is the problem being solved? It seems like ChatGPT is much more likely to be used by someone imitating themselves out of laziness (“I don’t feel like writing this blog post today”) than to imitate someone else out of malice and this scheme does nothing to solve it. I suppose someone could make a HN or Reddit account and pose as Fred Wilson and then ChatGPT makes it more credible. But are HN or Reddit going to allow GPG-style signed messages? Seems more likely, if this became an issue, they would move to usernames expressly tied to visible email addresses or domains that could be authenticated by receiving an email, adding a DNS entry, OAuth etc.

infotogivenm 4y ago
Signing everything is not terrible advice. But here the problem Fred is trying to solve is very simple. He is an investor in mirror.xyz and needs to make his money back. This is the only reason someone interested in signing something would also ask you to go through the hassle of using a blockchain.
- infotogivenm 4y ago
  To add to this a bit - I do think an embedded steganographic-type “signature” of chatgpt output could be enormously valuable - without it we can wave goodbye to the efficacy of things like school assignments. Such a signature is very different than a typical cryptographic one we have today however. But I expect instead of embedding such a signature they will just save hashes of generated content and sell a billion dollar “checking” service to schools.
  - dinosaurdynasty 4y ago
    https://scottaaronson.blog/?p=6823 https://scottaaronson.blog/?p=6823 (search for watermark) OpenAI did hire somebody who is investigating watermarking the output of models like GPT-3, so it's definitely possible.
    - infotogivenm 4y ago
      Ah that makes sense “watermark” is the word I was fumbling for here (I kept coming up with “fingerprint”)
  - kwhitefoot 4y ago
    School assignments should be what the student does to prepare for face to face evaluation; that is, they are a means not an end in themselves. We need to stop trying to get education on the cheap and put enough real, motivated, and properly trained humans in place to do it properly.
  - acdha 4y ago
    I agree that it would be useful but also considering it adversarially I don’t think it’s practical because the people we most want to stop will strip it and in most cases they can submit samples until they pass just as spammers have been doing forever. There might be a few cases where e.g. maybe OpenAI would be willing to ping turnitin.com but it seems like that approach wouldn’t work in general and would inevitably result in false-positives with potentially serious consequences. I think we’ll end up seeing two changes: restricting things like school or hiring to favor in person, dynamic interactions - which is better in many ways but costs more and has downsides for people with social anxiety - and more pressure to make the internet less anonymous, especially if that’s linked to physical devices similar to what Cloudflare and Apple have been doing to prevent the bulk attacks (it wouldn’t help much with a cheating student). Depressingly, all of that seems expensive and likely to have substantial collateral damage.
  - nonameiguess 4y ago
    How good is perceptual hashing for text? It seems trivial for a student with any modicum of intellect to delete a sentence and re-word a few others, same as you would do now after bullying a human nerd into writing your essay for you.
- radicalbyte 4y ago
  There is absolutely 0 need to use a blockchain to sign something :facepalm:
  - legutierr 4y ago
    Sure, you don't need a blockchain to sign something, but using a blockchain means that the signature is timestamped in an incontestable way, massively replicated and made globally retrievable. It also helps mitigate the possibility that the content will be deleted or suppressed.
    - infotogivenm 4y ago
      No blockchain in the world could store the hash of every post ever in a “retrievable way”. Even Elon understands this. The valid ideas that apply here are global labeling and account subkey ordering. I have never seen a solution for either that is end-to-end without a client syncing gigs of data, so various intermediaries are always trusted. In practice a federated (“permissioned” I suppose) audit log is better, cheaper, and doesn’t require your users spend tokens if you want to go down this path. Then of course there is the UX question of the validity of using the same keys you use to make irreversible financial transactions to sign your tweets.
      - legutierr 4y ago
        > No blockchain in the world could store the hash of every post ever in a “retrievable way”. Ethereum processes about 1MM transactions per day right now; other blockchains have higher throughput. It currently manages about 100MM accounts. Sure, I suppose that might not be enough to hash and sign all blog posts ever (has anyone ever counted?) but you could timestamp a lot of them. > In practice a federated (“permissioned” I suppose) audit log is better, cheaper, and doesn’t require your users spend tokens if you want to go down this path. Someone has to pay for the servers, etc., right? What's wrong with paying for that using a token? Blockchains may be expensive, but you get orders of magnitude more redundancy, and you don't need to worry about ever losing access. You get what you pay for.
- lancesells 4y ago
  How about all ChatGPT output gets signed instead so humanity doesn't take on the burden? But no, let's use Web3 and add something else to our workload because someone scraped humanity and turned it into a business.
- matthewmcg 4y ago
  Reminds me of the world’s greatest reply to a cease and desist letter[1]: “I feel you should be aware some asshole is signing your name to stupid letters.” [1]: https://news.lettersofnote.com/p/very-truly-yours https://news.lettersofnote.com/p/very-truly-yours
wccrawford 4y ago
I think it's funny that if everyone signed their blog posts, ChatGPT would try to "sign" its blog posts as well, and fail.
- cobbal 4y ago
  Fail if someone checks the signature, but chatGPT would definitely pass the xkcd test: https://m.xkcd.com/1181/ https://m.xkcd.com/1181/
- czbond 4y ago
  Chat would sign it with a different public/key pair - which upon verification would not match authors public key nor hash.
TacticalCoder 4y ago
> What problem is Fred trying to solve? > The blog post is already authenticated because it’s on avc.com. Fred Wilson has the tokens, social authority etc to ensure what shows up there under his name is from him. Haven't you answered your question? Most people don't own a domain, that's the problem signing solves. If I see signed messages from John Doe which look like human generated messages and I start following him and then I start seeing AI-generated spam from John Doe, I can tag the signing key John Doe used as "bot confidently generating non-sense". I cannot do that if John Doe doesn't have its own website.
powera 4y ago
The blockchain proves that the post was made when he says it was. Being on avc.com doesn't prove that; he could back-date posts.
- layer8 4y ago
  You don’t need a blockchain, there are cryptographic timestamp services for that.
- rushikesh98 4y ago
  The author can archive it on the internet archive. That will be proof of timestamp.
evrydayhustling 4y ago
I'm no Web3 proponent, but hard disagree here -- I think Fred is onto something clear. - Current experience is that most folks do not own their own domain or do not have audiences that will consume the content on their own domain, they instead depend on content being copied and distributed on other platforms. - Threat is that modern AI makes it much easier to convincingly impersonate you on other platforms -- including robust profiles and statements that match your style or even tweak real content. - Web3 potentially offers a platform-neutral, globally accessible place to claim ownership of content itself, so that people know if they are viewing a primary source or something edited. There is a TON of productization to be done and a network effect problem to solve (there must be a simple, widely used protocol for verification that can either be used by everyone or built into trusted tools). But the value prop for a public signature ledger is not some fantasy IMO.
esalman 4y ago
I agree with you. The problem here is ascertaining whether Fred actually wrote the blog post himself or used AI to generate the post and then pass it on as his own. He offers no solution to that.
mike_hearn 4y ago
Although not particularly useful for ChatGPT text output, signing with hardware-rooted keys embedded in camera chips would raise the bar for generating deepfakes. Now, does it matter, well, deepfakes were around for a few years now and this type of impersonation attack doesn't seem like a big problem.

infotogivenm 4y ago

Signing everything is not terrible advice. But here the problem Fred is trying to solve is very simple. He is an investor in mirror.xyz and needs to make his money back. This is the only reason someone interested in signing something would also ask you to go through the hassle of using a blockchain.

infotogivenm 4y ago
To add to this a bit - I do think an embedded steganographic-type “signature” of chatgpt output could be enormously valuable - without it we can wave goodbye to the efficacy of things like school assignments. Such a signature is very different than a typical cryptographic one we have today however. But I expect instead of embedding such a signature they will just save hashes of generated content and sell a billion dollar “checking” service to schools.
- dinosaurdynasty 4y ago
  https://scottaaronson.blog/?p=6823 https://scottaaronson.blog/?p=6823 (search for watermark) OpenAI did hire somebody who is investigating watermarking the output of models like GPT-3, so it's definitely possible.
  - infotogivenm 4y ago
    Ah that makes sense “watermark” is the word I was fumbling for here (I kept coming up with “fingerprint”)
- kwhitefoot 4y ago
  School assignments should be what the student does to prepare for face to face evaluation; that is, they are a means not an end in themselves. We need to stop trying to get education on the cheap and put enough real, motivated, and properly trained humans in place to do it properly.
- acdha 4y ago
  I agree that it would be useful but also considering it adversarially I don’t think it’s practical because the people we most want to stop will strip it and in most cases they can submit samples until they pass just as spammers have been doing forever. There might be a few cases where e.g. maybe OpenAI would be willing to ping turnitin.com but it seems like that approach wouldn’t work in general and would inevitably result in false-positives with potentially serious consequences. I think we’ll end up seeing two changes: restricting things like school or hiring to favor in person, dynamic interactions - which is better in many ways but costs more and has downsides for people with social anxiety - and more pressure to make the internet less anonymous, especially if that’s linked to physical devices similar to what Cloudflare and Apple have been doing to prevent the bulk attacks (it wouldn’t help much with a cheating student). Depressingly, all of that seems expensive and likely to have substantial collateral damage.
- nonameiguess 4y ago
  How good is perceptual hashing for text? It seems trivial for a student with any modicum of intellect to delete a sentence and re-word a few others, same as you would do now after bullying a human nerd into writing your essay for you.
radicalbyte 4y ago
There is absolutely 0 need to use a blockchain to sign something :facepalm:
- legutierr 4y ago
  Sure, you don't need a blockchain to sign something, but using a blockchain means that the signature is timestamped in an incontestable way, massively replicated and made globally retrievable. It also helps mitigate the possibility that the content will be deleted or suppressed.
  - infotogivenm 4y ago
    No blockchain in the world could store the hash of every post ever in a “retrievable way”. Even Elon understands this. The valid ideas that apply here are global labeling and account subkey ordering. I have never seen a solution for either that is end-to-end without a client syncing gigs of data, so various intermediaries are always trusted. In practice a federated (“permissioned” I suppose) audit log is better, cheaper, and doesn’t require your users spend tokens if you want to go down this path. Then of course there is the UX question of the validity of using the same keys you use to make irreversible financial transactions to sign your tweets.
    - legutierr 4y ago
      > No blockchain in the world could store the hash of every post ever in a “retrievable way”. Ethereum processes about 1MM transactions per day right now; other blockchains have higher throughput. It currently manages about 100MM accounts. Sure, I suppose that might not be enough to hash and sign all blog posts ever (has anyone ever counted?) but you could timestamp a lot of them. > In practice a federated (“permissioned” I suppose) audit log is better, cheaper, and doesn’t require your users spend tokens if you want to go down this path. Someone has to pay for the servers, etc., right? What's wrong with paying for that using a token? Blockchains may be expensive, but you get orders of magnitude more redundancy, and you don't need to worry about ever losing access. You get what you pay for.
lancesells 4y ago
How about all ChatGPT output gets signed instead so humanity doesn't take on the burden? But no, let's use Web3 and add something else to our workload because someone scraped humanity and turned it into a business.
matthewmcg 4y ago
Reminds me of the world’s greatest reply to a cease and desist letter[1]: “I feel you should be aware some asshole is signing your name to stupid letters.” [1]: https://news.lettersofnote.com/p/very-truly-yours https://news.lettersofnote.com/p/very-truly-yours

wccrawford 4y ago

I think it's funny that if everyone signed their blog posts, ChatGPT would try to "sign" its blog posts as well, and fail.

cobbal 4y ago
Fail if someone checks the signature, but chatGPT would definitely pass the xkcd test: https://m.xkcd.com/1181/ https://m.xkcd.com/1181/
czbond 4y ago
Chat would sign it with a different public/key pair - which upon verification would not match authors public key nor hash.

TacticalCoder 4y ago

> What problem is Fred trying to solve? > The blog post is already authenticated because it’s on avc.com. Fred Wilson has the tokens, social authority etc to ensure what shows up there under his name is from him. Haven't you answered your question? Most people don't own a domain, that's the problem signing solves. If I see signed messages from John Doe which look like human generated messages and I start following him and then I start seeing AI-generated spam from John Doe, I can tag the signing key John Doe used as "bot confidently generating non-sense". I cannot do that if John Doe doesn't have its own website.

powera 4y ago

The blockchain proves that the post was made when he says it was. Being on avc.com doesn't prove that; he could back-date posts.

layer8 4y ago
You don’t need a blockchain, there are cryptographic timestamp services for that.
rushikesh98 4y ago
The author can archive it on the internet archive. That will be proof of timestamp.

evrydayhustling 4y ago

I'm no Web3 proponent, but hard disagree here -- I think Fred is onto something clear. - Current experience is that most folks do not own their own domain or do not have audiences that will consume the content on their own domain, they instead depend on content being copied and distributed on other platforms. - Threat is that modern AI makes it much easier to convincingly impersonate you on other platforms -- including robust profiles and statements that match your style or even tweak real content. - Web3 potentially offers a platform-neutral, globally accessible place to claim ownership of content itself, so that people know if they are viewing a primary source or something edited. There is a TON of productization to be done and a network effect problem to solve (there must be a simple, widely used protocol for verification that can either be used by everyone or built into trusted tools). But the value prop for a public signature ledger is not some fantasy IMO.

esalman 4y ago

I agree with you. The problem here is ascertaining whether Fred actually wrote the blog post himself or used AI to generate the post and then pass it on as his own. He offers no solution to that.

mike_hearn 4y ago

Although not particularly useful for ChatGPT text output, signing with hardware-rooted keys embedded in camera chips would raise the bar for generating deepfakes. Now, does it matter, well, deepfakes were around for a few years now and this type of impersonation attack doesn't seem like a big problem.