3 ms·

On a personal level, I'm happy about this. Having all personal info trackable is a tiring aspect of modern life? Remember when social media didn't zero out the

by j-bos 2y ago

On a personal level, I'm happy about this. Having all personal info trackable is a tiring aspect of modern life? Remember when social media didn't zero out the exif on photos and anyone could easily grab a map to your family's home? On a society level? I'm stil ok with this. Watermarks are trivially removed by the motivated and unpleasant.

romanovcode 2y ago
I'm not sure what you are saying here. Do you really believe that internally OpenAI does not track and save important info based on your inputs creating a "persona"? This is any intelligence agencies wet dream. The functionality that was introduced that would save info about you as a person to "improve" responses cross-chats basically made a whole profile out of you as a person. Oh, and what a surprise - OpenAI directors since this year are ex-CIA or have very close connections to the agency.
- j-bos 2y ago
  No I believe and understand that. Doesn't mean I want more.
tgv 2y ago
It doesn't seem trackable. And it may not be perfect, but that's no reason to discard it. At the moment, it's being used at a large scale to avoid homework by lazy students, which is nearly everyone. How much further do you want tech to erode education?
- falcor84 2y ago
  Not the parent, but I personally would want tech to erode education exactly to the level where students aren't asked any more to spend time on things that machines can perform trivially for us. Education should prepare us for the real world of today, rather than some make-belief role play version of a bureaucratic office from the late 19th century, where we don't have computers and the only way you have of affecting the world is by writing memos with a pencil.
  - tgv 2y ago
    Education should train your brain, not "prepare for the real world," if only because you can't define how to do that, nor what real world needs are. Is math a real world need? Grammar? Geography? Sports? Biology? With the silly reduction to "things that machines can perform trivially for us" even reading and writing won't be needed. And like that, there's suddenly a great surplus of farm hands, miners, and opioid users.
    - falcor84 2y ago
      [dead]
  - xigoi 2y ago
    > where students aren't asked any more to spend time on things that machines can perform trivially for us. Why do people lift weights at the gym if machines can trivially do that?
    - falcor84 2y ago
      I'm pretty sure that if those electric muscle stimulation devices actually worked well enough to allow people to build muscles without needing to go to the gym, they'd do that, at least I would.
      - xigoi 2y ago
        Does ChatGPT allow you to learn without studying?
        falcor84 2y ago
        Yes, in a way. When I want to do something that I've never done before (typically as part of a larger project I'm working on), I often ask ChatGPT/Claude for advice and get really useful just-in-time explanations, as part of a conversation that I often continue throughout my work on that task, getting additional support and guidance. I usually learn a lot from these conversations, such that the next time I approach a similar situation, I can generally do so on my own. I'm not necessarily arguing that there's no more need for any ahead of time "study", but I think that with the equivalent of a personal tutor for everyone, we can achieve a lot more by learning while doing, instead of study being a dedicated activity. I might be a bit sentimental here, but some of these interactions with an LLM bring back memories of me being a kid working on some small project at home on a Saturday, and being able to come to my dad for advice and even hands-on assistance. And on a less sentimental note, I believe that effective use of an LLM while working on a project fits really well with Papert's Constructionism approach to learning.
- j-bos 2y ago
  My concern isn't the details of tracking, only the fact of it. As for eroding education, I believe 98% of homework is useless for learning. A kid that cheats with gpt is a kid that would cheat off of friends. The tech, in it's cheating and anti cheating, does not degrade or improve the antisocial effects on education.
  - tgv 2y ago
    Those two statements don't bear out. Homework reinforces skills, and practically all kids cheat when the bar is low enough, but that bar doesn't affect their behavior in direct social relations so easily (as you readily admit), and most are really quite honest. And it's not about tech's anti-social influence in this case, but on the intellectual state of humanity as a whole. We've made a whole generation addicted to their phone, and now you want to remove any bit of knowledge?
voidUpdate 2y ago
Its hard to remove this watermark, because the "watermark" is adjusting the probabilities of what tokens are generated, rather than just slapping "generated by chatgpt" over the top. You'd have to actually rewrite the text to remove it
- j-bos 2y ago
  It was once hard to remove watermarks from photos.
- vrighter 2y ago
  so the fact that it's watermarked literally means it was tampered with to produce some form of predetermined output. No other way to do it. And because they are entirely uncontrollable they have to add a lot of checks in the prompts. but we all know that hasn't worked. A series of manually built if statements is akin to an expert system. The first thing I learned about those decades ago was that they were mostly a failed experiment
sva_ 2y ago
> Watermarks are trivially removed by the motivated and unpleasant I heard some chatter from OpenAI folks some 1-2 years ago that the way they'd watermark text is that rather than just using random numbers in the top-k/p sampling process, you have a pseudorandom sequence of numbers that either follows a pattern or you save them directly. This way you could fairly trivially build a tool that determines with very high accuracy whether or not a sequence of words has been generated by your model. I think such a watermark can't be trivially removed unless you rewrite the text, or at least large portions of it.

by j-bos 2y ago

romanovcode 2y ago

I'm not sure what you are saying here. Do you really believe that internally OpenAI does not track and save important info based on your inputs creating a "persona"? This is any intelligence agencies wet dream. The functionality that was introduced that would save info about you as a person to "improve" responses cross-chats basically made a whole profile out of you as a person. Oh, and what a surprise - OpenAI directors since this year are ex-CIA or have very close connections to the agency.

j-bos 2y ago
No I believe and understand that. Doesn't mean I want more.

tgv 2y ago

It doesn't seem trackable. And it may not be perfect, but that's no reason to discard it. At the moment, it's being used at a large scale to avoid homework by lazy students, which is nearly everyone. How much further do you want tech to erode education?

falcor84 2y ago
Not the parent, but I personally would want tech to erode education exactly to the level where students aren't asked any more to spend time on things that machines can perform trivially for us. Education should prepare us for the real world of today, rather than some make-belief role play version of a bureaucratic office from the late 19th century, where we don't have computers and the only way you have of affecting the world is by writing memos with a pencil.
- tgv 2y ago
  Education should train your brain, not "prepare for the real world," if only because you can't define how to do that, nor what real world needs are. Is math a real world need? Grammar? Geography? Sports? Biology? With the silly reduction to "things that machines can perform trivially for us" even reading and writing won't be needed. And like that, there's suddenly a great surplus of farm hands, miners, and opioid users.
  - falcor84 2y ago
    [dead]
- xigoi 2y ago
  > where students aren't asked any more to spend time on things that machines can perform trivially for us. Why do people lift weights at the gym if machines can trivially do that?
  - falcor84 2y ago
    I'm pretty sure that if those electric muscle stimulation devices actually worked well enough to allow people to build muscles without needing to go to the gym, they'd do that, at least I would.
    - xigoi 2y ago
      Does ChatGPT allow you to learn without studying?
      - falcor84 2y ago
        Yes, in a way. When I want to do something that I've never done before (typically as part of a larger project I'm working on), I often ask ChatGPT/Claude for advice and get really useful just-in-time explanations, as part of a conversation that I often continue throughout my work on that task, getting additional support and guidance. I usually learn a lot from these conversations, such that the next time I approach a similar situation, I can generally do so on my own. I'm not necessarily arguing that there's no more need for any ahead of time "study", but I think that with the equivalent of a personal tutor for everyone, we can achieve a lot more by learning while doing, instead of study being a dedicated activity. I might be a bit sentimental here, but some of these interactions with an LLM bring back memories of me being a kid working on some small project at home on a Saturday, and being able to come to my dad for advice and even hands-on assistance. And on a less sentimental note, I believe that effective use of an LLM while working on a project fits really well with Papert's Constructionism approach to learning.
j-bos 2y ago
My concern isn't the details of tracking, only the fact of it. As for eroding education, I believe 98% of homework is useless for learning. A kid that cheats with gpt is a kid that would cheat off of friends. The tech, in it's cheating and anti cheating, does not degrade or improve the antisocial effects on education.
- tgv 2y ago
  Those two statements don't bear out. Homework reinforces skills, and practically all kids cheat when the bar is low enough, but that bar doesn't affect their behavior in direct social relations so easily (as you readily admit), and most are really quite honest. And it's not about tech's anti-social influence in this case, but on the intellectual state of humanity as a whole. We've made a whole generation addicted to their phone, and now you want to remove any bit of knowledge?

voidUpdate 2y ago

Its hard to remove this watermark, because the "watermark" is adjusting the probabilities of what tokens are generated, rather than just slapping "generated by chatgpt" over the top. You'd have to actually rewrite the text to remove it

j-bos 2y ago
It was once hard to remove watermarks from photos.
vrighter 2y ago
so the fact that it's watermarked literally means it was tampered with to produce some form of predetermined output. No other way to do it. And because they are entirely uncontrollable they have to add a lot of checks in the prompts. but we all know that hasn't worked. A series of manually built if statements is akin to an expert system. The first thing I learned about those decades ago was that they were mostly a failed experiment

sva_ 2y ago

> Watermarks are trivially removed by the motivated and unpleasant I heard some chatter from OpenAI folks some 1-2 years ago that the way they'd watermark text is that rather than just using random numbers in the top-k/p sampling process, you have a pseudorandom sequence of numbers that either follows a pattern or you save them directly. This way you could fairly trivially build a tool that determines with very high accuracy whether or not a sequence of words has been generated by your model. I think such a watermark can't be trivially removed unless you rewrite the text, or at least large portions of it.