7 ms·

Given the very (very, VERY) obvious concerns associated with malicious deployment of this tech, and the minimal/largely ineffectual countermeasures deployed by

Given the very (very, VERY) obvious concerns associated with malicious deployment of this tech, and the minimal/largely ineffectual countermeasures deployed by the founders, what surprises me the most is that YC gave this startup its stamp of approval. It used to be that they offered at least a basic sanity check to anything they funded. Is this now getting lost as they scale up their funding operations?

mahmoudfelfel 4y ago
What is in this demo is a very rate-limited, early version of our new model. We have many mitigations in place to increase the safety of our main product (Play.ht); I mentioned some of that here https://news.ycombinator.com/item?id=35331310 https://news.ycombinator.com/item?id=35331310
- taytus 4y ago
  Then why release this outdated version?
- neilv 4y ago
  The mitigations don't seem to address targeted attacks that I'd think you can assume will happen. How do you address the civil and criminal liability of that?
  - eganist 4y ago
    Are there even any criminal laws against this? I mean, it strikes me that there should be, but my non lawyerly self has never heard of any.
    - newonhere 4y ago
      Yes, absolutely. Using one‘s likeness without explicit consent first is illegal in (most of?) Europe and is a tricky subject in the US. For example look at Crispin Glover‘s lawsuit against Back to the Future II. https://en.wikipedia.org/wiki/Personality_rights https://en.wikipedia.org/wiki/Personality_rights
      - eganist 4y ago
        To be clear, I'm looking for references to criminal law, not civil or case law. This all seems a combination of the latter, and even here it's not obvious what applies in cases where a third party produces the infringing content.
        newonhere 4y ago
        I am not a lawyer. I had to briefly look up the difference between criminal law and case law to see what you mean. I have no idea if there are any criminal cases in the US about this. In Civil Law countries as opposed to Common Law countries the plaintiff could very well be the state for this type of legislation. For example, look at tech companies being fined for GDPR violations, same basic idea.
- CJefferson 4y ago
  Your mitigations are so bad, you must know that they are almost useless. No website has ever kept up with spam by individually checking every complaint, and you would only stop people after they've already recorded large amounts of output from a stolen voice. On the other hand, as you also know, you could easily put a much stronger safeguard on by making people say a short prepared statement and only clone voices who have said that statement. You do not appear to have done that. Why not, except it would make it harder to voice steal?
anoy8888 4y ago
Moral is the last thing YC cared about . In fact Paul wrote an essay about good founders being the kind of people who tend to break rules. Rules, laws , morality are for other people
- rockemsockem 4y ago
  I feel silly for saying this, but it's very obvious that rule breaking is not the same as acting immorally. Machines should be able to speak in realistic voices. Every cool future I can imagine includes that. Why muzzle it just because people can be scammed with it?? People get scammed by jpgs, random people calling and saying they're from the bank or the IRS. I'm really not interested in limiting what humanity can accomplish to account for folks getting scammed. Many of them will get scammed anyway. What we should do is make it easier for these individuals to learn how to not get scammed or to come up with scam insurance for the ridiculously gullible. I'm just spitballing, but banning cool technology because people can be scammed is overkill.
  - CJefferson 4y ago
    My problem isn't computers speaking in realistic voices. The problem is speaking in your voice, or my voice.
  - killdozer 4y ago
    Just because there's some bad stuff out there doesn't mean that opening the flood gates to higher amounts of more advanced bad stuff is acceptable. It's absolutely reasonable to look at a new technology and try to figure out if it is likely to bring about more harm than good and take action based on that assessment.
- hn_throwaway_99 4y ago
  I think it's impossible to write anything with nuance on the Internet, as it is always taken out of context and used as a gross caricature of the original point, as you have done. I'm quite familiar with pg's essay, and the idea that he's arguing that "Rules, laws , morality are for other people" is laughable. Sure, it's fair to argue against his point, but one of the main thing he highlights in his essay is clearly knowing the difference between rules and laws that are morally important, versus ones that came about through regulatory capture or "tradition". Most societies have at least some tradition of celebrating "good trouble".
kybernetyk 4y ago
I don’t want to topple US democracy. I just want to make walk through videos for my apps without my terrible accent coming through :) So thanks for funding this.
rcme 4y ago
The constant worry about malicious deployment is so tired in my opinion. The technology to clone voices exists. Your trust is audio recording should already be shaken. Trying to hobble this product on the grounds of "it's dangerous" just serves to limit creativity.
- drewwwwww 4y ago
  asking someone to take basic precautions like “have a fire extinguisher” or “hazmat labeling” just gets in the way of innovation!
  - MattRix 4y ago
    well no, their argument is more like: "Everyone should keep advancing the art of cooking, but just make sure they have a fire extinguisher handy" and your response is "Don't use fire to cook because there's a chance it could burn your house down".
    - Apocryphon 4y ago
      Well, jurisdictions are starting to ban gas stoves, so indoor cooking with fire might eventually become a thing of the past.
  - rcme 4y ago
    Generally hazmat labeling is only required when there is a clear and present harm possible. So far, the only argument presented is “theoretically someone could abuse the this” which is very different than “if you’re exposed to this, you will get cancer.” If a theoretical argument that the masses can’t be trusted due to the actions of a few appeals to you, then I imagine you also support banning encryption to prevent terrorism.
- jprete 4y ago
  The technology never should have existed in the first place. Adding to it is still adding additional harm.
- 4y ago
utopcell 4y ago
Can't you take a moment and appreciate the great technical achievement before you ?

mahmoudfelfel 4y ago

What is in this demo is a very rate-limited, early version of our new model. We have many mitigations in place to increase the safety of our main product (Play.ht); I mentioned some of that here https://news.ycombinator.com/item?id=35331310 https://news.ycombinator.com/item?id=35331310

taytus 4y ago
Then why release this outdated version?
neilv 4y ago
The mitigations don't seem to address targeted attacks that I'd think you can assume will happen. How do you address the civil and criminal liability of that?
- eganist 4y ago
  Are there even any criminal laws against this? I mean, it strikes me that there should be, but my non lawyerly self has never heard of any.
  - newonhere 4y ago
    Yes, absolutely. Using one‘s likeness without explicit consent first is illegal in (most of?) Europe and is a tricky subject in the US. For example look at Crispin Glover‘s lawsuit against Back to the Future II. https://en.wikipedia.org/wiki/Personality_rights https://en.wikipedia.org/wiki/Personality_rights
    - eganist 4y ago
      To be clear, I'm looking for references to criminal law, not civil or case law. This all seems a combination of the latter, and even here it's not obvious what applies in cases where a third party produces the infringing content.
      - newonhere 4y ago
        I am not a lawyer. I had to briefly look up the difference between criminal law and case law to see what you mean. I have no idea if there are any criminal cases in the US about this. In Civil Law countries as opposed to Common Law countries the plaintiff could very well be the state for this type of legislation. For example, look at tech companies being fined for GDPR violations, same basic idea.
CJefferson 4y ago
Your mitigations are so bad, you must know that they are almost useless. No website has ever kept up with spam by individually checking every complaint, and you would only stop people after they've already recorded large amounts of output from a stolen voice. On the other hand, as you also know, you could easily put a much stronger safeguard on by making people say a short prepared statement and only clone voices who have said that statement. You do not appear to have done that. Why not, except it would make it harder to voice steal?

anoy8888 4y ago

Moral is the last thing YC cared about . In fact Paul wrote an essay about good founders being the kind of people who tend to break rules. Rules, laws , morality are for other people

rockemsockem 4y ago
I feel silly for saying this, but it's very obvious that rule breaking is not the same as acting immorally. Machines should be able to speak in realistic voices. Every cool future I can imagine includes that. Why muzzle it just because people can be scammed with it?? People get scammed by jpgs, random people calling and saying they're from the bank or the IRS. I'm really not interested in limiting what humanity can accomplish to account for folks getting scammed. Many of them will get scammed anyway. What we should do is make it easier for these individuals to learn how to not get scammed or to come up with scam insurance for the ridiculously gullible. I'm just spitballing, but banning cool technology because people can be scammed is overkill.
- CJefferson 4y ago
  My problem isn't computers speaking in realistic voices. The problem is speaking in your voice, or my voice.
- killdozer 4y ago
  Just because there's some bad stuff out there doesn't mean that opening the flood gates to higher amounts of more advanced bad stuff is acceptable. It's absolutely reasonable to look at a new technology and try to figure out if it is likely to bring about more harm than good and take action based on that assessment.
hn_throwaway_99 4y ago
I think it's impossible to write anything with nuance on the Internet, as it is always taken out of context and used as a gross caricature of the original point, as you have done. I'm quite familiar with pg's essay, and the idea that he's arguing that "Rules, laws , morality are for other people" is laughable. Sure, it's fair to argue against his point, but one of the main thing he highlights in his essay is clearly knowing the difference between rules and laws that are morally important, versus ones that came about through regulatory capture or "tradition". Most societies have at least some tradition of celebrating "good trouble".

kybernetyk 4y ago

I don’t want to topple US democracy. I just want to make walk through videos for my apps without my terrible accent coming through :) So thanks for funding this.

rcme 4y ago

The constant worry about malicious deployment is so tired in my opinion. The technology to clone voices exists. Your trust is audio recording should already be shaken. Trying to hobble this product on the grounds of "it's dangerous" just serves to limit creativity.

drewwwwww 4y ago
asking someone to take basic precautions like “have a fire extinguisher” or “hazmat labeling” just gets in the way of innovation!
- MattRix 4y ago
  well no, their argument is more like: "Everyone should keep advancing the art of cooking, but just make sure they have a fire extinguisher handy" and your response is "Don't use fire to cook because there's a chance it could burn your house down".
  - Apocryphon 4y ago
    Well, jurisdictions are starting to ban gas stoves, so indoor cooking with fire might eventually become a thing of the past.
- rcme 4y ago
  Generally hazmat labeling is only required when there is a clear and present harm possible. So far, the only argument presented is “theoretically someone could abuse the this” which is very different than “if you’re exposed to this, you will get cancer.” If a theoretical argument that the masses can’t be trusted due to the actions of a few appeals to you, then I imagine you also support banning encryption to prevent terrorism.
jprete 4y ago
The technology never should have existed in the first place. Adding to it is still adding additional harm.
4y ago

utopcell 4y ago

Can't you take a moment and appreciate the great technical achievement before you ?