4 ms·

> Every audio file generated by Chatterbox includes Resemble AI's Perth (Perceptual Threshold) Watermarker - imperceptible neural watermarks that survive MP3 co

> Every audio file generated by Chatterbox includes Resemble AI's Perth (Perceptual Threshold) Watermarker - imperceptible neural watermarks that survive MP3 compression, audio editing, and common manipulations while maintaining nearly 100% detection accuracy. Am I misunderstanding, or can you trivially disable the watermark by simply commenting out the call to the apply_watermark function in tts.py? https://github.com/resemble-ai/chatterbox/blob/master/src/chatterbox/tts.py https://github.com/resemble-ai/chatterbox/blob/master/src/ch... I thought the point of this sort of watermark was that it was embedded somehow in the model weights, so that it couldn't easily be separated out. If you're going to release an open-source model that adds a watermark as a separate post-processing step, then why bother with the watermark at all?

jchw 1y ago
Possibly a sort of CYA gesture, kinda like how original Stable Diffusion had a content filter IIRC. Could also just be to prevent people from accidentally getting peanut butter in the toothpaste WRT training data, too.
- throw101010 1y ago
  Stable Diffusion or rather Automatic1111 which was initially the UI of choice for SD models had a joke/fake "watermark" setting too which was deliberately doing nothing besides poking fun at people who were thinking that open source projects would really waste time on developing something that could easily be stripped/reverted by the virtue of being open source anyways.
vunderba 1y ago
Yeah, there's even a flag to turn it off in the parser `--no-watermark`. I assumed they added it for downstream users pulling it in as a "feature" for their larger product.
echelon 1y ago
1. Any non-OpenAI, non-Google, non-ElevenLabs player is going to have to aggressively open source or they'll become 100% irrelevant. The TTS market leaders are obvious and deeply entrenched, and Resemble, Play(HT), et al. have to aggressively cater to developers by offering up their weights [1]. 2. This is CYA for that. Without watermarking, there will be cries from the media about abuse (from anti-AI outfits like 404Media [2] especially). [1] This is the right way to do it. Offer source code and weights, offer their own API/fine tuning so developers don't have to deal with the hassle. That's how they win back some market share. [2] https://www.404media.co/wikipedia-pauses-ai-generated-summaries-after-editor-backlash/ https://www.404media.co/wikipedia-pauses-ai-generated-summar...
- echelon 1y ago
  Nevermind, this is just ~3/10 open, or not really open at all [1]: https://github.com/resemble-ai/chatterbox/issues/45#issuecomment-2923044742 https://github.com/resemble-ai/chatterbox/issues/45#issuecom... > For now, that means we’re not releasing the training code, and fine-tuning will be something we support through our paid API (https://app.resemble.ai https://app.resemble.ai). This helps us pay the bills and keep pushing out models that (hopefully) benefit everyone. Big bummer here, Resemble. This is not at all open. For everyone stumbling upon this, there are better "open weights" models than Resemble's Chatterbox TTS: Zeroshot TTS: MaskGCT, MegaTTS3 Zeroshot VC: Seed-VC, MegaTTS3 These are really good robust models that score higher in openness. Unfortunately only Seed-VC is fully open. But all of the above still beat Resemble's Chatterbox in zero shot MOS (we tested a lot), especially the mega-OP Chinese models. (ByteDance slaps with all things AI. Their new secretive video model is better than Veo 3, if you haven't already seen it [2]!) You can totally ignore this model masquerading as "open". Resemble isn't really being generous at all here, and this is some cheap wool over the eyes trickery. They know they retain all of the cards here, and really - if you're just going to use an API, why not just use ElevenLabs? Shame on y'all, Resemble. This isn't "open" AI. The Chinese are going to wipe the floor with TTS. ByteDance released their model in a more open manner than yours, and it sounds way better and generalizes to voices with higher speaker similarity. Playing with open source is a path forward, but it has to be in good faith. Please do better. [1] "10/10" open includes: 1. model code, 2. training code, 3. fine tuning code, 4. inference code, 5. raw training data, 6. processed training data, 7. weights, 8. license to outputs, 9. research paper, 10. patents. For something to be a good model, it should have 7/10 or above. [2] https://artificialanalysis.ai/text-to-video/arena?tab=leaderboard&input=image https://artificialanalysis.ai/text-to-video/arena?tab=leader...
  - tedip 1y ago
    Cant make everyone happy :)
    - echelon 1y ago
      This space is getting pretty crowded. If you're going to drop weights on unsuspecting developers (who might not be familiar with TTS) and make them think that they'll fit their use case, that's a bit of a bait-and-switch. Chatterbox TTS is only available over API for fine tunes. That's an incredibly saturated market, and there are better quality and cheaper models for this. Chatterbox TTS is equivalent to already-released semi-open weights from ByteDance and other labs, and those models already sound and perform better. It'd be truly exciting if Chatterbox fine tunes could be done as open weights, similar to how Flux operates. Black Forest Labs has an entire open weights ecosystem built around them. While they do withhold their pro / highest quality variants, they always release open weights with training code for each commercial release. That's a much better model for courting open source developers. Another company doing "open weights" right is Lightricks with LTX-1. They have a commercial studio, but they release all of their weights and tuning code in the open. I don't see how this is a carrot for open source. It's an ad for the hosted API.
backnotprop 1y ago
[dead]

jchw 1y ago

Possibly a sort of CYA gesture, kinda like how original Stable Diffusion had a content filter IIRC. Could also just be to prevent people from accidentally getting peanut butter in the toothpaste WRT training data, too.

throw101010 1y ago
Stable Diffusion or rather Automatic1111 which was initially the UI of choice for SD models had a joke/fake "watermark" setting too which was deliberately doing nothing besides poking fun at people who were thinking that open source projects would really waste time on developing something that could easily be stripped/reverted by the virtue of being open source anyways.

vunderba 1y ago

Yeah, there's even a flag to turn it off in the parser `--no-watermark`. I assumed they added it for downstream users pulling it in as a "feature" for their larger product.

echelon 1y ago

1. Any non-OpenAI, non-Google, non-ElevenLabs player is going to have to aggressively open source or they'll become 100% irrelevant. The TTS market leaders are obvious and deeply entrenched, and Resemble, Play(HT), et al. have to aggressively cater to developers by offering up their weights [1]. 2. This is CYA for that. Without watermarking, there will be cries from the media about abuse (from anti-AI outfits like 404Media [2] especially). [1] This is the right way to do it. Offer source code and weights, offer their own API/fine tuning so developers don't have to deal with the hassle. That's how they win back some market share. [2] https://www.404media.co/wikipedia-pauses-ai-generated-summaries-after-editor-backlash/ https://www.404media.co/wikipedia-pauses-ai-generated-summar...

echelon 1y ago
Nevermind, this is just ~3/10 open, or not really open at all [1]: https://github.com/resemble-ai/chatterbox/issues/45#issuecomment-2923044742 https://github.com/resemble-ai/chatterbox/issues/45#issuecom... > For now, that means we’re not releasing the training code, and fine-tuning will be something we support through our paid API (https://app.resemble.ai https://app.resemble.ai). This helps us pay the bills and keep pushing out models that (hopefully) benefit everyone. Big bummer here, Resemble. This is not at all open. For everyone stumbling upon this, there are better "open weights" models than Resemble's Chatterbox TTS: Zeroshot TTS: MaskGCT, MegaTTS3 Zeroshot VC: Seed-VC, MegaTTS3 These are really good robust models that score higher in openness. Unfortunately only Seed-VC is fully open. But all of the above still beat Resemble's Chatterbox in zero shot MOS (we tested a lot), especially the mega-OP Chinese models. (ByteDance slaps with all things AI. Their new secretive video model is better than Veo 3, if you haven't already seen it [2]!) You can totally ignore this model masquerading as "open". Resemble isn't really being generous at all here, and this is some cheap wool over the eyes trickery. They know they retain all of the cards here, and really - if you're just going to use an API, why not just use ElevenLabs? Shame on y'all, Resemble. This isn't "open" AI. The Chinese are going to wipe the floor with TTS. ByteDance released their model in a more open manner than yours, and it sounds way better and generalizes to voices with higher speaker similarity. Playing with open source is a path forward, but it has to be in good faith. Please do better. [1] "10/10" open includes: 1. model code, 2. training code, 3. fine tuning code, 4. inference code, 5. raw training data, 6. processed training data, 7. weights, 8. license to outputs, 9. research paper, 10. patents. For something to be a good model, it should have 7/10 or above. [2] https://artificialanalysis.ai/text-to-video/arena?tab=leaderboard&input=image https://artificialanalysis.ai/text-to-video/arena?tab=leader...
- tedip 1y ago
  Cant make everyone happy :)
  - echelon 1y ago
    This space is getting pretty crowded. If you're going to drop weights on unsuspecting developers (who might not be familiar with TTS) and make them think that they'll fit their use case, that's a bit of a bait-and-switch. Chatterbox TTS is only available over API for fine tunes. That's an incredibly saturated market, and there are better quality and cheaper models for this. Chatterbox TTS is equivalent to already-released semi-open weights from ByteDance and other labs, and those models already sound and perform better. It'd be truly exciting if Chatterbox fine tunes could be done as open weights, similar to how Flux operates. Black Forest Labs has an entire open weights ecosystem built around them. While they do withhold their pro / highest quality variants, they always release open weights with training code for each commercial release. That's a much better model for courting open source developers. Another company doing "open weights" right is Lightricks with LTX-1. They have a commercial studio, but they release all of their weights and tuning code in the open. I don't see how this is a carrot for open source. It's an ad for the hosted API.

backnotprop 1y ago

[dead]