4 ms·
The weights are indeed open (both accessible and licensing-wise): you don't need to put that in square quotes. Training code is not. You can fine-tune the weigh
by fastball 1y ago
The weights are indeed open (both accessible and licensing-wise): you don't need to put that in square quotes. Training code is not. You can fine-tune the weights yourself with your own training code. Saying that isn't open is like saying ffmpeg isn't open because it doesn't do everything I need it to do and I have to wrap it with own code to achieve my goals.
- echelon 1y agoMachine learning assets are not binary "open" or "closed". There is a continuum of openness. To make a really poor analogy, this repo is like a version of Linux that you can't cross-compile or port. To make another really poor (but fitting) analogy, this is like an "open core" SaaS platform that you know you'll never be able to run the features that matter on your own. This repo scores really low on the "openness" continuum. In this case, you're very limited in what you can do with Chatterbox TTS. You certainly can't improve it or fit it to your data. > You can fine-tune the weights yourself with your own training code. This will never be built by anyone, and they know that. If it could be, they'd provide it themselves. If you're considering Chatterbox TTS, just use MegaTTS3 [1] instead. It's better by all accounts. [1] https://github.com/bytedance/MegaTTS3 https://github.com/bytedance/MegaTTS3
- fastball 1y agoWhy can't you improve it or fit it to your data? This can be cross-compiled/ported in the Linux analogy. The Linux analogy would be more like: a kernel dev wrote code for some part of the Linux kernel using JetBrains' CLion. He used features of CLion that made this process much easer than if he had written the code using `nano`. By your logic, the resulting kernel code is not "open" because the tooling used to create it is not open. This is, of course, nonsense. I agree that the project as a whole is less open than it could be, but the weights are indeed as open as they can be, no scare quotes required.
- echelon 1y agoI really don't think your analogy fits the absurdity of lacking the tooling. It's more like you have to decompile an N64 cartridge ROM and don't have the tools. But I don't want to play that game. I'll up the ante. I'll bet you money that nobody forks this and adds fine tuning for at least a year.
- eginhard 1y agoSomeone already did: https://github.com/stlohrey/chatterbox-finetuning https://github.com/stlohrey/chatterbox-finetuning And someone else fine-tuned it for German: https://huggingface.co/SebastianBodza/Kartoffelbox-v0.1 https://huggingface.co/SebastianBodza/Kartoffelbox-v0.1
- fastball 1y agoYou're supposed to wait to post this until I agree to the bet ;)
- echelon 1y agoI'm totally humbled by this. I haven't seen this level of involvement for a lot of the models I'm using, including several text to speech models. The rapidity of this is also quite shocking. I don't think Resemble anticipated this either, given their wording on the aforementioned ticket. There's probably a lot more work to do to ensure this works, adjusting learning rates, batching, etc., but it's all clearly being put into place and given attention. Even if this model has some finicky fine tuning behaviors, with this kind of willpower it'll be quickly overcome. I suppose I owe you, haha.
- dragonwriter 1y ago> This will never be built by anyone, and they know that. If it could be, they'd provide it themselves. Community fine-tuning code has been developed in the past for open-weights models without public first-party training code.
- dragonwriter 1y agoIt really weird to say ByteDance’s release is “more open” when the WaveVAE encoder isn't released at all, only the decoder, so new voices require submitting your sample to a public GDrive folder and getting extracted latents back through another public GDrive folder.
- pmarreck 1y agoFYI, the term is scare quotes (because they imply suspicion), not square quotes