3 ms·

It is explainable given that there was a single developer working on TTS. It is hard to compete with big academic teams/industry players this way. I also belie

by nshm 6y ago

It is explainable given that there was a single developer working on TTS. It is hard to compete with big academic teams/industry players this way. I also believe Mozilla team was restricted by a lack of computing resources. They had just a single 8GPU server or so.

posguy 6y ago
Said 8 GPU server was consistently in use for Mozilla DeepSpeech (now renamed Mozilla STT) in training models. Its impressive how far Mozilla got considering how limited their resources were.
qchris 6y ago
This is an area that I find unbelievably frustrating. A lack of computing resources in the current day is kind of insane. You can buy an 8GB GPU for <$1000. Even with the rest of the costs, the cost of hardware like this is a drop in the bucket when your main office is housed in Mountain View! Especially on a project that ends up being public-facing, these are missed opportunities where a little can go a long way.
- panpanna 6y ago
  We need a SETI@home approach to open source AI models. Only then we can break our dependency on Google and Facebook - and Mozilla for that matter.
  - nshm 6y ago
    It is going on, see https://github.com/learning-at-home/hivemind https://github.com/learning-at-home/hivemind and https://github.com/bytedance/fedlearner https://github.com/bytedance/fedlearner
- nmstoker 6y ago
  I take your point but according to the release details on the repo it was not 8Gb on one card but a server with 8 cards, each a Quadro RTX 6000 with 24Gb, and they're around £4k each currently, so the cost of the GPUs alone is £32k https://github.com/mozilla/STT/releases/tag/v0.8.2 https://github.com/mozilla/STT/releases/tag/v0.8.2
  - qchris 6y ago
    Ah, I see-- not an 8GB, 1-GPU server, but an 8-GPU server. That does make a bit of a difference, changing the cost from a new workstation to functionally a piece of capital equipment. Still, I'm not sure that my point about equipment costs falls short--even at (call it) $40K, you're probably talking less than 3 months of the company's all-in cost for the developer themself, amortized over multiple years.

by nshm 6y ago

posguy 6y ago

Said 8 GPU server was consistently in use for Mozilla DeepSpeech (now renamed Mozilla STT) in training models. Its impressive how far Mozilla got considering how limited their resources were.

qchris 6y ago

This is an area that I find unbelievably frustrating. A lack of computing resources in the current day is kind of insane. You can buy an 8GB GPU for <$1000. Even with the rest of the costs, the cost of hardware like this is a drop in the bucket when your main office is housed in Mountain View! Especially on a project that ends up being public-facing, these are missed opportunities where a little can go a long way.

panpanna 6y ago
We need a SETI@home approach to open source AI models. Only then we can break our dependency on Google and Facebook - and Mozilla for that matter.
- nshm 6y ago
  It is going on, see https://github.com/learning-at-home/hivemind https://github.com/learning-at-home/hivemind and https://github.com/bytedance/fedlearner https://github.com/bytedance/fedlearner
nmstoker 6y ago
I take your point but according to the release details on the repo it was not 8Gb on one card but a server with 8 cards, each a Quadro RTX 6000 with 24Gb, and they're around £4k each currently, so the cost of the GPUs alone is £32k https://github.com/mozilla/STT/releases/tag/v0.8.2 https://github.com/mozilla/STT/releases/tag/v0.8.2
- qchris 6y ago
  Ah, I see-- not an 8GB, 1-GPU server, but an 8-GPU server. That does make a bit of a difference, changing the cost from a new workstation to functionally a piece of capital equipment. Still, I'm not sure that my point about equipment costs falls short--even at (call it) $40K, you're probably talking less than 3 months of the company's all-in cost for the developer themself, amortized over multiple years.