6 ms·

Together’s $20M seed funding to build open-source AI and cloud platform

73 points by tim_sw 3y ago

Anil1331 3y ago
How is it different from Stability AI and other platforms ?
- orra 3y ago
  > Together’s $20M seed funding to build open-source This is a huge contrast. Stability AI’s bread and butter is proprietary, not open source. Together seem to have a track record of open sourcing both code and models. To be fair, Stability AI have open sourced StableLM, which is welcome. But overall Stability exaggerate and lie about their openness.
- mtillman 3y ago
  Different investors.
- etaioinshrdlu 3y ago
  Stability AI is also no longer open. They seem to be commercializing SDXL and it's not open - "yet".
- dmix 3y ago
  It doesn't sound like it's trying to build a single model, it wants to build a network/community of them. One of which is their own. Probably sharing/coordinating training resources and other support systems? The things that are usually out of financial/technical scope for most smaller OSS AI projects. I could see smaller companies with open models spending to be a part of that or other financially backed OSS projects. Maybe it's more hands-on than just a hosted service like huggingface? I don't really know the market, so just speculating.
deleted 3y ago
[deleted]
tikkun 3y ago
The homepage reads like PR speak to me. Does someone here know concretely what the product will likely be? Is it a replicate/modal/banana competitor, an OpenAI/Anthropic competitor, a Stability AI competitor, or something else
starlust2 3y ago
I've noticed a big trend by commercial entities claiming to be "Open Source" but actually charging for licenses. Just because your code is in a public repo on github doesn't mean it's open source. If it's not an open source license it's not open. This is especially true of AI projects.
- timtom39 3y ago
  This is the org behind RedPajama. So, to date, they have been open.
- slashdev 3y ago
  Open source is overrated, in my opinion. I’d rather a healthy commercial entity driving development and supporting it, with visible source that I can modify as a big plus. The whole OSI open source thing is nice, but it’s not going to matter to me much either way. I’m more worried that the open source project is no longer maintained or unsupported or development is slow and it gets obsolete. Your values, and thus your opinion, may be different.
  - pydry 3y ago
    Big tech seem to really dislike this model. E.g. the whole tantrum Amazon had over the Elastic Search relicensing or Google briefly banning SSPL from their code hosting platform.
- capableweb 3y ago
  Since you mention it in connection to Together/RedPajama, are you saying that what they are publishing isn't actually open source? Most of the stuff I see them publish is licensed under Apache-2.0 which is a FOSS license, so I find it weird to mention it here.
- latchkey 3y ago
  Vipul's razor... https://razor.sourceforge.net/ https://razor.sourceforge.net/ He's well known for producing open source.
- josephd79 3y ago
simonw 3y ago
This is the company behind RedPajama, which is a VERY big deal. It's the most realistic open source attempt at recreating the Facebook LLaMA LLM, from scratch, in a way that supports commercial usage. They released their full 2.6TB training dataset last month, and it's significant: https://simonwillison.net/2023/Apr/17/redpajama-data/ https://simonwillison.net/2023/Apr/17/redpajama-data/ They've also started releasing new, commercially-usable openly licensed LLM models trained on that data. You can try one of those out here: https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1?text=Datasette+is https://huggingface.co/togethercomputer/RedPajama-INCITE-Ins...
- ericflo 3y ago
  I would also check out their 3B model. I tested it on launch with LoRA fine-tuning and found it to be surprisingly capable despite its size. I think a lot of people are skipping past testing it because it only has 3B params. Edit: https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1 https://huggingface.co/togethercomputer/RedPajama-INCITE-Bas...
- sanxiyn 3y ago
  Well, MPT-7B is also commercially usable and openly licensed: https://www.mosaicml.com/blog/mpt-7b https://www.mosaicml.com/blog/mpt-7b
  - simonw 3y ago
    Yeah it's really promising. It's partially trained on that RedPajama data.
  - akrymski 3y ago
    So is https://github.com/Lightning-AI/lit-llama https://github.com/Lightning-AI/lit-llama
ushakov 3y ago
$20M for a seed round? What’s their moat? How will they earn money?
- capableweb 3y ago
  Their outlined "mission" seems to be: > to empower innovation and creativity by providing leading open-source generative AI models and a cloud platform that makes AI accessible to anyone, anywhere. Since it's generally hard to earn money on the first part, I'm guessing they expect to make money on the latter part, assuming "anyone, anywhere" meaning people that can do business with US entities + people who can afford their service.
  - ushakov 3y ago
    How is this different from let’s say huggingface? IMO, this sounds more like an excuse to cash out the $20M check
ryanschneider 3y ago
No mention of their alignment strategy, if any. I feel like alignment is even more important in open models since it’ll be much easier to train a “but evil” version, right?

73 points by tim_sw 3y ago

Anil1331 3y ago

How is it different from Stability AI and other platforms ?

orra 3y ago
> Together’s $20M seed funding to build open-source This is a huge contrast. Stability AI’s bread and butter is proprietary, not open source. Together seem to have a track record of open sourcing both code and models. To be fair, Stability AI have open sourced StableLM, which is welcome. But overall Stability exaggerate and lie about their openness.
mtillman 3y ago
Different investors.
etaioinshrdlu 3y ago
Stability AI is also no longer open. They seem to be commercializing SDXL and it's not open - "yet".
dmix 3y ago
It doesn't sound like it's trying to build a single model, it wants to build a network/community of them. One of which is their own. Probably sharing/coordinating training resources and other support systems? The things that are usually out of financial/technical scope for most smaller OSS AI projects. I could see smaller companies with open models spending to be a part of that or other financially backed OSS projects. Maybe it's more hands-on than just a hosted service like huggingface? I don't really know the market, so just speculating.

deleted 3y ago

[deleted]

tikkun 3y ago

The homepage reads like PR speak to me. Does someone here know concretely what the product will likely be? Is it a replicate/modal/banana competitor, an OpenAI/Anthropic competitor, a Stability AI competitor, or something else

starlust2 3y ago

I've noticed a big trend by commercial entities claiming to be "Open Source" but actually charging for licenses. Just because your code is in a public repo on github doesn't mean it's open source. If it's not an open source license it's not open. This is especially true of AI projects.

timtom39 3y ago
This is the org behind RedPajama. So, to date, they have been open.
slashdev 3y ago
Open source is overrated, in my opinion. I’d rather a healthy commercial entity driving development and supporting it, with visible source that I can modify as a big plus. The whole OSI open source thing is nice, but it’s not going to matter to me much either way. I’m more worried that the open source project is no longer maintained or unsupported or development is slow and it gets obsolete. Your values, and thus your opinion, may be different.
- pydry 3y ago
  Big tech seem to really dislike this model. E.g. the whole tantrum Amazon had over the Elastic Search relicensing or Google briefly banning SSPL from their code hosting platform.
capableweb 3y ago
Since you mention it in connection to Together/RedPajama, are you saying that what they are publishing isn't actually open source? Most of the stuff I see them publish is licensed under Apache-2.0 which is a FOSS license, so I find it weird to mention it here.
latchkey 3y ago
Vipul's razor... https://razor.sourceforge.net/ https://razor.sourceforge.net/ He's well known for producing open source.
josephd79 3y ago

simonw 3y ago

This is the company behind RedPajama, which is a VERY big deal. It's the most realistic open source attempt at recreating the Facebook LLaMA LLM, from scratch, in a way that supports commercial usage. They released their full 2.6TB training dataset last month, and it's significant: https://simonwillison.net/2023/Apr/17/redpajama-data/ https://simonwillison.net/2023/Apr/17/redpajama-data/ They've also started releasing new, commercially-usable openly licensed LLM models trained on that data. You can try one of those out here: https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1?text=Datasette+is https://huggingface.co/togethercomputer/RedPajama-INCITE-Ins...

ericflo 3y ago
I would also check out their 3B model. I tested it on launch with LoRA fine-tuning and found it to be surprisingly capable despite its size. I think a lot of people are skipping past testing it because it only has 3B params. Edit: https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1 https://huggingface.co/togethercomputer/RedPajama-INCITE-Bas...
sanxiyn 3y ago
Well, MPT-7B is also commercially usable and openly licensed: https://www.mosaicml.com/blog/mpt-7b https://www.mosaicml.com/blog/mpt-7b
- simonw 3y ago
  Yeah it's really promising. It's partially trained on that RedPajama data.
- akrymski 3y ago
  So is https://github.com/Lightning-AI/lit-llama https://github.com/Lightning-AI/lit-llama

ushakov 3y ago

$20M for a seed round? What’s their moat? How will they earn money?

capableweb 3y ago
Their outlined "mission" seems to be: > to empower innovation and creativity by providing leading open-source generative AI models and a cloud platform that makes AI accessible to anyone, anywhere. Since it's generally hard to earn money on the first part, I'm guessing they expect to make money on the latter part, assuming "anyone, anywhere" meaning people that can do business with US entities + people who can afford their service.
- ushakov 3y ago
  How is this different from let’s say huggingface? IMO, this sounds more like an excuse to cash out the $20M check

ryanschneider 3y ago

No mention of their alignment strategy, if any. I feel like alignment is even more important in open models since it’ll be much easier to train a “but evil” version, right?