6 ms·

How to train your own large language models

283 points by benwen 3y ago

faithsmith 3y ago
[dead]
VWWHFSfQ 3y ago
Did we ever get any resolution about what happened after this company threatened to sue their intern for making a side project that supposedly stole all their great ideas? I would like to know before I ever consider anything from them again.
- plondon514 3y ago
  The founder admitted his mistake and the ex-intern's site is back up and running https://riju.codes/ https://riju.codes/. I'm personally a fan of both Amjad's (CEO) and Radon's (intern) and realize that everyone makes mistakes. It's not a reason to discount the hard work of the people at replit.
  - seeEllArr 3y ago
    If you read the blog post from Radon, Amjad's apology comes across as "sorry I got caught" rather than any actual remorse.
  - wpietri 3y ago
    A badly behaved CEO is absolutely a reason to avoid using a whole company. Reading through the entire story leaves me with a bad taste in my mouth, especially this bit: "still refused to list any specific part of Replit he thought I had copied, even when I asked him for such details multiple times during the phone call, despite his continuing to claim both privately and publicly that I copied Replit unethically". I haven't used Replit, but reading about it and looking at riju.codes, I have a hard time believing that there was any secret sauce that was inappropriately used, and the sketchy refusal to give details makes me think it's more about a CEO establishing dominance over the little people than any serious IP concern.
    - wahnfrieden 3y ago
      It ends with another threat - it's ok you copied, but don't copy more things (what things?)
  - birdplant 3y ago
    That’s a very generous interpretation of what happened because it wasn’t a “mistake” when he threatened the intern, it was something he purposefully and intentionally did, and doubled down on, even after having significant time to reconsider. Only when there was widespread public criticism of his actions did he backpedal. I’m curious what he’s said or done to make you a fan?
    - minhazm 3y ago
      Seems like you're just arguing about the definition of the word "mistake". Intent has nothing to do with it. From Google (Oxford Dictionary): > an action or judgment that is misguided or wrong. So to admit that you made a mistake just means you were "misguided or wrong" which he definitely made clear he was. You're claiming that there was significant time to reconsider but the reality is that this all went down in a matter of hours from when the intern published his article. Sometimes people have bad judgement and the public and especially one's peers can help to see the error of their ways and improve for the better. If he was a repeat offender and this happened several times then I could totally understand it but there's no reason to keep bringing this up every single time anything related to Replit is mentioned on HN.
      - birdplant 3y ago
        No, whole thing didn’t go down over a few hours. It was weeks. Not sure why you’d lie about this to protect someone you don’t know. Furthermore, not only did did he refuse to acknowledge his lies, he continued to lie, doubled down on the previous ones, and to this day still continues to make deceptive statements. So yes, as his slimy behavior has continued it is relevant to bring this up every single time replit is mentioned here.
    - alpaca128 3y ago
      Mistakes are not necessarily accidental or guilt-free. The CEO absolutely made a mistake.
  - johnfn 3y ago
    The CEO refused to apologize, and instead doubled down, taking advantage of a massive power differential between himself and a random college grad. He only apologized when the differential evaporated after the post hit the top of HN with something like 3000 points. I don't know about you, but I don't find that to be particularly acceptable, nor a "mistake", and I'm happy to continue to punish a CEO's unethical behavior.
    - nextworddev 3y ago
      Sounds pretty toxic
      - Cypher 3y ago
        Why is he toxic? reminds me of those media influencers calling for others to lose their jobs, freedoms and lives because they wouldn't take the vaccine. I'll never forgive them for their toxicity and I don't think that makes me toxic.
  - antonvs 3y ago
    > I’m personally a fan of both Amjad's (CEO) Why?
  - berkle4455 3y ago
    Thanks for linking this. This is actually a superior offering to replit. They recently removed the ability to access a simple repl without logging in. Now you a) have to login and b) have to deal with this obtuse IDE-in-a-browser project creation shit. It's so many extra steps before I can run code. I just want a URL in which I can run some code. https://riju.codes/ https://riju.codes/ is literally that. Thanks!
    - ZephyrBlu 3y ago
      I ran into the same thing and finally made a Replit account. I'm just gonna use Riju from now on though. Using Replit with an account is way more janky than it was without needing to login.
      - berkle4455 3y ago
        > Using Replit with an account is way more janky than it was without needing to login. It’s such a massive miss by their product teams. I don’t need this half-baked IDE. I want an interface that lets me run code as quickly as possible without any intermediate steps.
  - detrites 3y ago
    Not at all familiar with the details of this, but just to generally observe the bigger the mistake - the harder the walk-back and the higher the chance a lesson was learned. No guarantees, but it's rare to get past the first stage.
- turk- 3y ago
  Story for those who didn't see it: https://intuitiveexplanations.com/tech/replit/ https://intuitiveexplanations.com/tech/replit/
  - boredemployee 3y ago
    That's weird. I would never do anything even remotely similar to what my (ex) employer does. CEO sounds like a douchebag tho.
    - thejosh 3y ago
      I've seen terms/clauses here in AU for full time employment, depending on the industry/niche, where you can't jump to the same industry within X months.
      - boredemployee 3y ago
        That's what happens when you have a society worried about money and not interested in true human development.
- antonvs 3y ago
  Shh, another post on the home page is about them hiring (YC W18), don’t interfere with the business model!
  - dang 3y ago
    That's just a coincidence. The job posts go into a queue and it's semi-random when they get placed. The current submission appearing at the same time is unrelated. At least I presume it is.
- CityOfThrowaway 3y ago
  This CEO also has a history of punching down on Twitter. A very bad look.
ftxbro 3y ago
How expensive is it? My understanding is that it's not reasonable to train an LLM from scratch by yourself, and that if you want one that isn't just very stupid then you need to spend between hundreds of thousands and hundreds of millions of dollars. But if you don't want to train from scratch then you can fine-tune existing models for cheaper.
- kkielhofner 3y ago
  Looking at what they're doing here probably not as much as you think. As you note, with the plethora of open/open-ish LLMs today and LoRA + PEFT you can fine tune with low VRAM and pretty quickly so even a single A100 or whatever cloud GPUs are just fine. I've even seen people pull it off in reasonable time on super cheap T4s, A10s, etc. I doubt anyone reading a blog post is attempting to train a "true" multi-billion param LLM from scratch.
  - ftxbro 3y ago
    [flagged]
    - kkielhofner 3y ago
      Ouch. Ok, I'll try again because I read the article and watched the demo video from Mosaic: - MosaicML claims to have some magic that efficiently autoscales with awareness of specific instance per hour pricing. - MosaicML claims to have auto-optimize magic that makes training 3-7x faster. - The incentives seem to be aligned because their value prop is likely "we'll markup 10% but save you 50%" (or whatever). It's too bad Replit didn't provide the costs for this post in the same way Stanford, etc have done with Alpaca, Vicuna, etc. They probably didn't for from-scratch training because the answer to your question is almost certainly "it depends" or "if you have to ask you can't afford it" (for now).
      - ftxbro 3y ago
        Thanks I understand better now. Sorry my other comment was rude. Probably I'm just frustrated that I will never be able to make a sentient bot because it will cost literally a billion dollars, and I just have to sit on sidelines and watch the billionaires play with their sentient bots.
koboll 3y ago
Ghostwriter is notably worse than GPT-4, so while it may be true in a sense that "Training a custom model allows us to tailor it to our specific needs and requirements", the reality is they'd be getting better results just using OpenAI right now. Probably true for almost every other use case. That said, I am patiently waiting and champing at the bit for the day this isn't true anymore. Cool to see the groundwork being laid for it.
- jackmott42 3y ago
  That could all change in a few months. We saw locally runnable, open source image generation catch up quick.
- birdplant 3y ago
  They’re competing directly with Microsoft (and getting crushed) because GitHub is their biggest competitor, so it makes sense that they wouldn’t want to use OpenAI products. Agree that Ghostwriter is subpar though.
- two_in_one 3y ago
  just my 5 cents. it should be easier to train small custom model which works off a big pre-trained one. getting latent state as an input. while big model does all the hard work. but, getting latent means it should be accessible. that's why open source models are so valuable, even if they are not that good in general. more over, open source models can be used in other projects in various setups.
- brucethemoose2 3y ago
  Stable Diffusion 1.5 is not SOTA, but in reality the sea of augmentations makes SD kinda unbeatable, if you are willing to put in the work to use them. I think LLMs could end up the same way, if the comminity consolidates around a good one.
- nemoniac 3y ago
  If one of your specific needs and requirements is that you do not share data with OpenAI then this is a viable option.
zacksiri 3y ago
This is interesting. I think the future of AI will not be re-creating something like ChatGPT, but using variations of these methodologies to train AI models for specific tasks. There are some advantages to not having to make an LLM that impresses every human being on the planet. Imagine training the AI to be good in only one specific thing. I think it will become much more precise and deterministic. This is just my hypothesis. I'm excited to see where this goes.
- kkielhofner 3y ago
  There is plenty of less attention-grabbing work being done on "domain specific LLMs" like BioMedLM[0], Med-PaLM[1], BloombergGPT[2], etc. That reminds me - I saw a somewhat-clever acronym variant for LLM that communicated this the other day but it escapes me ATM... [0] - https://www.mosaicml.com/blog/introducing-pubmed-gpt https://www.mosaicml.com/blog/introducing-pubmed-gpt [1] - https://cloud.google.com/blog/topics/healthcare-life-sciences/sharing-google-med-palm-2-medical-large-language-model https://cloud.google.com/blog/topics/healthcare-life-science... [2] - https://dev.to/reaminated/thoughts-on-bloomberggpt-and-domain-specific-llms-1mon https://dev.to/reaminated/thoughts-on-bloomberggpt-and-domai...
  - MrMovl 3y ago
    I'm wondering about just that, I want to have a minimal model with little overhead that I can train to a specific body of texts but it doesn't need to know about all the rest. So basically it should be good at conversation and able to learn what I teach it, nothing else. But I'm having trouble finding resources about how to achieve that.
  - zacksiri 3y ago
    Yeah that's awesome. I honestly think the next 'leap' in AI will come from these 'domain specific' models. Also I'm not talking about just 'prompt' output model. These ones are great and I'm sure they will be extremely impressive. However I'm talking more about being able to 'operate' something. Imagine this, an AI able to operate some specific API in a deterministic / reliable way. I'm talking about complex operations. SO the output is not so much a text prompt but an SOP and then actually Operating the SOP. Imagine going into an app and say "can you boot up a cluster on AWS and run a wordpress site, point domain example.com to the site". Imagine this "you know my database for app X, what was the latest snapshot", it replies with the date / time of the snapshot, and you reply with "can you move that snapshot from google cloud and create a new database from that snapshot on AWS cloud?", and it does it for you. That's what I look forward to.
zwaps 3y ago
I am leading a similar initiative and I have also used Databricks for the preprocessing. Most interesting is what happens between the preprocessing and the model training - the hand-off to the cluster workers. I guess the efficient option is to partition the data, set up shards in-advance and ideally cache or even copy the data to local workers during init. This, of course, breaks some of the promise of being able to scale training flexibly, for instance to experiment with the scaling of compute and data. A different way to go about it is to use a streaming/iterable dataset/loader implementation with its own sharding logic that reads from a central store of parquets with some reasonable row-group size. This gives full flexibility in terms of node/gpu/worker/batch_size for experimentation - e.g. literally as parameters in PyTorch. Of course, one has to also implement caching of remote data since the data is kept centrally. In my opinion, there is no satisfying/flexible solution for this, especially when one also wants to experiment with complex transformations or augmentations in the dataset/loader and remain portable across cloud offerings. So, this has to be implemented from scratch (not too difficult but still a lot of code). The coming datapipes also probably make this trivial. Would love to hear more experiences in how you set this up! Edit: I guess for NLP this is a good implementation and what Mosaic uses https://huggingface.co/docs/datasets/stream https://huggingface.co/docs/datasets/stream
jxi 3y ago
Is this just a high level description of the process? There isn’t really any actual code to run or tutorial to follow.
- detrites 3y ago
  Yes, they mention they plan to publish articles with specific details later on as well as to open source some models.
dhruvbird 3y ago
Did the blog mention any metrics like what the model size is etc... since that seems to be the one of the motivating factors?
lxe 3y ago
Shameless plug: I built a toy tool to really quickly get into llm finetuning by just pasting your training samples into a textbox: https://github.com/lxe/simple-llm-finetuner https://github.com/lxe/simple-llm-finetuner
- ukuina 3y ago
  This is brilliant! Thank you for creating and sharing this.
rmbyrro 3y ago
> "a student coding on their phone in India should have access to the same AI as a professional developer in Silicon Valley. To make this possible, we train custom models with reduced cost." In principle, that's great. But the reality is: whoever has the resources and benefits from something better will look for ways to get it. What they're communicating here is: the most resourceful developers on the planet aren't our ideal customer.

283 points by benwen 3y ago

[dead]

Did we ever get any resolution about what happened after this company threatened to sue their intern for making a side project that supposedly stole all their great ideas? I would like to know before I ever consider anything from them again.

plondon514 3y ago
The founder admitted his mistake and the ex-intern's site is back up and running https://riju.codes/ https://riju.codes/. I'm personally a fan of both Amjad's (CEO) and Radon's (intern) and realize that everyone makes mistakes. It's not a reason to discount the hard work of the people at replit.
- seeEllArr 3y ago
  If you read the blog post from Radon, Amjad's apology comes across as "sorry I got caught" rather than any actual remorse.
- wpietri 3y ago
  A badly behaved CEO is absolutely a reason to avoid using a whole company. Reading through the entire story leaves me with a bad taste in my mouth, especially this bit: "still refused to list any specific part of Replit he thought I had copied, even when I asked him for such details multiple times during the phone call, despite his continuing to claim both privately and publicly that I copied Replit unethically". I haven't used Replit, but reading about it and looking at riju.codes, I have a hard time believing that there was any secret sauce that was inappropriately used, and the sketchy refusal to give details makes me think it's more about a CEO establishing dominance over the little people than any serious IP concern.
  - wahnfrieden 3y ago
    It ends with another threat - it's ok you copied, but don't copy more things (what things?)
- birdplant 3y ago
  That’s a very generous interpretation of what happened because it wasn’t a “mistake” when he threatened the intern, it was something he purposefully and intentionally did, and doubled down on, even after having significant time to reconsider. Only when there was widespread public criticism of his actions did he backpedal. I’m curious what he’s said or done to make you a fan?
  - minhazm 3y ago
    Seems like you're just arguing about the definition of the word "mistake". Intent has nothing to do with it. From Google (Oxford Dictionary): > an action or judgment that is misguided or wrong. So to admit that you made a mistake just means you were "misguided or wrong" which he definitely made clear he was. You're claiming that there was significant time to reconsider but the reality is that this all went down in a matter of hours from when the intern published his article. Sometimes people have bad judgement and the public and especially one's peers can help to see the error of their ways and improve for the better. If he was a repeat offender and this happened several times then I could totally understand it but there's no reason to keep bringing this up every single time anything related to Replit is mentioned on HN.
    - birdplant 3y ago
      No, whole thing didn’t go down over a few hours. It was weeks. Not sure why you’d lie about this to protect someone you don’t know. Furthermore, not only did did he refuse to acknowledge his lies, he continued to lie, doubled down on the previous ones, and to this day still continues to make deceptive statements. So yes, as his slimy behavior has continued it is relevant to bring this up every single time replit is mentioned here.
  - alpaca128 3y ago
    Mistakes are not necessarily accidental or guilt-free. The CEO absolutely made a mistake.
- johnfn 3y ago
  The CEO refused to apologize, and instead doubled down, taking advantage of a massive power differential between himself and a random college grad. He only apologized when the differential evaporated after the post hit the top of HN with something like 3000 points. I don't know about you, but I don't find that to be particularly acceptable, nor a "mistake", and I'm happy to continue to punish a CEO's unethical behavior.
  - nextworddev 3y ago
    Sounds pretty toxic
    - Cypher 3y ago
      Why is he toxic? reminds me of those media influencers calling for others to lose their jobs, freedoms and lives because they wouldn't take the vaccine. I'll never forgive them for their toxicity and I don't think that makes me toxic.
- antonvs 3y ago
  > I’m personally a fan of both Amjad's (CEO) Why?
- berkle4455 3y ago
  Thanks for linking this. This is actually a superior offering to replit. They recently removed the ability to access a simple repl without logging in. Now you a) have to login and b) have to deal with this obtuse IDE-in-a-browser project creation shit. It's so many extra steps before I can run code. I just want a URL in which I can run some code. https://riju.codes/ https://riju.codes/ is literally that. Thanks!
  - ZephyrBlu 3y ago
    I ran into the same thing and finally made a Replit account. I'm just gonna use Riju from now on though. Using Replit with an account is way more janky than it was without needing to login.
    - berkle4455 3y ago
      > Using Replit with an account is way more janky than it was without needing to login. It’s such a massive miss by their product teams. I don’t need this half-baked IDE. I want an interface that lets me run code as quickly as possible without any intermediate steps.
- detrites 3y ago
  Not at all familiar with the details of this, but just to generally observe the bigger the mistake - the harder the walk-back and the higher the chance a lesson was learned. No guarantees, but it's rare to get past the first stage.
turk- 3y ago
Story for those who didn't see it: https://intuitiveexplanations.com/tech/replit/ https://intuitiveexplanations.com/tech/replit/
- boredemployee 3y ago
  That's weird. I would never do anything even remotely similar to what my (ex) employer does. CEO sounds like a douchebag tho.
  - thejosh 3y ago
    I've seen terms/clauses here in AU for full time employment, depending on the industry/niche, where you can't jump to the same industry within X months.
    - boredemployee 3y ago
      That's what happens when you have a society worried about money and not interested in true human development.
antonvs 3y ago
Shh, another post on the home page is about them hiring (YC W18), don’t interfere with the business model!
- dang 3y ago
  That's just a coincidence. The job posts go into a queue and it's semi-random when they get placed. The current submission appearing at the same time is unrelated. At least I presume it is.
CityOfThrowaway 3y ago
This CEO also has a history of punching down on Twitter. A very bad look.

ftxbro 3y ago

How expensive is it? My understanding is that it's not reasonable to train an LLM from scratch by yourself, and that if you want one that isn't just very stupid then you need to spend between hundreds of thousands and hundreds of millions of dollars. But if you don't want to train from scratch then you can fine-tune existing models for cheaper.

kkielhofner 3y ago
Looking at what they're doing here probably not as much as you think. As you note, with the plethora of open/open-ish LLMs today and LoRA + PEFT you can fine tune with low VRAM and pretty quickly so even a single A100 or whatever cloud GPUs are just fine. I've even seen people pull it off in reasonable time on super cheap T4s, A10s, etc. I doubt anyone reading a blog post is attempting to train a "true" multi-billion param LLM from scratch.
- ftxbro 3y ago
  [flagged]
  - kkielhofner 3y ago
    Ouch. Ok, I'll try again because I read the article and watched the demo video from Mosaic: - MosaicML claims to have some magic that efficiently autoscales with awareness of specific instance per hour pricing. - MosaicML claims to have auto-optimize magic that makes training 3-7x faster. - The incentives seem to be aligned because their value prop is likely "we'll markup 10% but save you 50%" (or whatever). It's too bad Replit didn't provide the costs for this post in the same way Stanford, etc have done with Alpaca, Vicuna, etc. They probably didn't for from-scratch training because the answer to your question is almost certainly "it depends" or "if you have to ask you can't afford it" (for now).
    - ftxbro 3y ago
      Thanks I understand better now. Sorry my other comment was rude. Probably I'm just frustrated that I will never be able to make a sentient bot because it will cost literally a billion dollars, and I just have to sit on sidelines and watch the billionaires play with their sentient bots.

koboll 3y ago

Ghostwriter is notably worse than GPT-4, so while it may be true in a sense that "Training a custom model allows us to tailor it to our specific needs and requirements", the reality is they'd be getting better results just using OpenAI right now. Probably true for almost every other use case. That said, I am patiently waiting and champing at the bit for the day this isn't true anymore. Cool to see the groundwork being laid for it.

jackmott42 3y ago
That could all change in a few months. We saw locally runnable, open source image generation catch up quick.
birdplant 3y ago
They’re competing directly with Microsoft (and getting crushed) because GitHub is their biggest competitor, so it makes sense that they wouldn’t want to use OpenAI products. Agree that Ghostwriter is subpar though.
two_in_one 3y ago
just my 5 cents. it should be easier to train small custom model which works off a big pre-trained one. getting latent state as an input. while big model does all the hard work. but, getting latent means it should be accessible. that's why open source models are so valuable, even if they are not that good in general. more over, open source models can be used in other projects in various setups.
brucethemoose2 3y ago
Stable Diffusion 1.5 is not SOTA, but in reality the sea of augmentations makes SD kinda unbeatable, if you are willing to put in the work to use them. I think LLMs could end up the same way, if the comminity consolidates around a good one.
nemoniac 3y ago
If one of your specific needs and requirements is that you do not share data with OpenAI then this is a viable option.

zacksiri 3y ago

This is interesting. I think the future of AI will not be re-creating something like ChatGPT, but using variations of these methodologies to train AI models for specific tasks. There are some advantages to not having to make an LLM that impresses every human being on the planet. Imagine training the AI to be good in only one specific thing. I think it will become much more precise and deterministic. This is just my hypothesis. I'm excited to see where this goes.

kkielhofner 3y ago
There is plenty of less attention-grabbing work being done on "domain specific LLMs" like BioMedLM[0], Med-PaLM[1], BloombergGPT[2], etc. That reminds me - I saw a somewhat-clever acronym variant for LLM that communicated this the other day but it escapes me ATM... [0] - https://www.mosaicml.com/blog/introducing-pubmed-gpt https://www.mosaicml.com/blog/introducing-pubmed-gpt [1] - https://cloud.google.com/blog/topics/healthcare-life-sciences/sharing-google-med-palm-2-medical-large-language-model https://cloud.google.com/blog/topics/healthcare-life-science... [2] - https://dev.to/reaminated/thoughts-on-bloomberggpt-and-domain-specific-llms-1mon https://dev.to/reaminated/thoughts-on-bloomberggpt-and-domai...
- MrMovl 3y ago
  I'm wondering about just that, I want to have a minimal model with little overhead that I can train to a specific body of texts but it doesn't need to know about all the rest. So basically it should be good at conversation and able to learn what I teach it, nothing else. But I'm having trouble finding resources about how to achieve that.
- zacksiri 3y ago
  Yeah that's awesome. I honestly think the next 'leap' in AI will come from these 'domain specific' models. Also I'm not talking about just 'prompt' output model. These ones are great and I'm sure they will be extremely impressive. However I'm talking more about being able to 'operate' something. Imagine this, an AI able to operate some specific API in a deterministic / reliable way. I'm talking about complex operations. SO the output is not so much a text prompt but an SOP and then actually Operating the SOP. Imagine going into an app and say "can you boot up a cluster on AWS and run a wordpress site, point domain example.com to the site". Imagine this "you know my database for app X, what was the latest snapshot", it replies with the date / time of the snapshot, and you reply with "can you move that snapshot from google cloud and create a new database from that snapshot on AWS cloud?", and it does it for you. That's what I look forward to.

zwaps 3y ago

I am leading a similar initiative and I have also used Databricks for the preprocessing. Most interesting is what happens between the preprocessing and the model training - the hand-off to the cluster workers. I guess the efficient option is to partition the data, set up shards in-advance and ideally cache or even copy the data to local workers during init. This, of course, breaks some of the promise of being able to scale training flexibly, for instance to experiment with the scaling of compute and data. A different way to go about it is to use a streaming/iterable dataset/loader implementation with its own sharding logic that reads from a central store of parquets with some reasonable row-group size. This gives full flexibility in terms of node/gpu/worker/batch_size for experimentation - e.g. literally as parameters in PyTorch. Of course, one has to also implement caching of remote data since the data is kept centrally. In my opinion, there is no satisfying/flexible solution for this, especially when one also wants to experiment with complex transformations or augmentations in the dataset/loader and remain portable across cloud offerings. So, this has to be implemented from scratch (not too difficult but still a lot of code). The coming datapipes also probably make this trivial. Would love to hear more experiences in how you set this up! Edit: I guess for NLP this is a good implementation and what Mosaic uses https://huggingface.co/docs/datasets/stream https://huggingface.co/docs/datasets/stream

jxi 3y ago

Is this just a high level description of the process? There isn’t really any actual code to run or tutorial to follow.

detrites 3y ago
Yes, they mention they plan to publish articles with specific details later on as well as to open source some models.

dhruvbird 3y ago

Did the blog mention any metrics like what the model size is etc... since that seems to be the one of the motivating factors?

lxe 3y ago

Shameless plug: I built a toy tool to really quickly get into llm finetuning by just pasting your training samples into a textbox: https://github.com/lxe/simple-llm-finetuner https://github.com/lxe/simple-llm-finetuner

ukuina 3y ago
This is brilliant! Thank you for creating and sharing this.

rmbyrro 3y ago

> "a student coding on their phone in India should have access to the same AI as a professional developer in Silicon Valley. To make this possible, we train custom models with reduced cost." In principle, that's great. But the reality is: whoever has the resources and benefits from something better will look for ways to get it. What they're communicating here is: the most resourceful developers on the planet aren't our ideal customer.