3 ms·

That's roughly akin to saying "if you have a wrench, you can fix a car", and posting a link to a YouTube tutorial with it.

bugglebeetle 3y ago
No, not really. The script does the vast majority of the work. The only challenges here would be knowing how to use Google Colab and formatting/splitting your training and test data. That’s the computer science equivalent of adding wiper fluid to your car.
- claytonjy 3y ago
  It's true we've made this easy, and that's awesome, but this is not what AI startups do; this is what other companies experimenting with AI do.
  - bugglebeetle 3y ago
    Ok, but that’s goalpost shifting. We’ve gone from there are not many people who can fine-tune a model (demonstrably untrue) to there are not many people who can do {???} that AI startups do. It’s unclear what is this special AI startup thing you’re referring to, but given that various fine-tuning strategies, like QLORA, emerged out of the open source community, this also seems unlikely to be true.
    - claytonjy 3y ago
      Yeah, that's fair. I could have been more precise about what an advantage research talent can be. As an example, the startup-employed AI researchers I know had already PEFT'd llama2 within a day or two of the weights being out, determined that wasn't good enough for their needs, and began a deeper fine tuning effort. That's not something I can do, nor can most people, and it's a serious competitive advantage for those who can. It's a rather different interpretation of "can adequately fine-tune" than "can follow a tutorial". When I think "AI startup", I think of the places where these people work. I don't think there's many of those people, and I think their presence is a big competitive advantage for their employers.
      - bugglebeetle 3y ago
        Understood. Apologies, I wasn’t trying to be combative. I agree that what you describe requires a special emphasis on AI stuff or at least a part of the org that has a research focus. I work on an R&D team at a legacy org and we do the latter.

bugglebeetle 3y ago

No, not really. The script does the vast majority of the work. The only challenges here would be knowing how to use Google Colab and formatting/splitting your training and test data. That’s the computer science equivalent of adding wiper fluid to your car.

claytonjy 3y ago
It's true we've made this easy, and that's awesome, but this is not what AI startups do; this is what other companies experimenting with AI do.
- bugglebeetle 3y ago
  Ok, but that’s goalpost shifting. We’ve gone from there are not many people who can fine-tune a model (demonstrably untrue) to there are not many people who can do {???} that AI startups do. It’s unclear what is this special AI startup thing you’re referring to, but given that various fine-tuning strategies, like QLORA, emerged out of the open source community, this also seems unlikely to be true.
  - claytonjy 3y ago
    Yeah, that's fair. I could have been more precise about what an advantage research talent can be. As an example, the startup-employed AI researchers I know had already PEFT'd llama2 within a day or two of the weights being out, determined that wasn't good enough for their needs, and began a deeper fine tuning effort. That's not something I can do, nor can most people, and it's a serious competitive advantage for those who can. It's a rather different interpretation of "can adequately fine-tune" than "can follow a tutorial". When I think "AI startup", I think of the places where these people work. I don't think there's many of those people, and I think their presence is a big competitive advantage for their employers.
    - bugglebeetle 3y ago
      Understood. Apologies, I wasn’t trying to be combative. I agree that what you describe requires a special emphasis on AI stuff or at least a part of the org that has a research focus. I work on an R&D team at a legacy org and we do the latter.