3 ms·
Ok, but that’s goalpost shifting. We’ve gone from there are not many people who can fine-tune a model (demonstrably untrue) to there are not many people who can
by bugglebeetle 3y ago
Ok, but that’s goalpost shifting. We’ve gone from there are not many people who can fine-tune a model (demonstrably untrue) to there are not many people who can do {???} that AI startups do. It’s unclear what is this special AI startup thing you’re referring to, but given that various fine-tuning strategies, like QLORA, emerged out of the open source community, this also seems unlikely to be true.
- claytonjy 3y agoYeah, that's fair. I could have been more precise about what an advantage research talent can be. As an example, the startup-employed AI researchers I know had already PEFT'd llama2 within a day or two of the weights being out, determined that wasn't good enough for their needs, and began a deeper fine tuning effort. That's not something I can do, nor can most people, and it's a serious competitive advantage for those who can. It's a rather different interpretation of "can adequately fine-tune" than "can follow a tutorial". When I think "AI startup", I think of the places where these people work. I don't think there's many of those people, and I think their presence is a big competitive advantage for their employers.
- bugglebeetle 3y agoUnderstood. Apologies, I wasn’t trying to be combative. I agree that what you describe requires a special emphasis on AI stuff or at least a part of the org that has a research focus. I work on an R&D team at a legacy org and we do the latter.