3 ms·
Is there any hope that something like unsloth studio will let people retrain models continually so that, when the day comes that there are no good open models b
by zdragnar 2mo ago
Is there any hope that something like unsloth studio will let people retrain models continually so that, when the day comes that there are no good open models being released, something like the final generation of qwen whatever can continue being relevant into the future?
- zeeveener 2mo agoIt would likely fall on the harness (and possibly local datasets) to encourage that the model reach out for up to date information instead of relying on it's own internal "knowledge". We already see a lot of that with `web_search` tooling, so I imagine it would just become more essential to have tools like that.
- zdragnar 2mo agoMaybe this is just my lack of experience thinking, but that seems really unlikely to scale without retraining. The context will balloon with updated syntax for languages and APIs for libraries and so forth. Imagine if, for example, something on the scale of custom elements / web components were introduced in this post-open world. You couldn't fit all the information needed in a local model's context to tell it how to write a new custom element of any real complexity, especially if it needed to interact with the new post-release web transport specification to interact with a new language's client for a new database.
- esseph 2mo agoYou can just make archives of the web and language specs, best practices, etc and RAG it via a pipeline every month or so.