3 ms·
Show HN: LunaLora: Multi-LoRA System to Combat Catastrophic Forgetting
- SphericalCowww 8mo agoThis is more of a conceptual idea from an AI hobbyist; hopefully, the big claims for motivation are not too distracting. After doing too many basic-level tutorials, this could be an interesting intermediate-level project to apply modern AI architectures. What is your opinion?
- sensarts 8mo ago[flagged]
- SphericalCowww 8mo agoJust to understand what you mean, you are worried that there are 2 or more LoRAs that have almost identical content but are distinct only in timestamps. Yeah, that could be a problem... Hmm..., perhaps the similarity for recency does need to be incorporated through orthogonality, but can somehow be implemented through weight preference, such that the new weight is influenced by the weight history. Thanks, that is a very good point, will take that into consideration!
- sensarts 8mo ago[flagged]