3 ms·
Try this, it's for running llms that won't fit in the gpu: https://github.com/FMInference/FlexGen https://github.com/FMInference/FlexGen
by version_five 4y ago
Try this, it's for running llms that won't fit in the gpu: https://github.com/FMInference/FlexGen https://github.com/FMInference/FlexGen
- gpm 4y agoCurrently that looks like it only supports facebook's opt and galactica models. Though they do appear to plan to add support for more models.