4 ms·
Great & motivational comment. Any pointers on where to start playing with the internals and sampling? Doesn’t need to be comprehensive, I just don’t know where
by creativenolo 2y ago
Great & motivational comment. Any pointers on where to start playing with the internals and sampling?
Doesn’t need to be comprehensive, I just don’t know where to jump off from.
- wruza 2y agoAfaiu “sampling” here, it is controlled with (not only?) topk and temp parameters in e.g. “text generation web ui”. You may find these in other frontends probably too. This ofc implies local models and that you have a decent cpu + min 64gb of ram to run above 7b-sized model. https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui https://huggingface.co/models?pipeline_tag=text-generation&sort=likes https://huggingface.co/models?pipeline_tag=text-generation&s...