3 ms·
Taking a step back, was it necessary to roll your own LLM (or whatever FM) API as opposed to using an off the shelf API
by nextworddev 2y ago
Taking a step back, was it necessary to roll your own LLM (or whatever FM) API as opposed to using an off the shelf API
- zackangelo 2y agoYes we’re running our own inference server because we do nonstandard things in the decoder loop (think logit masking, other types of sampling, etc).