3 ms·

Is it a fully custom inference engine or are you reusing parts of an existing stack? (llama.CPP, vLLM, etc.)

• i386 3mo ago
  Our skippy library is a patch queue on top of llama that allows us to access internal information, such as activations, and filter tensors on model load.
  • rao-v 3mo ago
    This really should be in the blogpost. It’s both useful info and basic courtesy to be explicit about which underlying inferencing engine you are using
    • sig_kill 3mo ago
      We didnt post it, we use a library (iroh) who featured us - so we are here answering any Q’s instead :)