3 ms·

Airgapped LLM inferrence server can't serve their output tokens, right?

• angry_octet 13d ago
  They can expose just their inference port, possible via some supervisor. The inference consumer can also be air gapped. This kind of segmentation is increasingly common for high value services.
  • what 13d ago
    Then it’s not air gapped…
    • angry_octet 12d ago
      It's air gapped from e.g. the internet.
• bibimsz 13d ago
  not at a high bitrate