3 ms·

I think existing players will have trouble developing a low latency solution like us whilst they are still running on non-deterministic hardware.

• WanderPanda 3y ago
  What do you mean by non-deterministic hardware? cuBLAS on a laptop GPU was deterministic when I tried it last iirc
  • tome 3y ago
    Non-deterministic timing characteristics.
  • frozenport 3y ago
    Tip of the ice-berg. DRAM needs to be refreshed every X cycles. This means you don't know the time it takes to read from memory. You could be reading at a refresh cycle. This circuitry also adds latency.
    • LtdJorge 3y ago
      OP says SRAM, which doesn't decay so no refreshing.
      • ndjdbdjdbev 3y ago
        Timing can simply mean the FETs that make up the logic circuits of a chip. The transition from high to low and low to high has a minimum safe time to register properly...
• nickpsecurity 3y ago
  While you’re here, I have a quick, off-topic question. We‘ve seen incredible results with GPT3-176B (Davinci) and GPT4 (MoE). Making attempts at open models that reuse their architectural strategies could have a high impact on everyone. Those models took 2500-25000 GPU’s to train, though. It would be great to have a low-cost option for pre training Davinci-class models. It would great if a company or others with AI hardware were willing to do production runs of chips sold at cost specifically to make open, permissive-licensed models. As in, since you’d lose profit, the cluster owner and users would be legally required to only make permissive models. Maybe at least one in each category (eg text, visual). Do you think your company or any other hardware supplier would do that? Or someone sell 2500 GPU’s at cost for open models? (Note to anyone involved in CHIPS Act: please fund a cluster or accelerator specifically for this.)
  • tome 3y ago
    Great idea, but Groq doesn't have a product suitable for training at the moment. Our LPUs shine in inference.