lneiman

3 ms·

1. 1.▲
   by lneiman 8mo ago

   What exactly is this?
2. 2.▲
   by lneiman 10mo ago

   Author here. We were hitting tail latency and low GPU utilization issues serving SLMs via Triton. I built a scrappy client-side router using Redis and Lua to track real-time GPU load. It boosted utilization by ~40% and improved latencies. H
3. 3.▲
   Client-side GPU load balancing with Redis and Lua(galileo.ai)

   54 points by lneiman 10mo ago|11 comments
4. 4.▲
   Building financial integration with Cadence in DoorDash(doordash.engineering)

   4 points by lneiman 4y ago|0 comments