3 ms·
I’m curious if anyone’s employer has set up their own LLM. My employer has a couple of A100 sitting around which could easily host a couple instance of 65B LLaM
by syntaxing 4y ago
I’m curious if anyone’s employer has set up their own LLM. My employer has a couple of A100 sitting around which could easily host a couple instance of 65B LLaMA or Alpaca. Convincing upper management to allow me is the hard part.
- tiborsaas 4y agoTeam up with someone from sales, marketing :)
- AnthonyMouse 4y agoI just run it on my desktop? 64GB of DDR4 is <$150.
- syntaxing 4y agoI'm assuming a quantized version?
- AnthonyMouse 4y agoThe 65B quantized model fits in 64GB of RAM, which I already had. Though RDIMMs on eBay are even cheaper than UDIMMs (just over $1/GB) and Broadwell-era Xeon workstations aren't that expensive if you want to run the unquantized version.
- swyx 4y agowhat on earth do you need to convince them apart from gestures at all this