agcat

13 ms·

181.
Show HN: Scale PDF Q&A App to 10K Users with GPUs – <$250/Mo(cookbook.inferless.com)
7 points by agcat 3y ago|2 comments
182.
Finetune Phi-2 with DPO(tutorials.inferless.com)
1 points by agcat 3y ago|1 comments
183.
by agcat 3y ago
Recently fine-tuned, quantized and deployed Phi-2 model with DPO technique and did 4 bit quantization using bitsandbytes and then deployed on Inferless.com, got around ~7 seconds of cold-start and ~21.34 tokens/second.
184.
Implement Fractional GPUs in Kubernetes to save upto 50% cost(huggingface.co)
1 points by agcat 3y ago|1 comments
185.
by agcat 3y ago
Check out this detailed tutorial featured on the Hugging Face blog about utilizing fractional GPUs in Kubernetes. It demonstrates how splitting a single GPU into seven smaller units can save up to 50% in costs, each unit having its own reso
186.
by agcat 3y ago
Curious to learn about the different, potentially more valuable problem that you mentioned about? Is this for Replicate OR Continue?
187.
by agcat 3y ago
We are building something in the AI Infrastructure space.
188.
Startup Pivots: How we Iterated on 4 Ideas in 12 weeks(aishwarya-48913.medium.com)
3 points by agcat 3y ago|6 comments
189.
by agcat 3y ago
Pivoting with Purpose: Dive into the journey of moving from weak PMF at nearly $1Mn ARR, to a more robust, founder-market fit product, leveraging customer discovery and first principle thinking. This can help if you're thinking of chan
190.
Deploying Hugging Face Models on Nvidia Triton Inference Server at Scale(inferless.com)
2 points by agcat 3y ago|1 comments
191.
by agcat 3y ago
If you are deploying ML models at scale, leveraging Hugging Face pipelines could be much faster at generating tokens & need lesser code to get started when you deploy with the NVIDIA triton inference server. This blog is quite helpful t
192.
by agcat 3y ago
This is a great list. If you want to read about deploying HF models on triton inference server at scale for inference. Find this great read - https://www.inferless.com/learn/nvidia-triton-inference-infe...
193.
by agcat 3y ago
AWS claims that Inferentia2 chip delivers 4x throughput increase and a 10x latency reduction compared to Inferentia. Have you tried the offering? Would like to hear your experience
194.
Have You Tried AWS Inferentia2 for ML Deployments?(huggingface.co)
1 points by agcat 3y ago|1 comments
195.
Hugging Face X Inferless: First Ever AI After-Party in Bengaluru(partiful.com)
1 points by agcat 3y ago|1 comments
196.
by agcat 3y ago
Opened up an After-party post our afternoon meetup on Deploying Generative AI models( https://lu.ma/ijcugt4n ). RSVP to enjoy cool community demos of Generative AI companies, AI ArtWorks, learnings, engaging tech conversation