pavelstoev

9 ms·

91.
by pavelstoev 3y ago
Very nice. Can we chat briefly ?
92.
by pavelstoev 3y ago
Often you are better off running certain workloads on lesser GPUs. But this requires certain tricky compiler-level optimizations. For example, can run certain LLM inference with comparable latency on cheaper A40s vs running on A100s. Could
93.
by pavelstoev 3y ago
please consider CoreWeave - great experience so far.
94.
by pavelstoev 3y ago
May also suggest the suite of open source DeepView tools and which are part of PyPi. Profile and predict your specific model training performance on a variety of GPUs. I wrote a linked in post with usage GIF here: https://www.lin
95.
by pavelstoev 3y ago
AI/ML is a disruptive tech with huge financial benefits. History has shown us that government regulation of disruptive technologies can often have unintended consequences and push those technologies into the shadows, where they are har
96.
by pavelstoev 3y ago
Training and inference on GPUs significantly underutilize …the GPUs. So tuning and various tricks need to be applied to achieve dramatic performance gains. If I am not good at cooking, giving me a larger kitchen will not make me faster or b
97.
by pavelstoev 3y ago
I have been using The quick brown fox jumps over the lazy dog
98.
by pavelstoev 3y ago
Or you can apply GPU optimizations for such ML workloads. By optimizing the way these models run on GPUs, significantly improve efficiency and slash costs by a factor of 10 or even more. These techniques include kernel fusion, memory access
99.
by pavelstoev 3y ago
Generally, Hidet outperforms other inference compilers - PyTorch Eager, ORT, TRT, TVM. For example, PyTorch Eager - too much framework overhead. ORT -doesn't do operator fusion. TRT - close-sourced and hard to fix if a model can not ru
100.
by pavelstoev 3y ago
Generally, Hidet outperforms other inference compilers - PyTorch Eager, ORT, TRT, TVM. For example, PyTorch Eager - too much framework overhead. ORT -doesn't do operator fusion. TRT - close-sourced and hard to fix if a model can not ru
101.
by pavelstoev 3y ago
How much they pay for GPU use ?
102.
FOSS tool to estimate GPU-based DNN training time and costs(pypi.org)
1 points by pavelstoev 3y ago|1 comments
103.
by pavelstoev 3y ago
DeepView.Predict - FOSS tool developed by CentML that predicts a deep neural network's training iteration execution time on a variety of GPUs. Know your model training costs before running it !
104.
by pavelstoev 4y ago
model-depending, you can train on lesser (cheaper) GPUs but system-level optimizations are needed. Which is what we provide at centml.ai
105.
by pavelstoev 4y ago
Training a ChatGPT-beating model for much less than $85,000is entirely feasible. At CentML, we're actively working on model training and inference optimization without affecting accuracy, which can help reduce costs and make such ambit
106.
by pavelstoev 4y ago
Hi Bea and thank you for sharing your creation. Looking good ! Couple of questions: 1) could you please describe your data privacy considerations. Like what happens to my documents after they are uploaded ? Are they stored somewhere (encryp
107.
by pavelstoev 4y ago
We have these in Florida, USA and I have seen these types in Ontario, Canada where I am from originally.
108.
by pavelstoev 4y ago
As the world marvels at the astonishing capabilities of OpenAI's GPT-4, I find myself contemplating the rapid acceleration of AI and machine learning, and the evolutionary impact it is having on our lives. Naturally, I turned to GPT-4
109.
by pavelstoev 4y ago
At CentML, we profiled GPU utilization on a larger AI/ML research institute cluster. 10% to 45% range, mostly in 10% utilization range. We then offered them software optimizers (which do not affect model accuracy) to get to the 90% uti
110.
by pavelstoev 4y ago
wait till you visit eastern europe where drivers believed strings or single CD handing on the rear-view mirror somehow makes the car stealth to a police radar.
111.
by pavelstoev 4y ago
It does not have to be. We have optimizations for all kinds of workloads - https://CentML.ai
112.
by pavelstoev 4y ago
And we can make Ray more efficient by optimizing GPU hardware utilization https://centml.ai/
113.
by pavelstoev 4y ago
Before spending $$$ on the hardware accelerators, there is still a lot to achieve using software optimizations for inference and trainings workloads. We optimized GPT-J, and others, for real. https://centml.ai/benchmarks&#x2
114.
by pavelstoev 4y ago
We do GPU-specific training and inference speedups, at CentML.
115.
by pavelstoev 4y ago
It is a loosing uphill battle to reduce the operational costs as deep learning models get larger and more complex. Nvidia CEO says hardware alone will not be able to address the growing compute demand. The solution is computational optimiza
116.
by pavelstoev 4y ago
We have done the same and published back in early 2020. THis paper does reference our work as [11]. Blog post with our human detection video here (eerie !): https://futurebnd.com/human-detection-classification-and-vis... an