treprinum

10 ms·

31.
by treprinum 2y ago
How about running your intelligent home with a voice assistant on your own computer? In privacy-oriented countries (Germany) that would be massive.
32.
by treprinum 2y ago
My comment was about Intel having a starter project, getting enthusiastic response from devs, network effects and iterate from there. They need a way to threaten Nvidia and just focusing on what they can't do won't bring them ther
33.
by treprinum 2y ago
Intel has Xeon Phi which was a spin-off of their first attempt at GPU so they have a lot of tech in place they can reuse already. They don't need to go with GDDRx/HBMx designs that require large dies.
34.
by treprinum 2y ago
4080 won't do video due to low RAM. The GPU doesn't have to be as fast there, it can be 5x slower which is still way faster than a CPU. And Intel can iterate from there.
35.
by treprinum 2y ago
4090 is 5x faster than M3 Max 128GB according to my tests but it can't even inference LLaMA-30B. The moment you hit that memory limit the inference is suddenly 30x slower than M3 Max. So a basic GPU with 128GB RAM would trash 4090 on t
36.
by treprinum 2y ago
Nothing in my comment says about pricing it at the M4 Max level. Apple charges as much because they can (typing this on an $8000 M3 Max). 128GB LPDDR5 is dirt cheap these days just Apple adds its premium because they like to. Nothing preven
37.
by treprinum 2y ago
Inference on very large LLMs where model + backprop exceed 48GB is already way faster on a 128GB MacBook than on NVidia unless you have one of those monstrous Hx00s with lots of RAM which most devs don't.
38.
by treprinum 2y ago
Sufficient for LLMs and image/video gen.
39.
by treprinum 2y ago
NVidia and AMD make $$$ on datacenter GPUs so it makes sense they don't want to discount their own high-end. Intel has nothing there so they can happily go for commodization of AI hardware like what Meta did when releasing LLaMA to the
40.
by treprinum 2y ago
They can use LPDDR5x, it would still massively accelerate inference of large local LLMs that need more than 48GB RAM. Any tensor swapping between CPU RAM and GPU RAM kills the performance.
41.
by treprinum 2y ago
M3/M4 Max MacBooks with 128GB RAM are already way better than an A6000 for very large local LLMs. So even if the GPU is as slow as the one in M3/M4 Max (<3070), and using some basic RAM like LPDDR5x it would still be way faster
42.
by treprinum 2y ago
llama.cpp and its derivatives say yes.
43.
by treprinum 2y ago
Why don't they just release a basic GPU with 128GB RAM and eat NVidia's local generative AI lunch? The networking effect of all devs porting their LLMs etc. to that card would instantly put them as a major CUDA threat. But beancou
44.
by treprinum 2y ago
text-embedding-3-large + SPLADE + RRF
45.
by treprinum 2y ago
Anticipate what kind of questions user might ask, pre-compute the answers and store them as natural sentences in a vector database.
46.
by treprinum 2y ago
I hoped for at least 192GB of RAM for LLMs as 48GB DDR5 are pretty normal nowadays.
47.
by treprinum 2y ago
Poor. My M3 Max/128GB is about 20x slower than 4090. For inference it's much better, still much slower than 4090 but it enables working with much larger LLMs albeit at ~10t/s (in comparison, Threadripper 2990WX/256GB doe
48.
by treprinum 2y ago
Who designed the icons in MacOS up to Mavericks and iOS 6? Those were IMO the peak icon design on Apple devices. Would be great to hire that person for some Linux UI.
49.
by treprinum 2y ago
Isn't that better than normal? It used to be 90% of startups going belly up within 3 years, and out of remaining 10%, 9% becoming zombies and 1% having a proper exit? This looks more like Pareto 80/20 which is way better.
50.
by treprinum 2y ago
Try The Quest: Everest next. https://www.thequesteverest.com/
51.
by treprinum 2y ago
I can confirm, they are banned but VPN or Tor can access them without any issues. So it's only to prevent normies from accessing them.
52.
by treprinum 2y ago
Set firm boundaries and reject/ghost customers that want to build their app for free and are "angry". Those never lead to anything profitable.
53.
by treprinum 2y ago
Does HP have a black magic corporate division?
54.
by treprinum 2y ago
Generative AI often uses Markov chains as its theoretical basis and is modeled in the language of conditional probabilities albeit not stuck to a single ancestor but to a chain of ancestors. One can often view LLMs as scaled out Markov chai
55.
by treprinum 2y ago
While the science behind it is shoddy, some people with neuro long covid report significant improvement from it. Perhaps there is some not yet unearthed mechanism making it work in some cases?
56.
by treprinum 2y ago
Didn't YouTube get popular on pirated content first? What was the main difference of the initial phase of YouTube and MegaUpload? They both went legal later.
57.
by treprinum 2y ago
Yeah, but the "AI" on CPU means some basic NPU doing (sparse) matrix multiplication, not AVX512.
58.
by treprinum 2y ago
All these points about Intel are just waiting for an answer to "Is Intel capable of changing its prevailing culture or not?". There is a reason why Intel fell behind and it wasn't due to a lack of engineering geniuses inside.
59.
by treprinum 2y ago
MKL runs on Ryzens just fine and newest Ryzens all have AVX512 which all new desktop/laptop Intels lack. "AI" on CPU is a gimmick, everybody does it on GPU anyway (outside 128GB RAM M3 Max that has some LLM use, but it's
60.
by treprinum 2y ago
The article had a separate "mobile" category for that where AMD had <20% market share.

More ›