iamnotagenius

9 ms·

31.
by iamnotagenius 2y ago
RAG works well no one argues, but using RAG completely defeats the purpose of LLMs as their primary goal is to be a smart creative, not a simple summarizer.
32.
by iamnotagenius 2y ago
[flagged]
33.
by iamnotagenius 2y ago
Well there is general perception of being "advertised as such" from those who push LLMs and keep hyping them up as "PhD level performing" systems.
34.
by iamnotagenius 2y ago
I think you are unintentionally participating in gaslighting. Hallucinations in LLM is a massive problem, and normally, all computational systems we dealt with would immediately considered unusable, if, instead refusing to provide informati
35.
by iamnotagenius 2y ago
"Practically speaking" GP is obviously blatantly right, as there is no LLM in existence capable of doing what Karpathy claims is possible.
36.
by iamnotagenius 2y ago
LLM are generative AIs and have to be used as such - to generate a report from facts, to summarise an article, to translate from one language to another, anything where we agree to sacrifice accuracy for gain in creativity. As storage of fa
37.
by iamnotagenius 2y ago
Wikipedia or even Russia today is not advertised to be a source of unbiased truth.
38.
by iamnotagenius 2y ago
No it is not neither eerily nor computationaly similar to AI hallucinations, as we do not have a dam idea how the brains think.
39.
by iamnotagenius 2y ago
Here is a short exchange with free tier chatgpt: Me: who is Trumps grandpa? ChatGPT: Donald Trump's paternal grandfather was Friedrich Trump (originally known as Friedrich Drumpf). He was born in 1869 in Kallstadt, Germany, and immigra
40.
by iamnotagenius 2y ago
I use AVX-512 on Alder and it does not increase the voltage above AVX-2 voltages, and even then power dissipation is considerably lower than AVX-2.
41.
by iamnotagenius 2y ago
very strange as I get on old i5-12400+DDR4 2 tok/sec with 14B/q8 model.
42.
by iamnotagenius 2y ago
The performance comes mostly from a fraction of memory bandwidth needed, as LLM are mostly memory constrained. Compute matters too, but usually far less than memory.
43.
by iamnotagenius 2y ago
would be great to have dynamic quants of V3-non-R1 version, as for some tasks it is good enough. Also would be very interesting to see degradation with dynamic quants on small/medium size MoEs, such as older Deepseek models, Mixtrals,
44.
by iamnotagenius 2y ago
Yes, you are right; I meant "consumer grade cpu".
45.
by iamnotagenius 2y ago
Imo the most "unhinged" cpus for AVX-512 are early batches of Alder Lakes which is the only cpu family that has nearly full coverage of all existing avx-512 subsets.
46.
by iamnotagenius 2y ago
requires obscene amount of memory for context.
47.
by iamnotagenius 2y ago
People hate being caught committing hypocrisy, you and me will be downvotoed to oblivion.
48.
by iamnotagenius 2y ago
have not tried yet. any specific query? I can try.
49.
by iamnotagenius 2y ago
well this is not how normally coding models used anyway.
50.
by iamnotagenius 2y ago
10-15t/s on 12400 with ddr5
51.
by iamnotagenius 2y ago
I just run llama-cli with the model. Every time I want some "awk" or "find" trickery, I just ask model. Good for throwaway python scripts too.
52.
by iamnotagenius 2y ago
No, but I use llama 3.2 1b and qwen2.5 1.5 as bash oneliner generator, always runnimg in console.
53.
by iamnotagenius 2y ago
Coding problems are never solved with rag.
54.
by iamnotagenius 2y ago
small models can be funny (12b model): tell me funny story about techbro and real developer who actually ships code Sure, here's a light-hearted story for you: --- Once upon a time in Silicon Valley, there was a shiny new startup calle
55.
by iamnotagenius 2y ago
How exactly is using what is essentially free energy worse than using heatpumps? You will build datacenters anyway; it _will_ happen, irrespective of using or not using heatpumps. Using 10Mw of energy from a DS would mean that you'd ne
56.
by iamnotagenius 2y ago
Sadly LLM itself is not impressive. Not creative, not exactly smartest. good but not great.
57.
by iamnotagenius 2y ago
Yes, it is good for suumarizing existing text, explaining something or coding; in short any generative/transformative tasks. Not good for information retrieval. Having said that even tiny Qwen 3b/7b coding llms turned out to be ve
58.
by iamnotagenius 2y ago
But you _will_ build datacenters, this was my point.
59.
by iamnotagenius 2y ago
1. The reality of ex-USSR is that no one will ever convince governments to invest in very costly infrastructure modification for efficiency; datacenters are far far easier to integrate into existing boiler based system. 2. The point was not
60.
by iamnotagenius 2y ago
istjohn is not right. Energy produced by datacenters is simply wasted. even if you will use heatpumps there is still only so much the can produce; using datacenters waste energy would lessen the the power neede for her pumps.

More ›