menaerus

29 ms·

511.
by menaerus 1y ago
Writing software would have been a much easier place if we had CPUs like that today :D
512.
by menaerus 1y ago
Report is packed with interesting details. Engineering challenges and solutions chapter especially show how things which are supposed and expected to work break when put through a massive scale. Really difficult bugs. Great writeup.
513.
by menaerus 1y ago
From their report: > Once a production environment has been set up, we estimate that the model can be realistically trained in approximately 90 days on 4096 GPUs, accounting for overheads. If we assume 560 W power usage per Grace-Hopper
514.
by menaerus 1y ago
People should just ignore such low-quality material and stop feeding the troll. Information found in both the first and second part of "Memory is slow, disk is fast" series is wrong on so many levels that it isn't worth corre
515.
by menaerus 1y ago
All of them no. Some of them almost certainly. Majority, we don't know yet.
516.
by menaerus 1y ago
> ... and has an idea of how they work shouldn't think its going to lead to "AGI" Not sure what level of understanding are you referring to but having learned and researched about the pretty much all LLM internals I think
517.
by menaerus 1y ago
> If it ever becomes necessary for the OOM Killer to kill processes, the decision of which processes to kill will be made based on something called the OOM score. Each process has an OOM score associated with it. > Every running proce
518.
by menaerus 1y ago
My understanding, and per Intel documentation, is that RAPL is giving them power consumption over the whole package therefore I believe 22W for Cores (W) in their figure is correct? Other figures such as instruction decoder they seem to ext
519.
by menaerus 1y ago
https://www.usenix.org/system/files/conference/cooldc16/cool... uses 84W TDP-rated Haswell-architecture Intel i7-4770, and authors constructed the synthetic microbenchmark with 1.67 IPC FPU and 3.86 IPC
520.
by menaerus 1y ago
Isn't TDP supposed to be an upper bound of how much power budget there is for a chip when it's running under maximum IPC (which implies AVX512 workload spread across all cores with all the test data in all the L1 caches)? I guess
521.
by menaerus 1y ago
Not true. To downclock the core you will need to have a sustained AVX workload that spends the power budget, and probably over many many cores and not a single one, and even then it's going to be undeterministic.
522.
by menaerus 1y ago
Misconstrued arguments. First paper is saying that they measured 3% for the floating-point workloads and 10% for the integer workloads. > Based on Figure 3, the power consumption of the instruction decoders is very small compared with th
523.
by menaerus 1y ago
Nonetheless I still think it's impressive considering that LLVM codebase is one of the most complex ones to be found in the wild.
524.
by menaerus 1y ago
Confirmation bias much?
525.
by menaerus 1y ago
What efficiency did you have in mind? Bandwidth-wise M4 is ~10x to ~30x lower.
526.
by menaerus 1y ago
RAG commonly implies some sort of vector database to be built and which will then be used for response augmentation. If it operates over the repo, I believe it will index your codebase using those vector embeddings.
527.
by menaerus 1y ago
Well, we don't really know if they aren't doing exactly that for their internal code repos, right? Conceptually, there is no difference between fine-tuning the LLM for being a law expert of specific country and fine-tuning the LLM
528.
by menaerus 1y ago
The reason for depending on Boost in this repo is just few search characters away - he needs HTTP/WebSocket implementation and Boost.Beast provides it. The actual bloat here in this repo is conan.
529.
by menaerus 1y ago
I don't think so. Tech is transformative in a way that I can't think of any similar in the last few decades really.
530.
by menaerus 1y ago
It says every employee unless there's some other source you're reading this? Although I would expect the truth to be closer to what you said.
531.
by menaerus 1y ago
Anything can be improved. Or almost anything. The question is by how much, 5%, 10% or 300%. In this case, I am not really I understand the problem oxide is set to solve so I can't really comment precisely but to me it sounds that if we
532.
by menaerus 1y ago
Like what type of tasks/tools are we talking about here, asking questions about the content from (PDF) documents or?
533.
by menaerus 1y ago
Too expensive from what I have seen - price for reasonably large GPU rigs that can host medium to large models is anywhere between ~5$/hr to ~9$/hr. That's ~40$-72$ for 8-hour working day or ~800$-1500$ for ~20 working days i
534.
by menaerus 1y ago
I started to appreciate that compilers can do only as much and from my experience auto-vectorization doesn't really shine that much, it leaves a lot of performance on the table, and then it also messes up with the hand optimized code.
535.
by menaerus 1y ago
> if I remove this assert(), then Clang does something stupid and 30% of CPU time is spent stalling on this single instruction, so meh, leave it in Classic compiler games and similar happened to me just recently when I wrote a micro-opti
536.
by menaerus 1y ago
Yeah, it might be the LSD then, basically no frontend involved after the first loop iteration, and then no bottleneck in the backend as well. So, what did you end up having in the code? Ugly and fast or nice and slow? :)
537.
by menaerus 1y ago
I still don't understand if you're really sending byte by byte over TCP/UDP or? Because that would be detrimental for performance.
538.
by menaerus 1y ago
Probably everything that there is including the instruction fusion, no decoding aka uop cache, ideal port dispatching, no data dependency etc. If it was a loop then perhaps even LSD.
539.
by menaerus 1y ago
It's interesting nonetheless. I wouldn't expect measuring such an IPC in the wild without having to craft the code in such an artificial way so that it hits that bound.
540.
by menaerus 1y ago
Server CPUs are hitting those numbers already for many years, including the AMD. The thing here is that Apple optimized their core for a different workload than the rest. I don't think there's a secret sauce AMD isn't aware o

More ›