ryao

57 ms·

961.
by ryao 2y ago
I have seen Gemini hallucinate ridiculous bugs in a file that had less than 1000 LOC when I was scratching my head over what was wrong. The issue turned out to be that the cbBLAS matrix multiplication functions expected column major indexin
962.
by ryao 2y ago
Google should release the weights under a MIT license like Deepseek.
963.
by ryao 2y ago
Install a Linux distribution and install your software in Wine.
964.
by ryao 2y ago
I recall MSN Messenger supporting phone calls during the Windows 98 days. There should be no fundamental reason why it could not handle VoIP.
965.
by ryao 2y ago
You can install Office 2016 in Wine. The newer versions have some sort of DRM that is illegal for Wine to support.
966.
by ryao 2y ago
Calling EMCAScript JavaScript was a huge mistake that is still biting us.
967.
by ryao 2y ago
How big are the active weights? That how much bandwidth you need per second per token.
968.
by ryao 2y ago
Given that OpenAI model outputs are littering the internet, is it even possible to train a new model on public webpages without indirectly using OpenAI’s model to train it?
969.
by ryao 2y ago
I would suspect the infinity fabric links are already saturated with the local RAM’s memory bandwidth such that you will not get more by accessing another socket’s RAM. Chips and Cheese suggests things are even worse than this as the per CC
970.
by ryao 2y ago
TCP checksums usually prevent corruption, unless the data is already corrupt prior to the checksum computation (which does happen rarely).
971.
by ryao 2y ago
It typically somewhere between 3 to 5 characters.
972.
by ryao 2y ago
Did you implement token generation for Deepseek R1 using PBLAS?
973.
by ryao 2y ago
The answer to your question is yes. There is an open issue with llama.cpp about this very thing: https://github.com/ggerganov/llama.cpp/issues/11333 The TLDR is that llama.cpp’s NUMA support is suboptimal, wh
974.
by ryao 2y ago
llama.cpp’s token generation speed does not scale with multiple CPU sockets just like it does not scale with multiple GPUs. Matthew Carrigan wrote: > Also, an important tip: Go into the BIOS and set the number of NUMA groups to 0. This w
975.
by ryao 2y ago
Maybe. It likely depends on whether GEMM can operate at/near full speed with streaming weights over PCI-E. I am not sure how to stream the weights over PCI-E for use by CUDA/PTX code offhand. It would be a R&D project.
976.
by ryao 2y ago
Like this? https://threadreaderapp.com/thread/1884244369907278106.html
977.
by ryao 2y ago
It would be interesting to see the performance if someone built a single socket version. I have a parts list here if anyone wants to try it: https://news.ycombinator.com/item?id=42868360
978.
by ryao 2y ago
Jevons’ paradox seems relevant: https://en.wikipedia.org/wiki/Jevons_paradox
979.
by ryao 2y ago
It might be possible to desolder the chips and solder larger capacity ones, but it is a risky thing to do, especially since there is no guarantee that it will work out. That said, a similar upgrade has been done on the raspberry pi 4, so it
980.
by ryao 2y ago
A better setup on paper would be this CPU and motherboard combination with 12x64GB DIMMs: https://www.newegg.com/p/N82E16819113866 https://www.newegg.com/supermicro-h13ssl-nt-amd-epyc-9004-se... As for
981.
by ryao 2y ago
This seems relevant: https://finance.yahoo.com/news/deepseek-temu-ai-analysts-132... People are already looking at it like Temu.
982.
by ryao 2y ago
The deployment cost is even lower when you don’t need to put a tower on the ground, which is the point of this. They can cover the entire U.S. to eliminate coverage gaps using the starlink satellites.
983.
by ryao 2y ago
My understanding is that the starlink satellites will use T-Mobile’s spectrum to communicate with the cellphones, which requires a partnership.
984.
by ryao 2y ago
They claimed that it only took $5 million of GPUs to train Deepseek v3, which was the base model. They did not claim that the total costs were $5 million. They omitted the costs of additional hardware, electricity, personnel, training datas
985.
by ryao 2y ago
When running inference workloads via something like llama.cpp, only 1 GPU is ever used at a time, so you would have 1 active GPU and 4 idle GPUs. That should make the power usage less insane in practice than you expect.
986.
by ryao 2y ago
Which one did you run?
987.
by ryao 2y ago
Dropbox reversed its stance on this. It added support for ZFS, XFS, ecryptfs and btrfs: https://help.dropbox.com/installs/system-requirements They say ecryptfs is only supported when it is backed by ext4, which is a bi
988.
by ryao 2y ago
The mq-deadline part is what confused me. That is a competing option for the Linux IO elevator that runs under ZFS. Anyway, I understand now and thanks for the data point. I added a list of questions to GitHub that could help narrow things
989.
by ryao 2y ago
It did not seem to directly address how one scenario is better than the other. The article felt like “do it this way; it is better and don’t worry how”. I should stress that I am only saying this because I find the omission curious. I do no
990.
by ryao 2y ago
It did not need to be. My pint was that upgrades being done by everyone did not eliminate all points of fragility.

More ›