sliken

9 ms·

91.
by sliken 2y ago
A framework desktop with a strix halo at 2/3rd the price and similar memory bandwidth looks pretty good in comparison.
92.
by sliken 2y ago
I'm tinkering with gemma 3 27B on a last gen 12 core ryzen. I get 5 tokens/sec.
93.
by sliken 2y ago
Sounds promising, hope it's true, much like a mac studio with the m2 max. Otherwise a 128GB amd strix with 256GB/sec memory bandwidth for $2k looks good.
94.
by sliken 2y ago
They should work just fine, sure peak bandwidth will be limited, but most workloads are not bottlenecked by I/O bandwidth on PCIe 4, but are limited by IOPs which should be decent on the 9100.
95.
by sliken 2y ago
Keep in mind the digits is part of their server line and server OS, not the random embedded dev kits often used for developers targeting car entertainment systems and often with abandoned kernel + driver. The nvidia digits uses DGOS, same a
96.
by sliken 2y ago
Yes, very, but so far they haven't mentioned the memory bandwidth.
97.
by sliken 2y ago
Right, like the mac studio ultra. Or a dual socket AMD Turin. Or a grace+hopper (assuming you offload to the hopper). Latency is dictated by the laws of physics, more bandwidth is easy, but not cheap.
98.
by sliken 2y ago
Small, quiet, and more energy efficient is a good start. I also saw that while it competed well with a 4070 at average frame rates, that the 1% lows were twice as fast. The speculation I saw was that the 12GB vram limit was causing issues
99.
by sliken 2y ago
It's got a rtx 4070 laptop chip equivalent on chip, so most are not going to need more GPU or anything else that needs a x16 PCIe. Looks pretty nice for a small PC that's quiet, energy efficient, and don't flinch if you try
100.
by sliken 2y ago
But no published memory bandwidth.
101.
by sliken 2y ago
Please don't misused channels. In DDR4 1 channel was 64 bits. In DDR5 1 channel is 32 bits. So a 128 bit wide DDR4 system had 2 channels, but a 128 bit wide DDR5 system has 4 channels. The latest AMD server is 12 DIMMS wide, but h
102.
by sliken 2y ago
Nvidia Grace is DDR5, hopper is HBM. Many servers like Intel and AMD's latest and greatest all use DDR5.
103.
by sliken 2y ago
Yes, those are the ones with a 128 bit memory bus that can reuse designs from previous generations. Nearly every laptop and desktop has has 128 bit memory for the last few decades, the strix halo is the first with 256 bit wide x86 targeted
104.
by sliken 2y ago
Problem is there is no socket available for thin/light/low power CPUs to have a 256 bit wide bus. So to ensure 256 bits @ 8000 MHz works well AMD did it in the same package. In theory they could A) ship a halo without ram B) desi
105.
by sliken 2y ago
Yes it's possible, but at the cost of 2-3x less memory bandwidth, which is a key feature to provide GPU and large language model performance.
106.
by sliken 2y ago
Sadly there's no alternative if you want 256 bits wide at the moment. Well a previous generation threadripper with a huge increase in power consumption and requiring a discrete GPU. BTW, careful with "channels". Todays norm
107.
by sliken 2y ago
SFFs and laptops share many parts, and of course the new mini desktop has 2 of the framework compatible bays for your choice of usb/network/audio that should work today and be able to be upgraded later. I don't do much comput
108.
by sliken 2y ago
Teslas turn on the brake lights when regeneratively braking.
109.
by sliken 2y ago
If you have a few 100 people in an area literally spending their waking hours worrying about having enough food. Areas without enough of the right nutrients are pretty common. People are pretty good at figuring out what makes them feel be
110.
by sliken 2y ago
Or buy 2 Nvidia digits for $6,000 to get 256GB vram.
111.
by sliken 2y ago
Keep in mind the strix halo APU has a 256 bit wide memory bus and the Mac Ultra has a 1024 bit wide memory bus. Here's hoping the Nvidia Digit (GB10 chip) has a 512 bit or 1024 bit wide interface, otherwise the Strix Halo will be the b
112.
by sliken 2y ago
I do want a 192GB Mac Ultra, I'm hoping the Nvidia Digit achieves similar at $3,000. Sadly no specifications or benchmarks, so tokens/sec is just a guess at this point.
113.
by sliken 2y ago
Maybe you want to configure 4 nodes to provide redundant network service. Or experiment with VLANs, firewalls, QOS, and related. Sure transcoding multiple 4k video streams is intensive, but there's plenty to be learned about performan
114.
by sliken 2y ago
Er, and how to you make the images? Or a family of tweaked images for different use cases. The make sure security standards, access to centralized services, etc is working? Doesn't that just move where you need CM a bit?
115.
by sliken 2y ago
You mean like over half of hardware added to the amazon cloud in the last year? Graviton, now in it's 4th gen, seems to be doing quite well. I believe other large cloud providers are working on similar home grown arms. Or maybe all t
116.
by sliken 2y ago
Arm's definitely trying to push on the laptop, tablet, desktop, and server markets. The fastest cluster on the top500 was arm for several years, most of the big clouds either have home grown arm servers (like graviton) or will soon. T
117.
by sliken 2y ago
Not to mention the GB10 where Nvidia mates a 20 core arm chip to a blackwell chip and puts it in a widget the size of a mac mini.
118.
by sliken 2y ago
Step #1 keep your CM files in Git. My favorite part of puppet is demonstrated well by the trifects: package { 'openssh-server': ensure => installed, } file { '/etc/ssh/sshd_config':
119.
by sliken 2y ago
They announced May, has that changed? Source?
120.
by sliken 2y ago
NVidia basically abandonned various tegra related SBCs. But this one runs DGOS which they have a few $b worth of hardware floating around in various clouds and is a derivative of ubuntu. Seems like this is MUCH more likely to have at least

More ›