rlupi

18 ms·

181.
by rlupi 3y ago
I am talking about training models on thousands of machines, each with thousands of GPU streaming processors. For data parallelism, if you want deterministic results, you need to merge weights (AllReduce, in the general case) in a determini
182.
by rlupi 3y ago
I haven't touched a 7900X, but AMD Rome and Milan have core complex (CCX) that share 16MB L3 cache. AMD Rome has significantly faster L3 cache than Intel ( https://www.anandtech.com/show/14694/amd-rome-epyc-2nd
183.
by rlupi 3y ago
1) If you read Nvidia GPUDirect RDMA page closely ( https://docs.nvidia.com/cuda/gpudirect-rdma/index.html ), you can find this paragraph: "Traditionally, resources like BAR windows are mapped to user or kernel
184.
by rlupi 3y ago
The inherent faults or even just speed differences[ ] of hardware. [ ] In the real world, a lot of resources are oversubscribed and not deterministic. Just think about how scheduling and power management work in a processor. Large model tra
185.
by rlupi 3y ago
I suggest designing APIs so that they can return warnings and still return useful data, as opposed to only success with data, or no errors. ( https://docs.influxdata.com/influxdb/cloud/api/#tag/Response...
186.
by rlupi 3y ago
The UN has identified 17 sustainable development goals ( https://sdgs.un.org/goals ) and the AI for good Global Summit https://aiforgood.itu.int/ (which was held last week) focused on exactly that. If you che
187.
by rlupi 3y ago
https://www.europarl.europa.eu/doceo/document/TA-9-2023-0237... > (11) This Regulation should apply to all categories of batteries placed on the market or put into service within the Union, regardless of wheth
188.
by rlupi 3y ago
> What nobody realized at the time was that was a capability based security system. that's a very interesting idea.
189.
by rlupi 3y ago
I have worked remotely for 10 years in smaller companies before going to a large FAANG and working in office for 10 years, then remotely since 2020. Working remotely for the smaller companies and startups was much faster faced and productiv
190.
by rlupi 3y ago
Or Naples https://www.youtube.com/watch?v=dCE1aUkeMe0
191.
by rlupi 3y ago
Does hw+sw count? Robots.
192.
by rlupi 3y ago
Interesting point. It makes me wonder, do having only experience in SV tech scene make you a worse entrepreneur than someone who has experience elsewhere where making a business sustainable earlier is necessary?
193.
by rlupi 3y ago
There are workloads that are data parallel, and scale like the GPU-heavy scale-out nodes that you describe. The other approach, which you do when models themselves are massive, is model parallelism. You split it into multiple parts that run
194.
by rlupi 3y ago
It would be interesting to see how fuzzy and subjective logic evolves when powered by different numerical systems. For example, surreal numbers are much more expressive than reals, and there are ways to express gaps as first order concepts
195.
by rlupi 3y ago
Out of curiosity. Do mostly use/want one GPU or the full server with all GPUs (8x A100 80GB, or 16x A100 40GB but I think only Google Cloud has those)? or a mix?
196.
by rlupi 3y ago
IMHO The biggest roadblock to use AI to rewrite code and blindly trust results is having good tests to validate results. This is mood for simulations and models. Also, neural ODEs are already taking traction. It won't be hard to replac
197.
by rlupi 3y ago
We are the start of a new S-curve. The current skilled employees will use their expertise to bridge the adoption of AI, bring the current use cases. The next wave of AI-native employees will build on their expertise, which we don't hav
198.
by rlupi 3y ago
In 2002, I wrote a CMS in Python and later Zope 2, in a tiny company with 3 SWEs. At a given point, it was used for internal sites in 20 banks, a couple of 5k+ employees companies and a division of Deutsche Bank that managed investment fund
199.
by rlupi 4y ago
ICI is interchip interconnect in TPUv4 pods. You can read more on paper that my collegues in Google platforms recently published https://arxiv.org/abs/2304.01433 https://cloud.google.com/blog/topic
200.
by rlupi 4y ago
This is totally unrelated to the language itself, but... You have my greatest appreciation for building proper responsive websites in all those links, with proper CSS styles and sizing (rem units)! I use a 40" 4K screen (no HiDPI) and
201.
by rlupi 4y ago
uhm? Every Smalltalk environment I know have great documentation, integrated right in the environment, and it is a better than Python for a beginner (I have been using Python since 1.6.1; I did Smalltalk professionally in the late 90s); sur
202.
by rlupi 4y ago
The world of servers has changed a lot in the recent past. Hardware complexity has exploded because now you have to use custom chips to be competitive: DPUs can accelerate data proccesing pipelines, large GPU clusters or custom chips like G
203.
by rlupi 4y ago
It looks like you don't know much about Europe. Let's start with browser. Opera, made in Norway. Search Engine. Ecosia, Qwant. Social network: Mastodon. Operating system: are you kidding right? Linux, Linus Torvalds is from Finlan
204.
by rlupi 4y ago
Well, this is just because they were looking for a specific kind of fault (performance regressions); CPUs tend to fail in a different way. (me: 10y+ Google SRE, 5y in ML and Borg SRE teams working very closely with hardware)
205.
by rlupi 4y ago
Nice work gathering all HTTP codes, but wouldn't this be simpler? lynx -dump https://developer.mozilla.org/en-US/docs/Web/HTTP/Status > HTTP-status-codes.txt grep 204 HTTP-status-codes.txt
206.
by rlupi 4y ago
Taking Schrems2 to the logical consequences, will we, from EU, have to leave HN or github too?