jmalicki

12 ms·

241.
by jmalicki 5mo ago
I can no longer edit this, but want to expand on my comment. I've seen those vision researchers want to train on H100s at the time and being told know, wait for the T4s. I've seen T4s running BERT models for document classificatio
242.
by jmalicki 5mo ago
The costs to hire management and "support staff" like TPMs that scale with SWEs that help them meet goals is proportional to SWEs - often that is taken for the higher end fully loaded costs, depending on how you define it. Office
243.
by jmalicki 5mo ago
What do you think are running on the T4 GPUs in AWS? A lot of the use cases I know of for them are mid-level computer vision models that don't need to be frontier level.
244.
by jmalicki 5mo ago
As long as the demand for GPUs keeps increasing, there are more data centers being built to house them. When you have waitlists for many many months for Blackwell GPUs, keeping the old ones around as long as customers are willing to pay for
245.
by jmalicki 5mo ago
GAAP is basically a standards body to recognize practices. When there are interesting stories that can't be told with GAAP metrics, accountants derive new metrics. Just because they haven't gone through the standardization proces
246.
by jmalicki 5mo ago
That's a super interesting question and I agree! I am only saying the modern period of compound economic growth clearly started at the black plague with good explanations as to why. Why other events did not have the same effects are v
247.
by jmalicki 5mo ago
And that is super interesting! This is history not science so it is super incomplete. We definitely know that the trajectory of the industrial revolution was kicked off at the plague. We don't fully understand why there major growth pe
248.
by jmalicki 5mo ago
It's a long arc, and as sibling comments say, there are many books about it. The elites being able to capture some of it was what allowed for science and the enlightenment to happen, which eventually led to the technology that inspired
249.
by jmalicki 5mo ago
I will try to steelman watwut, a lot of people presume compound exponential economic growth is basically inevitable as a law, so you can always go back to some point in time. However, economic growth was basically flat before the Black Plag
250.
by jmalicki 5mo ago
The economy went from 0% growth for 100s of years to 3% growth that lasted many hundreds of years. Yes, at each step keeping the 3% growth needed things to keep going, but the big thing was unleashing the nonzero growth rate, to get unstuc
251.
by jmalicki 5mo ago
There was a very marked change in the growth rate, which is why economic historians focus on that. Yes, there were multiple further steps that needed to happen, as you note. But the black plague got the population "unstuck" from
252.
by jmalicki 5mo ago
The growth started in the 14th. It was awhile before the industry happened, but the change in growth rate is strongly connected with the Black Plague. One of the many, many descriptions of this is here (many because this is the mainstream
253.
by jmalicki 5mo ago
I mean all part of the surplus - you weren't struggling to barely survive so could do other things with your time, and some of the people used that surplus to invest in efficiency.
254.
by jmalicki 5mo ago
One of the leading economic history theories of why the industrial revolution happened was that it was largely a result of the Black Plague. The theory is roughly that before the Black Plague, the population was stuck in Malthusian dynamics
255.
by jmalicki 5mo ago
Sure, but the moment they use the token the revenue is earned.
256.
by jmalicki 5mo ago
For Anthropic in particular, that we're talking about, API token costs are revenue that is earned in real-time not on a contract - so hypothetically a giant spike in token use on an API use contract could spike revenue. But I don&#x27
257.
by jmalicki 5mo ago
I'm not an accountant, but afaik run rate is not a GAAP recognized metric. Presumably investors who care want it to be more precisely defined. In practice usually I've seen it be extrapolating the previous month. E.g. if you ha
258.
by jmalicki 5mo ago
Revenue is an integral over the past year or past quarter. Run-rate is taking a recent measurement and multiplying it out so it will span a year. Basically, it assumes they keep all of their current contracts, and don't gain any new o
259.
by jmalicki 5mo ago
With RLHF and RLVR we are creating tons of new training data, that is much more focused than reading the Internet. Annotation shops are doing many billions per year in revenue creating newer data, and a lot of it is highly complex, focused
260.
by jmalicki 5mo ago
Good question! Maybe the capabilities of newer GPUs allow AWS to charge higher margins for them? I don't actually know.
261.
by jmalicki 5mo ago
I agree - I was more talking about naive JSON ser/deser that Lemire was not involved in.
262.
by jmalicki 5mo ago
For lifespan, AWS is still running a ton of T4 GPUs from 2018, that power a lot of computer vision models. A ton of these will have a long life, not all ML is about frontier LLMs.
263.
by jmalicki 5mo ago
I think it's an extremely important distinction because self supervised learning has real inherent reward signals. Something like clustering does not.
264.
by jmalicki 5mo ago
Unsupervised is basically clustering. Alphago is RL - winning or losing a game is a form of supervision. Unsupervised is something where there is no intrinsic reward signal. In pre training, predicting the next token and seeing that it ma
265.
by jmalicki 5mo ago
I don't know about this specifically, but I've seen a lot of big data jobs where 99% of the CPU was spent in JSON ser/deser. This might be a reasonable chunk of it.
266.
by jmalicki 5mo ago
pretraining isn't unsupervised, it is self-supervised - meaning it is moderately more scale limited.
267.
by jmalicki 5mo ago
A merely wet road (1mm of water) and one with 10cm can be hard to distinguish. If you avoid the former, you can't operate in rain at all.
268.
by jmalicki 5mo ago
It has lidar and radar, not just vision.
269.
by jmalicki 5mo ago
Pytorch dataloaders are often horribly inefficient, a lot of stuff there can benefit from Rust/C++
270.
by jmalicki 5mo ago
If you have ECC memory, you can actually monitor this. I've typically seen ~a dozen bitflips per year per machine when I looked at this on servers, except for the cases of a faulty RAM module. I am more worried about SSD corruption tha

More ›