jmalicki

10 ms·

181.
by jmalicki 4mo ago
If LLMs were already good at it, the AI labs wouldn't be paying this insane amount of money for people to generate training data to teach them.
182.
by jmalicki 4mo ago
Right now, this is a 10-figure run rate industry. They are generating a lot of this. Also remember it's not just quantity, it's roughly active learning - they're paying for training data that's at the classification bou
183.
by jmalicki 4mo ago
Yes, which is why for some things I've gotten paid as much as $1500 per training example generated. AI labs don't care about cost prohibitive.
184.
by jmalicki 4mo ago
There is one level that these training data give examples of specific static reasoning chains. Given exposure to enough reasoning chains, with training data that is designed around adversarial reasoning and teaching models to reason, these
185.
by jmalicki 4mo ago
There is also an endless fountain of things you come across every day and think "oh, wouldn't this complex solution to this low priority problem be cool", but noone ever implements it because it's too complex and the pro
186.
by jmalicki 4mo ago
I wish our actual world wasn't an implausible scifi novel!
187.
by jmalicki 4mo ago
Which is why rubrics as rewards are used.
188.
by jmalicki 4mo ago
That is a great example of the kind of thing they're paying people to create as training data. You write the prompt, and then write rubrics to judge the responses, and you found something the model failed at. Congratulations, you just
189.
by jmalicki 4mo ago
> And how much? Mercor, one of the larger vendors for contracting with experts to create bespoke data, says on their webpage they're paying $3M/day to their contractors for data. So well into the billions of dollars a year for
190.
by jmalicki 4mo ago
> How do they know [person] is an expert in [some field]? How do they find that person? They have a PhD from a top school, they are a licensed attorney, they are a licensed physician, a board certified cardiologist, etc. They are constan
191.
by jmalicki 4mo ago
I've done Mercor and other brands - the contracts move around, since the labs want the vendors to know they're just vendors and have to compete with each other. It seemed to be roughly resume and interview similar to getting hire
192.
by jmalicki 4mo ago
The most advanced training data is in the form of rubrics as rewards. A human asks a question, then writes rubrics to judge the LLMs response, so rather than evaluating a specific response, those rubrics can live on as the LLM evolves and g
193.
by jmalicki 4mo ago
As a side gig, I write novel software that solves problems no existing software does, that existing LLMs have difficulty reproducing, purely for the purpose of existing as LLM training data. There are journalists being hired to write Atlant
194.
by jmalicki 4mo ago
I find these internet arguments talking about LLMs as if they are trained by reading the internet to be wild. Yes, pretraining still exists. But for the past few years, pretraining by reading the internet is just the initial bootstrapping
195.
by jmalicki 4mo ago
Fair! At the same time, a lot of interest in this paper is specific to the AI industry (even if it's not only about that), and outside of high-end labor, the other constraints are new there.
196.
by jmalicki 4mo ago
It's not that supply was actually infinite, but you didn't realistically have situations where you said "I want to buy GPUs for a data center" only to be told "there's a 3 year waiting list." You might hav
197.
by jmalicki 4mo ago
The ratio of managers:staff engineers has been decreasing, so sort of, yes.
198.
by jmalicki 4mo ago
Then they get fired for poor performance and you hire new fresh people. Rinse, repeat.
199.
by jmalicki 4mo ago
If you can't at least guarantee write ordering you don't even have consistency. Fsync is often used when the data doesn't truly need to be on disk, because there aren't very good write ordering APIs exposed, even if th
200.
by jmalicki 4mo ago
"Every batch of writes called file.Write on the write-ahead log" You don't write to the WAL on a batch. > the author tested correctness after a crash. You mean the LLM?
201.
by jmalicki 4mo ago
Writing to disk for every write is required, otherwise you're not durable. Sure it's faster to never write to disk, then you reboot and you've lost data. /dev/null is a webscale database that is even faster!
202.
by jmalicki 4mo ago
You're saying charities like OpenAI that people have given money to haven't pushed the frontier on anything? You'd rather vibecode a SaaS or start a dropshipping business because that is somehow pushing the frontier more than
203.
by jmalicki 4mo ago
It is both absolutely gorgeous and luxurious, yet still at less than $6 million, pretty modest for someone capable of giving away half a billion.
204.
by jmalicki 4mo ago
The win for something like OpenAI isn't getting a ton of customers to pay $10-100/mo. It's getting businesses to pay $2k/mo or more per professional employee, like a lot of Anthropic customers. Anthropic is ahead of them
205.
by jmalicki 4mo ago
> would you want claude to be your boss? would you want claude to be your coworker? I've had worse. Mostly much better, but I've had worse.
206.
by jmalicki 4mo ago
People are rightly worried about that, but is there any indication that it nullifies any privacy contracts around the data? Is it: 1) We know that legally privacy terms to data are still binding, and those worried about it are freaking out
207.
by jmalicki 4mo ago
Which way do you think that goes? Are the ones who "get it" the ones who are captivated or see them as incremental?
208.
by jmalicki 4mo ago
Almost all computer PSUs I have ever seen are 110/220 since they don't make different models for Europe.
209.
by jmalicki 4mo ago
If you are going to do that, why not install a NEMA 5-20R receptacle, that has two independent circuits and is backwards compatible, as well as being rated for 20A per plug? The receptacle is the easy part, running the new circuit is the ha
210.
by jmalicki 4mo ago
If I wanted to use two circuits without running extension cords, every place I've lived would mean getting electrical rewired. If you're gonna get rewired you may as well install a 240v circuit, and some 120v 20a sockets while you

More ›