wrsh07

19 ms·

211.
by wrsh07 1y ago
Oh shoot I thought that still applied to 4.5 just in a more "effective compute" way (not 100x more parameters, but 100x more compute in training) But alas, it's not like 3nm fab means the literal thing either. Marketing alway
212.
by wrsh07 1y ago
My understanding for the original OpenAI and anthropic labels was essentially: gpt2 was 100x more compute than gpt1. Same for 2 to 3. Same for 3 to 4. Thus, gpt 4.5 was 10x more compute^ If anthropic is doing the same thing, then 3.5 would
213.
by wrsh07 1y ago
Does anyone know if this is still up-to-date? All three authors are large contributors to the field (the book _discrete and computational geometry_ by O'Rourke & Devadoss is excellent), Demaine has some origami in the collection at
214.
by wrsh07 1y ago
This whole line of thought is sort of funny. Yes you can try training a model on synthetic data in such a way that it experiences model collapse That doesn't mean there aren't ways to train a model incorporating synthetic data wit
215.
by wrsh07 1y ago
The junior engineers on my team are just supercharged and their job is different from when I was a junior engineer. I would say: ten years ago there was a huge shortage of engineers. Today, there is still lots of coding to be done, but ever
216.
by wrsh07 1y ago
For all of the skepticism I've seen of Sam Altman, listening to interviews with him (eg by Ben Thompson) he says he really does not want to create an ad tier for OpenAI. Even if you take him at his word, incentives are hard to ignore (
217.
by wrsh07 1y ago
I don't agree with Tyler on this point (although o3 really is a thing to behold) But reasonable people could argue that we've achieved AGI (not artificial super intelligence) https://marginalrevolution.com/marginal
218.
by wrsh07 1y ago
I think the issue is exactly the spikiness because of how AC electricity works (whereas if the data centered were DC - eg wired through a battery - it wouldn't be an issue) I expect you're right that GPU data centers are a particu
219.
by wrsh07 1y ago
I tried desperately to source this even before seeing your request. My current guess is that I heard it on a podcast (either a Dwarkesh interview or an episode of something else - maybe transistor radio? - featuring Dylan Patel). I'll
220.
by wrsh07 1y ago
The flip side of this is meta having a hack that keeps their GPUs busy so that the power draw is more stable during llm training (eg don't want a huge power drop when synchronizing batches)
221.
by wrsh07 2y ago
This is a somewhat ridiculous comment. The article says many things about what one should pay attention to in order to identify what species of woodpecker you're hearing, and I'm sure you could easily get a list of species that li
222.
by wrsh07 2y ago
This is fun! I like the timing challenge but for many people this is a turnoff (there's a great interview with Zach Gage^ where he discusses research - some people are unable to critically think / learn new things while under pres
223.
by wrsh07 2y ago
I thought everything was pretty easy to use as soon as I realized what clicking a button would do (a little trickiness if you open the tile while the button is nearly off the top of the screen but honestly really great) Because I don't
224.
by wrsh07 2y ago
It's hilarious reading the other comments. I'm on mobile but my first thought was how interesting and novel the site design was and how clearly communicated the problem they were trying to solve Cool post! It's refreshing to
225.
by wrsh07 2y ago
It is obviously machine learning. It is a machine learning algorithm. It is taught in machine learning classes. It is described as a machine learning algorithm literally everywhere. Here's wiki: > Linear regression is also a type of
226.
by wrsh07 2y ago
Does this mean if I simulate a neural network on pen and paper that stops being machine learning? All of these are machine learning techniques. Doing it by hand doesn't change anything. Today we use machines so it's machine learni
227.
by wrsh07 2y ago
Linear regression is literally the second lecture of the Stanford ML class. https://cs229.stanford.edu/ If you want to say "not neural networks" or not dnn or not llm, sure. But it's obviously machine learnin
228.
by wrsh07 2y ago
Maybe this just makes your point, but solar panels being 20% efficient is about "how efficiently it converts energy from the sun into usable electricity" All that matters is the actual power generation since there's no margin
229.
by wrsh07 2y ago
My impression was that the most likely scenario was China blockading Taiwan until forcing a surrender. In that case, the US and its allies would have time to respond although may be busy doing other things (eg responding to ships near Austr
230.
by wrsh07 2y ago
Seems like it no longer does in the US but it's incredibly frustrating that we wouldn't ban lead based ingredients more generally https://en.m.wikipedia.org/wiki/Grecian_Formula
231.
by wrsh07 2y ago
Idk it seems fairly normal to have a hobby (eg photography, 3d printing, espresso) where you are able to spend thousands. The main issue is that I'm not sure it's easy to get into pressing vinyl - what's the starter kit price
232.
by wrsh07 2y ago
Note that they define algorithm 2 (see figure of said name) as Exchange Sort, which is distinct from the algorithm they present
233.
by wrsh07 2y ago
But as a response to the parent saying "LLMs will be great at ts/js slop but not for infra" it's quite reasonable to say: here's an example of someone applying it to backend optimizations today. Fwiw, there are alwa
234.
by wrsh07 2y ago
Eh Google had pretty good reasons to not operate in China (not seeing them in this thread, don't recall the details precisely enough to relate here) Apple is deeply embedded in China (manufacturing) and benefits from a decent (but shri
235.
by wrsh07 2y ago
Nah, you can just request that in your prompt and then fail answers that are incorrect and/or don't include the think trace
236.
by wrsh07 2y ago
4o is going to be better for a straight up factual question (But eg I asked it about something Martin Short / John Mulaney said on SNL and it needed 2 prompts to get the correct answer..... the first answer wasn't making anything
237.
by wrsh07 2y ago
I'm pleasantly surprised by the quality. Like you, I haven't tried the others, but I have heard tips about what questions they excel at (product research, "what is the process for x" where x can be publish a book or prod
238.
by wrsh07 2y ago
One of the challenges with proposing the rewrite of a complex codebase is that it requires the person planning and executing the rewrite to understand the requirements and the source of complexity. I used to think that there existed some qu
239.
by wrsh07 2y ago
Fwiw, I work at a saas company and we do have some performance issues. It's about 50/50 split between not using react optimally and slow backend. If we were using svelte we would still have performance issues, but they would proba
240.
by wrsh07 2y ago
> The more likely and far more mundane outcome isn’t that LLM providers actively tip the scales, but rather that they just entrench existing winners 100%. This isn't that different from the previous status quo (googling how to build

More ›