wrsh07

26 ms·

391.
by wrsh07 2y ago
Honestly, I thought her Ted ai interview was balanced and reasonable. I don't recall her mentioning yc, but I might have missed it. That said, the interviewer tries to sensationalize the upcoming interview as much as possible in the in
392.
by wrsh07 2y ago
I would only find that satisfying (from a snobbish and impractical perspective) if we were able to have the model decide: 1) what encoding should this section use? 2) how should I train this encoding? A mixture of experts but for encodings
393.
by wrsh07 2y ago
You can always use the API which is dirt cheap? Just put $5 on and access via the playground They have better data policies and your $5 will go way farther than a 1 month subscription
394.
by wrsh07 2y ago
Granting that, the original point was that they're not excited about this particular paper unless (for example) it improves the networks' general reasoning abilities. The problem was never "my llm can't do addition"
395.
by wrsh07 2y ago
This is cool, but special casing digits is unsatisfying. It makes me think that the authors have correctly identified an issue (positional embeddings) but don't propose a general solution. I'm not sure if such a thing is possible,
396.
by wrsh07 2y ago
In the book he discusses the shkadov thruster which reflects the energy so the shell would move, too. Obviously if it's not reflective you could absorb the energy and use it as needed https://en.m.wikipedia.org/wiki&#x2
397.
by wrsh07 2y ago
I'm reading Stephen Webb's book (If the Universe Is Teeming with Aliens ... WHERE IS EVERYBODY), and he describes how a partial Dyson sphere can turn a star into a spaceship which blew my mind (just cover all but one side, the rel
398.
by wrsh07 2y ago
I don't think this is entirely right (not a lawyer) You can't hire an artist to draw black widow in the style of Scarlett Johansson's widow. The issue isn't how the art is made, it's whether the end result looks lik
399.
by wrsh07 2y ago
Sorry, in the context of this tweet and my message, I was referring to "senior leadership leaving" This sort of high level turnover isn't that uncommon. OpenAI is growing quickly and going from initial concept to hundreds of
400.
by wrsh07 2y ago
Greg has some agency in this He also stuck by Altman in November. I'm surprised by how much credit and how much hate goes to the CEO when there are clearly other people making their own decisions, too. Say whatever you want, they'
401.
by wrsh07 2y ago
I think we're maybe gleefully posting past each other, but the point I'm trying to hit is that business models matter. Stack overflow provides a service. It's a good service. They host a great q&a platform for developers
402.
by wrsh07 2y ago
I listed 3 things that are free. Personally, I don't think ChatGPT will start running ads in the next ten years. However, let's assume that it does. Lmsys is for research, I suspect if it runs ads it will be like godbolt (a small
403.
by wrsh07 2y ago
My question is "how valuable is your time?" I think at a festival it's a little tricky to value (if it pulled you away from seeing your favorite band play a song, maybe this cost you the equivalent of $X, where that's wh
404.
by wrsh07 2y ago
ChatGPT serves 3.5 for free. You can run llama locally for free. Lmsys is free.
405.
by wrsh07 2y ago
They do serve ads, we should probably stop pretending "funded by ads" is the same as free. Your attention isn't free.
406.
by wrsh07 2y ago
There's no reason to think the current next token prediction models are optimal for predicting sentences (they aren't!) > An algorithm may outperform another on a problem when neither is specialized to the problem https:/
407.
by wrsh07 2y ago
I think any next token predictor will benefit. Iiuc mamba is a next token predictor. I just skimmed the gradient article, but if their only change is swapping out the transformer block for the mamba block, I don't think it's alrea
408.
by wrsh07 2y ago
My understanding is that theirs is a pure hardware solution. The hardware is flexible enough to model any current NN architecture. (Incidentally, there are black box optimization algorithms, so a system as good as grok at inference might be
409.
by wrsh07 2y ago
There are better and worse algorithms. I'm not sure "there is no free lunch" always applies in a particularly meaningful way. Some things aren't on the pareto frontier.
410.
by wrsh07 2y ago
I'm sure some security researcher is doing this, but we could easily create a visualization of "who has contributed over time" and identify transitioning of maintainers automatically just from git. This might be worth doing a
411.
by wrsh07 2y ago
I found this mesmerizing. Particularly fun is to scrub forward and backward through the video to clarify where exactly you're looking. (I found it worked better on the embedded video in the article than the yt one, not sure why)
412.
by wrsh07 2y ago
I interpreted it as a peak vs off-peak thing You might be able to buy either a peak or off-peak paper ticket, but you have to choose. If you make the wrong decision, you'll end up paying £3.90 too much. With contactless, you literally
413.
by wrsh07 2y ago
"unproved national security problem" and yet Congress was fairly well galvanized to vote for this bill after their intelligence briefings - multiple senators seem to want the briefing declassified: https://www.npr.org&#
414.
by wrsh07 2y ago
I don't really understand why banning one short form video platform infringes on speech. Wouldn't that mean banning any web 2.0 platform infringes on my speech if I use that platform? Regardless of why the platform might be banned
415.
by wrsh07 2y ago
I'm not sure why you're being downvoted. The law doesn't ban TikTok. It was written carefully with the doj, and being passed by Congress holds real weight vs an executive order (as it should!)
416.
by wrsh07 2y ago
There's always the question of "how easy is it to do a bad thing?" With guns, it's very easy to kill someone from far away. With a car, it's very easy to kill someone right in front of your car. At some point it wil
417.
by wrsh07 3y ago
My understanding from this tweet thread [1] is that chinchilla probably overspecified some of the hyperparameters to the model tl;dr I'm looking forward to having lots of models (ideally models) trained with a wide range of parameters
418.
by wrsh07 3y ago
Whoops you are absolutely right!! Good point, I totally misread the if/else.
419.
by wrsh07 3y ago
This was interesting and new to me, but as other commenters indicate, part of the problem is that we're trying to find the bad thing rather than trying to verify it is the good thing There's a related concept of "failing open
420.
by wrsh07 3y ago
Wasting two tokens on start/end reasoning seems expensive to me (a priori) I am curious what that would yield though - in some ways that would be the most fun to analyze (when does it think a lot??) I would also be curious to see at wh

More ›