valine

27 ms·

421.
by valine 3y ago
This is a really good idea, I wish I had thought of it. Thinking of chat as a graph instead of a list is very weird and very cool. I think this could be especially useful in a research context where someone is trying to evaluate the perform
422.
by valine 3y ago
After getting access to the ChatGPT plugins beta I can say with confidence that you are utterly, hilariously, wrong. Trying language model plugins feel like using multitouch for the first time. It's the first new user-interface paradig
423.
by valine 3y ago
People didn't like Cortana because it sucked as an assistant. ChatGPT is actually useful.
424.
by valine 3y ago
The "As an AI language model" is a feature not a bug. There was a story on HN recently about this string showing up everywhere in Amazon reviews. If it makes spotting generated reviewers easier I'm all for it. https:/&#
425.
by valine 3y ago
I never said it was my code responsible for the race condition. ;)
426.
by valine 3y ago
Presumably the ChatGPT content that makes it onto the web is at the very least curated by humans, making that text on average slightly higher quality than the raw output of ChatGPT. If that's the case than you would expect model perfor
427.
by valine 3y ago
I’m smart enough to fix a race condition, but why would I spend a hour debugging when ChatGPT can find the issue in 10 seconds. Sure it might not be perfect, but having some initial code to fix the issue is a huge time save. Making ChatGPT
428.
by valine 3y ago
> I don't understand what people with a higher than average IQ are getting from llms People with higher than average IQ are smart enough to ask detailed and specific questions like “Fix the race condition in this [giant block of cod
429.
by valine 3y ago
That's why I put "motivation" in quotes. Also I was talking about training, where you compute a cost function and back propagate error. This is somewhat analogous to "motivation".
430.
by valine 3y ago
That's not even necessary. The whole point of RLHF is to prime the model to do what we want. The only time you need a reward system (ie jailbreak) with ChatGPT is when you want to do things that OpenAI specifically prohibited the model
431.
by valine 3y ago
ChatGPT, aside from the things we've specifically trained it not to do, rarely refuses to do what you ask. The much more capable Gpt4 is the same way. It's easy to personify the model and assume it gets bored the same way we get b
432.
by valine 4y ago
Pouring new concrete is cheap, they’ve already done it multiple times from damage due to test fires. Each time they reformulate the concrete and it needs less repair. After they get to orbit they can start putting payloads on board, and the
433.
by valine 4y ago
You can’t verify the reliability of an aircraft by flying it 1000 times because you’d be putting the life of the pilots at risk. Autonomous rockets have no such constraint. You could launch starship an unlimited number of times without putt
434.
by valine 4y ago
You actually don’t. With a high launch cadence you can make statistical inferences about the reliability of the system as a whole. This isn’t the 80s where you launch one disposable rocket every 6 months. Starship is fully reusable and will
435.
by valine 4y ago
Evaluating the reliability of every component is extremely expensive. If you had perfect knowledge of the reliability of every component, then sure. It would be cheaper to build a perfect rocket first, then launch it. The cost saving comes
436.
by valine 4y ago
The goal of this flight was always to prove they could get off the launch pad. They got past max-q, ergo it’s a success. The launch pad is intact, so the next starship rolling off the assembly line should launch in a couple months.
437.
by valine 4y ago
Also notable is that unlike most heavy lift rockets in history they have a functional assembly line that can rapidly churn these things out. We should only have to wait a couple months for the next test flight.
438.
by valine 4y ago
Not blowing up on the pad is a huge success. The rocket got past max-q, and they’ve proven beyond a shadow of a doubt that 33 engines can work together on one booster. I remember a couple years ago when folks like you were confident that it
439.
by valine 4y ago
You must be fun on road trips haha. I’ve never heard of someone enjoying a song more after the 100th time. I’d say there’s a sweet spot around the 3rd or 4th play for me, after that I enjoy the song less on each play.
440.
by valine 4y ago
Your favorite song loses its ability to trigger your dopamine when you listen to it 100 times in a row. Being able to generate unlimited music in the style of your favorite song sounds amazing.
441.
by valine 4y ago
Which Llama model have you played with? Unless it’s Llama 65B with 16f precision it’s not a fair comparison to gpt3.
442.
by valine 4y ago
AI models with obvious, broad, real world applications always seem get reproduced in public. Nvidia’s result is obviously great, but it’s still a long way from being useful. It reminds me of image generation models maybe 4 years ago. We nee
443.
by valine 4y ago
Screenshot for posterity https://imgur.com/a/AUgMwe2
444.
by valine 4y ago
I find it hilarious that if you ask google if google is down, it simply states “No”, despite showing no results. https://imgur.com/a/AUgMwe2
445.
by valine 4y ago
Cheap is relative I suppose. I’m running Vicuna 13b 16f locally and it needs 26GB of VRAM, which won’t even fit on a single RTX 4090. The next gen RTX Titan might have enough vram but that won’t come cheap. I’m expecting a price point above
446.
by valine 4y ago
Near term it’s a frustrating decision, but if these gpt4 vision LLMs are anything to go by it will prove to be the right decision in the long term.
447.
by valine 4y ago
Long term it’s better to have launch and landing on the same tower, if the goal is to rapidly reuse the rocket.
448.
by valine 4y ago
OpenAssistant has been collecting instruction/response data. They’ve already used that data to refine several llama models with good success. You can also bootstrap RLHF training data from the gpt4 api. Vicuna is probably the best publ
449.
by valine 4y ago
Fine tuning an existing model won’t get you to GPT4 level quality. You need a larger base model, and lots of fine tuning to have a chance.
450.
by valine 4y ago
Yeah I'm using GPU inference. The vicuna 13B model uses 26.3GB of VRAM with my setup. I'm running it split between two rtx4090s which gives me about 20 token/s.

More ›