valine

31 ms·

391.
by valine 3y ago
Yeah it handles it all gracefully. It’s definitely not just trapping the question.
392.
by valine 3y ago
A nearly omnipotent AI could probably get close to light speed, which would allow it to reach all edges of the galaxy in 100,000 or so years.
393.
by valine 3y ago
I think you’re getting hung up on the terminology. Call them “potential facts” if that makes you more comfortable. You can hold an idea in your head as potentially true without incorporating it into your world view. If you strip out every i
394.
by valine 3y ago
You misunderstood my comment. The GPT that filters the training set doesn’t need to recognize truth from lies. It needs to recognize which bits of text from the training set look like facts and label them as such. Everything in the training
395.
by valine 3y ago
That’s the “LLMs are a fuzzy jpeg of the web” theory, and it’s far, far from the scientific consensus. No one knows for sure exactly what happens inside a 500B parameter model. I’ll just leave you with GPT4’s response to your question “What
396.
by valine 3y ago
Or, more optimistically, AGI will accelerate scientific progress and we will have fusion, quantum computers, and self driving cars shortly after. If AIs had the tendency to recursively devour worlds you would expect at least one alien AI to
397.
by valine 3y ago
My hope would be that by labeling factual and fictional data it would coax the model into a state where it doesn’t invent facts outside of a creative writing context. Inventing falsehoods is practically the definition of creative writing, y
398.
by valine 3y ago
The datasets are massive, and curating them into buckets of fact/fiction by hand would be next to impossible. Automating curation of the datasets with the latest and greatest GPT sounds very possible however. GPT-supervised learning co
399.
by valine 3y ago
Relying on subject matter experts isn't a scalable solution. I would speculate that LLMs don't hallucinate when a fact is included a handful of times in the training set. I'd also speculate that our current LLMs can say wheth
400.
by valine 3y ago
Having a second adversarial network could absolutely be part of the solution, I wasn't arguing against that. My problem is with in context learning, ie the idea that including background information in the prompt can solve hallucinatio
401.
by valine 3y ago
The proposed solution to eliminate hallucination is to ground the model with external data. This is the approach taken with Bing Chat, and while it kinda works, it doesn't play to the strengths of LLMs. Every time Bing Chat searches so
402.
Enabling LLMs to Reason and Memorize with Self-Notes(arxiv.org)
1 points by valine 3y ago|0 comments
403.
by valine 3y ago
If this is possible with just from measuring blood flow imagine what we could extrapolate from electrodes implanted deep in the brain.
404.
by valine 3y ago
I tried GPT4, it failed. I also tried Vicuna which sort of passed. The conspiracy theorist in me thinks OpenAI did something to neuter the chess playing capabilities of their models. GPT4: Yes, a knight in chess can move from f6 to d4. The
405.
by valine 3y ago
I don’t buy into perplexity as a good benchmark for the usefulness of an LLM. In my experience playing with many LLaMA models of various sizes and quantization levels, the higher bit models perform significantly better on complex questions.
406.
by valine 3y ago
It’s useful because you can buy a Macbook with 64GB of RAM, and then use that RAM as VRAM to run your LLM.
407.
by valine 3y ago
Sure, but other than Apple no one has the tight integration needed to make that happen. If Apple leans into their CoreML stuff they have a real opportunity to steal the market from Nvidia. Long term maybe Nvidia is able to release release s
408.
by valine 3y ago
I hope they do, and I hope it forces Nvidia to release their own 48GB+ consumer card. 80GB is on my long term wish list as it would allow running a 65B model 8bit quantized. I don’t see local models exceeding ChatGPT performance until we ge
409.
by valine 3y ago
This is a well documented problem with all language models. ChatGPT with the Wolfram Alpha plugin can do math. That’s good enough for me.
410.
by valine 3y ago
The best of the best right now is probably Vicuna 13B. The 30B and 65B LLaMA models are better on benchmarks, but there isn’t a compelling instruct fine tuned version of those yet so they require a lot of prompt engineering. If you want to
411.
by valine 3y ago
Most of the repos for these new LLaMA derivatives are brand new, and put together by people who specialize in ML research not software engineering. I’m just happy to have access to their code. I’m sure most of these teams would be happy to
412.
by valine 3y ago
I’ve got 4 different llama models running locally with CUDA and can freely switch between them, including LLaVA which is a multimodal LLaMA variant. None of them are particularly difficult to get running, the trick is to search the project’
413.
by valine 3y ago
Ideally the model would insert an API call into its output to fetch the contacts first, then make the call. The scary part is asking the LLM to read your messages. Imagine someone texts you an LLM jailbreak. Siri might not be Siri anymore a
414.
by valine 3y ago
Why is that a problem? If you don't want the model to access your doors you disable it at the API level. Siri already has an option for that today.
415.
by valine 3y ago
How about a law? Then every farmer doesn’t have to negotiate repair terms.
416.
by valine 3y ago
There’s nothing particularly unsafe about LLMs. Giving Sirious access to a weather api and timers is not gonna break the world. If anything, giving LLMs access to apis makes them less likely to hallucinate.
417.
by valine 3y ago
One of the main benefits of right to repair is to keep tractors in service for longer. Your solution is for farmers to throw away their tractors and buy new ones? What’s to stop the new manufacturer from implementing a similar policy once t
418.
by valine 3y ago
I’d agree if it was a grift if Tesla was sitting on their hands and not making progress. Eight years really isn’t that long when it comes to solving hard problems. Tesla has a long attention span, and personally I think that’s a virtue whic
419.
by valine 3y ago
Predicting when an algorithmic problem will be solved is a fools game. I don’t think Musk was intending to deceive with FSD, he’s just wildly over optimistic and partially blinded by the financial upside. That being said, just because they
420.
by valine 3y ago
I’ve heard the argument made that LLMs are simply models overfit on the entirety of the internet. Double decent is a good counter point to that claim, it suggests something more interesting than simple overfitting is happening with large pa

More ›