valine

33 ms·

451.
by valine 4y ago
I have AIO liquid cooling for both cards. The radiators are annoying though, I might convert it to a custom loop if I ever add a third card.
452.
by valine 4y ago
I’ve got 2 rtx 4090s on an EATX motherboard. Been using them to run the full 13B un-quantized with a good deal of success. Getting about 20 tokens/s.
453.
by valine 4y ago
Are you using quantization? I’ve gotten very good results from the float16 13B vicuna model.
454.
by valine 4y ago
You can find the merged weights pretty easily online, I wouldn't hold your breath waiting for an official release given the licensing issues around LLaMA.
455.
by valine 4y ago
It’s hard to keep up with all developments around LLaMA. What’s the best RLHF alpaca like model you can download right now?
456.
by valine 4y ago
That means nothing. Studies can be funded from anywhere.
457.
by valine 4y ago
Even from an omniscient perspective it isn’t remotely clear that quantum waveform collapse isn’t entirely random. Quantum random number generators should be empirically nondeterministic.
458.
by valine 4y ago
Maybe ChatGPT is helping cut through bureaucratic tape. I know it’s helped me become much more persuasive in my communication. If your goal is to convince the US government to spend money testing your rotational detonation engine, your odds
459.
by valine 4y ago
Right, stable diffusion and control net depth network could give very good results if you have a source video.
460.
by valine 4y ago
OpenAI already has it, it’s part of the retrieval plugin: “A notable feature of the Retrieval Plugin is its capacity to provide ChatGPT with memory. By utilizing the plugin's upsert endpoint, ChatGPT can save snippets from the conversa
461.
by valine 4y ago
OpenAI has a moat, but not that large of a moat. Anyone with a couple million dollars can train a competitive LLM.
462.
by valine 4y ago
For me the most impactful use case is prototyping small apps that I sort of know how to build, but don't know the relevant libraries off the top of my head. My most recent example: I got an Onnx file from a guy with a small machine lea
463.
by valine 4y ago
Tesla uses a small language model for lane selection at intersections, they already shipped that to customers. Natural language is useful for systems with discrete elements and complex rules, which seems like a good fit for parsing intersec
464.
by valine 4y ago
I expect there is lots of low hanging fruit to be picked from LLMs. Imagine an LLM running locally on your car, refined to pick your lane based on the current traffic conditions and upcoming map data. Many of the problems with self driving
465.
by valine 4y ago
I always assumed it a name for the application of a language model, ie I’m using this language model as a programming copilot. Some models making better copilots than others.
466.
by valine 4y ago
They should train it on the monorepo
467.
by valine 4y ago
From the FAQ: "Can Bard help with coding? Bard can’t help you with coding just yet. Bard is still learning to code, and responses about code aren’t officially supported for now." So what exactly has the AI team at google been doin
468.
by valine 4y ago
Its not the truth. Young people are more difficult to persuade online. Technical literacy is on average higher than previous generations (though still not universal), and teens seem to be practically immune to current forms of online advert
469.
by valine 4y ago
The token limit is the problem, in general token limits can’t be changed after the model has been trained. Gpt4 has an exceptionally large 32k token limit, but even with 32k tokens you’d only get a few weeks of chat before the context windo
470.
by valine 4y ago
You could also take the conversation and use it as part of the reinforcement learning dataset. I feel like that's the closest thing to long term memory ChatGPT is capable of right now.
471.
by valine 4y ago
I think for now, the data requirements to train a SOTA LLM are so extreme we don’t have the luxury of being picky with the training data. We are getting close to the point where there isn’t enough human written text in existence to continue
472.
by valine 4y ago
If you can trust the model is at least as accurate as wikipedia then it becomes a drop in replacement for every task you do that requires wikipedia. There are a whole range of tasks that can’t be done today with an LLM because of the hallu
473.
by valine 4y ago
These language models are more than just autocomplete. Palm-E uses them for robotic planning and control, toolformer uses them to make arbitrary API calls, and gpt4 can build websites from napkin drawings. You really can’t imagine how an LL
474.
by valine 4y ago
While GPT-4 won’t, it’s almost certainly a piece of the AGI puzzle. AGI has the potential to solve world hunger.
475.
by valine 4y ago
The “for what” is to end world hunger, cure disease, and improve our understanding of the universe. The world is a miserable place full of overworked, underpaid people. What about the status quo do you really think is worth protecting?
476.
by valine 4y ago
The lion is an omnivore, it eats both the cabbage and the goat. The goat is for whatever reason, not interested in eating the cabbage. We know the lion will eat the cabbage because the riddle states the lion and cabbage cannot be left alone
477.
by valine 4y ago
The flow of the a river can be approximated with the Navier–Stokes equations. We might not be able to say with certainty it's an exact solution, but it's a useful approximation nonetheless. There was, past tense, no reason to beli
478.
by valine 4y ago
Neural networks are universal approximators. If cognition can be represented as a mathematical function then it can be approximated by a neural network. If cognition magically exists outside of math and science, then sure, all bets are off.
479.
by valine 4y ago
Just spitballing, I think you’d need a benchmark that contains novel logic puzzles, not contained in the training set, that don’t resemble any existing logic puzzles. The problem with the goat question is that the model is falling back on m
480.
by valine 4y ago
How do you know the model isn’t internally reasoning about the problem? It’s a 175B+ parameter model. If, during training, some collection of weights exist along the gradient that approximate cognition, then it’s highly likely the optimizer

More ›