valine

17 ms·

181.
by valine 3y ago
Maybe. Goliath 120B took two different llama variants and interwove the layers. Surprisingly Goliath 120B quantized to 2bit is outperforming llama 70B 4bit in many benchmarks. https://www.reddit.com/r/LocalLLaMA/co
182.
by valine 3y ago
With Thanksgiving this week that’s a good bet.
183.
by valine 3y ago
I wonder if separate LLMs can find each other’s logical mistakes. If I ask llama to find the logical mistake in Yi output, would that work better than llama finding a mistake in llama output? A logical mistake might imply a blind spot inher
184.
by valine 3y ago
If he has safety concerns with OpenAI, he must be mortified with his old company Meta dropping 70B llamas on HF.
185.
by valine 3y ago
Whatever the intended outcome, losing half your employees to Microsoft certainly undermines it.
186.
by valine 3y ago
Not a word from Ilya. I can’t wrap my mind around his motivation. Did he really fire Sam over “AI safety” concerns? How is that remotely rational.
187.
by valine 3y ago
The largest rocket ever built just got to space for the first time. Even without re-entry this is a milestone. They have a production line to build these things. They’ll roll out the next one and try again. It’s not like SLS where everythin
188.
by valine 3y ago
Altogether, it was a terrific flight and a big step forward for the Starship program. The biggest success was the survival of stage zero, which bodes well for a quick turnaround for the next flight. Hot staging was the second big success, a
189.
by valine 3y ago
OpenAI has the most capable language model in the world, that’s bordering on a national security asset. I could see the US government stepping in to provide funding.
190.
by valine 3y ago
If anything this is a power grab by the board away from Microsoft. Optimistically, this could be an attempt to return OpenAI to its original status as a true non-profit company. OpenAI lost most of its openness under Sam. They needed the Mi
191.
by valine 3y ago
You're describing a post scarcity world. The whole point of an economy is to manage scarcity, talking about economic value post AGI is nonsensical. The responsibility to produce and distribute goods post AGI will fall to the government
192.
by valine 3y ago
You don't need SLI to render with dual GPUs in cycles.
193.
by valine 3y ago
You absolutely do not need exponential growth for UBI in a world with heavy automation. UBI is needed in a world where a large percentage of jobs have been automated, that implies everything in your supply chain is run by robots. When robot
194.
by valine 3y ago
Why would it be limited to a few people? If the impacts are as far reaching as people expect an open source AGI will be inevitable. Useful algorithms are very hard to keep secret. I imagine a future with few jobs, where everyone is largely
195.
by valine 3y ago
I'd think of it less like teaching the model something new, and more like enforcing a behavior the model can already output. Any decent raw model can output function names and parameters with prompt engineering. To do function calling,
196.
by valine 3y ago
Going from C++ to python is a little unfair. How many examples would a human need to learn python coming from C++? Its probably more than you can fit in a LLM context window.
197.
by valine 3y ago
I use it frequently with my own custom UI framework. It’s never seen my framework before but it can output new, useable code with just a few examples. If that’s not generalization I don’t know what is.
198.
by valine 3y ago
With quantization you can run this on a Raspberry Pi.
199.
by valine 3y ago
Tesla doesn’t use steer by wire for the yolk. It’s a regular steering wheel that’s just shaped different.
200.
by valine 3y ago
200 years with AGI? I’d be shocked if there weren’t a few hundred billion humans spread across the solar system.
201.
by valine 3y ago
People say this like it’s a fact. Until the courts decide otherwise or your AI model is regurgitating copyrighted data verbatim, generative AI is probably not violating copyright.
202.
by valine 3y ago
The model wasn’t update after the initial training, they’re definitely using in-context learning. The advancement here, as I understand it, is that they’ve achieved in-context learning with a very small model. From the paper: >> Few-s
203.
by valine 3y ago
It’s called in-context learning. I wouldn’t call it a trick so much as the most exciting and useful feature of LLMs. A lot of research is being done to expand the context window which would allow you to prime the model with entire books or
204.
by valine 3y ago
LLMs can absolutely learn new words. Try it with GPT4, you can make up words with definitions and it will start using them in conversation.
205.
by valine 3y ago
For anyone wondering about the architecture this is a 1.4 million parameter transformer model with both an encoder and decoder. The vocabulary size is comically small, it only understands 8 words. It learns new ideas with very few examples,
206.
by valine 3y ago
That sounds like a lot of work for something that can be worked around trivially. Add a small amount of random noise to the images and you’re done. Most people are probably doing this already for data augmentation purposes.
207.
by valine 3y ago
Yeah that makes sense, their original wording was ambiguous.
208.
by valine 3y ago
I doubt it. To pull that off you need a high frame rate and lots of motion tracking.
209.
by valine 3y ago
You could run it through marching cubes and get a mesh, whether or not that mesh would be printable is a different question entirely. It might actually be better to slice the point cloud directly. Go layer by layer and trace the boundary of
210.
by valine 3y ago
I’ve used iPhone lidar to 3D scan, the results are no where near this level of fidelity.

More ›