6 ms·
Damn, that's some impressive speeds. At that rate it doesn't matter if the first try resulted in an unwanted answer, you'll be able to run once or twice more i
by asabla 2y ago
Damn, that's some impressive speeds.
At that rate it doesn't matter if the first try resulted in an unwanted answer, you'll be able to run once or twice more in a fast succession.
I hope their hardware stays relevant as this field continues to evolve
- tjoff 2y agoThe biggest time sink for me is validating answers so not sure I agree on that take. Fast iteration is a killer feature, for sure, but at this time I'd rather focus on quality for it to be worthwhile the effort.
- jeswin 2y ago> The biggest time sink for me is validating answers so not sure I agree on that take. But you're assuming that it'll always ne validated by humans. I'd imagine that most validation (and subsequent processing, especially going forward) will be done on machines.
- tjoff 2y agoIf that is the way to get quality, sure. Otherwise I feel that power consumption is the bigger issue than speed, though in this case they are interlinked.
- threatripper 2y agoHumans consume a lot of power and resources.
- croes 2y agoThe basic efficiency is pretty high.
- yunohn 2y agoHow does the next machine/LLM know what’s valid or not? I don’t really understand the idea behind layers of hallucinating LLMs.
- ben_w 2y agoBy comparison with reality. The initial LLMs had "reality" be "a training set of text", when ChatGPT came out everyone rapidly expanded into RLFH (reinforcement learning from human feedback), and now there's vision and text models the training and feedback is grounded on a much broader aspect of reality than just text.
- yunohn 2y agoCould you link to a paper or working POC that shows how this “turtles all the way down“ solution works?
- ben_w 2y agoI don't understand your question. This isn't turtles all the way down, it's grounded in real world data, and increasingly large varieties of it.
- croes 2y agoHow does the AI know it’s reality and not a fake image or text fed to the system?
- exe34 2y agoDoes the AI need to know or the curator of the dataset? If the curator took a camera and walked outside (or let a drone wander around for a while), do you believe this problem would still arise?
- ben_w 2y agoI refer you to Wachowski & Wachowski (1999)*, building on previous work including Descartes and A. J. Ayer. To whit: humans can't either, so that's an unreasonable question. More formally, the tripartite definition of knowledge is flawed, and everything you think you know has a Munchausen trilemma. * Genuinely part of my A-level in philosophy
- croes 2y agoAnd who validates the validation?
- exe34 2y agothe compiler/interpreter are assumed to work in this scenario.
- vineyardmike 2y agoIf you're using an LLM as a compressed version of a search index, you'll be constantly fighting hallucinations. Respectfully, you're not thinking big-picture enough. There are LLMs today that are amazing at coding, and when you allow it to iterate (eg. respond to compiler errors), the quality is pretty impressive. If you can run an LLM 3x faster, you can enable a much bigger feedback loop in the same period of time. There are efforts to enable LLMs to "think" by using Chain-of-thought, where the LLM writes out reasoning in a "proof" style list of steps. Sometimes, like with a person, they'd reach a dead-end logic wise. If you can run 3x faster, you can start to run the "thought chain" as more of a "tree" where the logic is critiqued and adapted, and where many different solutions can be tried. This can all happen in parallel (well, each sub-branch). Then there are "agent" use cases, where an LLM has to take actions on its own in response to real-world situations. Speed really impacts user-perception of quality.
- tjoff 2y agoIf the speed is used to get better quality with no more input from the user then sure, that is great. But that is not the only way to get better quality (though I agree that there are some low hanging fruit in the area).
- OhNoNotAgain_99 2y agoTo be honest most LLM's are reasonable at coding, they're not great. Sure they can code small stuff. But the can't refactor large software projects, or upgrade them.
- regularfry 2y agoUpgrading large java projects is exactly what AWS want you to believe their tooling can do, but the ergonomics aren't great. I think most of the capability problems with coding agents aren't the AI itself, it's that we haven't cracked how to let them interact with the codebase effectively yet. When I refactor something, I'm not doing it all at once, it's a step by step process. None of the individual steps are that complicated. Translating that over to an agent feels like we just haven't got the right harness yet.
- croes 2y agoExactly, validating and rewriting the prompt are the real time consuming tasks.