3 ms·
Correct, but morph is a LLM as well. In practice its basically Big LLM using small LLM as a tool call
by bhaktatejas922 1y ago
Correct, but morph is a LLM as well.
In practice its basically Big LLM using small LLM as a tool call
- Workaccount2 1y agoI see. How is this not going to get run over immediately by big players? Google's diffusion model is already in the wings, and it's both wicked fast and ~flash-lite intelligent.
- bhaktatejas922 1y agoyou could make the argument about any startup really. To me its the same reason they don't build the foundational model for legal, for sales, etc.. - everything comes at a cost. Allocating researcher time to this is attention not spent on the general frontier model - losing 1-2% there is the difference of billions of dollars for them
- nailer 1y agoGoogle's a great tech organization but they generally don't create dominant tech products like they used to back in the Maps / Mail days (this is nearly two decades ago). Google wrote AKYNIA. OpenAI wrote ChatGPT.
- bhaktatejas922 1y agofactual
- conartist6 1y agoConventional tools are also a threat. Nothing about this problem is AI-specific.
- thegeomaster 1y agoIt does require a level of contextual awareness, fuzziness and robustness against crazy inputs that in my mind would be very hard to achieve using classical approaches.
- conartist6 1y agoContext awareness, yep. Response to fuzzy inputs... idunno, why does it need that? The thing I think is really silly is that it tries to make incremental writes to a flat file really fast, which is an impossible goal. As the file gets bigger your writes will just get slower and slower and slower at a rate that increases linearly with the size of the file.
- thegeomaster 1y agoLLMs will generate unpredictable, very humanlike code edits in this form. They might use comments like "same function as above", "rest of the function with similar changes", "function ABC is no longer needed", "function ABC same as above", etc. Your code edit model must resolve all of these, or flag an error in case of too much ambiguity. I would think a classical algorithm would have a lot of trouble differentiating between, for example, "function replaced with a comment because the LLM wanted to remove it" and "function replaced with a comment because the LLM wanted to keep it the same". The other half of your comment is true, but we typically have a ceiling on the reasonable size of a code file. For decades we've had the conventional wisdom to refactor files that are beyond some threshold LoC (be it 200, 500 or whatever it is.) If that is sufficiently fast, you can parallelize such operations and provide a maximum edit time regardless of the change size.
- conartist6 1y agoI agree that sorting something that messy can't easily be done with a heuristic expert system, but I'm registering my concern that the whole approach is predicated on first making a huge mess with one LLM then cleaning up the mess with another. The classical approach is more like "change the definition of the problem until you don't need to make a big mess in the first place"
- thegeomaster 1y agoThis might be a controversial take, but this approach is just plain old engineering applied to LLMs. Instead of making an LLM perform both an accurate code edit AND follow a strict output schema, you split that into two problems: accurate code edit with a lax output schema, then application of that lax output schema to the original file. You can then use different models for the two tasks, reducing your probability of failure.
- thegeomaster 1y agoYou can run more intelligent traditional LLMs at higher speeds than the Google diffusion model. Even then, it runs nowhere near 4500tok/s, and such small models generally suck in terms of accuracy compared to a specialized, fine tuned one.
- furyofantares 1y agoBig LLM and small LLM, very Starbucks sizing vibes here.
- icy 1y agoThat would be Grande LLM and Tall LLM.