3 ms·

It could be used for coding if you gave it an AST. If you work at TypeSafe please try this. Side note: This is probably how LLMs would perform with better enc

It could be used for coding if you gave it an AST. If you work at TypeSafe please try this. Side note: This is probably how LLMs would perform with better encoders and next-latent prediction, so eventually those will beat this architecture out. Still amazing though.

ramon156 11d ago
I've implemented tree-sitter in pi before, and while it works, I have no real proof it saves me tokens, or is more accurate. I think a better implementation is a model that's trained for AST's, not just "use tool, see what happens". I'd love to do research on this when I have the time.
- vatsachak 11d ago
  Cool project! That's what I was insinuating through "better encoder"; the model creating more efficient representations of ASTs using something like JEPA
Escapado 11d ago
I saw the CEO reply elsewhere in the comments to some other question. Maybe he can shed some light on it. My gut feeling is that this is non-trivial and they did not get this to work (yet?), otherwise I can’t come up with a good reason as to why they would not demo that as I assume half of the crowd here (myself included) would line up as customers.
- vatsachak 11d ago
  Yeah it would be quite trivial to try and implement an auto regressive AST generator for STLC with Jev provided that you had bounded variable names and integers. As you said, if it worked, they would have demoed it haha
CompleteSkeptic 11d ago
the hard part for coding is actually state engineering (e.g. getting your dependencies in context) - we haven't even tried it yet (because my philosophy is we should automate the easy tasks before the hard and we've been working on getting the model smart on the former) we do think there's a lot of potential though and do want coding themed releases soon
- hunterbrooks 11d ago
  I could see Jev being great at finding key symbols in codebase before a code generation/code review task. I sent you guys an email (to hello@) about using Jev in Code Review for www.ellipsis.dev.
8note 10d ago
im not seeing it. youd ask it to pick a location on the ast to add something from the grammar? i dont see how this stays confined well enough? make a new output space every time? does that end up auto-regressive?

ramon156 11d ago

I've implemented tree-sitter in pi before, and while it works, I have no real proof it saves me tokens, or is more accurate. I think a better implementation is a model that's trained for AST's, not just "use tool, see what happens". I'd love to do research on this when I have the time.

vatsachak 11d ago
Cool project! That's what I was insinuating through "better encoder"; the model creating more efficient representations of ASTs using something like JEPA

Escapado 11d ago

I saw the CEO reply elsewhere in the comments to some other question. Maybe he can shed some light on it. My gut feeling is that this is non-trivial and they did not get this to work (yet?), otherwise I can’t come up with a good reason as to why they would not demo that as I assume half of the crowd here (myself included) would line up as customers.

vatsachak 11d ago
Yeah it would be quite trivial to try and implement an auto regressive AST generator for STLC with Jev provided that you had bounded variable names and integers. As you said, if it worked, they would have demoed it haha

CompleteSkeptic 11d ago

the hard part for coding is actually state engineering (e.g. getting your dependencies in context) - we haven't even tried it yet (because my philosophy is we should automate the easy tasks before the hard and we've been working on getting the model smart on the former) we do think there's a lot of potential though and do want coding themed releases soon

hunterbrooks 11d ago
I could see Jev being great at finding key symbols in codebase before a code generation/code review task. I sent you guys an email (to hello@) about using Jev in Code Review for www.ellipsis.dev.

8note 10d ago

im not seeing it. youd ask it to pick a location on the ast to add something from the grammar? i dont see how this stays confined well enough? make a new output space every time? does that end up auto-regressive?