4 ms·
in the era of LLMs, syntax might matter more than you think. The c form of `type name;` is ambiguous because it could actually be more than one thing depending
by dnautics 8mo ago
in the era of LLMs, syntax might matter more than you think.
The c form of `type name;` is ambiguous because it could actually be more than one thing depending on context. Even worse if you include macro sheananigans. The alternate (~rust/zig) is `var/const/mut name type` is unambiguous.
For humans, with rather long memory of what is going on in the codebase, this is ~"not a problem" for experts. But for an LLM, its knowledge is limited to the content that currently exists in your context, and conventions baked in with the training corpus, this matters. Of course it is ALSO a problem for humans if they are first looking at a codebase, and if the types are unusual.
- wavemode 8mo agoI hope that someday LLMs will interact with code mostly via language servers, rather than reading the code itself (which both frequently confuses the LLM, as you've noted, but is also simply a waste of tokens).
- dnautics 8mo agowhy? I suspect that writing code itself is extremely token efficient (unless like your keywords happen to be silly, super-long alien text). Like which do you think is more token-efficient? 1) <tool-call write_code "my_function(my_variable)"/> 2) <tool-call available_functions/> resp: <option> my_function </option> <option> your_function </option> <option> some_other_function </option> <option> kernel_function1 </option> <option> kernel_function2 </option> <option> imported_function1 </option> <option> imported_function2 </option> <option> ... </option> <tool-call write_function_call "my_function"/> resp: <option> my_variable </option> <option> other_variable_of_same_type </option> <tool-call write_variable "my_variable"/>
- wavemode 8mo agoNot sure I follow. You seem to have omitted the part of 1) explaining how the LLM knew that my_function even existed - presumably, it read the entire file to discover that, which is way more input tokens than your hypothetical available_functions response.
- dnautics 8mo agoReading files is not that input tokien heavy, I suspect. But anyways I omitted it because presumably it would have done so to gain local context in general.
- wavemode 8mo agoI still don't follow. Why doesn't that count as cost?
- dnautics 8mo agoA+B > A+C => B > C
- wavemode 8mo agoThe LLM doesn't need to read the whole file in example 2). Why would it?
- dnautics 8mo agoWhere would it inject the code?
- wavemode 8mo agoPresumably, the same MCP tool allowing it to read specific definitions would also allow it to write out specific definitions (rather than, again, having to write out the whole file just to make one change). Otherwise it defeats the purpose. This is the same way humans work on code. You never read an entire file top to bottom just to find out one thing, and you never type out the entire file just to make one change. This would, understandably, be very taxing on your working memory, and you'd probably make more mistakes.
- dnautics 8mo ago
- 0x457 8mo agoLSP is meant for IDEs and very deterministic calls. Its APIs are like this: give me a definition of <file> <row> <column> <lenght>. This makes sense for IDEs because all of those can be deterministically captures based of your cursor position. LLMs are notoriously bad at counting.
- wavemode 8mo agoI think one could easily build an MCP tool wrapping LSP which smooths over those difficulties. What the LLM needs is just a structured way to say "perform this code change" and a structured way to ask things like "what's the definition of this function?" or "what functions are defined in this module?" Not much different from what agents already do today inside of their harnesses, just without the part where they have to read entire files to find the definition of one thing.
- 0x457 8mo agoSo not using LSP, but rather using something in a middle that uses LSP as implementation detail. So far enabling LSP in Claude only added messages like "this is old diagnostic before my edit".
- cranberryturkey 8mo ago[dead]
- vidarh 8mo agoFine tuning is likely a bigger part of it. I've worked on fine tuning projects. There's a massive bias towards fone tuning for Python at several model providers for example, followed by JS.
- dheera 8mo agoHumans also have limited context. For LLMs it's mostly a question of pipeline engineering to pack the context and system prompt with the most relevant information, and allow tool use to properly understand the rest of the codebase. If done well I think they shouldn't have this particular issue. Current AI coding tools are mostly huge amounts of this pipeline innovation.
- dnautics 8mo agoI think we need a LLM equivalent of this part's of fitt's law: The fastest place to click under a cursor is the location of the cursor. For an LLM the least context-expensive feedback is no feedback at all, the LLM should be able to intuit the correct code in-place, at token generation.
- zahlman 8mo agoI suspect that the context-dependence in C is more an issue with the implementation than the overall syntactic philosophy.
- dnautics 8mo agoIt's a thing in c. foo * bar; Is that multiplication? Or a declaration of type foo*?
- zahlman 8mo agoRight; and my argument is that this isn't because the type expression `foo *` precedes the name `bar`; it's because the type "pointer to foo" is expressed in a way that could also be a prefix of a multiplication expression.
- dnautics 8mo agoOk fine but never here have I not been talking specifically about C
- teo_zero 8mo agoYes, it's a pity that some symbols have been overloaded with multiple meanings and the syntax can't disambiguate them. This case would not exist if another symbol had been chosen for dereference. A similar claim, equally not related to type declarations, is with & that serves two roles as a boolean operator and address-of, that makes (foo)&bar ambiguous. The list of C symbols that have been badly chosen at the beginning and now it's too late to correct continues with && for logical and, that prevents masking an address-of without adding spaces or parentheses in ``mask & &var``, /* for comments, that prevents dividing by a dereferenced value without adding spaces or parentheses in ``*a / *b``, the pre-increment ``--`` that is indistinguishable from a double application of the unary minus, and the original =+ etc. that have been later changed to += etc.