4 ms·

We should fix the diff tools too while we are at it. Why are we diffing by lines and not doing a semantic diff of the code? If the compiler doesn't care about i

by dalore 13y ago

We should fix the diff tools too while we are at it. Why are we diffing by lines and not doing a semantic diff of the code? If the compiler doesn't care about it, why should the diff?

spenczar5 13y ago
Not all languages are compiled - sometimes line breaks are meaningful, and it would be good if diff tools work across all languages.
RogerL 13y ago
Scaling. The same diff tool works for text files, latex and other markup files, for any computer language, and so on. I can write downstream tools that take the output of the diff and do further processing, without worrying about what the output might look like for brainfuck. And how long would we have to wait for Torvalds to add a C++ or Java extension to git? :)
rlpb 13y ago
You essentially want to replace Unix pipelines with an API then. Nobody else has managed to do this successfully, completely replacing text.
- Chris_Newton 13y ago
  Isn’t this the heart of the problem, though? Raw text is a limited medium, and working at the level of plain text analysis rather than semantic analysis does limit how much our tools can achieve. The UNIX philosophy of having many small, text-based tools and chaining them together represents a common platform, but it’s also a least common denominator platform. As long as we allow that glass ceiling to remain, the best our tools will ever do is push it incrementally higher, one tiny step at a time. If we want to make big leaps, we’re going to have to sacrifice some of that generality so we can use more powerful but specific tools. Unfortunately, that means that any new programming language wanting to take on the established standards needs not only a compiler/interpreter, but also the rest of the ecosystem: a comprehensive tool chain, ample library coverage, documentation and training resources, and so on. It is also, almost inevitably, going to need some standardised way of bridging the gap to today’s established languages for interoperability and backward compatibility purposes. This is why I think the view that programming languages should be designed optimally for humans is short-sighted, and I suspect most of the big success stories in the coming years will be languages that were designed with clean semantics and easy parsing in mind. Those languages will better support building that surrounding ecosystem, and a good language with a good ecosystem is more practically useful than a slightly better language with a limited ecosystem.
  - rlpb 13y ago
    > working at the level of plain text analysis rather than semantic analysis does limit how much our tools can achieve. We can add semantic analysis. I have no problem with that. But removing or harming the ability to do text analysis might not stop us from using more sophisticated tools, but it does stop us from putting tools together quickly in a way not previously predicted. This is why Unix is so powerful. If you haven't read The Art of Unix Programming, I urge you to do so. It puts the argument and examples forward far more convincingly than I can. Would moving to semantic tools harm the ability to use our existing text-based tools? I'd say so. A simple tool like diff works better, for example, if you take your big list of Python imports and put each one on a separate line, and keep them sorted. Patching works smoother this way too, reducing the likelyhood of the need for manual conflict resolution. If we eliminate doing this kind of arrangement by hand, and instead start relying on a semantic editor, we'll lose this ability. I have yet to see a tool that does semantic diffs, patches and merges better than diff, patch and git do.
    - Chris_Newton 13y ago
      But removing or harming the ability to do text analysis [...] does stop us from putting tools together quickly in a way not previously predicted. Well, OK, but we’ve been using these text-based tools for a year or two now, and I don’t see many radical advances taking place in how we use or combine them. Are you sure you’re not chasing an illusion here? The freeform text-based tools represent a great deal of flexibility, to be sure, but they were conceived at a time when flexible text manipulation was about as much as one could hope for. Today, we can do more. I have yet to see a tool that does semantic diffs, patches and merges better than diff, patch and git do. As long as everything is limited to manipulation of freeform text files, perhaps you never will. That doesn’t mean better tools aren’t possible; it just means they aren’t possible within the constraints you’re choosing to impose.
      - rlpb 13y ago
        > Well, OK, but we’ve been using these text-based tools for a year or two now, and I don’t see many radical advances taking place in how we use or combine them. Are you sure you’re not chasing an illusion here? I'm not claiming recent advances. I'm claiming existing power that has been around for decades, which we would lose if we compromised the text tooling available today. Have you read TAOUP? Do you understand the extent of the power that existing text tooling gives us today? Are you experienced in the advanced use of the existing tools, so you are able to make comparisons about their power? > As long as everything is limited to manipulation of freeform text files, perhaps you never will. That doesn’t mean better tools aren’t possible; it just means they aren’t possible within the constraints you’re choosing to impose. This is backwards. We move forward when people show how it can be done. Please show us how we can improve diffs, patches and merges by moving to semantic data structures over text, without compromising any existing capabilities. Even just illustrating specifics of how these tools might work, rather than implementing them, will do something for your argument. The onus is on you.

by dalore 13y ago

We should fix the diff tools too while we are at it. Why are we diffing by lines and not doing a semantic diff of the code? If the compiler doesn't care about it, why should the diff?

spenczar5 13y ago

Not all languages are compiled - sometimes line breaks are meaningful, and it would be good if diff tools work across all languages.

RogerL 13y ago

Scaling. The same diff tool works for text files, latex and other markup files, for any computer language, and so on. I can write downstream tools that take the output of the diff and do further processing, without worrying about what the output might look like for brainfuck. And how long would we have to wait for Torvalds to add a C++ or Java extension to git? :)

rlpb 13y ago

You essentially want to replace Unix pipelines with an API then. Nobody else has managed to do this successfully, completely replacing text.

Chris_Newton 13y ago
Isn’t this the heart of the problem, though? Raw text is a limited medium, and working at the level of plain text analysis rather than semantic analysis does limit how much our tools can achieve. The UNIX philosophy of having many small, text-based tools and chaining them together represents a common platform, but it’s also a least common denominator platform. As long as we allow that glass ceiling to remain, the best our tools will ever do is push it incrementally higher, one tiny step at a time. If we want to make big leaps, we’re going to have to sacrifice some of that generality so we can use more powerful but specific tools. Unfortunately, that means that any new programming language wanting to take on the established standards needs not only a compiler/interpreter, but also the rest of the ecosystem: a comprehensive tool chain, ample library coverage, documentation and training resources, and so on. It is also, almost inevitably, going to need some standardised way of bridging the gap to today’s established languages for interoperability and backward compatibility purposes. This is why I think the view that programming languages should be designed optimally for humans is short-sighted, and I suspect most of the big success stories in the coming years will be languages that were designed with clean semantics and easy parsing in mind. Those languages will better support building that surrounding ecosystem, and a good language with a good ecosystem is more practically useful than a slightly better language with a limited ecosystem.
- rlpb 13y ago
  > working at the level of plain text analysis rather than semantic analysis does limit how much our tools can achieve. We can add semantic analysis. I have no problem with that. But removing or harming the ability to do text analysis might not stop us from using more sophisticated tools, but it does stop us from putting tools together quickly in a way not previously predicted. This is why Unix is so powerful. If you haven't read The Art of Unix Programming, I urge you to do so. It puts the argument and examples forward far more convincingly than I can. Would moving to semantic tools harm the ability to use our existing text-based tools? I'd say so. A simple tool like diff works better, for example, if you take your big list of Python imports and put each one on a separate line, and keep them sorted. Patching works smoother this way too, reducing the likelyhood of the need for manual conflict resolution. If we eliminate doing this kind of arrangement by hand, and instead start relying on a semantic editor, we'll lose this ability. I have yet to see a tool that does semantic diffs, patches and merges better than diff, patch and git do.
  - Chris_Newton 13y ago
    But removing or harming the ability to do text analysis [...] does stop us from putting tools together quickly in a way not previously predicted. Well, OK, but we’ve been using these text-based tools for a year or two now, and I don’t see many radical advances taking place in how we use or combine them. Are you sure you’re not chasing an illusion here? The freeform text-based tools represent a great deal of flexibility, to be sure, but they were conceived at a time when flexible text manipulation was about as much as one could hope for. Today, we can do more. I have yet to see a tool that does semantic diffs, patches and merges better than diff, patch and git do. As long as everything is limited to manipulation of freeform text files, perhaps you never will. That doesn’t mean better tools aren’t possible; it just means they aren’t possible within the constraints you’re choosing to impose.
    - rlpb 13y ago
      > Well, OK, but we’ve been using these text-based tools for a year or two now, and I don’t see many radical advances taking place in how we use or combine them. Are you sure you’re not chasing an illusion here? I'm not claiming recent advances. I'm claiming existing power that has been around for decades, which we would lose if we compromised the text tooling available today. Have you read TAOUP? Do you understand the extent of the power that existing text tooling gives us today? Are you experienced in the advanced use of the existing tools, so you are able to make comparisons about their power? > As long as everything is limited to manipulation of freeform text files, perhaps you never will. That doesn’t mean better tools aren’t possible; it just means they aren’t possible within the constraints you’re choosing to impose. This is backwards. We move forward when people show how it can be done. Please show us how we can improve diffs, patches and merges by moving to semantic data structures over text, without compromising any existing capabilities. Even just illustrating specifics of how these tools might work, rather than implementing them, will do something for your argument. The onus is on you.