6 ms·

I think VCS is far from solved, it hasn't even started yet. So far we only have distributed VCS for sequences (mostly text / source code) and sometimes things w

I think VCS is far from solved, it hasn't even started yet. So far we only have distributed VCS for sequences (mostly text / source code) and sometimes things which can be serialized unambiguously like trees and tables. But we don't have a general solution to distributed VCS. We currently can't use distributed VCS for: - DAGs - Arbitrary graphs (Topology) - 3D models (Geometry) - Mutli-dimensional sequences: Images, Videos, etc. - most domain specific problems / formats - ... almost everything else

tomc1985 6y ago
Store all those things as text and you can stick them in git
- strombofulous 6y ago
  You won't really get a meaningful diff from, for example, an image or video
- marcosdumay 6y ago
  People do not want to represent most of them as text. Some of them need high amounts of compression to be viable. Others don't have any concept of locality that one can map into a single dimension. For some, locality of changes is even an anti-feature.
  - npsimons 6y ago
    > People do not want to represent most of them as text. "People" don't care what things are stored as. You could make GUI tools save to GraphViz's text based format (or heck PlantUML[0]) for graphs at least and it would work just fine. For assets which have to be truly binary, Perforce offers versioning of those. I know it's not open source and costs money, but if it's really a need, it's worth it. [0] - https://plantuml.com/ https://plantuml.com/
- Lichtso 6y ago
  You can do that, but you will only get some of the features (a distributed backup system with an undo function). The most important one IMO is merging and that does not work on arbitrary formats serialized to text. E.g. let two 3D artists fork a blender 3D file in GIT, both do changes and commit them. Until here everything seems fine, now they want to merge their changes ... and suddenly everything falls apart. Edit: To clarify, if you don't need merging than you also don't need distributed VCS and centralized VCS will be sufficient. That is why I explicitly talked about distributed VCS.
  - npsimons 6y ago
    This sounds like an intractable problem, and putting it politely, the fault lies in the file format, not the VCS.
    - Lichtso 6y ago
      You are right, in that it is very hard. That is why I said that I think distributed VCS is far from being solved. However, I think the "fault" lies neither in the VCS nor file format. It simply shows that merging requires domain knowledge and that future VCS will need to cooperate with file formats to achieve that.
leobakerhytch 6y ago
To add to that list: abstract syntax trees. It would take a pretty monumental effort to build, but I think a huge step forward could be made with a VCS that has knowledge of code structure, and the relationships between source files. Large scale refactoring is such a fragile affair with git. Conceptually though, it should be possible to cleanly and concisely represent such operations as renaming functions, reordering arguments, widening type parameters, etc. Stability of the programming language in question is of course highly desirable, if you were to build such a system. The pace of change of Go would be much more conducive to success than that of JavaScript (or TypeScript).
- npsimons 6y ago
  > To add to that list: abstract syntax trees. Are you joking? Because you can represent these as Lisp sexps in plain text.
  - deleted 6y ago
    [deleted]
npsimons 6y ago
> - DAGs > - Arbitrary graphs (Topology) GraphViz (https://graphviz.org/ https://graphviz.org/) and PlantUML (https://plantuml.com/ https://plantuml.com/). > - 3D models (Geometry) My experience here is much more limited, but I'm certain there are text based versions of 3D model formats. The names of these formats just doesn't come immediately to mind, but having worked with antenna models, we'd stay with text representations whenever possible, and it worked just fine. > - Mutli-dimensional sequences: Images, Videos, etc. > - most domain specific problems / formats > - ... almost everything else This is where things get tricky, and again, I don't have experience in these domains. I can say, I've worked on projects where binary assets were tracked in Perforce. Not sure how efficient it was with diffs, but I know it worked, and probably better than git, which yes, doesn't know anything about binaries. Note: I am not a fan of Perforce, for other reasons, but I am not blind to the fact that it handles binary versioning better than git. Note2: I will also add, SVGs are just one image format that can be stored as text. NetPBM (https://en.wikipedia.org/wiki/Netpbm https://en.wikipedia.org/wiki/Netpbm) has also been around forever.
l0b0 6y ago
Realistically, there seems to be only two ways you could solve this: built-in VCS support in the software used to edit these files or exporting them to line-based text and dealing with them like we currently do. Building support for millions of formats (and their countless variants when created by different software and even versions of the same software) into the VCS system itself I strongly suspect would be futile. The VCS would end up much bigger and more complex than the OS.

tomc1985 6y ago

Store all those things as text and you can stick them in git

strombofulous 6y ago
You won't really get a meaningful diff from, for example, an image or video
marcosdumay 6y ago
People do not want to represent most of them as text. Some of them need high amounts of compression to be viable. Others don't have any concept of locality that one can map into a single dimension. For some, locality of changes is even an anti-feature.
- npsimons 6y ago
  > People do not want to represent most of them as text. "People" don't care what things are stored as. You could make GUI tools save to GraphViz's text based format (or heck PlantUML[0]) for graphs at least and it would work just fine. For assets which have to be truly binary, Perforce offers versioning of those. I know it's not open source and costs money, but if it's really a need, it's worth it. [0] - https://plantuml.com/ https://plantuml.com/
Lichtso 6y ago
You can do that, but you will only get some of the features (a distributed backup system with an undo function). The most important one IMO is merging and that does not work on arbitrary formats serialized to text. E.g. let two 3D artists fork a blender 3D file in GIT, both do changes and commit them. Until here everything seems fine, now they want to merge their changes ... and suddenly everything falls apart. Edit: To clarify, if you don't need merging than you also don't need distributed VCS and centralized VCS will be sufficient. That is why I explicitly talked about distributed VCS.
- npsimons 6y ago
  This sounds like an intractable problem, and putting it politely, the fault lies in the file format, not the VCS.
  - Lichtso 6y ago
    You are right, in that it is very hard. That is why I said that I think distributed VCS is far from being solved. However, I think the "fault" lies neither in the VCS nor file format. It simply shows that merging requires domain knowledge and that future VCS will need to cooperate with file formats to achieve that.

leobakerhytch 6y ago

To add to that list: abstract syntax trees. It would take a pretty monumental effort to build, but I think a huge step forward could be made with a VCS that has knowledge of code structure, and the relationships between source files. Large scale refactoring is such a fragile affair with git. Conceptually though, it should be possible to cleanly and concisely represent such operations as renaming functions, reordering arguments, widening type parameters, etc. Stability of the programming language in question is of course highly desirable, if you were to build such a system. The pace of change of Go would be much more conducive to success than that of JavaScript (or TypeScript).

npsimons 6y ago
> To add to that list: abstract syntax trees. Are you joking? Because you can represent these as Lisp sexps in plain text.
- deleted 6y ago
  [deleted]

npsimons 6y ago

> - DAGs > - Arbitrary graphs (Topology) GraphViz (https://graphviz.org/ https://graphviz.org/) and PlantUML (https://plantuml.com/ https://plantuml.com/). > - 3D models (Geometry) My experience here is much more limited, but I'm certain there are text based versions of 3D model formats. The names of these formats just doesn't come immediately to mind, but having worked with antenna models, we'd stay with text representations whenever possible, and it worked just fine. > - Mutli-dimensional sequences: Images, Videos, etc. > - most domain specific problems / formats > - ... almost everything else This is where things get tricky, and again, I don't have experience in these domains. I can say, I've worked on projects where binary assets were tracked in Perforce. Not sure how efficient it was with diffs, but I know it worked, and probably better than git, which yes, doesn't know anything about binaries. Note: I am not a fan of Perforce, for other reasons, but I am not blind to the fact that it handles binary versioning better than git. Note2: I will also add, SVGs are just one image format that can be stored as text. NetPBM (https://en.wikipedia.org/wiki/Netpbm https://en.wikipedia.org/wiki/Netpbm) has also been around forever.

l0b0 6y ago

Realistically, there seems to be only two ways you could solve this: built-in VCS support in the software used to edit these files or exporting them to line-based text and dealing with them like we currently do. Building support for millions of formats (and their countless variants when created by different software and even versions of the same software) into the VCS system itself I strongly suspect would be futile. The VCS would end up much bigger and more complex than the OS.