4 ms·

How can your tools all accept the same kind of structured data without dictating its representation? I don't get it.

jcmhn 14y ago
Allow a shell variable to control output record and field separators. Default to space and newline if nothing is specified. ORS=: OFS=, ls -l # rw-r--r--:1:uname:gname:1025:Jun 1:somefile.txt,...,... Add a specifier for dates, usernames, groupnames, etc. DSF="%Y-%m-%d" ls -l
haberman 14y ago
When I talk about a "representation," I mean an in-memory format. For example, the "representation" of an HTML tree is the DOM. Yes, you have to agree on a serialization format (JSON, Protocol Buffers, etc), but that's not the same thing. From a serialization format you can represent the data however you see fit in your process. For example, a C++ user might represent a string as a std::string object whereas a Python user would represent it as a native Python string. The VM-based approach (like PowerShell) defines an in-memory tree representation, namely .NET objects. This means that you can't really interoperate with this stack unless you use .NET too, since you don't have an easy way of converting .NET objects to your own objects.
- stcredzero 14y ago
  Yes, but you could also just use IronPython and script in Python.
  - haberman 14y ago
    The more decisions you force on your users, the more reasons you give them to choose some other technology, and the less future-proof you are. For example, .NET is only really usable on Windows.
- akkartik 14y ago
  Just a nit: I think protocol buffers include representations, not just serialization formats. You need to have the schema of the proto to parse it correctly, know which fields are required, repeated, etc. Am I understanding you correctly?
  - Evbn 14y ago
    Schema isn't representation the way haberman is using it. He means something like implementation, which protobuf has in many languages in various VMs and can be easily ported to more.
  - haberman 14y ago
    It's true that many Protocol Buffer libraries include representations, but these are for convenience; Protocol Buffers are defined in terms of their serialization format and schema.
    - akkartik 14y ago
      I don't follow. Isn't schema the same as representation? It's the equivalent of the DTD for an XML document. To be super concrete, I can't read a file containing protos without knowing their type, what fields they contain, etc. I can however read JSON just fine without knowing the precise schema being encoded.

jcmhn 14y ago

Allow a shell variable to control output record and field separators. Default to space and newline if nothing is specified. ORS=: OFS=, ls -l # rw-r--r--:1:uname:gname:1025:Jun 1:somefile.txt,...,... Add a specifier for dates, usernames, groupnames, etc. DSF="%Y-%m-%d" ls -l

haberman 14y ago

When I talk about a "representation," I mean an in-memory format. For example, the "representation" of an HTML tree is the DOM. Yes, you have to agree on a serialization format (JSON, Protocol Buffers, etc), but that's not the same thing. From a serialization format you can represent the data however you see fit in your process. For example, a C++ user might represent a string as a std::string object whereas a Python user would represent it as a native Python string. The VM-based approach (like PowerShell) defines an in-memory tree representation, namely .NET objects. This means that you can't really interoperate with this stack unless you use .NET too, since you don't have an easy way of converting .NET objects to your own objects.

stcredzero 14y ago
Yes, but you could also just use IronPython and script in Python.
- haberman 14y ago
  The more decisions you force on your users, the more reasons you give them to choose some other technology, and the less future-proof you are. For example, .NET is only really usable on Windows.
akkartik 14y ago
Just a nit: I think protocol buffers include representations, not just serialization formats. You need to have the schema of the proto to parse it correctly, know which fields are required, repeated, etc. Am I understanding you correctly?
- Evbn 14y ago
  Schema isn't representation the way haberman is using it. He means something like implementation, which protobuf has in many languages in various VMs and can be easily ported to more.
- haberman 14y ago
  It's true that many Protocol Buffer libraries include representations, but these are for convenience; Protocol Buffers are defined in terms of their serialization format and schema.
  - akkartik 14y ago
    I don't follow. Isn't schema the same as representation? It's the equivalent of the DTD for an XML document. To be super concrete, I can't read a file containing protos without knowing their type, what fields they contain, etc. I can however read JSON just fine without knowing the precise schema being encoded.