3 ms·

Wait, Isn't the appeal of a stream of bytes that the OS is agnostic to whatever the application layers above are doing with it? That way, you can separate conce

by iamwil 7y ago

Wait, Isn't the appeal of a stream of bytes that the OS is agnostic to whatever the application layers above are doing with it? That way, you can separate concerns and evolve each layer independently. It's probably the reason the OS has had such a long staying power of decades. It seems to be the same strategy of IP (internet protocol)--that it's a narrow waist architecture--so the layers above and below it can evolve independently. How would versioning of the data structures work? Would it be append only? I'm pretty sure part of the reason abstractions are hard is because we don't know exactly how to model the domain the first couple of times around. So if these data structures in the OS aren't right the first time, we end up lugging it around and it'd be more expensive than the parsing that we currently do. When I look at the EDI spec, it just has a lot of cruft I don't need. And lastly, I'm afraid when we invoke meaning in a stream of bytes, different groups with different interests will have different ontologies they want to enforce. We see these in format wars and internet working groups. I'm not sure we can easily agree what these data structures are, and how to evolve them. Maybe there's a good way and I just haven't seen it.

pdimitar 7y ago
> Wait, Isn't the appeal of a stream of bytes that the OS is agnostic to whatever the application layers above are doing with it? That way, you can separate concerns and evolve each layer independently. Dude, we all know the supposed theoretical ideal state of things. And yet it almost never happens. > It's probably the reason the OS has had such a long staying power of decades. Or the fact that without an OS your hardware is a simple paperweight? Not sure what is your point here, can you clarify? > It seems to be the same strategy of IP (internet protocol)--that it's a narrow waist architecture--so the layers above and below it can evolve independently. We don't have that much choice there either. Networks have to be streams of bytes due to the physics and realities of our network hardware -- which aren't likely to change anytime soon. So I don't think yours is a transferable analogy. > How would versioning of the data structures work? Would it be append only? Yes. That's how FlatBuffers work and every single team I talked with are grateful that they moved to it and no longer use JSON. Has to mean something. (FlatBuffers also allows you deprecate and thus ignore parts of the structure you are transmiting. But indeed and as you alluded to, anything new can only be appended at the end of the byte stream.) > I'm pretty sure part of the reason abstractions are hard is because we don't know exactly how to model the domain the first couple of times around. Sure, agreed. But as an area IT already has a lot of experience and can tap into some past lessons already. Configuration: zero programming language constructs inside it, please! You need config with programming? Here's that small subset of LISP or Lua, you can only use that. I see nothing wrong with an approach like that. People will grumble a lot and will adapt to it as they always do. It's not like Ansible's YAML programming language is better, no? > And lastly, I'm afraid when we invoke meaning in a stream of bytes, different groups with different interests will have different ontologies they want to enforce. We see these in format wars and internet working groups. I'm not sure we can easily agree what these data structures are, and how to evolve them. Maybe there's a good way and I just haven't seen it. I also haven't seen it and I am sure most people don't as well but it doesn't mean we shouldn't try. Stuff like RDF were pretty good for describing many types of data, for example. And if somebody needs something drastically different, well, they shouldn't use that format. --- IMO the author's point isn't that tomorrow we can have the one ultimate programming language and data format to rule them all -- of course not. Their point is that nobody is even trying. And we actually have a lot of low-hanging fruit, the "everything is a string on the shell" being one of the best examples.
- iamwil 7y ago
  I'm not against trying. I'm just saying, ok, if we're going to try it, these are the things that immediately come to mind as hurdles to get over. I wish they addressed those hurdles in the post (or maybe in a later post).

by iamwil 7y ago

pdimitar 7y ago

> Wait, Isn't the appeal of a stream of bytes that the OS is agnostic to whatever the application layers above are doing with it? That way, you can separate concerns and evolve each layer independently. Dude, we all know the supposed theoretical ideal state of things. And yet it almost never happens. > It's probably the reason the OS has had such a long staying power of decades. Or the fact that without an OS your hardware is a simple paperweight? Not sure what is your point here, can you clarify? > It seems to be the same strategy of IP (internet protocol)--that it's a narrow waist architecture--so the layers above and below it can evolve independently. We don't have that much choice there either. Networks have to be streams of bytes due to the physics and realities of our network hardware -- which aren't likely to change anytime soon. So I don't think yours is a transferable analogy. > How would versioning of the data structures work? Would it be append only? Yes. That's how FlatBuffers work and every single team I talked with are grateful that they moved to it and no longer use JSON. Has to mean something. (FlatBuffers also allows you deprecate and thus ignore parts of the structure you are transmiting. But indeed and as you alluded to, anything new can only be appended at the end of the byte stream.) > I'm pretty sure part of the reason abstractions are hard is because we don't know exactly how to model the domain the first couple of times around. Sure, agreed. But as an area IT already has a lot of experience and can tap into some past lessons already. Configuration: zero programming language constructs inside it, please! You need config with programming? Here's that small subset of LISP or Lua, you can only use that. I see nothing wrong with an approach like that. People will grumble a lot and will adapt to it as they always do. It's not like Ansible's YAML programming language is better, no? > And lastly, I'm afraid when we invoke meaning in a stream of bytes, different groups with different interests will have different ontologies they want to enforce. We see these in format wars and internet working groups. I'm not sure we can easily agree what these data structures are, and how to evolve them. Maybe there's a good way and I just haven't seen it. I also haven't seen it and I am sure most people don't as well but it doesn't mean we shouldn't try. Stuff like RDF were pretty good for describing many types of data, for example. And if somebody needs something drastically different, well, they shouldn't use that format. --- IMO the author's point isn't that tomorrow we can have the one ultimate programming language and data format to rule them all -- of course not. Their point is that nobody is even trying. And we actually have a lot of low-hanging fruit, the "everything is a string on the shell" being one of the best examples.

iamwil 7y ago
I'm not against trying. I'm just saying, ok, if we're going to try it, these are the things that immediately come to mind as hurdles to get over. I wish they addressed those hurdles in the post (or maybe in a later post).