4 ms·

Interesting premise. Can't read the entire paper right now but going on the abstract alone I'm not able to think of an illuminating example of the need for meta

by causal 2y ago

Interesting premise. Can't read the entire paper right now but going on the abstract alone I'm not able to think of an illuminating example of the need for metaclass connectors

Joker_vD 2y ago
I think any of those openapi.yaml files in the wild is a good example that we need better metaclass connectors.
- jauntywundrkind 2y ago
  What are you asserting? Why?
knome 2y ago
Unless there are two languages called Objective-S, here's a site for the language: https://objective.st/ https://objective.st/ I'll have to look closer at this later, but it doesn't give me a good feeling on first skim.
- mpweiher 2y ago
  That's the one. Alas, the website precedes my so far best attempt at explaining what I've done, which is this paper, and I haven't had time to properly update the site since. And probably won't for some time: first give the presentation at Onward! in Pasadena next Wednesday, then finish writing the PhD thesis, then update the site. It is what it is ¯\_(ツ)_/¯
  - carapace 2y ago
    I think you're on to something. My only advice would be something like, don't neglect the debugging, uh, "stories".
    - mpweiher 2y ago
      Do you mean the stories of debugging the steaming pile of ... I mean my wonderful implementation of the language? Or how to debug these kinds of system? For the latter, I am actually very optimistic about making it possible to debug systems at a much higher and more meaningful level than currently, because you are making the high-level structure visible in the code instead of hand-compiling it down to some procedural implementation, with that hand-compiled procedural representation of your system being the thing that the the debugger sees. With both the polymorphic write streams and the storage combinators being so highly composable, it becomes fairly easy to insert logging/instrumentation/debugging elements between the components of your system.
      - carapace 2y ago
        The latter. ;) Sounds good.
jerf 2y ago
It isn't so much that there's no need for such things but that they are so pervasive and normal you can't hardly even "see" them. This is related to what killed the original OO conception as "your OO program should match your physical model", so, you have an Engine class that is directly related to the Drivetrain class (ignore "inheritance" versus "composition" and all such questions for this post), which is directly related to the Axle class, which is related to the directly Wheel class... what kills this model is that in practice you need stuff in between them. This model of OO is almost useless for anything except its original specification for simulations. Real-world OO however continues alive and well precisely in these "metaclass connectors", like, iterators, network call managers, and even the dreaded SomethingFactoryFactory. You need a WheelIterator and an CarVisitor and all kinds of other structural classes that are super useful to programmers but have no physical interpretation in OO, even if you are dealing with physical objects at all, which you often aren't. The paper points out that if we sometimes communicate over HTTP, and sometimes over a "Remote Procedure Call", and sometimes over some pipe or messaging system, to which I'd also add, sometimes via database communications or passing text files around and lots of other things, it could/would be helpful if our languages could interact with these things in ways other than procedure calls, which have a very particular abstraction they provide that is a poor match to these sorts of interactions [1]. On my personal list of "things I'd like to see new languages try" one of them is to make the default function call less capable, so that the language might more pervasively deal with "this function call failed" in a way modern languages can't very well... not, "this function ran and returned an error/exception" but this function call itself failed; couldn't find the code, couldn't reach the server, etc. Things like, every "function call" in the language being able to set a timeout rather than only certain special things. The purpose of this approach is for the foreignness of these various other methodologies to be something that no longer breaks function calls so thoroughly and causes these remote calls to be treated specially everywhere, where you have to specially wrap them in timeouts, specially handle a lot of failures you don't have to handle with normal functions, etc. As a side effect the language would potentially be more able to swap in "remote calls" for local calls more transparently than in modern languages, where local vs. remote calls act a lot like a "color" as referenced in the sync/async conversation, except it's even worse. Sync code wants to call sync code and async code wants to call async code, and if you need to cross the barrier you need an expensive and potentially lossy conversion process, but in this case, local code wants to call local code but there is no "remote code mode", the expensive and potentially lossy conversion is mandatory. Even when you stuff all that conversion code behind a single function call and sweep all the complexity somewhere where you can't see it at the call site, it's still there... it's just hidden better in some languages and frameworks and less well in others, and the question of the exact level of exposure that is best is one that can be debated for quite a while (it is definitely not clear that you should completely sweep away the differences even if you can, and there is a history of techs that take the sweeping too far [1]. (Yes, the same link twice.)) This paper and the accompanying language appears to take the opposite approach to what I was proposing, which was to loosen what a "function call" was until later abstractions could potentially harmonize the various approaches, and instead build an all-singing, all-dancing "connector" concept that envelops and abstracts all the various approaches to foreign communication, thus providing much more machinery than a normal function call. I consider it an interesting idea worth exploring, however, as someone who has worked with a lot of these "connectors" over the years (as many of us have, I am not claiming anything unique here), I observe that they have wildly varying characteristics and I question the ability of an all-singing, all-dancing abstraction to accommodate all or even a good subset of the possible options without becoming a poor lowest-common-denominator of all of them. For instance, even within two members of the same class, a message bus that is 0-or-1 has significantly different characteristics than 1-or-many delivery, and by "significantly" I don't mean the normal casual definition of "larger than average", but in this case, relevant to my design. It doesn't take much scale before any attempt to abstract around that isn't going to work. I've got cases where I speak to HTTP services and I may or may not add a client certificate and may or may not apply certain checks to my server's certificate, depending on how I route over the network to this service. It's hard to abstract that. And abstracting "this code may connect with either an HTTP JSON API or a transactional message bus", with all the accompanying differences in how much context the answer comes back with. HTTP has an associated socket, so even if you're "async" or even if you do something exotic like make the request and hand off the reply to a completely different thread, it's got much more context than a message coming in over the bus that initially is just "some kind of message" to your system and it has to laboriously work it down into a reply to what, how do I route that reply, etc. which is all much more complicated than the HTTP request, to say nothing of, what if I want want to send the message from one thread, but I want the reply to not only come back to that thread but also go somewhere else in my program, like a high-level metric tracking how many of each kind of reply I get, that is not inline with the original request/response cycle? I do not question that a lowest-common-denominator library could be constructed. Such things have been done. I would question whether it could meet real-world needs. For that someone would have to study their language directly for more time than I have for an HN post, because to really know if they succeeded you'd need to do more than just read the docs, but actually start building real programs and seeing how they do in the field. (Although reading the docs could be enough to tell they have not succeeded.) If they do succeed, then I give them significant props because I have just explained why I think it's hard. I have learned to be very skeptical of this sort of all-encompassing abstraction over things that in reality have such huge and significant differences between them despite superficial similarities that in the end I end up questioning if there was any "true" commonality to abstract over in the first place. I have been burned both in the using of such systems, and the occasional error I've made trying to build them. The ones that do work take massive, sustained usage from a large user base over years of development. While it may not be a connector in the sense this paper means, an ORM that tries to go beyond a simple table mapping but to expose the underlying capabilities of your database, for multiple databases, is an example of this. Ignoring the "are ORMs even good at all?" for a moment, as that is well-trod ground, the ones that are good and usable and functional are huge and take years upon years of bashing out in the real world before they get there. I would suspect that if something like what is described in this paper was going to get to the point that even "medium sized" cloud usage (to say nothing of Facebook or Google) could consider it a reliable technology would be a similarly sized task... the initial cut of the library would seem superficially appealing and seem like they're 90% of the way there, but upon examination it would turn out to be more like 10%. The real world is fairly harsh. [1]: https://news.ycombinator.com/item?id=30831309 https://news.ycombinator.com/item?id=30831309

by causal 2y ago

Interesting premise. Can't read the entire paper right now but going on the abstract alone I'm not able to think of an illuminating example of the need for metaclass connectors

Joker_vD 2y ago

I think any of those openapi.yaml files in the wild is a good example that we need better metaclass connectors.

jauntywundrkind 2y ago
What are you asserting? Why?

knome 2y ago

Unless there are two languages called Objective-S, here's a site for the language: https://objective.st/ https://objective.st/ I'll have to look closer at this later, but it doesn't give me a good feeling on first skim.

mpweiher 2y ago
That's the one. Alas, the website precedes my so far best attempt at explaining what I've done, which is this paper, and I haven't had time to properly update the site since. And probably won't for some time: first give the presentation at Onward! in Pasadena next Wednesday, then finish writing the PhD thesis, then update the site. It is what it is ¯\_(ツ)_/¯
- carapace 2y ago
  I think you're on to something. My only advice would be something like, don't neglect the debugging, uh, "stories".
  - mpweiher 2y ago
    Do you mean the stories of debugging the steaming pile of ... I mean my wonderful implementation of the language? Or how to debug these kinds of system? For the latter, I am actually very optimistic about making it possible to debug systems at a much higher and more meaningful level than currently, because you are making the high-level structure visible in the code instead of hand-compiling it down to some procedural implementation, with that hand-compiled procedural representation of your system being the thing that the the debugger sees. With both the polymorphic write streams and the storage combinators being so highly composable, it becomes fairly easy to insert logging/instrumentation/debugging elements between the components of your system.
    - carapace 2y ago
      The latter. ;) Sounds good.

jerf 2y ago

It isn't so much that there's no need for such things but that they are so pervasive and normal you can't hardly even "see" them. This is related to what killed the original OO conception as "your OO program should match your physical model", so, you have an Engine class that is directly related to the Drivetrain class (ignore "inheritance" versus "composition" and all such questions for this post), which is directly related to the Axle class, which is related to the directly Wheel class... what kills this model is that in practice you need stuff in between them. This model of OO is almost useless for anything except its original specification for simulations. Real-world OO however continues alive and well precisely in these "metaclass connectors", like, iterators, network call managers, and even the dreaded SomethingFactoryFactory. You need a WheelIterator and an CarVisitor and all kinds of other structural classes that are super useful to programmers but have no physical interpretation in OO, even if you are dealing with physical objects at all, which you often aren't. The paper points out that if we sometimes communicate over HTTP, and sometimes over a "Remote Procedure Call", and sometimes over some pipe or messaging system, to which I'd also add, sometimes via database communications or passing text files around and lots of other things, it could/would be helpful if our languages could interact with these things in ways other than procedure calls, which have a very particular abstraction they provide that is a poor match to these sorts of interactions [1]. On my personal list of "things I'd like to see new languages try" one of them is to make the default function call less capable, so that the language might more pervasively deal with "this function call failed" in a way modern languages can't very well... not, "this function ran and returned an error/exception" but this function call itself failed; couldn't find the code, couldn't reach the server, etc. Things like, every "function call" in the language being able to set a timeout rather than only certain special things. The purpose of this approach is for the foreignness of these various other methodologies to be something that no longer breaks function calls so thoroughly and causes these remote calls to be treated specially everywhere, where you have to specially wrap them in timeouts, specially handle a lot of failures you don't have to handle with normal functions, etc. As a side effect the language would potentially be more able to swap in "remote calls" for local calls more transparently than in modern languages, where local vs. remote calls act a lot like a "color" as referenced in the sync/async conversation, except it's even worse. Sync code wants to call sync code and async code wants to call async code, and if you need to cross the barrier you need an expensive and potentially lossy conversion process, but in this case, local code wants to call local code but there is no "remote code mode", the expensive and potentially lossy conversion is mandatory. Even when you stuff all that conversion code behind a single function call and sweep all the complexity somewhere where you can't see it at the call site, it's still there... it's just hidden better in some languages and frameworks and less well in others, and the question of the exact level of exposure that is best is one that can be debated for quite a while (it is definitely not clear that you should completely sweep away the differences even if you can, and there is a history of techs that take the sweeping too far [1]. (Yes, the same link twice.)) This paper and the accompanying language appears to take the opposite approach to what I was proposing, which was to loosen what a "function call" was until later abstractions could potentially harmonize the various approaches, and instead build an all-singing, all-dancing "connector" concept that envelops and abstracts all the various approaches to foreign communication, thus providing much more machinery than a normal function call. I consider it an interesting idea worth exploring, however, as someone who has worked with a lot of these "connectors" over the years (as many of us have, I am not claiming anything unique here), I observe that they have wildly varying characteristics and I question the ability of an all-singing, all-dancing abstraction to accommodate all or even a good subset of the possible options without becoming a poor lowest-common-denominator of all of them. For instance, even within two members of the same class, a message bus that is 0-or-1 has significantly different characteristics than 1-or-many delivery, and by "significantly" I don't mean the normal casual definition of "larger than average", but in this case, relevant to my design. It doesn't take much scale before any attempt to abstract around that isn't going to work. I've got cases where I speak to HTTP services and I may or may not add a client certificate and may or may not apply certain checks to my server's certificate, depending on how I route over the network to this service. It's hard to abstract that. And abstracting "this code may connect with either an HTTP JSON API or a transactional message bus", with all the accompanying differences in how much context the answer comes back with. HTTP has an associated socket, so even if you're "async" or even if you do something exotic like make the request and hand off the reply to a completely different thread, it's got much more context than a message coming in over the bus that initially is just "some kind of message" to your system and it has to laboriously work it down into a reply to what, how do I route that reply, etc. which is all much more complicated than the HTTP request, to say nothing of, what if I want want to send the message from one thread, but I want the reply to not only come back to that thread but also go somewhere else in my program, like a high-level metric tracking how many of each kind of reply I get, that is not inline with the original request/response cycle? I do not question that a lowest-common-denominator library could be constructed. Such things have been done. I would question whether it could meet real-world needs. For that someone would have to study their language directly for more time than I have for an HN post, because to really know if they succeeded you'd need to do more than just read the docs, but actually start building real programs and seeing how they do in the field. (Although reading the docs could be enough to tell they have not succeeded.) If they do succeed, then I give them significant props because I have just explained why I think it's hard. I have learned to be very skeptical of this sort of all-encompassing abstraction over things that in reality have such huge and significant differences between them despite superficial similarities that in the end I end up questioning if there was any "true" commonality to abstract over in the first place. I have been burned both in the using of such systems, and the occasional error I've made trying to build them. The ones that do work take massive, sustained usage from a large user base over years of development. While it may not be a connector in the sense this paper means, an ORM that tries to go beyond a simple table mapping but to expose the underlying capabilities of your database, for multiple databases, is an example of this. Ignoring the "are ORMs even good at all?" for a moment, as that is well-trod ground, the ones that are good and usable and functional are huge and take years upon years of bashing out in the real world before they get there. I would suspect that if something like what is described in this paper was going to get to the point that even "medium sized" cloud usage (to say nothing of Facebook or Google) could consider it a reliable technology would be a similarly sized task... the initial cut of the library would seem superficially appealing and seem like they're 90% of the way there, but upon examination it would turn out to be more like 10%. The real world is fairly harsh. [1]: https://news.ycombinator.com/item?id=30831309 https://news.ycombinator.com/item?id=30831309