5 ms·

That was my first question, too. From the fine article: > We've drawn from and built upon the contributions of many others (the E programming language, Agoric

That was my first question, too. From the fine article: > We've drawn from and built upon the contributions of many others (the E programming language, Agoric, and Cap'n Proto to name a few) and is actively being shaped by input from many developers of those projects. The feature list includes distributed garbage collection, from what I understand that's out of scope of capnproto rpc. Maybe Kenton is around and can add more details?

dlahoda 3y ago
agoric is ibc. i know ibc app layer and partially core layer. how at all, all things will work together. for example timeouts in ibc, client cannot timeout if server is dead for example. ibc goes hand in hand with identity and sagas.
- omginternets 3y ago
  IBC?
  - dannyobrien 3y ago
    https://cosmos.network/ibc/ https://cosmos.network/ibc/ I think?
kentonv 3y ago
Ian "zenhack" Denhardt, a major Cap'n Proto contributor, was working closely with the ocapn effort to try to ensure some level of compatibility. Tragically, he passed away in July, leaving that effort (and so much else) in limbo. Unfortunately I have not had the bandwidth to keep up on ocapn, so I can't really comment much myself. ------------ I'll say a bit about "distributed garbage collection". I'm not caught up on what, exactly, the ocapn people mean here. E always advertised support for "distributed acyclic garbage collection", which in practice meant reference counting. Cap'n Proto also implements reference counting in exactly the same way. But I personally wouldn't call it "garbage collection" if it can't collect cycles. Moreover, Cap'n Proto applications commonly expect deterministic destruction when references are dropped -- they wouldn't play well with GCs that might arbitrarily delay collection. My personal take is that distributed GC is simply not feasible. Garbage collection algorithms heavily lean on heuristics and amortization to achieve any kind of performance. In particular, most GCs will only actually execute when they detect "memory pressure", i.e., that the system is in need of freeing up some memory. If the system doesn't need memory, why waste time finding some? However, in a distributed system, it can easily be the case that the local system has plenty of memory, but is holding on to a reference to a remote object living on a system that does need memory. The reference may no longer be reachable. But, the local GC isn't going to run because it doesn't perceive a need. Meanwhile, the remote system has no way to know that the object it is hosting could be collected. To solve problems like this, you cannot just have each machine running its own local GC, you must have a single distributed GC that can respond to memory pressure anywhere by triggering sweeps of the entire distributed object graph. This seems extremely difficult to design, far more so if the systems involved do not trust each other. Maybe it's possible, but certainly there's quite a lot of research needed, and last I knew the ocapn people hadn't actually demonstrated anything that works. But my personal take is that GC in general is a terrible fit for systems programming, because it is so bad at handling any kind of external resource, which is what systems programming is all about. Cap'n Proto is unlikely to support anything more than reference counting.
- chubot 3y ago
  Yeah I always thought of distributed GC as a failed research idea from the 80's and 90's, kind of like transparent remote objects (as opposed to messages) I'd be interested to hear otherwise. And some details about a deployment
  - evgen 3y ago
    As an EC alum I can tell you that in the late 90s there was a lot of work on E to get both transparent remote objects and distributed garbage collection working. In fact, one of the first patents I remember us having approved was Arturo's garbage collector (https://patents.google.com/patent/US5991779A/en https://patents.google.com/patent/US5991779A/en, yes same guy as the Meta whistleblower from a week or so ago) but it was still a bit wonky in practice from what I remember.
    - chubot 3y ago
      OK it's great that it was tried But it feels like you're saying it was tried and didn't quite pan out? That's what my impression was

dlahoda 3y ago

agoric is ibc. i know ibc app layer and partially core layer. how at all, all things will work together. for example timeouts in ibc, client cannot timeout if server is dead for example. ibc goes hand in hand with identity and sagas.

omginternets 3y ago
IBC?
- dannyobrien 3y ago
  https://cosmos.network/ibc/ https://cosmos.network/ibc/ I think?

kentonv 3y ago

Ian "zenhack" Denhardt, a major Cap'n Proto contributor, was working closely with the ocapn effort to try to ensure some level of compatibility. Tragically, he passed away in July, leaving that effort (and so much else) in limbo. Unfortunately I have not had the bandwidth to keep up on ocapn, so I can't really comment much myself. ------------ I'll say a bit about "distributed garbage collection". I'm not caught up on what, exactly, the ocapn people mean here. E always advertised support for "distributed acyclic garbage collection", which in practice meant reference counting. Cap'n Proto also implements reference counting in exactly the same way. But I personally wouldn't call it "garbage collection" if it can't collect cycles. Moreover, Cap'n Proto applications commonly expect deterministic destruction when references are dropped -- they wouldn't play well with GCs that might arbitrarily delay collection. My personal take is that distributed GC is simply not feasible. Garbage collection algorithms heavily lean on heuristics and amortization to achieve any kind of performance. In particular, most GCs will only actually execute when they detect "memory pressure", i.e., that the system is in need of freeing up some memory. If the system doesn't need memory, why waste time finding some? However, in a distributed system, it can easily be the case that the local system has plenty of memory, but is holding on to a reference to a remote object living on a system that does need memory. The reference may no longer be reachable. But, the local GC isn't going to run because it doesn't perceive a need. Meanwhile, the remote system has no way to know that the object it is hosting could be collected. To solve problems like this, you cannot just have each machine running its own local GC, you must have a single distributed GC that can respond to memory pressure anywhere by triggering sweeps of the entire distributed object graph. This seems extremely difficult to design, far more so if the systems involved do not trust each other. Maybe it's possible, but certainly there's quite a lot of research needed, and last I knew the ocapn people hadn't actually demonstrated anything that works. But my personal take is that GC in general is a terrible fit for systems programming, because it is so bad at handling any kind of external resource, which is what systems programming is all about. Cap'n Proto is unlikely to support anything more than reference counting.

chubot 3y ago
Yeah I always thought of distributed GC as a failed research idea from the 80's and 90's, kind of like transparent remote objects (as opposed to messages) I'd be interested to hear otherwise. And some details about a deployment
- evgen 3y ago
  As an EC alum I can tell you that in the late 90s there was a lot of work on E to get both transparent remote objects and distributed garbage collection working. In fact, one of the first patents I remember us having approved was Arturo's garbage collector (https://patents.google.com/patent/US5991779A/en https://patents.google.com/patent/US5991779A/en, yes same guy as the Meta whistleblower from a week or so ago) but it was still a bit wonky in practice from what I remember.
  - chubot 3y ago
    OK it's great that it was tried But it feels like you're saying it was tried and didn't quite pan out? That's what my impression was