4 ms·

I used to use interface upgrades in rclone (a program which syncs your data to many different cloud providers) for all the optional features a backend (cloud pr

by nickcw 2y ago

I used to use interface upgrades in rclone (a program which syncs your data to many different cloud providers) for all the optional features a backend (cloud provider) might have, for example renaming a file. However the proxy problem became unmanageable. That's when I had a backend (eg the crypt backend which encrypts any other backend) which wraps another backend. Can the crypt backend rename things? That depends on what it is wrapping and you'd have to upgrade the interface call it and then get a special error to find out. Eventually I switched to a table of bound method pointers which were easy to test against nil to see whether they were implemented and could be filled with a very small amount of reflection. In my experience interface upgrades are useful but the proxy problem is very real so use sparingly only! I did suggest at one point (to rsc) an addition to Go which would allow interface proxies to take methods away from their method set at run time to fix this problem, so the proxy could only have the methods it could implement. I've no idea how difficult this would be to implement though.

lazypenguin 2y ago
Thanks for rclone, great tool!
memset 2y ago
Could you share a very small code snippet describing what you mean by bound method pointers? (Huge fan of rclone!)
- jerf 2y ago
  Something like this: https://go.dev/play/p/AVVz0qs2i1e https://go.dev/play/p/AVVz0qs2i1e This is much less convenient than using interfaces directly when that is possible, but comes with a lot more power. You have more insight into the "interface" value, you can use this to manually implement prototype-like behavior as seen in Javascript, you can deliberately make an object where one "method" is invoked on one object and one "method" is invoked on another, which is hard to read but may be just what you need in some situation, etc. There are a lot of ways to improvise on this tune. Also note most OO or OO-flavored languages can do this, Go has no special claim to it. But the tradeoffs are pretty similar in all those languages... you want to be sure this is what you need before you use it, because the design costs are significant.
38 2y ago
http://avtok.com/2014/11/05/interface-upgrades.html#the-proxy-problem http://avtok.com/2014/11/05/interface-upgrades.html#the-prox...
chrsig 2y ago
I face this a lot, and I think you have one very valid approach to it that I often employ. Especially if you have some centralized point to make a list of structs, or walk a tree to collect them. I think another other route is to provide lots of tooling to inspect the object graph -- include a pretty print function in the interface or something to that effect. There's equivalency between the two, the struct with pointers to implementation functions can implement an interface. Likewise, the implementation functions may call other implementations, so you haven't exactly solved the proxy problem, the state controlling which implementations becomes function local instead of on a struct. Unless you start to provide a struct with all of the context, at which point you've actually re-invented interfaces. I've grown wary of having too much of the details as function local like this because it hinders making more advanced tooling by hiding intermediate state from reflection. The interface provides the means to give other representations, and allows for embedding one within another, and gives runtime tooling to inspect underlying types. It gives more elegance and flexibility to the problem at language and standard library scale. One of the nice things is that it gives a sort of bidirectional refactoring path. If someone chose struct-of-pointers and it's at it's breaking point, those structs can stay as is and start to implement the interface with minimal effort. Likewise, if I need to boil down a hodgepodge of implementations into a really simple control flow, struct methods capture the struct and can be passed around as ordinary function pointers, so I can built a struct out of existing interface implementations. There's no real way around the complexity -- leaky abstractions are a thing. I think the author got it right: use interface upgrades and proxies sparingly. If they are the right tool for the job, then it's probably a good idea to go the extra mile to provide good documentation and instrumentation with extra attention to caller/callee expectations.
- nickcw 2y ago
  The main problem the method pointers solves for me is answering the question "can this object do X". The mere existence of a method you can type assert isn't enough. Returning a NotImplemented error isn't a great solution either as you might have to do quite a bit of work to get ready to call the method only to find it isn't implemented. Some bit of reflection magic which allowed objects to remove methods from their method sets at runtime would be perfect. You'd just use interface upgrades and you'd be sure that it would work. Also an available flag for each method would work too and might be neater than the function pointers.

by nickcw 2y ago

lazypenguin 2y ago

Thanks for rclone, great tool!

memset 2y ago

Could you share a very small code snippet describing what you mean by bound method pointers? (Huge fan of rclone!)

jerf 2y ago
Something like this: https://go.dev/play/p/AVVz0qs2i1e https://go.dev/play/p/AVVz0qs2i1e This is much less convenient than using interfaces directly when that is possible, but comes with a lot more power. You have more insight into the "interface" value, you can use this to manually implement prototype-like behavior as seen in Javascript, you can deliberately make an object where one "method" is invoked on one object and one "method" is invoked on another, which is hard to read but may be just what you need in some situation, etc. There are a lot of ways to improvise on this tune. Also note most OO or OO-flavored languages can do this, Go has no special claim to it. But the tradeoffs are pretty similar in all those languages... you want to be sure this is what you need before you use it, because the design costs are significant.

38 2y ago

http://avtok.com/2014/11/05/interface-upgrades.html#the-proxy-problem http://avtok.com/2014/11/05/interface-upgrades.html#the-prox...

chrsig 2y ago

I face this a lot, and I think you have one very valid approach to it that I often employ. Especially if you have some centralized point to make a list of structs, or walk a tree to collect them. I think another other route is to provide lots of tooling to inspect the object graph -- include a pretty print function in the interface or something to that effect. There's equivalency between the two, the struct with pointers to implementation functions can implement an interface. Likewise, the implementation functions may call other implementations, so you haven't exactly solved the proxy problem, the state controlling which implementations becomes function local instead of on a struct. Unless you start to provide a struct with all of the context, at which point you've actually re-invented interfaces. I've grown wary of having too much of the details as function local like this because it hinders making more advanced tooling by hiding intermediate state from reflection. The interface provides the means to give other representations, and allows for embedding one within another, and gives runtime tooling to inspect underlying types. It gives more elegance and flexibility to the problem at language and standard library scale. One of the nice things is that it gives a sort of bidirectional refactoring path. If someone chose struct-of-pointers and it's at it's breaking point, those structs can stay as is and start to implement the interface with minimal effort. Likewise, if I need to boil down a hodgepodge of implementations into a really simple control flow, struct methods capture the struct and can be passed around as ordinary function pointers, so I can built a struct out of existing interface implementations. There's no real way around the complexity -- leaky abstractions are a thing. I think the author got it right: use interface upgrades and proxies sparingly. If they are the right tool for the job, then it's probably a good idea to go the extra mile to provide good documentation and instrumentation with extra attention to caller/callee expectations.

nickcw 2y ago
The main problem the method pointers solves for me is answering the question "can this object do X". The mere existence of a method you can type assert isn't enough. Returning a NotImplemented error isn't a great solution either as you might have to do quite a bit of work to get ready to call the method only to find it isn't implemented. Some bit of reflection magic which allowed objects to remove methods from their method sets at runtime would be perfect. You'd just use interface upgrades and you'd be sure that it would work. Also an available flag for each method would work too and might be neater than the function pointers.