6 ms·

Wish I understood this better. Seems very interesting.

Jtsummers 5y ago
Pages 4 and 5 illustrate the central conceit: Instead of a set of fixed operations (like allocating a class instance or accessing properties/slots/fields of a class in a fixed way), you can reprogram how your classes behave (in CLOS, at both compile and runtime). By way of analogy, consider a GPU implementing a fixed-function pipeline versus having shader units. The former gets the job done, but only offers so many ways to control what gets rendered, the latter is far more capable in the longterm (at the cost of added complexity). Trying to get this to fit into a comment so I'm being handwavy a bit. Consider the Person class that they describe. Because it's sparse (not every field is populated), instead of having a large fixed size for all thousand fields (or however many) they change out the underlying representation to use a hash table. Now, anyone can represent their data using a map. However, just storing your data in a map doesn't let you continue to use the other language features that are built around classes and objects. In particular, take C++, if you stored your data in a map you'd lose this bit of handy syntax: Person p = ... p.age = ... Well, if age is stored in a map inside the person class, there is no age field to assign to. With a MOP, however, you can still have this more compact data representation format (since the data is sparse), but still have all the normal language syntax and semantics around the object (like accessing fields using the dot syntax). In Common Lisp you might use (or any other slot accessor): (with-slots (age height) person ...) And the change in representation doesn't change how the slot values are accessed (both for reading and for writing). Now, changing the underlying data representation is useful, but a relatively small thing. Consider something a bit more useful: Ada's protected objects. A protected object in Ada adds, basically, mutexes and condition variables via language syntax to control access to data shared across tasks (and other things, this is just the main utility of them). It's a special set of syntax added to provide this ability, in principle you can use MOP to extend CLOS to provide this behavior. Change your meta-class from standard-class to protected-class. Users still access objects (from their perspective) exactly like any other, but all the mutex and condition variable stuffs can be automatically baked in (no need to add extra logic in your methods). Now, this does introduce potential runtime complexity which eats into performance. And that's addressed further into the linked paper. It's not totally insurmountable, but it does have to be considered (any time you add indirection or potential indirection you have to decide if it's worth the cost).
- brundolf 5y ago
  On the surface, this: > With a MOP, however, you can still have this more compact data representation format (since the data is sparse), but still have all the normal language syntax and semantics around the object (like accessing fields using the dot syntax) just sounds like the JavaScript object model, and this: > Users still access objects (from their perspective) exactly like any other, but all the mutex and condition variable stuffs can be automatically baked in (no need to add extra logic in your methods). kinda sounds like computed properties/getters+setters (or simply "Properties" in C#), which many languages have at this point. I assume I'm missing or misunderstanding something?
  - db48x 5y ago
    Yes, you missed the point where you as the programmer can choose the semantics of the language as you go, rather than the language designer doing it ahead of time. Instead of having a language designer pick one spot in a large space of possible designs, the Metaobject Protocol gives the language user the ability to pick their own spot within that space. You can have sparse objects in the one project where they are needed, while not bothering in projects where they aren’t. You can have version controlled objects, or database–backed objects, or objects which implement inheritance differently. If you can think of it, you can probably do it.
    - brundolf 5y ago
      That still sounds like getters/setters, then (which you could use to implement properties-as-a-hash-map, for example). Maybe the ergonomics are different - we're still speaking pretty abstractly here so it's hard to tell - but it sounds like the same basic idea: class TimePeriod { private double _seconds; public double Hours { get { return _seconds / 3600; } set { if (value < 0 || value > 24) throw new ArgumentOutOfRangeException( $"{nameof(value)} must be between 0 and 24."); _seconds = value * 3600; } } }
      - db48x 5y ago
remexre 5y ago
The oversimplified view is that it's basically an API like Python's "dunder methods," but both more powerful and (iiuc) able to be compiled to reasonably fast code.
Rochus 5y ago
Kiczales and Bobrow are also the author of this book https://www.amazon.com/-/de/dp/0262610744 https://www.amazon.com/-/de/dp/0262610744 which has more details.
_old_dude_ 5y ago
The idea is to let users of a language, here CLOS, to be able to intercept meta-messages like instance creation, method call, property access, etc to provide their own implementations. It's like the doesNotUnderstand in SmallTalk, method_missing in Ruby or more recently .Net Dynamic Language Runtime or invokedynamic in Java.
- astrange 5y ago
  It's commonly used in Objective-C due to Smalltalkness - NSInvocation, XPC, OCMock, etc.
- nerdponx 5y ago
  I believe Lua has this with metatables, and of course Python has its "magic methods" and metaclasses.
- DaiPlusPlus 5y ago
  That’s a bad idea though, and the past 30+ years of OOP has laid-bare all of the problems associated with not having compile-time object type safety. Just because you can do something doesn’t mean you should… Outside of prototyping and quick-hacks, there is no legitimate reason to use `dynamic` in C#/.NET: instead, the moment your program receives or handles a “late-bound” (i.e. untyped) object your program should verify its actual type and then access it only through a wrapper type (be it an interface, struct, or class) which is how you can enforce guarantees about that object. This also applies to reflection (so have all of your `GetMethod` calls during the wrapper’s ctor and use Action/Func wrappers over MethodInfo).
  - db48x 5y ago
    It’s nothing like reflection or untyped objects.
    - DaiPlusPlus 5y ago
      I was referring to this: > It's like the [...] .NET Dynamic Language Runtime or invokedynamic in Java. My answer concerns the "DLR", aka `dynamic` type in C#/.NET, which is untyped. The GP is not correct when describing the DLR/`dynamic` as to "be able to intercept meta-messages like instance creation, method call, property access, etc to provide their own"... ...C#/.NET does not allow for this: you cannot intercept methods or otherwise have Smalltalk-style message-passing in .NET. Using `dyanmic` does allow you to intercept methods and define behaviour but only with `dynamic` objects managed by your own `IDynamicMetaObjectProvider`, which I concede you could use to implement this, but that would force you to type everything as `dynamic` which also means you lose static type safety. (pre-.NET Core, the CLR had hardcoded magic behaviour for `Proxy` types, which is how .NET Remoting worked with any object type: including sealed types, but this was removed in .NET Core and its replacement in `DispatchProxy` falls far short: it only works with interfaces (hence vtables), not static/non-virtual methods). -------------- What surprises me about .NET's CLR is that despite the (frustrating) simplicity of its type-system, that applications still don't have any real opportunities to extend the CLR's type-system to suit themselves. This includes anonymous refinement types via flow-analysis (TypeScript gets this right), the lack of support for real ADTs (F#'s tagged-unions exist in their own hierarchy: you can't have a tagged-union of arbitrary types, which basically makes them the same as Java's `enum` classes).
Banana699 5y ago
Basically (I don't understand this satisfactorily either) : represent meta concepts of the object system inside the object system itself. As always, the best way to make sense of it is with an example. One example is meta classes in python. Classes, those 'blueprints' that we create objects in their image? they are themselves but objects, their class is called 'type', it's a special kind class, a meta class (a class whose objects are themselves classes). You can inherit this type and create your own meta class, and that lets you do various cool things. For one thing, object creation in python is done by calling the object' s class as if it's a function. Now this is a method call (__call__), where do methods reside? in the class off course. So for classes, the __call__ reside in the meta class. When python says you can inherit this class, you listen very carefully. This means you can inject arbitary code during the instance creation process. Now every class that is an instance of your meta class can email you whenever somebody wants to create an object from it. It's like the interpreter is giving you a hook to something incredibly intimate and private to the VM, exposing it from within the interpreted language itself (and not as a C API for example) meta classes is a kind of MOP for python because it represents intimate details about an object system ('hey, what exactly happens when we ask a class for an object?') inside that object system itself ('it calls this particular method, which you are more than welcome to override and augment as you please without leaving the comfort of the language').

Pages 4 and 5 illustrate the central conceit: Instead of a set of fixed operations (like allocating a class instance or accessing properties/slots/fields of a class in a fixed way), you can reprogram how your classes behave (in CLOS, at both compile and runtime). By way of analogy, consider a GPU implementing a fixed-function pipeline versus having shader units. The former gets the job done, but only offers so many ways to control what gets rendered, the latter is far more capable in the longterm (at the cost of added complexity). Trying to get this to fit into a comment so I'm being handwavy a bit. Consider the Person class that they describe. Because it's sparse (not every field is populated), instead of having a large fixed size for all thousand fields (or however many) they change out the underlying representation to use a hash table. Now, anyone can represent their data using a map. However, just storing your data in a map doesn't let you continue to use the other language features that are built around classes and objects. In particular, take C++, if you stored your data in a map you'd lose this bit of handy syntax: Person p = ... p.age = ... Well, if age is stored in a map inside the person class, there is no age field to assign to. With a MOP, however, you can still have this more compact data representation format (since the data is sparse), but still have all the normal language syntax and semantics around the object (like accessing fields using the dot syntax). In Common Lisp you might use (or any other slot accessor): (with-slots (age height) person ...) And the change in representation doesn't change how the slot values are accessed (both for reading and for writing). Now, changing the underlying data representation is useful, but a relatively small thing. Consider something a bit more useful: Ada's protected objects. A protected object in Ada adds, basically, mutexes and condition variables via language syntax to control access to data shared across tasks (and other things, this is just the main utility of them). It's a special set of syntax added to provide this ability, in principle you can use MOP to extend CLOS to provide this behavior. Change your meta-class from standard-class to protected-class. Users still access objects (from their perspective) exactly like any other, but all the mutex and condition variable stuffs can be automatically baked in (no need to add extra logic in your methods). Now, this does introduce potential runtime complexity which eats into performance. And that's addressed further into the linked paper. It's not totally insurmountable, but it does have to be considered (any time you add indirection or potential indirection you have to decide if it's worth the cost).

brundolf 5y ago
On the surface, this: > With a MOP, however, you can still have this more compact data representation format (since the data is sparse), but still have all the normal language syntax and semantics around the object (like accessing fields using the dot syntax) just sounds like the JavaScript object model, and this: > Users still access objects (from their perspective) exactly like any other, but all the mutex and condition variable stuffs can be automatically baked in (no need to add extra logic in your methods). kinda sounds like computed properties/getters+setters (or simply "Properties" in C#), which many languages have at this point. I assume I'm missing or misunderstanding something?
- db48x 5y ago
  Yes, you missed the point where you as the programmer can choose the semantics of the language as you go, rather than the language designer doing it ahead of time. Instead of having a language designer pick one spot in a large space of possible designs, the Metaobject Protocol gives the language user the ability to pick their own spot within that space. You can have sparse objects in the one project where they are needed, while not bothering in projects where they aren’t. You can have version controlled objects, or database–backed objects, or objects which implement inheritance differently. If you can think of it, you can probably do it.
  - brundolf 5y ago
    That still sounds like getters/setters, then (which you could use to implement properties-as-a-hash-map, for example). Maybe the ergonomics are different - we're still speaking pretty abstractly here so it's hard to tell - but it sounds like the same basic idea: class TimePeriod { private double _seconds; public double Hours { get { return _seconds / 3600; } set { if (value < 0 || value > 24) throw new ArgumentOutOfRangeException( $"{nameof(value)} must be between 0 and 24."); _seconds = value * 3600; } } }
    - db48x 5y ago

remexre 5y ago

The oversimplified view is that it's basically an API like Python's "dunder methods," but both more powerful and (iiuc) able to be compiled to reasonably fast code.

Rochus 5y ago

Kiczales and Bobrow are also the author of this book https://www.amazon.com/-/de/dp/0262610744 https://www.amazon.com/-/de/dp/0262610744 which has more details.

_old_dude_ 5y ago

The idea is to let users of a language, here CLOS, to be able to intercept meta-messages like instance creation, method call, property access, etc to provide their own implementations. It's like the doesNotUnderstand in SmallTalk, method_missing in Ruby or more recently .Net Dynamic Language Runtime or invokedynamic in Java.

astrange 5y ago
It's commonly used in Objective-C due to Smalltalkness - NSInvocation, XPC, OCMock, etc.
nerdponx 5y ago
I believe Lua has this with metatables, and of course Python has its "magic methods" and metaclasses.
DaiPlusPlus 5y ago
That’s a bad idea though, and the past 30+ years of OOP has laid-bare all of the problems associated with not having compile-time object type safety. Just because you can do something doesn’t mean you should… Outside of prototyping and quick-hacks, there is no legitimate reason to use `dynamic` in C#/.NET: instead, the moment your program receives or handles a “late-bound” (i.e. untyped) object your program should verify its actual type and then access it only through a wrapper type (be it an interface, struct, or class) which is how you can enforce guarantees about that object. This also applies to reflection (so have all of your `GetMethod` calls during the wrapper’s ctor and use Action/Func wrappers over MethodInfo).
- db48x 5y ago
  It’s nothing like reflection or untyped objects.
  - DaiPlusPlus 5y ago
    I was referring to this: > It's like the [...] .NET Dynamic Language Runtime or invokedynamic in Java. My answer concerns the "DLR", aka `dynamic` type in C#/.NET, which is untyped. The GP is not correct when describing the DLR/`dynamic` as to "be able to intercept meta-messages like instance creation, method call, property access, etc to provide their own"... ...C#/.NET does not allow for this: you cannot intercept methods or otherwise have Smalltalk-style message-passing in .NET. Using `dyanmic` does allow you to intercept methods and define behaviour but only with `dynamic` objects managed by your own `IDynamicMetaObjectProvider`, which I concede you could use to implement this, but that would force you to type everything as `dynamic` which also means you lose static type safety. (pre-.NET Core, the CLR had hardcoded magic behaviour for `Proxy` types, which is how .NET Remoting worked with any object type: including sealed types, but this was removed in .NET Core and its replacement in `DispatchProxy` falls far short: it only works with interfaces (hence vtables), not static/non-virtual methods). -------------- What surprises me about .NET's CLR is that despite the (frustrating) simplicity of its type-system, that applications still don't have any real opportunities to extend the CLR's type-system to suit themselves. This includes anonymous refinement types via flow-analysis (TypeScript gets this right), the lack of support for real ADTs (F#'s tagged-unions exist in their own hierarchy: you can't have a tagged-union of arbitrary types, which basically makes them the same as Java's `enum` classes).

Banana699 5y ago

Basically (I don't understand this satisfactorily either) : represent meta concepts of the object system inside the object system itself. As always, the best way to make sense of it is with an example. One example is meta classes in python. Classes, those 'blueprints' that we create objects in their image? they are themselves but objects, their class is called 'type', it's a special kind class, a meta class (a class whose objects are themselves classes). You can inherit this type and create your own meta class, and that lets you do various cool things. For one thing, object creation in python is done by calling the object' s class as if it's a function. Now this is a method call (__call__), where do methods reside? in the class off course. So for classes, the __call__ reside in the meta class. When python says you can inherit this class, you listen very carefully. This means you can inject arbitary code during the instance creation process. Now every class that is an instance of your meta class can email you whenever somebody wants to create an object from it. It's like the interpreter is giving you a hook to something incredibly intimate and private to the VM, exposing it from within the interpreted language itself (and not as a C API for example) meta classes is a kind of MOP for python because it represents intimate details about an object system ('hey, what exactly happens when we ask a class for an object?') inside that object system itself ('it calls this particular method, which you are more than welcome to override and augment as you please without leaving the comfort of the language').