5 ms·

The Python dictionary dispatch pattern

• DarkNova6 3y ago
  Despite the danger of being considered too cynical... "When your OOP support sucks so bad, the people might as well just use maps."
  • phyrex 3y ago
    Not even, I believe objects in python are just maps with some syntax sugar. I’m not sure what this is buying over a static class aside from being slower and more awkward to use?
    • DarkNova6 3y ago
      Documentation, type hints and arguably readability. But I understand that this is a very python answer.
    • Pxtl 3y ago
      In the article he describes using this for user-provided mathematical expressions. I suppose that makes sense if you're implementing a light DSL in python... But maybe don't? Edit: I don't believe this is any slower, actually. It may even be marginally faster than a class, since a class adds some QOL features that will add a little overhead that would add a few branches to the code path. But if you're worried about speed, don't use Python anyways.
    • raverbashing 3y ago
      > I’m not sure what this is buying over a static class 1. It is not static. It can be configured, changed at runtime, etc 2. First order functions are very powerful as well
      • fluidcruft 3y ago
        Modifiability is also true of python objects and classes. This is basically reinventing obj.__dict__ The real difference is that since it's not an object, "self" isn't in the function signature which to be fair can be annoying for a collection of stateless functions. So this is actually closer to a module.
        • sanderjd 3y ago
          Yeah, you alluded to this at the end, but you don't need to define these as methods on an instance (requiring `self`) if they don't require state.
          • fluidcruft 3y ago
            To be fair, I have come prefer approaching the problem stated in the link in the exact opposite direction. Specifically the part about having a collection of functions with signatures that are similar/identical. The example in the post is of a simple signature, but if you have more complex (and long) signatures and a bunch of functions that use them, it might be worth thinking about replacing the argument list with a frozen dataclass. @dataclass(frozen=True) class MyArguments: a: int, b: str, c: list = None def func1(self): ... def func2(self): ... etc. You can also sprinkle in @property and functools things like @cached, @cached_property when desired. It also lets you create functions that transform MyArguments into different MyArguments.
            • sanderjd 3y ago
              Yeah I'm with you here too.
              • fluidcruft 3y ago
                The struggle to figure out what the hell to name MyArguments is real. But that's also true about picking a name for the dict-of-functions-that-have-a-similar-weird-signature. :D
      • sanderjd 3y ago
        (1) seems like a bug rather than a feature, to me, outside extremely narrow use cases. It's generally much better to be able to statically reason about what code exists and how it is structured and called, than to have to reason about runtime behavior to determine those things. I agree with (2), but I'm not sure the pattern from the OP is a good example of using that power. It seems like using a language feature - mapping a string to an anonymous function - when a better one exists: defining named functions. Named functions can be looked up dynamically by name in python, and are (I think) generally more clear to a reader, especially with the standard documentation and type hints.
        • raverbashing 3y ago
          1) is absolutely not a bug, though sure you don't want to overuse it" "Oh but you can replace this with a derived class", it's not as flexible (for example, mix and matching, etc) > than to have to reason about runtime behavior to determine those things. Sure if you only do CRUD you probably won't run into those cases. Math heavy, scientific code, simulations, etc might disagree
          • sanderjd 3y ago
            > "Oh but you can replace this with a derived class", it's not as flexible (for example, mix and matching, etc) What do you mean by "mix and matching"? I feel like a concrete example might help me understand what you're thinking of in this thread. > Sure if you only do CRUD you probably won't run into those cases. Math heavy, scientific code, simulations, etc might disagree Ha, it's funny because I've recently started doing more math-y / scientific / simulation work in python, after a career spent mostly doing CRUD-y things in not-python, and I've never been more convinced that it is best to make code as statically legible as possible. It's hard enough to make sure the math and the algorithms are right, I don't want to also be reasoning through dynamic abstractions. But again, maybe there's a concrete thing you're thinking of here that I'm not seeing!
            • raverbashing 3y ago
              > What do you mean by "mix and matching"? A bit of a contrived example, but think of multiple inheritance but dynamically. > maybe there's a concrete thing you're thinking of here Genetic programming would be the main use case of this (of table dispatch, not the "mix and match thing", which ok, is a bit of a corner case > I've never been more convinced that it is best to make code as statically legible as possible Given how the average academic code is, I don't think I can disagree with you much And I think it's also a question of syntax. For those kinds of things a lambda might be more readable, but a typed lambda is still a lambda.
    • bjourne 3y ago
      It's open so you can change the dispatch during runtime. It's also pretty nice to dispatch on strings as not all situations require complicated setup with class hierarchies, enumeration classes, or what have you.
  • tmpX7dMeXU 3y ago
    I can at best see this pattern as useful for some select use cases. At least, that’s when I’ve used it myself. Python’s class system generally works completely fine for me, as someone that’s been using it for the better part of a decade full-time.
• chriswarbo 3y ago
  Dictionary lookup is often a handy way to solve small tasks, when (a) there's no need to build a whole class hierarchy, etc. and (b) the solution is nicely declarative/data-oriented. The article talks about string keys, but Python allows many things to be used as keys. For example, instead of chains of `elif` we can look up `True` in a dictionary: >>> age = 25 >>> { ... (age < 18): "Forbidden", ... (age == 18): "OK, you're old enough now", ... (age > 18 and age < 20): "LOL teen emoji amirite?", ... (age >= 20 and age < 40): "Hi", ... (age >= 40): "Welcome", ... }[True] 'Hi' One thing that's annoying with this pattern is that Python calculates values eagerly, so we can't use it for expensive calculations or observable effects. To make the values lazy we can wrap them in `lambda:`, but at that point we're essentially re-implementing dynamic dispatch, so classes may be better (or a couple of `elif`s, at most). Note: that's what the article's doing ;)
  • frou_dh 3y ago
    That's neat but it's also kinda the definition of "clever code". When a later reader is going to have to pause and ask themselves what on earth is this.
    • regularfry 3y ago
      It's worse than that because anyone who's touched a Lisp will think "this is a cond system! I know this!" and get the semantics exactly wrong.
    • someguy7250 3y ago
      Honestly when I accepted that clever code is bad code, my passion for coding died and now I just want to coast until I get fired and then go work at a phone repair shop I think when I picked my major, I never understood what software engineering is, or even what engineering is. Honestly I shouldn't be complaining. So many people have it worse. Coding would still be fun if it weren't a job, where we bend tech backwards to please the users (instead of teaching users to do the right thing)
      • lacrimacida 3y ago
• anentropic 3y ago
  I don't understand what value the dictionary is providing here, I suspect none? How is this better than just functions in a module? (import the module as math_exprs, use getattr)
  • Znafon 3y ago
    You could modify it easily, for example the user could register an additional function to the mapping. That would not be as easy to do with a module and getattr(). You can also list the registered function with math_exprs.keys() and that might not be as easy with a module either.
    • anentropic 3y ago
      if all the keys are unknown until runtime then I guess the dict makes sense I don't think it makes much sense in the example code given
  • llimllib 3y ago
    You use it to dispatch to the appropriate function in a module based on the value of a runtime variable
  • alanwreath 3y ago
    this. At least with the current example and even the surrounding chatter in this thread, I don’t see any benefit and plenty of room for downsides. Some of which haven’t even been mentioned yet (e.g how this will play with IDEs) I don’t mean to poo poo on it. I really do like learning new patterns, but I haven’t yet grasped this one’s utility.
• nomdep 3y ago
  Terrible example, because he is writing the keys when calling the functions: mydict[“foobar”]() is basically a worse version of using a module mymod.foobar() The REAL value of this pattern is when the keys are going to change on runtime: for name in userfilters: im = filtersdict[name](im)
  • zerojames 3y ago
    Love this! Thank you for sharing (I'm the author).
  • sanderjd 3y ago
    What is value are you envisioning is associated with the dynamic `name` key in your example? In my mind, this is more a case where I might return an anonymous function that captures the name, but it's less clear to me what the dict is for.
  • lijok 3y ago
    Even when key change at runtime, wouldn't you use getattr on a module? for name in userfilters: filter_function = getattr(mymod, name, None) if filter_function is None: raise NotImplementedError(name) im = filter_function(im)
  • Izkata 3y ago
    > The REAL value of this pattern is when the keys are going to change on runtime Which is demonstrated in the article, see second example (third and fourth code blocks).
  • MaxBarraclough 3y ago
    Even when changes are going on at runtime, strings are rarely the right answer. There's a name for when this approach is taken so far as to become an antipattern: stringly typed programming. [0] Typically, enums should be used in place of the way strings are used in the post. The obvious advantages of enums: they use language-level identifiers, so the language is able to check whether you're referring to something that exists. IDE autocompletion will work properly, so you won't have to memorise which strings are accepted where. Enums can also nicely leverage a static type system in languages that support it (i.e. it should be a compile-time error to try to use a member value from an irrelevant enum). Dispatching over an enum may also have performance benefits, if that matters. That said the post does mention that the strings-as-keys approach is particularly useful in hand-written parsers, which sounds like a reasonable exception. [0] https://blog.codinghorror.com/new-programming-jargon/ https://blog.codinghorror.com/new-programming-jargon/
• b0b10101 3y ago
  It's a favourite pattern of mine too. I'm pretty sure it's bad practice but in one project i needed some form of parameterised extensibility (parsing a broad mix of files onto a standard format). So what i did was have other devs subclass an abstract base class I created (with some predefined attributes and methods to fill in) and then generated my execution dictionary based on the attributes of the subclasses of the base class.
  • sanderjd 3y ago
    This doesn't sound like a bad pattern to me. (Though I certainly think I have seen bad implementations of it.) But if the keys are known statically (as in the article), I think it's generally better to use named functions. But it's good to be aware of the capability, for cases like you're describing.
• denton-scratch 3y ago
  I'd be less than enthuisiastic about relying on a function named "yolo<something>". I take "You Only Live Once" (YOLO) to express approval of risk-taking, which isn't usually my intent when I rely on an API.
  • zerojames 3y ago
    YOLO is a model architecture in computer vision (You Only Look Once). https://blog.roboflow.com/whats-new-in-yolov8/ https://blog.roboflow.com/whats-new-in-yolov8/ In one example, dictionary dispatch decides what model to use: YOLOv5, v8, SAM, and others.
  • Madmallard 3y ago
    pretty sure yolo is an object recognition ML model
  • deleted 3y ago
    [deleted]
  • fluidcruft 3y ago
    Use this to build an affective programming interface: call "yolo", "y̴o̸l̶o̸", "ÿ̵͔̽̑̀̋͋̒͠o̷̢̫͉̠̭̭̔͗̑̎̍l̴̡̡̨̹͈̤̯̜̓̅͂̓̽͊͂̔o̴̭̙̼̺̹̭͚̊̔͑̽͆͝" or "y̸̛̬̣̯͇̆͑̂̍̈́̏́̆̒̌͆̐̉̑͒̆̐̄̄̒͒͘͝͝͝͝͝ơ̴̡̧̧̛̥͖͖͕̪̣̜̮̜͉̹̙̠̬̝̱̹͇̻͖̲̦̲͚̣̦̏̿͑͌̽̒̅̏̈̉̃͆̾̌̽̽̐̆̃̇͋̐͂̏̓͛̈̈́͊̆̔̒͂͂͛̃͂̀̽͛̾̎͋̏̑̀̌͑̏̒́̆͗́͊͂͂͑̎͘͘͝͠͠ͅͅl̵̨̨̧̛̺̖̫͇̬̻͕̠̬̮͎̖͎̟̱̟͙͉͓͔͖͕̗̜̝͚̠͔̹͎̭̻͙̭͚͎̝͍̯̭̖̠̳̱̩̱̠̜̯̱̠̻̯̻͚̙͖̭̟̹͆͆̀̆͌̓̇̍̇̏͂̽̇̇͛̓͘͜͜͝ͅͅͅỏ̷̢̹͎̺̞͚̥̲̐͋̈́̑̈̈͘̚" to set the tone appropriately.
• whalesalad 3y ago
  I use this pattern a lot. It's also nice to be able to do dict.get(key, lambda: raise ValueError(f'no handler for {key}')) to have a nice fallback or raise in a convenient one liner.
  • ofek 3y ago
    That's not at all valid syntax, this does not work.
    • whalesalad 3y ago
      yeah sorry the 'raise' is not valid in-line I did this from mobile - normally I would define the fallback handler elsewhere and/or place it in the map under a key like 'default'
• rtpg 3y ago
  I've found that lots of dictionary dispatch in Python ends up feeling a lot nicer as class attributes. You get nicer typing properties, autocomplete, don't need to type quotation marks in many places, and you don't end up in the weird "use lambdas or define functions and then refer to those in the dictionary". Of course if you're trying to do a registry-pattern style thing the constraints are different (though there I also think that you end up wanting to base your stuff on classes whose metaclass registers itself based on a string-y class attr or something...) Though in the end it's super dependent on what kind of code you're trying to write. No rule is absolute, especially in this category of aesthetics.
  • Izkata 3y ago
    If you're hardcoding it sure, but at that point you should consider just using a module with bare functions. This pattern is for when the lookup value comes from somewhere else, and to use it with attributes requires a more awkward getattr() call, and it has a less obvious error message if the value isn't in the lookup.
• est 3y ago
  why not just SUPPORTED_INFERENCE_MODELS = { "groundingdino": registry.grounding_dino_base, "yolov8": registry.yolov8_bas, ... }
• ijustlovemath 3y ago
  I prefer doing this directly in a registry/factory class, eg: class Registry: constructors = {} @classmethod def register(cls, other): # you can do it by an implicit property or the class name in simple cases. It's better not to use implementation details here, eg you should use an explicit class attribute # you can also verify that the other class holds certain invariants here, so it fails immediately cls.constructors[other.__name__] = other # cls.contructors[other.name] = other @classmethod def from_name(cls, name): # error handling code here return cls.constructors[name] # in something.py @Registry.register class Something: def __init__(self, foo, bar=None): self.foo = foo self.bar = None # In something_test.py import something import registry smth = registry.from_name("Something") s = smth('foo', bar=2) You'll want to have a registry for each base class (they largely all look the same except for the invariant checks), but this way you don't have to update a map, you just add the decorator to new classes as you build them. It also allows classes to be completely unaware of the registry and focus on their actual task. Additionally, the invariant checking can do all sorts of things that you can't do with a base class, eg making sure that overridden properties are still properties, checking that class attributes follow whatever formatting convention you have, etc etc
• js2 3y ago
  I sometimes use this pattern in combination with a decorator to register functions which I later dispatch to dynamically. Something like: funcs = {} def register(key): def decorator(func): funcs[key] = func return func return decorator @register("some_data_key") def do_something_with_some_value(value): return value # later.... def parse(data): return { key: func(data.get(key)) for key, func in funcs.items() }
• ramesh31 3y ago
  There's an equivalent JS pattern that I always suggest over using switch cases, as it's far more idiomatic: const case = 'case1' const result = { 'case1': 'value1', 'case2': 'value2', 'case3': 'value3', 'case4': 'value4', } [case] || 'defaultVal'
• calderwoodra 3y ago
  Anything that breaks IDE type hinting and usage and definition linking is a non-starter for me unless the entire scope of the function fits in about a dozen or so lines.