5 ms·
It's a nice idea, but i never like writing what amounts to a DSL in strings in my code (yes, that applies to in-code SQL as well, although that's often unavoida
by nerdponx 9y ago
It's a nice idea, but i never like writing what amounts to a DSL in strings in my code (yes, that applies to in-code SQL as well, although that's often unavoidable).
I prefer the `get_in()` method from Toolz: http://toolz.readthedocs.io/en/latest/api.html#toolz.dicttoolz.get_in http://toolz.readthedocs.io/en/latest/api.html#toolz.dicttoo...
- doublereedkurt1 9y agothe only string parsing is 'a.b.c', which is mostly equivalent to T.a.b.c so you can completely ignore that capability the very slight difference is that using T you must be explicit about attribute access vs key access whereas 'a.b.c' will try both
- radmin 9y agoMy thoughts also turned to toolz. Here's a example comparison: glom glom(target, ('system.planets', ['name'])) # ['earth', 'jupiter'] toolz list(pluck('name', get_in(('system', 'planets'), target))) # ['earth', 'jupiter']
- mhashemi 9y agoThe real advantage over toolz/get_in is in the breadth of types glom can support (http://glom.readthedocs.io/en/latest/api.html#setup-and-registration http://glom.readthedocs.io/en/latest/api.html#setup-and-regi...), the extensive fallback behavior (http://glom.readthedocs.io/en/latest/api.html#advanced-specifiers http://glom.readthedocs.io/en/latest/api.html#advanced-speci...), and "T" specifier (https://sedimental.org/glom_restructured_data.html#true-python-native https://sedimental.org/glom_restructured_data.html#true-pyth...), which allows performing object-oriented traversals and calls. Check the post for more examples of where path-based access falls short of the mark! (Sidenote: I find it very clunky that get_in() takes a "default" kwarg and _also_ a n"o_default" kwarg. Use a Sentinel object! http://boltons.readthedocs.io/en/latest/typeutils.html#boltons.typeutils.make_sentinel http://boltons.readthedocs.io/en/latest/typeutils.html#bolto... )
- nerdponx 9y agoSentinel is one of my favorite objects in Python.
- Sean1708 9y agoWhat's the advantage of make_sentinel vs just doing _MISSING = object()?
- nerdponx 9y agoA Sentinel is guaranteed unique and distinct from all other objects.
- Sean1708 9y agoI'm fairly sure object() also returns an object which is unique and distinct from all other objects. The only difference as far as I can see is that make_sentinel returns an object that has a unique and distinct type from all other objects, but I don't see why you'd be checking the type of your sentinels in Python.
- mhashemi 9y agoHehe, boltons.typeutils.make_sentinel should really be documented better for more experienced developers. A few small advantages: a nice repr, pickleability, and (back to the first advantage really) good rendering in a Sphinx autodoc context. :)
- doublereedkurt 9y agoa nice __repr__ is the main thing
- nerdponx 9y agoSorry to double reply, but yes the "T" spec is exactly what I would want. IMO it's yet another step better than using a "spec" made of untyped nested lists and dicts. DSL in strings = bad DSL in native syntax = good
- abuckenheimer 9y agoI feel the same way, although my instinct is generally to build a custom generator. Only costs a couple lines but is plain old python and quite explicit target = {'system': {'planets': [{'name': 'earth', 'moons': 1}, {'name': 'jupiter', 'moons': 69}]}} glom(target, {'moon_count': ('system.planets', ['moons'], sum)}) # vs def iter_moons(t): for planet in target['system']['planets']: yield planet['moons'] sum(iter_moons(target)) would have to combine with `defaultdict`s if your nested data is only sometimes there though
- mhashemi 9y agoThat's a great example! Mind if I use it? ;)
- ziikutv 9y agoYou can also use a list comprehension like so. >>> sum([x['moons'] for x in target['system']['planets']])
- jeremiahwv 9y agoFor simple cases like this it doesn't even cost a couple lines as sum() can take a generator expression: sum(planet['moons'] for planet in target['system']['planets'])
- divbzero 9y agoCombining ideas from parent and grandparent: sum(planet.get('moons', 0) for planet in target['system']['planets'])
- typon 9y agoWhy is writing DSLs strictly worse than writing complicated transformations built on top of the limited constructs provided by the language itself? (You said never)
- joshuamorton 9y agoThe issue isn't dsls, it's string dsls. Doing it within the language can provide stronger garauntees. As a simple example, you get some syntax checking.
- nerdponx 9y agoPrecisely. Until my autocomplete, syntax highlighting, and linting engines recognize it, I'll pass.
- mhashemi 9y agoI think y'all might be getting a bit up in arms about a non-issue. Seasoned glommers (and experienced Pythonists), will almost always be using T: http://glom.readthedocs.io/en/latest/api.html#object-oriented-access-and-method-calls-with-t http://glom.readthedocs.io/en/latest/api.html#object-oriente... While mentioned in the post, it's not front-and-center because that's the sort of Python superpower that can look scary to less-experienced devs.
- jkaptur 9y agoThey’re pretty comparable when the strings are constants. The fun begins when the string is dynamically created.
- mhashemi 9y agoI wouldn't recommend dynamically generating strings. Strings are just a shorthand for Path objects, use Path instead: http://glom.readthedocs.io/en/latest/api.html#specifier-types http://glom.readthedocs.io/en/latest/api.html#specifier-type...
- jeremiahwv 9y agoI agree, I don't like the magic string approach (even if it is mostly just dot-notation attribute lookup). However, there is some good stuff here, and nested data lookup when value existence is unknown is a pain point for me. In addition to the string based lookup, it looks like there is an attempt at a pythonic approach: from glom import T spec = T['system']['planets'][-1].values() glom(target, spec) # ['jupiter', 69] For me though, while I can understand what is going on, it doesn't feel pythonic. Here's what I would love to see: from glom import nested nested(target)['system']['planets'][-1].values() And I would love (perhaps debatably) for that to be effectively equivalent to: nested(target).system.planets[-1].values() Possible? --- edit: Ignore the above idea. I thought about this a bit more and the issue that your T object solves is that in my version: nested(target)['system'] the result is ambiguous. Is this the end of the path query and should return original non-defaulting dict, or the middle of the path query and should return a defaulting dict? Unknown. The T object is a good solution for this.
- emidln 9y agoObjects are overkill. Use an iterable of getitem lookups. They compose easily and don’t have much overhead (cpu or cognitive). From the library I use at work, inspired by clojure.core/get-in, it would look like this: def get_in(obj, lookup, default=None): """ Walk obj via __getitem__ for each lookup, returning the final value of the lookup or default. """ tmp = obj for l in lookup: try: tmp = tmp[l] except (KeyError, IndexError, TypeError): return default return tmp data = {“foo”: {“bar”: [“spam”, “eggs”]}} # find eggs get_in(data, [“foo”, “bar”, 1]) By using __getitem__ you naturally work with anything in the python ecosystem.
- Too 9y agoI prefer your nested() version. I don't see why your final example would be ambiguous either, if it always returns a defaulting dict which you always have to extract with .values() there is no ambiguity. This is similar to how Javas Optional<T> work. The problem could be that the type checker isn't good enough and writing code like if nested(target)['system'] == "lazer" could pass the type checker.