3 ms·
The history of the Schwartzian Transform (2016)
- ygritte 1y agoI'm so grateful for the Lisp translation. Perl is such a write-only language, my hair curls up just from looking at it.
- spauldo 1y agoTo be fair, this isn't what normal Perl usually looks like. Perl allows you to write like this but most people don't. This is more like Duff's Device in that it works and it's clever and you'd can anyone who put it in production code. That's not to say that idiomatic Perl doesn't have its quirks and oddities - Perl idioms do throw off people who aren't used to them. But this is a bit far even for Perl.
- dale_glass 1y agoI've definitely seen it in production code, but it was a good deal after 1994. By then it was a commonly known technique, so it was a pattern a Perl dev would be expected to recognize.
- kilna 1y agoIf the production code has to be maximally performant, you'd want the transform. It doesn't copy data around with a bunch of needless assignments. Pipelining the extract/sort/reconstitute in this manner is absolutely the right thing to do in production if production requirements are to keep it fast and with a small footprint. There's no excuse for not commenting it, but strictly-code-speaking there's no reason not to use this for production if the requisite needs are there.
- throwanem 1y agoEh. It's not worse than Ruby, and vice versa.
- heresie-dabord 1y ago> X is such a write-only language, my hair curls up just from looking at it. That would be a powerful language feature, but alas X.isWriteOnly() for X in aLanguagesIdonotUse is merely akin to X.hasTooManyRules() for X in aHumanLanguagesIdonotSpeak
- RodgerTheGreat 1y agoI wrote a blog post some time ago which contrasts the "Schwartzian transform" with the slightly clearer alternative used by most APL-family languages and (in my opinion) an even cleaner approach which is possible in languages with a first-class table type: https://beyondloom.com/blog/rankingoffruits.html https://beyondloom.com/blog/rankingoffruits.html
- inopinatus 1y agoThat was a lovely vignette, thanks. Much earlier in my career I had exactly the sense suggested that “something’s off” about the Schwartzian, even though it became a useful and idiomatic tool at the time. I only really understood the disquiet much later, when working with columnar data, and realising the positive qualities of declaring structs-of-arrays rather than arrays-of-structs, with subsequently excellent mechanical sympathy, and so on from there.
- throwanem 1y agoUnfortunately, the only such language to which most workaday devs have access is some dialect of SQL. Fortunately, at least there's that.
- mont_tag 1y agoPython's key-functions nicely encapsulate the whole process.
- oofabz 1y agoI find it interesting that the transform was controversial in the '90s. Today, it seems like a normal solution to the problem to me, and the controversy seems silly. I have much experience with the map function from Javascript. It is too simple to be objectionable. But in the '90s, I would also have had trouble understanding the transform. Lambdas/closures were unheard of to everyone except Lisp dweebs. Once I figured out what the code was doing, I would be suspicious of its performance and memory consumption. This was 1994! Kilobytes mattered and optimal algorithmic complexity was necessary for anything to be usable. Much safer to use a well understood for loop. I have plenty of experience making those fast, and that's what map() must be doing under the hood anyway. But I would have been wrong! map() isn't doing anything superfluous and I can't do it faster myself. The memory consumption of the temporary decorated array is worth it to parse the last word N times instead of N log N times. Lisp is certainly a slow language compared to C, but that's not because of its lambdas!
- zippyman55 1y agoThank you for posting! I always felt like such a badass in the late 1990s when I used this.
- ubercore 1y agoI believe I first came across this in the early xslt days, when I had a manager that wanted to write all software with an XML requirements spec, then use xslt to simultaneously generate code AND documentations. 1. It did not work because of course it didn't work 2. This meant that all data output of the system would done in XML and transformed, so I got to know xslt quite well 3. It did give me a fun moment at a conference where a speaker asked "Who knows the Schwartzian transform or has used it?" and only me and my coworkers raised their hands
- jiggawatts 1y agoI was curious to see what other languages do by default (or not), so I quickly tested a few: C# evaluates the lambda only once per item, in input order: https://dotnetfiddle.net/oyG4Cv https://dotnetfiddle.net/oyG4Cv Java uses "Comparators", which are called repeatedly: https://www.jdoodle.com/ia/1IO5 https://www.jdoodle.com/ia/1IO5 Rust has sort_by_key() which is encouraging... but also calls the key transformation many times: https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=131df402d96e040dca37bb60217bd019 https://play.rust-lang.org/?version=stable&mode=debug&editio... C++20 with std::range also calls the lambda repeatedly: https://godbolt.org/z/raa1766PG https://godbolt.org/z/raa1766PG Obviously, the Schwartzian Transform can be manually implemented in any modern language elegantly, but it's interesting that only C# uses it implicitly! PS: The fancy solution would be a temporary "Hashset<T,K>" used to map values to the transformed sort keys, so that the mapping Foo(T t): K would only be called once for each unique input value 't'. That way the transformation could be invoked less than O(n) times!
- ynzoqn 1y agoInspired by your behavior, I looked up the Wikipedia article on Schwartzian transform[1]. The article mentioned that Rust not only provides sort_by_key, but also sort_by_cached_key. :D [1]: https://en.wikipedia.org/wiki/Schwartzian_transform https://en.wikipedia.org/wiki/Schwartzian_transform
- jiggawatts 1y agoI keep saying to colleagues that simply knowing that something exists in a library somewhere is 90% of the skill of modern programming. I suspect the profession of software archeology form the Vernor Vinge science fiction novels will soon become a reality, especially now that old languages can be uplifted mechanically with AI.
- Sharlin 1y agoThis one is particularly easy to chance upon, though, by either browsing the docs of slices’ `sort*` methods (because they’re all clumped together on the page), or even simpler, noticing it when you type `foo.sort` and your editor suggests autocompletions.
- Sharlin 1y ago> Some people want Perl to be accessible at first glance to someone who doesn’t know the language. …oh boy. In retrospect it doesn’t seem like these people prevailed.
- kilna 1y agoPerl was ahead of its time, and likely because of it's Borg-like capacity to absorb a little bit of everything... it was unfortunately nigh on impossible to refactor. This is why Perl6 (nee Raku) took nearly 2 decades to be "done". It's a crying shame that it lost momentum, because many of the techniques that the features of Raku enable are as clever/succinct/revolutionary as the Schwartzian Transform was back in its day... but because the developer community is so very small now, Raku will have a very hard time catching up to the performance of competing languages.