8 ms·
My account was reset. I've been around since day two. :) You're right, it has some downfalls. But a lot of the time it simply doesn't matter. All the links at
by sillysaurusx 7y ago
My account was reset. I've been around since day two. :)
You're right, it has some downfalls. But a lot of the time it simply doesn't matter. All the links at https://www.tensorfork.com/tpus https://www.tensorfork.com/tpus are dynamic, and the speed you gain by being able to whip up a feature in 10 minutes is worth the pain of an occasional dead link.
On the other hand, I did some work for "deduplicating fnids": http://arclanguage.com/item?id=20996 http://arclanguage.com/item?id=20996 which the site is using, so the links possibly last much longer than the early-HN links.
(Basically, we calculate the fnid key based on the lexical environment, rather than using a random ID each time the page is loaded. So each link gets a unique ID based on the code structure rather than a random ID. Meaning, instead of millions of random links to store, you end up with a few tens of thousands.)
- DonHopkins 7y agoAre fnids anything like Fnords? https://en.wikipedia.org/wiki/Fnord https://en.wikipedia.org/wiki/Fnord Or are they more like Vermicious Knids? https://roalddahl.fandom.com/wiki/Vermicious_Knids https://roalddahl.fandom.com/wiki/Vermicious_Knids
- sillysaurusx 7y agoIt's short for function ID. If you want to route requests to specific closures, you have to have some sort of ID that you can send down to the user. Arc stores closures in a hash table keyed by random ID, but we use lexical structure plus lexical values (like username) to make the key deterministic. It greatly cut down on the number of closures that needed to be stored. Basically, if you have a closure that does a certain action – e.g. editing a comment – Arc generates new a closure every page refresh. That was the root reason for the "dead link problem" during the early days of HN. I reworked it to make the closure IDs deterministic. If you walk the source code from the point of the closure up to the root, and then you stick all the local variables in a list and use that as a key, then hash that, along with the actual source code, you end up with something that (a) has a very low probability of collision, and (b) is deterministic each time the page refreshes, unless the variables' values change. (E.g. if you store the current time in a local variable, the value changes, so the fnid will change as well, since that value is captured by the closure and therefore the closure has to do something different by definition.)
- throwanem 7y agoThat sounds like a complicated fix! How many tens of minutes did it take? And how many tens of minutes did the bug cost, in effort spent on comments that got eaten instead of posted? (I know there's no way to answer that last question, but that doesn't mean its answer is equal to zero.)
- throwanem 7y ago(Ah, never mind, I see this line of thought has already been explored nearby. I've nothing to add beyond what's already been said there.)
- sillysaurusx 7y agoIt was complicated, but at one point I was so in love with Arc that I wanted to give it a real shot at taking over the world. It seemed like a necessary change to make, since the moment someone brought up dead links as a slight against arc, I could point to the change and say, "Already fixed!" (I made a few other improvements, like having accurate line numbers in error messages, and a clojure-style REPL for emacs: https://www.youtube.com/watch?v=3v1KpEmuPeM https://www.youtube.com/watch?v=3v1KpEmuPeM)
- stianstr 7y agoFnord!
- chongli 7y agothe speed you gain by being able to whip up a feature in 10 minutes is worth the pain of an occasional dead link This is everything that is wrong with the software industry, summarized in one sentence. Speed gain enjoyed by developers is paid for by the users in pain. It used to be that developers would go through tremendous amounts of pain just to squeeze out a few instructions from a UI drawing routine in order to make it just a little bit smoother for the users. Now those developers are derided as "greybeards." I'm sorry, I probably could have found a less harsh and cynical way of writing all that, but I feel like the Internet is getting worse everyday and there's not enough urgency among tech people.
- sillysaurusx 7y agoNot at all. I share your frustration. Closures on a server are a powerful way of representing data flows. But they come at a cost: the links expire after some time. How do you strike a balance? The simplest way is to put in the extra time to make everything into a persistent link. But, that's equivalent to removing all the benefits of lexical scoping. If you've ever created an inner function before, you know how powerful that technique can be. You can encode the entire state of the program into closures -- no need to store things in databases. Want a password reset link? Spin up a closure, email the link, done. Literally identical to storing a reset token in a database, except there's no database. Another solution is to fix the root problem. Does the closure really need a random ID every time the page refreshes? The closure links die because they have to be GC'd periodically, to keep memory usage down. Even if you cache the page for 90 seconds, that's still 960 refreshes per day for logged-out users. Then if you have a few hundred regular users, that's at least another factor of two. And certain pages might create hundreds of closure links each refresh, so it quickly gets out of hand. Ironically, the solution was emacs -- in emacs, they store closures in a printable way. A closure is literally a list of variable values plus the function's source code. That got me thinking -- why not use that as the key, instead of making a random key each time the page refreshes? After all, if the function's lexical variables are identical, then it should produce identical results each time it's run. No need to create another one. That's what I did. It took a week or so, which is a week I'll never get back for building new features. But at least users won't have to deal with dead links anymore. Clever readers will note a theoretical security flaw: an attacker might be able to guess your function IDs if they knew the entire state of the closure + the closure's source code (which is the default case for an open source project). That might give them access to e.g. your admin links. But that's not an indictment of the technique; it's easily solved by concatenating the key with a random ID generated at startup, and hashing that. I'm just making a note of it here in case some reader wants to try implementing this idea in their own framework. The closure technique has nontrivial productivity speedups (that I think someone will rediscover some years from now). I hope the idea becomes more popular over time.
- sillysaurusx 7y ago(Darn, I linked to the wrong comment. http://arclanguage.com/item?id=20996 http://arclanguage.com/item?id=20996 has the writeup about the function deduplication in Arc.)
- dang 7y agoI've swapped that for http://arclanguage.com/item?id=21046 http://arclanguage.com/item?id=21046 in your GP comment.