5 ms·
Arc is underrated as an information management tool. There's something to be said for having a web framework that works out of the box. Rails is probably the on
by sillysaurusx 7y ago
Arc is underrated as an information management tool. There's something to be said for having a web framework that works out of the box. Rails is probably the only other framework that makes it as easy to "just make some forms that pass data around and run some code on that data." But not quite -- I haven't seen arc's closure-storing technique used in any other web framework.
The main issue that arc solves is that it gives you a full pipeline for managing "objects with properties" via the web. It's so flexible. I wrote a thread on how we're using it in production to manage our TPUs: https://twitter.com/theshawwn/status/1247570883306119170 https://twitter.com/theshawwn/status/1247570883306119170
- throwanem 7y agoYou haven't been around long enough, at least under the same name, to remember when there was an implicit time limit on the comment reply page, inflicted by those stored closures silently timing out. Having long comments so often eaten that way was actually the specific thing that annoyed me into first installing It's All Text. It's an interesting approach, as attempts to force statefulness on a stateless-by-design protocol go, but I don't know that I like how it scales.
- sillysaurusx 7y agoMy account was reset. I've been around since day two. :) You're right, it has some downfalls. But a lot of the time it simply doesn't matter. All the links at https://www.tensorfork.com/tpus https://www.tensorfork.com/tpus are dynamic, and the speed you gain by being able to whip up a feature in 10 minutes is worth the pain of an occasional dead link. On the other hand, I did some work for "deduplicating fnids": http://arclanguage.com/item?id=20996 http://arclanguage.com/item?id=20996 which the site is using, so the links possibly last much longer than the early-HN links. (Basically, we calculate the fnid key based on the lexical environment, rather than using a random ID each time the page is loaded. So each link gets a unique ID based on the code structure rather than a random ID. Meaning, instead of millions of random links to store, you end up with a few tens of thousands.)
- DonHopkins 7y agoAre fnids anything like Fnords? https://en.wikipedia.org/wiki/Fnord https://en.wikipedia.org/wiki/Fnord Or are they more like Vermicious Knids? https://roalddahl.fandom.com/wiki/Vermicious_Knids https://roalddahl.fandom.com/wiki/Vermicious_Knids
- sillysaurusx 7y agoIt's short for function ID. If you want to route requests to specific closures, you have to have some sort of ID that you can send down to the user. Arc stores closures in a hash table keyed by random ID, but we use lexical structure plus lexical values (like username) to make the key deterministic. It greatly cut down on the number of closures that needed to be stored. Basically, if you have a closure that does a certain action – e.g. editing a comment – Arc generates new a closure every page refresh. That was the root reason for the "dead link problem" during the early days of HN. I reworked it to make the closure IDs deterministic. If you walk the source code from the point of the closure up to the root, and then you stick all the local variables in a list and use that as a key, then hash that, along with the actual source code, you end up with something that (a) has a very low probability of collision, and (b) is deterministic each time the page refreshes, unless the variables' values change. (E.g. if you store the current time in a local variable, the value changes, so the fnid will change as well, since that value is captured by the closure and therefore the closure has to do something different by definition.)
- throwanem 7y agoThat sounds like a complicated fix! How many tens of minutes did it take? And how many tens of minutes did the bug cost, in effort spent on comments that got eaten instead of posted? (I know there's no way to answer that last question, but that doesn't mean its answer is equal to zero.)
- throwanem 7y ago(Ah, never mind, I see this line of thought has already been explored nearby. I've nothing to add beyond what's already been said there.)
- senderista 7y agoI think Seaside had the same issues with scaling.
- e12e 7y agoAnd I believe similar closures for keeping state?
- senderista 7y agoYes. An elegant approach with poor scalability and availability characteristics.
- dang 7y agoHN still uses it extensively for its more obscure operations—the ones that don't need to scale. We switched all the most common handlers to regular links years ago. It's hard to remember now that the most common complaint here (by far) used to be about those "Unknown or expired link" messages. In fact, you can tell from the histogram of https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=%22Unknown%20or%20expired%20link%22&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... when it was that we fixed this—six years ago—because that's when the complaints slow down to a trickle. As long as you don't use it for things that you need a lot of, it's a great approach that holds up well. The primary downside is that they all get discarded when we restart the server process. Another downside is that they don't work well with the back button. Edit: I just remembered another issue with them. Sometimes browsers pre-visit these links, which 'uses' them so that by the time the user goes to click on it, it has already expired. (Yes, this is an issue with using GET for these.)
- sillysaurusx 7y agoThe primary downside is that they all get discarded when we restart the server process. For solving this, http://arclanguage.com/item?id=20996 http://arclanguage.com/item?id=20996 has some theoretical ideas. It's way easier in elisp than in racket, but, the idea is to write out the function + the closure variables to disk. The hard part is that you'd have to fix up object references. (If two closures both capture x, and x is a hash table, then the values of x for both closures should be the same hash table after a reboot.) But it's doable. And of course, if it's possible to write the closures to disk, that means you can write them out to a database shared by multiple Arc webservers. As long as the state is also shared (perhaps the values of the variables can also be stored in the database?) then this means the technique can horizontally scale, just like any other. I spent like a year trying to brainstorm ways of showing that Arc can go toe-to-toe with any of the popular frameworks, with no downsides. "Reboots wipe the closures" implies "Arc can't scale horizontally," which would be a serious limitation in a corporate setting. But in principle I think you could write closures to disk. That would also result in a funny situation: if closures persist forever, it means that a closure could potentially be activated years after it was first stored. So it'll run with years-old code, rather than the latest version. :) But if people are using global names for functions, then it'll just call the latest versions of those functions, which will probably work fine in most cases.
- drcode 7y agoSome arc code I wrote back in the day, for generating textual descriptions from structured data using a web frontend, is still in production at a previous company of mine (as far as I know). It was indeed a useful tool. It's a shame arc didn't have persistent data structures (besides alists :-) and a native hashmap type though. For those confused, we're talking about this: http://www.paulgraham.com/arc.html http://www.paulgraham.com/arc.html
- DonHopkins 7y agoI'm sure somebody has written an ADVENTURE front-end for a LISP debugger. You are in a CONS cell. The there is a CAR to the left, leading off to another CONS, and a CDR to the right, containing NIL. The garbage collector briefly enters the CONS, marks it, quickly glances at the NIL in the CDR, smiles quietly to itself, and looking relieved, hurriedly sweeps away through the CAR. >GIVE CONS TO CDR You give the CONS cell to its CDR, creating a circular list.
- q92z8oeif 7y agoproblem with those frameworks that "just works" is that they get old pretty darn fast. it handles all your sql and auth cookies? too bad now there's an easy way every script kiddie can now login as admin or guess cookies without the security header du jour. It's all nice and all when it's being actively updated. But arc, rails, phoenix-ecto, node/react, drupal, spring, etc it all get old pretty soon when the core maintainers loose focus, and then instead of just keeping an eye on the latest best practices and implementing it yourself, you have to dive deep down into years of feature creep and bad coding practices to do the little thing you need to keep things going.
- saurik 7y ago> I haven't seen arc's closure-storing technique used in any other web framework. Apache Cocoon? https://cocoon.apache.org/2.2/core-modules/core/2.2/1373_1_1.html https://cocoon.apache.org/2.2/core-modules/core/2.2/1373_1_1...
- riffraff 7y agoI am ready to be corrected but I'm a fairly sure there have been a few other continuation-based web frameworks, Seaside in Smalltalk was the one that made the idea popular if I recall correctly. "Href considered harmful" comes from that.
- toolslive 7y agoI don't know Arc, but it seems you're referring to 'continuation based web frameworks'? They have been around since the dawn of the web. Here's criticism from 2006: http://www.interact-sw.co.uk/iangblog/2006/05/21/webcontinuations http://www.interact-sw.co.uk/iangblog/2006/05/21/webcontinua... Truth be told, the criticism is a bit unfair (you can solve the problems)
- rlander 7y agoDo you have any links to resources or blog posts explaining how to do web dev in Arc? A cursory google search turns up ArcGIS stuff.
- zem 7y agoi think seaside (a smalltalk web framework) pioneered the continuation-based technique, though it could have been around earlier. http://www.seaside.st/ http://www.seaside.st/ racket's built-in web server does it too.
- kgwgk 7y agoReferences to some previous uses: https://thelackthereof.org/docs/library/cs/continuations.pdf https://thelackthereof.org/docs/library/cs/continuations.pdf Including pg’s ViaWeb written in continuation-passing style: https://sep.yimg.com/ty/cdn/paulgraham/bbnexcerpts.txt https://sep.yimg.com/ty/cdn/paulgraham/bbnexcerpts.txt