5 ms·
That's what happens when you make something in a new and unproven language. And I think that it deserves tremendous respect that you have done so. You could ha
by mixmax 18y ago
That's what happens when you make something in a new and unproven language.
And I think that it deserves tremendous respect that you have done so. You could have hacked HN in python (or lisp?) in a weekend with the knowledge that it would scale, yet you decided to go with Arc. Very bold. And a great way to battletest a new language.
It's great to see people eat their own dogfood.
- dylanz 18y agoWow, had no idea this was written in Arc. +1 to what mixmax said. It's the only way to keep pushing the envelope.
- mixmax 18y agoBoldness drives innovation
- Jebdm 18y agoIf I'm not mistaken, "to battletest a new language" was the purpose of building this.
- rw 18y agoKeep in mind that mzscheme is battle tested. Arc is "just" a bunch of macros on top of PLT Scheme; what gives HN its [under]performing character is that foundational JIT compiler and 3m GC.
- pg 18y agoArc compiles into MzScheme, but it's not implemented as macros. You can see that from the source.
- elibarzilay 18y agoIn general, 3m is performing very well, much better than the conservative collector plt used to use by default (the boehm gc). As for the jit: many of the benefit of the jit are irrelevant in Arc since it doesn't use mzscheme modules; this is in addition to using an old mzscheme version, when the jit was rather new -- many many improvements were done since then.
- jrockway 18y agoI doubt Arc is the cause of any problems. I think this data model would be problematic in any language. You can't just let your processes die when you are low on memory, you need to evict unused pages from the cache.
- lacker 18y agoWell, in a language with existing web frameworks it would be convenient to use one of them and store your data with SQL. In Arc you would have to write your own SQL wrapper. I suspect if pg was not using Arc he would not be keeping everything in memory. That said, the lisp world could use a good web framework, so perhaps this will inspire some lispers to create one.
- jrockway 18y agoI suspect if pg was not using Arc he would not be keeping everything in memory. I don't want to speak for pg, but I highly doubt this. He chose the architecture he did for good reason. Looking things up in memory is much faster than looking them up in the database. The only issue is that the site is too big to fit entirely into memory now, so he needs to write something that cleanly manages the memory space. (BerkeleyDB calls this a pager; data is stored in pages that can be in memory or disk.) With that in place, the site should perform fine.
- pg 18y agoI would be keeping everything in memory (using lazy loading) regardless. It makes life simpler.
- lacker 18y agoYou wouldn't use SQL? Something like django or rails is pretty handy if you are already using those languages, as annoying as it can be.
- jrockway 18y agoSQL databases are a very poor fit for sites like news.yc. Things like threaded comments, for example, are hard to model efficiently. If you use an object database (or roll your own in-memory thing), though, it is very easy to model. Relational databases are certainly useful in some situations, but there are usually better options for websites. You should consider learning about other options before telling everyone to use Rails. Rails is not the only way to do things, and it's rarely the best way.