5 ms·

I sincerely advice against using node. (I'll get to that in a bit). For the DB, I personally like CouchDB. It's easy to get started with, and it lends itself ve

by hasenj 14y ago

I sincerely advice against using node. (I'll get to that in a bit). For the DB, I personally like CouchDB. It's easy to get started with, and it lends itself very well to horizontal scaling. As for why node is a bad choice: you can think of 2 categories of languages: 1) Those which allow you to get things done quickly but in which large code bases are hard to maintain because you need to carry too much information in your head, and they possibly don't scale very well because they tend to run in an interpreter (python, ruby, lisp). 2) Those which require some amount of boilerplate to get simple things done, but provide structure and modularity so it's easier to maintain large code bases, and they possibly produce high-performance code because they tend to be compiled or almost-compiled (C#, Java, Go). There's a trade off between "getting up to speed quickly" and "maintainable performant code". Node is bad because it fails at getting you up to speed quickly AND it doesn't make your code maintainable at large scale. This is because async-style programming is not natural to how we think. The "performance" should not be taken as a plus for Node because you can get it in other languages/platforms (Go, for example) without sacrificing maintainability.

aaronblohowiak 14y ago
The node debugging experience is quite nice. Unfortunately, I found myself having to use it. The issues with carrying around information in your head is true IFF you have a large monolithic app. SOA, which is advocated with compiled languages as well, helps to avoid this issue because each service is limited in scope. Then again, SOA itself is counter to "hello world" latency... I think this is important because if you find yourself dealing with a (example) rails app that is experiencing cognitive burden overload issues, splitting it out into a SOA can be a nice transition (eventually replacing parts with other languages if it is required.) Also, if you care about performance and quick execution and want a scripting environment, luvit is much faster than node, much lower memory usage and IIRC you can use lua continuations instead of async style. IMHO, node.js as an edge server providing connection handling and templating with backend services giving it JSON seems like a reasonable 3-tier architecture, especially because your edge tier can then be maintained by people with front-end experience.
- hasenj 14y ago
  What's SOA? And how does it help with cognitive overload issues?
  - aaronblohowiak 14y ago
    Service-Oriented Architecture. It helps because each service is self-contained, and only communicates with other services through defined interfaces and known boundaries. Because they are self contained, you then only have to have the mental overhead of the complexity of the service you are currently editing (so long as you do not break any of its contracts.) IE: Encapsulation and isolation, which is something that "scripting" languages generally stink at enforcing (on purpose!)
    - slurgfest 14y ago
      Besides shell and perl (which really do excel at a certain kind of task), calling a language a 'scripting language' is usually not much more than a dismissive insult by someone with a heavy investment in something like Java, which seems more serious mostly because it involves a lot of self-important boilerplate and has high levels of adoption by big old companies where you have to wear a tie.
      - aaronblohowiak 14y ago
        no insult. i am deeply invest in ruby and js. calling a scripting language in my book means no a priori verification of the code before beginning to run it. Of course this is a spectrum from Coq to Ruby..
- bmuon 14y ago
  > IMHO, node.js as an edge server providing connection handling and templating with backend services giving it JSON seems like a reasonable 3-tier architecture, especially because your edge tier can then be maintained by people with front-end experience. I have yet to try it, but this seems like a great approach. Manage most of the computing intensive tasks in whatever environment you like/need. Use Node to talk to APIs and render templates. With a well developed client framework you can even make only the first render in Node and the following in the client talking to the same APIs. This leaves the most complicated back-end problems in the hands of developers that like and understand those problems and creates a friendly environment for the front-end developer who usually focuses on building features. The front-end developer even has already some experience with async code because of events in the browser, so he'll probably get up to speed quickly with Node.
rdtsc 14y ago
Completely agree with the CouchDB choice. It has really elegant and unique features but combined with rock solid durability. You can configure it to fsync after every update it does as much as it possibly can together with the OS to make sure you data reaches the disk before returning a result. That is very important if you value your data. As for other features, I like the REST-ful interface, a continuous changes feed, master to master replication, and a usable interface to see and manage data -- Futon. I haven't yet found a product that comes with all those features.
- hasenj 14y ago
  Another thing I consider a great feature of CouchDB is the cached map-reduce views. It makes it easy to think and reason about indexing. I've seen this situation quite a bit with traditional SQL databases: somehow you end up with a giant complicated query, and it's becoming a bottle-neck and it's having a terrible impact on performance. Somehow you need to untangle the mess and figure out where the problem is. Sometimes you find out the query is just poorly written. Other times the problem is fixed by simply adding indexes to some table. This is a terrible situation I don't want to run into. I have a feeling MongoDB might also suffer from a smilar problem, because it doesn't have cached map-reduce views; it has a query language. In CouchDB, views are pretty much indexes, and they're pretty much the only way to do a query. I'd argue that because it's explicit like that, it's easy to reason about.
  - sbisbee 14y ago
    Yeah, we call it Incremental MapReduce and we _LOVE_ it. I swear, it's one of the biggest "ah ha!" moments whenever I do training for Cloudant's customers. The basic idea is that the intermediate Map function results are persisted, while most other system (read: hadoop) throws them out. If you're interested in some of the MapReduce history, where it's going, incremental indexing, etc., then you should read this: http://gigaom.com/cloud/why-the-days-are-numbered-for-hadoop-as-we-know-it/ http://gigaom.com/cloud/why-the-days-are-numbered-for-hadoop... Cheers.
- Andrex 14y ago
  Yeah, Couch is seriously underrated. HTTP API, Master-Master, append-only, these are things to love if you cherish your web app's data. I wish Couchbase hadn't abandoned the Apache version, or rather, I wish someone else had stepped in to take their place as "the Couch company" the same way 10gen is for Mongo. Cloudant is close, but their BigCouch stuff still isn't merged yet as far as I know, so if you go with them you're still using some weird Couch fork instead of the open source version. Which might be more of a psychological problem than an actual one, but it makes me really uneasy about the Couch ecosystem they're the only thing we have that's close to a gold-standard champion.
slurgfest 14y ago
There are all kinds of considerations with respect to node. But discounting all interpreted languages because they are interpreted is crazy biased. I don't at all accept (and it is not self-evident) that a high-boilerplate compiled language is necessary to have a large codebase or acceptable performance. Unless most of your storage is in memcached, the database is far more likely to be a bottleneck than your interpreter (!) There are a number of compilers for Lisp. Between these two issues with your post, I would not be convinced to accept your advice regarding node, either.
- hasenj 14y ago
  I'm aware that lisp can be compiled, that's why I said that dynamic languages "tend to be" interpreted (the implication being, they're not always interpreted). My point is about comparing dynamic and non-dynamic languages. Dynamic languages tend to have "magic" where complex logic occurs without you explicitly declaring it, at the expense of either a) maintainability or b) performance or c) both There's a trade off here: varying degrees of magic vs varying degrees of boilerplate. Boilerplate code can make things very explicit and thus easier to reason about. Now, NodeJS has the worst of both worlds. The callback mess is not easy to write, not easy to reason about, not easy to maintain, and there's no magic. That's a lot of sacrifice, it better be for good gain. But exactly do you gain? I don't see any gains. Performance? You can get that with other languages/platforms (like Go for example) without sacrificing code clarity.

by hasenj 14y ago

aaronblohowiak 14y ago

The node debugging experience is quite nice. Unfortunately, I found myself having to use it. The issues with carrying around information in your head is true IFF you have a large monolithic app. SOA, which is advocated with compiled languages as well, helps to avoid this issue because each service is limited in scope. Then again, SOA itself is counter to "hello world" latency... I think this is important because if you find yourself dealing with a (example) rails app that is experiencing cognitive burden overload issues, splitting it out into a SOA can be a nice transition (eventually replacing parts with other languages if it is required.) Also, if you care about performance and quick execution and want a scripting environment, luvit is much faster than node, much lower memory usage and IIRC you can use lua continuations instead of async style. IMHO, node.js as an edge server providing connection handling and templating with backend services giving it JSON seems like a reasonable 3-tier architecture, especially because your edge tier can then be maintained by people with front-end experience.

hasenj 14y ago
What's SOA? And how does it help with cognitive overload issues?
- aaronblohowiak 14y ago
  Service-Oriented Architecture. It helps because each service is self-contained, and only communicates with other services through defined interfaces and known boundaries. Because they are self contained, you then only have to have the mental overhead of the complexity of the service you are currently editing (so long as you do not break any of its contracts.) IE: Encapsulation and isolation, which is something that "scripting" languages generally stink at enforcing (on purpose!)
  - slurgfest 14y ago
    Besides shell and perl (which really do excel at a certain kind of task), calling a language a 'scripting language' is usually not much more than a dismissive insult by someone with a heavy investment in something like Java, which seems more serious mostly because it involves a lot of self-important boilerplate and has high levels of adoption by big old companies where you have to wear a tie.
    - aaronblohowiak 14y ago
      no insult. i am deeply invest in ruby and js. calling a scripting language in my book means no a priori verification of the code before beginning to run it. Of course this is a spectrum from Coq to Ruby..
bmuon 14y ago
> IMHO, node.js as an edge server providing connection handling and templating with backend services giving it JSON seems like a reasonable 3-tier architecture, especially because your edge tier can then be maintained by people with front-end experience. I have yet to try it, but this seems like a great approach. Manage most of the computing intensive tasks in whatever environment you like/need. Use Node to talk to APIs and render templates. With a well developed client framework you can even make only the first render in Node and the following in the client talking to the same APIs. This leaves the most complicated back-end problems in the hands of developers that like and understand those problems and creates a friendly environment for the front-end developer who usually focuses on building features. The front-end developer even has already some experience with async code because of events in the browser, so he'll probably get up to speed quickly with Node.

rdtsc 14y ago

Completely agree with the CouchDB choice. It has really elegant and unique features but combined with rock solid durability. You can configure it to fsync after every update it does as much as it possibly can together with the OS to make sure you data reaches the disk before returning a result. That is very important if you value your data. As for other features, I like the REST-ful interface, a continuous changes feed, master to master replication, and a usable interface to see and manage data -- Futon. I haven't yet found a product that comes with all those features.

hasenj 14y ago
Another thing I consider a great feature of CouchDB is the cached map-reduce views. It makes it easy to think and reason about indexing. I've seen this situation quite a bit with traditional SQL databases: somehow you end up with a giant complicated query, and it's becoming a bottle-neck and it's having a terrible impact on performance. Somehow you need to untangle the mess and figure out where the problem is. Sometimes you find out the query is just poorly written. Other times the problem is fixed by simply adding indexes to some table. This is a terrible situation I don't want to run into. I have a feeling MongoDB might also suffer from a smilar problem, because it doesn't have cached map-reduce views; it has a query language. In CouchDB, views are pretty much indexes, and they're pretty much the only way to do a query. I'd argue that because it's explicit like that, it's easy to reason about.
- sbisbee 14y ago
  Yeah, we call it Incremental MapReduce and we _LOVE_ it. I swear, it's one of the biggest "ah ha!" moments whenever I do training for Cloudant's customers. The basic idea is that the intermediate Map function results are persisted, while most other system (read: hadoop) throws them out. If you're interested in some of the MapReduce history, where it's going, incremental indexing, etc., then you should read this: http://gigaom.com/cloud/why-the-days-are-numbered-for-hadoop-as-we-know-it/ http://gigaom.com/cloud/why-the-days-are-numbered-for-hadoop... Cheers.
Andrex 14y ago
Yeah, Couch is seriously underrated. HTTP API, Master-Master, append-only, these are things to love if you cherish your web app's data. I wish Couchbase hadn't abandoned the Apache version, or rather, I wish someone else had stepped in to take their place as "the Couch company" the same way 10gen is for Mongo. Cloudant is close, but their BigCouch stuff still isn't merged yet as far as I know, so if you go with them you're still using some weird Couch fork instead of the open source version. Which might be more of a psychological problem than an actual one, but it makes me really uneasy about the Couch ecosystem they're the only thing we have that's close to a gold-standard champion.

slurgfest 14y ago

There are all kinds of considerations with respect to node. But discounting all interpreted languages because they are interpreted is crazy biased. I don't at all accept (and it is not self-evident) that a high-boilerplate compiled language is necessary to have a large codebase or acceptable performance. Unless most of your storage is in memcached, the database is far more likely to be a bottleneck than your interpreter (!) There are a number of compilers for Lisp. Between these two issues with your post, I would not be convinced to accept your advice regarding node, either.

hasenj 14y ago
I'm aware that lisp can be compiled, that's why I said that dynamic languages "tend to be" interpreted (the implication being, they're not always interpreted). My point is about comparing dynamic and non-dynamic languages. Dynamic languages tend to have "magic" where complex logic occurs without you explicitly declaring it, at the expense of either a) maintainability or b) performance or c) both There's a trade off here: varying degrees of magic vs varying degrees of boilerplate. Boilerplate code can make things very explicit and thus easier to reason about. Now, NodeJS has the worst of both worlds. The callback mess is not easy to write, not easy to reason about, not easy to maintain, and there's no magic. That's a lot of sacrifice, it better be for good gain. But exactly do you gain? I don't see any gains. Performance? You can get that with other languages/platforms (like Go for example) without sacrificing code clarity.