4 ms·

I don't want to enter into the details of the post, but it is impossible for me avoiding a reaction to this: "Do you honestly think that the PhDs at Google, Ama

by antirez 17y ago

I don't want to enter into the details of the post, but it is impossible for me avoiding a reaction to this: "Do you honestly think that the PhDs at Google, Amazon, Twitter, Digg, and Facebook created Cassandra, BigTable, Dynamo, etc. when they could have just used a RDBMS instead?" It is really impossible to argument something based on the fact that people that are supposed to be very smart are doing it. The only way to support arguments is by showing facts... shameless plug as I don't want to post a message just to say this, but isn't HN too slow lately? I'm at the point that I visit the site less often than I was used to do as I don't want to experience the delay.

blhack 17y ago
>shameless plug as I don't want to post a message just to say this, but isn't HN too slow lately? I'm at the point that I visit the site less often than I was used to do as I don't want to experience the delay. I've noticed that loading my comment and submission history is really slow, but loading everything else is as snappy as it's ever been. I have a feeling that the front-page and comments get heavily cached, while our comment and submission history does not.
- robconery 17y ago
  They should probably go NoSQL :)
  - ambition 17y ago
    There's no SQL behind Hacker News, the persistence code is custom and straight-to-files.
    - mistermann 17y ago
      In the true ThoughtWorks way. Sorry, couldn't resist. :)
fogus 17y ago
It is really impossible to argument something based on the fact that people that are supposed to be very smart are doing it. The only way to support arguments is by showing facts... I read that as the obligatory appeal to authority that seems to impress some people. The rest of the post however was extremely interesting and likely as fact-filled as it gets when it comes to these SQL vs. NoSQL arguments.
aphyr 17y ago
I think in some cases this kind of appeal to authority can be valid. Facebook has absolutely insane sparse matrices to handle. They handle enourmous volumes of traffic querying very specific (read: not cachable between users) datasets. Moreover, they've already invested mind-boggling amounts of capital into their stack. Same goes for Amazon with Dynamo. These people operate on scales that startups like us can't even comprehend; and they've found it worthwhile to write their own datastores for those scenarios. Moreover, their use of those databases has apparently contributed to their success. That, to me, is strongly suggestive evidence. That and HA/fault-tolerance is a no-brainer; Cassandra's scaling characteristics rock the socks off of any SQL DB I've used. The consistency tradeoff is well worth it for some use cases.
- wvenable 17y ago
  Absolutely. Facebook. Google. All great examples of the need for a different solutions. But I'm not sure about Digg. It seems like a very straight forward implementation would work for them. But given the small amount of information they've provided about their setup, it doesn't sound like they've ever gone for one. Compare them to StackOverflow, which at recent evidence, has about 10% of the traffic of Digg. They're running a very straight forward RDBMS configuration on rather pedestrian hardware. If Digg has a 50 node cluster (for example), StackOverflow should require at least a 5 node cluster.
  - aphyr 17y ago
    Yeah, I'm a little surprised that Digg has moved away for performance reasons. Maybe their data model is fundamentally more complex than StackOverflow? Or maybe SO has a better caching layer in front of the service?
    - mbreese 17y ago
      Or maybe Digg's model is flawed. I don't know if it is or not, but from everything I'd read it was far from optimal. I'd love to see more about it though. Now, relationship graph traversal is an issue for normalized relational systems, but in these cases, things could be split pretty from articles versus recommendations. One big problem I see in these comparisons is when a NoSQL person claims that their box is processing 5000 req/sec, what does that mean? Are they denormalizing this so much that it's equivalent to 500-1000 req/sec on a RDBMS? Another thing: when Digg was starting their type of site was very novel. There wasn't much out there that approached the scale and growth they experienced. I'm sure that StackOverflow has been designed with scaling in mind.
ubernostrum 17y ago
It is really impossible to argument something based on the fact that people that are supposed to be very smart are doing it. There's been a strong undercurrent of posts which basically consist of ad hominem "non-SQL databases are only for idiots who can't figure out how to manage an RDBMS properly". Pointing out that there are people who very definitely are not idiots and who can manage an RDBMS quite effectively, but who feel non-SQL databases are still appropriate for their use cases is, so far as I'm concerned, an acceptable rebuttal to that.
- VBprogrammer 17y ago
  Obviously its not a water-tight proof but its generally safe to assume a correlation between smart people and intelligent implementations.
  - torial 17y ago
    unless said smart people were in a committee :-)
    - mbreese 17y ago
      A person is smart. People are dumb.
deleted 17y ago
[deleted]

by antirez 17y ago

blhack 17y ago

>shameless plug as I don't want to post a message just to say this, but isn't HN too slow lately? I'm at the point that I visit the site less often than I was used to do as I don't want to experience the delay. I've noticed that loading my comment and submission history is really slow, but loading everything else is as snappy as it's ever been. I have a feeling that the front-page and comments get heavily cached, while our comment and submission history does not.

robconery 17y ago
They should probably go NoSQL :)
- ambition 17y ago
  There's no SQL behind Hacker News, the persistence code is custom and straight-to-files.
  - mistermann 17y ago
    In the true ThoughtWorks way. Sorry, couldn't resist. :)

fogus 17y ago

It is really impossible to argument something based on the fact that people that are supposed to be very smart are doing it. The only way to support arguments is by showing facts... I read that as the obligatory appeal to authority that seems to impress some people. The rest of the post however was extremely interesting and likely as fact-filled as it gets when it comes to these SQL vs. NoSQL arguments.

aphyr 17y ago

I think in some cases this kind of appeal to authority can be valid. Facebook has absolutely insane sparse matrices to handle. They handle enourmous volumes of traffic querying very specific (read: not cachable between users) datasets. Moreover, they've already invested mind-boggling amounts of capital into their stack. Same goes for Amazon with Dynamo. These people operate on scales that startups like us can't even comprehend; and they've found it worthwhile to write their own datastores for those scenarios. Moreover, their use of those databases has apparently contributed to their success. That, to me, is strongly suggestive evidence. That and HA/fault-tolerance is a no-brainer; Cassandra's scaling characteristics rock the socks off of any SQL DB I've used. The consistency tradeoff is well worth it for some use cases.

wvenable 17y ago
Absolutely. Facebook. Google. All great examples of the need for a different solutions. But I'm not sure about Digg. It seems like a very straight forward implementation would work for them. But given the small amount of information they've provided about their setup, it doesn't sound like they've ever gone for one. Compare them to StackOverflow, which at recent evidence, has about 10% of the traffic of Digg. They're running a very straight forward RDBMS configuration on rather pedestrian hardware. If Digg has a 50 node cluster (for example), StackOverflow should require at least a 5 node cluster.
- aphyr 17y ago
  Yeah, I'm a little surprised that Digg has moved away for performance reasons. Maybe their data model is fundamentally more complex than StackOverflow? Or maybe SO has a better caching layer in front of the service?
  - mbreese 17y ago
    Or maybe Digg's model is flawed. I don't know if it is or not, but from everything I'd read it was far from optimal. I'd love to see more about it though. Now, relationship graph traversal is an issue for normalized relational systems, but in these cases, things could be split pretty from articles versus recommendations. One big problem I see in these comparisons is when a NoSQL person claims that their box is processing 5000 req/sec, what does that mean? Are they denormalizing this so much that it's equivalent to 500-1000 req/sec on a RDBMS? Another thing: when Digg was starting their type of site was very novel. There wasn't much out there that approached the scale and growth they experienced. I'm sure that StackOverflow has been designed with scaling in mind.

ubernostrum 17y ago

It is really impossible to argument something based on the fact that people that are supposed to be very smart are doing it. There's been a strong undercurrent of posts which basically consist of ad hominem "non-SQL databases are only for idiots who can't figure out how to manage an RDBMS properly". Pointing out that there are people who very definitely are not idiots and who can manage an RDBMS quite effectively, but who feel non-SQL databases are still appropriate for their use cases is, so far as I'm concerned, an acceptable rebuttal to that.

VBprogrammer 17y ago
Obviously its not a water-tight proof but its generally safe to assume a correlation between smart people and intelligent implementations.
- torial 17y ago
  unless said smart people were in a committee :-)
  - mbreese 17y ago
    A person is smart. People are dumb.

deleted 17y ago

[deleted]