4 ms·

It also doesn't say anything. It'd be nice if he explained a little more of his position. A common thing I hear is that Rails simply puts the problem of scale i

by j2d2 18y ago

It also doesn't say anything. It'd be nice if he explained a little more of his position. A common thing I hear is that Rails simply puts the problem of scale into the database. The problem may not be a Rails problem, but the design of Rails doesn't solve the problem either. It just puts it somewhere else. I have not worked with Rails, though. This is just what I've read. I'd prefer if an insider like Evan Weaver would elaborate a little more on what problems they had and how they solved them. All I know is that some people say Rails is a problem and they have evidence for that and this guy says it's not and suggests nothing more.

nostrademons 18y ago
Many folks would say the database is where scaling problems belong. Lots and lots of programmer hours have already been expended in figuring out how to scale databases; no sense duplicating that effort in scaling your web framework. I'd say Twitter's problem is that they're a multicast messaging app with a database backend, a combination that tends not to work so well. As an intern in college, I worked on a financial J2EE app that tried to do everything through JMS messaging queues backed by Oracle. It had similar scalability and performance problems. In Twitter's defense, they didn't know they'd be a multicast messaging app when they started, and a database is a logical choice for what they did know they'd be (a website). They'll figure things out; they just need time and resources to rearchitect.
- j2d2 18y ago
  Many folks would say the database is where scaling problems belong. This makes sense but I've found the db to be a rather large bottleneck. On one hand, it's nice to store everything in a single blackbox and the apps don't have to care about data. On the other, databases can be slow and I figure some kind of caching is involved between them. I could see that a site would simply use the db to generate static pages which would then be cached, but as you said, multicast messaging with a database backend seems like a tricky combination. Edit: I realize I've kinda thought outloud here. I've summed up the general idea with the question below. I take the problem to be that going to the actual database is necessary. What are the possible solutions for something like this? Horizontal partitioning seems like the obvious answer. Is it that easy, though?
  - LogicHoleFlaw 18y ago
    Horizontal partitioning depends on your data. Do you have lots of joins and interdependent data? Partitioning will be tricky. Do you just need to store a lump of data relevant to one customer, many times over? Horizontal partitioning is easy. Relational databases are really cool. They're not the right technology for all apps though. The big wave-makers right now are things like Google's BigTable and Amazon's SimpleDB. They're implicitly horizontal but with reduced querying abilities compared to traditional SQL. It's a hard problem to scale a traditional relational DB across multiple independent commodity boxes like we see at Google or with AWS. The "stuff everything in a giant hashtable" approach scales nicely across a bunch of cheap boxes.

by j2d2 18y ago

nostrademons 18y ago

Many folks would say the database is where scaling problems belong. Lots and lots of programmer hours have already been expended in figuring out how to scale databases; no sense duplicating that effort in scaling your web framework. I'd say Twitter's problem is that they're a multicast messaging app with a database backend, a combination that tends not to work so well. As an intern in college, I worked on a financial J2EE app that tried to do everything through JMS messaging queues backed by Oracle. It had similar scalability and performance problems. In Twitter's defense, they didn't know they'd be a multicast messaging app when they started, and a database is a logical choice for what they did know they'd be (a website). They'll figure things out; they just need time and resources to rearchitect.

j2d2 18y ago
Many folks would say the database is where scaling problems belong. This makes sense but I've found the db to be a rather large bottleneck. On one hand, it's nice to store everything in a single blackbox and the apps don't have to care about data. On the other, databases can be slow and I figure some kind of caching is involved between them. I could see that a site would simply use the db to generate static pages which would then be cached, but as you said, multicast messaging with a database backend seems like a tricky combination. Edit: I realize I've kinda thought outloud here. I've summed up the general idea with the question below. I take the problem to be that going to the actual database is necessary. What are the possible solutions for something like this? Horizontal partitioning seems like the obvious answer. Is it that easy, though?
- LogicHoleFlaw 18y ago
  Horizontal partitioning depends on your data. Do you have lots of joins and interdependent data? Partitioning will be tricky. Do you just need to store a lump of data relevant to one customer, many times over? Horizontal partitioning is easy. Relational databases are really cool. They're not the right technology for all apps though. The big wave-makers right now are things like Google's BigTable and Amazon's SimpleDB. They're implicitly horizontal but with reduced querying abilities compared to traditional SQL. It's a hard problem to scale a traditional relational DB across multiple independent commodity boxes like we see at Google or with AWS. The "stuff everything in a giant hashtable" approach scales nicely across a bunch of cheap boxes.