5 ms·

Always wonder what kind of simple apps most people must write, if they not need joins? I will be happy if i got such simple tasks :)

by progx 11y ago

maze-le 11y ago
There is always the possibility to make a map/reduce over multiple datasets. It is not exactly a replacement for join, but you can cover aggregation over multiple datasets with a output-collection. Well... in my opinion this is much more complicated and error-prone than a join-operation on a relational database.
threeseed 11y ago
How exactly do you think eBay, GMail, Facebook etc work ? They aren't relying on relational database joins. If you want to write a truly scalable application you structure everything such that you do joins in your application layer. http://highscalability.com/ebay-architecture http://highscalability.com/ebay-architecture And in the case of MongoDB you avoid joins since it is a document database. You embed data instead.
- ngrilly 11y ago
  Not everybody works at eBay, GMail or Facebook scale. Most applications fit very well in a single server. For example, Stack Overflow runs on a single instance of SQL Server, replicated to a slave in another data center. In such a case, the convenience of joins and transactions is priceless. And even at scale, it makes sense to rely on joins and transactions. The perfect example is AdWords that runs of F1 and Spanner: "Our users needed complex queries and joins, which meant they had to carefully shard their data, and resharding data without breaking applications was challenging." http://static.googleusercontent.com/media/research.google.com/fr//pubs/archive/41344.pdf http://static.googleusercontent.com/media/research.google.co...
  - threeseed 11y ago
    Exactly. But many of us do have apps that are beyond the capabilities of a single instance. And in this situation the rule of thumb is to do joins in your application layer so you can store different types data (e.g. graph/document/relational/unstructured) in different systems and easily cache where needed. The fact that so many new databases have been created in the last decade suggests that there are a lot of people who do fall into this camp.
    - wcummings 11y ago
      There are also organizational requirements that push people away from joins (different teams responsible for warehousing different data in a large organization) and high performance applications which pre-summarize data (adtech).
    - theseoafs 11y ago
      A lot of us have apps that are beyond the performance requirements of Stack Overflow?
      - Pyxl101 11y ago
        Yes. It's not an especially exotic level of performance. Especially if you're talking about applications that handle traffic from other machines, as opposed to humans. As websites go, Stack Overflow is of reasonable size, but as systems go it's small. They have about 7.5m visits per day to Stack Overflow. That's about 86 requests per second, which perhaps at peak is several multiples larger than the average. I wouldn't consider that a particularly gargantuan website. That sounds like traffic that you could service with a reasonable fleet of web servers and database fleet, given caching. Quite a lot of actions on the site can be applied with eventual consistency, I'd imagine. http://stackexchange.com/sites?view=list#traffic http://stackexchange.com/sites?view=list#traffic I don't mean to say that the system or the problem that it's solving is trivial - I am sure it is difficult to get right. As websites go, it might be large, but as systems go it's not particularly high traffic among systems that receive traffic from machines rather than humans. Imagine that you operate a data center, and you want to sample CPU, memory, etc. from your machines every minute. If you collect 50 samples per machine per minute, and you have 258 machines, you'll be handling 86 samples per second. Storing 86 samples per second into a time series database is probably considerably easier than SO's website rendering, but it goes to show that high-traffic or high-frequency systems are common in companies beyond small to medium size. It is easy for cross-cutting concerns like this to have massive request volumes, far greater than the human-generated traffic to any website.

Always wonder what kind of simple apps most people must write, if they not need joins? I will be happy if i got such simple tasks :)

by progx 11y ago

Always wonder what kind of simple apps most people must write, if they not need joins? I will be happy if i got such simple tasks :)

maze-le 11y ago

There is always the possibility to make a map/reduce over multiple datasets. It is not exactly a replacement for join, but you can cover aggregation over multiple datasets with a output-collection. Well... in my opinion this is much more complicated and error-prone than a join-operation on a relational database.

threeseed 11y ago

How exactly do you think eBay, GMail, Facebook etc work ? They aren't relying on relational database joins. If you want to write a truly scalable application you structure everything such that you do joins in your application layer. http://highscalability.com/ebay-architecture http://highscalability.com/ebay-architecture And in the case of MongoDB you avoid joins since it is a document database. You embed data instead.

ngrilly 11y ago
Not everybody works at eBay, GMail or Facebook scale. Most applications fit very well in a single server. For example, Stack Overflow runs on a single instance of SQL Server, replicated to a slave in another data center. In such a case, the convenience of joins and transactions is priceless. And even at scale, it makes sense to rely on joins and transactions. The perfect example is AdWords that runs of F1 and Spanner: "Our users needed complex queries and joins, which meant they had to carefully shard their data, and resharding data without breaking applications was challenging." http://static.googleusercontent.com/media/research.google.com/fr//pubs/archive/41344.pdf http://static.googleusercontent.com/media/research.google.co...
- threeseed 11y ago
  Exactly. But many of us do have apps that are beyond the capabilities of a single instance. And in this situation the rule of thumb is to do joins in your application layer so you can store different types data (e.g. graph/document/relational/unstructured) in different systems and easily cache where needed. The fact that so many new databases have been created in the last decade suggests that there are a lot of people who do fall into this camp.
  - wcummings 11y ago
    There are also organizational requirements that push people away from joins (different teams responsible for warehousing different data in a large organization) and high performance applications which pre-summarize data (adtech).
  - theseoafs 11y ago
    A lot of us have apps that are beyond the performance requirements of Stack Overflow?
    - Pyxl101 11y ago
      Yes. It's not an especially exotic level of performance. Especially if you're talking about applications that handle traffic from other machines, as opposed to humans. As websites go, Stack Overflow is of reasonable size, but as systems go it's small. They have about 7.5m visits per day to Stack Overflow. That's about 86 requests per second, which perhaps at peak is several multiples larger than the average. I wouldn't consider that a particularly gargantuan website. That sounds like traffic that you could service with a reasonable fleet of web servers and database fleet, given caching. Quite a lot of actions on the site can be applied with eventual consistency, I'd imagine. http://stackexchange.com/sites?view=list#traffic http://stackexchange.com/sites?view=list#traffic I don't mean to say that the system or the problem that it's solving is trivial - I am sure it is difficult to get right. As websites go, it might be large, but as systems go it's not particularly high traffic among systems that receive traffic from machines rather than humans. Imagine that you operate a data center, and you want to sample CPU, memory, etc. from your machines every minute. If you collect 50 samples per machine per minute, and you have 258 machines, you'll be handling 86 samples per second. Storing 86 samples per second into a time series database is probably considerably easier than SO's website rendering, but it goes to show that high-traffic or high-frequency systems are common in companies beyond small to medium size. It is easy for cross-cutting concerns like this to have massive request volumes, far greater than the human-generated traffic to any website.