4 ms·

I honestly don't know why anyone would use Cascading or Scalding when Spark exists.

• saryant 11y ago
  Because they have legacy Hadoop to support?
  • kod 11y ago
    Spark interoperates with hdfs.
• evancasey 11y ago
  Huge spark fan here. Love the execution model, API, supporting libs etc. Unfortunately, Spark doesn't scale well on large datasets (10TB+). Sure, it's possible (and has been done), but right now there are too many rough edges to make it a better choice than Scalding/Cascading for data processing at scale. Most of this boils down to fine tuning certain Spark parameters, which is a pain when you're dealing with long-running, resource intensive workflows.
  • kod 11y ago
    You have any references to support that 10tb claim?
  • bsg75 11y ago
    What is the upper limit you have hit with Spark?
• fs111 11y ago
  Because Cascading does not break your apps by changing the API every minor release, that would be one. Also, being production grade software for many years is another.