4 ms·

This is a fair criticism and I think we used unclear phrasing here. It's not that we don't consider the JVM to be a modern toolchain in general. Rather we think

This is a fair criticism and I think we used unclear phrasing here. It's not that we don't consider the JVM to be a modern toolchain in general. Rather we think it's not a modern toolchain for implementing MapReduce because it forces users to work in a JVM compatible way. Docker is a better solution because it gives users the freedom to use what they're already using.

valarauca1 12y ago
Doesn't this approach make users send 1GB+ Docker images instead of small 5-10MB .jar files?
- jdoliner 12y ago
  Useful Docker images can actually be made quite a bit smaller than 1GB+ by using small distros like busybox and following best practices such as cleaning up temporary data between commits. That being said this is a legitimate concern because it affects how tight a developers loop is so it's something we'll probably have to address longterm.
- coffeemug 12y ago
  That sounds relatively easy to fix. I'd imagine a diff between two Docker images is relatively small compared to the size of the image. I'd imagine a deduplication scheme based on widely available tools (e.g. rsync) would solve this problems fairly easily. (Though I have no idea how hard it would be to implement; it doesn't sound too hard off the top of my head)
  - jdoliner 12y ago
    This is a very good point. In addition Docker itself tries to address this by using btrfs's diffing. Like a lot of things in Docker it's not perfect but it's getting there.
teacup50 12y ago
How is schlepping around non-portable Linux-specific binaries packaged into what is essentially a VM a "more modern toolchain"?
- shykes 12y ago
  (just my personal opinion below, I'm not affiliated with the grand-parent) The tooling around the JVM is much more mature the Docker ecosystem in a lot of ways. However, one way to justify the "more modern" label is that Docker can be used to package and manage every component of a Linux-based application stack, whereas JVM tools can only run the JVM-backed pieces. The modern application architecture is decisively service-oriented and polyglot, which makes a JVM-only toolbox less relevant.

valarauca1 12y ago

Doesn't this approach make users send 1GB+ Docker images instead of small 5-10MB .jar files?

jdoliner 12y ago
Useful Docker images can actually be made quite a bit smaller than 1GB+ by using small distros like busybox and following best practices such as cleaning up temporary data between commits. That being said this is a legitimate concern because it affects how tight a developers loop is so it's something we'll probably have to address longterm.
coffeemug 12y ago
That sounds relatively easy to fix. I'd imagine a diff between two Docker images is relatively small compared to the size of the image. I'd imagine a deduplication scheme based on widely available tools (e.g. rsync) would solve this problems fairly easily. (Though I have no idea how hard it would be to implement; it doesn't sound too hard off the top of my head)
- jdoliner 12y ago
  This is a very good point. In addition Docker itself tries to address this by using btrfs's diffing. Like a lot of things in Docker it's not perfect but it's getting there.

teacup50 12y ago

How is schlepping around non-portable Linux-specific binaries packaged into what is essentially a VM a "more modern toolchain"?

shykes 12y ago
(just my personal opinion below, I'm not affiliated with the grand-parent) The tooling around the JVM is much more mature the Docker ecosystem in a lot of ways. However, one way to justify the "more modern" label is that Docker can be used to package and manage every component of a Linux-based application stack, whereas JVM tools can only run the JVM-backed pieces. The modern application architecture is decisively service-oriented and polyglot, which makes a JVM-only toolbox less relevant.