3 ms·

Yes, we need new, standardizing APIs. However this made me cringe: > Exposing machines as the abstraction to developers unnecessarily complicates the engineeri

Yes, we need new, standardizing APIs. However this made me cringe: > Exposing machines as the abstraction to developers unnecessarily complicates the engineering, causing developers to build software constrained by machine-specific characteristics, like IP addresses and local storage. It reminded me of the old RPC approach of making potentially any method call a remote method call. It didn't work out, simply because remote call latencies are orders of magnitudes higher than for in-process calls, so you need different (coarser) APIs for them. By the same token, while abstractions are very welcome, they should still allow distinctions between local and non-local resources, for various definitions of "local".

dllthomas 12y ago
While I take your point, "local" is a leaky concept inside a data-center. On a fast network, "in cache on the neighboring computer" can be closer than "on my hard drive" for most purposes. It seems like what might be ideal would be a specification language that doesn't care where things are, an implementation that tries to deal with that automagically, and a way to specify portions (to all) precisely that is checked against the high-level specification.
- thrownaway2424 12y ago
  Absolutely. In-core on a neighboring machine can be accessed in less than 1ms. On-disk on the local machine (or any machine) can take tens of seconds.
  - chetanahuja 12y ago
    "On-disk on the local machine (or any machine) can take tens of seconds" Surely you meant 10's of milliseconds. But even so, fastest SSD random access latencies are in sub-millisecond ranges.
    - thrownaway2424 12y ago
      No I did not mean that. Contended local disk access under heavy random read/write workloads takes essentially forever.
- SEJeff 12y ago
  To further your point, you can get faster access directly via RDMA on an Infiniband network than the SATA bus can push data. The fastest SATA busses are around 16gbps I believe and you can get HDR IB switches that clock 50gbps today. With the direction things are moving, "the data center as a computer" is absolutely the right approach.
presspot 12y ago
One thing worth noting is that the indirection Ben describes is, basically, the indirection that is already in the Apache Mesos kernel. In real world environments (e.g., Twitter), there is negligible overhead. Also, locality and latency can both be expressed in terms of placement rules and schedulers on Mesos can use those rules to guarantee or express preference for task placement that optimizes around reduced latencies. The point that I take away is that the ability to express your needs in a declarative way (e.g., "place these two tasks such that they have such-and-such latency) is much more scalable, flexible and resilient than coding to machine-specific internals. The latter is easier to update and supports delegation of responsibilities. John Wilkes of Google puts it this way: "Our own experience has been that allowing our developers unfettered access to the internals of infrastructure systems has been a problem, and we're moving away from that model as fast as we can. Constructing large-scale complex systems with many interdependencies leads to brittle, fragile systems if they rely on internal implementation mechanisms. Allowing internal customers to rely on internal implementation mechanisms has made it hard to adopt new technologies, because we only know what knobs they set - not why. The fix for both is similar: describe the desired end state, not how to get there."
- perlgeek 12y ago
  > The point that I take away is that the ability to express your needs in a declarative way (e.g., "place these two tasks such that they have such-and-such latency) is much more scalable, flexible and resilient than coding to machine-specific internals. The latter is easier to update and supports delegation of responsibilities. That makes total sense to me, thanks.

dllthomas 12y ago

While I take your point, "local" is a leaky concept inside a data-center. On a fast network, "in cache on the neighboring computer" can be closer than "on my hard drive" for most purposes. It seems like what might be ideal would be a specification language that doesn't care where things are, an implementation that tries to deal with that automagically, and a way to specify portions (to all) precisely that is checked against the high-level specification.

thrownaway2424 12y ago
Absolutely. In-core on a neighboring machine can be accessed in less than 1ms. On-disk on the local machine (or any machine) can take tens of seconds.
- chetanahuja 12y ago
  "On-disk on the local machine (or any machine) can take tens of seconds" Surely you meant 10's of milliseconds. But even so, fastest SSD random access latencies are in sub-millisecond ranges.
  - thrownaway2424 12y ago
    No I did not mean that. Contended local disk access under heavy random read/write workloads takes essentially forever.
SEJeff 12y ago
To further your point, you can get faster access directly via RDMA on an Infiniband network than the SATA bus can push data. The fastest SATA busses are around 16gbps I believe and you can get HDR IB switches that clock 50gbps today. With the direction things are moving, "the data center as a computer" is absolutely the right approach.

presspot 12y ago

One thing worth noting is that the indirection Ben describes is, basically, the indirection that is already in the Apache Mesos kernel. In real world environments (e.g., Twitter), there is negligible overhead. Also, locality and latency can both be expressed in terms of placement rules and schedulers on Mesos can use those rules to guarantee or express preference for task placement that optimizes around reduced latencies. The point that I take away is that the ability to express your needs in a declarative way (e.g., "place these two tasks such that they have such-and-such latency) is much more scalable, flexible and resilient than coding to machine-specific internals. The latter is easier to update and supports delegation of responsibilities. John Wilkes of Google puts it this way: "Our own experience has been that allowing our developers unfettered access to the internals of infrastructure systems has been a problem, and we're moving away from that model as fast as we can. Constructing large-scale complex systems with many interdependencies leads to brittle, fragile systems if they rely on internal implementation mechanisms. Allowing internal customers to rely on internal implementation mechanisms has made it hard to adopt new technologies, because we only know what knobs they set - not why. The fix for both is similar: describe the desired end state, not how to get there."

perlgeek 12y ago
> The point that I take away is that the ability to express your needs in a declarative way (e.g., "place these two tasks such that they have such-and-such latency) is much more scalable, flexible and resilient than coding to machine-specific internals. The latter is easier to update and supports delegation of responsibilities. That makes total sense to me, thanks.