4 ms·

This is wrong on merit, and I am not sure why it is presented this way. The difference between a car company and a software company is economy of scale. I.e. e

This is wrong on merit, and I am not sure why it is presented this way. The difference between a car company and a software company is economy of scale. I.e. economy of scale dominate the physical world but does not exist in the software world since I can replicate software at zero cost. In addition, new tools and new processes for software has increased the productivity times fold, which means that you need fewer developers for new software. I predict two shifts in the tech world: 1) Move to the edge. Specially for AI, there is really no need for a central public cloud due to latency, privacy, and dedicated hardware chips. I.e. most of AI traffic is inference traffic which should be done on the edge. 2) Kubernetes operators for replacing cloud services. The value add of the public cloud is managing complexity.

mooted1 7y ago
If you read more of Ben's writing, he talks extensively about how software companies dominate market share through network effects and vertical integration. You don't hear him talk about economies of scale because marginal costs are negligible for software companies. Besides, network effects and vertical integration are sufficiently powerful to control the market. > In addition, new tools and new processes for software has increased the productivity times fold, which means that you need fewer developers for new software. There are other barriers to entry besides the cost of writing software, like product, sales, operations, and most importantly, network.
- streetcat1 7y ago
  However, the network effect in tech can be leapfrogged due to the zero marginal cost (as shown in this post). I.e. what network effect do you get from doing ML inference in the cloud? The case for big tech today is still the economy of scale and not network effects (maybe facebook have those, but it exists only if the interface to facebook does not change). The big tech players have economy of scale, due to their ability to use automation and offload the risk of managing complexity (I.e. one AWS engineer can manager 1000's of machines with AWS software). No wonder, that the software that manages the public cloud is still closed source. However, with Kubernetes operators, there is a way to move those capabilities into any Kubernetes cluser.
  - mooted1 7y ago
    Did you actually read the post? > The case for big tech today is still the economy of scale and not network effects (maybe facebook have those, but it exists only if the interface to facebook does not change). This is only true if you believe that the greatest cost of developing software is running hardware. The greatest cost of developing software is developing software. Not only are economies of scale in compute management negligible except at massive scale, the cost of compute has declined dramatically as the companies you've described have made their datacenters available for rent through the cloud. Yet the tech giants persist. Facebook, Google, Netflix, Amazon all have considerable network effects that you're not considering. For each of these companies, having so many customers provides benefits that accrue without diminishing returns, giving them a firm hold on market share. See https://stratechery.com/2015/aggregation-theory/ https://stratechery.com/2015/aggregation-theory/ Ben is saying that the only way to topple the giants is by working around them and leveraging new computing technologies better than them. He makes the (admittedly speculative) case that this is no longer possible because we can't bring compute any closer to the user than the mobile devices. > However, with Kubernetes operators, there is a way to move those capabilities into any Kubernetes cluser. Kubernetes, at the scale of technologies we're discussing, is a minor optimization. Introducing k8s costs more than it helps far until far into a company's infra maturity. Even if most companies deployed k8s in a manner that significantly reduced costs, it's not enough to overcome the massive advantages existing tech companies have accrued. Not to mention all of the big tech companies have internal cluster managers of their own.
    - yodon 7y ago
      Your comment would work equally well without the initial inflammatory "Did you actually read the post?" opening line.
    - mr__y 7y ago
      >this is no longer possible because we can't bring compute any closer to the user than the mobile devices. this is based on a very dubious assumption that bringing compute closer is the only path for innovation. and even that is not true, you could imagine compute being even closer with a direct brain interface (actually you could consider google glasses to be an attempt at bringing compute closer)
xapata 7y ago
> 1) Move to the edge. Specially for AI, there is really no need for a central public cloud due to latency, privacy, and dedicated hardware chips. I.e. most of AI traffic is inference traffic which should be done on the edge. Inferencing is done at the edge, but training must be done centrally.
- Slartie 7y ago
  Right now, the only market participant I see doing some inferencing at the edge is Apple with its photo analysis stuff that runs on the phone itself. Anyone else is busy building little dumb cubes with microphones and speakers that send sound bites into clouds and receive sound bites to play back (heck, even Apple does it this way with Siri). Or other dumb cubes that get plugged into a wall socket and that can switch lights that you plug into them by receiving commands from a cloud (even if the origin of the command is in the same room). Or dumb bulbs that get RGB values from a cloud server which inferred somehow that the owner must have come home recently and which then set the brightness of their RGB LEDs accordingly. Or software that lets you record sounds bites, send them into the cloud and receive transcripts back. Or software that sends all your photos to a cloud library where it is scanned and tagged so you can search for "bikes" or whatever in your photos. No matter what you look at in all that stuff that makes up what consumers currently consider to be "AI", it does inference (if it even does anything like that at all) on some cloud server. I don't like that development myself, but unfortunately that's how it is.
zebrafish 7y ago
Your point contradicts itself, does it not? Movement to the edge necessitates edge hardware and edge personnel to manage the hardware. Network effects are THE factor in software because the marginal cost tends towards zero with each incremental user in the network. The edge adds cost per node. Up until the point that users are paid to connect to the network and/or the network is directly linked to the user with the I/O line completely obviated, the economics of hardware and management underlying the network will tend towards economies of scale... which is the point Ben is trying to make.
foobiekr 7y ago
It really doesn't make a lot of sense to do AI at the edge (in terms of the various edge providers). But then a lot of edge cases don't make a lot of sense. The best edge use cases are fan-in (aggregation and data reduction), fan-out (replication and amplification - broadcasting, conferencing, video streaming, etc.) and caching (which is just a variant of fan-out). The rest of the cases are IMHO largely fictional - magical latency improvements talked about in the same context as applications that are grossly un-optimized in every way imaginable, AR/VR, etc. Especially the AR/VR thing. Beyond that the only thing left is cost arbitrage - selling bandwidth (mostly) cheaper than AWS. What's the use case for moving inference to the edge? Most of the inference will in fact be at the edge - in the device, which has plenty of capacity - but that's not the case you're describing.
- streetcat1 7y ago
  Why would you run AI in the cloud? It is a closed, expensive, high latency, etc. You might want to train in the cloud, maybe. For inference, I See 90% on the edge (I.e. outside of the clouds).

mooted1 7y ago

If you read more of Ben's writing, he talks extensively about how software companies dominate market share through network effects and vertical integration. You don't hear him talk about economies of scale because marginal costs are negligible for software companies. Besides, network effects and vertical integration are sufficiently powerful to control the market. > In addition, new tools and new processes for software has increased the productivity times fold, which means that you need fewer developers for new software. There are other barriers to entry besides the cost of writing software, like product, sales, operations, and most importantly, network.

streetcat1 7y ago
However, the network effect in tech can be leapfrogged due to the zero marginal cost (as shown in this post). I.e. what network effect do you get from doing ML inference in the cloud? The case for big tech today is still the economy of scale and not network effects (maybe facebook have those, but it exists only if the interface to facebook does not change). The big tech players have economy of scale, due to their ability to use automation and offload the risk of managing complexity (I.e. one AWS engineer can manager 1000's of machines with AWS software). No wonder, that the software that manages the public cloud is still closed source. However, with Kubernetes operators, there is a way to move those capabilities into any Kubernetes cluser.
- mooted1 7y ago
  Did you actually read the post? > The case for big tech today is still the economy of scale and not network effects (maybe facebook have those, but it exists only if the interface to facebook does not change). This is only true if you believe that the greatest cost of developing software is running hardware. The greatest cost of developing software is developing software. Not only are economies of scale in compute management negligible except at massive scale, the cost of compute has declined dramatically as the companies you've described have made their datacenters available for rent through the cloud. Yet the tech giants persist. Facebook, Google, Netflix, Amazon all have considerable network effects that you're not considering. For each of these companies, having so many customers provides benefits that accrue without diminishing returns, giving them a firm hold on market share. See https://stratechery.com/2015/aggregation-theory/ https://stratechery.com/2015/aggregation-theory/ Ben is saying that the only way to topple the giants is by working around them and leveraging new computing technologies better than them. He makes the (admittedly speculative) case that this is no longer possible because we can't bring compute any closer to the user than the mobile devices. > However, with Kubernetes operators, there is a way to move those capabilities into any Kubernetes cluser. Kubernetes, at the scale of technologies we're discussing, is a minor optimization. Introducing k8s costs more than it helps far until far into a company's infra maturity. Even if most companies deployed k8s in a manner that significantly reduced costs, it's not enough to overcome the massive advantages existing tech companies have accrued. Not to mention all of the big tech companies have internal cluster managers of their own.
  - yodon 7y ago
    Your comment would work equally well without the initial inflammatory "Did you actually read the post?" opening line.
  - mr__y 7y ago
    >this is no longer possible because we can't bring compute any closer to the user than the mobile devices. this is based on a very dubious assumption that bringing compute closer is the only path for innovation. and even that is not true, you could imagine compute being even closer with a direct brain interface (actually you could consider google glasses to be an attempt at bringing compute closer)

xapata 7y ago

> 1) Move to the edge. Specially for AI, there is really no need for a central public cloud due to latency, privacy, and dedicated hardware chips. I.e. most of AI traffic is inference traffic which should be done on the edge. Inferencing is done at the edge, but training must be done centrally.

Slartie 7y ago
Right now, the only market participant I see doing some inferencing at the edge is Apple with its photo analysis stuff that runs on the phone itself. Anyone else is busy building little dumb cubes with microphones and speakers that send sound bites into clouds and receive sound bites to play back (heck, even Apple does it this way with Siri). Or other dumb cubes that get plugged into a wall socket and that can switch lights that you plug into them by receiving commands from a cloud (even if the origin of the command is in the same room). Or dumb bulbs that get RGB values from a cloud server which inferred somehow that the owner must have come home recently and which then set the brightness of their RGB LEDs accordingly. Or software that lets you record sounds bites, send them into the cloud and receive transcripts back. Or software that sends all your photos to a cloud library where it is scanned and tagged so you can search for "bikes" or whatever in your photos. No matter what you look at in all that stuff that makes up what consumers currently consider to be "AI", it does inference (if it even does anything like that at all) on some cloud server. I don't like that development myself, but unfortunately that's how it is.

zebrafish 7y ago

Your point contradicts itself, does it not? Movement to the edge necessitates edge hardware and edge personnel to manage the hardware. Network effects are THE factor in software because the marginal cost tends towards zero with each incremental user in the network. The edge adds cost per node. Up until the point that users are paid to connect to the network and/or the network is directly linked to the user with the I/O line completely obviated, the economics of hardware and management underlying the network will tend towards economies of scale... which is the point Ben is trying to make.

foobiekr 7y ago

It really doesn't make a lot of sense to do AI at the edge (in terms of the various edge providers). But then a lot of edge cases don't make a lot of sense. The best edge use cases are fan-in (aggregation and data reduction), fan-out (replication and amplification - broadcasting, conferencing, video streaming, etc.) and caching (which is just a variant of fan-out). The rest of the cases are IMHO largely fictional - magical latency improvements talked about in the same context as applications that are grossly un-optimized in every way imaginable, AR/VR, etc. Especially the AR/VR thing. Beyond that the only thing left is cost arbitrage - selling bandwidth (mostly) cheaper than AWS. What's the use case for moving inference to the edge? Most of the inference will in fact be at the edge - in the device, which has plenty of capacity - but that's not the case you're describing.

streetcat1 7y ago
Why would you run AI in the cloud? It is a closed, expensive, high latency, etc. You might want to train in the cloud, maybe. For inference, I See 90% on the edge (I.e. outside of the clouds).