4 ms·

Not sure - are Google running their servers with custom-made processors as well?

quantumofalpha 5y ago
Kind of, I've seen some non-public Intel SKUs in the fleet but afaik they just had tweaked core counts and clocks for better perf/watt. They were also exploring non-intel architectures at some point, ppc especially but no significant prod deployment yet. And then I'm sure you've heard of special purpose hardware like TPUs
cube00 5y ago
That's covered earlier (full quote): > Those patches implement various internal APIs (e.g. for Google Fibers), provide hardware support, add performance optimizations, and contain other "tweaks that are needed to run binaries that we use at Google". I'd assume if you're making allowances for custom-made processors at the kernel level you wouldn't also want that leaking into your user land binaries as well.
- eru 5y ago
  Why not? They could have kernel accommodation for one feature, and user-land leakage for another. Also, for some features you need both kernel and userland cooperation. Just think of eg fuse or io_uring or mmap that are in the public kernel. You can surely imagine that Google might brew up similar features. (I used to work at Google. But I didn't have any special insider knowledge about their kernel stuff. Which is good, so I can't violate any lingering NDAs here with my speculation..)
jeffbee 5y ago
A lot of it is just mundane details. Some of it leaks out in ABSL, for example some boring trivia is at: https://github.com/abseil/abseil-cpp/blob/master/absl/base/internal/sysinfo.cc#L303-L313 https://github.com/abseil/abseil-cpp/blob/master/absl/base/i... They also have for years had major patches to TCP which Linux refuses to adopt. Some details at: https://www.ietf.org/proceedings/97/slides/slides-97-tcpm-tcp-options-for-low-latency-00.pdf https://www.ietf.org/proceedings/97/slides/slides-97-tcpm-tc... These won't make binaries "not run" as such but they are necessary for the correct and efficient operation of large-scale distributed systems. QUIC, Snap/Pony, and user-space thread scheduling are strictly superior to what happens inside the kernel and their development makes the delta between prodkernel and upstream kernel less relevant over time. They also, collectively, make Linux itself less relevant.
- pjmlp 5y ago
  Linux kernel is the most sucessful free beer UNIX clone, but it is by no means the only one, and if the trends on IoT and cloud infrastructure are any measure, in a couple of decades we might still live under POSIX monoculture to some extent, but Linux most likely won't be the kernel underneath.
H8crilA 5y ago
Not CPUs but a lot of other components are completely custom, developed in-house. A well known example: TPUs. Google is one of the world's largest manufacturer of servers and other DC equipment, actually, it's just that none of that is for sale (though a lot is for rent :) ).
- walrus01 5y ago
  Unless they've been building any factories I don't know about, their contractors are the manufacturers. I'm talking about companies based in Taiwan like Quanta, Compal, Clevo, and others. If you've ever been to a Computex Taipei you have seen the booths of the actual manufacturers of Google's hardware. https://en.wikipedia.org/wiki/Computex https://en.wikipedia.org/wiki/Computex
  - deleted 5y ago
    [deleted]
  - drivebycomment 5y ago
    This is a meaningless nitpicking. It is like nitpicking Apple is not a manufacturer of hardware since the actual assembly happens under some contractor. For all intent and purpose, and even technically, Google is the largest manufacturer of data center-sized supercomputers.
- ars 5y ago
  See this for an early example: https://news.ycombinator.com/item?id=542531 https://news.ycombinator.com/item?id=542531
- skunkworker 5y ago
  Like when that Pluto Switch got leaked back in 2012 https://www.wired.com/2012/09/pluto-switch/ https://www.wired.com/2012/09/pluto-switch/
eru 5y ago
Some of them, yes! At least for machine learning. They also have custom silicon in their phones (and the phones run a Linux).

Kind of, I've seen some non-public Intel SKUs in the fleet but afaik they just had tweaked core counts and clocks for better perf/watt. They were also exploring non-intel architectures at some point, ppc especially but no significant prod deployment yet. And then I'm sure you've heard of special purpose hardware like TPUs

cube00 5y ago

That's covered earlier (full quote): > Those patches implement various internal APIs (e.g. for Google Fibers), provide hardware support, add performance optimizations, and contain other "tweaks that are needed to run binaries that we use at Google". I'd assume if you're making allowances for custom-made processors at the kernel level you wouldn't also want that leaking into your user land binaries as well.

eru 5y ago
Why not? They could have kernel accommodation for one feature, and user-land leakage for another. Also, for some features you need both kernel and userland cooperation. Just think of eg fuse or io_uring or mmap that are in the public kernel. You can surely imagine that Google might brew up similar features. (I used to work at Google. But I didn't have any special insider knowledge about their kernel stuff. Which is good, so I can't violate any lingering NDAs here with my speculation..)

jeffbee 5y ago

A lot of it is just mundane details. Some of it leaks out in ABSL, for example some boring trivia is at: https://github.com/abseil/abseil-cpp/blob/master/absl/base/internal/sysinfo.cc#L303-L313 https://github.com/abseil/abseil-cpp/blob/master/absl/base/i... They also have for years had major patches to TCP which Linux refuses to adopt. Some details at: https://www.ietf.org/proceedings/97/slides/slides-97-tcpm-tcp-options-for-low-latency-00.pdf https://www.ietf.org/proceedings/97/slides/slides-97-tcpm-tc... These won't make binaries "not run" as such but they are necessary for the correct and efficient operation of large-scale distributed systems. QUIC, Snap/Pony, and user-space thread scheduling are strictly superior to what happens inside the kernel and their development makes the delta between prodkernel and upstream kernel less relevant over time. They also, collectively, make Linux itself less relevant.

pjmlp 5y ago
Linux kernel is the most sucessful free beer UNIX clone, but it is by no means the only one, and if the trends on IoT and cloud infrastructure are any measure, in a couple of decades we might still live under POSIX monoculture to some extent, but Linux most likely won't be the kernel underneath.

H8crilA 5y ago

Not CPUs but a lot of other components are completely custom, developed in-house. A well known example: TPUs. Google is one of the world's largest manufacturer of servers and other DC equipment, actually, it's just that none of that is for sale (though a lot is for rent :) ).

walrus01 5y ago
Unless they've been building any factories I don't know about, their contractors are the manufacturers. I'm talking about companies based in Taiwan like Quanta, Compal, Clevo, and others. If you've ever been to a Computex Taipei you have seen the booths of the actual manufacturers of Google's hardware. https://en.wikipedia.org/wiki/Computex https://en.wikipedia.org/wiki/Computex
- deleted 5y ago
  [deleted]
- drivebycomment 5y ago
  This is a meaningless nitpicking. It is like nitpicking Apple is not a manufacturer of hardware since the actual assembly happens under some contractor. For all intent and purpose, and even technically, Google is the largest manufacturer of data center-sized supercomputers.
ars 5y ago
See this for an early example: https://news.ycombinator.com/item?id=542531 https://news.ycombinator.com/item?id=542531
skunkworker 5y ago
Like when that Pluto Switch got leaked back in 2012 https://www.wired.com/2012/09/pluto-switch/ https://www.wired.com/2012/09/pluto-switch/

eru 5y ago

Some of them, yes! At least for machine learning. They also have custom silicon in their phones (and the phones run a Linux).