3 ms·

If we start shifting towards specialized hardware configurations to solve specialized problems, how does “move everything to the cloud communicating through fra

If we start shifting towards specialized hardware configurations to solve specialized problems, how does “move everything to the cloud communicating through fragmented docker images” fit? It just seems weird to me to see posts of spending lots of money on complicated scalable Kubernetes clusters, but at the same time people are putting their AI computation engines right on die so that they get good cache coherence. Maybe they’re more resolvable than it seems.

c54 4y ago
K8s GPU nodes are already kind of an example of this-- the app can still access the specialty capabilities of the hardware, we just have to make sure to tell the scheduler "hey run this app on this kind of node with this flavor of capabilities". I could imagine that kind of flavor capability growing and becoming more fleshed out Except k8s calls it 'taints' which is uh a weird choice.
- kevincox 4y ago
  Taints is a funny name when you look at it like that but taints are actually the opposite half of the story. If you want a Pod to be scheduled with a certain node you need to add affinity. The taints are to ensure that your average nginx pod doesn't run on the specialty hardware and block the pods that actually need it. In theory the scheduler could be smart enough to evict the pods that don't need the special node but IDK if the kube scheduler is smart enough. Even then it may he preferable to have the taint though to reduce the eviction rate.
bippingchip 4y ago
This is probably one of the biggest challenges ahead, likely more so than making hardware that still somehow scales despite Moore’s law slowing down and tapering out entirely. The increased scale of integration with advanced 3D packaging and chiplets is a nice way forward, but the heterogeneity of those solutions need innovation at the programming model as much as anything else: how do you make such a complex system accessible in a way that the programmer can actually use it efficiently? As much as I like semiengineering and their articles, they are very hardware and process technology focused and tend to miss the software side of things.

c54 4y ago

K8s GPU nodes are already kind of an example of this-- the app can still access the specialty capabilities of the hardware, we just have to make sure to tell the scheduler "hey run this app on this kind of node with this flavor of capabilities". I could imagine that kind of flavor capability growing and becoming more fleshed out Except k8s calls it 'taints' which is uh a weird choice.

kevincox 4y ago
Taints is a funny name when you look at it like that but taints are actually the opposite half of the story. If you want a Pod to be scheduled with a certain node you need to add affinity. The taints are to ensure that your average nginx pod doesn't run on the specialty hardware and block the pods that actually need it. In theory the scheduler could be smart enough to evict the pods that don't need the special node but IDK if the kube scheduler is smart enough. Even then it may he preferable to have the taint though to reduce the eviction rate.

bippingchip 4y ago

This is probably one of the biggest challenges ahead, likely more so than making hardware that still somehow scales despite Moore’s law slowing down and tapering out entirely. The increased scale of integration with advanced 3D packaging and chiplets is a nice way forward, but the heterogeneity of those solutions need innovation at the programming model as much as anything else: how do you make such a complex system accessible in a way that the programmer can actually use it efficiently? As much as I like semiengineering and their articles, they are very hardware and process technology focused and tend to miss the software side of things.