4 ms·

The term “serverless GPU” somehow wrecks my brain. Logically the absence of a server suggests its opposite, and the opposite of a server is a client, and client

by pavlov 4y ago

The term “serverless GPU” somehow wrecks my brain. Logically the absence of a server suggests its opposite, and the opposite of a server is a client, and client GPUs are the default. But this means “server GPU that’s available on-demand for very short-lived jobs” I guess.

touisteur 4y ago
Yeah, should be 'elastic' and 'very reduced runtime, possibly just inferencing', so exposing Triton to an API gateway and putting a custom load balancer and task queue facade ? Curious too. Also, GPUs do other things than inferencing, right?
ralusek 4y ago
I don't understand how people haven't gotten over this yet. When someone says serverless, I immediately understand that to mean "we've obfuscated the underlying server hardware from the consumer of this product." It means "you don't think about servers," not "there are no servers."
- pavlov 4y ago
  I’ve got used to the term, but when it’s in a new context like “serverless GPU”, its inherent absurdity resurfaces.
  - eurekin 4y ago
    Yup. My first intuition was "probably some server gpu (a100), but sold separately"
- gpm 4y ago
  Because if I'm thinking about the gpu I'm fundamentally thinking about the hardware, the server. Serverless responding to http requests. Sure. I write some code. It gets fed data and returns data. I don't have to know how many cores the server has, or what microcode version the CPU is, or how many other things are running on the server, or if I'm writing an interpreted language (probably the case) even what architecture the CPU is. But... I need to know all of that if I'm writing GPU code today.
  - rcme 4y ago
    Are you really thinking about the hardware when thinking about the GPU? For instance, if you use pytorch to write a NN, don't you kind of expect it to execute on the GPU without needing to get into the gritty details of it?
    - seabass-labrax 4y ago
      The specific version of Nvidia CUDA or AMD ROCm that the GPU supports is often very important; some software needs to be compiled from a specific branch or with specific settings to support a given version. Case in point: the official PyTorch website offers four distinct builds for various platforms, two of those being different versions of CUDA. I think it'll still be useful for plenty of people to choose which GPU runs their code, even if it's compatible with any GPU offered by the service. You might want to choose an older, cheaper GPU for basic parallel computation, but as supply catches up with demand for a newer and more energy-efficient model you'll then want to switch to that. There are only so many GeForce 4090s to go round :)
- 1letterunixname 4y ago
  It comes from the magical thinking of having too many marketing people and not enough engineers who actually build shit.
  - DonHopkins 4y ago
    Says the guy whose user name claims to have one letter, but actually contains one digit. ;)
- faeriechangling 4y ago
  I dunno, maybe it's because the jargon is stupid?
mgraczyk 4y ago
"Serverless" has been a standard term in industry for at least 7 years. Sometimes words don't map perfectly onto the subcomponents that form them. For example a "mailbox" isn't always a box, and doesn't always contain mail. You just have to learn to use the words and not worry so much about the etymology.
- pavlov 4y ago
  The tortured etymology becomes apparent again when these words are combined in new ways. “Serverless GPU” might be something like “mailbox SSD” in your example. What would that mean? It’s not obvious at first sight. The metaphor loses its power when it’s attached to a physical descriptor which is not a metaphor.
  - mgraczyk 4y ago
    No, it's obvious to everyone working with GPUs who knows what "serverless" means. It's a very standard construction in English. "Serverless GPU" means "GPU" that is "serverless". If you know what both words mean in the jargon, you know what they mean together. It's ok to not know what they mean, but arguing that it's "tortured" rings to me as misguided obtuseness.
    - Mike_12345 4y ago
      > It's a very standard construction in English. It's a new buzzword coined by the marketing team at Amazon in 2014. Somewhat confusing here as you are renting time on a GPU server described as "serverless".
      - mgraczyk 4y ago
        When I said "standard construction", I was referring to the combination of "serverless" and "GPU" to make "serverless GPU". Also you're factually wrong. Amazon didn't invent this term and it was widely used before 2014. Here's a 2012 article using the term. https://readwrite.com/why-the-future-of-software-and-apps-is-serverless/ https://readwrite.com/why-the-future-of-software-and-apps-is... I truly don't understand the bizarre and conspiracy-theory laden opposition to terms like this. It's a word, everyone who matters knows that it means.
1letterunixname 4y ago
It's a stupid misnomer. "Cloud" is still another. Perhaps "distributed scheduling and execution" is too straightforward than marketing wank.
- TeMPOraL 4y ago
  I mean it's all because marketers didn't want people to think of DCOM or CORBA when dealing with this "new" technology. (I can kind of understand it in case of CORBA - if you put the word "broker" in the name of your product, no one else will want to associate with it.)
thomastjeffery 4y ago
Something like "GPU for Serverless computing" would be much better. GPUs don't contain servers. Serverlesses contain GPUs.
DonHopkins 4y ago
The term should be "Serverless GPUless" to be fair and consistent. What's good for the goose is good for the gander.
dvh 4y ago
Serverless = pay per request Cloud computing = someone's else's computer
ukuina 4y ago
I, too, think of "classical serverless" rather than "neo-serverless": https://www.sqlite.org/serverless.html https://www.sqlite.org/serverless.html ------ Recently, folks have begun to use the word "serverless" to mean something subtly different from its intended meaning in this document. Here are two possible definitions of "serverless": Classic Serverless: The database engine runs within the same process, thread, and address space as the application. There is no message passing or network activity. Neo-Serverless: The database engine runs in a separate namespace from the application, probably on a separate machine, but the database is provided as a turn-key service by the hosting provider, requires no management or administration by the application owners, and is so easy to use that the developers can think of the database as being serverless even if it really does use a server under the covers.

by pavlov 4y ago

touisteur 4y ago

Yeah, should be 'elastic' and 'very reduced runtime, possibly just inferencing', so exposing Triton to an API gateway and putting a custom load balancer and task queue facade ? Curious too. Also, GPUs do other things than inferencing, right?

ralusek 4y ago

I don't understand how people haven't gotten over this yet. When someone says serverless, I immediately understand that to mean "we've obfuscated the underlying server hardware from the consumer of this product." It means "you don't think about servers," not "there are no servers."

pavlov 4y ago
I’ve got used to the term, but when it’s in a new context like “serverless GPU”, its inherent absurdity resurfaces.
- eurekin 4y ago
  Yup. My first intuition was "probably some server gpu (a100), but sold separately"
gpm 4y ago
Because if I'm thinking about the gpu I'm fundamentally thinking about the hardware, the server. Serverless responding to http requests. Sure. I write some code. It gets fed data and returns data. I don't have to know how many cores the server has, or what microcode version the CPU is, or how many other things are running on the server, or if I'm writing an interpreted language (probably the case) even what architecture the CPU is. But... I need to know all of that if I'm writing GPU code today.
- rcme 4y ago
  Are you really thinking about the hardware when thinking about the GPU? For instance, if you use pytorch to write a NN, don't you kind of expect it to execute on the GPU without needing to get into the gritty details of it?
  - seabass-labrax 4y ago
    The specific version of Nvidia CUDA or AMD ROCm that the GPU supports is often very important; some software needs to be compiled from a specific branch or with specific settings to support a given version. Case in point: the official PyTorch website offers four distinct builds for various platforms, two of those being different versions of CUDA. I think it'll still be useful for plenty of people to choose which GPU runs their code, even if it's compatible with any GPU offered by the service. You might want to choose an older, cheaper GPU for basic parallel computation, but as supply catches up with demand for a newer and more energy-efficient model you'll then want to switch to that. There are only so many GeForce 4090s to go round :)
1letterunixname 4y ago
It comes from the magical thinking of having too many marketing people and not enough engineers who actually build shit.
- DonHopkins 4y ago
  Says the guy whose user name claims to have one letter, but actually contains one digit. ;)
faeriechangling 4y ago
I dunno, maybe it's because the jargon is stupid?

mgraczyk 4y ago

"Serverless" has been a standard term in industry for at least 7 years. Sometimes words don't map perfectly onto the subcomponents that form them. For example a "mailbox" isn't always a box, and doesn't always contain mail. You just have to learn to use the words and not worry so much about the etymology.

pavlov 4y ago
The tortured etymology becomes apparent again when these words are combined in new ways. “Serverless GPU” might be something like “mailbox SSD” in your example. What would that mean? It’s not obvious at first sight. The metaphor loses its power when it’s attached to a physical descriptor which is not a metaphor.
- mgraczyk 4y ago
  No, it's obvious to everyone working with GPUs who knows what "serverless" means. It's a very standard construction in English. "Serverless GPU" means "GPU" that is "serverless". If you know what both words mean in the jargon, you know what they mean together. It's ok to not know what they mean, but arguing that it's "tortured" rings to me as misguided obtuseness.
  - Mike_12345 4y ago
    > It's a very standard construction in English. It's a new buzzword coined by the marketing team at Amazon in 2014. Somewhat confusing here as you are renting time on a GPU server described as "serverless".
    - mgraczyk 4y ago
      When I said "standard construction", I was referring to the combination of "serverless" and "GPU" to make "serverless GPU". Also you're factually wrong. Amazon didn't invent this term and it was widely used before 2014. Here's a 2012 article using the term. https://readwrite.com/why-the-future-of-software-and-apps-is-serverless/ https://readwrite.com/why-the-future-of-software-and-apps-is... I truly don't understand the bizarre and conspiracy-theory laden opposition to terms like this. It's a word, everyone who matters knows that it means.

1letterunixname 4y ago

It's a stupid misnomer. "Cloud" is still another. Perhaps "distributed scheduling and execution" is too straightforward than marketing wank.

TeMPOraL 4y ago
I mean it's all because marketers didn't want people to think of DCOM or CORBA when dealing with this "new" technology. (I can kind of understand it in case of CORBA - if you put the word "broker" in the name of your product, no one else will want to associate with it.)

thomastjeffery 4y ago

Something like "GPU for Serverless computing" would be much better. GPUs don't contain servers. Serverlesses contain GPUs.

DonHopkins 4y ago

The term should be "Serverless GPUless" to be fair and consistent. What's good for the goose is good for the gander.

dvh 4y ago

Serverless = pay per request Cloud computing = someone's else's computer

ukuina 4y ago

I, too, think of "classical serverless" rather than "neo-serverless": https://www.sqlite.org/serverless.html https://www.sqlite.org/serverless.html ------ Recently, folks have begun to use the word "serverless" to mean something subtly different from its intended meaning in this document. Here are two possible definitions of "serverless": Classic Serverless: The database engine runs within the same process, thread, and address space as the application. There is no message passing or network activity. Neo-Serverless: The database engine runs in a separate namespace from the application, probably on a separate machine, but the database is provided as a turn-key service by the hosting provider, requires no management or administration by the application owners, and is so easy to use that the developers can think of the database as being serverless even if it really does use a server under the covers.