4 ms·

Self hosting means hosting it yourself, not running it on Amazon. I think the distinction the author intends to make is between running something that can't be

Self hosting means hosting it yourself, not running it on Amazon. I think the distinction the author intends to make is between running something that can't be hosted elsewhere, like ChatGPT, versus running Llama-3 yourself. Overlooking that, the rest of the article feels a bit strange. Would we really have a use case where we can make use of those 157 million tokens a month? Would we really round $50 of energy cost to $100 a month? (Granted, the author didn't include power for the computer) If we buy our own system to run, why would we need to "scale your own hardware"? I get that this is just to give us an idea of what running something yourself would cost when comparing with services like ChatGPT, but if so, we wouldn't be making most of the choices made here such as getting four NVIDIA Tesla T4 cards. Memory is cheap, so running Llama-3 entirely on CPU is also an option. It's slower, of course, but it's infinitely more flexible. If I really wanted to spend a lot of time tinkering with LLMs, I'd definitely do this to figure out what I want to run before deciding on GPU hardware, then I'd get GPU hardware that best matches that, instead of the other way around.

williamstein 2y ago
> Self hosting means hosting it yourself, not running it on Amazon. No. I googled "self hosting", read the first few definitions, and they agree with the article, not you. E.g., wikipedia -- https://en.wikipedia.org/wiki/Self-hosting_(web_services) https://en.wikipedia.org/wiki/Self-hosting_(web_services)
- johnklos 2y ago
  The very first definition from the link you provide is: > Self-hosting is the practice of running and maintaining a website or service using a private web server, instead of using a service outside of someone's own control. Hosting anything on Amazon is not "using a private web server" and is the very definition of using "a service outside of someone's own control". The fact that the rest of the article talks about "enabled users to run their own servers on remote hardware or virtual machines" is just wrong. It's not "their own servers", and we don't have "more control over their data, privacy" when it's literally in the possession of others.
  - Majestic121 2y ago
    The second sentence is however : > The practice of self-hosting web services became more feasible with the development of cloud computing and virtualization technologies, which enabled users to run their own servers on remote hardware or virtual machines. The first public cloud service, Amazon Web Services (AWS), was launched in 2006, offering Simple Storage Service (S3) and Elastic Compute Cloud (EC2) as its initial products.[3] The mystery deepens
    - chasd00 2y ago
      I hate when terms get diluted like this. "self hosted", to me, means you own the physical machine. This reminds of how "air-gapped server" now means a route configuration vs an actual gap of air, no physical connection, between two networks. It really confuses things.
    - guilhas 2y ago
      That's a bit of newspeak I think we generally understand Iaas, Paas, and Saas, to be hosted offerings, managed and unmanaged... https://duckduckgo.com/?q=iaas+paas+saas&ia=web https://duckduckgo.com/?q=iaas+paas+saas&ia=web
- carom 2y ago
  I would say that is "cloud hosted", which is obviously very expensive compared to running on hardware you own (assuming you own a computer and a GPU). That was the comparison I was interested in, the fact that renting a computer is more expensive than the OpenAI API is not a surprising result.

williamstein 2y ago

> Self hosting means hosting it yourself, not running it on Amazon. No. I googled "self hosting", read the first few definitions, and they agree with the article, not you. E.g., wikipedia -- https://en.wikipedia.org/wiki/Self-hosting_(web_services) https://en.wikipedia.org/wiki/Self-hosting_(web_services)

johnklos 2y ago
The very first definition from the link you provide is: > Self-hosting is the practice of running and maintaining a website or service using a private web server, instead of using a service outside of someone's own control. Hosting anything on Amazon is not "using a private web server" and is the very definition of using "a service outside of someone's own control". The fact that the rest of the article talks about "enabled users to run their own servers on remote hardware or virtual machines" is just wrong. It's not "their own servers", and we don't have "more control over their data, privacy" when it's literally in the possession of others.
- Majestic121 2y ago
  The second sentence is however : > The practice of self-hosting web services became more feasible with the development of cloud computing and virtualization technologies, which enabled users to run their own servers on remote hardware or virtual machines. The first public cloud service, Amazon Web Services (AWS), was launched in 2006, offering Simple Storage Service (S3) and Elastic Compute Cloud (EC2) as its initial products.[3] The mystery deepens
  - chasd00 2y ago
    I hate when terms get diluted like this. "self hosted", to me, means you own the physical machine. This reminds of how "air-gapped server" now means a route configuration vs an actual gap of air, no physical connection, between two networks. It really confuses things.
  - guilhas 2y ago
    That's a bit of newspeak I think we generally understand Iaas, Paas, and Saas, to be hosted offerings, managed and unmanaged... https://duckduckgo.com/?q=iaas+paas+saas&ia=web https://duckduckgo.com/?q=iaas+paas+saas&ia=web
carom 2y ago
I would say that is "cloud hosted", which is obviously very expensive compared to running on hardware you own (assuming you own a computer and a GPU). That was the comparison I was interested in, the fact that renting a computer is more expensive than the OpenAI API is not a surprising result.