3 ms·

I think the important metric will be if we can compete against the price of AWS or Microsoft in running large LLMs, not their time to market. Competing on cost

I think the important metric will be if we can compete against the price of AWS or Microsoft in running large LLMs, not their time to market. Competing on cost against overpriced hyperscalers is not very hard, and $30K is a small investment, not a gamble. If it would fail, worst case you would only loose $3000-$6500 or so.

DoingIsLearning 2y ago
> $40K is a small investment, not a gamble. If it would fail, worst case you would only lose $3000-$6500 or so. As someone not familiar with investment sourcing or SME financing. Could you break down the maths/accounting? How do you go from sinking 40k in a business to losing 6.5k if you turn the lights off at the end?
- morphle 2y ago
  You buy the hardware (48 servers), rent part of a colocation rack with a 10 Gbps or 100 Gbps internet transit link, get a payment processor, make a webpage and GitHub demo with the API. Break down: $3000 labour, $20.5K hardware, $800 monthly rental fees, $376 car fees. When you shut down within a year, the $20.5K popular off the shelf hardware can easily be sold for $17K, a fact you can check from 25 years of data. I would invest more than the initial $30K on optimization after the servers have found paying customers and thus have proven commercial viability. I would invest in software development, finetuning, retraining and above all reverse engineering GPU and neural engine instruction sets and adapting these open source models to the more than 2 quadrillion operations per second that these 48 servers can do.
  - seanp2k2 2y ago
    So, $0 budget for software dev / sales / support?
    - morphle 2y ago
      I broke down the first $30K investment cost for release of the online API product, that does not need further software development, sales or support. You would be wise to do the software development I mentioned, do more sales and support than was covered under my initial $3000 labour fee. But that you can pay for with the revenues, it would not be the initial investment to see if it is viable as a business.
    - gloflo 2y ago
      That's what the AI is for, no? /s
    - tonyhart7 2y ago
      Well, when you run an AI company, you must test your product, right? What better way to test it than by building your own webpage, admin panel, etc.?
  - lossolo 2y ago
    Where can you get 10 or 100 Gbps flat with a full 48U rack and power for $800? Because if that exists, I want to buy them all.
    - morphle 2y ago
      Email in my profile. 100 Gps plus rack is 5 times the cost of 10 Gbps. You'll need 100 Gbps routers and smartnics too.
      - menaerus 2y ago
        48 servers at 60K also does not math to me at all. Even if I consider the second-hand ones, I could hardly find zen4 under ~10K. And this is without GPUs.
        lossolo 2y ago
        Yeah, this guy doesn't know what he's talking about. I buy hardware and transit, so I know he's just making up numbers.
        menaerus 2y ago
        The numbers were really too good to be true but there's always a chance that there may be something I am not seeing correctly so I wanted to give the benefit of the doubt. Otherwise, the idea is not too crazy regardless of much higher input costs.

DoingIsLearning 2y ago

> $40K is a small investment, not a gamble. If it would fail, worst case you would only lose $3000-$6500 or so. As someone not familiar with investment sourcing or SME financing. Could you break down the maths/accounting? How do you go from sinking 40k in a business to losing 6.5k if you turn the lights off at the end?

morphle 2y ago
You buy the hardware (48 servers), rent part of a colocation rack with a 10 Gbps or 100 Gbps internet transit link, get a payment processor, make a webpage and GitHub demo with the API. Break down: $3000 labour, $20.5K hardware, $800 monthly rental fees, $376 car fees. When you shut down within a year, the $20.5K popular off the shelf hardware can easily be sold for $17K, a fact you can check from 25 years of data. I would invest more than the initial $30K on optimization after the servers have found paying customers and thus have proven commercial viability. I would invest in software development, finetuning, retraining and above all reverse engineering GPU and neural engine instruction sets and adapting these open source models to the more than 2 quadrillion operations per second that these 48 servers can do.
- seanp2k2 2y ago
  So, $0 budget for software dev / sales / support?
  - morphle 2y ago
    I broke down the first $30K investment cost for release of the online API product, that does not need further software development, sales or support. You would be wise to do the software development I mentioned, do more sales and support than was covered under my initial $3000 labour fee. But that you can pay for with the revenues, it would not be the initial investment to see if it is viable as a business.
  - gloflo 2y ago
    That's what the AI is for, no? /s
  - tonyhart7 2y ago
    Well, when you run an AI company, you must test your product, right? What better way to test it than by building your own webpage, admin panel, etc.?
- lossolo 2y ago
  Where can you get 10 or 100 Gbps flat with a full 48U rack and power for $800? Because if that exists, I want to buy them all.
  - morphle 2y ago
    Email in my profile. 100 Gps plus rack is 5 times the cost of 10 Gbps. You'll need 100 Gbps routers and smartnics too.
    - menaerus 2y ago
      48 servers at 60K also does not math to me at all. Even if I consider the second-hand ones, I could hardly find zen4 under ~10K. And this is without GPUs.
      - lossolo 2y ago
        Yeah, this guy doesn't know what he's talking about. I buy hardware and transit, so I know he's just making up numbers.
        menaerus 2y ago
        The numbers were really too good to be true but there's always a chance that there may be something I am not seeing correctly so I wanted to give the benefit of the doubt. Otherwise, the idea is not too crazy regardless of much higher input costs.