3 ms·

The numbers are optimistically legit -- it's calculated based purely considering we have demand for all machines at all times. We don't have that right now, but

The numbers are optimistically legit -- it's calculated based purely considering we have demand for all machines at all times. We don't have that right now, but fairly optimistic that people will do it. That's why we don't recommend purchasing a new machine. Existing machine is no cost for you to run this. Electricity is one cost, but it will get paid off from every request it receives. Electricity is only deducted when you run an inference. If you have any questions, DM me @gajesh on Twitter.

stavros 6mo ago
You're not taking into account the thermal strain on the machine, though. A machine that's 100% utilized (even worse if it's in bursts) will last less than an idle machine.
- embedding-shape 6mo ago
  > A machine that's 100% utilized (even worse if it's in bursts) will last less than an idle machine. How much though? Say I have three Mac Minis next to each other, one that is completely idle but on, one that bursts 100% CPU every 10 minutes and one that uses 100% CPU all the time, what's the difference on how long the machines survives? Months, years or decades?
- washadjeffmad 6mo ago
  Not appreciably, and not before a 5-yr AppleCare+ warranty expires. Out of our >3000 currently active Apple Silicon Macs, failures due to non-physical damage are in the single digits per year. Of those, none have been from production systems with 24/7 uptime and continuous high load, which reflects your parenthetical. Perhaps we haven't met the other end of the bathtub curve yet, but we also won't be retaining any of these very far beyond their warranty period, much less the end of their support life.
  - stavros 6mo ago
    I think the point of this is more "use the machine you have at home" than "do a TCO analysis and see if it's profitable", though. People like to keep their machines working for longer, generally.
  - alsetmusic 6mo ago
    > Not appreciably, and not before a 5-yr AppleCare+ warranty expires. It’s three years for Macs, though I believe you can pay annually for longer. Five has never been a thing to my knowledge.
  - bbatsell 6mo ago
    AppleCare+ annual is perpetual as long as you keep paying it (and Apple offers to switch to that when your 3-year lump sum expires if you choose that instead). I’m guessing it ends whenever they officially discontinue hardware support, which has traditionally been about 7 years after the last unit is produced, but I haven’t reached that yet to know for sure.
mbesto 6mo ago
> That's why we don't recommend purchasing a new machine. Existing machine is no cost for you to run this. You misunderstood. If the ROI is there, there is enough capital in existence for you to accelerate your profit. So why even deal with the complexity of renting people's hardware when you can do it yourself?
- splintercell 6mo ago
  No, what he's saying is that he expects this to be the ROI in the future because his product is so good.
BuildTheRobots 6mo ago
I don't worry about bandwidth or constant CPU use, but the one thing that will kill my mac is burning out the SSD. The calculator gives numbers for nearly everything, but I can't obviously see how much space it needs for model storage or how many writes of temp files I should expect if I'm running flat out.
- sleepybrett 6mo ago
  put that stuff on an external disk perhaps, it will eventually crater, but it's easier to replace than macbook internal storage (how are they doing mac minis these days?)
  - neurostimulant 6mo ago
    Isn't high memory pressure will result in high memory paging which will wear down the internal ssd?
- MithrilTuxedo 6mo ago
  I assumed you'd want to run one immutable model that can fit in memory without any temp files.
dmitrygr 6mo ago
> Existing machine is no cost for you to run this. That is not at all how modern chips work. Idle chips are mostly powered down, non-idle ones are working and that causes real measurable wear and tear on the silicon. CPU, RAM, NAND all wear and tear measurably with use on current manufacturing processes. https://en.wikipedia.org/wiki/Electromigration https://en.wikipedia.org/wiki/Electromigration
- Barbing 6mo ago
  Like pitching "drive rideshares for only the cost of your time & gas"
- ta988 6mo ago
  The question is, do they wear faster than they become obsolete, as in much more expensive to run than buying a new one with higher compute/watt. (and you can also factor in the ability to run latest models at usable speed)
  - dmitrygr 6mo ago
    It’s complicated. When you design with modern PDKs, you consider the expected duty cycle of the device, expected temps, and the wear and tear on the silicon all together. That affects the layout of the chip as well as certain choices about widths of various features. Generally, one designs consumer SoCs to last 10 years with the expected duty cycle (low). With more wear you could run out of your “years" much faster, maybe even before the warranty.
LPisGood 6mo ago
My question is why did you have to design this to use an MDM instead of a simple program running in the terminal or something?

stavros 6mo ago

You're not taking into account the thermal strain on the machine, though. A machine that's 100% utilized (even worse if it's in bursts) will last less than an idle machine.

embedding-shape 6mo ago
> A machine that's 100% utilized (even worse if it's in bursts) will last less than an idle machine. How much though? Say I have three Mac Minis next to each other, one that is completely idle but on, one that bursts 100% CPU every 10 minutes and one that uses 100% CPU all the time, what's the difference on how long the machines survives? Months, years or decades?
washadjeffmad 6mo ago
Not appreciably, and not before a 5-yr AppleCare+ warranty expires. Out of our >3000 currently active Apple Silicon Macs, failures due to non-physical damage are in the single digits per year. Of those, none have been from production systems with 24/7 uptime and continuous high load, which reflects your parenthetical. Perhaps we haven't met the other end of the bathtub curve yet, but we also won't be retaining any of these very far beyond their warranty period, much less the end of their support life.
- stavros 6mo ago
  I think the point of this is more "use the machine you have at home" than "do a TCO analysis and see if it's profitable", though. People like to keep their machines working for longer, generally.
- alsetmusic 6mo ago
  > Not appreciably, and not before a 5-yr AppleCare+ warranty expires. It’s three years for Macs, though I believe you can pay annually for longer. Five has never been a thing to my knowledge.
- bbatsell 6mo ago
  AppleCare+ annual is perpetual as long as you keep paying it (and Apple offers to switch to that when your 3-year lump sum expires if you choose that instead). I’m guessing it ends whenever they officially discontinue hardware support, which has traditionally been about 7 years after the last unit is produced, but I haven’t reached that yet to know for sure.

mbesto 6mo ago

> That's why we don't recommend purchasing a new machine. Existing machine is no cost for you to run this. You misunderstood. If the ROI is there, there is enough capital in existence for you to accelerate your profit. So why even deal with the complexity of renting people's hardware when you can do it yourself?

splintercell 6mo ago
No, what he's saying is that he expects this to be the ROI in the future because his product is so good.

BuildTheRobots 6mo ago

I don't worry about bandwidth or constant CPU use, but the one thing that will kill my mac is burning out the SSD. The calculator gives numbers for nearly everything, but I can't obviously see how much space it needs for model storage or how many writes of temp files I should expect if I'm running flat out.

sleepybrett 6mo ago
put that stuff on an external disk perhaps, it will eventually crater, but it's easier to replace than macbook internal storage (how are they doing mac minis these days?)
- neurostimulant 6mo ago
  Isn't high memory pressure will result in high memory paging which will wear down the internal ssd?
MithrilTuxedo 6mo ago
I assumed you'd want to run one immutable model that can fit in memory without any temp files.

dmitrygr 6mo ago

> Existing machine is no cost for you to run this. That is not at all how modern chips work. Idle chips are mostly powered down, non-idle ones are working and that causes real measurable wear and tear on the silicon. CPU, RAM, NAND all wear and tear measurably with use on current manufacturing processes. https://en.wikipedia.org/wiki/Electromigration https://en.wikipedia.org/wiki/Electromigration

Barbing 6mo ago
Like pitching "drive rideshares for only the cost of your time & gas"
ta988 6mo ago
The question is, do they wear faster than they become obsolete, as in much more expensive to run than buying a new one with higher compute/watt. (and you can also factor in the ability to run latest models at usable speed)
- dmitrygr 6mo ago
  It’s complicated. When you design with modern PDKs, you consider the expected duty cycle of the device, expected temps, and the wear and tear on the silicon all together. That affects the layout of the chip as well as certain choices about widths of various features. Generally, one designs consumer SoCs to last 10 years with the expected duty cycle (low). With more wear you could run out of your “years" much faster, maybe even before the warranty.

LPisGood 6mo ago

My question is why did you have to design this to use an MDM instead of a simple program running in the terminal or something?