4 ms·

Even so, you can run models on a private cloud. No need to do anything but sense on an embedded device.

• hosh 4y ago
  That’s a huge burden on the natural resources (electricity). Beyond local control, I gave other examples of embedded AI in other comments — or more specifically, what you can do with an embedded LLM. Namely, being able to have a better human interface for complex settings, and being able to reprogram protocols (or anything that is “software-defined”) for future-proofing.
  • mirker 4y ago
    Integrating an accelerator into a dishwasher is also expensive. How many inferences are you going to get out of it before the hardware is outdated? Once a day for 5 years may not be enough to justify the cost. Compare that to having a single GPU in a server in your house that services all your IoT devices, which can be readily upgraded and requires no special software. The is no benefit to doing inference on device unless you are in the business of selling said devices.
    • hosh 4y ago
      A big part of TinyML is finding alternate paths for implementing AI in resource-constrained space. We already have SoC that is functionally not so much different from modern microcontrollers. Depending on economics of scale, it isn’t that big of a leap of imagination to see a microcontroller which includes 4-bit vector ops like a GPU.
  • STM32F030R8 4y ago
    >huge burden on natural resources Can you explain? I don’t see how sending data to the cloud is a huge burden compared to say an EV or your AC unit.
    • hosh 4y ago
      If by "private cloud", we mean carving out something from a public cloud, I'm still not sure I'd want something as important and private as AI to be in a cloud facility. Even if you put together a private cloud in a data center, it's still going to use up a lot more electrical resources compared to say, an iphone-sized usage, much less in a low-powered, embedded application. Also, there's a tendency for our civilization, when we make efficiency gains with breakthrough technologies, to then expand our usage. We don't do a great job of actually reducing overall energy expenditure.
• alex_sf 4y ago
  'private' cloud.
  • mirker 4y ago
    1 GPU in a NAS server is all you need. Pretty simple.
    • alex_sf 4y ago
      I agree, but that's colo/on-prem. Not a private cloud, which is typically just some allocation of dedicated resources from a public cloud.
      • JohnFen 4y ago
        Huh. I always thought of "private cloud" as meaning "my own server", not a carveout on a public cloud. I learned something new today. How does such a "private cloud" differ from using the "public cloud"? The two seem identical to me.
        • alex_sf 4y ago
          Well, it can. The definition of public vs. private cloud depends entirely on who is selling it to you. It's not even a binary at many of the large providers: you can have dedicated servers ("my own server") in a public cloud at AWS, for example. Personally, I keep it simple. If I can physically hit a box and no-one can get mad at me, then it's private and mine.
• JohnFen 4y ago
  A device that doesn't need a network connection to work beats a device that does every time.