5 ms·

That’s a huge burden on the natural resources (electricity). Beyond local control, I gave other examples of embedded AI in other comments — or more specifical

by hosh 4y ago

That’s a huge burden on the natural resources (electricity). Beyond local control, I gave other examples of embedded AI in other comments — or more specifically, what you can do with an embedded LLM. Namely, being able to have a better human interface for complex settings, and being able to reprogram protocols (or anything that is “software-defined”) for future-proofing.

mirker 4y ago
Integrating an accelerator into a dishwasher is also expensive. How many inferences are you going to get out of it before the hardware is outdated? Once a day for 5 years may not be enough to justify the cost. Compare that to having a single GPU in a server in your house that services all your IoT devices, which can be readily upgraded and requires no special software. The is no benefit to doing inference on device unless you are in the business of selling said devices.
- hosh 4y ago
  A big part of TinyML is finding alternate paths for implementing AI in resource-constrained space. We already have SoC that is functionally not so much different from modern microcontrollers. Depending on economics of scale, it isn’t that big of a leap of imagination to see a microcontroller which includes 4-bit vector ops like a GPU.
STM32F030R8 4y ago
>huge burden on natural resources Can you explain? I don’t see how sending data to the cloud is a huge burden compared to say an EV or your AC unit.
- hosh 4y ago
  If by "private cloud", we mean carving out something from a public cloud, I'm still not sure I'd want something as important and private as AI to be in a cloud facility. Even if you put together a private cloud in a data center, it's still going to use up a lot more electrical resources compared to say, an iphone-sized usage, much less in a low-powered, embedded application. Also, there's a tendency for our civilization, when we make efficiency gains with breakthrough technologies, to then expand our usage. We don't do a great job of actually reducing overall energy expenditure.

by hosh 4y ago

mirker 4y ago

Integrating an accelerator into a dishwasher is also expensive. How many inferences are you going to get out of it before the hardware is outdated? Once a day for 5 years may not be enough to justify the cost. Compare that to having a single GPU in a server in your house that services all your IoT devices, which can be readily upgraded and requires no special software. The is no benefit to doing inference on device unless you are in the business of selling said devices.

hosh 4y ago
A big part of TinyML is finding alternate paths for implementing AI in resource-constrained space. We already have SoC that is functionally not so much different from modern microcontrollers. Depending on economics of scale, it isn’t that big of a leap of imagination to see a microcontroller which includes 4-bit vector ops like a GPU.

STM32F030R8 4y ago

>huge burden on natural resources Can you explain? I don’t see how sending data to the cloud is a huge burden compared to say an EV or your AC unit.

hosh 4y ago
If by "private cloud", we mean carving out something from a public cloud, I'm still not sure I'd want something as important and private as AI to be in a cloud facility. Even if you put together a private cloud in a data center, it's still going to use up a lot more electrical resources compared to say, an iphone-sized usage, much less in a low-powered, embedded application. Also, there's a tendency for our civilization, when we make efficiency gains with breakthrough technologies, to then expand our usage. We don't do a great job of actually reducing overall energy expenditure.