4 ms·

“Having those calculations done on the chip instead of sending information to and from the cloud could enable faster, more secure, cheaper, and more scalable AI

by spyder 4y ago

“Having those calculations done on the chip instead of sending information to and from the cloud could enable faster, more secure, cheaper, and more scalable AI going into the future, and give more people access to AI power,” said H.-S Philip Wong, Huh... why is he comparing it to network (cloud) transfers and not to the current CPU-RAM transfers? I thought the benefit of compute-in-memory is reduction of CPU-RAM transfers and that's what they're also saying in the beginning of the article.

svnt 4y ago
You sell much higher quantities of chips if they are not being time-shared in the cloud. This statement is a between-the-lines message to potential partners and investors that they intend to sell to end users and not data centers, and capture that increase in volume.
KKKKkkkk1 4y ago
They are targeting edge devices like your phone or watch. Computations that use too much battery (e.g., because they do too many cache misses) have to go to the cloud. I am guessing the reason they're targeting edge is because the technology is energy efficient but slow. Any experts can chime in?
- c_o_n_v_e_x 4y ago
  Local inferencing eliminates the round trip latency to/from the cloud. Also, for wireless sensors, you save power by not having to turn on your radio transmitter and receiver on. For non-battery / line powered devices, thermal design is a big consideration at the edge. Running high powered GPUs in non-environmentally controlled environments is difficult. Low power AI chipsets can help reduce product cost, design cost (simpler thermal designs), and decrease product SWaPs.
senttoschool 4y ago
>Huh... why is he comparing it to network (cloud) transfers and not to the current CPU-RAM transfers? I thought the benefit of compute-in-memory is reduction of CPU-RAM transfers and that's what they're also saying in the beginning of the article. He's saying that they can now perform heavier calculations on the edge device using their novel chip design. Currently, these calculations have to be sent to the cloud to perform because chips today can't compute them without wasting too much battery. For example, Siri has recently implemented on-device AI processing for many queries because the neural engine inside Apple SoCs have gotten so much more powerful and efficient. Previously, all Siri requests were sent to the cloud.[0] [0]https://www.theguardian.com/technology/2021/jun/07/apple-overhauls-siri-to-address-privacy-concerns-and-improve-performance https://www.theguardian.com/technology/2021/jun/07/apple-ove...

by spyder 4y ago

svnt 4y ago

You sell much higher quantities of chips if they are not being time-shared in the cloud. This statement is a between-the-lines message to potential partners and investors that they intend to sell to end users and not data centers, and capture that increase in volume.

KKKKkkkk1 4y ago

They are targeting edge devices like your phone or watch. Computations that use too much battery (e.g., because they do too many cache misses) have to go to the cloud. I am guessing the reason they're targeting edge is because the technology is energy efficient but slow. Any experts can chime in?

c_o_n_v_e_x 4y ago
Local inferencing eliminates the round trip latency to/from the cloud. Also, for wireless sensors, you save power by not having to turn on your radio transmitter and receiver on. For non-battery / line powered devices, thermal design is a big consideration at the edge. Running high powered GPUs in non-environmentally controlled environments is difficult. Low power AI chipsets can help reduce product cost, design cost (simpler thermal designs), and decrease product SWaPs.

senttoschool 4y ago

>Huh... why is he comparing it to network (cloud) transfers and not to the current CPU-RAM transfers? I thought the benefit of compute-in-memory is reduction of CPU-RAM transfers and that's what they're also saying in the beginning of the article. He's saying that they can now perform heavier calculations on the edge device using their novel chip design. Currently, these calculations have to be sent to the cloud to perform because chips today can't compute them without wasting too much battery. For example, Siri has recently implemented on-device AI processing for many queries because the neural engine inside Apple SoCs have gotten so much more powerful and efficient. Previously, all Siri requests were sent to the cloud.[0] [0]https://www.theguardian.com/technology/2021/jun/07/apple-overhauls-siri-to-address-privacy-concerns-and-improve-performance https://www.theguardian.com/technology/2021/jun/07/apple-ove...