3 ms·

2 TOPS is not a lot for AI projects.

by asadm 1y ago

jdiaz97 1y ago
It's a lot for 40 USD. And not every AI project is a Language Model project.
- asadm 1y ago
  no its not. Just use a smaller rockchip, 1-2 actual TOPS in ~$20 as a board or $10 a piece. i am using one for my project.
OhMeadhbh 1y ago
meh. i write a lot of PROLOG, so 2 TOPS is A LOT of horsepower.
muxamilian 1y ago
They get this number by simply adding up all CPUs' processing speed. There is no NPU or similar: https://medium.com/@zlodeibaal/orange-pi-rv2-ai-board-scam-7c71a93d63f1 https://medium.com/@zlodeibaal/orange-pi-rv2-ai-board-scam-7...
- asadm 1y ago
  thats even worse. 0 TOPS! The whole point is offloading load from CPU and doing something in parallel while NPU does its thing.
- dlcarrier 1y ago
  Four of the eight cores have matrix accelerators. If you are using the full 2 TOPS of accelerated performance, you have four more otherwise symmetric cores free to run regular workloads, otherwise you have eight. I'd rather have this setup than 4 regular cores and a separate asymmetric accelerator that provides 2 TOPS to matrix/tensor calculations but otherwise sits idle.
numpad0 1y ago
Hopefully Adafruit or someone could get these[1] out of "Contact Us" jail. 16GB 30TOPS BF16 in M.2 2280 at $369. PCIe 2x8 low profile "Duo" configuration available at $799. Supposedly. I believe the theoretical performance is Strix territory if these could be clustered, but only if they mass manufacture these. 1: https://www.edgecortix.com/en/hardware https://www.edgecortix.com/en/hardware
- CapricornNoble 1y ago
  Heh, interesting you call this "Contact Us Jail". I met sales and engineering staff from EdgeCortix at DSEI Japan 2025, and was very enthusiastic to pick up some of their products as alternatives to the Hailo-8L RPi cards. So far....nothing. Not even an ETA on when they will be shipping hardware.
  - numpad0 1y ago
    DSEI! I wish I had some official reasons that I'd be obligated to go and have fun in events like that :p My fear is that they could be in endemic Japanese death spiral disease of hand-wavingly waiting for massive orders while not doing much to secure one. There are so many cool stuffs and perfect plans from Japanese large companies and government agencies that are pushed forward up to one step before volume production or commercial rollout and then stalled until funding dries up and everyone moves on. Which, if I were to explain by speculating, might be due to people prioritizing own job security over corporate revenues, due to low psychological safety and perceived invincibility of large organizations, as one among many consequences of employment ice age phenomenon. But whatever the mechanism is, it'll be sad if this one follows that path. Hopefully they make and ship them in substantial numbers. That should be their goal anyway.

2 TOPS is not a lot for AI projects.

by asadm 1y ago

2 TOPS is not a lot for AI projects.

jdiaz97 1y ago

It's a lot for 40 USD. And not every AI project is a Language Model project.

asadm 1y ago
no its not. Just use a smaller rockchip, 1-2 actual TOPS in ~$20 as a board or $10 a piece. i am using one for my project.

OhMeadhbh 1y ago

meh. i write a lot of PROLOG, so 2 TOPS is A LOT of horsepower.

muxamilian 1y ago

They get this number by simply adding up all CPUs' processing speed. There is no NPU or similar: https://medium.com/@zlodeibaal/orange-pi-rv2-ai-board-scam-7c71a93d63f1 https://medium.com/@zlodeibaal/orange-pi-rv2-ai-board-scam-7...

asadm 1y ago
thats even worse. 0 TOPS! The whole point is offloading load from CPU and doing something in parallel while NPU does its thing.
dlcarrier 1y ago
Four of the eight cores have matrix accelerators. If you are using the full 2 TOPS of accelerated performance, you have four more otherwise symmetric cores free to run regular workloads, otherwise you have eight. I'd rather have this setup than 4 regular cores and a separate asymmetric accelerator that provides 2 TOPS to matrix/tensor calculations but otherwise sits idle.

numpad0 1y ago

Hopefully Adafruit or someone could get these[1] out of "Contact Us" jail. 16GB 30TOPS BF16 in M.2 2280 at $369. PCIe 2x8 low profile "Duo" configuration available at $799. Supposedly. I believe the theoretical performance is Strix territory if these could be clustered, but only if they mass manufacture these. 1: https://www.edgecortix.com/en/hardware https://www.edgecortix.com/en/hardware

CapricornNoble 1y ago
Heh, interesting you call this "Contact Us Jail". I met sales and engineering staff from EdgeCortix at DSEI Japan 2025, and was very enthusiastic to pick up some of their products as alternatives to the Hailo-8L RPi cards. So far....nothing. Not even an ETA on when they will be shipping hardware.
- numpad0 1y ago
  DSEI! I wish I had some official reasons that I'd be obligated to go and have fun in events like that :p My fear is that they could be in endemic Japanese death spiral disease of hand-wavingly waiting for massive orders while not doing much to secure one. There are so many cool stuffs and perfect plans from Japanese large companies and government agencies that are pushed forward up to one step before volume production or commercial rollout and then stalled until funding dries up and everyone moves on. Which, if I were to explain by speculating, might be due to people prioritizing own job security over corporate revenues, due to low psychological safety and perceived invincibility of large organizations, as one among many consequences of employment ice age phenomenon. But whatever the mechanism is, it'll be sad if this one follows that path. Hopefully they make and ship them in substantial numbers. That should be their goal anyway.