7 ms·
BeagleY-AI: a 4 TOPS-capable $70 board from Beagleboard
- lproven 3y agoI honestly thought this thing was going to run TOPS-10 or TOPS-20. I'm a bit disappointed. https://en.wikipedia.org/wiki/TOPS-10 https://en.wikipedia.org/wiki/TOPS-10 https://en.wikipedia.org/wiki/TOPS-20 https://en.wikipedia.org/wiki/TOPS-20
- johnohara 3y agoGiven the recent HN post about an update to the OpenVMS project I thought the same thing when I saw "TOPS-capable." Hmm, maybe some kind of port from long-retired propeller heads at DEC/Compaq/HP. Nope. Looked up "4 TOPS" and learned the famed Motown group was still performing though some of the original members had passed. Finally determined that "TOPS" did indeed mean "trillion operations per second." Wasn't so long ago these types of things were only found inside some mountain laboratory surrounded by heavily-guarded 5 mile restricted access perimeters. ~$70.00 sometime in June 2024. Don't tell me the industry isn't still sprinting all-out 54 years after TOPS-10 was first introduced.
- shadowpho 3y ago4GB of RAM seems low. No NVME even though it has PCIe :(
- diggan 3y agoDoesn't PCIe mean you could trivially add support for NVME though via an extension board?
- shadowpho 3y agoHuge PITA. Do they even support raspberry pi hats? If yes that's another $25, plus now it blocks your fan unless you get the proprietary sideways fan hat. It also won't mount into any cheap cases anymore. This means it goes from "hey I can buy this for $70, throw into a case with 128gb nvme and run stuff" to "ok so $70 + $25 hat + different fan hat + $40 to get taller clearances"
- T-A 3y agoMeanwhile, in Taipei... https://www.extremetech.com/computing/intel-40-tops-is-the-new-baseline-for-an-npu https://www.extremetech.com/computing/intel-40-tops-is-the-n...
- weikju 3y agoreminds me of the "multimedia PC" of the 90s. But at least that branding didn't enforce cloud services and Microsoft telemetry
- sunshine-o 3y agoOk Microsoft, let's add a copilot key to our keyboards. But can we then remove the Windows key? unclear...
- Alupis 3y agoBeagleBoard is akin to a Raspberry Pi. It's not meant to be a powerhouse PC, or in this case, a powerhouse AI computing platform. It's meant for embedding, tinkering, and learning. To that end, 4GB of RAM on an AI Accelerator board is fine - the expected workloads will not consume a lot of RAM. This also makes the lack of NVMe sufficient as well. For more "horsepower" there is also the BeagleBone AI-64[1], which claims up to 8 TOPS. [1] https://www.beagleboard.org/boards/beaglebone-ai-64 https://www.beagleboard.org/boards/beaglebone-ai-64
- mmoskal 3y agoLooks like 1/10 of TOPS and 1/3 of mem bandwidth of Jetson Orin Nano (the TI chip seems able to run 3733MT/s at 32 bit, so 15GB/s) but much cheaper.
- Alupis 3y agoFor reference: - BeagleY-AI - 4 TOPs - $72.00 - BeagleBone AI-64 - 8 TOPs - $187.50 - Jetson Orin Nano Dev Kit - 40 TOPs - $499.95 The 20 TOPs Jetson Orin Nano seems to only be available in a commercial module, not in a dev kit. The commercial module is priced at $259.00.
- phlipski 3y agoKeep in mind that once you've prototyped a product on the BeagleY-AI board - you can actually BUY the TI AM62A74 chips in volume for ~ $25. Good luck sourcing the SoC on that Jetson Orin Nano board...
- reaperman 3y agoThe 5-year old Google Coral Edge TPU does 4 TOPS for $40-130. Comparing exact features, the $70 BeagleY-AI probably most directly compares to Coral's $130 option. https://coral.ai/products/ https://coral.ai/products/ https://www.aliexpress.us/item/3256806327903926.html?src=google&aff_fcid=f349642bbd2b4929abf62d2bcaee5733-1710095245201-07681-UneMJZVf&aff_fsk=UneMJZVf&aff_platform=aaf&sk=UneMJZVf&aff_trace_key=f349642bbd2b4929abf62d2bcaee5733-1710095245201-07681-UneMJZVf&terminal_id=8b34d788ea6d40939f9e0056c00383e3&afSmartRedirect=y&gatewayAdapt=glo2usa https://www.aliexpress.us/item/3256806327903926.html?src=goo...
- RantyDave 3y agoAI accelerator? Really? I thought AI acceleration in this day and age meant more than multiplying two matrices together. If not, what's CUDA all about?
- imtringued 3y agoYou are confusing programmable shaders and GPGPU with AI. AI only needs matrix multiplication and activation functions to be accelerated. For everything else the GPU is already so fast there is no point in further improvements.
- mynegation 3y agoI wish there was some reference of what can I actually do with 4 or 8 TOPS and 4 or 8 or whatever GiB of accelerator memory. Can I run speech recognition? Run a model for object detection in images? In video? LLMs - probably not. Stable Diffusion also seems out of question. But really - what can I run?
- deleted 3y ago[deleted]
- yjftsjthsd-h 3y agoAgreed! What I really want to know from any hardware accelerator is: * What can it accelerate? * What software supports it (and what's the state of driver support)? * How much faster or more power efficient is it vs CPU/GPU? * How exactly do I use it? (Okay, I buy the thing, it arrives in a box, I plug it in... what exact steps do I take to get to running real workloads on it?)
- randunel 3y agoThere's about two 240p object detection demo videos with source code, one picture object detection demo, a bunch of forum posts with incomplete code about some audio processing, and they're all about a different SBC. Software wise, there's not much to it, you're on your own.
- jdietrich 3y ago4 TOPS is last-gen smartphone territory - you aren't going to have a good time running big generative models, but it's a useful amount of performance for typical edge workloads like machine vision and robotics.
- sciencesama 3y agogoogle coral tpu is 4Tops btw
- hcfman 3y ago
- Havoc 3y agoOther boards have NPUs with 4 tops too plus more mem. Eg the orange pis. … but doesn’t look like any of the usual LLMs suspects can run on NPU so not sure it’s much use. I’ve seen some opencv code but that’s about it
- deleted 3y ago[deleted]
- sandwichukulele 3y agoI don't know anything about the Beagle. How is it different than say a Raspberry pi 5? It says open source hardware but what does that actually mean. I'm curious about buying one but I want to understand them better first
- Retr0id 3y agohttps://openbeagle.org/beagley-ai/beagley-ai https://openbeagle.org/beagley-ai/beagley-ai - at a glance, it looks like there's everything you need to be able to fab your own boards, if you really wanted to.
- a2800276 3y agoWell it's certainly a good idea to get a better understanding of potential purchases. Unmotivated impulse purchases just clutter up your mind, your house and are bad for the environment. "If you have to ask" you're almost certainly better off with a rasp5 or probably 4 if you are looking for a mini PC to get some use out of that monitor you have flying around. Raspberry has a larger community and is therefore more accessible when starting out. If you're looking to get a better understanding of the "low-level" workings of computers, consider an Arduino.
- Gibbon1 3y agoSimilar but Beaglebones have flash drive controller which they boot off so they do better at not corrupting themselves on power cycling. I don't know about the BeagleY-AI but the Beaglebone black's power sequencer has won't fix issues that cause the board to fail to start if the power comes on too slow. Open source hardware means they give you schematics and the board layout as an Altium project. You can take that and customize it to fit your needs. When it comes to processors that can run embedded linux that's actually common. Advantage if you don't modify the layout of the high speed buses your prototype will likely just work.
- moffkalast 3y ago> Quad-core 64-bit Arm®Cortex®-A53 CPU subsystem at 1.4GHz It's slower. > Memory 4GB LPDDR4 Has less and slower memory. > Arm Cortex-R5 subsystem for low-latency I/O and control Might use less power at idle, the Pi 5 is extremely power hungry when doing nothing (~5W). > Dual general-purpose C7x DSP with Matrix Multiply Accelerator (MMA) capable of 4 TOPs An AI chip with questionable software support. The rest seems about the same, it's loosely based on the same PCB layout.
- kitd 3y ago4 TOPS-capable So, reach out & it'll be there?
- fithisux 3y ago4GB RAM is not much.
- moffkalast 3y agoAnd it's not honest work either. LPDDR4 is now more expensive per GB and in smaller packages than LPDDR4X, plus it uses twice as much power. It's a lose-lose-lose situation not to use it.
- rcarmo 3y agoThis looks interesting, but the trouble with these small built in accelerators is that they were mostly designed when YOLO was the pinnacle of edge applications. These days they’re grossly underpowered…
- tucosan 3y agoPlus, they have way too little memory for any meaningful LLM workloads that go beyond simple inference tasks or running yolo.
- bradfa 3y agoGrossly underpowered compared to a fat stonking GPU, yes. But the whole BeagleY-AI board probably consumes less power at full tilt than just the fans to keep a modern 2-slot NVIDIA GPU cool. For embedded applications, the C7x DSPs are mighty. Lots of existing C6x DSP code can easily be recompiled to target the C7x with minimal effort as well, which is a big deal. Making your C6x code work efficiently on the C7x may require more than just a recompile, but being able to leverage existing C6x DSP codebases with minimal investment while getting the extra performance of the C7x is a very big deal. I don't think a quad-core ARM Cortex-A53 with a pair of C7x DSPs is trying to compete with an NVIDIA Hopper. But if you're making a $100 embedded product that has to handle video, this seems quite attractive.
- rcarmo 3y agoI’ve been trying to use rknn on RK3588 boards (which can go a bit higher in wattage due to the rest of the SOC and peripherals), and I get your point, but the overall landscape for non-GPU accelerators is still pretty much… just plain bad. I’m hoping for Ryzen APUs to be a good stopgap for larger models, or for enough support for Mali GPUs (which can add a little more compute) to be more usable, but in general, you have a huge abyss between “oh, that’s a face in that picture” and “here’s my current estimation of movement for this human”.
- teaearlgraycold 3y agoWhat do the cool kids use these days? I know YOLO is still relevant but I’m not sure what’s the new stuff.
- cjs_ac 3y agoI find it interesting that the single-board computer market seems to be coalescing around the Raspberry Pi B models as the standard form factor. This device in particular has almost all of the same IO connectors as the Raspberry Pi 5 (one microHDMI port is missing) in all the same places, so it should be compatible with Raspberry Pi 5 cases. I wonder whether the pinout on the 40-pin header is the same as that on the Pi? I think most of us here are familiar with the fact that amd64 machines are made entirely from commodity parts: ATX cases, ATX power supplies, and so on. I wonder whether there's a similar commodification in the offing for the Pi form factor?
- franciscop 3y agoBeware of the documentation though, which is literally a blank page "coming soon!": https://docs.beagleboard.org/latest/boards/beagley/ai/ https://docs.beagleboard.org/latest/boards/beagley/ai/
- Double_a_92 3y agoI can't wait for it to be sold out, or only being sold at shops that wan't like 40$ for international shipping...
- RobotToaster 3y agoIs the low level firmware open source? I know it isn’t on the rpi, which runs a proprietary broadcom version of microsoft threadx.
- synergy20 3y agoJetson Orin Nano has no video hardware encoder, probably no hardware decoder either, you need use CPU to do the heavy computing in 2024. Not sure about BeagleY-AI here, since it has DSP inside, probably also doing software encoding, using DSP instead of CPU though. For power efficiency, I would think the cheap hardware encoder is the way to go, surprised both are using software as the encoders.
- deleted 3y ago[deleted]
- andrewmcwatters 3y agoI think we need some sort of mlbench but for hardware ranking... I have no idea what these devices are capable of and neither do the vendors apparently.
- phlipski 3y agoMLCommons provides MLPerf which is a series of benchmarks anyone can run and compare results. For devices like the BeagleY-AI the "edge" category provides a series of various benchmarks to run covering things like object detection, speech recognition, and now smaller LLM's. https://mlcommons.org/benchmarks/inference-edge/ https://mlcommons.org/benchmarks/inference-edge/
- renewedrebecca 3y agoWith the two DSPs, seems like this would be a nice board to build a guitar effects pedal around.