17 ms·
Google Edge TPU Devices
- gtm1260 8y agoThis has been around for a while but has been stuck at 'Coming Soon' forever. Does anyone know what the status of this project actually is? I suspect that it has been stalled for some reason or the other.
- mondoshawan 8y agoOne of the Google engineers on the project here. No, we haven't stalled. Keep an eye out. :-)
- lanevorockz 8y agoGoogle has been marketing the TPUs for a long time but they were not even using them much as they were still on Nvidia stack. Not sure if it changed on the past 9 months but my guess is that they are able to run tensorflow on them.
- theDoug 8y agoThis is nonsense.
- TaylorAlexander 8y agoYep. Google definitely uses TPU internally.
- theDoug 8y agoWe do sell access to a huge variety of NVIDIA GPU via Google Cloud Platform[1], which is maybe where the poster's confusion came from. Shrug. 1: https://cloud.google.com/gpu https://cloud.google.com/gpu
- bpye 8y agoI don't know enough about this but how do these devices compare to the Sipeed MAIX devices [1] I saw mentioned on HN the other day? They seem to both support TensorFlow Lite but that's where my ability to understand their capabilities end. [1] - https://www.indiegogo.com/projects/sipeed-maix-the-world-first-risc-v-64-ai-module/x/20407509#/ https://www.indiegogo.com/projects/sipeed-maix-the-world-fir...
- joshvm 8y agoThe risk with crowd funded chips is support and longevity. The reason that the Raspberry Pi wins every time against technically superior competition is that it's well supported and there are reasonable supply guarantees. The same goes for big companies of course. Intel has a habit of releasing Iot platforms and then killing them. Let's hope the TPU lasts a bit longer. As for a comparison, it's impossible to say until Google releases benchmark information on the edge TPU, or some kind of datasheet for the SOM.
- ocdtrekkie 8y agoGiven Google's tendency to kill products and shift priorities rapidly, I think building a product or service dependent on a supply of their hardware is probably a pretty risky choice. I definitely have been shocked how fast Intel maker boards have come and gone though. It feels like Intel has written them off before anyone's tried to build a project using one. I have one sitting around here somewhere that's never so much as been powered on.
- joshvm 8y agoIt's very hard to beat the traction that the Pi has. I think because it's explicitly targeted towards people without any embedded experience, there's been a lot of pressure to make things work and to make the documentation somewhat organised. Intel made some nice little boards, but there wasn't much publicity and actually getting started with them wasn't easy at all because the docs were buried. They were usually modules designed for integration, not standalone devices. With the Pi you can buy a kit, plug in the SD card and boot ot desktop in minutes.
- walterbell 8y agoIn Nov 2018, Google engineers who designed the TPU gave a presentation about the use of open-source Chisel for designing the ASIC, https://youtube.com/watch?v=x85342Cny8c https://youtube.com/watch?v=x85342Cny8c From https://techcrunch.com/2018/07/25/google-is-making-a-fast-specialized-tpu-chip-for-edge-devices-and-a-suite-of-services-to-support-it/ https://techcrunch.com/2018/07/25/google-is-making-a-fast-sp... > Google will have the cloud TPU ... to handle training models for various machine learning-driven tasks, and then run the inference from that model on a specialized chip that runs a lighter version of TensorFlow that doesn’t consume as much power ... dramatically reduce the footprint required in a device that’s actually capturing the data ... Google will be releasing the chip on a kind of modular board not so dissimilar to the Raspberry Pi ... it’ll help entice developers who are already working with TensorFlow as their primary machine learning framework with the idea of a chip that’ll run those models even faster and more efficiently.
- mrlambchop 8y agoThis was an amazing sunday night rabbit hole to go down - thanks!
- fizixer 8y agoSo my guess is that Chisel is one of the many responses to the two horrors that are VHDL and Verliog. Unfortunately, Chisel is built on Scala, and I have no interest in learning Scala. Though I'm intrigued by the claim of using generators and not instances, and would be interested in a white paper that explains it in PL-agnostic terms (PL: programming language). Also have on my to-do list MyHDL [1], a Python solution to the same problem. (has anyone tried it and found to be better than VHDL/Verilog?) [1] http://www.myhdl.org/ http://www.myhdl.org/
- Cyph0n 8y ago> Unfortunately, Chisel is built on Scala, and I have no interest in learning Scala. That's a strange reason for not wanting to reap the benefits of Chisel. Care to explain your rationale? There is another compile-to-HDL "language" called SpinalHDL[1], so I would actually argue that Scala's metaprogramming features seem to be a good fit for this usecase. [1] https://github.com/SpinalHDL/SpinalHDL https://github.com/SpinalHDL/SpinalHDL
- joshvm 8y agoThis looks cool. Currently the only real options for amateur off-the-shelf (accelerated) edge ML are the Nvidia boards (but small carrier boards for the TX2 cost more than the module itself) or the Intel NCS which inexplicably blocks every other USB port on the host device due to its poorly designed case. There is the Movidius chip itself, but Intel won't sell you one unless you're a volume customer. The NCS also does bizarre things: the setup script will clobber an existing installation of opencv with no warning, for example. There are various optimised machine learning frameworks for ARM, but I'm only counting hardware accelerated boards here. I'm also not including the various kickstarter or indiegogo boards which might as well be vapour ware. There are no good, cheap, embedded boards with USB3 that I can find. There are a few Chinese boards with USB3, but none of them have anywhere near the quality of support that the Pi has. Then camera support. The Pi has a CSI port, but it's undocumented and only works with the Pi camera. The TX2 is pretty good, but you need to dig through the documentation to figure things out. USB is fine, but CSI is typically faster and frees up a valuable port. Finally another issue is fast storage. It's difficult to capture raw video on the Pi because you can't store anything faster than about 20MB/s. There are almost no boards that support SATA or similar (the TX2 does), so the ability to use USB3 storage would be welcome too. If this is offered at a reasonable price point, it could be a really nice tool for hobbyists. It looks like they're trying to keep GPIO pin compatibility with the Pi too.
- walterbell 8y ago> If this is offered at a reasonable price point Hopefully it will, since the voice and vision kits on the same AIY page are sold for $50 at Target. > There is the Movidius chip itself, but Intel won't sell you one unless you're a volume customer Single units are listed on this page, e.g. mini-PCIe board with Movidius VPU for $79: https://up-shop.org/25-up-ai-edge https://up-shop.org/25-up-ai-edge
- joshvm 8y agoGood find, although they're not out yet by the looks of it? And also pushing the price of a TX2 if you want the dev board plus a vision carrier (though it does have 3 VPUs). I was referring to the Movidius Dev Kit which exists, but seems impossible to buy as a consumer.
- dwrodri 8y agoA related edge computing AI accelerator: https://www.crowdsupply.com/up/ai-core-x https://www.crowdsupply.com/up/ai-core-x
- peterwwillis 8y agoWhen I see products developed by Google, I imagine them as Replicants. Developed by an advanced tech company, full of futuristic technology and amazing potential, and destined to die in ~4 years.
- jrockway 8y agoThe competitors don't really keep chips around for longer. Intel isn't manufacturing Skylake anymore. Nvidia isn't manufacturing Maxwell GPUs anymore. (Incidentally, Apple did appear to be using their 4-year old A8 SoC in the first HomePods, released in 2018, though.) Hardware and software are different things. We are all sad that Google Reader doesn't exist anymore, but every silicon product has basically been a flash in the pan. They make it, you buy it, and by the time it's shipped to you, it's announced as obsolete. That's the pace of that industry. Maybe with Google's attention span, they should have been a hardware company all along. They will fit right in.
- peterwwillis 8y agoMy example was inaccurate, because at least dead Replicants can be replaced with newer models, whereas dead Google products have no follow-up model and require completely replacing what you had created around that product. That's something seemingly unique to either vaporware start-ups, or Google.
- jacquesm 8y agoAt least you can still write software for those. If Google decides your particular flavor of chip is no longer supported then good luck. Besides that, good luck to acquire those chips in the first place, 'coming soon' without a stated delivery date may well translate into 'never'. I'll stick to the usual suspects before I get roped into some cloud based development system. Why does Google need access to my IP to begin with?
- xrisk 8y agoHow expensive is this likely to be? Feasible for a hobbyist to purchase?
- lanevorockz 8y agoPlenty of affordable options on the market already. For a hobbyst I would highly recommend to use tegra instead. It is compatible with all frameworks and won’t lock you down on google stack
- neuromancer2701 8y agoThe NXP® i.MX 8MQuad board is available for 150$ and has USB3 and PCIe. The TPU would probably be attached through one of those buses. I would bet around 250$ with the TPU which is pretty good and puts it around half the Jetson Tx2, 1/5 the Xavier. I wonder if the TPU could be used for SLAM not just object identification, now that would be useful
- voltagex_ 8y agoDo you know where I can buy the board itself? (without the TPU), I only saw "SOM" versions which I'm not equipped to use.
- Odz86 8y agoThe Dev Board is $149.99, the USB Accelerator is $74.99. (I was invited for Beta Program.)
- potatofarmer45 8y agoThis is a long time coming. I'm normally not a big fan of large companies building products in the embedded space that could potentially destroy competition and future innovation but this is needed. Nvidia's embedded boards are EXPENSIVE. So expensive it limits the applications dramatically. They also require a different skillset in people to set up which drives up the cost. We did an analysis for a security project that required visual inference. It turned out all the extra costs to setup with TX boards meant it actually made more sense to have mini desktops with consumer gtx cards. I am excited to see the performance of the inference module. If it's decent at a good price, that opens up so many pi/beagle/arduino applications that were limited by both cost and form factor of existing options.
- arthurcolle 8y agoWhich of Nvidia's boards are you referring to?
- lanevorockz 8y agoNvidia provides a line of embedded systems for accelerated compute called tegra. It’s pretty awesome kit but costs from 150-500, depending of the compute necessary. Probably a new one will be announced in a months time, hence google is trying to get ahead
- newfocogi 8y agoThe new Tegra Xavier kits costs ~$1300 and were first released to developers a few months ago
- TorKlingberg 8y agoTegra is Nvidia's line of embedded SoCs. Are you talking about the Jetson boards and modules?
- lanevorockz 8y agoNot so sure how much this kit will cost but I wouldn’t put my hopes it will be cheaper per compute than nvidia tegras. It could be a good alternative for lower end compute though.
- Rafuino 8y agoHow would someone compare the Edge TPU Accelerator with a Movidius Neural Compute Stick? https://software.intel.com/en-us/movidius-ncs https://software.intel.com/en-us/movidius-ncs
- m0zg 8y agoMovidius is not a TPU. It's more like a GPU, but with SIMD, DSP and even VLIW capabilities and with a _very_ wide memory bus (and massive throughput). It's rather impressive actually, but probably serious overkill for what really needs to be done during inference: https://en.wikichip.org/wiki/movidius/microarchitectures/shave_v2.0 https://en.wikichip.org/wiki/movidius/microarchitectures/sha.... Whereas a TPU is highly specialized for just, you guessed it, processing tensors, which basically means matrix and vector multiply. It's a systolic architecture, so it also (purportedly, since I don't have insider knowledge) stores the weights for the computation for the duration of the computation.
- sanxiyn 8y agoWe hardly know anything about Edge TPU. Yes, TPU uses systolic array, but there is really no reason to believe Edge TPU will use it. Edge TPU is not TPU.
- m0zg 8y agoIt has "TPU" right its name. If that's not a dead giveaway, I don't know what is. :-)
- TorKlingberg 8y agoAs far as I understand "TPU" is Google's brand name, so of course competing products are not TPUs. There is an overlap in what you can do with the devices, so a comparison of their strengths would be useful.
- lambda 8y agoAm I missing something, or are they not only not documenting the chip, they are also not even releasing the compiler, but requiring you to use a cloud based compiler: > You need to create a quantized TensorFlow Lite model and then compile the model for compatibility with the Edge TPU. We will provide a cloud-based compiler tool that accepts your .tflite file and returns a version that's compatible with the Edge TPU. This seems like a new low in software freedom, and pretty risky to depend on as Google is known to shutter services pretty often and could just decide to turn off their cloud-based compiler at any time they feel like.
- diminoten 8y agoGoogle's reputation for shuttering services is vastly overstated, and beating this particular dead horse every time Google creates something new just wastes people's time.
- mlindner 8y agoWell they decided to not only make it closed source but also lock it up behind a http frontend so you can't even reverse engineer it. Criticizing Google for shuttering things suddenly that companies are depending on is rightly justified.
- diminoten 8y agoLiterally not what they do, but sure, keep up the absurd lie.
- royal_ts 8y agoWhich paid Services are shutdown regularly?
- seanp2k2 8y agoThere are even dedicated sites for the products they've killed off over the years: https://www.wordstream.com/articles/retired-google-projects https://www.wordstream.com/articles/retired-google-projects https://gcemetery.co/ https://gcemetery.co/ https://killedbygoogle.com/ https://killedbygoogle.com/ https://en.wikipedia.org/wiki/Category:Discontinued_Google_s https://en.wikipedia.org/wiki/Category:Discontinued_Google_s ervices https://www.businessinsider.com/discontinued-google-products-2016-8 https://www.businessinsider.com/discontinued-google-products...
- polskibus 8y agoIs this anything worth considering, if it cannot be used for training?
- justinjlynn 8y ago> You need to create a quantized TensorFlow Lite model and then compile the model for compatibility with the Edge TPU. We will provide a cloud-based compiler tool that accepts your .tflite file and returns a version that's compatible with the Edge TPU. I seriously hope that's not the only way they're expecting people to compile models for this particular TPU.
- dejv 8y agoAs a person with access to their documentation I can confirm that this is currently the only way to compile model for this TPU. Also list of supported network architectures is very very short. Well this is just begining I am sure they are going to expand its capability.
- justinjlynn 8y agoI'd rather they enable independent development of models on the hardware they're selling. This is about as useful as a high performance electric car you can only charge at authorised dealerships. Dealerships which have the unfortunate habit of closing down after a few years or so.
- dejv 8y agoI bet there is some technical reason behind allowing just few architectures and I hope they will fix it in future releases. Currently you can't even run ResNet type of networks on it.
- justinjlynn 8y agoWell, one can hope.
- pilooch 8y agoWhat is the use of 100fps vision models other than being the input to a controller (e.g. driving, flying etc) ? A raspberry can hold up to 3fps with standard open source frameworks, and this is enough for many applications, e.g. construction site surveillance etc... Not criticizing, rather a genuine interest to understand the edge ML vision market.
- dejv 8y agoI did work on optical sorting machine: you have stream of fruits on very fast conveyor belt and then machine vision system scans the passing fruits, detects each object and reject (by firing stream of air) those that don't pass: those can be molded fruits, weird color or foreign material like rocks or leaves. 100 fps might be enough, but faster you go, faster your conveyor belt could be.
- ovi256 8y agoThe 100fps model is also much more efficient in W/Flop, or J/Flop, or W/fps which is very important for embeded and mobile applications. You can design your construction site surveillance system to record 10 frames a minute while sleeping the ML accelerator and then process 100 frames all at once in a few seconds, which reduces the duty cycle tremendously, improving battery / device life.
- michaelt 8y agoBarcode scanners routinely perform 100 scans per second. Of course, some would say machine learning is needless complexity if you just want to scan barcodes :)
- petra 8y agoSo they will sell develoent boards, without selling IC's. Seems like a nice way to gather ideas and data about new products.
- AndrewKemendo 8y agoAre they going after the Movidius [1] with this? [1]https://software.intel.com/en-us/movidius-ncs https://software.intel.com/en-us/movidius-ncs
- deleted 8y ago[deleted]