17 ms·
Epiphany-V: A 1024-core 64-bit RISC processor
- thechao 10y agoIs there a mirror anywhere?
- mmastrac 10y agohttps://webcache.googleusercontent.com/search?q=cache:XCsT2efmgU0J:https://www.parallella.org/2016/10/05/epiphany-v-a-1024-core-64-bit-risc-processor/+&cd=1&hl=en&ct=clnk&gl=ca https://webcache.googleusercontent.com/search?q=cache:XCsT2e... This PDF is a great technical overview as well: https://www.parallella.org/wp-content/uploads/2016/10/e5_1024core_soc.pdf https://www.parallella.org/wp-content/uploads/2016/10/e5_102...
- Coffeewine 10y agoThis is fascinating: The Epiphany-V was designed using a completely automated flow to translate Verilog RTL source code to a tapeout ready GDS, demonstrating the feasibility of a 16nm “silicon compiler”. The amount of open source code in the chip implementation flow should be close to 100% but we were forbidden by our EDA vendor to release the code. All non-proprietary RTL code was developed and released continuously throughout the project as part of the “OH!” open source hardware library.[20] The Epiphany-V likely represents the first example of a commercial project using a transparent development model pre-tapeout.
- codebook 10y agoI might be wrong. But if they automated the flow from RTL to GDS, the timing might not be optimal. I understand since they have lack of resources so that this is unavoidable but in normal chip design flow, the backend timing ECO is critical to achieve high frequency for all timing corners.
- Coffeewine 10y agoI agree completely, it's still impressive to me that they presumably managed a competitive offering with such a system. I imagine having it be a highly homogeneous design also helped.
- adapteva 10y agoDesign symmetry and regularity was the key. Harder to achieve that with a heterogeneous architecture.
- adapteva 10y agoYes, we are leaving 2X on the table in terms of peak frequency compared to well staffed chipzilla teams. Not ideal, but we have a big enough of a lead in terms of architecture that it kind of works.
- sitkack 10y agoThat seems analogous to human assembly optimization vs a compiler. But the time to market is greatly reduced, designs can be vetted and a 2.0 that is optimized for frequency can be shipped later.
- yellowapple 10y agoIIRC, human assembly optimization is unlikely to be better than a modern compiler nowadays. Same thing could very well happen for this "automated flow" if it starts incorporating its own optimization techniques.
- executesorder66 10y ago> but we were forbidden by our EDA vendor to release the code. Why? Is there anything that could be done to change that?
- LeifCarrotson 10y agoRTL = Register Transfer Logic, and EDA = Electronic Design Automation, for anyone else who was curious. I don't know what GDS stands for, but context indicates it's the actual physical description that's used to make the part. But I'm confused about what part of this is open and not open. Do they mean that they imported their Verilog into a proprietary tool, which generates the design? That doesn't make it open source in practice.
- adapteva 10y agoHW design is not that different from SW design. Comp table below: HW SW Verilog --> C/Java/etc EDA --> GCC/LLVM GDS --> Binary (elf) The GDS is completely tied up in NDAs due to the foundry. The EDA combines/translates open source code with proprietary blobs to produce a "super secret" GDS binary blob that gets sent to the foundry for manufacturing.
- LukeShu 10y agoFor anyone else who was confused by everything being on one line: HW SW Verilog --> C/Java/etc EDA --> GCC/LLVM GDS --> Binary (elf)
- NEDM64 10y agoFor everyone still confused Verilog --> imperative language EDA --> IDE + compiler GDS --> Assembly
- ChrisRus 10y ago> HW design is not that different from SW design. Shouldn't be. But it is.
- AceJohnny2 10y agoExcept the economics are vastly different. The complexity and cost of manufacturing, the computationally intensive cost of simulation and various checks and optimizations (be it clock timing or mask optimizations to etch features that are smaller than the wavelength used to etch them), all mean that you can't just "compile and publish", and turnaround times are months, not hours. And there are no open-source toolchains for any of this. It's a student project to implement a SW compiler, why isn't it to implement an RTL compiler?
- PhilWright 10y agoYou should investigate the RISC-V project. It is an open source RISC based ISA along with open source implementations of example processor cores. Then you could have had a processor that was completely open and did not include any proprietary code. https://riscv.org/ https://riscv.org/
- loeg 10y agoWhat's the practical application of a chip like this?
- ilanco 10y agoIn the paper they are suggesting deep learning, self-driving cars, autonomous drones and cognitive radio.
- fudged71 10y agoWhat is cognitive radio?
- ilanco 10y agoMaybe that's what they call speech recognition?
- zump 10y agoDynamically switching carrier frequencies to make better use of the spectrum. It is somewhat related to software-defined radio, in that SDR's are typically used to prototype cognitive radio. It hasn't really got mindshare though in the sense players like Qualcomm have all but ignored it and would rather work on proprietary comms schemes.
- mutagen 10y agoDynamic spectrum management, changing channels based on current usage and other factors. https://en.wikipedia.org/wiki/Cognitive_radio https://en.wikipedia.org/wiki/Cognitive_radio
- adapteva 10y agoIn general it was built for math and signal processing (broad field). Within those fields, more specifically it was designed initially for real time signal processing (image analysis, communication, decryption). Turns out that makes it a pretty good fit for other things as well (like neural nets..). Here is the publication list showing some of the apps. (for later, server is flooded now): http://parallella.org/publications http://parallella.org/publications
- zelon88 10y agoDid I read the specs wrong or are they claiming a 12x - 15x performance improvement over the Ivy Bridge Xeon in GFLOPS/watt? In a <2w package? http://www.adapteva.com/wp-content/uploads/2013/06/hpec12_olofsson_publish.pdf http://www.adapteva.com/wp-content/uploads/2013/06/hpec12_ol...
- dnautics 10y agoThat's not unreasonable.
- dnautics 10y agoI should clarify:. Presumably the parallella's RISC does away with a lot of the superscalar features of the x86 which are embedded in the xeon phi's One way to think about it is that things like branch prediction and speculative and out of order execution are like real-time JITting of your code. Not having that silicon can make things way more efficient.
- adapteva 10y agoThat's an older paper, but yes there have been more than one independent study showing 25x boost in terms of energy efficiency. See Ericsson FFT paper, OpenWall bcrypt paper, and others at parallella.org/publications.
- white-flame 10y agoThe Epiphany gains are certainly only achievable for massively pipelinable or embarrassingly parallel operations with very little intermediate state (e.g. streaming data, neural software, etc), not for random access large memory footprint crunching like the Xeon. There simply isn't the per-core memory (64KB), or external memory bandwidth, to go around otherwise. Xeon, Power, etc are kind of power pigs anyway, though they've got a lot of absolute oomph to show for it.
- adapteva 10y agoI am here, if anyone has questions. AMA! Andreas
- mos6502 10y agoWhat are the chances of seeing a new Parallella SBC with an Epiphany-V coprocessor coupled with a RISC-V main processor?
- adapteva 10y agoNot going to happen in the near term. There is no way to meet the price point needed to compete in the low cost SBC market with the Epiphany-V. Believe it or not, the $99 Parallella was priced too high to reach mass adoption.
- cevans01 10y agoHow about a evaluation board which plugs into the mezzanine connectors of the ZC706 evaluation kit? Something similar to the AD9361 FMCOMMS3/5 [1] Also: any more information on the ISA extensions for communications/deep learning? [1] https://wiki.analog.com/resources/eval/user-guides/ad-fmcomms3-ebz https://wiki.analog.com/resources/eval/user-guides/ad-fmcomm...
- adapteva 10y agoSure, there will be evaluation boards, they just won't be generally available at digikey and won't cost $99. More information about custom ISA will be disclosed once we have silicon back.
- ebcode 10y agoHaving seen that the $99 price point was too high, is one of your goals still "supercomputing for everyone"? Or has that dream been dashed?
- adapteva 10y ago
- cordite 10y agoBut can I run Erlang on it?
- adapteva 10y agoHah! You thought you would get us with that one.:-) Here is the link to the Erlang OTP developed at Uppsala University for Epiphany. https://github.com/margnus1/otp https://github.com/margnus1/otp
- cordite 10y agoIs this actually running Erlang processes on the epiphany cores or just erlang spawning special processes on the epiphany cores? I've seen the latter and was not impressed.
- fenollp 10y agoHey Andreas, I'm unable to find the feature branch bringing Parallella support to OTP https://github.com/margnus1/otp/branches https://github.com/margnus1/otp/branches Maybe it was merged upstream already? You came a long way since I saw you in London in 2013. 1024 cores came sooner that 2020! Amazing job.
- tomcam 10y agosnap
- 10y ago
- convolvatron 10y agoI read through the pdf summary and it doesn't look as if the shared memory is coherent (which would be silly anyways). But I couldn't find any discussion about synchronization support. Given the weak ordering of non-local references it seems difficult to map alot of workloads. My real guess is that I haven't seen part of the picture.
- adapteva 10y agoIt comes back to the programming model. Synchronization is all explicit. See publication list. Includes work on MPI, BSP, OpenMP, OpenCL, and OpenSHMEM. The work from US army research labs on OpenSHMEM is especially promising. It's a PGAS model.
- jamesaross 10y agoOpenSHMEM paper: https://arxiv.org/abs/1608.03545 https://arxiv.org/abs/1608.03545
- convolvatron 10y agogot it, thanks. it looks like the per-node memory controller has an atomic test and set edit: and also a global wired-or for a barrier.
- robryk 10y agoIf you're looking for weird synchronization primitives, look at the documentation of the DMA controller. It has a mode in which it stores bytes that are written to a particular address in a memory range in order the writes arrive. I haven't figured out a reasonable way to use that with multiple writers (except the trivial case of having a byte-based stream with bounded size), though.
- teraflop 10y agoYeah, I was thinking about that problem too. (It's not safe to blindly write somewhere unless you can be sure that nobody else is going to simultaneously clobber your data. You can't do any kind of atomic test-and-set or compare-and-swap operation on remote memory, so you don't have the usual building blocks for things like queues or semaphores.) The problem becomes a lot easier if you can reduce the multiple-writer case to the single-writer case. One idea that occurred to me is that since you have 1024 cores, it might make sense to dedicate a small fraction of them (say, 1/64) to synchronization. When you need to send a message to another process, you write to a nearby "router" that has a dedicated buffer to receive your data. The router can then serialize the with respect to other messages and put it into the receiver's buffer. Basically, you'd end up defining an "overlay network" on top of the native hardware support; you pay a latency cost, but you gain a lot of flexibility. EDIT: I may be completely wrong about the first paragraph; it looks like the TESTSET instruction might actually be usable on remote addresses. I assumed it didn't because the architecture documentation doesn't say anything about how such a capability would be implemented. But if it works, it would drastically simplify inter-node communication.
- AnimalMuppet 10y ago1024 64-bit cores? Cool. Very impressive. 64 MB on-chip memory? For 1024 cores? That's 64 K per core. That seems rather inadequate... though for some applications, it will be plenty.
- mekaj 10y agoDepending on the application this may be a reasonable trade-off.
- adapteva 10y agoYou need think of it as aggregate memory, not as per core memory to use if effectively. Are you aware of a chip with more than 64MB of on chip RAM?
- orbifold 10y agoThe latest generations of IBM Power processors have >64MB L3 caches on chip. The Power 7+ has 80MB per chip, the 12 core Power 8 96MB, according to Wikipedia the Power 9 will have 120MB.
- adapteva 10y agoGood data! That puts e5 in good company with some big-iron heavies.
- rbjorklin 10y agoI realize comparing to Intel is unfair but I think the Skylake Iris Pro 580 has 128MB on chip RAM. https://en.wikipedia.org/wiki/Intel_HD_and_Iris_Graphics#Skylake https://en.wikipedia.org/wiki/Intel_HD_and_Iris_Graphics#Sky...
- Narishma 10y agoThat's eDRAM though, and it's really on-module rather than on-chip. It's a separate die on the same module as the main SoC.
- valarauca1 10y agoTwo things immediately jump out Custom ISA extensions for deep learning, communication, and cryptography DARPA/MTO autonomous drones cognitive radio The radar geeks are gonna love to get their hands on ~250GFLOP, 4watt processor.
- microcolonel 10y agoCould be excellent for a dense automatic isolating array microphone; thousand other things. I'd love to see Parallella in embedded, they set a great example.
- nickpsecurity 10y agoI'm seeing NIDS for 10+Gbit links, DDOS mitigation, cache appliance for web servers, Erlang accelerator, BitTorrent accelerator, and so on. Quite a few possibilities. Also, something like this might be tuned for hardware synthesis, formal verification, or testing given all the resources that requires. Intel has a nice presentation showing what kind of computing resources go into their CPU work: https://www-ssl.intel.com/content/dam/www/public/us/en/documents/white-papers/intel-it-hpc-silicon-design-paper.pdf https://www-ssl.intel.com/content/dam/www/public/us/en/docum...
- valarauca1 10y agoI don't believe the chip has enough I/O for a 10Gb nic.
- adapteva 10y agoThe issue is not bandwidth, it's the system cost. The chip has 1024 IO pins. (more than enough for MANY 10Gb nics...)
- nickpsecurity 10y ago"The issue is not bandwidth, it's the system cost." What does that mean?
- rpiguy 10y agoWow from Kickstart to DARPA funding! How did I miss that?
- agumonkey 10y agoThey went surprisingly silent after the KS boards. I falsely assumed they left the business or went employee. Delightful surprised they found ways to keep searching.
- liveoneggs 10y agotoo bad the entire site is returning a 500 error now
- aerodog 10y agoi can't see anything on the site. is this for sale or just a proposed architecture? amazon seems only to be selling your 16-core device. was there a 64-core one? can't access your product offering.
- reportingsjr 10y agoThe tapeout is apparently at the foundry and they are expecting chips back in 4-5 months. (I gathered this info from a google cache of their blog)
- algorithm314 10y agoThe ISA is epiphany or risc-v?
- jamesaross 10y agoIt's backward compatible with Epiphany-III...so it's still Epiphany ISA with new instructions.
- algorithm314 10y agoI have read it but in the past he wrote a blog post that risc-v will be used as isa in future products.So maybe 64 bit risc-v with backwards compatibility with epiphane?(it sounds a bit strange)
- jamesaross 10y agoI think a heterogeneous product was implied in that post, but I don't blame you for the confusion. The Epiphany-V is still homogeneous because of the time/funding constraints.
- milcron 10y agoThe Epiphany core is a co-processor, and the "main" processor is a couple of ARM cores to run Linux/other. Maybe in the future they will offer boards with Risc-V main processors, and Epiphany co-processors. I'm not sure how feasible 1024 Risc-V cores would be (although it sounds awesome). Epiphany cores were designed for this sort of thing.
- adapteva 10y agoAgree, but people have all kinds of pre-conceived notions about co-processors so let's clarify some things: e5 can't self-boot, doesn't have virtual memory management, and doesn't have hardware caching, but otherwise they are "real" cores. Each RISC core can run a lightweight runtime/scheduler/OS and be a host.
- jecel 10y ago
- mechagodzilla 10y agoThe linked paper mentions a 500 MHz operating frequency, as well as mentioning a completely automated RTL-to-GDS flow. 500 MHz seems extraordinarily slow for a 16nm chip - was this just an explicit decision to take whatever the tools would give you so as to minimize back-end PD work? Also, given the performance target (high flops/w), how much effort did you spend on power optimization?
- adapteva 10y agoPaper stated that 500MHz number was arbitrary (had to fill in something for people to compare to). Agree that 500MHz with 16nm FinFet is ridiculously slow. We are not disclosing actual performance numbers until silicon returns in 4-5 months. 28nm Epiphany-IV silicon ran at 800MHZ.
- Tistel 10y agoI wonder if the Erlang/BEAM VM could take advantage of it. Erlang would be a beast. if any of the pure functional languages get running on it (for easy parallel), watch out. Nice work!
- protomyth 10y agoThe website is erroring out for me, so I wonder what the motherboard situation will be like for this chip. It would be really nice to be able to buy and ARM like we can buy an x86.
- rjammala 10y agoSeems like this url is really popular,I get this connection error: Error establishing a database connection
- imaginenore 10y agoMirror: http://archive.is/Zj9hG http://archive.is/Zj9hG
- witty_username 10y agoPrepend cache: to the URL to view Google's` cached version of this website.
- mden 10y agoFor the lazy - http://webcache.googleusercontent.com/search?q=cache%3Ahttps%3A%2F%2Fwww.parallella.org%2F2016%2F10%2F05%2Fepiphany-v-a-1024-core-64-bit-risc-processor%2F%3F&oq=cache%3Ahttps%3A%2F%2Fwww.parallella.org%2F2016%2F10%2F05%2Fepiphany-v-a-1024-core-64-bit-risc-processor%2F%3F&aqs=chrome..69i57j69i58.5527j0j4&sourceid=chrome&ie=UTF-8 http://webcache.googleusercontent.com/search?q=cache%3Ahttps...
- pjc50 10y agoInteresting, but for a very specialized market, somewhere in the corner between GPU and FPGA. Closest existing offer might be Tilera? Site is currently slashdotted so I can't comment on details like how much DRAM bandwidth you might actually have.
- nickpsecurity 10y agoTilera is what I thought of, too. It's actually where I'm getting my ideas of applications for Epiphany-V. They did a lot of the early proving work on architectures like this. Example: first 100Gbps NIDS I saw used a few Tilera chips to do that.
- planteen 10y agoKind of off topic, but are there any low-end/hobbyist Tilera boards? The Linux kernel has support for it. I've always thought you could stress multi-threaded code in interesting ways by running it on tons of cores.
- bra-ket 10y agohow do I connect external RAM to it, and what would be the cpu-to-memory bandwidth in that case
- adapteva 10y agoExternal RAM, up to 1 Petabyte would be connected through an external FPGA, containing an epiphany link, some glue logic, and a memory controller.
- bra-ket 10y agothanks, Andreas. is there an existing solution, or is it DIY project at this point?
- sargun 10y agoFrom my understanding the Zynq's memory controller can only handle ~4GB of memory. Am I missing something? Is there a way to connect more than 4GB -- if so, I'd be very interested.
- adapteva 10y agoLarger FPGA support 64-bits with custom memory controllers.
- sargun 10y agoWith the new chip, is there a memory controller on the board, or will you still need the FPGA's? Even with the new MPSoC, I think the memory controller is limited to 8GB. Do you know what the most efficient cost / GB confit is for a Epiphany + memory controller or FPGA
- noelwelsh 10y agoTying in to earlier discussion on C (https://news.ycombinator.com/item?id=12642467 https://news.ycombinator.com/item?id=12642467), it's interesting to imagine what a better programming model for a chip like this would look like. I know about the usual CSP / message passing stuff, and a bit about HPC languages like SISAL and SAC. Anyone have links to more modern stuff?
- deleted 10y ago[deleted]
- weatherlight 10y agoWhat are the benefits/advantages of choosing something like this over a traditional Arm/x86 or a GPU? My knowledge in this area is limited. :)
- Stubb 10y agoBest I can tell, Epiphany is designed as a co-processor, so it's not booting the OS and relies on a host (like an ARM/x86) to run the show and issue commands. The Epiphany cores have significantly more functionality than GPU cores, so they're useful for things beyond computing FFTs and other number-crunching tasks. For example, you could map active objects one-to-one onto Epiphany cores.
- jdmoreira 10y agothat's going to provide some interesting race conditions for sure :D
- technological 10y agoAnyone looking for cached link for the website http://webcache.googleusercontent.com/search?q=cache:https://www.parallella.org/2016/10/05/epiphany-v-a-1024-core-64-bit-risc-processor/ http://webcache.googleusercontent.com/search?q=cache:https:/...? Related Report - https://www.parallella.org/wp-content/uploads/2016/10/e5_1024core_soc.pdf https://www.parallella.org/wp-content/uploads/2016/10/e5_102...
- jokoon 10y agoWhat I don't understand with computer chips, is how really relevant the FLOPS unit is, because in most situations, what limits computation speed is always the memory speed, not the FLOPS. So for example a big L2 or L3 cache will make a CPU faster, but I don't know if a parallel task is always faster on a massively parallel architecture, and if so, how can I understand why it is the case? It seems to me that massively parallel architectures are just distributing the memory throughput in a more intelligent way.
- adapteva 10y agoYou have to look at all the numbers (I/O, on-chip memory, flops, threads) and see if the architecture fits your problem. Some algorithms like matrix matrix multiplication are FLOPS bounds. It's rare to see a HPC architecture (don't know if there is one?) that can't reach close to the theoretical flops with matrix matrix multiplication. Parallel architectures and parallel algorithm development go hand in hand.
- GirlsCanCode 10y agoCan I run the Erlang BEAM on one of these cores?
- wbsun 10y agoThe website is down. Maybe a good opportunity to demonstrate the scalability improvement with such 1024-core processor?
- bikamonki 10y agoError establishing a database connection
- api 10y agoWish I was still working on genetic programming and digital artificial life. This would be barrels of fun.
- williesleg 10y agoLike SPARC?
- Animats 10y agoSo each processor has 64KB of local memory and network connections to its neighbors? The NCube and the Cell went down that road. It didn't go well. Not enough memory per CPU. As a general purpose architecture, this class of machines is very tough to program. For a special purpose application such as deep learning, though, this has real potential.
- the_duke 10y agoPage is overwhelmed. Can anyone provide a summary?
- kirrent 10y agoFor those interested, Andreas did an interview on the Amp hour a while ago. http://www.theamphour.com/254-an-interview-with-andreas-olofsson-adatevas-ampliative-abacus/ http://www.theamphour.com/254-an-interview-with-andreas-olof... Congrats to everyone at adapteva. I remember talking to a couple of researchers who were using the prototype 64 core epiphany processor who seemed excited at how it could scale. I wonder how excited they'd be about this.
- erichocean 10y agoAny chance of adding 16-bit floating point support in Epiphany-VI?
- mhurd 10y agoTruly inspirational in showing what largely one person can do even in these times of huge fabs, expensive masks, and difficult, modern design rules http://meanderful.blogspot.com.au/2016/10/adapteva-tapes-out-epiphany-v-1024-core.html http://meanderful.blogspot.com.au/2016/10/adapteva-tapes-out...
- new299 10y agoI think my thoughts on the parallella stuff still hold: http://41j.com/blog/2012/10/my-take-on-the-adapteva-parallella/ http://41j.com/blog/2012/10/my-take-on-the-adapteva-parallel... Basically this is a recurring theme in computing, but the whole custom massively parallel thing rarely works out.
- sargun 10y agoWould anyone be interested in an Epiphany dedicated servers a la Rasberry Pi collocation (https://www.pcextreme.com/colocation/raspberry-pi https://www.pcextreme.com/colocation/raspberry-pi)? I've always wanted to play with these units, but buying one doesn't make a lot of sense for me (where would I put it?). I would be super interested in making them accessible to folks.
- adapteva 10y agoPAPER: https://www.parallella.org/wp-content/uploads/2016/10/e5_1024core_soc.pdf https://www.parallella.org/wp-content/uploads/2016/10/e5_102... (access until we resolve the hosting issues, wordpress completely hosed...)
- tempodox 10y agoError establishing a database connection Overload thru request storm?
- laxk 10y ago403 error now for the entire site.
- imre 10y agoWhat is the possible applications? i.e how to potentially make use of all the cores? Is it more like GPU programming?
- tiggilyboo 10y agoGood to see this here! I actually wrote a paper analyzing this architecture for one of my bachelor classes, been a few years but: http://simonwillshire.com/papers/efficient-parallelism/ http://simonwillshire.com/papers/efficient-parallelism/
- tiggilyboo 10y agoGood to see this here! I actually wrote a paper analyzing this architecture for one of my bachelor classes, been a few years but: http://simonwillshire.com/papers/efficient-parallelism/ http://simonwillshire.com/papers/efficient-parallelism/
- thesz 10y agoCray had always resisted the massively parallel solution to high-speed computing, offering a variety of reasons that it would never work as well as one very fast processor. He famously quipped "If you were plowing a field, which would you rather use: Two strong oxen or 1024 chickens?" I cannot see how this thing can be programmed efficiently (to at least 70% of computing capacity, as most vector machines can be programmed for).