13 ms·
Compiling my own SPARC CPU inside a cheap FPGA
- LeonM 7y ago> Alas, I am told by my friends that DDR controllers are no joke; they are not the playground of bored SW engineers. No, they definitely aren't funny. I've worked with FPGAs and DDR controllers at college and their can be a big PITA. Even with DDR controller libraries you can still run into all sorts of timing issues.
- ur-whale 7y agoWhen a SWE really starts to understand how complex a modern DDR controller is at the silicon level, two things happen: 1. You start to wonder how very, very far your code runs from the theoretical capabilities of the hardware and you start experiencing existential doubts. 2. You are overcome by a deep and immense feeling of gratitude towards whomever managed to force all that complexity to remain hidden underneath the simple memory abstraction your rely on to write day to day code.
- tpmx 7y agoAny links to reading materials for learning about that? Found this PDF from Samsung regarding the DDR4 interface itself: https://www.samsung.com/semiconductor/global.semi/file/resource/2017/11/DDR4_Device_Operations_Rev11_Oct_14-0.pdf https://www.samsung.com/semiconductor/global.semi/file/resou... (The JEDEC DDR4 spec is $284 to download...)
- lnsru 7y agoStart reading data sheets of memory chips. They have state diagrams. DDR2 is great technology to start with. Some random memory chip: https://www.micron.com/-/media/client/global/documents/products/data-sheet/dram/ddr2/512mbddr2.pdf https://www.micron.com/-/media/client/global/documents/produ... on page 9 you have state diagram. There is simulation model available.
- exikyut 7y agoWhere is this simulation model, and (here's hoping) is it cross-platform? Also, is it anything like http://www.visual6502.org/JSSim/ http://www.visual6502.org/JSSim/?
- lnsru 7y agoOn Micron’s website for most of their parts. Worked very well with Alteras’s memory controller some time ago. I needed to evaluate read write throughput for specific read/write patterns before designing printed circuit boards. I don’t unterstand what you mean with cross-platform.
- jacquesm 7y agoI try really hard to not go down that particular rabbit hole as it will always end up being 100% sure that software can't possibly work reliably or even at all. The degree to which we assume that our hardware is able to move stuff from point 'a' to point 'b' at a couple of billion times per second is scary.
- nickpsecurity 7y agoAlthough I don't know hardware, I read lots of stuff about developing it to get a better idea. Among the more interesting reads were slides about each process shrink along with challenges they brought. Especially from beginning of Deep Sub-Micron toward 28nm. The impression I got, esp by 28nm, is the hardware is inherently broken in quite a few ways. They have to correct the masks with algorithms, they do image recognition on circuits to spot patterns that act up, extra latches, variance across the chip/wafer, aging effects... list goes on. Miracle they even work at all. These things are also why I only trust old nodes for security. Sort of.
- l33tman 7y agoIt's very easy to make a DDR interface in an FPGA, it's difficult to make it run at the mark speed... (but this observation is generic for all FPGA-stuff really, it seems easy first, it's just another language etc.. but you need to learn timing analysis and constraints and optimization to use it for real)
- rjsw 7y agoThe Spartan-6 has a DDR memory controller built in.
- tverbeure 7y agoEdit: I was wrong and confused the Spartan 6 of the Pano G2 with the Spartan 3 of the G1. Spartan-6 does not. But Xilinx has a MIG (memory interface generator) that automatically creates the RTL for a memory controller that synthesizes to regular core logic. In addition, it also has IO cells with posedge and negedge FFs and calibrated delay lines. Getting the DDR DRAM to work on the Pano G2 shouldn’t be too hard. (Less hard than the G1, which has DRAM that isn’t supported by the MIG.)
- rwmj 7y agoThere's actually an open source DDR4 controller[1]. To my untrained eyes it looks quite complex. It's written in Python using Migen[2] to generate the HDL. [1] https://github.com/enjoy-digital/litedram https://github.com/enjoy-digital/litedram [2] https://github.com/m-labs/migen https://github.com/m-labs/migen
- deleted 7y ago[deleted]
- tpmx 7y agoHmm, yeah, that does look affordable. Lots of ads on ebay. "Lot of 25 Pano Logic Thin / Zero Desktop Client Black w/ Power Supply Buy now: US $170.00" I wonder what the thinking was that lead Pano Logic to put expensive FPGAs inside these units, instead of some more typical cheap ARM SoC? Edit: Ah, they operated in 2006-2012. I guess that was just before the rise of the very cheap/fast SoCs.
- lnsru 7y agoExactly. These are far far away from cheap FPGAs, Spartan 6 chips are just old, but not cheap. When thinking about cheap FPGAs I imagine these cheap things from Lattice for interface bridging. Cheap ARM SoCs were on ARM’s roadmap back then. Cortex A9 was huge innovation these days.
- tpmx 7y agoI guess they were thinking they'd be using very expensive FPGAs in the the first VC-funded iterations, and then move on to something cheaper. But they never got there... https://www.bizjournals.com/sanjose/blog/2012/11/pano-logic-shuts-down-leaving.html https://www.bizjournals.com/sanjose/blog/2012/11/pano-logic-... "... the company had doubled or tripled its revenue every year since 2008, when it had about $1 million in sales. ... The company was backed by about $38 million in funding from investors including Goldman Sachs, ComVentures, Foundation Capital, Fuse Capital, and Mayfield Fund."
- panpanna 7y agoI think you are correct. Fpgas are often used as a stepping stone towards cheaper asic or hybrid solutions. asic requires a high volume and a working design, neither of which you have early in the project.
- blattimwind 7y agoReminds me of the nVidia GSync module. Those used FPGAs which costed about 250 $/1ku iirc and 768 MB (1st gen) / 3 GB (2nd gen) of memory. Even at the hefty pricing of GSync screens it probably only covered BOM cost. (The large amount of memory is not because they need so much memory, but because they needed the bandwidth of a 3x 32 bit bus)
- rjsw 7y agoAnother design to target at one of these boxes could be Milkymist [1], uses the lm32 CPU and various peripheral cores. [1] https://github.com/m-labs/milkymist https://github.com/m-labs/milkymist
- Annatar 7y agoThis is awesome!!! (Ultra)SPARC CPU's are a joy to code for in assembler, and modern T3, T4 and T5's are number crunching monsters. How about synthesizing the GPL-licensed OpenSPARC T2 now? https://www.oracle.com/technetwork/systems/opensparc/opensparc-t2-page-1446157.html#t2-to-use https://www.oracle.com/technetwork/systems/opensparc/openspa... I'd love to have SmartOS backported on a FPGA-based, OpenSPARC T2, 19" 1U rack mountable server someday. Free hardware and software all the way.
- 4ad 7y agoThe Leon CPU used in the article (and by ESA) is a derivative of OpenSPARC T2. Edit: no it isn't, see below.
- rjsw 7y agoThe LEON3 CPU used in the article is a 32-bit SPARCv8, it isn't derived from the T2. You can see the 32-bit addresses in the Hello World example.
- 4ad 7y agoSee my parallel reply.
- Annatar 7y agoLeon is a 32-bit, SPARC V8 ISA implementation. It's also one of the rare radiation-hardened designs (for use in space equipment where cosmic radiation has a significant influence on correctness of operation). An OpenSPARC T2 is a SPARC V9b+VIS ISA. All UltraSPARC processors with a SPARC V9 ISA are 64-bit.
- 4ad 7y agoHaving written the SPARC64 compiler and runtime for Go (and the Solaris port), I assure you I know the difference between SPARC V8 and SPARC V9. It appears that I have been mislead, back in 2005 when OpenSPARC was released somebody from Sun told me that LEON is based on OpenSPARC, and I haven't bothered to check since. It seems to be a completely different design (it even uses VHDL instead of Verilog). So I was very mistaken about LEON. However, there have been other (embedded) SPARC V8 CPUs derived from OpenSPARC. I have worked on one myself (on the software side). This one was a very strange one because it only had two register windows (and the ABI didn't use register windows) and they disabled SMT. This type of modification is extremely easy to do on SPARC because of the way 64-bit works. There are no separate execution modes, there's simply another set of flags that's affected by instructions. The conditional instructions chose which set of flags to use. I really like SPARC, it's my favorite architecture.
- tverbeure 7y agoFor those interested in hacking in this: The Pano G2 FPGA is a monster, but prices on eBay have gone up a lot. My cheapest buy was 25 of them for $85 (including shipping!). They now go for around 1 for $30 if you’re lucky... or $200+ for many. The Pano G1 (with VGA instead of DVI) is cheaper but has a much smaller FPGA, though still large by hobby standards. The benefit of the G1 is that all interfaces are working now, including DRAM, USB, Ethernet. Last week, Skip Hansen got a full CP/M system running on one: https://github.com/skiphansen/pano_z80 https://github.com/skiphansen/pano_z80 USB on the G2 is hard. A bunch of people have tried and failed.
- tpmx 7y agoSo, Rev C is G2 [Edit: nope]. What's the cutoff between G1/G2? Is "Rev B" G1 or G2? (Granted, I mostly see Rev C.)
- tverbeure 7y agoThe revs that are posted on eBay are useless, even though that’s what’s advertised. Rev C is usually G1, rev B usually G2. But not always. ;-) The only reliable indicator to check which one to buy is looking at the pictures: VGA is G1, DVI is G2. (Some G2 are photographed with a DVI to VGA dongle plugged in, adding to the confusion.) There are 2 G2 versions: one with an LX100 and one with LX150. You can’t know which one you’re buying... LX100 is still a gigantic FPGA for hobby stuff.
- johndoe0815 7y agoFor a complete SPARCstation 5 implementation able to run SunOS, Solaris, BSD, Linux or NeXTstep, see http://temlib.org/ http://temlib.org/ - this fits on a Spartan6 XC6SLX45T FPGA, so you should be able to get it to work on the larger Spartan 6 FPGAs (if you can manage to build a working memory controller...). It seems that the Panos are only available on eBay US, any idea where to get them in Europe without all the shipping and tax hassle?
- jacquesm 7y agoThat's one way to get a computer where you're actually sure what runs on it.
- nickpsecurity 7y agoYou have to trust the FPGA, its toolchain, and any peripheral hardware and firmware with priveleged access.
- jacquesm 7y agoThat's true in a technical way but it is false in a practical one. An effective attack using compromised FPGAs or toolchains would be super hard to carry out undetected because of the decree of scrutiny the output would receive, besides, the attack itself would have to make substantial assumptions about the way the FPGA would be wired into the resulting circuit. I'm not saying it can't be done but it would extremely hard to carry out and I'm not aware of any such attacks ever being discovered in the wild. > any peripheral hardware and firmware with priveleged access. That would be a more feasible vector. But it all still will be much more secure than your average computer with a BMC.
- nickpsecurity 7y ago"An effective attack using compromised FPGAs or toolchains would be super hard to carry out undetected because of the decree of scrutiny the output would receive" FOSS vulnerabilities countered the many eyeballs argument a long time ago. There's even fewer people who know how to review hardware for flaws. I'm assuming it would be a targeted attack by default. That raises the bar. However, they could also leave a trigger that looks like a hardware flaw in the I/O interface. Intel has basically been doing that subversion with their ME flaws for some time now. Then, the only targeted part is just how to aim what's already there. "That would be a more feasible vector. But it all still will be much more secure than your average computer with a BMC. " True. Especially if you use an architecture like crash-safe.org or Cambrige's CHERI. I already advocate secure CPU's on FPGA's with dumb-as-allowed hardware if one can't get actual silicon. Also lets you throw in extra reliability features, too.
- roryrjb 7y agoIn one of the bash scripts in this article there's a lot of " || exit 1", you can enable this automatically by using "set -e" at the beginning of the script, this works for POSIX shell too. You can get extra safety in bash specifically by doing something like "set -euo pipefail" that will exit on errors, including pipeline failures and also on undefined variables.
- Annatar 7y agoThis will make the program instantly unportable, because not all Bourne shells are POSIX-compliant. Please don't do that. || exit 1 is portable.
- roryrjb 7y agoSo I guess I'm just used to POSIX compatible shells, it would be good to know about Bourne shells that aren't POSIX compatible (genuine question). But the shebang at the top of the script is bash so what I said is still correct in the context of the article. Stuff like multiple "|| exit 1"s makes people think that shell scripting is ugly and antiquated. I know that "#!/bin/sh" and "set -e" will work on all the BSDs, Solaris, Linux, macOS, what Unix is realistically left?
- ttsiodras 7y agoHi Rory. I am well aware of the flags you described - and indeed, they'd clean things up here. I am in fact tempted to go back and update the post - and either replace the shebang, or add the "early abort" flags. But TBH, keeping that script (that I wrote in 60 seconds) nice and clean was the least of my worries... Making the damn toolchain work had far, far higher priority :-) And after that, booting the LEON cores of course.
- roryrjb 7y agoYeah of course. I didn't mean to take anything away from the awesome work you are doing, this was nitpicking really, the script as it is, will work as intended, was really just highlighting this for the sake of others who may not be aware. Keep up the good work!
- ur-whale 7y agoThis article is the perfect summary of everything that's wrong with the existing HW development toolchains for FPGAs. The best bit is the windows-only version of the Xillinx gooware that in fact installs a Linux virtual box on windows to finally get to run the tools it needs. Oh, and yeah : there's a lame protection in there that checks it's running on a specific virtual box with a specific MAC address. Amazing (not in a good way).
- peter_d_sherman 7y agoThis article brings up an interesting question... What other cheap hardware products contain FPGA's that are potentially user accessible? I started a new message chain for this: Ask HN: What other cheap hardware products contain FPGA's? https://news.ycombinator.com/edit?id=21305355 https://news.ycombinator.com/edit?id=21305355
- alain94040 7y agoGreat article, I hope it helps motivate more people to give hardware design a try. the truth is that most of the HW designers I know are editing inside their Vendor-provided IDEs. Maybe true for FPGA designers, but not for ASIC designers in my experience. Another crazy difference I experienced was that builds are NOT deterministic Yes, hardware generation (synthesis, but mostly optimizations, placement and routing) are not deterministic. SW people are starting to experience that phenomenon with ML as well: you don't fully control what you get, but it works.
- qubex 7y agoOn a side-note of pedantry, we really need to stop using the term ‘compiling’ in the context of FPGAs and HDLs. To ‘compile’ is to assemble a dossier of documents and/or fill in forms - this is why Grace Hopper called her automatic code generation contraption a ‘compiler’: because, quite appropriately, it took the description of actions to be undertaken by the machine and fleshed them out in a ritualistic fashion in lower-level instructions. HDLs and FPGAs have very different principles and objectives. The best term is ‘instantiate’, because one creates an instance of a given hardware description upon the substrate of gates provided by the array. I’m sure I’ll be told I’m nit-picking, but those who do so would probably recoil in horror at the faux pas of some n00b saying a browser “compiles HTML” and tell them the correct term is ‘render’, and they’d be right. Please, let’s be careful and deliberate about the terms we use, can we please?
- ynx 7y agoI think you're not quite incorrect but your nitpicking isn't so much clarifying as redefining terms. Lowering HDLs into logic that can be flushed onto LUTs does involve an intermediate compilation step, even though it does also involve placing/relocating/routing elements on the FPGA, ultimately producing a file to be flushed to the chip. One could argue that 'compiling' is often conflated with 'linking' in producing application binaries, and in fact 'linking' also involves positioning/relocating elements in some fashion. 'compiling' is suitable and conceptually compatible enough that it clarifies rather than confuses.
- qubex 7y agoIt’s precisely this notion of ‘lowering’, as you call it, that preoccupies me. If unchecked, the idea that all forms of ‘lowering’ through abstraction layers is a form of ‘compilation’ (of sorts) will become the norm, and we’ll no longer have a unique term for what compilation proper actually means. And you’re entirely right in noting that nowadays ‘compiling’ has come to encompass and conflate the conceptually very distinct steps of compilation, linking, and assembly... a process I’m more comfortable with collectively referring to as ‘building’. It’s entirely possible that I might’ve (accidentally) redefined one or more meanings, and if I have, I apologise. It’s almost four in the morning and insomnia prevents me from sleeping but doesn’t necessarily maintain me at full alertness (tomorrow/today will be hell).
- jhallenworld 7y agoWell, for comparison, the cost of a current 100K LUT FPGA on a board is: $250 for Xilinx Artix-7: https://store.digilentinc.com/arty-a7-artix-7-fpga-development-board-for-makers-and-hobbyists/ https://store.digilentinc.com/arty-a7-artix-7-fpga-developme... $100 for Lattice ECP5 (85K LUT): https://www.latticestore.com/products/tabid/417/categoryid/59/productid/122774/default.aspx https://www.latticestore.com/products/tabid/417/categoryid/5...
- happycube 7y agoI haven't played with mine yet, but there's an open toolchain for the ECP5 :)
- panpanna 7y agoNote that you can't easily compare LUT count between vendors or even among a vendors different architectures. In any case, I think ecp5 is partially supported by open source tools which should make it much more interesting to hobbyists.
- freemint 7y agoAlso Lattice only does LUT4. (Their internal look up tables have 4 inputs) The Artix-7 has LUT6. From what i heard that means you need more than 1.6 times more LUT4 than LUT6. Not including the fact that Artix-7 includes other hardware features which are sometimes wired in to a desgin
- panpanna 7y agoThis use of expensive surplus hw reminds me of the nsa@home project http://nsa.unaligned.org/ http://nsa.unaligned.org/