12 ms·
Lanai, the mystery CPU architecture in LLVM
- throwawag 5y agoProbably shouldn't reveal this but, Google are using it internally for their accelerated networking in their cloud, pretty much all Lanai hardware is deployed in GCP infrastructure.
- Aperocky 5y agosecurity by obscurity?
- deleted 5y ago[deleted]
- jeffbee 5y agoWhat explains why Amazon has been relatively open about Nitro but Google has never said anything about how their NICs and SSDs work? It can't be because it's secret sauce; Amazon beats them on latency etc.
- jsolson 5y agoWe've been fairly open about how Google Cloud Networking works -- https://www.usenix.org/conference/nsdi18/presentation/dalton https://www.usenix.org/conference/nsdi18/presentation/dalton There's also some more recent coverage on IPUs that I don't have a handy link to.
- znpy 5y agoin fairnes, skimming the slide the name "lanai" doesn't seem to occour
- jsolson 5y agoIndeed.
- LosWochosWeek 5y agoI guess that's the "fairly" part of your comment?
- jsolson 5y agoAh, no, the fairly was merely that we omitted details and some things have changed. That said, the paper completely captures the use of Lanai in GCP's networking data plane.
- exikyut 5y agoCool, it went somewhere very wide and deep. I guess the only genuinely interesting question worth asking (and hoping for an answer to) is, given the back and forth about performance and Myrinet vs IB on here... how well does the design stack up in practice to an Ebit-scale environment?
- jsolson 5y agoYou should not have posted this, least of all because it's extraordinarily misleading. We use Lanai elsewhere, and it's in hardware powering Google's server networking networking (inclusive of GCP), but Lanai sadly does not play a substantial role in Andromeda networking. Feel free to ping me on corp (username in my profile here) to discuss.
- CalChris 5y ago> The world's best instruction mnemonic: PUNT, to switch between user and system contexts. That's pretty good but I still prefer PowerPC's EIEIO. https://www.ibm.com/docs/en/aix/7.2?topic=set-eieio-enforce-in-order-execution-io-instruction https://www.ibm.com/docs/en/aix/7.2?topic=set-eieio-enforce-...
- classichasclass 5y agoThere are many great Power mnemonics. VSX introduced one nearly as good, xxlxor (which, quite reasonably, is a logical XOR between VSX registers/FPRs). It's delightful to try to even pronounce. The best part is that eieio's derivation is totally plausibly serious from its stated function. (Remember, IBM doesn't have a corporate sense of humour.) It's also an easy, fairly lightweight speculation barrier apart from its official usage.
- thebester5 5y agoMy favorite PPC instruction is darn. It delivers a random number, either raw or NIST conditioned. We decided to use the same instruction name in an internal RISCV implementation, and fun times have been had complaining about the darn instruction not working.
- exikyut 5y agoAfter mumbling "x-x-l-x-o-r" my brain decided to pronounce it "excelsior". Huh.
- davidw 5y agoIf Google acquired it, the cynical take is that they ended up sitting on it and doing nothing or killing it.
- wmf 5y agoThe fact that they developed and upstreamed an LLVM backend indicates that they probably did something with it.
- rsstack 5y agoThe Google way: Force the rest of the world adjust to your new "cool thing", and then kill it because it wasn't that great.
- monocasa 5y agoGoogle isn't really forcing anyone to do anything in this case. Another RISC backend that was made clear would be ripped out of upstream the instant the Google maintainers stopped responding isn't really an imposition.
- uluyol 5y agoWould something like this be accepted in open source projects that are not significantly driven by Google? E.g. would Linux accept code for drivers / architectures that are not available to the public? I'm genuinely curious.
- JonChesterfield 5y agoxcore (mentioned in that thread) is pretty obscure and still in trunk last I looked. Extra backends don't carry that much of a maintenance cost, mostly patching them up on api changes. Weirder targets hit bugs that the common ones don't so there's a benefit from having them in tree too. I'm not sure that generalises beyond modular compilers though.
- 5y ago
- drewg123 5y agoI worked for Myricom 2001->2013, and Google 2013->2015. And, wow, I wish I could comment on this thread.. :) EDIT: Scott's Medium blog post linked from this article is fascinating: https://medium.com/swlh/myricom-an-hpc-story-and-lessons-learned-from-the-fall-of-an-industry-darling-35017e6373d8 https://medium.com/swlh/myricom-an-hpc-story-and-lessons-lea...
- CalChris 5y agoI just want to know why it needs to be in-tree.
- wyldfire 5y agoGoogle uses upstream llvm as their dev repo. I don't know if they do any development downstream.
- IshKebab 5y agoLLVM doesn't have a plugin system so it's that, or maintain a fork and deal with horrible merges all the time.
- Fnoord 5y agoSome important/big customer pays a maintainer to maintain it upstream instead of having to maintain/port it in-house. If there's two of these, it makes sense. Even with one it would otherwise require backporting. Might as well do it in public?
- q3k 5y agoYeah, that blog post was a great reference to get a general overview of What Happened At Myricom. ... and I just hope I didn't write anything factually wrong in TFA. :)
- deleted 5y ago[deleted]
- ncmncm 5y ago
- kleton 5y ago
- wheybags 5y agoI gotta say, if I was the BDFL of llvm, I would kick this out of tree without a second thought. Why on earth should a foss project support a private architecture? How many man-hours have been wasted waiting for the Lanai backend to compile? Not to mention applying project-wide refactoring to the Lanai code.
- judofyr 5y agoThere's more context in the thread where it was upstreamed: https://lists.llvm.org/pipermail/llvm-dev/2016-February/095118.html https://lists.llvm.org/pipermail/llvm-dev/2016-February/0951... Some notable comments: > I was going to mention it, but you guys know very well the drill, so unless this hardware changes fundamental parts of the middle end in ways that are unnatural to most other targets (doesn't seem that way), then I see no reason why not have it upstream. > I see no problem with having the backend upstream with the understanding that all the normal policies apply. Getting more people working on ToT is valuable to the community as a whole and provided it's "just another backend" with plenty of tests, the cost is low.
- klausa 5y agoA significant part of work on LLVM comes from Apple, which includes support for quirks of _their_ proprietary chips and platform; and the world is much better for those changes being upstreamed rather than living in a fork somewhere.
- mjw1007 5y agoThere's a big difference between "we are the only people who sell hardware using these chips" and "we are the only people who have access to hardware using these chips".
- deleted 5y ago[deleted]
- tedunangst 5y ago> How many man-hours have been wasted waiting for the Lanai backend to compile? Probably zero? Is it compiled by default?
- IncRnd 5y ago> Some of my recent long-term projects revolve around a little known CPU architecture called 'Lanai'. Unsurprisingly, very few people have heard of it, and even their Googling skills don't come in handy. I don't get this. Searching for [lanai cpu] shows tons of links on the LANai cpu architecture from Myricom, purchased by Google.
- illegalmemory 5y agoThis seems related https://github.com/chriseth/notes/blob/186d7ea0742336ed38e399cd7d7edc0eb5429f00/talks/truebit_devcon2/index.html#L251 https://github.com/chriseth/notes/blob/186d7ea0742336ed38e39... "TrueBit - Off-Chain Computations for Smart Contracts"
- q3k 5y agoThis is https://github.com/TrueBitProject/lanai https://github.com/TrueBitProject/lanai , which is in itself an interesting endeavor which I forgot to write about. I've added a mention about it to TFA.
- maximilianburke 5y agoHere's the thread on the LLVM development lists from when Lanai was proposed to be upstreamed with some good commentary about why it should be accepted even if Google isn't willing to go into details: https://discourse.llvm.org/t/rfc-lanai-backend/39874 https://discourse.llvm.org/t/rfc-lanai-backend/39874
- nwatson 5y agoFor the link, Norton reports: "This is a known dangerous webpage. It is highly recommended that you do NOT visit this page." The "page full report" doesn't reveal any particulars. I rarely get this warning elsewhere.
- q3k 5y agoThis is an old domain, and once upon a time some private malware sample links hosted on it got indexed. Years later and some software still think I'm evil :).
- trasz 5y agoRandom, tangentially related thing I just remembered: FreeBSD used to provide its own firmware for (iirc MIPS-like) core used in very early Broadcom NICs. There also used to be a custom firmware for some Adaptec controllers, along with an assembler tool to compile it during kernel build.
- drewg123 5y agoI think you're talking about the Alteon Tigon-II NICs (Alteon was acquired by Broadcom). Ken Merry and I modified Alteon firmware so that it supported zero-copy sockets by doing page-flipping on receive. See https://people.freebsd.org/~ken/zero_copy/ https://people.freebsd.org/~ken/zero_copy/
- p_l 5y agoAll LSI "SCRIPTS" HBAs had firmware as part of kernel/driver sources among the many platforms they were used. I believe linux used a simple assembler whose sources were included in kernel to generate the binaries (if not preprocessor tricks, even). *BSDs definitely had their own solution too.
- biorach 5y ago
- biorach 5y ago> The CEO capped the full time headcount at 49 people so he wouldn't be subject to a California law requiring him to provide his employees with a health care plan. ... words fail me
- cplusplusfellow 5y agoI believe this was disputed by a former employee elsewhere ITT.
- Melatonic 5y agoGiven the timing by Google I suspect this COULD be involved in what Paul Debevec was doing there with giant camera sphere arrays
- Melatonic 5y agoInteresting that this is being downvoted but not commented on.....
- hogehoge51 5y agoLanai …. Sounded familiar, of course myri. I used these NICs to implement wire speed 10g packet capture. They were a fraction of the cost of dedicated packet capture boards and had a nice api. These days I guess a Risc-v would be a perfect match for this sort of application, but back then it seemed every accelerator startup would implement a custom ISA.
- kyrra 5y agoGoogler, opinions are my own. Btw, the Lanai target in LLVM can be found here: https://github.com/llvm/llvm-project/tree/main/llvm/lib/Target/Lanai https://github.com/llvm/llvm-project/tree/main/llvm/lib/Targ.... Latest commit is only 24 days ago, so it looks to still be active. Though I'm not sure how much of that is generic target updates, vs target specific changes.
- zozbot234 5y agoHow would Lanai compare to a simple RV32I isa/core for this application area? The short description in OP doesn't really clarify what, if anything, might be specifically compelling about this arch.
- q3k 5y agoLanai is significantly older than RISC-V. These days you'd likely easily pick designing or using an existing RISC-V core for an application like this, possibly adapting it to your particular usecase. The decision is different, however, if you happen to have the entirety of the Lanai engineering team on board and own the entire Lanai IP portfolio. :)
- derefr 5y agoReminds me of the various uses that have come of the IP of the https://en.wikipedia.org/wiki/ARC_(processor) https://en.wikipedia.org/wiki/ARC_(processor) — including inside the Intel Management Engine! — since its ignominious start as the SNES SuperFX chip :)