8 ms·
FreeBSD gets pNFS support
- brian_herman 8y agoWhat is pNFS?
- jdhawk 8y agoParallel NFS, which was introduced in the NFS v4.1 protocol.
- brian_herman 8y agoThanks!
- kev009 8y agoIt will be fairly amusing to see this in a Billions dollar product line like NetApp or Dell EMC Isilon. I can't find any photos of Rick Macklem's setup but it was developed in a fairly.. steampunk fashion.. on several obsolete i386 laptops with <4GB RAM.
- parvenu74 8y agoWhy the hate? Just because FreeBSD supports the idea of making a commercial product from the code isn't a reason bag on Rick.
- ComputerGuru 8y agoI didn’t detect any hint of hate in the OP’s comment, more appreciation and irony.
- pinewurst 8y agoI believe NetApp has had shipping pNFS support in ONTAP for quite some time now. I don’t know how many active users though.
- loeg 8y agoIsilon hasn't used the FreeBSD nfsd since 2014 (OneFS 7.2). (And prior to that, it was a heavily modified version.)
- jim00338811 8y agoPanasas, who open sources pNFS, is a serious scale company competitive with Isilon and NetApp https://www.pcmag.com/article2/0,2817,2135264,00.asp https://www.pcmag.com/article2/0,2817,2135264,00.asp https://www.panasas.com/ https://www.panasas.com/
- GalacticDomin8r 8y agoSince Panasas contributes a fair amount to FreeBSD, perhaps they are happy as well. eg https://people.freebsd.org/~rpokala/2017-03-24.nvdimm.txt https://people.freebsd.org/~rpokala/2017-03-24.nvdimm.txt
- loeg 8y agoRavi never actually got around to committing that NVDIMM work :-(.
- mprovost 8y agoInteresting that this was tested at Framestore (the visual effects company), they used to be a Lustre shop years ago then moved to commercial NFS servers and must be back onto commodity hardware now.
- nomadlogic 8y agoI could see a use for both - using Lustre as a shared SAN for their flame/smoke/etc suites and using pNFS for workstation/render-node shared storage. at least that's how i've deployed things in the past. i could see this replacing isilon (which is built on freebsd) for the later use-case :)
- mozumder 8y agoIs NFS viable at all with untrusted clients on a LAN? Seems it requires all machines on a LAN to be trusted, with proper user IDs. It's fast though.
- jiveturkey 8y agoPerhaps you'd like to read up on NFS basics. The short answer is, you can use it with either loose or tight authentication. The "LAN" bit of your question is largely irrelevant.
- icedchai 8y agoYou can set up Kerberos authentication, but I've never actually seen this done in the wild. Most setups trust based on subnet.
- gnufx 8y agoIt's a lot more common than pNFS, which I've never seen. You really don't want anything else unless you have tightly-coupled, well-managed clients.
- xvf22 8y agoI wish there was a middle ground where you could issue a ticket to the server and not have to issue one per user but still have the traffic encrypted between NFS client and NFS mount.
- partycoder 8y agoSubversion? oh man, what a blast from the past.
- loeg 8y agoVery alpha code, YMMV, buyer beware. I don't want anyone to try this and be turned off FreeBSD if it doesn't work well. The author only has 100 Mbit networking, so performance bottlenecks may not be well-explored.
- kondro 8y agoI’m still surprised that NFS doesn’t have a TLS option. Or a way to grant access without relying on IP whitelists.
- asveikau 8y ago> Or a way to grant access without relying on IP whitelists. You can use Kerberos.
- 4ad 8y agoBut then you must use Kerberos authentication (password only) and can't use SSH public key authentication.
- KaiserPro 8y agoThats simply not true. GSSAPI allows you to forward kerberos tokens over ssh, with or without passwords or keys.
- 4ad 8y agoYou misunderstood. The problem is not forwarding Kerberos tokens, the problem is obtaining Kerberos tokens. Unless some extensions are employed, you need to use password authentication. Even if kerberos extensions are employed, you must still use kerberos auth instead of ssh public keys. If you try to use ssh public key authentication (instead of telling ssh to use kerberos authentication) you can't log-in to the machine because you can't access your home filesystem, which is kerberised. In fact, depending on your configuration, you might not be able to log-in at all since the user's filesystem might not be mounted at all (waiting for the automounter). If you want to store user directories on kerberized NFS, users must use kerberos authentication.
- gnufx 8y agoKerberos actually operates on keys. The nfs service key is typically a random one stashed in the keytab. I don't understand the objection to Kerberos tickets -- it's a single-sign-on system. You can obtain a ticket with a public key if you want, see e.g. http://www.h5l.org/manual/HEAD/info/heimdal/Setting-up-PK_002dINIT.html http://www.h5l.org/manual/HEAD/info/heimdal/Setting-up-PK_00... I don't know how easy it aould be to use SSH keys, but I once designed an authentication system for a scientific facility that was based on PGP keys and a web of trust reflecting administrative/social structures in the days before things like Eduroam and (viable?) NFS4, but not before AFS, which has been successfully used in the same way for a long time. I don't know why anyone would think "Kerberized" wouldn't involve Kerberos, but it's nothing specific to user directories. OrangeFS (nee PVFS) is a userland parallel filesystem (with a kernel driver in Linux) that can use two types of public key security if you want that sort of thing, but I gather that's not widely used.
- deleted 8y ago[deleted]
- davidmr 8y agoThis seems pretty cool in principle, but I got as far as a single nonreplicated MDS before I lost interest in trying it out at this stage. Admittedly, my background is in HPC where metadata performance and reliability requirements would preclude using something like this. Does anyone who is a more familiar with FreeBSD than I am (which is pretty much anyone who’s used it in the last 10 years) have any more information on long their term plans for at least replicated block storage failover or cache coherency in an active/active MDS setup?
- loeg 8y agopNFS in FreeBSD is one developer's toy project. I don't know of any widespread interest or plans for enterprise features.
- philg_jr 8y agoRed Hat worked pretty closely with NetApp and their implementation of pNFS. I've used it and it is very nice for mitigating any downtime while you perform maintenance on a controller.
- gnufx 8y agoHow does a parallel filesystem make maintenance easier? Unless it's specifically redundant, I'd expect the opposite.