NFS4.1 Multipath with Ganesha and CephFS

Hi Folks, so there is a small 3 node ESXi cluster I am trying to set up. Datastore is a CephFS exported by NFS Ganesha. Everything works, I can connect to datastore with some decent speeds etc. However, I am unable to wrap my head around failover setup with NFS4.1 storage. The datastore is created with IPs of three NFS servers exporting the same CephFS, ESXi host creates three connections to the servers, however only the first one remains active and the other two connections are dropped by ESXi. When the first NFS server gets stopped, no failover occurs, the connection is just marked down.

I set up everything according to the whitepaper there [https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/vmware-nfs-bestpractices-white-paper-en.pdf](https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/vmware-nfs-bestpractices-white-paper-en.pdf). iSCSI is out of question due to lack of support of LACP for binded NICs.

So, can somebody give me a definitive answer what is going with NFS4.1 and multipath setup?

View Reddit by lantianzView Source

Related Articles


  1. I’m a bot, *bleep*, *bloop*. Someone has linked to this thread from another place on reddit:

    – [/r/ceph] [(x-post from r/vmware) NFS4.1 Multipath with Ganesha and CephFS](https://www.reddit.com/r/ceph/comments/e608cm/xpost_from_rvmware_nfs41_multipath_with_ganesha/)

     *^(If you follow any of the above links, please respect the rules of reddit and don’t vote in the other threads.) ^([Info](/r/TotesMessenger) ^/ ^[Contact](/message/compose?to=/r/TotesMessenger))*

  2. Are your 3 IPs on different subnets/vlans?

    If everything is in the same subnet then multipath will function only as failover.

    Of you separate the 3 targets to 3 unique subnets then you will get the combined speed of your 3 links.

  3. I got this setup awhile back, and posted a blog about it:


    I had to drop to the command line for a few things on ESXi 6.5, as the web gui wouldn’t allow some of the options to browse the datastore. But I think those were resolved in a subsequent version of 6.5, and for sure fixed with 6.7

    I did have to have the different subnets configured, or else I didn’t get the aggregated bandwidth.

Leave a Reply

Your email address will not be published. Required fields are marked *