VMware

4 Node vSAN in Homelab

I’m planning out a vSAN cluster for my homelab. I’ll most likely use the following specifications:

* vSAN Enterprise 6.7
* 4x Core i3 4th Gen 16GB DDR3 Nodes
* 4x 1TB SATA SSD Capacity Tier
* 4x 128GB NVMe Cache Tier
* 4x 1GBe NICs

Most VMs will run on the nodes themselves, but some will be hosted on other nodes connecting to the vSAN cluster for storage.

To my questions:

* Does this sound reasonable in a homelab setting?
* I haven’t decided on whether to run RAID-1 or RAID-5. Any input on this? Regarding performance and fault tolerance mainly, but other considerations are welcome as well!
* How much of a bottleneck will the 1GBe NICs be?
* Adding 10GBe NICs aren’t an option unfortunately.
* I already have the SATA and NVMe SSDs around. Should I use the SATA SSDs for something else and just go with mechanical drives + NVMe? Will the 1GBe NICs be enough of a bottleneck to warrant this?
* Have I forgotten something?

Thank you so much! I’ve learnt so much from reading this subreddit.


View Reddit by GAGARIN0461View Source

Related Articles

7 Comments

  1. Not enough RAM. I had 32GB in each 3 node cluster and vSAN and vSAN took up at least 10GB on each node. Ended up just scrapping it. This is in the docs somewhere that I just didn’t look at until it happened.

  2. I would go with 10GbE since it’s so cheap now and it will make vmotions a lot faster. I would also go with R1 for the performance and again because the SSD are so much cheaper than they used to be. Also, I like your tiering strategy.

  3. The bottleneck will really depend on the amount of writes you’ll need to do in the backend.
    RAID5 will also increase network traffic.
    Another thing that might be a bottleneck is the 16gb ram. VSAN also uses RAM, see if you can upgrade that

  4. I would not run raid of any form with vsan, and only use a raid controller if it has hba mode. Vsan works best with direct access to the disks.

  5. Not enough RAM, also not worth doing anything DDR3 based right now- you want to be able to run the next version too. 10 gig is cheap for four nodes, $130 for a microtik switch.

  6. >I haven’t decided on whether to run RAID-1 or RAID-5. Any input on this?

    my impression where that you would run vSAN?

  7. I have a homelab with a somewhat similar (but older) setup:

    * 4x Mac Mini Late 2012
    * 200 GB Intel SSD (SATA)
    * 1TB spinning disk (SATA)
    * 16 GB RAM per node
    * 1x 1Gbe NIC per node
    * Hybrid VSAN
    * Super janky consumer network gear which I’m too embarrassed to name here

    As others have said, you’re a bit light on RAM, but as long as you’re not going to run anything crazy on it, it’ll work fine. Leave dedupe and compression turned off on your VSAN and it’ll probably be okay.

    1Gbe (and 16 GB RAM) works fine on a hybrid VSAN setup, but will be a bottleneck with an all-flash setup. But if those are the disks you’ve got, it should work fine for a homelab. I added a couple of iSCSI datastores to my setup (from a Netgear ReadyNAS), and my VSAN outperforms that consistently.

    For my money, part of the fun of a homelab is shoehorning the equipment you’ve got into a workable setup. As long as you’re not expecting enterprise-level performance and you’re okay with the occasional funkiness from nonstandard hardware, you should be just fine with that setup.

Leave a Reply

Close