proxmox
250 Virtual Machines on a Proxmox Mini PC
250 Virtual Machines on a Proxmox Mini PC
#Virtual #Machines #Proxmox #Mini
“VirtualizationHowto”
This was such a fun experiment to see how many virtual machines you can realistically get on a modern mini PC with 96 GB of memory. I tested using the Geekom AE7 with the Ryzen 7840HS processor with 16 threads and 96 GB of DDR5 memory. See just how far I was able to get running VMs on a mini PC!…
source
To see the full content, share this page by clicking one of the buttons below |
wtf, use ansible for it xd
Well, I'm not impressed. Most of these VMs are idle! Yes, I was expecting ALL VMs run the stress test utility or some program(s) that simulate user activity. Now that would be realistic.
HI , Can l run eve ng ccie lab on this?
What happens when you increase the Swap? Or is that not possible? I wonder how many stressed machines you can have before it comes crashing down.
Its a nit-pick but with 100% swap usage how much would that really hit performance on anything prod? That aside, this is a true testament to how far we have come in applied SFF. Thanks.
Brandon, Which small mini pc would you go with? MS-01 or MS-A1 or something else?
The randomly shutting down VM's comment makes me wonder if my current cloud provider is over provisioning!
I am thinking if we give proxmox in a custom system a dedicated nvme swap drive with the same amount of swap space as the amount of ram. Could that be a help for spinning up more or similar amounts with more workloads in each VM? We learnt that these days we don't need that much of swap space because of speeds we achieved. But we are the next level, so swap can be useful again with these fast hard drives?
Thanks for the demo and info. This is great, and makes me wonder if I should scrap my power hungry HP DL 360 G8. Have a great day
That’s a great test to perform. With all of them running, but no workloads being down on those VM‘s, it makes sense that you should be able to do that. However, if you have workloads going on any of those virtual machines, you absolutely will not be able to put that money on there.how many can you spend up the actually have viable workload on them. That would be interested in seeing
What about containers instead of VMs ? Thanks for this video.
Now try Windows Server 2022
Next you can run a test where you can test just how many VMs you can run on that machine with everyone one of the VMs running stress test.
It looks as though each VM was just a process.
stop using light mode jesus
Could you do the same test but using the MS-01 would be good to compare Intel vs AMD plus you could compare the eco threads and its impact in the home lab?
If only there would be these mini pc's with sfp+ or if I could be greedy two, 1 for my fault domain/block storage and 1 for my network access. That's okay, I enjoy my enterprise eqpt but hate my power bill
Hello, great demonstration there, i've got nothing more to add except one thing : the test was a little skewed, because as your video shows, there was 100% swap used all along. I don't know Proxmox, i'm more an ESX guy, but your hypervisor consumes RAM, as well as your VMs. Using 8 GB swap is almost similar as adding 8GB more RAM. Albeit SSDs / NVMEs are a much slower and wearable kind of RAM. We could imagine a scenario where you would use 1TB of NVME, and be able to run thousands of VMs, althrough as i've said earlier, much slower. 8GB is not a lot, but it's significant on a 96GB RAM system (12,5%).
But i DO agree with you, there never was a better time to run a homelab, even if i despair seeing more memory limits and connectivity on these mini-PCs !
I personnaly prefer the "shuttle" size, the shoebox format, for heat evacuation, dual 10Gbps fiber connectivity, and the 128GB RAM limit).
You are crazy, I hope you know that. That's why I love your channel!
Very interesting experiment!
You could also use Terraform or Ansible to automate the template cloning, must have been tiring doing the full clone 250 times!
Great! But, are those vm's payloaded with some software? Like lamp for example? Or other?
You could probably push it a bit further if you expand the swap space. sitting at 8GB fully used in the example you gave. But personally, I try to avoid any swap usage if possible since disks are so much slower.
Now try without swap
Great test. I would have like to see the power drawn from the system and also the temperatures it rose to. Those can have a negative impact in the life expectancy of your hardware, so please be aware of this if you try to consolidate your entire homelab into a miniPC 🙂
Excellent presentation. Thank you. Really enjoyed this one. Even more impressive since that CPU only has 8 actual cores with hyperthreading. I think this test really shows how good both AMD processors are and also Proxmox.
You made a comment about the IO Delay going up. Would like to see a video about this specifically. Back when I had Proxmox servers that had regular old hard disk drives with spinning platters, I don't remember ever seeing this type of behavior. It was only with SSDs that I started to notice it. It also appears that the type of SSD can make a huge difference in this and many posts I've read indicate that an "Enterprise grade" SSD is required for Proxmox. But I haven't found a good definition or explanation for what an Enterprise grade SSD means.
Nice! I'm curious what the Proxmox log was showing when it shut down those VMs on you. It was off the bottom of the screen in your recording. Then again, you may even have had that closed. But that would provide more details on the VMs that got shut down and possibly the exact reason why they were shut down.
I would like to see another version of this, instead of Ubuntu use Windows VMs.
Interesting but somewhat theoretical. I use VirtualBox since 2009 and it has a comparable function to share pages between VMs. I collected 70 desktop VMs, and often I run 2 at the same time: Xubuntu 24.04 LTS (2,560 MB) for social media and Ubuntu 16.04 ESM (2048 MB) exclusively for banking, encrypted by VirtualBox. The third one I often use is Windows XP (768 MB) as jukebox. I stopped using page sharing, since it takes the fun out of using desktop VMs. I use VMs for security and to reuse ancient VMs dear to me, and page sharing forces you to use the same VMs for everything.
My 2019 VM "server" is smaller than yours; Ryzen 5 2200G; 16GB of which max 4GB is used for L1ARC memory cache. The Host OS is a minimal install of Ubuntu 24.04 LTS with Conky; VirtualBox and OpenZFS. Another interesting property of my VMs is that they all use swap space (0 to 500 MB), and I don't care, because most ends up in the lz4 compressed L1ARC memory cache :). So a VM running on OpenZFS swaps to memory 🙂 Only if the swap size often exceeds 500MB, I tend to increase the VM memory size.
great, congrats video. a doubt:
do you created lxc/vms manually or automatically? is possible to create dynamically automated any lxc/vms? if yes, how?