proxmox
How to Set Up SR-IOV with Intel Flex 170
How to Set Up SR-IOV with Intel Flex 170
#Set #SRIOV #Intel #Flex
“Level1Linux”
Join Wendell as he takes you through the how to set up SR-IOV with intel.
Check out the forum post here:
Supermicro system here:…
source
To see the full content, share this page by clicking one of the buttons below |
I got a Rocket Lake machine (W-1370P), only to find out that even though one Intel page says it supports GVT-G, and another says it supports SR-IOV, it actually supports NEITHER. I guess Rocket Lake is kind of a forgotten/abandoned generation.
That's not the only thing missing on Rocket Lake… the ieh_edac.c driver is both missing from upstream and missing the PCI ID for Rocket Lake. I wonder if the IEH hardware is just broken? Supermicro has it disabled in some hidden setup options.
I have a brand new MS-01 sitting in front of me awaiting deployment. I've been stuck setting up ollama on my TrueNAS Scale install on my Poweredge T30 /w a RTX 2070. I've also been debating what to install on it (vs TrueNAS). I had Proxmox on my Poweredge, but I removed it and put TrueNAS bare metal. All my True Chart apps get stuck installing/starting. Once I solve that, I was going to setup my MS-01 using the 2.5G port at first – i'm awaiting on two shipments of 4x intel OPT/SR/10G modules and 2 pairs of MMF. I'm glad you made this video. I'm going to watch it multiple times before I start digging in this amazing piece of hardware (for the price). It's my primary home server with the PowerEdge (hopefully) being my mid-range AI box doing only that (inference, not training until a video card upgrade). The poweredge was my "first real server". I've worked support that had a datacenter on prem with 4U Compaq servers from 20 years ago, so seeing something like this in my home today is mind blowing.
Sooooo is it mother board?
This is amazing. Thanks for the detailed video and the even more detailed article on the forum. I can't wait to try this. 🙂
I've been struggling with this for about a week. It's spectacularly non-intuitive and confusing. I've got a Mini-PC with an i7-12700T (iHD 770) that I'm using for a low power Proxmox server, and I got really interested in SR-IOV not just for Plex/transcoding but also to attempt to hardware accelerate VMs with remote desktops to get 1080p60 fps + not-glitchy, low latency sound. Particularly for things like Software Defined Radio and other low-latency audio processing, if they're performant enough, SR-IOV VFs would be ideal. From the video it sounds like I could at least get a couple of those going at once, which is all I want. Even one would be great.
I've tried VirtIO-GL, and it actually works for hardware acceleration with the non-free version of the intel-media-va driver, but it's horrifically inefficient and dependent on software support that doesn't seem to exist in most (any?) remote desktop servers–I've actually managed to get 1080p60fps video out of Sunshine server on a 4 core 12500T VM with VirGL (host) and va-api (guest), but the audio was a choppy, high-latency useless mess. It's a shame because it feels like if VirGL was powerful enough, it'd be plenty for most home use cases. Hopefully its replacement Venus will do better.
As someone who actually wants to virtualize GUI-based OSes for testing/specific production use cases, it's been very frustrating to realize the best, non-maddening way to do it is to give a Proxmox server an entire real GPU, preferably one that supports vGPU functionality so I can slice it up between multiple VMs. Passing through a whole iGPU in Proxmox is an absolute nightmare that's not worth it (and doesn't seem to work with Intel iGPUs, anyway), and unless things improve dramatically in Linux 6.8, SR-IOV really isn't worth it for that use case, either, versus the effort to set it up and keep it working and relatively low performance.
This whole experiment has taught me a lot about Proxmox and Linux, but more than that taught me that my use cases pretty much demand a server with at least one slot for a GPU. Hopefully, one day iGPUs will be powerful enough to change that, but I suspect it'll be a while. It's disappointing, as my use cases are otherwise so small in scale that mini-PCs would be perfectly adequate.
I'm more convinced than ever that I need Minisforum to put out an update to the MS-01 that has room for actual usable GPUs. 🙂
Currently using my A380 as the host gpu for my passthrough setup. Had to fight through new vendor-reset issues and a startup script with that `echo 'device_specific' > /sys/bus/pci/devices/0000:03:00.0/reset_method` fix people got working, in order to get my 5700 xt passed through.
Wouldn't it be awesome if consumer Battlemage supported SR-IOV out of the box haha? Or if even a software update brought it to the A770?
Thank goodness Strong Bad's cousin is working on this. I would trust no one else.
13:30 — passthrough second physical gpu
Hummm. First time that YouTube showed me this channel , after I’ve been dubbed to the main L1 channel for YEARS!
I've not seen a company so loathe to supply it's products. The Flex series are essentially unobtainable in the retail environment. Nvidia might charge a lot for their professional cards, but at least there's availability from board vendors and not extreme pricing from server vendors.
Intel making this work with their own consumer GPU’s would be … nice and maybe get their sales up?
Can this be done on a rolling Arch host with kernel 6.8+ instead of Proxmox?
My Linux system has kernel 6.8.7 from the repo for this, I use RichARCH btw. With it I was able to get Arch Linux up and running simply and quickly with kickstart like automation features.
I want to build a Windows XP Cloud Gaming Rig, as in a linux host with 4 VMs with windows XP. These would be all set up with old lan games. I have friends I do LAN Parties with and We can't ever run old stuff properly on modern laptops, so I want the cloud setup to play these, I' want to try this with the iGPU maybe it has enough power for my use case!
I just can't believe it's 2024 and not one of three gpu manufacturers have thought to break ranks and enable gpu sharing pass through on VM's and containers for a consumer card / personal use. Every single person with a homelab / server would buy one.
If sr-iov starts working in Proxmox on consumer grade Intel GPUs (A770, etc.) they'll get my money. I only want accelerated desktops for ~4 VMs (2 more slices for jellyfin and frigate would also be nice).
Too funny, I wasted a bunch of time today trying to get an old HD 530 in some xeon thing from 8 years ago to passthrough to Windows 10 without 43'ing me through PVE. I did NOT succeed. Fun day though!
Wooooooo! SRIOV baby!
There's latency to take into account (not talking about gaming) but that needn't be a problem per se but rather that the use cases are specific, such as neural networks running on GPU in use cases whereby the latency is not the priority. So for example, there might be a probability-estimation or vector similarity comparison on some Fourier Transform to be done retroactively on some data rather than as fast as possible in real-time. The GPU uses with OpenCL could be handy (medical imagery, radiation technologies and so on). That then means heterogeneous computing because the device can be switch between for different kernels including on different microarchitectures. Therefore FPGA or emulation expansion card could be used (and passed through in VM, or Jail via SR-IOV or at least VMDq) so that exotic microarchitecures (most likely custom) can have the heterogeneous computing run on there, especially for testing and bug finding. Managing various VMs passing round the expansion device could be done with Ansible but moreover for the jails on account of the listening on multiple sockets. So then you can listen on multiple sockets but also with pass-through of NICs that support that (VMDq or SR-IOV).
This could also help dependency tree mitigation such as RISC microarchitectures, some of which have a tendency to be deciduous. That then can help with hybrid kernels (like SoC, System on A Chip) so a commonplace A53 might have a SoC made on it by emulating it in a FPGA or emulator-expansion card (even if it is slow). The R-and-D of that (or RISC-V, etc.) can then mean its long-term feasibility study is for something like say power-saving or ecologically sensitive applications. Then it can be studied at home in a home-lab, cost-saving, and saving for licencing such that localisation could be planned for as a reasonable revenue source.
My comment has no hate in it and I do no harm. I am not appalled or afraid, boasting or envying or complaining… Just saying. Psalms23: Giving thanks and praise to the Lord and peace and love. Also, I'd say Matthew6.
Next: "vWendell".
In the trend of everything being virtualized, this is probably the next logical step😉
What's wrong with WSL's paravirtualized GPU? Doesn't Windows' CUDA support show that paravirtualization is usable? QEMU has also had a paravirtualized GPU for years now, and SR-IOV seems like a step backwards.
thanks Wendell
Seems like we can all say if you into getting a pc for any gaming… make your money worth spent. at least a 4060 gpu and a 12th gen intel cpu or a amd 5000 series R5 at least.. just do it