I thought I had seen a pretty good share of things happen to Windows servers virtualized via VMware. This one kind of takes the cake; (no references to Cake Day intended). One of my users is a contractor helping our division to implement Power BI. It’s all fascinating work and I’ll eventually have to learn it myself. But today, I pulling out what little hair I have left.
Over the past 4 days, this Power BI server (which runs the “On-premises data gateway”) has been running out of drive space from 5 -30 GB at a time. When totalling up the size of all the folders on the C: drive, it only comes to about 50 GB. However, the C: drive is now 600 GB in size. (I had to add 100 GB on New Year’s Eve because the drive was showing that it was in the Red with only 10 GB of space left.)
Yesterday, it seemed to work just fine, but today, it’s gradually eating up drive space again and there’s no accounting for how it is being done. I began going through the event viewer logs in 48-hr increments. Each day, usually between 8-9 PM, I see a “Warning”…Event ID 157. When I look that up, the details say (and I’m not kidding),”Disk 1 has been surprise removed.” I’ve never seen an event description like that. When I started looking it up, most forums are saying that it’s an enumerator glitch that can occur when a type of failure occurs (i.e. VM quiescing to get a snapshot). At some point, the system believes that the drive has been removed and reports that to the system and it proceeds onward. I’d love to run the old SMART utility on it, but that only works with PHYSICAL drives.
I understand that the conventional wisdom is to ignore it, but it shows up at the same window of time in the evening and I’m almost certain that it’s got something to do with the drive space being eaten up constantly. If I didn’t know any better, I would probably surmise that it’s tied to some sort of corruption; which I’m not seeing. However, there are plenty of other system and application event logs coming out as either “Warning” or “Error”.
I’m not sure who originally built the VM and installed the software needed. Since SQL Server and some of its companion services play a role in this as well, I’m wondering whether or not to just build a new one from scratch; paying close attention to what’s being done. I’ve basically inherited a seriously flawed VM and my gut says it’s corruption and the short answer is to build a new Power BI Server; particularly since it’s not really “live” yet, but some Production systems are touching it. Before I make a recommendation such as rebuilding, I wanted to get some input from y’all as to whether or not I’m on the right track. Thanks in advance for any wisdom that you can share.
To see the full content, share this page by clicking one of the buttons below