Need help with our server setup!!!!

Hi! I badly need some advice.

Currently, we experienced delays (1 day turnaround time) in processing large
amounts of data (200GB-300GB)
Our company bought a server with ff. specs:
– 2x Intel Xeon Gold 5218
– 512GB RAM

And here’s our current process:

1. We use Elasticsearch to store raw data
2. Retrieve 200GB-300GB of data (from ES) to preprocess it using Python
3. Save data to DB (PostgreSQL)
4. Visualize data using Tableau
(Note: we are using VMware free license with our current setup)

Can you give me an advice on how can we maximize the specs of our server using VMware/Hyper-V?
What are the best setup with our scenario using VMware/Hyper-V?

View Reddit by dreamer2177View Source

Related Articles


  1. You need to profile your process and find out where the bottleneck is. I’m guessing it’s just inefficient code and not an actual underlying hardware issue. Without knowing the issue we can’t tell you how to make it better, because a hypervisor running a bunch of VMs will have the exact same issues you have now.

Leave a Reply

Your email address will not be published. Required fields are marked *