proxmox
They said Pi clusters were CRAZY! (Mars 400)
They said Pi clusters were CRAZY! (Mars 400)
#clusters #CRAZY #Mars
“Jeff Geerling”
Try out Twingate FREE for up to 5 users! Enable fine-grained access control to your resources:
Special thanks to Ambedded for sending over a Mars 400 for testing. I only found out about them through one…
source
To see the full content, share this page by clicking one of the buttons below |
Jeff, ya dummy!
Great video Jeff and looks like some interesting hardware. Who do you think would be the target audience for an machine like this? I'm not familiar with Ceph myself so don't know of it's benefits, but looking at your speeds I can't help feeling that a regular server would outperform in transfer speeds and come at a lower cost, although with higher power usage.
Interesting, but a little bigger than what I could use.
Jeff can you do some content on half rack 1u equipment?
I don’t know about the rest of the home lab community but I think that would be more interesting to us
Next up, you're going to link a raspberry pi cluster to a cluster of raspberry pi clusters over 10g to create a megacluster…. or a clusterf**k…. 😛
Link aggregation is still worth it sometimes! even though the core of your network is only 10Gbe, having the storage linked to the switch at X2 (or higher) to the switch means that a single node on the network can only ever consume 50% (or less) of the total available throughput, it's good at least for multi user access. in your case though maybe not worth it, but just thought i'd mention it!
I couldn't switch to a regular PC PSU for the Storinator so far that I've seen. My Storinator is the 60-drive model and it seems to have larger power requirements.
Also I run all SSDs in one of them and bought 8 hybrid converters to do 128 2.5" drive bays. But logistically, there are many issues.
1. How to connect all the SAS ports without expanders?
2. How to fit everything in the case with all those wires?
3. There are only 6 connectors for drives, but I need 8.
4. The PSUs only have 50A on +5V (250W) each whereas I need about 15A with some headroom per 16 drives.
Doing all this, I found some 12V to 5V converters from Corsair to do 20A each (100W). With 6 of these, I'll be in he safe zone.
Logistically this is a pain to setup, but I'll be doing it this weekend.
Wow. I've introduced to the Ambedded in 2018, back then when the Ambedded's founder husband was teaching me Openshift & Ceph. He told that her wife is starting a ARM-based ceph cluster in a box. At that time, i believe it is using Cortex A7 or A53, but with almost the same rack chassis and 8 node config.
in 2019, I've met her on the Taiwan Computex showcasing the Mars 400
Love the concept, less excited in the licensing aspect of it but appreciate that I as a lowly home labber am not the audience for this product.
This is pretty good, the 4GB of memory per node worries me though. If it's just the storage node, maybe it's fine.
yes dad i need this for school
Palo alto needs to calm down with this pos bs. Ceph is good for only one thing: demonstrate how good is truenas.
That chassy looks like my week-end project cobled together from old hw i've collected over the years. I get how you feel compelled to spam this but 3 nodes at 6k a pop is not really a proposition here.
The Mars 400 really makes for the textbook example of why industrial ARM has taken so long to take off, and why Ampere has done so well in contrast. It has no resale value, the company behind it has no real reputation to speak of and they gave it a heavily used product name, it has no refurb or reapplication opportunity, and it's running on non-distribute closed blobs with minimal oversight. It's an expensive e-waste hulk to dispose of the moment Ambedded decides to drop support in any form or fashion and by extension a ticking time bomb for any business that decides to rely on it for mission-critical tasks, not to mention a security risk. As a CSE, I wouldn't want that thing anywhere near any network, cluster, or farm I'm responsible for.
❤
It's easier to keep 80W of CPU cool than it is 200W of CPU … well no shit Sherlock…
lol oh boy. The first couple times I thought you were saying CIF really weirdly
I want one
Ok Jeff, enough with the raspberry pies now at this point these are relevant projects anymore
Can you use some of these clusters to crack the latest TI calculator signing keys for us? I can't afford to build one, myself.
Well, It's nice to have connections and get free stuff.
Why though? Rasberry Pis are so overpriced for how much of a bottleneck they are compared to actual X86 based computers.
You know its out of your price range when Ambedded just has a button to inquirer with no price given.
I wonder how many you could stack… 20? 50?
Firstly I just want to say I love this. It is awesome to see more arm clusters. That being said, I think starting at $3,500 this feels a bit lacking. Especially the hard drive mounting seem super clunky. The other thing that kinda grinds my gears is why documentation is "liscense" based. Where you need to pay for support to get their documentation. Actual support I understand, but the manual and faq etc. seems a bit excessive.
This is a really cool storage appliance for a homelab! I didn't know that I needed one until you reviewed it
And thanks for the free eBook!
Stop making ads.
Ceph cluster or storage in general on a pi which does not have ecc is just nuts..
Twingate doesn't inherently offer enhanced security by default. If you gain console access, such as through SSH for debugging purposes, you can easily move to other services and machines within the same Layer 2 network segments, bypassing TwinGate's access control. In contrast, I prefer Zerotier, which also supports ingress and egress filtering on a per-network basis, though it's not stateful, but it's free and can be self-hosted. NOW … I'm getting myself a MARS400 for the office – definitely some good stuff! 😉
Just running 4 of my Dell R710 servers is killing me on power. My last utility bill was $600. And most of them are usually just at idle. I am afraid to power up my entire 20 server cluster. The monthly cost savings of just a few of these machines would pay for themselves in short order. The only concern would be running my own software and os. I don't want to be locked into a specific ecosystem. I will have to do some research, but it might be worth moving over to these.
Jeff ya dummy, no one is gonna run those…
4:54 Kids these days with their "full-sized" 3.5" drives. Back in my day, we only had 5.25" drives in 3.25" high by 8 INCHES DEEP bays… and we liked it. 5MB should be more than enough for anyone!!
A single Intel i9 CPU consumes twice as much as this entire appliance! Can't complain about its efficiency.
I do have to say, though Jeff. Who is this video for? It doesn’t seem to follow your usual audience. I see this channel as a “amazing stuff you can do with pi hardware around the home.” It seems, as of late, you are focusing on a lot of very high end hardware and some very extreme cutting edge implementations of arm cpus. If we are moving more into the enterprise from the home because your personal growth depends on it, then perhaps you should branch this type of content into a new channel.
Jeff, Is it worth 10x the price of your original cluster?
Who needs this? Raises his hand …slowly at first, and then yells “I DO!”