How Many Pies in 2U?

[ejolson]

I've read the thread about a 3U case that can hold 70 Raspberry Pi 3B computers.

viewtopic.php?f=63&t=155030

I'm also aware of Bitscope and what looks looks like the custom racks they produce.

https://www.bitscope.com/product/blade/?p=index

I'm wondering if there is a 2U solution that would fit in a standard rack that can be configured similarly to

• 24 to 30 Pi 4B computers that can be individually remote power cycled.
• Internal gigabit switch with two 10GB uplink ports.
• some 3.5" HDs and 2.5" HDs.
• mATX or ITX server motherboard with IPMI connected to the HDs.
• Suitable power supply for all that.

The idea would be to create a complete setup in a 2U space that could be easily collocated and administered remotely. It would be nice if there were a company behind the product offering a warranty, but any detailed do it yourself project would also be interesting.

[deepo]

I found this 3D printer solution, which holds 13 RPi's that have a POE hat installed. So no need for other cabling.
So a lot of space behind the RPi's is not occupied.
https://www.youtube.com/watch?v=splC57efBFQ
https://www.thingiverse.com/thing:4078710

When you only have POE you can't power cycle the RPi, and there's no support for HDD's.
So maybe that's not what you're after.

The only other rack solution I have seen is this:
http://my.bitscope.com/store/?p=view&i=product+BR20A

But that's 5U and more for display I guess.

/Mogens

[ejolson]

I found this 3D printer solution, which holds 13 RPi's that have a POE hat installed. So no need for other cabling.
So a lot of space behind the RPi's is not occupied.
https://www.youtube.com/watch?v=splC57efBFQ
https://www.thingiverse.com/thing:4078710

When you only have POE you can't power cycle the RPi, and there's no support for HDD's.
So maybe that's not what you're after.

The only other rack solution I have seen is this:
http://my.bitscope.com/store/?p=view&i=product+BR20A

But that's 5U and more for display I guess.

/Mogens

I thought one of the advantages of POE is an ability to power cycle the Pi, assuming a reasonable POE switch.

Much as I like it, the Bitscope 5U box

https://www.bitscope.com/blog/FM/?p=GF13L

with 150 inside still requires a switch and server. That occupies more than 7U. Since collocation charges by the inch, it would be nice for everything to be bundled into a single 2U chassis.

In my opinion a 3D printed bracket will unlikely handle the heat generated by a high-density collection of Pi computers. Since they can run without SD cards, it makes sense to me to stack them on end with no plastic to block air flow.

[ejolson]

Inspired by the thread

viewtopic.php?f=63&t=275451#p1668659

I've just started the process of getting a 2U server chassis. Since the university surplus auction is quarantined because of the coronavirus, it's looks like I might end up with a new case rather than one with a server already in it.

That's just as well, because I might have started fixing that old server rather than making an integrated Pi box.

This is a proof of concept project, so it is unlikely I will actually fill the chassis with Raspberry Pi 4B computers. It is even less likely, unless space is donated, that the end result will be collocated in the Citadel. For this reason, I'm not going to bother with a proper server motherboard for the file server, but instead pull the mATX PC and power supply from that dented tower case a friend donated a few months ago.

viewtopic.php?f=63&t=227177&start=100#p1610492

While that AMD A6-5400K is similar in speed to the Pi 4B, it was free and has a power supply. If I could afford more Pi computers to place in the 2U chassis, I'd use a couple 4B's for serving files as well. Free is rarely better than cheap; however, it may be good enough.

[ejolson]

While that AMD A6-5400K is similar in speed to the Pi 4B, it was free and has a power supply.

I was doing some online shopping for 12V DC to 5V DC buck converters. Before pulling the trigger on something which might explode, I decided to open that dented silver box and actually look at the included ATX power supply.

To my surprise, that old power supply predates popular need for the extra 12V capacity to run modern GPUs. It likely predates the Pentium 4 and was apparently repurposed twice before I decided to use it to build a Pi microcloud. Instead of extra 12V capacity, it has a 40 ampere 5V rail. For comparison, a new 1600W Gold certified EVGA SuperNOVA provides only 24 amperes at 5V.

https://www.bhphotovideo.com/c/product/ ... _gold.html

Since there is plenty of 5V power already, there is no need to add the buck converter necessary for a supernova. It looks like free will be significantly better than expensive for this do-it-yourself microscaler. See also

viewtopic.php?f=63&t=275911#p1672336

[ejolson]

The 2U chassis came, a Compucase RA251L00F for less than the price of the 8GB model of Raspberry Pi. I've moved the motherboard, power supply and disk drives from free desktop PC to use as a file server for the pies that will otherwise fill the remaining half of the case. As this will be a self-contained unit I also need a small network switch (or two) to place inside. I'll post some pictures later showing how things are progressing.

Right now, I'm again astonished with what may be a lucky coincidence: That free motherboard has a parallel port header on it. Originally, I had thought to use the GPIO of one of the Pi's to reboot the others for remote administration. Then I thought maybe an 3.3V Arduino connected through USB would be easier. Now I'm pondering whether it might be most reliable to use the parallel port that's already there. I'd need a 5V to 3.3V level shifter and some way to use the 8 data lines to select one of maybe 13 different Pi computers.

Note that the estimated numbers of Pi's needed has fortunately decreased due to how much space was taken up by a birds nest made out of ATX power supply wires. If a Pi had been used for the file server instead of that Intel PC, there would be room many more Pi computers. The difficulty with multiplexing an 8-bit parallel printer port would also vanish.

Maybe I need to get another 2U chassis and try again. At the same time, without a donation of too many Pi computers, I wouldn't be able to justify any more for a prototype anyway. I sent an inquiry to Switch last week but missed the telephone tag. Will try again soon in order to get an idea how much one would have to charge to break even.

[ejolson]

While discussing with a sales person the implications of hundreds of WiFi radios from a micro cloud of Raspberry Pi 4B computers colocated in a high-security data center, I though I would post a few pictures of the current prototype.

Here is the free computer that was disassembled to use as the file server:



Here is the new 2U case:



The case without the Pi computers:



The wires for the mATX board and power supply take about half of the space that would otherwise have been available for more Pi computers. If this were not a prototype intended to write and debug the software, it would have made sense to simply fill the case with raspberries and maybe a bit of cream. As there appears room (circled in green) for about 12 single-board computers left in the case, that should still enough for a micro scaler to raise some eyebrows

[ejolson]

Some people are wondering why build a cloud out of Pi computers when the Pi works so well as a tinkering device for turning LED lights on and off. The new Pi 4B can even run a web browser and function as a desktop replacement, especially if you also replace the SD card with a USB3 HD. Curiously, it is not difficult to search and find click bait indicating desktop sales are declining and server sales are rising with revenue from cloud services growing the fastest:


https://www.nextplatform.com/2020/04/20 ... d-them-to/

While the market leaders are growing between 20 and 40 percent per year, the real miracle is that even the IBM cloud is growing. One reason for this remarkable growth is the use of original design manufactured hardware that is much cheaper, lower power and easier to maintain than the PC-compatible servers sold by Dell, HPE, Lenovo and others.

Since it is not possible for other companies to purchase the optimized cloud hardware used by the hyperscalers, it would seem not possible to compete. On the other hand, the Raspberry Pi 4B, though different in many ways, is surprisingly similar in terms of the minimal design philosophy and performance per watt. This suggests a cloud made out of raspberry pies might also experience 20 to 40 percent growth per year.

[ejolson]

It would appear this misguided project of squeezing a bunch of Pi computers into a 2U chassis has been completed. Except for an additional wire leading from the Pi Zero through a 330 ohm resistor to a spare LED on the front panel, the final hardware arrangement looks like



Note that the Pi computers have been unwisely mounted to blocks of 1x4 lumber fastened to existing mounting posts in the chassis. While including such kindling inside a high-density rack-mount server may seem dangerous, I'm hoping a coat of aluminized paint will hide the present folly in the same way that reclassifying the unsuccessful hospital cases as death-due-to-lack-of-breathing has noticeably reduced the coronavirus death toll.

The dog developer recommended M2.5 standoffs screwed into a piece of ultra-high molecular weight polyethelene with an ignition temperature of 349 degrees Celsius in place of the wood. Since wood chars at 120 degrees and ignites at 150, using wood is like a bandanna in place of an N95 mask. On the other hand, since the bandanna satisfies recommended public safety guidelines, it's clear people like to live dangerously, especially here on the liberal frontier. Along those lines, it seems the bullet holes next door have been patched and the house sold. Fortunately, the new residents scraped my automobile parked curbside with a scooter last month rather than hitting it full-on with one of those off-road quads.

Back on topic, here is a list of parts that should have been used for the build:

• Quarter-inch polyethelene board.
• Eighty-eight M2.5 standoffs.
• Two tiny 8-port gigabit switches.
• Fifteen custom cut network cables.
• Four USB2 hubs with 5V power.
• Four fuses for the USB2 hubs.
• Six 2, four 4 and two 8GB 4B's.
• Four Pi Zeros.
• ATX power with 40 amp 5V rail.
• AMD APU and mATX motherboard.
• Two 4GB memory sticks.
• Additional networking card.
• Two 3.5" hard disks.
• 2U server chassis and five fans.

Since the hard disks, power supply and Intel-compatible PC were recycled, the major cost of the complete system was the price of the Pi computers, the time taken to stuff everything into the box and the mental anguish that resulted from breaking three extra USB hubs before figuring out how to properly pry them apart and rewire them.

While it would have been possible to stuff a few more Pi computers into the case, I restricted the number to twelve to make sure the ATX power supply could handle them. I saw at least one ATX supply with a 50 amp 5V rail for sale, but most new supplies deliver only around 20 amps at 5V. As already mentioned, if the mATX motherboard and related hardware were replaced by another Raspberry Pi, it would be possible to fit in many more Pi.

I will reserve this thread for additional discussion of the hardware and start another thread on configuring the software. For now, please also see

viewtopic.php?t=274553
viewtopic.php?f=63&t=275911

Rather than placing bets on whether the wood will spontaneously burst into flame, if people are interested in speculation, it would be interesting to know if the 5 amp fuses used for the USB hubs delivering the power to the Pi computers will blow. Note each fuse delivers power to three Pi 4B computers and one Zero.

Will simultaneously running Rosetta at home on all the Pi 4B computers blow a fuse? Will a fuse blow when running the Pi pie chart program

viewtopic.php?p=1668056#p1668056

in stress mode on all nodes? What about cpuburn? Any thoughts?

[ejolson]

For reference, a thread on setting up software for the hardware described in this thread is being continued at

viewtopic.php?f=36&t=279176

[alkersan]

Just wanted to share what i've come with when was tinkering with rpi recently: album
I know it's not a 2U, and even not rack mountable, but maybe it will give you some extra ideas to try.

Upd: btw, this switch is managed and have a rather simple undocumented REST API in it's latest firmware, which allows to power cycle the PoE ports.

[ejolson]

Just wanted to share what i've come with when was tinkering with rpi recently: album
I know it's not a 2U, and even not rack mountable, but maybe it will give you some extra ideas to try.

Upd: btw, this switch is managed and have a rather simple undocumented REST API in it's latest firmware, which allows to power cycle the PoE ports.

It looks quite functional. Does each Pi have an SSD attached?

[alkersan]

Does each Pi have an SSD attached?

Unfortunately, I didn't complete that inverted USB plug-to-receptacle adapter yet, but the plan was to integrate a small usb-ssd drive into the sliding base and connect it to usb3 port of rpi with smallest footprint.