Does Anyone Have any Compiling Tips

[wisnoskij]

Trying to compile Alacrity, and running into out of memory errors even after I added 10g of swap. Trying to figure out if cargo maybe does not have access to use all that memory, or what is going on. htop is showing 850M RAM and 1.2G Swap, right now, and we started out with 64M and 0 while running in CLI mode so most of that should be the compiling.

RasPI 3 B+

[wisnoskij]

And it failed shortly after with a out of memory error. so likely within a few hundred M, what where the error was thrown.

Code: Select all

sl@raspberrypi:~ $ free
               total        used        free      shared  buff/cache   available
Mem:          944980      877744       38352           0       28884       28220
Swap:       10239996     1193216     9046780

[ncx]

After a certain point, you are better off cross compiling for smaller targets that are incapable of building the software natively. Unfortunately, creating 10x more swap space is not the solutions to memory issues. See: https://help.ubuntu.com/community/SwapFaq

[dickon]

I've not run a 32b kernel in a long, long time, so I may be wrong, but IIRC 32b processes on a 32b kernel are limited to either 2GB or 3GB of RAM, depending on kernel compilation options. 32b processes on a 64b kernel I believe get closer to 4GB of RAM. Try again under a 64b kernel; it may get further before bombing out, and if you're lucky, it may complete. Otherwise, cross-compile on a 64b system.

[wisnoskij]

I've not run a 32b kernel in a long, long time, so I may be wrong, but IIRC 32b processes on a 32b kernel are limited to either 2GB or 3GB of RAM, depending on kernel compilation options. 32b processes on a 64b kernel I believe get closer to 4GB of RAM. Try again under a 64b kernel; it may get further before bombing out, and if you're lucky, it may complete. Otherwise, cross-compile on a 64b system.

Thank you for your reply.

Sorry for being a bit of a noob, but that surprised my, running a 64 bit kernel on 32 bit Raspbian? I never would of thought that was a thing.
Do you recommend this over just upgrading to the some 64 bit OS?

[dickon]

As with i686 (and friends) and amd64, you can run aarch32 programs under an aarch64 kernel. As that's a one-line change to config.txt -- uncomment 'arm_64bit=1' and reboot -- it's worth a try. It may not help if the compile requires more than even a 64b kernel can provide to a 32b process -- in which case a cross-compile from something truly 64b will likely be your only[0] option -- but as it's a lot less disruptive than installing a whole new OS, it's worth a try.


[0] It's usually the linking phase which runs out of memory in my experience, and you can often mitigate it with partial links if the build system is attempting to do the whole thing in one go, but that's way, way outside anything I'm prepared to talk anyone through remotely, I'm afraid.

[hippy]

I've not run a 32b kernel in a long, long time, so I may be wrong, but IIRC 32b processes on a 32b kernel are limited to either 2GB or 3GB of RAM, depending on kernel compilation options.

A Pi 3B+ only has 1GB of RAM. Same as on my Pi 3B (non-plus). When I ran into 'not enough memory' issues I installed WSL (Ubuntu) on my Windows 10 box and cross-compiled on that.

[jahboater]

An 8GB Pi4 works too, running the 64-bit OS ...

[wisnoskij]

A Pi 3B+ only has 1GB of RAM. Same as on my Pi 3B (non-plus). When I ran into 'not enough memory' issues I installed WSL (Ubuntu) on my Windows 10 box and cross-compiled on that.

Ya, that would seem to be the correct way of doing this. But my only other computer runs windows and seems to have systemic hardware damage/wear. So their is no way it would support a VM any longer and it is so fragile I really don't like messing with it. That would also feel a little like cheating. Been talking to the Alacritty devs and they think you should not need much ram at all, so I am probably pretty close to getting it to work. And if it just does not work out, I would rather just use a more basic terminal anyways.

[wisnoskij]

An 8GB Pi4 works too, running the 64-bit OS ...

Ya, I think I can just install the beta 64 bit Raspbian on this pi as well, but am thinking of an upgrade. but the lead engineer seems to really hate the entire concept of a 64 bit OS https://www.raspberrypi.com/news/raspbe ... nt-1565609, so I don't know how much TLC those builds get.

[dickon]

I've not run a 32b kernel in a long, long time, so I may be wrong, but IIRC 32b processes on a 32b kernel are limited to either 2GB or 3GB of RAM, depending on kernel compilation options.

A Pi 3B+ only has 1GB of RAM. Same as on my Pi 3B (non-plus).

True, but not relevant. OP had '11GB' of RAM (1GB physical, 10GB virtual), and the process ran out. That's most likely because of an address space issue, not actual RAM availability.

[ncx]

I've not run a 32b kernel in a long, long time, so I may be wrong, but IIRC 32b processes on a 32b kernel are limited to either 2GB or 3GB of RAM, depending on kernel compilation options.

A Pi 3B+ only has 1GB of RAM. Same as on my Pi 3B (non-plus).

True, but not relevant. OP had '11GB' of RAM (1GB physical, 10GB virtual), and the process ran out. That's most likely because of an address space issue, not actual RAM availability.

I wouldn't consider swap as RAM.

The relevant part of the article I linked:

..."diminishing returns" means that if you need more swap space than twice your RAM size, you'd better add more RAM as Hard Disk Drive (HDD) access is about 10³ slower then RAM access, so something that would take 1 second, suddenly takes more then 15 minutes! And still more then a minute on a fast Solid State Drive (SSD)...

[dickon]

I wouldn't consider swap as RAM.

I would: it's just very slow RAM[0]. And if something is failing with 'out of memory', it means that the brk(2) / sbrk(2) syscall has failed, and with '11GB' of available 'RAM', that means address space exhaustion.

The relevant part of the article I linked:

..."diminishing returns" means that if you need more swap space than twice your RAM size, you'd better add more RAM as Hard Disk Drive (HDD) access is about 10³ slower then RAM access, so something that would take 1 second, suddenly takes more then 15 minutes! And still more then a minute on a fast Solid State Drive (SSD)...

I'm not going to dispute that. But memory use isn't always memory use -- think mmap(2) calls, for example, which don't usually take any significant memory until the underlying object is accessed, but can chew through address space like it's going out of fashion -- which I strongly suspect is the case in this instance. The 850MB used is a good indicator. Adding more swap isn't going to help because it's probably suffering from address space exhaustion, which a 64b kernel can do a little about.


[0] In fact, you can consider L1 cache to be memory, L2 cache to be slow cache, DRAM to be slower cache, and swap to be glacial, if it helps. Conceptually, at least, the process doesn't know or care what its instruction is actually accessing at any given point, just that all the magic underneath it works correctly.

[hippy]

I wouldn't consider swap as RAM.

I would: it's just very slow RAM[0].

I wouldn't. But each to his own.

And I might if I had the bare-faced cheek to sell my Pi Zero as having 8 GB RAM simply by leaving an SD Card inserted.

[jahboater]

I wouldn't consider swap as RAM.

I would: it's just very slow RAM[0].

I wouldn't. But each to his own.

Its all virtual memory, and apart from speed, should be indistinguishable to programs.

[ncx]

Its all virtual memory, and apart from speed, should be indistinguishable to programs.

Maybe it would have helped to clarify that I didn't necessarily mean it in the perspective of the program, but as someone reading this who might not be as informed. Generalizations like that without context (which was given, I appreciate it) can lead people to having silly misconceptions.

[Gavinmc42]

I used to have lots of issues on 3B+ compiling, increasing to 1GB swap helped.
Similar when the Pi4 1GB arrived.
Pi4 2GB problems started to be less.
Pi4 4GB nearly never have compile issues except dependencies now.

Most of the time now I use a 2GHz overclocked Pi400 running 32bit Raspberry OS.

On the Pi3B+ I wore out 3 uSD cards due to all the swapping on 10-13hour compiles.

My tip, get a Pi4 or Pi400.

[dickon]

I wouldn't consider swap as RAM.

I would: it's just very slow RAM[0].

I wouldn't. But each to his own.

OK, for the avoidance of doubt: I consider swap to be RAM *in the context of any given process*, which is what we were talking about. It isn't from a hardware perspective, or even a kernel.

Happier?

And I might if I had the bare-faced cheek to sell my Pi Zero as having 8 GB RAM simply by leaving an SD Card inserted.

That would be a bit rude, yes...

[Gavinmc42]

I actually went looking for extra ram in a USB drive format.
They did once exist
But then the 2GB Pi4 came out

With 4GB and USB 3 to SSD adapter(ASMedia chip) and 256GB SSD lots of issues just went away.

[swampdog]

I know nothing of rust but if it builds in parallel (aka effectively "make -j`nproc`") then see if there's a way to drop that to "make -j1" - especially if utilises the gcc linker behind the scenes.

eg: my new PC failed building my custom QT5 stuff whereupon I caught it using *7.9Gb* per linker process. There are various ways to fix it for that build which aren't important here.

Leave 'top' running in a terminal and watch it as it fails. See if you can catch the culprit.

[wisnoskij]

Using the 64 bit kernel fixed it. It looks like I was getting 99% done before, and this added swap space had it finish really quick.