RAM-based functions and routines

[hippy]

What is the magical incantation for getting CMake, Make, GCC, whatever, to warn or error when a routine or function which runs in RAM is calling a function in Flash ?

How should one resolve the issue when that does arise ?

As far as I can tell the only option is to clone the parts of the Pico SDK which contain the called functions in Flash, edit those to make them local 'run in RAM' functions, repeat for everything those call in Flash.

It seems it would be possible to scan one's own source see what is 'run in RAM' and should not call Flash, run through Pico SDK to find those routines, clone them as 'run in RAM', and repeat recursively but that would seem to be a lot of work, having to handle pre-processing directives and macros or one is likely to miss something.

Is there such a tool already ?

Or some way to use CMake to build object code with the RAM-based functions which uses normal includes, make that all 'run in RAM' and then link it with the rest of the code which runs in Flash ?

I know one can build an entire project to 'run in RAM' or 'copy to RAM' but in my case it's MicroPython which is too large to run from RAM but I'm needing to call routines which must, along with all they call, run solely in RAM.

So far my 'run in RAM' module calls 'save_and_disable_interrupts', 'flash_range_erase', flash_range_program' and 'watchdog_reboot'. The first is "inline" so I'm presuming it will be in-lined into RAM ( but is that guaranteed ? ), the erase and program are '__no_inline_not_in_flash_func' so safe to call, but 'watchdog_reboot' is in Flash and I haven't dug down into what that calls.

That should be possible to solve by hand but I am expecting my calls from RAM into Pico SDK to expand.

Code: Select all

.-----------------------------------.----------------------------.
| MicroPython - Run in Flash        | Pico SDK - Run in Flash    |
`-------.---------------------------^----------------------------'
        |                                          /|\
     /  |   .---------.    .-------------------.    |
    |   `-->| My func |--->| disable interupts |    |
    |       |         |--->| erase flash       |    |
RAM |       |         |--->| write flash       |    |
    |       |         |    `-------------------'    |
    |       |         |                             |
    |       |         |----- watchdog reboot -------'
     \      `---------'

[mschnell]

What is the magical incantation for getting CMake, Make, GCC, whatever, to warn or error when a routine or function which runs in RAM is calling a function in Flash ?

Why should it warn ? That is perfectly legal.
-Michael

[kilograham]

you can look for bl.*veneer in the .dis at a RAM address

[hippy]

What is the magical incantation for getting CMake, Make, GCC, whatever, to warn or error when a routine or function which runs in RAM is calling a function in Flash ?

Why should it warn ? That is perfectly legal.
-Michael

Yes, perfectly legal. What I am wanting is to be warned when it does that because it would be catastrophic for what I am doing. It could lead to jumping into erased or changed Flash, could mean executing code other than what was there when compiled and uploaded, with all the adverse consequences of doing that.

you can look for bl.*veneer in the .dis at a RAM address

Thanks. I'm guessing these veneers are cross-domain calls from RAM to Flash, Flash to RAM -

Code: Select all

1003deac <watchdog_reboot>:
1003deac:       b570            push    {r4, r5, r6, lr}
...
20007d10 <__watchdog_reboot_veneer>:
20007d10:       b401            push    {r0}
20007d12:       4802            ldr     r0, [pc, #8]    ; (20007d1c <__watchdog>
20007d14:       4684            mov     ip, r0
20007d16:       bc01            pop     {r0}
20007d18:       4760            bx      ip
20007d1a:       bf00            nop
20007d1c:       1003dead        .word   0x1003dead

Wondering why they use a veneer rather than just call direct ? Setting the lsb for Thumb ? Time to ask Google I guess.

But seems to work, and matching on just '<watchdog_reboot>:' seems good enough -

Code: Select all

#define TARGET_MUST_BE_IN_RAM(x) x

static void __no_inline_not_in_flash_func(burner)(uint8_t *addr, uint32_t size) {
        size = (size | 4095) + 1;
        save_and_disable_interrupts();
        TARGET_MUST_BE_IN_RAM(flash_range_erase(0, size));
        TARGET_MUST_BE_IN_RAM(flash_range_program(0, addr, size));
        TARGET_MUST_BE_IN_RAM(watchdog_reboot(0, 0, 0));
}

Code: Select all

pi@Pi3B:~/pico/micropython/ports/picopython $ python3 check_domain.py
In ./core/extmod_moduos.c
  Line  152 : Function 'burner'
        157 :   'watchdog_reboot'                Calls Flash

Turns out checking the smaller '*.elf.map' is quicker and also provides a hint as to what I've got to clone to fix things ...

Code: Select all

.text.watchdog_reboot
                0x1003deac       0x40 CMakeFiles/picopython.dir/home/pi/pico/micropython/lib/pico-sdk/src/rp2_common/hardware_watchdog/watchdog.c.obj
                0x1003deac                watchdog_reboot

[alastairpatrick]

Maybe it would be useful to set up the memory protection unit (MPU) to cause a hard fault if flash is accessed while running (hopefully) only from RAM. The SDK has no support for MPU but the registers are documented in the RP2040 datasheet. There are two MPUs, one for each core.

[hippy]

Using MPU is in an interesting idea though 'if (&watchdog_reboot < 0x20000000)' may be easier to implement.

I prefer to avoid the issue at build time, before running the executable. While a safety net to avoid catastrophe it doesn't help much in telling me where things have gone wrong as, not knowing what's now in Flash or where, there's no easy alternative but to hang in a tight loop or add a whole lot more code.

[MikeDB]

What is the magical incantation for getting CMake, Make, GCC, whatever, to warn or error when a routine or function which runs in RAM is calling a function in Flash ?

Why should it warn ? That is perfectly legal.
-Michael

Not if you haven't got a Flash it isn't. We use external download at boot and it's a nightmare catching these odds and ends.

[dthacher]

I just use copy to RAM so that I do not have to mess with this. I would avoid runtime solutions. The issue with the script is it cannot see hidden symbols in the SDK.

To fix it you will need a dependency database for all SDK symbols. Hippy is likely correct that it is easier to mark the SDK functions themselves manually. A script will move through your code looking for all the functions marked to be scanned.

Once the list of SDK functions have been found those plus all following dependencies will be entered into a custom linker script. The benefit to this solution is it can be done pre build. This should ensure there is no symbols called in flash from RAM. This will also force the linker's hand. This will keep the code mostly clean and maintain performance. This should be easier to implement once the dependencies are known. (Note this will change with every commit to the SDK.) You will need to find and remove duplicates.

Any ideas on how to build this database without manual work?

[hippy]

I have come to the conclusion it's not at all easy. I am not convinced walking the Pico SDK will help because, even if you can identify everything, you won't know where things will end up after GCC and the linker have performed their magic; RAM, Flash and possibly both. The code we are calling might be local, not in the Pico SDK.

Even my top-level marking scheme falls to pieces pretty quickly. Consider 'TARGET_MUST_BE_IN_RAM(debug("Oops"))'. That 'debug' may not be a function, may expand to some function we don't know the name of, may depend on what options have been set elsewhere, may be in-lined code, or may be nothing at all.

I have decided that's not the best approach; that it's better to mark a function as 'Everything this function calls must be in RAM'. That makes things a little trickier at the top-level but easier overall, while avoiding issues with GCC in-lining stuff. The top-level stuff is minimal, what then needs to get cloned, included, edited to make it work, means effort expended increases exponentially and this helps minimise that -

Code: Select all

#define MUST_BE_IN_RAM(func) __no_inline_not_in_flash_func(func)

#define ONLY_CALLS_RAM(func) MUST_BE_IN_RAM(func)

static void ONLY_CALLS_RAM(actual_burner)(uint8_t *addr, uint32_t size);

static void MUST_BE_IN_RAM(burner)(uint8_t *addr, uint32_t size) {
        size = (size | 4095) + 1;
        save_and_disable_interrupts();
        actual_burner(addr, size);
}

static void ONLY_CALLS_RAM(actual_burner)(uint8_t *addr, uint32_t size) {
        flash_range_erase(0, size);
        flash_range_program(0, addr, size);
        watchdog_reboot(0, 0, 0);
}

Walking the disassembly of the executable code and building a call tree to discover all functions which can end up in Flash may be the solution but that won't handle function pointers and dynamic code. That's probably not an issue for what I and most will need but it means it's not a universal and infallible solution.

If there are existing tools for producing a call tree that may avoid a lot of effort.

[MikeDB]

If there are existing tools for producing a call tree that may avoid a lot of effort.

There are such tools but the problem lies in the toolset. For many MCUs you define where all the different parts of the code, stack and heap go as many have lots of different lumps of memory, often on different buses with different speeds, some cached, some not, often not always contiguous either, so one of the first jobs in any design is partitioning the memory map, followed by creating the clock tree to optimise speed and power consumption.

[hippy]

It's a bit easier for the RP2040 and what I am doing because I'm only worrying about execution flow and jumping from 0x2nnnnnnn into 0x1nnnnnnn.

I did discover GCC's '-fcallgraph-info' for creating call trees which was promising in tests but then I discovered that's not supported by GCC 7.3.1 which is what 'apt' brings in for Raspberry Pi OS 32-bit Buster.

So I thought I'd go DIY, create the call tree myself from the executable disassembly and that was easier than expected.

But the results weren't. I was expecting my call to 'watchdog_reboot' to be an issue because we already know it uses a veneer to jump into Flash, but I wasn't expecting the call to '__no_inline_not_in_flash_func(flash_range_erase)' to potentially call into Flash ...

Code: Select all

Walking './build/picopython.dis'
FUNC 200000C0 actual_burner
     20000618   flash_range_erase
     200078A0     __hard_assertion_failure_veneer
FAIL 1003DB4D       hard_assertion_failure
     200005B0     flash_init_boot2_copyout
     20033C5D       <Unknown>
     20032E40       <Unknown>
FAIL 10000100       __VECTOR_TABLE
     200005E4     flash_enable_xip_via_boot2
     00004649     Calls ROM function
     00005845     Calls ROM function
     00004552     Calls ROM function
     00004346     Calls ROM function
     20000690   flash_range_program
     00005052     Calls ROM function
     20007AB0   __watchdog_reboot_veneer
FAIL 1003DEAD     watchdog_reboot

At least the changes needed aren't too bad, and I can get my app to do a lot of the work, tell me what must be done -

Code: Select all

FILE burn.fix
FIND flash_range_erase
EDIT clone_flash_range_erase
ALSO #include "clone_hardware_flash_flash.c"

FILE burn.fix
FIND watchdog_reboot
EDIT clone_watchdog_reboot
ALSO #include "clone_hardware_watchdog_watchdog.c"

FILE clone_hardware_flash_flash.c.fix
FROM ~/pico/micropython/lib/pico-sdk/src/rp2_common/hardware_flash/flash.c
FIND flash_range_erase
EDIT ONLY_CALLS_RAM(clone_flash_range_erase)

FILE clone_hardware_watchdog_watchdog.c.fix
FROM ~/pico/micropython/lib/pico-sdk/src/rp2_common/hardware_watchdog/watchdog.c
FIND watchdog_reboot
EDIT ONLY_CALLS_RAM(clone_watchdog_reboot)

So it's build, check, fix, build, check, until it says it's okay. I'll let you know how well that goes.

[dthacher]

I was under the impression that I could cherry pick C symbols in the linker script to a specific memory. It was inline that the compiler and linker could mess around with it. A function that was not inlined had its own memory location independent of the caller's.

[hippy]

I was under the impression that I could cherry pick C symbols in the linker script to a specific memory.

I am afraid I have no idea so can't comment on that. I am wanting to do this using stock configuration so no messing with linker scripts for me. I'm not really worried about where things end up only if particular RAM routines call into Flash when they shouldn't.

[hippy]

Not tested, and ignore the false positive in 'clone_flash_enable_xip_via_boot2' which is a copy up to address constant rather than a jump or call, "time for bed", said Zebedee.

Code: Select all

FUNC 20000214 actual_burner
     20000104   clone_flash_range_erase.constprop.0
     200000C0     clone_flash_init_boot2_copyout
     20033E9D       <Unknown>
     20032F80       <Unknown>
FAIL 10000100       __VECTOR_TABLE
     200000F4     clone_flash_enable_xip_via_boot2
     00004649     Calls ROM function
     00005845     Calls ROM function
     00004552     Calls ROM function
     00004346     Calls ROM function
     20000164   clone_flash_range_program.constprop.1
     00005052     Calls ROM function
     200001F4   clone_watchdog_reboot.constprop.2
     200001C4     clone_watchdog_enable.constprop.3
     4005B000       <Unknown>
     0001FFFC       Calls ROM function
     40012008       <Unknown>
     4005A000       <Unknown>
     40058000     <Unknown>