DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Tue Sep 19, 2023 1:08 am

I've come up against a stubborn little bug and I can't see to find it's source. It's happens when I have Lua run a C function that sets some values in RAM, from what I can tell there shouldn't be a possibility of a buffer overrun as all calls are setting variables directly and not using pointers. Prior to this call there is writes from Lua to some RAM but they are contained and seem to be working on the areas they are meant to. I'm perplexed and not sure how to start debugging.

Any suggestions?

arg001
Posts: 484
Joined: Tue Jan 23, 2018 10:06 am

Re: Troubles with Lua

Tue Sep 19, 2023 9:48 am

What sort of variables? Have you run out of stack?

DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Tue Sep 19, 2023 1:25 pm

arg001 wrote:
Tue Sep 19, 2023 9:48 am
What sort of variables? Have you run out of stack?
The variables are global, they set register that core 1 uses. I have 17,000 bytes free of RAM and have taken the steps to remove all usage of traditional malloc and friends, I have a memory pool allocator of my own to replace these.

How do I know if I'm running out of stack?

hippy
Posts: 15129
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Troubles with Lua

Tue Sep 19, 2023 3:22 pm

DarkElvenAngel wrote:
Tue Sep 19, 2023 1:08 am
It's happens when I have Lua run a C function that sets some values in RAM
You haven't really described what's happening. Are these RAM variables not being set, aren't being set correctly, some are but some aren't, other variables or RAM is being corrupted, or is the RAM set correctly but then changes ?

DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Tue Sep 19, 2023 5:06 pm

Sorry I never know how much information to give sometimes I feel like I give too much.

So far as I can tell the memory is getting set the problem is core 1 doesn't do what it should be with the information and that's it I can't diagnose the problem.

What I'm doing leading up to this is failure is Lua loads a script that reads from disk and copies data to a ram buffer this works fine. I can run this process over and over with no issues. if I then have the core 1 switch modes by changing the values in RAM then the program goes into an unrecoverable state.

Core 1 is running VGA output. And unrecoverable is the screen either goes half blue (seems to have stopped this) or goes black (this is what happens now) The core doesn't crash.

The odd this is I run other scripts that does the same operations it doesn't change the display mode number and it's fine. I have the function keys set up to change the mode number if I press that then it's fine and doesn't crash and after pressing that I can switch modes again with Lua no issues. and repeating the previous steps no longer causes the issue.

arg001
Posts: 484
Joined: Tue Jan 23, 2018 10:06 am

Re: Troubles with Lua

Tue Sep 19, 2023 8:04 pm

DarkElvenAngel wrote:
Tue Sep 19, 2023 1:25 pm

How do I know if I'm running out of stack?
Wierd memory corruption problems start happening....

The default linker script puts the core0 stack in scratchY and the core1 stack in scratchX, so if you haven't manually placed anything else in there you've got 4K each, and if core0 uses more than 4K it will start overwriting core1's stack; if core1 uses too much it will start overwriting the heap (typically).

You can define PICO_USE_STACK_GUARDS which will cause the SDK startup code to program the MPU such that the last 32 bytes of each stack are read-only and so you will (probably) get an exception if you use too much stack rather than blundering on and corrupting RAM.

Or you can do your own stuff to see how things are going - fill the stack with 0xdeadbeef or similar before starting and see how much gets overwritten, find out where the stack pointer is at any given moment by taking the address of a local variable in a small function etc.

hippy
Posts: 15129
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Troubles with Lua

Tue Sep 19, 2023 8:27 pm

Only thing I can think of at the moment is to add more shared variables, or an array, so Core 1 can write progress, increment counts, and Core 0 can show those and maybe whatever's odd or just not happening might make itself clear.

Also there's UART debug reporting but that might be 'data overload' at VGA refresh rates.

DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Tue Sep 19, 2023 8:36 pm

Are the addresses of the stacks known?

You say default linker script sets the size, could I make the stacks bigger somehow? I don't think the stacks are too small but it could be something to try.
by hippy » Tue Sep 19, 2023 4:27 pm

Only thing I can think of at the moment is to add more shared variables, or an array, so Core 1 can write progress, increment counts, and Core 0 can show those and maybe whatever's odd or just not happening might make itself clear.

Also there's UART debug reporting but that might be 'data overload' at VGA refresh rates.
I haven't enabled UART yet... I could add the array thing is if core 1 corrupts to output I wouldn't see the responce. also if I used uart on the VGA core wouldn't that comprise the timing?

DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Tue Sep 19, 2023 8:55 pm

A little update I've been trying to work out how to merge some work I did on the week on another computer with my WIP here so I've been doing many rebuilds to make sure it's right.

So I found something on switching to mode 0 cause the screen corruption. At least in this latest build. Maybe there is a memory read out of bounds issue?

I added memory dumping and it seems the act of dumping the memory fixes the issue? I don't really understand it if I follow the steps to make the issue it will still happen but if I dump the memory after each step then nothing happens and it works as expected?

What does dumping the memory do.

UPDATE:

So I tried adding the mode change to the script, and now it doesn't muck up anymore....

Oh but it does and now the screen is half blue...

DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Wed Sep 20, 2023 3:15 pm

I've check for memory out of boundary writes and the Lua script doesn't attempt to do it.

unscii-load.lua

Code: Select all

uses("vram")
uses("io")

function loadu(f)
 local H = io.open(f)
 local i = 0
 for _=0,255 do
	H:read(6) -- Discard first six bytes 
	for _=0,7 do
		local d = H:read(2)
		vram.write("font",i,tonumber(d,16)) -- load data memory
		i=i+1
	end
	H:read("L") -- Discard any extra data and line endings
 end
 H:close()
end
unscii-demo.lua

Code: Select all

uses "pt"
pt.set_textmode(2)
dofile "unscii-load.lua"
print ("Hello World")
print("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*()")
loadu("unscii-8.hex")
pt.sleep(3000)
loadu("unscii-8-thin.hex")
pt.sleep(3000)
loadu("unscii-8-alt.hex")
pt.sleep(3000)
loadu("unscii-8-mcr.hex")
pt.sleep(3000)
loadu("unscii-8-fantasy.hex")
pt.sleep(3000)
-- load_font(i) where i is the index. 
-- memcpy(_VBUFFERS.FONT_BUFFER,&font_rom[i * 0x100 * GLYPH_HEIGHT], 0x100 * GLYPH_HEIGHT);
pt.load_font(0) -- copy font from ROM back to RAM 2048 bytes
print("all done!")
The vram library has limit checks in place that don't allow for writes outside a selected range in the case of the script it uses the font range. I did see that if the data is a table it could write past the end of the boundary but in this case the data is a number. The load_font function is bound checked as well so it cannot access outside it's address space a simple i &= 3 is used and the ROM holds 8 k of data.

arg001
Posts: 484
Joined: Tue Jan 23, 2018 10:06 am

Re: Troubles with Lua

Wed Sep 20, 2023 3:55 pm

DarkElvenAngel wrote:
Tue Sep 19, 2023 8:36 pm
Are the addresses of the stacks known?

You say default linker script sets the size, could I make the stacks bigger somehow? I don't think the stacks are too small but it could be something to try.
Look in the <projectname>.elf.map file generated in the build, and in particular at the symbols:

__StackTop
__StackBottom
__StackOneTop
__StackOneBottom

The 'Top's will always be 0x20041000/0x20042000 if you are using the standard linker script, but the Bottoms may vary if you've got variables allocated to scratch_x/scratch_y

For a test, it's probably easier to turn on the stack guards (PICO_USE_STACK_GUARDS=1) rather than trying to enlarge the stacks.

DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Wed Sep 20, 2023 4:47 pm

arg001 wrote:
Wed Sep 20, 2023 3:55 pm
DarkElvenAngel wrote:
Tue Sep 19, 2023 8:36 pm
Are the addresses of the stacks known?

You say default linker script sets the size, could I make the stacks bigger somehow? I don't think the stacks are too small but it could be something to try.
Look in the <projectname>.elf.map file generated in the build, and in particular at the symbols:

__StackTop
__StackBottom
__StackOneTop
__StackOneBottom

The 'Top's will always be 0x20041000/0x20042000 if you are using the standard linker script, but the Bottoms may vary if you've got variables allocated to scratch_x/scratch_y

For a test, it's probably easier to turn on the stack guards (PICO_USE_STACK_GUARDS=1) rather than trying to enlarge the stacks.
Okay that's made things worse now almost nothing works, so the stack on core 0 is too small? how do I make that larger?

arg001
Posts: 484
Joined: Tue Jan 23, 2018 10:06 am

Re: Troubles with Lua

Wed Sep 20, 2023 7:43 pm

DarkElvenAngel wrote:
Wed Sep 20, 2023 4:47 pm

Okay that's made things worse now almost nothing works, so the stack on core 0 is too small? how do I make that larger?
So you're now getting exceptions? And you are confident that it's core0 rather than core1?

Changing it is a bit fiddly, especially if this is your first exposure to linker scripts.

You can specify your own linkerscript via:

Code: Select all

pico_set_linker_script(my_project ${CMAKE_SOURCE_DIR}/my_script.ld);
and start by copying the one out of the pico SDK (src/rp2_common/pico_standard_link/memmap_default.ld)

The default arrangement has core0 stack at the very top (in scratch_y) and the core1 stack below it in scratch_x, so to increase the core0 stack you've got to move the core1 stack. I'd suggest to start by swapping them, then once you have core0 stack in scratch_x it can grow down into main RAM below it, and you just need to protect against the heap growing up towards it (where you say you have your own heap).

Code: Select all


    /* stack limit is poorly named, but historically is maximum heap ptr */
    __StackLimit = ORIGIN(RAM) + LENGTH(RAM);
    __StackOneTop = ORIGIN(SCRATCH_X) + LENGTH(SCRATCH_X);
    __StackTop = ORIGIN(SCRATCH_Y) + LENGTH(SCRATCH_Y);
    __StackOneBottom = __StackOneTop - SIZEOF(.stack1_dummy);
    __StackBottom = __StackTop - SIZEOF(.stack_dummy);
    PROVIDE(__stack = __StackTop);
So __StackLimit is the critical thing here, which is currently set to the end of main RAM (which is the same thing as the start of SCRATCH_X), and you just want to reduce that a bit.

I think that __StackBottom is only used for the stack guards, but the way it's set up with these .stack_dummy entries which are created from C arrays looks quite tricky to make it straddle the boundary between main RAM and SCRATCH_X.

DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Wed Sep 20, 2023 11:20 pm

arg001 wrote:
Wed Sep 20, 2023 7:43 pm
DarkElvenAngel wrote:
Wed Sep 20, 2023 4:47 pm

Okay that's made things worse now almost nothing works, so the stack on core 0 is too small? how do I make that larger?
So you're now getting exceptions? And you are confident that it's core0 rather than core1?

Changing it is a bit fiddly, especially if this is your first exposure to linker scripts.

You can specify your own linkerscript via:

Code: Select all

pico_set_linker_script(my_project ${CMAKE_SOURCE_DIR}/my_script.ld);
and start by copying the one out of the pico SDK (src/rp2_common/pico_standard_link/memmap_default.ld)

The default arrangement has core0 stack at the very top (in scratch_y) and the core1 stack below it in scratch_x, so to increase the core0 stack you've got to move the core1 stack. I'd suggest to start by swapping them, then once you have core0 stack in scratch_x it can grow down into main RAM below it, and you just need to protect against the heap growing up towards it (where you say you have your own heap).

Code: Select all


    /* stack limit is poorly named, but historically is maximum heap ptr */
    __StackLimit = ORIGIN(RAM) + LENGTH(RAM);
    __StackOneTop = ORIGIN(SCRATCH_X) + LENGTH(SCRATCH_X);
    __StackTop = ORIGIN(SCRATCH_Y) + LENGTH(SCRATCH_Y);
    __StackOneBottom = __StackOneTop - SIZEOF(.stack1_dummy);
    __StackBottom = __StackTop - SIZEOF(.stack_dummy);
    PROVIDE(__stack = __StackTop);
So __StackLimit is the critical thing here, which is currently set to the end of main RAM (which is the same thing as the start of SCRATCH_X), and you just want to reduce that a bit.

I think that __StackBottom is only used for the stack guards, but the way it's set up with these .stack_dummy entries which are created from C arrays looks quite tricky to make it straddle the boundary between main RAM and SCRATCH_X.
I can't see exception, I'm doing bad by assuming that could be the problem. Since core 1 is continuing to run I concluded that core 0 is having issues. This isn't the best troubleshooting.

I've found a pattern, reading data from the sdcard with Lua and writing to memory seems to be where the problem pokes up. If I reset the registers between then it works but I don't think the this is the issue. since each of these steps are calling scripts to run a new instance of Lua, this could have something to with things. I have run a long test of copying the 2k of data from flash to RAM 256 times and this was fine. But reading from SD card I get 3 times if I do a register reset between then nothing. It's very odd and maybe the stack isn't the issue.

The issue maybe the io lua library or fatfs. since everything runs in a memory pool and that's cleared on exit I'm not so sure...

The stack seemed logical since setting the (PICO_USE_STACK_GUARDS=1) cause all things to fall down. and when I tried to change the size it still was problems. Now that it's back things are working. This could be a house of cards and pulling one out has that effect.

I've left this bit alone and I'm working on other incomplete areas.

I need some way to verify what is going on with someone else that is more proficient at debugging on the Pico. The goal is to make the core as crash proof as possible or at least recoverable sometimes reset isn't enough.


DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Stack location

Mon Sep 25, 2023 1:19 am

This is a helpful article, I see that if core 0 is going over the stack limit it could destroy the stack on core 1. I'm going to have to see what I can do about this...

I've figured out how to get my linker script to be used. Now I've done as arg001 has suggested and swapped the scratch_x and scratch_y and success this works!

So how do I change the stack so it's 32K seems this was required in BBC Basic as well so no wonder I'm having these problems. But when I try the solution they proposed in the thread it was not working I'm guessing since I have core 1 in uses as well..

UPDATE:
I have an experimental branch set up now where I've updated Lua from 5.4.4 to 5.4.6 so far everything is working my concern is stack overflow is now masked behind the scenes and this will come back to bite me.

hippy
Posts: 15129
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Stack location

Mon Sep 25, 2023 11:14 am

DarkElvenAngel wrote:
Mon Sep 25, 2023 1:19 am
So how do I change the stack so it's 32K
Which stack, for Core 0 or for Core 1 ?

If you have swapped the stacks, put Core 1 Stack at the top of memory, Core 0 Stack below it, I can't see why Core 1 Stack used for video would need to be 32KB. And Core 0 Stack can grow as much as it needs to until it crashes into RAM used by the code so it doesn't seem to me that you have to do anything beyond swapping the stacks and removing the guard for Core 0 Stack.

If you want to limit Core 0 Stack to 32KB that would seem to me to simply be a matter of setting the guard for Core 0 Stack which is done at runtime; top of Core 0 Stack minus 32KB or something like that.

arg001
Posts: 484
Joined: Tue Jan 23, 2018 10:06 am

Re: Troubles with Lua

Mon Sep 25, 2023 11:31 am

Unfortunately, the SDK code for setting up the guards is a bit convoluted. The stacks are declared as arrays of words, and the size of those arrays only really matters for the purpose of setting up the guards, but it's not well parameterised. Part of the problem is that if you are declaring a stack as an array, then that array's got to be placed in a linker section - which is fine for scratch_x or scratch_y, but in this case we want it to overlap scratch_x and a bit of space in main RAM. It's all quite difficult to do in a generalised way - which is presumably why the SDK authors didn't do it. For something like this it's tempting to throw away the SDK implementation and do something special-purpose that's all in one place.

The other thing that needs to know the bottom of the stack is the heap code - you want heap allocations to fail in a sensible way rather than just allocating up until it collides with the stack.

IMO, having the stack guards enabled is highly valuable if end-user code can cause arbitrary amounts of stack to be used - which is probably the case for many language interpreters. The stack guards as implemented by the SDK have no runtime penalty (if you weren't using the MPU for something else), but you really want an exception handler to tell you that this was what happened, rather than just triggering a crash. Maybe if you are normally using SWD debugging then the exceptions are obvious, but I'm not normally doing that so I have one that spits out exception details on the serial port (I remember someone posted one here that's much better than mine - prints much more detail).

DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Mon Sep 25, 2023 1:56 pm

First off I'll answer the question about cores and what stack seemed to corrupt the other. In the default SDK Linker if I have this right the stack for core 0 if it overflowed would do so into core 1 and that would corrupt the video drive and cause unpredictable results. By swapping the the order of the stacks I've made it so core 0 can overflow into hopefully unused memory. (HEAP Space?) This could be bad if I was using dynamic memory allocations, since I'm not this is more like a time bomb that may never go off.

If I use the stack guards then yes I can confirm the core 0 overflows and I've tried to edit the linker which results in either the code not able to start at all or in one case the video output being corrupted. So I'm left with the time bomb that might be fine or not.

For example I dug through the linker and found these lines

Code: Select all

 MEMORY
{
    FLASH(rx) : ORIGIN = 0x10000000, LENGTH = 2048k
    RAM(rwx) : ORIGIN =  0x20000000, LENGTH = 256k
    SCRATCH_X(rwx) : ORIGIN = 0x20040000, LENGTH = 4k
    SCRATCH_Y(rwx) : ORIGIN = 0x20041000, LENGTH = 4k
}
And then add these lines later that I have flipped the stacks with.

Code: Select all

/* stack limit is poorly named, but historically is maximum heap ptr */
    __StackLimit = ORIGIN(RAM) + LENGTH(RAM);
    __StackOneTop = ORIGIN(SCRATCH_Y) + LENGTH(SCRATCH_Y);
    __StackTop = ORIGIN(SCRATCH_X) + LENGTH(SCRATCH_X);
    __StackOneBottom = __StackOneTop - SIZEOF(.stack1_dummy);
    __StackBottom = __StackTop - SIZEOF(.stack_dummy);
    PROVIDE(__stack = __StackTop);
You would think you could change something here to make the stack larger however that doesn't work. For example I found and tried this change. /viewtopic.php?p=1903641#p1904021

Code: Select all

    __StackLimit = ORIGIN(RAM) + LENGTH(RAM) - (24 * 1024);
I suspect that since the VGA driver is in use this requires more complex changes. For now things are working and the problem is covered by this lovely carpet.

arg001
Posts: 484
Joined: Tue Jan 23, 2018 10:06 am

Re: Troubles with Lua

Mon Sep 25, 2023 4:23 pm

DarkElvenAngel wrote:
Mon Sep 25, 2023 1:56 pm
First off I'll answer the question about cores and what stack seemed to corrupt the other. In the default SDK Linker if I have this right the stack for core 0 if it overflowed would do so into core 1 and that would corrupt the video drive and cause unpredictable results. By swapping the the order of the stacks I've made it so core 0 can overflow into hopefully unused memory. (HEAP Space?) This could be bad if I was using dynamic memory allocations, since I'm not this is more like a time bomb that may never go off.
That's my understanding too. I thought you'd previously said you had a custom heap implementation (so were still using heap, just not the SDK's implementation of it), in which case that wants to know where the bottom of stack is.
You would think you could change something here to make the stack larger however that doesn't work. For example I found and tried this change. /viewtopic.php?p=1903641#p1904021
Yes, that doesn't really work - it doesn't change the value of _StackOneBottom/_StackBottom which are still defined relative to the top of the stack, and taking the size from sizeof(.stack1.dummy) which is a linker section containing in fact only one object, which is defined in pico_multicore/multicore.c:

Code: Select all

static uint32_t __attribute__((section(".stack1"))) core1_stack[PICO_CORE1_STACK_SIZE / sizeof(uint32_t)];
And for core0 you've got the equivalent (but written in assembler rather than C!) in pico_standard_link/crt0.S:

Code: Select all

spacer_section .stack
// align to allow for memory protection (although this alignment is pretty much
ignored by linker script)
.p2align 5
    .equ StackSize, PICO_STACK_SIZE
.space StackSize
So in principle you can just change the symbol PICO_CORE1_STACK_SIZE and PICO_STACK_SIZE (bearing in mind the extra gotcha that PICO_STACK_SIZE is also used as the default for PICO_CORE1_STAC_SIZE).

Except that doesn't immediately work either, because those .stack sections are assumed to fit inside scratch_x/scratch_y after any other globals that have been explicitly placed in scratch_x/y. - and if you make them bigger it will just fail to link because it won't fit in scratch_x - you want it to sit across the boundary of normal RAM and scratch_x.

You could change PICO_STACK_SIZE and kludge the definition of SCRATCH_X at the top of the linker script so that SCRATCH_X includes part of the main RAM - but that's ugly.

I don't believe that those stack arrays are used for anything other than defining that sizeof(stack1_dummy), so If you know that you aren't using anything explicitly placed in scratch_x/y, you could just define __StackBottom directly (no longer based on sizeof(stack_dummy)) and it should just work.

But doing the job properly is a big mess, which is probably why nobody has sorted it out yet...

hippy
Posts: 15129
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Troubles with Lua

Mon Sep 25, 2023 4:41 pm

DarkElvenAngel wrote:
Mon Sep 25, 2023 1:56 pm
In the default SDK Linker if I have this right the stack for core 0 if it overflowed would do so into core 1 and that would corrupt the video drive and cause unpredictable results. By swapping the the order of the stacks I've made it so core 0 can overflow into hopefully unused memory.
Yes, that is correct. Swapping them should work and allow Core 0 stack unconstrained growth with guard turned off...

Code: Select all

#                 Default          Changed
#            .--------------.  .--------------.
# 0x20000000 |              |  |              | \
#            |              |  |     /|\      |  |-- RAM (256KB)
# 0x2003FFFF |              |  |      |       | /
#            |--------------|  |- - - | - - - |
# 0x20040000 |     /|\      |  |      |       | \
#            |      |       |  |      |       |  |-- Scratch X (4KB)
#            | Core 1 Stack |  | Core 0 Stack | /
#            |--------------|  |--------------|
# 0x20041000 |     /|\      |  |     /|\      | \
#            |      |       |  |      |       |  |-- Scratch Y (4KB)
#            | Core 0 Stack |  | Core 1 Stack | /
#            `--------------'  `--------------'
# 0x20042000
DarkElvenAngel wrote:
Mon Sep 25, 2023 1:56 pm
If I use the stack guards then yes I can confirm the core 0 overflows
I had missed that. Swapping the stacks and removing the guard should allow things to work as above.
DarkElvenAngel wrote:
Mon Sep 25, 2023 1:56 pm
then add these lines later that I have flipped the stacks with.

Code: Select all

/* stack limit is poorly named, but historically is maximum heap ptr */
    __StackLimit = ORIGIN(RAM) + LENGTH(RAM);
    __StackOneTop = ORIGIN(SCRATCH_Y) + LENGTH(SCRATCH_Y);
    __StackTop = ORIGIN(SCRATCH_X) + LENGTH(SCRATCH_X);
    __StackOneBottom = __StackOneTop - SIZEOF(.stack1_dummy);
    __StackBottom = __StackTop - SIZEOF(.stack_dummy);
    PROVIDE(__stack = __StackTop);
It's all a bit tricky given the multiple items, ordering and relationships between them - '.stack*', '.stack1*', '__Stack*', '__StackOne*', 'SCRATCH_X' and 'SCRATCH_Y'.

When I did it I recall I had to make four changes, and I only swapped SCRATCH_X and SCRATCH_Y ...

Code: Select all

.stack1_dummy (COPY): .... > SCRATCH_Y			/* was _X */
.stack_dummy (COPY): .... > SCRATCH_X			/* was _Y */
__StackOneTop = ORIGIN(SCRATCH_Y) + LENGTH(SCRATCH_Y);	/* was _X */
__StackTop = ORIGIN(SCRATCH_X) + LENGTH(SCRATCH_X);	/* was _Y */
The rest remained 'as was'. That seemed to work with how the Pico SDK uses things but I'll admit I can't remember the exact why and how. I'd have to sit down and re-do it all from first principles to figure it out. The key thing is how Pico SDK uses what it uses, then making sure that's what's in the link delivers what you want in reality, not as how it would have been by default.

I simply disabled the guards in my 'CMakeLists.txt', never got round to figuring out how to re-enable them as the RP2040 datasheet doesn't cover that in any meaningful way at all.

I took such stack swapping out in my MicroPython port because I suspected that was causing me problems with my use of doubles. Now I know it's not I am inclined to put it back in. I might well take that 'from first principles' approach to make sure I have it right.
Last edited by hippy on Mon Sep 25, 2023 5:57 pm, edited 1 time in total.

hippy
Posts: 15129
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Troubles with Lua

Mon Sep 25, 2023 5:46 pm

And here we have it ...

From 'multicore.c' we can see initialising Core 1 Stack involves using '__StackOneBottom'.

So we can presume '__StackOneBottom', '__StackOneTop', '.stack1_dummy' and 'stack1' relate to Core 1.

And that '__StackBottom', '__StackTop', '.stack_dummy' and 'stack' relate to Core 0.

Let's help get our brains in order, have a logical renaming ...

Code: Select all

SCRATCH_20040000	SCRATCH_X
SCRATCH_20041000	SCRATCH_Y

Code: Select all

Core0_Top		__StackTop
Core0_Bottom		__StackBottom
Core0_Scratch		SCRATCH_20040000
Core0_Dummy		stack_dummy
Core0			stack

Code: Select all

Core1_Top		__StackOneTop
Core1_Bottom		__StackOneBottom
Core1_Scratch		SCRATCH_20041000
Core1_Dummy		stack1_dummy
Core1			stack1
Using those invented definitions our linker script should contain ...

Code: Select all

    .Core0_Dummy (COPY):
    {
        *(.Core0*)
    } > Core0_Scratch

Code: Select all

    .Core1_Dummy (COPY):
    {
        *(.Core1*)
    } > Core1_Scratch

Code: Select all

    Core0_Top    = ORIGIN(Core0_Scratch) + LENGTH(Core0_Scratch);
    Core0_Bottom = Core0_Top - SIZEOF(.Core0_Dummy);

Code: Select all

    Core1_Top    = ORIGIN(Core1_Scratch) + LENGTH(Core1_Scratch);
    Core1_Bottom = Core1_Top - SIZEOF(.Core1_Dummy);
I found that a bit confusing because it defines the lower 'bottom' address in terms of the higher 'top' address minus its size. I would have defined the 'bottom' and then 'top' as 'bottom' plus size. It's the same at the end of the day. Why it's SIZEOF() rather than LENGTH() I don't know. I presume one may be bytes the other 32-bit words, haven't worried about it.

We can then do a first pass substitution back towards what it was ...

Code: Select all

    .stack_dummy (COPY):
    {
        *(.stack*)
    } > SCRATCH_20040000

Code: Select all

    .stack1_dummy (COPY):
    {
        *(.stack1*)
    } > SCRATCH_20041000

Code: Select all

    __StackTop       = ORIGIN(SCRATCH_20040000) + LENGTH(SCRATCH_20040000);
    __StackBottom    = __StackTop - SIZEOF(.stack_dummy);

Code: Select all

    __StackOneTop    = ORIGIN(SCRATCH_20041000) + LENGTH(SCRATCH_20041000);
    __StackOneBottom = __StackOneTop - SIZEOF(.stack1_dummy);
And finally we can replace 'SCRATCH_20040000' with 'SCRATCH_X' and 'SCRATCH_20041000' with 'SCRATCH_Y' ...

Code: Select all

    .stack_dummy (COPY):
    {
        *(.stack*)
    } > SCRATCH_X

Code: Select all

    .stack1_dummy (COPY):
    {
        *(.stack1*)
    } > SCRATCH_Y

Code: Select all

    __StackTop       = ORIGIN(SCRATCH_X) + LENGTH(SCRATCH_X);
    __StackBottom    = __StackTop - SIZEOF(.stack_dummy);

Code: Select all

    __StackOneTop    = ORIGIN(SCRATCH_Y) + LENGTH(SCRATCH_Y);
    __StackOneBottom = __StackOneTop - SIZEOF(.stack1_dummy);
And that makes for the four changes I thought it was.

The ordering of '.stack1_dummy' and '.stack_dummy' in the linker script is immaterial, they are used to determine size of the entity later in the script, not its location, the order of setting the 'Top' and 'Bottom' externals less so.

So that's it. Those are the changes needed. The existing stack initialisations of Core 0 and Core 1 will still work, just that they end up at different places, swapped.

Now, as to any stack sizes when a stack is initialised, I presume they don't count, are ignored, when guards are disabled for the build. No matter what size is specified lack of guards will allow the stacks to smash through their limits.

DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Mon Oct 02, 2023 12:08 am

Things are moving along now. My stack issues are back under control, I've upgraded successfully to the newest Lua release, and started the write a new fs library to fill in the gaps in the Lua standard libraries.

I've removed a bunch of this post because I missed that I put quotes around a define.... *Face Palm* yeah I sorted it out.

DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Mon Oct 02, 2023 11:42 am

My FS library is almost complete and now I'm looking at the list of functions. I was thinking I should include file copy and chmod since the Lua on the Pico cannot execute shell commands.

I've included the following "cd","mkdir","rmdir", "currentdir", "dir" what if anything looks like it's missing?

DarkElvenAngel
Posts: 2781
Joined: Tue Mar 20, 2018 9:53 pm

Re: Troubles with Lua

Tue Oct 03, 2023 1:03 am

Okay this is not what I would expect tell me what you think?

Code: Select all

if 0 then print (true) else print (false) end
The output is true

I found this odd behaviour I don't know what to make of this!?

Oh it's even documented as being this way I never saw this https://www.lua.org/pil/2.2.html
Beware that, unlike some other scripting languages, Lua considers both zero and the empty string as true in conditional tests.

Return to “SDK”