BBC BASIC on the Raspberry Pi Pico?

[RichardRussell]

Could you describe what mythread really does and ideas to get rid of it?

(continued from the Cobol in schools thread)

myThread is used to achieve a non-blocking console input mechanism that works on all platforms. BBC BASIC needs to be able to check for 'pending' console input without actually consuming any characters; this is relatively straightforward on Unix-style systems because it's possible to call select() on stdin, but there is no fully-functional equivalent in Windows. So a 'universal' solution is to read stdin in a separate thread, where it can safely block without stalling the interpreter.

If you need to "get rid of it" you could revert to the select() method, assuming it (or an equivalent) works on the Pico. That would of course kill Windows compatibility, but that may not be an issue if in any case you need to customise the existing code specifically for the Pico. But is it not possible to create new threads on the Pico? That seems a surprising limitation if so.

[ejolson]

Could you describe what mythread really does and ideas to get rid of it?

(continued from the Cobol in schools thread)

myThread is used to achieve a non-blocking console input mechanism that works on all platforms. BBC BASIC needs to be able to check for 'pending' console input without actually consuming any characters; this is relatively straightforward on Unix-style systems because it's possible to call select() on stdin, but there is no fully-functional equivalent in Windows. So a 'universal' solution is to read stdin in a separate thread, where it can safely block without stalling the interpreter.

If you need to "get rid of it" you could revert to the select() method, assuming it (or an equivalent) works on the Pico. That would of course kill Windows compatibility, but that may not be an issue if in any case you need to customise the existing code specifically for the Pico. But is it not possible to create new threads on the Pico? That seems a surprising limitation if so.

Thanks for creating this new topic to continue the discussion started at

viewtopic.php?p=1893614#p1893614

My understanding is the official Pico SDK is not an operating system, but only tools which could be used to create one. A port with just the SDK should be possible, but in that case one would need to write custom routines for console input and LittleFS.

There appear to be options such as

https://www.rt-thread.io/

which already support the Raspberry Pico with threads and Posix compatibility. This has the advantage of allowing BBC Basic to also run on other microcontrollers, but the disadvantage of having to learn one more thing and the possibility of bugs.

With the Pico I'm somewhat weary of the hardware divider across context switches and how to deal with the locking on writes to flash memory when using both cores. Hopefully the RT Thread developers have already worked out how this works. If anyone has familiarity with RT Thread, it would be nice to hear how well it works.

Does RT Thread use the faster floating point libraries from the official Pico SDK?

[RichardRussell]

My understanding is the official Pico SDK is not an operating system, but only tools which could be used to create one.

My understanding is limited to what it says here: "The SDK is designed to provide an API and programming environment that is familiar .. to non-embedded C developers" and "Standard C/C++ libraries are supported" which made it sound as though porting something like BBC BASIC should be relatively straightforward (since it relies only on 'standard C libraries'). But perhaps the devil is in the detail, and the support isn't as comprehensive as one might like.

[ejolson]

My understanding is the official Pico SDK is not an operating system, but only tools which could be used to create one.

My understanding is limited to what it says here: "The SDK is designed to provide an API and programming environment that is familiar .. to non-embedded C developers" and "Standard C/C++ libraries are supported" which made it sound as though porting something like BBC BASIC should be relatively straightforward (since it relies only on 'standard C libraries'). But perhaps the devil is in the detail, and the support isn't as comprehensive as one might like.

My impression is the Pico SDK provides about seventy percent of what's described as the C standard library in Appendix B of the second edition of The C Programming Language by Kernighan and Ritchie.

That appendix takes about 16 pages. It would be nice if there were document arranged in the same order that explained the differences in the SDK. Maybe there already is such a document. If so, does anyone know where might I find it?

[ejolson]

If you need to "get rid of it" you could revert to the select() method, assuming it (or an equivalent) works on the Pico.

I think I got rid of the threads with

Code: Select all

static unsigned char kbget (void)
{
    unsigned char ch = 0 ;
    char c=getchar_timeout_us(100);
    if(c!=PICO_ERROR_TIMEOUT) ch=c;
    return ch ;
}

Does that look like a plausible solution?

I presently get the output

Code: Select all

Welcome to minicom 2.7.1

OPTIONS: I18n 
Compiled on Jan  2 2019, 08:25:32.
Port /dev/ttyACM0, 17:10:46

Press CTRL-A Z for help on special keys

BBC BASIC for Pico Console v0.36
(C) Copyright R. T. Russell, 2021
Bad program

In an irritating sort of way messages printed to stderr on the Pico seem to vanish. Thus, I had to change the output from stderr to stdout to see the bad program error.

Note that I swapped out the complicated way userRAM was being allocated with

Code: Select all

    MaximumRAM = MINIMUM_RAM;
    userRAM = malloc(MaximumRAM);

Could that be the problem? Alternatively, maybe the interpreter is trying to interpret the buffer before any program has been loaded. I wonder how to fix that.

[DarkElvenAngel]

I'm very interested in this project I've been looking for a more feature rich version of BASIC to run on the pico in particular my PT52 terminal.

following with interest.

[ejolson]

I'm very interested in this project I've been looking for a more feature rich version of BASIC to run on the pico in particular my PT52 terminal.

following with interest.

I've updated the code that allocates memory as

Code: Select all

    MaximumRAM = MINIMUM_RAM;
    userRAM = malloc(MaximumRAM);
    if(userRAM) bzero(userRAM,MaximumRAM);

and the bad code error went away, however, the interpreter is still crashing in the routine that clears the variables at the beginning. I suspect something isn't getting initialized but haven't located where.

There is sometimes code of the form

Code: Select all

#ifdef _WIN32
// do Windows stuff
#endif
#ifdef __linux__
// do equivalent Linux stuff
#endif
#ifdef __APPLE__
// do equivalent Apple stuff
#endif

to which I've added a section

Code: Select all

#ifdef PICO
// do equivalent Pico stuff
#endif

However, it's possible I've missed a block somewhere.

The code for apicall_ looks astonishing at the moment, so that may be a problem too. If anyone is interested in looking at the currently broken code, please let me know and I'll post it.

[ejolson]

If anyone is interested in looking at the currently broken code, please let me know and I'll post it.

It would appear clear crashes at

Code: Select all

    link00 = 0 ;

which looks innocent, except for the fact this label is defined in bbdata_arm_32.s and that file might not be Pico friendly.

After commenting "link00=0" out I now get the BBC Basic '>' prompt but the interpreter immediately crashes after that. Any clues what to do next would be appreciated. I might need some help figuring out what that assembler file is supposed to do.

Another thing I'm wondering about is how important a functioning dlsym routine is for a working interpreter.

[RichardRussell]

I think I got rid of the threads with

Code: Select all

static unsigned char kbget (void)
{
    unsigned char ch = 0 ;
    char c=getchar_timeout_us(100);
    if(c!=PICO_ERROR_TIMEOUT) ch=c;
    return ch ;
}

Does that look like a plausible solution?

Yes, although do you need a non-zero timeout? Even 100μs could be a significant time if a program is calling INKEY(0) frequently,

Note that I swapped out the complicated way userRAM was being allocated with

Code: Select all

    MaximumRAM = MINIMUM_RAM;
    userRAM = malloc(MaximumRAM);

Could that be the problem?

The reason mmap is used in the original is that if the assembler is to be used (which I know it can't be at the moment) the allocated memory needs to be flagged as executable, which malloc'd memory usually isn't. mmap zero-fills which I don't think malloc is guaranteed to, so perhaps you need to add a memset

It would appear clear crashes at

Code: Select all

    link00 = 0 ;

A possible reason is that link00 is not aligned. I thought ARMv6 CPUs could cope with non-aligned memory but if not that particular write can be replaced with:

Code: Select all

memset(&link00, 0, 4);

But I can't guarantee you won't hit other alignment issues.

[RichardRussell]

After commenting "link00=0" out I now get the BBC Basic '>' prompt but the interpreter immediately crashes after that. Any clues what to do next would be appreciated.

Assuming you have a debugger, what is the crash reporting as the cause? If it's another alignment error you could be in trouble because I've never tried to run my BBC BASIC on a CPU that enforces strict alignment. If you have got --no_unaligned_access in your gcc flags take it out.

I might need some help figuring out what that assembler file is supposed to do.

Its purpose is to force internal interpreter variables to fixed addresses, because some BASIC programs assume they can access them using indirection. If it's assembling and linking correctly, there's not a problem specifically from that cause other than, potentially, alignment.

Another thing I'm wondering about is how important a functioning dlsym routine is for a working interpreter.

Not at all, it's only used by the SYS statement.

[Gavinmc42]

Looking forwards to seeing BBC Basic graphics examples on the Pico VGA/HDMI

[RichardRussell]

I've pushed a fix for the link00 alignment fault (only bbmain.c is affected). link00 is the only global variable that I know for sure is unaligned, but if you get more faults let me know and I will investigate. You definitely do not want the --no_unaligned_access flag; I added it to the RPi makefile because I couldn't be certain that gcc would realise that the ARMv7 can handle unaligned accesses, but it probably wasn't necessary and the other targets don't have it.

BBC BASIC makes extensive use of dynamic data structures, such as linked lists, which are typically not aligned. But since these are accessed via void* or char* pointers gcc should be aware that they may be unaligned and emit code appropriately. On most modern CPUs no special measures are necessary (unaligned accesses may suffer a performance hit, and won't be atomic, but will work) but, having checked, the Cortex-M0+ is ARMv6-M architecture which does enforce strict alignment, whereas the ARMv6 doesn't (or at least can be configured not to).

[RichardRussell]

Looking forwards to seeing BBC Basic graphics examples on the Pico VGA/HDMI

ejolson is attempting to port the Console Mode edition of BBC BASIC - no graphics or sound! That's not to say that, if successful, there might not be an attempt to incorporate them later but for the moment that's not the objective.

[jamesh]

Kilograham has a bit perfect BBC emulator running on the Pico. I don't think BBC BASCI should be too much of a stretch from that.

[RichardRussell]

Kilograham has a bit perfect BBC emulator running on the Pico. I don't think BBC BASCI should be too much of a stretch from that.

There's not much in common between a 6502 emulator and a BBC BASIC interpreter, certainly not enough to draw a conclusion about one from the other. Indeed even BBC BASIC interpreters differ significantly in their implementation; if one looks at (say) Sophie Wilson's ARM BASIC V, Dave Daniels' Brandy BASIC and my BBC BASIC for SDL 2.0 they're very different under the hood, and are likely to present different difficulties in porting, despite presenting an almost identical user interface.

[ejolson]

Kilograham has a bit perfect BBC emulator running on the Pico. I don't think BBC BASCI should be too much of a stretch from that.

There's not much in common between a 6502 emulator and a BBC BASIC interpreter, certainly not enough to draw a conclusion about one from the other.

One of the features that attracts me to Richard's interpreter, unlike Python, is that (if the port gets finished) it would include a built-in assembler that could be used to write performance critical code for GPIO.

While C is usually fast enough and if not also includes an inline assembler, that's a cross development environment requiring repeated uploads of executables not to mention a host with a cross compiler.

BBC Basic would be self contained. Thus, one would have more convenience than Python with assembler as needed for speed. The thing to keep in mind is that this project is not about emulation or retro computing but something modern and hopefully practical.

[ejolson]

I've pushed a fix for the link00 alignment fault (only bbmain.c is affected). link00 is the only global variable that I know for sure is unaligned, but if you get more faults let me know and I will investigate.

Unfortunately, the suggested fix doesn't work. More surprisingly

Code: Select all

        for(int i=0;i<4;i++){
            ((char *)&link00)[i]=0;
        }

also doesn't work, however

Code: Select all

        for(int i=0;i<4;i++){
            ((volatile char *)&link00)[i]=0;
        }

works just fine.

Now one gets to the prompt, but after that it still crashes. Is there a way to track down other alignment problems?

Could the gcc optimizer assume all global symbols are aligned. Is there a way to turn off that assumption? I might be stuck.

[RichardRussell]

Is there a way to track down other alignment problems?

Not really; as I said, my BBC BASIC makes extensive use of unaligned data structures (it was after all originally developed for the Z80 and then the x86, which do not care about alignment). The C code assumes that memory addressed by a void* or a char* pointer may be unaligned; if that's not the case for the toolchain on the Pico you are probably stuck.

By all means try reducing the optimisation level. It would be interesting to know if that helps, but even if it does it's probably not a practical solution because of the impact on overall performance.

Could the gcc optimizer assume all global symbols are aligned.

Probably, but link00 is the only such unaligned global variable AFAIK.

[jahboater]

Unfortunately, the suggested fix doesn't work. More surprisingly

Code: Select all

        for(int i=0;i<4;i++){
            ((char *)&link00)[i]=0;
        }

also doesn't work, however

Code: Select all

        for(int i=0;i<4;i++){
            ((volatile char *)&link00)[i]=0;
        }

works just fine.

Now one gets to the prompt, but after that it still crashes. Is there a way to track down other alignment problems?

It may be worth using memcpy() or memset() instead, its usually safer and just as fast.
Also this option might be of interest:

-Wcast-align=strict

-Wcast-align checks alignment for the target (and therefore never complains for many modern CPU's).
The =strict checks the pointer cast alignment for any platform.

[ejolson]

Unfortunately, the suggested fix doesn't work. More surprisingly

Code: Select all

        for(int i=0;i<4;i++){
            ((char *)&link00)[i]=0;
        }

also doesn't work, however

Code: Select all

        for(int i=0;i<4;i++){
            ((volatile char *)&link00)[i]=0;
        }

works just fine.

Now one gets to the prompt, but after that it still crashes. Is there a way to track down other alignment problems?

It may be worth using memcpy() or memset() instead, its usually safer and just as fast.
Also this option might be of interest:

-Wcast-align=strict

-Wcast-align checks alignment for the target (and therefore never complains for many modern CPU's).
The =strict checks the pointer cast alignment for any platform.

The proposed solution used memset and it also crashed. My guess is the optimizer recognized a call to the standard library and wondrously replaced it with a single instruction that tries to make an unaligned write. I haven't experimented, but my suspicion is the compiler is careful when writing through pointers that might not be aligned but in this case sees a global symbol and assumes it must be aligned.

Maybe there is a keyword that can be used to tell the compiler a certain global symbol is not aligned. Right now my plan is to look at the symbol map to see if there are any more odd addresses.

The last time I worked with a system that had such alignment requirements was the DEC Alpha. Does everything need to be aligned on 4-byte boundaries? Is it possible to trap the exception on the Pico when an unaligned access occurs?

I wonder if it is possible on a per process basis to set the ARMv6 core on the Pi Zero to signal unaligned memory accesses.

[jahboater]

The proposed solution used memset and it also crashed. My guess is the optimizer recognized a call to the standard library and wondrously replaced it with a single instruction that tries to make an unaligned write. I haven't experimented, but my suspicion is the compiler is careful when writing through pointers that might not be aligned but in this case sees a global symbol and assumes must be aligned.

Yes. memset and memcpy are now part of the ISO C standard, so the compiler is free to replace them with inline code that has the same effect.
And memxxx is guaranteed to work regardless of the alignment of the pointer(s) and also for random lengths. So if the compiler cannot prove the alignment, and the platform requires it, then it must split up the data accordingly or call the library function.

Maybe there is a keyword that can be used to tell the compiler a certain global symbol is not aligned. Right now my plan is to look at the symbol map to see if there are any more odd addresses.

There is all the alignment stuff in C99.
For example:

static _Alignas(4) xxx link00;

see stdalign.h for nicer names (like stdbool.h).

_Alignas(n) is more commonly used to force over-alignment, perhaps for SIMD access.
#define aligned _Alignas(16)

I don't know how to tell the compiler something is not aligned.
I haven't seen the code, but why cant link00 just be declared correctly (throughout) and then it will automatically be suitably aligned?

[jahboater]

In

Code: Select all

        for(int i=0;i<4;i++){
            ((char *)&link00)[i]=0;
        }

it may be that the compiler replaces the entire loop with a single STR instruction (because it thinks link00 is aligned).
Which explains why declaring it volatile works.

Horrible!

I am surprised that "memset(&link00, 0, 4)" doesn't work.
Some kind of UB?

[jahboater]

Perhaps there is something wrong with the declared target CPU?

Code: Select all

gcc -Q --help=target

Shows the defaults, see "-munaligned-access [enabled]" (or not) near the end.

See also __ARM_FEATURE_UNALIGNED

Code: Select all

-munaligned-access
-mno-unaligned-access
   Enables (or disables) reading and writing of 16- and 32- bit values from addresses
   that are not 16- or 32- bit aligned.  By default unaligned access is disabled for all
   pre-ARMv6, all ARMv6-M and for ARMv8-M Baseline architectures, and enabled for all
   other architectures.  If unaligned access is not enabled then words in packed data
   structures are accessed a byte at a time.

   The ARM attribute "Tag_CPU_unaligned_access" is set in the generated object file to
   either true or false, depending upon the setting of this option.  If unaligned access
   is enabled then the preprocessor symbol "__ARM_FEATURE_UNALIGNED" is also defined.

[RichardRussell]

I don't know how to tell the compiler something is not aligned.

I thought that, in early versions of C at least, one could use __attribute__ ((packed)) with simple variables as well as with structures.

But link00 is not the problem: the assignment can simply be omitted or a workaround using volatile used. The issue which I don't know how to solve is loading an int (etc.) indirectly from an unaligned pointer, which happens commonly in my code.

I assumed that here the compiler would treat p as (potentially) unaligned:

Code: Select all

char *p = some_external_function();
int n = *(int *)p;

In other words, I assumed that the (int *) cast was saying 'treat p as a pointer to an int' without additionally saying 'I promise that p is aligned'.

If that's not the case, is there no idiom in C which will load an int through an arbitrarily aligned pointer?

[ejolson]

Could the gcc optimizer assume all global symbols are aligned.

Probably, but link00 is the only such unaligned global variable AFAIK.

I'm not too worried about performance, because BBC Basic ideally has a built-in assembler for when performance of the interpreter is insufficient. I'm hoping the compiler will fix up memory accesses cast through pointers with less restrictive alignment characteristics, but that may not be the case.

Interestingly, the Z-code interpreter that I ported last week seems to have avoided the alignment problem due to the fact that the Z-machine is big endian so byte-level conversion routines were already in place.