C language program reading UART loses characters

[Entropia]

Hi

I've coded a simple program using the tutorial found in here: http://www.raspberry-projects.com/pi/pr ... g-the-uart

My application is to receive serial data from a sensor board. The sensor board can be configured to any baud rate I wish. It's using ordinary 8 bit data frames with one stop bit and no parity bits. The problem that I am experiencing is that Raspberry Pi 2 Model B V1.1 keeps losing characters. It is usually the very first character that gets lost. I have noticed that the code from the link above returns arrays of characters, in my case, 7 characters at a time (why exactly 7?) out of packet that consists of maybe 40 characters. This means that every 8th character gets lost. Is there any way to ensure that all characters are received reliably? I have tried varying the baud rate but it does not make any difference.

If it makes any difference, I must say that the sensor board UART's high level is around 2.5 volts, but this is way above the input high level low threshold of the BCM SOC. I have verified with a logic analyzer that the sensor board does output all characters consistently, but the RasPi just loses some of them for some reason.

Any ideas? Thanks.

[rurwin]

First of all, have you done everything under the heading of "Turning off the UART functioning as a serial console" in the link you gave?

[joan]

...also post your code (in code quotes).

[ElEscalador]

Watching. Because although I found a workaround, I have a similar problem reading serial data from my Roomba when polling the sensors - I'd often receive only the first part or the last part of the data packets of 50ish characters. I don't know if it can help you and it's not the ideal solution for me but it works: Because I know the packet length and what some of the characters are, I basically have the Roomba send a constant stream to the Pi and throw away characters until the first few match, then I grab the whole thing and do one final check with known characters along the way and at the end - then I either accept or trash the whole packet. It's "fast enough" but I don't understand why I had to do it this way...and for a moving robot project - I don't think there such a thing as "too fast." Slower baud rates didn't help at all (and again, slower is not better for me), and I don't even have a voltage shortage. I end up trashing about 50% of packets for being incomplete.

[danjperron]

I did check the webpage you provide and I hope you do something to check what you received.

When you read a serial data ,

1- You should check for the first character. Often garbage could be in front.

2- Read until the end of character. Is it a terminating cariage return or a fix length packet?

3- Check the integrity of the data if possible. A checksum or a CRC.

The current webpage just gather the data and It is perfect to get and send the data. But I don't see anything there to check what you received and this as to be implemented into your code.

This is very important and you should always consider to do the check. This way if you get bad data it is easy to recuperate from it.

[Entropia]

Hi guys.

rurwin: I have disabled the serial console. I doubt I would even be able to open the serial port in the first place without doing so.

joan: My code is almost exactly as stated on the link I provided. Only modification is that I open the serial port in read only mode (no need to write since TXD is unconnected).Here it is nevertheless, for completeness.

Code: Select all

#include <stdio.h>
#include <unistd.h>                     //Used for UART
#include <fcntl.h>                      //Used for UART
#include <termios.h>                    //Used for UART


int main(void)
{
        printf("\nUART test\n\n");

        int uart0_filestream = -1;

        uart0_filestream = open("/dev/ttyAMA0", O_RDONLY | O_NOCTTY | O_NDELAY);                //Open in non blocking read/write mode
        if (uart0_filestream == -1)
        {
                printf("Error - Unable to open UART.  Ensure it is not in use by another application\n");
                return -1;
        }

        struct termios options;
        tcgetattr(uart0_filestream, &options);
        options.c_cflag = B9600 | CS8 | CLOCAL | CREAD;         //<Set baud rate
        options.c_iflag = IGNPAR;
        options.c_oflag = 0;
        options.c_lflag = 0;
        tcflush(uart0_filestream, TCIFLUSH);
        tcsetattr(uart0_filestream, TCSANOW, &options);

        printf("init complete, listening...\n\n");


        while(1)
        {

        if (uart0_filestream != -1)
        {
                // Read up to 255 characters from the port if they are there
                unsigned char rx_buffer[256];
                int rx_length = read(uart0_filestream, (void*)rx_buffer, 255);          //Filestream, buffer to store in, number of bytes to read (max)
                if (rx_length < 0)
                {
                        //An error occured (will occur if there are no bytes)
                }
                else if (rx_length == 0)
                {
                        //No data waiting
                }
                else
                {
                        //Bytes received
                        rx_buffer[rx_length] = '\0';
                        printf("%i bytes read : %s\n", rx_length, rx_buffer);
                }
        }

        }

}

ElEscalador: That's not possible in my case. Messages that I am to receive are of varying length and they don't repeat. I can't afford to lose a single character. The data is wireless telemetry from building management sensors.

danjperron: What do you mean by checking the first character? I know that the sensor board sends clean data and that there are no garbage before actual payload starts. The packet is of varying length but it is always terminated by a linefeed. However, it doesn't solve anything if I lose characters between start and end. Since I have also built the sensor board (it's a PIC18 MCU + a wireless radio) I could add a CRC to the message. However, all of your suggestions are merely bandaids. They don't solve the real problem which is that the Pi loses characters in reception. I understand that the webpage I linked only shows barebones UART reception and transmission. However, as I have already spotted a problem just by dumping characters from UART to terminal, there is no point in progressing further before I get the reception working reliably. I have no trouble programming the rest of the program. I am relatively new to Raspberry Pi but have years of professional experience in embedded engineering. That's why I have to turn to you guys, because I have no expertise on this Broadcom SOC.

Here is an example of a communication (please notice the indicated CRC error in the message is from the wireless side, not the UART):

The sensor board sends

Code: Select all

3 --> 4 CRC MISMATCH! (Batt Low) LAFL, PIR inactive LUX:1470 RSSI:9\n

And this is what I receive on the Pi

Code: Select all

3 --> 4CRC MISATCH! (att Low LAFL, IR inacive LUX1470 RSI:9\n

Thank you for replies so far!

[rurwin]

Post your code.

[Entropia]

Post your code.

My code is already in my earlier post. I compile it using GCC "gcc -o test test.c" and it produces and reproduces the error for me.

[Entropia]

I did another check with the logic analyzer.

Attached to this post is logic analyzer capture (digital with analyzer and analog) directly from the Rasp Pi GPIO pins.

screenshot.png (29.36 KiB) Viewed 10779 times

This is what the program spits out:

Code: Select all

7 bytes read :  --> 2
7 bytes read : KLUFL T
7 bytes read : :24.70
7 bytes read : H:0 LUX
7 bytes read : 430)  R
5 bytes read : SI:5

[rurwin]

You said that you've tried changing the baud rate, have you tried setting it as low as 300 baud? Does that affect the behaviour at all?


Let's see the output of "setserial -a /dev/ttyAMA0" as well.

[PeterO]

"cooked" vs "raw" on the device ?

Have you tried putting the device into raw mode with cfmakeraw() ?

PeterO

[RaTTuS]

H.... Only modification is that I open the serial port in read only mode (no need to write since TXD is unconnected).Here it is nevertheless, for completeness.
...!

^^ umm you need RX and TX connected to make serial connections .. it is not a one wire system

[Entropia]

rurwin: I just tried 300 baud. The output to terminal is visibly very slow but all the same problems still persist. It's definitely not a baud rate issue. There doesn't seem to be a program called setserial on my Raspbian install but I installed it via Aptitude.

Code: Select all

/dev/ttyAMA0, Line 0, UART: undefined, Port: 0x0000, IRQ: 83
        Baud_base: 187500, close_delay: 50, divisor: 0
        closing_wait: 3000
        Flags: spd_normal

PeterO: added cfmakeraw(&options); just before I enter the main while-loop in the program. No visible difference. I don't exactly understand what this does (I have practically zero experience with serial ports in UNIX/Linux systems).

RaTTuS: Did you bother to read my posts? I am not interesting in writing, just reading. One wire is enough (plus GND).

[PeterO]

H.... Only modification is that I open the serial port in read only mode (no need to write since TXD is unconnected).Here it is nevertheless, for completeness.
...!

^^ umm you need RX and TX connected to make serial connections .. it is not a one wire system

No. RX + Ground is ok for receive only.

PeterO

[RaTTuS]

r...

RaTTuS: Did you bother to read my posts? I am not interesting in writing, just reading. One wire is enough (plus GND).

I did - I have been assuming [with my previous usages of RPI] that simplex connections did not work - I could be wrong

[edit]
thanks PeterO

[PeterO]

OK about raw mode not helping
Have you tried using minicom to display the incoming characters ?
PeterO

[rurwin]

It wasn't baud rate issues per-se that I was wondering about, rather whether it was a buffering issue. You're still seeing the characters come in in clumps of eight with one missing, even at 300 baud? That doesn't sound right. At that speed they should definitely come in as individual characters.

I agree with Peter -- try minicom as a sanity check. If that can read all the characters then it must be possible.

Edit: The cfmakeraw must be before

Code: Select all

        tcsetattr(uart0_filestream, TCSANOW, &options);

tcgetattr gets the attributes of the serial driver, then you change then, then you write them back with tcsetattr.

ReEdit: Also see this: viewtopic.php?p=805096#p805096

[PeterO]

I agree with Peter -- try minicom as a sanity check. If that can read all the characters then it must be possible.

Yes, that was my reasoning, if it works then it eliminates faulty hardware as the cause.

Edit: The cfmakeraw must be before

Code: Select all

        tcsetattr(uart0_filestream, TCSANOW, &options);

Good point ! Here is the code I use (all be it on a USB device)

Code: Select all

bool openSerial(char *devName)
{
        int err;
        char *deviceName;
        
        deviceName = (devName == NULL) ? "/dev/ttyUSB0" : devName;
        
        KX3fd = open(deviceName,O_RDWR | O_NONBLOCK);
        if(KX3fd == -1)
        {
            printf("Failed to open KX3 device %s, %m\n",deviceName);
            return(false);
        }


        err = tcgetattr(KX3fd,&KX3termios);
        if(err != 0)
        {
                perror("tcgetattr failed:");
                return(false);
        }

        cfsetospeed(&KX3termios,B38400);
        KX3termios.c_cc[VMIN] = 0;
        KX3termios.c_cc[VTIME] = 0;

        cfmakeraw(&KX3termios);

        KX3termios.c_cflag &= ~CRTSCTS;
        tcsetattr(KX3fd,TCSANOW,&KX3termios);

        tcflush(KX3fd,TCIFLUSH);

        return(true);
}

[rurwin]

I think the finger is pointing at

Code: Select all

   options.c_cc[VMIN] = 0;
   options.c_cc[VTIME] = 0;

as the culprit. Either of those would explain the bunching into groups of eight. Although I still would not expect to lose any.


Edit: vvvv Yes. The solution. Their lack is the culprit.

[PeterO]

I think the finger is pointing at

Code: Select all

   options.c_cc[VMIN] = 0;
   options.c_cc[VTIME] = 0;

as the culprit. Either of those would explain the bunching into groups of eight. Although I still would not expect to lose any.

Culprit or Solution ? As far as I can see they are in my code (which works on USB devices) but not in Entropia's (which doesn't work on UART)
PeterO

[Entropia]

Thanks for the great amount of replies.

I installed minicom via apt-get but it doesn't seem to receive anything from ttyAMA0. I did turn off all handshaking so it can't be that... Next I tried screen /dev/ttyAMA0 9600 (for 9600 baud rate) but that doesn't show anything either. Very strange! (All this time I have the logic analyzer hooked up and it's triggering so the sensor board is definitely transmitting.)

I checked rurwin's and PeterO's messages and that one link to another thread on this forum and came up with this code:

Code: Select all

#include <stdio.h>
#include <unistd.h>                     //Used for UART
#include <fcntl.h>                      //Used for UART
#include <termios.h>                    //Used for UART


int main(void)
{
        printf("\nUART test\n\n");

        int uart0_filestream = -1;

        uart0_filestream = open("/dev/ttyAMA0", O_RDWR | O_NOCTTY | O_NDELAY);          //Open in non blocking read/write mode
        if (uart0_filestream == -1)
        {
                printf("Error - Unable to open UART.  Ensure it is not in use by another application\n");
                return -1;
        }

        struct termios options;
        //tcgetattr(uart0_filestream, &options);
        options.c_cflag = CS8 | CLOCAL | CREAD;         //<Set baud rate
        options.c_iflag = 0;
        options.c_oflag = 0;
        options.c_lflag = 0;
        options.c_cc[VMIN] = 0;
        options.c_cc[VTIME] = 0;
        //tcflush(uart0_filestream, TCIFLUSH);


        if (cfsetospeed(&options, B9600) || cfsetispeed(&options, B9600))
        {
                printf("erz\n");
                return 0;
        }

        cfmakeraw(&options);
        tcflush(uart0_filestream, TCIFLUSH);

        tcsetattr(uart0_filestream, TCSANOW, &options);


        printf("init complete, listening...\n\n");


        while(1)
        {

        if (uart0_filestream != -1)
        {
                // Read up to 255 characters from the port if they are there
                unsigned char rx_buffer[256];
                int rx_length = read(uart0_filestream, (void*)rx_buffer, 255);          //Filestream, buffer to store in, number of bytes to read (max)
                if (rx_length < 0)
                {
                        //An error occured (will occur if there are no bytes)
                }
                else if (rx_length == 0)
                {
                        //No data waiting
                }
                else
                {
                        //Bytes received
                        rx_buffer[rx_length] = '\0';
                        printf("%i bytes read : %s\n", rx_length, rx_buffer);
                }
        }

        }

}

But it still behaves exactly as before. No visible differences, still missing one character every seven bytes. (This was the behavior even at 300 baud.)

I'm having hard time believing the UART peripheral on the SOC is defective... I bet this is some sort of configuration problem.

[Entropia]

I also tried PeterO's snippet (had to modify slightly to compile).

Here is the code:

Code: Select all

#include <stdio.h>
#include <unistd.h>                     //Used for UART
#include <fcntl.h>                      //Used for UART
#include <termios.h>                    //Used for UART

struct termios KX3termios;
int KX3fd = -1;


int openSerial(char *devName)
{
        int err;
        char *deviceName;

        deviceName = (devName == NULL) ? "/dev/ttyUSB0" : devName;

        KX3fd = open(deviceName,O_RDWR | O_NONBLOCK);
        if(KX3fd == -1)
        {
            printf("Failed to open KX3 device %s, %m\n",deviceName);
            return 0;
        }


        err = tcgetattr(KX3fd,&KX3termios);
        if(err != 0)
        {
                printf("tcgetattr failed:");
                return 0;
        }

        cfsetospeed(&KX3termios,B9600);
        KX3termios.c_cc[VMIN] = 0;
        KX3termios.c_cc[VTIME] = 0;

        cfmakeraw(&KX3termios);

//        KX3termios.c_cflag &= ~CRTSCTS;
        tcsetattr(KX3fd,TCSANOW,&KX3termios);

        tcflush(KX3fd,TCIFLUSH);

        return 1;
}

int main(void)
{
        printf("\nUART test\n\n");

        openSerial("/dev/ttyAMA0");


        printf("init complete, listening...\n\n");


        while(1)
        {

        if (KX3fd != -1)
        {
                // Read up to 255 characters from the port if they are there
                unsigned char rx_buffer[256];
                int rx_length = read(KX3fd, (void*)rx_buffer, 255);             //Filestream, buffer to store in, number of bytes to read (max)
                if (rx_length < 0)
                {
                        //An error occured (will occur if there are no bytes)
                }
                else if (rx_length == 0)
                {
                        //No data waiting
                }
                else
                {
                        //Bytes received
                        rx_buffer[rx_length] = '\0';
                        printf("%i bytes read : %s\n", rx_length, rx_buffer);
                }
        }

        }

}

Edit: Had erroneous baud rate, once corrected we're back to square one. Every eight character is missing.

What a problem to have. I really don't want to add a UART-to-USB bridge if I already have a hardware UART to begin with...

[Entropia]

I just checked an FTDI FT232RL based USB device. Works as expected in minicom.

[PeterO]

My money is on there still being some other process trying to use the UART.
PeterO

[Entropia]

How can I check if any other process is accessing the port?