Detailed Description of the RHT03 / DHT22 / AM2302 sensors

[paulv]

The RHT03 / DHT22 / AM2302 sensors - with the SPI interface method

I was finally able to make the final decision on what driver to use for my DHT22 / AM2302 sensor.
After trying out a few solutions, I initially settled on the Adafruit one, but I didn't feel comfortable enough with it, due to the long time it takes to get a good and reliable reading. (you may have to read it a couple of times and average it out, adding to the time)
This is a problem with most solutions, because the DHT, when triggered, is blurting out it's data pretty fast, and you need to capture all of it. The problem of course with our Pi and Debian is that it's not a real-time system. The house-holding tasks of the OS can interrupt the data capture, and you may have to do it all over again.

Unless you go pretty deep into the kernel, and prioritize the reading task and/or disable interrupts for a while, this is more of a hit and miss proposition. The Adafruit driver is typical for this solution on a Pi and that means it's slow. You also need to wrap some good code around it, to avoid the pitfalls in our program. If you just want to have the temperature and relative humidity for kicks, that's probably OK, but if you want to do something more serious, you need to be aware of the inherent issues with these DHT devices.

A few months ago, I found a promising solution from one of the forum contributors, and that appealed to me. http://www.raspberrypi.org/forums/viewt ... 50#p506650
Daniel uses the SPI interface to capture the burst of data from the DHT, and this interface does not get interrupted by the OS. It acts like a logic analyzer or digital scope. Although the code worked fine, as soon as I embedded it in my application, I experienced OS crashes due to the number of open files. At the time, I could not figure it out and decided to use the Adafruit solution.

Last week, I wanted to try the SPI solution again, and to my dismay, had the same open file errors. This time I got some more help and together we were able to identify and correct the problem I was having. http://www.raspberrypi.org/forums/viewt ... on#p637420

My application is a web-based thermostat program that is pretty complex, mostly because I want it to be as fool- & fail-proof as possible. There are several posts on this Forum about the hardware I designed for this project.
At this moment I use a couple of DS18B20 sensors for the temperatures (inside, outside and attic), but wanted to add humidity for the inside. Because the temperature drives the HVAC system, I want it to be as robust and reliable as possible. Before I include any code that is not mine, I want to understand it, and test it myself before I start to use it.

This is also a great learning experience as well for me. Because there are so many posts in the various Forums on the Web, this one included, I also wanted to document my findings, so we can all share the collected "wisdom".

Let's start with the datasheet. There are many around, but this one is one of the best: http://www.dlnmh9ip6v2uc.cloudfront.net ... /RHT03.pdf
If you click on the link, it will download the PDF.

You'll probably want to read the document first, but here is a quick summary:
The DHT needs to be triggered before it will send the information.
This is how that works:

Code: Select all

Initialization:

					  Pi pulls up for           Sensor pulls up for
					     20-40 uSec              80 uSec to Ack
_____					      _____                ____________         ____
     |                   |     |              |            |       |
     |___________________|     |______________|            |_______|
								       Sensor pulls low			    50uSec	 Start data
	 Pi pulls Low for >1Ms		for 80 uSec to Ack                   transmission

After that, the DHT blurts out the 40 bits of data as follows:

Code: Select all

Bit transmission for all 40 bits: 

		   26-28 uSec for a "0"            70 uSec for a "1"
_____           _________            ________________________
     |         |         |          |                        |
     |_________|         |__________|                        |___________>>>
	 50uSec Low           50uSec Low                                    50uSec Low
	 to signal start 
	 of bit transmission
	 

If you are able to capture this burst of activity without any distortions or interruptions, you can now start to analyze it and convert the 40 bits to meaningful data. Sounds simple, but if you Google around, or search on this Forum, it's maybe not so easy after all.

Daniel (danjperron) decided to use the SPI bus, and really only uses the MISO and MOSI pins. This is how the ports need to connect to the DHT.:

schematic pic.jpg (35.64 KiB) Viewed 21183 times

Note the use of a diode to split the signal in a driving and receiving portion. Also note the pull-up resistor of 1K. In many documents, you will find a resistor of 4K7, and that is what I initially used as well. If you look at the signals with a scope, you will see that the 4K7 is not sufficient to overcome the capacitance on the pin to pull it high quick enough. Here is the signal with the 4K7:

4K7 pull-up pic.jpg (55.14 KiB) Viewed 21183 times

And here with a 1K resistor.

1K pull-up pic.jpg (51.82 KiB) Viewed 21183 times

A significant improvement, which will aid us in the capturing and decoding of the bits later on. In both screenshots, the A-trace (top) is the MISO signal, and the B-trace (bottom) is the MOSI signal. The MISO signal is what is coming from the DHT, and the MOSI is the "clock signal" we're sending to it.

Next we're going to examine the wake-up sequence, but because I can only upload 3 files, I'll continue in the next post.
Stay tuned!

[paulv]

Let's look at the wake-up sequence with a scope.

Wake-up pic.jpg (55.26 KiB) Viewed 21173 times

We see that the Pi pulls the bus low. The spec says for a minimum of 1 m Sec, in reality we see 1.8 m Sec. So far so good. Before we look at the data, let's take a look at the whole sequence.

Rd burst pic.jpg (58.64 KiB) Viewed 21173 times

At about 4 m Sec. after the wake-up, the DHT has already finished sending it's data, but the Pi is still listening. You can see that because the pulses go towards Ground when they are transmitted, but raise a little when done. So our code listens for another 5.5 m Sec and that gives us some room for improvement in the code.
Let's look at the wake-up sequence in more detail now in the following screenshot.

Start of burst pic.jpg (58.1 KiB) Viewed 21173 times

On the left-hand side, you see the end of the wake-up signal from the picture above. After the initial low of 1.8 m Sec, the Pi now needs to pull the bus high for 20-40 u Sec. This is the first pulse and if you look at the screenshots in the previous post (labeled 1K or 4K7), you'll see it's exactly 20u Sec. Don't let the 2 u Sec. "clock" pulses coming from the B-trace confuse you. The DHT then acknowledges the wake-up by pulling the bus low for 80 u Sec. Then the DHT pulls it high again for another 80 u Sec. The DHT is now ready to start sending the 40 bits of data, and it does that by lowering the bus for 50 u Sec followed by the data bit. The data bit can be a "0" or a "1". According to the specification, when it sends a "0", the data bit stays high for 26-28 u Sec, and for a "1", it stays high for 70 u Sec.
On this screenshot, we see the first 8 data bits. These bits are 0 0 0 0 0 0 0 1 for the first byte, which is the MSB for the two bytes that contain the relative humidity data.

I'm out of files again, so up to the next post.

[paulv]

Let us now look at one complete burst of the data coming out of the DHT.

Data Transfer pic2.jpg (60.95 KiB) Viewed 21155 times

I have edited this screenshot to make it easier to look at the 5 bytes of data, and already decoded the individual bits. (You'll probably have to right-click on the picture and then select View Image to see it better) We'll go in more detail later on, but let's have a quick look to satisfy our curiosity.
Byte[1] shows 00000001
Byte[2] shows 11010001
Byte[3] shows 00000000
Byte[4] shows 11110000
Byte[5] shows 11000010

Byte[1] and [2] hold the data for the relative humidity
Byte[2] and [3] hold the data for the temperature
Byte[5] holds the check sum.

The relative humidity is made-up of MSB Byte[1] and LSB Byte[2]
So, the value is 0000000111010001 which is 465 in decimal.
That value needs to be divided by 10, so the rel. hum. is 46,5%

The temperature is made-up of MSB Byte[3] and LSB Byte[4]
That value is 0000000011110000 which is 240 in decimal.
It also needs to be divided by 10, so the temp is 24.0 degrees Celsius.

The last Byte[5] is the check sum, and that is the addition of all 4 bytes.
so: 0000000 + 111010001 + 00000000 + 11110000 = 0111000010, of which we only use the 8 LSB bits, which gives 11000010, and that happens to match the check sum we got from the DHT.
Our decoding was accurate and we have a valid transmission to pass on.

Next, we will look at the source code in more detail to see how Daniel solved the driving of the DHT through the SPI bus and also how he implemented the decoding of the data. We will also discuss where there is room for improvement, and where the deviations from the specs are and if they really matter.

Stay tuned!

[danjperron]

Nicely done !

Daniel

[paulv]

So, we have now seen that the DHT will respond to our wake-up call and will blurt out the 40 bits that make up the relative humidity data, the temperature and a check sum. So far, this was all in real-time and in hardware.

How do we capture the data and make sense of what we're receiving.
As mentioned earlier, Daniel figured out how to use the SPI interface to take a snap-shot of the data coming in.

What is required in our Python program is that we import the driver to do so.
This driver needs to be installed first, because it is not part of the standard distribution. Here is what you need to do:

Code: Select all

To install spidev:
    sudo apt-get update
    sudo apt-get install python-setuptools -y
    sudo apt-get install python-dev -y
    sudo apt-get install git-core -y
    git clone git://github.com/doceme/py-spidev
    cd py-spidev/
    sudo python setup.py install (you can ignore the warnings)
    

To install the SPI kernel module, you need to do this in earlier kernel versions:

Code: Select all

To enable the SPI kernel module:
    sudo nano /etc/modprobe.d/raspi-blacklist.conf
    put # in front of blacklist spi-bcm2708
    sudo modprobe spi-bcm2708 or
    sudo reboot

In later versions (Jessie) you need to modify the /boot/config.txt file and make the spi driver active. You can do that with raspi-config, or manually by removing the '#' in front of this line to make it acive, like so:

Code: Select all

dtparam=spi=on

After you did that, save the file and reboot the Pi to load the driver.

In the Python program you can now import the SPI device driver with:

Code: Select all

import spidev

Daniel wrote a Class to handle the interface to the device driver, and to process the data. I will try to explain this bit by bit.

Code: Select all

class dht22spi():

    def __init__(self, DeviceType="DHT22"):

        self.DHT22Type = (DeviceType == "DHT22")
        self.spi = spidev.SpiDev()
        self.speed = 500000 # 500 KHz, 20 u Sec.
        self.spi.open(0,0)
        self.max_speed_hz=self.speed
        self.Bits = [128, 64, 32, 16, 8, 4, 2, 1]
        self.DHT22Reg = [0, 0, 0, 0, 0]

First off, he wrote the Class to handle more than just the DHT22, it can also handle the DHT11, which I will not cover. In order to synchronize the program with the data that we will receive, he establishes a time reference of 500 KHz, or 20 u Sec. periods. These periods are the basis for the analysis later on. He then sets up a byte structure with the power of 2, and the 5 DHT22 registers, or bytes, that it will blurt out.

The unexpected quirk with SPI is that to receive a byte, you need to send a byte. Daniel created an array of data that will serve both. To prepare that array, he fills it with "1" 's and "0" 's, and makes it as large as needed to cover the whole process.

Code: Select all

        # 10ms should be good enough to get a good snapshot with all data
        MaxBits = 0.010 * self.speed
        self.ArraySize = int(MaxBits / 8)
        self.TotalBits = self.ArraySize * 8
        self.D = bytearray(self.ArraySize)

        # set first 1.5 ms of the data to Low and the rest to High
        End = (0.0015 * self.speed) / 8

        for i in range(self.ArraySize):
            if i > End:
                self.D[i] = 255
            else:
                self.D[i] = 0

Next are two helper functions:

Code: Select all

    def GetBit(self,index):
        '''
        Process the 8 Bits in the 5 Bytes
        '''
        byteIndex = index / 8
        if (self.Data[byteIndex] &  self.Bits[index % 8]) == 0:
            return False
        return True

    def GetNextDelay(self):
        '''
        This function skips over the "zero" parts to find the "one" parts
        in the returned data stream
        '''
        if self.LastIndex  >= self.TotalBits:
            return 0
        Index = self.LastIndex
        LastBit = self.GetBit(Index)
        while Index < self.TotalBits:
            if not (self.GetBit(Index) == LastBit):
                Delta = Index - self.LastIndex
                Deltaus = Delta * 1000000 / self.speed
                self.LastIndex = Index
                return Deltaus
            Index += 1

And then the meat of the Class, the Read function:

Code: Select all

    def Read(self):
        '''
        This function reads the data stream from the SPI input and indexes over
        the captured data to find the data elements coming from the DHT22

        It overwrites the array that was created in the init section, and then
        locates the "ones" in the data stream.

        It returns three tuples : Temperature, Rel_Humidity and a boolean flag
        for the Checksum.
        '''
        self.LastIndex = 0

        self.spi.open(0,0)
        self.spi.max_speed_hz=self.speed
        self.Data = list(self.D)
        self.spi.xfer2(self.Data)
        self.spi.close()

#        print "Data = \n", self.Data

        # first find & skip over the wake-up section of 1,8 Msec
        self.GetNextDelay()
        # find & skip over the 20 usec. of the Host pull-up
        self.GetNextDelay()
        # find & skip over the 70 usec. of the Sensor pull-low
        self.GetNextDelay()
        # find & skip over the 70 usec. of the Sensor pull-high
        self.GetNextDelay()
        # we have arrived at the 5 bytes (40 bits) of data

        for Idx in range(5):
            self.DHT22Reg[Idx]=0
            for bitidx in range(8):
                # find & skip over the 54 mSec. of data start
                self.GetNextDelay()
                # read the period during which the bits are high
                # the specification says:
                # 26-28 usec for a "0", 70 usec for a "1"
                # our bits are 24 uSec for a "0" and 62 uSec for a "1"
                delai = self.GetNextDelay()
                if delai < 12: # too short, invalid data
                    return [None, None, False]

                if delai > 80: # too long, invalid data
                    return [None, None, False]
                # there is enough of a difference, so we can safely use 43 uSec
                # to differentiate between a "0" or a "1"
                if delai > 43: # bit is a "1"
                    self.DHT22Reg[Idx] |= self.Bits[bitidx]

        # Verify the check sum to see if we have valid data in the captured stream
        # Add the 4 data bytes, only use the resulting lower 8 as the check sum
        Checksum = 0;
        for Idx in range(4):
            Checksum += self.DHT22Reg[Idx]
        Checksum %= 256

        # if the received check sum is the same as we calculated, continue
        if Checksum == self.DHT22Reg[4]:
            # Add the High byte to the Low byte and divide by 10
            Temp = ((self.DHT22Reg[2] & 127) * 256  + self.DHT22Reg[3]) / 10.0
            # Temp can be negative
            if (self.DHT22Reg[2] & 128) == 128:
                Temp = -Temp
            # add the high byte to the low byte and divide by 10
            Humidity = (self.DHT22Reg[0] * 256 + self.DHT22Reg[1]) / 10.0
        else:
            return[None, None, False] # bad check sum
        print Temp, Humidity
        return[Temp, Humidity, True]

Without going into too much details, lets look at the data first, in the next post...

[paulv]

The data captured through the SPI interface and loaded into the array "self.Data", looks like this:

Code: Select all

[
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 
192, 0, 0, 0, 7, 255, 255, 255, 255, 0, 0, 0, 255, 
224, 0, 0, 63, 252, 0, 0, 7, 255, 0, 0, 0, 255, 224, 
0, 0, 63, 252, 0, 0, 7, 255, 0, 0, 0, 255, 224, 0, 
0, 63, 255, 255, 255, 19, 2, 0, 0, 0, 255, 255, 255, 
255, 0, 0, 1, 255, 255, 255, 254, 0, 0, 3, 255, 255, 
255, 252, 0, 0, 7, 255, 0, 0, 0, 255, 255, 255, 255, 
0, 0, 0, 255, 255, 255, 255, 0, 0, 1, 255, 255, 255, 
254, 0, 0, 3, 255, 255, 255, 252, 0, 0, 0, 63, 252, 
0, 0, 3, 255, 0, 0, 0, 255, 224, 0, 0, 31, 252, 0, 0, 
3, 255, 128, 0, 0, 255, 240, 0, 0, 31, 254, 0, 0, 3, 
255, 0, 0, 0, 3, 255, 255, 255, 252, 0, 0, 7, 255, 255, 
255, 248, 0, 0, 7, 255, 255, 255, 240, 0, 0, 15, 255, 
255, 255, 240, 0, 0, 31, 254, 0, 0, 3, 255, 255, 255, 
252, 0, 0, 7, 255, 0, 0, 0, 255, 224, 0, 0, 1, 255, 255, 
255, 254, 0, 0, 3, 255, 255, 255, 252, 0, 0, 3, 255, 
255, 255, 252, 0, 0, 7, 255, 0, 0, 0, 255, 224, 0, 
0, 63, 255, 255, 255, 192, 0, 0, 127, 248, 0, 0, 7, 
255, 0, 0, 15, 255, 255, 255, 255, 255, 255, 255, 255, 
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 
255, 255, 255, 255, 255, 255, ...
]

This is an array with a decimal representation of the burst of data coming from the DHT22. It looks very confusing and complicated, and it is at first sight, so how do we make heads or tail from this jumble...
Here is the start up diagram again

Code: Select all

Initialization:

					      Pi pulls up for        Sensor pulls up for
					         20-40 uSec           80 uSec to Ack
_____					      _____                ____________         __________
     |                   |     |              |            |       |
     |___________________|     |______________|            |_______|
								        Sensor pulls low			   50uSec	 Start data
	 Pi pulls Low for >1Ms		for 80 uSec to Ack                   transmission

The first burst of "0" 's represents the wake-up call from the Pi, with the 1.8 m Sec period, we can skip that.
Now we get to the period of the 20 u Sec Ack. This is represented by the 255 and the 192. These decimal numbers represent the bits in a high sample. (high samples have the data, low samples signal that data is coming next) So 255d represents 11111111b and 192d is 11000000b, together 10 high bits. These 10 bits represent a 20 u Sec period, or 2 u Sec per bit. This is exactly the Ack pulse coming from the Pi.
OK, next we have 3 "0" 's, representing the 80 u Sec. low period for the DHT to Ack the wake-up call. The DHT then responds by pulling the bus high for 80 u Sec. Let's check that. The next numbers are : 7d 255d 255d 255d and 255d. that means 3+8+8+8+8 bits = 35 high bits x 2 u Sec = 70 u Sec. Hmm, not quite 80, but this really does not matter.
OK, next we have 50 u Sec. of "0" 's to tell that data is coming next. That data is 255d and 244d or 8 + 3 = 11 bits x 2 uSec = 22 u Sec. This is not quite the 26-28 u Sec of the spec. for a "0", but again, it is not critical. The reason is that the "1" is more than 2 times longer (at 70 u Sec), and we can rather easily differentiate between a "0" and a "1", as longs as we know what to look for, and the data sample is valid.
Let's continue, we see a 63d and a 252d, meaning 00111111b+11111100b or 12 high bits, a "0" again. and so on...

It's a little difficult to see what is going on in this data blurt, so let's switch to one that I edited to create groups.

Code: Select all

[ Wake-up period of > 1 Msec (1.8Msec) We can skip this.
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,

This is more interesting. This represents the first byte of data, the MSB for the relative humidity.

Code: Select all

Ack: 0, 0, 255, 192,                             pulled high for 20 uSec
Ack: 0, 0, 0, 15, 255, 255, 255, 255,            pulled-high for 72 uSec
B 1: 0, 0, 1, 255, 192,                          1+8+2 = 11 bits x 2 uSec = 22 uSec -> "0"
B 2: 0, 0, 127, 248,                             7+5   = 12 bits -> "0" 
B 3: 0, 0, 15, 255,                              4+8   = 12 bits -> "0"
B 4: 0, 0, 1, 255, 192,                          1+8+2 = 13 bits -> "0" 
B 5: 0, 0, 63, 248,                              6+5   = 11 bits -> "0"
B 6: 0, 0, 7, 255,                               3+8   = 11 bits -> "0"
B 7: 0, 0, 0, 255, 192,                          8+2   = 10 bits -> "0"
B 8: 0, 0, 63, 255, 255, 255, 128,               6+8+8+8+1 = 31 bits x 2 uSec = 61 uSec -> "1"
Byte[1] : 00000001b

And here is the LSB byte with the decoding already done.

Code: Select all

B 9: 0, 0, 1, 255, 255, 255, 254, 
B10: 0, 0, 3, 255, 255, 255, 252, 
B11: 0, 0, 7, 255, 
B12: 0, 0, 0, 255, 255, 255, 255, 
B13: 0, 0, 0, 255, 255, 255, 255, 
B14: 0, 0, 1, 255, 255, 255, 254, 
B15: 0, 0, 3, 255, 128, 
B16: 0, 0, 255, 255, 255, 255, 
Byte[2] : 11011101b
MSB Byte[1] + [2] = 111011101b = 477d / 10d = 47.7% rel. hum

Here is the MSB for the temperature

Code: Select all

B17: 0, 0, 0, 3, 255, 
B18: 0, 0, 0, 255, 224, 
B19: 0, 0, 31, 254, 
B20: 0, 0, 3, 255, 128, 
B21: 0, 0, 255, 240, 
B22: 0, 0, 31, 254, 
B23: 0, 0, 3, 255, 128, 
B24: 0, 0, 255, 224,
Byte[3] : 00000000b

And the LSB and the decoding

Code: Select all

B25: 0, 0, 0, 255, 255, 255, 255, 
B26: 0, 0, 0, 255, 255, 255, 255, 
B27: 0, 0, 0, 255, 255, 255, 255, 
B28: 0, 0, 1, 255, 192, 
B29: 0, 0, 127, 255, 255, 255, 128, 
B30: 0, 0, 255, 255, 255, 255, 
B31: 0, 0, 0, 255, 224, 
B32: 0, 0, 31, 252,
Byte[4] : 11001010b 
Byte[3] + [4] = 11001010b = 236d / 10 = 23.6 degree C

And finally the fourth byte that contains the check sum as calculated by the DHT, and our own calculation, to see if the data transfer was without errors

Code: Select all

B33: 0, 0, 0, 127, 255, 255, 255, 128, 
B34: 0, 0, 127, 255, 255, 255, 128, 
B35: 0, 0, 255, 240, 
B36: 0, 0, 31, 254, 
B37: 0, 0, 3, 255, 255, 255, 252, 
B38: 0, 0, 7, 255, 
B39: 0, 0, 0, 255, 255, 255, 255, 
B40: 0, 0, 0, 255, 192,
Byte[5] : 11001010 Checksum == 00000001b+11011101b+00000000b+11001010b %= 256 is also 11001010

And finally, the transmission ended with the output going high.

Code: Select all

0, 1, ( output goes high, Transmission ended)
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 
255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 
255, 255, 255, 255, 255, ...
]

Next we will look at some of the details in the code, stay tuned.

[danjperron]

Wow!

Thank you Mr. teacher. This is well explained!


But I wouldn't put the close inside the Read() function unless you move the spi.open() also.

You are using the class once and trow it . But I do have code that I import the class in my main application and it is always available. Using the close() into the read will crash my application.


There is 2 ways to fix it.

1- Move

Code: Select all

self.spi.open(0,0)
self.spi.max_speed_hz=self.speed

into the read function


or

2- remove the close, create a close function and call it after the Read().



The first option is the best since it will work if you use the class as disposable or use it as main part of your application

[paulv]

Oops, I stand corrected and will update the code.
I experimented with both variations, and it was working, but I was not really using the Class as intended.
Thanks for pointing that out.
BTW, I'm not the teacher, just the scribe.
Paul

[paulv]

Let's continue with our data processing.
In the previous post, we saw how we can rather easily interpret the data that we received by pencil and paper, now that we know how to do it. The key here is that it will only work as long is that data burst is valid. The SPI method makes that almost a certainty, and it seems to be a lot better (and therefore faster) as compared to catching the data on the fly, and hoping the OS leaves that moment of about 5-6 m Sec undisturbed. More testing is required, but the theory behind it and the measurements look impressive.
In any case, once the data is captured by whatever method, you can process it using the various code examples that are available on the web for the Pi. There are examples in C and Python. I have shown a Python example, that wraps around the SPI driver.

Let's now go back to the measurements that I showed in one of the earlier posts.

Rd burst pic.jpg (58.64 KiB) Viewed 20961 times

As I mentioned earlier, this screenshot shows that we're collecting a 10 m Sec. sample of data, but that is really too much. We don't need much more after the data burst from the DHT has been received. We can easily fix that in the code for the Class. Instead of 10 m Sec., 6 is long enough to collect the burst, so we can fix that with:

Code: Select all

MaxBits = 0.006 * self.speed

We're getting good data out of the Class, and I have not seen any problems so far after several days of testing.
However, if we are using the results from the Class, we still need to account for a few error conditions that can happen, which is why I wrote a wrapper around the Class.

Code: Select all

def get_dht22():
    '''
    Function to read the temperature and humidity values from the DHT22.

    It uses the dht22spi class.
    '''
    try:
        if TRACE : print "get_dht22"
        i = 0
        crc = 0
        avg_tmp = 0.0
        avg_hum = 0.0

        while (i < 3):
            dht22 = dht22spi()
            value = dht22.Read()
            if value[2]:
                # there is a valid crc in the recording
                temp = value[0]
                avg_tmp = avg_tmp + temp

                hum = value[1]
                avg_hum = avg_hum + hum

                i += 1
            else:
                crc += 1
                write_log("warning", "Bad crc {0} from DHT22 sensor n={1}".format(crc, i))
                if DEBUG:
                    if i > 1 : # it seems we have the most crc's with the first reading
                        write_log ("error", "Bad crc from DHT22 sensor n={0}".format(i))
                # limit the number of tries
                if crc > 3:
                    break
                # continue the loop but skip the bad readings
            # A little time to get a stable reading from the device (2 sec. is minimum)
            sleep(3)

        # if there are no good temp readings, we might as well jump ship here.
        if (avg_tmp == 0.0):
            write_log ("error", "No good readings from the DHT22 sensor")
            return(current_temp, humidity) # global values

        # average and round the result to 1 decimal digit
        temp = round(avg_tmp/(i), 1)
        hum = round(avg_hum/(i), 1)

        return(temp, hum) 

    except Exception as e:
        e = format_exception(e)
        with open(exception_log, "a") as fout:
            fout.write(e)
        write_log("error", "Unexpected Exception in get_dht22() n={0} : \n{1}".format(i, e))
        subprocess.call(['logger "Unexpected Exception in get_dht22()"'], shell=True)
        mail_err_log()
        return(current_temp, humidity) # use the previous readings so we can continue

This code will take three readings from the DHT and average the results. I do this to filter out small anomalies from the individual readings from the DHT and I also handle bad check sums (CRC's) . The goal is to get a solid reading before I process it further and base the decisions on heating and cooling on the temperatures. I'd rather be safe than sorry.

While testing the code, I experimented with the time between readings to see what would happen. The specification is pretty adamant with a minimum of 2 seconds between readings, so in the code I use 3 seconds. When I dropped the time between readings below 2 seconds, I was getting check sum errors on the first reading, but the second one was often OK, and the third one almost always, until I decreased the time even more until I had no more good readings. I did not manage the "hang" the DHT however, something others have reported.

My system actually uses several temperature sensors, but obviously, the inside one is the most important. I have additional code in the above routine to check for readings that are +/- 10% different from the previous one, that was taken 30 seconds ago (my looping time). When I was using the Adafruit code for the DHT22, I noticed large drops in the values of the DHT readings. This may very well be due to the way that code was written, but I don't want to have my HVAC react on that.

In any case, this is the last installment of my post(s) to describe the DHT sensor with the SPI method in more detail, and I hope you got something out of it. Comments and suggestions are welcome here, but try to stick to this topic please, so we keep this post as a reference.

[paulv]

Here is an update to what looks like an anomaly in the reporting that I did.
Remember that the measured frequencies/periods that I showed on the scope screen where all a bit off the mark.
In a very early version of the last post, I wanted to compensate for the small difference by tweaking the SPI clock from 500 to 555 KHz to extend the pulse from 18 to 20 uSec. Well, after verifying, it didn't work, so I took the text out again. Only some early birds could have seen it.

Here is what I found since then.
Although the SPI clock was set to 500 Khz, I measured a pulse width of about 18 u sec. (positive 16.4, negative 2) Look at the second picture in the first post called 1K pull-up. It should have been 16.0 (8 bits x 2 uSec) + 2 uSec. ( see the explanation below for the +2 uSec.) During my further investigations and learning about the SPI interface, I wrote a little program to vary the frequency so I could see what the issue was.
The spidev specification http://www.tightdev.net/SpiDev_Doc.pdf didn't mention anything special about the frequency, so I looked further. Turns out, the code in the kernel driver (spi-bcm2708) handles the frequency setting a little "strange". It uses a rounded down multiplier of 2 to give you only some frequencies, and with a large granularity in the higher frequencies. Here is a table coming from a web page I found when trying to figure this all out. cdiv is the divisor

Code: Select all

 cdiv    speed
    2    125.0 MHz
    4     62.5 MHz
    8     31.2 MHz
   16     15.6 MHz
   32      7.8 MHz
   64      3.9 MHz
  128     1953 kHz
  256      976 kHz
  512      488 kHz
 1024      244 kHz
 2048      122 kHz
 4096       61 kHz
 8192     30.5 kHz
16384     15.2 kHz
32768     7629 Hz

Here is the web page http://elinux.org/index.php?title=RPi_SPI So it didn't matter what I set the frequency to, because anything between 488 and 975 MHz would still result in a 488 KHz clock.
Another forum contributor, notro, also found this quirk, and also noticed that the accuracy is not what you would expect. http://www.raspberrypi.org/forums/viewt ... 2&p=347073

So, something learned again, and I'm satisfied with what I've measured. In the DHT22 case, it does not matter much because, as I explained earlier, with the SPI solution, the timings from the DHT22 are not that critical, and especially the data stream of "ones" (26-28 uS) and "zeros" (70 uS) are more than a factor 2 "apart".

[joan]

To make this really useful you need to quantify the reliability of the readings. In particular I'd like to know statistics like the number of good reads versus no reads or bad checksums over a several day period.

[paulv]

After re-reading my posts, I realize that I possibly did not clearly explain the issue with the 2 micro Sec. pulses enough, so let me try to do that now.

To do this, I need to explain a bit more of the code that Daniel wrote. I use a piece of my testing code, which is not a Class, so it's a little easier to read:

Code: Select all

speed = 1000 * 500              # 500 KHz (in reality 488 KHz)
spi = spidev.SpiDev()
MaxBits = 0.006 * speed         # 3,000 Sample size for 6 mSec.(is 7 mSec)
ArraySize = int(MaxBits / 8)    # 375 Bytes
TotalBits = ArraySize * 8       # 3,000 Bits
D = bytearray(ArraySize)
End = int((0.0016 * speed) / 8) # Marker for 1.6 mSec of bytes

So what he did here was to set-up the parameters to create an array of 3,000 bits, or 375 bytes, which is a calculated duration of 6 mSec with a 500KHz clock. (in reality this is 7 mSec.)
Daniel now creates the array that will be sent over the MISO bus to the DHT22. Remember, and this is a bit strange, but important to understand; to receive a byte over the bus, you need to send a byte. The nifty trick here is that the first 100 bytes in the data we will send has zeroes that will wake-up the DHT22, and in the remainder, the DHT will send out it's data over the MOSI bus and it will get captured into the same array. So we send data to the DHT and it returns data back that we capture into the same array.

Code: Select all

for i in range(ArraySize):
if i > End:
    D[i] = 255 # every byte is 8 bits of 1
else:
    D[i] = 0

Data = list(D) # convert D into a list of 8-bit byte values [0, 0, ...]

So basically, the array is a list of bytes, with the first 100 Bytes with zeros, followed by the remaining 275 bytes of one's. This is what get's sent out:

Actually sent.jpg (34.56 KiB) Viewed 20838 times

A duration of 5 Milli-Seconds with one's. However, this is without the 1K pull-up resistor on the MISO bus!
If you add the effect of the pull-up resistor on the bus, the situation changes completely.

With pull-up on MISO.jpg (33.89 KiB) Viewed 20838 times

The MISO bus is now pulled-high by default, and when we send out out a zero bit, the bus gets pulled down. Now we can see the effect of the first 100 bytes. And that is indeed 2 mSec. So far so good. Now we can get to the mystery of the 2 uSec pulses. Let's zoom in on the details:

a byte of one's.jpg (41.1 KiB) Viewed 20838 times

What we see here is a view of the MISO bus (top trace) and the MOSI bus (bottom trace), but without the DHT22 present to distort the picture. So we're looking purely at the SPI activity on the two buses that are connected together through the diode. What we see here is the transmission of bytes going out, consisting of 8 bits with a one (255d), which takes 16.4 uSec (or 2 uSec. per bit). Remember that the MISO bus is pulled high, so when we send a 1, we actually don't do anything to that bus. So where does the 2 uSec low comes from? Well, we have created an array of bytes, and turned that into a list. The last section in the list looks like [..., 255, 255, 255, ...]. The 2 uSec "pulse" is actually the time Python needs in-between the sending of the individual bytes in the list, and is a result of that and the bus speed of 500MHz. This "pulse" is also on the MOSI bus as you can see. When the DHT is activated by the activity on the MISO bus, it will start to send it's data on the "floated" bus, which is in this state for 5 mSec. This is why you see the 2 uSec pulses transposed on the data coming from the DHT22. Luckily, it does not matter for our capturing and processing later on.

I hope this helps some more to understand this.

[ondrej1024]

Hi Paul,

did you ever try to test the maximum cable length? On the DHT22 datasheet it says that the sensor communication works over a distance of 20m but later on I found this statement:

"With 3.3V supply voltage, cable length shall not be greater than 100cm. Otherwise, the line voltage drop will lead to the sensor power supply, resulting in measurement error."

Since we are powering the DHT22 via the RPi 3.3V line, I guess that applies to our case. I have always used short cables (50cm) but it would be interesting to know if the limit is really just 1m or if we can get reliable readings also at greater distances.

Ondrej

[paulv]

Hi Ondrej,
No, I did not test out the maximum cable length for the DHT22. In my case the sensor is only inches away from my Pi. Just enough to eliminate the generated heat from the Pi itself in the readings. On the other hand, I have three DS18B20 sensors that are also powered by the 3V3. One reads the inside temperature and that is located about 8 inches away from the Pi. One is located in the attic, about 10 feet away from the Pi, and the last one is measuring the outside temperature, and that one is about 30 feet away from the Pi. All three sensors are on the same 1-Wire bus, and I use telephone cable to connect them. The attic one and the outside one have a 100nF from 3V3 to ground. The bus is protected at the Pi by Schottkey diodes from the data bus to ground and to 3V3. The pull-up is 4K7 and I have 100 Ohm in series in the data line to W-1, and a 56pF cap from the data line to ground to filter spikes. This setup has been working for more than a year now, without any issues. And I do keep track of the check sum errors and excessive re-reads.

Paul

[paulv]

Hi Joan,
Unfortunately, I missed your good question about the reliability from last Wednesday until just now. Sorry!
I have been using the SPI method in it's described form for more than a week now, and I have not seen a single check sum error.

In my thermostat program, I use three readings 3 seconds apart, and repeat that every 30 seconds. My very extended logging of the thermostat program while it is running on a test_rig (to test new code), tracks every individual reading of the DHT22. For 8 days that would work out at just shy of 70,000 readings without an error. I also have another Pi where I do my development on, and this is the one I shot the screenshots for this post with. When I kept to the minimum of 2 seconds between readings, I also there did not catch any check sum errors, but I did not keep records long enough that would add to the above test results.

In contrast, I used the Adafruit driver for just over 3 months, in the same setup. That driver does not pass on or show check sum errors, but I know that under the hood of the driver, there must have been a lot of retries, because the time it took was sometimes excessive (every now and then it took up to 19 seconds to read the 3 samples)
The SPI method takes a steady 9.19-9.27 seconds for the same three readings in a threaded environment. If you take the 2 x 3 seconds sleep time out between the 3 readings, than 3.1-3.2 seconds for 3 or just over 1 second per reading is quite good I think.

There also have not been any strange results in the SPI readings, as I experienced with the Adafruit driver, where the temperature or the humidity reading would drop more than 20% for several seconds before returning to normal. This was the main reason for me to re-investigate the SPI method again.

My thermostat program on the test_rig continues to run 24x7 and I collect a lot of data from it before I commit the code to the real thermostat. As soon as I see an error I will report back. Don't hold your breath though!

Paul

[joan]

Hi Joan,
Unfortunately, I missed your good question about the reliability from last Wednesday until just now. Sorry!
I have been using the SPI method in it's described form for more than a week now, and I have not seen a single check sum error.
...

Thanks. That is pretty much what I'd expect. It is always useful to have expectations supported by concrete evidence though!

[ondrej1024]

Hi guys,

Here is a question to you DHT experts

I tried to read the DHT sensor with the SPI method also on the AriettaG25 board:
http://www.acmesystems.it/arietta

Here I see a different behaviour in the startup sequence. To start the communication I am sending the sequence of zeros to the sensor, as usual. While on the RPi we observe that the sensor never responds before the RPi pulls the data line high on the MOSI pin, on the AriettaG25 I see the sensor responding already before I send ones on the MOSI line. Therefore I will loose some data if the initial zero period is too long.

Do you have an explanation for this? I have the diode between MISO and MOSI connected just like on the RPi.

Ondrej

[danjperron]

It is possible that the data line MOSI go to tristate between byte transfer. The way to get ridd if this is to add a D-flipflop.

MOSI on D
spi clock on the flipflop clock and use the Q flipflop output to the diode.

Daniel

[ondrej1024]

Hi Daniel, thanks for your thoughts on this. It sounds reasonable and I was also suspecting something similar. If the DHT sensor starts sending data, then the data line must have been pulled high somehow. So I guess this could very well be caused by the tristate between the 8bit data transfers. I wonder if this is actually hardware implementation specific or depends on the SPI driver. In the SPI section of the Atmel SAM9G25 (the uP used on the AriettaG25) datasheet it doesn't mention anything about what happens on the MOSI pin between data transfers.

Using the flipflop is a good idea but I would like to solve the issue without adding further HW. A workaround would be to find the correct length of the initial zero period. Sending exactly 8us of zeros seems to work most of the time but is really just a dirty hack.
Using 16bit data transfers will probably not make much of a difference, there will only be half of the tristate conditions but they are still there. Don't know if increasing the SPI speed would change anything. Maybe the tristate condition is shorter and will not be detected by the sensor? Need to do some tests.

Ondrej

[danjperron]

Well the other approach could be way simpler. Instead of using the MOSI, use a standard gpio pin on the diode instead.

1 - Setup the SPI.
2- Pull the GPIO PIN LOW.
3- wait 1ms.
4- Pull the GPIO High and start the SPI. You need to pull the gpio high just before you start the SPI transfer.

B.T.W. Does this SPI is used for other purpose in this board. Maybe this is why it doesn't work properly.

Daniel

[ondrej1024]

Great, I like this idea. I could use the same number of pins and no extra HW component. Just need to add handling the GPIO pin in the SW.
AFAIK, the SPI device I am using on the Arietta is not used for anything else and should be available for the programmers purpose. I really suspect the SPI driver for this platform to behave differently when performing the full duplex transfer with the SPI_IOC_MESSAGE control.
Ondrej

[tchiwam]

Here's another possibility for someone who need his SPI pins for other projects. I often get my 42 bits with this one and the 18us+80us are a good hint if the read will work or not.
clock_gettime

Sample rate is ~ : 0.096417us (10MSample/s) using GPIO
dt: 0.004147
samples:43011
I'm just starting with this but I think this might have potential

Code: Select all

 i=0;
  clock_gettime(CLOCK_REALTIME,&tn[n]);
  n++;
  while(n<85 && i<100000)
  {
    r=GET_GPIO(27);
    if(last != r)
    {
       clock_gettime(CLOCK_REALTIME,&tn[n]);
       state[n] = last;
       last = r;
       n++;
    }
    i++;
  }
  printf("\n N:%ld samples:%d \n",n,i);
  for(i=0;i<n;i++)
  {
    if(i>1)
    {
      if( state[i] !=0 )
      {
	printf("n:%03d %ld.%09ld %01ld ",i,(long)tn[i].tv_sec,(long)tn[i].tv_nsec, state[i]>>27);
	dt = (long)(tn[i].tv_nsec-tn[i-1].tv_nsec);
	printf("dt:%05ld ",dt);
	if(dt<55000)
	{
	  printf("0");
	}
	if(dt>55000)
	{
	  printf("1");
	}
	nbit++;
	printf("\n");
      }
    }
  }
  printf("nbit:%ld \n ", nbit);

Code: Select all

n:002 1422223218.775358000 1 dt:17000 0
n:004 1422223218.775518997 1 dt:80999 1
n:006 1422223218.775598996 1 dt:26000 0
n:008 1422223218.775678995 1 dt:26000 0
n:010 1422223218.775759994 1 dt:27000 0
n:012 1422223218.775839993 1 dt:27000 0
n:014 1422223218.775931991 1 dt:37999 0
n:016 1422223218.775999990 1 dt:25999 0
n:018 1422223218.776079989 1 dt:26000 0
n:020 1422223218.776158988 1 dt:25000 0
n:022 1422223218.776251987 1 dt:26000 0
n:024 1422223218.776380985 1 dt:74999 1
n:026 1422223218.776507983 1 dt:73999 1
n:028 1422223218.776634981 1 dt:73999 1
n:030 1422223218.776762979 1 dt:73999 1
n:032 1422223218.776890977 1 dt:73999 1
n:034 1422223218.777017976 1 dt:72999 1
n:036 1422223218.777144974 1 dt:72999 1
n:038 1422223218.777237972 1 dt:25999 0
n:040 1422223218.777317971 1 dt:25999 0
n:042 1422223218.777397970 1 dt:26000 0
n:044 1422223218.777477969 1 dt:26000 0
n:046 1422223218.777558968 1 dt:27000 0
n:048 1422223218.777638967 1 dt:27000 0
n:050 1422223218.777718965 1 dt:25999 0
n:052 1422223218.777797964 1 dt:24999 0
n:054 1422223218.777938962 1 dt:73999 1
n:056 1422223218.778065960 1 dt:72999 1
n:058 1422223218.778145959 1 dt:25999 0
n:060 1422223218.778225958 1 dt:26000 0
n:062 1422223218.778353956 1 dt:73999 1
n:064 1422223218.778433955 1 dt:26000 0
n:066 1422223218.778561953 1 dt:73999 1
n:068 1422223218.778640952 1 dt:25000 0
n:070 1422223218.778731951 1 dt:27000 0
n:072 1422223218.778858949 1 dt:72999 1
n:074 1422223218.778939948 1 dt:26000 0
n:076 1422223218.779019947 1 dt:27000 0
n:078 1422223218.779146945 1 dt:73999 1
n:080 1422223218.779226944 1 dt:26000 0
n:082 1422223218.779306942 1 dt:25999 0
n:084 1422223218.779433941 1 dt:72999 1
nbit:42 

$ printf "%d\n" 0x007F
127
$ printf "%d\n" 0x00CA
202
$ printf "%04X\n" $(( 0xCA + 0x7F ))
0149

[danjperron]

@tchiwam The only problem with your code, like the other one which uses user space, is when the cpu usage start to be high. you start to loose time slice and you start to loose timing.

You could increase the priority but if something need time it will steal your time slice and then nothing works.

The SPI at least doesn't care about time slice.

Try to run motion with your code and read the DHT22.

Daniel

[Tarcas]

Paul,
You mentioned that "All three sensors are on the same 1-Wire bus." I'm wondering how you connected them so that you can poll the temperature from each individually. Any way I can think to do that would trigger all three at once (or none at all, ever) and you'd get a mess when all three respond at once.

I ask because I'm considering doing something similar to you, placing several of these (DHT 11 and 22) sensors around my house. Instead of using them as a thermostat though (at least yet) I'd just like to start creating temperature graphs of the patterns in various areas over time (to answer questions like how hot and how cold does it get in my garage, attic, basement, various indoor areas, etc, how much does it fluctuate over the course of a day, and how does it compare with the temperature outside and nearby indoor areas?) Eventually I may incorporate it into my home automation system by doing things like turning on an attic fan automatically or opening various windows or blinds, but that's a discussion for another time and another post. Right now I'd just like to know how you use this system with more than a single sensor per Pi.

Thanks,
-Tarcas

[Tarcas]

Hi Paul,

did you ever try to test the maximum cable length? On the DHT22 datasheet it says that the sensor communication works over a distance of 20m but later on I found this statement:

"With 3.3V supply voltage, cable length shall not be greater than 100cm. Otherwise, the line voltage drop will lead to the sensor power supply, resulting in measurement error."

Since we are powering the DHT22 via the RPi 3.3V line, I guess that applies to our case. I have always used short cables (50cm) but it would be interesting to know if the limit is really just 1m or if we can get reliable readings also at greater distances.

Ondrej

Since the DHT sensors never actually pull the signal line high (only low or floating) you can actually power the sensor from 5v and use a pullup to 3.3v, and it works fine. I've done this with a DHT11 with no ill effects. This trick should let you use longer cables with sensors attached to the Pi with no trouble.