Stereomic
Posts: 9
Joined: Sun Sep 05, 2021 3:17 pm

Read power meter over usb

Sun Sep 05, 2021 8:36 pm

Hi all,

I have been experimenting with my pico in micropython for quite a while now, but I am afraid I got stuck and need some help on how to proceed.

My project:
I want to read data telegrams from the digital power meter installed by my electricity provider. It follows the DMSR 5.02 P1 standard https://www.netbeheernederland.nl/_uplo ... fce1f1.pdf, (chapter 5.7 onwards is relevant).
I also purchased an RJ11 to USB converter with a FTDI chip inside.
After installing a ftdi VCP driver on my laptop, I could perfectly read the incoming telegrams using putty with the correct baudrate and everything. So the meter and the USB converter work as expected.
Now I want to replace the laptop by the pico and I wrote some micropython code using eg “select.select” and the “sys.stdin.readline” . When I test my code by connecting the pico to my laptop and provide sample telegrams over the REPL, all works fine and the telegrams are parsed correctly.
However, connecting the pico to the power meter using the USB converter is not working: not a single line is read. I did use a separate 5v supply to vbus of the pico because the RJ11 of the usb converter only had 3 connections (gnd, data and data_req, but no power) and I connected all gnd’s.

My questions:
1) In my investigations I came across the infamous “usb host” functionality that is missing in micropython. Is that what is causing my problem? Is there a way to fix this without the need of c/c++?
2) Alternative is to get rid of the usb converter. Then I would need eg an optocoupler to bring the DMSR signal down to 3v and use a PIO state machine to handle communication. Could this work and what would decoding look like? (Once I have a string I know what to do)
3) other alternatives?

Thanks a lot for pointing me in the right direction!
Stereomic

fdufnews
Posts: 355
Joined: Fri Oct 07, 2011 5:37 pm

Re: Read power meter over usb

Mon Sep 06, 2021 6:46 am

I think you just need optocouplers in both direction to connect the power meter to the Pico. One on Data Request line the other on the Data line.
No need to use the PIO. You can use one of the UART.
The optocouplers shall be faster enough in order to transmit @ 115200 Baud.

ex_piro
Posts: 9
Joined: Sun Jul 18, 2021 10:32 am

Re: Read power meter over usb

Mon Sep 06, 2021 10:23 am

fdufnews wrote:
Mon Sep 06, 2021 6:46 am
I think you just need optocouplers in both direction to connect the power meter to the Pico. One on Data Request line the other on the Data line.
No need to use the PIO. You can use one of the UART.
The optocouplers shall be faster enough in order to transmit @ 115200 Baud.
According to the schematic in the PDF Stereomic provided, it looks like optocouplers and 5 volt power is included in the metering equipment on the RJ12 port.

Stereomic
Posts: 9
Joined: Sun Sep 05, 2021 3:17 pm

Re: Read power meter over usb

Mon Sep 06, 2021 10:58 am

ex_piro wrote:
Mon Sep 06, 2021 10:23 am
According to the schematic in the PDF Stereomic provided, it looks like optocouplers and 5 volt power is included in the metering equipment on the RJ12 port.
[/quote]

Correct, but the RJ11 to usb convrter doesn’t use it. In case I use UART I can indeed use the 5v from the meter to build the data request line and power the pico. However I think I still need additional optocouplers as the pico cannot send nor accept 5v I/O.

Stereomic
Posts: 9
Joined: Sun Sep 05, 2021 3:17 pm

Re: Read power meter over usb

Mon Sep 06, 2021 11:01 am

fdufnews wrote:
Mon Sep 06, 2021 6:46 am
I think you just need optocouplers in both direction to connect the power meter to the Pico. One on Data Request line the other on the Data line.
No need to use the PIO. You can use one of the UART.
The optocouplers shall be faster enough in order to transmit @ 115200 Baud.
Thanks a lot for your input fdufnews!
Could you please elaborate a bit on the UART part? Is it enough to simply provide it with the baudrate etc, like I did before in Putty? How to configure the UART so it is able to read the DMSR messages?
- I only define the rx and rts pin?
- “invert = data line” how does this syntax work, isit included in the pico port?
- is the data request line handled automatically when performing a “readline”?

Thanks again!
Stereomic

ame
Posts: 3732
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Read power meter over usb

Tue Sep 07, 2021 7:45 am

Looks pretty straightforward, having read the PDF.

You don't need the USB interface. You just need two GPIO lines on the Pico. And you can power the Pico with the 5V power output from the RJ11 socket (pins 1 & 6).

To get data out, you need to drive pin 2 (data request) high. The document says it needs 5V (min. 4V, max. 5.5V) but you might find that 3.3V from the Pico GPIO is sufficient to turn on the opto-isolator inside. If not you can hard-wire it to 5V for testing, and hook up a transistor buffer later. When pin 2 is high then the meter will send data on pin 5.

Pin 5 is an open-collector output, so you need to connect it to the GPIO on the Pico and turn on the pull-up for that pin, or add a pull-up resistor from the pin to 3.3V.

Then, if you set up a serial object with a baud rate of 115200 you should see data. You might have to invert the sense of the incoming bits. Or you might not.

It's all very clear from the PDF. Work through that and ask questions if something is not clear.

Stereomic
Posts: 9
Joined: Sun Sep 05, 2021 3:17 pm

Re: Read power meter over usb

Tue Sep 07, 2021 12:00 pm

ame wrote:
Tue Sep 07, 2021 7:45 am
Then, if you set up a serial object with a baud rate of 115200 you should see data. You might have to invert the sense of the incoming bits. Or you might not.
Thanks ame!
It might be straightforward to you, but I am still learning a lot here…
Can you please clarify the “serial object” you refer to? I was convinced the serial library is not available on the micropython port for the Pico…
Regards,
Stereomic

User avatar
B.Goode
Posts: 12983
Joined: Mon Sep 01, 2014 4:03 pm
Location: UK

Re: Read power meter over usb

Tue Sep 07, 2021 12:14 pm

Stereomic wrote:
Tue Sep 07, 2021 12:00 pm
ame wrote:
Tue Sep 07, 2021 7:45 am
Then, if you set up a serial object with a baud rate of 115200 you should see data. You might have to invert the sense of the incoming bits. Or you might not.
Thanks ame!
It might be straightforward to you, but I am still learning a lot here…
Can you please clarify the “serial object” you refer to? I was convinced the serial library is not available on the micropython port for the Pico…
Regards,
Stereomic


Maybe the RP2 implementation of MicroPython refers to it as UART?

https://docs.micropython.org/en/latest/ ... serial-bus

ame
Posts: 3732
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Read power meter over usb

Tue Sep 07, 2021 1:40 pm

Yup. The UART.

Although if it didn't exist you could bit-bang it in software.

I don't actually have a Pico, but the Pico is basically a microcontroller, so most microcontroller techniques can be applied.

When you hook up the meter then make sure the serial output (pin 5) is connected to one of the UART RX pins (as shown/described in the document linked by B.Goode).

hippy
Posts: 10745
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Read power meter over usb

Tue Sep 07, 2021 2:29 pm

Given the Pico only has to receive it could make sense to use a PIO bit-banged 'UART' receiver for that - That would be my preference.

Stereomic
Posts: 9
Joined: Sun Sep 05, 2021 3:17 pm

Re: Read power meter over usb

Wed Sep 08, 2021 8:42 am

hippy wrote:
Tue Sep 07, 2021 2:29 pm
Given the Pico only has to receive it could make sense to use a PIO bit-banged 'UART' receiver for that - That would be my preference.
Agree, that is what I meant with question 2 in my very first post.
I hope to find some time this weekend to work on that.
Will probably start from an example I found of a bit-banged UART and strip that down to receiving only.
Still a bit uncetain on how to decode the incoming bytes, so I might come back here with some more questions on that…

Thanks all!
Stereomic

fdufnews
Posts: 355
Joined: Fri Oct 07, 2011 5:37 pm

Re: Read power meter over usb

Wed Sep 08, 2021 12:33 pm

You need a UART.
The Pico has two UARTs.
Why would you want to do a UART with PIO when you have everything you need ready to use

dbrion06
Posts: 681
Joined: Tue May 28, 2019 11:57 am

Re: Read power meter over usb

Wed Sep 08, 2021 12:56 pm

seems they need an UAR (no transmission: pico "only" receives) and an ordinary pin to ask for data....

hippy
Posts: 10745
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Read power meter over usb

Wed Sep 08, 2021 1:51 pm

fdufnews wrote:
Wed Sep 08, 2021 12:33 pm
You need a UART.
The Pico has two UARTs.
Why would you want to do a UART with PIO when you have everything you need ready to use
It's easier or just as easy
It is more suited to the task
It is more appropriate to the task
Any GPIO pin can be used for receiving
It leaves the UART's available for other uses
There are known bugs with UART configuration - https://github.com/raspberrypi/pico-sdk/issues/548

For this particular project I don't see it would make much difference which one uses assuming those bugs don't bite.

User avatar
bensimmo
Posts: 5456
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: Read power meter over usb

Wed Sep 08, 2021 2:09 pm

Can't help with the code or pico part, but as a reference if it help in the future, this can be done with ESP8266 /esp32 type devices https://esphome.io/components/sensor/dsmr.html

And a wiring diagram somebody has here (3v3 gpio also) https://trmm.net/Smartmeter/

If it to get electricity and gas usage esphome is easily integrated into homeassistant for their energy (elec/gas/solar/individual dashboard and building the firmware).

Just there for reference and ideas.

Feel free to ignore.

Here in the UK we just have blinking leds nothing smart about our smart meters.
If you get the microPython working, it should be usable on the ESP too, assuming no PIO used and just plain hard or softserial UART

hippy
Posts: 10745
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Read power meter over usb

Wed Sep 08, 2021 2:38 pm

My proof of concept PIO UART code. Testing doesn't require any external hardware links, transmits and receives on the same GPIO pin -

Code: Select all

from   machine import Pin
from   rp2     import asm_pio, PIO, StateMachine
import time

TX_PIN = 0
RX_PIN = TX_PIN
BAUD   = 115200

@asm_pio(sideset_init = PIO.OUT_HIGH,
         out_init     = PIO.OUT_HIGH,
         out_shiftdir = PIO.SHIFT_RIGHT,
         fifo_join    = PIO.JOIN_TX)
def PioTx8N1():
    pull()            .side(1)  [7]
    set(x, 7)         .side(0)  [7]
    label("txloop")
    out(pins, 1)
    jmp(x_dec, "txloop")        [6]

@asm_pio(set_init     = PIO.IN_HIGH,
         in_shiftdir  = PIO.SHIFT_RIGHT,
         fifo_join    = PIO.JOIN_RX)
def PioRx8N1():
    wait( 0, pin, 0)
    set(x, 7)                   [10]
    label("rxloop")
    in_(pins, 1)
    jmp(x_dec, "rxloop")        [6]
    push()

# Need to initialise RX first if using same pin for TX

rxPin = Pin(RX_PIN, Pin.IN, Pin.PULL_UP)
rx = StateMachine(0, PioRx8N1, freq=BAUD * 8, in_base=rxPin)
rx.active(1)

txPin = Pin(TX_PIN, Pin.OUT, value=1)
tx = StateMachine(1, PioTx8N1, freq=BAUD * 8, sideset_base=txPin, out_base=txPin)
tx.active(1)

def Tx(n):
    print("TX  >  {:>02X}  {:>08b}".format(n, n))
    tx.put(n)
    
def Rx():
    n = rx.get() >> 24
    print("RX  <  {:>02X}  {:>08b}".format(n, n))
    return n

def AnyAvailable():
    return rx.rx_fifo()

# Flush any data caused by confguration glitches

time.sleep(0.1)
if AnyAvailable():
    print("Flushing\n")
    while AnyAvailable():
        discard = Rx()
        if not AnyAvailable():
            time.sleep(0.1)
    print("")
    
# Run loop-back test

print("Loop-back testing")
while True:
    for n in range(256):
        print("")
        Tx(n)
        discard = Rx()
        time.sleep(1)

Stereomic
Posts: 9
Joined: Sun Sep 05, 2021 3:17 pm

Re: Read power meter over usb

Fri Sep 17, 2021 8:48 am

Hi folks, finally found some time to work on this again...

Before putting all the effort in a SM solution, I quickly tried UART as some have been suggesting here.
1) I keep getting the error "bad RTS pin", both when I put it in or outside of the default UART0 and UART1 pin ranges. I also tried initializing the pin, same error.
2) apparently the "invert=" parameter of the UART expects an int value. I set it to 1, hoping this would reflect the RX line (and 0 would be TX)?

Why isn't all this stuff better documented?

Any help? If I lose too much time on this I probably will still go for the SM solution.

Thanks,
stereomic

ame
Posts: 3732
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Read power meter over usb

Fri Sep 17, 2021 9:36 am

Can you specify that you don't need RTS? Or assign it to an unused pin?

invert probably inverts the sense of Tx and Rx, but you are Rx only, so it won't matter.

ame
Posts: 3732
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Read power meter over usb

Fri Sep 17, 2021 9:39 am

What is the SM solution?

Also, you could do this on an Arduino if the Pico doesn't cut it.

hippy
Posts: 10745
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Read power meter over usb

Fri Sep 17, 2021 11:49 am

Stereomic wrote:
Fri Sep 17, 2021 8:48 am
Why isn't all this stuff better documented?
"You get what you pay for"

Raspberry Pi don't consider MicroPython anything to do with them and the MicroPython team has very limited resources.

You could consider sponsoring MicroPython or even try to convince Raspberry Pi they should be spending some of their millions of pounds of profit on supporting MicroPython considering it is helping them generate those profits - https://github.com/sponsors/micropython

Stereomic
Posts: 9
Joined: Sun Sep 05, 2021 3:17 pm

Re: Read power meter over usb

Fri Sep 17, 2021 11:59 am

ame wrote:
Fri Sep 17, 2021 9:39 am
What is the SM solution?

Also, you could do this on an Arduino if the Pico doesn't cut it.
SM = state machine = pio

Yes arduino might simplify because all 5v. But I have rpi pico and no arduino...

Stereomic
Posts: 9
Joined: Sun Sep 05, 2021 3:17 pm

Re: Read power meter over usb

Fri Sep 17, 2021 12:07 pm

hippy wrote:
Fri Sep 17, 2021 11:49 am
Stereomic wrote:
Fri Sep 17, 2021 8:48 am
Why isn't all this stuff better documented?
"You get what you pay for"
Fair point. Indeed sad that rpi doesn't really support them...

After trial and error I found rts must be on pin 3. Can I add this to the 'ReadTheDocs' somehow?

No news on the invert yet, still waiting for my optocouplers and transistors to ship in. The 3v proved too low to trigger the rts at the meter end ...

Stereomic

hippy
Posts: 10745
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Read power meter over usb

Fri Sep 17, 2021 12:22 pm

Stereomic wrote:
Fri Sep 17, 2021 8:48 am
1) I keep getting the error "bad RTS pin", both when I put it in or outside of the default UART0 and UART1 pin ranges. I also tried initializing the pin, same error.
If this is the "Send me some data" signal I am not convinced it needs to use an actual UART RTS signal, and it would possibly be better if it did not.

As to why you cannot specify the pin I do not know. Perhaps post the code you have so others can try it, replicate the issue, identify what the problem may be, get things fixed if there is a bug.
Stereomic wrote:
Fri Sep 17, 2021 8:48 am
2) apparently the "invert=" parameter of the UART expects an int value. I set it to 1, hoping this would reflect the RX line (and 0 would be TX)?
From the MicroPython source code, 'machine_uart.c' -

Code: Select all

#define UART_INVERT_TX (1)
#define UART_INVERT_RX (2)
#define UART_INVERT_MASK (UART_INVERT_TX | UART_INVERT_RX)

STATIC const char *_invert_name[] = {"None", "INV_TX", "INV_RX", "INV_TX|INV_RX"};
My interpretation of that is -

Code: Select all

.-----.---------------.---------------.
|  0  |  TX Normal    |  RX Normal    |
|  1  |  TX Inverted  |  RX Normal    |
|  2  |  TX Normal    |  RX Inverted  |
|  3  |  TX Inverted  |  RX Inverted  |
`-----^---------------^---------------'
But I am not sure you would need any inversion. - Cross-posted with your mention of opto which probably does invert so would need signal inversion. I would use "3" as that guarantees inversion of RX, and it doesn't matter if TX gets inverted or not as it's not used. Once working you can change it to "2" if you feel like it.

But if you are wanting to invert the UART RTS signal to the meter then "invert" isn't what you want. And more reason not to use the UART RTS for that.

And, if 3V is too low to trigger the meter, then how is adding an opto going to create more than 3V at the meter ?

ame
Posts: 3732
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Read power meter over usb

Fri Sep 17, 2021 9:19 pm

Stereomic wrote:
Fri Sep 17, 2021 8:48 am
Hi folks, finally found some time to work on this again...

Before putting all the effort in a SM solution, I quickly tried UART as some have been suggesting here.
1) I keep getting the error "bad RTS pin", both when I put it in or outside of the default UART0 and UART1 pin ranges. I also tried initializing the pin, same error.
2) apparently the "invert=" parameter of the UART expects an int value. I set it to 1, hoping this would reflect the RX line (and 0 would be TX)?

Why isn't all this stuff better documented?

Any help? If I lose too much time on this I probably will still go for the SM solution.

Thanks,
stereomic
Apparently there is a parameter flow which specifies the flow control options, and can be set to zero. So, try that?

Mentioned here.
https://forum.micropython.org/viewtopic.php?t=1669

Also, if 3.3V is not sufficient to turn on the meter output (which was indeed stated in the documentation, but worth a try) just connect the meter pin directly to 5V. It will be on all the time then, spewing data everywhere, which is what you need for testing. You can add a transistor or something later if you want to be able to control the 5V signal.

hippy
Posts: 10745
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Read power meter over usb

Sat Sep 18, 2021 11:14 am

ame wrote:
Fri Sep 17, 2021 9:19 pm
If 3.3V is not sufficient to turn on the meter output (which was indeed stated in the documentation, but worth a try) just connect the meter pin directly to 5V. It will be on all the time then, spewing data everywhere, which is what you need for testing. You can add a transistor or something later if you want to be able to control the 5V signal.
I would concur with that. Given data is only sent every second and it's pretty easy to determine data framing I wouldn't even worry about getting continuous data would keep the request signal tied to 5V unless it actually proves to be problematic.

Return to “MicroPython”