MartinHa
Posts: 3
Joined: Tue Sep 07, 2021 2:51 pm

Docker macvlan not working

Tue Sep 07, 2021 3:01 pm

I tried to setup a Docker Container using macvlan network driver. But whatever I tried I cannot ping from inside the container to my network router not can I ping from my Laptop the Container hosted on my Pi4.

I setup Docker on a RPi4 8GB with Pi OS 32bit Lite.
I installed Docker via the script at https://get.docker.com
After that I created a macvlan:

Code: Select all

docker network create -d macvlan --subnet=10.0.1.199/24 --gateway=10.0.1.1 --ip-range 10.0.1.160/27 -o parent=wlan0 pub_net
I installed Portainer to get some UI (unclear to me why Portainer requires two steps to create a macvlan network)
Then I installed libseccomp2 via buster-backports as Alpine based Docker container do no longer work otherwise (this does not seem to apply to 64bit Pi OS)
I created an Alpine based container using the created macvlan network, it created correctly an instance having ip 10.0.1.160:

Code: Select all

/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
12: eth0@if3: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP 
    link/ether 02:42:0a:00:01:a0 brd ff:ff:ff:ff:ff:ff
    inet 10.0.1.160/24 brd 10.0.1.255 scope global eth0
       valid_lft forever preferred_lft forever
I tried creating the macvlan via Portainer and also tried using ipvlan but none of them works. I am also aware that by default the Container host cannot reach the containers in such a config, so I am always checking with my Router IP and my Laptop)

epoch1970
Posts: 7071
Joined: Thu May 05, 2016 9:33 am
Location: France

Re: Docker macvlan not working

Wed Sep 08, 2021 8:24 am

Not sure what “doesn’t work” actually means, but did you add “denyinterfaces” with the macvlan interfaces name to /etc/dhcpcd.conf under RaspiOS?
E.g. for veth interfaces commonly used by Docker, and assuming your macvlan interfaces are named mv-something, you want at the beginning of /etc/dhcpcd.conf:

Code: Select all

denyinterfaces veth* mv*
If you don’t exclude interfaces created and managed by Docker from the scope of dhcpcd, dhcpcd will try to configure them when they come up in the OS. An endless fight between Docker and dhcpcd ensues…

Edit. Also the M-DOWN interface flag, which somehow (?) differs from LOWERLAYERDOWN, seems to indicate wlan0 isn’t up. Macvlan creates (unauthenticated) MACs and that doesn’t work if the wlan interface is in client mode. Ipvlan should work, assuming wlan0 is running ok.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

MartinHa
Posts: 3
Joined: Tue Sep 07, 2021 2:51 pm

Re: Docker macvlan not working

Wed Sep 08, 2021 10:08 am

As described in the intro I cannot ping from container to router or from laptop to container :(

On the pi itself there seem to be no devices created by adding a macvlan to docker:

Code: Select all

ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether e4:5f:01:48:b9:89 brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether e4:5f:01:48:b9:8a brd ff:ff:ff:ff:ff:ff
    inet 10.0.1.199/24 brd 10.0.1.255 scope global dynamic noprefixroute wlan0
       valid_lft 787315sec preferred_lft 679315sec
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:04:34:84:b3 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
(I left some inet6 entries out...)

when creating a ipvlan I get a veth* and di-* interface on the Pi

Code: Select all

5: di-83dc6b2aafbf: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 
    link/ether 2e:ee:18:70:f6:27 brd ff:ff:ff:ff:ff:ff
    inet 169.254.248.78/16 brd 169.254.255.255 scope global noprefixroute di-83dc6b2aafbf
       valid_lft forever preferred_lft forever
    inet6 fe80::b947:8e98:6b15:cb1f/64 scope link 
       valid_lft forever preferred_lft forever
    inet6 fe80::2cee:18ff:fe70:f627/64 scope link 
       valid_lft forever preferred_lft forever
10: vethf6eff98@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether e2:ef:73:07:2f:13 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::e0ef:73ff:fe07:2f13/64 scope link 
       valid_lft forever preferred_lft forever
also tried adding vi* to denyinterfaces for now - ping is also not possible using ipvlan in any direction

Code: Select all

denyinterfaces veth* mv* di*
in that case the di-* interface is not assigned any private ip but still ping does not work

Code: Select all

5: di-83dc6b2aafbf: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 
    link/ether ee:8b:76:96:1b:ab brd ff:ff:ff:ff:ff:ff
    inet6 fe80::ec8b:76ff:fe96:1bab/64 scope link 
       valid_lft forever preferred_lft forever
7: vetha60f6ae@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default 
    link/ether 52:07:57:1b:0a:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::5007:57ff:fe1b:ac3/64 scope link 
       valid_lft forever preferred_lft forever

epoch1970
Posts: 7071
Joined: Thu May 05, 2016 9:33 am
Location: France

Re: Docker macvlan not working

Wed Sep 08, 2021 3:02 pm

Keep this line

Code: Select all

denyinterfaces veth* mv* di*
in dhcpcd.conf, it won't hurt.
I'd suggest you start by making a macvlan network work on top of eth0. Wireless is finicky as always. And also to leave portainer aside for a while.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

MartinHa
Posts: 3
Joined: Tue Sep 07, 2021 2:51 pm

Re: Docker macvlan not working

Fri Sep 24, 2021 12:21 pm

So indeed I found deep down in the docker github an entry stating that macvlan is not supported using a wifi host interface.
On macOS macvlan is not supported at all.
ipvlan should be supported with a wifi device, but somehow this did not work on Raspberry OS.

I switched to Pi to LAN now after moving it. This seems to work without any problems.

epoch1970
Posts: 7071
Joined: Thu May 05, 2016 9:33 am
Location: France

Re: Docker macvlan not working

Fri Sep 24, 2021 4:06 pm

MartinHa wrote:
Fri Sep 24, 2021 12:21 pm
ipvlan should be supported with a wifi device, but somehow this did not work on Raspberry OS.
Not quite true, but as you noticed it has limited use. This is on a Pi3B with the built-in interface. I just updated raspios for you ;)

Code: Select all

root@raspberrypi:/home/pi# uname -a
Linux raspberrypi 5.10.60-v7+ #1449 SMP Wed Aug 25 15:00:01 BST 2021 armv7l GNU/Linux

root@raspberrypi:/home/pi# wpa_cli -i wlan0 status
bssid=01:02:03:04:05:06
freq=2442
ssid=myssid
id=0
mode=station
pairwise_cipher=CCMP
group_cipher=CCMP
key_mgmt=WPA2-PSK
wpa_state=COMPLETED
ip_address=192.168.1.239
p2p_device_address=ba:27:eb:54:43:7a
address=b8:27:eb:54:43:7a
uuid=e8404a89-26b5-5775-a991-1adc60f61655

root@raspberrypi:/home/pi# ip link add link wlan0 name ipvl0 type ipvlan mode l2 bridge

root@raspberrypi:/home/pi# ip address show ipvl0
7: ipvl0@wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether b8:27:eb:54:43:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.239/24 brd 192.168.1.255 scope global dynamic noprefixroute ipvl0
       valid_lft 79031sec preferred_lft 69150sec
    inet6 fe80::4aa4:8f8f:2a56:47ed/64 scope link 
       valid_lft forever preferred_lft forever

root@raspberrypi:/home/pi# dhcpcd -k ipvl0
sending commands to master dhcpcd process

root@raspberrypi:/home/pi# ip address show ipvl0
7: ipvl0@wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether b8:27:eb:54:43:7a brd ff:ff:ff:ff:ff:ff

root@raspberrypi:/home/pi# ip address add 192.168.1.11/24 dev ipvl0 

root@raspberrypi:/home/pi# ip address show ipvl0
7: ipvl0@wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether b8:27:eb:54:43:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.11/24 scope global ipvl0
       valid_lft forever preferred_lft forever

root@raspberrypi:/home/pi# ip route add default via 192.168.1.1 dev ipvl0 metric 512

root@raspberrypi:/home/pi# ip route show default
default via 192.168.1.1 dev wlan0 proto dhcp src 192.168.1.239 metric 303 
default via 192.168.1.1 dev ipvl0 metric 512 

root@raspberrypi:/home/pi# ping -I wlan0 -c 1 8.8.8.8
PING 8.8.8.8 (8.8.8.8) from 192.168.1.239 wlan0: 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=114 time=59.8 ms

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 59.826/59.826/59.826/0.000 ms

root@raspberrypi:/home/pi# ping -I ipvl0 -c 1 8.8.8.8
PING 8.8.8.8 (8.8.8.8) from 192.168.1.11 ipvl0: 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=114 time=55.6 ms

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 55.628/55.628/55.628/0.000 ms
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

Return to “Networking and servers”