Bug#875733: lxc.mount.auto = cgroup:mixed doesn't seem to work in Stretch anymore

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug#875733: lxc.mount.auto = cgroup:mixed doesn't seem to work in Stretch anymore

Yves-Alexis Perez-2
Package: lxc
Version: 1:2.0.7-2
Severity: normal

Hi,

In Jessie I was using a container setup with LXC and unprivileged
containers. By unprivileged, I mean container config had a bunch of
lxc.cap.drop lines, especially including sys_admin.

That means the init system inside the container (systemd) is not able to
do any privileged operation, including mounts, so the mounts need to be
done before starting the containers. It worked fine in Jessie (both host
and guests) with lines suchs as:

auto = proc:mixed sys:ro cgroup:mixed

Which takes care of mounting /proc, /sys and /sys/fs/cgroup for the
container.

Now in Stretch with lxc 2.0.7-2, it doesn't work anymore. Console output
for a Jessie container shows:

Failed to mount tmpfs at /sys/fs/cgroup: Operation not permitted

While for a Stretch container I have:

Failed to mount tmpfs at /sys/fs/cgroup: Operation not permitted
Failed to mount cgroup at /sys/fs/cgroup/systemd: No such file or
directory
[!!!!!!] Failed to mount API filesystems, freezing.
Freezing execution.

So it looks like systemd is trying to mount /sys/fs/cgroup and fails
(because it doesn't have CAP_SYS_ADMIN, which is expected). That means
lxc somehow failed to mount /sys/fs/cgroup in the container, which looks
like a regression from Jessie.

I'll setup a more simple container and config so I can provide it and
some logs to you so you can reproduce.

Regards,
--
Yves-Alexis

-- System Information:
Debian Release: 9.1
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.9.0-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=fr_FR.utf8, LC_CTYPE=fr_FR.utf8 (charmap=UTF-8), LANGUAGE=fr_FR.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages lxc depends on:
ii  init-system-helpers  1.48
ii  libapparmor1         2.11.0-3
ii  libc6                2.24-11+deb9u1
ii  libcap2              1:2.25-1
ii  libgnutls30          3.5.8-5+deb9u2
ii  liblxc1              1:2.0.7-2
ii  libseccomp2          2.3.1-2.1
ii  libselinux1          2.6-3+b1
ii  lsb-base             9.20161125
ii  python3              3.5.3-1
ii  python3-lxc          1:2.0.7-2

Versions of packages lxc recommends:
pn  bridge-utils  <none>
ii  debootstrap   1.0.89
ii  dirmngr       2.1.18-6
pn  dnsmasq-base  <none>
ii  gnupg         2.1.18-6
ii  iptables      1.6.0+snapshot20161117-6
pn  libpam-cgfs   <none>
pn  lxcfs         <none>
ii  openssl       1.1.0f-3
ii  rsync         3.1.2-1
pn  uidmap        <none>

Versions of packages lxc suggests:
pn  apparmor     <none>
pn  btrfs-tools  <none>
ii  lvm2         2.02.168-2

-- debconf information:
* lxc/directory: /srv/lxc
  lxc/shutdown: /usr/bin/lxc-halt
  lxc/title:
  lxc/auto:

Reply | Threaded
Open this post in threaded view
|

Bug#875733: lxc.mount.auto = cgroup:mixed doesn't seem to work in Stretch anymore

Yves-Alexis Perez-2
On Thu, 2017-09-14 at 09:23 +0200, Yves-Alexis Perez wrote:
> Package: lxc
> Version: 1:2.0.7-2
> Severity: normal
>
> I'll setup a more simple container and config so I can provide it and
> some logs to you so you can reproduce.

lxc-create -n test -t debian

I added:

lxc.autodev = 1
lxc.mount.auto = proc:mixed
lxc.mount.auto = sys:mixed
lxc.mount.auto = cgroup:mixed
lxc.cap.drop = sys_admin

to the lxc configuration but I think for now only the two last line matter:
dropping CAP_SYS_ADMIN will prevent systemd to do the mounts itself,
lxc.mount.auto = cgroup:mixed should have lxc mount /sys/fs/cgroup properly
(and thus systemd should be happy), but it's not working.

I'm starting with:

lxc-start -n test -o /tmp/lxc.log -l DEBUG -F
Failed to mount tmpfs at /dev/shm: Operation not permitted
Failed to mount tmpfs at /run: Operation not permitted
Failed to mount tmpfs at /run/lock: Operation not permitted
Failed to mount tmpfs at /sys/fs/cgroup: Operation not permitted
Failed to mount cgroup at /sys/fs/cgroup/systemd: No such file or directory
[!!!!!!] Failed to mount API filesystems, freezing.
Freezing execution.

and I'm attaching the lxc.log here. There are some more errors in the console
logs because I don't setup some of the mounts, but they don't look critical
since they don't prevent the boot.

Regards,
--
Yves-Alexis

lxc.log (19K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#875733: lxc.mount.auto = cgroup:mixed doesn't seem to work in Stretch anymore

Evgeni Golov-2
control: found -1 1:2.0.8-2

Hi,

TL;DR: I can reproduce the "does not create cgroups" behaviour, but I
don't know why yet.

On Thu, Sep 14, 2017 at 10:01:41AM +0200, Yves-Alexis Perez wrote:

> On Thu, 2017-09-14 at 09:23 +0200, Yves-Alexis Perez wrote:
> > Package: lxc
> > Version: 1:2.0.7-2
> > Severity: normal
> >
> > I'll setup a more simple container and config so I can provide it and
> > some logs to you so you can reproduce.
>
> lxc-create -n test -t debian
>
> I added:
>
> lxc.autodev = 1
> lxc.mount.auto = proc:mixed
> lxc.mount.auto = sys:mixed
> lxc.mount.auto = cgroup:mixed

This is default in LXC 2.0 [3].

> lxc.cap.drop = sys_admin
>
> to the lxc configuration but I think for now only the two last line matter:
> dropping CAP_SYS_ADMIN will prevent systemd to do the mounts itself,
> lxc.mount.auto = cgroup:mixed should have lxc mount /sys/fs/cgroup properly
> (and thus systemd should be happy), but it's not working.
>
> I'm starting with:
>
> lxc-start -n test -o /tmp/lxc.log -l DEBUG -F
> Failed to mount tmpfs at /dev/shm: Operation not permitted
> Failed to mount tmpfs at /run: Operation not permitted
> Failed to mount tmpfs at /run/lock: Operation not permitted
> Failed to mount tmpfs at /sys/fs/cgroup: Operation not permitted
> Failed to mount cgroup at /sys/fs/cgroup/systemd: No such file or directory
> [!!!!!!] Failed to mount API filesystems, freezing.
> Freezing execution.

as mentioned on IRC, the behaviour I see is a bit different.
I am using the official Debian Vagrant boxes [1][2], where I just did:
# apt install lxc (1:1.0.6-6+deb8u6 on jessie, 1:2.0.7-2 on stretch)
# lxc-create -n debian8onX -t debian -- -r jessie
# lxc-create -n debian9onX -t debian -- -r stetch

The Jessie version needed two small tweaks to the Debian template to be able to bootstrap Stretch.

Without any config changes of the containers, they start just fine with `lxc-start -n <name> -d` and I can attach to them using `lxc-attach -n <name>`.

Jessie host:

root@debian8on8:~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.3  0.0  27900  4316 ?        Ss   11:10   0:00 /sbin/init
root        19  0.2  0.0  32968  4348 ?        Ss   11:10   0:00 /lib/systemd/systemd-journald
root        68  0.0  0.0  55188  5448 ?        Ss   11:10   0:00 /usr/sbin/sshd -D
root        71  0.0  0.0  12668  1852 tty4     Ss+  11:10   0:00 /sbin/agetty --noclear tty4 linux
root        72  0.0  0.0  12668  1864 tty1     Ss+  11:10   0:00 /sbin/agetty --noclear tty1 linux
root        73  0.0  0.0  12668  1860 tty3     Ss+  11:10   0:00 /sbin/agetty --noclear tty3 linux
root        74  0.0  0.0  12668  1872 tty2     Ss+  11:10   0:00 /sbin/agetty --noclear tty2 linux
root        75  0.0  0.0  14240  2244 console  Ss+  11:10   0:00 /sbin/agetty --noclear --keep-baud console 115200 38400 9600 vt102
root        82  0.0  0.0  21868  3704 ?        S    11:10   0:00 /bin/bash
root        83  0.0  0.0  19076  2332 ?        R+   11:10   0:00 ps aux

root@debian8on8:~# mount |grep cgroup
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset,clone_children)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)

root@debian9on8:~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.6  0.0  56664  6564 ?        Ss   11:49   0:00 /sbin/init
root        16  0.1  0.0  46092  5744 ?        Ss   11:49   0:00 /lib/systemd/systemd-journald
root        46  0.0  0.0  69944  5704 ?        Ss   11:49   0:00 /usr/sbin/sshd -D
root        48  0.0  0.0  12672  1756 tty4     Ss+  11:49   0:00 /sbin/agetty --noclear tty4 linux
root        49  0.0  0.0  12672  1664 tty2     Ss+  11:49   0:00 /sbin/agetty --noclear tty2 linux
root        50  0.0  0.0  12672  1740 tty3     Ss+  11:49   0:00 /sbin/agetty --noclear tty3 linux
root        51  0.0  0.0  12672  1660 tty1     Ss+  11:49   0:00 /sbin/agetty --noclear tty1 linux
root        52  0.0  0.0  14316  2076 console  Ss+  11:49   0:00 /sbin/agetty --noclear --keep-baud console 115200,38400,9600 vt220
root        54  0.0  0.0  19828  3560 ?        S    11:50   0:00 /bin/bash
root        55  0.0  0.0  38276  3268 ?        R+   11:50   0:00 ps aux

root@debian9on8:~# mount |grep cgroup
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset,clone_children)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)

Stretch host:

root@debian8on9:~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  28116  4308 ?        Ss   12:14   0:00 /sbin/init
root        38  0.0  0.0  32968  4236 ?        Ss   12:14   0:00 /lib/systemd/systemd-journald
root        87  0.0  0.0  55188  5392 ?        Ss   12:14   0:00 /usr/sbin/sshd -D
root        90  0.0  0.0  12668  1856 tty2     Ss+  12:14   0:00 /sbin/agetty --noclear tty2 linux
root        91  0.0  0.0  12668  1880 tty1     Ss+  12:14   0:00 /sbin/agetty --noclear tty1 linux
root        92  0.0  0.0  12668  1780 tty3     Ss+  12:14   0:00 /sbin/agetty --noclear tty3 linux
root        93  0.0  0.0  12668  1876 tty4     Ss+  12:14   0:00 /sbin/agetty --noclear tty4 linux
root        94  0.0  0.0  14240  2112 console  Ss+  12:14   0:00 /sbin/agetty --noclear --keep-baud console 115200 38400 9600 vt102
root       101  0.0  0.0  21868  3832 pts/3    Ss   12:14   0:00 /bin/bash
root       102  0.0  0.0  19076  2448 pts/3    R+   12:14   0:00 ps aux

root@debian8on9:~# mount |grep cgroup
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)

root@debian9on9:~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  56756  6572 ?        Ss   12:14   0:00 /sbin/init
root        36  0.0  0.0  46092  6144 ?        Ss   12:14   0:00 /lib/systemd/systemd-journald
root        75  0.0  0.0  69944  5448 ?        Ss   12:14   0:00 /usr/sbin/sshd -D
root        77  0.0  0.0  12672  1652 pts/0    Ss+  12:14   0:00 /sbin/agetty --noclear tty1 linux
root        78  0.0  0.0  12672  1740 pts/3    Ss+  12:14   0:00 /sbin/agetty --noclear tty4 linux
root        79  0.0  0.0  12672  1652 pts/1    Ss+  12:14   0:00 /sbin/agetty --noclear tty2 linux
root        80  0.0  0.0  12672  1760 pts/2    Ss+  12:14   0:00 /sbin/agetty --noclear tty3 linux
root        81  0.0  0.0  14316  2152 pts/2    Ss+  12:14   0:00 /sbin/agetty --noclear --keep-baud console 115200,38400,9600 vt220
root        83  0.0  0.0  19828  3612 pts/3    Ss   12:14   0:00 /bin/bash
root        84  0.0  0.0  38276  3240 pts/3    R+   12:14   0:00 ps aux

root@debian9on9:~# mount |grep cgroup
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)

So far this looks pretty boring and normal.
Now I added

    lxc.mount.auto = cgroup:mixed
    lxc.cap.drop = sys_admin

to the respective container configs, and re-did the above excercise.

Jessie host:

root@debian8on8:~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.6  0.0  27084  2000 ?        S    11:25   0:00 /sbin/init
root         3  0.0  0.0  21868  3836 ?        S    11:25   0:00 /bin/bash
root         4  0.0  0.0  19076  2516 ?        R+   11:25   0:00 ps aux

root@debian8on8:~# mount | grep cgroup
cgroup_root on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,size=10240k,mode=755)
cgroup_root on /sys/fs/cgroup/perf_event type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/perf_event/lxc/debian8 type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup_root on /sys/fs/cgroup/blkio type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/blkio/lxc/debian8 type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup_root on /sys/fs/cgroup/net_cls,net_prio type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/net_cls,net_prio/lxc/debian8 type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup_root on /sys/fs/cgroup/freezer type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/freezer/lxc/debian8 type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup_root on /sys/fs/cgroup/devices type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/devices/lxc/debian8 type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup_root on /sys/fs/cgroup/cpu,cpuacct type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/cpu,cpuacct/lxc/debian8 type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup_root on /sys/fs/cgroup/cpuset type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/cpuset/lxc/debian8 type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset,clone_children)

root@debian9on8:~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.2  0.0  56304  2980 ?        S    11:50   0:00 /sbin/init
root         3  0.0  0.0  19828  3560 ?        S    11:50   0:00 /bin/bash
root         4  0.0  0.0  38276  3264 ?        R+   11:51   0:00 ps aux

root@debian9on8:~# mount |grep cgroup
cgroup_root on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,size=10240k,mode=755)
cgroup_root on /sys/fs/cgroup/perf_event type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/perf_event/lxc/debian9 type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup_root on /sys/fs/cgroup/blkio type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/blkio/lxc/debian9 type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup_root on /sys/fs/cgroup/net_cls,net_prio type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/net_cls,net_prio/lxc/debian9 type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup_root on /sys/fs/cgroup/freezer type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/freezer/lxc/debian9 type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup_root on /sys/fs/cgroup/devices type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/devices/lxc/debian9 type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup_root on /sys/fs/cgroup/cpu,cpuacct type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/cpu,cpuacct/lxc/debian9 type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup_root on /sys/fs/cgroup/cpuset type tmpfs (ro,relatime,size=10240k,mode=755)
cgroup on /sys/fs/cgroup/cpuset/lxc/debian9 type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset,clone_children)

Stretch host:

root@debian8on9:~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  27084  2088 ?        Ss   12:21   0:00 /sbin/init
root        26  0.0  0.0  21868  3476 pts/3    Ss   12:23   0:00 /bin/bash
root        28  0.0  0.0  19076  2452 pts/3    R+   12:23   0:00 ps aux

root@debian8on9:~# mount |grep cgroup
<empty>

root@debian9on9:~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  56304  3008 ?        Ss   12:21   0:00 /sbin/init
root        23  0.0  0.0  19828  3620 pts/3    Ss   12:21   0:00 /bin/bash
root        24  0.0  0.0  38276  3168 pts/3    R+   12:21   0:00 ps aux

root@debian9:~# mount |grep cgroup
<empty>

As you can see, in both cases systemd did not properly start up (no sshd, no gettys, no journald).
But on Stretch host we are also missing the whole cgroup setup, which should have been done by LXC.

Now you asked me to start the containers in foreground, to catch the output of systemd. And also start them with /bin/sh as "init".

Jessie host:

root@debian8:~# lxc-start -n debian8on8 -F
Failed to mount tmpfs at /dev/shm: Operation not permitted

root@debian8:~# lxc-start -n debian8on8 -F -- /bin/sh
# lxc-start: Input/output error - failed to read

root@debian8:~# lxc-start -n debian9on8 -F -- /bin/sh
# lxc-start: Input/output error - failed to read


Stretch host:

root@debian9:~# lxc-start  -n debian9on9 -F
Failed to mount tmpfs at /dev/shm: Operation not permitted
Failed to mount tmpfs at /run: Operation not permitted
Failed to mount tmpfs at /run/lock: Operation not permitted
Failed to mount tmpfs at /sys/fs/cgroup: Operation not permitted
Failed to mount cgroup at /sys/fs/cgroup/systemd: No such file or directory
[!!!!!!] Failed to mount API filesystems, freezing.
Freezing execution.
root@debian9:~# lxc-start  -n debian9on9 -F -- /bin/sh
/bin/sh: 0: can't access tty; job control turned off
# mount
/dev/vda1 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
none on /dev type tmpfs (rw,relatime,size=492k,mode=755)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
sysfs on /sys/devices/virtual/net type sysfs (rw,relatime)
sysfs on /sys/devices/virtual/net type sysfs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
lxcfs on /proc/cpuinfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/diskstats type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/meminfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/stat type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/swaps type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/uptime type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
devpts on /dev/console type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
devpts on /dev/tty1 type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
devpts on /dev/tty2 type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
devpts on /dev/tty3 type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
devpts on /dev/tty4 type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
#

root@debian9:~# lxc-start  -n debian8on9 -F
Failed to mount tmpfs at /dev/shm: Operation not permitted
root@debian9:~# lxc-start  -n debian8on9 -F -- /bin/sh
/bin/sh: 0: can't access tty; job control turned off
# mount
/dev/vda1 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
none on /dev type tmpfs (rw,relatime,size=492k,mode=755)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
sysfs on /sys/devices/virtual/net type sysfs (rw,relatime)
sysfs on /sys/devices/virtual/net type sysfs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
lxcfs on /proc/cpuinfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/diskstats type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/meminfo type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/stat type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/swaps type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
lxcfs on /proc/uptime type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
devpts on /dev/console type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
devpts on /dev/tty1 type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
devpts on /dev/tty2 type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
devpts on /dev/tty3 type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)
devpts on /dev/tty4 type devpts (rw,relatime,gid=5,mode=620,ptmxmode=666)

As you see, a Stretch guest always ends up in "Failed to mount API filesystems, freezing.", even though the errors before are slightly different.
And a Jessie guest just fails completelly :/

Either way, you are right, the cgroups are missing in Stretch, and I don't yet understand why.

[1] https://app.vagrantup.com/debian/boxes/jessie64
[2] https://app.vagrantup.com/debian/boxes/stretch64
[3] https://github.com/lxc/lxc/commit/f24a52d5f588ff4e4575046903fb9498c376d833

Reply | Threaded
Open this post in threaded view
|

Bug#875733: [pkg-lxc-devel] Bug#875733: lxc.mount.auto = cgroup:mixed doesn't seem to work in Stretch anymore

Evgeni Golov-2
On Sun, Sep 17, 2017 at 10:40:27AM +0200, Evgeni Golov wrote:
> TL;DR: I can reproduce the "does not create cgroups" behaviour, but I
> don't know why yet.
>
> Either way, you are right, the cgroups are missing in Stretch, and I don't yet understand why.

This happens because cgfsng_mount() is a NOOP when cgroup namespaces are
supported [4].

This seems intentional, but I don't know who is suposed to used the
namespaced cgroups then.

[4] https://github.com/lxc/lxc/blob/master/src/lxc/cgroups/cgfsng.c#L1627-L1628

Reply | Threaded
Open this post in threaded view
|

Bug#875733: lxc.mount.auto = cgroup:mixed doesn't seem to work in Stretch anymore

Matthijs Kooijman-3
In reply to this post by Yves-Alexis Perez-2
Package: lxc
Version: 1:2.0.7-2+deb9u1
Followup-For: Bug #875733

Hi folks,

I also ran into this exact issue. It seems upstream fixed this bug, see
https://github.com/lxc/lxc/issues/1737

I've backported this fix (along with some other commits it needs) to the
Debian stretch version, which works as expected. I've attached a patch
to the Debian packaging that does this. Since this is a regression from
earlier Debian versions, I guess this would be worth including in
stretch update?

One caveat to note: In my setup, I had `lxc.cgroup.use=@all` in my
`lxc.conf` file, which prevented this fix from working. See
https://github.com/lxc/lxc/issues/2084 for more details.

Gr.

Matthijs

0001-Backport-upstream-commits-to-fix-running-without-CAP.patch (29K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#875733: Some testing

Yves-Alexis Perez-2
In reply to this post by Yves-Alexis Perez-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

I did some testing with the patch from
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=875733#31 on top of stretch
LXC (1:2.0.7-2+deb9u2). At first sight, it seems to work (I can start LXC with
lxc.cap.drop = sys_admin), but somehow I can't start multiple LXC correctly
anymore.

For example the sequence:

lxc-start -n test
lxc-attach -n test "echo OK"
lxc-start -n test2
lxc-attach -n test2 "echo OK"
lxc-attach -n test "echo OK"
lxc-attach: cgroups/cgfsng.c: cgfsng_attach: 1830 No such file or directory -
Failed to attach 14680 to /sys/fs/cgroup/systemd//lxc/www-1/cgroup.procs
                               lxc-attach: attach.c: lxc_attach: 992 Expected
to receive sequence number 0: No such file or directory.

It might be some kind of race condition because it doesn't always happen with
two containers, sometimes it's three.

Regards,
- --
Yves-Alexis
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEE8vi34Qgfo83x35gF3rYcyPpXRFsFAlvz7bkACgkQ3rYcyPpX
RFuToQf/a5JS7roF/wwCRM19OEwgdGTe1fnxuhqtDr5OBViBQoONOutTqb3QIXOe
gLzQd+yeRVrKMg9o3Ynxcb9alYQzPKGTAuvmC5tqXSGMpIFVML8PpP+tN5rJ6Xdm
D2Ulo7piPnZVnUbY7LOPDfx8sr4n1/oUPvq8/kVG/Hdj42KyEP0OQ5kwYPIlXdS2
6kNXCE9pBZrKeMjfUNynL7Xt16f2giv2Y67h9FfSNcutH6GCbdCer82Bb3y7UNtM
HBbuUeexMBFl4uW/a+I5hyG3MaXthtMR3l508WUsJHqMvA6SVHHNxdCYxUNwL3jq
Web8oKolr2nfvk1jnFJkkbtU7S08lA==
=VS7/
-----END PGP SIGNATURE-----