Bug#928096: systemd does not play well with nfs root - unable to shutdown/reboot

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug#928096: systemd does not play well with nfs root - unable to shutdown/reboot

Gianluigi Tiesi
Package: live-build
Version: 1:20190311
Severity: normal

Hi, I'm playing with images using nfs root, my host is debian sid amd64,
I build images for buster amd64

I'm using qemu:

kvm \
    -m 2048 \
    -vga qxl \
    -serial stdio \
    -usb -device usb-tablet \
    -soundhw hda \
    -netdev user,id=net0,host=192.168.76.1,net=192.168.76.0/24,dhcpstart=192.168.76.9 \
    -device e1000,netdev=net0 \
    -drive file=disk.img,if=ide,format=qcow2 \
    -kernel binary/live/vmlinuz \
    -initrd binary/live/initrd.img \
    -append "boot=live persistence components netboot=nfs locales=it_IT.UTF-8 nfsroot=192.168.76.1:/home/sherpya/live-default/binary console=ttyS0,115200 console=tty"


/etc/exports:

/home/sherpya/live-default/binary *(ro,async,no_root_squash,no_subtree_check,insecure)

The system works fine but I'm unable to shutdown or reboot:

[  OK  ] Reached target Reboot.
[   72.684441] systemd-shutdow: 51 output lines suppressed due to ratelimiting
[   77.162859] nfs: server 192.168.76.1 not responding, still trying
[   77.164759] nfs: server 192.168.76.1 not responding, still trying
[   77.170156] nfs: server 192.168.76.1 not responding, still trying
[  116.073320] systemd-journald[517]: Failed to send WATCHDOG=1 notification message: Connection refused
[  236.055922] systemd-journald[517]: Failed to send WATCHDOG=1 notification message: Transport endpoint is not connected
[  242.509427] INFO: task systemd-shutdow:1 blocked for more than 120 seconds.
[  242.513657]       Not tainted 4.19.0-4-amd64 #1 Debian 4.19.28-2
[  242.516653] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  242.520144] systemd-shutdow D    0     1      0 0x00000000
[  242.522682] Call Trace:
[  242.523495]  ? __schedule+0x2a2/0x870
[  242.524414]  schedule+0x28/0x80
[  242.525263]  io_schedule+0x12/0x40
[  242.526140]  __lock_page_or_retry+0x305/0x340
[  242.527137]  ? page_cache_tree_insert+0xe0/0xe0
[  242.528157]  filemap_fault+0x18d/0x770
[  242.529136]  ? alloc_set_pte+0x494/0x560
[  242.530229]  ? filemap_map_pages+0x139/0x3a0
[  242.531329]  ext4_filemap_fault+0x2c/0x40 [ext4]
[  242.532422]  __do_fault+0x36/0x130
[  242.533341]  __handle_mm_fault+0xe6f/0x1280
[  242.534325]  ? isolate_lru_page+0x1c7/0x260
[  242.535299]  handle_mm_fault+0xda/0x200
[  242.536332]  __get_user_pages+0x240/0x6d0
[  242.537369]  populate_vma_page_range+0x6d/0x70
[  242.538413]  __mm_populate+0x9d/0x140
[  242.539315]  __x64_sys_mlockall+0xf1/0x180
[  242.540298]  do_syscall_64+0x53/0x100
[  242.541210]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  242.542361] RIP: 0033:0x7ff5f7803717
[  242.543360] Code: Bad RIP value.
[  242.544234] RSP: 002b:00007ffea0b288f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000097
[  242.545776] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007ff5f7803717
[  242.547204] RDX: 0000000000000007 RSI: 000055b0fbbd028e RDI: 0000000000000003
[  242.548865] RBP: 000055b0fbbd3420 R08: 000000000000021f R09: 000055b0fd399290
[  242.550633] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[  242.553432] R13: 00007ffea0b28c80 R14: 0000000000000000 R15: 0000000000000000
[  296.047709] systemd-journald[517]: Failed to send WATCHDOG=1 notification message: Transport endpoint is not connected

In a previous build (some days ago) it was working.
I have the same problem using files from debian live iso

I suppose the system cannot shutdown ethernet if the root is on nfs, and nfs live mount cannot
be unmounted.

I've seen around someone suggesting to make systemd ignore interface shutdown problems


-- Package-specific info:

-- System Information:
Debian Release: 10.0
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.9.0-9-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)

Versions of packages live-build depends on:
ii  debootstrap  1.0.114

Versions of packages live-build recommends:
ii  apt-utils                       1.8.0
ii  bzip2                           1.0.6-9
ii  cpio                            2.12+dfsg-9
ii  file                            1:5.35-4
pn  live-boot-doc                   <none>
pn  live-config-doc                 <none>
pn  live-manual-html | live-manual  <none>
ii  wget                            1.20.1-1.1
ii  xz-utils                        5.2.4-1

Versions of packages live-build suggests:
ii  e2fsprogs  1.45.0-1
pn  mtd-utils  <none>
pn  parted     <none>

-- no debconf information

Reply | Threaded
Open this post in threaded view
|

Bug#928096: systemd does not play well with nfs root - unable to shutdown/reboot

Luca Boccassi-3
On Sun, 2019-04-28 at 00:08 +0200, Gianluigi Tiesi wrote:

> Package: live-build
> Version: 1:20190311
> Severity: normal
>
> Hi, I'm playing with images using nfs root, my host is debian sid
> amd64,
> I build images for buster amd64
>
> I'm using qemu:
>
> kvm \
>     -m 2048 \
>     -vga qxl \
>     -serial stdio \
>     -usb -device usb-tablet \
>     -soundhw hda \
>     -netdev
> user,id=net0,host=192.168.76.1,net=192.168.76.0/24,dhcpstart=192.168.
> 76.9 \
>     -device e1000,netdev=net0 \
>     -drive file=disk.img,if=ide,format=qcow2 \
>     -kernel binary/live/vmlinuz \
>     -initrd binary/live/initrd.img \
>     -append "boot=live persistence components netboot=nfs
> locales=it_IT.UTF-8 nfsroot=192.168.76.1:/home/sherpya/live-
> default/binary console=ttyS0,115200 console=tty"
>
>
> /etc/exports:
>
> /home/sherpya/live-default/binary
> *(ro,async,no_root_squash,no_subtree_check,insecure)
>
> The system works fine but I'm unable to shutdown or reboot:
>
> [  OK  ] Reached target Reboot.
> [   72.684441] systemd-shutdow: 51 output lines suppressed due to
> ratelimiting
> [   77.162859] nfs: server 192.168.76.1 not responding, still trying
> [   77.164759] nfs: server 192.168.76.1 not responding, still trying
> [   77.170156] nfs: server 192.168.76.1 not responding, still trying
> [  116.073320] systemd-journald[517]: Failed to send WATCHDOG=1
> notification message: Connection refused
> [  236.055922] systemd-journald[517]: Failed to send WATCHDOG=1
> notification message: Transport endpoint is not connected
> [  242.509427] INFO: task systemd-shutdow:1 blocked for more than 120
> seconds.
> [  242.513657]       Not tainted 4.19.0-4-amd64 #1 Debian 4.19.28-2
> [  242.516653] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [  242.520144] systemd-shutdow D    0     1      0 0x00000000
> [  242.522682] Call Trace:
> [  242.523495]  ? __schedule+0x2a2/0x870
> [  242.524414]  schedule+0x28/0x80
> [  242.525263]  io_schedule+0x12/0x40
> [  242.526140]  __lock_page_or_retry+0x305/0x340
> [  242.527137]  ? page_cache_tree_insert+0xe0/0xe0
> [  242.528157]  filemap_fault+0x18d/0x770
> [  242.529136]  ? alloc_set_pte+0x494/0x560
> [  242.530229]  ? filemap_map_pages+0x139/0x3a0
> [  242.531329]  ext4_filemap_fault+0x2c/0x40 [ext4]
> [  242.532422]  __do_fault+0x36/0x130
> [  242.533341]  __handle_mm_fault+0xe6f/0x1280
> [  242.534325]  ? isolate_lru_page+0x1c7/0x260
> [  242.535299]  handle_mm_fault+0xda/0x200
> [  242.536332]  __get_user_pages+0x240/0x6d0
> [  242.537369]  populate_vma_page_range+0x6d/0x70
> [  242.538413]  __mm_populate+0x9d/0x140
> [  242.539315]  __x64_sys_mlockall+0xf1/0x180
> [  242.540298]  do_syscall_64+0x53/0x100
> [  242.541210]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [  242.542361] RIP: 0033:0x7ff5f7803717
> [  242.543360] Code: Bad RIP value.
> [  242.544234] RSP: 002b:00007ffea0b288f8 EFLAGS: 00000246 ORIG_RAX:
> 0000000000000097
> [  242.545776] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
> 00007ff5f7803717
> [  242.547204] RDX: 0000000000000007 RSI: 000055b0fbbd028e RDI:
> 0000000000000003
> [  242.548865] RBP: 000055b0fbbd3420 R08: 000000000000021f R09:
> 000055b0fd399290
> [  242.550633] R10: 0000000000000000 R11: 0000000000000246 R12:
> 0000000000000000
> [  242.553432] R13: 00007ffea0b28c80 R14: 0000000000000000 R15:
> 0000000000000000
> [  296.047709] systemd-journald[517]: Failed to send WATCHDOG=1
> notification message: Transport endpoint is not connected
>
> In a previous build (some days ago) it was working.
> I have the same problem using files from debian live iso
>
> I suppose the system cannot shutdown ethernet if the root is on nfs,
> and nfs live mount cannot
> be unmounted.
>
> I've seen around someone suggesting to make systemd ignore interface
> shutdown problems
If you have tried with debian live iso too, I don't think this has
anything to do with live-build?

--
Kind regards,
Luca Boccassi

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#928096: systemd does not play well with nfs root - unable to shutdown/reboot

Gianluigi Tiesi
On 4/28/19 8:14 PM, Luca Boccassi wrote:
> If you have tried with debian live iso too, I don't think this has
> anything to do with live-build?

Isn't debian live iso made using live-build?
Unfortunately I was not able to make a non systemd live iso


--
Gianluigi Tiesi <[hidden email]>
EDP Project Leader
Netfarm S.r.l. - http://www.netfarm.it/
Free Software: http://oss.netfarm.it/

Q: Because it reverses the logical flow of conversation.
A: Why is putting a reply at the top of the message frowned upon?

Reply | Threaded
Open this post in threaded view
|

Bug#928096: systemd does not play well with nfs root - unable to shutdown/reboot

Luca Boccassi-3
On Wed, 2019-06-05 at 11:56 +0200, Gianluigi Tiesi wrote:
> On 4/28/19 8:14 PM, Luca Boccassi wrote:
> > If you have tried with debian live iso too, I don't think this has
> > anything to do with live-build?
>
> Isn't debian live iso made using live-build?
> Unfortunately I was not able to make a non systemd live iso

It does not, it uses live-wrapper. Did you see this problem with
official debian live images? Or custom-built with live-build?

--
Kind regards,
Luca Boccassi

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#928096: systemd does not play well with nfs root - unable to shutdown/reboot

Gianluigi Tiesi
On 6/5/19 12:01 PM, Luca Boccassi wrote:

> On Wed, 2019-06-05 at 11:56 +0200, Gianluigi Tiesi wrote:
>> On 4/28/19 8:14 PM, Luca Boccassi wrote:
>>> If you have tried with debian live iso too, I don't think this has
>>> anything to do with live-build?
>>
>> Isn't debian live iso made using live-build?
>> Unfortunately I was not able to make a non systemd live iso
>
> It does not, it uses live-wrapper. Did you see this problem with
> official debian live images? Or custom-built with live-build?
>

Both, but a build (live-build) made some days before reporting the bug
was working

--
Gianluigi Tiesi <[hidden email]>
EDP Project Leader
Netfarm S.r.l. - http://www.netfarm.it/
Free Software: http://oss.netfarm.it/

Q: Because it reverses the logical flow of conversation.
A: Why is putting a reply at the top of the message frowned upon?