Quantcast

Bug#860833: 50mounted-tests hangs at dmsetup when using biosgrub partition on a left over fat32 partition

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Bug#860833: 50mounted-tests hangs at dmsetup when using biosgrub partition on a left over fat32 partition

Marga Manterola
Package: os-prober
Version: 1.74
Severity: important

I'm using a recipe that includes this snippet:
              1 1 1 free                                      \
                $iflabel{ gpt }                               \
                $reusemethod{ }                               \
                method{ biosgrub }                            \

This snippet is present in the following automated recipes:

The biosgrub method does not touch the partition in any way. It leaves the contents as is.  If the installation is done in EFI mode, nothing is written to this BIOS partition. Which means that if there happened to be a fat32 partition that started on that sector (and was larger than 1 MB), the fat32 bits are still there after partman has run, in a way that confuses the lsblk command into thinking that the filesystem is "vfat"

This was always the case but it was not a problem with previous os-prober versions because there was no dmsetup being executed.  Now, with the new dmsetup commands, this triggers problems because the mount point enters a weird state.

This is the error in the log when it tries to mount the partition the first time:

Apr 20 08:30:12 50mounted-tests: debug: mounting /dev/mapper/osprober-linux-nvme0n1p1 (/dev/nvme0n1p1) as vfat failed: mount: mounting /dev/mapper/osprober-linux-nvme0n1p1 on /var/lib/os-prober/mount failed: Input/output error

The first time os-prober runs, it gets this error and it then claims to have removed the device mapper entry.  However, the next time os-prober runs (for some reason, during one d-i session, it runs 4 times) it hangs while running dmsetup create -r osprober-linux-nvme0n1p1

This hangs completely and never ends, a possible workaround is to kill the dmsetup command.  After that, the next 2 os-prober runs detect that there's a problem and don't hang.

I believe this is quite an important issue as any machine that has a previously formatted vfat will completely hang while installing. Before you claim no machine will have a previously formatted vfat: the affected machine is a new "Non Windows" machine coming from HP with FreeDOS installed.

After a lot of debugging with Philipp Kern, we were able to find out that dmsetup was hanging on a udev cookie semaphore.  The udev outside of the chroot is compiled udev_sync, and it doesn't have dmsetup rules, so it seems that it's not sending the dmsetup udevcomplete signal and thus dmsetup hangs forever. Another possible workaround then is to send the right dmsetup udevcomplete signal, and then installation proceeds.

It's not clear to me how the dmsetup code block would ever work without proper udev support outside the chroot.  It's also not clear why the first time it fails sort of gracefully and then hangs very ungracefully the second time.
--
Cheers,
Marga
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Bug#860833: 50mounted-tests hangs at dmsetup when using biosgrub partition on a left over fat32 partition

Philipp Kern-6
On 04/20/2017 08:34 PM, Marga Manterola wrote:
>> After a lot of debugging with Philipp Kern, we were able to find out
> that dmsetup was hanging on a udev cookie semaphore.  The udev outside
> of the chroot is compiled udev_sync, and it doesn't have dmsetup rules,
> so it seems that it's not sending the dmsetup udevcomplete signal and
> thus dmsetup hangs forever. Another possible workaround then is to send
> the right dmsetup udevcomplete signal, and then installation proceeds.

One small correction: The *dmsetup* outside of the chroot is compiled
*without* udev_sync as it's coming from the udeb. It also doesn't ship
with 55-dm.rules so it wouldn't call `dmsetup udevcomplete' even if it
were compiled with udev_sync.

Kind regards
Philipp Kern

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Bug#860833: 50mounted-tests hangs at dmsetup when using biosgrub partition on a left over fat32 partition

Marga Manterola
Adding some more data from logs and stuff now that I got the machine to finish installation successfully.

This is the first os-prober run:
Apr 21 08:35:48 os-prober: debug: running /usr/lib/os-probes/50mounted-tests on /dev/nvme0n1p1
Apr 21 08:35:48 50mounted-tests: debug: creating device mapper device /dev/mapper/osprober-linux-nvme0n1p1
Apr 21 08:35:48 kernel: [  953.534341] FAT-fs (dm-4): FAT read failed (blocknr 6158)
Apr 21 08:35:48 50mounted-tests: debug: mounting /dev/mapper/osprober-linux-nvme0n1p1 (/dev/nvme0n1p1) as vfat failed: mount: mounting /dev/mapper/osprober-linux-nvme0n1p1 on /var/lib/os-prober/mount failed: Input/output error
Apr 21 08:35:48 50mounted-tests: debug: remove device mapper device /dev/mapper/osprober-linux-nvme0n1p1

This is the second run:
Apr 21 08:35:53 os-prober: debug: running /usr/lib/os-probes/50mounted-tests on /dev/nvme0n1p1
Apr 21 08:35:53 50mounted-tests: debug: creating device mapper device /dev/mapper/osprober-linux-nvme0n1p1
Apr 21 08:38:58 init: starting pid 260, tty '/dev/tty2': '-/bin/sh'
Apr 21 08:40:41 50mounted-tests: debug: mounting /dev/mapper/osprober-linux-nvme0n1p1 (/dev/nvme0n1p1) as vfat failed: mount: special device /dev/mapper/osprober-linux-nvme0n1p1 does not exist

You can see that I logged in there and took a few minutes to gather data and then unblock it.  This is what I did (with some information obscured)

Trailing part of ps fax:

31842 ?  S+ 0:00   \_ /bin/sh /etc/grub.d/30_os-prober
31848 ?  S+ 0:00       \_ /bin/sh /etc/grub.d/30_os-prober
31849 ?  S+ 0:00           \_ /bin/sh /usr/bin/os-prober
31931 ?  S+ 0:00           |   \_ /bin/sh /usr/lib/os-probes/50mounted-tests /dev/nvme0n1p1
31938 ?  S+ 0:00           |       \_ /bin/sh /usr/lib/os-probes/50mounted-tests /dev/nvme0n1p1
31943 ?  S+ 0:00           |           \_ dmsetup create -r osprober-linux-nvme0n1p1
31850 ?  S+ 0:00           \_ tr   ^   
31851 ?  S+ 0:00           \_ paste -s -d  

$ dmsetup table 
osprober-linux-nvme0n1p1: 0 2048 linear 259:1 0
<LVM volume1>: 0 747470848 linear 253:0 117188608
<LVM volume2>: 0 117186560 linear 253:0 2048
nvme0n1p4_crypt: 0 998707200 crypt aes-xts-plain64 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0 259:4 4096
<LVM swap>: 0 134045696 linear 253:0 864659456

$ ipcs -s 

------ Semaphore Arrays --------
key        semid      owner      perms      nsems     
0x0d4ddd11 0          root       600        1  

And then I unblocked the process by doing (inside the chroot):
$ dmsetup udevcomplete 0x0d4ddd11

These are the third and fourth os-prober runs:
Apr 21 08:42:49 os-prober: debug: running /usr/lib/os-probes/50mounted-tests on /dev/nvme0n1p1
Apr 21 08:42:49 50mounted-tests: debug: creating device mapper device /dev/mapper/osprober-linux-nvme0n1p1
Apr 21 08:42:49 50mounted-tests: debug: mounting /dev/mapper/osprober-linux-nvme0n1p1 (/dev/nvme0n1p1) as vfat failed: mount: special device /dev/mapper/osprober-linux-nvme0n1p1 does not exist

Apr 21 08:43:53 os-prober: debug: running /usr/lib/os-probes/50mounted-tests on /dev/nvme0n1p1
Apr 21 08:43:53 50mounted-tests: debug: creating device mapper device /dev/mapper/osprober-linux-nvme0n1p1
Apr 21 08:43:53 50mounted-tests: debug: mounting /dev/mapper/osprober-linux-nvme0n1p1 (/dev/nvme0n1p1) as vfat failed: mount: special device /dev/mapper/osprober-linux-nvme0n1p1 does not exist

I have a machine where I can reproduce this every time (since the partition never gets written to), so let me know if you want me to gather any additional information.

Thanks!

--
Cheers,
Marga
Loading...