Installation tests with GRUB on GPT disks

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Installation tests with GRUB on GPT disks

John Paul Adrian Glaubitz
Hi!

I have been testing the installation of GRUB in debian-installer on disks
with GPT labels. Unfortunately, that doesn't work as grub-install
complains it cannot find the filesystem for /dev/vdiska1.

Looking at the partition table with parted, it seems that the actual problem
is a corrupted GPT partition table (see below).

Since a new partition table is created by debian-installer, it shouldn't be
corrupted at this point. So, I'm wondering what problem we are running
into.

I will have a look at the partman-partitioning source code but if someone
wants to perform their own tests in the meantime, the test installation
image can be found here:

> https://people.debian.org/~glaubitz/debian-cd/debian-10.0-sparc64-NETINST-1.iso

Adrian

=============================================================================

root@debian:/# parted /dev/vdiska                                              
GNU Parted 3.2                                                                  
Using /dev/vdiska                                                              
Welcome to GNU Parted! Type 'help' to view a list of commands.                  
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p                                                              
Error: /dev/vdiska: unrecognised disk label
Model: Unknown (unknown)                                                  
Disk /dev/vdiska: 17.2GB
Sector size (logical/physical): 512B/8192B
Partition Table: unknown
Disk Flags:
(parted) q                                                                
root@debian:/# fdisk -l /dev/vdiska
The primary GPT table is corrupt, but the backup appears OK, so that will be used.
Disk /dev/vdiska: 16 GiB, 17179869184 bytes, 33554432 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes                            
Disklabel type: gpt                                                            
Disk identifier: 2D6D55DD-FA7D-41E8-8920-919C31964B57                          
                                                                               
Device          Start      End  Sectors  Size Type                              
/dev/vdiska1     2048 29687807 29685760 14.2G Linux filesystem                  
/dev/vdiska2 29687808 33552383  3864576  1.9G Linux swap                        
root@debian:/#

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

pEpkey.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

Frank Scheiner
Hi,

On 05/16/2018 01:29 PM, John Paul Adrian Glaubitz wrote:

> Hi!
>
> I have been testing the installation of GRUB in debian-installer on disks
> with GPT labels. Unfortunately, that doesn't work as grub-install
> complains it cannot find the filesystem for /dev/vdiska1.
>
> Looking at the partition table with parted, it seems that the actual problem
> is a corrupted GPT partition table (see below).
>
> Since a new partition table is created by debian-installer, it shouldn't be
> corrupted at this point. So, I'm wondering what problem we are running
> into.

Maybe it wasn't created correctly by partman:

On [1] Eric has a ca. 7 MiB partition between 1 MiB at the beginning of
the disk and the remainder of the disk. It's garbled on GitHub, so here
it is reformatted:

```

Number Start  End    Size   File system Name      Flags
1      1049kB 8389kB 7340kB bios_grub   bios_grub
2      8389kB 1151MB 1143MB ext3        boot
3      1151MB 600GB  599GB  lvm
```

[1]:
https://github.com/esnowberg/grub2-sparc/wiki#installation-instructions-for-sparc-t4-and-above

During my testing end of February/beginning of March this partition was
needed, because otherwise the gpt label was corrupted after grub-install
ran IIRC - i.e. similar to what you saw. I don't know the exact size
requirements for this partition though, so just rounded up to 10 MiB
which worked fine.

I submitted a patch ([2]) which adds specific recipes for gpt capable
systems which add the additional 10 MiB partition. Partman than creates
this partition after the first 1 MiB on the disk in question. The post
also contains an alternative approach which might also work (i.e. just
extending the sparc recipes).

[2]: https://lists.debian.org/debian-sparc/2018/03/msg00007.html

Frank

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

John Paul Adrian Glaubitz
On 05/16/2018 02:44 PM, Frank Scheiner wrote:

> Maybe it wasn't created correctly by partman:
>
> On [1] Eric has a ca. 7 MiB partition between 1 MiB at the beginning of the disk and the remainder of the disk. It's garbled on GitHub, so here it is reformatted:
>
> ```
>
> Number Start  End    Size   File system Name      Flags
> 1      1049kB 8389kB 7340kB bios_grub   bios_grub
> 2      8389kB 1151MB 1143MB ext3        boot
> 3      1151MB 600GB  599GB  lvm
> ```
>
> [1]: https://github.com/esnowberg/grub2-sparc/wiki#installation-instructions-for-sparc-t4-and-above
>
> During my testing end of February/beginning of March this partition was needed, because otherwise the gpt label was corrupted after grub-install ran IIRC - i.e. similar to what you saw. I don't know the exact size requirements for this partition though, so just rounded up to 10 MiB which worked fine.

Ah, this makes sense. I'll see whether I can add this type manually.

> I submitted a patch ([2]) which adds specific recipes for gpt capable systems which add the additional 10 MiB partition. Partman than creates this partition after the first 1 MiB on the disk in question. The post also contains an alternative approach which might also work (i.e. just extending the sparc recipes).

Yes, there are some things I wanted to test first before adopting the patch.

I don't like it yet as is.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

John Paul Adrian Glaubitz
On 05/16/2018 02:48 PM, John Paul Adrian Glaubitz wrote:
>> During my testing end of February/beginning of March this partition was needed, because otherwise the gpt label was corrupted after grub-install ran IIRC - i.e. similar to what you saw. I don't know the exact size requirements for this partition though, so just rounded up to 10 MiB which worked fine.
>
> Ah, this makes sense. I'll see whether I can add this type manually.

Ok, just adding a 10 MB bios_grub partition resolved the issue and GRUB
installed without any problems :).

root@debian:~# parted /dev/vdiska
GNU Parted 3.2
Using /dev/vdiska
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p                                                                
Model: Unknown (unknown)
Disk /dev/vdiska: 17.2GB
Sector size (logical/physical): 512B/8192B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system     Name  Flags
 1      1049kB  10.5MB  9437kB                        bios_grub
 2      10.5MB  15.2GB  15.2GB  ext4
 3      15.2GB  17.2GB  1968MB  linux-swap(v1)

(parted)

>> I submitted a patch ([2]) which adds specific recipes for gpt capable systems which add the additional 10 MiB partition. Partman than creates this partition after the first 1 MiB on the disk in question. The post also contains an alternative approach which might also work (i.e. just extending the sparc recipes).
>
> Yes, there are some things I wanted to test first before adopting the patch.

Adding those recipes for partman-auto will resolve the issue. However, we
should probably add a warning like we had for SILO which reminds users
they have to add a bios_grub partition on a GPT machine:

> https://anonscm.debian.org/git/d-i/attic/silo-installer.git/tree/check.d/silo_check

Does any of the other architectures which use GRUB require a bios_grub partition?

If yes, we should just add sparc and sparc64 to this test. Otherwise, we
will have to add it from scratch, probably to the grub-installer package.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

Frank Scheiner
On 05/16/2018 03:03 PM, John Paul Adrian Glaubitz wrote:
> Adding those recipes for partman-auto will resolve the issue.

Should we add additional recipes or better modify the existing sparc recipe?

> However, we
> should probably add a warning like we had for SILO which reminds users
> they have to add a bios_grub partition on a GPT machine:
>
>> https://anonscm.debian.org/git/d-i/attic/silo-installer.git/tree/check.d/silo_check
>
> Does any of the other architectures which use GRUB require a bios_grub partition?

The "default" recipes use a bios_grub partition (see e.g. [1]). It's 1
MiB in size, but I expect that when partitioned on a gpt capable system
this will result in 1 MiB space at the beginning of the disk and a 1 MiB
partition after it. I assume this is created automatically on gpt
capable systems, because of:

```
$iflabel{ gpt }
```

I don't know if this is required for GRUB2 on i386/amd64 though, but
maybe yes, so GRUB2 doesn't touch the 1 MiB space at the beginning.

[1]:
https://salsa.debian.org/installer-team/partman-auto/blob/master/recipes/atomic

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

John Paul Adrian Glaubitz
On 05/16/2018 03:25 PM, Frank Scheiner wrote:
> On 05/16/2018 03:03 PM, John Paul Adrian Glaubitz wrote:
>> Adding those recipes for partman-auto will resolve the issue.
>
> Should we add additional recipes or better modify the existing sparc recipe?

If just adding

```
10 10 10 free
        $iflabel{ gpt }
        $reusemethod{ }
        method{ biosgrub } .
```

does trick, then we should use this, I guess.

If we need to add new recipes, then we should name the folder "recipes-sparc-sun4u_gpt"
and make "sparc64" the link files (to be consistent with the existing recipes).

Did you test whether this becomes a no-op one systems with Sun-label?

I can test whether it does the right thing on GPT labels, provided that
the recipe files can be found somewhere on the disk where I can change
them.

>> Does any of the other architectures which use GRUB require a bios_grub partition?
>
> The "default" recipes use a bios_grub partition (see e.g. [1]). It's 1 MiB in size, but I expect that when partitioned on a gpt capable system this will result in 1 MiB space at the beginning of the disk and a 1 MiB partition after it. I assume this is created automatically on gpt capable systems, because of:
>
> ```
> $iflabel{ gpt }
> ```
>
> I don't know if this is required for GRUB2 on i386/amd64 though, but maybe yes, so GRUB2 doesn't touch the 1 MiB space at the beginning.

Pretty sure it's not as x86-based systems use an EFI partition instead.

I think, we need to add such a warning at some point. But for the time
being, just extending partman-auto should be enough to be able to
kick out "silo-installer".

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

Frank Scheiner
On 05/16/2018 05:23 PM, John Paul Adrian Glaubitz wrote:
> ```
> 10 10 10 free
> $iflabel{ gpt }
> $reusemethod{ }
> method{ biosgrub } .
> ```
> Did you test whether this becomes a no-op one systems with Sun-label?

I don't remember, but I'd say no, because I wrote "[...] if the
assumption is correct that "$iflabel{ gpt }" line will only make this
part effective on GPT partitioned disks" in early March.

I'll give it a try on my T5220 with kernel and initrd from the image you
provided today and report back.

>
> I can test whether it does the right thing on GPT labels, provided that
> the recipe files can be found somewhere on the disk where I can change
> them.

The recipes are located below `/lib/partman/` in the installer
environment. They can be modified before entering the partitioning step
(`nano` is available in the installer environment) - even multiple times
if you can repeat the partitioning step (e.g. in expert mode).

Frank

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

John Paul Adrian Glaubitz
On 05/16/2018 06:57 PM, Frank Scheiner wrote:

> On 05/16/2018 05:23 PM, John Paul Adrian Glaubitz wrote:
>> ```
>> 10 10 10 free
>>     $iflabel{ gpt }
>>     $reusemethod{ }
>>     method{ biosgrub } .
>> ```
>> Did you test whether this becomes a no-op one systems with Sun-label?
>
> I don't remember, but I'd say no, because I wrote "[...] if the assumption is correct that "$iflabel{ gpt }" line will only make this part effective on GPT
> partitioned disks" in early March.
>
> I'll give it a try on my T5220 with kernel and initrd from the image you provided today and report back.

Thanks. We should definitely test this before committing those changes.

>> I can test whether it does the right thing on GPT labels, provided that
>> the recipe files can be found somewhere on the disk where I can change
>> them.
>
> The recipes are located below `/lib/partman/` in the installer environment. They can be modified before entering the partitioning step (`nano` is available in
> the installer environment) - even multiple times if you can repeat the partitioning step (e.g. in expert mode).

Yes, that's what I already expected. I just hadn't looked whether it's
actually the case the recipes are in there.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

Frank Scheiner
On 05/16/2018 07:11 PM, John Paul Adrian Glaubitz wrote:
>> I'll give it a try on my T5220 with kernel and initrd from the image you provided today and report back.
>
> Thanks. We should definitely test this before committing those changes.

Unfortunately booting the CDROM kernel and initrd via GRUB2 over the
network did not work as expected by me:

After the installer started it wants to load its components from the
non-existing CDROM, of course... :-(

There is no command line switch for the installer (kernel) that will
make it behave like a netboot installer, right?

So I wrote the image to a CDRW and started the installation from there.

The partitioning worked correctly with the added snippet and created a
layout that resembles the recipe without the snippet:

```
~ # fdisk -l

Disk /dev/sda: 149.1 GiB, 160041885696 bytes, 312581808 sectors

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

/lib/partman/recipes-sparc64 # nano 30atomic

## Added this part on top ##
## 10 10 10 free
## $iflabel{ gpt }
## $reusemethod{ }
## method{ biosgrub } .

/lib/partman/recipes-sparc64 # archdetect

sparc64/sun4v_sun

## Partitioning took place ##

/lib/partman/recipes-sparc64 # fdisk -l
Disk /dev/sda: 149.1 GiB, 160041885696 bytes, 312581808 sectors
Geometry: 255 heads, 63 sectors/track, 19457 cylinders
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: sun

Device         Start       End   Sectors   Size Id Type         Flags
/dev/sda1          0    192779    192780  94.1M  1 Boot
/dev/sda2     192780 296656289 296463510 141.4G 83 Linux native
/dev/sda3          0 312576704 312576705   149G  5 Whole disk
/dev/sda4  296656290 312576704  15920415   7.6G 82 Linux swap
```

If it creates the bios_grub "partition" on your gpt capable machine with
the snippet included, it could be save for inclusion in d-i/partman-auto.

Frank

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

John Paul Adrian Glaubitz
On 05/16/2018 08:15 PM, Frank Scheiner wrote:
> Unfortunately booting the CDROM kernel and initrd via GRUB2 over the network did not work as expected by me:
>
> After the installer started it wants to load its components from the non-existing CDROM, of course... :-(
>
> There is no command line switch for the installer (kernel) that will make it behave like a netboot installer, right?

This is expected. You cannot use the cdrom debian-installer initrd for netboot
and vice versa. If you want to perform a netboot, you have to use the initrd
for netboot which is inside the debian-installer-images tarball as found
on the FTP servers in the pool-sparc64/main/d/debian-installer subdirectory.

Since the d-i current image is outdated, use the one from here:

> https://people.debian.org/~glaubitz/debian-cd/debian-installer-images_20171204_sparc64.tar.gz

Please note that even the "NETINST" images are still CD images. Real netboot
images just require a kernel and initrd. The netboot initrd contains the
necessary network drivers and is designed to fetch udebs over the internet.

> So I wrote the image to a CDRW and started the installation from there.
>
> The partitioning worked correctly with the added snippet and created a layout that resembles the recipe without the snippet:
>> (...)
> If it creates the bios_grub "partition" on your gpt capable machine with the snippet included, it could be save for inclusion in d-i/partman-auto.

Thanks for the confirmation. I will test later as I didn't have the time yet.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

John Paul Adrian Glaubitz
On 05/16/2018 08:53 PM, John Paul Adrian Glaubitz wrote:
>> So I wrote the image to a CDRW and started the installation from there.
>>
>> The partitioning worked correctly with the added snippet and created a layout that resembles the recipe without the snippet:
>>> (...)
>> If it creates the bios_grub "partition" on your gpt capable machine with the snippet included, it could be save for inclusion in d-i/partman-auto.
>
> Thanks for the confirmation. I will test later as I didn't have the time yet.

Ok, I just successfully tested this change.

Two questions left:

1. Can you test whether we can drop the /boot partition on Sun
   partition tables?

2. Now that we can use GRUB, what about updating the default
   partitioning scheme? Maybe one that is inspired by the layout
   used on ppc64el:

   > https://salsa.debian.org/installer-team/partman-auto/tree/master/recipes-ppc64el

3. GRUB installs fine now on Sun partition tables without any further ado?

4. And, just to be safe: Your previous mail contained hashes
   before the partition layout lines (#). You did not actually
   have those in your atomic file, correct?

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

Frank Scheiner
On 05/16/2018 09:16 PM, John Paul Adrian Glaubitz wrote:
> Ok, I just successfully tested this change.
>
> Two questions left:
>
> 1. Can you test whether we can drop the /boot partition on Sun
>     partition tables?

I actually don't remember if I tested such a configuration. At least
d-i/grub-installer should support this now. I'll check that on my T5220
and report back.

Just a thought:

Could it happen that - depending on the size of the root FS and the
amount of packages installed - the GRUB image gets installed so far away
from the start of a disk - it usually gets installed at the end of an
installation - that older machines with older OBP versions could have
problems to access it via block lists? Are there any such limits?

If yes, a separate smaller partition for "/boot" might be required in
such a case.

>
> 2. Now that we can use GRUB, what about updating the default
>     partitioning scheme? Maybe one that is inspired by the layout
>     used on ppc64el:
>
>     > https://salsa.debian.org/installer-team/partman-auto/tree/master/recipes-ppc64el

You want to add "$defaultignore{ }" for the "/boot" partition, so it's
only active by default for installations using LVM? Maybe there are also
other use cases for a separate "/boot" partition than LVM. Such a change
would make it harder for people with those other use cases then.

>
> 3. GRUB installs fine now on Sun partition tables without any further ado?

For my tested configuration - t5220, atomic recipe, single disk - yes.

>
> 4. And, just to be safe: Your previous mail contained hashes
>     before the partition layout lines (#). You did not actually
>     have those in your atomic file, correct?

Correct, those # marks were just for "markup" and not included in the
actual recipe I modified from the installer environment.

Frank

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

John Paul Adrian Glaubitz
On 05/16/2018 10:26 PM, Frank Scheiner wrote:
> On 05/16/2018 09:16 PM, John Paul Adrian Glaubitz wrote:
>> Ok, I just successfully tested this change.
>>
>> Two questions left:

I just realized I can't count to four :).

>> 1. Can you test whether we can drop the /boot partition on Sun
>>     partition tables?
>
> I actually don't remember if I tested such a configuration. At least d-i/grub-installer should support this now. I'll check that on my T5220 and report back.

Ok, thank you.

> Just a thought:
>
> Could it happen that - depending on the size of the root FS and the amount of packages installed - the GRUB image gets installed so far away from the start of a
> disk - it usually gets installed at the end of an installation - that older machines with older OBP versions could have problems to access it via block lists?
> Are there any such limits?
>
> If yes, a separate smaller partition for "/boot" might be required in such a case.

That's a very good heads-up. I completely forgot about block lists.

I agree, it could potentially cause problems. So, maybe we should keep
the /boot partition. But I think we can drop the "bootable" flag, can't
we?

>>
>> 2. Now that we can use GRUB, what about updating the default
>>     partitioning scheme? Maybe one that is inspired by the layout
>>     used on ppc64el:
>>
>>     > https://salsa.debian.org/installer-team/partman-auto/tree/master/recipes-ppc64el
>
> You want to add "$defaultignore{ }" for the "/boot" partition, so it's only active by default for installations using LVM? Maybe there are also other use cases
> for a separate "/boot" partition than LVM. Such a change would make it harder for people with those other use cases then.

Well, people can still do a complete manual partitioning, so we don't force
anything. I am also fully open to any further suggestions.

I just thought that - at least with GPT partitioning - we don't need a
separate /boot partition. Maybe we should add "$iflabel { sun }" for
the separate /boot partition. Would that work?

>> 3. GRUB installs fine now on Sun partition tables without any further ado?
>
> For my tested configuration - t5220, atomic recipe, single disk - yes.

Thanks for the confirmation.

>> 4. And, just to be safe: Your previous mail contained hashes
>>     before the partition layout lines (#). You did not actually
>>     have those in your atomic file, correct?
>
> Correct, those # marks were just for "markup" and not included in the actual recipe I modified from the installer environment.

Ok, thanks.

So, to summarize:

1. Can you test whether adding "$iflabel { sun }" creates the
   separate /boot partition for you?

2. Do you need "$bootable { }" for /boot to work with GRUB?

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

John Paul Adrian Glaubitz
On 05/16/2018 10:44 PM, John Paul Adrian Glaubitz wrote:
> So, to summarize:
>
> 1. Can you test whether adding "$iflabel { sun }" creates the
>    separate /boot partition for you?

FWIW, this does the right thing for me on GPT disk labels:

root@debian:~# parted /dev/vdiska
GNU Parted 3.2
Using /dev/vdiska
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: Unknown (unknown)
Disk /dev/vdiska: 17.2GB
Sector size (logical/physical): 512B/8192B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system     Name  Flags
 1      1049kB  10.5MB  9437kB                        bios_grub
 2      10.5MB  15.1GB  15.1GB  ext4
 3      15.1GB  17.2GB  2038MB  linux-swap(v1)

(parted)

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

Frank Scheiner
In reply to this post by John Paul Adrian Glaubitz
On 05/16/2018 08:53 PM, John Paul Adrian Glaubitz wrote:

> On 05/16/2018 08:15 PM, Frank Scheiner wrote:
>> Unfortunately booting the CDROM kernel and initrd via GRUB2 over the network did not work as expected by me:
>>
>> After the installer started it wants to load its components from the non-existing CDROM, of course... :-(
>>
>> There is no command line switch for the installer (kernel) that will make it behave like a netboot installer, right?
>
> This is expected. You cannot use the cdrom debian-installer initrd for netboot
> and vice versa. If you want to perform a netboot, you have to use the initrd
> for netboot which is inside the debian-installer-images tarball as found
> on the FTP servers in the pool-sparc64/main/d/debian-installer subdirectory.

Ah, good to know.

>
> Since the d-i current image is outdated, use the one from here:
>
>> https://people.debian.org/~glaubitz/debian-cd/debian-installer-images_20171204_sparc64.tar.gz
>
> Please note that even the "NETINST" images are still CD images. Real netboot
> images just require a kernel and initrd. The netboot initrd contains the
> necessary network drivers and is designed to fetch udebs over the internet.

I had a look at the netboot "boot.img". The problem is - or was - that I
didn't know how to extract the initrd from this TILO image.

In the meantime I found a way to extract the initrd from the TILO image.
But trying an installation with it and the CDROM kernel (which I assume
is identical to the kernel inside of the TILO image) showed some issues
(missing Debian ports keys, defaults to buster, debootstrap fails to
install initramfs-tools, etc.) that prevented a successful installation.

We could start a separate thread for these issues and we should maybe
also change from TILO image to just kernel and netboot initrd or only
netboot initrd (using the CDROM kernel in addition), as we now know how
to netboot these with GRUB.

So for testing I again used the ISO.

BTW although this ISO ([1]) already has d-i/silo-installer removed, the
d-i/grub-installer on it does not yet include the patch you committed on
May 15th. But one can fix that during installation.

[1]:
https://people.debian.org/~glaubitz/debian-cd/debian-10.0-sparc64-NETINST-1.iso

Frank

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

Frank Scheiner
In reply to this post by John Paul Adrian Glaubitz
On 05/16/2018 10:44 PM, John Paul Adrian Glaubitz wrote:
>>> 1. Can you test whether we can drop the /boot partition on Sun
>>>      partition tables?
>>
>> I actually don't remember if I tested such a configuration. At least d-i/grub-installer should support this now. I'll check that on my T5220 and report back.
>
> Ok, thank you.

Ok, after testing I can confirm that at least on my T5220 a separate
partition for `/boot` is not needed. I'll try to check what my oldest
UltraSPARC machines make out if it.

> So, to summarize:
>
> 1. Can you test whether adding "$iflabel { sun }" creates the
>     separate /boot partition for you?

Just tested this now and yes:

```
50 500 100 ext2

         $iflabel { sun }

         method{ format }

         format{ }

         use_filesystem{ }

         filesystem{ ext2 }

         mountpoint{ /boot } .
```

...creates the separate partition for `/boot` on my T5220 and:

>
> 2. Do you need "$bootable { }" for /boot to work with GRUB?

...no, the bootable flag seems to be unneeded by GRUB (or OBP), as the
system booted without an issue after the installation finished.

One thing to note is, that per default GRUB uses `quiet` in the kernel
command line, so depending on the speed of a machine and/or its disk
drives, it might at first look that nothing happens after GRUB has
loaded kernel and initrd although the kernel is actually booting.

Frank

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

John Paul Adrian Glaubitz
In reply to this post by Frank Scheiner
On 05/18/2018 10:15 PM, Frank Scheiner wrote:

>> Since the d-i current image is outdated, use the one from here:
>>
>>> https://people.debian.org/~glaubitz/debian-cd/debian-installer-images_20171204_sparc64.tar.gz
>>
>> Please note that even the "NETINST" images are still CD images. Real netboot
>> images just require a kernel and initrd. The netboot initrd contains the
>> necessary network drivers and is designed to fetch udebs over the internet.
>
> I had a look at the netboot "boot.img". The problem is - or was - that I didn't know how to extract the initrd from this TILO image.
>
> In the meantime I found a way to extract the initrd from the TILO image. But trying an installation with it and the CDROM kernel (which I assume is identical to
> the kernel inside of the TILO image) showed some issues (missing Debian ports keys, defaults to buster, debootstrap fails to install initramfs-tools, etc.) that
> prevented a successful installation.

Please search the debian-sparc mailing list regarding the netboot topic,
we have discussed netboot in the past. Maybe this discussion can shed
some light onto your questions:

> https://lists.debian.org/debian-sparc/2017/04/msg00002.html

> We could start a separate thread for these issues and we should maybe also change from TILO image to just kernel and netboot initrd or only netboot initrd
> (using the CDROM kernel in addition), as we now know how to netboot these with GRUB.

Yes, please.

> BTW although this ISO ([1]) already has d-i/silo-installer removed, the d-i/grub-installer on it does not yet include the patch you committed on May 15th. But
> one can fix that during installation.

Yes, I'm aware of that. That's because the uploads are usually performed
by Christian Perrier after he sees there have been commits to any of the
debian-installer components. This should be resolved within the next
days or weeks - hopefully rather days.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

John Paul Adrian Glaubitz
In reply to this post by Frank Scheiner
On 05/18/2018 10:16 PM, Frank Scheiner wrote:
>> Ok, thank you.
>
> Ok, after testing I can confirm that at least on my T5220 a separate partition for `/boot` is not needed. I'll try to check what my oldest UltraSPARC machines
> make out if it.

Yes, but as you correctly mentioned, without a separate /boot partition, there
is always the risk that the blocklist method fails if the GRUB files are put
too far at the end of the / filesystem. We should maybe forward this question
upstream.

>> So, to summarize:
>>
>> 1. Can you test whether adding "$iflabel { sun }" creates the
>>     separate /boot partition for you?
>
> Just tested this now and yes:
>
> ```
> 50 500 100 ext2
>         $iflabel { sun }
>         method{ format }
>         format{ }
>         use_filesystem{ }
>         filesystem{ ext2 }
>         mountpoint{ /boot } .
> ```
>
> ...creates the separate partition for `/boot` on my T5220 and:

And it disabled /boot for GPT disk labels, works as expected. Thus,
I have already committed the changes to partman-auto.

>> 2. Do you need "$bootable { }" for /boot to work with GRUB?
>
> ...no, the bootable flag seems to be unneeded by GRUB (or OBP), as the system booted without an issue after the installation finished.

Ok. This can probably just be considered a leftover from SILO that
we can get rid of then. But it also doesn't seem to hurt at the
moment.

> One thing to note is, that per default GRUB uses `quiet` in the kernel command line, so depending on the speed of a machine and/or its disk drives, it might at
> first look that nothing happens after GRUB has loaded kernel and initrd although the kernel is actually booting.

I *think* the GRUB parameters are set in finish-install, but
I am not sure.

See: https://salsa.debian.org/installer-team/finish-install/tree/master/finish-install.d

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

Frank Scheiner
In reply to this post by John Paul Adrian Glaubitz
On 05/19/2018 09:49 AM, John Paul Adrian Glaubitz wrote:
>> In the meantime I found a way to extract the initrd from the TILO image. But trying an installation with it and the CDROM kernel (which I assume is identical to
>> the kernel inside of the TILO image) showed some issues (missing Debian ports keys, defaults to buster, debootstrap fails to install initramfs-tools, etc.) that
>> prevented a successful installation.
>
> Please search the debian-sparc mailing list regarding the netboot topic,
> we have discussed netboot in the past. Maybe this discussion can shed
> some light onto your questions:
>
>> https://lists.debian.org/debian-sparc/2017/04/msg00002.html

Thanks for this hint, I believe I read this already in the past, but
then forgot about it and hence - maybe again - wrongly assumed that the
installer image is a TILO image where it is not.

Reply | Threaded
Open this post in threaded view
|

Re: Installation tests with GRUB on GPT disks

Frank Scheiner
In reply to this post by John Paul Adrian Glaubitz
On 05/19/2018 10:03 AM, John Paul Adrian Glaubitz wrote:

> On 05/18/2018 10:16 PM, Frank Scheiner wrote:
>>> Ok, thank you.
>>
>> Ok, after testing I can confirm that at least on my T5220 a separate partition for `/boot` is not needed. I'll try to check what my oldest UltraSPARC machines
>> make out if it.
>
> Yes, but as you correctly mentioned, without a separate /boot partition, there
> is always the risk that the blocklist method fails if the GRUB files are put
> too far at the end of the / filesystem. We should maybe forward this question
> upstream.

Yeah, I just thought about figuring out by experiment if there are any
limits on the older UltraSPARC machines available to me.

>> One thing to note is, that per default GRUB uses `quiet` in the kernel command line, so depending on the speed of a machine and/or its disk drives, it might at
>> first look that nothing happens after GRUB has loaded kernel and initrd although the kernel is actually booting.
>
> I *think* the GRUB parameters are set in finish-install, but
> I am not sure.
>
> See: https://salsa.debian.org/installer-team/finish-install/tree/master/finish-install.d

I had a look, but this seems to be defined elsewhere. I traced it back
to the grub-ieee1275 package which creates `/etc/default/grub` in its
`postinst` script. This includes the var `GRUB_CMDLINE_LINUX_DEFAULT`
which defaults to "quiet" as per [1] => [2] => [3].

[1]: https://salsa.debian.org/grub-team/grub/blob/master/debian/templates.in

[2]: https://salsa.debian.org/grub-team/grub/blob/master/debian/rules#L441

[3]: https://salsa.debian.org/grub-team/grub/blob/master/debian/rules#L85

Hm, I actually don't want to divert from the defaults - although maybe
the defaults sometimes might lead to irritations (no output for dozens
of seconds). Few people have that much patience. ;-)

Cheers,
Frank

12