Grub, UEFI Secure Boot and netboot - help!

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Grub, UEFI Secure Boot and netboot - help!

Steve McIntyre
Hey folks,

We have a bit of a problem with PXE booting Grub in Buster, as shown
in #928750:

 * On all supported arches, we *used* to generate a Grub netboot image
   inside d-i, with a prefix setting of
   "debian-installer/$arch/grub". The prefix is important, as it's how
   Grub finds its config file, modules etc that it loads. Things are
   still like this in Stretch.

 * In Buster, we can no longer do this on arches which support Secure
   Boot. To keep the SB signature, we now re-use the existing signed
   binaries that have come directly from the Grub build (and the
   Debian signing infrastructure). There's just one minor problem with
   this - this means that we're stuck with the hard-coded prefix baked
   into the grubnetXXX.efi binary. This is currently set to "/grub",
   and this means that to get a functional amd64 (say) PXE
   installation working the user has to add a "/grub" symlink on their
   TFTP server, something like:

     /grub -> /debian-installer/amd64/grub

   (assuming that /debian-installer is the root of the netboot tree).

I can see a couple of options here, but I'm not sure either of them
are good. Comments would be most welcome!

  1. Update the docs to mention this - this is a new thing needed to
     get netboot working with Buster. It's *currently* inconsistent,
     as ia64 and armhf (as non-SB arches) are still using the old
     prefix setting. For the sake of consistency (in docs etc.), I
     propose to also update the d-i build for those arches to use the
     same prefix. But I acknowledge that will break existing
     setups. :-(

  2. Alternatively, we could tweak the netboot prefix setting as built
     by grub. I'm worried that this may also break things for some
     users. Do we assume (can we?) that all our grub netboot users are
     installer users (so we could use /debian-installer/$ARCH/grub)?
     If so, that might be a way to go. But is it a valid assumption?
     We'd be forcing all our grub netboot binaries to only sensibly
     work for d-i, and that worries me too.

Any other suggestions on what we could do? Let me know what you
think...

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
"The problem with defending the purity of the English language is that
 English is about as pure as a cribhouse whore. We don't just borrow words; on
 occasion, English has pursued other languages down alleyways to beat them
 unconscious and rifle their pockets for new vocabulary."  -- James D. Nicoll

Reply | Threaded
Open this post in threaded view
|

Re: Grub, UEFI Secure Boot and netboot - help!

Domenico Andreoli-3
On Mon, Jun 10, 2019 at 03:37:41AM +0100, Steve McIntyre wrote:
> Hey folks,

Hi,

>
> We have a bit of a problem with PXE booting Grub in Buster, as shown
> in #928750:
>
>  * On all supported arches, we *used* to generate a Grub netboot image
>    inside d-i, with a prefix setting of
>    "debian-installer/$arch/grub". The prefix is important, as it's how
>    Grub finds its config file, modules etc that it loads. Things are
>    still like this in Stretch.
>
>  * In Buster, we can no longer do this on arches which support Secure
>    Boot. To keep the SB signature, we now re-use the existing signed
>    binaries that have come directly from the Grub build (and the
>    Debian signing infrastructure). There's just one minor problem with
>    this - this means that we're stuck with the hard-coded prefix baked
>    into the grubnetXXX.efi binary. This is currently set to "/grub",
>    and this means that to get a functional amd64 (say) PXE
>    installation working the user has to add a "/grub" symlink on their
>    TFTP server, something like:
>
>      /grub -> /debian-installer/amd64/grub
>
>    (assuming that /debian-installer is the root of the netboot tree).
>
> I can see a couple of options here, but I'm not sure either of them
> are good. Comments would be most welcome!
>
>   1. Update the docs to mention this - this is a new thing needed to
>      get netboot working with Buster. It's *currently* inconsistent,
>      as ia64 and armhf (as non-SB arches) are still using the old
>      prefix setting. For the sake of consistency (in docs etc.), I
>      propose to also update the d-i build for those arches to use the
>      same prefix. But I acknowledge that will break existing
>      setups. :-(
These are two options:

  1a. docs update (does not break anything)
  1b. d-i build update (might break something)

>
>   2. Alternatively, we could tweak the netboot prefix setting as built
>      by grub. I'm worried that this may also break things for some
>      users. Do we assume (can we?) that all our grub netboot users are
>      installer users (so we could use /debian-installer/$ARCH/grub)?
>      If so, that might be a way to go. But is it a valid assumption?
>      We'd be forcing all our grub netboot binaries to only sensibly
>      work for d-i, and that worries me too.
>
> Any other suggestions on what we could do? Let me know what you
> think...
>
Is this question for Buster or Buster+x?  Those solutions requiring a
change in grub (and a new signature from M$) are not be ok for Buster.

For Buster I find acceptable only 1a. I prefer a sub-optimal solution
now to an improved solution later, especially under release.

For Buster+x, is it possible to make grub search multiple prefixes?

Regards,
Domenico

--
3B10 0CA1 8674 ACBA B4FE  FCD2 CE5B CF17 9960 DE13

signature.asc (321 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Grub, UEFI Secure Boot and netboot - help!

Domenico Andreoli-3
I apologize for the signature mess, I removed the offending key resulting
from some sbuild setup I attempted some time ago.

Dom

--
3B10 0CA1 8674 ACBA B4FE  FCD2 CE5B CF17 9960 DE13

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Grub, UEFI Secure Boot and netboot - help!

John Paul Adrian Glaubitz
In reply to this post by Steve McIntyre
Hi Steve!

On 6/10/19 4:37 AM, Steve McIntyre wrote:

>   1. Update the docs to mention this - this is a new thing needed to
>      get netboot working with Buster. It's *currently* inconsistent,
>      as ia64 and armhf (as non-SB arches) are still using the old
>      prefix setting. For the sake of consistency (in docs etc.), I
>      propose to also update the d-i build for those arches to use the
>      same prefix. But I acknowledge that will break existing
>      setups. :-(
>
>   2. Alternatively, we could tweak the netboot prefix setting as built
>      by grub. I'm worried that this may also break things for some
>      users. Do we assume (can we?) that all our grub netboot users are
>      installer users (so we could use /debian-installer/$ARCH/grub)?
>      If so, that might be a way to go. But is it a valid assumption?
>      We'd be forcing all our grub netboot binaries to only sensibly
>      work for d-i, and that worries me too.

I don't have a strong opinion here regarding GRUB on ia64. netboot with GRUB
on ia64 is currently untested on ia64, so I'm fine with breaking changes as
long as the configuration is kept in sync - where possible - with the other
EFI architectures and in a working state.

Thanks,
Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: Grub, UEFI Secure Boot and netboot - help!

Ian Campbell-5
In reply to this post by Steve McIntyre
On Mon, 2019-06-10 at 03:37 +0100, Steve McIntyre wrote:
> Any other suggestions on what we could do? Let me know what you
> think...

Is signing an extra, d-i specific, grubnetXX.efi image out of the
question?

Is the hard coded prefix a single prefix or is there a possibility of
searching a list? It's been a long time since I've played with any of
this but I have a vague recollection of once upon a time using (or
trying to use, maybe I'm remembering a failed experiment) a memdisk
(builtin to the grub image) containing an initial config file which
then was a bit more flexible about chaining to the next thing. I can't
find any evidence of that setup in any of the places I thought it might
be related to though :-/

Ian.

Reply | Threaded
Open this post in threaded view
|

Re: Grub, UEFI Secure Boot and netboot - help!

Steve McIntyre
In reply to this post by Domenico Andreoli-3
On Mon, Jun 10, 2019 at 11:08:35AM +0200, Domenico Andreoli wrote:
>On Mon, Jun 10, 2019 at 03:37:41AM +0100, Steve McIntyre wrote:

...

>> I can see a couple of options here, but I'm not sure either of them
>> are good. Comments would be most welcome!
>>
>>   1. Update the docs to mention this - this is a new thing needed to
>>      get netboot working with Buster. It's *currently* inconsistent,
>>      as ia64 and armhf (as non-SB arches) are still using the old
>>      prefix setting. For the sake of consistency (in docs etc.), I
>>      propose to also update the d-i build for those arches to use the
>>      same prefix. But I acknowledge that will break existing
>>      setups. :-(
>
>These are two options:
>
>  1a. docs update (does not break anything)
>  1b. d-i build update (might break something)
>>
>>   2. Alternatively, we could tweak the netboot prefix setting as built
>>      by grub. I'm worried that this may also break things for some
>>      users. Do we assume (can we?) that all our grub netboot users are
>>      installer users (so we could use /debian-installer/$ARCH/grub)?
>>      If so, that might be a way to go. But is it a valid assumption?
>>      We'd be forcing all our grub netboot binaries to only sensibly
>>      work for d-i, and that worries me too.
>>
>> Any other suggestions on what we could do? Let me know what you
>> think...
>
>Is this question for Buster or Buster+x?  Those solutions requiring a
>change in grub (and a new signature from M$) are not be ok for Buster.

This is definitely for Buster. Changes in *Grub* are fine - we sign
that ourselves. It's updated versions of Shim that are problematic.

>For Buster I find acceptable only 1a. I prefer a sub-optimal solution
>now to an improved solution later, especially under release.
>
>For Buster+x, is it possible to make grub search multiple prefixes?

I think it would be quite a big change...

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
  Armed with "Valor": "Centurion" represents quality of Discipline,
  Honor, Integrity and Loyalty. Now you don't have to be a Caesar to
  concord the digital world while feeling safe and proud.

Reply | Threaded
Open this post in threaded view
|

Re: Grub, UEFI Secure Boot and netboot - help!

Steve McIntyre
In reply to this post by John Paul Adrian Glaubitz
On Mon, Jun 10, 2019 at 11:17:52AM +0200, John Paul Adrian Glaubitz wrote:

>Hi Steve!
>
>On 6/10/19 4:37 AM, Steve McIntyre wrote:
>>   1. Update the docs to mention this - this is a new thing needed to
>>      get netboot working with Buster. It's *currently* inconsistent,
>>      as ia64 and armhf (as non-SB arches) are still using the old
>>      prefix setting. For the sake of consistency (in docs etc.), I
>>      propose to also update the d-i build for those arches to use the
>>      same prefix. But I acknowledge that will break existing
>>      setups. :-(
>>
>>   2. Alternatively, we could tweak the netboot prefix setting as built
>>      by grub. I'm worried that this may also break things for some
>>      users. Do we assume (can we?) that all our grub netboot users are
>>      installer users (so we could use /debian-installer/$ARCH/grub)?
>>      If so, that might be a way to go. But is it a valid assumption?
>>      We'd be forcing all our grub netboot binaries to only sensibly
>>      work for d-i, and that worries me too.
>
>I don't have a strong opinion here regarding GRUB on ia64. netboot with GRUB
>on ia64 is currently untested on ia64, so I'm fine with breaking changes as
>long as the configuration is kept in sync - where possible - with the other
>EFI architectures and in a working state.

ACK, thanks for the quick response. It was specifically ia64 that I
was worried about here in (1) above, hence my deliberate CC. armhf in
Debian has only very recently gained UEFI support, so there's no
release history to worry about.

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
"This dress doesn't reverse." -- Alden Spiess

Reply | Threaded
Open this post in threaded view
|

Re: Grub, UEFI Secure Boot and netboot - help!

Steve McIntyre
In reply to this post by Ian Campbell-5
Hi Ian,

On Mon, Jun 10, 2019 at 02:32:23PM +0100, Ian Campbell wrote:
>On Mon, 2019-06-10 at 03:37 +0100, Steve McIntyre wrote:
>> Any other suggestions on what we could do? Let me know what you
>> think...
>
>Is signing an extra, d-i specific, grubnetXX.efi image out of the
>question?

Oh, that's a thought. Possibly, I guess. Minor tweaks to the grub
packaging and to the d-i build. That's probably the easiest route, in
fact! Colin - how does that sound for you?

>Is the hard coded prefix a single prefix or is there a possibility of
>searching a list?

The prefix variable is very much a single value, yes. It's used and
dereferenced all over the place inside grub as a single const char
*. Not something I'm about to play with!

>It's been a long time since I've played with any of this but I have a
>vague recollection of once upon a time using (or trying to use, maybe
>I'm remembering a failed experiment) a memdisk (builtin to the grub
>image) containing an initial config file which then was a bit more
>flexible about chaining to the next thing. I can't find any evidence
>of that setup in any of the places I thought it might be related to
>though :-/

Right. That's how various other things work - it's how we control
things for CD boot, for example. But network boot is a slightly
different configuration. It's simply a single binary rather than a FAT
image containing the binary and config.

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
Who needs computer imagery when you've got Brian Blessed?

Reply | Threaded
Open this post in threaded view
|

Re: Grub, UEFI Secure Boot and netboot - help!

Colin Watson
On Mon, Jun 10, 2019 at 04:15:22PM +0100, Steve McIntyre wrote:

> On Mon, Jun 10, 2019 at 02:32:23PM +0100, Ian Campbell wrote:
> >On Mon, 2019-06-10 at 03:37 +0100, Steve McIntyre wrote:
> >> Any other suggestions on what we could do? Let me know what you
> >> think...
> >
> >Is signing an extra, d-i specific, grubnetXX.efi image out of the
> >question?
>
> Oh, that's a thought. Possibly, I guess. Minor tweaks to the grub
> packaging and to the d-i build. That's probably the easiest route, in
> fact! Colin - how does that sound for you?
>
> >Is the hard coded prefix a single prefix or is there a possibility of
> >searching a list?
>
> The prefix variable is very much a single value, yes. It's used and
> dereferenced all over the place inside grub as a single const char
> *. Not something I'm about to play with!

It'd be totally possible to make the grubnet memdisk try multiple
prefixes: there's already a memdisk that looks in a couple of locations
under $prefix for grub.cfg, so it could separate it into a list and try
each one until it finds something usable.

However, that approach embeds the debian-installer path even in images
that have nothing to do with d-i; so I think on balance I prefer the
idea of adding another d-i-specific image.  Steve is working on that
now.

> >It's been a long time since I've played with any of this but I have a
> >vague recollection of once upon a time using (or trying to use, maybe
> >I'm remembering a failed experiment) a memdisk (builtin to the grub
> >image) containing an initial config file which then was a bit more
> >flexible about chaining to the next thing. I can't find any evidence
> >of that setup in any of the places I thought it might be related to
> >though :-/
>
> Right. That's how various other things work - it's how we control
> things for CD boot, for example. But network boot is a slightly
> different configuration. It's simply a single binary rather than a FAT
> image containing the binary and config.

Moot given the above, but this isn't actually true - search for
grub-netboot.cfg and memdisk-netboot.fat in build-efi-images.

--
Colin Watson                                       [[hidden email]]

Reply | Threaded
Open this post in threaded view
|

Re: Grub, UEFI Secure Boot and netboot - help!

Steve McIntyre
On Mon, Jun 10, 2019 at 04:35:33PM +0100, Colin Watson wrote:

>On Mon, Jun 10, 2019 at 04:15:22PM +0100, Steve McIntyre wrote:
>>
>> The prefix variable is very much a single value, yes. It's used and
>> dereferenced all over the place inside grub as a single const char
>> *. Not something I'm about to play with!
>
>It'd be totally possible to make the grubnet memdisk try multiple
>prefixes: there's already a memdisk that looks in a couple of locations
>under $prefix for grub.cfg, so it could separate it into a list and try
>each one until it finds something usable.
>
>However, that approach embeds the debian-installer path even in images
>that have nothing to do with d-i; so I think on balance I prefer the
>idea of adding another d-i-specific image.  Steve is working on that
>now.

Yup. Simple changes. \o/

>> >It's been a long time since I've played with any of this but I have a
>> >vague recollection of once upon a time using (or trying to use, maybe
>> >I'm remembering a failed experiment) a memdisk (builtin to the grub
>> >image) containing an initial config file which then was a bit more
>> >flexible about chaining to the next thing. I can't find any evidence
>> >of that setup in any of the places I thought it might be related to
>> >though :-/
>>
>> Right. That's how various other things work - it's how we control
>> things for CD boot, for example. But network boot is a slightly
>> different configuration. It's simply a single binary rather than a FAT
>> image containing the binary and config.
>
>Moot given the above, but this isn't actually true - search for
>grub-netboot.cfg and memdisk-netboot.fat in build-efi-images.

I sit corrected. :-)

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
"Further comment on how I feel about IBM will appear once I've worked out
 whether they're being malicious or incompetent. Capital letters are forecast."
 Matthew Garrett, http://www.livejournal.com/users/mjg59/30675.html