Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Lucas Nussbaum-4
Package: apt-setup
Severity: normal

Hi,

We use d-i weekly images[1] to create Debian testing system images. Our
installation process is fully automated with preseeding.

The installation process hangs on the apt-setup/use_mirror question (see
screenshot). It did not with d-i alpha 4.

See attached log and preseeding file.

- Lucas

[1] https://cdimage.debian.org/cdimage/weekly-builds/amd64/iso-cd/debian-testing-amd64-netinst.iso

di-use-mirror.png (16K) Download Attachment
syslog (258K) Download Attachment
preseed.cfg (13K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Lucas Nussbaum-4
On 21/01/19 at 10:55 +0100, Lucas Nussbaum wrote:

> Package: apt-setup
> Severity: normal
>
> Hi,
>
> We use d-i weekly images[1] to create Debian testing system images. Our
> installation process is fully automated with preseeding.
>
> The installation process hangs on the apt-setup/use_mirror question (see
> screenshot). It did not with d-i alpha 4.
>
> See attached log and preseeding file.
>
> - Lucas

Hi,

After further investigations, it seems that the bug is related to:

> Jan 21 08:02:09 kernel: [    0.000000] Command line: BOOT_IMAGE=/install.amd/vmlinuz vga=788 initrd=/install.amd/initrd.gz --- quiet  preseed/url=http://10.0.2.2:8000/preseed.cfg auto locale=en_US kbd-chooser/method=us netcfg/get_hostname=localhost netcfg/get_domain=localdomain fb=false debconf/frontend=noninteractive console-setup/ask_detect=false console-keymaps-at/keymap=us keyboard-configuration/xkb-keymap=us

Which is a bit stupid: instead of passing many parameters on the
cmdline, we could just use 'auto url=....'

Still, there's a regression in d-i, when installing with
install preseed/url=https://blop.info/pub/preseed.cfg

With debian-buster-DI-alpha4-amd64-netinst.iso, the installer asks
questions about language, location, keyboard, hostname, and domainname,
but then proceeds until the end of the installation (which is expected).

With the current debian-testing-amd64-netinst.iso, the installer asks
the same questions, proceeds with the installation, but then, near the
end, asks the 'Use a network mirror?' question as described in this bug.

Lucas

Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Wolfgang Schweer-3
On Mon, Jan 21, 2019 at 10:50:31PM +0100, Lucas Nussbaum wrote:
> On 21/01/19 at 10:55 +0100, Lucas Nussbaum wrote:
> > Package: apt-setup
> > Severity: normal
[..]
> With debian-buster-DI-alpha4-amd64-netinst.iso, the installer asks
> questions about language, location, keyboard, hostname, and domainname,
> but then proceeds until the end of the installation (which is expected).
>
> With the current debian-testing-amd64-netinst.iso, the installer asks
> the same questions, proceeds with the installation, but then, near the
> end, asks the 'Use a network mirror?' question as described in this bug.

I'm pretty sure that this is due to the Debian Edu related changes to
debian-cd, see:
https://salsa.debian.org/images-team/debian-cd/commit/15b482d49e642e21e983dba27a47b4fc2d8b90b4 
We proposed/did those to be able to have a BD ISO image with COMPLETE=0
(for the image to be small enough) with 'bluray/not_complete' instead of
'not_complete' written to $CDDIR/.disk/cd_type (the default for the
COMPLETE=0 case). The patch had been taken from the former Debian Edu CD
build system; obviously some more thinking about the compliance with the
more general case would have been good, sorry.

I guess the following change could amend things:

diff --git a/tools/start_new_disc b/tools/start_new_disc
index 0b64a7e..83838d1 100755
--- a/tools/start_new_disc
+++ b/tools/start_new_disc
@@ -212,11 +212,7 @@ if [ $DISKNUM = 1 ] ; then
                 echo "bluray/not_complete" > $CDDIR/.disk/cd_type
             fi
         else
-            if [ "$MAXCDS"x = "1"x ]; then
-                echo "cd/single/not_complete" > $CDDIR/.disk/cd_type
-            else
-                echo "cd/not_complete" > $CDDIR/.disk/cd_type
-            fi
+            echo "not_complete" > $CDDIR/.disk/cd_type
         fi
     fi

Please check.

Wolfgang

signature.asc (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Steve McIntyre
Hi Wolfgang!

On Wed, Jan 23, 2019 at 11:07:29AM +0100, Wolfgang Schweer wrote:

>
>I'm pretty sure that this is due to the Debian Edu related changes to
>debian-cd, see:
>https://salsa.debian.org/images-team/debian-cd/commit/15b482d49e642e21e983dba27a47b4fc2d8b90b4 
>We proposed/did those to be able to have a BD ISO image with COMPLETE=0
>(for the image to be small enough) with 'bluray/not_complete' instead of
>'not_complete' written to $CDDIR/.disk/cd_type (the default for the
>COMPLETE=0 case). The patch had been taken from the former Debian Edu CD
>build system; obviously some more thinking about the compliance with the
>more general case would have been good, sorry.
>
>I guess the following change could amend things:
>
>diff --git a/tools/start_new_disc b/tools/start_new_disc
>index 0b64a7e..83838d1 100755
>--- a/tools/start_new_disc
>+++ b/tools/start_new_disc
>@@ -212,11 +212,7 @@ if [ $DISKNUM = 1 ] ; then
>                 echo "bluray/not_complete" > $CDDIR/.disk/cd_type
>             fi
>         else
>-            if [ "$MAXCDS"x = "1"x ]; then
>-                echo "cd/single/not_complete" > $CDDIR/.disk/cd_type
>-            else
>-                echo "cd/not_complete" > $CDDIR/.disk/cd_type
>-            fi
>+            echo "not_complete" > $CDDIR/.disk/cd_type
>         fi
>     fi
>
>Please check.

So I've been looking through this code again, and the corresponding
code in apt-setup that uses these values. Last time I played with
apt-setup, I added a table to describe what d-i will do based on the
information in cd_type, to explain exactly what d-i expects:

# Various different image types look different here:
#
# Image Type                       cd_type
###################################################
# netinst                          "not_complete"
# full CD sets (default desktop)   "full_cd"
# desktop-specific CD images       "full_cd/single"
# DVD                              "dvd"
# bluray                           "bluray"
# multi-arch CD/DVD                "not_complete"
# live                             "live"
#
# It can make sense to offer to scan more media here in most cases,
# but... on live or blu-ray it's unlikely to help; the
# desktop-specific image is designed specifically to work with only a
# single image. Hopefully the following makes sense.

The changes you've imported from the debian-edu fork of debian-cd
clearly don't match up with these, and that's a problem. So *for now*
I'm reverting them so we can build a d-i alpha 5 CD/DVD release that
we're already overdue for. Apologies for not getting back to you
earlier and reviewing the suggested changes back then. That's my fault
for being too busy and not looking at this code before it was merged.

We'll come back to this again shortly. To help with that, could you
describe exactly what debian-edu is expecting here please, i.e. what
the settings in cd_type mean for the debian-edu installer? I'm worried
that we may not have a clear solution here that can match the current
expectations of both d-i and and the debian-edu setup.

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
Who needs computer imagery when you've got Brian Blessed?

Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Wolfgang Schweer-3
Hi Steve,

On Fri, Jan 25, 2019 at 11:58:08PM +0000, Steve McIntyre wrote:

> So I've been looking through this code again, and the corresponding
> code in apt-setup that uses these values. Last time I played with
> apt-setup, I added a table to describe what d-i will do based on the
> information in cd_type, to explain exactly what d-i expects:
>
> # Various different image types look different here:
> #
> # Image Type                       cd_type
> ###################################################
> # netinst                          "not_complete"
> # full CD sets (default desktop)   "full_cd"
> # desktop-specific CD images       "full_cd/single"
> # DVD                              "dvd"
> # bluray                           "bluray"
> # multi-arch CD/DVD                "not_complete"
> # live                             "live"
Thanks for checking the code and providing this table.

The Debian Edu BD image (bluray) is generated using COMPLETE=0, see:
https://salsa.debian.org/images-team/setup/blob/master/buster/cronjob.weekly#L268 
because otherwise it turned out to be too big (~21 GiB).
This image is intended for offline installations, no mirror useable.

With this setting the Edu image differs from the stock BD/DLBD ones
where COMPLETE=0 is missing and the default (COMPLETE=1) takes effect,
see:
https://salsa.debian.org/images-team/setup/blob/master/buster/cronjob.weekly#L169

> The changes you've imported from the debian-edu fork of debian-cd
> clearly don't match up with these, and that's a problem.

Agreed; sorry for that.

> We'll come back to this again shortly. To help with that, could you
> describe exactly what debian-edu is expecting here please, i.e. what
> the settings in cd_type mean for the debian-edu installer?

I tried to understand the code that is used to write the content, see:
https://salsa.debian.org/images-team/debian-cd/blob/master/tools/start_new_disc#L179

If I understood correctly, for all cases with COMPLETE=0 the content of
cd_type is 'not_complete'.

If the EDU BD image has 'blueray/not_complete' then this content matches
the blueray*) case in apt-setup, see:
https://sources.debian.org/src/apt-setup/1:0.145/generators/50mirror/#L104
and the image is usable for offline installation.
This is the only change that is actually needed for the Edu BD image.

Commit
https://salsa.debian.org/images-team/debian-cd/commit/15b482d49e642e21e983dba27a47b4fc2d8b90b4
incorrectly altered the setting 'not_complete' to 'cd/not_complete' for
netinst, causing this bug. Sorry for not noticing it.
 
> I'm worried that we may not have a clear solution here that can match
> the current expectations of both d-i and and the debian-edu setup.

Hopefully I managed to clarify the Edu setup intention.

Wolfgang

signature.asc (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Wolfgang Schweer-3
On Sat, Jan 26, 2019 at 11:24:40AM +0100, Wolfgang Schweer wrote:
> On Fri, Jan 25, 2019 at 11:58:08PM +0000, Steve McIntyre wrote:
> > We'll come back to this again shortly. To help with that, could you
> > describe exactly what debian-edu is expecting here please, i.e. what
> > the settings in cd_type mean for the debian-edu installer?
[..]
> If the EDU BD image has 'blueray/not_complete' then this content matches
> the blueray*) case in apt-setup, see:
> https://sources.debian.org/src/apt-setup/1:0.145/generators/50mirror/#L104
> and the image is usable for offline installation.
> This is the only change that is actually needed for the Edu BD image.
[..]
> > I'm worried that we may not have a clear solution here that can match
> > the current expectations of both d-i and and the debian-edu setup.

Please check if the attached patch would be a solution.

Wolfgang

0001-Set-adjusted-cd_type-for-incomplete-BD-image.patch (1K) Download Attachment
signature.asc (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Steve McIntyre
In reply to this post by Wolfgang Schweer-3
[ Dropping Lucas from CC here... ]

Hi Wolfgang,

Finally getting back to this - it's been a busy couple of weeks...!

On Sat, Jan 26, 2019 at 11:24:40AM +0100, Wolfgang Schweer wrote:

>Hi Steve,
>
>On Fri, Jan 25, 2019 at 11:58:08PM +0000, Steve McIntyre wrote:
>> So I've been looking through this code again, and the corresponding
>> code in apt-setup that uses these values. Last time I played with
>> apt-setup, I added a table to describe what d-i will do based on the
>> information in cd_type, to explain exactly what d-i expects:
>>
>> # Various different image types look different here:
>> #
>> # Image Type                       cd_type
>> ###################################################
>> # netinst                          "not_complete"
>> # full CD sets (default desktop)   "full_cd"
>> # desktop-specific CD images       "full_cd/single"
>> # DVD                              "dvd"
>> # bluray                           "bluray"
>> # multi-arch CD/DVD                "not_complete"
>> # live                             "live"
>
>Thanks for checking the code and providing this table.
>
>The Debian Edu BD image (bluray) is generated using COMPLETE=0, see:
>https://salsa.debian.org/images-team/setup/blob/master/buster/cronjob.weekly#L268 
>because otherwise it turned out to be too big (~21 GiB).
>This image is intended for offline installations, no mirror useable.
>
>With this setting the Edu image differs from the stock BD/DLBD ones
>where COMPLETE=0 is missing and the default (COMPLETE=1) takes effect,
>see:
>https://salsa.debian.org/images-team/setup/blob/master/buster/cronjob.weekly#L169

Right.

>> The changes you've imported from the debian-edu fork of debian-cd
>> clearly don't match up with these, and that's a problem.
>
>Agreed; sorry for that.
>
>> We'll come back to this again shortly. To help with that, could you
>> describe exactly what debian-edu is expecting here please, i.e. what
>> the settings in cd_type mean for the debian-edu installer?
>
>I tried to understand the code that is used to write the content, see:
>https://salsa.debian.org/images-team/debian-cd/blob/master/tools/start_new_disc#L179
>
>If I understood correctly, for all cases with COMPLETE=0 the content of
>cd_type is 'not_complete'.

Correct.

>If the EDU BD image has 'blueray/not_complete' then this content matches
>the blueray*) case in apt-setup, see:
>https://sources.debian.org/src/apt-setup/1:0.145/generators/50mirror/#L104
>and the image is usable for offline installation.
>This is the only change that is actually needed for the Edu BD image.

Right - that's the information I was looking for. :-) If the Edu
images are doing nothing special (in terms of Edu setup) then that
makes life easier for me!

>Commit
>https://salsa.debian.org/images-team/debian-cd/commit/15b482d49e642e21e983dba27a47b4fc2d8b90b4
>incorrectly altered the setting 'not_complete' to 'cd/not_complete' for
>netinst, causing this bug. Sorry for not noticing it.
>  
>> I'm worried that we may not have a clear solution here that can match
>> the current expectations of both d-i and and the debian-edu setup.
>
>Hopefully I managed to clarify the Edu setup intention.

Definitely, thanks!

With a type of bluray/not_complete, we'll then get the following
behaviour from the current apt-setup code:

 * 41cdset will ask if you want to scan more media (the multi-line
   "if" code around L60 will *not* match)

 * 50mirror will *not* ask about using a mirror, as you say above
   (will match "bluray*" in L104

Is that what you're looking for? If not, we can tweak further to get
the right behaviour. To be fair, the interfaces here are not great,
and well overdue for some changes to make things clearer...

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
"C++ ate my sanity" -- Jon Rabone

Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Wolfgang Schweer-3
Hi Steve,

On Mon, Feb 11, 2019 at 02:27:47AM +0000, Steve McIntyre wrote:
> With a type of bluray/not_complete, we'll then get the following
> behaviour from the current apt-setup code:
>
>  * 41cdset will ask if you want to scan more media (the multi-line
>    "if" code around L60 will *not* match)

Thanks for pointing this out.
 
>  * 50mirror will *not* ask about using a mirror, as you say above
>    (will match "bluray*" in L104
>
> Is that what you're looking for?

Yes. And I guess Debian Edu could get around the scan issuue using this
additional preseeding:

apt-cdrom-setup apt-setup/cdrom/set-double boolean true
apt-cdrom-setup apt-setup/cdrom/set-failed boolean true
apt-cdrom-setup apt-setup/cdrom/set-first boolean false
apt-cdrom-setup apt-setup/cdrom/set-next boolean false

> If not, we can tweak further to get the right behaviour.

Good to know.

> To be fair, the interfaces here are not great, and well overdue for
> some changes to make things clearer...

Well, that's how things tend to be sometimes.

Thanks a lot for your work!

Wolfgang

signature.asc (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Steve McIntyre
On Mon, Feb 11, 2019 at 01:32:58PM +0100, Wolfgang Schweer wrote:

>Hi Steve,
>
>On Mon, Feb 11, 2019 at 02:27:47AM +0000, Steve McIntyre wrote:
>> With a type of bluray/not_complete, we'll then get the following
>> behaviour from the current apt-setup code:
>>
>>  * 41cdset will ask if you want to scan more media (the multi-line
>>    "if" code around L60 will *not* match)
>
>Thanks for pointing this out.
>
>>  * 50mirror will *not* ask about using a mirror, as you say above
>>    (will match "bluray*" in L104
>>
>> Is that what you're looking for?
>
>Yes. And I guess Debian Edu could get around the scan issuue using this
>additional preseeding:
>
>apt-cdrom-setup apt-setup/cdrom/set-double boolean true
>apt-cdrom-setup apt-setup/cdrom/set-failed boolean true
>apt-cdrom-setup apt-setup/cdrom/set-first boolean false
>apt-cdrom-setup apt-setup/cdrom/set-next boolean false

If you like, or we can tweak the setup in 41cdset to deal with
bluray/not_complete too. That's an easy change to make.

>> If not, we can tweak further to get the right behaviour.
>
>Good to know.
>
>> To be fair, the interfaces here are not great, and well overdue for
>> some changes to make things clearer...
>
>Well, that's how things tend to be sometimes.
>
>Thanks a lot for your work!

No problem, sorry for keeping you waiting on this.

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
"This dress doesn't reverse." -- Alden Spiess

Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Wolfgang Schweer-3
On Mon, Feb 11, 2019 at 02:06:33PM +0000, Steve McIntyre wrote:

> >Yes. And I guess Debian Edu could get around the scan issuue using this
> >additional preseeding:
> >
> >apt-cdrom-setup apt-setup/cdrom/set-double boolean true
> >apt-cdrom-setup apt-setup/cdrom/set-failed boolean true
> >apt-cdrom-setup apt-setup/cdrom/set-first boolean false
> >apt-cdrom-setup apt-setup/cdrom/set-next boolean false
>
> If you like, or we can tweak the setup in 41cdset to deal with
> bluray/not_complete too. That's an easy change to make.
 
Ok, that is a more general solution. Please do so if time allows.

Wolfgang

signature.asc (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Steve McIntyre
On Mon, Feb 11, 2019 at 03:34:24PM +0100, Wolfgang Schweer wrote:

>On Mon, Feb 11, 2019 at 02:06:33PM +0000, Steve McIntyre wrote:
>> >Yes. And I guess Debian Edu could get around the scan issuue using this
>> >additional preseeding:
>> >
>> >apt-cdrom-setup apt-setup/cdrom/set-double boolean true
>> >apt-cdrom-setup apt-setup/cdrom/set-failed boolean true
>> >apt-cdrom-setup apt-setup/cdrom/set-first boolean false
>> >apt-cdrom-setup apt-setup/cdrom/set-next boolean false
>>
>> If you like, or we can tweak the setup in 41cdset to deal with
>> bluray/not_complete too. That's an easy change to make.
>
>Ok, that is a more general solution. Please do so if time allows.

Done. I've just pushed your simple debian-cd change and uploaded a new
apt-setup package to match.

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
"I suspect most samba developers are already technically insane... Of
 course, since many of them are Australians, you can't tell." -- Linus Torvalds

Reply | Threaded
Open this post in threaded view
|

Bug#919982: apt-setup: preseeded installation hangs at "Use a network mirror?"

Wolfgang Schweer-3
Hi Steve,

On Mon, Feb 11, 2019 at 05:32:33PM +0000, Steve McIntyre wrote:
> Done. I've just pushed your simple debian-cd change and uploaded a new
> apt-setup package to match.
 
Great! Thank you very much.

Wolfgang

signature.asc (981 bytes) Download Attachment