Multiple console support

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Multiple console support

Wookey-6
Arm64 (arm in general in fact) has a rather fundamental problem with
D-I, which is that both serial and display are sensible default
devices for the installer to run on. Which is 'correct' depends very
much on the hardware and the circumstances. You may be installing a
server in rack, or a dev board with no display, in which case serial
is ideal, or you may have a chromebook or an ARM desktop machine with
a screen plugged in and no easy access to the serial console.

This problem doesn't arise on x86 where there is 'always' a screen (or
some BIOs magic to reflect what would be on the screen to serial).

Steve (McIntyre) and I have been thinking about what to do about this,
so did some investigation and came up with a plan. Essentially it was
to run d-i on both if they are configured/available. This way anyone
looking at just one or the other will see D-I as they expect. The
patch is not intrusive and essentially nothing changes if there is
only one console so this should be a low-risk change.

There are some further questions about whether we enable just
specified consoles (from kernel command line), or all
available+enabled consoles, but we can mess with that once the basic
support is in.

So far I have done a proof-of-concept hack and demonstrated that
running two instances does in fact work nicely without anything
obvious breaking. The console selection still needs some work/checking
(I've run out of time for that tonight, but it can easily be fished
out of the kernel command line args. (or we could get fancier).

The changes are in reopen-console (in rootskel), which runs the
initial d-i menu via an inittab entry, using steal-ctty to make sure
file handles and controlling tty are set up correctly.

Essentially all that has to happen is run extra copies on the other console:

--------------
  echo /dev/$console > /var/run/console-device
        echo /dev/$extraconsole > /var/run/extraconsole-device
 fi
 
# Some other session may have console as ctty. Steal it from them
# Run D-I on other console if one given
if [ -n $extraconsole ]; then
    ( exec /sbin/steal-ctty $(cat /var/run/extraconsole-device) "$@" & )
fi
exec /sbin/steal-ctty $(cat /var/run/console) "$@"
-----------
(attached as rootskel-multiple-consoles.patch)

The only other place this affects is
packages/finish-install.d/90console which reads
/var/run/console-device when tidying up at the end in order to write
inittab entries for the used console device (serial, xen, etc).

We also added some error-checking to steal-ctty.c which it might be
smart to include, because it currently segfaults if you don't give it
the right number of parameters, and silently fails if you run it in a
context where the calling process is not a session-master.

That's in steal-ctty-errorcheck.patch

Wookey
--
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

steal-ctty-errorcheck.patch (722 bytes) Download Attachment
rootskel-multiple-consoles.patch (947 bytes) Download Attachment
signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

Steve McIntyre
[ adding a CC to the debian-arm list - review there welcome too... ]

On Sat, Jan 19, 2019 at 04:27:05AM +0000, Wookey wrote:
>Arm64 (arm in general in fact) has a rather fundamental problem with
>D-I, which is that both serial and display are sensible default
>devices for the installer to run on. Which is 'correct' depends very
>much on the hardware and the circumstances. You may be installing a
>server in rack, or a dev board with no display, in which case serial
>is ideal, or you may have a chromebook or an ARM desktop machine with
>a screen plugged in and no easy access to the serial console.

ACK. There are also extra complications to confuse the issue. A Cavium
ThunderX "server" machine can also make quite a reasonable workstation
- just put it in a different case with a PCIe video card and USB
peripherals. People are selling these today! But the motherboard
firmware will tell the kernel to use ttyAMA0 as the default console
using ACPI (the SPCR table).

>This problem doesn't arise on x86 where there is 'always' a screen (or
>some BIOs magic to reflect what would be on the screen to serial).
>
>Steve (McIntyre) and I have been thinking about what to do about this,
>so did some investigation and came up with a plan. Essentially it was
>to run d-i on both if they are configured/available. This way anyone
>looking at just one or the other will see D-I as they expect. The
>patch is not intrusive and essentially nothing changes if there is
>only one console so this should be a low-risk change.
>
>There are some further questions about whether we enable just
>specified consoles (from kernel command line), or all
>available+enabled consoles, but we can mess with that once the basic
>support is in.

Yup. I saw this working yesterday - yay!

>So far I have done a proof-of-concept hack and demonstrated that
>running two instances does in fact work nicely without anything
>obvious breaking. The console selection still needs some work/checking
>(I've run out of time for that tonight, but it can easily be fished
>out of the kernel command line args. (or we could get fancier).

Nod. Potentially we might end up of running on multiple
consoles. Exactly how we define the set is not yet decided. Two
possible options I've thought of:

 * add lots more console= options to the grub.cfg for arm64 (to cover
   all the possibilities), then let d-i startup work out which devices
   exist from /proc/cmdlinge and spawn d-i on each.

   (This will look slightly messy if people look at grub.cfg, but it
    does allow for easy user-controllable changes to add/remove/change
    the settings.)

 * add that arch-specific list into d-i somwhere such that
   reopen-console-linux can grab it and use it, similarly.

   (slightly cleaner, but harder to over-ride. Maybe a hybrid is best
   - add a default list here and *also* parse /proc/cmdline for
   extras?)

>The changes are in reopen-console (in rootskel), which runs the
>initial d-i menu via an inittab entry, using steal-ctty to make sure
>file handles and controlling tty are set up correctly.
>
>Essentially all that has to happen is run extra copies on the other console:
>
>--------------
> echo /dev/$console > /var/run/console-device
> echo /dev/$extraconsole > /var/run/extraconsole-device
> fi
>
># Some other session may have console as ctty. Steal it from them
># Run D-I on other console if one given
>if [ -n $extraconsole ]; then
>    ( exec /sbin/steal-ctty $(cat /var/run/extraconsole-device) "$@" & )
>fi
>exec /sbin/steal-ctty $(cat /var/run/console) "$@"
>-----------
>(attached as rootskel-multiple-consoles.patch)
>
>The only other place this affects is
>packages/finish-install.d/90console which reads
>/var/run/console-device when tidying up at the end in order to write
>inittab entries for the used console device (serial, xen, etc).
>
>We also added some error-checking to steal-ctty.c which it might be
>smart to include, because it currently segfaults if you don't give it
>the right number of parameters, and silently fails if you run it in a
>context where the calling process is not a session-master.
>
>That's in steal-ctty-errorcheck.patch
>
>Wookey
>--
>Principal hats:  Linaro, Debian, Wookware, ARM
>http://wookware.org/

>commit 860e5132001dc8a9fb032ada312d590cabc497db
>Author: Wookey <[hidden email]>
>Date:   Fri Jan 18 18:44:09 2019 +0000
>
>    Add steal-ctty error checks
>
>diff --git a/src/sbin/steal-ctty.c b/src/sbin/steal-ctty.c
>index 0f3b14f..05a775f 100644
>--- a/src/sbin/steal-ctty.c
>+++ b/src/sbin/steal-ctty.c
>@@ -22,8 +22,12 @@ int main(int argc, char ** argv)
>     while (fd > 2) {
>         close(fd--);
>     }
>-    ioctl(0, TIOCSCTTY, (char *) 1);
>-    execvp(argv[2], &argv[2]);
>+    if (-1 == ioctl(0, TIOCSCTTY, (char *) 1)) {
>+      perror("steal-ctty:ioctl");
>+    }
>+    if (-1 == execvp(argv[2], &argv[2])) {
>+      perror("steal-ctty:execvp");
>+    }
>     /* never reached. */
>     return 0;
> }

Happy to take this straight away, tbh. Kibi?

>commit 053b796076981da6fbe38e8e7663f822d935b79b
>Author: Wookey <[hidden email]>
>Date:   Fri Jan 18 18:45:03 2019 +0000
>
>    run two installers
>
>diff --git a/src/sbin/reopen-console-linux b/src/sbin/reopen-console-linux
>index 87a7f7a..431bff0 100755
>--- a/src/sbin/reopen-console-linux
>+++ b/src/sbin/reopen-console-linux
>@@ -70,7 +70,13 @@ if ! [ -f /var/run/console-device ]; then
> console=console
> fi
> echo /dev/$console > /var/run/console-device
>+ echo /dev/$extraconsole > /var/run/extraconsole-device
> fi
>
>-# Some other session may have it as ctty. Steal it from them
>-exec /sbin/steal-ctty $(cat /var/run/console-device) "$@"
>+# Some other session may have console as ctty. Steal it from them
>+# Run D-I on other consoles if they are given
>+if [ -n $extraconsole ]; then
>+    ( exec /sbin/steal-ctty $(cat /var/run/extraconsole-device) "$@" & )
>+fi
>+exec /sbin/steal-ctty $(cat /var/run/console) "$@"
>+

2 minor nits here:

 1. You're checking for $extraconsole being set, but you're not using
    that, you're using the contents of
    /var/run/extraconsole-device. The two are not guaranteed to be the
    same. Instead, just check for the existence/size of the file (-s
    FILE)?

 2. We might want there to be more than one extra console here,
    depending on the hardware config. Maybe move to $extraconsoles and
    /var/run/extraconsole-devices and a loop to start on all of them?

We might *also* want to think about what console(s) the installed
system will use, but that's the next stage... :-)

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
"I suspect most samba developers are already technically insane... Of
 course, since many of them are Australians, you can't tell." -- Linus Torvalds

Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support

Richard Hector
In reply to this post by Wookey-6
On 19/01/19 5:27 PM, Wookey wrote:
> This problem doesn't arise on x86 where there is 'always' a screen (or
> some BIOs magic to reflect what would be on the screen to serial).

Is that true? I'm sure the Soekris box I've got (in storage)
(admittedly, probably not a supported cpu any more) only has serial. And
only server-class machines generally do the redirection, don't they?
There are plenty of use cases for a headless machine that doesn't need
to be expensive, where serial would be useful and a screen inconvenient.

Is there a disadvantage to doing the same thing on all arches?

Richard


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

Ian Campbell-5
In reply to this post by Steve McIntyre
On Sat, 2019-01-19 at 11:08 +0000, Steve McIntyre wrote:
> >So far I have done a proof-of-concept hack and demonstrated that
> >running two instances does in fact work nicely without anything
> >obvious breaking. The console selection still needs some work/checking
> >(I've run out of time for that tonight, but it can easily be fished
> >out of the kernel command line args. (or we could get fancier).
>
> Nod. Potentially we might end up of running on multiple
> consoles.

IIRC (and it hasn't changed in the years since I knew this stuff) we
already have this on some of the netconsole install images for
arm{el,hf}, you end up with an installer on serial and one on the ssh
connection. Not an identical situation since I think the second one is
only spawned when you login via ssh, but some sort of precedence I
guess!

Ian.

Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support

peter green-2
In reply to this post by Wookey-6
On 19/01/19 04:27, Wookey wrote:

> Arm64 (arm in general in fact) has a rather fundamental problem with
> D-I, which is that both serial and display are sensible default
> devices for the installer to run on. Which is 'correct' depends very
> much on the hardware and the circumstances. You may be installing a
> server in rack, or a dev board with no display, in which case serial
> is ideal, or you may have a chromebook or an ARM desktop machine with
> a screen plugged in and no easy access to the serial console.
>
> This problem doesn't arise on x86 where there is 'always' a screen (or
> some BIOs magic to reflect what would be on the screen to serial).
>
> Steve (McIntyre) and I have been thinking about what to do about this,
> so did some investigation and came up with a plan. Essentially it was
> to run d-i on both if they are configured/available. This way anyone
> looking at just one or the other will see D-I as they expect. The
> patch is not intrusive and essentially nothing changes if there is
> only one console so this should be a low-risk change.
One concern I would have is preseeding or other more-automated install modes. Presumablly this works out ok on a "normal" install because the instance of the installer that the user is *NOT* interacting with just sits there doing nothing but I would question if that is a reasonable assumption in general.

Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

Ben Hutchings-3
In reply to this post by Steve McIntyre
On Sat, 2019-01-19 at 11:08 +0000, Steve McIntyre wrote:
[...]
>  * add lots more console= options to the grub.cfg for arm64 (to cover
>    all the possibilities), then let d-i startup work out which devices
>    exist from /proc/cmdlinge and spawn d-i on each.
[...]

I think you should look in /proc/consoles, not /proc/cmdline.  The
format is documented in
https://www.kernel.org/doc/Documentation/filesystems/proc.txt

Ben.

--
Ben Hutchings
Hoare's Law of Large Problems:
   Inside every large problem is a small problem struggling to get out.


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

Wookey-6
On 2019-01-19 20:11 +0000, Ben Hutchings wrote:

> On Sat, 2019-01-19 at 11:08 +0000, Steve McIntyre wrote:
> [...]
> >  * add lots more console= options to the grub.cfg for arm64 (to cover
> >    all the possibilities), then let d-i startup work out which devices
> >    exist from /proc/cmdlinge and spawn d-i on each.
> [...]
>
> I think you should look in /proc/consoles, not /proc/cmdline.  The
> format is documented in
> https://www.kernel.org/doc/Documentation/filesystems/proc.txt
Interesting.

Currently reopen-consoles does:
if d-i console not already recorded
 set console using kernel 'handover' message (in dmesg) if running pre 2.6.38 kernel
 unless it's set to serial on a non-serial tty, in which case unset it
 make list of 'enabled consoles' from 1st 260k of dmesg
 if one matching device, then set as console
 if still not found (only) one, set to last one in kernel command line args
 if still not found one, default to /dev/console
 record chosen console
fi
start d-i on recorded console.

I'm not entirely convinced that all this logic is actually needed/optimum,
but I don't know the history of it.

How exactly does /proc/consoles fit into that? The docs say it is
"registered system consoles". Does that correspond to the same list of
'enabled consoles' the above is currently fishing out of dmesg? Does
it include any/all listed explicitly on the kernel command line?

My current code leaves all this alone and just records/uses all
enabled consoles on the command line, not just the last one, but
ideally we'd autodetect and/or hardcode all the sensible available
consoles and run on them.

If 'read /proc/consoles (and check the devices exist)' does this, then
that sounds very sensible.

Wookey
--
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

Ben Hutchings-3
On Sun, 2019-01-20 at 01:21 +0000, Wookey wrote:

> On 2019-01-19 20:11 +0000, Ben Hutchings wrote:
> > On Sat, 2019-01-19 at 11:08 +0000, Steve McIntyre wrote:
> > [...]
> > >  * add lots more console= options to the grub.cfg for arm64 (to cover
> > >    all the possibilities), then let d-i startup work out which devices
> > >    exist from /proc/cmdlinge and spawn d-i on each.
> > [...]
> >
> > I think you should look in /proc/consoles, not /proc/cmdline.  The
> > format is documented in
> > https://www.kernel.org/doc/Documentation/filesystems/proc.txt
>
> Interesting.
>
> Currently reopen-consoles does:
> if d-i console not already recorded
>  set console using kernel 'handover' message (in dmesg) if running pre 2.6.38 kernel
>  unless it's set to serial on a non-serial tty, in which case unset it
>  make list of 'enabled consoles' from 1st 260k of dmesg
>  if one matching device, then set as console
>  if still not found (only) one, set to last one in kernel command line args
>  if still not found one, default to /dev/console
>  record chosen console
> fi
> start d-i on recorded console.
>
> I'm not entirely convinced that all this logic is actually needed/optimum,
> but I don't know the history of it.
You can certainly remove anything that relates to old kernel versions.

> How exactly does /proc/consoles fit into that? The docs say it is
> "registered system consoles". Does that correspond to the same list of
> 'enabled consoles' the above is currently fishing out of dmesg?

/proc/consoles shows everything on the global console_drivers list.
Every time a console is added to the list the kernel logs a message
with the format "%sconsole [%s%d] enabled\n".  So these should match,
except that (1) it is also possible to unregister consoles, and reopen-
consoles doesn't account for that (2) the kernel log might have rolled
over so that reopen-console-linux doesn't see the messages.

> Does it include any/all listed explicitly on the kernel command line?

It's a bit more complicated than that.  The kernel has a table of up to
8 "preferred" console devices, which can be specified on the kernel
command, or through Device Tree or platform code, or by a hypervisor.
The device specified last (which usually means the last console=
argument on the command line) is the most preferred.

If some preferred devices are specified, then console_drivers will
include all the console devices that are preferred *and* have been
found to exist, and the most preferred (if it exists) will be first.

Otherwise, the kernel seems to enable the first console device found to
exist.

> My current code leaves all this alone and just records/uses all
> enabled consoles on the command line, not just the last one, but
> ideally we'd autodetect and/or hardcode all the sensible available
> consoles and run on them.
>
> If 'read /proc/consoles (and check the devices exist)' does this, then
> that sounds very sensible.

Reading /proc/consoles is exactly what you should do.

Ben.

--
Ben Hutchings
You can't have everything.  Where would you put it?


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support

Steve McIntyre
In reply to this post by Richard Hector
On Sun, Jan 20, 2019 at 01:08:24AM +1300, Richard Hector wrote:

>On 19/01/19 5:27 PM, Wookey wrote:
>> This problem doesn't arise on x86 where there is 'always' a screen (or
>> some BIOs magic to reflect what would be on the screen to serial).
>
>Is that true? I'm sure the Soekris box I've got (in storage)
>(admittedly, probably not a supported cpu any more) only has serial. And
>only server-class machines generally do the redirection, don't they?
>There are plenty of use cases for a headless machine that doesn't need
>to be expensive, where serial would be useful and a screen inconvenient.
>
>Is there a disadvantage to doing the same thing on all arches?

Good question! :-)

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
We don't need no education.
We don't need no thought control.

Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support

Steve McIntyre
In reply to this post by peter green-2
On Sat, Jan 19, 2019 at 02:38:46PM +0000, peter green wrote:

>On 19/01/19 04:27, Wookey wrote:
>>
>> Steve (McIntyre) and I have been thinking about what to do about this,
>> so did some investigation and came up with a plan. Essentially it was
>> to run d-i on both if they are configured/available. This way anyone
>> looking at just one or the other will see D-I as they expect. The
>> patch is not intrusive and essentially nothing changes if there is
>> only one console so this should be a low-risk change.
>
>One concern I would have is preseeding or other more-automated
>install modes. Presumablly this works out ok on a "normal" install
>because the instance of the installer that the user is *NOT*
>interacting with just sits there doing nothing but I would question
>if that is a reasonable assumption in general.

Ah, yes. Definitely worth checking on once we have something up and
running. We might need to make sure that only one of the running d-i
instances works like this.

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
Can't keep my eyes from the circling sky,
Tongue-tied & twisted, Just an earth-bound misfit, I...

Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

Steve McIntyre
In reply to this post by Ben Hutchings-3
On Sun, Jan 20, 2019 at 03:02:33AM +0000, Ben Hutchings wrote:
>On Sun, 2019-01-20 at 01:21 +0000, Wookey wrote:

...

>> My current code leaves all this alone and just records/uses all
>> enabled consoles on the command line, not just the last one, but
>> ideally we'd autodetect and/or hardcode all the sensible available
>> consoles and run on them.
>>
>> If 'read /proc/consoles (and check the devices exist)' does this, then
>> that sounds very sensible.
>
>Reading /proc/consoles is exactly what you should do.

Aha, thanks! I genuinely wasn't aware of /proc/consoles, and it does
sound like exactly the right thing to be looking at. :-)

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
Into the distance, a ribbon of black
Stretched to the point of no turning back

Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

Steve McIntyre
In reply to this post by Ian Campbell-5
On Sat, Jan 19, 2019 at 01:16:43PM +0000, Ian Campbell wrote:

>On Sat, 2019-01-19 at 11:08 +0000, Steve McIntyre wrote:
>> >So far I have done a proof-of-concept hack and demonstrated that
>> >running two instances does in fact work nicely without anything
>> >obvious breaking. The console selection still needs some work/checking
>> >(I've run out of time for that tonight, but it can easily be fished
>> >out of the kernel command line args. (or we could get fancier).
>>
>> Nod. Potentially we might end up of running on multiple
>> consoles.
>
>IIRC (and it hasn't changed in the years since I knew this stuff) we
>already have this on some of the netconsole install images for
>arm{el,hf}, you end up with an installer on serial and one on the ssh
>connection. Not an identical situation since I think the second one is
>only spawned when you login via ssh, but some sort of precedence I
>guess!

Yup, the netconsole case was exactly my inspiration for going down
this route. :-)

--
Steve McIntyre, Cambridge, UK.                                [hidden email]
You lock the door
And throw away the key
There's someone in my head but it's not me

Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

wookey-4
In reply to this post by Ben Hutchings-3
On 2019-01-20 03:02 +0000, Ben Hutchings wrote:

> /proc/consoles shows everything on the global console_drivers list.
> Every time a console is added to the list the kernel logs a message
> with the format "%sconsole [%s%d] enabled\n".  So these should match,
> except that (1) it is also possible to unregister consoles, and reopen-
> consoles doesn't account for that (2) the kernel log might have rolled
> over so that reopen-console-linux doesn't see the messages.
>
> > Does it include any/all listed explicitly on the kernel command line?
>
> It's a bit more complicated than that.  The kernel has a table of up to
> 8 "preferred" console devices, which can be specified on the kernel
> command, or through Device Tree or platform code, or by a hypervisor.
> The device specified last (which usually means the last console=
> argument on the command line) is the most preferred.
>
> If some preferred devices are specified, then console_drivers will
> include all the console devices that are preferred *and* have been
> found to exist, and the most preferred (if it exists) will be first.
>
> Otherwise, the kernel seems to enable the first console device found to
> exist.

> Reading /proc/consoles is exactly what you should do.

Checking this on a booted thunderx machine (with no explicit kernel cmdline options) it lists
ttyAMA0

If I boot with explicit console=tty0 console=ttyAMA0 on the kernel cmdline
then they both appear in /proc/consoles, AMA0 last

If I boot with explicit console=tty0 on the kernel cmdline
then they both appear in /proc/consoles, AMA0 still last


So /proc/consoles does indeed correspond to the set of enabled
consoles listed by dmesg, however the last-listed on the cmdline is
not necessarily the last in the list. Perhaps this is influenced by
the device EFI specifies as the default console?

What it doesn't do (which I was hoping for) is automagically note that
there is a display attached (that could be used as a console) unless you
tell it to look.

Anyone know what would it take to make tty0 appear as a valid console
on a thunderx machine without having to explicitly list it on the
kernel command line? This is what we'd ideally like to happen.

I've modified reopen-console-linux to use /proc/console and run D-I on all found.
I'll test that on tue when back at suitable machine, and post here when it shows signs of working.

Wookey
--
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

Ian Campbell-2
On Tue, 2019-01-22 at 04:31 +0000, Wookey wrote:

> On 2019-01-20 03:02 +0000, Ben Hutchings wrote:
> > Reading /proc/consoles is exactly what you should do.
>
> Checking this on a booted thunderx machine (with no explicit kernel cmdline options) it lists
> ttyAMA0
>
> If I boot with explicit console=tty0 console=ttyAMA0 on the kernel cmdline
> then they both appear in /proc/consoles, AMA0 last
>
> If I boot with explicit console=tty0 on the kernel cmdline
> then they both appear in /proc/consoles, AMA0 still last

Do the various flags not differ between the different cases?

Ian.

Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

wookey-4
On 2019-01-22 08:23 +0000, Ian Campbell wrote:

> On Tue, 2019-01-22 at 04:31 +0000, Wookey wrote:
> > On 2019-01-20 03:02 +0000, Ben Hutchings wrote:
> > > Reading /proc/consoles is exactly what you should do.
> >
> > Checking this on a booted thunderx machine (with no explicit kernel cmdline options) it lists
> > ttyAMA0
> >
> > If I boot with explicit console=tty0 console=ttyAMA0 on the kernel cmdline
> > then they both appear in /proc/consoles, AMA0 last
> >
> > If I boot with explicit console=tty0 on the kernel cmdline
> > then they both appear in /proc/consoles, AMA0 still last
>
> Do the various flags not differ between the different cases?
You are right. I wasn't taking note of those:

E=enabled
C=preferred console
p=used for printk buffer
a=safe to use when CPU is offline

console=tty0
tty0                 -WU (EC p  )    4:7
ttyAMA0              -W- (E  p a)  204:64

console=ttyAMA0
tty0                 -WU (E  p  )    4:7
ttyAMA0              -W- (EC p a)  204:64

console=tty0 console=ttyAMA0
tty0                 -WU (E  p  )    4:7
ttyAMA0              -W- (E  p a)  204:64

Any idea how we should choose a D-I primary console when none of them
is marked 'preferred'? Or should D-i do away with the concept and try
to treat them all equally (which is a slightly more intrusive change).

Currently I use the one marked 'C' or the last one if none.

Wookey
--
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

Ian Campbell-5
On Wed, 2019-01-23 at 03:41 +0000, Wookey wrote:

> You are right. I wasn't taking note of those:
>
> E=enabled
> C=preferred console
> p=used for printk buffer
> a=safe to use when CPU is offline
>
> console=tty0
> tty0                 -WU (EC p  )    4:7
> ttyAMA0              -W- (E  p a)  204:64
>
> console=ttyAMA0
> tty0                 -WU (E  p  )    4:7
> ttyAMA0              -W- (EC p a)  204:64
>
> console=tty0 console=ttyAMA0
> tty0                 -WU (E  p  )    4:7
> ttyAMA0              -W- (E  p a)  204:64
>
> Any idea how we should choose a D-I primary console when none of them
> is marked 'preferred'? Or should D-i do away with the concept and try
> to treat them all equally (which is a slightly more intrusive
> change).
>
> Currently I use the one marked 'C' or the last one if none.

I wonder if the lack of a 'C' in the final entry is considered a bug?

Looking at the kernel code I see:

   fs/proc/consoles.c:             { CON_CONSDEV,          'C' },

where CON_CONSDEV is:

   include/linux/console.h:#define CON_CONSDEV     (2) /* Last on the command line */

So it being lacking on ttyAMA0 in that case seems wrong.

Even if the bug were fixed it seems sensible to deal with this case,
last one (with sufficient other flags set) seems like as good as
anything...

Ian.

Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

wookey-4
On 2019-01-23 08:35 +0000, Ian Campbell wrote:
> On Wed, 2019-01-23 at 03:41 +0000, Wookey wrote:

> > Any idea how we should choose a D-I primary console when none of them
> > is marked 'preferred'? Or should D-i do away with the concept and try
> > to treat them all equally (which is a slightly more intrusive
> > change).

> Even if the bug were fixed it seems sensible to deal with this case,
> last one (with sufficient other flags set) seems like as good as
> anything...

OK. I've had 'fun with shell' getting this right and gone through a
couple of iterations.

After some thought, so far as I can see the 'preferred' console
concept is no longer useful once D-I runs on more than one. We don't
know which one the user is going to actually use, and the whole point
is that all the instances should work the same. We're not trying the
choose the 'right' console any more, just exposing the interface on
all the ones that (should) work.

The only reason for keeping it would be to keep the reopen-console
logic more like it currently is, so that the preferred console got to
replace the reopen-consoles process (via exec) and the others didn't
(becoming child processes of reopen-consoles dash instance). But you
end up with rather cleaner code if in fact you just run D-I on all enabled
consoles and treat them equivalently (as child processes). New version
(with debug still in) attached to show what I mean (or see below).

However this does lead to a question about D-I and inittab respawn logic.

Currently reopen-console ends with
exec D-I
so D-I replaces the process.

It is started with this inittab line:
::respawn:/sbin/reopen-console /sbin/debian-installer

Do we ever make use of D-I finishing in such a way that init finds the
process is gone and respawns? My assumption is that we don't want this
to happen, at least in general (otherwise the installer would restart
when you finished, rather confusingly). But perhaps there are error
cases when this is wanted?

The reason it matters is because it affects how the multiple D-Is on
different consoles should be started and what we should do when one
quits, or all quit.

My current code does this:
for cons in $(cat /var/run/console-devices)
do
        # Some other session may have console as ctty. Steal it from them
        ( exec /sbin/steal-ctty $cons "$@" & )
done
#don't return to init
sleep infinity

Which just starts a D-I on all the consoles, then sits forever (to
stop init from re-running this code over and over).  You _could_ have
some extra logic to make one special, and exec that one, and keep the
others as child-processes, but I'm not convinced that this achieves
anything (except complexity, and potentially confusingly different
behaviour between consoles), unless the respawning is important in
some way I have failed to grok so far?

(You'll note the above is rather nicer than my first effort with two
different /var/run/* files, and $console and $extraconsoles). Having
just the one 'consoles used' list file makes it all cleaner.  (I've
fixed up finish-install/finish-install.d/90console to deal with more
than one console listed in that file too)

Possibly what we'd really like is that if you quit _any_ console D-I
instance then it would return and respawn, but a) I can't work out how
to do that (you can only exec one thing) and b) does it actually help?

So, unless anyone can see a problem with this approach, I'll finish
this off. Trying to do it with separate /var/run/consoles and
/var/run/extra-consoles files was getting very messy.

Wookey
--
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

reopen-console-linux (1K) Download Attachment
signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

wookey-4
On 2019-01-25 03:45 +0000, Wookey wrote:
> So, unless anyone can see a problem with this approach, I'll finish
> this off. Trying to do it with separate /var/run/consoles and
> /var/run/extra-consoles files was getting very messy.

OK. This took way longer than I hoped as it was not entirely trivial
to get everything working.

Attached is a patch which provides working multiconsole support for
linux (not hurd or bsd, sorry).

After getting the proc/console choosing code working nicely the
installer was still mysteriously not working (nothing on consoles
except debug) unless I let it respawn in which case it sort of worked
(things appeared but input oddness and continuous respawning isn't much
use to anyone).

I was confused for a while as to what was going on but eventually worked it out.

Just to recap:
The objective here is to run D-I on all the enabled consoles, (and
ideally not fiddle with the code any more than is needed).

First step was upgrading the console parsing code to use
/proc/consoles to get the list, noting the preferred console if there
is one marked as such.

The main complication is that reopen-consoles is
run twice by init, once with debian-installer-startup and once with debian-installer:
# main rc script
::sysinit:/sbin/reopen-console /sbin/debian-installer-startup
# main setup program
::respawn:/sbin/reopen-console /sbin/debian-installer

The first run of reopen-console works out which consoles to use, and
writes it in a file, (/var/run/console-devices), the second run just
uses that file.
debian-installer-startup is just a shell script that runs through all the
debian-installer-startup.d rc scripts, like S01mount, S04countcpus-linux-hppa, S10syslog.

debian-installer actually runs the installer, on the second invocation
of reopen-consoles.

So the original plan was just to run $@ (debian-installer) on the
found consoles. But doing that for the rc scripts is not helpful. Most
of it is idempotent, but you end up with two syslogs and two klogds
and running it all twice on different consoles really isn't right.

So I added an --all-consoles option to declare that we want something
(/sbin/debian-installer in this case) run on all the consoles.

# main rc script
::sysinit:/sbin/reopen-console /sbin/debian-installer-startup
# main setup program
::respawn:/sbin/reopen-console --all-consoles /sbin/debian-installer

It's also worth noting that 'steal-ctty' cannot set the process as
'controlling tty' when two are run, because only one process can be
controlling tty in a session. So I did add error checking to that (it
had none before), but had to take it out again for the 'set ctty'
IOCTL as it's correct for it to fail in the muti-console case.

So what happens now is that the rc-scripts are run on the 'preferred'
console (or the first console listed if none are marked preferred),
then D-I is run on all of them. It does not return to init in this
case (as previously discussed).

This is an important improvement so despite it having ended up rather
late in the day, I hope we can include this for buster. I'm happy to
do more work on tidying up any breakage if we find any.

Future work:

All this faffage has made me realise that a better approach to all
this would probably be to get rid of all this 'steal-ctty' bodgery,
and use init to do it's job: it's designed to run multiple things on
multiple consoles, and deal with file handles and them failing etc.

The catch is that to make this work we'd have to use sysinit: to run
the console-choosing code, then write a new inittab containing one
respawn:/sbin/debian-installer for each console instance, then HUP
init. This should do exactly the right thing (assuming that busybox
init isn't too thick to get HUPing right).

That is way cleaner and I'm happy to have a go at that, but it feels
more intrusive and unless I'm very lucky it may take a while so I
sugest we go with the above for now, as it works already.

One bug I just noticed in the bit we did today is that the 'default'
preferred console (for when one is not indicated by the kernel) avoids
the existence-check for the /dev file, so that should be
improved. It's not hard, but I'll resist doing it now and sending an
untested patch :-)

Wookey
--
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

multiple-consoles.patch (5K) Download Attachment
signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

wookey-4
On 2019-02-09 04:11 +0000, Wookey wrote:
> On 2019-01-25 03:45 +0000, Wookey wrote:
>
> Attached is a patch which provides working multiconsole support for
> linux (not hurd or bsd, sorry).
>

> One bug I just noticed in the bit we did today is that the 'default'
> preferred console (for when one is not indicated by the kernel) avoids
> the existence-check for the /dev file, so that should be
> improved. It's not hard, but I'll resist doing it now and sending an
> untested patch :-)

OK. Updated version attached which is more robust about choosing the
'preferred' console.

I did some tests and discovered that, at least on thunderx, you get
slightly different behaviour WRT kernel command line options than the
old code, but I don't think it really matters.

Essentially the set of consoles is now 'those listed on the kernel
command line + whatever UEFI says is the default console'. Previously
you got 'whatever UEFI says is the default console, or the last one on
the kernel cmd line'. i.e the kernel no longer overrides the UEFI console,
it gets added. Now that D-I works on all provided consoles this doesn't
really matter much.

So on thunderx, for example, you get this:
default (nothing specified):
consoles:  /dev/ttyAMA0
preferred: /dev/ttyAMA0

cmdline: console=tty0
consoles:  /dev/tty0 /dev/ttyAMA0
preferred: /dev/tty0

cmdline: console=tty0 console=ttyAMA0
consoles:  /dev/tty0 /dev/ttyAMA0
preferred: /dev/tty0

cmdline: console=ttyAMA0
consoles:  /dev/ttyAMA0
preferred: /dev/ttyAMA0

Which all seems reasonable.

Testing this on some other machines/arches is the next thing to do, although
just checking what /proc/consoles shows tells you what should happen.


Wookey
--
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

debian-installer-multiple-consoles.patch (5K) Download Attachment
signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Multiple console support in d-i

wookey-4
In reply to this post by Steve McIntyre
On 2019-01-19 11:08 +0000, Steve McIntyre wrote:
> On Sat, Jan 19, 2019 at 04:27:05AM +0000, Wookey wrote:

[Re: adding multiple console support to D-I, including changing
/var/run/console-device with one device line to be
/var/run/console-devices with 1 or more lines]

> >The only other place this affects is
> >packages/finish-install.d/90console which reads
> >/var/run/console-device when tidying up at the end in order to write
> >inittab entries for the used console device (serial, xen, etc).

So here is the patch for that so that the right file is looked in and
any serial devices are dealt with as before, alowing for the fact there
may be more than one console device listed.

This code is not well-tested yet, but I think it does the right
thing. Review welcome. I can't see any reason why running through this
more than once should change anything, but I could be missing something.

I note that there is a load of upstart support in here. Can anyone
thing of a reason to keep that? I suspect it should go. Happy to do that if we agree.

Wookey
--
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

finish-install-multiconsole.patch (4K) Download Attachment
signature.asc (849 bytes) Download Attachment
12