Re: Bug#839046: debootstrap: enable --merged-usr by default

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Bug#839046: debootstrap: enable --merged-usr by default

Holger Levsen-2
On Mon, Feb 05, 2018 at 08:19:33PM +0100, Julien Cristau wrote:

> On Sat, Feb  3, 2018 at 09:16:40 +0100, Marco d'Itri wrote:
> > On Dec 23, md wrote:
> > > On Dec 20, Julien Cristau <[hidden email]> wrote:
> > > > > This change was reverted in 1.0.87 as dpkg-shlibdeps didn't cope
> > > > > properly with a merged-usr system. Thus reopening this bug report for
> > > > > that version.
> > > > >
> > > > > The dpkg-shlibdeps bugs has been fixed [1] in the mean time. So it would
> > > > > be great if this bug report could be re-considered.
> > > > That'll be after stretch now.
> > > Stretch was been released long ago: please re-enable --merged-usr in
> > > debootstrap.
> > There has not been any negative feedback, can we move on please?
> Is there buy-in from the dpkg maintainer?
cc:ing them.


--
cheers,
        Holger

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bug#839046: debootstrap: enable --merged-usr by default

Guillem Jover
On Mon, 2018-02-05 at 22:32:05 +0000, Holger Levsen wrote:

> On Mon, Feb 05, 2018 at 08:19:33PM +0100, Julien Cristau wrote:
> > On Sat, Feb  3, 2018 at 09:16:40 +0100, Marco d'Itri wrote:
> > > On Dec 23, md wrote:
> > > > On Dec 20, Julien Cristau <[hidden email]> wrote:
> > > > > > This change was reverted in 1.0.87 as dpkg-shlibdeps didn't cope
> > > > > > properly with a merged-usr system. Thus reopening this bug report for
> > > > > > that version.
> > > > > >
> > > > > > The dpkg-shlibdeps bugs has been fixed [1] in the mean time. So it would
> > > > > > be great if this bug report could be re-considered.

> > > > > That'll be after stretch now.

> > > > Stretch was been released long ago: please re-enable --merged-usr in
> > > > debootstrap.

> > > There has not been any negative feedback, can we move on please?

> > Is there buy-in from the dpkg maintainer?

As I've mentioned in the past, I think the usrmerge filesystem layout
has merit and can solve some issues, and to state it very clearly, I
have no technical issue with it *what*so*ever*. But the same can be
said about the non-usrmerge layout with multiple mount-points, which
while not general anymore in Debian, can still be used perfectly fine
on controlled subsets of packages and custom built kernels w/o reliance
on an initramfs.

What I still find to be terrible is the way it's been tried to be
deployed in Debian, via the usrmerge package, which does not support
reverting the change (#848626), for which there's (AFAIK) no way to
select not using this irreversible hack from d-i, which breaks
dpkg-query --search (at least #848622 and #858331). This seems like
a nice PoV for people to play with it, in the same way local admins
can use, to some extent, symlinks to redirect subtrees to other mount
points, but I don't see how this can be seen as a production-ready
solution shipped by default, TBH.

In any case, I looked the other day into implementing the
--map-pathname option for dpkg-query, and I've got most of the code
ready. The problem is that this requires adding support for config
files and config fragments to dpkg-query, which has the additional
problem of making it possible to mess with the --showformat option
and breaking the expectations from maintscripts, for example. The
other problem is with the search behavior changing depending on the
packages providing those mappings being installed (because dpkg would
certainly not provide them).


So I'll eventually try to find a solution for the dpkg-query issue,
because it's a long-standing paper-cut in dpkg for local admins. But
I'd not be very amused if this hack is enabled by default again and
suddenly RC bugs start popping up in dpkg again, and people start
pressuring with RCness to get those fixed again because well, it's
obviously breaking people's systems…


Thanks,
Guillem

Reply | Threaded
Open this post in threaded view
|

Re: Bug#839046: debootstrap: enable --merged-usr by default

Ansgar Burchardt-8
In reply to this post by Holger Levsen-2
Guillem Jover writes:
> In any case, I looked the other day into implementing the
> --map-pathname option for dpkg-query, and I've got most of the code
> ready. The problem is that this requires adding support for config
> files and config fragments to dpkg-query, which has the additional
> problem of making it possible to mess with the --showformat option
> and breaking the expectations from maintscripts, for example. The
> other problem is with the search behavior changing depending on the
> packages providing those mappings being installed (because dpkg would
> certainly not provide them).

So who should provide them?  debootstrap?  base-files?

The correct long-term fix is probably for packages to eventually install
to the location in /usr anyway.  That doesn't work without some
transition period of 1-2 releases though.

Ansgar

Reply | Threaded
Open this post in threaded view
|

Re: Bug#839046: debootstrap: enable --merged-usr by default

Raphaël Hertzog-3
In reply to this post by Guillem Jover
On Mon, 12 Feb 2018, Ansgar Burchardt wrote:

> Guillem Jover writes:
> > In any case, I looked the other day into implementing the
> > --map-pathname option for dpkg-query, and I've got most of the code
> > ready. The problem is that this requires adding support for config
> > files and config fragments to dpkg-query, which has the additional
> > problem of making it possible to mess with the --showformat option
> > and breaking the expectations from maintscripts, for example. The
> > other problem is with the search behavior changing depending on the
> > packages providing those mappings being installed (because dpkg would
> > certainly not provide them).
>
> So who should provide them?  debootstrap?  base-files?

It certainly makes sense for debootstrap to create those files given it
creates the symlinks in the first place.

An alternative solution could be to reuse the usrmerge package and let
debootstrap install this package if it exists. That way the usrmerge
package would exist until Debian switched entirely to put everything into
/usr/bin.

> The correct long-term fix is probably for packages to eventually install
> to the location in /usr anyway.  That doesn't work without some
> transition period of 1-2 releases though.

Ack.

On Mon, 12 Feb 2018, Guillem Jover wrote about usrmerge:
> This seems like a nice PoV for people to play with it, in the same way
> local admins can use, to some extent, symlinks to redirect subtrees to
> other mount points, but I don't see how this can be seen as a
> production-ready solution shipped by default, TBH.

Note that the change in debootstrap does not install usrmerge currently.
It only creates the required symlinks.

<reminder>
But this is enough to confuse "dpkg -S". This used to break dpkg-shlibdeps
and was the main reason for the initial revert. Fortunately dpkg-shlibdeps has
been fixed to try multiple paths until it can find the package owning the
library.
</reminder>

It might be that we will find other tools confused by "dpkg -S" non-answer
on some /lib/* or /usr/lib/* paths and some end users will definitely be surprised by
this.

Obviously we can document the problem in release notes but it would be
better to have a clean solution like suggested by Guillem:

> In any case, I looked the other day into implementing the
> --map-pathname option for dpkg-query, and I've got most of the code
> ready.

If this is forthcoming in the buster timeframe, it seems reasonable to
go ahead and re-enable the merged-usr by default. However to be able to
ship the new configuration files once their format is known, it would be
better for debootstrap to install a package whose role will be to provide
those configuration files ASAP after dpkg starts to support them.

> The problem is that this requires adding support for config
> files and config fragments to dpkg-query, which has the additional
> problem of making it possible to mess with the --showformat option
> and breaking the expectations from maintscripts, for example.

Forbid those options there? Do not parse those files if you detect
an environment variable DPKG_RUNNING_VERSION?

> The other problem is with the search behavior changing depending on the
> packages providing those mappings being installed (because dpkg would
> certainly not provide them).

That's the whole point of it so I would not consider this a problem.

> So I'll eventually try to find a solution for the dpkg-query issue,
> because it's a long-standing paper-cut in dpkg for local admins. But
> I'd not be very amused if this hack is enabled by default again and
> suddenly RC bugs start popping up in dpkg again, and people start
> pressuring with RCness to get those fixed again because well, it's
> obviously breaking people's systems…

Are you considering both "usrmerge" and "debootstrap creating symlinks" as
hacks even once they would provide mapping data for dpkg --search?

I believe that we have had quite some testing already last time and I
would be surprised if we got many more RC bugs on dpkg that had to be fixed
on a short timeframe. I guess nobody can give you any assurance but
I'm sure that you can downgrade those bugs pointing to this discussion
and showing that this was part of the deal.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: https://www.freexian.com/services/debian-lts.html
Learn to master Debian: https://debian-handbook.info/get/

Reply | Threaded
Open this post in threaded view
|

Re: Bug#839046: debootstrap: enable --merged-usr by default

Julien Cristau-6
On 02/13/2018 04:29 PM, Raphael Hertzog wrote:
> I believe that we have had quite some testing already last time and I
> would be surprised if we got many more RC bugs on dpkg that had to be fixed
> on a short timeframe. I guess nobody can give you any assurance but
> I'm sure that you can downgrade those bugs pointing to this discussion
> and showing that this was part of the deal.
>
If we break user systems we don't get to downgrade bugs on the basis
that "we knew we were doing a half assed job with this transition".
That's not part of the deal we're making with our users.

Cheers,
Julien

Reply | Threaded
Open this post in threaded view
|

Re: Bug#839046: debootstrap: enable --merged-usr by default

Raphaël Hertzog-3
On Tue, 13 Feb 2018, Julien Cristau wrote:

> On 02/13/2018 04:29 PM, Raphael Hertzog wrote:
> > I believe that we have had quite some testing already last time and I
> > would be surprised if we got many more RC bugs on dpkg that had to be fixed
> > on a short timeframe. I guess nobody can give you any assurance but
> > I'm sure that you can downgrade those bugs pointing to this discussion
> > and showing that this was part of the deal.
>
> If we break user systems we don't get to downgrade bugs on the basis
> that "we knew we were doing a half assed job with this transition".
> That's not part of the deal we're making with our users.

Granted, but RC bugs against dpkg does not translate into "break user
systems". I would be very surprised if we found out a tool using dpkg
--search that would break the system on which it is running because
it did not get an answer to a specific dpkg --search query.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: https://www.freexian.com/services/debian-lts.html
Learn to master Debian: https://debian-handbook.info/get/