Is there still a point in installing libgcrypt to /lib instead of /usr/lib

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Andreas Metzler-4
Hello,

afaict we are moving to a usrmerge setup, i.e. with /lib just a
symlink to /usr/lib. So shouldn't packages start installing stuff to
/usr/lib instead of /lib? I would like to do that for libgcrypt, since
I would be able to shorten debian/rules by stopping to split stuff
between /lib (.so) and /usr/lib (.a).

TIA, cu Andreas
--
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Ben Hutchings-3
On Sat, 2020-02-15 at 18:31 +0100, Andreas Metzler wrote:
> Hello,
>
> afaict we are moving to a usrmerge setup, i.e. with /lib just a
> symlink to /usr/lib. So shouldn't packages start installing stuff to
> /usr/lib instead of /lib? I would like to do that for libgcrypt, since
> I would be able to shorten debian/rules by stopping to split stuff
> between /lib (.so) and /usr/lib (.a).

Since stretch, it's a requirement that any separate /usr is mounted by
an initramfs before the normal init system is started:
https://www.debian.org/releases/stretch/amd64/release-notes/ch-information.en.html#late-mounting-usr

So although usrmerge is optional, I believe there is no need to install
libraries under /lib except for the dynamic linkers (which the kernel
loads using absolute paths).

Ben.

> TIA, cu Andreas
--
Ben Hutchings
Any sufficiently advanced bug is indistinguishable from a feature.


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Sven Joachim
On 2020-02-15 18:29 +0000, Ben Hutchings wrote:

> On Sat, 2020-02-15 at 18:31 +0100, Andreas Metzler wrote:
>> Hello,
>>
>> afaict we are moving to a usrmerge setup, i.e. with /lib just a
>> symlink to /usr/lib. So shouldn't packages start installing stuff to
>> /usr/lib instead of /lib? I would like to do that for libgcrypt, since
>> I would be able to shorten debian/rules by stopping to split stuff
>> between /lib (.so) and /usr/lib (.a).
>
> Since stretch, it's a requirement that any separate /usr is mounted by
> an initramfs before the normal init system is started:
> https://www.debian.org/releases/stretch/amd64/release-notes/ch-information.en.html#late-mounting-usr
>
> So although usrmerge is optional, I believe there is no need to install
> libraries under /lib except for the dynamic linkers (which the kernel
> loads using absolute paths).

True, but there seem to be a relatively high number of systems where an
old unowned version of some library is lying around under /lib (possibly
because the dpkg database became corrupted at some point and so dpkg
forgot about the file; see the dpkg bug #949395), and when that library
starts be installed under /usr/lib, this will trigger symbol lookup
errors and the like.  See #896019 and #948318 for examples.

Cheers,
       Sven

Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Marco d'Itri
On Feb 15, Sven Joachim <[hidden email]> wrote:

> True, but there seem to be a relatively high number of systems where an
> old unowned version of some library is lying around under /lib (possibly
> because the dpkg database became corrupted at some point and so dpkg
> forgot about the file; see the dpkg bug #949395), and when that library
> starts be installed under /usr/lib, this will trigger symbol lookup
> errors and the like.  See #896019 and #948318 for examples.
Somebody reported a similar problem about libcrypt.so.1, which moved
from /lib/ (provided by libc) to /usr/lib/ (provided by libxcrypt).

Since libcrypt.so.1 has been in /usr/lib/ for three months now without
any other unexpected issues then I think that we can be very confident
that there is no reason whatsoever to install anything outside of /usr
anymore.

--
ciao,
Marco

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Michael Biebl-3
In reply to this post by Sven Joachim
Am 15.02.20 um 19:48 schrieb Sven Joachim:

> True, but there seem to be a relatively high number of systems where an
> old unowned version of some library is lying around under /lib (possibly
> because the dpkg database became corrupted at some point and so dpkg
> forgot about the file; see the dpkg bug #949395), and when that library
> starts be installed under /usr/lib, this will trigger symbol lookup
> errors and the like.  See #896019 and #948318 for examples.

While I think the underlying issue should be investigated (tbh the
thought that dpkg get's confused / its db corrupted so does not properly
clean up those old files is quite disconcerting), couldn't we just
switch the order of /lib and /usr/lib and make /usr/lib the preferred path?



signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Guillem Jover
In reply to this post by Marco d'Itri
On Sat, 2020-02-15 at 20:35:58 +0100, Marco d'Itri wrote:
> On Feb 15, Sven Joachim <[hidden email]> wrote:
> > True, but there seem to be a relatively high number of systems where an
> > old unowned version of some library is lying around under /lib (possibly
> > because the dpkg database became corrupted at some point and so dpkg
> > forgot about the file; see the dpkg bug #949395), and when that library
> > starts be installed under /usr/lib, this will trigger symbol lookup
> > errors and the like.  See #896019 and #948318 for examples.

> Somebody reported a similar problem about libcrypt.so.1, which moved
> from /lib/ (provided by libc) to /usr/lib/ (provided by libxcrypt).

If the problem was with the new pathname disappearing, then that's just
yet another instance of the usrmerge-via-symlinks collateral damage.

Moving a pathname to an aliased directory across different binary
packages can make the new pathname disappear depending on the unpack
order. Because, if dpkg unpacks the package now owning the new pathname,
and then unpacks the package that has stopped owning the old pathname,
when it removes that, it will end up removing the aliased pathname and
will not notice any Replaces takeover due to the directory aliasing.

> Since libcrypt.so.1 has been in /usr/lib/ for three months now without
> any other unexpected issues then I think that we can be very confident
> that there is no reason whatsoever to install anything outside of /usr
> anymore.

Then the libcrypt package is broken on dist upgrade and can render
usrmerge-via-symlinks systems unusable. Package doing a proper /usr-merge
migration within the same binary package are of course perfectly fine
everywhere, and also of course non-usrmerged systems are always fine.

The irony in all this is that the usrmerge hack that got introduced to
make the switch faster and easier, has been consistently making a
proper /usr-merged switch more fragile and difficult… I need to create
a wiki with the increasing list of issues and a big fat note stating
that these broken deployments are unsupported by dpkg.

Regards,
Guillem

Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Guillem Jover
In reply to this post by Andreas Metzler-4
Hi!

On Sat, 2020-02-15 at 18:31:32 +0100, Andreas Metzler wrote:
> afaict we are moving to a usrmerge setup, i.e. with /lib just a
> symlink to /usr/lib. So shouldn't packages start installing stuff to
> /usr/lib instead of /lib? I would like to do that for libgcrypt, since
> I would be able to shorten debian/rules by stopping to split stuff
> between /lib (.so) and /usr/lib (.a).

Doing a proper /usr-merged migration is what we should have done from
the beginning. I've been doing that with all the library packages I
maintain that were under /lib. That includes acl, attr, libaio, libbsd
and libmd, and I know others have been doing this too, so there's
plenty of precedent with this.

Thanks,
Guillem

Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Michael Biebl-3
In reply to this post by Guillem Jover
Am 15.02.20 um 23:11 schrieb Guillem Jover:

> On Sat, 2020-02-15 at 20:35:58 +0100, Marco d'Itri wrote:
>> On Feb 15, Sven Joachim <[hidden email]> wrote:
>>> True, but there seem to be a relatively high number of systems where an
>>> old unowned version of some library is lying around under /lib (possibly
>>> because the dpkg database became corrupted at some point and so dpkg
>>> forgot about the file; see the dpkg bug #949395), and when that library
>>> starts be installed under /usr/lib, this will trigger symbol lookup
>>> errors and the like.  See #896019 and #948318 for examples.
>
>> Somebody reported a similar problem about libcrypt.so.1, which moved
>> from /lib/ (provided by libc) to /usr/lib/ (provided by libxcrypt).
>
> If the problem was with the new pathname disappearing, then that's just
> yet another instance of the usrmerge-via-symlinks collateral damage.
>
> Moving a pathname to an aliased directory across different binary
> packages can make the new pathname disappear depending on the unpack
> order. Because, if dpkg unpacks the package now owning the new pathname,
> and then unpacks the package that has stopped owning the old pathname,
> when it removes that, it will end up removing the aliased pathname and
> will not notice any Replaces takeover due to the directory aliasing.
>
>> Since libcrypt.so.1 has been in /usr/lib/ for three months now without
>> any other unexpected issues then I think that we can be very confident
>> that there is no reason whatsoever to install anything outside of /usr
>> anymore.
>
> Then the libcrypt package is broken on dist upgrade and can render
> usrmerge-via-symlinks systems unusable. Package doing a proper /usr-merge
> migration within the same binary package are of course perfectly fine
> everywhere, and also of course non-usrmerged systems are always fine.
Those issues happen on non-usr-merged systems.



signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Guillem Jover
On Sat, 2020-02-15 at 23:27:08 +0100, Michael Biebl wrote:
> Those issues happen on non-usr-merged systems.

The one report against dpkg sure. I'm talking about the ones with
disappearing pathnames, in case that was part of "similar". But if it
was not, then libcrypt is still just broken on usrmerge-via-symlink
systems, like libgcc-s1 was for a bit when it tried to do the same by
moving directories across binary packages. But I guess I should not
care much given that I consider those system broken already anyway…

Regards,
Guillem

Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Marco d'Itri
In reply to this post by Guillem Jover
On Feb 15, Guillem Jover <[hidden email]> wrote:

> > Somebody reported a similar problem about libcrypt.so.1, which moved
> > from /lib/ (provided by libc) to /usr/lib/ (provided by libxcrypt).
> If the problem was with the new pathname disappearing, then that's just
> yet another instance of the usrmerge-via-symlinks collateral damage.
No: the problem was an old libcrypt.so.1 left in /lib/.
This cannot have anything to do with merged-usr: if /lib/ and /usr/lib/
on this system were actually merged then it would not be possible to
have two different libraries around.

> The irony in all this is that the usrmerge hack that got introduced to
The irony is that in your quest to fight usrmerge, which at this point
has been the default for all new installs and obviously did not cause
any major troubles, you ranted at lenght about something which does not
appear to be a real-life problem (people tend to report bugs which break
PAM and perl...).

--
ciao,
Marco

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

smcv
In reply to this post by Michael Biebl-3
On Sat, 15 Feb 2020 at 21:24:38 +0100, Michael Biebl wrote:
> While I think the underlying issue should be investigated (tbh the
> thought that dpkg get's confused / its db corrupted so does not properly
> clean up those old files is quite disconcerting), couldn't we just
> switch the order of /lib and /usr/lib and make /usr/lib the preferred path?

Maybe, but it could result in the opposite bug in some cases. Consider
the SONAME libfoo.so.0, and assume the multiarch tuple is MA.

1. libfoo.so.0.1.2 is in /lib/MA
2. maintainer moves libfoo.so.0.2.3 from /usr/lib/MA to /lib/MA because
   some early-boot service needs libfoo.so.0
3. somehow libfoo.so.0.1.2 is not deleted during the upgrade
4. ldconfig creates libfoo.so.0 -> libfoo.so.0.1.2 in /lib/MA
   and libfoo.so.0 -> libfoo.so.0.2.3 in /usr/lib/MA
(several years later)
5. now we have libfoo.so.0 -> libfoo.so.0.1.2 in /lib/MA and
   libfoo.so.0 -> libfoo.so.0.4.2 in /usr/lib/MA
6. we change ld.so.conf.d so that /usr/lib/MA is preferred over /lib/MA
7. programs that depend on libfoo0 (>= 0.4.2) get libfoo.so.0.1.2, and
   are broken, because symbols introduced since 0.1.2 are missing

However, in many cases we might dodge that bullet in practice because the
transition to multiarch happened between steps 4 and 5, and /usr/lib/MA
and /lib/MA are both preferred over /usr/lib and /lib. For example,
libdbus-1-3 transitioned from /usr/lib to /lib in 1.2.16-1, from /lib
to /lib/MA in 1.4.12-3, and from /lib/MA to /usr/lib/MA in 1.13.10-1;
so it might get a bug equivalent to #896019 if we don't reverse the
priorities of /lib/MA and /usr/lib/MA, but it can't have the bug I
describe above if we do reverse their priorities.

    smcv

Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

smcv
In reply to this post by Guillem Jover
On Sat, 15 Feb 2020 at 23:21:35 +0100, Guillem Jover wrote:
> Doing a proper /usr-merged migration is what we should have done from
> the beginning. I've been doing that with all the library packages I
> maintain that were under /lib. That includes acl, attr, libaio, libbsd
> and libmd, and I know others have been doing this too, so there's
> plenty of precedent with this.

To be clear, what Guillem means by "a proper /usr-merged migration"
here is changing individual library packages, so that the path to their
libraries that exists in the data.tar.* and in the dpkg database changes
from for example /lib/MA/libfoo.so.0 to /usr/lib/MA/libfoo.so.0. This is
not the same thing that a lot of other people mean by "the /usr merge".

I think we have consensus that consolidating all static OS files into /usr
(removing the distinction between /usr and the static parts of the root
filesystem) is the route that Debian is taking. I think we do not have
consensus on how that is to be achieved.

I would be grateful if people who advocate transitioning individual
packages, and people who consider the approach taken by usrmerge and
debootstrap to be sufficient, could refer to their preferred route in a
way that makes it clear which one they are advocating. Saying we should
do a transition "properly" is tautologous - of course we should! - but
when people disagree about what the proper way to do it is, it becomes an
ambiguous recommendation that doesn't guide anyone to do the right thing.

The approach that transitions individual packages solves some bugs
(notably, if you ask dpkg which package owns /usr/lib/MA/libfoo.so.0,
you get the right answer, which has a lot of desirable consequences). It
also appears to *cause* some bugs. The ones I know about are:

* packages that hard-code paths into /lib stop working on systems that
  have *not* undergone the usrmerge-style /usr merge, because
  /lib/MA/libfoo.so.0 no longer exists (for example see #950715
  involving libgcc.so.1 and the gold linker);
* there is a class of bugs on systems that have *not* undergone the
  usrmerge-style /usr merge, involving old libraries lingering in /lib/MA
  (see #949395 for a summary of the instances that I know about), which
  are very hard to debug because they are unreproducible, to do with the
  state of an individual system, and are related to upgrades that happened
  years in the past and whose logs expired long ago;
* when paths migrate between package names and between paths at the same
  time, there can be undetected file conflicts on systems that *have*
  undergone the usrmerge-style /usr merge (for example see #950624,
  again in libgcc.so.1)

The term "/usr merge" or "merged /usr" more commonly refers to making
/usr/lib and /lib, /usr/bin and /bin, etc., *equivalent* (with symlinks
/lib -> usr/lib, etc.), as implemented by the usrmerge package and
debootstrap --merged-usr.

In many other distributions, for example Fedora, there was a flag day at
which merged /usr became mandatory. This makes various classes of bugs
like #949395 and #950715 cease to exist: for example, it doesn't matter
if /lib/MA/libfoo.so.0 is hard-coded somewhere, because that path still
exists (realpath would resolve it to /usr/lib/MA/libfoo.so.0).
(However, this can't solve everything: #950624 would still have been a
bug if we had had a merged /usr flag-day.)

In Debian we have (as usual) made life more difficult for ourselves by the
/usr merge being optional, which means that in any transition or upgrade
scenario, maintainers have to consider two cases: one where the system
has undergone the /usr merge, and one where it has not. For example,
#950624 only happens on systems that have undergone the /usr merge,
while #950715 and #949395 only happen on systems that have not. Testing
on any single system cannot detect all of these: to detect all the bugs,
we have to try both configurations.

    smcv

Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Marco d'Itri
On Feb 16, Simon McVittie <[hidden email]> wrote:

> To be clear, what Guillem means by "a proper /usr-merged migration"
> here is changing individual library packages, so that the path to their
Everything I suppose, not just libraries.

> I think we have consensus that consolidating all static OS files into /usr
> (removing the distinction between /usr and the static parts of the root
> filesystem) is the route that Debian is taking. I think we do not have
> consensus on how that is to be achieved.
Do we really have a consensus? Is everybody persuaded now that there is
no point in continuing to support non-merged systems?

> I would be grateful if people who advocate transitioning individual
> packages, and people who consider the approach taken by usrmerge and
> debootstrap to be sufficient, could refer to their preferred route in a
> way that makes it clear which one they are advocating. Saying we should
I never considered usrmerge (the package) to be the complete solution
but only a transition method.
I fully support plans to start moving everything from /bin /sbin
/lib to /usr, but I did not want to wait the many years that this will
take to have a real merged-/usr system.
Libraries are easy to move since the linker will find them no matter
where they are installed, but moving the binaries too will require
merging older systems. And to do this we need some work in the usrmerge
package. Short summary: it used to work very well on a live system, but
nowadays systemd creates a lot of bind mounts so it should be run from
the initramfs. But I have not been able to actually do this: if anybody
wants to have a look at it there is a branch in the repository.
I see no point in only moving libraries.

--
ciao,
Marco

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Sam Hartman-3
>>>>> "Marco" == Marco d'Itri <[hidden email]> writes:
    >> I think we have consensus that consolidating all static OS files
    >> into /usr (removing the distinction between /usr and the static
    >> parts of the root filesystem) is the route that Debian is
    >> taking. I think we do not have consensus on how that is to be
    >> achieved.
    Marco> Do we really have a consensus? Is everybody persuaded now
    Marco> that there is no point in continuing to support non-merged
    Marco> systems?

That is not my understanding of the TC decision no.
My understanding without rereading a complex decision is that for now
we're still supporting both.
But that it's certainly fine if packages move things to /usr if they do
so in a way that avoids breaking things.

Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

smcv
In reply to this post by Marco d'Itri
On Tue, 18 Feb 2020 at 12:44:18 +0100, Marco d'Itri wrote:
> On Feb 16, Simon McVittie <[hidden email]> wrote:
> > I think we have consensus that consolidating all static OS files into /usr
> > (removing the distinction between /usr and the static parts of the root
> > filesystem) is the route that Debian is taking. I think we do not have
> > consensus on how that is to be achieved.
>
> Is everybody persuaded now that there is
> no point in continuing to support non-merged systems?

No. Sorry, I phrased that badly. The consensus that I think we have is:
we are no longer attempting to support systems booting without /usr
mounted, and therefore it is not a bug if programs and libraries on the
rootfs have dependencies in /usr. (That's a less strong guarantee than
the one you are probably hoping for.)

That means that libraries and executables whose paths are not hard-coded
anywhere are allowed to migrate from the root filesystem to /usr (modulo
warnings about possible bugs: see elsewhere in this thread).

However, it doesn't give us a solution for what should happen to things
that are canonically on the root filesystem and *do* have their absolute
paths hard-coded somewhere, most critically /lib*/ld*.so.* and /bin/sh.

One of those mechanisms to consolidate files into /usr is to redirect files
en masse "behind dpkg's back", making use of dpkg's support for unpacking
through symlinks-to-directories. This is how debootstrap --merged-usr
works, and also how a system with usrmerge installed continues to work
afterwards. It has the advantage that it works for all files, even those
that get their paths hard-coded in various places: after undergoing this
transition, /PATH and /usr/PATH can be used interchangeably (for any
PATH in one of the directories that was historically on the rootfs).
This is the result that I think most people mean when they say "the /usr
merge" or "merged /usr".

The other mechanism is migrating things from the rootfs into /usr
individually, on a per-package basis. GLib's move from /lib/MULTIARCH
to /usr/lib/MULTIARCH is a typical example. I think it's misleading to
refer to this as "merged /usr" or "/usr merge" - although it can be
*a step towards* that - because in most cases, after this transition,
the two paths are *not* equivalent (one exists, the other doesn't),
unless the first mechanism has *also* been used.

> Libraries are easy to move since the linker will find them no matter
> where they are installed, but moving the binaries too will require
> merging older systems.

Yes.

I think some people (perhaps including Guillem?) might be advocating
including executables in the second mechanism described above by
moving them, on a per-package basis, to the root filesystem, and
creating compatibility symlinks on the root filesystem if it has not
undergone the /usr merge, like /bin/plymouth -> /usr/bin/plymouth and
/sbin/iptables -> /usr/sbin/iptables. However, so far, this is rare
(plymouth and iptables are among only a few examples on my laptop)
and if adopted, it seems likely to be a slow transition: most of the
affected packages are exactly those that are critical for boot, so their
maintainers tend to be quite conservative. One thing to watch out for
with this approach is that the symlinks must be created from the
maintainer scripts instead of being shipped in the data.tar.*, because
otherwise they would break the usrmerge-style approach to achieving
merged /usr.

Another issue with this approach is that it has quite a lot of moving parts
(see recent upgrade issues with libgcc1 and libgcc-s1 for an example of
how even a very experienced maintainer can reach unexpected situations in
a transition like this).

    smcv

Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Marco d'Itri
On Feb 18, Simon McVittie <[hidden email]> wrote:

> No. Sorry, I phrased that badly. The consensus that I think we have is:
> we are no longer attempting to support systems booting without /usr
> mounted, and therefore it is not a bug if programs and libraries on the
> rootfs have dependencies in /usr. (That's a less strong guarantee than
> the one you are probably hoping for.)
We do not just have a consensus, this has also been a fact for a long
time now:

kmod (25-2) unstable; urgency=medium

  * Moved the libraries to /usr/lib/. (Closes: #894566)

 -- Marco d'Itri <[hidden email]>  Sat, 17 Nov 2018 01:56:00 +0100

> However, it doesn't give us a solution for what should happen to things
> that are canonically on the root filesystem and *do* have their absolute
> paths hard-coded somewhere, most critically /lib*/ld*.so.* and /bin/sh.
This does not matter as long as we have to support un-merged-/usr
systems.

> I think some people (perhaps including Guillem?) might be advocating
> including executables in the second mechanism described above by
> moving them, on a per-package basis, to the root filesystem, and
> creating compatibility symlinks on the root filesystem if it has not
> undergone the /usr merge, like /bin/plymouth -> /usr/bin/plymouth and
> /sbin/iptables -> /usr/sbin/iptables. However, so far, this is rare
> (plymouth and iptables are among only a few examples on my laptop)
> and if adopted, it seems likely to be a slow transition: most of the
Slow, and also a lot of work for no significant benefits.

--
ciao,
Marco

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Andreas Henriksson-3
Hello,

On Tue, Feb 18, 2020 at 10:10:05PM +0100, Marco d'Itri wrote:
> On Feb 18, Simon McVittie <[hidden email]> wrote:
>
> > No. Sorry, I phrased that badly. The consensus that I think we have is:
> > we are no longer attempting to support systems booting without /usr
> > mounted, and therefore it is not a bug if programs and libraries on the
> > rootfs have dependencies in /usr. (That's a less strong guarantee than
> > the one you are probably hoping for.)
> We do not just have a consensus, this has also been a fact for a long
> time now:

.. and is completely unrelated to merged-usr (?!).

>
> kmod (25-2) unstable; urgency=medium
>
>   * Moved the libraries to /usr/lib/. (Closes: #894566)
>
>  -- Marco d'Itri <[hidden email]>  Sat, 17 Nov 2018 01:56:00 +0100
>
> > However, it doesn't give us a solution for what should happen to things
> > that are canonically on the root filesystem and *do* have their absolute
> > paths hard-coded somewhere, most critically /lib*/ld*.so.* and /bin/sh.
> This does not matter as long as we have to support un-merged-/usr
> systems.
>
> > I think some people (perhaps including Guillem?) might be advocating
> > including executables in the second mechanism described above by
> > moving them, on a per-package basis, to the root filesystem, and
> > creating compatibility symlinks on the root filesystem if it has not
> > undergone the /usr merge, like /bin/plymouth -> /usr/bin/plymouth and
> > /sbin/iptables -> /usr/sbin/iptables. However, so far, this is rare
> > (plymouth and iptables are among only a few examples on my laptop)
> > and if adopted, it seems likely to be a slow transition: most of the
> Slow, and also a lot of work for no significant benefits.

As I see it there are two possible ways forward:

a/ make merged-usr mandatory in one stable release and in the next
   we can just install the files under /usr. No per-package symlinks
   needed.
b/ someone volunteers to write a debhelper addon which runs after
   dh_install, detects files in /lib, /bin and /sbin, moves them
   into /usr and generates the needed postinst code doing the compat
   symlinks for non-merged systems.


The third option that I see as a no-go is that each package maintainer
implements this on their own because:
1. Getting this finished will take way too long.
2. Manually written maintainer scripts is well known source of bugs.
3. Broken maintainer scripts are a horrible user experience.
4. People will keep repeating the mistake of shipping the symlinks
   in the package (once they realize manually written maintainer scripts
   are horrible?!) and thus break their package on merged-usr systems.
5. Doing the work this way will consume too much resources for no gain.
6. People being fine with the current status quo will not like 5/
   further contributing to 1/.


I think it's time we either try to figure out if/how we can actually
get a decision on doing a/ soon, or for those who can't live with
one release making merged-usr mandatory then start working on
that debhelper needed to do b/ now!

(Also when/if doing b/ or the 'no-go' option someone might want to think
about how per-package symlinks generated in postinst is going to play
nice with Essential: yes packages which are supposed to work
unconfigured.)

Regards,
Andreas Henriksson

Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Guillem Jover
In reply to this post by smcv
On Sun, 2020-02-16 at 11:59:56 +0000, Simon McVittie wrote:
> I would be grateful if people who advocate transitioning individual
> packages, and people who consider the approach taken by usrmerge and
> debootstrap to be sufficient, could refer to their preferred route in a
> way that makes it clear which one they are advocating. Saying we should
> do a transition "properly" is tautologous - of course we should! - but
> when people disagree about what the proper way to do it is, it becomes an
> ambiguous recommendation that doesn't guide anyone to do the right thing.

I've been consistently calling the concept of merging /* into /usr/* as
merged-/usr and the specific approach of using directory symlinks as
merged-/usr-via-symlinks (although I think that's confusing as the
other approach does use symlink farms), so I think using either
merged-/usr-via-aliased-dirs or merged-/usr-via-symlink-dirs is more
clear (will be renaming the buildinfo tainted tag). The approach I've
been proposing I'd call merged-/usr-via-moves-and-symlink-farms or
something along those lines.

I've dumped this all into <https://wiki.debian.org/Teams/Dpkg/MergedUsr>
and updated the Dpkg FAQ:

<https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F>

> The approach that transitions individual packages solves some bugs
> (notably, if you ask dpkg which package owns /usr/lib/MA/libfoo.so.0,
> you get the right answer, which has a lot of desirable consequences). It
> also appears to *cause* some bugs. The ones I know about are:
>
> * packages that hard-code paths into /lib stop working on systems that
>   have *not* undergone the usrmerge-style /usr merge, because
>   /lib/MA/libfoo.so.0 no longer exists (for example see #950715
>   involving libgcc.so.1 and the gold linker);

For any pathname that has been hardcoded a symlink can be used for
backwards compat, nothing unlike /bin or /sbin here. This looks just
like a normal bug from a botched transition, nothing special.

> * there is a class of bugs on systems that have *not* undergone the
>   usrmerge-style /usr merge, involving old libraries lingering in /lib/MA
>   (see #949395 for a summary of the instances that I know about), which
>   are very hard to debug because they are unreproducible, to do with the
>   state of an individual system, and are related to upgrades that happened
>   years in the past and whose logs expired long ago;

merged-/usr (in general) seems irrelevant here, if this is really a bug
in dpkg, then it needs fixing regardless of any approach taken, as there
is nothing in dpkg really distinguishing between / and /usr as it is
pathname neutral.

I've pending to comment on that bug, but I was skimming over the involved
code and didn't see anything obviously wrong that would forget pathnames.
But my first suspect would be that we are not doing fsync() on the parent
directories, the second would be perhaps related to multiarch refcounting
logic, but ISTR this having happened in old glibc perhaps before multiarch
was deployed, but would need to check.

> * when paths migrate between package names and between paths at the same
>   time, there can be undetected file conflicts on systems that *have*
>   undergone the usrmerge-style /usr merge (for example see #950624,
>   again in libgcc.so.1)

This is one of the issues inherent in merged-/usr-via-aliased-dirs.

> In Debian we have (as usual) made life more difficult for ourselves by the
> /usr merge being optional,

I fully agree, but not on the reason for the root cause. Allowing the
merged-/usr-via-aliased-dirs hack at all, has meant that we cannot do
fully automatic migrations via debhelper with no maintainer scripts
involved whatsoever. :(

> which means that in any transition or upgrade
> scenario, maintainers have to consider two cases: one where the system
> has undergone the /usr merge, and one where it has not. For example,
> #950624 only happens on systems that have undergone the /usr merge,
> while #950715 and #949395 only happen on systems that have not. Testing
> on any single system cannot detect all of these: to detect all the bugs,
> we have to try both configurations.

The difference between these issues IMO is one between bugs (a botched
migration with no compat symlinks, and a potential dpkg bug), and design
flaws inherent in the merged-/usr-via-aliased-dirs approach.

I'm not sure how anyone could claim that the merged-/usr-via-aliased-dirs
approach is not a hack, even if it might have some possible benefits,
when it circumvents the package manager completely (we might as well
ditch it and go use bare tarballs or something :/…), and implies action
at a distance where the filesystem layout is injected from something
that is not packaged at all and should not be packaged (or we'd be forced
to hardcode bootstrapping ordering into each bootstrapper forever) so we
also lose locality from the .deb.

That approach would be acceptable in a distant future once and iff we ship
no objects under top root directories, which for now would imply breaking
ABI/APIs. Doing the proposed mandatory merged-/usr-via-aliased-dirs for a
release cycle, and moving everything en mass, still does not fix the
aliasing problems for all the paths that are hardcoded on non-/usr
locations. This would mean painting ourselves into an even tinier and
messier corner than where we are now…

IMO the technically better and more sound solution would be to devise
a way to revert the mess caused by the merged-/usr-via-aliased-dirs
approach, and then convert everything automatically via debhelper.

Thanks,
Guillem

Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

The Wanderer
On 2020-02-18 at 20:50, Guillem Jover wrote:

> On Sun, 2020-02-16 at 11:59:56 +0000, Simon McVittie wrote:
>
>> I would be grateful if people who advocate transitioning
>> individual packages, and people who consider the approach taken by
>> usrmerge and debootstrap to be sufficient, could refer to their
>> preferred route in a way that makes it clear which one they are
>> advocating. Saying we should do a transition "properly" is
>> tautologous - of course we should! - but when people disagree about
>> what the proper way to do it is, it becomes an ambiguous
>> recommendation that doesn't guide anyone to do the right thing.
>
> I've been consistently calling the concept of merging /* into /usr/*
> as merged-/usr and the specific approach of using directory symlinks
> as merged-/usr-via-symlinks (although I think that's confusing as
> the other approach does use symlink farms), so I think using either
> merged-/usr-via-aliased-dirs or merged-/usr-via-symlink-dirs is more
> clear (will be renaming the buildinfo tainted tag). The approach
> I've been proposing I'd call merged-/usr-via-moves-and-symlink-farms
> or something along those lines.
As a tangent, because this has never made sense to me:

Is there a reason this is all looking to merge /* into /usr/* instead of
the other way around?

The latter looks to me as if it would ultimately make it possible to
drop the /usr directory entirely (once we've had a release or three in
which it contains nothing but symlinks) - whereas the former means we'd
have the extra four characters at the head of nearly every path to
installed software forever, for no apparent benefit, and would also seem
to mean we'd have to keep the symlinks in / around potentially forever,
even after the transition has been complete for yonks.

I'm not sure I'd support dropping /usr, but I don't think I can remember
having ever run across (or come up with) any arguments against it which
don't seem like they would also be arguments against doing this merge at
all, so the fact that the merge is apparently being done in this
direction has consistently puzzled me.

Does keeping /usr have some kind of benefit in its own right which I'm
not seeing, even for cases when it's on the root filesystem?

(Answers in the forms of pointers to Websites, or even to archives of
past discussion where this would be made clear, are entirely
acceptable.)

--
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is there still a point in installing libgcrypt to /lib instead of /usr/lib

Marco d'Itri
In reply to this post by Guillem Jover
On Feb 19, Guillem Jover <[hidden email]> wrote:

> For any pathname that has been hardcoded a symlink can be used for
> backwards compat, nothing unlike /bin or /sbin here. This looks just
> like a normal bug from a botched transition, nothing special.
Creating symlinks in /bin and /sbin DOES NOT result in a merged-/usr
system, because the content of /usr would not be decoupled anymore from
the content of /.
A merged-/usr system must have /bin /sbin /lib* symlinks to /usr.

What you are proposing is NOT an alternative implementation of
merged-/usr but something else, which has no significant benefits.

--
ciao,
Marco

signature.asc (235 bytes) Download Attachment
12