Bug#794495: Mock fails to track installed packages

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug#794495: Mock fails to track installed packages

Vishvananda Ishaya
Package: mock
Version: 1.2.3-1

Mock fails to track installed packages properly due to the rpm package changing the default rpmdb directory to ~/.rpmdb from /var/lib/rpm.

There is an existing patch to remove the extra /buildroot/.rpmdb dir that gets created during the install, but we actually want the rpmdb to be in /var/lib/rpm

To see an example of the problem, try:
/usr/bin/mock --init
followed by
/usr/bin/mock --yum-cmd -- -C list installed
It will show an empty list of packages because it can't find the rpmdb. This means that yum will attempt to reinstall packages that are already installed when doing rpm builds leading to subtle differences in dependency ordering when building on another distro.

I have tried passing in the correct macro using -D:
 /usr/bin/mock -D '_dbpath %{_var}/lib/rpm' --init
I have tried editing my user's .rpmmacros to add:
%_dbpath %{_var}/lib/rpm
Neither of these works.
Editing /usr/lib/rpm/macros to manually undo the .rpmdb patch works:
%_dbpath        %{_var}/lib/rpm

I'm not sure what the best fix is here. This may also need to be reported against rpm, because the best option might be to remove the patch to /usr/lib/rpm/macros and instead document how a user could manually add the change to their .rpmmacros if they want it to include:
%_dbpath %(echo $HOME)/.rpmdb

Vish
Reply | Threaded
Open this post in threaded view
|

Bug#794495: Mock fails to track installed packages

Tzafrir Cohen
tags -1 + patch
reassign -1 rpm
thanks

Hi,

Sorry for not answering earlier,

On Mon, Aug 03, 2015 at 06:08:49PM +0000, Vishvananda Ishaya wrote:

> Package: mock
> Version: 1.2.3-1
>
> Mock fails to track installed packages properly due to the rpm package
> changing the default rpmdb directory to ~/.rpmdb from /var/lib/rpm.
>
> There is an existing patch to remove the extra /buildroot/.rpmdb dir that
> gets created during the install, but we actually want the rpmdb to be in
> /var/lib/rpm
>
> To see an example of the problem, try:
> /usr/bin/mock --init
> followed by
> /usr/bin/mock --yum-cmd -- -C list installed
> It will show an empty list of packages because it can't find the rpmdb.
> This means that yum will attempt to reinstall packages that are already
> installed when doing rpm builds leading to subtle differences in dependency
> ordering when building on another distro.
>
> I have tried passing in the correct macro using -D:
>  /usr/bin/mock -D '_dbpath %{_var}/lib/rpm' --init
> I have tried editing my user's .rpmmacros to add:
> %_dbpath %{_var}/lib/rpm
> Neither of these works.
Mock runs as root mostly. Maybe it reads root's .rpmmacros?

> Editing /usr/lib/rpm/macros to manually undo the .rpmdb patch works:
> %_dbpath        %{_var}/lib/rpm
>
> I'm not sure what the best fix is here. This may also need to be reported
> against rpm, because the best option might be to remove the patch to
> /usr/lib/rpm/macros and instead document how a user could manually add the
> change to their .rpmmacros if they want it to include:
> %_dbpath %(echo $HOME)/.rpmdb

This is a Debian-specific change in the package rpm. RPM automatically
attempts to create its database if it does not exist. On Debian it does
not exist. This creates a harmless but noisy and annoying error message
whenever you try to use rpm on the Debian system.

Why would you use rpm on a Debian system?

* rpm -qp: annoying message
* rpm operations as root inside an RPM-based system in a chroot: that
  fix breaks it.

I can't think of any way to fix this in mock or in yum. I think that the
proper way to fix it is in rpm: instead of editing _dbpath, don't
complain when you get EPERM trying to open the packages database. A
patch is attached (against 4.12.0.2, as it seems that patches fail to
apply for 4.13.0.1 in master).

I did not look deeper into it. Maybe it's possible to avoid opening the
database on rpm -qp?

--
Tzafrir Cohen         | [hidden email] | VIM is
http://tzafrir.org.il |                    | a Mutt's
[hidden email] |                    |  best
[hidden email]    |                    | friend

0001-Add-hush_rpmdb_warning.patch.patch (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#794495: Mock fails to track installed packages

Michal Čihař
Hi

On Tue, 2017-04-18 at 14:12 +0200, Tzafrir Cohen wrote:

> This is a Debian-specific change in the package rpm. RPM
> automatically
> attempts to create its database if it does not exist. On Debian it
> does
> not exist. This creates a harmless but noisy and annoying error
> message
> whenever you try to use rpm on the Debian system.
>
> Why would you use rpm on a Debian system?
>
> * rpm -qp: annoying message
> * rpm operations as root inside an RPM-based system in a chroot: that
>   fix breaks it.
Can't you just edit the file when constructing chroot? Or am I missing
something here?

> I can't think of any way to fix this in mock or in yum. I think that
> the
> proper way to fix it is in rpm: instead of editing _dbpath, don't
> complain when you get EPERM trying to open the packages database. A
> patch is attached (against 4.12.0.2, as it seems that patches fail to
> apply for 4.13.0.1 in master).

With your patch it still does complain, what would make the most usual
use case for rpm (alien) pretty annoying:

$ fakeroot alien blacs-doc-1.1-1.2.x86_64.rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
error: cannot open Packages index using db5 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
blacs-doc_1.1-2.2_amd64.deb generated

I don't think this is acceptable.

> I did not look deeper into it. Maybe it's possible to avoid opening
> the
> database on rpm -qp?

AFAIK it was not possible as it at least tries to open the keys
database to verify signatures (which usually doesn't exist on Debian).

I'm open to better solutions, but they can not break existing
workflows.

--
        Michal Čihař | https://cihar.com/ | https://weblate.org/

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#794495: Mock fails to track installed packages

Mihai Moldovan
Hi Michal


I don't want to be condescending, but can we please, pretty-please stop breaking
rpm in Debian?

I understand that rpm is not intended to be used as a package manager on Debian
systems, but the rpmdb-in-homedir patch does more harm than good, causing
trouble in totally legitimate use cases as described in this bug report.

Users should be educated and preferably not even try to install RPM packages in
the first place, but putting the rpmdb into a user's home directory won't really
do any good.

Users wanting to use the rpm package within Debian to install rpm packages could
either just edit the global macro file and reset _dbpath to /var/lib/rpm, create
/var/lib/rpm and symlinks from ~/.rpmdb to that location or do even weirder things.

The point is that even with this patch, rpm won't be unusable in the way you'd
like to make it, even if that requires very minor tweaking.


However, without tinkering, it breaks other software like mock or libsolv's RPM
backend. One could argue about the latter's use(fulness) on Debian, but breaking
mock is breaking a legitimate use case in which the rpm package really NEEDS to
install software correctly.

mock is some piece of software for building RPM packages (source or binaries) in
a chroot - quite like sbuild for Debian packages. I do use it a lot, but this
patch breaks mock in a very subtle way.

Looong explanation coming up: mock creates a new chroot and then uses an RPM
package manager (for instance yum, but potentially also dnf) to install a base
set of packages in there. Again, pretty much like debootstrap. For that to work,
the package manager is executed with the --installroot parameter, which
instructs it to essentially carry out all operations in the chroot - including
using the rpmdb in there. mock also sets HOME to /builddir, hence the rpmdb will
be created in /chroot/builddir/.rpmdb. Everything's fine until then, but then
mock fully deletes the home directory (within the chroot, of course) and
recreates it from scratch/skel. Boom, the rpmdb went bust.

This wasn't a huge problem per se, since in the worst case the package manager
would just install more packages than necessary when installing a source RPM's
build dependencies. Recently, however, rpm and dnf got support for rich
dependencies, which are also heavily used by Fedora at least (and likely will be
used by RHEL 8 as well). Such "rich dependencies" can be used to pull in
optional dependencies if a different package is already installed, for instance,
to automatically enhance functionality - notably, the "different package" need
not be any any direct or indirect dependency at all.

A real-world breaking example (with dnf, since yum doesn't support rich
dependencies, but I have dnf packages for Debian and would like to introduce
them at some point):

redhat-rpm-config has a dependency such as this:
Requires: (annobin if gcc)

This will pull in "annobin" as a dependency of redhat-rpm-config if the "gcc"
package is installed, but redhat-rpm-config does not depend upon "gcc".

What now happens is breakage: gcc is part of the base package set, so is
installed when creating the chroot. Since the rpmdb goes bust, the package
manager won't know it's installed and when a build-dependency (indirectly) pulls
in redhat-rpm-config, which is NOT part of base, it won't pull in annobin.

This then leads to build failures, since the annobin plugin is being used
unconditionally at build time. If it's not installed, builds fail.


Such breakage is subtle, and it's likely not the only thing to go haywire with
the patched rpm binaries.

Yes, I COULD go ahead and add .rpmdb to mock's config file option
"exclude_from_homedir_cleanup", but that would likely create even more problems.
Newer mock versions (currently not available in Debian, but I guess the mock
package will be updated eventually) can bootstrap the build system using the
system-provided package manager and then use the chroot-based package manager
for installing build dependencies. With the system rpm library/binary patched to
put the rpmdb in ${HOME}, but Fedoras/CentOS's not, we'd run into a very nice
/builddir/.rpmdb vs. /var/lib/rpm dance yet again.


Yet another argument: it makes other maintainer's lifes more difficult since
software like yum, dnf or libsolv have to be specifically patched to even work.


I can only re-iterate: Please consider dropping this patch.



Mihai


signature.asc (916 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#794495: Mock fails to track installed packages

Michal Čihař
Hello

Sorry, I didn't read your long email. I just want to point out that rpm
package is available for adoption, see https://bugs.debian.org/923352

--
        Michal Čihař | https://cihar.com/ | https://weblate.org/


Mihai Moldovan píše v Čt 05. 09. 2019 v 15:55 +0200:

> Hi Michal
>
>
> I don't want to be condescending, but can we please, pretty-please
> stop breaking
> rpm in Debian?
>
> I understand that rpm is not intended to be used as a package manager
> on Debian
> systems, but the rpmdb-in-homedir patch does more harm than good,
> causing
> trouble in totally legitimate use cases as described in this bug
> report.
>
> Users should be educated and preferably not even try to install RPM
> packages in
> the first place, but putting the rpmdb into a user's home directory
> won't really
> do any good.
>
> Users wanting to use the rpm package within Debian to install rpm
> packages could
> either just edit the global macro file and reset _dbpath to
> /var/lib/rpm, create
> /var/lib/rpm and symlinks from ~/.rpmdb to that location or do even
> weirder things.
>
> The point is that even with this patch, rpm won't be unusable in the
> way you'd
> like to make it, even if that requires very minor tweaking.
>
>
> However, without tinkering, it breaks other software like mock or
> libsolv's RPM
> backend. One could argue about the latter's use(fulness) on Debian,
> but breaking
> mock is breaking a legitimate use case in which the rpm package
> really NEEDS to
> install software correctly.
>
> mock is some piece of software for building RPM packages (source or
> binaries) in
> a chroot - quite like sbuild for Debian packages. I do use it a lot,
> but this
> patch breaks mock in a very subtle way.
>
> Looong explanation coming up: mock creates a new chroot and then uses
> an RPM
> package manager (for instance yum, but potentially also dnf) to
> install a base
> set of packages in there. Again, pretty much like debootstrap. For
> that to work,
> the package manager is executed with the --installroot parameter,
> which
> instructs it to essentially carry out all operations in the chroot -
> including
> using the rpmdb in there. mock also sets HOME to /builddir, hence the
> rpmdb will
> be created in /chroot/builddir/.rpmdb. Everything's fine until then,
> but then
> mock fully deletes the home directory (within the chroot, of course)
> and
> recreates it from scratch/skel. Boom, the rpmdb went bust.
>
> This wasn't a huge problem per se, since in the worst case the
> package manager
> would just install more packages than necessary when installing a
> source RPM's
> build dependencies. Recently, however, rpm and dnf got support for
> rich
> dependencies, which are also heavily used by Fedora at least (and
> likely will be
> used by RHEL 8 as well). Such "rich dependencies" can be used to pull
> in
> optional dependencies if a different package is already installed,
> for instance,
> to automatically enhance functionality - notably, the "different
> package" need
> not be any any direct or indirect dependency at all.
>
> A real-world breaking example (with dnf, since yum doesn't support
> rich
> dependencies, but I have dnf packages for Debian and would like to
> introduce
> them at some point):
>
> redhat-rpm-config has a dependency such as this:
> Requires: (annobin if gcc)
>
> This will pull in "annobin" as a dependency of redhat-rpm-config if
> the "gcc"
> package is installed, but redhat-rpm-config does not depend upon
> "gcc".
>
> What now happens is breakage: gcc is part of the base package set, so
> is
> installed when creating the chroot. Since the rpmdb goes bust, the
> package
> manager won't know it's installed and when a build-dependency
> (indirectly) pulls
> in redhat-rpm-config, which is NOT part of base, it won't pull in
> annobin.
>
> This then leads to build failures, since the annobin plugin is being
> used
> unconditionally at build time. If it's not installed, builds fail.
>
>
> Such breakage is subtle, and it's likely not the only thing to go
> haywire with
> the patched rpm binaries.
>
> Yes, I COULD go ahead and add .rpmdb to mock's config file option
> "exclude_from_homedir_cleanup", but that would likely create even
> more problems.
> Newer mock versions (currently not available in Debian, but I guess
> the mock
> package will be updated eventually) can bootstrap the build system
> using the
> system-provided package manager and then use the chroot-based package
> manager
> for installing build dependencies. With the system rpm library/binary
> patched to
> put the rpmdb in ${HOME}, but Fedoras/CentOS's not, we'd run into a
> very nice
> /builddir/.rpmdb vs. /var/lib/rpm dance yet again.
>
>
> Yet another argument: it makes other maintainer's lifes more
> difficult since
> software like yum, dnf or libsolv have to be specifically patched to
> even work.
>
>
> I can only re-iterate: Please consider dropping this patch.
>
>
>
> Mihai
>


signature.asc (849 bytes) Download Attachment