AMDGPU+OpenCL with Debian?

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

AMDGPU+OpenCL with Debian?

Steffen Möller
Hello,

Running Debian unstable, I failed to set up OpenCL to work with BOINC
and an AMD RX580. The card worked just fine with the monitor, but it was
not recognised as an OpenCL device by BOINC. When I then tried to
install the 19.10 release of the driver AMD provides on
https://www.amd.com/en/support/graphics/radeon-500-series/radeon-rx-500-series/radeon-rx-580
on our distribution, the kernel module did not compile.

I then installed Ubuntu and basically did not need to do anything - it
just worked upon installing boinc-client-opencl. I could also install
the .debs provided by AMD with no difficulty. Are there others on this
list with similar experiences under Debian? Is there something we
can/want to do to help that situation?

Cheers,

Steffen

Reply | Threaded
Open this post in threaded view
|

RE:AMDGPU+OpenCL with Debian?

PICCA Frederic-Emmanuel
Same here... with WXX100 cards.
what about rocm packaging ?
________________________________________
De : Steffen Möller [[hidden email]]
Envoyé : lundi 17 juin 2019 20:14
À : [hidden email]
Objet : AMDGPU+OpenCL with Debian?

Hello,

Running Debian unstable, I failed to set up OpenCL to work with BOINC
and an AMD RX580. The card worked just fine with the monitor, but it was
not recognised as an OpenCL device by BOINC. When I then tried to
install the 19.10 release of the driver AMD provides on
https://www.amd.com/en/support/graphics/radeon-500-series/radeon-rx-500-series/radeon-rx-580
on our distribution, the kernel module did not compile.

I then installed Ubuntu and basically did not need to do anything - it
just worked upon installing boinc-client-opencl. I could also install
the .debs provided by AMD with no difficulty. Are there others on this
list with similar experiences under Debian? Is there something we
can/want to do to help that situation?

Cheers,

Steffen

Reply | Threaded
Open this post in threaded view
|

Re: AMDGPU+OpenCL with Debian?

Carsten Schoenert
Hi,

Am 17.06.19 um 21:15 schrieb PICCA Frederic-Emmanuel:
> Same here... with WXX100 cards.
> what about rocm packaging ?

(I'm not using an AMD graphic card but ...)

there was recently a new article added to the Debian wiki regarding this
topic about using the official AMDGPU driver. Maybe this helps solving
some questions?

https://wiki.debian.org/How%20to%20install%20official%20AMDGPU%20linux%20driver%20with%20kernel%204.19.x%20on%20Stretch%20and%20Buster

--
Regards
Carsten Schoenert

Reply | Threaded
Open this post in threaded view
|

Re: AMDGPU+OpenCL with Debian?

Mo Zhou
In reply to this post by PICCA Frederic-Emmanuel
On 2019-06-18 03:15, PICCA Frederic-Emmanuel wrote:
> Same here... with WXX100 cards.
> what about rocm packaging ?

Not easy. Involves a toolchain.
Is ROCm promising enough to challenge CUDA?
(Although ROCm has already beaten CUDA in
terms of license).

Both tensorflow and pytorch supports
ROCm. I'm interested in a formal
benchmark for comparison with cuda.
Scattered user comments are not always
consistent to each other.

The proprietary CUDA compiler's
support for gcc and clang is always
lagging behind the latest toolchain
state in sid. That makes the maintenance
work less interesting. At this point
ROCm should be better.
I hope AMD could help me get rid of
nvidia non-free blobs someday in the
future.

Reply | Threaded
Open this post in threaded view
|

Re: AMDGPU+OpenCL with Debian?

Steffen Möller
In reply to this post by Carsten Schoenert

On 17.06.19 22:16, Carsten Schoenert wrote:

> Am 17.06.19 um 21:15 schrieb PICCA Frederic-Emmanuel:
>> Same here... with WXX100 cards.
>> what about rocm packaging ?
> (I'm not using an AMD graphic card but ...)
>
> there was recently a new article added to the Debian wiki regarding this
> topic about using the official AMDGPU driver. Maybe this helps solving
> some questions?
>
> https://wiki.debian.org/How%20to%20install%20official%20AMDGPU%20linux%20driver%20with%20kernel%204.19.x%20on%20Stretch%20and%20Buster

Thank you for that pointer, Carsten. I think that likely would have
helped. Still, when working on a new system, would you follow these
instructions or rather install Ubuntu? As helpful as such instructions
are, most folks don't want to be bothered. They want their graphics card
to work. When Debian describes tinkering with GPUs, then this should be
something constructive, like for flashing some new bios with extra feats
- not for the basics.

Best,

Steffen


Reply | Threaded
Open this post in threaded view
|

Re: AMDGPU+OpenCL with Debian?

J Kinney
In reply to this post by Steffen Möller
Hello,

> Thank you for that pointer, Carsten. I think that likely would have
> helped. Still, when working on a new system, would you follow these
> instructions or rather install Ubuntu? As helpful as such instructions
> are, most folks don't want to be bothered.

I am a "regular" Debian user - not a developer.  If Debian makes any attempt to move in the direction of the amount of hand-holding Ubuntu does, I am throwing my laptop out the window and going to live in the woods to eat berries and learn spear fishing or some shite.  I do not need a 6 month bug cycle with patches applied on top of patches.  I want the solid stable Debian system I have been using for over ten years that I have total control over and can fix with simple instructions like those provided for this AMD graphics issue.  There are two other mainstream OS's out there specifically designed for people who don't want to learn how a computer works, and there is Ubunutu for the rest.  Please leave Debian the way it is in this regard so that I can continue to have the joy of learning more about how my system works when minor issues like this come up.  Thanks for listening and thank you very much for the work you all do.

Very Best!

Jason

Reply | Threaded
Open this post in threaded view
|

Re: AMDGPU+OpenCL with Debian?

Moritz Mühlenhoff-2
In reply to this post by Mo Zhou
Mo Zhou <[hidden email]> schrieb:
> On 2019-06-18 03:15, PICCA Frederic-Emmanuel wrote:
>> Same here... with WXX100 cards.
>> what about rocm packaging ?
>
> Not easy. Involves a toolchain.
> Is ROCm promising enough to challenge CUDA?
> (Although ROCm has already beaten CUDA in
> terms of license).

You may find https://phabricator.wikimedia.org/T148843/#5078403
(and later) interesting, which describes some of the tests done
with ROCm running on Buster with the 4.19 Buster kernel; aside from
firmware-amd-graphics running entirely on free software (the only
non-free package (hsa-ext-rocr-dev) is optional and was axed out
in the tests).

Cheers,
        Moritz

Reply | Threaded
Open this post in threaded view
|

Re: AMDGPU+OpenCL with Debian?

Rebecca N. Palmer-2
In reply to this post by Steffen Möller
Summary: try installing mesa-opencl-icd.

It's a known issue that there is currently no way for an OpenCL-using
package to ask for "the correct OpenCL driver for this hardware": it can
ask for "any OpenCL driver" (letting apt choose) or "an OpenCL driver
chosen by the packager", either of which might be the wrong one for the
user's hardware.

Debian currently has:
beignet-opencl-icd - Intel GPUs
mesa-opencl-icd - AMD/ATI GPUs (not sure exactly which ones - very new
ones might need ROCm)
pocl-opencl-icd - CPUs
Nvidia needs non-free.

I proposed [0] fixing this by creating a metapackage for "all OpenCL
drivers" (similar to the ones for graphics).  However, having unusable
OpenCL drivers installed can trigger bugs: [1] in llvm, and some
applications that treat "no hardware for this driver" as "fatal error"
instead of "try the next driver".

[0]
https://alioth-lists.debian.net/pipermail/pkg-opencl-devel/Week-of-Mon-20140203/000076.html
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=852746

Reply | Threaded
Open this post in threaded view
|

Re: AMDGPU+OpenCL with Debian?

Michael Kesper-5
In reply to this post by Moritz Mühlenhoff-2
Hi Moritz,

On 18.06.19 22:55, Moritz Mühlenhoff wrote:
> You may find https://phabricator.wikimedia.org/T148843/#5078403
> (and later) interesting,

This seems to require wikimedia authentication.
Is there some information publicly available about it?

Best wishes
Michael


signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: AMDGPU+OpenCL with Debian?

Moritz Mühlenhoff-2
Michael Kesper <[hidden email]> schrieb:
> On 18.06.19 22:55, Moritz M=C3=BChlenhoff wrote:
>> You may find https://phabricator.wikimedia.org/T148843/#5078403
>> (and later) interesting,=20
>
> This seems to require wikimedia authentication.
> Is there some information publicly available about it?

Ah, that's just an artefact of the URL, simply use
https://phabricator.wikimedia.org/T148843 which doesn't need any
login.

Cheers,
        Moritz

Reply | Threaded
Open this post in threaded view
|

Re: AMDGPU+OpenCL with Debian?

Enrico Weigelt, metux IT consult-3
In reply to this post by Rebecca N. Palmer-2
On 19.06.19 09:09, Rebecca N. Palmer wrote:

Hi,

> I proposed [0] fixing this by creating a metapackage for "all OpenCL
> drivers" (similar to the ones for graphics).  However, having unusable
> OpenCL drivers installed can trigger bugs: [1] in llvm, and some
> applications that treat "no hardware for this driver" as "fatal error"
> instead of "try the next driver".

So, installing an opencl-based package pulls in *all* cl driver stacks ?

Please don't do that. This IMHO clearly belongs into the operator's
hands (or possibly some installer magic).

--mtx

--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[hidden email] -- +49-151-27565287

Reply | Threaded
Open this post in threaded view
|

Re: OpenCL / handling of hardware-specific packages

Rebecca N. Palmer-2


On 01/07/2019 18:10, Enrico Weigelt, metux IT consult wrote:
> On 19.06.19 09:09, Rebecca N. Palmer wrote:
>> I proposed [0] fixing this by creating a metapackage for "all OpenCL
>> drivers" (similar to the ones for graphics).  However, having unusable
>> OpenCL drivers installed can trigger bugs: [1] in llvm, and some
>> applications that treat "no hardware for this driver" as "fatal error"
>> instead of "try the next driver".
>
> So, installing an opencl-based package pulls in *all* cl driver stacks ?

If we do the above, yes by default, but the user can prevent this by
explicitly installing any one.

>
> Please don't do that. This IMHO clearly belongs into the operator's
> hands

Do you mean "not as long as it would cause the above bugs" or "not
ever"?  If the latter, is it because of wasted storage/bandwidth or
something else?

Do you also believe that the existing hardware-related -all packages
(printer-driver-all(-enforce), va-driver-all, vdpau-driver-all,
xserver-xorg-input-all, xserver-xorg-video-all) should not exist /
should not be installed by default?

> (or possibly some installer magic).

Do we have such a mechanism?  I agree this would be better if it existed.

The AppStream metadata format includes a field for "hardware this works
with", and beignet-opencl-icd has one, but I don't know if any existing
tools use this field.


Reply | Threaded
Open this post in threaded view
|

Re: OpenCL / handling of hardware-specific packages

Guillem Jover
Hi!

On Mon, 2019-07-01 at 22:59:51 +0100, Rebecca N. Palmer wrote:

> On 01/07/2019 18:10, Enrico Weigelt, metux IT consult wrote:
> > Please don't do that. This IMHO clearly belongs into the operator's
> > hands
>
> Do you mean "not as long as it would cause the above bugs" or "not ever"?
> If the latter, is it because of wasted storage/bandwidth or something else?
>
> Do you also believe that the existing hardware-related -all packages
> (printer-driver-all(-enforce), va-driver-all, vdpau-driver-all,
> xserver-xorg-input-all, xserver-xorg-video-all) should not exist / should
> not be installed by default?

As long as we do not have an out-of-the-box way to automatically
propose/install hardware-specific packages to match the current system,
the «-all» package pattern is the best we've got, which gives a good
default, but makes it possible to removed unnecessary packages.

> > (or possibly some installer magic).
>
> Do we have such a mechanism?  I agree this would be better if it existed.
>
> The AppStream metadata format includes a field for "hardware this works
> with", and beignet-opencl-icd has one, but I don't know if any existing
> tools use this field.

The closest thing I'm aware of is isenkram, but re-checking it now, it
looks it's now tailored for hot-plugged hardware? Its README contains
interesting information on alternatives and similar solutions. CCed
Petter for further input.

Thanks,
Guillem

Reply | Threaded
Open this post in threaded view
|

Re: OpenCL / handling of hardware-specific packages

Ben Hutchings-3
On Tue, 2019-07-02 at 14:29 +0200, Guillem Jover wrote:
[...]
> The closest thing I'm aware of is isenkram, but re-checking it now, it
> looks it's now tailored for hot-plugged hardware? Its README contains
> interesting information on alternatives and similar solutions. CCed
> Petter for further input.

Linux doesn't distinguish between devices found in an initial
enumeration or hot-plugged later, so I doubt that isenkram is limited
in that way.

Ben.

--
Ben Hutchings
Any sufficiently advanced bug is indistinguishable from a feature.



signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: OpenCL / handling of hardware-specific packages

Petter Reinholdtsen-5
In reply to this post by Guillem Jover

Thank you for the heads up, I am not following debian-devel@ these days.
Please keep me on Cc.

[Guillem Jover]
> The closest thing I'm aware of is isenkram, but re-checking it now, it
> looks it's now tailored for hot-plugged hardware? Its README contains
> interesting information on alternatives and similar solutions. CCed
> Petter for further input.

Not quite sure why you believe it is tailored for hot-plugged hardware.
What should be updated to avoid such impression?  It can definitely
handle any package with hardware mapping in AppStream.

Isenkram provide a background daemon 'isenkramd' for handling
hot-plugged hardware, and various scripts for one off runs related to
more fixed hardware.  You can run 'isenkram-lookup' to get a list of
packages suited for the currently identified hardware,
'isenkram-pkginstall' to get a interactive list of packages with the
option to install them, 'isenkram-autoinstall-firmware' to automatically
install the firmware relevant for the current machine, and even
'tasksel' after installing the isenkram-cli package to get a tasksel
option to install all the packages listed by isenkram-lookup.

--
Happy hacking
Petter Reinholdtsen

Reply | Threaded
Open this post in threaded view
|

Re: OpenCL / handling of hardware-specific packages

Guillem Jover
On Tue, 2019-07-02 at 16:40:14 +0200, Petter Reinholdtsen wrote:
> [Guillem Jover]
> > The closest thing I'm aware of is isenkram, but re-checking it now, it
> > looks it's now tailored for hot-plugged hardware? Its README contains
> > interesting information on alternatives and similar solutions. CCed
> > Petter for further input.
>
> Not quite sure why you believe it is tailored for hot-plugged hardware.
> What should be updated to avoid such impression?  It can definitely
> handle any package with hardware mapping in AppStream.

I've to confess, I just skimmed over it, but precisely the few first
things I checked, were what might have mislead me:

  ,---
  $ apt show isenkram
  …
  Description: Suggest packages to install when inserting new hardware (GUI popup)
   Try to figure out which packages to suggest for use with a freshly
   inserted hardware device.
  `---

  .--- README ---
  The Isenkram project
  ====================

  This is a simple system to propose to the logged in user any Debian
  packages that would be useful to support a given given piece of
  inserted hardware.

  …
  `---

  ,--- NOTES ---
  Make it easier to get pluggable hardware working in Debian
  ==========================================================

  …
  `---

Thanks,
Guillem

Reply | Threaded
Open this post in threaded view
|

Re: OpenCL / handling of hardware-specific packages

Enrico Weigelt, metux IT consult-3
In reply to this post by Rebecca N. Palmer-2
On 01.07.19 23:59, Rebecca N. Palmer wrote:

Hi,

>> So, installing an opencl-based package pulls in *all* cl driver stacks ?
>
> If we do the above, yes by default, but the user can prevent this by
> explicitly installing any one.

Ok, that's fine, as long as it doesn't cause the already mentioned problems.

>> Please don't do that. This IMHO clearly belongs into the operator's
>> hands
>
> Do you mean "not as long as it would cause the above bugs" or "not
> ever"?  If the latter, is it because of wasted storage/bandwidth or
> something else?

Bandwidth and install time is one issue, another is storage (yes, some
folks actually care about storage, eg. w/ containers :p), another is
reducing code and therefore potential bugs and attack surface.
In general, I'd like to keep my systems as minimal as possible.

> Do you also believe that the existing hardware-related -all packages
> (printer-driver-all(-enforce), va-driver-all, vdpau-driver-all,
> xserver-xorg-input-all, xserver-xorg-video-all) should not exist /
> should not be installed by default?

I don't have a problem with those packages, but there shouldn't
be dependencies on them, so that suddenly hundreds of packages
come in when just one is needed.

Maybe there should be some selection mechanism (not sure how
that could be implemented, yet).

>> (or possibly some installer magic).
>
> Do we have such a mechanism?  I agree this would be better if it existed.

Honestly, I don't know. Perhaps that deserves a more detailed
discussion. At that point it would also be nice if we could
configure things like which editor or mailer to install.

Maybe this could be done w/ some virtual package + equivs magic.
I'll have to think about this ...

> The AppStream metadata format includes a field for "hardware this works
> with", and beignet-opencl-icd has one, but I don't know if any existing
> tools use this field.

I don't like the idea of making driver packages depending
on some desktop stuff :o

IMHO, it should be handled on dpkg/apt level.


--mtx

--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[hidden email] -- +49-151-27565287