Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

classic Classic list List threaded Threaded
63 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Sam Hartman-5

Hi. Almost two weeks ago [1] I  started a discussion on whether we
wanted to increase the strength of our recommendation of the dh
sequencer from debhelper.
This message is a consensus call summarizing my reading of the
discussion.

  [1]
  https://lists.debian.org/msgid-search/tsla7fqjzyv.fsf@...

My approach for calling consensus is heavily influenced by RFC 7282 [2],
which talks about some approaches for evaluating rough consensus within
the IETF.  That document is not normative policy there and certainly has
no force here, but the authors have spent a lot of time thinking about
how to understand what an opinionated group of technologists mean when
writing to a mailing list.  In judging consensus, it's much more about
whether issues have been considered and whether the community believes
the issues have been adequately resolved than numbers of people on any
side of a given issue.

  [2] https://tools.ietf.org/html/rfc7282

Please send any comments by 2019-06-16.
Please distinguish cases where you believe I've misread the discussion
from new contributions intended to change the community's view.

At the core of this discussion are a number of key issues that were
reflected in comments throughout the discussion.

* People who make changes across the archive such as enabling hardening,
  cross-building, bootstrapping, etc benefit significantly from more
  uniformity in packaging practices.  The time they spend working on
  packages that use dh is significantly less.  That is, people working
  on making Debian more reproducible benefit from when we adopt more
  uniform practices.

* People who spend time fixing the archive, fixing RC bugs, etc don't
  like to see churn in areas that are already working.  If it's working
  well, don't break it.  Developers doing NMUs do not always perform
  adequate QA of their changes.

* Traditionally we've valued respecting the maintainer's preference
  above most other factors.  In the limit, short of orphaning packages
  or removing maintainers, we don't actually have tools to get
  maintainers to do things they don't want to.  Per our constitution no
  one in Debian is obligated to do anything besides not stand in the way
  of others.  And yet this discussion is about pushing things towards
  uniformity and in a small way away from maintainer preference.

Recommendation
==============

There are some exceptions where we think using dh is the wrong choice;
see below.  We have a strong consensus that other than in exceptional
circumstances, new packages should use dh.

There's a weaker consensus that existing packages should use dh, but
migrating working packages to use dh is often not the best use of our
resources and can be harmful when it introduces bugs.  A number of
people spoke against choosing dh for existing packages.  I looked at the
reasons expressed.  A number of these focused on concerns about
introducing bugs.  When I read through all the messages I think it was
fair to interpret a lot of these concerns as asking us to be careful
about how we spend our resources.  It is generally harmful to convert a
package to dh without adequate testing.  It often makes sense to spend
time on something other than converting working packages to dh unless
doing so fixes something else or makes something easier.

But if a maintainer asks if in an ideal world should their package use
dh, the rough consensus of this discussion is that other than
exceptional circumstances, "yes".

Exceptional Circumstances
=========================

This is not a complete list of reasons not to use dh.  That list will
likely evolve over time, but here are reasons identified in the
discussion.

* If you're actively working on something new, innovation is still
  something we care about.  We're not trying to close down development
  of the next great thing.

* Cdbs has a few features that dh doesn't have.  The Haskell tools in
  cdbs ar critical to our current Haskell infrastructure.  Cdbs flavors
  don't seem to have a dh analogue.  For examples compare
  https://salsa.debian.org/multimedia-team/snd/blob/master/debian/rules
  to
  https://salsa.debian.org/multimedia-team/soundscaperenderer/blob/master/debia
  n/rules .  If cdbs is doing something for you, then it probably makes
  sense to stay with it for now.  But it sounds like there's discussion
  of retiring cdbs longterm.  So if you're using cdbs for things dh is
  perfectly good at, the cdbs exception is much weaker and possibly may
  not apply.

* Consistency with teams.  If your team is using something else, then it
  might be worth having a discussion within the team about whether
  that's still the best choice, but consistency within teams is
  important.

* We need to go back and check if there are any bootstrapping concerns
  that should affect build system decisions for specific packages.

There was strong consensus that we should have a lintian tag that can be
overridden to indicate why a package is not using debhelper.

I think there is rough consensus (although rougher than some other
things) that individual maintainer preference is not in and of itself a
justification for not using dh.  That is, I think the people  arguing
for how using dh made their jobs either in making large changes to
Debian made a better case than those arguing for maintainer preference.
However Mo Zhou reminded us that we need to keep things fun.  If a
maintainer really wants to use something other than dh there's a good
chance that there is some underlying requirement dh is not meeting.  And
that probably is a legitimate exception.

we're not coming after people with pitchforks if they don't use dh.  It
might simply mean their packages have a bug.  It certainly doesn't mean
they are obligated to fix that bug, although best practice in our
community is to work with submitters of patches to review those patches.

Is Not Using DH a Bug?
======================

It's certainly not an RC bug.  There was some talk of it eventually
being an RC bug if a new package doesn't use DH (and doesn't fall under
an exception).  I don't think there's consensus for that today.

It's obviously not a bug if one of the exceptions applies, but once the
lintian tag exists, overriding that tag sounds to me like best practice
to document the exception.  (Presumably for things like Haskell we'd
want Lintian to be smart enough to figure that out on its own)

To some extent I'm extrapolating from implications of the rest of the
consensus call for the rest of this section.  There was some discussion
of whether not using dh should be a normal bug, but the comments about
that were inconsistent with the rest of the discussion.  But if the
consensus call above that existing packages (absent exceptional
circumstances) should use dh stands, that's approximately the definition
of a normal bug.  So my reading is that absent exceptional
circumstances, not using dh is a normal severity bug.

It doesn't mean you should file that bug and it doesn't mean that you
should go fix that bug.  We definitely didn't get the kind of support
we'd be needing for a mass bug filing or anything like that.  It
wouldn't serve a point.  This isn't atypical.  There are a lot of things
lintian flags that are technically bugs, but we wouldn't want to mass
file all lintian tags (even if we could filter out false positives) as
bugs.

This paragraph is very much my interpretation.  I'd personally say that
if you're going to file a bug that a particular package doesn't use dh,
have a good reason and document it in that bug.  Your reason might be "I
want to contribute; I'm willing to dedicate time and updating the
packaging would make it much more appealing to work on."  Often your
reason will be that there's some other problem, migrating to dh will fix
that problem, and between the time you're willing to spend and the time
you hope the maintainer will spend it's worth doing a good job of that
migration.

My interpretation of our standard practices is that maintainers have
wide discretion in which bugs they work on.  That said, if someone
submits a patch, it's good if you review it.  It's fine to ask them to
do the necessary testing work and it's fine to hold them to the same
high standards that you hold yourself to.  If they are less experienced
with the package it might make sense for them to do tests that make up
for that experience gap.  None of this  changes any of that or asks
maintainers to treat bugs about dh differently than other bugs.

Best Practices for Testing DH Conversions
=========================================

* Run a debdiff of the binaries to see what has changed

* Use diffoscope

* Run autopkgtests

* Test piuparts

Generally look at the packaging and explain any changes carefully.

So I should Go NMU the Archive to DH
====================================

ABSOLUTELY NOT!

We had a long discussion of NMUs and dh that was probably my fault and
unnecessary.

Through a round about set of messages we seemed to come to a consensus
that is roughly the same as our existing NMU policy.

I don't see a consensus to change how NMUs regarding dh are handled.

Generally doing an NMU to change packaging style is not a good idea.
There are cases where it might be appropriate.  Feel free to read the
longer discussion and to look at discussions of NMUs in Developer's
Reference.

Why are We Doing this
=====================

https://lists.debian.org/msgid-search/20190513173542.GA24063@...

is fairly typical of this.  There was also some discussion in the
initial message.

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Vincent Bernat-3
 ❦ 25 mai 2019 13:26 -04, Sam Hartman <[hidden email]>:

> * People who make changes across the archive such as enabling hardening,
>   cross-building, bootstrapping, etc benefit significantly from more
>   uniformity in packaging practices.  The time they spend working on
>   packages that use dh is significantly less.  That is, people working
>   on making Debian more reproducible benefit from when we adopt more
>   uniform practices.

It has also been said that non-unformity makes also the life of
everybody more difficult when they look at a random package. This
includes non-DD/DM people. We have a reputation of having difficult
packaging practices. We uphold this reputation as long as we have so
many ways to do the same thing.
--
Say what you mean, simply and directly.
            - The Elements of Programming Style (Kernighan & Plauger)

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Jonas Smedegaard-2
Quoting Vincent Bernat (2019-05-26 11:34:39)

>  ❦ 25 mai 2019 13:26 -04, Sam Hartman <[hidden email]>:
>
> > * People who make changes across the archive such as enabling
> >   hardening, cross-building, bootstrapping, etc benefit
> >   significantly from more uniformity in packaging practices.  The
> >   time they spend working on packages that use dh is significantly
> >   less.  That is, people working on making Debian more reproducible
> >   benefit from when we adopt more uniform practices.
>
> It has also been said that non-unformity makes also the life of
> everybody more difficult when they look at a random package. This
> includes non-DD/DM people. We have a reputation of having difficult
> packaging practices. We uphold this reputation as long as we have so
> many ways to do the same thing.
We "uphold this reputation" by maintaining many packages, which is good.


 - Jonas

--
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Vincent Bernat-3
 ❦ 26 mai 2019 12:04 +02, Jonas Smedegaard <[hidden email]>:

>> > * People who make changes across the archive such as enabling
>> >   hardening, cross-building, bootstrapping, etc benefit
>> >   significantly from more uniformity in packaging practices.  The
>> >   time they spend working on packages that use dh is significantly
>> >   less.  That is, people working on making Debian more reproducible
>> >   benefit from when we adopt more uniform practices.
>>
>> It has also been said that non-unformity makes also the life of
>> everybody more difficult when they look at a random package. This
>> includes non-DD/DM people. We have a reputation of having difficult
>> packaging practices. We uphold this reputation as long as we have so
>> many ways to do the same thing.
>
> We "uphold this reputation" by maintaining many packages, which is
> good.
Do we? I am now using nix to get packages for stuff not in Debian. Our
package count is artificially inflated by *-perl packages, golang-*
packages which may not be present in some other distributions. But for
some ecosystems, we are severely behind. We may argue we are better on
some metrics, but this has nothing to do with the fact we have so many
ways to build a package.
--
English literature's performing flea.
                -- Sean O'Casey on P. G. Wodehouse

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Adrian Bunk-3
In reply to this post by Vincent Bernat-3
On Sun, May 26, 2019 at 11:34:39AM +0200, Vincent Bernat wrote:
>...
> We have a reputation of having difficult
> packaging practices. We uphold this reputation as long as we have so
> many ways to do the same thing.

  [citation needed]

I do honestly not know what statements/comparisons from people outside
Debian are the basis for this claim, and whether making dh more mandatory
is even related to them.

It is a problem for people making their first contributions to Debian to
get them into unstable. The problem here is lack of sponsors willing to
do proper reviews and then uploads. Usually the package in question is
already using dh.

Often the most difficult part of packaging are the unique rules the
Debian ftp team requires for debian/copyright that are not required in
distributions with actual lawyers. That's a completely separate topic.

It is perfectly possible that there is something else I am not aware
of, and you are assuming everyone in this discussion is.

It would therefore be really useful if you could send some links to
statements from people outside Debian *why* they consider Debian to
have difficult packaging practices.

cu
Adrian

--

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Reply | Threaded
Open this post in threaded view
|

Difficult Packaging Practices

Sam Hartman-3
>>>>> "Adrian" == Adrian Bunk <[hidden email]> writes:

    Adrian> It is a problem for people making their first contributions
    Adrian> to Debian to get them into unstable. The problem here is
    Adrian> lack of sponsors willing to do proper reviews and then
    Adrian> uploads. Usually the package in question is already using
    Adrian> dh.

My experience is consistent with the above.
I can't send links because most of my conversations with people getting
started contributing to Debian  have been that: actual conversations
with lungs compressing air to blow out mouths.

Unfortunately, I've found that it's often a combination of factors.

The one first cited is  the difficulty finding a sponsor.
But in at least some cases it's more complex than that.

for example in one case someone was talking about finding a sponsor.  I
said that I didn't normally sponsor packages I couldn't adequately test,
so that limited what I'd sponsor, but if the contributor could find
something within that set I'd sponsor it.

He came up with something.
Then he said but really the hard part there was not the sponsorship, but
the dependency problem
Apparently upstream had forked some library already in Debian and you
had to use the fork for the package to work.

I think it's a combination of a lot of things.  We have high standards,
a lot of complexity, and you have to get most or all of that right to
contribute.  You have to have a package that meets our standards.  You
have to have a copyright file that meets our standards.  You have to be
able to figure out our processes.  You have to be willing to follow our
processes.  And you eventually have to deal with the PGP mess.

If you don't find value in the things where we have high standards,
Debian doesn't make a lot of sense.
If you just want to get upstream's idea of their package onto a system
with their release schedule and their recommended dependency versions,
there are better ways than getting a package into Debian.


--Sam

Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Philip Hands
In reply to this post by Adrian Bunk-3
Adrian Bunk <[hidden email]> writes:

...
> Often the most difficult part of packaging are the unique rules the
> Debian ftp team requires for debian/copyright that are not required in
> distributions with actual lawyers. That's a completely separate topic.

That seems needlessly snide, and glosses over the fact that we are bound
by constraints that other distributions are not, such as giving a damn
about the license conditions we inflict on our downstreams, which I
suspect the lawyers you allude to are not being retained to contemplate.

Cheers, Phil.
--
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/    http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,    GERMANY

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Difficult Packaging Practices

Jérémy Lal-2
In reply to this post by Sam Hartman-3


Le dim. 26 mai 2019 à 23:01, Sam Hartman <[hidden email]> a écrit :
>>>>> "Adrian" == Adrian Bunk <[hidden email]> writes:

    Adrian> It is a problem for people making their first contributions
    Adrian> to Debian to get them into unstable. The problem here is
    Adrian> lack of sponsors willing to do proper reviews and then
    Adrian> uploads. Usually the package in question is already using
    Adrian> dh.

My experience is consistent with the above.
I can't send links because most of my conversations with people getting
started contributing to Debian  have been that: actual conversations
with lungs compressing air to blow out mouths.

Unfortunately, I've found that it's often a combination of factors.

The one first cited is  the difficulty finding a sponsor.

There's a gap between getting some unknown software into debian,
and getting well-known software into debian.
Some well-determined upstream develop can manage to get software
into debian, and even get it into stable, and then just quit maintaining it !
On the other hand, when a fellow DD uploads a software it usually means
that piece is worth it. So i'm not that keen on making it easier to get any kind
of software into debian.
 
But in at least some cases it's more complex than that.

for example in one case someone was talking about finding a sponsor.  I
said that I didn't normally sponsor packages I couldn't adequately test,
so that limited what I'd sponsor, but if the contributor could find
something within that set I'd sponsor it.

He came up with something.
Then he said but really the hard part there was not the sponsorship, but
the dependency problem
Apparently upstream had forked some library already in Debian and you
had to use the fork for the package to work.

It might also mean the upstream software is still maturing and not ready for
prime time.

I think it's a combination of a lot of things.  We have high standards,
a lot of complexity, and you have to get most or all of that right to
contribute.  You have to have a package that meets our standards.  You
have to have a copyright file that meets our standards.

When it's hard to make upstream conform to DFSG it's either easy to fix,
or impossible to fix. So it's not that hard to deal with.
 
You have to be
able to figure out our processes.  You have to be willing to follow our
processes.  And you eventually have to deal with the PGP mess.

Are you referring to the identity check ?
That mess is onto the uploader's hand.
You don't need to have your identity checked as an upstream author, and
the identity check is the best part of subscribing to a community.
 
If you don't find value in the things where we have high standards,
Debian doesn't make a lot of sense.
If you just want to get upstream's idea of their package onto a system
with their release schedule and their recommended dependency versions,
there are better ways than getting a package into Debian.

There are even ways that are supported by software available in Debian !
(thinking of flatpak but many other ways to allow users to install software easily).

--
Jérémy 
Reply | Threaded
Open this post in threaded view
|

Re: Difficult Packaging Practices

Russ Allbery-2
In reply to this post by Sam Hartman-3
Sam Hartman <[hidden email]> writes:

> I think it's a combination of a lot of things.  We have high standards,
> a lot of complexity, and you have to get most or all of that right to
> contribute.  You have to have a package that meets our standards.  You
> have to have a copyright file that meets our standards.  You have to be
> able to figure out our processes.  You have to be willing to follow our
> processes.  And you eventually have to deal with the PGP mess.

> If you don't find value in the things where we have high standards,
> Debian doesn't make a lot of sense.  If you just want to get upstream's
> idea of their package onto a system with their release schedule and
> their recommended dependency versions, there are better ways than
> getting a package into Debian.

This is my experience as well.  I've occasionally tried to get people at
work (at various jobs) interested in packaging software for Debian,
without all that much success.  The problem seems less any one specific
thing and more that they're perfectly content with a Debian package
created by running fpm on some install tree, and don't see the point in
doing any more work than that.  This is usually in the context where
there's some other config management system in use, so to them all the
packaging format is good for is as a container of files with a version
number attached.

It's not that they don't understand the merits of having what they think
of as the base operating system properly maintained and integrated; it's
more that they don't see any value in doing that work for the incremental
thing that they're adding.  They cobble together some combination of local
config management and a deployment method until it works and then move on
to some (from their perspective) more interesting problem.

--
Russ Allbery ([hidden email])               <http://www.eyrie.org/~eagle/>

Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Andreas Tille-5
In reply to this post by Vincent Bernat-3
On Sun, May 26, 2019 at 07:28:55PM +0200, Vincent Bernat wrote:
> > We "uphold this reputation" by maintaining many packages, which is
> > good.
>
> Do we? I am now using nix to get packages for stuff not in Debian. Our
> package count is artificially inflated by *-perl packages, golang-*
> packages which may not be present in some other distributions. But for
> some ecosystems, we are severely behind. We may argue we are better on
> some metrics, but this has nothing to do with the fact we have so many
> ways to build a package.

Some Debian Med people are concerned about the droping usage of Debian
Med packages since people prefer BioConda[1] over it.  There is even a
scientific paper (I've only seen a printed version not online yet) who
compares ways to package biology software.  We are way better than other
distributions - but we are lagging begind BioConda a lot.  We have some
upstreams who are doing Debian packaging by the help of the Debian Med
team but that's just a minor fraction.  Lots of BioConda packages are
maintained by Upstream since they consider it easy.

In short:  Our "reputation" is scaring people away to favour other
techniques.

Kind regards

       Andreas.

[1] https://bioconda.github.io/

--
http://fam-tille.de

Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Scott Kitterman-5
In reply to this post by Sam Hartman-5


On May 25, 2019 5:26:47 PM UTC, Sam Hartman <[hidden email]> wrote:
>
>Hi. Almost two weeks ago [1] I  started a discussion on whether we
>wanted to increase the strength of our recommendation of the dh
>sequencer from debhelper.
>This message is a consensus call summarizing my reading of the
>discussion.
...

>
>Please send any comments by 2019-06-16.
>Please distinguish cases where you believe I've misread the discussion
>from new contributions intended to change the community's view.
>
>...
>Is Not Using DH a Bug?
>======================
>
>It's certainly not an RC bug.  There was some talk of it eventually
>being an RC bug if a new package doesn't use DH (and doesn't fall under
>an exception).  I don't think there's consensus for that today.
>
>It's obviously not a bug if one of the exceptions applies, but once the
>lintian tag exists, overriding that tag sounds to me like best practice
>to document the exception.  (Presumably for things like Haskell we'd
>want Lintian to be smart enough to figure that out on its own)
>
>To some extent I'm extrapolating from implications of the rest of the
>consensus call for the rest of this section.  There was some discussion
>of whether not using dh should be a normal bug, but the comments about
>that were inconsistent with the rest of the discussion.  But if the
>consensus call above that existing packages (absent exceptional
>circumstances) should use dh stands, that's approximately the
>definition
>of a normal bug.  So my reading is that absent exceptional
>circumstances, not using dh is a normal severity bug.
>
>It doesn't mean you should file that bug and it doesn't mean that you
>should go fix that bug.  We definitely didn't get the kind of support
>we'd be needing for a mass bug filing or anything like that.  It
>wouldn't serve a point.  This isn't atypical.  There are a lot of
>things
>lintian flags that are technically bugs, but we wouldn't want to mass
>file all lintian tags (even if we could filter out false positives) as
>bugs.
>
>This paragraph is very much my interpretation.  I'd personally say that
>if you're going to file a bug that a particular package doesn't use dh,
>have a good reason and document it in that bug.  Your reason might be
>"I
>want to contribute; I'm willing to dedicate time and updating the
>packaging would make it much more appealing to work on."  Often your
>reason will be that there's some other problem, migrating to dh will
>fix
>that problem, and between the time you're willing to spend and the time
>you hope the maintainer will spend it's worth doing a good job of that
>migration.
>
>My interpretation of our standard practices is that maintainers have
>wide discretion in which bugs they work on.  That said, if someone
>submits a patch, it's good if you review it.  It's fine to ask them to
>do the necessary testing work and it's fine to hold them to the same
>high standards that you hold yourself to.  If they are less experienced
>with the package it might make sense for them to do tests that make up
>for that experience gap.  None of this  changes any of that or asks
>maintainers to treat bugs about dh differently than other bugs.

On what basis is it a bug of any kind?

A lintian check does not a buggy package make.

Assuming a package that is otherwise working correctly, you have a policy compliant, fully functional package.  That seems to me like the very definition of not a bug.

If we want to make not using dh except in certain situations a bug, it seems like something for a policy should kind of item.  We have an existing process for updating policy, so this should probably be kicked over there for further evaluation.

Scott K

Reply | Threaded
Open this post in threaded view
|

Re: Difficult Packaging Practices

Ben Finney-3
In reply to this post by Sam Hartman-3
Sam Hartman <[hidden email]> writes:

> If you just want to get upstream's idea of their package onto a system
> with their release schedule and their recommended dependency versions,
> there are better ways than getting a package into Debian.

In the Debian mentors forum (that is, the chat channel, the mailing
list, etc.) we make a point of saying: that's fine!

Not every package needs to be in Debian, for its users to install that
package. Someone who wants what you describe above can learn how to make
a Debian package that will only be side-loaded from that community's own
repository, or whatever.

Perhaps that needs to be stated louder or more consistently?

--
 \       “Truth is stranger than fiction, but it is because fiction is |
  `\     obliged to stick to possibilities, truth isn't.” —Mark Twain, |
_o__)                                          _Following the Equator_ |
Ben Finney

Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Jonas Smedegaard-2
In reply to this post by Andreas Tille-5
Quoting Andreas Tille (2019-05-27 06:29:05)

> On Sun, May 26, 2019 at 07:28:55PM +0200, Vincent Bernat wrote:
> > > We "uphold this reputation" by maintaining many packages, which is
> > > good.
> >
> > Do we? I am now using nix to get packages for stuff not in Debian.
> > Our package count is artificially inflated by *-perl packages,
> > golang-* packages which may not be present in some other
> > distributions. But for some ecosystems, we are severely behind. We
> > may argue we are better on some metrics, but this has nothing to do
> > with the fact we have so many ways to build a package.
>
> Some Debian Med people are concerned about the droping usage of Debian
> Med packages since people prefer BioConda[1] over it.  There is even a
> scientific paper (I've only seen a printed version not online yet) who
> compares ways to package biology software.  We are way better than
> other distributions - but we are lagging begind BioConda a lot.  We
> have some upstreams who are doing Debian packaging by the help of the
> Debian Med team but that's just a minor fraction.  Lots of BioConda
> packages are maintained by Upstream since they consider it easy.
>
> In short: Our "reputation" is scaring people away to favour other
> techniques.
We agree that some get scared away from Debian.  Question is why.

Like rpm, Debian arguably has a single unified _core_ structure:
debian/rules must be a make file.  hat Sam, me, and Vincent disagree on
in this subthread is is lack of a unified _framework_ like short-form dh
sequencer being mandatory is what scares people off.

Does that paper you talk about point to _causes_ Debian packaging being
more scary? Is it a) complexities related to hardening, cross-building,
bootstrapping etc. or b) lack of a single¹ unified build framework, or
c) that Debian is "different" in package naming and code availability
than other system package managers or or misc. language-specific package
managers, or d) something else than what you snipped from the beginning
of this subthread?


 - Jonas

--
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Difficult Packaging Practices

Vincent Bernat-3
In reply to this post by Ben Finney-3
 ❦ 27 mai 2019 16:15 +10, Ben Finney <[hidden email]>:

>> If you just want to get upstream's idea of their package onto a system
>> with their release schedule and their recommended dependency versions,
>> there are better ways than getting a package into Debian.
>
> In the Debian mentors forum (that is, the chat channel, the mailing
> list, etc.) we make a point of saying: that's fine!
>
> Not every package needs to be in Debian, for its users to install that
> package. Someone who wants what you describe above can learn how to make
> a Debian package that will only be side-loaded from that community's own
> repository, or whatever.
People using tools like fpm will never get familiar with our tools and
will never be contributors.
--
The man who sets out to carry a cat by its tail learns something that
will always be useful and which never will grow dim or doubtful.
                -- Mark Twain

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Jonas Meurer
In reply to this post by Adrian Bunk-3
Adrian Bunk:

> On Sun, May 26, 2019 at 11:34:39AM +0200, Vincent Bernat wrote:
>> ...
>> We have a reputation of having difficult
>> packaging practices. We uphold this reputation as long as we have so
>> many ways to do the same thing.
>
>   [citation needed]
>
> I do honestly not know what statements/comparisons from people outside
> Debian are the basis for this claim, and whether making dh more mandatory
> is even related to them.
>
> [...]
>
> Often the most difficult part of packaging are the unique rules the
> Debian ftp team requires for debian/copyright that are not required in
> distributions with actual lawyers. That's a completely separate topic.
>
> It is perfectly possible that there is something else I am not aware
> of, and you are assuming everyone in this discussion is.
>
> It would therefore be really useful if you could send some links to
> statements from people outside Debian *why* they consider Debian to
> have difficult packaging practices.
Unfortunately I don't have *links* either, but when introducing people
into the world of Debian packaging recently, I always got the impression
that they were heavily overwhelmed by the complexity of the Debian
ecosystem.

Depending on the software you packages, doing the initial packaging
already requires a lot of knowledge about library handling, doc build
systems, makefiles, the filesystem hierarchy standard, language-specific
toolchains, etc.

To properly build the package you have to learn either sbuild or
pbuilder. Which involves understanding and creating chroots/VMs/...

For proper version controlling, things like git-buildpackage (and/or
dgit) and the "3.0 (quilt)" format need to be understood.

And for testing, you need to learn about piuparts, autopkgtest, as well
as again chroots and/or containers for local testing.

That's a very high bar for entering the world of Debian packaging.

My opinion is that more uniformity in packaging practices will bring a
bit more simplicity as well. Therefore I applaud Sam's initiative to
require DH whereever it's sensible.

I think that Debian would gain a lot if the vast majority of packages
were packaged using DH and development would happen in Git on Salsa
using a common Git format. I agree that there should be exceptions.

Cheers
 jonas


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Difficult Packaging Practices

Sam Hartman-3
In reply to this post by Russ Allbery-2
>>>>> "Russ" == Russ Allbery <[hidden email]> writes:

    Russ> This is my experience as well.  I've occasionally tried to get
    Russ> people at work (at various jobs) interested in packaging
    Russ> software for Debian, without all that much success.  The
    Russ> problem seems less any one specific thing and more that
    Russ> they're perfectly content with a Debian package created by
    Russ> running fpm on some install tree, and don't see the point in
    Russ> doing any more work than that.  This is usually in the context
    Russ> where there's some other config management system in use, so
    Russ> to them all the packaging format is good for is as a container
    Russ> of files with a version number attached.

I've actually had somewhat better experience getting Debian packages
built at work.
At least in environments where we had a fairly pure Debian ecosystem (or
say Debian+windows) I've been able to get people to build packages using
.dscs.  And if I set up the gitlab CI integration, that works fairly
well.

At my current job we even have interest in making some of the stuff
comply with policy and actually make its way back into Debian.
And I think we're possibly getting close to activation energy to do some
of that.

But the challenge is that the quality bar you need for a local
environment is much lower than for inclusion in a global OS.

I think the only reason I've had different experience than Russ is two
factors.  First, I didn't show people fpm (and they weren't aware).
Second, I think the environments I'm talking about are a lot smaller
than Russ where some of the social concerns really do matter.

At the first Cloud sprint we saw something similar.  The folks from
Google wanted to talk to us about getting some of their cloud APIs into
Debian.
But they wanted an approach that allowed them to build for all
distributions.  They wanted us to accept fpm input as source.
They didn't actually want to do the work that is Debian specific in
packaging.
I ended up saying that unfortunately, unless someone wanted to commit to
doing that work, it wasn't a package we as a community want to accept.
I tried to be more polite than that and to explain why those were our
norms.

To be clear, I'm sure there are a lot of ways our processes and
documentation can be improved.
However there are some reasons why what we're asking people to do is
hard.  And I don't think we want to give up on those.

Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Alex Mestiashvili-4
In reply to this post by Andreas Tille-5


On 5/27/19 6:29 AM, Andreas Tille wrote:

> On Sun, May 26, 2019 at 07:28:55PM +0200, Vincent Bernat wrote:
>>> We "uphold this reputation" by maintaining many packages, which is
>>> good.
>>
>> Do we? I am now using nix to get packages for stuff not in Debian. Our
>> package count is artificially inflated by *-perl packages, golang-*
>> packages which may not be present in some other distributions. But for
>> some ecosystems, we are severely behind. We may argue we are better on
>> some metrics, but this has nothing to do with the fact we have so many
>> ways to build a package.
>
> Some Debian Med people are concerned about the droping usage of Debian
> Med packages since people prefer BioConda[1] over it.  There is even a
> scientific paper (I've only seen a printed version not online yet) who
> compares ways to package biology software.  We are way better than other
> distributions - but we are lagging begind BioConda a lot.  We have some
> upstreams who are doing Debian packaging by the help of the Debian Med
> team but that's just a minor fraction.  Lots of BioConda packages are
> maintained by Upstream since they consider it easy.
>
> In short:  Our "reputation" is scaring people away to favour other
> techniques.
>
> Kind regards
>
>        Andreas.
>
> [1] https://bioconda.github.io/
>

Debian packaging is also pretty easy if one doesn't care about FHS,
Mutli-Arch, licensing, reproducibility, autopkgtests, hardening and so
on. I believe the list is quite long.

One of the killer features of conda is that conda software can be
managed without root. In some cases it is very important. And of course
that conda is available on osx and windows.

Best,
Alex









       

Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Sam Hartman-3
In reply to this post by Scott Kitterman-5
>>>>> "Scott" == Scott Kitterman <[hidden email]> writes:
    Scott> If we want to make not using dh except in certain situations
    Scott> a bug, it seems like something for a policy should kind of
    Scott> item.  We have an existing process for updating policy, so
    Scott> this should probably be kicked over there for further
    Scott> evaluation.

So a discussion like this tends to serve as inputs to discussions in the
rest of the project including policy.  However, I'd hope that delegates
like the policy editors or the TC would be guided by project consensus.

And If this consensus call stands, my next action as DPL will be to talk
to various people who are impacted about next steps.

The project is not well served by having each team rehash all the global
discussions.
Absolutely, whoever takes this on on the policy front (TC or policy
editors) should figure out how to say this in policy.  Absolutely if
they find factors we failed to consider globally they should come back
to us.  If they read the discussion and believe I made the wrong
consensus call for that discussion, they should come back and say so.

But no I think we as a community right here get to decide what's a bug
if we want to, and I think it's reasonable for us to hold our delegates
accountable for implementing that decision or having a good reason to do
otherwise.

--Sam

Reply | Threaded
Open this post in threaded view
|

Re: Difficult Packaging Practices

Sam Hartman-3
In reply to this post by Ben Finney-3
>>>>> "Ben" == Ben Finney <[hidden email]> writes:

    Ben> Sam Hartman <[hidden email]> writes:
    >> If you just want to get upstream's idea of their package onto a
    >> system with their release schedule and their recommended
    >> dependency versions, there are better ways than getting a package
    >> into Debian.

    Ben> In the Debian mentors forum (that is, the chat channel, the
    Ben> mailing list, etc.) we make a point of saying: that's fine!

    Ben> Not every package needs to be in Debian, for its users to
    Ben> install that package.

I think I was trying to agree with you.


I'd actually recommend understanding people's pain points for other
approaches and making sure there aren't things we can do to make it
easier.

As an example, Ubuntu makes it easy to add side-loaded channels (ppas).
we don't.

The fast-paced repository discussion from earlier this year is another
example of where we could address a pain point.

Another pain point is people who would like to do the work of making
something stable but who need a fixed set of dependencies; the space
where containers might be a solution.
We as a distribution haven't done a lot there beyond including flatpak
and snap.

--Sam

Reply | Threaded
Open this post in threaded view
|

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

Sam Hartman-3
In reply to this post by Jonas Meurer
>>>>> "Jonas" == Jonas Meurer <[hidden email]> writes:


    Jonas> My opinion is that more uniformity in packaging practices
    Jonas> will bring a bit more simplicity as well. Therefore I applaud
    Jonas> Sam's initiative to require DH whereever it's sensible.

Hi.
I'm acting as a facilitator here not as a proponent.
My job as DPL in this discussion is not to guide us to requiring dh.
My job is to understand what we as a project want and then to ask the
appropriate teams in our project to implement this.

It would be inappropriate for me as DPL to actually be trying to get us
to require dh until we as a community decide (as I believe we've done)
that in a lot of circumstances, that's what we want.


--Sam

1234