Summary: Git Packaging Round 2 [comments by 11/05/2019]

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Summary: Git Packaging Round 2 [comments by 11/05/2019]

Sam Hartman-5

This is a summary of discussions we had in August and September around
how we want to use salsa.  Presented so those involved in the discussion
can see if I'm calling consensus correctly and as a last opportunity for
others to comment.  A few tweaks from my original proposed
recommendations.  If you were OK with that version you can likely skip
this entirely, although I'd appreciate it if you'd search for "HOW YOU
CAN HELP" and glance at that section at least.


This message focuses on  the areas of the original recommendation that
received significant comment and includes the current recommendation at
the end.

As a reminder, this recommendation is purely optional; it is intended to
help those looking for best practices or newcomers searching for a
recommendation.  If it doesn't fit your needs, then find something that
does.

Discussion Comments
====================

Teams
*****

We were reminded at several points that the best practice is to find a
team and to maintain your package using their practices.  That was
always the intent, but I'm proposing to update and clarify this from the
top just to be extra clear.

Debian Group When no Specific Team
**********************************

TL; DR: I think there is consensus behind recommending the Debian group
on Salsa when there is not an existing team to use.  This consensus is
not complete; some disagreed.  If after reading the summary people want
to have a quick vote, I think that would be fine.

In more detail: I proposed that the debian group would be a sensible
default if there is not a team project to use.  We had a wide ranging
discussion.

There are two big concerns with the debian group.  Technically, any DD
has push access to any repository in the group.  Also, as Ansgar pointed
out, the wiki page [1] says that it is reasonable to commit with no
prior discussion.

  [1]:
  https://wiki.debian.org/Salsa/Doc#Collaborative_Maintenance:_.22Debian.22_group

My original text implied that the Debian group was a lot closer to
low-threshhold NMUs than a complete free-for-all.  In practice, I think
that's true: I think that the actual social conventions that get used in
practice are more conservative than the permissions granted technically
and by that wiki page.

Russ agreed that Debian is a sensible default [2].  Enrico agreed.
After re-reading the proposal, Jonas agreed that this was a reasonable
recommendation.  He would not have supported something stronger like a
policy which limited maintainers' flexibility when this recommendation
does not meet their needs.

  [2]: https://lists.debian.org/msgid-search/875zm0or60.fsf@...

Ian Jackson thought that recommending  the debian group would be reasonable if
we did three things:

* Document which maintainer branch format a maintainer is using

* Update documentation on when it is reasonable to push to the debian
  group

* document the above

David Bremner is uncomfortable with the idea of having all DDs have push
access to thousands of packages.  He doesn't think we have the social
conventions for that.
However he said he would be OK with the recommendation if we did as Ian
proposed (documenting branch format, documenting when to push).

Ansgar argued that documenting the branch format is unnecessary given
all the work you need to do to contribute to a package.  Several
disagreed arguing that it helped them do their work.

Bernd Zeimetz argued we didn't need a recommendation.  We could just
list places people could host with advantages and disadvantages.  Matt
Zagrabelny and Enrico argued that  having specific recommendations
lowered the bar to contributing because you did not need to evaluate the
advantages and disadvantages for your environment.  This is consistent
with feedback we received during the dh discussion and during the DPL
campaign.  (Enrico's message was posted to a related thread on -project,
not to debian-devel)


Ansgar argued that a recommendation could be harmful because we take a
long time to update documentation/recommendations and it would get out
of date.  Several people pointed out that under that argument we
wouldn't have documentation at all.
Even if the slope is not that slippery, the arguments about lowering the
bar to entry apply here too.  However having the recommendation become
out of date remains a residual risk.


Scott Kitterman is concerned that we are loosening the boundaries of
individual responsibility around what it means to be a maintainer too
far.  He's concerned that if we weaken the maintainer model too much,
that everyone will have responsibility.  He believes that is the same as
no one having responsibility.

It became clear in the discussion that the alternative to the debian
group is repositories hosted under individual salsa accounts.

The debian group is widely used today.  It certainly dominates any
individual salsa account.  It also dominates (at least in terms of
vcs-git in unstable) individual (non-team) accounts combined.  There are
some teams that are bigger than the debian group.  Since we recommend
team package maintenance, that's great to hear.  (I posted some stats on
the thread.  People argued over my methodology and choice of stats, but
in any interpretation proposed, the debian group is significant and
dominates individual accounts).

Conclusiony Thing
-----------------

* Make it clear that team packaging is preferred; emphasize that more.

* If you are using debian group you should document your branch format.
  Update to say that.

* It would be valuable to better understand what social conventions have
  evolved around pushing to debian.  These should be documented even if
  there is not a formal change to the rules around pushing.  I disagree
  with Ian Jackson and David Bremner that we need to block a general
  recommendation on what may be a somewhat involved project of
  documenting that social convention.  I think it is great work.  I
  think this recommendation will be far more valuable when it completes.
  A lot of people are using the debian group today successfully with few
  complaints even though that work hasn't been done.  So I don't think a
  compelling argument for blocking a recommendation on better
  understanding social conventions has been made.

* I believe that there is a rough consensus in favor of the
  recommendation.  In judging consensus I'm presuming a strong consensus
  behind having recommendations in this space; I'm giving a lot of
  weight to the lowering barriers to entry argument.  I'm also giving
  significant weight to the debian group being up and operational and
  used by a lot of people.  Yes that's partially because it was
  recommended as part of the Alioth migration.  I don't see that
  counting against anything.  But this consensus is still kind of
  rough.  If people believe I've made the wrong call here, I think it's
  entirely reasonable to hold a GR and understand the view of the
  project today.

* It's clear there is no consensus in favor of alternatives.  For
  example there is not a consensus in favor of saying that best practice
  is to put packages in individual accounts.  That's a reasonable thing
  to do, but we don't have consensus in favor of that being the best
  practice.  So, this is a question between debian group and no
  recommendation at all.

Github and Non-Free Services
****************************

too much has been written on this already.  There is no consensus to
forbid using Github.  More discussion on this topic will not serve the
project at this time.

The recommendation against using Github appears to be relatively
uncontroversial.  Some disagree, but I call the consensus in favor of
retaining that.


Salsa CI
********

The salsa admins wrote and let us know that right now we don't have
enough resources on Salsa to recommend the Salsa CI pipeline for all
projects.  So I'm removing that recommendation.

A couple people disagreed with this and argued that I should include the
recommendation even over the Salsa Admins' objection.  Part of giving
people responsibility is letting them make choices over what they are
ready to support.  The project can let the Salsa Admins know that we as
a community want stronger support for Salsa CI.  We can work with them.
They can ask for more help if they need it.  Eventually, we can evaluate
whether the people running a service are doing what the project needs.
But while they have that responsibility, they need our support.  And
that means they get to decide when they are not ready for something.

It seems clear that a lot of people want to see Salsa CI support improve
and I think it's reasonable for us to ask for periodic feedback both
from the Salsa Admins and the Salsa CI team on how that is going.


Changes from Previous Version
*****************************


* Add Existing Team first

* Non-DDs should ask for a debian group repo or use personal namespace.

* For merge requests, note that you should be as responsive as BTS if
you enable merge requests at all.

* remove Salsa CI recommendation:-(

* Do not make a recommendation for VCS technology if the upstream is not
  in git.  Ian Jackson gave compelling arguments why there's no clear
  consensus in this case.

                                AND NOW
                                  THE
                             Recommendation

HOW YOU CAN HELP
===============

There are a number of tasks where we need help to make these
recommendations more useful.  Help would be appreciated in the following
areas.  If you are willing to take on tasks please write back and let us
know:

* Exploring what current social conventions are around pushing to other
  people's repositories in the debian group on salsa and documenting
  them.  This is more about documenting what people do than about
  documenting what permissions people have.

* We've talked a number of times about how it would be great to do a
  better job of documenting what branch format (patches unapplied,
  patches applied, dpm, packaging only, etc) a repository uses.  It
  would be great if someone would work on figuring out how to document
  that.  It would be even better if someone would work with tool authors
  like the authors of git-buildpackage, dgit, debcherry,  etc and see if
  there are ways these tools could automatically maintain this
  information.  This is an area where cross-tool coordination could make
  all our lives easier.

* In the "Account Transition" section, I talk about some of the issues
  that can come up as people become DDs, retire, or otherwise have their
  account change.  It would be very helpful to get someone to consider
  these issues and figure out what if any changes are needed.  That
  probably involves working with Salsa Admin, DSA, and NM among others.

* Once we get done here we'll need people to help fold this in to
  developer's reference or other appropriate documentation sources.  I'm
  not going to have time to do that.

* Not quite related to this round.  We had a discussion of using native
  packages.  My conclusion from that is that there are a lot of issues
  to consider when choosing to use a native format but that sometimes it
  is a reasonable choice.  It would be really helpful to get all the
  issues people brought up in that discussion onto a wiki page into a
  non-judgmental way.  I think using native formats will always be
  somewhat advanced, but I know I'd benefit from the wiki page.  If no
  one else gets to this I'll get there in a few months probably.
 

Existing Teams
==============

If there is an existing packaging team that your package fits into, join
that team and package it there.  Use their recommended Git packaging
practices.

See the Team Recommendation below if you are setting up a new team.


General Recommendation
======================


This recommendation is intended for people who are simply looking for an
approach that will work well with approaches others are using.  If you
are packaging something in a large team, follow their existing
practices.  If you are setting up a new team, see the recommendation for
teams.  If you have your own ideas about what you'd like to do and
choose to disregard this recommendation, that is also fine.  If you
do not already know the answer you want, this recommendation is for you.

If you are a Debian Developer packaging a package for inclusion in
Debian, you should store your packaging information in one repository
per package on salsa.debian.org in the debian group.  That is you should
create a repository under https://salsa.debian.org/debian .

You should make sure that at least one person sets their notification
level to 'watch' rather than global.  This way you will receive
notifications of merge requests and changes.

Hopefully you will choose to monitor merge requests for your
repository.  If not, turn off merge requests.  If you enable merge
requests, you should be as responsive to merge requests as you are
patches in the BTS.

You should document the branch format (such as patches unapplied (gbp
pq)) that your repository uses.  Best practice for documenting this is
still evolving.  The best current option is to document your branch
format in debian/README.source.  Alternatively if you use `dgit
push-source`, your source format is documented in maintainer tags.


If you are not a Debian Developer, you cannot directly create a
repository in the debian group.  If you're willing to wait for a Debian
Developer to create a repository for you and grant you access, do that.
If that wait would be long enough to frustrate you or demotivate you,
you should create a repository in a your personal namespace on
salsa.debian.org and store your package there.

By creating a repository in the debian group, you grant access to all
developers.  That way people performing NMUs can directly commit their
changes.  It will also make it easier if you later orphan the package to
preserve version control history, URIs and merge request history.


Notes and Limitations
---------------------

The debian group is great for relatively unrelated packages.  It may not
be appropriate for a large number of related packages (especially when
maintained by a team) for these reasons:

* It is hard to examine the group of related packages without also
  seeing other unrelated packages

* You cannot subscribe to watch a group of related packages with one
  click

* Access to people who are not Debian Developers needs to be granted on
  a per-repository basis in the debian group

Some teams with a large number of packages have adopted a
monolithic format where a single repository contains information for
many packages.  It is not the intent of this recommendation to judge
that approach either positively or negatively; this recommendation is
targeted at a very different audience.

This recommendation is not appropriate in cases when Debian Developers
should not have push access to the repository.  For example if you are
mirroring the repository from another service and do not want changes
pushed to this replica, using the debian group is inappropriate.
Two-way mirroring is desired when possible.


Packages not on Salsa
=====================

Some people choose not to host their packages on Salsa for a variety of
reasons.  Examples include wanting to host Debian packaging in the same
repository as upstream development when the Debian and upstream
maintainers are related.  Even if you do not host your packaging work on
Salsa you should:

* Host your packaging work in Git assuming that the upstream does not
  use some other version control system.  There is no recommendation
  when upstream uses different version control.  There are advantages to
  using Git as well as advantages to using what the upstream chooses.

* Use a public repository where in-progress and ongoing work are
  available to the public.  Do not just push when you release.

* Set the vcs-git and vcs-browser fields in debian/control to point to
  your repository.

* Document the maintainer branch format you use.

You are encouraged to mirror your repository to Salsa so that people can
find more of the Debian packaging in one place.

Non-Free Services
-----------------

It is important to many members of our community that they be able to do
work within Debian without using non-free software or web services that
depend on non-free software.  Github is a common example of a web
service that uses non-free software.
Clearly community members can use the BTS, make NMUs and interact with
mailing lists.

However, not being able to interact with packaging Git repos can
significantly reduce developer's effectiveness.

Evaluate the impact on our community before using a non-free service
like Github for Debian packaging.  Would your needs be met by mirroring
from free services to non-free services?

Using non-free services for Debian packaging is not recommended but is
permitted.

Some have suggested that we forbid the use of non-free services for
Debian packaging.  First, we'd rather public git repositories be
available than for example have people claim they are not using VCS at
all.  Secondly, we don't even have a mechanism to codify such a policy,
nor is it clear that it would be desirable to have formal policy for
what tools people use prior to upload.

If the use of a non-free service is preventing an active member of our
community from contributing to a team, the team should work to find a
solution so that member can contribute effectively.



Recommendation for Teams
========================

The debian group on Salsa may not be the best choice for teams
maintaining a large number of packages.  The team packages will be mixed
together with other packages and that may be confusing.

Often, a team will want their own group to segregate their packages.

In this case the team should create a team group on Salsa and store
their packages there.  This will make looking at changes to these
packages, merge requests for the set of packages, and CI information
easier.

It is not possible to grant the debian group access to all packages in a
group at once.  However, if a team wishes, they may grant push access to
the debian group on a package-by-package basis.

Best practice is for multiple people to have owner privileges on the
team.  If the team owner leaves Debian, having a second team owner will
make managing the team easier.  Similarly if an account is locked for
security reasons, having a second owner will make the transition
smoother.


Account Transitions
===================

This isn't so much a case where we have a recommendation as where it
seems like we need to do more work.

There are some awkward issues around Salsa accounts:

* When you retire from Debian or are removed, your salsa account becomes
  inaccessible.  That means you no longer have access to teams and repos
  that you are granted access to.

* Granting access to repositories in personal namespace becomes
  problematic.  You will typically no longer be able to push to
  repositories in your personal namespace.

* If you are the only active team owner for a team, gaining access to
  the team will require administrator action and will be relatively
  slow.

* You will need to regain access to any repositories that you should
  have access to.

Depending on why an account is locked, the above may be strongly desired
or deeply frustrating.

There's been enough discussion of this issue that it seems like someone
should take on the task of trying to figure out what we as a project
want.  Solutions might include:

* Procedures that get followed when accounts are closed.  We probably
  don't want more effort added to the tasks of NM front desk, MIA or
  DAM.  But if additional volunteers stepped forward who wanted to help
  with these transitions, it might make sense to approach things that
  way.

* Changing how Salsa and Debian LDAP interact.  This would involve
  figuring out what we want and working with the Salsa admins and DSA.

 

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

Holger Levsen-2
On Mon, Oct 07, 2019 at 09:22:46PM -0400, Sam Hartman wrote:
> [...] as a last opportunity for
> others to comment.

what's the deadline to grok this 20k and respond?


--
cheers,
        Holger

-------------------------------------------------------------------------------
               holger@(debian|reproducible-builds|layer-acht).org
       PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

Raphael Hertzog-3
On Tue, 08 Oct 2019, Holger Levsen wrote:
> On Mon, Oct 07, 2019 at 09:22:46PM -0400, Sam Hartman wrote:
> > [...] as a last opportunity for
> > others to comment.
>
> what's the deadline to grok this 20k and respond?

It's in the subject: [comments by 11/05/2019]

November 5th.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: https://www.freexian.com/services/debian-lts.html
Learn to master Debian: https://debian-handbook.info/get/

signature.asc (535 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

gregor herrmann-3
In reply to this post by Sam Hartman-5
On Mon, 07 Oct 2019 21:22:46 -0400, Sam Hartman wrote:

> Ansgar argued that documenting the branch format is unnecessary given
> all the work you need to do to contribute to a package.  Several
> disagreed arguing that it helped them do their work.

I have an idea where Ansgar might be coming from; maybe I'm wrong but
in any case I have the following reservation or request for
clarification:
 
> Conclusiony Thing
> -----------------
[…]
> * If you are using debian group you should document your branch format.
>   Update to say that.

Ok, here it says that the branch format should be documented for the
Debian group (which reads like "not for teams if teams have different
policies / general documentation somewhere").
 
> General Recommendation
> ======================
>
> This recommendation is intended for people who are simply looking for an
> approach that will work well with approaches others are using.  If you
> are packaging something in a large team, follow their existing
> practices.  
[…]
> You should document the branch format (such as patches unapplied (gbp
> pq)) that your repository uses.  Best practice for documenting this is
> still evolving.  The best current option is to document your branch
> format in debian/README.source.  Alternatively if you use `dgit
> push-source`, your source format is documented in maintainer tags.

Does this paragraph refer to repos in the Debian group or to all
repos, i.e. team-maintained ones?

The reason I'm asking is that I have very bad memories of the times
when Policy required to document the use of quilt in
debian/README.source (although quilt was very visible in both
debian/control and debian/rules at that time); and literally
thousands of boilerplate debian/README.source files were added; and
some years later removed after a Policy change. I'm pretty sure there
are still quite a few left in the archive.

So if this recommendation is targetted at repos in the Debian group
only (and teams can ignore it) I'm not enthusiastic by the prospect
of outdated documentation there, but well … If it's meant to apply to
team-maintained repos as well I'd consider this a mistake.

(Maybe a sentence like "The following paragraphs refer to the Debian
group, for teams see below under 'Recommendations for Teams'."
in/after the first paragraph would help to disambiguate the scope at
least.)
 

Cheers,
gregor

--
 .''`.  https://info.comodo.priv.at -- Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
   `-   NP: Janis Joplin: Cry Baby

signature.asc (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

Sam Hartman-3
>>>>> "gregor" == gregor herrmann <[hidden email]> writes:

    >> General Recommendation ======================

This entire section effectively only applies to individual packages that
are not in a team.

Would "When there is no Team to use" or similar be a better headline
and address your concern?
    >> This recommendation is intended for people who are simply looking
    >> for an approach that will work well with approaches others are
    >> using.  If you are packaging something in a large team, follow
    >> their existing practices.
    gregor> […]
    >> You should document the branch format (such as patches unapplied
    >> (gbp pq)) that your repository uses.  Best practice for
    >> documenting this is still evolving.  The best current option is
    >> to document your branch format in debian/README.source.
    >> Alternatively if you use `dgit push-source`, your source format
    >> is documented in maintainer tags.

    gregor> Does this paragraph refer to repos in the Debian group or to
    gregor> all repos, i.e. team-maintained ones?

As implied by above, effectively non-team stuff.
That recommendation technically applies to:

* debian group

* Things created in personal namespaces by non-DDs who don't want to
  wait for someone to create a debian-group repo for them.

but yes, I was explicitly not wanting to make teams document source
package format in README.source.

Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

gregor herrmann-3
On Wed, 09 Oct 2019 12:40:12 -0400, Sam Hartman wrote:

> >>>>> "gregor" == gregor herrmann <[hidden email]> writes:
>
>     >> General Recommendation ======================
>
> This entire section effectively only applies to individual packages that
> are not in a team.
>
> Would "When there is no Team to use" or similar be a better headline
> and address your concern?

Yes, something like this would make it clearer IMO.

> That recommendation technically applies to:
>
> * debian group
>
> * Things created in personal namespaces by non-DDs who don't want to
>   wait for someone to create a debian-group repo for them.

Maybe add this as an intro?
 
> but yes, I was explicitly not wanting to make teams document source
> package format in README.source.

Great, thanks for the clarification.


Cheers,
gregor

--
 .''`.  https://info.comodo.priv.at -- Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
   `-   NP: Janis Joplin: Cry Baby

signature.asc (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

Theodore Y. Ts'o
In reply to this post by Sam Hartman-5
One thing that is been left unclear is what does it mean to "use
salsa"?  For example, the e2fsprogs git repository is hosted at
multiple locations:

  * https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git
  * https://github.com/tytso/e2fsprogs.git
  * https://git.code.sf.net/p/e2fsprogs/code
  * https://repo.or.cz/e2fsprogs.git

I push changes to all of them when do updates from my development
system, and they all have my debian packaging branches.  Which one is
the "master" repo?  There's really no such thing.  I suppose we could
call git.kernel.org the "master" because it was the first but
technically, the bitkeeper repository predates them all.  :-)

So I could create a Salsa repo for e2fsprogs and add it to the list;
but what does that actually mean?  What does it mean to have a Vcs-Git
line pointing at git.kernel.org versus salsa.debian.org?  It surely
doesn't mean anything about access rights, whether it's "any random
Debian person can check in arbitrary things to the repo --- there are
some packages that are in groups that have very tight access controls,
and that's probably a good thing.  I'm much more comfortable knowing
that stealing some random Debian maintainer's git credentials is not
enough to install trojan horses into the openssh package!

And suppose I did create a Salsa repo for e2fsprogs, which could be
changed by anyone in the debian group.  And suppose someone adds
something to the git repo which is totally wrong, and which bypassed
any kind of code review.  No problem!  I'd just do a force push and
the commit in Salsa would Go Away.  Or is that sort of thing frowned
upon with having a git repository on Salsa?  

As a result, I'd argue that when we talk about "forcing" people to use
Salsa, it's actually kind of underspecified what might be meant by
that.  If a developer has their git repository on github, or
git.kernel.org, or on their own private server, what value does it add
to have another copy on Salsa?  As far as I'm concerned, it neither
adds much value, nor does it cost much.

It's when you start saying that it must be the *canonical* repository,
and it doesn't matter what random DD's push to it; once they've pushed
to it, it must be preserved ***forever*** without any forced pushes or
rewinds, that it starts to make more of a difference.

Cheers,

                                                - Ted

Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

Sam Hartman-3

TL;DR: These are only guidelines.  You're free to ignore them.  So I
think the issue you bring up isn't a big deal in this case.

In more detail:

>>>>> "Theodore" == Theodore Y Ts'o <[hidden email]> writes:

    Theodore> One thing that is been left unclear is what does it mean
    Theodore> to "use salsa"?

Use Salsa" as a term doesn't seem to appear in the recommendation text.

I'd assume that your evaluation of the recommendation might go like
this:

* No existing team

* Hmm, I'm a Debian developer packaging for debian.  So, let's look at
the general recommendation.  Wonder along.  Get to notes and limitations
and decide probably that you don't want all developers to have push
access because you don't want to force push when you disagree with
changes.

* So you decide that the general recommendation is not for you  and get
to the "Packages not on Salsa" recommendation.  Perhaps you decide to
mirror to salsa as suggested there.
I'm fairly sure you're already following the rest of the guidelines in
that section.

    Theodore> So I could create a Salsa repo for e2fsprogs and add it to
    Theodore> the list; but what does that actually mean?  What does it
    Theodore> mean to have a Vcs-Git line pointing at git.kernel.org
    Theodore> versus salsa.debian.org?  It surely doesn't mean anything
    Theodore> about access rights, whether it's "any random Debian
    Theodore> person can check in arbitrary things to the repo ---

Mostly it makes things more consistent about where to find your code for
people thinking more about a lot of Debian packages than about
e2fsprogs.
Does it have huge value?  It might some day if we get to a point where
we do require everyone to at least mirror to salsa.
Today it makes it easier for us to do things like that in the future.

Is it worth you doing given how many  repos you already have?
Don't know; perhaps you have enough that it is easy.  Perhaps adding one
more is more effort and you don't see the point.


    Theodore> And suppose I did create a Salsa repo for e2fsprogs, which
    Theodore> could be changed by anyone in the debian group.  And
    Theodore> suppose someone adds something to the git repo which is
    Theodore> totally wrong, and which bypassed any kind of code review.
    Theodore> No problem!  I'd just do a force push and the commit in
    Theodore> Salsa would Go Away.  Or is that sort of thing frowned
    Theodore> upon with having a git repository on Salsa?

I bet we don't have a consensus.  I personally wouldn't put a repo that
commonly had master force-pushed in the debian group on Salsa.  I
wouldn't mind force-pushing rarely say if I changed branch formats or
there was a significant oops.

I would prefer not to have to develop such a consensus now.
I agree that it's a great thing to converge on in the future.

    Theodore> As a result, I'd argue that when we talk about "forcing"
    Theodore> people to use Salsa, it's actually kind of underspecified
    Theodore> what might be meant by that.

I agree.
But since we're not talking about forcing people to use Salsa, I think
it is OK.
We're giving people best practice guidelines to use as a starting point.
Unlike the dh discussion where there are some behaviors we consider
buggy, these are guidelines you can use if they help you or ignore if
they do not.

I think it's OK to be less specified for such guidelines.

We may get more specific in the future.  But I think if we try to do
that all in one step, it will be too big to ever take.

--Sam

Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

Tomas Pospisek ML
In reply to this post by Sam Hartman-5
Hi Sam & Debianistas,

this is far TLDR for me. That is not meant as a critique, but as a
feedback so you have a data point from some random Debianer's available
CPU resources.

(in general I'm fine to declare best practices for whatever issue so
that people can orient themselves on where to head to, provided that
those "best practices" aren't "too far out")

Am 08.10.19 um 03:22 schrieb Sam Hartman:
>
> [...]

Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

Sam Hartman-3
In reply to this post by Sam Hartman-5
I think we have about two weeks left in the review period.

Just as a reminder we do need comments.
Even if people generally agree, we do need at least a few comments to
that effect.

Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

Enrico Zini
On Tue, Oct 22, 2019 at 09:58:00PM -0400, Sam Hartman wrote:

> I think we have about two weeks left in the review period.
> Just as a reminder we do need comments.
> Even if people generally agree, we do need at least a few comments to
> that effect.

I like the current proposal for a default suggestion for how to lay out
repositories on salsa when one is not working with a team and one has no
better ideas. I think it would both ease the path of people[1]
(re)learning debian packaging, and of people[1] who can do packaging
and, having no particular needs, like to stick to standards to ease
further work through uniformity.

I liked Ansgar's concern about the standard risking to become obsolete.
As a recommendation for a default layout, I think there should be a way
to keep the recommendations aligned to current project expectations, and
I'd like some upgrading-checklist-style document to assist in keeping
things aligned.

Hopefully the recommendations won't change too often, or the effort
saved by not having to explore multiple options while setting up a
packaging workflow, would be spent in keeping the workflow up to date.

I have no particular ideas about how to keep a recommendation up to
date, though I like the idea of having in Debian the possibility of
doing that a lot, as I see it valuable for a broader range of things.

While over-normating things in Debian would stifle its valuable
diversity, I think that being able to recommend as much standards as
possible would make it much easier to navigate that diversity.


Enrico

[1] like often me
--
GPG key: 4096R/634F4BD1E7AD5568 2009-05-08 Enrico Zini <[hidden email]>

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

Thorsten Glaser-7
In reply to this post by Sam Hartman-5
Sam Hartman dixit:

>depend on non-free software.  Github is a common example of a web
>service that uses non-free software.

So is Salsa. It does not use the packaged version of GitLab,
which is in a sorry state anyway, and not suitable for a stable
release, but the proprietary “open core” versioin from the
GitLab people.

I prefer to not host my packages on Salsa; for this reason, and
because of the experience of having to change all the VCS headers
multiple times, move repos, etc. when using the Debian-provided
hosting platform. I can host very well on my own, using a Free
Software platform.

bye,
//mirabilos
--
I believe no one can invent an algorithm. One just happens to hit upon it
when God enlightens him. Or only God invents algorithms, we merely copy them.
If you don't believe in God, just consider God as Nature if you won't deny
existence. -- Coywolf Qi Hunt

Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

Richard Laager
In reply to this post by Sam Hartman-5
This all looks very good.

Presumably the repository / Salsa project name should match the source package name? If so, that might be good to note, at least as the default.

I'd love to see more information about a recommended branch structure. FWIW, I've been using branches named for each release (e.g. "sid" is the default, but I also have "buster" for a (proposed) stable update, will likely soon have "buster-backports"). This works really well, and also scales to also having branches for Ubuntu (e.g. I have "bionic" and "cosmic" too due to some SRUs). It also keeps "master" available for the upstream master branch. This seems so obvious and wonderful to me, but I'm not sure how popular it is.

I'm currently maintaining my packages on GitHub, but I'm going to move them to Salsa shortly to follow the consensus recommendation. I'm not a DD (and not a DM either, though I'm finally getting around to applying). Unfortunately, this means I have the, IMHO obnoxious, -guest suffix on my Salsa account. I certainly don't want to create things under that namespace, and it's doubly painful because if I ever become a DD, my username would change. So I'm asking my sponsor to create projects in the debian namespace. Of course, that's the recommendation anyway, so the -guest might be an inadvertent feature because it's encouraging me to use the debian team. :) I understand that DMs are quite literally second-class citizens (and non-DM package maintainers are third-class), but I really wish that things like Salsa usernames didn't call that out.

If this message is threaded wrong, my apologies. While I am now, I was not subscribed to debian-devel at the time. I have tried to set In-Reply-To directly, but this is my first attempt at doing so. In these situations, I would normally download the mbox archive, import it, and reply to the message that way, but mbox archives are seemingly unavailable (at least to non-DDs?) for Debian lists.

--
Richard


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

wferi-2
In reply to this post by Sam Hartman-5
Sam Hartman <[hidden email]> writes:

> General Recommendation
> ======================
> [...]
> You should document the branch format (such as patches unapplied (gbp
> pq)) that your repository uses.  Best practice for documenting this is
> still evolving.  The best current option is to document your branch
> format in debian/README.source.

I'm not sure this best current option (README.source) is something I'd
actually recommend.  Granted, it's annoying to have to figure the git
workflows, and a freeform documentation text like README.source helps if
you have to do it a couple of times, but it still isn't a thing you can
automate.  Thus I think that the overall burden it creates outweighs its
value.  Also, from the Policy: "maintainers are encouraged to document
in a debian/README.source file any source package with a particularly
complex or unintuitive source layout or build system" -- do we intend to
recommend something "particularly complex or unintuitive"?

I have to admit I haven't read the full discussion.  Have we got a list
of items we need to know about the "branch format"?  Your text mentions
patches applied or unapplied, which dpkg-source (and, according to you,
dgit) already handles, so a utility could provide the answer if there's
at least one patch in the package; why document it then?  If the
heuristic is deemed too fragile or we want to store this information
even in the absence of patches, perhaps we could invent a new (local)
dpkg-source option to override it and require using that (just an idea).
Another obvious item is the main packaging branch, which is declared by
Vcs-Git ("ideally", says the Policy -- shouldn't we make this stronger?)

If we really want to go beyond these, the next step would be discovering
the actual names of the DEP-14 entities, I guess.  This would probably
start with identifying the tooling (gbp, git-debrebase, git-dpm, etc.)
then reading their configuration (if any).  This would give answers for
the previous items as well.  But do we target mass operations on this
scale of variability at all?

Anyway: I think we should recommend setups which allow reliable
determination of the needed information, thus not requiring manual
textual documentation.
--
Thanks,
Feri

Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2 [comments by 11/05/2019]

Nicholas D Steeves
Richard Laager <[hidden email]> writes:

> This all looks very good.
>
> Presumably the repository / Salsa project name should match the source
> package name? If so, that might be good to note, at least as the
> default.
>

I'm curious what other people think about this point, because it could
cause a lot of churn either in source package renames or salsa project
renames and uploads to fix VCS links.

> I'd love to see more information about a recommended branch
> structure. FWIW, I've been using branches named for each release
> (e.g. "sid" is the default, but I also have "buster" for a (proposed)
> stable update, will likely soon have "buster-backports"). This works
> really well, and also scales to also having branches for Ubuntu
> (e.g. I have "bionic" and "cosmic" too due to some SRUs). It also
> keeps "master" available for the upstream master branch. This seems so
> obvious and wonderful to me, but I'm not sure how popular it is.
>

https://dep-team.pages.debian.net/deps/dep14

It sounds like you're already using something similar :-)

[1] One modification to DEP14 that I'd like to see is a requirement to set
the Debian development branch as the default branch on Salsa, so that a
debcheckout will put the user in the right place to get work done.  Eg:
"debian/next", "debian/sid", "debian/devel", etc.  This also makes it
easier for new contributors to submit MRs based on the correct branch.

[snip]
> If this message is threaded wrong, my apologies. While I am now, I was
> not subscribed to debian-devel at the time. I have tried to set
> In-Reply-To directly, but this is my first attempt at doing so. In
> these situations, I would normally download the mbox archive, import
> it, and reply to the message that way, but mbox archives are seemingly
> unavailable (at least to non-DDs?) for Debian lists.
>

I didn't notice a threading problem; however your email had long lines
that I had to manually reflow.


[hidden email] writes:

> Sam Hartman <[hidden email]> writes:
>
>> General Recommendation
>> ======================
>> [...]
>> You should document the branch format (such as patches unapplied (gbp
>> pq)) that your repository uses.  Best practice for documenting this is
>> still evolving.  The best current option is to document your branch
>> format in debian/README.source.
>
> I'm not sure this best current option (README.source) is something I'd
> actually recommend.  Granted, it's annoying to have to figure the git
> workflows, and a freeform documentation text like README.source helps if
> you have to do it a couple of times, but it still isn't a thing you can
> automate.  Thus I think that the overall burden it creates outweighs its
> value.  Also, from the Policy: "maintainers are encouraged to document
> in a debian/README.source file any source package with a particularly
> complex or unintuitive source layout or build system" -- do we intend to
> recommend something "particularly complex or unintuitive"?
>
[2] I wonder if the investigative work Ian did for tag2upload can be
recuperated for the development of an extension of the source format
specification.  Namely, a shorthand git-project format.  One would then
consult the table for repos that declare a style using a shorthand key
term that one isn't familiar with.  "other" would need to be an option
for a while.  I'm not sure if it should go in L2 of "source/format" or
somewhere else.

> I have to admit I haven't read the full discussion.  Have we got a list
> of items we need to know about the "branch format"?  Your text mentions
> patches applied or unapplied, which dpkg-source (and, according to you,
> dgit) already handles, so a utility could provide the answer if there's
> at least one patch in the package; why document it then?  If the
> heuristic is deemed too fragile or we want to store this information
> even in the absence of patches, perhaps we could invent a new (local)
> dpkg-source option to override it and require using that (just an
> idea).

The utility would parse [2].  I'm not sure about using heuristics and
suspect that the "branch format" list will need to be manually
maintained.  It might also benefit from codification in Policy for the
following case: a decade from now someone adopts a package, but the
"branch format" document/section has substantially changed since the
last upload.  Codification in Policy would allow the person adopting the
package to reference a historical copy of Policy to more easily decipher
the branch format.

> Another obvious item is the main packaging branch, which is declared by
> Vcs-Git ("ideally", says the Policy -- shouldn't we make this stronger?)
>

See [1].  I'd prefer for the general case to be solved on salsa rather
than using extra Vcs-Git metadata.  To get the affect of "debcheckout
foo && cd foo" and be ready for work ISTM that debcheckout (and other
tools) would need to parse this and switch to the correct branch.  How
can this approach solve the problem of getting the wrong working branch
after a salsa fork and clone?  eg: How to avoid raising the barrier of
entry to new contributors?

> If we really want to go beyond these, the next step would be discovering
> the actual names of the DEP-14 entities, I guess.  This would probably
> start with identifying the tooling (gbp, git-debrebase, git-dpm, etc.)
> then reading their configuration (if any).  This would give answers for
> the previous items as well.  But do we target mass operations on this
> scale of variability at all?
>
> Anyway: I think we should recommend setups which allow reliable
> determination of the needed information, thus not requiring manual
> textual documentation.
I agree that this would be the ideal.  How would one distinguish between
git-debrebase, git-dpm, and manually cherry picked upstream commit?


Cheers,
Nicholas

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Git Branch Names / DEP-14 (Was: Re: Summary: Git Packaging Round 2 [comments by 11/05/2019])

Richard Laager
On 11/5/19 9:51 PM, Nicholas D Steeves wrote:
> Richard Laager <[hidden email]> writes:
>> I'd love to see more information about a recommended branch
>> structure. FWIW, I've been using branches named for each release
>> (e.g. "sid" is the default, but I also have "buster" for a (proposed)
>> stable update, will likely soon have "buster-backports"). This works
>> really well, and also scales to also having branches for Ubuntu
>> (e.g. I have "bionic" and "cosmic" too due to some SRUs). It also
>> keeps "master" available for the upstream master branch. This seems so
>> obvious and wonderful to me, but I'm not sure how popular it is.

As I moved my first package to Salsa and was rethinking everything, I
switched from "sid" to "unstable", for reasons that will come up below.

> https://dep-team.pages.debian.net/deps/dep14
>
> It sounds like you're already using something similar :-)

I had forgotten about DEP-14 until it was mentioned earlier today by
Feri <[hidden email]>. I'll likely rename my branches again to add the
debian/ prefix, to be in compliance with DEP-14.

DEP-14 specifies three options for development releases:

debian/master (the recommended default)
debian/sid
debian/unstable

Having three names increases inconsistency between packages, which seems
to me contrary to the goal of DEP-14.

If someone is working with both unstable and experimental, then they
must use two branches to differentiate them. DEP-14 says to do so with
debian/experimental for experimental. So far, so good. For unstable,
DEP-14 says to use debian/sid or debian/unstable. Why not A) pick *one*
of those, and B) always use it, never using debian/master?

As for _which_ one (debian/sid or debian/unstable)... The DEP-14
recommended branch names for stable release are debian/CODENAME. The
recommended branch name for experimental is debian/experimental. These,
plus debian/unstable, but not debian/master or debian/sid, are
consistent with distribution names from debian/changelog and the
.changes file. [0] Therefore, it seems like one should use debian/unstable.

[0] https://www.debian.org/doc/debian-policy/ch-controlfields.html#id25

--
Richard


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Git Branch Names / DEP-14

Russ Allbery-2
Richard Laager <[hidden email]> writes:

> If someone is working with both unstable and experimental, then they
> must use two branches to differentiate them.  DEP-14 says to do so with
> debian/experimental for experimental. So far, so good. For unstable,
> DEP-14 says to use debian/sid or debian/unstable. Why not A) pick *one*
> of those, and B) always use it, never using debian/master?

My normal use of experimental does not involve maintaining unstable and
experimental branches simultaneously.  I essentially never do that;
instead, I maintain one development branch, which I upload to either
experimental or to unstable based on things like where we are in the
release cycle or whether I need to stage some change.  If I'm uploading to
experimental, I don't fix bugs in unstable until the experimental release
is ready for upload to unstable.  (The exception is if we're in the middle
of a release and I need to fix something that will go into the next
release, but at that point I just use debian/<codename> for the unstable
upload that will propagate to that release.)

I know some people do more of a two-branch setup, but I think my approach
is reasonably common for a lot of packages.

In that case, both debian/sid and debian/unstable are, well, wrong if the
latest version was uploaded to experimental.  debian/master correctly
captures the semantics of what I'm doing: it's the master development
branch, which is going into either unstable or experimental as
appropriate.

--
Russ Allbery ([hidden email])              <https://www.eyrie.org/~eagle/>

Reply | Threaded
Open this post in threaded view
|

Re: Git Branch Names / DEP-14

Simon McVittie-7
On Tue, 05 Nov 2019 at 20:40:43 -0800, Russ Allbery wrote:
> My normal use of experimental does not involve maintaining unstable and
> experimental branches simultaneously.
...
> I know some people do more of a two-branch setup

One common reason to need to use experimental more actively is if your
upstream has a relatively long-running "latest feature development"
branch that is explicitly not suitable to be in a stable release, such
as dbus 1.odd.z, GNOME 3.odd.z, and Linux/Xorg/Mesa release candidates.

dbus and GNOME both use the versioning scheme popularized by pre-2.6
Linux kernels, where version x.even.z is considered stable and will
receive security fixes, but version x.odd.z is not security-supported,
and is only suitable for use in contexts where you can guarantee to
replace it with the next stable branch as soon as it becomes available.

    smcv

Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2

Sam Hartman-3
In reply to this post by Sam Hartman-5

TL;DR: I'd feel a lot more comfortable if a couple of people would
explicitly review wether I correctly captured the discussion in the
summary.

So, we've received a number of comments on aspects of the discussion.
That plus the original discussion leads me to believe we really are
interested in this issue and that there's community interest in moving
forward.

But many of the comments have explicitly said they didn't follow the
discussion.  The summaries are half the check that the facilitator is
doing a good job in a consensus discussion.  The other half is having
some people involved sanity check the summary.

It would really help me have the confidence to move forward if a couple
of people did that:

* If you were involved enough that you can read the summary and say
  "Yeah, that's more or less what happened," please please do that. (If
  you think I got something wrong in the summary, then please say that too)

* If you're willing to read the summary and compare against the actual
  discussion, I'd greatly appreciate that, especially if we cannot get
  people involved to confirm from their memory.  If you do that, I
  recommend using a threaded mail reader  and being prepared to skip
  reading subthreads that get too repetative or off-topic.  When I did
  my re-read to prepare the summary, I trimmed out most of the
  discussion of non-free services from my re-read.  With that trimmed,
  it took me an hour and a half to take notes on the entire discussion.
  I do understand that's a significant time investment to ask others to
  make.

* If you aren't able to spend that time or have personal recollections,
  but think the level of review I and others have already done is
  adequate, please say that.

Thanks,

--Sam

Reply | Threaded
Open this post in threaded view
|

Re: Summary: Git Packaging Round 2

Sean Whitton
Hello,

On Wed 06 Nov 2019 at 11:10AM -05, Sam Hartman wrote:

> * If you were involved enough that you can read the summary and say
>   "Yeah, that's more or less what happened," please please do that. (If
>   you think I got something wrong in the summary, then please say that too)

I read almost all the mail from the discussion and when I read your
summary I did not think that anything was incorrect.  HTH!

--
Sean Whitton

signature.asc (847 bytes) Download Attachment
12