Survey: git packaging practices / repository format

classic Classic list List threaded Threaded
102 messages Options
1234 ... 6
Reply | Threaded
Open this post in threaded view
|

Survey: git packaging practices / repository format

Ian Jackson-2
While trying to write the dgit FAQ, and some of the relevant docs, it
has become even more painfully obvious that we lack a good handle on
what all the different ways are that people use git to do their Debian
packaging, and what people call these formats/workflows, and what
tools they use.

Can you please look through the table below and see if I have covered
everything you do ?

In particular:
 - have I missed a git repository and history layout
 - have I missed a primary tool that should be mentioned
 - are any of the details wrong for workflows that you use ?

 Main packaging     Delta from upstream     Tools for manipulating
  git branch         represented as          delta from upstream,
  contains                                   building .dsc, etc.

 Unmodified         debian/patches             gbp, gbp pq
  upstream files,    (only)                    quilt / dquilt
 plus debian/*                                 Manual patch editing
 incl. d/patches

 Modified           Direct changes             git merge
  upstream files,    to upstream files          (.dsc: 1.0-with-diff or
 plus debian/*.                                 single-debian-patch)
 Maybe d/patches, depending.
 History has direct merges from upstream.

 Modified           Direct changes to          git-debrebase
  upstream files,    upstream files.
 plus debian/*
 Sometimes d/patches.
 History is special git-debrebase rebasing topic branch format.

 Modified           Direct changes to          git-dpm
  upstream files,    upstream files
 plus debian/*,    
 plus d/patches,  
 plus .git-dpm    
 History is special git-dpm rebasing topic branch format.

 Only debian/*,     d/patches, only;           gbp ?
 with d/patches     Baseline upstream:         quilt/dquilt ?
                     changelog version =>
                     upstream git tag

 Only debian/*,     d/patches, only;           gbp ?
 with d/patches     Baseline upstream:         quilt/dquilt ?
                     changelog version =>
                     .orig tarball(s)

 Template debian/*. Patches in package-        language-specific
 One branch for      specific subdirectory;     monorepo tooling,
  many packages.                                found in same branch
 Tooling to make    Baseline upstream is
  d/control etc.     named by reference somehow
  during build

Thanks,
Ian.

--
Ian Jackson <[hidden email]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Chris Lamb -2
Dear Ian,

> Can you please look through the table below and see if I have covered
> everything you do ?
>
> In particular:
>  - have I missed a git repository and history layout
>  - have I missed a primary tool that should be mentioned
>  - are any of the details wrong for workflows that you use ?

[…]

I am genuinely not being snarky here nor the "logical fallacy spotter"
pub bore, but don't forget that not everybody can follow debian-devel
and thus there may be some blind spots or observation bias at work in
the results.

Best of luck with the survey, naturally, I look forward to the outcome.


Regards,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      [hidden email] 🍥 chris-lamb.co.uk
       `-

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Simon McVittie-7
In reply to this post by Ian Jackson-2
On Tue, 28 May 2019 at 16:51:10 +0100, Ian Jackson wrote:
>  Unmodified         debian/patches             gbp, gbp pq
>   upstream files,    (only)                    quilt / dquilt
>  plus debian/*                                 Manual patch editing
>  incl. d/patches

Do you intend "manual patch editing" to include, for example, fetching
patches from upstream's gitweb or mailing list or equivalent, or
generating patches from a parallel VCS checkout of the upstream code?

>  Only debian/*,     d/patches, only;           gbp ?
>  with d/patches     Baseline upstream:         quilt/dquilt ?
>                      changelog version =>
>                      .orig tarball(s)

gbp pq cannot be used for this: it relies on the first setup that I
quoted above. To manage patches in this kind of repository, you need to
either use quilt or similar, or some out-of-band mechanism.

When I've done this in the past, I usually fetched patches from
upstream's VCS repository via out-of-band mechanisms and dropped them
into debian/patches. Rebasing interdependent patches to apply cleanly
in this kind of repository is hard to do, and I would recommend doing
something else if your patch series is non-trivial.

These days I use gbp pq instead, and my only packaging-only repositories
are for non-textual game data that is too large for convenient use of
git and rarely meaningful to patch: if I wanted to add or change files
in those repositories, I'd put them in debian/ and arrange for them to
overwrite the ones that upstream provided, rather than trying to do
incremental patches.

(If I wanted a better workflow for mostly-non-text-based packages,
I'd be looking into storing the binary blobs with git-annex or git-lfs
rather than plain git.)

Some other classes of package I've encountered (I don't maintain these
and wouldn't recommend their layouts, but they exist):

Relying on debuild -i (e.g. gcc-8)
==================================

Tree contains: usually unmodified upstream files, but could be any of
your examples
Changes to upstream source are: any of your examples, except that changes
to /.gitignore don't appear in d/patches even if other delta does
Patches managed by: any of your examples
Special build tool: use debuild -i (or
debuild --extend-diff-ignore='(^|/).gitignore$')

Debian Linux kernel
===================

Tree contains: an incomplete debian/ directory, notably without d/control,
and no upstream source
Changes to upstream source are: d/patches only
Baseline upstream: changelog version => .orig tarball
Patches managed by: ???
Special build tool: there is a pre-build step to generate d/control

Ubuntu Linux kernel
===================

Tree contains: upstream Linux source code plus an incomplete debian/
and a debian.master/ directory that I do not really understand
Changes to upstream source are: ???
Patches managed by: ???
Special build tool: there is a pre-build step to generate the missing
parts of debian/ from source that includes debian.master/

xorg-team (e.g. mesa)
=====================

Tree contains: upstream files from a git tag (not identical to the
upstream `make dist` tarball), sometimes with extra commits cherry-picked
Changes to upstream source are: applied directly or via d/patches,
sometimes a mixture within the same package
Baseline upstream: changelog version => .orig tarball; files that
exist in git but not in the upstream tarball are compensated for by
extend-diff-ignore in debian/source/local-options
Patches managed by: a mixture of git cherry-pick and quilt

Regards,
    smcv

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Sam Hartman-3
In reply to this post by Ian Jackson-2
Thanks for this work.

I've reviewed your table and believe everything I've used is there.

I think this survey is really important.

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Ian Jackson-2
In reply to this post by Chris Lamb -2
Chris Lamb writes ("Re: Survey: git packaging practices / repository format"):
> Dear Ian,
> > Can you please look through the table below and see if I have covered
> > everything you do ?
> >
> > In particular:
> >  - have I missed a git repository and history layout
> >  - have I missed a primary tool that should be mentioned
> >  - are any of the details wrong for workflows that you use ?>
...
> I am genuinely not being snarky here nor the "logical fallacy spotter"
> pub bore, but don't forget that not everybody can follow debian-devel
> and thus there may be some blind spots or observation bias at work in
> the results.

You make a good point.

I want to catch everyone rather than look for what is used the most,
so it's not a popularity contest.  I will probably put the results on
the wiki at some point, perhaps after we've had a bunfight over
terminology :-).

If folks think it worthwhile I guess I could post to d-d-a, but let's
let it run here for a bit first.

Ian.

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Ian Jackson-2
In reply to this post by Ian Jackson-2
Ian Jackson writes ("Survey: git packaging practices / repository format"):
> While trying to write the dgit FAQ, and some of the relevant docs, it
> has become even more painfully obvious that we lack a good handle on
> what all the different ways are that people use git to do their Debian
> packaging, and what people call these formats/workflows, and what
> tools they use.
>
> Can you please look through the table below and see if I have covered
> everything you do ?

A very important thing I forgot to say:

Please can we leave aside discussion of the merits or otherwise of
each of these formats/workflows.

Perhaps we can talk about that (again!) at some point, but it tends to
derail any conversation about git packaging stuff and I don't want
this thread derailed.

Also I want people to be able to say what they do without feeling they
may be criticised for it.  For that reason, I should also say: please
feel free to reply privately.

Thanks,
Ian.

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Enrico Weigelt, metux IT consult-3
In reply to this post by Ian Jackson-2
On 28.05.19 17:51, Ian Jackson wrote:> While trying to write the dgit
FAQ, and some of the relevant docs, it
> has become even more painfully obvious that we lack a good handle on
> what all the different ways are that people use git to do their Debian
> packaging, and what people call these formats/workflows, and what
> tools they use.
>
> Can you please look through the table below and see if I have covered
> everything you do ?

<snip>

I'm always cloning the upstream repo, branch off at their release tag
and add all necessary chanages as individual git commits - first come
the generic (non-deb specific) patches, then the deb specific ones.
No text-based patches, or even magic rewriting within the build process.
The HEAD is exactly the debianized source tree, which is then fed to
dck-buildpackage.


--mtx

--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[hidden email] -- +49-151-27565287

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Ian Jackson-2
In reply to this post by Simon McVittie-7
Simon McVittie writes ("Re: Survey: git packaging practices / repository format"):
> On Tue, 28 May 2019 at 16:51:10 +0100, Ian Jackson wrote:
> >  Unmodified         debian/patches             gbp, gbp pq
> >   upstream files,    (only)                    quilt / dquilt
> >  plus debian/*                                 Manual patch editing
> >  incl. d/patches
>
> Do you intend "manual patch editing" to include, for example, fetching
> patches from upstream's gitweb or mailing list or equivalent, or
> generating patches from a parallel VCS checkout of the upstream code?

Obviously that needs to be included here.  I'm struggling with a brief
way to express it.  Maybe that is a fool's errand, and it would help
people recognise their own practices if it were less brief.

> >  Only debian/*,     d/patches, only;           gbp ?
> >  with d/patches     Baseline upstream:         quilt/dquilt ?
> >                      changelog version =>
> >                      .orig tarball(s)
>
> gbp pq cannot be used for this: it relies on the first setup that I
> quoted above. To manage patches in this kind of repository, you need to
> either use quilt or similar, or some out-of-band mechanism.

git-buildpackage can build such things, I recently learned.

I have snipped the parts of your message that might be read as an
opinion about what is best, for the reasons explained in my last mail
:-).

> Some other classes of package I've encountered (I don't maintain these
> and wouldn't recommend their layouts, but they exist):
>
> Relying on debuild -i (e.g. gcc-8)
> ==================================
>
> Tree contains: usually unmodified upstream files, but could be any of
> your examples
> Changes to upstream source are: any of your examples, except that changes
> to /.gitignore don't appear in d/patches even if other delta does
> Patches managed by: any of your examples
> Special build tool: use debuild -i (or
> debuild --extend-diff-ignore='(^|/).gitignore$')

I'm not sure I understand this one.

Is this actually a packaging format or patch management workflow, or
is it something else ?

I am leaving aside the .gitignore thing for this survey.  It is a
wrinkle.  Most of the tools I mention in the survey have an opinion
about the .gitignore thing but it is just confusing detail in this
context.

> Debian Linux kernel
> ===================
>
> Tree contains: an incomplete debian/ directory, notably without d/control,
> and no upstream source
> Changes to upstream source are: d/patches only
> Baseline upstream: changelog version => .orig tarball
> Patches managed by: ???
> Special build tool: there is a pre-build step to generate d/control

Thanks, I will add this to my survey.  I assume "patches are managed
by" is the same as any other only-debian/* tree.  The wrinkle is the
need for a special build tool.

> Ubuntu Linux kernel
> ===================
>
> Tree contains: upstream Linux source code plus an incomplete debian/
> and a debian.master/ directory that I do not really understand
> Changes to upstream source are: ???
> Patches managed by: ???
> Special build tool: there is a pre-build step to generate the missing
> parts of debian/ from source that includes debian.master/

Same here.

> xorg-team (e.g. mesa)
> =====================
>
> Tree contains: upstream files from a git tag (not identical to the
> upstream `make dist` tarball), sometimes with extra commits cherry-picked
> Changes to upstream source are: applied directly or via d/patches,
> sometimes a mixture within the same package
> Baseline upstream: changelog version => .orig tarball; files that
> exist in git but not in the upstream tarball are compensated for by
> extend-diff-ignore in debian/source/local-options
> Patches managed by: a mixture of git cherry-pick and quilt

Likewise.

Thanks,
Ian.

--
Ian Jackson <[hidden email]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Ian Jackson-2
In reply to this post by Simon McVittie-7
Simon McVittie writes ("Re: Survey: git packaging practices / repository format"):

> xorg-team (e.g. mesa)
> =====================
>
> Tree contains: upstream files from a git tag (not identical to the
> upstream `make dist` tarball), sometimes with extra commits cherry-picked
> Changes to upstream source are: applied directly or via d/patches,
> sometimes a mixture within the same package
> Baseline upstream: changelog version => .orig tarball; files that
> exist in git but not in the upstream tarball are compensated for by
> extend-diff-ignore in debian/source/local-options
> Patches managed by: a mixture of git cherry-pick and quilt

I'm not sure if you know the answer to this, but I thought of a
question or two:

Apart from the extra files, these orig tarballs must be identical to
the files in git, or dpkg-source would complain about the differences.
So presumably when you say "changelog version => .orig tarball" you
actually mean => some git tag, from which a tarball is made ?

Or do you mean the git tag is the one corresponding to some upstream
version ?  Does it have a systematic name ?  Is it made by upstream ?
The more I think about this the more puzzled I seem.

Ian.

--
Ian Jackson <[hidden email]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Ian Jackson-2
In reply to this post by Enrico Weigelt, metux IT consult-3
Hi, thanks for replying.  You have an interesting workflow which I
think I need to ask some questions about before I can document it
fully.

Enrico Weigelt, metux IT consult writes ("Re: Survey: git packaging practices / repository format"):
> I'm always cloning the upstream repo, branch off at their release tag
> and add all necessary chanages as individual git commits - first come
> the generic (non-deb specific) patches, then the deb specific ones.
> No text-based patches, or even magic rewriting within the build process.
> The HEAD is exactly the debianized source tree,

What source format do you use ?  What is in debian/source/format, if
anything ?  Do you handle orig tarballs at all ?

When you go to a new upstream, you make a new git branch, then ?
Do you publish this git branch anywhere ?

> which is then fed to dck-buildpackage.

What is that ?  manpages.debian.org wasn't any help.
Did you mean dpkg-buildpackage ?

Ian.

--
Ian Jackson <[hidden email]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Sam Hartman-3
In reply to this post by Ian Jackson-2
>>>>> "Ian" == Ian Jackson <[hidden email]> writes:

    Ian> If folks think it worthwhile I guess I could post to d-d-a, but
    Ian> let's let it run here for a bit first.

I don't have an opinion on whether you should post to d-d-a.
I know I'll reference this in my upcoming bits from the DPL because
discussion #2 will be the first git discussion and this seems like data
that may help us in that discussion.

You could mention it in your blog to get it onto planet.
You could submit a micronews  (see the debian publicity team wiki page)
to get more circulation that way.

--Sam

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Simon McVittie-7
In reply to this post by Ian Jackson-2
On Tue, 28 May 2019 at 21:20:50 +0100, Ian Jackson wrote:
> Simon McVittie writes ("Re: Survey: git packaging practices / repository format"):
> > xorg-team (e.g. mesa)
> > =====================
> >
> > Tree contains: upstream files from a git tag (not identical to the
> > upstream `make dist` tarball), sometimes with extra commits cherry-picked
...
> > Baseline upstream: changelog version => .orig tarball; files that
> > exist in git but not in the upstream tarball are compensated for by
> > extend-diff-ignore in debian/source/local-options
>
> I'm not sure if you know the answer to this, but I thought of a
> question or two:
>
> Apart from the extra files, these orig tarballs must be identical to
> the files in git, or dpkg-source would complain about the differences.

You might reasonably assume that, but no, they are not. mesa (and probably
other xorg-team packages) uses v1.0 dpkg-source format combined with
dh --with quilt, so deliberate Debian changes can be either direct
changes to the upstream source code, or quilt patches in d/patches,
or a mixture. Additionally, mesa uses d/source/local-options to ignore
files that only exist in the upstream git tag (which is what gets merged
into the packaging git branch), but not in the upstream `make dist` output
produced from that tag (which is used as the .orig tarball).

My understanding is that this unusual difference between the .orig
tarball and what's in git is an attempt to "square the circle" between
two colliding design principles: "the .orig tarball should be upstream's
official binary artifact" (in this case Automake `make dist` output,
including generated files like Makefile.in but not non-critical source
files like .gitignore) and "what's in git should match upstream's git
repository" (including .gitignore but
not usually Makefile.in).

Most other packaging git repository layouts that I've seen choose one
of those two design principles to follow (whichever one the maintainer
thinks is more important), and ignore the other one. For instance, many
Autotools packages have upstream release tarballs committed to their
upstream or upstream/latest branch, like I do for dbus: their maintainers
have chosen to follow the first of those design principles, because they
think the second is less important.

> So presumably when you say "changelog version => .orig tarball" you
> actually mean => some git tag, from which a tarball is made ?

No, I really do mean the binary artifact released by upstream from
`make dist` output, even though what's committed to the packaging
git repository does not correspond 1:1 to that. I'm aware that this
contradicts some of the assumptions made by both dgit and the 3.0 (quilt)
source package format.

    smcv

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Enrico Weigelt, metux IT consult-3
In reply to this post by Ian Jackson-2
On 28.05.19 22:30, Ian Jackson wrote:
> Hi, thanks for replying.  You have an interesting workflow which I
> think I need to ask some questions about before I can document it
> fully.

I'd call it the 'git-only-workflow' ;-)

The main reasons behind are:
* i wanna be able to easily rebase onto upstream anytime
* i wanna keep generic changes separate from the distro-specific stuff
  (usually I try to do very generic, so it can go into mainline, eg.
  instead of directly patching things like pathes, etc, I'm adding
  new build options, ...)
* i wanna easily bring generic changes upstream
* i don't ever like to cope w/ text-based patches anymore (all these
  apply/unapply cycles really suck :p) - git is much easier to handle,
  IMHO
* i wanna have exactly the build tree in my git repo
* i don't wanna versioning patches (reading diffs of diffs are not quite
  useful :o)

Actually, the workflow is a tiny bit more complex: i'm using layered
branches (regularily rebasing):

layer 0: upstream releases
layer 1: per release maintenance branches w/ generic (hopefully
         upstreamable) fixes - based on the corresponding upstream
         release tags (or potentially their maint branches)
layer 2: per distro and release debianized branches
         (sometimes some layer 1.5 for really generic deb stuff)

Branches and tags have a canonical naming - ref name prefixes,
canonical version numbers, ... (eg. anyting for debian stretch is
prefixed 'stretch/' ...)

Years ago, I've already tried to form layer 1 into a greater,
cross-distro community, where stabelization efforts are shared between
many distros (kinda send-patches.org but w/ high grade of normalization
and automation. It was called the 'oss-qm' project (github org with same
name). But the interest from dist maintainers was asymptotically
approaching zero, from below.

> Enrico Weigelt, metux IT consult writes ("Re: Survey: git packaging practices / repository format"):
>> I'm always cloning the upstream repo, branch off at their release tag
>> and add all necessary chanages as individual git commits - first come
>> the generic (non-deb specific) patches, then the deb specific ones.
>> No text-based patches, or even magic rewriting within the build process.
>> The HEAD is exactly the debianized source tree,
>
> What source format do you use ?  What is in debian/source/format, if
> anything ?

Usually "3.0 (quilt)", but I actually don't really care so much. Just
picked that some time, as it just worked, and never really though about
it anymore :p

> Do you handle orig tarballs at all ?

No. I'm exclusively using docker-buildpackage, which directly operates
on the final source tree - no intermediate steps like unpacking,
patching, etc.

One of the fine things (besides simplicity) is that if anything goes
wrong, I can just jump into the container (it intentionally doesn't
clean up failing containers) and directly work from there (the git
repo is also there).

> When you go to a new upstream, you make a new git branch, then ?

git checkout <prev-maint-branch> -b <new-maint-branch>
git rebase <new-upstream-tag>

And then see it it works, finxing things, etc.
Of course, I'll also care about self-consistent and understandable
commits - git history is documentation, not rotating backup ;-)

> Do you publish this git branch anywhere ?

https://github.com/oss-qm

(from time to time I also send patches upstream)

>> which is then fed to dck-buildpackage.
>
> What is that ?  

https://github.com/metux/docker-buildpackage

It's a little tool that sets up build containers (also creates base
images on-demand), including build tools, extra repos, etc, runs
the build in the container and finally pulls out the debs.

The main audience are folks that maintain extra repos (eg.
customizations, backports, etc) - that's one of the things I'm
regularily doing for my clients.

I've got another toolkit ontop of that, which helps maintaining
whole repos, including managing git repos and their remotes,
dependency handling, etc. It's actually not a standalone tool,
but a fundation for easily setting up your own customized build
environment. I'm using it for all my customers who get apt repos,
but also for backports and depotterization.

(Note: the 'master' branch currently is crappy, more a playgound, w/
lot's of things that have to be cleaned up ... for production use
fork from 'base' branch.)

> manpages.debian.org wasn't any help.

It's not in official Debian. I've announced it long go, but nobody
here really cared. I couldn't even convice debian maintainers for
little less insane scm workflows (just look at the kernel :p), but
failed, so I don't waste my time anymore, instead just clean up the
mess for those packages that I actually need.


--mtx

--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[hidden email] -- +49-151-27565287

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Enrico Weigelt, metux IT consult-3
In reply to this post by Ian Jackson-2
On 28.05.19 22:08, Ian Jackson wrote:

Hi,

> Please can we leave aside discussion of the merits or otherwise of
> each of these formats/workflows.
>
> Perhaps we can talk about that (again!) at some point, but it tends to
> derail any conversation about git packaging stuff and I don't want
> this thread derailed.

I understand you point, but I believe we really should discuss this.
(maybe based on some specific examples)

OTOH, I'll only participate in such discussions, if I see that it's
really going forward ... already tried that several times in recent
years, but no success, so I just gave up :(


--mtx

--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[hidden email] -- +49-151-27565287

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Enrico Weigelt, metux IT consult-3
In reply to this post by Simon McVittie-7
On 29.05.19 01:39, Simon McVittie wrote:

Hi,

> You might reasonably assume that, but no, they are not. mesa (and probably
> other xorg-team packages) uses v1.0 dpkg-source format combined with
> dh --with quilt, so deliberate Debian changes can be either direct
> changes to the upstream source code, or quilt patches in d/patches,
> or a mixture. Additionally, mesa uses d/source/local-options to ignore
> files that only exist in the upstream git tag (which is what gets merged
> into the packaging git branch), but not in the upstream `make dist` output
> produced from that tag (which is used as the .orig tarball).

hmm, sounds quite complicated ... anyone here who could explain why
exactly they're doing it that way ?

by the way: that's IMHO an important information we should also collect:
why exactly some particular workflow was picked

> My understanding is that this unusual difference between the .orig
> tarball and what's in git is an attempt to "square the circle" between
> two colliding design principles: "the .orig tarball should be upstream's
> official binary artifact" (in this case Automake `make dist` output,
> including generated files like Makefile.in but not non-critical source
> files like .gitignore) and "what's in git should match upstream's git
> repository" (including .gitignore but
> not usually Makefile.in).

Since we have git, I've completely given up the orig tarball - I'm just
basing on their release tags. And, of course, there shouldn't be
anything autogenerated in the git repo - always recreate everything
(*especially* autotools-generated stuff). The orig tarball, IMHO, is a
long obsolete ancient relic.

For upstreams that don't have a git repo yet, I setup an importer first,
and call that my upstream.


--mtx

--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[hidden email] -- +49-151-27565287

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Enrico Weigelt, metux IT consult-3
In reply to this post by Simon McVittie-7
On 28.05.19 19:31, Simon McVittie wrote:

Hi,

> Debian Linux kernel
> ===================
>
> Tree contains: an incomplete debian/ directory, notably without d/control,
> and no upstream source
> Changes to upstream source are: d/patches only
> Baseline upstream: changelog version => .orig tarball
> Patches managed by: ???
> Special build tool: there is a pre-build step to generate d/control

I'm handling the kernel very differently (actually the offical packages
never actually built at my site), similar to what I've described in
my other mails - layered branches:

* layer 0: upstream tag (linus or greg)
* layer 1: generic patches for making upstream's 'make dep-pkg' work
           with usual debian workflows (eg. not creating debian/rules
           from there anymore, but a generic one instead)
* layer 2: dist and target specific customizations (changelos, .config,
           etc ...)

The whole thing is again is built via dck-buildpackage (dpkg-
buildpackage should also work, but I never called it manually anymore,
since i've wrote dck-buildpackage).

Note that I don't even try to create some one-fits-all superpackage for
all archs, flavours, etc. - instead I'm using separate layer 2 branches
for that.

(for maintaining lots of kernel configs based on some meta config, I've
got a separate tool)


--mtx

--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[hidden email] -- +49-151-27565287

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

David Bremner
In reply to this post by Ian Jackson-2
Ian Jackson <[hidden email]> writes:


>
>  Main packaging     Delta from upstream     Tools for manipulating
>   git branch         represented as          delta from upstream,
>   contains                                   building .dsc, etc.
>
>  Unmodified         debian/patches             gbp, gbp pq
>   upstream files,    (only)                    quilt / dquilt
>  plus debian/*                                 Manual patch editing
>  incl. d/patches
>
>  Modified           Direct changes             git merge
>   upstream files,    to upstream files          (.dsc: 1.0-with-diff or
>  plus debian/*.                                 single-debian-patch)
>  Maybe d/patches, depending.
>  History has direct merges from upstream.

With modified upstream files in the main branch, plus debian/*, but
usually no d/patches I use a seperate (manually
rebased) branch for patches, and export those at dsc creation time using
a gitpkg hook

With unmodified upstream files in the main branch, plus debian/*, but
usually no d/patches, I use git-debcherry to generate a quilt series at
dsc build time.

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Ian Jackson-2
In reply to this post by Enrico Weigelt, metux IT consult-3
Enrico Weigelt, metux IT consult writes ("Re: Survey: git packaging practices / repository format"):
> hmm, sounds quite complicated ... anyone here who could explain why
> exactly they're doing it that way ?
>
> by the way: that's IMHO an important information we should also collect:
> why exactly some particular workflow was picked

I don't want to get into that in this thread.  It is too close to a
discussion of what is the best way to do things.  I certainly don't
want to challenge people by asking "why".

I want people to feel free to describe things they have seen, or which
they do, without feeling criticised.  "sounds quite complicated"
sounds like a criticism to me.  And I don't want the thread derailed.

I would prefer to focus on "what".

(Also, I feel that I personally have a pretty good handle on the
advantages and disadvantages of various approaches and I can usually
see why people have done things, even things that I think are a bad
idea.  So I don't need help with that.)

Ian.

--
Ian Jackson <[hidden email]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Ian Jackson-2
In reply to this post by Enrico Weigelt, metux IT consult-3
Enrico Weigelt, metux IT consult writes ("Re: Survey: git packaging practices / repository format"):
> I'd call it the 'git-only-workflow' ;-)
...
> It's not in official Debian. I've announced it long go, but nobody
> here really cared. I couldn't even convice debian maintainers for
> little less insane scm workflows (just look at the kernel :p), but
> failed, so I don't waste my time anymore, instead just clean up the
> mess for those packages that I actually need.

Oh.  I think I have misunderstood.  I think you are describing a git
workflow you use as a *downstream* of Debian, not as a maintainer
*within* Debian.

And I think what you are saying is that you don't use source packages
(.dsc) except maybe in the innards somewhere of your machinery.
I think that is a good way for a downstream to work.  Certainly when I
modify anything locally I don't bothere with that .dsc stuff.

But my aim in this thread was to capture how people work *within*
Debian, where a maintainer is still required to produce a .dsc.

(Sorry that maybe I have wasted your time answering my questions.)

Regards,
Ian.

--
Ian Jackson <[hidden email]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Reply | Threaded
Open this post in threaded view
|

Re: Survey: git packaging practices / repository format

Ian Jackson-2
In reply to this post by Enrico Weigelt, metux IT consult-3
Enrico Weigelt, metux IT consult writes ("Re: Survey: git packaging practices / repository format"):

> On 28.05.19 22:08, Ian Jackson wrote:
> > Please can we leave aside discussion of the merits or otherwise of
> > each of these formats/workflows.
> >
> > Perhaps we can talk about that (again!) at some point, but it tends to
> > derail any conversation about git packaging stuff and I don't want
> > this thread derailed.
>
> I understand you point, but I believe we really should discuss this.
> (maybe based on some specific examples)

Not in this thread, please.  There have been many other threads about
these issues.

Ian.

--
Ian Jackson <[hidden email]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

1234 ... 6