git & Debian packaging sprint report

classic Classic list List threaded Threaded
41 messages Options
123
Reply | Threaded
Open this post in threaded view
|

git & Debian packaging sprint report

Sean Whitton
Hello,

Over the weekend, Ian Jackson and I met in Cambridge, U.K. to work on
the design and implementation of tools and processes relating to git &
Debian packaging.

Main achievement
----------------

We designed and implemented a system to make it possible for DDs to
upload new versions of packages by simply pushing a specially formatted
git tag to salsa.

Please see this blog post to learn about how it works:
https://spwhitton.name/blog/entry/tag2upload/

While the cloud service part of this system has not yet been deployed,
and so you can't just tag to upload yet, the blog post explains how you
can run the cloud service in an ad-hoc mode on your laptop, and thereby
get a feel for how it works.

You can also read git-debpush(1) in sid.[1]

Other achivements
-----------------

1. In-depth design discussions for:

   - git-ffqrebase, a tool for Debian users and downstreams to rebase
     delta queues on top of Debian's source code; and

   - better git-merge support for git-debrebase.  Confirmed the
     resolution UX/workflow, so implementation is now unblocked.  Some
     notes are in #931552.

2. Improved the git packaging survey[2] by setting up inner pages,
   deciding on a template for those, and filling some of them in.

3. Made some plans to improve the layout of dgit's documentation.  Thank
   you to Colin Watson for the feedback which prompted this.

4. Triage of bugs filed against src:dgit.

5. Various smaller fixes/improvements to dgit.

6. Wrote up some requirements as input to an upcoming BoF.[3]

7. Produced blog post linked above.

Reimbursement
-------------

There will be one travel reimbursement request, for an amount less than
$100.

Acknowledgements
----------------

Thank you to donators to Debian for travel funding, and the people who
live at Ian Jackson's place (including Ian) for hosting.

[1]  https://manpages.debian.org/git-debpush
[2]  https://wiki.debian.org/GitPackagingSurvey
[3]  https://debconf19.debconf.org/talks/68-one-git-to-package-them-all-and-on-the-salsa-find-them/

--
Sean Whitton

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

gregor herrmann-3
On Wed, 10 Jul 2019 09:10:40 +0100, Sean Whitton wrote:

> Main achievement
> ----------------
>
> We designed and implemented a system to make it possible for DDs to
> upload new versions of packages by simply pushing a specially formatted
> git tag to salsa.
>
> Please see this blog post to learn about how it works:
> https://spwhitton.name/blog/entry/tag2upload/

This sounds very exciting, I'm really looking forward to typing `git
debpush'! Thank you very much for this work.


Cheers,
gregor

--
 .''`.  https://info.comodo.priv.at -- Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
   `-   NP: Beatles

signature.asc (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Antonio Terceiro-3
In reply to this post by Sean Whitton
Hi,

Thanks for the report.

On Wed, Jul 10, 2019 at 09:10:40AM +0100, Sean Whitton wrote:

> Hello,
>
> Over the weekend, Ian Jackson and I met in Cambridge, U.K. to work on
> the design and implementation of tools and processes relating to git &
> Debian packaging.
>
> Main achievement
> ----------------
>
> We designed and implemented a system to make it possible for DDs to
> upload new versions of packages by simply pushing a specially formatted
> git tag to salsa.
>
> Please see this blog post to learn about how it works:
> https://spwhitton.name/blog/entry/tag2upload/
>
> While the cloud service part of this system has not yet been deployed,
> and so you can't just tag to upload yet, the blog post explains how you
> can run the cloud service in an ad-hoc mode on your laptop, and thereby
> get a feel for how it works.
>
> You can also read git-debpush(1) in sid.[1]
If the uploads will be done by a service, this means that all of them
will be signed by a single key and not by the DD key. Is ftp-master ok
with that?

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Andrej Shadura-2
In reply to this post by Sean Whitton
Hi,

On Wed, 10 Jul 2019 at 10:10, Sean Whitton <[hidden email]> wrote:
> Over the weekend, Ian Jackson and I met in Cambridge, U.K. to work on
> the design and implementation of tools and processes relating to git &
> Debian packaging.

Sean, are you coming to Curitiba? If yes, how about organising a BoF
on Git packaging workflows?

--
Cheers,
  Andrej

Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Sam Hartman-3
>>>>> "Andrej" == Andrej Shadura <[hidden email]> writes:

    Andrej> Hi,
    Andrej> On Wed, 10 Jul 2019 at 10:10, Sean Whitton <[hidden email]> wrote:
    >> Over the weekend, Ian Jackson and I met in Cambridge, U.K. to
    >> work on the design and implementation of tools and processes
    >> relating to git & Debian packaging.

    Andrej> Sean, are you coming to Curitiba? If yes, how about
    Andrej> organising a BoF on Git packaging workflows?

He's not, but if you read the entire report you'd see a link to


https://debconf19.debconf.org/talks/68-one-git-to-package-them-all-and-on-the-salsa-find-them/

We'll be starting off early in the week trying to understand and catalog
requirements, and I'm sure the fun won't end there!

Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Scott Kitterman-5
In reply to this post by Sean Whitton


On July 10, 2019 8:10:40 AM UTC, Sean Whitton <[hidden email]> wrote:

>Hello,
>
>Over the weekend, Ian Jackson and I met in Cambridge, U.K. to work on
>the design and implementation of tools and processes relating to git &
>Debian packaging.
>
>Main achievement
>----------------
>
>We designed and implemented a system to make it possible for DDs to
>upload new versions of packages by simply pushing a specially formatted
>git tag to salsa.
>
>Please see this blog post to learn about how it works:
>https://spwhitton.name/blog/entry/tag2upload/
>
>While the cloud service part of this system has not yet been deployed,
>and so you can't just tag to upload yet, the blog post explains how you
>can run the cloud service in an ad-hoc mode on your laptop, and thereby
>get a feel for how it works.
...
Thanks for the detailed explanation.

Has there been any analysis of the security implications of this proposed service?

If I am understanding the description correctly, the transformation from git tag (which is signed and can be verified) to a source package (which can be signed and verified) will happen on an internet facing server (typically this would happen on a local developer machine) and, unless there is additional magic around key management that isn't described in the blog post, the private key for a key the archive trusts would also be there.

It seems to me that there is potential for a significant new attack surface that ought to be carefully assessed before this gets anywhere near wired up to feed into the archive from any kind of 'cloud' service.

Scott K

Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Sean Whitton
Hello Scott,

On Fri 12 Jul 2019 at 04:30am +00, Scott Kitterman wrote:

> Has there been any analysis of the security implications of this
> proposed service?

Nothing formal, though of course we were thinking about it while we were
working on it.

> If I am understanding the description correctly, the transformation
> from git tag (which is signed and can be verified) to a source package
> (which can be signed and verified) will happen on an internet facing
> server (typically this would happen on a local developer machine) and,
> unless there is additional magic around key management that isn't
> described in the blog post, the private key for a key the archive
> trusts would also be there.
>
> It seems to me that there is potential for a significant new attack
> surface that ought to be carefully assessed before this gets anywhere
> near wired up to feed into the archive from any kind of 'cloud'
> service.
The current plan is for this machine to be firewalled such that it talks
only to salsa.  For exactly the sort of reasons you describe, you won't
be able to use this with arbitrary git hosts.

The only untrusted input is the git tags before their signature has been
verified against the Debian keyring.  Maybe we could isolate fetching
and checking those tags from the part of the service which fetches the
whole git tree to produce a source package.

--
Sean Whitton

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Ansgar Burchardt-5
In reply to this post by Antonio Terceiro-3
On Thu, 2019-07-11 at 17:58 -0300, Antonio Terceiro wrote:
> > We designed and implemented a system to make it possible for DDs to
> > upload new versions of packages by simply pushing a specially
> > formatted git tag to salsa.
[...]
> If the uploads will be done by a service, this means that all of them
> will be signed by a single key and not by the DD key. Is ftp-master
> ok with that?

Depends on a lot of things.  As far as I understand this work is in a
very early stage and a first brainstorming session on what problem this
is intended to solve, why one should consider doing it, and possible
requirements is planned for DebConf[1].

Without having any of these, it is hard to comment on anything.

Ansgar

  [1] https://debconf19.debconf.org/talks/68-one-git-to-package-them-all-and-on-the-salsa-find-them/

Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Sean Whitton
Hello,

On Fri 12 Jul 2019 at 02:06PM +02, Ansgar Burchardt wrote:

> Depends on a lot of things.  As far as I understand this work is in a
> very early stage and a first brainstorming session on what problem this
> is intended to solve, why one should consider doing it, and possible
> requirements is planned for DebConf[1].
>
> Without having any of these, it is hard to comment on anything.

Figuring out how this is going to be deployed as Debian infrastructure
is indeed at an early stage.

I don't think it is accurate to think of the whole project as being at
an early stage.  From my perspective, the hardest problem to solve was
conversion of git trees to uploads, on the server side, in a way that is
user friendly.  We take ourselves to have solved that problem, which to
my mind renders this project beyond the early stages of development.

--
Sean Whitton

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Ansgar Burchardt-8
Sean Whitton writes:

> On Fri 12 Jul 2019 at 02:06PM +02, Ansgar wrote:
>> Depends on a lot of things.  As far as I understand this work is in a
>> very early stage and a first brainstorming session on what problem this
>> is intended to solve, why one should consider doing it, and possible
>> requirements is planned for DebConf[1].
>>
>> Without having any of these, it is hard to comment on anything.
>
> Figuring out how this is going to be deployed as Debian infrastructure
> is indeed at an early stage.
>
> I don't think it is accurate to think of the whole project as being at
> an early stage.  From my perspective, the hardest problem to solve was
> conversion of git trees to uploads, on the server side, in a way that is
> user friendly.  We take ourselves to have solved that problem, which to
> my mind renders this project beyond the early stages of development.

What is the DebConf BOF then for?  It says it is "a requirements
collecting BOF for that [git push] discussion"?

Ansgar

Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Sean Whitton
Hello,

On Sun 14 Jul 2019 at 09:48AM +02, Ansgar wrote:

> What is the DebConf BOF then for?  It says it is "a requirements
> collecting BOF for that [git push] discussion"?

To my mind, the BoF and the work at the sprint are two efforts to
improve things that are running in parallel.  I think we can expect
the them to enhance each other.

If the BoF or e-mail discussion comes up with requirements which are
impossible for our solution to satisfy, then we misjudged what to spend
our time on at the sprint.  Ian and I have been working on this stuff
for long enough that we felt able to make a judgement about what would
be useful, however.

--
Sean Whitton

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Sam Hartman-3
>>>>> "Sean" == Sean Whitton <[hidden email]> writes:

    Sean> If the BoF or e-mail discussion comes up with requirements
    Sean> which are impossible for our solution to satisfy, then we
    Sean> misjudged what to spend our time on at the sprint.  Ian and I
    Sean> have been working on this stuff for long enough that we felt
    Sean> able to make a judgement about what would be useful, however.

The BOF will almost certainly come up with requirements that are
impossible for any solution to meet.
THe BOF is not judging requirements, only collecting them.

People proposing solutions can use the BOF input where it is helpful.
We as a project can use that input in guiding discussions too.  However
just because something is proposed as a requirement at the BOF doesn't
mean we will be able to (or will choose to) meet that requirement.

Also, the BOF was proposed by me in the hopes of getting things moving.
We've had a lot of activity since I submitted that original BOF
proposal.  If Sean and Ian think they already have enough information to
put something together, I think that's great.  My BOF proposal was
intended to enable work, not to slow it down.

--Sam

Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Michael Kesper-5
In reply to this post by Sean Whitton
Hi Sean, hi all,

On 12.07.19 09:00, Sean Whitton wrote:

> On Fri 12 Jul 2019 at 04:30am +00, Scott Kitterman wrote:
>
>> Has there been any analysis of the security implications of this
>> proposed service?
>
> Nothing formal, though of course we were thinking about it while we were
> working on it.
>
>> If I am understanding the description correctly, the transformation
>> from git tag (which is signed and can be verified) to a source package
>> (which can be signed and verified) will happen on an internet facing
>> server (typically this would happen on a local developer machine) and,
>> unless there is additional magic around key management that isn't
>> described in the blog post, the private key for a key the archive
>> trusts would also be there.
>>
>> It seems to me that there is potential for a significant new attack
>> surface that ought to be carefully assessed before this gets anywhere
>> near wired up to feed into the archive from any kind of 'cloud'
>> service.
>
> The current plan is for this machine to be firewalled such that it talks
> only to salsa.  For exactly the sort of reasons you describe, you won't
> be able to use this with arbitrary git hosts.
>
> The only untrusted input is the git tags before their signature has been
> verified against the Debian keyring.  Maybe we could isolate fetching
> and checking those tags from the part of the service which fetches the
> whole git tree to produce a source package.
Nonetheless it seems to me you are moving from trusting local signing
to trusting upload by salsa, thereby making salsa more attractive for
attackers.

Best wishes
Michael
 



signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Sean Whitton
Hello Michael,

On Mon 15 Jul 2019 at 01:16PM +02, Michael Kesper wrote:

> Nonetheless it seems to me you are moving from trusting local signing
> to trusting upload by salsa, thereby making salsa more attractive for
> attackers.

I don't follow -- the git tag is PGP-signed, locally, by the uploader.
Just like how they would PGP-sign, locally, the .dsc and .changes.

--
Sean Whitton

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Russ Allbery-2
In reply to this post by Sean Whitton
Sean Whitton <[hidden email]> writes:

> The current plan is for this machine to be firewalled such that it talks
> only to salsa.  For exactly the sort of reasons you describe, you won't
> be able to use this with arbitrary git hosts.

> The only untrusted input is the git tags before their signature has been
> verified against the Debian keyring.  Maybe we could isolate fetching
> and checking those tags from the part of the service which fetches the
> whole git tree to produce a source package.

Just to make sure I fully understand the model, is the idea that this
system will verify the signature on the Git tag, construct a source
package from the signed archive, and then sign the resulting source
package with some internal key?

If so, I think that security model is roughly equivalent to the automatic
signing of binary packages by buildds, so probably doesn't introduce a new
vulnerability, but my understanding was that the identity of the signature
on the source package was used in various other places.  Presumably we
would need to introduce some new metadata so that the uploader is mapped
properly to the Git tag signer, rather than to some internal identity of
the source package construction service.

Also, doesn't the archive publish the signed *.dsc files currently?  I
believe this would mean that we would lose some published information from
those files that we currently have (namely which DD and which key signed
the package, which could be useful data in some incident response
scenarios).  That said, there's been some discussion for some time about
having the archive sign all the *.dsc files instead of keeping the
uploader signature, which may be from an expired or unverifiable key
(particularly for packages that haven't been uploaded in some time).

There are also some interesting nuances here around handling DM packages,
where not everyone with a key in the keyring can upload every package,
although the obvious way to address that is probably for this service to
do the same DM checks that ftpmaster would normally do.

--
Russ Allbery ([hidden email])               <http://www.eyrie.org/~eagle/>

Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Sean Whitton
Hello,

On Mon 15 Jul 2019 at 10:22AM -07, Russ Allbery wrote:

> Just to make sure I fully understand the model, is the idea that this
> system will verify the signature on the Git tag, construct a source
> package from the signed archive, and then sign the resulting source
> package with some internal key?

Assuming that by 'archive' you mean the committish at which the
DD-signed tag points, then yes, that's the model.

> If so, I think that security model is roughly equivalent to the automatic
> signing of binary packages by buildds, so probably doesn't introduce a new
> vulnerability, but my understanding was that the identity of the signature
> on the source package was used in various other places.  Presumably we
> would need to introduce some new metadata so that the uploader is mapped
> properly to the Git tag signer, rather than to some internal identity of
> the source package construction service.

Right.  This might be needed.

> Also, doesn't the archive publish the signed *.dsc files currently?  I
> believe this would mean that we would lose some published information from
> those files that we currently have (namely which DD and which key signed
> the package, which could be useful data in some incident response
> scenarios).  That said, there's been some discussion for some time about
> having the archive sign all the *.dsc files instead of keeping the
> uploader signature, which may be from an expired or unverifiable key
> (particularly for packages that haven't been uploaded in some time).

As you say, DD signatures on .dscs are only reliably useful in sid,
where they're quite likely to be verifiable.

So perhaps having them signed by this bot would be an improvement for
some usecases.

> There are also some interesting nuances here around handling DM packages,
> where not everyone with a key in the keyring can upload every package,
> although the obvious way to address that is probably for this service to
> do the same DM checks that ftpmaster would normally do.

Right.  dgit-infrastructure already has code to do that.  We just
haven't hooked it up yet.

--
Sean Whitton

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Ansgar Burchardt-8
In reply to this post by Russ Allbery-2
Russ Allbery writes:
> If so, I think that security model is roughly equivalent to the automatic
> signing of binary packages by buildds, so probably doesn't introduce a new
> vulnerability,

It doesn't rely on strong cryptographic hashes to guarantee integrity.
To quote Wikipedia:

+---
| Revision control systems such as Git, Mercurial, and Monotone use
| SHA-1 not for security but to identify revisions and to ensure that
| the data has not changed due to accidental corruption.
+---[ https://en.wikipedia.org/wiki/SHA-1#Data_integrity ]

But developers could instead just sign artifacts using a strong
cryptographic hash that will be included in the source package; for
example the .orig.tar and .debian.tar which can be made reproducible
(git-archive is supposed to be reproducible; compression might not be so
just sign the uncompressed version).

We shouldn't go back to trusting SHA-1.

> There are also some interesting nuances here around handling DM packages,
> where not everyone with a key in the keyring can upload every package,
> although the obvious way to address that is probably for this service to
> do the same DM checks that ftpmaster would normally do.

We have other permissions checks as well; they shouldn't be
reimplemented in different places.  Instead the archive (dak) should
know who signed the package.

Ansgar

Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Russ Allbery-2
Ansgar Burchardt <[hidden email]> writes:
> Russ Allbery writes:

>> If so, I think that security model is roughly equivalent to the
>> automatic signing of binary packages by buildds, so probably doesn't
>> introduce a new vulnerability,

> It doesn't rely on strong cryptographic hashes to guarantee integrity.
> To quote Wikipedia:

> +---
> | Revision control systems such as Git, Mercurial, and Monotone use
> | SHA-1 not for security but to identify revisions and to ensure that
> | the data has not changed due to accidental corruption.
> +---[ https://en.wikipedia.org/wiki/SHA-1#Data_integrity ]

> But developers could instead just sign artifacts using a strong
> cryptographic hash that will be included in the source package; for
> example the .orig.tar and .debian.tar which can be made reproducible
> (git-archive is supposed to be reproducible; compression might not be so
> just sign the uncompressed version).

> We shouldn't go back to trusting SHA-1.

I'm dubious that we really care that much about a preimage attack on
SHA-1, but sure, if there's some easy way to use something different, that
would be more future-proof.

--
Russ Allbery ([hidden email])               <http://www.eyrie.org/~eagle/>

Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Ansgar Burchardt-8
Russ Allbery writes:

> Ansgar Burchardt writes:
>> Russ Allbery writes:
>>> If so, I think that security model is roughly equivalent to the
>>> automatic signing of binary packages by buildds, so probably doesn't
>>> introduce a new vulnerability,
>
>> It doesn't rely on strong cryptographic hashes to guarantee integrity.
>> To quote Wikipedia:
>
>> +---
>> | Revision control systems such as Git, Mercurial, and Monotone use
>> | SHA-1 not for security but to identify revisions and to ensure that
>> | the data has not changed due to accidental corruption.
>> +---[ https://en.wikipedia.org/wiki/SHA-1#Data_integrity ]
>
>> But developers could instead just sign artifacts using a strong
>> cryptographic hash that will be included in the source package; for
>> example the .orig.tar and .debian.tar which can be made reproducible
>> (git-archive is supposed to be reproducible; compression might not be so
>> just sign the uncompressed version).
>
>> We shouldn't go back to trusting SHA-1.
>
> I'm dubious that we really care that much about a preimage attack on
> SHA-1, but sure, if there's some easy way to use something different, that
> would be more future-proof.

SHA-1 isn't going to get stronger in the future.  The TLS world has
already moved on, OpenPGP is still in the slow process to move on,
Release/Packages stopped using it[1], there is no reason to continue
using it.

There is another advantage to developers signing the artifacts: one
would need much less trust in the service building the source files as
it could only manipulate the .dsc, .changes and compression used.

It also has one downside: `git tag` alone won't be enough to generate
the required information, but then a special-purpose tool was proposed
here already.

The client tool could possibly also just create the .dsc and .changes,
except for hashes of the compressed files, and the web service just
recreate the tarball and compress them.  That would require near zero
trust in the web service, but still allow developers to no longer upload
source packages which might be large.  (A bit similar to not having to
trust buildds by making packages reproducible.)

Ansgar

  [1] Though we still have MD5sum for some suites for jigdo...

Reply | Threaded
Open this post in threaded view
|

Re: git & Debian packaging sprint report

Russ Allbery-2
Ansgar Burchardt <[hidden email]> writes:

> SHA-1 isn't going to get stronger in the future.  The TLS world has
> already moved on, OpenPGP is still in the slow process to move on,
> Release/Packages stopped using it[1], there is no reason to continue
> using it.

Well, the reason to continue using it is that Git uses it and we use Git,
and it may simplify the workflow.

You're not wrong, of course, but preimage attacks are very hard.  MD5 is
still probably robust against preimage attacks, let alone SHA-1.  By all
means, let's future-proof as much as possible, but I'm not sure worrying
about SHA-1 preimage resistance is the most important design principle in
this case.  At some point, Git itself will switch away from SHA-1, and we
can then obviously follow.

That said, there's enough custom logic going on here that it may be easy
to add something that you describe, in which case, great.

> The client tool could possibly also just create the .dsc and .changes,
> except for hashes of the compressed files, and the web service just
> recreate the tarball and compress them.

I think experience with pristine-tar indicates that recreating tarballs is
unfortunately not trivial.

--
Russ Allbery ([hidden email])               <http://www.eyrie.org/~eagle/>

123