https://tracker.debian.org/pkg/dballe

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

https://tracker.debian.org/pkg/dballe

Enrico Zini
Hello,

some time ago I uploaded a new version of dballe, which went through NEW
because of a change in binary package names (SONAME bump, IIRC). It took
two weeks to go through NEW and I turned my energy towards other things
since then.

Now I see that things got stuck for the transition to testing, and I'm
asking for help figuring out what to do.

I have this:

> This package is part of the ongoing testing transition known as
> python3.8. Please avoid uploads unrelated to this transition, they
> would likely delay it and require supplementary work from the release
> managers. On the other hand, if your package has problems preventing
> it to migrate to testing, please fix them as soon as possible. You can
> probably find supplementary information in the debian-release archives
> or in the corresponding release.debian.org bug.

And I have this:

> Not built on buildd: arch all binaries uploaded by enrico, a new
> source-only upload is needed to allow migration

I was asked to do a binary upload to go through NEW, and now I'm asked
to do a source only upload to go to testing. There are surely good
reasons for that, and I wish there weren't.

I don't really have a new version to upload, but I suppose I need to at
least bump the debian version with no other changes in the package for
it to be accepted.

Then, if I did that, would I be helping the python3.8 transition by
enabling a migration to testing, or getting in the way by delaying the
transition and requiring supplementary work from the release managers?

I feel a bit caught in a bureaucratic corner. Please help me with some
directions to get out of that.


Enrico

--
GPG key: 4096R/634F4BD1E7AD5568 2009-05-08 Enrico Zini <[hidden email]>

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Paul Gevers-4
Hi Enrico,

On 29-12-2019 09:52, Enrico Zini wrote:
> some time ago I uploaded a new version of dballe, which went through NEW
> because of a change in binary package names (SONAME bump, IIRC). It took
> two weeks to go through NEW and I turned my energy towards other things
> since then.
>
> Now I see that things got stuck for the transition to testing, and I'm
> asking for help figuring out what to do.

I'm happy to help.

> I have this:
>
>> This package is part of the ongoing testing transition known as
>> python3.8. Please avoid uploads unrelated to this transition, they
>> would likely delay it and require supplementary work from the release
>> managers. On the other hand, if your package has problems preventing
>> it to migrate to testing, please fix them as soon as possible. You can
>> probably find supplementary information in the debian-release archives
>> or in the corresponding release.debian.org bug.

The python3.8 transition is not a classical transition, so this normally
helpful comment from tracker.d.o doesn't really apply. Please ignore it.
If you would follow the link to the transition page, you would find a
link to a transition bug (#942106) and it will explain (already in the
title) that it is different from most transitions. (While writing this
response I realize it may be possible for me to "hide" this info on the
tracker, but I haven't done that before).

> And I have this:
>
>> Not built on buildd: arch all binaries uploaded by enrico, a new
>> source-only upload is needed to allow migration
>
> I was asked to do a binary upload to go through NEW, and now I'm asked
> to do a source only upload to go to testing. There are surely good
> reasons for that, and I wish there weren't.

I agree with this. And from your phrasing your not really interested in
the explanation, so find that at the bottom [1].

> I don't really have a new version to upload, but I suppose I need to at
> least bump the debian version with no other changes in the package for
> it to be accepted.

Unfortunately, indeed.

> Then, if I did that, would I be helping the python3.8 transition by
> enabling a migration to testing, or getting in the way by delaying the
> transition and requiring supplementary work from the release managers?

As explained above, you'll not be interfering, so go ahead.

> I feel a bit caught in a bureaucratic corner. Please help me with some
> directions to get out of that.

Hope this helps.

Paul

[1] Source packages that build binaries unknown to the archive currently
need these binaries to be uploaded by the maintainers for reviewing by
ftp-master in NEW. IIRC there have been multiple proposals to avoid
these binaries from either being needed or being uploaded to the Debian
archive, but so far the current tooling requires this. On the other
hand, the release team has decided that we want all binaries in our
releases to be build on buildd. For that we have enhanced our migration
software to check for non-buildd uploads and add a block for them.
Unfortunately, once uploaded we can't fix the situation for arch:all
packages as binNMU'ing arch:all is currently broken. There is slow
progress to improve that situation, see e.g. bug #916601.


signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Roberto C. Sánchez
On Sun, Dec 29, 2019 at 12:32:24PM +0100, Paul Gevers wrote:

>
> [1] Source packages that build binaries unknown to the archive currently
> need these binaries to be uploaded by the maintainers for reviewing by
> ftp-master in NEW. IIRC there have been multiple proposals to avoid
> these binaries from either being needed or being uploaded to the Debian
> archive, but so far the current tooling requires this. On the other
> hand, the release team has decided that we want all binaries in our
> releases to be build on buildd. For that we have enhanced our migration
> software to check for non-buildd uploads and add a block for them.
> Unfortunately, once uploaded we can't fix the situation for arch:all
> packages as binNMU'ing arch:all is currently broken. There is slow
> progress to improve that situation, see e.g. bug #916601.
>

I've encountered this issue myself in the past with a SONAME bump in a
library package.  It was rather surprising.

Would it not be possible to eliminate the need for the second
unnecessary upload by requiring two signed .changes files to go into
NEW?  A signed binary changes which would form the basis of the FTP
master review and a signed source changes to enter the archive if the
package is approved?

When I build packages I always end up with both changes files, so
requiring both for NEW processing would be a triviality (for me, at
least) and eliminate this peculiarity as well.

Regards,

-Roberto

--
Roberto C. Sánchez

Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Theodore Y. Ts'o
In reply to this post by Enrico Zini
On Sun, Dec 29, 2019 at 09:52:44AM +0100, Enrico Zini wrote:
> Hello,
>
> some time ago I uploaded a new version of dballe, which went through NEW
> because of a change in binary package names (SONAME bump, IIRC). It took
> two weeks to go through NEW and I turned my energy towards other things
> since then.

Wow, two weeks?  I uploaded a new version of f2fs-tools back in July,
with the same issue (SONAME bump), and it's still not gotten through
NEW.

I had assumed everyone was waiting 5-6+ months to get through NEW....

                      - Ted

Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Enrico Zini
In reply to this post by Paul Gevers-4
On Sun, Dec 29, 2019 at 12:32:24PM +0100, Paul Gevers wrote:

> > Now I see that things got stuck for the transition to testing, and I'm
> > asking for help figuring out what to do.
>
> I'm happy to help.

Thanks! <3

> The python3.8 transition is not a classical transition, so this normally
> helpful comment from tracker.d.o doesn't really apply. Please ignore it.

Ack, wonderful.

> As explained above, you'll not be interfering, so go ahead.

Perfect, I have gone ahead with a new source-only upload.


Enrico

--
GPG key: 4096R/634F4BD1E7AD5568 2009-05-08 Enrico Zini <[hidden email]>

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

NEW processing time (Was: https://tracker.debian.org/pkg/dballe)

Sebastiaan Couwenberg
In reply to this post by Theodore Y. Ts'o
On 12/29/19 3:53 PM, Theodore Y. Ts'o wrote:

> On Sun, Dec 29, 2019 at 09:52:44AM +0100, Enrico Zini wrote:
>> some time ago I uploaded a new version of dballe, which went through NEW
>> because of a change in binary package names (SONAME bump, IIRC). It took
>> two weeks to go through NEW and I turned my energy towards other things
>> since then.
>
> Wow, two weeks?  I uploaded a new version of f2fs-tools back in July,
> with the same issue (SONAME bump), and it's still not gotten through
> NEW.
>
> I had assumed everyone was waiting 5-6+ months to get through NEW....

It seems to differ depending on the package. Every month the qgis
package has to go through NEW due to SONAME bumps as well, and this is
usually processed within a week or two. Possibly because the changes
since the last time it was in NEW is limited.

In March netcdf will be in NEW for a year, a number of subsequent
upstream releases have accumulated there. I suspect this may be another
case where the package got stuck an needs an intervention to get it
acceptable again.

Asking for explicit review of a package via email or IRC tends to work
reasonably well.

Kind Regards,

Bas

--
 GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1

Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Paul Wise via nm
In reply to this post by Paul Gevers-4
On Sun, Dec 29, 2019 at 11:32 AM Paul Gevers wrote:

> [1] Source packages that build binaries unknown to the archive currently
> need these binaries to be uploaded by the maintainers for reviewing by
> ftp-master in NEW. IIRC there have been multiple proposals to avoid
> these binaries from either being needed or being uploaded to the Debian
> archive, but so far the current tooling requires this.

There has been some recent work on this part of the problem, I
focussed on throwing away binaries for all sourceful uploads and ivodd
focussed on doing that for NEW uploads only. Since ivodd's patch is
further along than mine, I hope he will extend it to all sourceful
uploads.

https://lists.debian.org/msgid-search/5ec2e979cd7d7ec9bf386fbf77e3399c7aeeb473.camel@...
https://salsa.debian.org/pabs/dak/commits/discard-binaries
https://lists.debian.org/msgid-search/87sgm2lhwc.fsf@...
https://lists.debian.org/msgid-search/de353950121612a86ca706cbdf36153b25b9fa36.camel@...
https://lists.debian.org/msgid-search/27641434-e45a-404f-254f-daea899164a8@...
https://salsa.debian.org/ivodd/dak/tree/new-throwaway-binary-v10

--
bye,
pabs

https://wiki.debian.org/PaulWise

Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Paul Wise via nm
In reply to this post by Roberto C. Sánchez
On Sun, Dec 29, 2019 at 1:29 PM Roberto C. Sánchez wrote:

> Would it not be possible to eliminate the need for the second
> unnecessary upload by requiring two signed .changes files to go into
> NEW?  A signed binary changes which would form the basis of the FTP
> master review and a signed source changes to enter the archive if the
> package is approved?

Another approach is to simply have dak discard binaries from all
sourceful uploads (in dak parlance that means .changes files that have
a .dsc) (and save them to an audit directory). The buildds currently
only produce non-sourceful uploads so all their binaries get accepted
fine. For bootstrap scenarios, maintainers can just do an additional
binary-only upload. See the patches myself/ivodd posted recently for a
work in progress on this.

--
bye,
pabs

https://wiki.debian.org/PaulWise

Reply | Threaded
Open this post in threaded view
|

Re: NEW processing time (Was: https://tracker.debian.org/pkg/dballe)

Theodore Y. Ts'o
In reply to this post by Sebastiaan Couwenberg
On Sun, Dec 29, 2019 at 10:51:36PM +0100, Sebastiaan Couwenberg wrote:

> > Wow, two weeks?  I uploaded a new version of f2fs-tools back in July,
> > with the same issue (SONAME bump), and it's still not gotten through
> > NEW.
> >
> > I had assumed everyone was waiting 5-6+ months to get through NEW....
>
> It seems to differ depending on the package. Every month the qgis
> package has to go through NEW due to SONAME bumps as well, and this is
> usually processed within a week or two. Possibly because the changes
> since the last time it was in NEW is limited.
>
> In March netcdf will be in NEW for a year, a number of subsequent
> upstream releases have accumulated there. I suspect this may be another
> case where the package got stuck an needs an intervention to get it
> acceptable again.
>
> Asking for explicit review of a package via email or IRC tends to work
> reasonably well.

Thanks, I'll try that.  I hadn't sent any e-mail pings because I
didn't to nag what appeared to be a massively overloaded ftp-master
team....

                                                - Ted

Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Kurt Roeckx
In reply to this post by Paul Wise via nm
On Mon, Dec 30, 2019 at 02:52:54AM +0000, Paul Wise wrote:

> On Sun, Dec 29, 2019 at 1:29 PM Roberto C. Sánchez wrote:
>
> > Would it not be possible to eliminate the need for the second
> > unnecessary upload by requiring two signed .changes files to go into
> > NEW?  A signed binary changes which would form the basis of the FTP
> > master review and a signed source changes to enter the archive if the
> > package is approved?
>
> Another approach is to simply have dak discard binaries from all
> sourceful uploads (in dak parlance that means .changes files that have
> a .dsc) (and save them to an audit directory). The buildds currently
> only produce non-sourceful uploads so all their binaries get accepted
> fine. For bootstrap scenarios, maintainers can just do an additional
> binary-only upload. See the patches myself/ivodd posted recently for a
> work in progress on this.

Is it .deb and .changes file that you would move?

Note that the name of the .changes file by the maintainer and the
buildd will be the same, and dak will reject it if that .changes
file already exists.


Kurt

Reply | Threaded
Open this post in threaded view
|

Re: NEW processing time (Was: https://tracker.debian.org/pkg/dballe)

Thomas Goirand-3
In reply to this post by Sebastiaan Couwenberg
On 12/29/19 10:51 PM, Sebastiaan Couwenberg wrote:

> On 12/29/19 3:53 PM, Theodore Y. Ts'o wrote:
>> On Sun, Dec 29, 2019 at 09:52:44AM +0100, Enrico Zini wrote:
>>> some time ago I uploaded a new version of dballe, which went through NEW
>>> because of a change in binary package names (SONAME bump, IIRC). It took
>>> two weeks to go through NEW and I turned my energy towards other things
>>> since then.
>>
>> Wow, two weeks?  I uploaded a new version of f2fs-tools back in July,
>> with the same issue (SONAME bump), and it's still not gotten through
>> NEW.
>>
>> I had assumed everyone was waiting 5-6+ months to get through NEW....
>
> It seems to differ depending on the package. Every month the qgis
> package has to go through NEW due to SONAME bumps as well, and this is
> usually processed within a week or two. Possibly because the changes
> since the last time it was in NEW is limited.
>
> In March netcdf will be in NEW for a year, a number of subsequent
> upstream releases have accumulated there. I suspect this may be another
> case where the package got stuck an needs an intervention to get it
> acceptable again.
>
> Asking for explicit review of a package via email or IRC tends to work
> reasonably well.

+1

Especially when explaining to the FTP masters that the approval is
needed to fix an ongoing Python2 removal process (which was the case
with Ceph recently for example).

Cheers,

Thomas Goirand (zigo)

Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Mattia Rizzolo-5
In reply to this post by Kurt Roeckx
On Mon, Dec 30, 2019 at 11:29:52AM +0100, Kurt Roeckx wrote:
> Note that the name of the .changes file by the maintainer and the
> buildd will be the same, and dak will reject it if that .changes
> file already exists.

That's true only in case of policy queues nowadays.

--
regards,
                        Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540      .''`.
More about me:  https://mapreri.org                             : :'  :
Launchpad user: https://launchpad.net/~mapreri                  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Paul Wise via nm
In reply to this post by Kurt Roeckx
On Mon, 2019-12-30 at 11:29 +0100, Kurt Roeckx wrote:

> Is it .deb and .changes file that you would move?

That would be the idea yeah.

> Note that the name of the .changes file by the maintainer and the
> buildd will be the same, and dak will reject it if that .changes
> file already exists.

I expect that could be dealt with easily in the dak patches, for e.g.
by renaming the .changes to include the fingerprint of the signing key
or by naming the morgue directory after the signing key fingerprint.

--
bye,
pabs

https://wiki.debian.org/PaulWise

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Kurt Roeckx
In reply to this post by Mattia Rizzolo-5
On Mon, Dec 30, 2019 at 01:39:14PM +0100, Mattia Rizzolo wrote:
> On Mon, Dec 30, 2019 at 11:29:52AM +0100, Kurt Roeckx wrote:
> > Note that the name of the .changes file by the maintainer and the
> > buildd will be the same, and dak will reject it if that .changes
> > file already exists.
>
> That's true only in case of policy queues nowadays.

What is a policy queue?

All the recent rejects I get seems to be stable/security uploads.


Kurt

Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Wouter Verhelst
In reply to this post by Paul Wise via nm
On Mon, Dec 30, 2019 at 02:47:45AM +0000, Paul Wise wrote:

> On Sun, Dec 29, 2019 at 11:32 AM Paul Gevers wrote:
>
> > [1] Source packages that build binaries unknown to the archive currently
> > need these binaries to be uploaded by the maintainers for reviewing by
> > ftp-master in NEW. IIRC there have been multiple proposals to avoid
> > these binaries from either being needed or being uploaded to the Debian
> > archive, but so far the current tooling requires this.
>
> There has been some recent work on this part of the problem, I
> focussed on throwing away binaries for all sourceful uploads and ivodd
> focussed on doing that for NEW uploads only. Since ivodd's patch is
> further along than mine, I hope he will extend it to all sourceful
> uploads.

You'll make it unnecessarily harder to bootstrap environments that need
themselves to build if you do that.

Throwing it away for NEW makes sense, but not for regular uploads, IMO.

--
To the thief who stole my anti-depressants: I hope you're happy

  -- seen somewhere on the Internet on a photo of a billboard

Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Paul Wise via nm
On Mon, Jan 13, 2020 at 4:11 PM Wouter Verhelst wrote:

> You'll make it unnecessarily harder to bootstrap environments that need
> themselves to build if you do that.

The idea here is that bootstrap builds are special and so they should
be very explicit rather than happen as a side effect of regular upload
workflows. Since they are relatively rare, making them explicit via
this mechanism shouldn't be too much of an imposition and on balance
might be the right way to go.

--
bye,
pabs

https://wiki.debian.org/PaulWise

Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Sam Hartman-3
>>>>> "Paul" == Paul Wise <[hidden email]> writes:

    Paul> On Mon, Jan 13, 2020 at 4:11 PM Wouter Verhelst wrote:
    >> You'll make it unnecessarily harder to bootstrap environments that need
    >> themselves to build if you do that.

    Paul> The idea here is that bootstrap builds are special and so they should
    Paul> be very explicit rather than happen as a side effect of regular upload
    Paul> workflows. Since they are relatively rare, making them explicit via
    Paul> this mechanism shouldn't be too much of an imposition and on balance
    Paul> might be the right way to go.

Bootstrap uploads of compilers etc are actually more common than I
thought before I started following debian-release.
They are common enough that requiring interactions from ftpmaster and/or
release team would make being a developer of such packages suck
significantly more.

If we throw away binaries by default, I do believe we need a mechanism
for maintainers to signal that this is a bootstrap upload.

--Sam

Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Johannes Schauer-3
Quoting Sam Hartman (2020-01-17 09:45:43)

> >>>>> "Paul" == Paul Wise <[hidden email]> writes:
>
>     Paul> On Mon, Jan 13, 2020 at 4:11 PM Wouter Verhelst wrote:
>     >> You'll make it unnecessarily harder to bootstrap environments that need
>     >> themselves to build if you do that.
>
>     Paul> The idea here is that bootstrap builds are special and so they should
>     Paul> be very explicit rather than happen as a side effect of regular upload
>     Paul> workflows. Since they are relatively rare, making them explicit via
>     Paul> this mechanism shouldn't be too much of an imposition and on balance
>     Paul> might be the right way to go.
>
> Bootstrap uploads of compilers etc are actually more common than I
> thought before I started following debian-release.
> They are common enough that requiring interactions from ftpmaster and/or
> release team would make being a developer of such packages suck
> significantly more.
>
> If we throw away binaries by default, I do believe we need a mechanism for
> maintainers to signal that this is a bootstrap upload.
or have a mechanism that allows maintainers to tell buildds "please build this
source package with build profiles X and Y enabled". That would then build the
binary packages necessary to do a full build in a second upload.

In an even better world, dak would figure out itself, that a source package
first has to be built with build profiles X and Y and then schedule a full
build afterwards.

Thanks!

cheers, josch

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Sam Hartman-3
>>>>> "Johannes" == Johannes Schauer <[hidden email]> writes:

    Johannes> or have a mechanism that allows maintainers to tell buildds "please build this
    Johannes> source package with build profiles X and Y enabled". That would then build the
    Johannes> binary packages necessary to do a full build in a second upload.

That doesn't help.

If I need version y+3 of rustcc to build y+5 of rustcc and only y is in
the archive, no build profile is going to help me.

There are a lot of packages that need themselves to build themselves in
Debian.
It's not just a new arch issue.

Reply | Threaded
Open this post in threaded view
|

Re: https://tracker.debian.org/pkg/dballe

Richard Laager
On 1/17/20 6:55 AM, Sam Hartman wrote:

>>>>>> "Johannes" == Johannes Schauer <[hidden email]> writes:
>
>     Johannes> or have a mechanism that allows maintainers to tell buildds "please build this
>     Johannes> source package with build profiles X and Y enabled". That would then build the
>     Johannes> binary packages necessary to do a full build in a second upload.
>
> That doesn't help.
>
> If I need version y+3 of rustcc to build y+5 of rustcc and only y is in
> the archive, no build profile is going to help me.
If I'm following correctly: The packager would use rustcc >= y+3 (in
practice, likely rustcc y+5) to locally build rustcc y+5 and then do a
binary upload. But if dak (or whatever, I'm not so familiar with the
server side here) throws away the binary, then we're sunk. So there
needs to be a way to note that the binary needs to be kept.

Assuming I'm on the right track, here are a couple of questions:

In such a case, would the source package have a Build-Depends of >= y+3?
It seems like it would.

Could that be the way to indicate a bootstrap upload? In other words, if
the package has a Build-Depends on one of the binary packages it
produces that is _not_ satisfied in the suite it's being uploaded to but
is satisfied by this upload, then it's a bootstrap upload, and the
binaries should be kept. This would avoiding adding a new field for this
and would ensure the Build-Depends are set correctly in bootstrapping
scenarios.

Regardless of how the "keep the binaries" flag is implemented... should
the uploaded binary be published, or should the package be rebuilt? I
think I'd argue for the latter. The uploaded binary package should be
installed on the buildd and then the package should be rebuilt from
source, with _that_ result being put into the archive. This doesn't
protect against the trojaned-compiler problem, but it does at least
ensure that the package builds from source.

--
Richard


signature.asc (849 bytes) Download Attachment
12