Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Mattia Rizzolo-5
user [hidden email]
usertags 926242 reproducible
thanks

On Tue, Apr 02, 2019 at 09:30:58AM -0400, Chris Lamb wrote:
> However, it would be great if we had some continuous testing of this,
> with the usual bells-and-whistles of running/publishing diffoscope
> reports, etc.
>
> Due to the d-i testing release cycle it would be great to find any
> regressions so they can get merged in time for whatever the next
> alpha/beta is particularly as we (completely correctly!) get more
> conservative with any changes respect to the upcoming release of
> "buster."

Does the installer need anything special?  I thought d-i was just like
any other package when it came to regular building it.

> Whilst I think of it, there is also the separate issue of ensuring we
> generate a .buildinfo (or .buildinfo-like) build attestation document so
> that others can reproduce the build at a later date and further ensure
> this is published or otherwise available somewhere for official
> releases… but I was hoping it would become more obvious what we needed
> (without guessing) once we have testing.

If d-i is like I assume it be, there should a regular buildinfo as well,
see https://buildinfo.debian.net/sources/debian-installer for what I
mean.

> Mattia, is this something you could proof-of-concept…?

So the reason src:debian-installer does not build at this moment, it's
because it *is* a bit of a snowflake as it wants to access a debian
archive while it builds.  Currently pbuilder doesn't have a "whistlist"
method to block all networking but to a particular site, which is what
would be needed to solve this nicely.

One way to workaround this problem of src:debian-installer, would be for
our building script to instruct pbuilder to not block the network when
it's building this special package.  I think that's acceptable, given
that d-i is not a random package... :)


Am I missing some other detail?

--
regards,
                        Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540      .''`.
more about me:  https://mapreri.org                             : :'  :
Launchpad user: https://launchpad.net/~mapreri                  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Chris Lamb -2
Hey Mattia et al.,

> Does the installer need anything special?  I thought d-i was just like
> any other package when it came to regular building it.

For some reason I thought that src:debian-installer was a special-case
package (or simply used for its build-depends) and it does not
"really" build the images. Obviously, I hope you are absolutely right
and this is much easier than I thought to resolve.

(I think I got some of this preconception from various README files
which imply one should build via the Makefile, not via building
src:debian-installer.)

Thinking about it, a d-i *release* might trigger or other go through
some other codepaths — perhaps even manual ones — that might lead to
an unreproducible package.. and thus we would definitely want to
exercise these.

> If d-i is like I assume it be, there should a regular buildinfo as well,
> see https://buildinfo.debian.net/sources/debian-installer for what I
> mean.

Nod. However, we would need to double check that the src:debian-
installer .buildinfo includes or otherwise takes into consideration:

 a) The actual generated ISOs/.img/netboot images etc. and not just
    the binary packages as usual. (I mean we only really care about
    the former here, right?)

 b) Any sources downloaded directly from the archive.

As above, perhaps there is some kind of "release" codepath we need to
definitely and specifically test too? -boot can you chime in here on
this angle?

> So the reason src:debian-installer does not build at this moment [..]

For completeness (and/or people following along):

  http://tests.reproducible-builds.org/debian-installer

… specifically:

  make[2]: Entering directory '/build/1st/debian-installer-20190118/build'
  WARNING: mirror 'http://deb.debian.org/debian' appears to be invalid; skipping
  WARNING: mirror 'http://deb.debian.org/debian' appears to be invalid; skipping
  […]
  Building dependency tree...
  E: Unable to locate package acpi-modules-4.19.0-1-amd64-di
  E: Couldn't find any package by glob 'acpi-modules-4.19.0-1-amd64-di'
  E: Couldn't find any package by regex 'acpi-modules-4.19.0-1-amd64-di'
  […]

> One way to workaround this problem of src:debian-installer, would be for
> our building script to instruct pbuilder to not block the network when
> it's building this special package.

I think that's fine, especially if we add this to the log and that
it's well-commented as a magic exception. You happy to go-ahead with
this?


Best wishes,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      [hidden email] 🍥 chris-lamb.co.uk
       `-

Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Samuel Thibault-8
Hello,

Chris Lamb, le mer. 03 avril 2019 05:01:44 -0400, a ecrit:
> > Does the installer need anything special?  I thought d-i was just like
> > any other package when it came to regular building it.
>
> For some reason I thought that src:debian-installer was a special-case
> package (or simply used for its build-depends) and it does not
> "really" build the images.

It doesn't build the netinst/CD/DVD iso images indeed (debian-cd handles
that). But it builds the initrd used there (and the netboot mini.iso).

Samuel

Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Cyril Brulebois-4
Samuel Thibault <[hidden email]> (2019-04-03):

> Hello,
>
> Chris Lamb, le mer. 03 avril 2019 05:01:44 -0400, a ecrit:
> > > Does the installer need anything special?  I thought d-i was just like
> > > any other package when it came to regular building it.
> >
> > For some reason I thought that src:debian-installer was a special-case
> > package (or simply used for its build-depends) and it does not
> > "really" build the images.
>
> It doesn't build the netinst/CD/DVD iso images indeed (debian-cd handles
> that). But it builds the initrd used there (and the netboot mini.iso).
Right. Check the tarball (!) produced by building src:debian-installer;
that's what gets installed in installer-$arch directories in the
archive; then consumed by debian-cd to produce “full-blown” installation
images.

(You might have seen some special casing in dak; that's us!)


Cheers,
--
Cyril Brulebois ([hidden email])            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Chris Lamb -2
Hey Cyril & Samuel,

> > It doesn't build the netinst/CD/DVD iso images indeed (debian-cd handles
> > that). But it builds the initrd used there (and the netboot mini.iso).
>
> Right. Check the tarball (!) produced by building src:debian-installer;
> that's what gets installed in installer-$arch directories in the
> archive; then consumed by debian-cd to produce “full-blown” installation
> images.

TIL. However, as these generated files do not appear in the binary
debian-installer package it is likely that that our testing framework
will (after the mooted networking exception is made) entirely-
correctly report that the src:debian-installer package is reproducible
as its declared artifects contain only documentation. This will be
somewhat misleading about the true reproducibility status of our installer.

Just throwing out ideas here but perhaps this binary package could
contain at least the hashes of the generated files you mention? That way,
at least if they vary between our test builds then we will implicitly see
that in the package's reproducibility status.

However, as this would not incorporate anything that debian-cd does
with them to produce the "full-blown" images I suspect that this will
not be enough to cover everything. Just to underline this point in a
silly way we would not be aware of, for example, debian-cd running
"echo $RANDOM >> /target/ somefile.txt", even with the above hack.

Thanks for your input btw. :)


Best wishes,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      [hidden email] 🍥 chris-lamb.co.uk
       `-

Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Holger Levsen-2
Hi,

kudos for the progress so far!

On Wed, Apr 03, 2019 at 05:50:26AM -0400, Chris Lamb wrote:
> Just throwing out ideas here but perhaps this binary package could
> contain at least the hashes of the generated files you mention?

or we setup debian-cd builds as well..!


--
tschau,
        Holger

-------------------------------------------------------------------------------
               holger@(debian|reproducible-builds|layer-acht).org
       PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Mattia Rizzolo-5
In reply to this post by Chris Lamb -2
On Wed, Apr 03, 2019 at 05:50:26AM -0400, Chris Lamb wrote:

> Hey Cyril & Samuel,
>
> > > It doesn't build the netinst/CD/DVD iso images indeed (debian-cd handles
> > > that). But it builds the initrd used there (and the netboot mini.iso).
> >
> > Right. Check the tarball (!) produced by building src:debian-installer;
> > that's what gets installed in installer-$arch directories in the
> > archive; then consumed by debian-cd to produce “full-blown” installation
> > images.
>
> TIL. However, as these generated files do not appear in the binary
> debian-installer package it is likely that that our testing framework
> will (after the mooted networking exception is made) entirely-
> correctly report that the src:debian-installer package is reproducible
> as its declared artifects contain only documentation. This will be
> somewhat misleading about the true reproducibility status of our installer.
It doesn't contain only documentation.
src:debian-installer also builds a
debian-installer-images_$(VERSION)_$(ARCH).tar.gz that does contain
binary stuff, including the initrd, kernel image, mini.iso, etc etc.

> However, as this would not incorporate anything that debian-cd does
> with them to produce the "full-blown" images I suspect that this will
> not be enough to cover everything. Just to underline this point in a
> silly way we would not be aware of, for example, debian-cd running
> "echo $RANDOM >> /target/ somefile.txt", even with the above hack.

Right, to cover this we would need to do a full build of the cd image.
I have no idea whatsoever how that's done (and, as others said, it's
outside of the debian-boot office, it's within debian-cd).

In the meantime I did what I mentioned,
https://salsa.debian.org/qa/jenkins.debian.net/commit/e3117ca244b230c04e324814e20c02032026a5cf
and the build for unstable/arm64 is running.

--
regards,
                        Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540      .''`.
more about me:  https://mapreri.org                             : :'  :
Launchpad user: https://launchpad.net/~mapreri                  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Chris Lamb -2
Dear Mattia,

> > TIL. However, as these generated files do not appear in the binary
> > debian-installer package it is likely that that our testing framework
> > will (after the mooted networking exception is made) entirely-
> > correctly report that the src:debian-installer package is reproducible
> > as its declared artifects contain only documentation. This will be
> > somewhat misleading about the true reproducibility status of our installer.
>
> It doesn't contain only documentation.
> src:debian-installer also builds a
> debian-installer-images_$(VERSION)_$(ARCH).tar.gz that does contain
> binary stuff, including the initrd, kernel image, mini.iso, etc etc.

Whilst it may build indeed these files they do not appear in the
binary package:

    $ find | head
    .
    ./usr
    ./usr/share
    ./usr/share/doc
    ./usr/share/doc/debian-installer
    ./usr/share/doc/debian-installer/talks
    ./usr/share/doc/debian-installer/talks/fosdem07
    ./usr/share/doc/debian-installer/talks/fosdem07/README
    ./usr/share/doc/debian-installer/talks/fosdem07/fosdem1.tgz
    ./usr/share/doc/debian-installer/talks/d-i_debconf7

    $ find | grep images

    $

Thus, they cannot affect the reproducibility status of the debian-
installer source package. This prompted my paragraph regarding at
least including these files' hashes, etc.

> Right, to cover this we would need to do a full build of the cd image.
> I have no idea whatsoever how that's done (and, as others said, it's
> outside of the debian-boot office, it's within debian-cd).

I see I've been conflating the -boot and -cd work here, sorry. Well,
if full debian-cd rebuilds are out of scope for now (!) we should
at-least cover any "debian-boot" related stuff.

Now, I am certain I am missing something but would this latter
approach just (!) involve adding another Jenkins job that tracks the
HEAD of debian- installer.git (ooh, another advantage; time) which
calls "make -C build release" in two environments?


Regards,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      [hidden email] 🍥 chris-lamb.co.uk
       `-

Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Mattia Rizzolo-5
On Wed, Apr 03, 2019 at 06:21:39AM -0400, Chris Lamb wrote:
> Whilst it may build indeed these files they do not appear in the
> binary package:

> Thus, they cannot affect the reproducibility status of the debian-
> installer source package. This prompted my paragraph regarding at
> least including these files' hashes, etc.

Yes they do!

The tricky part is that that "tarball" we have been talking about is not
saved anywhere.  Once it is uploaded to ftp-master it's unpacked and
then discarded, so you can't quite get your hands on it without doing a
local build of debian-installer.

Furthermore now d-i as released wouldn't build anyway because it tries
to look up linux 4.19.0-1 instead of 4.19.0-4... And after changing that
it tries to look for the non-existent hyperv-modules-4.19.0-4-amd64-di,
and here I don't want to go digging in d-i… :( - so I can't show you
that tarball I'm talking about right now...

--
regards,
                        Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540      .''`.
more about me:  https://mapreri.org                             : :'  :
Launchpad user: https://launchpad.net/~mapreri                  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Chris Lamb -2
Hi Mattia,

> > Whilst it may build indeed these files they do not appear in the
> > binary package:
>
> > Thus, they cannot affect the reproducibility status of the debian-
> > installer source package. […]
>
> Yes they do!

Oh, hang on, does src:debian-installer's .changes file include these
extra files? That might be what I'm missing.

My conception is that as we call diffoscope on the two .changes files
it will report they are reproducible as, well, the binary package will
(likely…) be identical.

> The tricky part is that that "tarball" we have been talking about is not
> saved anywhere.

Nod, I guess we would want that for true jenkins.debian.org "please
save the build artifacts" support but for now getting a red/green
light would be interesting in/of itself.


Best wishes,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      [hidden email] 🍥 chris-lamb.co.uk
       `-

Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Cyril Brulebois-4
Chris Lamb <[hidden email]> (2019-04-03):

> Hi Mattia,
>
> > > Whilst it may build indeed these files they do not appear in the
> > > binary package:
> >
> > > Thus, they cannot affect the reproducibility status of the debian-
> > > installer source package. […]
> >
> > Yes they do!
>
> Oh, hang on, does src:debian-installer's .changes file include these
> extra files? That might be what I'm missing.
Yes:

    kibi@armor:~/debian-installer$ dcmd debian-installer_20190118_amd64.changes
    debian-installer-images_20190118_amd64.tar.gz
    debian-installer_20190118_amd64.buildinfo
    debian-installer_20190118_amd64.deb
    debian-installer_20190118_amd64.changes

The tarball is uploaded alongside the other files; “just” treated
“somewhat differently” by dak.
 

Cheers,
--
Cyril Brulebois ([hidden email])            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Samuel Thibault-8
In reply to this post by Chris Lamb -2
Chris Lamb, le mer. 03 avril 2019 06:45:28 -0400, a ecrit:

> > > Whilst it may build indeed these files they do not appear in the
> > > binary package:
> >
> > > Thus, they cannot affect the reproducibility status of the debian-
> > > installer source package. […]
> >
> > Yes they do!
>
> Oh, hang on, does src:debian-installer's .changes file include these
> extra files? That might be what I'm missing.

It seems so, see in

https://buildd.debian.org/status/fetch.php?pkg=debian-installer&arch=amd64&ver=20190118&stamp=1547858681&raw=0

Checksums-Sha1:
 726c95c9add3222b6c4b744f8cf1e18afd44437e 528261542 debian-installer-images_20190118_amd64.tar.gz
 324185c8187548dd86e301910751e1f83413ae5c 10730 debian-installer_20190118_amd64.buildinfo
 870d941832bcdd7fbde0b04974d5e46c812d76b2 734340 debian-installer_20190118_amd64.deb
Checksums-Sha256:
 698b1c32e0f3d1da174ed96af64040d65eac955ccec110d7b651b615ff3c74ae 528261542 debian-installer-images_20190118_amd64.tar.gz
 633371431ea7a425163f7a8360d56e1fa401fffcfff8612fe80078cdb40add4b 10730 debian-installer_20190118_amd64.buildinfo
 7e82f489df6d36a37bc9c810cb24d4cfbdc1eb95e5aa539d19691c28d24c6dfa 734340 debian-installer_20190118_amd64.deb
Files:
 843efa1eba8e58e04bd174647431fbc6 528261542 raw-installer - debian-installer-images_20190118_amd64.tar.gz
 9daa2e9db4355b189c60132a97eb4103 10730 devel optional debian-installer_20190118_amd64.buildinfo
 4c9e4745b8dd0e673af2548f11bc5796 734340 devel optional debian-installer_20190118_amd64.deb

Samuel

Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Mattia Rizzolo-5
In reply to this post by Chris Lamb -2
On Wed, Apr 03, 2019 at 06:45:28AM -0400, Chris Lamb wrote:
> My conception is that as we call diffoscope on the two .changes files
> it will report they are reproducible as, well, the binary package will
> (likely…) be identical.

Yup.  It would.

> > The tricky part is that that "tarball" we have been talking about is not
> > saved anywhere.
>
> Nod, I guess we would want that for true jenkins.debian.org "please
> save the build artifacts" support but for now getting a red/green
> light would be interesting in/of itself.

I think it would work out of the box already.

Now, regarding building d-i as a normal package, I hit a bit of a
readblock because it fails while trying to download the files in the
nodes running in the future.
What d-i does it copying the url from the host's (or, well, the
chroot's) /etc/apt/sources.list, but it seems it doesn't also pick up an
eventual [check-valid-until=no] placed on the same line :\

--
regards,
                        Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540      .''`.
more about me:  https://mapreri.org                             : :'  :
Launchpad user: https://launchpad.net/~mapreri                  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Chris Lamb -2
[dropping CCs already subscribed to debian-boot]

Hi Mattia,

> Now, regarding building d-i as a normal package, I hit a bit of a
> readblock because it fails while trying to download the files in the
> nodes running in the future.

Sounds about right:

  https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/debian-installer.html

    get-packages udeb  
    make[10]: 'sources.list.udeb' is up to date.
    Ign:1 copy:/build/debian-installer-20190410/2nd/build localudebs/ InRelease
    Ign:2 copy:/build/debian-installer-20190410/2nd/build localudebs/ Release
    Ign:3 copy:/build/debian-installer-20190410/2nd/build localudebs/ Packages
    Ign:3 copy:/build/debian-installer-20190410/2nd/build localudebs/ Packages
    Ign:3 copy:/build/debian-installer-20190410/2nd/build localudebs/ Packages
    Get:3 copy:/build/debian-installer-20190410/2nd/build localudebs/ Packages [20 B]
    Get:4 http://cdn-fastly.deb.debian.org/debian buster InRelease [163 kB]
    Reading package lists...
    E: Release file for http://deb.debian.org/debian/dists/buster/InRelease is expired (invalid since 391d 12h 5min 44s). Updates for this repository will not be applied.

> What d-i does it copying the url from the host's (or, well, the
> chroot's) /etc/apt/sources.list, but it seems it doesn't also pick up an
> eventual [check-valid-until=no] placed on the same line :\

Nod. Any further thoughts on this? It would be great to see where we
are at here...


Best wishes,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      [hidden email] 🍥 chris-lamb.co.uk
       `-

Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Vagrant Cascadian-4
On 2019-05-18, Chris Lamb wrote:

>> Now, regarding building d-i as a normal package, I hit a bit of a
>> readblock because it fails while trying to download the files in the
>> nodes running in the future.
>
> Sounds about right:
>
>   https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/debian-installer.html
>
>     get-packages udeb  
>     make[10]: 'sources.list.udeb' is up to date.
>     Get:3 copy:/build/debian-installer-20190410/2nd/build localudebs/ Packages [20 B]
...

>     Get:4 http://cdn-fastly.deb.debian.org/debian buster InRelease [163 kB]
>     Reading package lists...
>     E: Release file for http://deb.debian.org/debian/dists/buster/InRelease is expired (invalid since 391d 12h 5min 44s). Updates for this repository will not be applied.
>
>> What d-i does it copying the url from the host's (or, well, the
>> chroot's) /etc/apt/sources.list, but it seems it doesn't also pick up an
>> eventual [check-valid-until=no] placed on the same line :\
>
> Nod. Any further thoughts on this? It would be great to see where we
> are at here...
Colin Watson removed it back in 2011:

  https://salsa.debian.org/installer-team/debian-installer/commit/fa965c32ca8bfa2ff14886c6f0dca131532815c7

commit fa965c32ca8bfa2ff14886c6f0dca131532815c7
Author: Colin Watson <[hidden email]>
Date:   Mon Mar 14 18:08:25 2011 +0000

    Skip the option field in sources.list lines, if present.                                                  

diff --git a/build/util/gen-sources.list.udeb b/build/util/gen-sources.list.udeb                              
index e86b4fa66..9f140100e 100755
--- a/build/util/gen-sources.list.udeb
+++ b/build/util/gen-sources.list.udeb
@@ -36,6 +36,7 @@ get_mirrors() {
                [ -s $file ] || continue
                grep '^deb[[:space:]]' $file | \
                   grep -v '^deb[[:space:]]\+cdrom:' | \                                                      
+                  sed 's,^deb \[[^]]*\] ,deb ,' | \
                   grep -v '\(security.debian.org\|volatile.debian.\(net\|org\)\)' | \                        
                   grep '[[:space:]]main' | \
                   awk '{print $1 " " $2}' | \


Happen to remember what it breaks to have these present? If it was a
workaround, perhaps it is no longer needed?


live well,
  vagrant

signature.asc (233 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Colin Watson
On Fri, May 24, 2019 at 06:56:35PM -0700, Vagrant Cascadian wrote:

> Colin Watson removed it back in 2011:
>
>   https://salsa.debian.org/installer-team/debian-installer/commit/fa965c32ca8bfa2ff14886c6f0dca131532815c7
>
> commit fa965c32ca8bfa2ff14886c6f0dca131532815c7
> Author: Colin Watson <[hidden email]>
> Date:   Mon Mar 14 18:08:25 2011 +0000
>
>     Skip the option field in sources.list lines, if present.                                                  
>
> diff --git a/build/util/gen-sources.list.udeb b/build/util/gen-sources.list.udeb                              
> index e86b4fa66..9f140100e 100755
> --- a/build/util/gen-sources.list.udeb
> +++ b/build/util/gen-sources.list.udeb
> @@ -36,6 +36,7 @@ get_mirrors() {
>                 [ -s $file ] || continue
>                 grep '^deb[[:space:]]' $file | \
>                    grep -v '^deb[[:space:]]\+cdrom:' | \                                                      
> +                  sed 's,^deb \[[^]]*\] ,deb ,' | \
>                    grep -v '\(security.debian.org\|volatile.debian.\(net\|org\)\)' | \                        
>                    grep '[[:space:]]main' | \
>                    awk '{print $1 " " $2}' | \
>
>
> Happen to remember what it breaks to have these present? If it was a
> workaround, perhaps it is no longer needed?

I'm not certain even after going through my IRC and email logs around
that time, but given the timing I suspect that it was a workaround for
multiarch systems where sources.list contained some lines with
[arch=...] options to limit them to only some architectures.  I don't
remember exactly how this broke the d-i build; I assume it must have
done at the time or I wouldn't have made that change.

It would be worth somebody trying out a d-i build on a system with this
kind of configuration to see if it still breaks, but otherwise I suspect
we can drop this.

--
Colin Watson                                       [[hidden email]]

Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Chris Lamb -2
Dear Colin,

> >   https://salsa.debian.org/installer-team/debian-installer/commit/fa965c32ca8bfa2ff14886c6f0dca131532815c7
[…]
> I'm not certain even after going through my IRC and email logs around
> that time, but given the timing I suspect that it was a workaround for
> multiarch systems where sources.list contained some lines with
> [arch=...] options to limit them to only some architectures.

Thank for looking into this.

> It would be worth somebody trying out a d-i build on a system with this
> kind of configuration to see if it still breaks
  ^^^^^^^^^^^^^^^^^^^^^

Just to clarify, building d-i on a system with [arch=...] foo in its
/etc/apt/sources.list?


Best wishes,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      [hidden email] 🍥 chris-lamb.co.uk
       `-

Reply | Threaded
Open this post in threaded view
|

Re: Bug#926242: jenkins.debian.org: Please test reproducibility status of Debian Installer images

Colin Watson
On Sun, May 26, 2019 at 11:05:40AM +0100, Chris Lamb wrote:
> Dear Colin,
> > It would be worth somebody trying out a d-i build on a system with this
> > kind of configuration to see if it still breaks
>   ^^^^^^^^^^^^^^^^^^^^^
>
> Just to clarify, building d-i on a system with [arch=...] foo in its
> /etc/apt/sources.list?

This is all from dubious memory, but I suspect my setup at the time was
roughly an amd64 system with:

  deb [arch=amd64] <local partial mirror>
  deb <official mirror>

... on the grounds that my local partial mirror didn't have space for
both amd64 and i386.

--
Colin Watson                                       [[hidden email]]