CI for kraken

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

CI for kraken

Nadiya Sitdykova
Hello All, 

I've been working on autopkgtest test-suite for kraken package and encountered a problem.

debian/control have jellyfish in Depends, but kraken-build fails with message:

      Found jellyfish v2.2.6
      Kraken requires jellyfish version 1

I was able to install manually jellyfish v1.1.5-1, but only when I added wheezy repository to my source.list. It seems that sid doesn't have older versions of jellyfish. 

But when I changed debian/control so it have jellyfish (= 1.1.5-1) in Depends, I wasn't able to run autopkgtest in a chroot successfully. Although autpkgtest with null works just fine, all tests pass.
I attach the log after executing 

      autopkgtest kraken_0.10.5~beta-3_amd64.changes -- schroot sid-amd64

I'm stuck on this problem, and any help would be much appreciated.

P.S. I commited my changes in case someone would like to reproduce the problem

Regards, Nadiya

kraken_autpkgtest.log (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: CI for kraken

Andreas Tille-5
Hi Nadiya,

I'm back now but need some time to recover from backlog.  Since this is
a more complex issue than just uploading I'll put this issue back for
some later inspection.  May be somebody else might beat me in giving a
proper answer.  The only thing I know is that I vaguely remember that
kraken really needs jellyfish version 1 and I somehow discussed this
with kraken authors in the past.  At that time (summer 2016? - may be
you seek mailing list archive for the discussion since I usually forward
any technically relevant information) upstream claimed to work on kraken2
which should be independently from jellyfish.

So far for a short answer

      Andreas.

On Tue, Jul 11, 2017 at 07:09:37PM -0700, Nadiya Sitdykova wrote:

> Hello All,
>
> I've been working on autopkgtest test-suite for kraken package and
> encountered a problem.
>
> debian/control have jellyfish in Depends, but kraken-build fails with
> message:
>
>       Found jellyfish v2.2.6
>       Kraken requires jellyfish version 1
>
> I was able to install manually jellyfish v1.1.5-1, but only when I added
> wheezy repository to my source.list. It seems that sid doesn't have older
> versions of jellyfish.
>
> But when I changed debian/control so it have jellyfish (= 1.1.5-1) in
> Depends, I wasn't able to run autopkgtest in a chroot successfully.
> Although autpkgtest with null works just fine, all tests pass.
> I attach the log after executing
>
>       autopkgtest kraken_0.10.5~beta-3_amd64.changes -- schroot sid-amd64
>
> I'm stuck on this problem, and any help would be much appreciated.
>
> P.S. I commited my changes in case someone would like to reproduce the
> problem
>
> Regards, Nadiya

> nadiya@debian:~$ autopkgtest kraken_0.10.5~beta-3_amd64.changes -- schroot sid-amd64
> autopkgtest [21:41:52]: version 4.4
> autopkgtest [21:41:52]: host debian; command line: /usr/bin/autopkgtest 'kraken_0.10.5~beta-3_amd64.changes' -- schroot sid-amd64
> autopkgtest [21:41:53]: testbed dpkg architecture: amd64
> autopkgtest [21:41:53]: testbed running kernel: Linux 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2 (2017-06-12)
> autopkgtest [21:41:53]: @@@@@@@@@@@@@@@@@@@@ source kraken_0.10.5~beta-3.dsc
> dpkg-source: warning: extracting unsigned source package (/tmp/autopkgtest.t5jdFh/kraken_0.10.5~beta-3.dsc)
> dpkg-source: info: extracting kraken in kraken-0.10.5~beta
> dpkg-source: info: unpacking kraken_0.10.5~beta.orig.tar.xz
> dpkg-source: info: unpacking kraken_0.10.5~beta-3.debian.tar.xz
> dpkg-source: info: applying kraken_tmp_dir.patch
> autopkgtest [21:41:53]: testing package kraken version 0.10.5~beta-3
> autopkgtest [21:41:53]: build not needed
> autopkgtest [21:41:53]: test run-unit-test: preparing testbed
> Get:1 file:/tmp/autopkgtest.t5jdFh/binaries  InRelease
> Ign:1 file:/tmp/autopkgtest.t5jdFh/binaries  InRelease
> Get:2 file:/tmp/autopkgtest.t5jdFh/binaries  Release [816 B]
> Get:2 file:/tmp/autopkgtest.t5jdFh/binaries  Release [816 B]
> Get:3 file:/tmp/autopkgtest.t5jdFh/binaries  Release.gpg
> Ign:3 file:/tmp/autopkgtest.t5jdFh/binaries  Release.gpg
> Get:4 file:/tmp/autopkgtest.t5jdFh/binaries  Packages [2518 B]
> Reading package lists...
> Reading package lists... Done
> Building dependency tree      
> Reading state information... Done
> Correcting dependencies...Starting pkgProblemResolver with broken count: 1
> Starting 2 pkgProblemResolver with broken count: 1
> Investigating (0) autopkgtest-satdep:amd64 < 0 @iU mK Nb Ib >
> Broken autopkgtest-satdep:amd64 Depends on kraken:amd64 < none | 0.10.5~beta-3 @un uH > (>= 0~)
>   Considering kraken:amd64 0 as a solution to autopkgtest-satdep:amd64 -2
>   Removing autopkgtest-satdep:amd64 rather than change kraken:amd64
> Done
>  Done
> Starting pkgProblemResolver with broken count: 0
> Starting 2 pkgProblemResolver with broken count: 0
> Done
> The following packages will be REMOVED:
>   autopkgtest-satdep
> 0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
> 1 not fully installed or removed.
> After this operation, 0 B of additional disk space will be used.
> (Reading database ... 13390 files and directories currently installed.)
> Removing autopkgtest-satdep (0) ...
> blame: arg:kraken-dbgsym_0.10.5~beta-3_amd64.deb deb:kraken-dbgsym arg:kraken_0.10.5~beta-3_amd64.deb deb:kraken kraken_0.10.5~beta-3.dsc
> badpkg: Test dependencies are unsatisfiable. A common reason is that your testbed is out of date with respect to the archive, and you need to use a current testbed or run apt-get update or use -U.
> autopkgtest [21:41:56]: ERROR: erroneous package: Test dependencies are unsatisfiable. A common reason is that your testbed is out of date with respect to the archive, and you need to use a current testbed or run apt-get update or use -U.


--
http://fam-tille.de

Reply | Threaded
Open this post in threaded view
|

Re: CI for kraken

Andreas Tille-5
In reply to this post by Nadiya Sitdykova
Hi Nadiya,

On Tue, Jul 11, 2017 at 07:09:37PM -0700, Nadiya Sitdykova wrote:

> I've been working on autopkgtest test-suite for kraken package and
> encountered a problem.
>
> debian/control have jellyfish in Depends, but kraken-build fails with
> message:
>
>       Found jellyfish v2.2.6
>       Kraken requires jellyfish version 1
>
> I was able to install manually jellyfish v1.1.5-1, but only when I added
> wheezy repository to my source.list. It seems that sid doesn't have older
> versions of jellyfish.
>
> But when I changed debian/control so it have jellyfish (= 1.1.5-1) in
> Depends, I wasn't able to run autopkgtest in a chroot successfully.
> Although autpkgtest with null works just fine, all tests pass.
> I attach the log after executing
>
>       autopkgtest kraken_0.10.5~beta-3_amd64.changes -- schroot sid-amd64
>
> I'm stuck on this problem, and any help would be much appreciated.

I think the only proper solution for this problem is to provide a
jellyfish1 package in addition to jellyfish.  I think a sensible way
to do this is to copy jellyfish git to jellyfish1 checkout the latest
release of version 1 and make this new master.  Would you volunteer
to do this (provided my explanation was sufficient)?

Kind regards

        Andreas.

--
http://fam-tille.de

Reply | Threaded
Open this post in threaded view
|

Re: CI for kraken

Nadiya Sitdykova
Hello Andreas, 

Unfortunately your explanation wasn't sufficient for me. 
Do you mean that I should clone both jellyfish and jellyfish1, than copy jellyfish git file to the jellyfish1 directory? 
And than how can I checkout the latest release of version 1 and make this new master?

Regards, Nadiya

On Fri, Jul 14, 2017 at 11:15 PM, Andreas Tille <[hidden email]> wrote:
Hi Nadiya,

On Tue, Jul 11, 2017 at 07:09:37PM -0700, Nadiya Sitdykova wrote:
> I've been working on autopkgtest test-suite for kraken package and
> encountered a problem.
>
> debian/control have jellyfish in Depends, but kraken-build fails with
> message:
>
>       Found jellyfish v2.2.6
>       Kraken requires jellyfish version 1
>
> I was able to install manually jellyfish v1.1.5-1, but only when I added
> wheezy repository to my source.list. It seems that sid doesn't have older
> versions of jellyfish.
>
> But when I changed debian/control so it have jellyfish (= 1.1.5-1) in
> Depends, I wasn't able to run autopkgtest in a chroot successfully.
> Although autpkgtest with null works just fine, all tests pass.
> I attach the log after executing
>
>       autopkgtest kraken_0.10.5~beta-3_amd64.changes -- schroot sid-amd64
>
> I'm stuck on this problem, and any help would be much appreciated.

I think the only proper solution for this problem is to provide a
jellyfish1 package in addition to jellyfish.  I think a sensible way
to do this is to copy jellyfish git to jellyfish1 checkout the latest
release of version 1 and make this new master.  Would you volunteer
to do this (provided my explanation was sufficient)?

Kind regards

        Andreas.

--
http://fam-tille.de


Reply | Threaded
Open this post in threaded view
|

Re: CI for kraken

Andreas Tille-5
Hi Nadiya,

On Wed, Jul 19, 2017 at 08:09:19PM -0700, Nadiya Sitdykova wrote:
> Unfortunately your explanation wasn't sufficient for me.
> Do you mean that I should clone both jellyfish and jellyfish1, than copy
> jellyfish git file to the jellyfish1 directory?
> And than how can I checkout the latest release of version 1 and make this
> new master?

I'm not a Git expert but I did the following - may be there is a more
efficient way to do things.

  cp -a jellyfish jellyfish1
  cd jellyfish1
  git reset --hard debian/1.1.5-1
  sed -i 's/jellyfish/jellyfish1/g' debian/control
  vi debian/control # doing some manual fixes in debian/control
  cme fix dpkg-control
  dch -i
-->
jellyfish1 (1.1.5-1) UNRELEASED; urgency=medium

  * Create separate jellyfish1 package since there exist packages that
    explicitly require jellyfish version 1

 -- Andreas Tille <[hidden email]>  Thu, 20 Jul 2017 09:16:24 +0200

  git commit -a -m"create jellyfish1 package"
  git checkout upstream
  git reset --hard upstream/1.1.5
  git checkout pristine-tar
  git reset --hard 8b4a53ecb71b5739d050fbb926c5298840318093
  # this goes down to the import of 1.1.11 which is the latest version of the 1.x series
  vi .git/config
  # this is a hack since I have no idea about all those Git settings.  I simply left
  # the [core] section and used my inject-into-alioth-git script which creates a new
  # repository on aliot ( https://anonscm.debian.org/viewvc/debian-med/trunk/helper-scripts/inject-into-alioth-git?view=markup )
  inject-into-alioth-git
  # forgot to remove branches
  git branch -D debian/jessie-backports
  git push origin :debian/jessie-backports
  git branch -D libjsoncpp
  git push origin :libjsoncpp
  # fix pristine-tar branch
  git checkout pristine-tar
  for i in jellyfish_* ; do git mv $i `echo $i | sed "s/^jellyfish/&1/"` ; done
  git commit -a -m"Rename pristine-tar information files"
  git checkout master
  # after realising that upstream branch seems not to fit 1.1.11
  # reimport this tarball, deleting upstream/1.1.11 tag first
  git tag -d upstream/1.1.11
  git push origin :refs/tags/upstream/1.1.11
  uscan --verbose --force-download
  gbp import-orig --pristine-tar ../jellyfish1_1.1.11.orig.tar.gz


This leads IMHO to some workable Git repository of 1.x series of
jellyfish.  I also added a commit to fetch only 1.x series in watch
file.  Now the task is to redo some packaging changes like separating
libjellyfish-1.x - but only in case this would be really needed for
kraken.  There is no point in over-engineering.  Our fokus should be
to get kraken reliably working and not copying jellyfish 2.x layout
since this is most probably not used and just draining time.

I tested as far that gbp-build now starts building but there is some
issue in the build time tests.  To clean up the repository it might be
sensible to remove all tags refering to 2.x series.

Do you think you can continue from here and check, whether kraken
works with the resulting package?  If something remains unclear feel
free to keep on asking as usual.

Kind regards

      Andreas.

--
http://fam-tille.de

Reply | Threaded
Open this post in threaded view
|

Re: CI for kraken

Andreas Tille-5
(as you might have seen from commit logs I've done some

   for i in `git tag | grep '/2'`; do git tag -d $i; git push origin :refs/tags/$i ; done

to remove useless tags)

On Thu, Jul 20, 2017 at 09:46:15AM +0200, Andreas Tille wrote:

> Hi Nadiya,
>
> On Wed, Jul 19, 2017 at 08:09:19PM -0700, Nadiya Sitdykova wrote:
> > Unfortunately your explanation wasn't sufficient for me.
> > Do you mean that I should clone both jellyfish and jellyfish1, than copy
> > jellyfish git file to the jellyfish1 directory?
> > And than how can I checkout the latest release of version 1 and make this
> > new master?
>
> I'm not a Git expert but I did the following - may be there is a more
> efficient way to do things.
>
>   cp -a jellyfish jellyfish1
>   cd jellyfish1
>   git reset --hard debian/1.1.5-1
>   sed -i 's/jellyfish/jellyfish1/g' debian/control
>   vi debian/control # doing some manual fixes in debian/control
>   cme fix dpkg-control
>   dch -i
> -->
> jellyfish1 (1.1.5-1) UNRELEASED; urgency=medium
>
>   * Create separate jellyfish1 package since there exist packages that
>     explicitly require jellyfish version 1
>
>  -- Andreas Tille <[hidden email]>  Thu, 20 Jul 2017 09:16:24 +0200
>
>   git commit -a -m"create jellyfish1 package"
>   git checkout upstream
>   git reset --hard upstream/1.1.5
>   git checkout pristine-tar
>   git reset --hard 8b4a53ecb71b5739d050fbb926c5298840318093
>   # this goes down to the import of 1.1.11 which is the latest version of the 1.x series
>   vi .git/config
>   # this is a hack since I have no idea about all those Git settings.  I simply left
>   # the [core] section and used my inject-into-alioth-git script which creates a new
>   # repository on aliot ( https://anonscm.debian.org/viewvc/debian-med/trunk/helper-scripts/inject-into-alioth-git?view=markup )
>   inject-into-alioth-git
>   # forgot to remove branches
>   git branch -D debian/jessie-backports
>   git push origin :debian/jessie-backports
>   git branch -D libjsoncpp
>   git push origin :libjsoncpp
>   # fix pristine-tar branch
>   git checkout pristine-tar
>   for i in jellyfish_* ; do git mv $i `echo $i | sed "s/^jellyfish/&1/"` ; done
>   git commit -a -m"Rename pristine-tar information files"
>   git checkout master
>   # after realising that upstream branch seems not to fit 1.1.11
>   # reimport this tarball, deleting upstream/1.1.11 tag first
>   git tag -d upstream/1.1.11
>   git push origin :refs/tags/upstream/1.1.11
>   uscan --verbose --force-download
>   gbp import-orig --pristine-tar ../jellyfish1_1.1.11.orig.tar.gz
>
>
> This leads IMHO to some workable Git repository of 1.x series of
> jellyfish.  I also added a commit to fetch only 1.x series in watch
> file.  Now the task is to redo some packaging changes like separating
> libjellyfish-1.x - but only in case this would be really needed for
> kraken.  There is no point in over-engineering.  Our fokus should be
> to get kraken reliably working and not copying jellyfish 2.x layout
> since this is most probably not used and just draining time.
>
> I tested as far that gbp-build now starts building but there is some
> issue in the build time tests.  To clean up the repository it might be
> sensible to remove all tags refering to 2.x series.
>
> Do you think you can continue from here and check, whether kraken
> works with the resulting package?  If something remains unclear feel
> free to keep on asking as usual.
>
> Kind regards
>
>       Andreas.
>
> --
> http://fam-tille.de
>
>

--
http://fam-tille.de

Reply | Threaded
Open this post in threaded view
|

Re: CI for kraken

Nadiya Sitdykova
Hello Andreas, 

I can see a git repository jellyfish1, but I'm not sure how to tell kraken to use this particular package. 
I tried to write jellyfish1 in debian/control Depends and it didn't work. Also I tried jellyfish (= 1.1.5-1) and it still gives the same error message.
And when I try just jellyfish it doesn't work again with message 

Found jellyfish v2.2.6
Kraken requires jellyfish version 1

Sorry, I must be missing something important.

Regards, Nadiya

On Thu, Jul 20, 2017 at 12:54 AM, Andreas Tille <[hidden email]> wrote:
(as you might have seen from commit logs I've done some

   for i in `git tag | grep '/2'`; do git tag -d $i; git push origin :refs/tags/$i ; done

to remove useless tags)

On Thu, Jul 20, 2017 at 09:46:15AM +0200, Andreas Tille wrote:
> Hi Nadiya,
>
> On Wed, Jul 19, 2017 at 08:09:19PM -0700, Nadiya Sitdykova wrote:
> > Unfortunately your explanation wasn't sufficient for me.
> > Do you mean that I should clone both jellyfish and jellyfish1, than copy
> > jellyfish git file to the jellyfish1 directory?
> > And than how can I checkout the latest release of version 1 and make this
> > new master?
>
> I'm not a Git expert but I did the following - may be there is a more
> efficient way to do things.
>
>   cp -a jellyfish jellyfish1
>   cd jellyfish1
>   git reset --hard debian/1.1.5-1
>   sed -i 's/jellyfish/jellyfish1/g' debian/control
>   vi debian/control # doing some manual fixes in debian/control
>   cme fix dpkg-control
>   dch -i
> -->
> jellyfish1 (1.1.5-1) UNRELEASED; urgency=medium
>
>   * Create separate jellyfish1 package since there exist packages that
>     explicitly require jellyfish version 1
>
>  -- Andreas Tille <[hidden email]>  Thu, 20 Jul 2017 09:16:24 +0200
>
>   git commit -a -m"create jellyfish1 package"
>   git checkout upstream
>   git reset --hard upstream/1.1.5
>   git checkout pristine-tar
>   git reset --hard 8b4a53ecb71b5739d050fbb926c5298840318093
>   # this goes down to the import of 1.1.11 which is the latest version of the 1.x series
>   vi .git/config
>   # this is a hack since I have no idea about all those Git settings.  I simply left
>   # the [core] section and used my inject-into-alioth-git script which creates a new
>   # repository on aliot ( https://anonscm.debian.org/viewvc/debian-med/trunk/helper-scripts/inject-into-alioth-git?view=markup )
>   inject-into-alioth-git
>   # forgot to remove branches
>   git branch -D debian/jessie-backports
>   git push origin :debian/jessie-backports
>   git branch -D libjsoncpp
>   git push origin :libjsoncpp
>   # fix pristine-tar branch
>   git checkout pristine-tar
>   for i in jellyfish_* ; do git mv $i `echo $i | sed "s/^jellyfish/&1/"` ; done
>   git commit -a -m"Rename pristine-tar information files"
>   git checkout master
>   # after realising that upstream branch seems not to fit 1.1.11
>   # reimport this tarball, deleting upstream/1.1.11 tag first
>   git tag -d upstream/1.1.11
>   git push origin :refs/tags/upstream/1.1.11
>   uscan --verbose --force-download
>   gbp import-orig --pristine-tar ../jellyfish1_1.1.11.orig.tar.gz
>
>
> This leads IMHO to some workable Git repository of 1.x series of
> jellyfish.  I also added a commit to fetch only 1.x series in watch
> file.  Now the task is to redo some packaging changes like separating
> libjellyfish-1.x - but only in case this would be really needed for
> kraken.  There is no point in over-engineering.  Our fokus should be
> to get kraken reliably working and not copying jellyfish 2.x layout
> since this is most probably not used and just draining time.
>
> I tested as far that gbp-build now starts building but there is some
> issue in the build time tests.  To clean up the repository it might be
> sensible to remove all tags refering to 2.x series.
>
> Do you think you can continue from here and check, whether kraken
> works with the resulting package?  If something remains unclear feel
> free to keep on asking as usual.
>
> Kind regards
>
>       Andreas.
>
> --
> http://fam-tille.de
>
>

--
http://fam-tille.de


Reply | Threaded
Open this post in threaded view
|

Re: CI for kraken

Andreas Tille-5
Hello Nadiya,

On Thu, Jul 20, 2017 at 04:38:08PM -0700, Nadiya Sitdykova wrote:
>
> I can see a git repository jellyfish1, but I'm not sure how to tell kraken
> to use this particular package.

Well, for sure you add this to the dependencies.

> I tried to write jellyfish1 in debian/control Depends and it didn't work.
> Also I tried jellyfish (= 1.1.5-1) and it still gives the same error
> message.
> And when I try just jellyfish it doesn't work again with message
>
> Found jellyfish v2.2.6
> Kraken requires jellyfish version 1
>
> Sorry, I must be missing something important.

I have no time to track this down before Monday, but here are some
ideas:

  1. We need to make yellyfish and jellyfish1 co-installable -
     currently the executables have the same name - so jellyfish1
     should have /usr/bin/jellyfish1
  2. Since kraken will not seek for /usr/bin/jellyfish1 we either
     need to put the jellyfish1 executable in /usr/lib and provide
     the binary above as link.  Than we could use a shell wrapper
     around kraken and set the PATH accordingly to make sure kraken
     will find the right executable.
     Alternatively we can simply patch kraken to use jellyfish1.
     This might be easier to debug to make really sure that kraken
     is using the correct version and it should definitely prevent
     that it will find the 2.2.6 version.

So far for my ideas.  If these are not helpful I need to check at
beginning of next week.

Kind regards

        Andreas.

--
http://fam-tille.de

Reply | Threaded
Open this post in threaded view
|

Re: CI for kraken

Nadiya Sitdykova
Hello Andreas, 

1. I don't know how to make it happen, maybe you can give some hints?
2. I think I know the place where jellyfish is called, so it shouldn't be a problem to patch kraken to use jellyfish1

Regards, Nadiya

On Thu, Jul 20, 2017 at 9:30 PM, Andreas Tille <[hidden email]> wrote:
Hello Nadiya,

On Thu, Jul 20, 2017 at 04:38:08PM -0700, Nadiya Sitdykova wrote:
>
> I can see a git repository jellyfish1, but I'm not sure how to tell kraken
> to use this particular package.

Well, for sure you add this to the dependencies.

> I tried to write jellyfish1 in debian/control Depends and it didn't work.
> Also I tried jellyfish (= 1.1.5-1) and it still gives the same error
> message.
> And when I try just jellyfish it doesn't work again with message
>
> Found jellyfish v2.2.6
> Kraken requires jellyfish version 1
>
> Sorry, I must be missing something important.

I have no time to track this down before Monday, but here are some
ideas:

  1. We need to make yellyfish and jellyfish1 co-installable -
     currently the executables have the same name - so jellyfish1
     should have /usr/bin/jellyfish1
  2. Since kraken will not seek for /usr/bin/jellyfish1 we either
     need to put the jellyfish1 executable in /usr/lib and provide
     the binary above as link.  Than we could use a shell wrapper
     around kraken and set the PATH accordingly to make sure kraken
     will find the right executable.
     Alternatively we can simply patch kraken to use jellyfish1.
     This might be easier to debug to make really sure that kraken
     is using the correct version and it should definitely prevent
     that it will find the 2.2.6 version.

So far for my ideas.  If these are not helpful I need to check at
beginning of next week.

Kind regards

        Andreas.

--
http://fam-tille.de


Reply | Threaded
Open this post in threaded view
|

Re: CI for kraken

Andreas Tille-6
Hi Nadiya,

On Mon, Aug 07, 2017 at 12:55:21PM -0700, Nadiya Sitdykova wrote:
> 1. I don't know how to make it happen, maybe you can give some hints?
> 2. I think I know the place where jellyfish is called, so it shouldn't be a
> problem to patch kraken to use jellyfish1

Lets simply sit together later to get this done. :-)
 
Greetings to all who do not have the pleasure to be in Montreal
and meet great people (including newcomers like Nadiya)

     Andreas.

--
http://fam-tille.de