Bug#920545: python-intervaltree breaks python-intervaltree-bio autopkgtest

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug#920545: python-intervaltree breaks python-intervaltree-bio autopkgtest

Paul Gevers-4
Source: python-intervaltree, python-intervaltree-bio
Control: found -1 python-intervaltree/3.0.2-1
Control: found -1 python-intervaltree-bio/1.0.1-2
X-Debbugs-CC: [hidden email]
Severity: important
User: [hidden email]
Usertags: breaks needs-update

Dear maintainers,

With a recent upload of python-intervaltree the autopkgtest of
python-intervaltree-bio fails in testing when that autopkgtest is run
with the binary packages of python-intervaltree from unstable. It passes
when run with only packages from testing. In tabular form:
                        pass            fail
python-intervaltree     from testing    3.0.2-1
python-intervaltree-bio from testing    1.0.1-2
all others              from testing    from testing

I copied some of the output at the bottom of this report.

Currently this regression is blocking the migration of
python-intervaltree to testing [1]. Due to the nature of this issue, I
filed this bug report against both packages. Can you please investigate
the situation and reassign the bug to the right package? If needed,
please change the bug's severity.

More information about this bug and the reason for filing it can be found on
https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation

Paul

[1] https://qa.debian.org/excuses.php?package=python-intervaltree

https://ci.debian.net/data/autopkgtest/testing/amd64/p/python-intervaltree-bio/1785537/log.gz

=================================== FAILURES
===================================
________________________________ test_knownGene
________________________________

    def test_knownGene():
        # To speed up testing, we'll download the file and reuse the
downloaded copy
        knownGene_url =
'http://hgdownload.cse.ucsc.edu/goldenpath/hg19/database/knownGene.txt.gz'
        # Mirror. Slightly faster and more stable, I believe:
        knownGene_url =
'http://kt.era.ee/distribute/pyintervaltree/knownGene.txt.gz'

        # To speed up testing, we'll download the file and reuse the
downloaded copy
        knownGene_file, headers = urlretrieve(knownGene_url)

        knownGene_localurl = 'file:///%s' % os.path.abspath(knownGene_file)
        knownGene =
GenomeIntervalTree.from_table(url=knownGene_localurl, decompress=True) #
Py3 downloads .gz files to local files with names not ending with .gz
        assert len(knownGene) == 82960
>       result = knownGene[b'chr1'].search(100000, 138529)
E       AttributeError: 'IntervalTree' object has no attribute 'search'

.pc/offline-test-data.patch/tests/genomeintervaltree_test.py:28:
AttributeError
 test_knownGene[file:///tmp/autopkgtest-lxc.z3ra1nek/downtmp/build.Szw/src/debian/data/]

base_url =
'file:///tmp/autopkgtest-lxc.z3ra1nek/downtmp/build.Szw/src/debian/data/'

    def test_knownGene(base_url):
        # To speed up testing, we'll download the file and reuse the
downloaded copy
        knownGene_url = base_url + 'knownGene.txt.gz'

        # To speed up testing, we'll download the file and reuse the
downloaded copy
        knownGene_file, headers = urlretrieve(knownGene_url)

        knownGene_localurl = 'file:///%s' % os.path.abspath(knownGene_file)
        knownGene =
GenomeIntervalTree.from_table(url=knownGene_localurl, decompress=True) #
Py3 downloads .gz files to local files with names not ending with .gz
        assert len(knownGene) == 82960
>       result = knownGene[b'chr1'].search(100000, 138529)
E       AttributeError: 'IntervalTree' object has no attribute 'search'


signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#920545: python-intervaltree breaks python-intervaltree-bio autopkgtest

Hilko Bengen
control: tag -1 patch

* Paul Gevers:

> With a recent upload of python-intervaltree the autopkgtest of
> python-intervaltree-bio fails in testing when that autopkgtest is run
> with the binary packages of python-intervaltree from unstable. It passes
> when run with only packages from testing. In tabular form:
>                         pass            fail
> python-intervaltree     from testing    3.0.2-1
> python-intervaltree-bio from testing    1.0.1-2
> all others              from testing    from testing

Oops, my fault, sorry about that. And hooray for autopkgtests, I guess.

Apparently, the .search() method was removed in intervaltree-3.0.

Replacing all instances of .search(B,E) with .overlap(B,E) in the
intervalltree-bio test code makes the tests pass once more. Here's a
patch.

Cheers,
-Hilko

0001-Replace-all-.search-calls-with-.overlap-in-tests-and.patch (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#920545: closed by Andreas Tille <tille@debian.org> (Bug#920545: fixed in python-intervaltree-bio 1.0.1-3)

Paul Gevers-4
In reply to this post by Paul Gevers-4
Hi Andreas,

Thanks for picking this up so quickly, however...

On 27-01-2019 00:39, Debian Bug Tracking System wrote:
> #920545: python-intervaltree breaks python-intervaltree-bio autopkgtest

the test now fails with the following error, which seems to result from
the fact that you patched a test file:

==================================== ERRORS
====================================
 ERROR collecting
.pc/offline-test-data.patch/tests/genomeintervaltree_test.py _
import file mismatch:
imported module 'genomeintervaltree_test' has this __file__ attribute:

/tmp/autopkgtest-lxc.ozcajq0h/downtmp/build.kh1/src/.pc/0001-Replace-all-.search-calls-with-.overlap-in-tests-and.patch/tests/genomeintervaltree_test.py
which is not the same as the test file we want to collect:

/tmp/autopkgtest-lxc.ozcajq0h/downtmp/build.kh1/src/.pc/offline-test-data.patch/tests/genomeintervaltree_test.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for
your test file modules
=============================== warnings summary
===============================

Paul


signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#920545: closed by Andreas Tille <tille@debian.org> (Bug#920545: fixed in python-intervaltree-bio 1.0.1-3)

Andreas Tille-2
Control: reopen -1
Control: tags -1 help upstream
Control: forwarded -1 https://github.com/konstantint/intervaltree-bio/issues/8

Hi Paul,

On Sun, Jan 27, 2019 at 09:32:33PM +0100, Paul Gevers wrote:

> Thanks for picking this up so quickly, however...
>
> On 27-01-2019 00:39, Debian Bug Tracking System wrote:
> > #920545: python-intervaltree breaks python-intervaltree-bio autopkgtest
>
> the test now fails with the following error, which seems to result from
> the fact that you patched a test file:
>
> ==================================== ERRORS
> ====================================
>  ERROR collecting
> .pc/offline-test-data.patch/tests/genomeintervaltree_test.py _
> import file mismatch:
> imported module 'genomeintervaltree_test' has this __file__ attribute:
>
> /tmp/autopkgtest-lxc.ozcajq0h/downtmp/build.kh1/src/.pc/0001-Replace-all-.search-calls-with-.overlap-in-tests-and.patch/tests/genomeintervaltree_test.py
> which is not the same as the test file we want to collect:
>
> /tmp/autopkgtest-lxc.ozcajq0h/downtmp/build.kh1/src/.pc/offline-test-data.patch/tests/genomeintervaltree_test.py
> HINT: remove __pycache__ / .pyc files and/or use a unique basename for
> your test file modules
> =============================== warnings summary
> ===============================

That's really strange.  I have no idea why this test is fiddling around
with filed in .pc (no idea why a test attempts to work on hidden
directories).  I have forwarded the issue upstream (but for sure any
patch from Debian is welcome.

Kind regards

      Andreas.


--
http://fam-tille.de

Reply | Threaded
Open this post in threaded view
|

Bug#920545: closed by Andreas Tille <tille@debian.org> (Bug#920545: fixed in python-intervaltree-bio 1.0.1-3)

Paul Gevers-4
Hi,

On Mon, 28 Jan 2019 08:36:39 +0100 Andreas Tille <[hidden email]> wrote:
> That's really strange.  I have no idea why this test is fiddling around
> with filed in .pc (no idea why a test attempts to work on hidden
> directories).  I have forwarded the issue upstream (but for sure any
> patch from Debian is welcome.

I have decided that I'll have this regression ignored to enable
python-intervaltree(|-bio) migration to buster. However, please keep on
trying to fix this issue as it will mean that it will not catch any
future regressions.

Paul


signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#920545: Fixed in git repo

Olivier Sallou-3
In reply to this post by Paul Gevers-4
Hi,
I have pushed a fix to get autopkgtests done in python-intervaltree-bio
I have not uploaded package, as we are in freeze transition, so patch is in git only
Commit: 4f67ea8b607fa47520b553ce8776a37cf62a1713

I ran locally autopkgtest with this patch and everything was fine

autopkgtest -B python-intervaltree-bio_1.0.1-4_all.deb  python-intervaltree-bio_1.0.1-4.dsc -- null

Olivier
Reply | Threaded
Open this post in threaded view
|

Bug#920545: python-intervaltree breaks python-intervaltree-bio autopkgtest

Graham Inggs-3
Hi Olivier

On 2019/03/09 17:56, olivier sallou wrote:
> I have pushed a fix to get autopkgtests done in python-intervaltree-bio
> I have not uploaded package, as we are in freeze transition, so patch is in
> git only
> Commit: 4f67ea8b607fa47520b553ce8776a37cf62a1713
> Link:
> https://salsa.debian.org/med-team/python-intervaltree-bio/commit/4f67ea8b607fa47520b553ce8776a37cf62a1713

I've just uploaded python-intervaltree-bio including this patch to
Ubuntu and confirm that it fixes the autopkgtest there.

python-intervaltree should get "Breaks: python-intervaltree-bio (<<
1.0.1-4)" or similar added.

Regards
Graham

Reply | Threaded
Open this post in threaded view
|

Bug#920545: [Debian-med-packaging] Bug#920545: python-intervaltree breaks python-intervaltree-bio autopkgtest

Graham Inggs-3
Hi Olivier

Paul says a targeted fix for the autopkgtest will be eligible for an unblock.

Regards
Graham

Reply | Threaded
Open this post in threaded view
|

Bug#920545: python-intervaltree breaks python-intervaltree-bio autopkgtest

Andreas Tille-2
In reply to this post by Graham Inggs-3
Hi Graham,

On Thu, Mar 28, 2019 at 04:58:38PM +0200, Graham Inggs wrote:

> Hi Olivier
>
> On 2019/03/09 17:56, olivier sallou wrote:
> > I have pushed a fix to get autopkgtests done in python-intervaltree-bio
> > I have not uploaded package, as we are in freeze transition, so patch is in
> > git only
> > Commit: 4f67ea8b607fa47520b553ce8776a37cf62a1713
> > Link:
> > https://salsa.debian.org/med-team/python-intervaltree-bio/commit/4f67ea8b607fa47520b553ce8776a37cf62a1713
>
> I've just uploaded python-intervaltree-bio including this patch to Ubuntu
> and confirm that it fixes the autopkgtest there.

I'm using a pbuilder hook to build packages and get:

autopkgtest [07:19:35]: test command2:  python3 -m pytest --ignore=.pc --verbose --datadir debian/data
autopkgtest [07:19:35]: test command2: [-----------------------
ImportError while loading conftest '/tmp/autopkgtest.chaq95/tree/conftest.py'.
py._path.local.LocalPath.ImportMismatchError: ('conftest', '/build/python-intervaltree-bio-1.0.1/conftest.py', local('/tmp/autopkgtest.chaq95/tree/conftest.py'))
autopkgtest [07:19:36]: test command2: -----------------------]
autopkgtest [07:19:36]: test command2:  - - - - - - - - - - results - - - - - - - - - -
command2             FAIL non-zero exit status 4
autopkgtest [07:19:36]: test command2:  - - - - - - - - - - stderr - - - - - - - - - -
ImportError while loading conftest '/tmp/autopkgtest.chaq95/tree/conftest.py'.
py._path.local.LocalPath.ImportMismatchError: ('conftest', '/build/python-intervaltree-bio-1.0.1/conftest.py', local('/tmp/autopkgtest.chaq95/tree/conftest.py'))
autopkgtest [07:19:36]: @@@@@@@@@@@@@@@@@@@@ summary
command1             FAIL non-zero exit status 4
command2             FAIL non-zero exit status 4
autopkgtest [07:19:36]: Binaries: resetting testbed apt configuration


with the current state in Git.  Are you sure you have exactly the status
from Git (now head at 86cc4e0eafcc1cd2e2916ecf7c476495f5c16f1f - but I
simply added myself to Uploaders - no change to test).

 
> python-intervaltree should get "Breaks: python-intervaltree-bio (<<
> 1.0.1-4)" or similar added.

That should probably be a separate bug report.

Kind regards

         Andreas.

--
http://fam-tille.de

Reply | Threaded
Open this post in threaded view
|

Bug#920545: python-intervaltree breaks python-intervaltree-bio autopkgtest

Graham Inggs-3
Hi Andreas

On 2019/03/29 09:45, Andreas Tille wrote:
> with the current state in Git.  Are you sure you have exactly the status
> from Git (now head at 86cc4e0eafcc1cd2e2916ecf7c476495f5c16f1f - but I
> simply added myself to Uploaders - no change to test).

I ran debdiff on what I uploaded to Ubuntu
https://launchpad.net/ubuntu/+source/python-intervaltree-bio/1.0.1-3ubuntu1
versus git @ 86cc4e0eafcc1cd2e2916ecf7c476495f5c16f1f and I confirm the
only changes were in changelog and uploaders.

>> python-intervaltree should get "Breaks: python-intervaltree-bio (<<
>> 1.0.1-4)" or similar added.
>
> That should probably be a separate bug report.

It's probably not worth an upload since the new python-intervaltree has
already migrated to testing.

Regards
Graham

Reply | Threaded
Open this post in threaded view
|

Bug#920545: python-intervaltree breaks python-intervaltree-bio autopkgtest

Andreas Tille-5
In reply to this post by Paul Gevers-4
Control: tags -1 help

Hi,

I removed Python2 support from this package in Git[1].  Unfortunately the
autopkgtest failure is for Python3.  Graham, I understood your
mail that you've found a solution - but I do not understand what
this might be.

Any hint / patch / team upload that would fix the package (preferably)
based on the latest status in Git[1] would be really welcome.

Kind regards

    Andreas.

[1] https://salsa.debian.org/med-team/python-intervaltree-bio

--
http://fam-tille.de