Bug#938501: Please port smalt to Python3 (Was: Bug#938501: smalt: Python2 removal in sid/bullseye)

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug#938501: Please port smalt to Python3 (Was: Bug#938501: smalt: Python2 removal in sid/bullseye)

Andreas Tille-5
Hi again,

I just took James Bonfield in CC since I'm unsure whether my mail might
have reached someone who is currently working at sanger.ac.uk.  Just to
stretch the importance of the issue have a look at the Python2 EOL clock:

   https://pythonclock.org/

In short: Please be so kind and verify the patch linked below and fix
the remaining issues with your deeper knowledge of the code.

Thanks a lot

      Andreas.


On Thu, Sep 05, 2019 at 03:39:35PM +0200, Andreas Tille wrote:

> Control: tags -1 upsteam
> Control: forwarded -1 Hannes Ponstingl <[hidden email]>
>
> Hi Hannes,
>
> as you can read below Debian will remove Python2 since it is EOL.
> I tried to port your test scripts using the 2to3 tool.  The result
> of the automatic conversion can be found here:
>
>     https://salsa.debian.org/med-team/smalt/blob/master/debian/patches/2to3.patch
>
> Unfortunately there are some remaining issues in the test suite which
> are probably not very hard to fix.  I wonder whether you might want
> to have a look at these and could prepare an official Python3 release
> of your nice tool.
>
> Here is the output of the test suite:
>
>
> PASS: splitReads_test.py
> PASS: results_split_test.py
> PASS: ouform_cigar_test.py
> Traceback (most recent call last):
>   File "./sample_test.py", line 154, in <module>
>     compare_mapping(oufilnam1, oufilnam3)
>   File "./sample_test.py", line 108, in compare_mapping
>     if cmp(cig1.qnam, cig2.qnam):
> NameError: name 'cmp' is not defined
> FAIL: sample_test.py
> PASS: cigar_test.py
> Discrepancy:
> cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
> cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
> FAIL: mthread_test.py
> Traceback (most recent call last):
>   File "./ioform_test.py", line 45, in <module>
>     samnam = df.unpack(READ_PREFIX + ".sam")
>   File "/build/smalt-0.7.6/test/testdata.py", line 71, in unpack
>     oufil.write(lin)
> TypeError: write() argument must be str, not bytes
> FAIL: ioform_test.py
> PASS: xali_test.py
> Traceback (most recent call last):
>   File "./bam_cigar_test.py", line 254, in <module>
>     isOK = testSAMfilesAreIdentical(sambamnam, samoufilnam)
>   File "./bam_cigar_test.py", line 149, in testSAMfilesAreIdentical
>     linA = infilA.readline()
>   File "/usr/lib/python3.7/codecs.py", line 322, in decode
>     (result, consumed) = self._buffer_decode(data, self.errors, final)
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 260: invalid start byte
> FAIL: bam_cigar_test.py
> =================================
> 4 of 9 tests failed
> Please report to [hidden email]
> =================================
>
>
> Kind regards
>
>       Andreas.
>
>
> On Fri, Aug 30, 2019 at 07:52:50AM +0000, Matthias Klose wrote:
> > Package: src:smalt
> > Version: 0.7.6-8
> > Severity: normal
> > Tags: sid bullseye
> > User: [hidden email]
> > Usertags: py2removal
> >
> > Python2 becomes end-of-live upstream, and Debian aims to remove
> > Python2 from the distribution, as discussed in
> > https://lists.debian.org/debian-python/2019/07/msg00080.html
> >
> > Your package either build-depends, depends on Python2, or uses Python2
> > in the autopkg tests.  Please stop using Python2, and fix this issue
> > by one of the following actions.
> >
> > - Convert your Package to Python3. This is the preferred option.  In
> >   case you are providing a Python module foo, please consider dropping
> >   the python-foo package, and only build a python3-foo package.  Please
> >   don't drop Python2 modules, which still have reverse dependencies,
> >   just document them.
> >  
> >   This is the preferred option.
> >
> > - If the package is dead upstream, cannot be converted or maintained
> >   in Debian, it should be removed from the distribution.  If the
> >   package still has reverse dependencies, raise the severity to
> >   "serious" and document the reverse dependencies with the BTS affects
> >   command.  If the package has no reverse dependencies, confirm that
> >   the package can be removed, reassign this issue to ftp.debian.org,
> >   make sure that the bug priority is set to normal and retitle the
> >   issue to "RM: PKG -- removal triggered by the Python2 removal".
> >
> > - If the package has still many users (popcon >= 300), or is needed to
> >   build another package which cannot be removed, document that by
> >   adding the "py2keep" user tag (not replacing the py2remove tag),
> >   using the [hidden email] user.  Also any
> >   dependencies on an unversioned python package (python, python-dev)
> >   must not be used, same with the python shebang.  These have to be
> >   replaced by python2/python2.7 dependencies and shebang.
> >
> >   This is the least preferred option.
> >
> > If the conversion or removal needs action on another package first,
> > please document the blocking by using the BTS affects command, like
> >
> >   affects <bug number of blocking py2removal bug> + src:smalt
> >
> > If there is no py2removal bug for that reverse-dependency, please file
> > a bug on this package (similar to this bug report).
> >
> > If there are questions, please refer to the wiki page for the removal:
> > https://wiki.debian.org/Python/2Removal, or ask for help on IRC
> > #debian-python, or the [hidden email] mailing list.
> >
> > _______________________________________________
> > Debian-med-packaging mailing list
> > [hidden email]
> > https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-med-packaging
>
> --
> http://fam-tille.de
>
> _______________________________________________
> Debian-med-packaging mailing list
> [hidden email]
> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-med-packaging

--
http://fam-tille.de

Reply | Threaded
Open this post in threaded view
|

Bug#938501: Please port smalt to Python3 (Was: Bug#938501: smalt: Python2 removal in sid/bullseye) [EXT]

James Bonfield
Hi all,

Hannes is still around, but has moved jobs.  I'm including Zemin Ning
in this reply whose team Hannes was in when he wrote Smalt.

I don't know the extent of the issue having not used Smalt myself, but
the main tool itself is C so I doubt the python bit is a major
component.

Regardless, it's not something I'm involved with personally.

James


On Tue, Oct 08, 2019 at 05:33:42PM +0200, Andreas Tille wrote:

> Hi again,
>
> I just took James Bonfield in CC since I'm unsure whether my mail might
> have reached someone who is currently working at sanger.ac.uk.  Just to
> stretch the importance of the issue have a look at the Python2 EOL clock:
>
>    https://urldefense.proofpoint.com/v2/url?u=https-3A__pythonclock.org_&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=1ojoaFsu0HVyuhxGkiUBSnPjQJLSXJQyv1jrE-FfnMw&e= 
>
> In short: Please be so kind and verify the patch linked below and fix
> the remaining issues with your deeper knowledge of the code.
>
> Thanks a lot
>
>       Andreas.
>
>
> On Thu, Sep 05, 2019 at 03:39:35PM +0200, Andreas Tille wrote:
> > Control: tags -1 upsteam
> > Control: forwarded -1 Hannes Ponstingl <[hidden email]>
> >
> > Hi Hannes,
> >
> > as you can read below Debian will remove Python2 since it is EOL.
> > I tried to port your test scripts using the 2to3 tool.  The result
> > of the automatic conversion can be found here:
> >
> >     https://urldefense.proofpoint.com/v2/url?u=https-3A__salsa.debian.org_med-2Dteam_smalt_blob_master_debian_patches_2to3.patch&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=j7oR-RSaMB5DM2SvBCK63XeKcftyATF9QB_A56HxSCs&e= 
> >
> > Unfortunately there are some remaining issues in the test suite which
> > are probably not very hard to fix.  I wonder whether you might want
> > to have a look at these and could prepare an official Python3 release
> > of your nice tool.
> >
> > Here is the output of the test suite:
> >
> >
> > PASS: splitReads_test.py
> > PASS: results_split_test.py
> > PASS: ouform_cigar_test.py
> > Traceback (most recent call last):
> >   File "./sample_test.py", line 154, in <module>
> >     compare_mapping(oufilnam1, oufilnam3)
> >   File "./sample_test.py", line 108, in compare_mapping
> >     if cmp(cig1.qnam, cig2.qnam):
> > NameError: name 'cmp' is not defined
> > FAIL: sample_test.py
> > PASS: cigar_test.py
> > Discrepancy:
> > cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
> > cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
> > FAIL: mthread_test.py
> > Traceback (most recent call last):
> >   File "./ioform_test.py", line 45, in <module>
> >     samnam = df.unpack(READ_PREFIX + ".sam")
> >   File "/build/smalt-0.7.6/test/testdata.py", line 71, in unpack
> >     oufil.write(lin)
> > TypeError: write() argument must be str, not bytes
> > FAIL: ioform_test.py
> > PASS: xali_test.py
> > Traceback (most recent call last):
> >   File "./bam_cigar_test.py", line 254, in <module>
> >     isOK = testSAMfilesAreIdentical(sambamnam, samoufilnam)
> >   File "./bam_cigar_test.py", line 149, in testSAMfilesAreIdentical
> >     linA = infilA.readline()
> >   File "/usr/lib/python3.7/codecs.py", line 322, in decode
> >     (result, consumed) = self._buffer_decode(data, self.errors, final)
> > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 260: invalid start byte
> > FAIL: bam_cigar_test.py
> > =================================
> > 4 of 9 tests failed
> > Please report to [hidden email]
> > =================================
> >
> >
> > Kind regards
> >
> >       Andreas.
> >
> >
> > On Fri, Aug 30, 2019 at 07:52:50AM +0000, Matthias Klose wrote:
> > > Package: src:smalt
> > > Version: 0.7.6-8
> > > Severity: normal
> > > Tags: sid bullseye
> > > User: [hidden email]
> > > Usertags: py2removal
> > >
> > > Python2 becomes end-of-live upstream, and Debian aims to remove
> > > Python2 from the distribution, as discussed in
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.debian.org_debian-2Dpython_2019_07_msg00080.html&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=Xgujf6sRlZuNDRpgPdw-HM2EVf1K2YRkzazDPAt9sK4&e= 
> > >
> > > Your package either build-depends, depends on Python2, or uses Python2
> > > in the autopkg tests.  Please stop using Python2, and fix this issue
> > > by one of the following actions.
> > >
> > > - Convert your Package to Python3. This is the preferred option.  In
> > >   case you are providing a Python module foo, please consider dropping
> > >   the python-foo package, and only build a python3-foo package.  Please
> > >   don't drop Python2 modules, which still have reverse dependencies,
> > >   just document them.
> > >  
> > >   This is the preferred option.
> > >
> > > - If the package is dead upstream, cannot be converted or maintained
> > >   in Debian, it should be removed from the distribution.  If the
> > >   package still has reverse dependencies, raise the severity to
> > >   "serious" and document the reverse dependencies with the BTS affects
> > >   command.  If the package has no reverse dependencies, confirm that
> > >   the package can be removed, reassign this issue to ftp.debian.org,
> > >   make sure that the bug priority is set to normal and retitle the
> > >   issue to "RM: PKG -- removal triggered by the Python2 removal".
> > >
> > > - If the package has still many users (popcon >= 300), or is needed to
> > >   build another package which cannot be removed, document that by
> > >   adding the "py2keep" user tag (not replacing the py2remove tag),
> > >   using the [hidden email] user.  Also any
> > >   dependencies on an unversioned python package (python, python-dev)
> > >   must not be used, same with the python shebang.  These have to be
> > >   replaced by python2/python2.7 dependencies and shebang.
> > >
> > >   This is the least preferred option.
> > >
> > > If the conversion or removal needs action on another package first,
> > > please document the blocking by using the BTS affects command, like
> > >
> > >   affects <bug number of blocking py2removal bug> + src:smalt
> > >
> > > If there is no py2removal bug for that reverse-dependency, please file
> > > a bug on this package (similar to this bug report).
> > >
> > > If there are questions, please refer to the wiki page for the removal:
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.debian.org_Python_2Removal&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=LtGkMxuth36_KHtGeql2HwKH7xEIKZ6BNAP0GOH9aDE&e= , or ask for help on IRC
> > > #debian-python, or the [hidden email] mailing list.
> > >
> > > _______________________________________________
> > > Debian-med-packaging mailing list
> > > [hidden email]
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=QNdw7jPPPozYeLfZMk5O-mjcTGurrKIidxWrdwjOlao&e= 
> >
> > --
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=vbfIEwXiPAwzyQefpXbHmZpdmW1CI-xSYZm96PkCsmw&e= 
> >
> > _______________________________________________
> > Debian-med-packaging mailing list
> > [hidden email]
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=QNdw7jPPPozYeLfZMk5O-mjcTGurrKIidxWrdwjOlao&e= 
>
> --
> https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=vbfIEwXiPAwzyQefpXbHmZpdmW1CI-xSYZm96PkCsmw&e= 

--
James Bonfield ([hidden email])
The Sanger Institute, Hinxton, Cambs, CB10 1SA


--
 The Wellcome Sanger Institute is operated by Genome Research
 Limited, a charity registered in England with number 1021457 and a
 company registered in England with number 2742969, whose registered
 office is 215 Euston Road, London, NW1 2BE.

Reply | Threaded
Open this post in threaded view
|

Bug#938501: Please port smalt to Python3 (Was: Bug#938501: smalt: Python2 removal in sid/bullseye) [EXT]

Andreas Tille-6
Hi James,

thanks for forwarding.  I agree that its probably a small issue
in the Python3 code.  Hope Zemin will be able to fix it soon.
@Zemin: Please feel free to ask for help from our Python team
in case you might run into any trouble.

Kind regards
    Andreas.


On Tue, Oct 08, 2019 at 05:24:47PM +0100, James Bonfield wrote:

> Hi all,
>
> Hannes is still around, but has moved jobs.  I'm including Zemin Ning
> in this reply whose team Hannes was in when he wrote Smalt.
>
> I don't know the extent of the issue having not used Smalt myself, but
> the main tool itself is C so I doubt the python bit is a major
> component.
>
> Regardless, it's not something I'm involved with personally.
>
> James
>
>
> On Tue, Oct 08, 2019 at 05:33:42PM +0200, Andreas Tille wrote:
> > Hi again,
> >
> > I just took James Bonfield in CC since I'm unsure whether my mail might
> > have reached someone who is currently working at sanger.ac.uk.  Just to
> > stretch the importance of the issue have a look at the Python2 EOL clock:
> >
> >    https://urldefense.proofpoint.com/v2/url?u=https-3A__pythonclock.org_&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=1ojoaFsu0HVyuhxGkiUBSnPjQJLSXJQyv1jrE-FfnMw&e= 
> >
> > In short: Please be so kind and verify the patch linked below and fix
> > the remaining issues with your deeper knowledge of the code.
> >
> > Thanks a lot
> >
> >       Andreas.
> >
> >
> > On Thu, Sep 05, 2019 at 03:39:35PM +0200, Andreas Tille wrote:
> > > Control: tags -1 upsteam
> > > Control: forwarded -1 Hannes Ponstingl <[hidden email]>
> > >
> > > Hi Hannes,
> > >
> > > as you can read below Debian will remove Python2 since it is EOL.
> > > I tried to port your test scripts using the 2to3 tool.  The result
> > > of the automatic conversion can be found here:
> > >
> > >     https://urldefense.proofpoint.com/v2/url?u=https-3A__salsa.debian.org_med-2Dteam_smalt_blob_master_debian_patches_2to3.patch&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=j7oR-RSaMB5DM2SvBCK63XeKcftyATF9QB_A56HxSCs&e= 
> > >
> > > Unfortunately there are some remaining issues in the test suite which
> > > are probably not very hard to fix.  I wonder whether you might want
> > > to have a look at these and could prepare an official Python3 release
> > > of your nice tool.
> > >
> > > Here is the output of the test suite:
> > >
> > >
> > > PASS: splitReads_test.py
> > > PASS: results_split_test.py
> > > PASS: ouform_cigar_test.py
> > > Traceback (most recent call last):
> > >   File "./sample_test.py", line 154, in <module>
> > >     compare_mapping(oufilnam1, oufilnam3)
> > >   File "./sample_test.py", line 108, in compare_mapping
> > >     if cmp(cig1.qnam, cig2.qnam):
> > > NameError: name 'cmp' is not defined
> > > FAIL: sample_test.py
> > > PASS: cigar_test.py
> > > Discrepancy:
> > > cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
> > > cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
> > > FAIL: mthread_test.py
> > > Traceback (most recent call last):
> > >   File "./ioform_test.py", line 45, in <module>
> > >     samnam = df.unpack(READ_PREFIX + ".sam")
> > >   File "/build/smalt-0.7.6/test/testdata.py", line 71, in unpack
> > >     oufil.write(lin)
> > > TypeError: write() argument must be str, not bytes
> > > FAIL: ioform_test.py
> > > PASS: xali_test.py
> > > Traceback (most recent call last):
> > >   File "./bam_cigar_test.py", line 254, in <module>
> > >     isOK = testSAMfilesAreIdentical(sambamnam, samoufilnam)
> > >   File "./bam_cigar_test.py", line 149, in testSAMfilesAreIdentical
> > >     linA = infilA.readline()
> > >   File "/usr/lib/python3.7/codecs.py", line 322, in decode
> > >     (result, consumed) = self._buffer_decode(data, self.errors, final)
> > > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 260: invalid start byte
> > > FAIL: bam_cigar_test.py
> > > =================================
> > > 4 of 9 tests failed
> > > Please report to [hidden email]
> > > =================================
> > >
> > >
> > > Kind regards
> > >
> > >       Andreas.
> > >
> > >
> > > On Fri, Aug 30, 2019 at 07:52:50AM +0000, Matthias Klose wrote:
> > > > Package: src:smalt
> > > > Version: 0.7.6-8
> > > > Severity: normal
> > > > Tags: sid bullseye
> > > > User: [hidden email]
> > > > Usertags: py2removal
> > > >
> > > > Python2 becomes end-of-live upstream, and Debian aims to remove
> > > > Python2 from the distribution, as discussed in
> > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.debian.org_debian-2Dpython_2019_07_msg00080.html&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=Xgujf6sRlZuNDRpgPdw-HM2EVf1K2YRkzazDPAt9sK4&e= 
> > > >
> > > > Your package either build-depends, depends on Python2, or uses Python2
> > > > in the autopkg tests.  Please stop using Python2, and fix this issue
> > > > by one of the following actions.
> > > >
> > > > - Convert your Package to Python3. This is the preferred option.  In
> > > >   case you are providing a Python module foo, please consider dropping
> > > >   the python-foo package, and only build a python3-foo package.  Please
> > > >   don't drop Python2 modules, which still have reverse dependencies,
> > > >   just document them.
> > > >  
> > > >   This is the preferred option.
> > > >
> > > > - If the package is dead upstream, cannot be converted or maintained
> > > >   in Debian, it should be removed from the distribution.  If the
> > > >   package still has reverse dependencies, raise the severity to
> > > >   "serious" and document the reverse dependencies with the BTS affects
> > > >   command.  If the package has no reverse dependencies, confirm that
> > > >   the package can be removed, reassign this issue to ftp.debian.org,
> > > >   make sure that the bug priority is set to normal and retitle the
> > > >   issue to "RM: PKG -- removal triggered by the Python2 removal".
> > > >
> > > > - If the package has still many users (popcon >= 300), or is needed to
> > > >   build another package which cannot be removed, document that by
> > > >   adding the "py2keep" user tag (not replacing the py2remove tag),
> > > >   using the [hidden email] user.  Also any
> > > >   dependencies on an unversioned python package (python, python-dev)
> > > >   must not be used, same with the python shebang.  These have to be
> > > >   replaced by python2/python2.7 dependencies and shebang.
> > > >
> > > >   This is the least preferred option.
> > > >
> > > > If the conversion or removal needs action on another package first,
> > > > please document the blocking by using the BTS affects command, like
> > > >
> > > >   affects <bug number of blocking py2removal bug> + src:smalt
> > > >
> > > > If there is no py2removal bug for that reverse-dependency, please file
> > > > a bug on this package (similar to this bug report).
> > > >
> > > > If there are questions, please refer to the wiki page for the removal:
> > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.debian.org_Python_2Removal&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=LtGkMxuth36_KHtGeql2HwKH7xEIKZ6BNAP0GOH9aDE&e= , or ask for help on IRC
> > > > #debian-python, or the [hidden email] mailing list.
> > > >
> > > > _______________________________________________
> > > > Debian-med-packaging mailing list
> > > > [hidden email]
> > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=QNdw7jPPPozYeLfZMk5O-mjcTGurrKIidxWrdwjOlao&e= 
> > >
> > > --
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=vbfIEwXiPAwzyQefpXbHmZpdmW1CI-xSYZm96PkCsmw&e= 
> > >
> > > _______________________________________________
> > > Debian-med-packaging mailing list
> > > [hidden email]
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=QNdw7jPPPozYeLfZMk5O-mjcTGurrKIidxWrdwjOlao&e= 
> >
> > --
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=vbfIEwXiPAwzyQefpXbHmZpdmW1CI-xSYZm96PkCsmw&e= 
>
> --
> James Bonfield ([hidden email])
> The Sanger Institute, Hinxton, Cambs, CB10 1SA
>
>
> --
>  The Wellcome Sanger Institute is operated by Genome Research
>  Limited, a charity registered in England with number 1021457 and a
>  company registered in England with number 2742969, whose registered
>  office is 215 Euston Road, London, NW1 2BE.
>

--
http://fam-tille.de

Reply | Threaded
Open this post in threaded view
|

Bug#938501: Please port smalt to Python3 (Was: Bug#938501: smalt: Python2 removal in sid/bullseye) [EXT]

Hannes Ponstingl
In reply to this post by Andreas Tille-5
Hi Andreas,

Thank you for pointing me to this issue once more.
Please bear with me until I get a moment to check the patch/update the SMALT test suite to python 3.

It is not a problem, but I have to slot this in with the main obligations of my current role.

Kind regards,
Hannes

On 08/10/2019, 16:33, "Andreas Tille" <[hidden email]> wrote:

    Hi again,
   
    I just took James Bonfield in CC since I'm unsure whether my mail might
    have reached someone who is currently working at sanger.ac.uk.  Just to
    stretch the importance of the issue have a look at the Python2 EOL clock:
   
       https://urldefense.proofpoint.com/v2/url?u=https-3A__pythonclock.org_&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=XTh5PlieT8KhSRAcJV6w_A&m=KXRG53toHduo0G9U-G8lXQ7R1ZFLgHXlL6pjvR12rzI&s=g-PIxNtQvdPqurLJIeCBAx6cbYUpsQj2kEZ57K5Dhks&e= 
   
    In short: Please be so kind and verify the patch linked below and fix
    the remaining issues with your deeper knowledge of the code.
   
    Thanks a lot
   
          Andreas.
   
   
    On Thu, Sep 05, 2019 at 03:39:35PM +0200, Andreas Tille wrote:
    > Control: tags -1 upsteam
    > Control: forwarded -1 Hannes Ponstingl <[hidden email]>
    >
    > Hi Hannes,
    >
    > as you can read below Debian will remove Python2 since it is EOL.
    > I tried to port your test scripts using the 2to3 tool.  The result
    > of the automatic conversion can be found here:
    >
    >     https://urldefense.proofpoint.com/v2/url?u=https-3A__salsa.debian.org_med-2Dteam_smalt_blob_master_debian_patches_2to3.patch&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=XTh5PlieT8KhSRAcJV6w_A&m=KXRG53toHduo0G9U-G8lXQ7R1ZFLgHXlL6pjvR12rzI&s=HE73hWE00-z91NAO-Of277mmEjoxLCnPh7ns2AfJt88&e= 
    >
    > Unfortunately there are some remaining issues in the test suite which
    > are probably not very hard to fix.  I wonder whether you might want
    > to have a look at these and could prepare an official Python3 release
    > of your nice tool.
    >
    > Here is the output of the test suite:
    >
    >
    > PASS: splitReads_test.py
    > PASS: results_split_test.py
    > PASS: ouform_cigar_test.py
    > Traceback (most recent call last):
    >   File "./sample_test.py", line 154, in <module>
    >     compare_mapping(oufilnam1, oufilnam3)
    >   File "./sample_test.py", line 108, in compare_mapping
    >     if cmp(cig1.qnam, cig2.qnam):
    > NameError: name 'cmp' is not defined
    > FAIL: sample_test.py
    > PASS: cigar_test.py
    > Discrepancy:
    > cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
    > cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
    > FAIL: mthread_test.py
    > Traceback (most recent call last):
    >   File "./ioform_test.py", line 45, in <module>
    >     samnam = df.unpack(READ_PREFIX + ".sam")
    >   File "/build/smalt-0.7.6/test/testdata.py", line 71, in unpack
    >     oufil.write(lin)
    > TypeError: write() argument must be str, not bytes
    > FAIL: ioform_test.py
    > PASS: xali_test.py
    > Traceback (most recent call last):
    >   File "./bam_cigar_test.py", line 254, in <module>
    >     isOK = testSAMfilesAreIdentical(sambamnam, samoufilnam)
    >   File "./bam_cigar_test.py", line 149, in testSAMfilesAreIdentical
    >     linA = infilA.readline()
    >   File "/usr/lib/python3.7/codecs.py", line 322, in decode
    >     (result, consumed) = self._buffer_decode(data, self.errors, final)
    > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 260: invalid start byte
    > FAIL: bam_cigar_test.py
    > =================================
    > 4 of 9 tests failed
    > Please report to [hidden email]
    > =================================
    >
    >
    > Kind regards
    >
    >       Andreas.
    >
    >
    > On Fri, Aug 30, 2019 at 07:52:50AM +0000, Matthias Klose wrote:
    > > Package: src:smalt
    > > Version: 0.7.6-8
    > > Severity: normal
    > > Tags: sid bullseye
    > > User: [hidden email]
    > > Usertags: py2removal
    > >
    > > Python2 becomes end-of-live upstream, and Debian aims to remove
    > > Python2 from the distribution, as discussed in
    > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.debian.org_debian-2Dpython_2019_07_msg00080.html&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=XTh5PlieT8KhSRAcJV6w_A&m=KXRG53toHduo0G9U-G8lXQ7R1ZFLgHXlL6pjvR12rzI&s=HjxauC5_s56WX-Za34l9eg_sw_Guf5QO1udZvd6fZys&e= 
    > >
    > > Your package either build-depends, depends on Python2, or uses Python2
    > > in the autopkg tests.  Please stop using Python2, and fix this issue
    > > by one of the following actions.
    > >
    > > - Convert your Package to Python3. This is the preferred option.  In
    > >   case you are providing a Python module foo, please consider dropping
    > >   the python-foo package, and only build a python3-foo package.  Please
    > >   don't drop Python2 modules, which still have reverse dependencies,
    > >   just document them.
    > >  
    > >   This is the preferred option.
    > >
    > > - If the package is dead upstream, cannot be converted or maintained
    > >   in Debian, it should be removed from the distribution.  If the
    > >   package still has reverse dependencies, raise the severity to
    > >   "serious" and document the reverse dependencies with the BTS affects
    > >   command.  If the package has no reverse dependencies, confirm that
    > >   the package can be removed, reassign this issue to ftp.debian.org,
    > >   make sure that the bug priority is set to normal and retitle the
    > >   issue to "RM: PKG -- removal triggered by the Python2 removal".
    > >
    > > - If the package has still many users (popcon >= 300), or is needed to
    > >   build another package which cannot be removed, document that by
    > >   adding the "py2keep" user tag (not replacing the py2remove tag),
    > >   using the [hidden email] user.  Also any
    > >   dependencies on an unversioned python package (python, python-dev)
    > >   must not be used, same with the python shebang.  These have to be
    > >   replaced by python2/python2.7 dependencies and shebang.
    > >
    > >   This is the least preferred option.
    > >
    > > If the conversion or removal needs action on another package first,
    > > please document the blocking by using the BTS affects command, like
    > >
    > >   affects <bug number of blocking py2removal bug> + src:smalt
    > >
    > > If there is no py2removal bug for that reverse-dependency, please file
    > > a bug on this package (similar to this bug report).
    > >
    > > If there are questions, please refer to the wiki page for the removal:
    > > https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.debian.org_Python_2Removal&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=XTh5PlieT8KhSRAcJV6w_A&m=KXRG53toHduo0G9U-G8lXQ7R1ZFLgHXlL6pjvR12rzI&s=Flt2-tweYkXweqyy-FkEpeI7Jh_D46g4Cu2i2_EtLGE&e= , or ask for help on IRC
    > > #debian-python, or the [hidden email] mailing list.
    > >
    > > _______________________________________________
    > > Debian-med-packaging mailing list
    > > [hidden email]
    > > https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=XTh5PlieT8KhSRAcJV6w_A&m=KXRG53toHduo0G9U-G8lXQ7R1ZFLgHXlL6pjvR12rzI&s=hWA2JRzCt65NC5cOzhEUd0oJMyolBH6kYPj9J-puy0I&e= 
    >
    > --
    > https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=XTh5PlieT8KhSRAcJV6w_A&m=KXRG53toHduo0G9U-G8lXQ7R1ZFLgHXlL6pjvR12rzI&s=4X9FCUvuHdF6GJwGIPkIJXdQqNs1vPmb1NCxqVPlZjY&e= 
    >
    > _______________________________________________
    > Debian-med-packaging mailing list
    > [hidden email]
    > https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=XTh5PlieT8KhSRAcJV6w_A&m=KXRG53toHduo0G9U-G8lXQ7R1ZFLgHXlL6pjvR12rzI&s=hWA2JRzCt65NC5cOzhEUd0oJMyolBH6kYPj9J-puy0I&e= 
   
    --
    https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=XTh5PlieT8KhSRAcJV6w_A&m=KXRG53toHduo0G9U-G8lXQ7R1ZFLgHXlL6pjvR12rzI&s=4X9FCUvuHdF6GJwGIPkIJXdQqNs1vPmb1NCxqVPlZjY&e= 
   




--
 The Wellcome Sanger Institute is operated by Genome Research
 Limited, a charity registered in England with number 1021457 and a
 company registered in England with number 2742969, whose registered
 office is 215 Euston Road, London, NW1 2BE.
Reply | Threaded
Open this post in threaded view
|

Bug#938501: Please port smalt to Python3 (Was: Bug#938501: smalt: Python2 removal in sid/bullseye) [EXT]

Zemin Ning
In reply to this post by Andreas Tille-6
Thanks Andreas.

Hannes has moved to other teams a few years ago. In my memory, Python was
not used in the main SMALT code. However, it might be used in some testing
cases. I don't know if Hannes has the time to fix it.

Best regards,


Zemin

On Tue, 8 Oct 2019, Andreas Tille wrote:

> Hi James,
>
> thanks for forwarding.  I agree that its probably a small issue
> in the Python3 code.  Hope Zemin will be able to fix it soon.
> @Zemin: Please feel free to ask for help from our Python team
> in case you might run into any trouble.
>
> Kind regards
>    Andreas.
>
>
> On Tue, Oct 08, 2019 at 05:24:47PM +0100, James Bonfield wrote:
>> Hi all,
>>
>> Hannes is still around, but has moved jobs.  I'm including Zemin Ning
>> in this reply whose team Hannes was in when he wrote Smalt.
>>
>> I don't know the extent of the issue having not used Smalt myself, but
>> the main tool itself is C so I doubt the python bit is a major
>> component.
>>
>> Regardless, it's not something I'm involved with personally.
>>
>> James
>>
>>
>> On Tue, Oct 08, 2019 at 05:33:42PM +0200, Andreas Tille wrote:
>>> Hi again,
>>>
>>> I just took James Bonfield in CC since I'm unsure whether my mail might
>>> have reached someone who is currently working at sanger.ac.uk.  Just to
>>> stretch the importance of the issue have a look at the Python2 EOL clock:
>>>
>>>    https://urldefense.proofpoint.com/v2/url?u=https-3A__pythonclock.org_&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=1ojoaFsu0HVyuhxGkiUBSnPjQJLSXJQyv1jrE-FfnMw&e=
>>>
>>> In short: Please be so kind and verify the patch linked below and fix
>>> the remaining issues with your deeper knowledge of the code.
>>>
>>> Thanks a lot
>>>
>>>       Andreas.
>>>
>>>
>>> On Thu, Sep 05, 2019 at 03:39:35PM +0200, Andreas Tille wrote:
>>>> Control: tags -1 upsteam
>>>> Control: forwarded -1 Hannes Ponstingl <[hidden email]>
>>>>
>>>> Hi Hannes,
>>>>
>>>> as you can read below Debian will remove Python2 since it is EOL.
>>>> I tried to port your test scripts using the 2to3 tool.  The result
>>>> of the automatic conversion can be found here:
>>>>
>>>>     https://urldefense.proofpoint.com/v2/url?u=https-3A__salsa.debian.org_med-2Dteam_smalt_blob_master_debian_patches_2to3.patch&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=j7oR-RSaMB5DM2SvBCK63XeKcftyATF9QB_A56HxSCs&e=
>>>>
>>>> Unfortunately there are some remaining issues in the test suite which
>>>> are probably not very hard to fix.  I wonder whether you might want
>>>> to have a look at these and could prepare an official Python3 release
>>>> of your nice tool.
>>>>
>>>> Here is the output of the test suite:
>>>>
>>>>
>>>> PASS: splitReads_test.py
>>>> PASS: results_split_test.py
>>>> PASS: ouform_cigar_test.py
>>>> Traceback (most recent call last):
>>>>   File "./sample_test.py", line 154, in <module>
>>>>     compare_mapping(oufilnam1, oufilnam3)
>>>>   File "./sample_test.py", line 108, in compare_mapping
>>>>     if cmp(cig1.qnam, cig2.qnam):
>>>> NameError: name 'cmp' is not defined
>>>> FAIL: sample_test.py
>>>> PASS: cigar_test.py
>>>> Discrepancy:
>>>> cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
>>>> cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
>>>> FAIL: mthread_test.py
>>>> Traceback (most recent call last):
>>>>   File "./ioform_test.py", line 45, in <module>
>>>>     samnam = df.unpack(READ_PREFIX + ".sam")
>>>>   File "/build/smalt-0.7.6/test/testdata.py", line 71, in unpack
>>>>     oufil.write(lin)
>>>> TypeError: write() argument must be str, not bytes
>>>> FAIL: ioform_test.py
>>>> PASS: xali_test.py
>>>> Traceback (most recent call last):
>>>>   File "./bam_cigar_test.py", line 254, in <module>
>>>>     isOK = testSAMfilesAreIdentical(sambamnam, samoufilnam)
>>>>   File "./bam_cigar_test.py", line 149, in testSAMfilesAreIdentical
>>>>     linA = infilA.readline()
>>>>   File "/usr/lib/python3.7/codecs.py", line 322, in decode
>>>>     (result, consumed) = self._buffer_decode(data, self.errors, final)
>>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 260: invalid start byte
>>>> FAIL: bam_cigar_test.py
>>>> =================================
>>>> 4 of 9 tests failed
>>>> Please report to [hidden email]
>>>> =================================
>>>>
>>>>
>>>> Kind regards
>>>>
>>>>       Andreas.
>>>>
>>>>
>>>> On Fri, Aug 30, 2019 at 07:52:50AM +0000, Matthias Klose wrote:
>>>>> Package: src:smalt
>>>>> Version: 0.7.6-8
>>>>> Severity: normal
>>>>> Tags: sid bullseye
>>>>> User: [hidden email]
>>>>> Usertags: py2removal
>>>>>
>>>>> Python2 becomes end-of-live upstream, and Debian aims to remove
>>>>> Python2 from the distribution, as discussed in
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.debian.org_debian-2Dpython_2019_07_msg00080.html&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=Xgujf6sRlZuNDRpgPdw-HM2EVf1K2YRkzazDPAt9sK4&e=
>>>>>
>>>>> Your package either build-depends, depends on Python2, or uses Python2
>>>>> in the autopkg tests.  Please stop using Python2, and fix this issue
>>>>> by one of the following actions.
>>>>>
>>>>> - Convert your Package to Python3. This is the preferred option.  In
>>>>>   case you are providing a Python module foo, please consider dropping
>>>>>   the python-foo package, and only build a python3-foo package.  Please
>>>>>   don't drop Python2 modules, which still have reverse dependencies,
>>>>>   just document them.
>>>>>
>>>>>   This is the preferred option.
>>>>>
>>>>> - If the package is dead upstream, cannot be converted or maintained
>>>>>   in Debian, it should be removed from the distribution.  If the
>>>>>   package still has reverse dependencies, raise the severity to
>>>>>   "serious" and document the reverse dependencies with the BTS affects
>>>>>   command.  If the package has no reverse dependencies, confirm that
>>>>>   the package can be removed, reassign this issue to ftp.debian.org,
>>>>>   make sure that the bug priority is set to normal and retitle the
>>>>>   issue to "RM: PKG -- removal triggered by the Python2 removal".
>>>>>
>>>>> - If the package has still many users (popcon >= 300), or is needed to
>>>>>   build another package which cannot be removed, document that by
>>>>>   adding the "py2keep" user tag (not replacing the py2remove tag),
>>>>>   using the [hidden email] user.  Also any
>>>>>   dependencies on an unversioned python package (python, python-dev)
>>>>>   must not be used, same with the python shebang.  These have to be
>>>>>   replaced by python2/python2.7 dependencies and shebang.
>>>>>
>>>>>   This is the least preferred option.
>>>>>
>>>>> If the conversion or removal needs action on another package first,
>>>>> please document the blocking by using the BTS affects command, like
>>>>>
>>>>>   affects <bug number of blocking py2removal bug> + src:smalt
>>>>>
>>>>> If there is no py2removal bug for that reverse-dependency, please file
>>>>> a bug on this package (similar to this bug report).
>>>>>
>>>>> If there are questions, please refer to the wiki page for the removal:
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.debian.org_Python_2Removal&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=LtGkMxuth36_KHtGeql2HwKH7xEIKZ6BNAP0GOH9aDE&e= , or ask for help on IRC
>>>>> #debian-python, or the [hidden email] mailing list.
>>>>>
>>>>> _______________________________________________
>>>>> Debian-med-packaging mailing list
>>>>> [hidden email]
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=QNdw7jPPPozYeLfZMk5O-mjcTGurrKIidxWrdwjOlao&e=
>>>>
>>>> --
>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=vbfIEwXiPAwzyQefpXbHmZpdmW1CI-xSYZm96PkCsmw&e=
>>>>
>>>> _______________________________________________
>>>> Debian-med-packaging mailing list
>>>> [hidden email]
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=QNdw7jPPPozYeLfZMk5O-mjcTGurrKIidxWrdwjOlao&e=
>>>
>>> --
>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=vbfIEwXiPAwzyQefpXbHmZpdmW1CI-xSYZm96PkCsmw&e=
>>
>> --
>> James Bonfield ([hidden email])
>> The Sanger Institute, Hinxton, Cambs, CB10 1SA
>>
>>
>> --
>>  The Wellcome Sanger Institute is operated by Genome Research
>>  Limited, a charity registered in England with number 1021457 and a
>>  company registered in England with number 2742969, whose registered
>>  office is 215 Euston Road, London, NW1 2BE.
>>
>
> --
> https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=3I_p6ZxdWYkfke7BGXRytA&m=jvxTSvXv20nWY_BYG1f0G4HCsvR_lRdqywIhvCGBnMI&s=C0YBsqEJzoB7n-zxstKVL7D7AGjU73TWp2UYbiQpDrQ&e=
>


--
 The Wellcome Sanger Institute is operated by Genome Research
 Limited, a charity registered in England with number 1021457 and a
 company registered in England with number 2742969, whose registered
 office is 215 Euston Road, London, NW1 2BE.

Reply | Threaded
Open this post in threaded view
|

Bug#938501: Please port smalt to Python3 (Was: Bug#938501: smalt: Python2 removal in sid/bullseye) [EXT]

Hannes Ponstingl
Hi Zemin, James, Andreas,

Python was used for the test code (and internally there was a python interface to some of the C code).
I will update that to python 3 - not a problem, just bear with me.

Best,
Hannes

On 09/10/2019, 09:38, "Zemin Ning" <[hidden email]> wrote:

    Thanks Andreas.
   
    Hannes has moved to other teams a few years ago. In my memory, Python was
    not used in the main SMALT code. However, it might be used in some testing
    cases. I don't know if Hannes has the time to fix it.
   
    Best regards,
   
   
    Zemin
   
    On Tue, 8 Oct 2019, Andreas Tille wrote:
   
    > Hi James,
    >
    > thanks for forwarding.  I agree that its probably a small issue
    > in the Python3 code.  Hope Zemin will be able to fix it soon.
    > @Zemin: Please feel free to ask for help from our Python team
    > in case you might run into any trouble.
    >
    > Kind regards
    >    Andreas.
    >
    >
    > On Tue, Oct 08, 2019 at 05:24:47PM +0100, James Bonfield wrote:
    >> Hi all,
    >>
    >> Hannes is still around, but has moved jobs.  I'm including Zemin Ning
    >> in this reply whose team Hannes was in when he wrote Smalt.
    >>
    >> I don't know the extent of the issue having not used Smalt myself, but
    >> the main tool itself is C so I doubt the python bit is a major
    >> component.
    >>
    >> Regardless, it's not something I'm involved with personally.
    >>
    >> James
    >>
    >>
    >> On Tue, Oct 08, 2019 at 05:33:42PM +0200, Andreas Tille wrote:
    >>> Hi again,
    >>>
    >>> I just took James Bonfield in CC since I'm unsure whether my mail might
    >>> have reached someone who is currently working at sanger.ac.uk.  Just to
    >>> stretch the importance of the issue have a look at the Python2 EOL clock:
    >>>
    >>>    https://urldefense.proofpoint.com/v2/url?u=https-3A__pythonclock.org_&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=1ojoaFsu0HVyuhxGkiUBSnPjQJLSXJQyv1jrE-FfnMw&e=
    >>>
    >>> In short: Please be so kind and verify the patch linked below and fix
    >>> the remaining issues with your deeper knowledge of the code.
    >>>
    >>> Thanks a lot
    >>>
    >>>       Andreas.
    >>>
    >>>
    >>> On Thu, Sep 05, 2019 at 03:39:35PM +0200, Andreas Tille wrote:
    >>>> Control: tags -1 upsteam
    >>>> Control: forwarded -1 Hannes Ponstingl <[hidden email]>
    >>>>
    >>>> Hi Hannes,
    >>>>
    >>>> as you can read below Debian will remove Python2 since it is EOL.
    >>>> I tried to port your test scripts using the 2to3 tool.  The result
    >>>> of the automatic conversion can be found here:
    >>>>
    >>>>     https://urldefense.proofpoint.com/v2/url?u=https-3A__salsa.debian.org_med-2Dteam_smalt_blob_master_debian_patches_2to3.patch&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=j7oR-RSaMB5DM2SvBCK63XeKcftyATF9QB_A56HxSCs&e=
    >>>>
    >>>> Unfortunately there are some remaining issues in the test suite which
    >>>> are probably not very hard to fix.  I wonder whether you might want
    >>>> to have a look at these and could prepare an official Python3 release
    >>>> of your nice tool.
    >>>>
    >>>> Here is the output of the test suite:
    >>>>
    >>>>
    >>>> PASS: splitReads_test.py
    >>>> PASS: results_split_test.py
    >>>> PASS: ouform_cigar_test.py
    >>>> Traceback (most recent call last):
    >>>>   File "./sample_test.py", line 154, in <module>
    >>>>     compare_mapping(oufilnam1, oufilnam3)
    >>>>   File "./sample_test.py", line 108, in compare_mapping
    >>>>     if cmp(cig1.qnam, cig2.qnam):
    >>>> NameError: name 'cmp' is not defined
    >>>> FAIL: sample_test.py
    >>>> PASS: cigar_test.py
    >>>> Discrepancy:
    >>>> cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
    >>>> cigar:A:60 SIM_000000000_MAL11_001337747_10_F_75m/1 1 75 + MAL11 1337747 1337821 + 75 M 75
    >>>> FAIL: mthread_test.py
    >>>> Traceback (most recent call last):
    >>>>   File "./ioform_test.py", line 45, in <module>
    >>>>     samnam = df.unpack(READ_PREFIX + ".sam")
    >>>>   File "/build/smalt-0.7.6/test/testdata.py", line 71, in unpack
    >>>>     oufil.write(lin)
    >>>> TypeError: write() argument must be str, not bytes
    >>>> FAIL: ioform_test.py
    >>>> PASS: xali_test.py
    >>>> Traceback (most recent call last):
    >>>>   File "./bam_cigar_test.py", line 254, in <module>
    >>>>     isOK = testSAMfilesAreIdentical(sambamnam, samoufilnam)
    >>>>   File "./bam_cigar_test.py", line 149, in testSAMfilesAreIdentical
    >>>>     linA = infilA.readline()
    >>>>   File "/usr/lib/python3.7/codecs.py", line 322, in decode
    >>>>     (result, consumed) = self._buffer_decode(data, self.errors, final)
    >>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 260: invalid start byte
    >>>> FAIL: bam_cigar_test.py
    >>>> =================================
    >>>> 4 of 9 tests failed
    >>>> Please report to [hidden email]
    >>>> =================================
    >>>>
    >>>>
    >>>> Kind regards
    >>>>
    >>>>       Andreas.
    >>>>
    >>>>
    >>>> On Fri, Aug 30, 2019 at 07:52:50AM +0000, Matthias Klose wrote:
    >>>>> Package: src:smalt
    >>>>> Version: 0.7.6-8
    >>>>> Severity: normal
    >>>>> Tags: sid bullseye
    >>>>> User: [hidden email]
    >>>>> Usertags: py2removal
    >>>>>
    >>>>> Python2 becomes end-of-live upstream, and Debian aims to remove
    >>>>> Python2 from the distribution, as discussed in
    >>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.debian.org_debian-2Dpython_2019_07_msg00080.html&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=Xgujf6sRlZuNDRpgPdw-HM2EVf1K2YRkzazDPAt9sK4&e=
    >>>>>
    >>>>> Your package either build-depends, depends on Python2, or uses Python2
    >>>>> in the autopkg tests.  Please stop using Python2, and fix this issue
    >>>>> by one of the following actions.
    >>>>>
    >>>>> - Convert your Package to Python3. This is the preferred option.  In
    >>>>>   case you are providing a Python module foo, please consider dropping
    >>>>>   the python-foo package, and only build a python3-foo package.  Please
    >>>>>   don't drop Python2 modules, which still have reverse dependencies,
    >>>>>   just document them.
    >>>>>
    >>>>>   This is the preferred option.
    >>>>>
    >>>>> - If the package is dead upstream, cannot be converted or maintained
    >>>>>   in Debian, it should be removed from the distribution.  If the
    >>>>>   package still has reverse dependencies, raise the severity to
    >>>>>   "serious" and document the reverse dependencies with the BTS affects
    >>>>>   command.  If the package has no reverse dependencies, confirm that
    >>>>>   the package can be removed, reassign this issue to ftp.debian.org,
    >>>>>   make sure that the bug priority is set to normal and retitle the
    >>>>>   issue to "RM: PKG -- removal triggered by the Python2 removal".
    >>>>>
    >>>>> - If the package has still many users (popcon >= 300), or is needed to
    >>>>>   build another package which cannot be removed, document that by
    >>>>>   adding the "py2keep" user tag (not replacing the py2remove tag),
    >>>>>   using the [hidden email] user.  Also any
    >>>>>   dependencies on an unversioned python package (python, python-dev)
    >>>>>   must not be used, same with the python shebang.  These have to be
    >>>>>   replaced by python2/python2.7 dependencies and shebang.
    >>>>>
    >>>>>   This is the least preferred option.
    >>>>>
    >>>>> If the conversion or removal needs action on another package first,
    >>>>> please document the blocking by using the BTS affects command, like
    >>>>>
    >>>>>   affects <bug number of blocking py2removal bug> + src:smalt
    >>>>>
    >>>>> If there is no py2removal bug for that reverse-dependency, please file
    >>>>> a bug on this package (similar to this bug report).
    >>>>>
    >>>>> If there are questions, please refer to the wiki page for the removal:
    >>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.debian.org_Python_2Removal&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=LtGkMxuth36_KHtGeql2HwKH7xEIKZ6BNAP0GOH9aDE&e= , or ask for help on IRC
    >>>>> #debian-python, or the [hidden email] mailing list.
    >>>>>
    >>>>> _______________________________________________
    >>>>> Debian-med-packaging mailing list
    >>>>> [hidden email]
    >>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=QNdw7jPPPozYeLfZMk5O-mjcTGurrKIidxWrdwjOlao&e=
    >>>>
    >>>> --
    >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=vbfIEwXiPAwzyQefpXbHmZpdmW1CI-xSYZm96PkCsmw&e=
    >>>>
    >>>> _______________________________________________
    >>>> Debian-med-packaging mailing list
    >>>> [hidden email]
    >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__alioth-2Dlists.debian.net_cgi-2Dbin_mailman_listinfo_debian-2Dmed-2Dpackaging&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=QNdw7jPPPozYeLfZMk5O-mjcTGurrKIidxWrdwjOlao&e=
    >>>
    >>> --
    >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=wodoR_G062E4YLZ-xu5t6g&m=XK--XBUkRyktzXOkgsZpq_yu16VL7H7Y75QlpOEnwGc&s=vbfIEwXiPAwzyQefpXbHmZpdmW1CI-xSYZm96PkCsmw&e=
    >>
    >> --
    >> James Bonfield ([hidden email])
    >> The Sanger Institute, Hinxton, Cambs, CB10 1SA
    >>
    >>
    >> --
    >>  The Wellcome Sanger Institute is operated by Genome Research
    >>  Limited, a charity registered in England with number 1021457 and a
    >>  company registered in England with number 2742969, whose registered
    >>  office is 215 Euston Road, London, NW1 2BE.
    >>
    >
    > --
    > https://urldefense.proofpoint.com/v2/url?u=http-3A__fam-2Dtille.de&d=DwIBAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=3I_p6ZxdWYkfke7BGXRytA&m=jvxTSvXv20nWY_BYG1f0G4HCsvR_lRdqywIhvCGBnMI&s=C0YBsqEJzoB7n-zxstKVL7D7AGjU73TWp2UYbiQpDrQ&e=
    >
   




--
 The Wellcome Sanger Institute is operated by Genome Research
 Limited, a charity registered in England with number 1021457 and a
 company registered in England with number 2742969, whose registered
 office is 215 Euston Road, London, NW1 2BE.