dh_python2 and large /usr/share/pyshared

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

dh_python2 and large /usr/share/pyshared

Ole Streicher-2
Hi,

I am trying to create a python package (astropy) that has a few arch
specific modules built (.so files), and a rather large amount of shared
code in /usr/share/pyshared/. Therefore, lintian complains (well: rather
informs) about

I: python-astropy: arch-dep-package-has-big-usr-share 4137kB 87%

How do I create an arch independend package that contains these files?
Since in tmp/ everything is still in usr/lib/, I cannot just put a

/usr/share/pyshared

into the .install file.

Best regards

Ole


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/87d343rkhe.fsf@...

Reply | Threaded
Open this post in threaded view
|

Re: dh_python2 and large /usr/share/pyshared

Jakub Wilk-4
* Olе Streicher <[hidden email]>, 2012-07-10, 12:14:

>I am trying to create a python package (astropy) that has a few arch
>specific modules built (.so files), and a rather large amount of shared
>code in /usr/share/pyshared/. Therefore, lintian complains (well:
>rather informs) about
>
>I: python-astropy: arch-dep-package-has-big-usr-share 4137kB 87%
>
>How do I create an arch independend package that contains these files?
>Since in tmp/ everything is still in usr/lib/, I cannot just put a
>
>/usr/share/pyshared
>
>into the .install file.

I know NeuroDebian folks have been splitting their packages this way
(see e.g. pymvpa2 debian/rules).

But to be frank, I'd rather not do that. You will likely end up with
either dependency loop or a package that is not useful for anything when
installed alone.

--
Jakub Wilk


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20120710103704.GA4844@...

Reply | Threaded
Open this post in threaded view
|

Re: dh_python2 and large /usr/share/pyshared

Ole Streicher-2
Jakub Wilk <[hidden email]> writes:
> * Olе Streicher <[hidden email]>, 2012-07-10, 12:14:
>>I: python-astropy: arch-dep-package-has-big-usr-share 4137kB 87%

>> How do I create an arch independend package that contains these files?

> [...] I'd rather not do that. You will likely end up with either
> dependency loop or a package that is not useful for anything when
> installed alone.

Debian Developer's Reference section 6.7.5 (Architecture-independent
data) recommends to do so, and I would just trust Lintian's
recommendation here.

Or are they outdated with todays disk capacities?

Best

Ole


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/878verrj0u.fsf@...

Reply | Threaded
Open this post in threaded view
|

Re: dh_python2 and large /usr/share/pyshared

Scott Kitterman-5


[hidden email] wrote

>Jakub Wilk <[hidden email]> writes:
>> * Olе Streicher <[hidden email]>, 2012-07-10, 12:14:
>>>I: python-astropy: arch-dep-package-has-big-usr-share 4137kB 87%
>
>>> How do I create an arch independend package that contains these
>files?
>
>> [...] I'd rather not do that. You will likely end up with either
>> dependency loop or a package that is not useful for anything when
>> installed alone.
>
>Debian Developer's Reference section 6.7.5 (Architecture-independent
>data) recommends to do so, and I would just trust Lintian's
>recommendation here.
>
>Or are they outdated with todays disk capacities?

Neither devref nor Lintian are suicide pacts. It's good general advice, but not to the point of creating dependency loops or broken packages.

Scott K


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/fce11db8-489c-4442-bb2e-8c6a7c01e72b@...

Reply | Threaded
Open this post in threaded view
|

Re: dh_python2 and large /usr/share/pyshared

Ole Streicher-2
Scott Kitterman <[hidden email]> writes:
>>Debian Developer's Reference section 6.7.5 (Architecture-independent
>>data) recommends to do so, and I would just trust Lintian's
>>recommendation here.
>>
>>Or are they outdated with todays disk capacities?
>
> Neither devref nor Lintian are suicide pacts. It's good general
> advice, but not to the point of creating dependency loops or broken
> packages.

My case looks exactly like what devref and Lintian write about:
significant amount of architecture-dependent data. The problem of
dependency loops or broken packages is also quite common in this case:
Just by looking through packages that end with "n-data", the first I
found is f.e.  "amsn-data" [1], which is only usable with the "amsn"
package, but doesn't have it in its dependency list. The case that the
-data file is really usable independent from the main package seems to
be a rare special case.

I don't see a fundamental difference between this example and my one --
except that amsn-data is 29 MB, any in my case this would be ~4 MB; but
there I would trust the limit that is set in lint.

Since I am still quite a newcomer: what is the problem with dependency
loops, especially as long as they are within the same source package?

Independent of this, my question remains how one would create an
architecture independend subpackage with dh_python2.

Best regards

Ole

--
[1] http://packages.debian.org/squeeze/amsn-data
    more examples: gmerlin-data, hugin-data, john-data,
    konversation-data, nateon-data, ...


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/ytzobnn3gcq.fsf@...

Reply | Threaded
Open this post in threaded view
|

Re: dh_python2 and large /usr/share/pyshared

Yaroslav Halchenko
In reply to this post by Scott Kitterman-5

On Tue, 10 Jul 2012, Scott Kitterman wrote:

> >>>I: python-astropy: arch-dep-package-has-big-usr-share 4137kB 87%
> >>> How do I create an arch independend package that contains these
> >files?
> >> [...] I'd rather not do that. You will likely end up with either
> >> dependency loop or a package that is not useful for anything when
> >> installed alone.
> >Debian Developer's Reference section 6.7.5 (Architecture-independent
> >data) recommends to do so, and I would just trust Lintian's
> >recommendation here.
> >Or are they outdated with todays disk capacities?

> Neither devref nor Lintian are suicide pacts. It's good general advice, but not to the point of creating dependency loops or broken packages.

since our works were mentioned ;) -- yes, we generally split out python
extensions (and anything else arch-dep) into python-*-lib packages
primarily to minimize impact on the archive where we could achieve that
-- we have a dozen of ports, so any MB makes ten-fold impact (and then
do not forget snapshot.d.o).

To overcome any dependency loop, arch:all depends on arch:any -lib and
"broken packages" indeed could happen with annoying one when
arch:any is installed without arch:all package.  Then module is
'visible' since all the paths leading there present but most probably
would not be importable/usable.  It is easily resolvable (through
installation of the corresponding arch:all package) and I rarely hear
about it this from users.  That is why I keep following this scheme for
any new package of such kind despite that it requires a bit of ad-hoc
actions in debian/rules.

Hope this of some help ;-)

--
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik       


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20120711141651.GO21226@...

Reply | Threaded
Open this post in threaded view
|

suffix for packages with (optional?) Python extensions

Piotr Ozarowski-3
[Yaroslav Halchenko, 2012-07-11]
> since our works were mentioned ;) -- yes, we generally split out python
> extensions (and anything else arch-dep) into python-*-lib packages

can we agree on a common suffix for such¹ packages and add a suggestion
to Debian Python Policy?

I use -ext (python-sqlalchemy-ext) but now I see that there are also
-accel (python-reportlab-accel) and -lib (python-guppy-lib)

[¹] packages with split out (optional?) Python extensions
--
Piotr Ożarowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Stefano Rivera-15
Hi Piotr (2012.07.13_05:00:45_+0200)
> can we agree on a common suffix for such¹ packages and add a suggestion
> to Debian Python Policy?
>
> I use -ext (python-sqlalchemy-ext) but now I see that there are also
> -accel (python-reportlab-accel) and -lib (python-guppy-lib)

I also know of a -helpers (veusz-helpers)

+1 to some standardisation. Propose a policy patch?

SR

--
Stefano Rivera
  http://tumbleweed.org.za/
  H: +27 21 461 1230 C: +27 72 419 8559


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20120713032033.GD1912@...

Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Yaroslav Halchenko
In reply to this post by Piotr Ozarowski-3
> can we agree on a common suffix for such¹ packages and add a suggestion
> to Debian Python Policy?

> I use -ext (python-sqlalchemy-ext) but now I see that there are also
> -accel (python-reportlab-accel) and -lib (python-guppy-lib)

good idea... Now I somewhat like -ext more than -lib -- it is more Pythonic --
but I guess I am the biggest abuser with the -lib (codespeak is not mine).
Also -lib might actually be more factual -- it is not necessarily native
Pythonic extensions which would be provided there but might be other .so's
(native dynamic libraries etc -- just look inside python-numpy since
python-numpy-ext is apparently a transitional package since 2007 I found on my
drive ;) )

$> dpkg -l python-*-{lib,ext,accel}
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                     Version           Architecture      Description
+++-========================-=================-=================-======================================================
ii  python-brian-lib         1.3.1-1+b1        amd64             simulator for spiking neural networks -- extensions
ii  python-bson-ext          2.2-2             amd64             C-coded extension to the python-bson package
un  python-codespeak-lib     <none>                              (no description available)
ii  python-dipy-lib          0.5.0-3           amd64             toolbox for analysis of MR diffusion imaging data -- e
un  python-guppy-lib         <none>                              (no description available)
ii  python-mlpy-lib          2.2.0~dfsg1-2+b1  amd64             low-level implementations and bindings for mlpy
ii  python-mvpa-lib          0.4.8-1           amd64             low-level implementations and bindings for PyMVPA
ii  python-mvpa2-lib         2.1.0-1           amd64             low-level implementations and bindings for PyMVPA v. 2
ii  python-numpy-ext         1:1.3.0-3         all               NumPy adds a fast array facility to the Python languag
ii  python-pandas-lib        0.8.0-1           amd64             low-level implementations and bindings for pandas
ii  python-pymongo-ext       2.2-2             amd64             C-coded extension to the python-pymongo package
ii  python-reportlab-accel   2.5-1.1           amd64             C coded extension accelerator for the ReportLab Toolki
un  python-scikits-learn-lib <none>                              (no description available)
ii  python-skimage-lib       0.6-1             amd64             Optimized low-level algorithms for scikits-image
ii  python-sklearn-lib       0.11.0-2          amd64             low-level implementations and bindings for scikit-lear
ii  python-sqlalchemy-ext    0.7.8-1           amd64             SQL toolkit and Object Relational Mapper for Python -
ii  python-statsmodels-lib   0.4.0-2           amd64             low-level implementations and bindings for statsmodels



On Thu, 12 Jul 2012, Piotr Ożarowski wrote:

> [Yaroslav Halchenko, 2012-07-11]
> > since our works were mentioned ;) -- yes, we generally split out python
> > extensions (and anything else arch-dep) into python-*-lib packages


> [¹] packages with split out (optional?) Python extensions
--
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik       


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20120713033030.GU21226@...

Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Barry Warsaw
In reply to this post by Piotr Ozarowski-3
On Jul 12, 2012, at 09:00 PM, Piotr Ożarowski wrote:

>can we agree on a common suffix for such¹ packages and add a suggestion
>to Debian Python Policy?

+1

>I use -ext (python-sqlalchemy-ext) but now I see that there are also
>-accel (python-reportlab-accel) and -lib (python-guppy-lib)

I suppose I have a mild preference for the -ext shade of bikeshed.

-Barry

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Scott Kitterman-5
On Friday, July 13, 2012 11:48:57 AM Barry Warsaw wrote:

> On Jul 12, 2012, at 09:00 PM, Piotr Ożarowski wrote:
> >can we agree on a common suffix for such¹ packages and add a suggestion
> >to Debian Python Policy?
>
> +1
>
> >I use -ext (python-sqlalchemy-ext) but now I see that there are also
> >-accel (python-reportlab-accel) and -lib (python-guppy-lib)
>
> I suppose I have a mild preference for the -ext shade of bikeshed.

To make sure I understand this ...

For packages with a large mix of arch all Python and arch any extensions, they
could be split into two binaries:

1.  python{3}-foo which is arch all and follows the current naming convention
of foo being the name you import.  It would depend on the arch any python-foo-
ext package.

2.  python{3}-foo-ext which is arch any and Suggests/Recommends (not sure
which) python{3}-foo.

Is that right?

How would this work for packages where the extension is not built on all
architectures?

Scott K


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/1366059.DrSicR9nDd@scott-latitude-e6320

Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Ben Finney-5
In reply to this post by Barry Warsaw
Barry Warsaw <[hidden email]> writes:

> On Jul 12, 2012, at 09:00 PM, Piotr Ożarowski wrote:
> >I use -ext (python-sqlalchemy-ext) but now I see that there are also
> >-accel (python-reportlab-accel) and -lib (python-guppy-lib)
>
> I suppose I have a mild preference for the -ext shade of bikeshed.

Python terminology for these is already “extension”, so that lends
weight to the ‘-ext’ suffix.

‘-lib’ is confusing because a ‘python-foo’ package is generally a
library (of Python code) anyway. The packages under discussion here are
distinct from normal Python libraries.

‘-accel’ is specific to only one kind of extension (those that are for
the purpose of increasing speed), and doesn't describe all Python
extensions (e.g. those that, regardless of speed, are for the purpose of
interfacing with non-Python code).

So I think ‘-ext’ is the current winner on the basis of existing
terminology.

--
 \       “As the most participatory form of mass speech yet developed, |
  `\    the Internet deserves the highest protection from governmental |
_o__)                   intrusion.” —U.S. District Court Judge Dalzell |
Ben Finney

attachment0 (851 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Julien Cristau-11
In reply to this post by Scott Kitterman-5
On Fri, Jul 13, 2012 at 11:57:01 -0400, Scott Kitterman wrote:

> 1.  python{3}-foo which is arch all and follows the current naming convention
> of foo being the name you import.  It would depend on the arch any python-foo-
> ext package.
>
all -> any package dependencies are often icky, if you want them to be
strictly versioned.  Probably not a showstopper, but something to be
aware of.

Cheers,
Julien
--
Julien Cristau          <[hidden email]>
Logilab        http://www.logilab.fr/
Informatique scientifique & gestion de connaissances


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20120716075759.GA7089@...

Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Scott Kitterman-5


Julien Cristau <[hidden email]> wrote:

>On Fri, Jul 13, 2012 at 11:57:01 -0400, Scott Kitterman wrote:
>
>> 1.  python{3}-foo which is arch all and follows the current naming
>convention
>> of foo being the name you import.  It would depend on the arch any
>python-foo-
>> ext package.
>>
>all -> any package dependencies are often icky, if you want them to be
>strictly versioned.  Probably not a showstopper, but something to be
>aware of.

Right now I'd just like to understand what is being proposed.

Scott K


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/063f9746-b6c6-40d9-bbea-637f1ae651c0@...

Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Yaroslav Halchenko

On Mon, 16 Jul 2012, Scott Kitterman wrote:
> >> 1.  python{3}-foo which is arch all and follows the current naming
> >convention
> >> of foo being the name you import.  It would depend on the arch any
> >python-foo-
> >> ext package.

> >all -> any package dependencies are often icky, if you want them to be
> >strictly versioned.  Probably not a showstopper, but something to be
> >aware of.

> Right now I'd just like to understand what is being proposed.

look at any of mine python-* packages having corresponding -lib package.
e.g.

$> apt-get source python-nipy
...
$> grep -e Package: -e Architecture:  nipy-0.2.0\~rc2+git27-g7b9b5a5/debian/control        
Package: python-nipy
Architecture: all
Package: python-nipy-lib
Architecture: any
Package: python-nipy-lib-dbg
Architecture: any
Package: python-nipy-doc
Architecture: all


--
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik       


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20120716142418.GY21226@...

Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Scott Kitterman-5
On Monday, July 16, 2012 10:24:18 AM Yaroslav Halchenko wrote:

> On Mon, 16 Jul 2012, Scott Kitterman wrote:
> > >> 1.  python{3}-foo which is arch all and follows the current naming
> > >
> > >convention
> > >
> > >> of foo being the name you import.  It would depend on the arch any
> > >
> > >python-foo-
> > >
> > >> ext package.
> > >
> > >all -> any package dependencies are often icky, if you want them to be
> > >strictly versioned.  Probably not a showstopper, but something to be
> > >aware of.
> >
> > Right now I'd just like to understand what is being proposed.
>
> look at any of mine python-* packages having corresponding -lib package.
> e.g.
>
> $> apt-get source python-nipy
> ...
> $> grep -e Package: -e Architecture:
> nipy-0.2.0\~rc2+git27-g7b9b5a5/debian/control Package: python-nipy
> Architecture: all
> Package: python-nipy-lib
> Architecture: any
> Package: python-nipy-lib-dbg
> Architecture: any
> Package: python-nipy-doc
> Architecture: all

OK.  python-nipy depends on python-nipy-lib.  Makes sense.

Package: python-nipy-lib
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}, ${python:Depends}
Provides: ${python:Provides}
XB-Python-Version: ${python:Versions}
Description: Analysis of structural and functional neuroimaging data
 NiPy is a Python-based framework for the analysis of structural and
 functional neuroimaging data.
 .
 This package provides architecture-dependent builds of the libraries.

Is python-nipy-lib useful on it's own?  It seems odd to me that it doesn't at
least Suggest python-nipy.

Scott K


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/2045675.bODldYHci7@scott-latitude-e6320

Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Yaroslav Halchenko-10

On Mon, 16 Jul 2012, Scott Kitterman wrote:
> OK.  python-nipy depends on python-nipy-lib.  Makes sense.

> Is python-nipy-lib useful on it's own?  

nope -- moreover it might be somewhat  detrimental -- module might
appear to be "installed" while only extensions are there.  That is the
only disadvantage of such an approach.

> It seems odd to me that it doesn't at
> least Suggest python-nipy.

and that is where I think circular dependencies are coming -- although I
do not remember details and why I haven't had Suggest -- it clearly
worth adding -- may be it is ok now ;-) ?

--
Yaroslav O. Halchenko
Postdoctoral Fellow,   Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik       


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20120716150659.GD21226@...

Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Scott Kitterman-5
On Monday, July 16, 2012 11:06:59 AM Yaroslav Halchenko wrote:

> On Mon, 16 Jul 2012, Scott Kitterman wrote:
> > OK.  python-nipy depends on python-nipy-lib.  Makes sense.
> >
> > Is python-nipy-lib useful on it's own?
>
> nope -- moreover it might be somewhat  detrimental -- module might
> appear to be "installed" while only extensions are there.  That is the
> only disadvantage of such an approach.
>
> > It seems odd to me that it doesn't at
> > least Suggest python-nipy.
>
> and that is where I think circular dependencies are coming -- although I
> do not remember details and why I haven't had Suggest -- it clearly
> worth adding -- may be it is ok now ;-) ?

I think it is worth getting consensus on how this should be done before you
make a change.

I think at least suggests, but I think recommends is better if it doesn't
behave as a dependency loop.  I haven't looked into how this gets handled yet.  
Anyone?

Scott K


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/1347516.j6B7HhzdZl@scott-latitude-e6320

Reply | Threaded
Open this post in threaded view
|

Re: suffix for packages with (optional?) Python extensions

Julien Cristau-11
On Mon, Jul 16, 2012 at 11:37:17 -0400, Scott Kitterman wrote:

> I think at least suggests, but I think recommends is better if it doesn't
> behave as a dependency loop.  I haven't looked into how this gets handled yet.  
> Anyone?
>
Recommends should be fine AFAIK, since they don't impose any
unpack/configuration ordering on the package manager.

Cheers,
Julien
--
Julien Cristau          <[hidden email]>
Logilab        http://www.logilab.fr/
Informatique scientifique & gestion de connaissances


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20120716155806.GF7089@...