Bug#815202: packages: machines and sponsors information is outdated

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug#815202: packages: machines and sponsors information is outdated

Frank Lichtenheld
On Sat, 20 Feb 2016 09:26:23 +0800 Paul Wise <[hidden email]> wrote:

> Package: www.debian.org
> Severity: minor
> User:Â [hidden email]
> Usertags: packages
>
> The packages site says the two packages mirrors are piatti and rore but
> these were decommissioned a long time ago. It would be best to generate
> the machines and sponsors info from Debian LDAP on a regular basis so
> that this information never gets out of date. Some combination of
> the description, purpose, sponsor and allowedGroups LDAP fields should
> be enough to find the right hosts and display the right info. picconi
> and pkgmirror-1and1 are the current hosts for this service.

Patches welcome ;)

--
Frank Lichtenheld <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Bug#815202: packages: machines and sponsors information is outdated

Stéphane Blondon


Le 29/03/2016 10:31, Frank Lichtenheld a écrit :
> On Sat, 20 Feb 2016 09:26:23 +0800 Paul Wise <[hidden email]> wrote:
>> Some combination of
>> the description, purpose, sponsor and allowedGroups LDAP fields should
>> be enough to find the right hosts and display the right info. picconi
>> and pkgmirror-1and1 are the current hosts for this service.
>
> Patches welcome ;)
>


Sorry, it's not a patch...


Perhaps the script https://db.debian.org/machines.cgi could be a code
base. picconi and pkgmirror-1and1 are listed in this page:

"""
picconi.debian.org packages.debian.org master [...]
pkgmirror-1and1.debian.org packages.debian.org mirror [...]
"""

I don't know in which repository ./machines.cgi is (at least not in the
website repo).


--
Stéphane


signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#815202: packages: machines and sponsors information is outdated

Paul Wise via nm
On Fri, Apr 1, 2016 at 7:21 AM, Stéphane Blondon wrote:

> I don't know in which repository ./machines.cgi is (at least not in the
> website repo).

DSA would very much like to kill the existing codebase behind the
db.d.o website in favour of a rewrite in django:

https://github.com/Debian/ud

The existing codebase is available here:

https://anonscm.debian.org/cgit/mirror/userdir-ldap-cgi.git/tree/machines.cgi

--
bye,
pabs

https://wiki.debian.org/PaulWise

Reply | Threaded
Open this post in threaded view
|

Bug#815202: packages: machines and sponsors information is outdated

Stéphane Blondon
Le 01/04/2016 06:43, Paul Wise a écrit :
> DSA would very much like to kill the existing codebase behind the
> db.d.o website in favour of a rewrite in django:
>
> https://github.com/Debian/ud

There are no commit since two years, so I'm not sure it's still alive.



> The existing codebase is available here:
>
> https://anonscm.debian.org/cgit/mirror/userdir-ldap-cgi.git/tree/machines.cgi

I tried to extract the useful code from the script. It depends on the
libnet-ldapapi-perl package and the configuration of the machine which
is running the LDAP.

The syntax is correct but I can't run it on the machine (I don't have
the access rights).

However, I wonder if this perl script is the best strategy:

- Perhaps providing the full data in JSON format by db.d.o would provide
an easy access for several Debian services/tools.
- Or providing a python script which could be reused by the django site
in the future?

Whatever the chosen solution, I can't try to it due to lack of access
rights. (I think it's normal I don't have it.)

--
Stéphane

packagemachines.pl (3K) Download Attachment
signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#815202: packages: machines and sponsors information is outdated

Paul Wise via nm
On Tue, Apr 5, 2016 at 9:05 AM, Stéphane Blondon wrote:

> There are no commit since two years, so I'm not sure it's still alive.

Nevertheless, it is the way forward.

> I tried to extract the useful code from the script. It depends on the
> libnet-ldapapi-perl package and the configuration of the machine which
> is running the LDAP.

The Debian LDAP can be accessed from any machine, anonymous access
will only return the public data (including hostname stuff). So you
shouldn't need to run on db.d.o nor read the config file at all.

> - Perhaps providing the full data in JSON format by db.d.o would provide
> an easy access for several Debian services/tools.

Direct LDAP access should be machine-readable enough for almost all
services and others can easily add a proxy script.

> - Or providing a python script which could be reused by the django site
> in the future?

I agree that any future machines list in ud (one doesn't exist yet)
should have some filtering options.

--
bye,
pabs

https://wiki.debian.org/PaulWise

Reply | Threaded
Open this post in threaded view
|

Bug#815202: packages: machines and sponsors information is outdated

Stéphane Blondon
Le 05/04/2016 07:06, Paul Wise a écrit :

> On Tue, Apr 5, 2016 at 9:05 AM, Stéphane Blondon wrote:
>
>> There are no commit since two years, so I'm not sure it's still alive.
>
> Nevertheless, it is the way forward.
>
>> I tried to extract the useful code from the script. It depends on the
>> libnet-ldapapi-perl package and the configuration of the machine which
>> is running the LDAP.
>
> The Debian LDAP can be accessed from any machine, anonymous access
> will only return the public data (including hostname stuff). So you
> shouldn't need to run on db.d.o nor read the config file at all.
Thank you for these infos, I ignored them. :-)

I attached 2 scripts (one in Perl, one in Python) allowing to get the
names of the machines providing packages.debian.org service. I think
some checks about LDAP errors could be added but it's a start.

I guess the perl script could be included in order to fix the current
bug report.

--
Stéphane

packagemachines.pl (828 bytes) Download Attachment
packagemachines.py (858 bytes) Download Attachment
signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#815202: packages: machines and sponsors information is outdated

Stéphane Blondon
In reply to this post by Paul Wise via nm


Le 05/04/2016 07:06, Paul Wise a écrit :
> On Tue, Apr 5, 2016 at 9:05 AM, Stéphane Blondon wrote:
>
>> There are no commit since two years, so I'm not sure it's still alive.
>
> Nevertheless, it is the way forward.


Perhaps I am wrong but
https://github.com/Debian/ud is the future for the ldap web interface
(db.debian.org), not for packages.debian.org, isn't it?



To merge the previous Perl patch to packages.debian.org:
- clone http://anonscm.debian.org/cgit/webwml/packages.git/ repository,
check out to the debian-master branch
- move the template static/about/sponsors.tmpl templates/html/
- create a new module lib/Packages/DoAboutSponsors.pm. The request to
the LDAP would be inserted inside.
- modify (at least) lib/Packages/Dispatcher.pm by:
 - including DoAboutSponsors.pm
 - modify do_dispatch to bind the url to the new module. (I didn't
understand it completely)

(I fail to write the patch so I wrote the previous paragraph in hope  it
could help someone else.)


>> - Or providing a python script which could be reused by the django site
>> in the future?
>
> I agree that any future machines list in ud (one doesn't exist yet)
> should have some filtering options.

I can do a pull/request to github with a modified version of the
previous code if you want.


--
Stéphane


signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#815202: packages: machines and sponsors information is outdated

Paul Wise via nm
On Mon, Apr 18, 2016 at 5:47 AM, Stéphane Blondon wrote:

> https://github.com/Debian/ud is the future for the ldap web interface
> (db.debian.org), not for packages.debian.org, isn't it?

Correct.

--
bye,
pabs

https://wiki.debian.org/PaulWise