Re: wicd-daemon-run_1.0_amd64.changes REJECTED

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: wicd-daemon-run_1.0_amd64.changes REJECTED

Dmitry Bogatov-3

[ Added runit maining list in thread ]

[2018-11-27 19:00] Bastian Blank <[hidden email]>
> All those *-run packages
> - are tiny (under 100 bytes of content),
> - generated (so why different source packages?) and
> - there sole purpose is providing support for another init system.
>
> Please let's reach consensus on debian-devel on how to manage support for
> runit.  We also don't provide extra binary packages for systemd services or
> traditional init scripts.

Hello, debian-devel@. Above is REJECT from ftp-master, suggesting to
discuss issue here, so...

==== Background ===

I maintain runit init and supervision system [^1] in Debian [^2]. Runit
init system is compatible to some extend with SysVInit, and able to use
traditional init.d scripts.

But to use supervision possiblities, service should be defined with
format, native to runit. Here is how it typical runscript
looks like:

        $ tree /etc/sv/wicd-daemon
        /etc/sv/wicd-daemon/
        ├── log
        │   ├── run
        │   └── supervise -> /var/lib/runit/log/supervise/wicd-daemon
        ├── run
        └── supervise -> /var/lib/runit/supervise/wicd-daemon

which is provided by `-run' package:

        $ dpkg -L wicd-daemon-run
        [...]
        /etc/sv/wicd-daemon/log
        /etc/sv/wicd-daemon/log/run
        /etc/sv/wicd-daemon/run
        /var/log/runit/wicd-daemon

Essential part is `run' script -- program that will do its work in
foreground. In this particular case it looks like following:

        #!/bin/sh -eu
        exec /usr/sbin/wicd --keep-connection --no-stdout --no-stderr --no-daemon

Note `--no-daemon' option. Logging is expected to go on stdout, which is
piped to script in `/etc/sv/wicd-daemon/log/run', looking, usually like
this:

        #!/bin/sh
        chown -R '_log-wicd-daemon' '/var/log/runit/wicd-daemon'
        exec chpst -u '_log-wicd-daemon' svlogd -tt '/var/log/runit/wicd-daemon'

svlogd(8) is logging utility with built-in timestamping and log
rotating. Note that it runs under dedicated user.
All boring work (creating dedicated user and correct symlinks) is
handled automaticall by `dh-runit' debhelper.

=== The problem ===

Question is what is the best way to package runscript. There is around
1300 daemons, so question is how to manage 1300 runscripts.

I believed (and still believe, despite of REJECT), that best way is

0. One source package, providing single binary package per runscript.

   src:{foo}-run -> bin:{foo}-run -> /etc/sv/{foo}

I considered another approaches, and all of them I find inferior:

1. Provide runscript by {foo}.

  It is infeasible due two reasons:

  1.1 Technical. Most -run packages provide dedicated system user to run
                 logging process. It would introduce cruft on systems
                 of users, that install {foo}, but do not use Runit.

                 It is possible to run all logging processes as 'daemon'
                 or 'nobody', but it reduces process isolation and does
                 not solve...

  1.2 Social. Maintainer of {foo} can rightfully refuse to maintain
              support for `runit', not mandated by Policy

2. Add bin:{foo}-run into {foo}.

   It is infeasible due same social consideration.

3. Make single src:runscripts, providing many bin:{foo}-run

   I expect at least following problems:

   3.1 Scalability.
     We have little experience with source packages, providing dozens of
     binary packages, this one will produce hundreds.

   3.2 Changelogs.
     Changelogs for single {foo}-run will contain entries for all other
     {bar}-run packages. That will make them huge and signal/noise ratio will
     fall drastically.

   3.3 Updates.
     Every update of single line of single script will rebuild whole
     package. It is cruel to `sid' users.

   3.4 Epoch inflation

     If for some reason, maintainer of {foo} decides to provide
     {foo}-run package himself, and after that drop it, we get epoch
     bump for src:runscripts. If then maintainer {bar} will decide to
     provide {bar}-run himself, he will have to bump epoch too.

     This way we will eventually get many packages with epoch number
     as big, as 10. But it is only aesthetics.

4. Make single bin:runscripts

   It will be havoc and madness.

     * Hundreds of runscripts, most of which are useless and confusing
       for user.

     * Hundreds extra system users {it will push the limits, since we
       have ~800 slots for them}.

     * Slowed boot for no good reason {at least three extra `stat' for
       every unused runscript and one shell invocation}

     * Updates problem (see above)

     * Changelog problem (see above)

I propose to discuss what is the best way to package runscripts, how to
deal with problems I expect with approaches 1-4, what problems with
approach 0 I miss, and, maybe, there is even exist perfect approach 5?

[^1] http://smarden.org/runit
[^2] https://tracker.debian.org/runit

Reply | Threaded
Open this post in threaded view
|

Re: wicd-daemon-run_1.0_amd64.changes REJECTED

Ansgar Burchardt-8
Dmitry Bogatov writes:
> I believed (and still believe, despite of REJECT), that best way is
>
> 0. One source package, providing single binary package per runscript.
>
>    src:{foo}-run -> bin:{foo}-run -> /etc/sv/{foo}

We generally try to avoid tiny packages in the archive; having 1000+
automatically generated source and binary packages in the archive seems
like a suboptimal solution.

Neither systemd, sysvinit or upstart required extra binary packages.

> 2. Add bin:{foo}-run into {foo}.
>
>    It is infeasible due same social consideration.

That would still result in many tiny binary packages.
>
> 3. Make single src:runscripts, providing many bin:{foo}-run

Same.

As a possible alternative: ship the runscript and some metadata (which
systemd service(s) and/or sysvinit script(s) this corresponds with;
which system users would be needed; ...) either in the service package
(preferred long-term) or a "runscripts" package (maybe easier for
initial experiments).

Then have runit provide a command that creates the system users, sets up
the runit service and disables the systemd service (which I think was
still missing from the *-run packages).

Ansgar

Reply | Threaded
Open this post in threaded view
|

Re: wicd-daemon-run_1.0_amd64.changes REJECTED

Michael Stone-2
In reply to this post by Dmitry Bogatov-3
On Wed, Nov 28, 2018 at 06:48:05PM +0000, Dmitry Bogatov wrote:
>I believed (and still believe, despite of REJECT), that best way is
>
>0. One source package, providing single binary package per runscript.

No, that's horrible. I agree with the REJECT.

Reply | Threaded
Open this post in threaded view
|

Re: wicd-daemon-run_1.0_amd64.changes REJECTED

Tollef Fog Heen
In reply to this post by Dmitry Bogatov-3
]] Dmitry Bogatov

> which is provided by `-run' package:
>
> $ dpkg -L wicd-daemon-run
> [...]
> /etc/sv/wicd-daemon/log
> /etc/sv/wicd-daemon/log/run
> /etc/sv/wicd-daemon/run
> /var/log/runit/wicd-daemon

Does it also provide an init script?  Else, it's RC buggy according to
9.11 in Policy.

> #!/bin/sh -eu
> exec /usr/sbin/wicd --keep-connection --no-stdout --no-stderr --no-daemon
>
> Note `--no-daemon' option. Logging is expected to go on stdout, which is
> piped to script in `/etc/sv/wicd-daemon/log/run', looking, usually like
> this:

How do you ensure that those settings are kept in sync with any command
line flag changes the maintainer makes in their package?

> 1. Provide runscript by {foo}.
>
>   It is infeasible due two reasons:
>
>   1.1 Technical. Most -run packages provide dedicated system user to run
>                  logging process. It would introduce cruft on systems
>                  of users, that install {foo}, but do not use Runit.

I don't think a small number of log users would be a particularly high
cost to bear.  You could also create those using triggers when runit is
installed.

[...]

>   1.2 Social. Maintainer of {foo} can rightfully refuse to maintain
>               support for `runit', not mandated by Policy

I think
https://lists.debian.org/debian-devel-announce/2014/08/msg00001.html
should be sufficient here:

   For the record, the TC expects maintainers to continue to support the
   multiple available init systems in Debian.  That includes merging
   reasonable contributions, and not reverting existing support without
   a compelling reason.

Cheers,
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Reply | Threaded
Open this post in threaded view
|

Re: Re: wicd-daemon-run_1.0_amd64.changes REJECTED

Lorenzo Puliti
In reply to this post by Dmitry Bogatov-3
Hi,
sorry for the extended quote, it's for reference in the debian-runit
mailing list (i'm subscribed there and you drop the CC)

Ansgar Burchardt:
>We generally try to avoid tiny packages in the archive; having 1000+
>automatically generated source and binary packages in the archive seems
>like a suboptimal solution.

> Neither systemd, sysvinit or upstart required extra binary packages.

But none of the above is a process supervisor that can run under another init system.

>As a possible alternative: ship the runscript and some metadata (which
>systemd service(s) and/or sysvinit script(s) this corresponds with;
>which system users would be needed; ...) either in the service package
>(preferred long-term) or a "runscripts" package (maybe easier for
>initial experiments).

>Then have runit provide a command that creates the system users, sets up
>the runit service and disables the systemd service (which I think was
>still missing from the *-run packages).

That will work for runit-init, but what about runit-sysv and runit-systemd?
Let's say I have systemd (as init), runit-systemd and a foo daemon installed;
and 'runscripts' package ship a run script for foo. How can I detect if the user wants to
manage foo with runit or with systemd?
Lorenzo



Reply | Threaded
Open this post in threaded view
|

Re: wicd-daemon-run_1.0_amd64.changes REJECTED

Russ Allbery-2
Lorenz <[hidden email]> writes:

> That will work for runit-init, but what about runit-sysv and
> runit-systemd?  Let's say I have systemd (as init), runit-systemd and a
> foo daemon installed; and 'runscripts' package ship a run script for
> foo. How can I detect if the user wants to manage foo with runit or with
> systemd?

I think a command would work for that case as well.  What I'm imagining
would look something like this:

- If runit-init is installed, it installs a trigger that runs the command
  for any change to the runit metadata directory.  That command sets up
  the users, runit configuration, and does whatever other actions are
  needed to maintain a consistent system.

- If runit-init is not installed, by default all services are run through
  the regular init system.  However, the local system administrator can
  run the command manually, specifying a specific service, and it then
  sets up that service to run via runit in a similar way and also disables
  the systemd unit or init script.

- runit-sysv and runit-systemd install a different trigger that runs the
  script with a flag that says to update all configuration that was
  previously manually enabled by the system administrator (so that you can
  update service definitions or delete ones when packages are removed),
  and ignore all the other configurations.

Note that a lot of the runit metadata can probably be derived from systemd
units for services that have unit files.  For example, if the systemd unit
runs the daemon as a different user, runit can probably use the same user,
and the systemd unit may well also run the daemon in the foreground since
systemd prefers that for the same reasons runit does.  So it's conceivable
that you could get out of shipping explicit runit data for a lot of
packages, or ship something that just notes that the unit file can be
autoconverted.  This would cut down on the maintenance burden of the
primary package maintainer a lot.

--
Russ Allbery ([hidden email])               <http://www.eyrie.org/~eagle/>

Reply | Threaded
Open this post in threaded view
|

Re: wicd-daemon-run_1.0_amd64.changes REJECTED

Ansgar Burchardt-8
In reply to this post by Lorenzo Puliti
Lorenz writes:

> Ansgar Burchardt:
>>As a possible alternative: ship the runscript and some metadata (which
>>systemd service(s) and/or sysvinit script(s) this corresponds with;
>>which system users would be needed; ...) either in the service package
>>(preferred long-term) or a "runscripts" package (maybe easier for
>>initial experiments).
>
>>Then have runit provide a command that creates the system users, sets up
>>the runit service and disables the systemd service (which I think was
>>still missing from the *-run packages).
>
> That will work for runit-init, but what about runit-sysv and runit-systemd?
> Let's say I have systemd (as init), runit-systemd and a foo daemon installed;
> and 'runscripts' package ship a run script for foo. How can I detect
> if the user wants to manage foo with runit or with systemd?

You let the user configure it.

Ansgar

Reply | Threaded
Open this post in threaded view
|

Re: wicd-daemon-run_1.0_amd64.changes REJECTED

Ian Jackson-2
In reply to this post by Russ Allbery-2
Russ Allbery writes ("Re: wicd-daemon-run_1.0_amd64.changes REJECTED"):
> I think a command would work for that case as well.  What I'm imagining
> would look something like this:

Stepping back a bit I think the ideal situation is this:

 * All packages have sysvinit scripts for compatibility.

 * Some packages have metadata for additional init systems,
   including systemd and/or runit.

 * When runit-INITSYSTEM is installed (where INITSYSTEM is sysvinit or
   systemd), and a package is installed that provides runit metadata,
   the daemon is run via runit.

 * When runit-init is installed, obviously, the runit metadata is
   used for packages that provide runit metadata.

 * Packages might provide `sysvinit scripts' which actually work
   via runit, and Depend on runit (but not runit-init).  That is fine
   since they will work with all init systems.

> - If runit-init is installed, it installs a trigger that runs the command
>   for any change to the runit metadata directory.  That command sets up
>   the users, runit configuration, and does whatever other actions are
>   needed to maintain a consistent system.

I think some of this should be done by runit-INITSYSTEM packages too,
since in those cases the daemon should be run via runit and all the
things should be set up.

> Note that a lot of the runit metadata can probably be derived from
> systemd units for services that have unit files.  For example, if
> the systemd unit runs the daemon as a different user, runit can
> probably use the same user, and the systemd unit may well also run
> the daemon in the foreground since systemd prefers that for the same
> reasons runit does.  So it's conceivable that you could get out of
> shipping explicit runit data for a lot of packages, or ship
> something that just notes that the unit file can be autoconverted.
> This would cut down on the maintenance burden of the primary package
> maintainer a lot.

It might be worth doing this as a dh_* command.  It is generally
better to do these things at build time where a greater variety of
tooling is available, and debugging is easier, than shipping input
files to be converted on end user systems.

Cf the emacsen compilation stuff, which I was just debugging a
package's interaction with yesterday.  I understand why it is that way
and I don't want to change it, but we should avoid that approach where
we can.

Ian.

--
Ian Jackson <[hidden email]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Reply | Threaded
Open this post in threaded view
|

Re: wicd-daemon-run_1.0_amd64.changes REJECTED

Josh Triplett-9
Ian Jackson wrote:
> Stepping back a bit I think the ideal situation is this:
>
>  * All packages have sysvinit scripts for compatibility.

Preferably in a package maintained by someone who actually uses that
daemon with sysvinit, rather than one maintained by someone who doesn't.
(And bugs in the use of that package with sysvinit then belong to that
separate package, where the work can be done by people who want that
functionality. Also, s/All packages/All packages capable of running with
sysvinit/.)

Small packages are not a bug. Our archive not handling them well (to the
extent it doesn't, which seems questionable) is a bug. And it's
absolutely a feature to have runit scripts and sysvinit scripts and
configuration snippets of many sorts maintained in separate packages.

- Josh Triplett

Reply | Threaded
Open this post in threaded view
|

Re: wicd-daemon-run_1.0_amd64.changes REJECTED

Matthew Vernon-2
Josh Triplett <[hidden email]> writes:

> Ian Jackson wrote:
>> Stepping back a bit I think the ideal situation is this:
>>
>>  * All packages have sysvinit scripts for compatibility.
>
> Preferably in a package maintained by someone who actually uses that
> daemon with sysvinit, rather than one maintained by someone who doesn't.
> (And bugs in the use of that package with sysvinit then belong to that
> separate package, where the work can be done by people who want that
> functionality. Also, s/All packages/All packages capable of running with
> sysvinit/.)

This seems like an odd idea to me; the init script for foo is always
going to be tightly coupled with foo. It seems straighforward to me that
both it should live in foo's source package and be maintained by the foo
maintainer - if they don't use its init script, then they can take
patches from people who are more expert in init scripts. The same would
apply to a foo systemd service file where the foo maintainer ran
sysvinit on their systems.

To try and pick a less contentious example: I maintain pcre2, which
includes a udeb. I am in no way, shape, or form an expert in the
installer (I use fai at work, frex, and my home system hasn't been
re-installed in years). It would be unhelpful to say "the udeb should be
in a package mainted by someone who actually uses pcre in the
installer".

Regards,

Matthew

--
"At least you know where you are with Microsoft."
"True. I just wish I'd brought a paddle."
http://www.debian.org

Reply | Threaded
Open this post in threaded view
|

Packaging runscripts for Runit init system

Dmitry Bogatov-3
In reply to this post by Tollef Fog Heen

[ Forgot to mention. Please keep me in CC. ]

> [ Tollef Fog Heen ]
>  For the record, the TC expects maintainers to continue to support the
>  multiple available init systems in Debian.  That includes merging
>  reasonable contributions, and not reverting existing support without
>  a compelling reason.

I did not remember this quote from TC. Sounds like solution.

Not that I think it is the best way, but since Debian already provide
sysvinit and systemd scripts as part of main package, I will follow this
tradition. Probably, we could add several sentences into policy,
deprecating *-run packages.

Idea with creating logging users with trigger on `runit' installation
also sounds nice. I will definitely take a look.

Lorenzo, another bright side is that it solves ownership question. There
will be no question, what to do if at time of -run package installation
daemon already running.

Users of runit-sysv/runit-systemd (which I always considered as
non-priority) will have to manually choose, who of sysvinit/runit/systemd
will manage given daemon.

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: wicd-daemon-run_1.0_amd64.changes REJECTED

Guillem Jover
In reply to this post by Josh Triplett-9
On Thu, 2018-11-29 at 13:56:40 -0800, Josh Triplett wrote:
> Preferably in a package maintained by someone who actually uses that
> daemon with sysvinit, rather than one maintained by someone who doesn't.
> (And bugs in the use of that package with sysvinit then belong to that
> separate package, where the work can be done by people who want that
> functionality. Also, s/All packages/All packages capable of running with
> sysvinit/.)

I very much disagree over this as a general principle, be it for
sysvinit or systemd, or whatever other feature a package might provide
that might be optionally used, but the maintainer does not use.

This is the antithesis of what a distribution is supposed to be doing.

> Small packages are not a bug. Our archive not handling them well (to the
> extent it doesn't, which seems questionable) is a bug. And it's
> absolutely a feature to have runit scripts and sysvinit scripts and
> configuration snippets of many sorts maintained in separate packages.

And I still object to this characterization, because the metadata is
the lesser of the problems when it comes to overhead. Few small packages
might be fine, tons of them is an organizational bug:

  <https://lists.debian.org/debian-devel/2015/09/msg00141.html>

Thanks,
Guillem