Cron, anacron, cronie, systemd-timers

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Cron, anacron, cronie, systemd-timers

Marc Haber-3
[This goes to -devel, the cron and cronie maintainers and the
participants of the discussion in #897138 ("O: anacron"). If there is a
better forum to discuss this, please point me in the direction]

Hi,

with buster being freshly out of the door, it is the time to take
technical decisions. I would like to point your attention the cron
situation in Debian. The vast majority of our systems is still running
vixie cron, which has been dead upstream for two decades. The Debian
maintainer is doing the code maintenance, with a Debian revision of
currently 134. Machines that do not run continuously usually use anacron
in addition to cron, which has had its last upstream releas in the year
2000 and is sans Debian maintenacne for more than a year.

On the other hand, there is cronie, which is used in the Red Hat world
for years, is a well-tested code base, maintained upstream (in a rather
slow pace), but only has an eight years old package in experimental on
the Debian side.

And on the third side, we have systemd timers, which are not suited as
a complete replacement. There is code to transform crontab entries into
systemd timer units, but functionality that cron delivers, such as
in-sequence execution of the cron.{daily|weekly|monthly} jobs and the
ability to send cron-job output per e-mail (which can be a nuisance, but
is still functionality a lot of code depends upon).

Without having a closer look at cronie (it basically works noiselessly
on CentOS systems with me not noticing for months that it's not vixie
cron), my gut feeling says that moving over to systemd timers completly
might not be a good solution, but using cronie in the future might be
helpful. Christian, the cronie maintainer, also being an uploader for
vixie cron, will probably help to make necessary adaptions for a
seamless migration.

What is the issue that keeps cronie out of unstable? If it just a matter
of personpower, or are there technical reasons? If it's just a matter of
personpower, I'd like to help.

Are there any points towards keeping vixie cron and anacron around? I
would love to hear your opinion.

Greetings
Marc


--
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Reply | Threaded
Open this post in threaded view
|

Re: Cron, anacron, cronie, systemd-timers

Christian Kastner-3
Hi,

one of the cron maintainers here, and also the cronie maintainer.

On 07.07.19 17:00, Marc Haber wrote:
> On the other hand, there is cronie, which is used in the Red Hat world
> for years, is a well-tested code base, maintained upstream (in a rather
> slow pace), but only has an eight years old package in experimental on
> the Debian side.

[...]

> Without having a closer look at cronie (it basically works noiselessly
> on CentOS systems with me not noticing for months that it's not vixie
> cron), my gut feeling says that moving over to systemd timers completly
> might not be a good solution, but using cronie in the future might be
> helpful. Christian, the cronie maintainer, also being an uploader for
> vixie cron, will probably help to make necessary adaptions for a
> seamless migration.
>
> What is the issue that keeps cronie out of unstable? If it just a matter
> of personpower, or are there technical reasons? If it's just a matter of
> personpower, I'd like to help.

It's mainly been personpower; however, that is improving. Any additional
help is welcome, of course.

cronie was uploaded to experimental ages ago, but it's been rotting
there ever since.

My goal for it was to patch is so that it is as compatible with our
vixie cron (which has quite a few extensions!), to make a transition
from cron to cronie as the main cron daemon as painless as possible.

This goal was unattainable with the current vixie cron -134, which is
still source format 1.0, because it's really hard to carry over features
properly from a single huge diff.

Over the past couple of years, I've spent a considerable amount of
effort in converting the format to source format 3.0 (quilt), but I
never quite finished and got distracted with other stuff. Nevertheless,
I completed this conversion in February. I'll push it over the next few
days (I was just waiting for buster to be released before doing anything
drastic).

My current plan it to move from vixie cron 3.0pl1-134 directly to a
(patched) cronie as soon as possible, so that the default daemon can be
switched to cronie in time the next release.

I've abandoned the plan of "upgrading" from 3.0pl1-134 to 4.1-1 before
switching to cronie (which is based on 4.1): it's just too much work,
and I've come to realized that this would just be a waited effort.

Regards,
Christian

Reply | Threaded
Open this post in threaded view
|

Re: Cron, anacron, cronie, systemd-timers

Marc Haber-3
Hi,

On Sun, 7 Jul 2019 19:18:17 +0200, Christian Kastner <[hidden email]>
wrote:
>one of the cron maintainers here, and also the cronie maintainer.

thanks for the swift answer.

>> What is the issue that keeps cronie out of unstable? If it just a matter
>> of personpower, or are there technical reasons? If it's just a matter of
>> personpower, I'd like to help.
>
>It's mainly been personpower; however, that is improving. Any additional
>help is welcome, of course.
>
>cronie was uploaded to experimental ages ago, but it's been rotting
>there ever since.
>
>My goal for it was to patch is so that it is as compatible with our
>vixie cron (which has quite a few extensions!), to make a transition
>from cron to cronie as the main cron daemon as painless as possible.

It is good to know where things are going. Would you mind if I created
a wiki page with this road map laid out about where Debian's cron
world is going? That would, though, only make sense if you could find
the time to cross-read it and add omissions and point out factual
errors.

>Over the past couple of years, I've spent a considerable amount of
>effort in converting the format to source format 3.0 (quilt), but I
>never quite finished and got distracted with other stuff. Nevertheless,
>I completed this conversion in February. I'll push it over the next few
>days (I was just waiting for buster to be released before doing anything
>drastic).

\o/

It is good to know that there is considerable work going on, most of
it being on the source code level, which is unfortunately something I
cannot be helpful because I'm not a sufficiently good C programmer. I
was not even aware that vixie cron and cronie share a common code base
that makes such a migration feasible.

>My current plan it to move from vixie cron 3.0pl1-134 directly to a
>(patched) cronie as soon as possible, so that the default daemon can be
>switched to cronie in time the next release.

Great! That is good to know. Are you using a mailing list for this
effort that I can subscribe to and see where I can be helpful? Or is
this mainly a one-man show anyway?

Greetings
Marc
--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber         |   " Questions are the         | Mailadresse im Header
Mannheim, Germany  |     Beginning of Wisdom "     |
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Reply | Threaded
Open this post in threaded view
|

Re: Cron, anacron, cronie, systemd-timers

Christian Kastner-3
On 09.07.19 09:32, Marc Haber wrote:
> It is good to know where things are going. Would you mind if I created
> a wiki page with this road map laid out about where Debian's cron
> world is going?

Not at all, on the contrary -- thanks for the offer!

> That would, though, only make sense if you could find
> the time to cross-read it and add omissions and point out factual
> errors.

Of course.

>> Over the past couple of years, I've spent a considerable amount of
>> effort in converting the format to source format 3.0 (quilt), but I
>> never quite finished and got distracted with other stuff. Nevertheless,
>> I completed this conversion in February. I'll push it over the next few
>> days (I was just waiting for buster to be released before doing anything
>> drastic).
>
> \o/
>
> It is good to know that there is considerable work going on, most of
> it being on the source code level, which is unfortunately something I
> cannot be helpful because I'm not a sufficiently good C programmer. I
> was not even aware that vixie cron and cronie share a common code base
> that makes such a migration feasible.

There's a lot of non-C stuff that needs to be done, too, in case you're
interested in that?

For example, I'm almost certain by now that the system crontab file and
dirs (/etc/crontab, /etc/cron.*) need to be moved out into a separate
config package so that alternative cron implementations can use them.
Currently, each implementation ships its own crontabs, and switching
between them (eg: switching from cron to bcron) is a pain.

>> My current plan it to move from vixie cron 3.0pl1-134 directly to a
>> (patched) cronie as soon as possible, so that the default daemon can be
>> switched to cronie in time the next release.
>
> Great! That is good to know. Are you using a mailing list for this
> effort that I can subscribe to and see where I can be helpful? Or is
> this mainly a one-man show anyway?

There used to be one until the move to salsa, and I'm ashamed to say I
forgot to opt-in to the migration.

I'll set one up for cronie and then report back to this list.

Regards,
Christian

Reply | Threaded
Open this post in threaded view
|

Re: Cron, anacron, cronie, systemd-timers

Marc Haber-3
On Tue, 16 Jul 2019 20:55:41 +0200, Christian Kastner <[hidden email]>
wrote:
>On 09.07.19 09:32, Marc Haber wrote:
>> It is good to know where things are going. Would you mind if I created
>> a wiki page with this road map laid out about where Debian's cron
>> world is going?
>
>Not at all, on the contrary -- thanks for the offer!

Done. https://wiki.debian.org/cron

>> It is good to know that there is considerable work going on, most of
>> it being on the source code level, which is unfortunately something I
>> cannot be helpful because I'm not a sufficiently good C programmer. I
>> was not even aware that vixie cron and cronie share a common code base
>> that makes such a migration feasible.
>
>There's a lot of non-C stuff that needs to be done, too, in case you're
>interested in that?

I have also created a Wiki page in the cron salsa project outlining
what needs to be done. https://salsa.debian.org/debian/cron/wikis/home

Would it be possible that you push your changes to cron and cronie to
branches on salsa or fork the projects so that your work becomes
visible?

>For example, I'm almost certain by now that the system crontab file and
>dirs (/etc/crontab, /etc/cron.*) need to be moved out into a separate
>config package so that alternative cron implementations can use them.
>Currently, each implementation ships its own crontabs, and switching
>between them (eg: switching from cron to bcron) is a pain.

I think it would probably be good to do that inside the cron source
package first to get the package dependencies sorted out. What should
the crontab's package be named? Should the crontab(1) binary also move
(I don't think so, cronie will most probably have its own
implementation)? Should the crontab package eventually move to its own
crontab source package or should it move to cronie (my least favorite
variant being to keep it with vixie cron)?

>I'll set one up for cronie and then report back to this list.

It's also in the TODO list ;-)

Greetings
Marc
--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber         |   " Questions are the         | Mailadresse im Header
Mannheim, Germany  |     Beginning of Wisdom "     |
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Reply | Threaded
Open this post in threaded view
|

Re: Cron, anacron, cronie, systemd-timers

Simon McVittie-7
In reply to this post by Marc Haber-3
On Sun, 07 Jul 2019 at 17:00:53 +0200, Marc Haber wrote:
> And on the third side, we have systemd timers, which are not suited as
> a complete replacement. There is code to transform crontab entries into
> systemd timer units, but functionality that cron delivers, such as
> in-sequence execution of the cron.{daily|weekly|monthly} jobs and the
> ability to send cron-job output per e-mail (which can be a nuisance, but
> is still functionality a lot of code depends upon).

Presumably you got distracted by the parenthesized clauses and lost the
end of this sentence, which was meant to end with "functionality that
cron delivers ... is lost"?

> in-sequence execution of the cron.{daily|weekly|monthly} jobs

As noted in another thread, this isn't actually missing from systemd-cron,
which still uses run-parts for these (although
https://github.com/systemd-cron/systemd-cron/issues/47 requests the
behaviour that it seems we both assumed it already had).

> ability to send cron-job output per e-mail

I think sending cron job output by email can be either a positive or a
negative, depending on the system it's happening on. On a traditional
Unix server with a local MTA and monitored local email accounts, it's
certainly expected functionality, and perhaps the best way to contact a
sysadmin - but not every Debian system falls into that category any more,
with many desktop systems having personal email travel directly between
the user's MUA and remote IMAP/SMTP servers like GMail, the same way it
typicaly would on Windows or macOS. On such systems, if there is a local
MTA at all, it will often deliver system-level email to a mbox that the
user probably isn't aware of and certainly never reads.

It has been a recurring issue that there is no good way to notify the
user of a typical desktop/laptop system about things going wrong during
boot, package installation or normal operation. The system log (syslog or
systemd Journal) is certainly imperfect, but it seems closer to a solution
than email does - it already a concept of priority, has configurable
data retention, interleaves messages from concurrent events in multiple
daemons, records segfaults (and other core dumps if systemd-coredump
is used), and includes at least a subset of messages from system boot
and daemons (admittedly rather less complete on sysvinit systems, where
each daemon has to implement syslog support for itself rather than just
logging to stdout/stderr).

    smcv

Reply | Threaded
Open this post in threaded view
|

Re: Cron, anacron, cronie, systemd-timers

Marc Haber-3
On Mon, 29 Jul 2019 19:13:15 +0100, Simon McVittie <[hidden email]>
wrote:

>On Sun, 07 Jul 2019 at 17:00:53 +0200, Marc Haber wrote:
>> And on the third side, we have systemd timers, which are not suited as
>> a complete replacement. There is code to transform crontab entries into
>> systemd timer units, but functionality that cron delivers, such as
>> in-sequence execution of the cron.{daily|weekly|monthly} jobs and the
>> ability to send cron-job output per e-mail (which can be a nuisance, but
>> is still functionality a lot of code depends upon).
>
>Presumably you got distracted by the parenthesized clauses and lost the
>end of this sentence, which was meant to end with "functionality that
>cron delivers ... is lost"?

Presumably, yes. Happens to me all the time.

>> in-sequence execution of the cron.{daily|weekly|monthly} jobs
>
>As noted in another thread, this isn't actually missing from systemd-cron,
>which still uses run-parts for these (although
>https://github.com/systemd-cron/systemd-cron/issues/47 requests the
>behaviour that it seems we both assumed it already had).

I am not sure which way is the better one. I can think of both methods
having advantages and disadvantages and would probably try the other
one if it was implemented or just available by throwing one switch.

>> ability to send cron-job output per e-mail
>
>I think sending cron job output by email can be either a positive or a
>negative, depending on the system it's happening on.

Sending cron output by e-mail has the advantage of getting the admin's
attention right away in a place where people usually look a anyway. It
has the disadvantage of not scaling at all, so the journal is the
better place for a site running more than, say, a hundred systems. But
those places tend to have monitoring mechanisms in place that can be
used to lead the administrator's attention to those systems with cron
output present.

Having cron output going to a journal at a place that doesn't have
sophisticated monitoring is hiding information and hiding problems,
which is bad. If sending cron output per mail is the default, the pain
is likely to increase with growing number of systems, motivating those
places to change things. Starting with hiding information isn't a good
thing from this point of view.

> On a traditional
>Unix server with a local MTA and monitored local email accounts, it's
>certainly expected functionality, and perhaps the best way to contact a
>sysadmin - but not every Debian system falls into that category any more,
>with many desktop systems having personal email travel directly between
>the user's MUA and remote IMAP/SMTP servers like GMail, the same way it
>typicaly would on Windows or macOS. On such systems, if there is a local
>MTA at all, it will often deliver system-level email to a mbox that the
>user probably isn't aware of and certainly never reads.

Same effect as having the information in the journal, with the notable
exception that the journal is rotated.

>It has been a recurring issue that there is no good way to notify the
>user of a typical desktop/laptop system about things going wrong during
>boot, package installation or normal operation. The system log (syslog or
>systemd Journal) is certainly imperfect, but it seems closer to a solution
>than email does - it already a concept of priority, has configurable
>data retention, interleaves messages from concurrent events in multiple
>daemons, records segfaults (and other core dumps if systemd-coredump
>is used), and includes at least a subset of messages from system boot
>and daemons (admittedly rather less complete on sysvinit systems, where
>each daemon has to implement syslog support for itself rather than just
>logging to stdout/stderr).

A single, stand-alone system is the third case, yes, but I think that
have lost the majority of those users to the more "user-friendly"
derivatives already anyway.

Greetings
Marc
--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber         |   " Questions are the         | Mailadresse im Header
Mannheim, Germany  |     Beginning of Wisdom "     |
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

Reply | Threaded
Open this post in threaded view
|

Re: Cron, anacron, cronie, systemd-timers

Alexandre Detiste
In reply to this post by Marc Haber-3
Hi Mark,

Thanks for notifying me of this discussion at
https://github.com/systemd-cron/systemd-cron/issues/47

I have been trough some issues recently (apt-cacher-ng that spew
errors after the release)
and I had to dig trough the cron-daily unit journal to locate the problem
and I untersand this is a pain for others too.

> > in-sequence execution of the cron.{daily|weekly|monthly} jobs
> >As noted in another thread, this isn't actually missing from systemd-cron,
> >which still uses run-parts for these (although /issues/47 requests the
> >behaviour that it seems we both assumed it already had).
>
> I am not sure which way is the better one. I can think of both methods
> having advantages and disadvantages and would probably try the other
> one if it was implemented or just available by throwing one switch.

I did the boring work: autoconf, the Makefile...

It is alive here but needs some testing and feedback,
see how it works through the daily, weekly jobs...

> throwing one switch
The idea is to have RUN_PARTS as a bool in /etc/crontab but it
doesn't work yet if there isn't any job defined at all.
(the python generator returns nothing)...

Greets,

Alexandre Detiste