Adoption of Nix?

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Adoption of Nix?

Artyom Shalkhakov
Hello,


Cite from the homepage:

> Nix is a purely functional package manager. It allows multiple
> versions of a package to be installed side-by-side, ensures that
> dependency specifications are complete, supports atomic
> upgrades and rollbacks, allows non-root users to install software,
> and has many other features.

The claims that I think are valuable are:
- *all* dependencies of a package are automatically found by Nix,
  no exceptions,
- updates and rollbacks are atomic, an update can never break
  your system.

What do you think of adopting Nix as a package management
tool for Debian? I would like to accentuate that I seek
an informative discussion, not a holy war.

Cheers,
Artyom Shalkhakov.


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

jackyf.devel (Bugzilla)
Artyom Shalkhakov wrote:
> Hello,
Hello,

>> Nix is a purely functional package manager. It allows multiple
>> versions of a package to be installed side-by-side, ensures that
>> dependency specifications are complete, supports atomic
>> upgrades and rollbacks, allows non-root users to install software,
>> and has many other features.
>
> The claims that I think are valuable are:
> - *all* dependencies of a package are automatically found by Nix,
>   no exceptions,
Hmm... Nix probably use libastral, doesn't it? Even for C/C++ programs there
is no way to 100% automatically determine entire list of runtime
libraries/tools needed for some particular program (consider runtime library
opening and all non-library dependencies).

> - updates and rollbacks are atomic, an update can never break
>   your system.
This cannot be true. Consider package maintainer scripts. And, for example.
purge of config files cannot be reverted.

> What do you think of adopting Nix as a package management
> tool for Debian? I would like to accentuate that I seek
> an informative discussion, not a holy war.
Nix, as mentioned on its homepage, installs some info in /nix (hey, FHS) and
keeping all versions of packages/libraries -> system bloat and a hell for
security team. It has nothing to do with our apt infrastructure, it doesn't
understand it and invented its own wheel. I think no way for Nix in Debian. We
have excellent dpkg, we have not-so-excellent, but rather good apt, and
significant amount of Debian users choose Debian just only because of apt. IMO.

--
Eugene V. Lyubimkin aka JackYF, JID: jackyf.devel(maildog)gmail.com
Ukrainian C++ Developer, Debian Maintainer, APT contributor


signature.asc (268 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Artyom Shalkhakov
Hi Eugene,

2008/12/24 Eugene V. Lyubimkin <[hidden email]>:
>> The claims that I think are valuable are:
>> - *all* dependencies of a package are automatically found by Nix,
>>   no exceptions,
> Hmm... Nix probably use libastral, doesn't it? Even for C/C++ programs there
> is no way to 100% automatically determine entire list of runtime
> libraries/tools needed for some particular program (consider runtime library
> opening and all non-library dependencies).

This is not about libastral, it's about pure functions (those without
side-effects).

Regarding "runtime library opening" (I suppose, you meant dlopen and friends),
then I suppose, you've found an exception to the rule, but maybe you are wrong.
I'm not a developer of Nix, so I can't say more.

>> - updates and rollbacks are atomic, an update can never break
>>   your system.
> This cannot be true. Consider package maintainer scripts. And, for example.
> purge of config files cannot be reverted.

It can always be reverted if you don't "destructively update" (overwrite) files
and if you can guarantee that filenames do not clash.

> It has nothing to do with our apt infrastructure, it doesn't
> understand it and invented its own wheel. I think no way for Nix in Debian. We
> have excellent dpkg, we have not-so-excellent, but rather good apt, and
> significant amount of Debian users choose Debian just only because of apt. IMO.

I'm not interested in your opinion if it isn't backed by facts, I'm interested
in *informative discussion*. I don't say that dpkg/apt are bad, on the contrary,
I think they are good, but we aren't talking about personal tastes.

It looks like you completely misunderstood the idea, so lurk before
you post. Thanks.

Cheers,
Artyom Shalkhakov.


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Michael Banck
On Wed, Dec 24, 2008 at 05:17:28PM +0600, Artyom Shalkhakov wrote:
> It looks like you completely misunderstood the idea, so lurk before
> you post. Thanks.

You can't post a two-sentence summary of a new package manager to
debian-devel, say "discuss" and then shoot down replies by claiming the
person has not done any research.

If you want to be taken serious, write a paper about Nix and/vs.
dpkg/apt, and post the link to it here together with an introductory
summary/abstract or something.


Michael


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Josselin Mouette
In reply to this post by Artyom Shalkhakov
Le mercredi 24 décembre 2008 à 17:17 +0600, Artyom Shalkhakov a écrit :
> > It has nothing to do with our apt infrastructure, it doesn't
> > understand it and invented its own wheel. I think no way for Nix in Debian. We
> > have excellent dpkg, we have not-so-excellent, but rather good apt, and
> > significant amount of Debian users choose Debian just only because of apt. IMO.
>
> I'm not interested in your opinion if it isn't backed by facts

Which facts are you missing?
      * Nix doesn’t follow the FHS and invents its own filesystem
        hierarchy instead.
      * Nix keeps all versions of all packages installed; this not a
        feature but a bug, and quite a serious one.
      * Nix doesn’t allow to transparently replace a library by a newer
        version, which is an even more critical issue, which makes it
        impossible to remotely consider its use on production machines.
      * The implementation sounds full of gross hacks (like grepping all
        files for the hash of the directory with dependencies).
      * Installing each package in its own directory means that no
        shared configuration and no preservation of configuration upon
        upgrades.

Furthermore, even if it wasn’t as badly flawed, you seem to be
underestimating the amount of work to change a package manager in a
distribution. Any changes to the package manager must be
backwards-compatible.

If you want to do something useful, I suggest that you grab the
interesting ideas from nix (like binary deltas) and propose patches for
APT to implement them.

Cheers,
--
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Paul Wise via nm
On Wed, Dec 24, 2008 at 8:42 PM, Josselin Mouette <[hidden email]> wrote:

> If you want to do something useful, I suggest that you grab the
> interesting ideas from nix (like binary deltas) and propose patches for
> APT to implement them.

Debian already has binary deltas (debdelta), but it isn't well
integrated into apt (#498778). It also (understandably) supported on
the ftp-master/mirrors side.

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Vsevolod Velichko
In reply to this post by Artyom Shalkhakov
Hello.

While I was writing, Josselin Mouette said almost all I wanted to say,
but I'll add a little :)

2008/12/24 Artyom Shalkhakov <[hidden email]>:

>>> The claims that I think are valuable are:
>>> - *all* dependencies of a package are automatically found by Nix,
>>>   no exceptions,
>> Hmm... Nix probably use libastral, doesn't it? Even for C/C++ programs there
>> is no way to 100% automatically determine entire list of runtime
>> libraries/tools needed for some particular program (consider runtime library
>> opening and all non-library dependencies).
>
> This is not about libastral, it's about pure functions (those without
> side-effects).

Well, as I see, it uses it's own package format, which is
wrapper-description around everything - source, deb or rpm. Does it
really have any sense? We have our deb and src packages, do we really
need any wrappers, that make us possible to install rpms? For what
purposes? Surely, dpkg always allows you to rollback any installed
packages. You just sometimes have to rollback half of all your
packages - in accordance with dependencies.
I've just looked to the structure of that package format - it also
requires to write dependencies - so what in it deals with 'em better?
I really don't understand. Can it work with sections like "Recommends"
or "Suggests"?
And, of course, for the 2-3 versions of each package will make debian
security team curse you for ages. Consider it :)

--
Best wishes,
Velichko Vsevolod


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

jackyf.devel (Bugzilla)
In reply to this post by Artyom Shalkhakov
Artyom Shalkhakov wrote:

>> Hmm... Nix probably use libastral, doesn't it? Even for C/C++ programs there
>> is no way to 100% automatically determine entire list of runtime
>> libraries/tools needed for some particular program (consider runtime library
>> opening and all non-library dependencies).
>
> This is not about libastral, it's about pure functions (those without
> side-effects).
>
> Regarding "runtime library opening" (I suppose, you meant dlopen and friends),
> then I suppose, you've found an exception to the rule, but maybe you are wrong.
> I'm not a developer of Nix, so I can't say more.
There are packages which runtime dependencies cannot be determined without
looking to source code or contacting the author/looking to the home page.
I maintains such a package. "dlopen" and friends is another example. Which
means that "find all dependencies with no exceptions" is not true.

>>> - updates and rollbacks are atomic, an update can never break
>>>   your system.
>> This cannot be true. Consider package maintainer scripts. And, for example.
>> purge of config files cannot be reverted.
>
> It can always be reverted if you don't "destructively update" (overwrite) files
> and if you can guarantee that filenames do not clash.
If edited by administrator config file was deleted, then or it cannot be
reverted, or it was not purged. Most other stuff can be reverted in theory...
but again, Debian package maintainer scripts don't support downgrading (in
general), and there are reasons for it.

>> It has nothing to do with our apt infrastructure, it doesn't
>> understand it and invented its own wheel. I think no way for Nix in Debian. We
>> have excellent dpkg, we have not-so-excellent, but rather good apt, and
>> significant amount of Debian users choose Debian just only because of apt. IMO.
>
> I'm not interested in your opinion if it isn't backed by facts,
One big fact is: Debian have tens (or even hundreds) of tools that use apt
infrastructure, including both user side and archive maintenance side. Nix, in
any way it operates, suggests other API to maintain packages. Who is supposed
to rewrite all this stuff for Nix?

> It looks like you completely misunderstood the idea, so lurk before
> you post. Thanks.
Yes, you are probably right: I don't understand how Nix may be useful for
Debian (and for GNU/Linux also).

--
Eugene V. Lyubimkin aka JackYF, JID: jackyf.devel(maildog)gmail.com
Ukrainian C++ Developer, Debian Maintainer, APT contributor


signature.asc (268 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Artyom Shalkhakov
Sorry, I forgot to forward this to debian-devel.

---------- Forwarded message ----------
From: Artyom Shalkhakov <[hidden email]>
Date: 2008/12/24
Subject: Re: Adoption of Nix?
To: "Eugene V. Lyubimkin" <[hidden email]>


2008/12/24 Eugene V. Lyubimkin <[hidden email]>:
> Which means that "find all dependencies with no exceptions" is not true.

This is how Nix developers put it:

> Runtime dependencies are found by scanning binaries for the hash parts
> of Nix store paths (such as r8vvq9kq…). This sounds risky, but it works
> extremely well.

(See <http://nixos.org/about.html>, section called "Complete dependencies".)

> If edited by administrator config file was deleted, then or it cannot be
> reverted, or it was not purged. Most other stuff can be reverted in theory...
> but again, Debian package maintainer scripts don't support downgrading (in
> general), and there are reasons for it.

Take another point of view: every Nix package exists in an ideal world where the
only packages it knows about are it's dependencies (and their precise versions).

> One big fact is: Debian have tens (or even hundreds) of tools that use apt
> infrastructure, including both user side and archive maintenance side. Nix, in
> any way it operates, suggests other API to maintain packages. Who is supposed
> to rewrite all this stuff for Nix?

You're probably right: nobody is going for a full rewrite. I guess I
should inspect
both Nix and dpkg more closely, and if I can find a one-to-one mapping between
the two, then we can go for an automatic migration.

> Yes, you are probably right: I don't understand how Nix may be useful for
> Debian (and for GNU/Linux also).

That's too bad for you. Shallow thinking doesn't get you anywhere.

Cheers,
Artyom Shalkhakov.
Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Artyom Shalkhakov
In reply to this post by Vsevolod Velichko
Sorry, I forgot about the debian-devel for the second time. :(


---------- Forwarded message ----------
From: Artyom Shalkhakov <[hidden email]>
Date: 2008/12/24
Subject: Re: Adoption of Nix?
To: Всеволод Величко <[hidden email]>


Hi Vsevolod,

2008/12/24 Всеволод Величко <[hidden email]>:

> Well, as I see, it uses it's own package format, which is
> wrapper-description around everything - source, deb or rpm. Does it
> really have any sense?

"Every problem in computer science can be solved by adding
a layer of indirection", as the saying goes.

> We have our deb and src packages, do we really need any
> wrappers, that make us possible to install rpms? For what
> purposes?
> Surely, dpkg always allows you to rollback any installed
> packages. You just sometimes have to rollback half of all your
> packages - in accordance with dependencies.

> I've just looked to the structure of that package format - it also
> requires to write dependencies - so what in it deals with 'em better?
> I really don't understand.

The difference is *purity*, which means that Nix expressions
are *deterministic*. And that's what really makes them better.

> Can it work with sections like "Recommends" or "Suggests"?

I don't know this yet, but I think it's nearly trivial to add.

> And, of course, for the 2-3 versions of each package will make debian
> security team curse you for ages. Consider it :)

Thanks for the advice, point taken. :)

Cheers,
Artyom Shalkhakov.

PS do you work for Nigma, an "intelligent search engine"?
Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

jackyf.devel (Bugzilla)
In reply to this post by Artyom Shalkhakov
Artyom Shalkhakov wrote:
> 2008/12/24 Eugene V. Lyubimkin <[hidden email]>:
>> Which means that "find all dependencies with no exceptions" is not true.
>
> This is how Nix developers put it:
>
>> Runtime dependencies are found by scanning binaries for the hash parts
>> of Nix store paths (such as r8vvq9kq…). This sounds risky, but it works
>> extremely well.
> (See <http://nixos.org/about.html>, section called "Complete dependencies".)
I read this part, however I haven't understand it. Nix creates cryptographic
hashes, ok, and how can this work for runtime deps? Shared libraries are best
found by ldd, but most of other stuff still cannot be deduced, because binary
obviously doesn't contain these hashes.

>> If edited by administrator config file was deleted, then or it cannot be
>> reverted, or it was not purged. Most other stuff can be reverted in theory...
>> but again, Debian package maintainer scripts don't support downgrading (in
>> general), and there are reasons for it.
>
> Take another point of view: every Nix package exists in an ideal world where the
> only packages it knows about are it's dependencies (and their precise versions).
So, Nix will install packages (i.e. files) to non-standard directories. Who
will patch programs to look not in /etc/<package>.conf, but in /nix/...? Same
question about /usr/share.

Keep in mind that Debian policy explicitly forbid using non-FHS paths by
packages. Packages that don't obey this must be patched or leave Debian.
I doubt Nix can get an exception from this rule.

--
Eugene V. Lyubimkin aka JackYF, JID: jackyf.devel(maildog)gmail.com
Ukrainian C++ Developer, Debian Maintainer, APT contributor


signature.asc (268 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Steve Kemp
In reply to this post by Artyom Shalkhakov
> > Yes, you are probably right: I don't understand how Nix may be useful for
> > Debian (and for GNU/Linux also).
>
> That's too bad for you. Shallow thinking doesn't get you anywhere.

  As promoter/recommender surely the onus is upon you to demonstrate:

    1. Nix is good.
    2. Nix is better than what currently exists.
    3. Nix would be a good fit for Debian.

  I believe you'll struggle, not least because you do not seem to
 have a thorough understanding of what is actually involved in
 a packaging system.  (Perhaps a comparison to the auto-package
 format is in order?)

Steve
--
# The Debian Security Audit Project.
http://www.debian.org/security/audit


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Drake Wilson-3
In reply to this post by Artyom Shalkhakov
Quoth Artyom Shalkhakov <[hidden email]>, on 2008-12-24 17:17:28 +0600:
> It looks like you completely misunderstood the idea, so lurk before
> you post. Thanks.

Debian List Search, list "devel", author match "artyom shalkhakov":
two matches, all in this thread, not including your most recent two
messages, also in this thread.  Earliest post date: today.

(I'm not a prolific d-d poster myself---mostly a lurker---but I also
don't grant myself the social authority to drop "lurk before you post"
on people's heads.)

Quoth Artyom Shalkhakov <[hidden email]>, on 2008-12-24 22:17:35 +0600:
> That's too bad for you. Shallow thinking doesn't get you anywhere.

And a purity war against a huge established packaging system base
won't get you much of anywhere either unless you're willing to do an
awful lot of the work and demonstrate that the result is both superior
in practice and sufficiently continuous that it's not just an entirely
different system.

(Actually, realistically, I think you're unlikely to get anywhere
with this regardless, for other reasons.)

Quoth Artyom Shalkhakov <[hidden email]>, on 2008-12-24 15:08:20 +0600:
> I would like to accentuate that I seek an informative discussion,
> not a holy war.

Yet I see practical issues being raised, and responses mainly accusing
them of "completely misunderstanding" and "shallow thinking".

   ---> Drake Wilson


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Vsevolod Velichko
In reply to this post by Vsevolod Velichko
Hi.

2008/12/24 Artyom Shalkhakov <[hidden email]>:
>> Well, as I see, it uses it's own package format, which is
>> wrapper-description around everything - source, deb or rpm. Does it
>> really have any sense?
>
> "Every problem in computer science can be solved by adding
> a layer of indirection", as the saying goes.

Yep, and the next year we'll introduce new level of abstraction - for
different versions of nix package format. Pluralitas non est ponenda
sine necessitate, said Occam, as I remember.

>
>> We have our deb and src packages, do we really need any
>> wrappers, that make us possible to install rpms? For what
>> purposes?
>> Surely, dpkg always allows you to rollback any installed
>> packages. You just sometimes have to rollback half of all your
>> packages - in accordance with dependencies.
>
>> I've just looked to the structure of that package format - it also
>> requires to write dependencies - so what in it deals with 'em better?
>> I really don't understand.
>
> The difference is *purity*, which means that Nix expressions
> are *deterministic*. And that's what really makes them better.
I see nothing "purer" in them. Show the difference, please.
>
>> Can it work with sections like "Recommends" or "Suggests"?
>
> I don't know this yet, but I think it's nearly trivial to add.
But it's not still added, really?
And how about packaging many binary packages from the one source?

> PS do you work for Nigma, an "intelligent search engine"?
>
Yes, I'm developer there. :)

--
Best wishes,
Velichko Vsevolod


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

George Danchev
In reply to this post by Steve Kemp
On Wednesday 24 December 2008 18:55:20 Steve Kemp wrote:

> > > Yes, you are probably right: I don't understand how Nix may be useful
> > > for Debian (and for GNU/Linux also).
> >
> > That's too bad for you. Shallow thinking doesn't get you anywhere.
>
>   As promoter/recommender surely the onus is upon you to demonstrate:
>
>     1. Nix is good.
>     2. Nix is better than what currently exists.
>     3. Nix would be a good fit for Debian.

Hm, Nix seems to be born in academia, and based on by someone's PhD thesis, so
there might be some good ideas to consider out of it, but the whole story
smells like the promoter is trying to sell mercedeses to Daimler (i.e.
package management to Debian;-) without getting the whole picture.

>   I believe you'll struggle, not least because you do not seem to
>  have a thorough understanding of what is actually involved in
>  a packaging system.  (Perhaps a comparison to the auto-package
>  format is in order?)

If the promoter doesn't mind I'd also suggest comparision to the openpkg.org.

<citation>Guess he wasn't too popular at the end, huh?</citation>

--
pub 4096R/0E4BD0AB 2003-03-18 <people.fccf.net/danchev/key pgp.mit.edu>


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Ron Johnson
In reply to this post by Drake Wilson-3
On 12/24/08 10:55, Drake Wilson wrote:

> Quoth Artyom Shalkhakov <[hidden email]>, on 2008-12-24 17:17:28 +0600:
>> It looks like you completely misunderstood the idea, so lurk before
>> you post. Thanks.
>
> Debian List Search, list "devel", author match "artyom shalkhakov":
> two matches, all in this thread, not including your most recent two
> messages, also in this thread.  Earliest post date: today.
>
> (I'm not a prolific d-d poster myself---mostly a lurker---but I also
> don't grant myself the social authority to drop "lurk before you post"
> on people's heads.)
>
> Quoth Artyom Shalkhakov <[hidden email]>, on 2008-12-24 22:17:35 +0600:
>> That's too bad for you. Shallow thinking doesn't get you anywhere.
>
> And a purity war against a huge established packaging system base
> won't get you much of anywhere either unless you're willing to do an
> awful lot of the work and demonstrate that the result is both superior
> in practice and sufficiently continuous that it's not just an entirely
> different system.
>
> (Actually, realistically, I think you're unlikely to get anywhere
> with this regardless, for other reasons.)
>
> Quoth Artyom Shalkhakov <[hidden email]>, on 2008-12-24 15:08:20 +0600:
>> I would like to accentuate that I seek an informative discussion,
>> not a holy war.
>
> Yet I see practical issues being raised, and responses mainly accusing
> them of "completely misunderstanding" and "shallow thinking".

This is the kind of foolishness I pulled when I was fresh out of Uni
and thought Academia knew everything.

--
Ron Johnson, Jr.
Jefferson LA  USA

I like my women like I like my coffee - purchased at above-market
rates from eco-friendly organic farming cooperatives in Latin America.


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Daniel Burrows
In reply to this post by Artyom Shalkhakov
  Hi there.  As I'm sure everyone knows, I'm not exactly unbiased here
since I've done a lot of work on the apt system (although nix looks more
like a replacement for dpkg).

  This is the same package manager that was posted on lambda-the-ultimate
a while back, right?  Since you didn't provide a link, I'll provide one.
According to Google, we're talking about this:

    http://nixos.org

  My first impression from reading blurbs on news sites was that they
either found some seriously deep magic, or they're ignoring a lot of
the practical issues that are involved in managing packages within a
large Linux distribution -- and I suspect the latter rather than the
former.  As an academic research project, they are within their rights
to do that, and for some use cases it doesn't matter, but it doesn't
mean we should adopt their software in Debian.

  To take a few excerpts from their site:

====
Nix stores packages in the Nix store, usually the directory /nix/store,
where each package has its own unique subdirectory such as

/nix/store/r8vvq9kq18pz08v249h8my6r9vs7s0n3-firefox-2.0.0.1/
====

  Never mind that this breaks the FHS -- I'll pretend for now that
we've amended policy to allow this, or that we've stuck it in /var
with some horrible bind-mounting to make it appear in the right places.

  That's a terrible user interface decision!  This is Unix, and
filenames are part of the user interface.  That file name, aside from
breaking all user expectations (as per my note about the FHS), is
completely unmemorable, means that packages with the same name aren't
necessarily sorted together in directory listings, breaks tab-completion,
and includes a long string of (to the user) meaningless gobbledygook.
At the very *least*, you should put the package name first, to fix the
tab-completion and sorting problems:

/nix/store/firefox-2.0.0.1-r8vvq9kq18pz08v249h8my6r9vs7s0n3

  but then what if I have two firefox-2.0.0.1s installed?  How do I
know which one is which?

  I hope nix at least has stow-like abilities to create a unified /bin
directory, but that doesn't help when you want to track down the files
of a program for whatever reason.

====
Multiple versions

You can have multiple versions or variants of a package installed at the
same time. This is especially important when different applications have
dependencies on different versions of the same package — it prevents the
“DLL hell”. Because of the hashing scheme, different versions of a package
end up in different paths in the Nix store, so they don’t interfere with
each other.
====

  That's fine as long as your hard drives (never mind the flash devices
embedded systems use, where dpkg is already painfully heavy) are infinitely
large, or you don't install very many versions of very many packages.  The
thought of doing this to track "unstable" terrifies me; I suspect that even
a large hard drive would fill up a few weeks, months tops.  And you can't
automatically purge versions, because you never know which ones a user
wants.

  Presumably there's a way to turn this off.  In fact, I would expect
that we *would* turn this off by default, with a manual override for
particular packages, if we used it in Debian, because I can't see it
being usable for a whole distribution otherwise.  On the other hand:

====
An important consequence is that operations like upgrading or uninstalling
an application cannot break other applications, since these operations
never “destructively” update or delete files that are used by other
packages.
====

  That sounds like they haven't thought hard about the problems around
upgrades and removals, which are not trivial.  (there's a research team
at the University of Paris they could talk to about this, if they
haven't already)  Because of that, I suspect that we *can't* disable
the "install multiple versions" feature -- it sounds like the package
manager fundamentally relies on this to do anything.

  In addition to my earlier comments: what if I have multiple Web
servers or database servers installed (or multiple versions of one of
them)?  Which one runs at startup, and what if I have packages that
specifically depend on another one?

====
Complete dependencies
====

  As other people have written, their claims are at best overblown.
e.g., while it can tell that I use Python, there's no possible way
it can tell which versions of Python I'm compatible with.  It also
sounds like maybe they bind programs to the exact binary of the library
they're built with, which would mean that you have to rebuild all the
reverse dependencies of a library every time you rebuild the library!
(that's so outrageous that I'm sure I must just be incorrectly
extrapolating from the summary on their Web site)

  Also: what about programs that refer to a file, but can function
without it?  This seems to have the same problem as other harnesses
that "automatically" detect dependencies through file access, in that
it will see a program probing for some functionality and assume that
it requires that functionality.

  In short: they have a clever way of detecting dependencies, that has
have a different combination of advantages and disadvantages compared
to our own system.  I can't say which one is better without reading up
more thoroughly on what they do, but unless they have pure magic (not
to mention some heavy-duty static program analysis from machine code)
there's no way they can eliminate the need to manually add or tweak
dependencies.  Our dependency detection works in most of the cases where
you could reasonably expect it to, and it doesn't seem to be burdensome
in practice for package maintainers to add the missing dependencies.

  I can't remember the last time I saw a stable Debian package with
incomplete dependencies, and even in unstable it seems very rare these
days.

====
Garbage collection

When you install a package like this…

$ nix-env --uninstall firefox

the package isn’t deleted from the system right away (after all, you might
want to do a rollback, or it might be in the profiles of other users).
====

  That's great, as long as they include the ability for me to say "I
really want to remove firefox, please show me all the packages that are
keeping it on my system."  Of course, we already have that in Debian.

====
Multi-user support

Starting at version 0.11, Nix has multi-user support. This means that
non-privileged users can securely install software. Each user can have a
different profile, a set of packages in the Nix store that appear in the
user’s PATH. If a user installs a package that another user has already
installed previously, the package won’t be built or downloaded a second
time. At the same time, it is not possible for one user to inject a Trojan
horse into a package that might be used by another user.
====

  I like this, and I've thought from time to time about how we could
theoretically integrate it into dpkg.  I'm a little worried about the
interaction with garbage collection: it sounds to me like a normal user
can DoS the system by depending on someone else's package so that it
never gets garbage collected.

====
Atomic upgrades and rollbacks

Since package management operations never overwrite packages in the Nix
store but just add new versions in different paths, they are atomic. So
during a package upgrade, there is no time window in which the package has
some files from the old version and some files from the new version — which
would be bad because a program might well crash if it’s started during that
period.
====

  I have to say: this is a very nice feature, and the one thing I see
that would be worth thinking about overhauling our package system to
get.

  However, the language above is somewhat misleading, on two points.
First, I think it overstates the problems that can occur due to a
partially installed upgrade.  That's certainly a theoretical
possibility, but dpkg goes to a lot of trouble to minimize the time
window during which an inconsistent package is visible.  In practice,
I've never seen this happen.  What I *have* seen is packages failing
between being installed and being configured, which gets me to my
second point:

  Rolling back software upgrades isn't as easy as just reverting the
files.  Many packages need some amount of post-install configuration,
and this can't be done in an acceptable way without breaking atomic
upgrades and rollbacks.  Typical examples include:

   1. Upgrading configuration files or databases to new formats.
   2. Creating users in the system password file.
   3. Starting or stopping daemons.
   4. Running programs to initialize system-wide caches of data
      so they are available when the program runs.

  Take (1), for instance.  Where are the package's configuration files
and databases stored?  I'll take databases as an example, but most of
these comments (except the size-related ones) apply to configuration
files as well.  If they're installed inside the package, then

  (a) Different versions of the package will have different databases,
      even if they're compatible!  This might be a feature, but then
      again, it might be a bug (what if I have two programs configured
      to use different builds of a library that has a systemwide
      database -- one of them can put stuff into the database and it
      won't be visible to the other one).
  (b) Rolling back to a previous version of the package will *destroy*
      any data you've added since the upgrade (obviously unacceptable).
  (c) You'll need a way to copy the database on upgrade, or upgrades
      will lose all the user's data (again, obviously unacceptable).
      The upgrade process will also have to upgrade the database format
      to the new version.  You also have to use disk space to store
      multiple copies of the database (which might be substantial).

  If the databases are installed systemwide, on the other hand, then
you cannot perform the upgrade atomically when the database format
changes: at some point, either the new version of the program will be
installed and the systemwide database will have the old version, or the
old version will be installed and the systemwide database will have the
new version.  You might say that this isn't really a problem, in which
case I will point out that the same applies to how dpkg does things. :-)

  The only way to deal with (b) is to take the data from the new version
and somehow inject it into the old version, but this raises its own
problems.

  (i)   If the user has relied on the "multiple package versions"
        feature of nix and has different configurations and/or database
        contents in the different package versions, you've just wiped out
        or otherwise messed with the configuration / data of the old
        version.
  (ii)  If the upgrade required a database or configuration file format
        upgrade, then you need to "downgrade" the contents of the
        database or configuration file.  But this isn't possible in
        general: first, it is often not possible *in principle* because
        the new format contains features that cannot be expressed in the
        old format.  Even if that's not an issue, upstream authors will
        often provide upgrade scripts but not downgrade scripts, or
        they'll only support downgrades to particular versions of the
        program.

        Also, who performs the auto-downgrade?  A package script has to
        do it (there's no way the package manager can know about the
        specific procedure for downgrading every configuration file and
        database on the system), and that means that the package install
        script has to contain information about how to downgrade to any
        version that the user could possibly have on his or her system.
        In practice, you would end up just supporting downgrades to just
        a few recent versions.
  (iii) Unlike upgrade scripts, downgrade scripts are likely to be
        poorly tested and unreliable.  For that matter, *upgrade*
        scripts from more than a few versions back are often not well
        tested; that's one of the reasons we recommend that people
        upgrade only between consecutive Debian releases (there just
        aren't enough people looking at what happens when you skip
        releases, especially more than one).

  I won't bore you with 2-4, and anyway this is the most intractable
of the problems.

  Please understand, *this* is the reason that dpkg and apt do not
officially support downgrades.  Replacing files with earlier ones is
trivial, and dpkg is happy to do that all day for you.  We already store
snapshots of old .debs online, and we could easily arrange for apt to
download them and/or to archive every version you install for future
rollback.  The problem is that it's very difficult to reliably downgrade
packages *in a manner that automatically results in them working*.

====
Functional package language

Packages are built from Nix expressions, which is a simple functional
language. A Nix expression describes everything that goes into a package
build action (a “derivation”): other packages, sources, the build script,
environment variables for the build script, etc. Nix tries very hard to
ensure that Nix expressions are deterministic: building a Nix expression
twice should yield the same result.
====

  Ultimately, in order to build a program, you have to invoke external
processes (unless Nix includes a C compiler), so you're only
deterministic in the sense of "except the changes introduced by running
random stuff as part of the build process", which is to say not entirely.
Pedantry aside, I think this might still be a useful piece of technology
(just going by the above description), but I don't see a really
compelling advantage over the build system we use now.  Also, the only
place where we currently have nondeclarative stuff in the Debian build
process is the shell snippets in debian/rules; everything else is
declarative.  (some of our build systems go further than that)

  I think the confusion of trying to introduce a new build system would
outweigh any marginal benefit from the improvements they've made, but
again I haven't read the details, just the summary.  Of course, you
could always introduce this the same way that dbs and the various other
flavor-of-the-year build systems have been introduced; we have enough
confusion there already, why not increase it a little more?

====
Transparent source/binary deployment

(snip description of gentoo-like abilities)
====

  I think that's an interesting idea, but I suspect it would be less
work to build this capability into apt than to completely replace our
entire packaging infrastructure.

====
Binary patching
====

  Same comment as above: nice feature, but it's easier to extend the
existing infrastructure.

====
Nix Packages collection

We provide a large set of Nix expressions containing hundreds of existing
Unix packages, the Nix Packages collection (Nixpkgs).
====

  We provide a large set of Debian source packages containing tens of
thousands of existing Unix packages, the Debian package archive. (SCNR)



  With all that said, I think that Nix is a useful program, both because
it explores some new ideas in package management, and also because I can
see it being very useful in some niche areas where current package
managers are lacking.  User-level package management is a great thing in
some environments and this seems like$ a decent solution to that problem
(the hard part is getting the packages to dynamically adapt to their
install directory, but the way they have things set up, you have to do
that anyway).  It also looks like a great way for ISVs to distribute
their software: it's distribution-independent, and it can apparently
guarantee that you bind against exactly the libraries you QAed against.
Won't help with the kernel, X server, etc, but it avoids most of the
versioning issues that can bite commercial software.

  In general, it looks like a nice way of handling binary packages that
aren't part of the core distribution, without the disgusting hacks you
see in things like autopackage.

  For Debian-wide use, though, I don't think that Nix is worth the
trouble.

  Daniel


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Bernd Eckenfels
In reply to this post by George Danchev
In article <[hidden email]> you wrote:
> Hm, Nix seems to be born in academia, and based on by someone's PhD thesis, so
> there might be some good ideas to consider out of it, but the whole story
> smells like the promoter is trying to sell mercedeses to Daimler (i.e.
> package management to Debian;-) without getting the whole picture.

There are quite a lot home grown package managment systems offering
versioned program directoris. Nix is not the only one doing that. Those are
especially used in academia where larger multi user shell servers are
common.

Greetings
Bernd


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Stefano Zacchiroli
In reply to this post by Artyom Shalkhakov
On Wed, Dec 24, 2008 at 03:08:20PM +0600, Artyom Shalkhakov wrote:
> Cite from the homepage:
> > Nix is a purely functional package manager. It allows multiple

Hi all, I've studied a bit NixOS and written something about its
problems in a recent paper [1] (last paragraph of section
3). Interestingly enough, all the critics I've raised have been
confirmed by the authors of Nix, which I've met at the HotSWUp
workshop this year.

Summarizing the main objections I've raised against Nix are:

- a weird intermix between build-time dependencies and runtime
  configuration, both in some cases get mixed into arguments to the
  build functions, while it is clear that in Debian we prefer a more
  clear distinction among the two

- not everything which compose a distribution can be made functional,
  by the authors' own admission. The "solution" to similar problem is
  in some case just living with that (e.g., while updating the user
  database), in some other cases they just get rid of the
  non-functional part.

  It is for example the case of the ldconfig cache, they just live
  without that. Such approach is surely interesting for the properties
  you gain (atomicity of upgrades), but not something I want in a
  production system. Also consider that they have packaged _way_ less
  software than Debian, it is likely that more and more inherently
  imperative parts of a system will be found by packaging more stuff.

- the garbage collection thingy (as other already observed in this
  thread) is a PITA for security as you cannot completely get rid of
  security flawed code (or, if you do that you loose the "applications
  never break" property)

- maintainer scripts are not turned in functional (and revertible)
  expressions, so the only way to "undo" them is living without them,
  or restore whole parts of the filesystem (with the disk consumption
  problems other observed)

All in all, the authors are conducing a (really nice!) experiment, but
they are aware themselves that it is not ready to scale to a system of
the size of Debian.

What is, IMO, way more interesting for us is the evolution of DistNix,
a distributed version of the Nix package manager (think, e.g., at
upgrading of cluster of machines all together) [2]. What they have in
that part of their system is a language for specifying assignments of
services (data-base, web server, MTA, ...) to machines and
inter-machine dependencies. Those information are used to guide
cluster upgrades granting transactional properties.

Unfortunately (for us) the needed underlying assumption is that
upgrades on the single machines are transactional, which is not
something we can currently grant on top of APT.

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..|  .  |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Adoption of Nix?

Stefano Zacchiroli
On Sun, Dec 28, 2008 at 11:57:09AM +0100, Stefano Zacchiroli wrote:
> problems in a recent paper [1] (last paragraph of section
<snip>
> upgrading of cluster of machines all together) [2]. What they have in

Dangling references:

[1] http://upsilon.cc/~zack/research/publications/hotswup-package-upgrade.pdf
[2] http://www.st.ewi.tudelft.nl/~dolstra/pubs/atomic-hotswup2008-submitted.pdf

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..|  .  |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

12