Binary compatibility policy for security updates and point releases

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Binary compatibility policy for security updates and point releases

Jakob Leben
Hello,

I have not been able to find clear information about Debian policies for ABI compatibility across point releases and security updates. I would assume that no ABI changes at all (backwards compatible or not) are allowed in point releases and security updates. Still, I have not found a clear and unambiguous statement to this effect in Debian documentation.

Could someone please confirm the answer to my question? Would Debian developers consider improving documentation on this topic?

Best regards,
Jakob

Reply | Threaded
Open this post in threaded view
|

Re: Binary compatibility policy for security updates and point releases

Ximin Luo-5
Jakob Leben:
> Hello,
>
> I have not been able to find clear information about Debian policies for ABI compatibility across point releases and security updates. I would assume that no ABI changes at all (backwards compatible or not) are allowed in point releases and security updates. Still, I have not found a clear and unambiguous statement to this effect in Debian documentation.
>
> Could someone please confirm the answer to my question? Would Debian developers consider improving documentation on this topic?
>

In general, if no documented guarantee exists then you should assume there is no guarantee.

If you are a developer and you link against a particular version of a library in Debian 9.1 and its ABI changes in 9.2, it's a simple task to rebuild it, and in fact that is what all reverse-dependencies of that library in Debian would themselves have to do. Not a big deal; as a developer you are supposed to deal with these issues and make it transparent to your end users.

X

--
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Reply | Threaded
Open this post in threaded view
|

Re: Binary compatibility policy for security updates and point releases

Ondřej Surý-4
Jakob,

the backward incompatible ABI changes are generally something we avoid in stable releases.

There might be occasional exception to the rule where it cannot be avoided, but it is not something we take lightly.

Ondrej
--
Ondřej Surý <[hidden email]>

> On 17 Mar 2019, at 08:10, Ximin Luo <[hidden email]> wrote:
>
> Jakob Leben:
>> Hello,
>>
>> I have not been able to find clear information about Debian policies for ABI compatibility across point releases and security updates. I would assume that no ABI changes at all (backwards compatible or not) are allowed in point releases and security updates. Still, I have not found a clear and unambiguous statement to this effect in Debian documentation.
>>
>> Could someone please confirm the answer to my question? Would Debian developers consider improving documentation on this topic?
>>
>
> In general, if no documented guarantee exists then you should assume there is no guarantee.
>
> If you are a developer and you link against a particular version of a library in Debian 9.1 and its ABI changes in 9.2, it's a simple task to rebuild it, and in fact that is what all reverse-dependencies of that library in Debian would themselves have to do. Not a big deal; as a developer you are supposed to deal with these issues and make it transparent to your end users.
>
> X
>
> --
> GPG: ed25519/56034877E1F87C35
> GPG: rsa4096/1318EFAC5FBBDBCE
> https://github.com/infinity0/pubkeys.git
>

Reply | Threaded
Open this post in threaded view
|

Re: Binary compatibility policy for security updates and point releases

Jakob Leben
In reply to this post by Ximin Luo-5
On Sun, Mar 17, 2019 at 12:11 AM Ximin Luo <[hidden email]> wrote:

In general, if no documented guarantee exists then you should assume there is no guarantee.

That's what I thought too.


If you are a developer and you link against a particular version of a library in Debian 9.1 and its ABI changes in 9.2, it's a simple task to rebuild it, and in fact that is what all reverse-dependencies of that library in Debian would themselves have to do. Not a big deal; as a developer you are supposed to deal with these issues and make it transparent to your end users.


Well, there are use cases that are not so simple. For example: I might deploy Debian 9.1 on an embedded machine sold to a client on the other side of the world. I have a system for updating my own software which is also deployed on that machine, but not the rest of the Debian system. Now, if ABI might change between 9.2, then I have no guarantee that if I test my software update with 9.2, it will be work as expected on the client's machine with Debian 9.1. However, building and testing my software with Debian 9.1 might be a problem. For example, the official Debian Docker images (https://hub.docker.com/_/debian) are only provided for the latest point release. Finding older point releases from other sources is also not the simplest, and seems to be discouraged and not perfectly supported. See for example the notes here: https://cdimage.debian.org/mirror/cdimage/archive/

Note that this use case requires that ABI does not change at all - not even in a backwards compatible way, because in that case I might inadvertently start using a new symbol introduced in a library in 9.2 which is not present on the client's 9.1 system.

For these reasons, I would appreciate if the documentation was a little more transparent about binary (in)compatibilities across point releases and security updates.

Best regards,

Jakob
Reply | Threaded
Open this post in threaded view
|

Re: Binary compatibility policy for security updates and point releases

Ximin Luo-5
Jakob Leben:
> I have a system for updating my own software which is also deployed on that machine, but not the rest of the Debian system. Now, if ABI might change between 9.2, then I have no guarantee that if I test my software update with 9.2, it will be work as expected on the client's machine with Debian 9.1.

If you Depends: libfoo8, this will prevent dpkg from upgrading the library from libfoo8 to libfoo9 until you provide a new version that Depends: libfoo9 instead. The 8/9 numbering is bumped whenever ABI changes.

X

--
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Reply | Threaded
Open this post in threaded view
|

Re: Binary compatibility policy for security updates and point releases

Sam Hartman-3
In reply to this post by Jakob Leben
>>>>> "Jakob" == Jakob Leben <[hidden email]> writes:

    Jakob>    Well, there are use cases that are not so simple. For
    Jakob> example: I might deploy Debian 9.1 on an embedded machine
    Jakob> sold to a client on the other side of the world. I have a
    Jakob> system for updating my own software which is also deployed on
    Jakob> that machine, but not the rest of the Debian system. Now, if
    Jakob> ABI might change between 9.2, then I have no guarantee that
    Jakob> if I test my software update with 9.2, it will be work as
    Jakob> expected on the client's machine with Debian 9.1. However,
    Jakob> building and testing my software with Debian 9.1 might be a
    Jakob> problem. For example, the official Debian Docker images
    Jakob> (https://hub.docker.com/_/debian) are only provided for the
    Jakob> latest point release. Finding older point releases from other
    Jakob> sources is also not the simplest, and seems to be discouraged
    Jakob> and not perfectly supported. See for example the notes here:
    Jakob> https://cdimage.debian.org/mirror/cdimage/archive/ Note that
    Jakob> this use case requires that ABI does not change at all - not
    Jakob> even in a backwards compatible way, because in that case I
    Jakob> might inadvertently start using a new symbol introduced in a
    Jakob> library in 9.2 which is not present on the client's 9.1
    Jakob> system.  For these reasons, I would appreciate if the
    Jakob> documentation was a little more transparent about binary
    Jakob> (in)compatibilities across point releases and security
    Jakob> updates.

I think the best you're going to get is that we're aware of the issue
and that we try and make ABIs backward compatible (although not frozen)
between point releases.

Some other factors like minimizing changes overall between point
releases tend to make ABIs fairly stable (no new symbols), but I'm aware
of cases where new symbols have been introduced to fix important bugs or
security issues.

However, there are cases where we've broken ABI compatibility within a
stable release.  Haskell is one of the more obvious cases: simply
rebuilding a dependency can break the ABI of a Haskell library.

If you distribute your software as Debian packages, then you can express
your requirements as dependencies and at least know whether the customer
has a new enough point release (and even handle knowing whether you have
the right Haskell dependencies).

If not, in practice things will work fairly well, but you are correct
that there are corner cases that can be problematic.

We certainly are awary of the issue, but it is something that needs to
be balanced against other tradeoffs.

--Sam

Reply | Threaded
Open this post in threaded view
|

Re: Binary compatibility policy for security updates and point releases

Ximin Luo-5
In reply to this post by Ximin Luo-5
Sam Hartman:

>>>>>> "Ximin" == Ximin Luo <[hidden email]> writes:
>
>     Ximin> Jakob Leben:
>     >> I have a system for updating my own software which is also
>     >> deployed
>
>     Ximin> If you Depends: libfoo8, this will prevent dpkg from
>     Ximin> upgrading the library from libfoo8 to libfoo9 until you
>     Ximin> provide a new version that Depends: libfoo9 instead. The 8/9
>     Ximin> numbering is bumped whenever ABI changes.
>
> This is simply not true.
> We don't change sonames when we add symbols to an ABI or make backward
> compatible changes.
> The original poster talked about how this was an issue.
>

OK, I was a bit imprecise with my wording. In the case of added ABI symbols (backwards-compatible changes), the dependency would look like libfoo8 (>= X.Y), which is automated by the dpkg-shlibdeps stuff. That is assuming the library uses symbols files, which it should be doing if it is trying to provide a finer-grained level of ABI compatibility (as opposed to bumping the SOVERSION on every change).

As a reverse-dependency of the library (e.g. Jakob's program) the process should be mostly transparent if you're using the standard Debian build scripts, which uses dpkg-shlibdeps to auto-generate Depends:.

X

--
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Reply | Threaded
Open this post in threaded view
|

Re: Binary compatibility policy for security updates and point releases

Florian Weimer
In reply to this post by Jakob Leben
* Jakob Leben:

> Well, there are use cases that are not so simple. For example: I might
> deploy Debian 9.1 on an embedded machine sold to a client on the other side
> of the world. I have a system for updating my own software which is also
> deployed on that machine, but not the rest of the Debian system. Now, if
> ABI might change between 9.2, then I have no guarantee that if I test my
> software update with 9.2, it will be work as expected on the client's
> machine with Debian 9.1.

Why do you care about ABI breakage in particular?  What about
regressions in functionality?

Practically speaking, you will have to run your own testing (probably
focusing on functionality).  ABI testing is rather hard, and while
there are quite a few tools for it, interpreting the results still
needs expert knowledge.  Sometimes we waive test failures, believing
they do not matter, when it fact they do.  Only testing will find out.

Reply | Threaded
Open this post in threaded view
|

Re: Binary compatibility policy for security updates and point releases

Gilles Filippini-2
In reply to this post by Ondřej Surý-4
Hi,

Ondřej Surý a écrit le 17/03/2019 à 08:43 :
> Jakob,
>
> the backward incompatible ABI changes are generally something we avoid in stable releases.
>
> There might be occasional exception to the rule where it cannot be avoided, but it is not something we take lightly.

Are these incompatible changes from security updates advertised somewhere?

On a related matter we've just experienced at $work a scientific
software breakage after a recent jessie security upgrade. The breakage
is caused by the upgrade of ghostscript to 9.26a~dfsg-0+deb8u1. In this
new ghostscript release the pswrite driver used by our software was
deprecated.
The fix is easy (use the ps2write driver instead), but my colleagues are
now very concerned about Debian security upgrades which can introduce
such non backward compatible changes.
Shouldn't backward incompatibilities introduced by security upgrades be
advertised somewhere? At least in the NEWS.Debian file?

Thanks in advance for any input,

_g.



signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Binary compatibility policy for security updates and point releases

Bastian Blank
On Tue, Mar 19, 2019 at 10:50:04PM +0100, Gilles Filippini wrote:
> Are these incompatible changes from security updates advertised somewhere?
>
> On a related matter we've just experienced at $work a scientific
> software breakage after a recent jessie security upgrade.

Jessie does not longer get security updates since last june.[1]  The
current LTS state uses different rules.

Bastian

[1]: https://www.debian.org/News/2018/20180623
--
If I can have honesty, it's easier to overlook mistakes.
                -- Kirk, "Space Seed", stardate 3141.9