what about an special QA package priority?

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

what about an special QA package priority?

Luciano Bello-3
Hi list,
        I was thinking about the Debian/OpenSSL debacle. Clearly it not easy to
manage a hard meticulous QA process in all packages. In the other hand, there
are packages more critical than others, which are more delicate to security.
        Sometimes, those packages have different priorities in the policy meaning.
Maybe we can implement this as an Optional header in the control.
        The point is: if we can create critical QA category for delicate packages in
the security sense we can have mandatory QA requirement. For example:
 - It should be checked with debugging tools (like valgrind :P)
 - It should maintained by a team
 - It should a public VCS
 - Its patches should be sign-off by reviewers (Raphael Hertzog (hertzog@)
proposed something like this)

        You can extend or reduce this list. We can discuss about the implementation.
But I mainly want to know your opinion.
        Please, paste the URL if you discussed this in the pass.

luciano

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Miriam Ruiz-4
2008/5/20 Luciano Bello <[hidden email]>:
> But I mainly want to know your opinion.

I think it would be a good idea to have something like that :)

Greetings,
Miry


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

William Pitcock-2
In reply to this post by Luciano Bello-3
Hi,

On Tue, 2008-05-20 at 17:21 -0300, Luciano Bello wrote:
> Hi list,
> I was thinking about the Debian/OpenSSL debacle. Clearly it not easy to
> manage a hard meticulous QA process in all packages. In the other hand, there
> are packages more critical than others, which are more delicate to security.
> Sometimes, those packages have different priorities in the policy meaning.
> Maybe we can implement this as an Optional header in the control.
> The point is: if we can create critical QA category for delicate packages in
> the security sense we can have mandatory QA requirement. For example:
>  - It should be checked with debugging tools (like valgrind :P)

Isn't valgrind how we got into this mess to begin with?

>  - It should maintained by a team
>  - It should a public VCS
>  - Its patches should be sign-off by reviewers (Raphael Hertzog (hertzog@)
> proposed something like this)
>
> You can extend or reduce this list. We can discuss about the implementation.
> But I mainly want to know your opinion.
> Please, paste the URL if you discussed this in the pass.
>
> luciano
I think for critical packages, valgrind prettyness isn't something to
care about (unless the interest is generating suppressions).

William


signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Nicolas FRANCOIS (Nekral)
In reply to this post by Luciano Bello-3
Hi,

On Tue, May 20, 2008 at 05:21:07PM -0300, Luciano Bello wrote:
> Hi list,
> I was thinking about the Debian/OpenSSL debacle. Clearly it not easy to
> manage a hard meticulous QA process in all packages. In the other hand, there
> are packages more critical than others, which are more delicate to security.
> Sometimes, those packages have different priorities in the policy meaning.
> Maybe we can implement this as an Optional header in the control.
> The point is: if we can create critical QA category for delicate packages in
> the security sense we can have mandatory QA requirement.

It will be hard to define this list of "delicate" packages.
For example, I'm not sure I would have put openssl in the list a few weeks
ago.
I would have first think about setuid/setgid programs, servers, with high
popcon packages first.

> For example:
>  - It should be checked with debugging tools (like valgrind :P)

It's not always needed.
It may show some bad practices, but having a command line utility which
allocate some resources (memory, syslog, files, PAM, ...) and does not
free them directly (i.e. it relies on system to do the cleanup on exit) is
not an issue.

If you want to improve quality, you can also recommend using checkers
(splint, security checkers), code metrics tools (e.g. pmccabe), unit tests

It might be good to recommend these tools (including valgrind), and to
provide some web services to provide the reports of these tools (IIRC,
there is already some servers with rats reports; for other checkers,
formalizing some debian/rules rules could help to to start the checkers).
But I don't think it will be possible to have them mandatory.

>  - It should maintained by a team

We will only be able to advertise that these packages need comaintainers.
Or is there a defined response for the "delicate" packages with no
teams/comaintainers.

>  - It should a public VCS
>  - Its patches should be sign-off by reviewers (Raphael Hertzog (hertzog@)
> proposed something like this)

ACK for both.

> You can extend or reduce this list. We can discuss about the implementation.
> But I mainly want to know your opinion.

I really appreciate the idea, but it might be hard to implement.

--
Nekral


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Ron Johnson
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/20/08 18:42, Nicolas François wrote:
> Hi,
>
> On Tue, May 20, 2008 at 05:21:07PM -0300, Luciano Bello wrote:
[snip]

>
>> For example:
>>  - It should be checked with debugging tools (like valgrind :P)
>
> It's not always needed.
> It may show some bad practices, but having a command line utility which
> allocate some resources (memory, syslog, files, PAM, ...) and does not
> free them directly (i.e. it relies on system to do the cleanup on exit) is
> not an issue.
>

Still, though, it's Bad Practice not to clean up after yourself,
even though it's "just" a command line utility.  Who knows what
weird, unexpected side effects there might be from running such an
app within a tight bash loop.

- --
Ron Johnson, Jr.
Jefferson LA  USA

ESPN makes baseball players better.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIM4mtS9HxQb37XmcRArSXAJ92VD0i7lBncKAt65cOo2J2s7aq8wCfUsfz
NHsVsPSaxuuaWonjTRuLJ0o=
=ee7/
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

John Hasler
Ron Johnson writes:
> Still, though, it's Bad Practice not to clean up after yourself, even
> though it's "just" a command line utility.  Who knows what weird,
> unexpected side effects there might be from running such an app within a
> tight bash loop.

None.  If the process exits it exits.  It doesn't matter how quickly it is
started again.
--
John Hasler


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Bernd Eckenfels
In reply to this post by Ron Johnson
In article <[hidden email]> you wrote:
> even though it's "just" a command line utility.  Who knows what
> weird, unexpected side effects there might be from running such an
> app within a tight bash loop.

none*. And not cleaning up yourself also improves performance for short
running apps.

Gruss
Bernd

* unless we talk persistent resources like files or ipc.


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Miriam Ruiz-4
In reply to this post by Luciano Bello-3
2008/5/20 Luciano Bello <[hidden email]>:
> Hi list,
>        I was thinking about the Debian/OpenSSL debacle. Clearly it not easy to
> manage a hard meticulous QA process in all packages. In the other hand, there
> are packages more critical than others, which are more delicate to security.
>        Sometimes, those packages have different priorities in the policy meaning.
> Maybe we can implement this as an Optional header in the control.

Thinking about it again, I wonder if that could be implemented using
Enrico's DebTags. I think they would be perfect for this.

Miry


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Enrico Zini
On Wed, May 21, 2008 at 09:00:45AM +0200, Miriam Ruiz wrote:

> Thinking about it again, I wonder if that could be implemented using
> Enrico's DebTags. I think they would be perfect for this.

Something like #436161 would be the place to start: it's about time to
resume that work.


Ciao,

Enrico

--
GPG key: 1024D/797EBFAB 2000-12-05 Enrico Zini <[hidden email]>

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Ron Johnson
In reply to this post by Bernd Eckenfels
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/20/08 23:11, Bernd Eckenfels wrote:
> In article <[hidden email]> you wrote:
>> even though it's "just" a command line utility.  Who knows what
>> weird, unexpected side effects there might be from running such an
>> app within a tight bash loop.
>
> none*. And not cleaning up yourself also improves performance for short
> running apps.

How so?

> Gruss
> Bernd
>
> * unless we talk persistent resources like files or ipc.

That's the point.  And what if there's a kernel (or would that be a
glibc?) bug?

Since you can't know the future of your software (more than once,
I've seen a one-off script or program morph-expand into an important
and much larger app) or the OS it will run on in the future, it's
always good to clean up after yourself.

- --
Ron Johnson, Jr.
Jefferson LA  USA

ESPN makes baseball players better.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFINCb2S9HxQb37XmcRAq4WAKCd+UGDDIKarUy7YuznusgS1ZxIeACfadoc
3uC4lFzrlrkckOFSJtJWJbQ=
=Z30s
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

peter green-2
In reply to this post by Luciano Bello-3
>> none*. And not cleaning up yourself also improves performance for short
>> running apps.
>How so?

The libraries request memory from the kernel in pages (4k on i386, will vary
on other architectures), they then run thier own heap management system within
those pages. Some memory managers will return pagess to the OS when they become
completely empty others will not.

When the application quits the kernel cleans it up, every page it owns is reclaimed
without having to even look at the memory manager structures inside.

in other words freeing the memory you have allocated before quitting takes time
and achives nothing usefull.




--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Luciano Bello-3
In reply to this post by Nicolas FRANCOIS (Nekral)
El Mar 20 May 2008, Nicolas François escribió:
> It will be hard to define this list of "delicate" packages.
> For example, I'm not sure I would have put openssl in the list a few weeks
> ago.
> I would have first think about setuid/setgid programs, servers, with high
> popcon packages first.

I agree, we should sharpen the definition of "delicate" packages:
- setuid/setgid programs.
- network servers with high popcon (how much is high?)
- packages which implements cryptographic algorithms (like python-crypto)

What about compilers and interpreters (like gcc and perl)? Kernel and drivers?

luciano

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Andreas Bombe-3
In reply to this post by Ron Johnson
On Wed, May 21, 2008 at 08:43:19AM -0500, Ron Johnson wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 05/20/08 23:11, Bernd Eckenfels wrote:
> > In article <[hidden email]> you wrote:
> >> even though it's "just" a command line utility.  Who knows what
> >> weird, unexpected side effects there might be from running such an
> >> app within a tight bash loop.
> >
> > none*. And not cleaning up yourself also improves performance for short
> > running apps.
>
> How so?

The cleanup is pointless and takes CPU time.  Consider a program that
does a lot of malloc()s which it uses until it exits.  If it really
wants to cleanup, it needs to free() every single one which means
updating memory allocation structures for every single block of memory
that gets freed.

And all that for nothing, as the process's whole memory space gets
unmapped on exit no matter its contents (including the state of the
malloc implementation's allocation management structures).

--
Andreas Bombe <[hidden email]>    GPG key 0x04880A44


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Bernd Eckenfels
In reply to this post by Luciano Bello-3
In article <[hidden email]> you wrote:
> What about compilers and interpreters (like gcc and perl)? Kernel and drivers?

Everything which is part of the TCB (libs, login, resolvercache, init, root
cron tools, etc).

And of course all network clients and all other programs :)

Gruss
Bernd


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Ron Johnson
In reply to this post by Andreas Bombe-3
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/21/08 20:08, Andreas Bombe wrote:

> On Wed, May 21, 2008 at 08:43:19AM -0500, Ron Johnson wrote:
>>
>> On 05/20/08 23:11, Bernd Eckenfels wrote:
>>> In article <[hidden email]> you wrote:
>>>> even though it's "just" a command line utility.  Who knows what
>>>> weird, unexpected side effects there might be from running such an
>>>> app within a tight bash loop.
>>> none*. And not cleaning up yourself also improves performance for short
>>> running apps.
>>
>> How so?
>
> The cleanup is pointless and takes CPU time.  Consider a program that
> does a lot of malloc()s which it uses until it exits.  If it really
> wants to cleanup, it needs to free() every single one which means
> updating memory allocation structures for every single block of memory
> that gets freed.
>
> And all that for nothing, as the process's whole memory space gets
> unmapped on exit no matter its contents (including the state of the
> malloc implementation's allocation management structures).

I guess that working in the "enterprise" attunes me to the real
possibility that little apps which do one thing then terminate can
easily morph into big apps that run "forever".  Cleaning up after
yourself just leaves fewer surprises for the guy who comes after you
and makes unanticipated modifications.

- --
Ron Johnson, Jr.
Jefferson LA  USA

ESPN makes baseball players better.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFINNxsS9HxQb37XmcRAlOQAKDO4woqICg8GGO8DMknhxVjJEjW2wCgjYtM
ON91J1pRnNvqsTg2eS4Mst4=
=gey7
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Guillem Jover
In reply to this post by Bernd Eckenfels
On Wed, 2008-05-21 at 06:11:46 +0200, Bernd Eckenfels wrote:
> In article <[hidden email]> you wrote:
> > even though it's "just" a command line utility.  Who knows what
> > weird, unexpected side effects there might be from running such an
> > app within a tight bash loop.
>
> none*. And not cleaning up yourself also improves performance for short
> running apps.
>
> * unless we talk persistent resources like files or ipc.

There's also the case of PIE applications, and someone else dlopening
them, althought I don't think this is that common right now.

regards,
guillem


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Stefano Zacchiroli
In reply to this post by Luciano Bello-3
On Tue, May 20, 2008 at 05:21:07PM -0300, Luciano Bello wrote:
> I was thinking about the Debian/OpenSSL debacle. Clearly it not easy to
> manage a hard meticulous QA process in all packages. In the other hand, there
> are packages more critical than others, which are more delicate to security.

The more I think at this proposal of yours, the more I get convinced
that the only reasonable definition of delicate is "used by a lot of
people" (i.e. score high in popcoon).

As previously noted in this thread other criteria are subjective, and
even apparently innocuous packages can open the flank to really serious
security problems.

So, basically, I welcome your proposal, but IMO its simplest and most
effective implementation would be: ``packages scoring high in popcon
have to be maintained by teams using some Vcs-*''. To that feel free to
add the bells and whistles you want (like valgrind :-P).

Cheers.

--
Stefano Zacchiroli -*- PhD in Computer Science ............... now what?
zack@{upsilon.cc,cs.unibo.it,debian.org}  -<%>-  http://upsilon.cc/zack/
(15:56:48)  Zack: e la demo dema ?    /\    All one has to do is hit the
(15:57:15)  Bac: no, la demo scema    \/    right keys at the right time

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Don Armstrong
In reply to this post by Luciano Bello-3
On Tue, 20 May 2008, Luciano Bello wrote:
>  - It should be checked with debugging tools (like valgrind :P)
>  - It should a public VCS

These should be encouraged, and in the cases where packages aren't in
a public VCS or QAed properly before upload, the deficiencies should
be politely pointed out and maintainers encouraged to rectify.

>  - It should maintained by a team

Team maintenance doesn't automatically make a package better.[1]
Furthermore, I don't believe there are many (possibly any!) packages
in Debian where the package is "important" and the current maintainer
wouldn't accept help. [And if there are, that's a problem which we can
deal with on a case-by-case basis.]

>  - Its patches should be sign-off by reviewers (Raphael Hertzog (hertzog@)
> proposed something like this)

There isn't enough manpower to do this. While more review is good,
blocking development and bug fixing to wait on review is just not
sustainable and scalable. [It's not like it's hard for people to
interdiff diff.gz's now and see what has changed in each patch; only a
few people not directly involved with the package appear to be doing
this.]

That said, it'd be wonderfull for multiple people to prove me wrong by
reviewing all of the patches in base and above, and keep up with the
development of those packages while doing so. But I'm not going to
hold my breath for it; and we shouldn't hamstring development for it
either.


Don Armstrong

1: It basically boils down to a problem of manpower; see various other
threads which have gone over this in the past.

--
A Bill of Rights that means what the majority wants it to mean is worthless.
 -- U.S. Supreme Court Justice Antonin Scalia

http://www.donarmstrong.com              http://rzlab.ucr.edu


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Steinar H. Gunderson
In reply to this post by Stefano Zacchiroli
On Fri, May 23, 2008 at 06:03:51PM +0200, Stefano Zacchiroli wrote:
> So, basically, I welcome your proposal, but IMO its simplest and most
> effective implementation would be: ``packages scoring high in popcon
> have to be maintained by teams using some Vcs-*''.

Why do you want to force the use of a VCS onto a team?

/* Steinar */
--
Homepage: http://www.sesse.net/


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what about an special QA package priority?

Luciano Bello-3
In reply to this post by Don Armstrong
El Vie 23 May 2008, Don Armstrong escribió:
> >  - It should maintained by a team
>
> Team maintenance doesn't automatically make a package better.[1]
> Furthermore, I don't believe there are many (possibly any!) packages
> in Debian where the package is "important" and the current maintainer
> wouldn't accept help. [And if there are, that's a problem which we can
> deal with on a case-by-case basis.]

Is not about accept help. It about considering the package as unmaintained if
there is not a team to maintain it. In same packages, we can not depend on
only two pairs of eyes.

> >  - Its patches should be sign-off by reviewers (Raphael Hertzog
> > (hertzog@) proposed something like this)
>
> There isn't enough manpower to do this. While more review is good,
> blocking development and bug fixing to wait on review is just not
> sustainable and scalable. [It's not like it's hard for people to
> interdiff diff.gz's now and see what has changed in each patch; only a
> few people not directly involved with the package appear to be doing
> this.]

Of course at first is not easy. But we should go to an scenario where all the
local patches was reported to upstream (to apply them in the next release) or
be justified by more than one developer.

I'm just saying the platitude. We need to improve our process. We must learn
something from the Debian/OpenSSL debacle.


signature.asc (196 bytes) Download Attachment
12