Bug#930428: debootstrap should ensure matching _apt uid

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Michael Schaller-2
Package: debootstrap
Version: 1.0.114
Severity: normal

Dear Maintainer,

At the end of the 'apt' package installation the '_apt' user will be created without specifying a fixed uid. This typically results in a differing '_apt' uid between the host system and the bootstrapped system. The differing '_apt' uid is problematic in case the host system has firewall rules specific to the '_apt' user and that typically leads to Apt downloads failing inside a chroot.

For more details see:
* https://lists.debian.org/debian-devel/2019/04/msg00259.html
* https://robots.org.uk/PbuilderFirewallSetup

Unfortunately this issue isn't easy to detect/troubleshoot, particularly firewall rules on the '_apt' uid and that there is an uid mismatch inside a chroot. This could be improved a lot if debootstrap could avoid this issue if it would ensure that the '_apt' user in the bootstrapped system has the same uid as on the host system.

Thoughts? Would the debootstrap maintainers accept patches for this issue?

Best,

Michael Schaller

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Michael Schaller-2
Patch proposal:
https://salsa.debian.org/installer-team/debootstrap/merge_requests/32

Patch note: This patch introduces the 'setup_users_groups' function
which is inspired by the existing 'setup_dselect_method' function.

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Ansgar Burchardt-5
In reply to this post by Michael Schaller-2
Michael Schaller writes:

> At the end of the 'apt' package installation the '_apt' user will be
> created without specifying a fixed uid. This typically results in a
> differing '_apt' uid between the host system and the bootstrapped
> system. The differing '_apt' uid is problematic in case the host
> system has firewall rules specific to the '_apt' user and that
> typically leads to Apt downloads failing inside a chroot.
>
> For more details see:
> * https://lists.debian.org/debian-devel/2019/04/msg00259.html
> * https://robots.org.uk/PbuilderFirewallSetup
>
> Unfortunately this issue isn't easy to detect/troubleshoot,
> particularly firewall rules on the '_apt' uid and that there is an uid
> mismatch inside a chroot. This could be improved a lot if debootstrap
> could avoid this issue if it would ensure that the '_apt' user in the
> bootstrapped system has the same uid as on the host system.

(I don't maintain debootstrap.)

I don't think it is a good idea to require debootstrap to know about
such details.

For limiting network access, I would recommend instead using network
namespaces (to only provide limited network access for all processes)
and/or user namespaces (if filtering for single UIDs is really needed).
These do not require any uids to match between in- and outside.

Ansgar

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Ansgar Burchardt-5
In reply to this post by Michael Schaller-2
Ansgar Burchardt writes:
> (I don't maintain debootstrap.)
>
> I don't think it is a good idea to require debootstrap to know about
> such details.
>
> For limiting network access, I would recommend instead using network
> namespaces (to only provide limited network access for all processes)
> and/or user namespaces (if filtering for single UIDs is really needed).
> These do not require any uids to match between in- and outside.

And sadly the submitter's address bounced my mail as the mail provider
the submitter uses cannot parse RFC-5321 mail addresses correctly.

Ansgar

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Trek-2
In reply to this post by Michael Schaller-2
On Thu, 20 Jun 2019 09:32:17 +0200
Ansgar Burchardt <[hidden email]> wrote:

> I don't think it is a good idea to require debootstrap to know about
> such details.

_apt user is standard to debian, but not its uid

the _apt user is created by the apt postinst, that cannot know anything
about the host system from where debootstrap was launched, so
debootstrap seems to me the only place where this functionality can be
added


> For limiting network access, I would recommend instead using network
> namespaces (to only provide limited network access for all processes)
> and/or user namespaces (if filtering for single UIDs is really
> needed). These do not require any uids to match between in- and
> outside.

filtering out the root user is a pretty common security practice and
setting an iptables rule on uids is simple for system administrators

using namespaces, how can you block any user but not the _apt user if it
is not already created?

just my 2 cents :)
ciao!

P.S.: the patch seems ok to me, I don't like hard-conding the _apt user
line in /etc/passwd, as apt postinst uses adduser, but it's not clear
to me when adduser is installed during debootstrap

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Ansgar Burchardt-9
In reply to this post by Michael Schaller-2
Trek writes:
> Ansgar Burchardt wrote:
>> For limiting network access, I would recommend instead using network
>> namespaces (to only provide limited network access for all processes)
>> and/or user namespaces (if filtering for single UIDs is really
>> needed). These do not require any uids to match between in- and
>> outside.
>
> filtering out the root user is a pretty common security practice and
> setting an iptables rule on uids is simple for system administrators

So you don't run sshd (requires root with network access)?  That seems
rather uncommon to me.

> using namespaces, how can you block any user but not the _apt user if it
> is not already created?

You look up which uid the _apt user inside the chroot has and use that.

> P.S.: the patch seems ok to me, I don't like hard-conding the _apt user
> line in /etc/passwd, as apt postinst uses adduser, but it's not clear
> to me when adduser is installed during debootstrap

You cannot use `adduser` as debootstrap might install binaries you
cannot execute (in the first stage).

But the effects of the patch are different from calling adduser, for
example the _apt user it creates has no entry in /etc/shadow.  Such
inconsistencies are not good.

Ansgar

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Philipp Kern-2
In reply to this post by Ansgar Burchardt-5
On 20/06/2019 09:50, Ansgar Burchardt wrote:

> Ansgar Burchardt writes:
>> (I don't maintain debootstrap.)
>>
>> I don't think it is a good idea to require debootstrap to know about
>> such details.
>>
>> For limiting network access, I would recommend instead using network
>> namespaces (to only provide limited network access for all processes)
>> and/or user namespaces (if filtering for single UIDs is really needed).
>> These do not require any uids to match between in- and outside.
>
> And sadly the submitter's address bounced my mail as the mail provider
> the submitter uses cannot parse RFC-5321 mail addresses correctly.

Well, you can use -submitter@ if you already know that your domain is
problematic. Even re-reading the RFC I'm not sure why that's a bug. RFC
5321 references RFC 1035's definition of the label, which specifies that
a <letter> needs to be first in the label. I didn't immediately find
anything updating that part of RFC 1035. RFC 2181 also specifies that
applications can impose additional restrictions on top of labels.

I'm happy to file an internal bug report if there is actually supporting
documentation rather than just trying out the boundaries of
deliverability. (Where I mostly wish you good luck. It's not a fight I
want to have, which is also why I mostly stopped using my @debian.org
address.)

Kind regards
Philipp Kern

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Philipp Kern-2
In reply to this post by Ansgar Burchardt-9
On 20/06/2019 20:22, Ansgar Burchardt wrote:

> Trek writes:
>> Ansgar Burchardt wrote:
>>> For limiting network access, I would recommend instead using network
>>> namespaces (to only provide limited network access for all processes)
>>> and/or user namespaces (if filtering for single UIDs is really
>>> needed). These do not require any uids to match between in- and
>>> outside.
>> filtering out the root user is a pretty common security practice and
>> setting an iptables rule on uids is simple for system administrators
> So you don't run sshd (requires root with network access)?  That seems
> rather uncommon to me.

There is a difference between running an sshd that only listens and
allowing outbound connections as root, though. But that's a tangent.

>> using namespaces, how can you block any user but not the _apt user if it
>> is not already created?
> You look up which uid the _apt user inside the chroot has and use that.

Yeah, but that scales poorly if you have a centralized firewall policy.
It means that you need to maintain dynamic rules. I know it's possible
and you can dedicate a chain to it. At the same time I think this
problem is actually common enough that it deserves a solution.

>> P.S.: the patch seems ok to me, I don't like hard-conding the _apt user
>> line in /etc/passwd, as apt postinst uses adduser, but it's not clear
>> to me when adduser is installed during debootstrap
> You cannot use `adduser` as debootstrap might install binaries you
> cannot execute (in the first stage).
>
> But the effects of the patch are different from calling adduser, for
> example the _apt user it creates has no entry in /etc/shadow.  Such
> inconsistencies are not good.

Yeah, that's certainly not desirable. But there's also a limited amount
of places (like /etc/shadow) that need to be touched in addition.

Kind regards
Philipp Kern

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Ansgar Burchardt-5
In reply to this post by Michael Schaller-2
Philipp Kern writes:
> 20/06/2019 20:22, Ansgar Burchardt wrote:
>> You look up which uid the _apt user inside the chroot has and use that.
>
> Yeah, but that scales poorly if you have a centralized firewall
> policy. It means that you need to maintain dynamic rules. I know it's
> possible and you can dedicate a chain to it. At the same time I think
> this problem is actually common enough that it deserves a solution.

If _apt deserves a special solution, I would suggest assigning the _apt
user a static uid instead of patching debootstrap.

Ansgar

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Trek-2
In reply to this post by Michael Schaller-2
On Thu, 20 Jun 2019 22:31:15 +0200
Ansgar Burchardt <[hidden email]> wrote:

> If _apt deserves a special solution, I would suggest assigning the
> _apt user a static uid instead of patching debootstrap.

it seems to me the simplest approach, from a technical point of view,
and it's the one I'm using since _apt user was introduced (making sure
uids match)

ciao!

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Ben Hutchings-3
In reply to this post by Philipp Kern-2
On Thu, 2019-06-20 at 20:33 +0200, Philipp Kern wrote:

> On 20/06/2019 09:50, Ansgar Burchardt wrote:
> > Ansgar Burchardt writes:
> > > (I don't maintain debootstrap.)
> > >
> > > I don't think it is a good idea to require debootstrap to know about
> > > such details.
> > >
> > > For limiting network access, I would recommend instead using network
> > > namespaces (to only provide limited network access for all processes)
> > > and/or user namespaces (if filtering for single UIDs is really needed).
> > > These do not require any uids to match between in- and outside.
> >
> > And sadly the submitter's address bounced my mail as the mail provider
> > the submitter uses cannot parse RFC-5321 mail addresses correctly.
>
> Well, you can use -submitter@ if you already know that your domain is
> problematic. Even re-reading the RFC I'm not sure why that's a bug. RFC
> 5321 references RFC 1035's definition of the label, which specifies that
> a <letter> needs to be first in the label.
[...]

No, RFC 1035 says that starting each label with a letter "will result
in fewer problems with many applications".  But RFC 1123 says a label
*can* begin with a digit, and that there is no ambiguity with IP
literals because TLDs start with a letter.

Ben.

--
Ben Hutchings
Beware of programmers who carry screwdrivers. - Leonard Brandwein


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Philipp Kern-2
On 2019-06-21 20:36, Ben Hutchings wrote:

> On Thu, 2019-06-20 at 20:33 +0200, Philipp Kern wrote:
>> On 20/06/2019 09:50, Ansgar Burchardt wrote:
>> > Ansgar Burchardt writes:
>> > > (I don't maintain debootstrap.)
>> > >
>> > > I don't think it is a good idea to require debootstrap to know about
>> > > such details.
>> > >
>> > > For limiting network access, I would recommend instead using network
>> > > namespaces (to only provide limited network access for all processes)
>> > > and/or user namespaces (if filtering for single UIDs is really needed).
>> > > These do not require any uids to match between in- and outside.
>> >
>> > And sadly the submitter's address bounced my mail as the mail provider
>> > the submitter uses cannot parse RFC-5321 mail addresses correctly.
>>
>> Well, you can use -submitter@ if you already know that your domain is
>> problematic. Even re-reading the RFC I'm not sure why that's a bug.
>> RFC
>> 5321 references RFC 1035's definition of the label, which specifies
>> that
>> a <letter> needs to be first in the label.
> [...]
>
> No, RFC 1035 says that starting each label with a letter "will result
> in fewer problems with many applications".  But RFC 1123 says a label
> *can* begin with a digit, and that there is no ambiguity with IP
> literals because TLDs start with a letter.

Thanks for the clarification (although the "will result in fewer
problems" is kind of what I meant then, I guess).

It turns out what he did was not apparent in the mail I replied to and
the problem was instead in the local part (using quotes and whitespace
in it), which - while potentially valid - is also such an odd corner
case that I think we'd be hard pressed to find anyone else who does
that.

And then again, if your whole goal is to test the boundaries of
deliveries (and potentially to avoid spam while doing so), you are
somewhat on your own in the modern Internet. :)

Kind regards
Philipp Kern

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Philipp Kern-2
In reply to this post by Trek-2
On 2019-06-21 07:51, Trek wrote:
> On Thu, 20 Jun 2019 22:31:15 +0200
> Ansgar Burchardt <[hidden email]> wrote:
>
>> If _apt deserves a special solution, I would suggest assigning the
>> _apt user a static uid instead of patching debootstrap.
>
> it seems to me the simplest approach, from a technical point of view,
> and it's the one I'm using since _apt user was introduced (making sure
> uids match)

Adding [hidden email]. APT maintainers, please see the context in the bug.
Do you think there should be logic in debootstrap to handle the case of
trying to have the same UID within a chroot and outside, or could you
apply for a static UID assignment? I would also prefer the latter, but I
honestly don't know how messy the migration would be...

(If so, I guess this bug should be reassigned to apt.)

Kind regards and thanks
Philipp Kern

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Michael Schaller-2
> But the effects of the patch are different from calling adduser, for
> example the _apt user it creates has no entry in /etc/shadow.  Such
> inconsistencies are not good.

Oops. Added a fix for that to the merge request.


> P.S.: the patch seems ok to me, I don't like hard-conding the _apt user
> line in /etc/passwd, as apt postinst uses adduser, but it's not clear
> to me when adduser is installed during debootstrap

adduser and apt are installed as part of the base packages.
base-passwd is installed earlier as part of the required packages.
Hence I also added a fix that moves the setup_* calls for apt directly
before the base package installation.

With that I get the following log output:
$ debootstrap ...
...
I: Configuring required packages...
...
I: Configuring base-passwd...
...
I: Unpacking the base system...
I: Added _apt user with uid 103
I: Unpacking adduser...
I: Unpacking apt...
...


> Do you think there should be logic in debootstrap to handle the case of
> trying to have the same UID within a chroot and outside, or could you
> apply for a static UID assignment? I would also prefer the latter, but I
> honestly don't know how messy the migration would be...

I would prefer if _apt would use a reserved uid (reserved by
base-passwd). I presume that the migration of the existing _apt user
would be messy though, particularly because of existing firewall
rules. So I suggest reserving a completely new user name / uid in
base-passwd for that purpose. As the _apt user seems to only be used
for fetches the new user could be named _apt_fetch.

On Sun, Jun 23, 2019 at 3:18 PM Philipp Kern <[hidden email]> wrote:

>
> On 2019-06-21 07:51, Trek wrote:
> > On Thu, 20 Jun 2019 22:31:15 +0200
> > Ansgar Burchardt <[hidden email]> wrote:
> >
> >> If _apt deserves a special solution, I would suggest assigning the
> >> _apt user a static uid instead of patching debootstrap.
> >
> > it seems to me the simplest approach, from a technical point of view,
> > and it's the one I'm using since _apt user was introduced (making sure
> > uids match)
>
> Adding [hidden email]. APT maintainers, please see the context in the bug.
> Do you think there should be logic in debootstrap to handle the case of
> trying to have the same UID within a chroot and outside, or could you
> apply for a static UID assignment? I would also prefer the latter, but I
> honestly don't know how messy the migration would be...
>
> (If so, I guess this bug should be reassigned to apt.)
>
> Kind regards and thanks
> Philipp Kern

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Johannes Schauer-3
In reply to this post by Philipp Kern-2
Hi all,

Quoting Philipp Kern (2019-06-23 15:14:34)

> On 2019-06-21 07:51, Trek wrote:
> >> If _apt deserves a special solution, I would suggest assigning the
> >> _apt user a static uid instead of patching debootstrap.
> > it seems to me the simplest approach, from a technical point of view,
> > and it's the one I'm using since _apt user was introduced (making sure
> > uids match)
> Adding [hidden email]. APT maintainers, please see the context in the bug.
> Do you think there should be logic in debootstrap to handle the case of
> trying to have the same UID within a chroot and outside, or could you
> apply for a static UID assignment? I would also prefer the latter, but I
> honestly don't know how messy the migration would be...
with my mmdebstrap-maintainer hat on, I wanted to quickly chime in and express
my support for the _apt user having a reproducible user id. The status quo is,
that the apt user id depends on the order in which the maintainer scripts are
executed. Because of this I had to disable some mmdebstrap tests where I
compare the mmdebstrap chroot against the debootstrap chroot because the _apt
uid would be different. One of the goals of mmdebstrap is to be a
proof-of-concept of moving more and more of the mechanics that are currently
hardcoded in debootstrap into apt and dpkg. So from my perspective, fixing the
_apt uid is one piece of the puzzle that would make the life of debootstrap
alternatives like mmdebstrap easier.

Thanks!

cheers, josch

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Michael Schaller-2
Looks like there is no reply from the Apt maintainers. Should I open a
bug against apt?

Also could the proposed patch be added to debootstrap as a temporary
workaround until this is fixed in Apt?

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Julian Andres Klode-4
On Mon, Jul 01, 2019 at 02:47:08PM +0200, Michael Schaller wrote:
> Looks like there is no reply from the Apt maintainers. Should I open a
> bug against apt?

The _apt user is used in stable, and various Ubuntu LTS, and so far,
nobody cared about us creating it dynamically. So this is not a matter
of urgency.

>
> Also could the proposed patch be added to debootstrap as a temporary
> workaround until this is fixed in Apt?

There is nothing to fix here in apt atm.

The only sensible option long term would be to migrate to a different
user I guess, and that sucks, because the user name is nice.

But this is a long-term effort, and not something that will take
place in the short term.

Adding a workaround to debootstrap sounds reasonable, but probably
to be done post-buster release.

--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

Reply | Threaded
Open this post in threaded view
|

Bug#930428: debootstrap should ensure matching _apt uid

Michael Schaller-2
Congrats to the successful Buster launch, everyone.
Also thanks to Julian for the reply on Apt.

Julian, should I open a bug for Apt?

Everyone, what do you think about the proposed patch to debootstrap?
Is that something you'd be willing to carry until Apt gets fixed
(which might or might not happen)?