APT do not work with Squid as a proxy because of pipelining default

classic Classic list List threaded Threaded
38 messages Options
12
Reply | Threaded
Open this post in threaded view
|

APT do not work with Squid as a proxy because of pipelining default

Petter Reinholdtsen
I am bothered by <URL: http://bugs.debian.org/565555 >, and the fact
that apt(-get,itude) do not work with Squid as a proxy.  I would very
much like to have apt work out of the box with Squid in Squeeze.  To
fix it one can either change Squid to work with pipelining the way APT
uses, which the Squid maintainer and developers according to the BTS
report is unlikely to implement any time soon, or change the default
setting in apt for Aquire::http::Pipeline-Depth to zero (0).  I've
added a file like this in /etc/apt/apt.conf.d/ to solve it locally:

  Aquire::http::Pipeline-Depth 0;

My question to all of you is simple.  Should the APT default be
changed or Squid be changed?  Should the bug report be reassigned to
apt or stay as a bug with Squid?

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20100517050500.GC3633@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Goswin von Brederlow-2
Petter Reinholdtsen <[hidden email]> writes:

> I am bothered by <URL: http://bugs.debian.org/565555 >, and the fact
> that apt(-get,itude) do not work with Squid as a proxy.  I would very
> much like to have apt work out of the box with Squid in Squeeze.  To
> fix it one can either change Squid to work with pipelining the way APT
> uses, which the Squid maintainer and developers according to the BTS
> report is unlikely to implement any time soon, or change the default
> setting in apt for Aquire::http::Pipeline-Depth to zero (0).  I've
> added a file like this in /etc/apt/apt.conf.d/ to solve it locally:
>
>   Aquire::http::Pipeline-Depth 0;
>
> My question to all of you is simple.  Should the APT default be
> changed or Squid be changed?  Should the bug report be reassigned to
> apt or stay as a bug with Squid?
>
> Happy hacking,

Given that squid already has a patch, although only for newer versions,
this really seems to be a squid bug. As such it should be fixed in
squid as not only apt might trigger the problem.

That said setting the Pipeline-Depth to 0 as default or when a proxy is
configured might be advisable. Adding a apt.conf.d sniplet to the stable
apt should be a trivial change. Much simpler than fixing squid itself.

And in testing/unstable one can fix it properly or update squid to 3.0.

My 2c,
   Goswin


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/87fx1r57tp.fsf@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Florian Weimer
In reply to this post by Petter Reinholdtsen
* Petter Reinholdtsen:

> I am bothered by <URL: http://bugs.debian.org/565555 >, and the fact
> that apt(-get,itude) do not work with Squid as a proxy.  I would very
> much like to have apt work out of the box with Squid in Squeeze.  To
> fix it one can either change Squid to work with pipelining the way APT
> uses, which the Squid maintainer and developers according to the BTS
> report is unlikely to implement any time soon, or change the default
> setting in apt for Aquire::http::Pipeline-Depth to zero (0).  I've
> added a file like this in /etc/apt/apt.conf.d/ to solve it locally:
>
>   Aquire::http::Pipeline-Depth 0;

Maybe it's safe to use pipelining when a proxy is not used?  This is
how things have been implemented in browsers, IIRC.

On the other hand, you probably still need to somewhat complex retry
logic, but I guess you need that anyway (if the first download fails,
try without pipelining etc.).


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/87pr0umhhn.fsf@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Robert Collins
In reply to this post by Goswin von Brederlow-2
Due to the widespread usage of intercepting proxies, its very hard, if
not impossible, to determine if a proxy is in use. Its unwise, at
best, to assume that no proxy configured == no proxy processing your
traffic :(.

-Rob


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/AANLkTik5TeAAZBKfZgWoN2SwQtDR_tb_tG1PnaGtEuJ0@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Marvin Renich
* Robert Collins <[hidden email]> [100517 17:42]:
> Due to the widespread usage of intercepting proxies, its very hard, if
> not impossible, to determine if a proxy is in use. Its unwise, at
> best, to assume that no proxy configured == no proxy processing your
> traffic :(.
>
> -Rob

IANADD, but if I had filed bug #565555, I would have selected severity
critical ("makes unrelated software on the system break"), and similarly
for any other transparent proxy in Debian that fails to work
transparently.

The proxy may not be on a Debian system, but wouldn't the following
logic in apt catch enough of the problem cases to be a useful
workaround:

    If Acquire::http::Pipeline-Depth is not set and Acquire::http::Proxy
    is set, use 0 for Pipeline-Depth; use current behavior
    otherwise.

Documenting this problem somewhere that an admin would look when seeing
the offending "Hash sum mismatch" message would also help.  Turning off
pipelining by default for everybody seems like the wrong solution to
this problem.

...Marvin


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20100518002527.GK1461@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Robert Collins
Given that pipelining is broken by design, that the HTTP WG has
increased the number of concurrent connections that are recommended,
and removed the upper limit - no. I don't think that disabling
pipelining hurts anyone - just use a couple more concurrent
connections.

-Rob


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/AANLkTikNKCqq8kRQc9zupeLG9gQrglou3zeZgly7-EIe@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Frank Lin PIAT
On Tue, 2010-05-18 at 14:02 +1200, Robert Collins wrote:
> Given that pipelining is broken by design, that the HTTP WG has
> increased the number of concurrent connections that are recommended,
> and removed the upper limit - no. I don't think that disabling
> pipelining hurts anyone - just use a couple more concurrent
> connections.

Lots of [new] users are using Debian in Non-Debian infrastructure, which
may use unpatched squid. They would get a bad initial perception of
Debian, if it wasn't working with standard setup.

My 2cents,

Franklin


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/1274160733.4972.94.camel@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Goswin von Brederlow-2
In reply to this post by Marvin Renich
Marvin Renich <[hidden email]> writes:

> * Robert Collins <[hidden email]> [100517 17:42]:
>> Due to the widespread usage of intercepting proxies, its very hard, if
>> not impossible, to determine if a proxy is in use. Its unwise, at
>> best, to assume that no proxy configured == no proxy processing your
>> traffic :(.
>>
>> -Rob
>
> IANADD, but if I had filed bug #565555, I would have selected severity
> critical ("makes unrelated software on the system break"), and similarly
> for any other transparent proxy in Debian that fails to work
> transparently.
>
> The proxy may not be on a Debian system, but wouldn't the following
> logic in apt catch enough of the problem cases to be a useful
> workaround:
>
>     If Acquire::http::Pipeline-Depth is not set and Acquire::http::Proxy
>     is set, use 0 for Pipeline-Depth; use current behavior
>     otherwise.
>
> Documenting this problem somewhere that an admin would look when seeing
> the offending "Hash sum mismatch" message would also help.  Turning off
> pipelining by default for everybody seems like the wrong solution to
> this problem.
>
> ...Marvin

Maybe apt should check size and try to resume the download. I'm assuming
it gets the right header but then the data ends prematurely?

Could you try to capture a tcpdump of the actual traffic between apt and
the proxy?

MfG
        Goswin


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/87eih9itvi.fsf@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Marvin Renich
In reply to this post by Robert Collins
* Robert Collins <[hidden email]> [100517 22:03]:
> Given that pipelining is broken by design, that the HTTP WG has
> increased the number of concurrent connections that are recommended,
> and removed the upper limit - no. I don't think that disabling
> pipelining hurts anyone - just use a couple more concurrent
> connections.
>
> -Rob

I was unaware that pipelining was considered "broken by design", so I
was trying to say that if there was an easy way for apt to choose
between pipelining and no pipelining (if it wasn't specifically set by
the admin) that would handle most of the cases, that was better than
disabling by default a feature that was beneficial to many.

If pipelining is considered broken, and concurrency is preferred, I'm
perfectly happy with that.

...Marvin


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20100518120244.GL1461@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Marvin Renich
In reply to this post by Goswin von Brederlow-2
* Goswin von Brederlow <[hidden email]> [100518 02:53]:

> Marvin Renich <[hidden email]> writes:
> > Documenting this problem somewhere that an admin would look when seeing
> > the offending "Hash sum mismatch" message would also help.  Turning off
> > pipelining by default for everybody seems like the wrong solution to
> > this problem.
> >
> > ...Marvin
>
> Maybe apt should check size and try to resume the download. I'm assuming
> it gets the right header but then the data ends prematurely?
>
> Could you try to capture a tcpdump of the actual traffic between apt and
> the proxy?
>
> MfG
>         Goswin

Fortunately, I am not behind a proxy, so I can't check this.  :-)

...Marvin


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20100518120413.GM1461@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Mike Hommey
In reply to this post by Florian Weimer
On Mon, May 17, 2010 at 09:54:28PM +0200, Florian Weimer wrote:

> * Petter Reinholdtsen:
>
> > I am bothered by <URL: http://bugs.debian.org/565555 >, and the fact
> > that apt(-get,itude) do not work with Squid as a proxy.  I would very
> > much like to have apt work out of the box with Squid in Squeeze.  To
> > fix it one can either change Squid to work with pipelining the way APT
> > uses, which the Squid maintainer and developers according to the BTS
> > report is unlikely to implement any time soon, or change the default
> > setting in apt for Aquire::http::Pipeline-Depth to zero (0).  I've
> > added a file like this in /etc/apt/apt.conf.d/ to solve it locally:
> >
> >   Aquire::http::Pipeline-Depth 0;
>
> Maybe it's safe to use pipelining when a proxy is not used?  This is
> how things have been implemented in browsers, IIRC.

Mozilla browsers have had pipelining disabled for years, because
reality is that a whole lot of servers don't implement it properly if at
all.

Mike


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/20100518120913.GA8700@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Luigi Gangitano
In reply to this post by Goswin von Brederlow-2
Il giorno 17/mag/2010, alle ore 09.02, Goswin von Brederlow ha scritto:
> Given that squid already has a patch, although only for newer versions,
> this really seems to be a squid bug. As such it should be fixed in
> squid as not only apt might trigger the problem.

Goswin, can you please point me to the patch you mention?

> That said setting the Pipeline-Depth to 0 as default or when a proxy is
> configured might be advisable. Adding a apt.conf.d sniplet to the stable
> apt should be a trivial change. Much simpler than fixing squid itself.
>
> And in testing/unstable one can fix it properly or update squid to 3.0.

I assume that squid3 is not affected by this bug, do you confirm this? If the patch you mentioned is related to squid3 a backport may or may not be feasible, but should try. :-)

Regards,

L

--
Luigi Gangitano -- <[hidden email]> -- <[hidden email]>
GPG: 1024D/924C0C26: 12F8 9C03 89D3 DB4A 9972  C24A F19B A618 924C 0C26


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/6BA9DD68-4527-4D44-A836-647352505831@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Brian M. Carlson
In reply to this post by Mike Hommey
On Tue, May 18, 2010 at 02:09:13PM +0200, Mike Hommey wrote:
> Mozilla browsers have had pipelining disabled for years, because
> reality is that a whole lot of servers don't implement it properly if at
> all.

Actually, I've had pipelining enabled for some time, and it works just
fine for me.  I have had zero problems with it.  And this is with
Iceweasel.

--
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Goswin von Brederlow-2
In reply to this post by Luigi Gangitano
Luigi Gangitano <[hidden email]> writes:

> Il giorno 17/mag/2010, alle ore 09.02, Goswin von Brederlow ha scritto:
>> Given that squid already has a patch, although only for newer versions,
>> this really seems to be a squid bug. As such it should be fixed in
>> squid as not only apt might trigger the problem.
>
> Goswin, can you please point me to the patch you mention?
>
>> That said setting the Pipeline-Depth to 0 as default or when a proxy is
>> configured might be advisable. Adding a apt.conf.d sniplet to the stable
>> apt should be a trivial change. Much simpler than fixing squid itself.
>>
>> And in testing/unstable one can fix it properly or update squid to 3.0.
>
> I assume that squid3 is not affected by this bug, do you confirm this? If the patch you mentioned is related to squid3 a backport may or may not be feasible, but should try. :-)
>
> Regards,
>
> L

It was mentioned in an earlier mail that the issue was fixed in squid 3
but the patch doesn't apply to 2.x. No idea where that patch is, check
the previous mails.

MfG
        Goswin


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/877hn13yz0.fsf@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Roger Lynn-3
In reply to this post by Petter Reinholdtsen
On 18/05/10 03:10, Robert Collins wrote:
> Given that pipelining is broken by design, that the HTTP WG has
> increased the number of concurrent connections that are recommended,
> and removed the upper limit - no. I don't think that disabling
> pipelining hurts anyone - just use a couple more concurrent
> connections.

But apt has been using pipelining for years. Why has this only just
become a problem? Not all proxies dislike pipelining - Polipo is an
example of one that works well with it. It also works with at least some
proprietary/commercial proxies too. And if transparent proxies can't
cope with pipelining then they're broken and not very transparent. I
think if this was a significant problem it would have been noticed a
long time ago. However disabling pipelining if a proxy is configured is
probably a good idea to ensure compatibility and is commonly done in
browsers, but it's not necessary for direct connections.

Roger


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/4BF31D2E.7020209@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Robert Collins
Well, I don't know why something has 'suddenly' become a problem: its
a known issue for years. The HTTP smuggling
[http://www.watchfire.com/resources/HTTP-Request-Smuggling.pdf]
attacks made that very obvious 5 years ago now.

http://en.wikipedia.org/wiki/HTTP_pipelining has a decent overview.

Its nice an interesting that some recent software has it on, but that
is generally because the authors don't realise how broken it is,
IMNSHO :).

-Rob


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/AANLkTikdEp4HoGhNs30kc-jRYXqBa9Fmudd9-dvjFMU0@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Brian May-11
On 19 May 2010 13:51, Robert Collins <[hidden email]> wrote:
> Well, I don't know why something has 'suddenly' become a problem: its
> a known issue for years. The HTTP smuggling
> [http://www.watchfire.com/resources/HTTP-Request-Smuggling.pdf]
> attacks made that very obvious 5 years ago now.

>From my Internet connection, that link seems to be a redirect to
http://www-01.ibm.com/software/rational/offerings/websecurity/, which
doesn't say anything about http security issues.

> http://en.wikipedia.org/wiki/HTTP_pipelining has a decent overview.

I cannot see anything about brokenness of HTTP pipelining here... Did
I miss something?
--
Brian May <[hidden email]>


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/AANLkTinCd4PaQGE1xZpwxwn3ILEnm1fzAnVA4QYG5D-v@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Robert Collins
Bah, link staleness.

http://www.cgisecurity.com/lib/HTTP-Request-Smuggling.pdf just worked for me.

Also, I realise that there may be a disconnect here: squid *shouldn't*
break if a client attempts to pipeline through it - if it is, thats a
bug to be fixed, squid just will not read the second request until the
first one is completed.

-Rob


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/AANLkTik7h4lPnXPYHPWb1x2QLyrv_c6d9fMIqybbg9mp@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Petter Reinholdtsen
In reply to this post by Roger Lynn-3

[Roger Lynn]
> But apt has been using pipelining for years. Why has this only just
> become a problem?

It has been a problem in Debian Edu for years.  Just recently I
figured out the cause and a workaround.

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/2flpr0sjvpa.fsf@...

Reply | Threaded
Open this post in threaded view
|

Re: APT do not work with Squid as a proxy because of pipelining default

Bjørn Mork
Petter Reinholdtsen <[hidden email]> writes:
> [Roger Lynn]
>> But apt has been using pipelining for years. Why has this only just
>> become a problem?
>
> It has been a problem in Debian Edu for years.  Just recently I
> figured out the cause and a workaround.

And FWIW I have experienced this problem for years too, but never
figured out why until this discussion came up.  And I do want to claim
more than common user knowledge of http proxy servers.  Still, it never
occured to me that my spurious apt problems could be caused by proxies.
And no, it's not just squid - I've been seeing the exact same at work
where the office network have some intercepting proxy solution from
websense.

Anyway, this is definitely the type of problem that can and do exist for
years without that necessarily causing a massive number of bug reports
against apt.  I still do not think that is an argument against fixing
it?

Can we please agree that in the real world
1) RFC1123 beats any other standard: "Be liberal in what you accept, and
   conservative in what you send", and
2) http proxy servers cannot always process pipelined requests due to
   the complexity this adds (complexity is always bad for security), and
3) http clients cannot know whether their requests are proxied
?

The sum of these three points is that a http client should never send
pipelined requests.  



Bjørn


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]
Archive: http://lists.debian.org/87vdak5lkg.fsf@...

12