Bug#953017: linux-image-amd64: Regression from "mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()"

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug#953017: linux-image-amd64: Regression from "mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()"

Chen-Yu Tsai
Package: linux-image-amd64
Version: 5.4.13-1~bpo10+1
Severity: important
Tags: upstream patch

Dear Maintainer,

A performance regression stemming from the patch "mm/vmalloc: Sync
unmappings in __purge_vmap_area_lazy()" in all mainline and current
stable kernels, except 3.16.y, was reported by multiple persons [1].
The regression involves any activity that exercises vmalloc a lot,
such as creating threads with CONFIG_VMAP_STACK=y or tty allocation
(for example by SSH servers).

A fix [2] for this was posted last October and was picked up in the
-mm tree last November [3]. However this fix did not make it into
v5.6-rc or any other release. It is still only in the -mm tree.

AFAICT this regression only impacts x86 platforms, as it is the only
platform that has a custom vmalloc_sync_all() instead of the standard
no-op stub.

As per my report to upstream, I currently have one production server
running with the offending patch reverted. I also have another with
the fix applied, and that seems to work well.

Please consider adding the fix to Debian kernel images until upstream
kernels have it.

[1] https://www.spinics.net/lists/stable/msg349763.html
[2] https://patchwork.kernel.org/patch/11181159/
[3] https://www.spinics.net/lists/mm-commits/msg141749.html

-- System Information:
Debian Release: 10.3
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: armhf, i386, arm64

Kernel: Linux 5.2.0-0.bpo.2-amd64 (SMP w/8 CPU cores)
Locale: LANG=zh_TW.UTF-8, LC_CTYPE=zh_TW.UTF-8 (charmap=UTF-8), LANGUAGE=zh_TW.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)

Versions of packages linux-image-amd64 depends on:
ii  linux-image-5.4.0-0.bpo.3-amd64  5.4.13-1~bpo10+1

linux-image-amd64 recommends no packages.

linux-image-amd64 suggests no packages.

-- no debconf information

Reply | Threaded
Open this post in threaded view
|

Bug#953017: Fixes in Upstream

Chen-Yu Tsai
The fix for this issue has been merged in v5.6-rc7 and is part of the
v5.6 release. The commit in upstream is:

    763802b53a42 x86/mm: split vmalloc_sync_all()

This has also been backported to all current LTS kernels except 3.16
in the following releases:

    v4.4.218
    v4.9.218
    v4.14.175
    v4.19.113
    v5.4.28

Reply | Threaded
Open this post in threaded view
|

Bug#953017: closed by Ben Hutchings <ben@decadent.org.uk> (Re: Bug#953017: Fixes in Upstream)

Chen-Yu Tsai
In reply to this post by Chen-Yu Tsai
The stable kernel is still at v4.19.98 though.

On Wed, Apr 15, 2020 at 8:36 AM Debian Bug Tracking System
<[hidden email]> wrote:

>
> This is an automatic notification regarding your Bug report
> which was filed against the src:linux package:
>
> #953017: linux-image-amd64: Regression from "mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()"
>
> It has been closed by Ben Hutchings <[hidden email]>.
>
> Their explanation is attached below along with your original report.
> If this explanation is unsatisfactory and you have not received a
> better one in a separate message then please contact Ben Hutchings <[hidden email]> by
> replying to this email.
>
>
> --
> 953017: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=953017
> Debian Bug Tracking System
> Contact [hidden email] with problems
>
>
>
> ---------- Forwarded message ----------
> From: Ben Hutchings <[hidden email]>
> To: [hidden email]
> Cc:
> Bcc:
> Date: Wed, 15 Apr 2020 01:32:18 +0100
> Subject: Re: Bug#953017: Fixes in Upstream
> Version: 5.5.13-1
>
> On Thu, 2020-04-09 at 13:40 +0800, Chen-Yu Tsai wrote:
> > The fix for this issue has been merged in v5.6-rc7 and is part of the
> > v5.6 release. The commit in upstream is:
> >
> >     763802b53a42 x86/mm: split vmalloc_sync_all()
> >
> > This has also been backported to all current LTS kernels except 3.16
> > in the following releases:
> >
> >     v4.4.218
> >     v4.9.218
> >     v4.14.175
> >     v4.19.113
> >     v5.4.28
>
> ...and 5.5.12, so this is fixed in unstable.
>
> Ben.
>
> --
> Ben Hutchings
> It is a miracle that curiosity survives formal education.
>                                                       - Albert Einstein
>
>
>
>
>
> ---------- Forwarded message ----------
> From: Chen-Yu Tsai <[hidden email]>
> To: Debian Bug Tracking System <[hidden email]>
> Cc:
> Bcc:
> Date: Tue, 03 Mar 2020 17:51:31 +0800
> Subject: linux-image-amd64: Regression from "mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()"
> Package: linux-image-amd64
> Version: 5.4.13-1~bpo10+1
> Severity: important
> Tags: upstream patch
>
> Dear Maintainer,
>
> A performance regression stemming from the patch "mm/vmalloc: Sync
> unmappings in __purge_vmap_area_lazy()" in all mainline and current
> stable kernels, except 3.16.y, was reported by multiple persons [1].
> The regression involves any activity that exercises vmalloc a lot,
> such as creating threads with CONFIG_VMAP_STACK=y or tty allocation
> (for example by SSH servers).
>
> A fix [2] for this was posted last October and was picked up in the
> -mm tree last November [3]. However this fix did not make it into
> v5.6-rc or any other release. It is still only in the -mm tree.
>
> AFAICT this regression only impacts x86 platforms, as it is the only
> platform that has a custom vmalloc_sync_all() instead of the standard
> no-op stub.
>
> As per my report to upstream, I currently have one production server
> running with the offending patch reverted. I also have another with
> the fix applied, and that seems to work well.
>
> Please consider adding the fix to Debian kernel images until upstream
> kernels have it.
>
> [1] https://www.spinics.net/lists/stable/msg349763.html
> [2] https://patchwork.kernel.org/patch/11181159/
> [3] https://www.spinics.net/lists/mm-commits/msg141749.html
>
> -- System Information:
> Debian Release: 10.3
>   APT prefers stable-updates
>   APT policy: (500, 'stable-updates'), (500, 'stable')
> Architecture: amd64 (x86_64)
> Foreign Architectures: armhf, i386, arm64
>
> Kernel: Linux 5.2.0-0.bpo.2-amd64 (SMP w/8 CPU cores)
> Locale: LANG=zh_TW.UTF-8, LC_CTYPE=zh_TW.UTF-8 (charmap=UTF-8), LANGUAGE=zh_TW.UTF-8 (charmap=UTF-8)
> Shell: /bin/sh linked to /bin/dash
> Init: sysvinit (via /sbin/init)
>
> Versions of packages linux-image-amd64 depends on:
> ii  linux-image-5.4.0-0.bpo.3-amd64  5.4.13-1~bpo10+1
>
> linux-image-amd64 recommends no packages.
>
> linux-image-amd64 suggests no packages.
>
> -- no debconf information

Reply | Threaded
Open this post in threaded view
|

Bug#953017: closed by Ben Hutchings <ben@decadent.org.uk> (Re: Bug#953017: Fixes in Upstream)

Salvatore Bonaccorso-4
Hi,

On Thu, Apr 16, 2020 at 05:52:08PM +0800, Chen-Yu Tsai wrote:
> The stable kernel is still at v4.19.98 though.

It is perfectly fine OTOH to actually close the bug in multiple
versions, which happened here. In fact the change for buster is as
well pending (cf.
https://salsa.debian.org/kernel-team/linux/-/merge_requests/213/diffs?commit_id=3be8cdd48491a034f252ecedd943635efd39c8a5).

So it will be as well in the next update targetting buster in the next
point release.

Regards,
Salvatore