perl/experimental FTBFS on m68k, sh4

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

perl/experimental FTBFS on m68k, sh4

Niko Tyni-3
Hi,

perl 5.30.0-3 in experimental has an overhauled build system based on
dh. Somewhere in the conversion process m68k and sh4 builds broke
in a strange way, with

  /bin/sh: 67: cannot create _cflags.c: Permission denied

cascading in many later issues.

This comes from running cflags.SH in the build directory, but I don't
really understand why it's getting EPERM and why only on these particular
architectures. There are some bits setting ccflags for these architectures
in debian/config.debian, and this might be a quoting problem with them
or something like that, but I haven't figured it out yet.

Based on data from two uploads, this seems to be deterministic and
really only affecting these two architectures. Are there porter machines
available to test this on, or (even better) would somebody be willing
to dig into this and provide a patch?

Thanks for your work on Debian,
--
Niko Tyni   [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: perl/experimental FTBFS on m68k, sh4

John Paul Adrian Glaubitz
Hi!

On 6/26/19 11:24 PM, Niko Tyni wrote:
> perl 5.30.0-3 in experimental has an overhauled build system based on
> dh. Somewhere in the conversion process m68k and sh4 builds broke
> in a strange way, with
>
>   /bin/sh: 67: cannot create _cflags.c: Permission denied
>
> cascading in many later issues.

Wild shot in the dark: I think this is most likely this bug again [1]
and we found another candidate for a package that needs to be built with
64-bit offsets.

A porterbox wouldn't help - although I have one for sh4 available - as
this is very likely a qemu problem.

Thanks for notifying us, I'll have a look tomorrow.

Adrian

> [1] https://sourceware.org/bugzilla/show_bug.cgi?id=23960

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: perl/experimental FTBFS on m68k, sh4

John Paul Adrian Glaubitz
On 6/26/19 11:53 PM, John Paul Adrian Glaubitz wrote:
> Wild shot in the dark: I think this is most likely this bug again [1]
> and we found another candidate for a package that needs to be built with
> 64-bit offsets.

So, I have rebuilt coreutils with 64-bit offsets now but that didn't help.

Will build a patched version of glibc now with the workaround to see if this
is actually bug I am suspecting. Maybe I need other build flags for coreutils
to sort this issue out.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: perl/experimental FTBFS on m68k, sh4

John Paul Adrian Glaubitz
On 7/2/19 12:31 PM, John Paul Adrian Glaubitz wrote:
> So, I have rebuilt coreutils with 64-bit offsets now but that didn't help.
>
> Will build a patched version of glibc now with the workaround to see if this
> is actually bug I am suspecting. Maybe I need other build flags for coreutils
> to sort this issue out.

Doesn't help either. There must be something else afoul.

I wish the error message would be more meaningful:

Doing variable substitutions on .SH files...
/bin/sh: 67: cannot create _cflags.c: Permission denied
cflags.SH: cc       = m68k-linux-gnu-gcc
cflags.SH: ccflags  = -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
cflags.SH: stdflags =
cflags.SH: optimize = -O2 -g
cflags.SH: warn     =  -Wall
Extracting cflags (with variable substitutions)
/bin/sh: 424: cannot create cflags: Permission denied
/bin/sh: 446: cannot create cflags: Permission denied
./chmod: cannot access 'cflags': No such file or directory

This will be fun trying to find out where the regression is coming from. *sigh*

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: perl/experimental FTBFS on m68k, sh4

John Paul Adrian Glaubitz
Hi!

On 7/6/19 12:19 AM, John Paul Adrian Glaubitz wrote:
> Doesn't help either. There must be something else afoul.
>
> I wish the error message would be more meaningful:
> (...)
> This will be fun trying to find out where the regression is coming from. *sigh*

So, some more information: This issue affects qemu-user only and can be reproduced
on armhf as well. If this is an issue with Perl in general, it means that this
particular version of Perl will not work on any version of qemu-user independent
of the architecture used.

@Niki: Did you make any changes in particular to the Perl configuration itself? I
       would like to find out what change exactly triggered the bug and file a
       bug report either against qemu-user or the component in question. This
       bug should definitely get fixed otherwise users won't be able to use
       qemu-user anymore. It's not specific to m68k and sh4.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Reply | Threaded
Open this post in threaded view
|

Re: perl/experimental FTBFS on m68k, sh4

Niko Tyni-3
On Sun, Jul 07, 2019 at 12:07:27PM +0200, John Paul Adrian Glaubitz wrote:

> So, some more information: This issue affects qemu-user only and can be reproduced
> on armhf as well. If this is an issue with Perl in general, it means that this
> particular version of Perl will not work on any version of qemu-user independent
> of the architecture used.

Thanks for your work narrowing this down!

I can reproduce it with qemu-armhf. It's not a general issue with Perl,
just with the build machinery. See below.
 
> @Niki: Did you make any changes in particular to the Perl configuration itself? I
>        would like to find out what change exactly triggered the bug and file a
>        bug report either against qemu-user or the component in question. This
>        bug should definitely get fixed otherwise users won't be able to use
>        qemu-user anymore. It's not specific to m68k and sh4.

The change that triggered this is passing -Dmksymlinks to Configure.
This creates a symlink farm of the source in a build subdirectory.
The idea here is that the different flavours (static, shared, debug)
can now be built and tested in parallel in different directories.

The qemu-user specific thing is that $0 (= /proc/pid/cmdline) contains
an absolute path to the binary there regardless of how it was actually
invoked.

On normal systems:

  % sh -c 'echo $0'
  sh

On qemu-user:

  $ sh -c 'echo $0'
  /usr/bin/sh

It looks like the cflags.SH gets called in a different way in a
subdirectory build, from Configure:2013 or so. In a subdirectory build
it gets run with 'sh < cflags.SH' while in a "normal" build it gets run
as 'sh cflags.SH'. The former causes $0 to contain '/usr/bin/sh' under
qemu-user, and this then gets treated as an absolute path argument. So
the code changes to the same path (here /usr/bin) and tries to create
files there.

Looking at https://patchwork.kernel.org/patch/9633901/ I suspect
the $0 handling in qemu-user is not easily fixed. I suppose the the
Perl build machinery should handle this case just like it handles
other quirks in weird systems and environments.

I'll file a bug separately.
--
Niko

Reply | Threaded
Open this post in threaded view
|

Re: perl/experimental FTBFS on m68k, sh4

John Paul Adrian Glaubitz
Hi Niko!

On 7/8/19 6:01 PM, Niko Tyni wrote:

> The qemu-user specific thing is that $0 (= /proc/pid/cmdline) contains
> an absolute path to the binary there regardless of how it was actually
> invoked.
>
> On normal systems:
>
>   % sh -c 'echo $0'
>   sh
>
> On qemu-user:
>
>   $ sh -c 'echo $0'
>   /usr/bin/sh
>
> It looks like the cflags.SH gets called in a different way in a
> subdirectory build, from Configure:2013 or so. In a subdirectory build
> it gets run with 'sh < cflags.SH' while in a "normal" build it gets run
> as 'sh cflags.SH'. The former causes $0 to contain '/usr/bin/sh' under
> qemu-user, and this then gets treated as an absolute path argument. So
> the code changes to the same path (here /usr/bin) and tries to create
> files there.
>
> Looking at https://patchwork.kernel.org/patch/9633901/ I suspect
> the $0 handling in qemu-user is not easily fixed. I suppose the the
> Perl build machinery should handle this case just like it handles
> other quirks in weird systems and environments.
>
> I'll file a bug separately.
Fantastic. Thanks so much for narrowing this down. Much appreciated.

This particular issue drove me nuts and I couldn't figure out what
the problem is. Glad you found out what the problem was.

PS: Can you CC me for the bug report you are filing?

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [hidden email]
`. `'   Freie Universitaet Berlin - [hidden email]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913